I'm building two targets from the same C source files - an executable and a shared library. The shared library requires the code to be relocatable (compiled with -fPIC), but I cannot use relocatable code in my executable for various reasons. Is there a standard/common way to handle this situation in a Makefile?
I suggest you have two directories for object files, one for standard objects and one for relocatable objects. Something like this:
$(STANDARD_OBJS) = $(addprefix $(STANDARD_OBJS_DIR)/, $(OBJS))
$(RELOC_OBJS) = $(addprefix $(RELOC_OBJS_DIR)/, $(OBJS))
$(RELOC_OBJS_DIR)/%.o: CCFLAGS += -fPic
$(RELOC_OBJS_DIR)/%.o $(STANDARD_OBJS_DIR)/%.o: $(SRC)/%.cc
g++ $< -o $#
(There are more sophisticated variations, but this will get you started.)
inject a few rules so that target of your binary build depends and uses static_lib.
shared_lib: whatever.so ...
static_lib: whatever.a ...
binary: static_lib
Related
I'm starting a new project and thinking of using gcc 6.3.1 -MM to generate the dependencies into a file called Make.Dep, that I'll include from Makefile.
The -M option outputs all headers, including system headers. The -MM option doesn't output system headers, but I'm still buried in literally thousands of vendor and package headers such as Sybase and Boost, which I don't think will change (and if they do I'm happy to have to do a full rebuild manually).
Obviously I could wrap gcc -MM in a perl script or what have you that knows what directories I consider packages, but is there some more widely-accepted solution?
Note that one of my vendors' headers look for specific gcc-defined pre-processor symbols to configure their portability. I'd rather not curate a set of such symbols manually to allow dependency generation with some non-gcc method (e.g., makedepend).
Instead of -I, use -isystem to state directories that you don't wish to be output with -MM.
This is not mentioned currently at https://gcc.gnu.org/onlinedocs/gcc/Preprocessor-Options.html despite it seeming to be very closely tied to -M and -MM options.
Example: this creates correct dependencies of foo.cpp and bar.cpp, including Sybase headers:
gcc -MM -I/opt/nmr/sap/sybaseASE/sybclient-16.0.3-7/OCS-16_0/include foo.cpp bar.cpp
Example: this does the same, but not including Sybase headers:
gcc -MM -isystem /opt/nmr/sap/sybaseASE/sybclient-16.0.3-7/OCS-16_0/include foo.cpp bar.cpp
Here is a sample Makefile implementation for gmake. The patsubst function is a pattern substitution using % as a part that matches on the "before" that is then captured and used in the "after." isystem appears to need a space after it, but this is easy to generate with patsubst as the percent sign keeps the space from being truncated. The minus option on -include tells gmake not to complain if the file named doesn't exist. This allows you to use gmake to make depend and produce Make.Dep even before there is a Make.Dep. Finally, this assumes $(PkgIncDirs) hold package include directories none of which should be changing, while $(ProjIncDirs) would be include directories inside the project that you'd want dependencies to be generated for.
depend:
gcc -MM $(CFlags) $(Defines) $(patsubst -I%, -isystem %, $(PkgIncDirs)) $(ProjIncDirs) $(Source) >Make.Dep
-include Make.Dep
I was wondering if there was any sample code for Makefiles (make) and CMakeLists.txt (cmake) that both do the same thing (the only difference being that one is written in make and the other in cmake).
I tried looking for 'cmake vs make', but I never found any code comparisons. It would be really helpful to understand the differences, even if just for a simple case.
The following Makefile builds an executable named prog from the sources
prog1.c, prog2.c, prog3.c and main.c. prog is linked against libmystatlib.a
and libmydynlib.so which are both also built from source. Additionally, prog uses
the library libstuff.a in stuff/lib and its header in stuff/include. The
Makefile by default builds a release target, but offers also a debug target:
#Makefile
CC = gcc
CPP = g++
RANLIB = ar rcs
RELEASE = -c -O3
DEBUG = -c -g -D_DEBUG
INCDIR = -I./stuff/include
LIBDIR = -L./stuff/lib -L.
LIBS = -lstuff -lmystatlib -lmydynlib
CFLAGS = $(RELEASE)
PROGOBJS = prog1.o prog2.o prog3.o
prog: main.o $(PROGOBJS) mystatlib mydynlib
$(CC) main.o $(PROGOBJS) $(LIBDIR) $(LIBS) -o prog
debug: CFLAGS=$(DEBUG)
debug: prog
mystatlib: mystatlib.o
$(RANLIB) libmystatlib.a mystatlib.o
mydynlib: mydynlib.o
$(CPP) -shared mydynlib.o -o libmydynlib.so
%.o: %.c
$(CC) $(CFLAGS) $(INCDIR) $< -o $#
%.o: %.cpp
$(CPP) $(CFLAGS) $(INCDIR) -fPIC $< -o $#
Here is a CMakeLists.txtthat does (almost) exactly the same, with some comments to underline the
similarities to the Makefile:
#CMakeLists.txt
cmake_minimum_required(VERSION 2.8) # stuff not directly
project(example) # related to building
include_directories(${CMAKE_SOURCE_DIR}/stuff/include) # -I flags for compiler
link_directories(${CMAKE_SOURCE_DIR}/stuff/lib) # -L flags for linker
set(PROGSRC prog1.c prog2.c prog3.c) # define variable
add_executable(prog main.c ${PROGSRC}) # define executable target prog, specify sources
target_link_libraries(prog mystatlib mydynlib stuff) # -l flags for linking prog target
add_library(mystatlib STATIC mystatlib.c) # define static library target mystatlib, specify sources
add_library(mydynlib SHARED mydynlib.cpp) # define shared library target mydynlib, specify sources
#extra flags for linking mydynlib
set_target_properties(mydynlib PROPERTIES POSITION_INDEPENDENT_CODE TRUE)
#alternatively:
#set_target_properties(mydynlib PROPERTIES COMPILE_FLAGS "-fPIC")
In this simple example, the most important differences are:
CMake recognizes which compilers to use for which kind of source. Also, it
invokes the right sequence of commands for each type of target. Therefore, there
is no explicit specification of commands like $(CC) ..., $(RANLIB) ... and so on.
All usual compiler/linker flags dealing with inclusion of header files, libraries, etc.
are replaced by platform independent / build system independent commands.
Debugging flags are included by either setting the variable CMAKE_BUILD_TYPE to "Debug",
or by passing it to CMake when invoking the program: cmake -DCMAKE_BUILD_TYPE:STRING=Debug.
CMake offers also the platform independent inclusion of the '-fPIC' flag (via
the POSITION_INDEPENDENT_CODE property) and many others. Still, more obscure settings can be implemented by hand in CMake just as well as in a Makefile (by using COMPILE_FLAGS
and similar properties). Of course CMake really starts to shine when third party
libraries (like OpenGL) are included in a portable manner.
The build process has one step if you use a Makefile, namely typing make at the command line. For CMake, there are two steps: First, you need to setup your build environment (either by typing cmake <source_dir> in your build directory or by running some GUI client). This creates a Makefile or something equivalent, depending on the build system of your choice (e.g. make on Unixes or VC++ or MinGW + Msys on Windows). The build system can be passed to CMake as a parameter; however, CMake makes reasonable default choices depending on your system configuration. Second, you perform the actual build in the selected build system.
Sources and build instructions are available at https://github.com/rhoelzel/make_cmake.
Grab some software that uses CMake as its buildsystem (there's plenty of opensource projects to choose from as an example). Get the source code and configure it using CMake. Read resulting makefiles and enjoy.
One thing to keep in mind that those tools don't map one-to-one. The most obvious difference is that CMake scans for dependencies between different files (e.g. C header and source files), whereas make leaves that to the makefile authors.
If this question is about a sample Makefile output of the CMakeList.txt file then please check the cmake-backend sources and generate one such Makefile. If it is not then adding to the reply of #Roberto I am trying to make it simple by hiding the details.
CMake function
While Make is flexible tool for rules and recipe, CMake is a layer of abstraction that also adds the configuration feature.
My plain CMakeLists.txt will look like the following,
cmake_minimum_required(VERSION 2.8)
project(example)
file(GLOB testapp_SOURCES *.cc)
add_executable(testapp ${testapp_SOURCES})
Note, that CMake hides how the build can be done. We only specified what is the input and output.
The CMakeLists.txt contains list of function-calls that are defined by cmake.
(CMake function) Vs Make rules
In Makefile the rules and recipes are used instead of functions . In addition to function-like feature, rules and recipes provide chaining. My minimalistic Makefile will look like the following,
-include "executable.mk"
TARGETS=testapp.bin
all:${TARGETS}
While the executable.mk will look like the following,
SOURCES=$(wildcard *.cpp)
OBJECTS=$(SOURCES:.cpp=.o)
DEPS=$(SOURCES:.cpp=.d)
%.bin:$(OBJECTS)
$(CC) $(CFLAGS) -o $# $^ $(LFLAGS) $(LIBS)
.PHONY: all clean
clean:
$(RM) $(OBJECTS) $(DEPS) $(TARGETS)
-include $(DEPS)
Starting from the scratch I shall start with a Makefile like the following,
all: testapp.bin
testapp.bin:sourcea.o sourcb.o
$(CC) $(CFLAGS) -o $# $^ $(LFLAGS) $(LIBS)
.PHONY: all clean
clean:
$(RM) $(OBJECTS) testapp.bin
I got this snippet from here and modified it. Note that some implicit-rules are added to this file which can be found in the makefile-documentation. Some implicit variables are also relevant here.
Note, that Makefile provides the detail recipe showing how the build can be done. It is possible to write executable.mk to keep the details defined in one file. In that way the makefile can be reduced as I showed earlier.
Internal Variables in CMake and Make
Now getting little advanced, in CMake we can set a compiler flag like the following,
set(CMAKE_C_FLAGS "-Wall")
Please find out more about CMake default variables in CMakeCache.txt file.
The CMake code above will be equivalent to Make code below,
CFLAGS = -Wall
Note that CFLAGS is an internal variable in Make, the same way, CMAKE_C_FLAGS is internal variable in CMake .
adding include and library path in CMake
We can do it in cmake using functions.
target_include_directories(testapp PRIVATE "myincludes")
list(APPEND testapp_LIBRARIES
mytest mylibrarypath
)
target_link_libraries(testapp ${testapp_LIBRARIES})
Vs adding include and library path in Make
We can add include and libraries by adding lines like the following,
INCLUDES += -Imyincludes
LIBS += -Lmylibrarypath -lmytest
Note this lines above can be generated from auto-gen tools or pkg-config. (though Makefile is not dependent of auto-config tools)
CMake configure/tweek
Normally it is possible to generate some config.h file just like auto-config tools by using configure_file function. It is possible to do more trick writing custom functions. And finally we can select a config like the following,
cmake --build . --config "Release"
It is possible to add some configurable option using the option function.
Makefile configure/tweak
If somehow we need to compile it with some debug flag, we can invoke the make like,
make CXXFLAGS=NDEBUG
I think internal variables, Makefile-rules and CMake-functions are good start for the comparison, good luck with more digging.
I have a small project that builds a number of targets from the same source files. The targets require building the source files with different compile flags. This is actually on cygwin so I'll use that as the concrete example although I suppose this is a generic problem.
So this would be an example Makefile:
a: CFLAGS =
a: a.o c.o
b: CFLAGS = -mno-cygwin
b: b.o c.o
This works in principle, building a will compile with CFLAGS unset, and building b will compile with CFLAGS set to -mno-cygwin. But only if c.o does not already exist.
So doing
> make a b
in a virgin directory will first compile a.c and c.c using an empty CFLAGS. Then it will try to build b using CFLAGS=-mno-cygwin. But since c.o already exists it will not recompile c.c resulting in linker errors since object files need to have the same setting of this flag.
Again, the cygwin-flag is just one concrete example, and I am looking for a generic solution.
What I have tried is to introduce an extra target checking the current CFLAGS and remove all object files if it does not match:
ARCH = `echo $(CFLAGS)`
checkTarget:
-#if test "`cat .arch`" != "$(ARCH)"; then \
rm *.o; \
/bin/echo -n $(ARCH) > .arch; \
fi
Inserting this target as a dependency for the two targets ensures that all object files are recompiled if required.
a: CFLAGS =
a: checkTarget a.o c.o
b: CFLAGS = -mno-cygwin
b: checkTarget b.o c.o
This however recompiles every object file, which is unnecessary, and becomes a problem in a larger project.
Is there a better way to do this?
EDIT:
In the comments to the single answer included a hint that you could make the object files depend on the content of the CFLAGS. I can't figure out how to do that, except cat'ing them to a temporary file and compare to the previous ones and then copy that over a depending file. Is there no better way?
What we need here is two (or more) versions of c.o, compiled with different flags and to be linked into different executables. You could do this with contrived filenames like c_for_a.o, but it's tidier to keep them in different directories, for_a/ and for_b/:
a for_a/%: CFLAGS =
a: a.o for_a/c.o
b for_b/%: CFLAGS = -mno-cygwin
b: b.o for_b/c.o
(If this isn't clear I can fill in the rest of the Makefile.)
EDIT:
If you don't want to keep multiple c.o's, you will have to recompile c.o sometimes. There is a small improvement possible on the checkTarget idea.
When you are building a target like b, and c.o already exists, it matters whether that c.o was built with the CFLAGS suited to b, and you use checkTarget to record that information. But you don't care what CFLAGS were used, only whether they were b's CFLAGS. So you don't have to record anything in the checkTarget file, just update it when you build a new target:
a: CFLAGS =
b: CFLAGS = -mno-cygwin
# You may be able to combine the a and b rules...
a: a.o c.o checkTarget
# build c.o
touch checkTarget
# build the target
b: b.o c.o checkTarget
# build c.o
touch checkTarget
# build the target
# Need this to create checkTarget the first time
checkTarget:
#touch $#
Although there where some suggestions on how to do what I initially asked for here (as pointed to by slowdog), I took the decision to take Beta's different-names-schema one step further and put all objects for the various variants in subdirectories following the description in Beta's second suggestion in his answer to this question.
In essence that made my Makefile look like:
A : AOBJDIR = .a
AOBJECTS = $(addprefix $(AOBJDIR)/,$(ASRCS:.c=.o))
$(AOBJECTS): $(AOBJDIR)/%o: %.c
$(CC) $(CFLAGS) -MMD -o $# -c $<
$(AOBJDIR) :
#mkdir $(AOBJDIR)
-include $(AOBJECTS:.o=.d)
a: $(AOBJDIR) $(AOBJECTS)
$(LINK) ...
So the first part names a subdirectory to be used for object files of 'a' and creates a list of object files in that directory by converting the source filenames and adding the subdirectory prefix.
Then follows the default rule for those object files (with dependency generation thrown in for good measure).
Next two are ensuring that the subdirectory exists, and that the dependency information is included.
Finally the link rule with an extra dependency on the subdirectory itself to ensure that it is created if it does not exist.
This block can be repeated for b, in my example, by exchanging all a's. So if I only could figure out how to package this block into something more general that can be parameterized I would be happy.
I understand that CFLAGS (or CXXFLAGS for C++) are for the compiler, whereas CPPFLAGS is used by the preprocessor.
But I still don't understand the difference.
I need to specify an include path for a header file that is included with #include -- because #include is a preprocessor directive, is the preprocessor (CPPFLAGS) the only thing I care about?
Under what circumstances do I need to give the compiler an extra include path?
In general, if the preprocessor finds and includes needed header files, why does it ever need to be told about extra include directories? What use is CFLAGS at all?
(In my case, I actually found that BOTH of these allow me to compile my program, which adds to the confusion... I can use CFLAGS OR CPPFLAGS to accomplish my goal (in autoconf context at least). What gives?)
The implicit make rule for compiling a C program is
%.o:%.c
$(CC) $(CPPFLAGS) $(CFLAGS) -c -o $# $<
where the $() syntax expands the variables. As both CPPFLAGS and CFLAGS are used in the compiler call, which you use to define include paths is a matter of personal taste. For instance if foo.c is a file in the current directory
make foo.o CPPFLAGS="-I/usr/include"
make foo.o CFLAGS="-I/usr/include"
will both call your compiler in exactly the same way, namely
gcc -I/usr/include -c -o foo.o foo.c
The difference between the two comes into play when you have multiple languages which need the same include path, for instance if you have bar.cpp then try
make bar.o CPPFLAGS="-I/usr/include"
make bar.o CFLAGS="-I/usr/include"
then the compilations will be
g++ -I/usr/include -c -o bar.o bar.cpp
g++ -c -o bar.o bar.cpp
as the C++ implicit rule also uses the CPPFLAGS variable.
This difference gives you a good guide for which to use - if you want the flag to be used for all languages put it in CPPFLAGS, if it's for a specific language put it in CFLAGS, CXXFLAGS etc. Examples of the latter type include standard compliance or warning flags - you wouldn't want to pass -std=c99 to your C++ compiler!
You might then end up with something like this in your makefile
CPPFLAGS=-I/usr/include
CFLAGS=-std=c99
CXXFLAGS=-Weffc++
The CPPFLAGS macro is the one to use to specify #include directories.
Both CPPFLAGS and CFLAGS work in your case because the make(1) rule combines both preprocessing and compiling in one command (so both macros are used in the command).
You don't need to specify . as an include-directory if you use the form #include "...". You also don't need to specify the standard compiler include directory. You do need to specify all other include-directories.
You are after implicit make rules.
To add to those who have mentioned the implicit rules, it's best to see what make has defined implicitly and for your env using:
make -p
For instance:
%.o: %.c
$(COMPILE.c) $(OUTPUT_OPTION) $<
which expands
COMPILE.c = $(CXX) $(CXXFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c
This will also print # environment data. Here, you will find GCC's include path among other useful info.
C_INCLUDE_PATH=/usr/include
In make, when it comes to search, the paths are many, the light is one... or something to that effect.
C_INCLUDE_PATH is system-wide, set it in your shell's *.rc.
$(CPPFLAGS) is for the preprocessor include path.
If you need to add a general search path for make, use:
VPATH = my_dir_to_search
... or even more specific
vpath %.c src
vpath %.h include
make uses VPATH as a general search path so use cautiously. If a file exists in more than one location listed in VPATH, make will take the first occurrence in the list.
I installed httpd on Ubuntu 18.04 using the CPPFLAGS variable for the -DLINUX flag. When run, CPPFLAGS scans the code from top to bottom, file by file, looking for directives before compiling, and will not be extended by other meaningful things like size optimization, flags that do not increase the size of the output file; under the type of processor; to reduce the size of the code and speed up the program; disable all variables except case. The only difference between CPPFLAGS and CFLAGS is that CFLAGS can be set to specify additional switches to be passed to the compiler. That is, the CFLAGS environment variable creates a directory in the installation path (eg CFLAGS=-i/opt/include) to add debugging information to the executable target's path: include general alarm messages; turning off alarm information; independent location generation; display compiler driver, preprocessor, compiler version number.
Standard way to set CPPFLAGS:
sudo ./configure --enable-unixd=DLINUX #for example
list of some known variables:
CPPFLAGS - is the variable name for flags to the C preprocessor.
CXXFLAGS - is the standard variable name for flags to the C++ compiler.
CFLAGS is - the standard name for a variable with compilation flags.
LDFLAGS - should be used for search flags/paths (-L) - i.e. -L/usr/lib (/usr/lib are library binaries).
LDLIBS - for linking libraries.
CPPFLAGS seems to be an invention of GNU Make, referenced in some of its built-in recipes.
If your program is built by some Free software distributions, you may find that some of them require packages to interpolate this variable, using CPPFLAGS for passing down options like -D_WHATEVER=1 for passing down a macro definition.
This separation is a poor idea and completely unnecessary in the GNU environment because:
There is a way to run gcc to do preprocessing only (while ignoring compiler options unrelated to preprocessing).
The stand-alone GNU cpp is tolerant to compiler options, such as -W warnings that do not pertain to preprocessing and even code generation options like -fstrict-aliasing and the linker-pass through like -Wl,--whatever.
So generally speaking, build systems that need to call the stand-alone preprocessor for whatever reason can just pass it $(CFLAGS).
As an application developer writing a Makefile, you cannot rely on the existence of CPPFLAGS. Users who are not insider experts in open source building won't know about CPPFLAGS, and will do things like make CFLAGS=-Dfoo=bar when building your program. If that doesn't work, they will be annoyed.
As a distro maintainer, you cannot rely on programs to pull in CPPFLAGS; even otherwise well-behaved ones that pull in CFLAGS, LDFLAGS and LDLIBS.
It's easy enough for the application developers to write GNU Make code to separate preprocessor flags out of $(CFLAGS):
cpp_only_flags := $(foreach arg, \
$(CFLAGS), \
$(or $(filter -D%,$(arg)), \
$(filter -U%,$(arg)), \
$(filter -I%,$(arg)), \
$(filter -iquote%,$(arg)), \
$(filter -W%,$(arg)), \
$(filter -M%,$(arg)))) \
$(CPPFLAGS) # also pull this in
all:
#echo cpp_only_flags == $(cpp_only_flags)
Demo:
$ make CFLAGS="-Wall -I/path/to/include -W -UMAC -DFOO=bar -o foo.o -lm"
cpp_only_flags == -Wall -I/path/to/include -W -UMAC -DFOO=bar
In the case of the GNU compiler and preprocessor, this is probably unnnecessary; but it illustrates a technique that could be used for non-GNU compilers and preprocessors, in a build system based on GNU Make.
I wonder where to put the optimization and debugging options in Makefile: linking stage or compiling stage? I am reading a Makefile:
ifeq ($(STATIC),yes)
LDFLAGS=-static -lm -ljpeg -lpng -lz
else
LDFLAGS=-lm -ljpeg -lpng
endif
ifeq ($(DEBUG),yes)
OPTIMIZE_FLAG = -ggdb3 -DDEBUG
else
OPTIMIZE_FLAG = -ggdb3 -O3
endif
ifeq ($(PROFILE),yes)
PROFILE_FLAG = -pg
endif
CXXFLAGS = -Wall $(OPTIMIZE_FLAG) $(PROFILE_FLAG) $(CXXGLPK)
test: test.o rgb_image.o
$(CXX) $(CXXFLAGS) -o $# $^ $(LDFLAGS)
Makefile.depend: *.h *.cc Makefile
$(CC) -M *.cc > Makefile.depend
clean:
\rm -f absurdity *.o Makefile.depend TAGS
-include Makefile.depend
What surprises me is CXXFLAGS is used in linking. I know it is also used in the implicit rule for compiling to generate .o files but is it necessary to use it again for linking? Specifically, where should I put optimization and debugging: linking stage or compiling stage?
Short answer:
optimization: needed at compiler time
debug flag: needed at compile time
debugging symbols: need at both compile and linking time
Take note that the linker decides what bits of each object file and library need to be included in the final executable. It could throw out the debugging symbols (I don't know what the default behavior is), so you need to tell it not to.
Further, the linker will silently ignore options which do not apply to it.
To the comments:
The above are very general claims based on knowing what happens at each stage of compilation, so no reference.
A few more details:
optimization: takes two major forms: peephole optimization can occur very late, because it works on a few assembly instructions at a time (I presume that in the GNU tool chain the assembler is responsible for this step), but the big gains are in structural optimizations that are generally accomplished by re-writing the Abstract Syntax Tree (AST) which is only possible during compilation.
debug flag: In your example this is a preprocessor directive, and only affects the first part of the compilation process.
debugging symbols: Look up the ELF file format (for instance), you'll see that various bits of code and data are organized into different blocks. Debugging symbols are stored in the same file along as the code they relate to, but are necessarily kept separate from the actual code. As such, any program that manipulates these files could just dump it. Therefore both the compiler and the linker need to know if you want them or not.