I want to make static XGBoost library, but I'm failing at this easy task.
Firstly, I simply tried to take all .o files and via ar rcs wrapper/libxgboostwrapper.a ./* make static library, but I failed. In fact, when I'm making other stuff which depends on XGBoost it fails with errors: undefined reference to XGBoosterFree.
This function is defined in wrapper part and there is no file wrapper/libxgboostwrapper.o. So in order to make a static library firstly we have to create wrapper/libxgboostwrapper.o, secondly, run ar rcs on all needed .o files.
How I modified Makefile.
Firstly, I slightly changed names of some variables.
ifeq ($(OS), Windows_NT)
LIBRABIT = subtree/rabit/lib/librabit_empty.a
DLIB = wrapper/xgboost_wrapper.dll
SLIB = wrapper/xgboost_wrapper.lib
else
LIBRABIT = subtree/rabit/lib/librabit.a
DLIB = wrapper/libxgboostwrapper.so
SLIB = wrapper/libxgboostwrapper.a
endif
Added new variable.
WRAP = wrapper/libxgboostwrapper.o
Modified target.
TARGET = $(BIN) $(OBJ) $(DLIB) $(SLIB) $(WRAP)
Added dependencies for wrapper/libxgboostwrapper.o exactly the same as for wrapper/libxgboostwrapper.so.
wrapper/libxgboostwrapper.o: wrapper/xgboost_wrapper.cpp src/utils/*.h src/*.h src/learner/*.hpp src/learner/*.h updater.o gbm.o io.o $(LIBRABIT) $(LIBDMLC)
After everything is done for $(BIN), $(MOCKBIN), $(DLIB), I'm trying to compile (or make?) my $(WRAP).
$(WRAP) :
$(CXX) $(CFLAGS) -fPIC -o $# $(filter %.cpp %.o %.c %.a %.cc, $^) $(LDFLAGS) $(DLLFLAGS)
After all this stuff finally I'm making wrapper/libxgboostwrapper.a.
$(SLIB) : updater.o gbm.o io.o $(WRAP) $(LIBRABIT) $(LIBDMLC)
ar rcs $# $^
It fails at step 5 with errors like:
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 0 has invalid symbol index 10
What am I doing wrong? For me it seems like compilation .so and .o files shuld be the same except flag -shared is not used.
This may take a few iterations, since you seem to have a few misconceptions about how these files work.
STEP 1: First, let's compile a source file to produce an object file:
g++ -o wrapper/xgboostwrapper.o -c wrapper/xgboost_wrapper.cpp
(Don't call the object file "libxgboostwrapper.o". The "lib" prefix is for a library, the ".o" suffix is for an object file, when you use them both it's just confusing.)
Do not proceed until that works.
STEP 2: Write a rule in your makefile to do it:
wrapper/xgboostwrapper.o: wrapper/xgboost_wrapper.cpp
$(CXX) $(CFLAGS) -o $# $<
Do not proceed until that works.
STEP 3: Figure out which header files it actually needs, and add those as prerequisites. (It's a bad idea to use wildcards to add all existing headers as prerequisites.):
wrapper/xgboostwrapper.o: wrapper/xgboost_wrapper.cpp src/utils/foo.h src/xgboost.h
$(CXX) $(CFLAGS) -o $# $<
Try that much and tell us how it went, then we can proceed.
Related
My makefile always recompiles everything in directory if one header is changed. It's not a problem now but since I'm adding more to my program this is becoming and issue. I don't want to wait for a whole recompile if I add a new variable to a header of a separate class object.
Here is my makefile:
CXX = g++
CPPFLAGS = -I -lm -lsfml-graphics -lsfml-audio -lsfml-window -lsfml-system
OBJ = CR_Main.o CarRental.o CR_Button.o CR_LoginMenu.o CR_TextBox.o CR_UserCreation.o CR_CheckBox.o
DEPS = CarRental.hpp CR_Button.hpp CR_LoginMenu.hpp CR_TextBox.hpp CR_UserCreation.hpp CR_CheckBox.hpp
%.o: %.cpp $(DEPS)
$(CXX) -c -o $# $< $(CPPFLAGS)
CRC.exe: $(OBJ)
$(CXX) -o $# $^ $(CPPFLAGS)
.PHONY: clean
clean:
del *.o *.exe
Thanks in advance!
EDIT:
I was wondering why is it compiling everything in my directory if only 1 out 6 .hpp files are modified on one line? Is something wrong with my makefile or is that how it is?
why is it compiling everything in my directory if only 1 out 6 .hpp files are modified on one line? Is something wrong with my makefile or is that how it is?
"Wrong" might be too strong a word, but yes, the behavior you describe is a consequence of how your makefile is written.
This rule ...
%.o: %.cpp $(DEPS)
$(CXX) -c -o $# $< $(CPPFLAGS)
... says, roughly, that you can build .o files from corresponding .cpp files plus all the files named in variable DEPS. This implies that if that's the rule make selects for building a given .o file, and any of those prerequisites is newer than the target, then the target is out of date and needs to be rebuilt. You have named all your headers in DEPS and you have not provided any other rules for building .o files, so yes, if any of your headers changes, all of the .o files will be rebuilt.
The most simple-minded alternative would be to write a separate rule for each .o, naming the prerequisites of that file only. That is, the corresponding .cpp file and whichever headers it #includes, directly or indirectly.
But you can save yourself a little typing by instead removing the $(DEPS) part from your existing rule, and adding an additional rule for each .o that does not have a recipe but names all the header prerequisites for that file.
Or if, as it appears, you have consistent relationships between source file names and header names, you might do something like this:
CXX = g++
CPPFLAGS = -I.
LIBS = -lsfml-graphics -lsfml-audio -lsfml-window -lsfml-system -lm
MAIN_OBJ = CR_Main.o
MODULE_OBJS = CarRental.o CR_Button.o CR_LoginMenu.o CR_TextBox.o CR_UserCreation.o CR_CheckBox.o
$(MAIN_OBJ): CR_Main.cpp $(MODULE_OBJS:.o=.h)
$(CXX) $(CPPFLAGS) -c -o $# $<
%.o: %.cpp %.h
$(CXX) $(CPPFLAGS) -c -o $# $<
CRC.exe: $(MAIN_OBJ) $(MODULE_OBJS)
$(CXX) -o $# $^ $(LIBS)
# Extra dependencies (guesses for the sake of example):
CarRental.o CR_LoginMenu.o CR_UserCreation.o: CR_TextBox.h CR_CheckBox.h
# No recipe here
.PHONY: clean
clean:
del $(MAIN_OBJ) $(MODULE_OBJS) CRC.exe
Ultimately, though, what you would really like to do is generate all the header dependencies automatically. That makes the project much easier to maintain once you get it initially set up. You can find lots of information about that on the web, some of it in the GNU make manual.
I have a domain specific language compiler (homemade) which takes a file x.inflow and generates two files: x.c and x.h. The C file is compiled in the conventional manner and the generated header file has to be included into any file that calls the functions defined within it.
The header files therefore have to be generated before any C files that use them are compiled. My current Makefile, below, works fine except for the first build from clean where it can try and compile main.c before the header file that it includes has been created.
NAME = simplest
OBJ = $(patsubst %.c,%.o,$(wildcard *.c)) \
$(patsubst %.inflow,%.o,$(wildcard *.inflow))
CC = gcc
CFLAGS = -g -Wall
$(NAME): $(OBJ)
$(CC) $(CFLAGS) -o $# $^ $(CLIBS)
# Dependencies for existing .o files.
-include $(OBJ:.o=.d)
# Compile an inflow file into both a .c and .h file.
# Note that this rule has two targets.
%.c %.h: %.inflow
inflow $<
# Compile object files and generate dependency information.
%.o: %.c
$(CC) -MD -MP -c $(CFLAGS) -o $# $<
Obviously, I can fix this for specific cases by adding, for example (where simplest.h is a generated header):
main.o: simplest.h
But is there a general way to force one type of pattern rule (%.c %.h: %.inflow) to be run before any invokations of another (%.o: %.c)?
Well, you can force any target to be run before any other target with order-only prerequisites. So for example, you can write:
%.o : %.c | simplest.h
$(CC) -MD -MP -c $(CFLAGS) -o $# $<
which will ensure that no target that uses this pattern rule to build will be invoked before the target simplest.h is created. However, I don't think you can put patterns in an order-only prerequisite. To be honest, I've never tried it so it's possible that it works, I'm not sure.
If not, you could just list all the order-only prerequisites in the %.o pattern rule; this would ensure that all the inflow files are generated before any of the object files are built. That's probably OK.
It seems the problem is twofold:
Make doesn't know that it needs to generate simplest.h before compiling main.c.
You don't want to have to explicitly tell Make about the dependency (and remember to update it when it changes).
Rather than force Make to evaluate rules in a set order, you can solve your problem by letting Make create the dependencies for you. Check out this section of the Gnu Make manual: http://www.gnu.org/software/make/manual/make.html#Automatic-Prerequisites
When you run Make, it will scan your source files and gather their dependencies for you (and you won't have to explicitly list that main.o depends on simplest.h).
I am new to make files and I put this together with a bit of trial & error. This code is used to compile my c++ program.
My main.cpp file in the same folder as the makefile. I have a lib/ folder that contains the headers main depends on.
The following makefile results in a correct and complete compilation of my code. But I was expecting that I would find *.o objects left behind. (Note that I've tried to make both with and without the "clean" rule, and I get the same results both times.)
#
# Makefile
#
CXX = g++
CCFLAGS = -O3 -I/sw/include -L/sw/lib
## /sw/include and /sw/lib contain dependencies for files in my lib/
LDFLAGS = -lpng
OPTS = $(CCFLAGS) $(LDFLAGS)
SOURCES = $(wildcard lib/*.cpp) main.cpp
OBJECTS = $(SOURCES: .cpp = .o)
TARGET = spirals
$(TARGET): $(OBJECTS)
$(CXX) $(OPTS) $^ -o $#
.PHONY: depend
depend:
g++ -MM $(SOURCES) > depend
## generate dependencies list
include depend
.PHONY: clean
clean:
rm -f *.o lib/*.o $(TARGET)
Also, in case it matters, I'm on MacOSX and my program is designed in xcode. (I know that xcode has its own build flow, but I'm designing a command-line program for a linux system and I'd like to test compilation & linking in a bash environment instead of only going through xcode.)
Questions:
Am I correct to expect makefiles to produce *.o files that stick around once the main target has been created?
If so, why doesn't my makefile do this?
If you observe what command your $(TARGET) rule causes to be run:
g++ -O3 -I/sw/include -L/sw/lib -lpng lib/bar.cpp lib/foo.cpp main.cpp -o spirals
you'll see that $(OBJECTS) in fact contains *.cpp files, and there are no *.o files sticking around because you haven't asked for any.
The problem is here:
OBJECTS = $(SOURCES:.cpp=.o)
In your GNU makefile as written, this substitution reference is written with excess spaces, so never matches anything and $(OBJECTS) ends up the same as $(SOURCES). Rewrite it as above and it'll do what you expect.
(Other notes: -lpng needs to go at the end of the link command to work in general, so you should introduce another make variable (traditionally called $(LDLIBS)) so as to arrange that. Especially as someone new to makefiles, you would do better to spell out your dependencies explicitly rather than playing games with $(wildcard) and a computed $(OBJECTS). -I options are needed during compilation while -L options are used during linking, so it would be good to arrange separate $(CXXFLAGS)/$(LDFLAGS) variables used in separate rules so they are only added when required.)
i need to build the same source tree twice,
1 - with normal cflags to build the project binary
2 - with cflags plus -fPIC to build a static library that would be some sort of SDK to develop project dynamic modules.
Using only one Makefile, what is the best approach to accomplish this?
It would be nice to do some sort of :
all: $(OBJECTS)
lib_rule: $(OBJECTS)
CFLAGS += -fPIC
.cpp.o:
$(CC) -c $< -o $# $(CFLAGS)
But obviously it can't be done.
Thanks
One thing I've used in the past is a different extension:
.cpp.o:
$(CC) -c $< -o $# $(CFLAGS)
.cpp.lo:
$(CC) -c $< -o $# $(CFLAGS) $(EXTRA_CFLAGS)
You then build your static library from the .lo files and you binary from the .o files:
prog: a.o b.o
libsdk.a: a.lo b.lo
Assuming you are using GNU Make, you can use some built in functions to only have to maintain the list of objects one time:
OBJS = a.o b.o
LOBJS = $(patsubst %.o, %.lo, $(OBJS))
GNU make offers also "Target-specific Variable Values". Consider the following Makefile:
# Makefile
CFLAGS := My Cflags
all: $(OBJECTS)
#echo "$# CFLAGS is: " $(CFLAGS)
lib_rule: CFLAGS += extended by -fPIC
lib_rule: $(OBJECTS)
#echo "$# CFLAGS is: " $(CFLAGS)
# Makefile - end.
$ make all
all CFLAGS is: My Cflags
$ make lib_rule
lib_rule CFLAGS is: My Cflags extended by -fPIC
$
(Please note: if you copy and paste the example, remember to re-add the tabstops in front of the command lines. I always get caught by that.)
Instead of placing the compiled .o files in the same directory as the source, I create them in labeled sub-directories. In your case, you can have the static library files created as source_dir/lib/*.o and your normal files as source_dir/bin/*.o. In your different build targets after you set up your unique CFLAGS, simply generate a DIR_NAME value holding the name of the appropriate sub-folder. You can use this variable when you create the paths for the compiler to use when building and when linking.
In a different make tool such as CMake, you can express something like that much more easily.
For instance, you could well do
set(sources ABC.cpp DEF.cpp XYZ.cpp)
ADD_LIBRARY(mylib STATIC ${sources})
add_executable(myExecutable ${sources} main.cpp)
Or, you could repeatedly build the same directory with different flags by including it several times from the directory's logical parent, i.e.
set(MyTweakFlag 2)
add_subdirectory("libDir" "libDir2")
set(MyTweakFlag 3)
add_subdirectory("libDir" "libDir3")
...and then use if() or whatnot in the child directory to set the right flags.
Particularly if you have many such configurations, using make becomes quite fragile; make won't correctly find the transitive closure of recursive make dependancies (and certainly won't correctly find the dependancy on the makefile itself - if you change flags, say) so if you're going to do complicated makefile magic: do it with a better tool!
(CMake simply happens to be what I replaced make with, but there are various other replacements possible, of course)
I have the following makefile that I use to build a program (a kernel, actually) that I'm working on. Its from scratch and I'm learning about the process, so its not perfect, but I think its powerful enough at this point for my level of experience writing makefiles.
AS = nasm
CC = gcc
LD = ld
TARGET = core
BUILD = build
SOURCES = source
INCLUDE = include
ASM = assembly
VPATH = $(SOURCES)
CFLAGS = -Wall -O -fstrength-reduce -fomit-frame-pointer -finline-functions \
-nostdinc -fno-builtin -I $(INCLUDE)
ASFLAGS = -f elf
#CFILES = core.c consoleio.c system.c
CFILES = $(foreach dir,$(SOURCES),$(notdir $(wildcard $(dir)/*.c)))
SFILES = assembly/start.asm
SOBJS = $(SFILES:.asm=.o)
COBJS = $(CFILES:.c=.o)
OBJS = $(SOBJS) $(COBJS)
build : $(TARGET).img
$(TARGET).img : $(TARGET).elf
c:/python26/python.exe concat.py stage1 stage2 pad.bin core.elf floppy.img
$(TARGET).elf : $(OBJS)
$(LD) -T link.ld -o $# $^
$(SOBJS) : $(SFILES)
$(AS) $(ASFLAGS) $< -o $#
%.o: %.c
#echo Compiling $<...
$(CC) $(CFLAGS) -c -o $# $<
#Clean Script - Should clear out all .o files everywhere and all that.
clean:
-del *.img
-del *.o
-del assembly\*.o
-del core.elf
My main issue with this makefile is that when I modify a header file that one or more C files include, the C files aren't rebuilt. I can fix this quite easily by having all of my header files be dependencies for all of my C files, but that would effectively cause a complete rebuild of the project any time I changed/added a header file, which would not be very graceful.
What I want is for only the C files that include the header file I change to be rebuilt, and for the entire project to be linked again. I can do the linking by causing all header files to be dependencies of the target, but I cannot figure out how to make the C files be invalidated when their included header files are newer.
I've heard that GCC has some commands to make this possible (so the makefile can somehow figure out which files need to be rebuilt) but I can't for the life of me find an actual implementation example to look at. Can someone post a solution that will enable this behavior in a makefile?
EDIT: I should clarify, I'm familiar with the concept of putting the individual targets in and having each target.o require the header files. That requires me to be editing the makefile every time I include a header file somewhere, which is a bit of a pain. I'm looking for a solution that can derive the header file dependencies on its own, which I'm fairly certain I've seen in other projects.
As already pointed out elsewhere on this site, see this page:
Auto-Dependency Generation
In short, gcc can automatically create .d dependency files for you, which are mini makefile fragments containing the dependencies of the .c file you compiled.
Every time you change the .c file and compile it, the .d file will be updated.
Besides adding the -M flag to gcc, you'll need to include the .d files in the makefile (like Chris wrote above).
There are some more complicated issues in the page which are solved using sed, but you can ignore them and do a "make clean" to clear away the .d files whenever make complains about not being able to build a header file that no longer exists.
You could add a 'make depend' command as others have stated but why not get gcc to create dependencies and compile at the same time:
DEPS := $(COBJS:.o=.d)
-include $(DEPS)
%.o: %.c
$(CC) -c $(CFLAGS) -MM -MF $(patsubst %.o,%.d,$#) -o $# $<
The '-MF' parameter specifies a file to store the dependencies in.
The dash at the start of '-include' tells Make to continue when the .d file doesn't exist (e.g. on first compilation).
Note there seems to be a bug in gcc regarding the -o option. If you set the object filename to say obj/_file__c.o then the generated _file_.d will still contain _file_.o, not obj/_file_c.o.
This is equivalent to Chris Dodd's answer, but uses a different naming convention (and coincidentally doesn't require the sed magic. Copied from a later duplicate.
If you are using a GNU compiler, the compiler can assemble a list of dependencies for you. Makefile fragment:
depend: .depend
.depend: $(SOURCES)
rm -f ./.depend
$(CC) $(CFLAGS) -MM $^>>./.depend;
include .depend
There is also the tool makedepend, but I never liked it as much as gcc -MM
You'll have to make individual targets for each C file, and then list the header file as a dependency. You can still use your generic targets, and just place the .h dependencies afterwards, like so:
%.o: %.c
#echo Compiling $<...
$(CC) $(CFLAGS) -c -o $# $<
foo.c: bar.h
# And so on...
Basically, you need to dynamically create the makefile rules to rebuild the object files when the header files change. If you use gcc and gnumake, this is fairly easy; just put something like:
$(OBJDIR)/%.d: %.c
$(CC) -MM -MG $(CPPFLAGS) $< | sed -e 's,^\([^:]*\)\.o[ ]*:,$(#D)/\1.o $(#D)/\1.d:,' >$#
ifneq ($(MAKECMDGOALS),clean)
include $(SRCS:%.c=$(OBJDIR)/%.d)
endif
in your makefile.
Over and above what #mipadi said, you can also explore the use of the '-M' option to generate a record of the dependencies. You might even generate those into a separate file (perhaps 'depend.mk') which you then include in the makefile. Or you can find a 'make depend' rule which edits the makefile with the correct dependencies (Google terms: "do not remove this line" and depend).
Simpler solution: Just use the Makefile to have the .c to .o compilation rule be dependent on the header file(s) and whatever else is relevant in your project as a dependency.
E.g., in the Makefile somewhere:
DEPENDENCIES=mydefs.h yourdefs.h Makefile GameOfThrones.S07E01.mkv
::: (your other Makefile statements like rules
::: for constructing executables or libraries)
# Compile any .c to the corresponding .o file:
%.o: %.c $(DEPENDENCIES)
$(CC) $(CFLAGS) -c -o $# $<
None of the answers worked for me. E.g. Martin Fido's answer suggests gcc can create dependency file, but when I tried that it was generating empty (zero bytes) object files for me without any warnings or errors. It might be a gcc bug. I am on
$ gcc --version gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-16)
So here's my complete Makefile that works for me; it's a combination of solutions + something that wasn't mentioned by anyone else (e.g. "suffix replacement rule" specified as .cc.o:):
CC = g++
CFLAGS = -Wall -g -std=c++0x
INCLUDES = -I./includes/
# LFLAGS = -L../lib
# LIBS = -lmylib -lm
# List of all source files
SRCS = main.cc cache.cc
# Object files defined from source files
OBJS = $(SRCS:.cc=.o)
# # define the executable file
MAIN = cache_test
#List of non-file based targets:
.PHONY: depend clean all
## .DEFAULT_GOAL := all
# List of dependencies defined from list of object files
DEPS := $(OBJS:.o=.d)
all: $(MAIN)
-include $(DEPS)
$(MAIN): $(OBJS)
$(CC) $(CFLAGS) $(INCLUDES) -o $(MAIN) $(OBJS) $(LFLAGS) $(LIBS)
#suffix replacement rule for building .o's from .cc's
#build dependency files first, second line actually compiles into .o
.cc.o:
$(CC) $(CFLAGS) $(INCLUDES) -c -MM -MF $(patsubst %.o,%.d,$#) $<
$(CC) $(CFLAGS) $(INCLUDES) -c -o $# $<
clean:
$(RM) *.o *~ $(MAIN) *.d
Notice I used .cc .. The above Makefile is easy to adjust for .c files.
Also notice importance of these two lines :
$(CC) $(CFLAGS) $(INCLUDES) -c -MM -MF $(patsubst %.o,%.d,$#) $<
$(CC) $(CFLAGS) $(INCLUDES) -c -o $# $<
so gcc is called once to build a dependency file first, and then actually compiles a .cc file. And so on for each source file.
I believe the mkdep command is what you want. It actually scans .c files for #include lines and creates a dependency tree for them. I believe Automake/Autoconf projects use this by default.