I need to build sources to binary file and two a static library.
Here is an example (I replaced recipes with ';' for brevity):
objects := a.o b.o ...
.PHONY: all build build_lib
all: build build_lib
build: bin/app
build_lib: bin/libapp.a
bin/app: $(objects) ;
bin/libapp.a $(objects) ;
obj/%.o: %.cpp ;
Will there be problems with parallel build? Can two make processes try to rebuild the same *.o file at once, making a broken build?
I guessed that, they can, so I rewritten the code this way:
objects := a.o b.o ...
.PHONY: all build build_lib
all: $(objects) | bin/app bin/libapp.a
build: bin/app
build_lib: bin/libapp.a
bin/app: $(objects) ;
bin/libapp.a $(objects) ;
obj/%.o: %.cpp ;
But the --debug=b output still shows:
Processing target file `all'.
File `all' does not exist.
Processing target file `bin/app'.
File `bin/app' does not exist.
Processing target file `obj/client.o'.
Need to rebuild target `obj/client.o'.
...
File `sb_all' does not exist.
File `bin/app' does not exist.
File `bin/libapp.a' does not exist.
File `sb_all' does not exist.
File `bin/app' does not exist.
File `bin/libapp.a' does not exist.
...
Need to rebuild target `bin/app'.
g++ -lgd ...
Need to rebuild target `bin/libapp.a'.
ar ...
File `all' does not exist.
Target file `all' rebuilt successfully.
So it seems that my $(objects) target does not run before order-only prerequisites, or do I incorrectly read output? And did I need this change anyway?
No, there is no problem with it. Make will not have any problem with parallelism and multiple targets (in the same instance of make) depending on the same prerequisite. If you have recursive instances of make and multiple different make instances try to build the same target you'll have problems.
Order-only doesn't have any impact on parallelism at all. Make will still invoke things in parallel if possible. The only way to impact the order in which rules are run is to declare a prerequisite relationship between those targets. Here you're just saying that both the higher-level targets must be built before the all target, so that doesn't do anything to reduce parallelism.
Luckily as I said above, you don't have to. As long as your makefile correctly defines the dependency relationship between any two targets, make will handle the bigger picture just fine.
Related
Im fairly new to makefiles. I want to compile multiple executables through my makefile, and it was to my understanding that having a target with multiple entries would run the recipe for that target through all entries. My example is:
$(EXE): $(OBJS)
g++ -o $# $< -L$(LIBPATH) -lOSApi -lrt -lpthread
My EXE variable contains all files that should be created, something like: prog1 prog2 and so on. My OBJS contains prog1.o prog2.o and so on.
When running make i create all .o files perfectly, but i only create one executable. I have tried replacing $# with $(EXE) and such, but no luck so far.
Any help would be appreciated.
EDIT:
I found the solution through MadScientist, who suggested to add an all target, and then changing my executable target to:
$(EXE): % : %.o
g++ -o $# $< -L$(LIBPATH) -lOSApi -lrt -lpthread
.PHONY: all clean
all: $(EXE)
Which to my understanding makes every target in my EXE target dependant on its corresponding .o file.
It would help greatly if you provided a full (small) sample. In the question you don't show us what the definition of EXE or OBJS is which makes it hard to say exactly.
Also, please be sure to format your question correctly.
By default make only builds the FIRST target in the makefile. It doesn't build ALL the targets in the makefile. So, if EXE contains multiple targets and the first rule in your makefile is $(EXE) : ... then only the first target in that list will be built.
You should add a new target before the above, saying that you want the default to build all the exe's. You can call it anything you like but the convention is to call it all:
all: $(EXES)
(you can also add a .PHONY: all for safety). Now the first target in the makefile is all, and as prerequisites it will build all the targets in the EXES variable.
I tried to use .DELETE_ON_ERROR target in makefile in order to delete both $(OBJ)
and executable files if the recipe fails, but it doesn't work. If I put an error inside any object file than while compiling the pattern rule an error occurs and it stops. The old object file is still on its place but I expect .DELETE_ON_ERROR to remove it.
Can anyone test the code? Can -include $(DEP) or flag -DDBG influence? The goal is to delete both the .o file that failed and the executable.
OUTPUT = executable
CPP := $(shell find $(SRC) -type f -name "*.cpp")
OBJ := $(CPP:.cpp=.o)
DEP := $(OBJ:.o=.d)
CXX := g++
CXXFLAGS =-MMD -MP -DDBG
INCLUDES = -I.
.DELETE_ON_ERROR :
$(OUTPUT): $(OBJ)
$(CXX) $^ -o $#
%.o: %.cpp
$(CXX) $(CXXFLAGS) $(INCLUDES) -c $< -o $#
-include $(DEP)
.PHONY : clean
clean:
rm -rf $(OBJ) $(DEP)
EDIT: According to the Ondrej K. solution to fix this problem you need to add #touch command before compilator in order to make the object files changed (the docs read "delete the target of a rule if it has changed".). So, the code should look like this:
%.o: %.cpp
#touch $#
$(CXX) $(CXXFLAGS) $(INCLUDES) -c $< -o $#
Not sure what failure you're seeing, but I am afraid there really isn't a good way for you to do that. .o files and executable ($(OUTPUT)) are separate rules. If the latter fails, former is already out of consideration. See the documentation:
.DELETE_ON_ERROR:
If .DELETE_ON_ERROR is mentioned as a target anywhere in the makefile, then make will delete the target of a rule if it has changed and its recipe exits with a nonzero exit status, just as it does when it receives a signal. See Errors in Recipes.
In other words, if your the target producing a binary object failed after .o target itself got updated, make would prune the changed file. But if your executable did not link, it won't go back and delete object files.
Not sure it'd be nice, but if you really needed to, you could probably achieve this by refactoring your makefile to basically have direct exec + objs from source prerequisites rule with a single recipe. Obvious downside, such rule would mean single .c file change causing all files being recompiled (basically negating substantial benefit of using make).
EDIT: I'll expand on the comment a bit to clarify. What you seem to want is: in case there is a broken .c file and compilation fails, remove the old .o file. That is quite clearly not how .DELETE_ON_ERROR works though. If the .o file already got updated, and then the rule failed, it would remove it ("delete the target of a rule if it has changed"), but in case of a mentioned syntactical problem, the compiler would fail before it would produced an .o file.
So, if for instance you updated your (pattern) rule for compilation so that it first touches (effectively updates timestamp) on the .o file and then tries to compile. After the compiler call and rule failed make would consider the target of the failed root to have been updated and remove it. Alternatively you could also change to rule to first try to rm the expected '.o' file in which case you actually wouldn't need to use .DELETE_ON_ERROR (and if there is no change in the relevant sources, the rule does not get used, so it's actually not as terrible as it sounds). Either way is not exactly very clean, but leads towards the behavior I understand you're describing.
It is possible that the Compiler crashes while writing the Output file. In this case, there is a corrupt output file that is newer than its sources. Make will stop due to the error, but on next run, it won't recompile the output file as it is newer than ist sources - and the make will fail again and again in the build step.
With the .DELETE_ON_ERROR rule, make will delete the Output file if the compiler (or whatever build step failed) exits with an error after touching (and corrupting) the Output file, so it will be recompiled on next run. (if the Compiler failed without touching the old output file, it will always be recompiled on next run anyway)
Most makefiles have a structure such as this:
.PHONY: prebuild
all: $(TARGET)
prebuild: Makefile
$(shell DEPDIR=$(DEPDIR) mkdir -p $(DEPDIR)/../common >/dev/null)
# do other work related to preparing for the object files to be built such as run a script to modify a header file included by $(TARGET).c
$(TARGET): $(TARGET).c prebuild
$(CC) $(CFLAGS) -o $(TARGET) $(TARGET).c
The implicit rules know how to build $(TARGET).o from $(TARGET).c, and doesn't do any work if $(TARGET).o is already newer than $(TARGET).c. This happens when make is run multiple times without changing the source file.
However, building the all target above will seemingly always rerun the $(CC) $(CFLAGS) -o $(TARGET) $(TARGET).c link to link the application and create the application binary. This happens even if that binary already exists and doesn't need to be recreated. In some larger projects, this process can take a long time (tens of seconds), which is sometimes not desirable.
Edit #1: The issue has to do something with an extra phony target that I do want to run ONCE before the object files are built. In my case, I'm running a script which takes Makefile variables and possibly updates a header file that is included in the C file. But, if the Makefile doesn't change, the prebuild target isn't run. However, the $(TARGET) target is still run even if prebuild doesn't do anything (for instance, because Makefile wasn't changed). FYI: because of the structure of my build system, I have prebuild run always because my build system is used for a variety of applications that can dynamically redefine prebuild.
How can this Makefile be restructured to not do the linking again when not necessary?
Edit #2:
Here's a simplified example that seems to illustrate my issue:
Before running, create a new directory and touch a b
.PHONY: prebuild main all
all: main
prebuild: a Makefile
#echo prebuild ran
main: prebuild
#echo main ran
When I run, I get this output:
prebuild ran
main ran
This is what happens no matter how many times I run make, even though the prerequisite a nor Makefile doesn't change. What I expect to happen is prebuild doesn't run (because a and Makefile don't change) and main also doesn't run because prebuild doesn't run. Clearly, I'm misunderstanding something.
The problem is that extra dependency triggering your rebuild.
Try this:
.PHONY: all
OUTPUTDIR=common/
TARGET=finalexe
all: $(OUTPUTDIR)/$(TARGET)
$(OUTPUTDIR)/$(TARGET): $(TARGET).c | $(OUTPUTDIR)
$(CC) $(CFLAGS) -o $# $(TARGET).c
$(OUTPUTDIR):
mkdir -p $#
In this above example, 'finalexe' will be created if A. it doesn't yet exist or B. if finalexe.c was modified. The timestamp on the OUTPUTDIR is not checked.
The question was edited after MadScientist's answer. See history for the original makefile, but the problem stays the same.
I have a small makefile:
DEPFLAGS=-MD -Mo $(OUTDIR)/$*.Td
POSTCOMPILE=#mv -f $(OUTDIR)/$*.Td $(OUTDIR)/$*.d && touch $#
VPATH=../src
OUTDIR=../out
SOURCES:=$(notdir $(wildcard ../src/*.c))
OBJECTS:=$(SOURCES:%.c=$(OUTDIR)/%.o)
all: $(OBJECTS) $(OBJECTS:%.o=%.d)
$(OUTDIR)/%.o : %.c
$(OUTDIR)/%.o : %.c $(OUTDIR)/%.d
#$(CC) $(DEPFLAGS) -c $< -o $#
#$(POSTCOMPILE)
$(OUTDIR)/%.d : ;
.PRECIOUS: $(OUTDIR)/%.d
Directory structure looks like:
src
contains file.c
out
empty, after make: contains file.o and file.d
make
contains the makefile
When I call the makefile everything works fine and two files are generated: file.o and file.d
However, when I delete file.d nothing happens. I would expect that make finds a missing dependency for file.c and starts a rebuild. Why doesn't it happen?
Make version is 3.81 built for i386-pc-mingw32 under Windows 7.
Marking a file as .PRECIOUS does not remove all aspects of it's "intermediateness". All it does is prevent it from being deleted, but this feature of intermediate files is still in effect:
If an ordinary file b does not exist, and make considers a target that depends on b, it invariably creates b and then updates the target from b. But if b is an intermediate file, then make can leave well enough alone. It won’t bother updating b, or the ultimate target, unless some prerequisite of b is newer than that target or there is some other reason to update that target.
This is why your .d file is not recreated. In order for it to be recreated you need to ensure it's not an intermediate file. Fortunately this is trivial to do: you just need to mention the files explicitly somewhere as a target or prerequisite. You can do it like this:
all: $(OBJECTS) $(SOURCES:%.c=$(OUTDIR)/%.d)
Or if you prefer like this:
depends: $(SOURCES:%.c=$(OUTDIR)/%.d)
which would allow you to run make depends to update the dependency files, if you wanted to.
I'll just point out in passing that this method of managing dependencies is considered outdated. There's a better, more advanced way it can be done described here among other places.
(I'll be a horrific necromancer here, but I've ran into same problem, and found that actual issue isn't one mentioned in answer or comments here)
Dependency rule generated by compiler by default sports file name with ALL suffixes replaced by single suffix .o and path removed. Which doesn't match the pattern of rule in makefile.
For gcc 4.x and later correct options would be
$(OUTDIR)/%.o : %.c $(OUTDIR)/%.d
#$(CC) -MF $(OUTDIR)/$*.Td -MT $# -c $< -o $#
Mo flag no longer exist, you have to use only MF flag to specify dependency file name.MT flag allows to provide a literal line for target name.
I have found the following code from http://mrbook.org/tutorials/make/:
CC=g++
CFLAGS=-c -Wall
LDFLAGS=
SOURCES=main.cpp hello.cpp factorial.cpp
OBJECTS=$(SOURCES:.cpp=.o)
EXECUTABLE=hello
all: $(SOURCES) $(EXECUTABLE)
$(EXECUTABLE): $(OBJECTS)
$(CC) $(LDFLAGS) $(OBJECTS) -o $#
.cpp.o:
$(CC) $(CFLAGS) $< -o $#
My question is: Do we really need the line
all: $(SOURCES) $(EXECUTABLE)
?
What will be wrong if I do not keep the line or does it have some other purpose?
From the perspective of the behavior of make
The line:
all: $(SOURCES) $(EXECUTABLE)
is special in this case as it is the first target defined in the Makefile. Since there is no .DEFAULT_GOAL defined that makes it the default goal and so what make will aim to build if no target is specified on the command line.
In this specific case all its presence is serving it to allow you to use the command make all.
It is also ensuring that make all and make will not work as expected if there is a file called all present as make will consider that when evaluating the rule.
The normal practice is to define such targets as phony so that they will not be effected by the presence of a file with their name.
For example: .PHONY : all.
The reasons it does not serve any other purpose in this example are:
$(SOURCES) is a redundant dependency as $(EXECUTABLE) depends on it already through $(OBJECTS) which depends on $(SOURCES) through the suffix rule .cpp.o:
The other dependence $(EXECUTABLE) is the next target defined, so removing the all target will make $(EXECUTABLE) the default goal.
From the perspective of convention
The all target is a Standard Target and "should be the default target".
Some users and tools may assume the target all is present.
For example the IDE eclipse by default expects Makefiles to have the target all which it will use for incremental builds. Without the all target extra configuration is required for eclipse be able to build the project.
When you run make without specifying a target, it will run the first target in the file. It's a convention that this target is responsible for creating a deployable file (package or executable) for installation on your platform. The name all is simply a convention (at least in the GNU project), and could be anything you like (for example $(name), compile or deployable).
I don't know why the $(SOURCES) variable appears as a prerequisite of the all target. That's pretty useless.
In your case, since you have only one executable, it's not necessary to have all. But many makefiles built multiple "final targets" which do not depend on each other. In that case it's traditional to have an all target which depends on them all so you can build them all with one command.
And since it's traditional, people put it into makefiles with only one target as well, for consistency.