makefile parallel clean+compile issue - makefile

I have a simple makefile with 3 build rules:
clean (that cleans the .o)
debug (compiles my code with debgging stuff)
release (compiles my code with optimization stuff)
sometimes I want to switch between debug mode and release so I would issue this
make clean debug -j8
or
make clean release -j8
that has a drawback because while it's doing the clean stuff, the -j8 allows make to jump some command since the .o are still there Then those .o are removed by the clean rule and the compiler complains because it can't find those .o
I could do something like
make clean; make debug -j8
but since I use an odd makefile in another dir, the command becomes
make -C ../src -f nMakefile clean ; make -C ../src -f nMakefile -j8 release
that is more annoying. I was wondering if there was an hiddedn-guru-mode-rule that allows me to do it in one line
Hope it's clear enough...

I needed to solve this very same problem, and the solution I came up was to parse the MAKECMDGOALS for clean, and dispatch a shell command to do the actual cleaning work; RATHER than clean the build as a target. This way, any MAKECMDGOALS that include "clean" will clean the build as part of that build, first, sequentially, rather than clean running asynchronously as its own target.
-include $(deps)
bin/%.o : %.cpp
#mkdir -p $#D
g++ $(flags) $(includes) -MMD -c $< -o $#
.PHONY : clean
clean:
#echo rm -rf bin/
ifneq ($(filter clean,$(MAKECMDGOALS)),)
$(shell rm -rf bin/)
endif

As I stated above, the normal practice is to have different sub directories for the object files. As you are running in parallel I would think you need to enforce serial execution so that clean is completed before release. One way of doing it could be:
clean_release: clean
+#$(MAKE) -s --no-print-directory release
or if you prefer
clean_release:
+#$(MAKE) -s --no-print-directory clean && $(MAKE) -s --no-print-directory release

Related

Need to run make twice after make clean

All I want to do is gather source files from different source directories into one folder and then do the build of those make files. After a make clean I have to run make command twice to do the build.
So first I run make clean, then i run make, which throws an error saying no -o files found. But when iIrun the make again, the build goes through and generates the build files.
My makefile looks like below
.PHONY: dirs all clean
all: dirs $(OBJ_DIR) $(OBJ_FILES)
"$(CC) -rvn fLib.a $(OBJ_FILES)
# clean build target. Remove all files without reporting errors in case they don't exist.
clean:
#rm -rf fLib.a $(OBJ_DIR)
# Build target for creating flat header file folder for SharedTrackerAPI (FLAT_INC_DIR)
# due to too long paths in Windows 7 build
dirs:
#echo 'Making flat source and header folders.'
#mkdir -p $(OBJ_DIR)
#for f in $(SRC_FILES); do cp $$f $(OBJ_DIR); done
#mkdir -p $(FLAT_INC_DIR)
#OLD_CWD=$(CURDIR)
#cd $(FLAT_INC_DIR)
#find $(STA_RADAR_TRACKER_IFACE) -name '*.h' | xargs -i cp -l {} $(FLAT_INC_DIR)
#cd $(OLD_CWD)
$(OBJ_DIR)/%.o: $(OBJ_DIR)/%.cpp
"$(TASKING_CTC_BIN)"/cctc.exe $(CXXFLAGS) -c -o $# $< $(CC_INCLUDE_PATH)
What am I doing wrong that I have to run make twice after a make clean.
The problem is that although dirs will place the source files in the flat source directory, Make doesn't know that. Before it executes the dirs rule, it has already determined that it knows no way to build the object files.
The quick and dirty solution is to tell Make "trust me, it'll be fine"; one way to do that is to modify the object rule like this:
$(OBJ_DIR)/%.o:
"$(TASKING_CTC_BIN)"/cctc.exe $(CXXFLAGS) -c -o $# $(OBJ_DIR)/$*.cpp $(CC_INCLUDE_PATH)
If you stop there, you will have a working solution.
If you want a more clean, efficient and flexible makefile, you must rethink the approach to finding source files. I see no good reason to use the flat source file approach, but if you really want to use it, here is a good way:
vpath %.cpp $(dir $(SRC_FILES))
$(OBJ_DIR)/%.cpp: %.cpp
#cp $< $#
Now you can get rid of dir and use your unmodified object rule:
$(OBJ_DIR)/%.o: $(OBJ_DIR)/%.cpp
"$(TASKING_CTC_BIN)"/cctc.exe $(CXXFLAGS) -c -o $# $< $(CC_INCLUDE_PATH)
How to handle the header files (FLAT_INC_DIR) is up to you, but I recommend vpath again.

Makefile skips dependency

I've created a makefile for my little project
.SUFFIXES:
%.cpp:
$(COMP) -c -o $(subst .cpp,.o,$#) $(SRCDIR)$# $(CFLAGS)
platformL: COMP:=gcc
platformL: $(FILES)
$(COMP) -o $(NAME) $(subst .cpp,.o,$(FILES)) $(CFLAGS)
rm $(subst .cpp,.o,$(FILES))
platformW: COMP:=wine gcc
platformW: $(FILES)
$(COMP) -o $(NAME).exe $(subst .cpp,.o,$(FILES)) $(CFLAGS)
rm $(subst .cpp,.o,$(FILES))
default: platformL platformW
echo Done!
Everything worked fine until I branched to 2 different platforms, 'make' command executes only my platformL branch. After spending some time with it I discovered that adding '.PHONY' won't fix the problem. Also, it appears that only the first branch from the top gets executed (I have put the lines of platformW before platformL and only Windows compilation was performed).
How can I make it execute both branches?
Make always builds the first explicit target (and all prerequisites of the first explicit target) in the makefile, by default. That's all it will build by default.
You can either specify multiple things to build on the command line, like make platformL platformW, or you can add a new first target that depends on all the other targets you want built. By tradition that target is named all but you can call it whatever you want:
all: platformL platformW
.PHONY: all
...
platformL: ...
...
platformW: ...

How to make a 'rebuild' rule out of 'build' and 'clean' in Makefile?

I have a Makefile containing rules for building a small project and cleaning it. For example:
CC:=gcc
LD:=gcc
SOURCES:=$(wildcard src/*.c)
OBJS:=$(SOURCES:src/%.c=build/%.o)
TARGET:=bin/program
all: $(TARGET)
$(TARGET): $(OBJS)
#mkdir -p bin
$(LD) $+ -o $#
build/%.o: src/%.c
#mkdir -p build
$(CC) $+ -c -o $#
clean:
rm -rf $(OBJS)
rm -rf $(TARGET)
rmdir bin
rmdir build
.PHONY: clean all
I am now interested in creating a rule rebuild which would perform clean and all in that order. I do not see how properly achieve the correct ordering.
The solutions I have seen are wrong to my knowledge.
rebuild: clean all
.PHONY: rebuild
Is naturally wrong, because there is no guarantee that dependencies are actually performed in the order of their appearance. all may execute before clean.
I have seen answers suggesting order-only dependencies, e.g.
rebuild: | clean all
.PHONY: rebuild
To my knowledge, this does not solve the problem. If you say a: | b c it means that a depends on b and c, but if b or c is taken, it does not force executing the a rule. It has nothing to do with ordering the dependencies.
The only option I see right now is launching a new instance of make, by having
rebuild : clean
make build
I would really like to avoid launching a new make instance for doing something simple like that!
I did some reasearch on SO. I have seen similar questions but no correct answer. To my knolwedge, making a target .PHONY or using order-only dependencies is not a solution.
First, it's not true that in a rule:
rebuild: clean all
that all could be built before clean when running serially (that is, without parallelism -j enabled). Make does always build prerequisites in the order they are listed in the makefile (there is a special case for the rule containing the recipe but that's not relevant here). When parallel builds are used then make still walks the dependency tree in the same order but because rules are built in parallel they may not be started in the same order.
However, you're right that this rule is not a great idea for other reasons (directory caching, etc.)
I recommend you use recursive make invocations to do this:
.PHONY: rebuild
rebuild:
$(MAKE) clean
$(MAKE) all
There is a way to do it without recursion; the price is a small amount of redundancy:
.PHONY: rebuild
$(TARGET) rebuild: $(OBJS)
#mkdir -p bin
$(LD) $+ -o $(TARGET) # note that I have replaced $# with $(TARGET)
rebuild: | clean
You can use MAKECMDGOALS to conditionally add a dependency between clean and all when the goal is rebuild. Something like this:
ifeq (rebuild,$(findstring rebuild,${MAKECMDGOALS}))
all: clean
rebuild: all
endif
Now, I don't really see the benefit of doing this in the makefile when there are other trivial and safe ways to do it (just "make clean && make all" might be a better option)

Running all targets at once in a Makefile

I have a Makefile as below
all:
bison -d -v parser.y
flex -o parser.lex.c parser.lex
gcc -o cparser parser.lex.c parser.tab.c -lfl -lm
clean:
rm parser.tab.h parser.tab.c parser.output parser.lex.c
When i ran make in terminal then it only runs target all.
I have tried by adding
.PHONY:all clean and .PHONY:clean even though it only runs all .
What are the changes i should make in Makefile so that it runs all target ?
System Configurations:
Ubuntu 14.10
GNU Make 4.0
You want to run all and clean when you just type make? That's usually not useful since that would clean up what you just built (though in your case I see that your clean target doesn't actually do that).
To do this you want something like.
all: cparser clean
cparser:
bison -d -v parser.y
flex -o parser.lex.c parser.lex
gcc -o cparser parser.lex.c parser.tab.c -lfl -lm
clean:
rm parser.tab.h parser.tab.c parser.output parser.lex.c
Though that is still a fairly poor makefile and doesn't take advantage of any of the benefits that make affords you (like intelligently rebuilding things only when their prerequisites change).
I would suggest you probably want something more like this.
.PHONY: all
all: cparser
parser.lex: parser.y
bison -d -v parser.y
parser.lex.% parser.tab.%: parser.lex
flex -o parser.lex.c parser.lex
cparser: parser.lex.c parser.tab.c
gcc -o cparser parser.lex.c parser.tab.c -lfl -lm
.PHONY: clean
clean:
rm parser.tab.h parser.tab.c parser.output parser.lex.c cparser
Assuming I recall correctly that flex will generate both the parser.lex.c and parser.tab.c files at the same time.
Note also that the prerequisites are chained and that clean now cleans the binary as well. You don't need to run make clean to have things rebuild correctly this way.
If you really want to make all targets in the Makefile (you really don't want to, this is usually not how you use makefiles), you can add a target that has all the other targets of the Makefile as it's prerequisites:
.PHONY: all_targets
all_targets: all clean
.PHONY: all
all:
# ... command script
.PHONY: clean
clean:
# ... command script
In this particular example, making all followed by clean is complete nonsense, since all would build all files and clean would remove them. If you want to clean all files before building all (and you don't want to do this either), you can in fact just turn around the prerequisites of all_targets:
.PHONY: all_targets
all_targets: clean all
.PHONY: all
all:
# ...
.PHONY: clean
clean:
# ...
Again, this is really not how you want to use make, since it is designed to skip all the unnecessary work in the build process (that is, not recompiling those files that do not need to be recompiled, etc).
Also remember that you can call make with more than just one target to build. So in your example, you could also build all targets by executing
make all clean
Note that the targets will be built in the order they were passed to the command line, so the first target to be built here is all, followed by clean (which again is nonsense).
If you want to read more about how to correctly use makefiles, I suggest you to grab the book Managing Projects with GNU Make. It features a quite comprehensive explanation of almost all (GNU) Make features.
Edit: As tripleee pointed out, the all_targets target in the above examples should of course be a phony target.

Pre-build step in makefile

How can I run a script, which must execute before all other makefile commands? And it will be nice (but not mandatory) to the script is not executed if there is nothing to build.
I've searched SO and Google, but can't find anything.
I have this workaround:
# myscript.bat output is empty
CHEAT_ARGUMENT = (shell myscript.bat)
CFLAGS += -DCHEAT_ARGUMENT=$(CHEAT_ARGUMENT)
AFLAGS += -DCHEAT_ARGUMENT=$(CHEAT_ARGUMENT)
But it's very ugly. Is there other way to run "pre-build step" in makefile?
I propose two solutions. The first mimics what NetBeans IDE generates:
CC=gcc
.PHONY: all clean
all: post-build
pre-build:
#echo PRE
post-build: main-build
#echo POST
main-build: pre-build
#$(MAKE) --no-print-directory target
target: $(OBJS)
$(CC) -o $# $(OBJS)
clean:
rm -f $(OBJS) target
The second one is inpired by what Eclipse IDE generates:
CC=gcc
.PHONY: all clean
.SECONDARY: main-build
all: pre-build main-build
pre-build:
#echo PRE
post-build:
#echo POST
main-build: target
target: $(OBJS)
$(CC) -o $# $(OBJS)
#$(MAKE) --no-print-directory post-build
clean:
rm -f $(OBJS) target
Note that in the first one, pre and post builds are always called regardless of whether the main build is determined to be up to date or not.
In the second one, the post-build step is not executed if the state of the main build is up to date. While the pre-build step is always executed in both.
Depending on your make version, something like the following should at least avoid running dozens of times if CFLAGS and AFLAGS are evaluated dozens of times:
CHEAT_ARG := $(shell myscript)
Note the colon.
This runs exactly once. Never more than once, but also never less than once. Choose your own tradeoffs.
You could add a special target to your Makefile and have all your build rules depend on that:
run-script:
myscript
.o.c: run-script
$(CC) $(CFLAGS) -o $# $<
.o.S: run-script
$(AS) $(AFLAGS) -o $# $<
Depending on what your script actually does, putting it to run once in a stage before the Makefile (configure stage in autoconf terms) could make even more sense (and be less work).
What you are proposing seems a bit "un-make-like". Why not just run the command in whatever makefile target you need it to go before?
Example, if you need it to run before linking foo:
foo: ${OBJS}
my-command-goes-here
${CC} -o $# ${OBJS} ${LIBS}
Thank you for answers. ndim helped me much, asveikau. The final file is one binary executable, so I can use now something like this:
run-script:
myscript
$(AXF_FILE): run-script $(OBJ_DIRS) $(OBJ_FILES)
$(LINK) #......
It will run myscript once. {AXF_FILE} value depends on myscript and I must run it before. And in this case myscript runs always, not only when rebuild is needed.
After, The Simplest Answer came to my mind:
all: run-script $(AXF_FILE)
That's all ;) (Of course, any target can be used instead of "all")
Edit: this method execute script after $(AXF_FILE) is calculated too. So it's possible to get wrong value of AXF_FILE.
Now only the first answer by ndim works as I need.

Resources