I have some variables defined in my Makefile which I pass to a compiler when I need to build my software. However, I need to make sure that these variables satisfy certain boolean identities at the level of the Makefile. I am currently doing something like the following.
SHELL := /bin/bash
NUMBER := 1
all: test.x
%.x: %.c ASSERT
#echo "Compiling $< -> $# with NUMBER=$(NUMBER)."
#gcc $< -D NUMBER=$(NUMBER) -o $#
ASSERT: Makefile
#if (( $(NUMBER) < 0 || $(NUMBER) > 4 )); then \
echo "ERROR: NUMBER must be positive and less than 4."; \
exit 1; \
fi
#echo "Ran ASSERT. No errors found."
The problem with the above setup is that it always runs the compilation recipe even if the target already exists and nothing has changed.
How can I run Makefile assertions properly so that targets are made only when there are changes in the Makefile variables?
You have said that all your object files depend on the prerequisite ASSERT. That prerequisite does not exist, so make will try to create it. Once make tries to create it, it will be considered "very new" and so any target that depends on it will be considered out of date and rebuilt.
Then the next time make runs, it sees that the prerequisite ASSERT does not exist, so it will try to create it and it will be considered "very new". Etc. etc.
If you don't want everything to be rebuilt, then your recipe for ASSERT must create the file ASSERT when it succeeds:
ASSERT: Makefile
#...
#touch $#
#echo "Ran ASSERT. No errors found."
Just remember that this won't have any impact if you run make NUMBER=10 rather than editing the makefile. Since the makefile isn't changed, ASSERT won't be out of date and so its recipe won't be invoked.
ETA
If all you want to do is verify that no one can use a NUMBER value that's illegal, ever, you can just test that directly with make rather than using a recipe to do it:
SHELL := /bin/bash
NUMBER := 1
$(if $(filter $(NUMBER), 0 1 2 3 4),\
$(info NUMBER is good),\
$(error ERROR: NUMBER must be positive and less than 4.))
all: test.x
%.x: %.c ASSERT
#echo "Compiling $< -> $# with NUMBER=$(NUMBER)."
#gcc $< -D NUMBER=$(NUMBER) -o $#
As your assertion is not related to *.c, can you make all depend on ASSERT:
SHELL := /bin/bash
NUMBER := 1
all: ASSERT test.x
%.x: %.c
#echo "Compiling $< -> $# with NUMBER=$(NUMBER)."
#gcc $< -D NUMBER=$(NUMBER) -o $#
ASSERT: Makefile
#if (( $(NUMBER) < 0 || $(NUMBER) > 4 )); then \
echo "ERROR: NUMBER must be positive and less than 4."; \
exit 1; \
fi
#echo "Ran ASSERT. No errors found."
Related
I have a makefile which produces an executable from several object files and I include a version number in each object file as it is compiled. However, I want the version number to be incremented only when a particular object file is created (ptarget below, the one containing main). I tried to do this using a conditional statement in the recipe:
ptarget:=$(addsuffix .obj,$(ouf))
%.obj : %.cpp
$(CXX) $(CXXFLAGS) $< -Fo$#
$(info $(ptarget))
$(info $#)
ifeq ($#, $(ptarget))
perl $(perlDir)versionBump/bump.pl -inc -f $(versionFile)
endif
I can see from the info commands that only when ptarget is built that $# == $(ptarget) -- I also tried using strip command to make sure no hidden whitespace, but the perl command to increment the version is never executed, it starts with a tab.
Just to add, this all works just fine without the conditional but the increment happens multiple times during a build, which is what I am trying to avoid. This example suggests it should work but...
This is a very common misunderstanding about make. ifeq is a make statement and is evaluated when make parses the Makefile. What you need is a conditional that gets evaluated when the recipe is executed by the shell, that is a shell if:
%.obj : %.cpp
$(CXX) $(CXXFLAGS) $< -Fo$#
$(info $(ptarget))
$(info $#)
if [ "$#" = "$(ptarget)" ]; then \
perl $(perlDir)versionBump/bump.pl -inc -f $(versionFile); \
fi
Do not forget the line continuations (the trailing \).
I have a makefile that for various reasons relies on a supporting python script to run every time and grab files from several external locations, copy into working directory, and run through a separate preprocessor before compiling.
This makefile must be able to be run in parallel (-j8) so the order of processing cannot be guaranteed.
In trying to explicitly specify prerequisites, I have created a situation where make skips all object files, goes straight to linking, and fails because the necessary objects do not exist. On a second run, all the objects already exist (the preprocess step skips the files that already exist) and all the files are compiled and linked properly.
When run without -j# everything works fine, but the moment I add -j2, the skipping begins.
Following is an example make file:
GEN_FILES := file1.cpp file2.cpp file3.cpp
CXX_FILES := bin_main.cpp $(GEN_FILES)
OBJ_FILES := $(patsubst %.cpp,%.o,$(CXX_FILES))
.PHONY : all clean prepare
all : bin_file
prepare :
# Copy and preprocess all source files
[ -f file1.cpp ] || cp d1/file1.cpp .
[ -f file2.cpp ] || cp d2/file2.cpp .
[ -f file3.cpp ] || cp d3/file3.cpp .
$(OBJ_FILES) : prepare
bin_file : $(OBJ_FILES)
[ -f file1.o ] && [ -f file2.o ] && [ -f file3.o ] && touch bin_file
%.o : %.cpp
#echo "Compiling $<..."
[ -f $< ] && touch $#
clean :
$(RM) *.o
$(RM) file*
$(RM) bin_file
How can I get this to build in one go, first running prepare to collect all files and then compiling and linking as necessary?
As code_fodder mentions the issue is the creation of the source files.
Basically what happens is, you have not told make how to create these source files, so as far as make knows they don't exist and there's no way to create them. So when make wants to build, for example, file1.o it looks at your pattern rule and finds it could build file1.o from file1.cpp. So then it looks for how to build file1.cpp. No file1.cpp exists, and there is no rule that make knows of that will build it, so make ignores that pattern rule as not matching.
Then make sees the target:
$(OBJ_FILES) : prepare
so it thinks there's no recipe needed to create the object files, and just runs the link line. The next time through, make sees the prepared source files (from the previous build) and then it can use your pattern rule.
If you change your pattern rule to a static pattern rule, where you explicitly tell make exactly what rule to use instead of providing it with a possible rule to use that it can ignore if it doesn't match (which is what a pattern rule is), you'll see the error:
$(OBJ_FILES): %.o : %.cpp
#echo "Compiling $<..."
sleep 1
[ -f $< ] && touch $#
will tell you:
make: *** No rule to make target 'file1.cpp', needed by 'file1.o'. Stop.
Remember, make is looking for a matching pattern rule BEFORE it actually builds anything: it doesn't want to build every possible prerequisite of every possible matching pattern rule, to decide whether or not at the end of it the rule can be used. The rule is matched based on the current state of the filesystem plus rules you have given make about changes it could make. Make has no idea that if it were to invoke the prepare target the source files it was looking for would magically come into existence.
Your basic problem is that this statement is the wrong dependency relationship:
$(OBJ_FILES) : prepare
It's not really true that the object files depend on prepare; what's true is that the PREPARED SOURCE FILES depend on prepare. The object files depend only the "prepared" source files, as your pattern rules shows. This rule should be written, instead:
$(GEN_FILES): prepare
If you do this with -j everything will wait as you want.
Yeah, this gets messy / difficult. The problem you have is that you can specify prerequisite lists - that can work in order, but as soon as you start to use -j then make can start processing prerequisites in any old order. So bin_file requires $(OBJ_FILES) which require prepare. Then %.o requires the same named %.cpp file - which it can do for main.o, but not the filex.o since they don't exist yet - but it tries anyway and fails - in the mean time make (in parallel) is potentially starting to generate the .cpp files, but by this time its too late...etc...
My Prerequisites Build Pattern
I use a very specific prerequisites pattern of my own design - some might frown upon - but I have carefully considered this over the years and found it to be optimal for me.
I create a rule called build or something - which requires build_prerequisites target and then calls make to do the actual build once this is complete:
.PHONY: build
build: build_prerequisites
build:
#echo "start_build"
#$(MAKE) bin_file
This means that build_prerequisites is always run first before the recipe runs. You cant seem to achieve the same forcing of order (at least not easily) using just dependencies. I.e. a list of dependencies can be run in any order with -j, but the rule recipe is always run last.
Now we have this pattern we can fill in the rest. First the build_prerequisites target which does your file generation - I am using echo in my example because I don't have your python script:
.PHONY: build_prerequisites
build_prerequisites:
#echo "build_prerequisites"
echo "create file1" > file1.cpp
echo "create file2" > file2.cpp
echo "create file3" > file3.cpp
Finally add in the c++ compile and link stages - these will be run with the single recursive make call from build - i.e. $(MAKE) bin_file (again I am using echo to create the files in my example):
%.o : %.cpp
#echo "compiling: $<"
##echo "$(CXX) $(SRC_INCLUDES) $(LIB_INCLUDES) $(CXXFLAGS) -c $< -o $#"
#echo "touch" > $#
bin_file : $(OBJ_FILES)
#echo "linking: $<"
#echo $(CXX) $(SRC_INCLUDES) $^ $(LIB_INCLUDES) $(LDFLAGS) -o $#
#echo "touch" > $#
Output
Here is the output from my test program (using echo) and main.cpp already exists usingn -j10:
make -j10
build_prerequisites
echo "create file1" > file1.cpp
echo "create file2" > file2.cpp
echo "create file3" > file3.cpp
start_build
make[1]: Entering directory '/mnt/d/software/ubuntu/make'
compile: bin_main.cpp
compile: file1.cpp
compile: file2.cpp
compile: file3.cpp
link: bin_main.o
g++ bin_main.o file1.o file2.o file3.o -o bin_file
make[1]: Leaving directory '/mnt/d/software/ubuntu/make'
Note: if I put a sleep 1 in the "compile" rule - this still takes only 1 second for all 4 files to compile.
Put it all together
GEN_FILES := file1.cpp file2.cpp file3.cpp
CXX_FILES := bin_main.cpp $(GEN_FILES)
OBJ_FILES := $(patsubst %.cpp,%.o,$(CXX_FILES))
###### STAGE 1
.PHONY: build
build: build_prerequisites
build:
#echo "start_build"
#$(MAKE) bin_file
.PHONY: build_prerequisites
build_prerequisites:
#echo "build_prerequisites"
copy_and_pp_files.py $(CXX_FILES) $(SEARCH_DIRS) .
copy_and_pp_files.py $(CFG_FILES) $(SEARCH_DIRS) .
###### STAGE 2
%.o : %.cpp
#echo "compiling: $<"
#$(CXX) $(SRC_INCLUDES) $(LIB_INCLUDES) $(CXXFLAGS) -c $< -o $#
bin_file : $(OBJ_FILES)
#echo "linking: $<"
#$(CXX) $(SRC_INCLUDES) $^ $(LIB_INCLUDES) $(LDFLAGS) -o $#
###### OTHER RULES
.PHONY: clean
clean :
#$(RM) *.o
#$(RM) file*
I have attempted to use your actual code, but I have no way to test this so there may be a bug in there. I split it up into 2 "stages" for clarity. Stage 1 is done in your makeor make build call, then state 2 is done in the recursive make call in the build recipe.
Assume I have a build-target foo:
foo:foo.c
$(CC) $(CFLAGS) $(ARGS) -c foo.c -o foo
Now, ARGS is something that I pass on the command line:
$ make ARGS:=-DX=1 foo
So, I need to bypass make's cleverness, because the foo target does not only depend on which files have changed, but also on the value of ARGS.
Is there something in make to do this? My hack (see answer) doesn't seem to be the most elegant but it works. Anything better?
Here is a general solution to your specific problem.
You want to be able to depend on a variable as a prerequisite. That is, you can make it a prerequisite to any target in your makefile, and when the value of the variable changes, you rebuild those targets.
Here is a function that does that, you use this function to declare a variable to be dependable, and then you can use it as a prerequisite.
Note that if the variable is not used on the command line, it will still mean that variable still has a value, namely, the empty string.
define DEPENDABLE_VAR
.PHONY: phony
$1: phony
#if [[ `cat $1 2>&1` != '$($1)' ]]; then \
echo -n $($1) > $1 ; \
fi
endef
#declare ARGS to be dependable
$(eval $(call DEPENDABLE_VAR,ARGS))
foo:foo.c ARGS
$(CC) $(CFLAGS) $(ARGS) -c foo.c -o foo
In fact, we could omit the need for "declaration", and just write a similar function that will make all variables dependable by default. But I don't like that. I prefer that the users that modify makefiles I write, declare their intentions explicitly. It is good for them :)
My solution was to create a dummy phony target:
.PHONY:dummy
dummy:
#:
and have foo depend on dummy if ARGS is nonempty:
foo:foo.c $(patsubst %,dummy,$(ARGS))
Note on Mark's excellent answer
The bare necessities of Mark's answer are actually very simple. It really boils down to just:
.PHONY: phony
ARGS: phony
#if [[ `cat ARGS 2>&1` != '$(ARGS)' ]]; then echo -n $(ARGS) >ARGS; fi
The rest of his code is just to let you reproduce the recipe for other variable names without repeating yourself. Useful in practice, but the above version will help you see what's going on more easily.
In fact, my answer can even be made general (like Mark's) for any variable name, but in a less complicated way as follows:
.PHONY: phony
.ARG~%: phony
#if [[ `cat .ARG~$* 2>&1` != '$($*)' ]]; then echo -n $($*) >.ARG~$*; fi
Then you simply add .ARG~MYVARNAME as a dependency to any target to make that target depend on variable MYVARNAME.
Note that the dot in front of .ARG~% simply causes it to create a dependency-tracking file that is 'hidden' in linux.
So in your case, you would do:
foo: foo.c .ARG~ARGS
$(CC) $(CFLAGS) $(ARGS) -c foo.c -o foo
I don't understand how the other solutions are supposed to work. If the ARGS target is .PHONY or depends on a .PHONY, then it will always be run, right?
Here is my solution using the $(file) function in newer versions of gmake:
.PHONY: FORCE
define DEPENDABLE_VAR
$(1):
echo -n $($(1)) > $(1)
ifneq ("$(file <$(1))","$($(1))")
$(1): FORCE
endif
endef
#declare ARGS to be dependable
$(eval $(call DEPENDABLE_VAR,ARGS))
foo: foo.c ARGS
touch foo
And the result:
~/stuff/makevars> make foo ARGS=1
echo -n 1 > ARGS
touch foo
~/stuff/makevars> make foo ARGS=1
make: 'foo' is up to date.
~/stuff/makevars> make foo ARGS=2
echo -n 2 > ARGS
touch foo
~/stuff/makevars> make foo ARGS=2
make: 'foo' is up to date.
So I am writing a makefile that will take some files (*.in) as input to my C++ program and compare their output (results.out) to given correct output (*.out).
Specifically I have files t01.in, t02.in, t03.in, t04.in, and t05.in.
I have verified that $TESTIN = t01.in t02.in t03.in t04.in t05.in.
The problem is that it seems to run the %.in: %.out block only for three of these files, 1,3, and 4. Why is it doing this?
OUTPUT = chart
COMPILER = g++
SOURCES = chart.cpp
HEADERS =
OBJS = $(SOURCES:.cpp=.o)
TESTIN = tests/*.in
all: $(OUTPUT)
$(OUTPUT): $(OBJS)
$(COMPILER) *.o -o $(OUTPUT)
%.o: %.cpp
clear
$(COMPILER) -c $< -o $#
test: $(TESTIN)
%.in: %.out
./$(OUTPUT) < $# > tests/results.out
printf "\n"
ifeq ($(diff $< tests/results.out), )
printf "\tTest of "$#" succeeded for stdout.\n"
else
printf "\tTest of "$#" FAILED for stdout!\n"
endif
Additionally, if there is a better way of accomplishing what I am trying to do, or any other improvements I could make to this makefile (as I am rather new at this), suggestions would be greatly appreciated.
EDIT: If I add a second dependency to the block (%.in: %.out %.err), it runs the block for all five files. Still no idea why it works this way but not the way before.
First, I don't see how TESTIN can be correct. This line:
TESTIN = tests/*.in
is not a valid wildcard statement in Make; it should give the variable TESTIN the value tests/*.in. But let's suppose it has the value t01.in t02.in t03.in t04.in t05.in or tests/t01.in tests/t02.in tests/t03.in tests/t04.in tests/t05.in, or wherever these files actually are.
Second, as #OliCharlesworth points out, this rule:
%.in: %.out
...
is a rule for building *.in files, which is not what you intend. As for why it runs some tests and not others, here is my theory:
The timestamp of t01.out is later than that of t01.in, so Make decides that it must "rebuild" t01.in; likewise t03.in and t04.in. But the timestamp of t02.out is earlier than that of t02.in, so Make does not attempt to "rebuild" t02.in; likewise t05.in. The timestamps of t02.err and t05.err are later than those of t02.in and t05.in, respectively, so when you add the %.err prerequisite, Make runs all tests. You can test this theory by checking the timestamps and experimenting with touch.
Anyway, let's rewrite it. We need a new target for a new rule:
TESTS := $(patsubst %.in,test_%,$(TESTIN)) # test_t01 test_t02 ...
.PHONY: $(TESTS) # because there will be no files called test_t01, test_t02,...
$(TESTS): test_%: %.in %.out
./$(OUTPUT) < $< > tests/results.out
Now for the conditional. Your attempted conditional is in Make syntax; Make will evaluate it before executing any rule, so tests/result.out will not yet exist, and variables like $< will not yet be defined. We must put the conditional inside the command, in shell syntax:
$(TESTS): test_%: %.in %.out
./$(OUTPUT) < $< > tests/results.out
if diff $*.out tests/results.out >/dev/null; then \
echo Test of $* succeeded for stdout.; \
else echo Test of $* FAILED for stdout!; \
fi
(Note that only the first line of the conditional must begin with a TAB.)
I want to do something like this, where I want to run a svn commit if a file has changed. The file has a timestamp which always changes. So if anything more than timestamp changes , then I want to commit the file.
The makefile will have something similar to this. But the If condition is not working properly. It is getting executed even when if is not satisfied. Can someone help me out what is the problem in If here.
UPDATE_STATE_FILE :
$(eval NO_LINES_CHANGES_IN_STATE = $(shell svn di STATE/build.state --diff-cmd=diff -x --normal | grep "^[<>]" | wc -l))
#echo $(NO_LINES_CHANGES_IN_STATE)
ifneq ($(strip $(NO_LINES_CHANGES_IN_STATE)), 2)
ifneq ($(strip $(NO_LINES_CHANGES_IN_STATE)), 0)
#echo $(NO_LINES_CHANGES_IN_STATE)
$(SVN) commit;
$(SVN) update;
endif
endif
You cannot mix make conditionals inside command rules. Make conditionals are like preprocessor statements in C or C++; they are handled as the file is read in, before any processing (like running rules) is performed.
If you want conditionals inside the rules you have to write the rule using shell conditionals, not make conditionals:
UPDATE_STATE_FILE :
#NO_LINES_CHANGES_IN_STATE=`svn di STATE/build.state --diff-cmd=diff -x --normal | grep "^[<>]" | wc -l`; \
echo $$NO_LINES_CHANGES_IN_STATE; \
if [ $$NO_LINES_CHANGES_IN_STATE -ne 2 ] && [ $$NO_LINES_CHANGES_IN_STATE -ne 0 ]; then \
echo $$NO_LINES_CHANGES_IN_STATE; \
$(SVN) commit; \
$(SVN) update; \
fi
After reading the answer given by #MadScientist, I came up with a different approach.
Not knowing if shell conditionals will work in any environment (windows vs. linux), I wrapped the rules inside the conditional instead of having a conditional inside the rule. i.e.
ifdef MY_FLAG
%.o: %.c
$(CC) -o $# -c $^ $(CFLAGS)
else
%.o: %.c
#$(CC) -o $# -c $^ $(CFLAGS)
endif
Good luck to anyone visiting here.
EDIT
As pointed out by James Moore in the comments, this different approach requires care and notice as to when and how variables are defined in relation to the placement of the if statement in the control flow.