What does this code in Makefile exactly do and is there a way to avoid using it and still make the code compile successfully?
work-obj93.cf: sources.mk
$(RM) $#
(for i in $(SOURCES); do \
$(GHDLC) -i $(GHDLFLAGS) $$i; \
done) || ($(RM) $#; exit 1)
work-obj93.cf: sources.mk
This first line tells "make" that the file "work-obj93.cf" depends on "sources.mk". If you change the latter, the former needs to be rebuilt, as the following lines define. Most probably your Makefile includes "sources.mk", which defines SOURCES.
$(RM) $#
The first command to execute removes the target file. $(RM) inserts the appropriate real command for your system, probably rm on your OS. $# is the placeholder for target file, an automatic make variable.
The target file is the "library" for GHDL. To do a clean build if the set of sources changes, it seems as if this file shall be removed and rebuilt.
(for i in $(SOURCES); do $(GHDLC) -i $(GHDLFLAGS) $$i; done) || ($(RM) $#; exit 1)
The second rule calls GHDL with the "import option" to parse and add all sources to its library.
The first part for i in $(SOURCES); do ...; done loops over all files given in SOURCES. The loop command $(GHDLC) -i $(GHDLFLAGS) $$i calls GHDL. If GHDL finds an error, it returns with a non-zero value that is used by the shell ((...) || ($(RM) $#; exit 1)) to remove the target file (the library) and to stop the build. EDIT: As HardcoreHenry pointed out, here is an error, because the loop runs through all sources; only if the last call of GHDL returns an error code, the reaction takes place.
The decision about necessity of this rule is up to you. You might want to read GHDL's documentation, and do some experiments, if removing the rule still gives correct builds under all circumstances. Try to add a new source file in "sources.mk", to rename a source file, or to remove some source file, of course without introducing VHDL errors.
Related
As part of my makefile I need to download and build ZLib. However I want to ensure that when I download ZLib, it is correct by comparing the sha256 of the downloaded .tar.gz against the known correct sha256 value. This need to work on multiple platforms.
I have so far something like the following, however the value of ZLIB_SHA256_ACTUAL always seems to be blank when I compare it with ZLIB_SHA256, so my makefile always exits with an error because the checksums are not the same. I am newish to Makefiles, can someone tell me what I am doing wrong please?
ZLIB_VER = 1.2.11
ZLIB_SHA256 = c3e5e9fdd5004dcb542feda5ee4f0ff0744628baf8ed2dd5d66f8ca1197cb1a1
SHA256_CMD = sha256sum
ifeq ($(PLATFORM), OS_MACOSX)
SHA256_CMD = openssl sha256 -r
endif
ifeq ($(PLATFORM), OS_SOLARIS)
SHA256_CMD = digest -a sha256
endif
libz.a:
-rm -rf zlib-$(ZLIB_VER)
curl -O -L http://zlib.net/zlib-$(ZLIB_VER).tar.gz
ZLIB_SHA256_ACTUAL = $(SHA256_CMD) zlib-$(ZLIB_VER).tar.gz
ifneq ($(ZLIB_SHA256), $(ZLIB_SHA256_ACTUAL))
$(error zlib-$(ZLIB_VER).tar.gz checksum mismatch, expected="$(ZLIB_SHA256)" actual="$(ZLIB_SHA256_ACTUAL)")
endif
tar xvzf zlib-$(ZLIB_VER).tar.gz
cd zlib-$(ZLIB_VER) && CFLAGS='-fPIC' ./configure --static && make
cp zlib-$(ZLIB_VER)/libz.a .
A makefile consists of two different programming languages in one file. Most of the file uses makefile syntax, that make understands and parses. But the recipes of the rules use shell syntax, which make doesn't try to interpret: it just passes the contents of the recipe to the shell to interpret.
The recipe is the part of the makefile indented with a TAB character, after a target definition. So in your example above, the target definition is libz.a: and all the lines after that which are indented with a TAB, are recipe lines. They are passed to the shell, not run by make.
The recipe is a single block of lines; you cannot intersperse recipe lines with makefile lines. Once make sees the first non-recipe line, that's the end of the recipe and make starts treating the remaining lines as if they were makefile lines.
Let's look at your rule:
libz.a:
-rm -rf zlib-$(ZLIB_VER)
curl -O -L http://zlib.net/zlib-$(ZLIB_VER).tar.gz
OK, this is fine: you've created a target libz.a and provided two command lines, which are valid shell commands, in your recipe.
ZLIB_SHA256_ACTUAL = $(SHA256_CMD) zlib-$(ZLIB_VER).tar.gz
OK, now you have problems; this is a make variable assignment, not a shell command, but since you've indented it with a TAB make will not interpret it: make will just pass it to the shell. That's not a valid shell command (in the shell, variable assignments cannot have spaces around the equal sign); this is trying to run a program named literally ZLIB_SHA256_ACTUAL and pass it the arguments = and the expansion of the SHA256_CMD variable. Even if this was recognized as a make assignment it wouldn't do what you want since it would just set the value of the variable to the string openssl sha256 -r zlib-1.2.11.tar.gz: you want to run that command and set the variable to the output.
Then the next lines:
ifneq ($(ZLIB_SHA256), $(ZLIB_SHA256_ACTUAL))
$(error zlib-$(ZLIB_VER).tar.gz checksum mismatch, expected="$(ZLIB_SHA256)" actual="$(ZLIB_SHA256_ACTUAL)")
endif
Again, this is wrong because these are make commands but you've put them into a recipe which means they'll be passed to the shell, but the shell doesn't know anything about them.
However, they never get the chance to be passed to the shell because the one thing make does with a recipe before it sends it off to the shell is expand all make variables and functions. So, when make expands this it runs the error function and that immediately fails and make never has a chance to try to run the recipe.
This is the tricky part of make. Maybe I've just confused you with all of the above stuff.
The short, simple answer is: you have to use shell commands to perform operations in a recipe. You cannot use make commands (like ifeq etc.), and if you want to set variables in a recipe they have to be shell variables, not make variables.
So, you want something like this, which uses shell syntax not make syntax for the variable assignment and test.
EDIT Note your SHA generation command doesn't print just the SHA it also prints the name of the file, so you can't compare them as strings: they'll never be the same. You need to do something fancier; there are many ways to go about it. Here I decided to use case to do the comparison:
libz.a:
-rm -rf zlib-$(ZLIB_VER)
curl -O -L http://zlib.net/zlib-$(ZLIB_VER).tar.gz
ZLIB_SHA256_ACTUAL=`$(SHA256_CMD) zlib-$(ZLIB_VER).tar.gz`; \
case "$$ZLIB_SHA256_ACTUAL " in \
($(ZLIB_SHA256)\ *) : ok ;; \
(*) echo zlib-$(ZLIB_VER).tar.gz checksum mismatch, expected=\"$(ZLIB_SHA256)\" actual=\"$$ZLIB_SHA256_ACTUAL\"; \
exit 1 ;; \
esac
tar xvzf zlib-$(ZLIB_VER).tar.gz
cd zlib-$(ZLIB_VER) && CFLAGS='-fPIC' ./configure --static && $(MAKE)
cp zlib-$(ZLIB_VER)/libz.a .
Note that each logical line in the recipe is passed to a new instance of the shell, so if you want to set a shell variable and test its value you have to combine physical lines into one logical line with the backslash/newline syntax.
Also, when running a sub-make in a recipe you should always use the variable $(MAKE) and never use just make.
I was thinking about using Make for small checks for my dev setup. One thing I want is to check that a particular string exists in a file before doing some action. If I wanted to create the entire file it would be trivial
action: filename
...
filename:
echo 'blah' >> filename
But how can this logic be applied to actions, like grep? My dependency isn't that a file exists, it's that the file has correct content.
I'm asking specifically about Make and not other solutions like chef/puppet
You can run any shell commands you want in a make recipe. As many of them as you want also.
So if you need to run grep before doing something else just do that.
Just remember that every line in a recipe is run in its own shell session so they don't share state.
So this:
action: filename
...
filename:
grep -qw blah $# || echo 'blah' > $#
runs grep on filename (via the automatic variable for the current target $#) looking for whole words and quitting on the first match (-q).
If grep finds blah then it will return success and the || will short-circuit and the recipe is done. If grep fails then the || will trigger and the echo will run.
You might be tempted to do things that require the inverse logic do X only if Y is true:
filename:
grep -qw blah $# && echo blah2 > $#
but that doesn't work correctly. When grep fails the && short-circuits and make sees a recipe failure and bails the make process out with an error.
You need this instead.
filename:
! grep -qw blah $# || echo blah2 > $#
to invert the logic and ensure that the "failure" from grep is seen as success as far as make is concerned.
That all being said in this specific example if filename exists at all then that recipe won't ever run as it has no prerequisites so make will always consider it up to date. To work around that you need to give the file a prerequisite that will force it to be considered out of date. Specifically a force target.
Don't follow the advice about .PHONY for this case though. .PHONY targets should never be prerequisites of non-.PHONY targets.
Expanding on what #john wrote I got the following to work:
TEST_FILE=filename
.PHONY: ${TEST_FILE}
string=testing
filecheck=$(shell grep -qw ${string} ${TEST_FILE} || echo ${TEST_FILE})
all: ${filecheck}
${TEST_FILE}:
echo 'changing the file'
echo ${string} >> ${TEST_FILE}
Here the file on which I'm operating is a .PHONY target. I think that's ok because I'm actually not creating the file, just modifying it. This will work if the file does not exist, or exists without the needed string.
You could add a test in the target's recipe (As Etan posted before I could complete this answer...). If you do want to do this using just make logic, you could do something along the lines of:
actions: $(if $(shell grep -q $$string filename && echo y),filename,)
filename:
echo blah >> $#
If filename contains the string, then there will be an actions: filename dependency, and filename will be built when you build actions. Notice, though that this will check whether the string exists in filename at the time the makefile is parsed -- if filename is generated, or modified in this makefile, then it would not effect whether the action is run. If you want to test right before overwriting the file, then you would use a bash if statement in the recipe itself.
I'm using a Makefile to compile my project. I get to a point which is:
$(MAKE) <some flags>; \
$(UPLOAD_SCRIPT)
The $(MAKE) line actually compiles the code, but I only want the upload script to run if the make was successful (i.e. no errors). Is there a way to do this? I'm imagining something with exit codes, storing the result in a variable, and an if statement, but I'm not super familiar with Makefiles.
Chain the 2 commands using && like this instead:
mytarget:
$(MAKE) <some flags> && $(UPLOAD_SCRIPT)
If you have more lines, and do not want to make your line look very long using &&, you can use set -e, so that the shell stops on the first error.
-e When this option is on, if a simple command fails for any of the
reasons listed in Consequences of Shell Errors or returns an exit
status value >0, and is not part of the compound list following a
while, until, or if keyword, and is not a part of an AND or OR list,
and is not a pipeline preceded by the ! reserved word, then the shell
shall immediately exit.
mytarget:
set -e; \
cmd1; \
cmd2
I'm running the following kind of pipeline:
digestA: hugefileB hugefileC
cat $^ > $#
rm $^
hugefileB:
touch $#
hugefileC:
touch $#
The targets hugefileB and hugefileC are very big and take a long time to compute (and need the power of Make). But once digestA has been created, there is no need to keep its dependencies: it deletes those dependencies to free up disk space.
Now, if I invoke 'make' again, hugefileB and hugefileC will be rebuilt, whereas digestA is already ok.
Is there any way to tell 'make' to avoid to re-comile the dependencies ?
NOTE: I don't want to build the two dependencies inside the rules for 'digestA'.
Use "intermediate files" feature of GNU Make:
Intermediate files are remade using their rules just like all other files. But intermediate files are treated differently in two ways.
The first difference is what happens if the intermediate file does not exist. If an ordinary file b does not exist, and make considers a target that depends on b, it invariably creates b and then updates the target from b. But if b is an intermediate file, then make can leave well enough alone. It won't bother updating b, or the ultimate target, unless some prerequisite of b is newer than that target or there is some other reason to update that target.
The second difference is that if make does create b in order to update something else, it deletes b later on after it is no longer needed. Therefore, an intermediate file which did not exist before make also does not exist after make. make reports the deletion to you by printing a rm -f command showing which file it is deleting.
Ordinarily, a file cannot be intermediate if it is mentioned in the makefile as a target or prerequisite. However, you can explicitly mark a file as intermediate by listing it as a prerequisite of the special target .INTERMEDIATE. This takes effect even if the file is mentioned explicitly in some other way.
You can prevent automatic deletion of an intermediate file by marking it as a secondary file. To do this, list it as a prerequisite of the special target .SECONDARY. When a file is secondary, make will not create the file merely because it does not already exist, but make does not automatically delete the file. Marking a file as secondary also marks it as intermediate.
So, adding the following line to the Makefile should be enough:
.INTERMEDIATE : hugefileB hugefileC
Invoking make for the first time:
$ make
touch hugefileB
touch hugefileC
cat hugefileB hugefileC > digestA
rm hugefileB hugefileC
And the next time:
$ make
make: `digestA' is up to date.
If you mark hugefileB and hugefileC as intermediate files, you will get the behavior you want:
digestA: hugefileB hugefileC
cat $^ > $#
hugefileB:
touch $#
hugefileC:
touch $#
.INTERMEDIATE: hugefileB hugefileC
For example:
$ gmake
touch hugefileB
touch hugefileC
cat hugefileB hugefileC > digestA
rm hugefileB hugefileC
$ gmake
gmake: `digestA' is up to date.
$ rm -f digestA
$ gmake
touch hugefileB
touch hugefileC
cat hugefileB hugefileC > digestA
rm hugefileB hugefileC
Note that you do not need the explicit rm $^ command anymore -- gmake automatically deletes intermediate files at the end of the build.
I would recommend you to create pseudo-cache files that are created by the hugefileB and hugeFileC targets.
Then have digestA depend on those cache files, because you know they will not change again until you manually invoke the expensive targets.
See also .PRECIOUS:
.PRECIOUS : hugefileA hugefileB
.PRECIOUS
The targets which .PRECIOUS depends on are given the following special treatment: if make is killed or interrupted during the execution of their recipes, the target is not deleted. See Interrupting or Killing make. Also, if the target is an intermediate file, it will not be deleted after it is no longer needed, as is normally done. See Chains of Implicit Rules. In this latter respect it overlaps with the .SECONDARY special target.
You can also list the target pattern of an implicit rule (such as ‘%.o’) as a prerequisite file of the special target .PRECIOUS to preserve intermediate files created by rules whose target patterns match that file’s name.
Edit: On re-reading the question, I see that you don't want to keep the hugefiles; maybe do this:
digestA : hugefileA hugefileB
grep '^Subject:' %^ > $#
for n in $^; do echo > $$n; done
sleep 1; touch $#
It truncates the hugefiles after using them, then touches the output file a second later, just to ensure that the output is newer than the input and this rule won't run again until the empty hugefiles are removed.
Unfortunately, if only the digest is removed, then running this rule will create an empty digest. You'd probably want to add code to block that.
The correct way is to not delete the files, as that removes the information that make uses to determine whether to rebuild the files.
Recreating them as empty does not help because make will then assume that the empty files are fully built.
If there is a way to merge digests, then you could create one from each of the huge files, which is then kept, and the huge file automatically removed as it is an intermediate.
I have a makefile that defines several rules where the target is a foreach function.
$(foreach var,$(list), $($(var)_stuff) $($(var)_more_stuff)):
#echo Building $# from $^...
$(CC) $(FLAGS) ...
Is there any way to get make to quit when encountering an error without going through the entire list.
One workaround is to "manually" invoke exit on failure.
For example, assume we have a directory called scripts with a number of shell scripts (with filenames that end with .sh) that we want to execute.
Then a variable declaration like this:
LIST_OF_SCRIPTS ?= $(wildcard scripts/*.sh)
will give us a list of those scripts, and a target like this:
run-all-scripts
#$(foreach scriptfile,$(LIST_OF_SCRIPTS),$(scriptfile);)
will run all of those scripts, but as you note, the foreach loop will keep going whether or not one of the scripts returns an error code. Adding a || exit to the command will force the subcommand to exit on error, which Make will then treat as a failure.
E.g.,
run-all-scripts
#$(foreach scriptfile,$(LIST_OF_SCRIPTS),$(scriptfile) || exit;)
will do what you want (I believe).
Specifically, using your pseudo-code example, I think you want something like this:
$(foreach var,$(list), $($(var)_stuff) $($(var)_more_stuff)):
#echo Building $# from $^...
($(CC) $(FLAGS) ...) || exit
(where all I've changed is wrapping the (CC) $(FLAGS) ... bit in parens and appending || exit to make it fail on error).
The foreach is completely evaluated and substituted before any of the rules are executed. So the behaviour of this should be identical to as if you had hardcoded the rule without using the foreach. In other words, it's not directly relevant to the problem.
There are only a few possible explanations for what you're seeing, mostly described in the manual here:
You are running Make with -k or --keep-going
You are running Make with -i or --ignore-errors
Your targets is defined as prerequisites of the special .IGNORE target
Your recipe starts with a -
Your recipe isn't actually returning a non-zero exit status
Not sure about your example, but maybe problem is in ; - look at Makefile : show and execute:
dirs = $(shell ls)
clean:
$(foreach dir,$(dirs),echo $(dir);)
produce:
$ make clean
echo bin; echo install.sh; echo Makefile; echo README.md; echo utils;
So make check exit code only for last command: echo utils.