I'm using a makefile to check some configuration files before checking into version control. I am having difficulty with one of the rules, which needs to essentially just run
/usr/local/bin/check-program config/file1.conf
/usr/local/bin/check-program config/file2.conf
[...]
The check-program does not accept multiple arguments, so each config file needs to be checked individually. All the configuration files are in one subdirectory and all end in .config.
As there is no source/dependency relationship I haven't been able to find the correct makefile syntax to get this to run. As a work-around, a for-loop will do the check for me, but this will neither exit on failure nor skip a file that has already been checked.
Here is a cut-down version of what I have so far:
SHELL := /bin/sh
WD := $(shell pwd)
HOST := $(shell hostname -s)
BPCFG := $(wildcard config/*.conf)
all : check-bp
check-bp : $(BPCFG)
for file in $(BPCFG); do \
/usr/local/bin/check-program $$file; \
echo ""; \
done
You can do this with sentinel files. For each .conf file, have the check step create a .conf.checked file if it succeeds.
SHELL := /bin/sh
WD := $(shell pwd)
HOST := $(shell hostname -s)
BPCFG := $(wildcard config/*.conf)
all : $(BCFG:%=%.checked)
%.conf.checked: %.conf
/usr/local/bin/check-program $< && touch $#
As for the for loop not exiting on failure—that’s just the way shell works. Try
for ((i = 0; i < 10; i++)); do if (( $i < 9 )); then false; else true; fi; done
The command will fail 9 times out of ten, but all that counts is the the exit status of the last run through the loop.
To fix this, change the SHELL line to SHELL = /bin/sh -e to make the shell’s default behaviour be to abort scripts when a command fails. Since that will make shell commands abort even when you don’t mind that some things return non-zero exit statues, you may have to add || true to the end of commands like grep Warning error.log.
Related
I'm using GNU Make to build a dynamic web site but I need to build two versions. As a net result currently I run my makefile using two command line incantations. This is inefficient and can result in errors (I don't always remember to run both versions or make a typing error and they fail) thus I want to make it one build.
The Command Line incantations are:
sudo make release build=test
sudo make release build=release
The two incantations activate ifeq blocks that set the path and modify some files.
Part of the much simplified (to help readability) top level makefile:
subs = today tomorrow
files = index.php foot.php
ifeq ($(build),test)
export path = /var/www/html/cgi-test
$(shell cp -f head-test.php head.php)
$(shell sed -i '/"DB"/c\ define("DB", "db_test");' subs.php)
else ifeq ($(build),release)
export path = /var/www/html/cgi-new
$(shell cp -f head-release.php head.php)
$(shell sed -i '/"DB"/c\ define("DB", "db_new");' subs.php)
endif
.PHONY: all go
all:
$(MAKE) go
#for ALL in $(subs);\
do $(MAKE) -C $$ALL all || exit $$?;\
done;
go:
cp $(files) $(path)/.
The sub-makefiles have a very similar structure but don't need the ifeq blocks because the files and paths have already been setup.
I think I can simply move the shell commands into the .PHONY rules but I can't do that with the exports because I get errors "export: : bad variable name".
I could do it with a batch file and call the makefile twice but that sidesteps the problem rather than cures it and I wish to learn from the process.
So can anybody show me the way to do this in a makefile?
Thanks to Tripleee here is the answer that finally worked back ported to match my starting post. The one major change is that I have gone back to 'all' as the rule I expect to start the build habits die hard! - Thanks
.PHONY: all both rel-test rel-release
cgi-test := cgi-test
db-test := db_test
cgi-release := cgi-new
db-release := db_new
subs = today tomorrow
files = index.php foot.php
all: both
both: rel-test rel-release
rel-test rel-release: rel-%:
cp -f head-$*.php head.php
sed -i '/"DB"/c\ define("DB", "$(db-$*)");' subs.php
$(MAKE) go path=/var/www/html/strutts/$(cgi-$*)
#for ALL in $(subs);\
do $(MAKE) build=$* path=/var/www/html/strutts/$(cgi-$*) -C $$ALL all || exit $$?;\
done;
Something like this?
.PHONY: both rel-test rel-release
both: rel-test rel-release
cgi-test := cgi-test
db-test := db_test
cgi-release := cgi-new
db-release := db_new
rel-%:
cp -f head-$*.php head.php
sed -i '/"DB"/c\ define("DB", "$(db-$*)")' subs.php
$(MAKE) release build=$* path=/var/www/html/$(cgi-$*)
The reason the export can't be moved into a recipe is that you are using the export command of make itself, not the shell's command with the same name.
You absolutely should not use sudo unless you specifically require the output files to be owned and only writable by root. Even then, running as much as possible as a regular user would be proper hygiene; maybe add a sudo command inside the Makefile to copy the files to their final location.
I would like to run a command before and after each target in a makefile.
so something like this
pre:
#echo pre
#echo running | mailx -s "Start {target}" user#foo.com
post:
#echo post
#echo post | mailx -s "Finish {target}" user#foo.com
j:
long_running_command && echo $# > $#
k: j
long_running_command2 && echo $# > $#
I would like to run pre and post for j and k. Ideally, I would like to get an email for each task that starts and stops.
One way to do it is to modify all the recipes in your makefile to invoke some command. You can put it into a variable so it doesn't look too gross:
START = mailto --subject 'Started target $#' me#my.host
END = mailto --subject 'Finished target $#' me#my.host
j:
$(START)
long_running_command && echo $# > $#
$(END)
k: j
$(START)
long_running_command2 && echo $# > $#
$(END)
The nice thing about this is you can pick and choose which targets you want it for; maybe some of them don't need it. The disadvantage is if the command fails you won't get any "end" email at all.
If you really want to do it for every single target, then you can write a shell script that mimics the shell's behavior but also sends mail, while running the command.
$ cat mailshell
#!/bin/sh
# get rid of the -c flag
shift
mailto --subject "started $*" me#my.host
/bin/sh -c "$#"
r=$?
mailto --subject "ended $* with exit code $r" me#my.host
exit $r
(note this is totally untested but you get the idea I hope). Then in your makefile, set SHELL to that shell:
SHELL := mailshell
j:
long_running_command && echo $# > $#
k: j
long_running_command2 && echo $# > $#
I guess you could still pick and choose by setting SHELL as a target-specific variable only for those targets you wanted to use the shell.
One downside of this is that if you have recipes that have multiple lines you'll get an email for each line individually. You can work around this by enabling .ONESHELL: to force the entire recipe to be passed to a single shell. However, I believe that this may require your mailshell tool to be more sophisticated.
If you only have one command per recipe, you can do this by changing the configuration for the shell the commands are run in.
Have the config file run the pre commands directly and trap EXIT to run the after commands in.
For example:
$ cat Makefile
SHELL := BASH_ENV=/dev/fd/3 3<<<'echo before; trap "echo after" EXIT' /bin/bash
default:
echo default
other:
echo first
echo second
$ make default
echo default
before
default
after
However this may not be what you want if a recipe runs several commands, as the before and after code will run each time.
$ make other
echo first
before
first
after
echo second
before
second
after
And I don't know of a way (outside of recursive Makefiles) to have different recipes use different shells, so this won't work if you only want to set before/after for several recipes.
In my Makefile, I check if a file exists, and execute different code depending on the result of that check.
Previously, there was only one relevant file, OUTLIB_CACHE.
OUTLIB_CACHE := filename
OUTLIB_CACHE_EXITS := $(shell if [ -f $(OUTLIB_CACHE) ]; then echo "1"; fi;)
ifneq ($(OUTLIB_CACHE_EXISTS),1)
# do something
endif
Now I am introducing a second relevant file, storing both of the file names in a list:
OUTLIB_CACHE_LIST := filename1 filename2
I want to set the OUTLIB_CACHE_EXISTS flag to true if and only if all the files in OUTLIB_CACHE_LIST exist. What would be the best way to do this?
You could substitute each element of OUTLIB_CACHE_LIST with a command, then execute the resulting commands in a shell:
OUTLIB_CACHE_MISSING := $(shell $(patsubst %,test -e "%" || echo missing;,$(OUTLIB_CACHE_LIST)))
If all the members exist, then the output of the shell command will be empty, else it will contain one word for each missing file. You can test for emptiness:
ifneq ($(OUTLIB_CACHE_MISSING),)
# do something
endif
If you want to know which files are missing, you can't just replace ! with %, because patsubst only replaces the first % it finds. Instead, you could use foreach:
OUTLIB_CACHE_MISSING := $(shell $(foreach f,$(OUTLIB_CACHE_LIST),test -e "$f" || echo "$f";))
Putting this all together in a testable example:
OUTLIB_CACHE_LIST := /etc /bin
#OUTLIB_CACHE_LIST += /nonexistent
OUTLIB_CACHE_MISSING := $(shell $(foreach f,$(OUTLIB_CACHE_LIST),test -e "$f" || echo "$f";))
.PHONY: all
all:
ifneq ($(OUTLIB_CACHE_MISSING),)
false "$(OUTLIB_CACHE_MISSING)"
else
true "No missing files"
endif
Uncomment the second line to select the first branch of the if.
N.B. You wanted to know whether the files exist, so I've used test -e rather than -f or -r. You'll know which test is actually appropriate for your case...
There is no need to use shell functions to test for the existence of multiple files in make. I use the following construct to make make output decent error messages for a very convoluted makefile. I presume you can adopt it to your needs if need be. The original idea (probably) comes from https://stackoverflow.com/a/20566812/1905491.
$(foreach p,$(ALL_INCLUDES),$(if $(wildcard $(p)),,$(info $(p) does not exist!) $(eval err:=yes)))
$(if $(err),$(error Aborting),)
Simply define OUTLIB_CACHE_EXISTS as follows
OUTLIB_CACHE_EXISTS := $(shell if ls $(OUTLIB_CACHE_LIST) >/dev/null 2>&1; then echo "1"; fi)
This should work
ifeq ($(OUTLIB_CACHE_LIST),$(wildcard $(OUTLIB_CACHE_LIST)))
# do something
endif
My Makefile is:
.PHONY: check
check:
ifneq $(shell echo 123), $(shell echo 123)
$(error Not equal)
endif
When I run, I've got the error:
$ make
Makefile:3: *** Not equal. Stop.
But this should happen only when they're different, but they're not. Why?
ifneq cannot be indented. the way you've written it, it's being run via a shell command which means the $(error) is being evaluated first by the make command.
i'm guessing you want the make check to actually run two commands only when make check is invoked, and compare their output. you can do:
.PHONY: check
check:
if [ "`echo 123`" != "`echo 123`" ]; then \
echo "Not equal"; \
exit 1; \
fi
According to GNU Make docs, Conditional Parts cannot be used to control shell commands at the time of execution, since conditionals control what make actually "sees" in the makefile.
So to perform condition during compilation process, shell syntax is preferred, e.g.
SHELL := /bin/bash -e
.PHONY: check
check:
#test "$(shell echo 123)" = "$(shell echo 123)" \
|| { echo Not equal; exit 2; } \
&& { echo Equal; }
In my Makefile, I need to test if the current directory is an SVN repo or not and if it is not I want to indicate an error using the $(error) directive in Makefile.
So I plan to use the return value of $(shell svn info .) but I'm not sure how to get this value from within the Makefile.
Note: I'm not trying to get the return value in a recipe, but rather in the middle of the Makefile.
Right now I'm doing something like this, which works just because stdout is blank when it is an error:
SVN_INFO := $(shell svn info . 2> /dev/null)
ifeq ($(SVN_INFO),)
$(error "Not an SVN repo...")
endif
I'd still like to find out if it is possible to get the return value instead within the Makefile.
How about using $? to echo the exit status of the last command?
SVN_INFO := $(shell svn info . 2> /dev/null; echo $$?)
ifeq ($(SVN_INFO),1)
$(error "Not an SVN repo...")
endif
If you want to preserve the original output then you need to do some tricks. If you are lucky enough to have GNU Make 4.2 (released on 2016-05-22) or later at your disposal you can use the .SHELLSTATUS variable as follows.
var := $(shell echo "blabla" ; false)
ifneq ($(.SHELLSTATUS),0)
$(error shell command failed! output was $(var))
endif
all:
#echo Never reached but output would have been $(var)
Alternatively you could use a temporary file or play with Make's eval to store the string and/or the exit code into a Make variable. The example below gets this done but I would certainly like to see a better implementation than this embarrassingly complicated version.
ret := $(shell echo "blabla"; false; echo " $$?")
rc := $(lastword $(ret))
# Remove the last word by calculating <word count - 1> and
# using it as the second parameter of wordlist.
string:=$(wordlist 1,$(shell echo $$(($(words $(ret))-1))),$(ret))
ifneq ($(rc),0)
$(error shell command failed with $(rc)! output was "$(string)")
endif
all:
#echo Never reached but output would have been \"$(string)\"
This worked fine for me - based on #eriktous' answer with a minor modification of redirecting stdout as well to skip the output from svn info on a valid svn repo.
SVN_INFO := $(shell svn info . 1>&2 2> /dev/null; echo $$?)
ifneq ($(SVN_INFO),0)
$(error "Not an SVN repo...")
endif
Maybe something like this?
IS_SVN_CHECKED_OUT := $(shell svn info . 1>/dev/null 2>&1 && echo "yes" || echo "no")
ifne ($(IS_SVN_CHECKED_OUT),yes)
$(error "The current directory must be checked out from SVN.")
endif
I use .NOTPARALLEL and a make function:
.NOTPARALLEL:
# This function works almost exactly like the builtin shell command, except it
# stops everything with an error if the shell command given as its argument
# returns non-zero when executed. The other difference is that the output
# is passed through the strip make function (the shell function strips only
# the last trailing newline). In practice this doesn't matter much since
# the output is usually collapsed by the surroundeing make context to the
# same result produced by strip.
SHELL_CHECKED = \
$(strip \
$(if $(shell (($1) 1>/tmp/SC_so) || echo nonempty), \
$(error shell command '$1' failed. Its stderr should be above \
somewhere. Its stdout is in '/tmp/SC_so'), \
$(shell cat /tmp/SC_so)))