In my Makefile, I check if a file exists, and execute different code depending on the result of that check.
Previously, there was only one relevant file, OUTLIB_CACHE.
OUTLIB_CACHE := filename
OUTLIB_CACHE_EXITS := $(shell if [ -f $(OUTLIB_CACHE) ]; then echo "1"; fi;)
ifneq ($(OUTLIB_CACHE_EXISTS),1)
# do something
endif
Now I am introducing a second relevant file, storing both of the file names in a list:
OUTLIB_CACHE_LIST := filename1 filename2
I want to set the OUTLIB_CACHE_EXISTS flag to true if and only if all the files in OUTLIB_CACHE_LIST exist. What would be the best way to do this?
You could substitute each element of OUTLIB_CACHE_LIST with a command, then execute the resulting commands in a shell:
OUTLIB_CACHE_MISSING := $(shell $(patsubst %,test -e "%" || echo missing;,$(OUTLIB_CACHE_LIST)))
If all the members exist, then the output of the shell command will be empty, else it will contain one word for each missing file. You can test for emptiness:
ifneq ($(OUTLIB_CACHE_MISSING),)
# do something
endif
If you want to know which files are missing, you can't just replace ! with %, because patsubst only replaces the first % it finds. Instead, you could use foreach:
OUTLIB_CACHE_MISSING := $(shell $(foreach f,$(OUTLIB_CACHE_LIST),test -e "$f" || echo "$f";))
Putting this all together in a testable example:
OUTLIB_CACHE_LIST := /etc /bin
#OUTLIB_CACHE_LIST += /nonexistent
OUTLIB_CACHE_MISSING := $(shell $(foreach f,$(OUTLIB_CACHE_LIST),test -e "$f" || echo "$f";))
.PHONY: all
all:
ifneq ($(OUTLIB_CACHE_MISSING),)
false "$(OUTLIB_CACHE_MISSING)"
else
true "No missing files"
endif
Uncomment the second line to select the first branch of the if.
N.B. You wanted to know whether the files exist, so I've used test -e rather than -f or -r. You'll know which test is actually appropriate for your case...
There is no need to use shell functions to test for the existence of multiple files in make. I use the following construct to make make output decent error messages for a very convoluted makefile. I presume you can adopt it to your needs if need be. The original idea (probably) comes from https://stackoverflow.com/a/20566812/1905491.
$(foreach p,$(ALL_INCLUDES),$(if $(wildcard $(p)),,$(info $(p) does not exist!) $(eval err:=yes)))
$(if $(err),$(error Aborting),)
Simply define OUTLIB_CACHE_EXISTS as follows
OUTLIB_CACHE_EXISTS := $(shell if ls $(OUTLIB_CACHE_LIST) >/dev/null 2>&1; then echo "1"; fi)
This should work
ifeq ($(OUTLIB_CACHE_LIST),$(wildcard $(OUTLIB_CACHE_LIST)))
# do something
endif
Related
I'm using GNU Make to build a dynamic web site but I need to build two versions. As a net result currently I run my makefile using two command line incantations. This is inefficient and can result in errors (I don't always remember to run both versions or make a typing error and they fail) thus I want to make it one build.
The Command Line incantations are:
sudo make release build=test
sudo make release build=release
The two incantations activate ifeq blocks that set the path and modify some files.
Part of the much simplified (to help readability) top level makefile:
subs = today tomorrow
files = index.php foot.php
ifeq ($(build),test)
export path = /var/www/html/cgi-test
$(shell cp -f head-test.php head.php)
$(shell sed -i '/"DB"/c\ define("DB", "db_test");' subs.php)
else ifeq ($(build),release)
export path = /var/www/html/cgi-new
$(shell cp -f head-release.php head.php)
$(shell sed -i '/"DB"/c\ define("DB", "db_new");' subs.php)
endif
.PHONY: all go
all:
$(MAKE) go
#for ALL in $(subs);\
do $(MAKE) -C $$ALL all || exit $$?;\
done;
go:
cp $(files) $(path)/.
The sub-makefiles have a very similar structure but don't need the ifeq blocks because the files and paths have already been setup.
I think I can simply move the shell commands into the .PHONY rules but I can't do that with the exports because I get errors "export: : bad variable name".
I could do it with a batch file and call the makefile twice but that sidesteps the problem rather than cures it and I wish to learn from the process.
So can anybody show me the way to do this in a makefile?
Thanks to Tripleee here is the answer that finally worked back ported to match my starting post. The one major change is that I have gone back to 'all' as the rule I expect to start the build habits die hard! - Thanks
.PHONY: all both rel-test rel-release
cgi-test := cgi-test
db-test := db_test
cgi-release := cgi-new
db-release := db_new
subs = today tomorrow
files = index.php foot.php
all: both
both: rel-test rel-release
rel-test rel-release: rel-%:
cp -f head-$*.php head.php
sed -i '/"DB"/c\ define("DB", "$(db-$*)");' subs.php
$(MAKE) go path=/var/www/html/strutts/$(cgi-$*)
#for ALL in $(subs);\
do $(MAKE) build=$* path=/var/www/html/strutts/$(cgi-$*) -C $$ALL all || exit $$?;\
done;
Something like this?
.PHONY: both rel-test rel-release
both: rel-test rel-release
cgi-test := cgi-test
db-test := db_test
cgi-release := cgi-new
db-release := db_new
rel-%:
cp -f head-$*.php head.php
sed -i '/"DB"/c\ define("DB", "$(db-$*)")' subs.php
$(MAKE) release build=$* path=/var/www/html/$(cgi-$*)
The reason the export can't be moved into a recipe is that you are using the export command of make itself, not the shell's command with the same name.
You absolutely should not use sudo unless you specifically require the output files to be owned and only writable by root. Even then, running as much as possible as a regular user would be proper hygiene; maybe add a sudo command inside the Makefile to copy the files to their final location.
I'm using a makefile to check some configuration files before checking into version control. I am having difficulty with one of the rules, which needs to essentially just run
/usr/local/bin/check-program config/file1.conf
/usr/local/bin/check-program config/file2.conf
[...]
The check-program does not accept multiple arguments, so each config file needs to be checked individually. All the configuration files are in one subdirectory and all end in .config.
As there is no source/dependency relationship I haven't been able to find the correct makefile syntax to get this to run. As a work-around, a for-loop will do the check for me, but this will neither exit on failure nor skip a file that has already been checked.
Here is a cut-down version of what I have so far:
SHELL := /bin/sh
WD := $(shell pwd)
HOST := $(shell hostname -s)
BPCFG := $(wildcard config/*.conf)
all : check-bp
check-bp : $(BPCFG)
for file in $(BPCFG); do \
/usr/local/bin/check-program $$file; \
echo ""; \
done
You can do this with sentinel files. For each .conf file, have the check step create a .conf.checked file if it succeeds.
SHELL := /bin/sh
WD := $(shell pwd)
HOST := $(shell hostname -s)
BPCFG := $(wildcard config/*.conf)
all : $(BCFG:%=%.checked)
%.conf.checked: %.conf
/usr/local/bin/check-program $< && touch $#
As for the for loop not exiting on failure—that’s just the way shell works. Try
for ((i = 0; i < 10; i++)); do if (( $i < 9 )); then false; else true; fi; done
The command will fail 9 times out of ten, but all that counts is the the exit status of the last run through the loop.
To fix this, change the SHELL line to SHELL = /bin/sh -e to make the shell’s default behaviour be to abort scripts when a command fails. Since that will make shell commands abort even when you don’t mind that some things return non-zero exit statues, you may have to add || true to the end of commands like grep Warning error.log.
I'm new to makefiles, and I'm stuck in this problem: how to create symbolic links to all the files in a directory found, all this inside a rule:
I want to know if a call to "find" retrieves any result. If I assign the result to a variable with "eval" (what is the only way I know to set a variable inside a rule), "ifdef" evaluation doesn't work.
I have also tried comparing to "" with "ifneq" But doesn't work.
Probably I'm following a wrong direction, so if someone has an alternative approach, I'll be very thankful.
Here the whole Makefile: (interesting line marked with ######)
Thank you a lot!
DEFAULT_YCS_ROOT := /opt/yujin
all:
ifdef YCS_ROOT
#echo Looking for Yujin maps in ${YCS_ROOT}
else
$(eval YCS_ROOT := ${DEFAULT_YCS_ROOT})
#echo WARNING: YCS_ROOT is undefined. Looking for Yujin maps in default path ${YCS_ROOT}
endif
$(eval MAPS_DIR := $(shell find ${YCS_ROOT} -name yujin_maps))
#echo ${MAPS_DIR} "${MAPS_DIR}"
ifdef MAPS_DIR ######
###### misguided alternative.... ifneq (${MAPS_DIR}, " ")
$(eval DEST_DIR := $(shell pwd)/resources/yaml/stage/maps)
#echo Creating symbolic links on ${DEST_DIR} for every PGM map on ${MAPS_DIR}/maps
rm ${DEST_DIR}/*.pgm
for f in ${MAPS_DIR}/maps/*.pgm; do ln -s $$f ${DEST_DIR}/; done
else
#echo ERROR: Yujin maps package not found in ${YCS_ROOT}. Define YCS_ROOT variable properly
endif
First, assuming you are using GNUMake, there is an easier way to handle YCS_ROOT and DEFAULT_YCS_ROOT:
DEFAULT_YCS_ROOT := /opt/yujin
YCS_ROOT ?= DEFAULT_YCS_ROOT
This will assign the value of DEFAULT_YCS_ROOT to YCS_ROOT if and only if YCS_ROOT is undefined.
For the variables MAPS_DIR and DEST_DIR, you are mixing Make syntax with shell syntax. I think the easiest approach is to define both variables outside the rule:
MAPS_DIR := $(shell find ${YCS_ROOT} -name yujin_maps)
DEST_DIR := $(shell pwd)/resources/yaml/stage/maps
You should test all of this to make certain that the variables have the values you expect. It seems that you expect only one directory in MAPS_DIR; I will follow that convention, but it is unsafe. Once the makefile so far is working, you can add the rule. You can use a shell loop:
all:
rm ${DEST_DIR}/*.pgm
for f in ${MAPS_DIR}/maps/*.pgm; do echo ln -s $$f ${DEST_DIR}/; done
or use Make syntax (which gives some advantages):
MAPS := $(wildcard $(MAPS_DIR)/maps/*.pgm)
DEST_MAPS := $(patsubst $(MAPS_DIR)/maps/%.pgm, ${DEST_DIR}/%.pgm, $(MAPS))
all: $(DEST_MAPS)
${DEST_DIR}/%.pgm: $(MAPS_DIR)/maps/%.pgm
rm $#
ln -s $< ${DEST_DIR}/
In my Makefile, I need to test if the current directory is an SVN repo or not and if it is not I want to indicate an error using the $(error) directive in Makefile.
So I plan to use the return value of $(shell svn info .) but I'm not sure how to get this value from within the Makefile.
Note: I'm not trying to get the return value in a recipe, but rather in the middle of the Makefile.
Right now I'm doing something like this, which works just because stdout is blank when it is an error:
SVN_INFO := $(shell svn info . 2> /dev/null)
ifeq ($(SVN_INFO),)
$(error "Not an SVN repo...")
endif
I'd still like to find out if it is possible to get the return value instead within the Makefile.
How about using $? to echo the exit status of the last command?
SVN_INFO := $(shell svn info . 2> /dev/null; echo $$?)
ifeq ($(SVN_INFO),1)
$(error "Not an SVN repo...")
endif
If you want to preserve the original output then you need to do some tricks. If you are lucky enough to have GNU Make 4.2 (released on 2016-05-22) or later at your disposal you can use the .SHELLSTATUS variable as follows.
var := $(shell echo "blabla" ; false)
ifneq ($(.SHELLSTATUS),0)
$(error shell command failed! output was $(var))
endif
all:
#echo Never reached but output would have been $(var)
Alternatively you could use a temporary file or play with Make's eval to store the string and/or the exit code into a Make variable. The example below gets this done but I would certainly like to see a better implementation than this embarrassingly complicated version.
ret := $(shell echo "blabla"; false; echo " $$?")
rc := $(lastword $(ret))
# Remove the last word by calculating <word count - 1> and
# using it as the second parameter of wordlist.
string:=$(wordlist 1,$(shell echo $$(($(words $(ret))-1))),$(ret))
ifneq ($(rc),0)
$(error shell command failed with $(rc)! output was "$(string)")
endif
all:
#echo Never reached but output would have been \"$(string)\"
This worked fine for me - based on #eriktous' answer with a minor modification of redirecting stdout as well to skip the output from svn info on a valid svn repo.
SVN_INFO := $(shell svn info . 1>&2 2> /dev/null; echo $$?)
ifneq ($(SVN_INFO),0)
$(error "Not an SVN repo...")
endif
Maybe something like this?
IS_SVN_CHECKED_OUT := $(shell svn info . 1>/dev/null 2>&1 && echo "yes" || echo "no")
ifne ($(IS_SVN_CHECKED_OUT),yes)
$(error "The current directory must be checked out from SVN.")
endif
I use .NOTPARALLEL and a make function:
.NOTPARALLEL:
# This function works almost exactly like the builtin shell command, except it
# stops everything with an error if the shell command given as its argument
# returns non-zero when executed. The other difference is that the output
# is passed through the strip make function (the shell function strips only
# the last trailing newline). In practice this doesn't matter much since
# the output is usually collapsed by the surroundeing make context to the
# same result produced by strip.
SHELL_CHECKED = \
$(strip \
$(if $(shell (($1) 1>/tmp/SC_so) || echo nonempty), \
$(error shell command '$1' failed. Its stderr should be above \
somewhere. Its stdout is in '/tmp/SC_so'), \
$(shell cat /tmp/SC_so)))
In my GNUmakefile, I would like to have a rule that uses a temporary directory. For example:
out.tar: TMP := $(shell mktemp -d)
echo hi $(TMP)/hi.txt
tar -C $(TMP) cf $# .
rm -rf $(TMP)
As written, the above rule creates the temporary directory at the time that the rule is parsed. This means that, even I don't make out.tar all the time, many temporary directories get created. I would like to avoid my /tmp being littered with unused temporary directories.
Is there a way to cause the variable to only be defined when the rule is fired, as opposed to whenever it is defined?
My main thought is to dump the mktemp and tar into a shell script but that seems somewhat unsightly.
In your example, the TMP variable is set (and the temporary directory created) whenever the rules for out.tar are evaluated. In order to create the directory only when out.tar is actually fired, you need to move the directory creation down into the steps:
out.tar :
$(eval TMP := $(shell mktemp -d))
#echo hi $(TMP)/hi.txt
tar -C $(TMP) cf $# .
rm -rf $(TMP)
The eval function evaluates a string as if it had been typed into the makefile manually. In this case, it sets the TMP variable to the result of the shell function call.
edit (in response to comments):
To create a unique variable, you could do the following:
out.tar :
$(eval $#_TMP := $(shell mktemp -d))
#echo hi $($#_TMP)/hi.txt
tar -C $($#_TMP) cf $# .
rm -rf $($#_TMP)
This would prepend the name of the target (out.tar, in this case) to the variable, producing a variable with the name out.tar_TMP. Hopefully, that is enough to prevent conflicts.
A relatively easy way of doing this is to write the entire sequence as a shell script.
out.tar:
set -e ;\
TMP=$$(mktemp -d) ;\
echo hi $$TMP/hi.txt ;\
tar -C $$TMP cf $# . ;\
rm -rf $$TMP ;\
I have consolidated some related tips here: Multi-line bash commands in makefile
Another possibility is to use separate lines to set up Make variables when a rule fires.
For example, here is a makefile with two rules. If a rule fires, it creates a temp dir and sets TMP to the temp dir name.
PHONY = ruleA ruleB display
all: ruleA
ruleA: TMP = $(shell mktemp -d testruleA_XXXX)
ruleA: display
ruleB: TMP = $(shell mktemp -d testruleB_XXXX)
ruleB: display
display:
echo ${TMP}
Running the code produces the expected result:
$ ls
Makefile
$ make ruleB
echo testruleB_Y4Ow
testruleB_Y4Ow
$ ls
Makefile testruleB_Y4Ow
I dislike "Don't" answers, but... don't.
make's variables are global and are supposed to be evaluated during makefile's "parsing" stage, not during execution stage.
In this case, as long as the variable local to a single target, follow #nobar's answer and make it a shell variable.
Target-specific variables, too, are considered harmful by other make implementations: kati, Mozilla pymake. Because of them, a target can be built differently depending on if it's built standalone, or as a dependency of a parent target with a target-specific variable. And you won't know which way it was, because you don't know what is already built.