Consider the following makefile
# If current commit is not part of a PR, the PULL_REQUEST_ID variable will be empty.
PULL_REQUEST_ID := $(shell git ls-remote origin 'pull/*/head' | grep -F -f <(git rev-parse HEAD) | awk -F'/' '{print $3}')
# if PULL_REQUEST_ID is empty, I want 'make deploy-to-staging' to be no-op
.PHONY deploy-to-staging
deploy-to-staging: update-some-files apply-those-files-to-k8s
Use the conditional directive ifneq.
deploy-to-staging: update-some-files apply-those-files-to-k8s
ifneq ($(PULL_REQUEST_ID),'')
git push ... # or whatever
else
echo "Refusing to deploy non-pull-request to staging"
endif
If all of your actions to deploy are prerequisites of a target, you can conditionally define those prerequisites, i.e.:
$ cat Makefile
ifneq ($(PULL_REQUEST_ID),)
deploy-to-staging: update-some-files apply-those-files-to-k8s
endif
.PHONY: deploy-to-staging
deploy-to-staging:
.PHONY: update-some-files
update-some-files:
echo Updating...
.PHONY: apply-those-files-to-k8s
apply-those-files-to-k8s:
echo Applying...
Output:
$ make deploy-to-staging
make: Nothing to be done for 'deploy-to-staging'.
$ make deploy-to-staging PULL_REQUEST_ID=foo
echo Updating...
Updating...
echo Applying...
Applying...
Related
I want to run Terraform fmt command only if my .tf/.tpl files in the current directory or subdirectories have changed. Here is my makefile, not sure if this is even possible with makefile or not:
TF_VERSION=0.10.7
.PHONY: format
README.md: ./variables.tf
terraform-docs md . > README.md
format: $(wildcard *.tf) $(wildcard */*.tf) $(wildcard */*.tpl)
docker run -ti -v $(shell pwd):/tmp hashicorp/terraform:${TF_VERSION} fmt /tmp
help:
#grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
.DEFAULT_GOAL := format
The problem is that make runs format every time (even without any changes), and if I remove the last line, I have to run make format otherwise it just runs the first rule.
Use an empty file, whose modification time records the last time the rule was run:
format_timestamp: $(wildcard *.tf) ...
docker run -ti -v ...
touch $#
I am trying to write a simple Makefile to build .expected files and compare them but I am failing.
APSSCHED=../../bin/apssched
BASE=.:../../base:../../examples
FLAGS=-DCOT
EXAMPLES=../../examples/
CASES=simple-binding1 simple-binding2
# skipping lines doesn't work ...
# run command and skip the first line
%.aps:
${APSSCHED} ${FLAGS} -p ${BASE} ${EXAMPLES}/$* | tail -n +2
# get all cases as an array to pipe it to different make targets
# maybe overcomplicating
cases:
echo ${CASES} | \
awk '{split($$0,numbers," ")} END {for(n in numbers){ print numbers[n] }}'
# create all .expected files from ${CASES}
build.expected:
$(MAKE) cases | xargs -n1 -I file /bin/bash -c '$(MAKE) file.build.expected'
# create single .expected file
%.build.expected:
$(MAKE) $*.aps > $*.expected
# compare result with
%.compare:
$(MAKE) $*.aps | diff $*.expected -
# run command for all cases and diff the result with corresponding expected
all:
$(MAKE) cases | xargs -n1 -I file /bin/bash -c '$(MAKE) file.compare'
clean.expected:
rm *.expected
Running make without any target and nothing happens.
echo simple-binding1 simple-binding2 | \
awk '{split($0,numbers," ")} END {for(n in numbers){ print numbers[n] }}'
simple-binding1
simple-binding2
I think the issue is with my cases target. I am not sure if I am on the right track.
I appreciate any help or hint.
I would avoid re-running make just to call a different target - it's a performance hit and may be unreliable (depending on rest of the Makefile) since separate calls may not be able to track dependencies correctly.
Moreover, I would avoid using | - every time a command is concatenated with pipe, exit code of piped command would be exit code of the last command. So a call like command | tail would return the exit code of tail (which would almost always succeed). Even if the command has failed, it would be covered with exit code 0 from tail and make will not detect the error and will not stop.
Thus said, I tried to rewrite your approach by just creating dependencies between the targets, like so:
$ cat Makefile
APSSCHED=../../bin/apssched
EXAMPLES=../../examples
BASE=.:../../base:$(EXAMPLES)
FLAGS=-DCOT
CASES=simple-binding1 simple-binding2
# Just for reproducing
$(EXAMPLES)/%.aps: ;
# Generate output and store it in a file
%.output: $(EXAMPLES)/%.aps
# echo is only for reproducing
echo $(APSSCHED) $(FLAGS) -p $(BASE) $< > $#
# Copy actual output as expected
%.expected: %.output
cp -f $< $#
# Compare actual output with expected
.PHONY: %.compare
%.compare: %.output | %.expected
diff $| $<
# Generate and verify all outputs
.PHONY: all
all: $(addsuffix .compare,$(CASES))
# Regenerate expected output
.PHONY: build.expected
build.expected: $(addsuffix .expected,$(CASES))
.PHONY: clean.expected
clean.expected:
-rm -f *.expected
Now the make build.expected will create expected output files, while make all or make will check the actual output against expected:
$ make build.expected
echo ../../bin/apssched -DCOT -p .:../../base:../../examples ../../examples/simple-binding1.aps > simple-binding1.output
cp -f simple-binding1.output simple-binding1.expected
echo ../../bin/apssched -DCOT -p .:../../base:../../examples ../../examples/simple-binding2.aps > simple-binding2.output
cp -f simple-binding2.output simple-binding2.expected
rm simple-binding1.output simple-binding2.output
$ make
echo ../../bin/apssched -DCOT -p .:../../base:../../examples ../../examples/simple-binding1.aps > simple-binding1.output
diff simple-binding1.expected simple-binding1.output
echo ../../bin/apssched -DCOT -p .:../../base:../../examples ../../examples/simple-binding2.aps > simple-binding2.output
diff simple-binding2.expected simple-binding2.output
rm simple-binding1.output simple-binding2.output
Is it possible to read the dependencies of a target inside a Makefile?
I would like to do something like the following:
.INTERMEDIATE: temp1.txt
.INTERMEDIATE: temp2.txt
print-intermediates:
#echo "Temp files: $(dependencies-of .INTERMEDIATE)"
The output of make print-intermediate would be
$ make print-intermediate
Temp files: temp1.txt temp2.txt
How can I access the list of dependencies of the .INTERMEDIATE target?
Recursive call of make.
There is no built-in function for this purpose. But there are some workarounds.
For example you can use recursive call of make with command line argument -p.
.INTERMEDIATE: temp1.txt
.INTERMEDIATE: temp2.txt
deps = $(shell $(MAKE) -qp none | sed -n "/$(strip $(1)):/ p;" | \
sed -e "s/$(strip $(1)): //g")
print-intermediates:
#echo "Temp files: " $(call deps,.INTERMEDIATE)
.PHONY: none
none:;
I'm trying to create a makefile (GNU make) that does the following:
A script generates a bunch of files--filenames not known in advance.
Each one of these files is converted to a different file.
After all are converted, all of these files are combined into a single output file.
How do I create a makefile with a "bellcurve"-patterned dependency graph, where the intermediate source and target files are not known in advance?
Conceptually I'm doing the following:
combined.pdf: $(filter combined.pdf, $(wildcard *.pdf))
cat *.pdf > combined.pdf
%.pdf: %.svg
cp $^ $#
$(wildcard *.svg):
# recipe is for simple example
# actually the *.svg files are not known in advance
echo a > a.svg
echo b > b.svg
echo c > c.svg
.PHONY: clean
clean:
${RM} *.svg *.pdf *.d
Of course this doesn't work: Make evaluates the targets and sources before it runs the target that actually creates the svg. Also, there's no way to make sure all svgs are converted before they are combined.
I realized I could create dependencies and include them into the makefile, but I had trouble getting this to work too:
.PHONY: clean
include deps.d
combined.pdf: deps.d
cat *.pdf > combined.pdf
%.pdf: %.svg
cp $^ $#
deps.d:
## recipe is for simple example
## actually the *.svg files are not known in advance
echo a > a.svg
echo b > b.svg
echo c > c.svg
## we know what files exist now, so we can establish dependencies
## "a.pdf : a.svg"
echo *.svg : > deps.d
## combined.pdf: a.pdf b.pdf c.pdf
ls *.svg \
| awk '{targetfn=$$0; sub(/\.svg$$/, ".pdf", targetfn); print targetfn, ":", $$0;}' \
>> deps.d
## combined.pdf: a.pdf b.pdf c.pdf
echo combined.pdf : $$(echo *.svg | sed -e 's/\.svg/\.pdf/g') >> deps.d
clean:
${RM} *.pdf *.svg *.d
However this still isn't connecting the dependency graph properly. When I run this, make quits as follows:
Makefile:3: deps.d: No such file or directory
echo a > a.svg
echo b > b.svg
echo c > c.svg
echo *.svg : > deps.d
ls *.svg \
| awk '{targetfn=$0; sub(/\.svg$/, ".pdf", targetfn); print targetfn, ":", $0;}' \
>> deps.d
echo combined.pdf : $(echo *.svg | sed -e 's/\.svg/\.pdf/g') >> deps.d
make: Nothing to be done for `a.svg'.
I still seem to have the problem that the make doesn't know about the rules in deps.d.
Also, this still doesn't solve the problem of building all the dependencies. I thought of using a marker file like this:
%.pdf: %.svg
cp $^ $#
## if all svgs are converted, touch a target allpdfs
if [ $(ls -1 *.svg | wc -l) -eq $(ls -1 *.pdf | grep -v combined\.pdf | wc -l) ]; touch allpdfs; fi
But there's no way to inform make that "allpdfs" may be created by this rule.
I'm surprised that moving the include directive makes a difference (what version of Make are you using?), but there is a simpler way. Your use of deps.d is in effect a recursive use of Make -- Make is arranging to execute itself a second time -- so we might as well make it official:
combined.pdf: ALL_SVGS
$(MAKE) ALL_PDFS
rm -f $# # just in case it exists already
cat *.pdf > $#
.PHONY: ALL_SVGS
ALL_SVGS:
# recipe is for simple example
# actually the *.svg files are not known in advance
echo a > a.svg
echo b > b.svg
echo c > c.svg
# These variables will be empty in the first execution of Make
SVGS = $(wildcard *.svg)
PDFS = $(patsubst %.svg,%.pdf,$(SVGS))
.PHONY: ALL_PDFS
ALL_PDFS: $(PDFS))
%.pdf: %.svg
cp $^ $#
This isn't an answer exactly, because I don't know why this works, but I discovered that if I move the include directive after the target that creates the included file, everything works.
I.e. do this:
deps.d:
....
include deps.d
Because my deps.d includes enough dependency information, there's no need to have an intermediate target allpdfs file. Everything Just Works, even with make -j.
However, I don't know why this works. The include documentation isn't enlightening me.
UPDATE
I noticed the following note at the very bottom of the make manual discussing Automatic Prerequisites:
Note that the ‘.d’ files contain target definitions; you should be sure to place the include directive after the first, default goal in your makefiles or run the risk of having a random object file become the default goal. See How Make Works.
So what happened is that the first rule inside the generated deps.d became the default target, causing the mysterious premature completion of the build. So the solution is just to make sure include directives are not before your intended default target.
I was just working on this exact problem in a slightly different setting. Here is a clean solution - no need for recursion and such (and you can tweak the sed if you like):
include deps.d
combined.pdf:
cat *.pdf > combined.pdf
%.pdf: %.svg
cp $^ $#
deps.d:
echo a > a.svg
echo b > b.svg
echo c > c.svg
echo 'combined.pdf:' *.svg | sed 's/\.svg/\.pdf/g' > deps.d
Enjoy!
In my Makefile, I need to test if the current directory is an SVN repo or not and if it is not I want to indicate an error using the $(error) directive in Makefile.
So I plan to use the return value of $(shell svn info .) but I'm not sure how to get this value from within the Makefile.
Note: I'm not trying to get the return value in a recipe, but rather in the middle of the Makefile.
Right now I'm doing something like this, which works just because stdout is blank when it is an error:
SVN_INFO := $(shell svn info . 2> /dev/null)
ifeq ($(SVN_INFO),)
$(error "Not an SVN repo...")
endif
I'd still like to find out if it is possible to get the return value instead within the Makefile.
How about using $? to echo the exit status of the last command?
SVN_INFO := $(shell svn info . 2> /dev/null; echo $$?)
ifeq ($(SVN_INFO),1)
$(error "Not an SVN repo...")
endif
If you want to preserve the original output then you need to do some tricks. If you are lucky enough to have GNU Make 4.2 (released on 2016-05-22) or later at your disposal you can use the .SHELLSTATUS variable as follows.
var := $(shell echo "blabla" ; false)
ifneq ($(.SHELLSTATUS),0)
$(error shell command failed! output was $(var))
endif
all:
#echo Never reached but output would have been $(var)
Alternatively you could use a temporary file or play with Make's eval to store the string and/or the exit code into a Make variable. The example below gets this done but I would certainly like to see a better implementation than this embarrassingly complicated version.
ret := $(shell echo "blabla"; false; echo " $$?")
rc := $(lastword $(ret))
# Remove the last word by calculating <word count - 1> and
# using it as the second parameter of wordlist.
string:=$(wordlist 1,$(shell echo $$(($(words $(ret))-1))),$(ret))
ifneq ($(rc),0)
$(error shell command failed with $(rc)! output was "$(string)")
endif
all:
#echo Never reached but output would have been \"$(string)\"
This worked fine for me - based on #eriktous' answer with a minor modification of redirecting stdout as well to skip the output from svn info on a valid svn repo.
SVN_INFO := $(shell svn info . 1>&2 2> /dev/null; echo $$?)
ifneq ($(SVN_INFO),0)
$(error "Not an SVN repo...")
endif
Maybe something like this?
IS_SVN_CHECKED_OUT := $(shell svn info . 1>/dev/null 2>&1 && echo "yes" || echo "no")
ifne ($(IS_SVN_CHECKED_OUT),yes)
$(error "The current directory must be checked out from SVN.")
endif
I use .NOTPARALLEL and a make function:
.NOTPARALLEL:
# This function works almost exactly like the builtin shell command, except it
# stops everything with an error if the shell command given as its argument
# returns non-zero when executed. The other difference is that the output
# is passed through the strip make function (the shell function strips only
# the last trailing newline). In practice this doesn't matter much since
# the output is usually collapsed by the surroundeing make context to the
# same result produced by strip.
SHELL_CHECKED = \
$(strip \
$(if $(shell (($1) 1>/tmp/SC_so) || echo nonempty), \
$(error shell command '$1' failed. Its stderr should be above \
somewhere. Its stdout is in '/tmp/SC_so'), \
$(shell cat /tmp/SC_so)))