I was thinking about using Make for small checks for my dev setup. One thing I want is to check that a particular string exists in a file before doing some action. If I wanted to create the entire file it would be trivial
action: filename
...
filename:
echo 'blah' >> filename
But how can this logic be applied to actions, like grep? My dependency isn't that a file exists, it's that the file has correct content.
I'm asking specifically about Make and not other solutions like chef/puppet
You can run any shell commands you want in a make recipe. As many of them as you want also.
So if you need to run grep before doing something else just do that.
Just remember that every line in a recipe is run in its own shell session so they don't share state.
So this:
action: filename
...
filename:
grep -qw blah $# || echo 'blah' > $#
runs grep on filename (via the automatic variable for the current target $#) looking for whole words and quitting on the first match (-q).
If grep finds blah then it will return success and the || will short-circuit and the recipe is done. If grep fails then the || will trigger and the echo will run.
You might be tempted to do things that require the inverse logic do X only if Y is true:
filename:
grep -qw blah $# && echo blah2 > $#
but that doesn't work correctly. When grep fails the && short-circuits and make sees a recipe failure and bails the make process out with an error.
You need this instead.
filename:
! grep -qw blah $# || echo blah2 > $#
to invert the logic and ensure that the "failure" from grep is seen as success as far as make is concerned.
That all being said in this specific example if filename exists at all then that recipe won't ever run as it has no prerequisites so make will always consider it up to date. To work around that you need to give the file a prerequisite that will force it to be considered out of date. Specifically a force target.
Don't follow the advice about .PHONY for this case though. .PHONY targets should never be prerequisites of non-.PHONY targets.
Expanding on what #john wrote I got the following to work:
TEST_FILE=filename
.PHONY: ${TEST_FILE}
string=testing
filecheck=$(shell grep -qw ${string} ${TEST_FILE} || echo ${TEST_FILE})
all: ${filecheck}
${TEST_FILE}:
echo 'changing the file'
echo ${string} >> ${TEST_FILE}
Here the file on which I'm operating is a .PHONY target. I think that's ok because I'm actually not creating the file, just modifying it. This will work if the file does not exist, or exists without the needed string.
You could add a test in the target's recipe (As Etan posted before I could complete this answer...). If you do want to do this using just make logic, you could do something along the lines of:
actions: $(if $(shell grep -q $$string filename && echo y),filename,)
filename:
echo blah >> $#
If filename contains the string, then there will be an actions: filename dependency, and filename will be built when you build actions. Notice, though that this will check whether the string exists in filename at the time the makefile is parsed -- if filename is generated, or modified in this makefile, then it would not effect whether the action is run. If you want to test right before overwriting the file, then you would use a bash if statement in the recipe itself.
Related
Updated my question as it seemed to be not clear enough!
I was listing when to use make over bash. One thing I like about make is its declarative way of describing necessary steps; we can write a rule by relying on other rules knowing how to provide necessary files (or other external states).
I'm wondering how I can get the same benefit for a value not a file, without changing outer world (like leaving a temporary file).
hello.txt: (here, tell that it needs to resolve person's name)
# Here, person's name is available.
echo Hello $(var_name) > $#
We can imperatively prepare a necessary value with $(call prepare_name, ...) at the beginning of a command in a rule, but that's not what I'm after here.
I posted my attempts as an answer when I opened this question. Hopefully that adds more info on what I'm trying to achieve.
It's not overly clear what you're after, however to clarify a few concepts:
A target must be dependent on other targets. It cannot be dependent on a variable name. It can be dependent on the value of a variable, if that variable resolves to a target name.
So you could do:
VAR=some_target
hello.txt: $(VAR)
echo "hello $^" > $#
some_target:
touch $#
You CANNOT do:
VAR=some_target
hello.txt: VAR
and expect it to work (make would try to build VAR which likely doesn't exist and it would fail).
I'm assuming from the question that you want make to request the variable name of a person, and put that into hello.txt. In that case you would likely want to store the name in a temporary file and use that for the output:
.getname.txt:
#read -p "enter name" name > $#
hello.txt: .getname.txt
#echo "hello $$(cat $$<)" > $#
This will update .getname.txt if it didn't previously exist (so it will not necessarily ask on every invokation of make...). You could make .getname.txt be a .PHONY target, and it will run every time.
If you do want to run every time, then you can simply do:
hello.txt:
#read -p "enter name: " name && echo "hello $$name" > $#
.PHONY: hello.txt
Which will invoke the hello.txt rule regardless of whether hello.txt already exists, and will always prompt the user for a name and rebuild hello.txt.
I can think of a way using eval function. Below suppose foo is a value obtained by a complex calculation.
hello.txt: var_name
echo Hello $($<) > $#
.PHONY: var_name
var_name:
$(eval $# = foo)
Or with .INTERMEDIATE target, this also works, but I feel it's more complicated.
var_name = var-name.txt
hello.txt: $(var_name)
echo Hello $$(< $<) > $#
.PHONY: $(var_name)
.INTERMEDIATE: $(var_name)
$(var_name):
rm -f $# # In case the var file already exists
echo bar > $#
Another way could be to use a target-specific variable. It's not listing a variable as a prerequisite, but I still don't need to think about how to get var_name when writing echo Hello ....
define get_name
echo foo
endef
hello.txt: var_name = $(call get_name)
hello.txt:
echo Hello $(var_name) > $#
As noted in other answers, make track dependencies between files, using timestamps. The regular solution for handling a value will be to store it in a file (or to generate it into a file). Assuming that there is significant work to do whenever the data is changing, you can follow one of the patterns below to implement dependency check on the file value.
The following makefile snapshot will trigger rebuild of complex-result, only when the content of var-value is modified. This is useful when the content of var-value is continuously regenerated, but does not change very frequently.
all: complex-result
last-value.txt: var-value.txt
cmp -s $< $# || cat <$^ > $#
complex-result: last-value.txt
echo Buildig for "$$(cat var-value.txt)"
touch $#
Or more realistic example: trigger a build if the value (content) of any file was modified, using md5 checksum,
all: complex-result
last-value.txt: $((wildcard *.data)
md5sum $^ > $#
last-value.txt: var-value.txt
cmp -s $< $# || cat <$^ > $#
complex-result: last-value.txt
echo Building for "$$(cat var-value.txt)"
touch $#
Make manual says I can have only one recipe per target. But I need to run different building tools depending on which dependencies has changed.
Something like this:
target.txt: subtarget.txt
cat $? >> $#
target.txt: header.txt
date >>$#
Is there some trick to archive it?
I'm not exactly sure what you want to do, but maybe double-colon rules will help you?
target.txt:: subtarget.txt
cat $? >> $#
target.txt:: header.txt
date >>$#
I want to know how I can convert this shell script to a Makefile and use make all to perform all the script's operations. I have two directories pictures and thumbs, where thumbs is empty before running the script and pictures contains some .jpg files. Finally, this is the shell script that I want to convert to a Makefile:
#!/bin/bash
DIR="thumbs"
if [ "$(ls -A $DIR)" ]; then
p=$(find pictures/|grep "jpg"|cut -d"/" -f2)
for i in $p
do
m=$(ls -l pictures/$i | cut -d" " -f7)
n=$(ls -l thumbs/$i | cut -d" " -f7)
if [ "${m//':'}" -gt "${n//':'}" ] ;then
rm thumbs/$i
convert -thumbnail 100 pictures/$i thumbs/$i
fi
done
else
find pictures/ |cut -d"/" -f2 | grep "jpg"| \
awk '{system("convert-thumbnail 100 pictures/" $0 " thumbs/" $0)}'
fi
From Makefile you can call a program.
Example
$ cat a.sh
echo From Makefile
$ cat Makefile
all:
./a.sh
Test :
$ make
./a.sh
From Makefile
You can implement the behavior of your shell code much more simply as a Makefile, at least if you are willing to rely on GNU make. I interpret this to be what you're asking (modulo dependency on GNU make in particular). This is a pretty functional rough cut:
THUMBS = $(patsubst pictures/%,thumbs/%,$(wildcard pictures/*jpg*))
all: $(THUMBS)
thumbs/%: pictures/% thumbs
convert -thumbnail 100 '$<' '$#'
thumbs:
mkdir -p '$#'
.PHONY: all
Notes:
the THUMBS make variable gets set to a list of the thumbnail images you want to manage, based on expanding the shell glob pictures/*jpg* and replacing each occurrence of pictures/ with thumbs/.
The pattern is chosen to match your shell code, but perhaps you really want something more like $(wildcard pictures/*.jpg)
File names with whitespace in them are going to present a tricky problem if you need to worry about them; file names with certain other special characters too, albeit a bit less so
the patsub and wildcard functions are GNU extensions
You could also merge the definition of THUMBS into the rule for all, and avoid a separate variable
The rule for thumbs/%: pictures/% thumbs uses GNU-specific pattern rule syntax; this particular form is hard to express to POSIX make.
The thumbs directory is created if absent, but errors will occur if there is an ordinary file of that name in the way
make all (or just make) will update all out-of-date thumbnails; it does not rely on the same date comparison logic as the original script (which is a good thing)
The .PHONY rule is just to be careful. It prevents the existence of an actual file named "all" from interfering with make's operation.
I have a rule like this in a makefile:
target : dependencies
rm -rd somedirectory
runcodecoverage.exe # this generates somefile
$(eval COVERAGE=$(shell grep "blah" somefile))
#echo $(COVERAGE)
When I run this file the first time (after make cleaning) the echo doesn't print anything. But the second and times after that, it prints the correct result. If I replace the $(eval ..) line with just grep "blah" somefile I get the result I want, so the problem must be in the use of $(eval) and $(shell). Why is this happening?
Edit: I solved this by adding a new dependency, so it now looks like this:
generatesomefile :
runcodecoverage.exe # this generates somefile
target : dependencies generatesomefile
rm -rd somedirectory
$(eval COVERAGE=$(shell grep "blah" somefile))
#echo $(COVERAGE)
It seems like $(eval) was being substituted with the result of grep as soon as the "target" target was entered, even though I wanted it to be run after runcodecoverage.exe had run. In this sense the answer I accepted as correct wasn't quite right - the docs say this on variable expansion:
Rule Definition
A rule is always expanded the same way, regardless of the form:
immediate : immediate ; deferred
deferred
The eval function doesn't work the way you think it does.
The first time you run Make, it expands the eval function and carries out the variable assignment before executing any rule. And since there is no somefile, the grep returns nothing and COVERAGE stays empty. When Make executes the rule, it passes the empty variable to the echo which duly reports nothing.
The second time you run Make, once again it expands the eval function, executes the grep on somefile (which was built in the first run), and stores the result in COVERAGE. Then it executes the rule and passes COVERAGE to the echo, which puts it up on your screen.
The $(eval ...) runs when your Makefile is parsed, not when the recipe is executed.
It's not clear to me what you expect to happen; setting a Make variable from within a recipe does not seem like a sustainable approach, but it depends a lot on where and how you need to use this variable. If all you need is to output the coverage from the same recipe, replace the last couple of lines with just
grep "blah" somefile
I have a makefile that defines several rules where the target is a foreach function.
$(foreach var,$(list), $($(var)_stuff) $($(var)_more_stuff)):
#echo Building $# from $^...
$(CC) $(FLAGS) ...
Is there any way to get make to quit when encountering an error without going through the entire list.
One workaround is to "manually" invoke exit on failure.
For example, assume we have a directory called scripts with a number of shell scripts (with filenames that end with .sh) that we want to execute.
Then a variable declaration like this:
LIST_OF_SCRIPTS ?= $(wildcard scripts/*.sh)
will give us a list of those scripts, and a target like this:
run-all-scripts
#$(foreach scriptfile,$(LIST_OF_SCRIPTS),$(scriptfile);)
will run all of those scripts, but as you note, the foreach loop will keep going whether or not one of the scripts returns an error code. Adding a || exit to the command will force the subcommand to exit on error, which Make will then treat as a failure.
E.g.,
run-all-scripts
#$(foreach scriptfile,$(LIST_OF_SCRIPTS),$(scriptfile) || exit;)
will do what you want (I believe).
Specifically, using your pseudo-code example, I think you want something like this:
$(foreach var,$(list), $($(var)_stuff) $($(var)_more_stuff)):
#echo Building $# from $^...
($(CC) $(FLAGS) ...) || exit
(where all I've changed is wrapping the (CC) $(FLAGS) ... bit in parens and appending || exit to make it fail on error).
The foreach is completely evaluated and substituted before any of the rules are executed. So the behaviour of this should be identical to as if you had hardcoded the rule without using the foreach. In other words, it's not directly relevant to the problem.
There are only a few possible explanations for what you're seeing, mostly described in the manual here:
You are running Make with -k or --keep-going
You are running Make with -i or --ignore-errors
Your targets is defined as prerequisites of the special .IGNORE target
Your recipe starts with a -
Your recipe isn't actually returning a non-zero exit status
Not sure about your example, but maybe problem is in ; - look at Makefile : show and execute:
dirs = $(shell ls)
clean:
$(foreach dir,$(dirs),echo $(dir);)
produce:
$ make clean
echo bin; echo install.sh; echo Makefile; echo README.md; echo utils;
So make check exit code only for last command: echo utils.