To demonstrate a major difference of how make treats single-line recipe than that of .ONESHELL-multiple-line recipe, I will use a sed command, where, we assume, that either of the following:
sed -e 'p; p'
sed -e 'p
p'
are mostly equivalent. Right?
What we find, however, is that Make fails only on the single-line recipe.
Yeah! That's right. Make handles perfectly fine the multiple-line version (due to .ONESHELL, of course), but cannot do the same for a single-line recipe.
It defies common-sense, but here is the proof.
The Makefile:
.ONESHELL :
# Define a MULTIPLE-line recipe.
define define_cmd_foo
all : cmd_foo = echo xxx | sed -ne '
s/xxx/yyy/
p'
endef
# Define a SINGLE-line recipe.
define define_cmd_bar
all : cmd_bar = echo xxx | sed -ne 's/xxx/yyy/; p'
endef
# cmd_foo is multiple-line (3 line) recipe:
# echo xxx | sed -ne '
# s/xxx/yyy/
# p'
$(define_cmd_foo)
# cmd_foo is a one and single-line recipe:
# echo xxx | sed -ne 's/xxx/yyy/; p'
$(define_cmd_bar)
# This is a 1st file in a double-colon (linked list of) target 'all'.
# This will exeucte a multipe-line recipe.
all ::
$(cmd_foo)
# This is a 2nd file in a double-colon (linked list of) target 'all'.
# This will exeucte a single-line recipe.
all ::
$(cmd_bar)
Executing, we get:
echo xxx | sed -ne '
s/xxx/yyy/
p'
yyy
echo xxx | sed -ne 's/xxx/yyy/
/bin/sh: 1: Syntax error: Unterminated quoted string
makefile:34: recipe for target 'all' failed
make: *** [all] Error 2
Can you see here 2 commands?
The first command executes a multiple-line recipe, and succeeds.
In fact, the output for the first recipe: yyy, is exactly what we expected.
Any output, from the 5th line and the following lines - of the output - above, belongs to the execution of the other single-line recipe, which looks like something went wrong there. Very wrong!
Any reason, why a perfect good recipe goes well with .ONESHELL for a multiple-line version, but fails in a plain one-line version?
Related
I have the following code:
LOCAL_VERSION := $(shell some_binary -v | head -n 1)
REMOTE_VERSION := $(shell curl -s https://example.com/key)
all:
ifeq($(REMOTE_VERSION), $(LOCAL_VERSION))
#echo yes
endfi
But I am getting this:
user:tmp user$ make
ifeq(v0.11.1, v0.11.1)
/bin/sh: -c: line 0: syntax error near unexpected token `v0.11.1,'
/bin/sh: -c: line 0: `ifeq(v0.11.1, v0.11.1)'
make: *** [all] Error
I am on Mac OSX, but it's using GNU Make anyway.
ifeq should not be indented, e.g.
LOCAL_VERSION := $(shell some_binary -v | head -n 1)
REMOTE_VERSION := $(shell curl -s https://example.com/key)
all:
ifeq ($(REMOTE_VERSION), $(LOCAL_VERSION))
#echo yes
else
#echo NO
endif
The issue is not that ifeq is indented in the recipe, the problem is that it was indented using a tab. If you indent using space, the code runs as expected.
From Make manual:
5.1 Recipe Syntax
Each line in the recipe must start with a tab (or the first character in the value of the .RECIPEPREFIX variable; see Special Variables), except that the first recipe line may be attached to the target-and-prerequisites line with a semicolon in between. Any line in the makefile that begins with a tab and appears in a “rule context” (that is, after a rule has been started until another rule or variable definition) will be considered part of a recipe for that rule. Blank lines and lines of just comments may appear among the recipe lines; they are ignored.
I am already working on a script to replace value of a variable "SUBDIRS" in a Makefile from shell script.
I used below command and it works fine but exits after doing for first occurrene of "SUBDIRS" and makefile is incomplete.
sed -z -i "s/\(SUBDIRS = \).*/\1$(tr '\n' ' ' < changed_fe_modules.log)\n/g" Makefile
Now I want to keep my Makefile as it is and only replace 3 occurrences of "SUBDIRS= abcdefgh" and update Makefile properly.
Please suggest how to just replace all 3 occurrences and keep Makefile also end to end as original.
Makefile input sample:
Makefile Desired output sample:
Right now, current command is giving me below output: it exits after first replacement and file is incomplete.
This will be very hard to do.
The reason you're seeing this behavior is that you're using the -z option with sed. The -z option separates lines with NUL characters, not newlines. This means the entire file (up to the first NUL character, which there isn't one here) is treated as a single "line" for the purposes of sed's pattern matching.
So this regex:
\(SUBDIRS = \).*
the .* here matches the entire rest of the file after the first SUBDIRS = match. Then you replace the entire rest of the file with the contents of the changed_fe_modules.log file. After that there's nothing left to match, so sed is done.
If your original makefile listed all the SUBDIRS on a single line, not using backslash/newline separators, it would be simple; you can just use:
sed -i "s/^SUBDIRS = .*/SUBDIRS = $(tr '\n' ' ' < changed_fe_modules.log)/" Makefile
If you have to use the backslash/newline you probably won't be able to make this change using sed. You'll need to use something more powerful like Perl which has non-greedy matching capabilities.
ETA
You could also write it in plain shell:
new_subdirs=$(tr '\n' ' ' < changed_fe_modules.log)
line_cont=false
in_subdirs=false
while read -r line; do
if $line_cont; then
case $line in
(*\\) : still continuing ;;
(*) line_cont=false ;;
esac
$in_subdirs || printf '%s\n' "$line"
continue
fi
case $line in
(SUBDIRS =*)
echo "SUBDIRS = $new_subdirs"
in_subdirs=true ;;
(*) printf '%s\n' "$line"
in_subdirs=false ;;
esac
case $line in
(*\\) line_cont=true ;;
esac
done < Makefile > Makefile.new
mv -f Makefile.new Makefile
(note, completely untested)
Using GNU Make 4.1, in my Makefile I can create a timestamp log file with this::
flush_log:
#echo "==> flush logs"
cat file > file_`date +%FT%T%Z`.log
But who to put:
file_`date +%FT%T%Z`.log
in a var inside the Makefile e.g to make a wc on it ?
I've try (with no success):
flush_log:
#echo "==> flush logs"
logfile:=file_$(shell date +%FT%T%Z).log
cat my_log > $(logfile)
echo `wc -l $(logfile)`
I get this error:
$ make flush_log
==> flush logs
logfile:=file_2016-12-24T20:09:52CET.log
/bin/sh: 1: logfile:=file_2016-12-24T20:09:52CET.log: not found
Makefile:7: recipe for target 'flush_log' failed
make: *** [flush_log] Error 127
$
I was following recommendation from https://stackoverflow.com/a/14939434/3313834 and Simply expanded variables https://www.gnu.org/software/make/manual/html_node/Flavors.html#Flavors
Every line in a makefile (that appears after a rule statement) that is indented with a TAB character is part of the rule's recipe. Lines in the rule's recipe are passed to the shell for handling, they're not parsed by make (except to expand variable/function references).
So your line logfile:=file... is being passed to the shell and being interpreted by the shell... and there is no valid := operator in the shell, so the shell thinks that entire line is a single word and tries to run a program with that name, which obviously doesn't exist.
You probably want to create a make variable, which must be done outside of the recipe, like this:
logfile := file_$(shell date +%FT%T%Z).log
flush_log:
#echo "==> flush logs"
cat my_log > $(logfile)
echo `wc -l $(logfile)`
I have a file sedstr.sh containing a function sedstr:
#!/bin/bash
function sedstr {
# From stackoverflow.com/a/29626460/633251 (Thanks Ed!)
old="$1"
new="$2"
file="${3:--}"
escOld=$(sed 's/[^^]/[&]/g; s/\^/\\^/g' <<< "$old")
escNew=$(sed 's/[&/\]/\\&/g' <<< "$new")
sed -i.tmp "s/\<$escOld\>/$escNew/g" "$file" # added -i.tmp
echo "sedstr done"
}
I have an external file "test" to be edited in place with these contents:
My last name is Han.son and I need help.
If the makefile works, I'll have a new last name.
I want to call the sedstr function with its arguments from a makefile. Nothing should be returned, but the external file should be edited. Here is a small makefile that doesn't work:
all: doEdit
doEdit:
$(shell ./sedstr.sh) # I was hoping this would bring the function into the scope, but nay
$(shell sedstr 'Han.son', 'Dufus', test)
How can I call this function using variables in the makefile? The error is:
make: sedstr: Command not found
make: Nothing to be done for `all'.
Each line in a make recipe is executed in its own shell.
Similarly, so is each call to $(shell).
They don't share state.
To do what you want ould would need a recipe line of
$(shell . ./sedstr.sh; sedstr 'Han.son' 'Dufus' test)
That being said there's no reason to use $(shell) here at all as you are already in a shell context and as you can just as easily (and more correctly) use a recipe line of
. ./sedstr.sh; sedstr 'Hans.son' 'Dufus' test
And yes, the commas in the original are just incorrect.
You can call the function from inside sedstr.sh. At the end
sedstr "$1" "$2" "$3"
EDIT
Or see other answer
I have a file, tmp containing:
{
"VolumeId": "vol--22222222",
}
and a makefile, containing only:
MYFILE =latest
x:
\cp tmp $(MYFILE)
grep VolumeId $(MYFILE)
#echo $(shell grep VolumeId $(MYFILE))
If I run make x, I get:
\cp tmp latest
grep VolumeId latest
"VolumeId": "vol--22222222",
VolumeId: vol--22222222,
as expected. If I modify the file tmp, replacing 2's wit 4's, I get:
\cp tmp latest
grep VolumeId latest
"VolumeId": "vol--44444444444",
VolumeId: vol--22222222,
...huh? The greps return different results! The second contains data from the file before the copy.
I run
rm latest ; make x
I get:
\cp tmp latest
grep VolumeId latest
"VolumeId": "vol--4444444444",
What's going on here?
GNU Make 3.81. Ubuntu 12.04 on VMWare 5.
Update 1
Here's a more explicit example
CMD0 =$(shell date +"%s.%N")
CMD1 =date +"%s.%N"
CMD2 =date +"%s.%N"
y:
#sleep 2
#date +"%s.%N" # 2nd
#echo $(CMD0) # 1st A
#sleep 2
#date +"%s.%N" # 3rd
#sleep 2
#$(CMD1) # 4th
#sleep 2
#echo $(shell $(CMD2) ) # 1st B
Output:
1381596581.761093768
1381596579.743610973
1381596583.769058027
1381596585.774766561
1381596579.751625601
Its look like all the $(shell ... ) commands get evaluated together before any line of the y recipe.
This is a little surprising (and counter-intuitive - what's the rationale for this odd design?). It has important consequences if the shell commands have side effects. Also, I can't find any documentation in make manual that describes this order of evaluation.
Update 2
What's especially odd about the above is that its hard to come up with a mental model for the order of evaluation. Is it that any rule of the form, $(function ... ) is executed before each line of the recipe? If so, why is $(CMD0) evaluated before the recipe, but not $(CMD1). Its true that CMD0 contains a $(f ... ) - does make look at this, even though CMD0 is declared as a delayed evaluated variable (i.e.declared with = not :=)?
Update 3
Reducing it down to essential components:
notSafe:
backup-everything # Format executed even if backup fails
echo $(shell format-disk) | tee log.txt #
CMDX =$(shell format-disk)
alsoNotSafe:
backup-everything # Format executed even if backup fails
echo $(CMDX) | tee log.txt # Even though CMDX is a delayed evaluation variable
CMDZ =format-disk
safe:
backup-everything # Works as expected.
$(CMDZ) | tee log.txt # Evaluation of CMDZ is delayed until line is executed.
Mad. Who designed this?
You're not telling us everything: when you run it the 1st time, you echo an empty line. Instead you get a grep file not found error.
And that's the decisive hint: $-expressions are evaluated by any variant of make before running the commands of the rule. I.e. in each later run you see the file contents of the previous run.
Btw. your leading backslash is spurious and causes the command to be passed to the Shell, instead of being directly executed by make. With makepp's builtin commands (prefixed by &) and fixed dependencies that would be:
latest: tmp
&cp $(input) $(output)
&grep VolumeId $(output)
#&echo $(&grep VolumeId $(output))
That doesn't change the problem, but by using the proper rule variables, makes it obvious what's wrong.