I have a Makefile that looks like this:
run_experiment1:
python some_file.py \
--data_file /somewhere/here \
--k ${k}
run_all_experiments:
for k in 1 2 3 4 5 ; do \
run_experiment1 k=$$k ; \
done
When I run make run_all_experiments I get:
for k in 1 2 3 4 5 ; do \
run_experiment1 k= ; \
done
/bin/sh: 2: run_experiment1: not found
/bin/sh: 2: run_experiment1: not found
/bin/sh: 2: run_experiment1: not found
/bin/sh: 2: run_experiment1: not found
/bin/sh: 2: run_experiment1: not found
Makefile:84: recipe for target 'run_all_experiments' failed
make: *** [run_all_experiments] Error 127
What I can immediately notice is that there doesn't seem to be a value being input for k like I'd expect, and the command's not being found. How should I go about fixing this? Thanks.
This is really an antipattern. You want to structure your Makefile so that make itself is in charge.
The immediate and obvious fix is that, like the error message hints, the correct command here is make run_experiment1:
.PHONY: run_experiment1 run_all_experiments
run_experiment1:
python some_file.py \
--data_file /somewhere/here \
--k $$k # notice doubled dollar sign to escape it from make
run_all_experiments:
for k in 1 2 3 4 5 ; do \
$(MAKE) run_experiment1 k=$$k ; \
done
However, as outlined above, I would probably refactor it to something like
.PHONY: run_experiment% run_all_experiments
run_experiment%:
python some_file.py \
--data_file /somewhere/here \
--k $*
run_all_experiments: $(patsubst %,run_experiment%,1 2 3 4 5)
Tangentially, targets which do not correspond to a file with that name should be declared .PHONY.
I am learning Makefile and trying to implement parallelism. I am aware of the "-j" option. However, for example having the following makefile (on Windows)-
all: a b c d
a:
# some build rule
b:
# some build rule with parallelism
c:
# some build rule
d:
#some build rule
I am trying to run make all with only target "b" running in parallel. Passing the -j option with the build rule for "b" doesn't work. Any pointers?
You could get b's recipe to run in the background as so:
all: a b c d
#echo running $#
.PHONY: a b c d all
a c d: | b
#echo -n _$#0 && \
sleep 1 && echo -n _$#1 && \
sleep 1 && echo _$#2
b:
#(echo -n _$#0 && \
sleep 2 && echo -n _$#1 && \
sleep 2 && echo -n _$#2\
) &
Which outputs:
_b0_a0_a1_b1_a2
_c0_c1_b2_c2
_d0_d1_d2
running all
The order-only dependency on b makes b run first, otherwise it wouldn't start until after a completes with -j1... It does of course mean that you have to build b if you build either a c or d.
Alternatively, (and I'm not recommending this) you could use some manual locking mechanism such as flock to prevent a, c, and d from running in parallel (note that the flock only protects a single shell, so you would have to collapse your recipes into a single line protected by flock for this to work).
I have a Makefile that runs pandoc. I want to turn a list of extensions:
PANDOC_EXTENSIONS = \
multiline_tables \
some_other_extension
into a string that looks like:
PANDOC_EXTENSION_LIST = +multiline_tables+some_other_extension
which will then be passed as a command line option to pandoc like this:
pandoc --from$(PANDOC_EXTENSION_LIST) ...
It's trivial in almost any programming language, but I can't figure out how to do this with the patsubst or subst functions, since make doesn't really have lists. Any ideas?
Here:
Makefile
PANDOC_EXTENSIONS = \
multiline_tables \
some_other_extension
$(foreach word,$(PANDOC_EXTENSIONS),$(eval PANDOC_EXTENSION_LIST := $(PANDOC_EXTENSION_LIST)+$(word)))
.PHONY: all
all:
echo $(PANDOC_EXTENSION_LIST)
Which runs like:
$ make
echo +multiline_tables+some_other_extension
+multiline_tables+some_other_extension
As this illustrates, GNU make really does have lists. A sequence of whitespace-separated words is a list.
Based on example in documentation:
empty:=
space:=$(empty) $(empty)
PANDOC_EXTENSIONS = \
multiline_tables \
some_other_extension
all:
#echo +$(subst ${space},+,${PANDOC_EXTENSIONS})
The result:
$ gmake
+multiline_tables+some_other_extension
This question is related to Convert Cygwin path to Windows path in a makefile but it is not the same.
I need to convert a Windows path like:
C:\src\bin
into a Unix path like:
/c/src/bin
Inside a makefile, I can use the following code to convert such paths:
slashedpath = $(subst \\,\/,$(windowspath))
unixpath = $(shell cygpath -u $(slashedpath))
How can I perform the same conversion in a makefile that is being processed by GNU Make, when the cygpath function is not available?
p.s.
What if $(windowspath) contains multiple paths? How to convert them all ?
The makefile:
windowspath=C:\src\bin
unixpath=$(subst \,/,$(subst C:\,/c/,$(windowspath)))
all:
#echo "$(windowspath)"
#echo "$(unixpath)"
gives the output:
C:\src\bin
/c/src/bin
This will also work if $(windowspath) contains multiple paths. Tested on GNU Make 4.2.1 for i686-pc-cygwin, and also on GNU Make 3.81 built for i686-redhat-linux-gnu.
I was surprised that this worked.
Update: This second version will handle various drives such as C:, D:, etc. Some of these ideas are from Eric Melski's answer to In GNU Make, how do I convert a variable to lower case?. If the Makefile is:
DRIVE = $(subst \
A:,/a,$(subst B:,/b,$(subst C:,/c,$(subst D:,/d,$(subst \
E:,/e,$(subst F:,/f,$(subst G:,/g,$(subst H:,/h,$(subst \
I:,/i,$(subst J:,/j,$(subst K:,/k,$(subst L:,/l,$(subst \
M:,/m,$(subst N:,/n,$(subst O:,/o,$(subst P:,/p,$(subst \
Q:,/q,$(subst R:,/r,$(subst S:,/s,$(subst T:,/t,$(subst \
U:,/u,$(subst V:,/v,$(subst W:,/w,$(subst X:,/x,$(subst \
Y:,/y,$(subst Z:,/z,$1))))))))))))))))))))))))))
drive = $(subst \
a:,/a,$(subst b:,/b,$(subst c:,/c,$(subst d:,/d,$(subst \
e:,/e,$(subst f:,/f,$(subst g:,/g,$(subst h:,/h,$(subst \
i:,/i,$(subst j:,/j,$(subst k:,/k,$(subst l:,/l,$(subst \
m:,/m,$(subst n:,/n,$(subst o:,/o,$(subst p:,/p,$(subst \
q:,/q,$(subst r:,/r,$(subst s:,/s,$(subst t:,/t,$(subst \
u:,/u,$(subst v:,/v,$(subst w:,/w,$(subst x:,/x,$(subst \
y:,/y,$(subst z:,/z,$1))))))))))))))))))))))))))
windowspath = c:\src\bin D:\FOO\BAR
unixpath = $(subst \,/,$(call DRIVE,$(call drive,$(windowspath))))
all:
#echo Original: "$(windowspath)"
#echo Modified: "$(unixpath)"
then the output to make is:
Original: c:\src\bin D:\FOO\BAR
Modified: /c/src/bin /d/FOO/BAR
Update 2: The most straight-forward approach, and the most flexible, is to use a standard regular-expression handler such as perl or sed, if these are available. For example, with GNU sed, this Makefile will work as required:
windowspath = c:\src\bin D:\FOO\BAR
unixpath = $(shell echo '$(windowspath)' | \
sed -E 's_\<(.):_/\l\1_g; s_\\_/_g')
all:
#echo Original: "$(windowspath)"
#echo Modified: "$(unixpath)"
Explanation of sed:
s_\<(.):_/\l\1_g For every word starting with something like A: or a:, replace the start with /a.
s_\\_/_g Replace all backslashes with forward slashes.
I got confused with Makefile. I am trying to run a simple command in the Makefile but it gives me the error "/bin/bash: line 3: :=: command not found". I am using shell to run this makefile
This is my part of my Makefile:
all:
vlog Benchmarks/$(NAME)/Syn/*.v
$(eval tux_number := 1)
$(eval range := 1)
$(eval ssh_log := 255)
echo "Start Range: ${range}"
echo "tux-number: ${tux_number}"
while [[ $$range -le 50 ]] ; do \
ssh -l yazdanbakhsh tux-$(tux_number).cae.wisc.edu exit ; \
echo "range: ${range}" ; \
eval $$range := $$((${range}+1)) ; \
done
Thanks
all:
#range=1; \
while [ $$range -le 10 ] ; \
do echo Range: $$range; \
let range=range+1 ; \
done;
Note that the whitespace in front of #range... is the only TAB.
Just to fix your obvious problems with Makefile syntax, here is an attempt at refactoring your attempt into valid code.
tux_number := 1
ssh_log := 255 # not used anywhere
all:
vlog Benchmarks/$(NAME)/Syn/*.v
echo "Start Range: 1" # This is probably no longer very useful output
echo "tux-number: ${tux_number}"
range=1; while [ $$range -le 50 ] ; do \
ssh -l yazdanbakhsh tux-$(tux_number).cae.wisc.edu exit ; \
echo "range: $$range" ; \
range=$$(expr "$$range + 1); \
done
Notice how tux_number and ssh_log are Makefile variables, while range only exists in the shell which executes the while loop. I have avoided the Bashisms in order to make this portable. (If portability is not important, you might want to refactor it back to Bash syntax and use for ((range=1; range<=50; range++)); do... instead.)
Your use of eval is misguided. As you can see, I simply lifted out the Makefile variables outside the recipe where they don't belong. What you were doing was (1) have Make evaluate the expression range := 1 (which evaluates to itself) and (2) use the output as a shell command in a recipe. Since it's not a valid shell command, you got the syntax error from Bash. Without further ado, I'll just take the easy way out here and say that eval is a complex subject, and until you get more experience with Make, it's probably just best to forget that it exists.
In order to properly make use of Make's facilities, I would make this parallelizable, i.e. split it up into 50 individual targets. This is a bit clumsy (there's probably a better way to define range here), but at least it should illustrate a number of differences to your approach. (If you don't insist on having range count up from 1, making it zero-based would make this a little less clumsy. This exploits the fact that the empty string is harmless in a shell snippet, so we can use it instead of a zero prefix. Again, this could be simplifed if you don't care about the human readability of the range index.)
digits := 0 1 2 3 4 5 6 7 8 9
deca := "" 1 2 3 4
range := $(filter-out ""0,$(foreach d,$(deca),$(foreach i,$(digits),$d$i))) 50
# Or, at the expense of an external process,
# range := $(shell perl -le 'print $$_ for 1..50')
.PHONY: all
all: $(patsubst %,ssh-%,$(range))
.PHONY: ssh-%
ssh-%:
ssh -l yazdanbakhsh tux-$(tux_number).cae.wisc.edu exit
echo "range: $*"
This can be run with something like make -j 5 to execute these in parallel batches of five, for example.
Incidentally, the commented-out $(shell ...) call might be the actual answer to your question, if what you really wanted to do was to use Make to drive an external program to calculate something for you.