i need to make a makefile which compares my written command grep output with .out file. The input text file which grep searches is in tests/*.test.
Beta's answer was very helpfull, except i cannot understand why this makefile
INPUT_DIR=tests
OUTPUT_DIR=outputs
FILES=$(wildcard ${INPUT_DIR}/*.test)
TEST_LIST = ${FILES:${INPUT_DIR}/%.test=%.diff}
.PHONY: all clean distclean
all: ${TEST_LIST}
echo ${FILES}
echo ${TEST_LIST}
${TEST_LIST}: ${INPUT_DIR}/%.test ${OUTPUT_DIR}/%.out
./grep apple $< >STDOUT
diff STDOUT ${OUTPUT_DIR}/$*.out > $#
${INPUT_DIR}/%.test:
echo $#
${OUTPUT_DIR}/%.out:
echo $#
clean:
rm -f ${FILES}
using make makefile all -n
make: Nothing to be done for `makefile'.
echo tests/%.test
echo outputs/%.out
./grep apple tests/%.test >STDOUT diff STDOUT outputs/.out > 1.diff
./grep apple tests/%.test >STDOUT diff STDOUT outputs/.out > 2.diff
./grep apple tests/%.test >STDOUT diff STDOUT outputs/.out > 3.diff
echo tests/1.test tests/2.test tests/3.test
echo 1.diff 2.diff 3.diff
and using make makefile all
make: Nothing to be done for `makefile'.
echo tests/%.test tests/%.test
echo outputs/%.out outputs/%.out
./grep apple tests/%.test >STDOUT
Your question is unclear, but since you use the file STDOUT as an intermediate file, and you use it for every comparison, and you use it for nothing else, I advise you to combine the rules:
%.diff: ${INPUT_DIR}/%.test ${OUTPUT_DIR}/%.out
./grep apple $< >STDOUT # Do you really have an executable called "grep"?
diff STDOUT ${OUTPUT_DIR}/$*.out > $#
Also, your all rule doesn't do what you expect. Try this:
all:
#echo you must specify a target, e.g. outputs/foo.diff
Related
The working example below always runs touch fileA with each run of make. How can I stop this?
all: analysis.txt
analysis.txt: countries
touch $#
countries: usa.txt mexico.txt
$(countries): %.txt
%.txt:
echo "bash ./cityGenerator $*" > $#
fileA depends on workflowB. So as long as workflowB is out of date, fileA will be out of date and the touch will be run.
workflowB, as shown here, is marked .PHONY so it will ALWAYS be considered out of date by make, so fileA will ALWAYS be rebuilt.
Even if it weren't .PHONY, there is no rule to create a file named workflowB so it would still always be out of date.
As for how you can stop it, that depends entirely on what these things do which you haven't told us, and why you structured workflowB that way which you also haven't told us.
One way to solve it would be to remove the .PHONY and give workflowB a recipe that touches a file, then it would be up to date.:
workflowB: workflowB_files
touch $#
But of course, that might defeat the purpose of this (which, again, you haven't told us).
Your analysis.txt depends on countries which is considered as a file, but such file is never created, hence the rule is always run. You can find out by the debug output:
$ make -dr
...
Finished prerequisites of target file 'analysis.txt'.
Prerequisite 'countries' of target 'analysis.txt' does not exist.
Must remake target 'analysis.txt'.
touch analysis.txt
...
It seems like you are treating countries as a variable, so maybe this could be reworked:
$ cat Makefile
all: analysis.txt
countries := usa.txt mexico.txt
analysis.txt: $(countries)
touch $#
%.txt:
echo "bash ./cityGenerator $*" > $#
Now there is no countries file considered and make behaves as expected:
$ make
echo "bash ./cityGenerator usa" > usa.txt
echo "bash ./cityGenerator mexico" > mexico.txt
touch analysis.txt
$ make
make: Nothing to be done for 'all'.
I am trying to write a simple Makefile to build .expected files and compare them but I am failing.
APSSCHED=../../bin/apssched
BASE=.:../../base:../../examples
FLAGS=-DCOT
EXAMPLES=../../examples/
CASES=simple-binding1 simple-binding2
# skipping lines doesn't work ...
# run command and skip the first line
%.aps:
${APSSCHED} ${FLAGS} -p ${BASE} ${EXAMPLES}/$* | tail -n +2
# get all cases as an array to pipe it to different make targets
# maybe overcomplicating
cases:
echo ${CASES} | \
awk '{split($$0,numbers," ")} END {for(n in numbers){ print numbers[n] }}'
# create all .expected files from ${CASES}
build.expected:
$(MAKE) cases | xargs -n1 -I file /bin/bash -c '$(MAKE) file.build.expected'
# create single .expected file
%.build.expected:
$(MAKE) $*.aps > $*.expected
# compare result with
%.compare:
$(MAKE) $*.aps | diff $*.expected -
# run command for all cases and diff the result with corresponding expected
all:
$(MAKE) cases | xargs -n1 -I file /bin/bash -c '$(MAKE) file.compare'
clean.expected:
rm *.expected
Running make without any target and nothing happens.
echo simple-binding1 simple-binding2 | \
awk '{split($0,numbers," ")} END {for(n in numbers){ print numbers[n] }}'
simple-binding1
simple-binding2
I think the issue is with my cases target. I am not sure if I am on the right track.
I appreciate any help or hint.
I would avoid re-running make just to call a different target - it's a performance hit and may be unreliable (depending on rest of the Makefile) since separate calls may not be able to track dependencies correctly.
Moreover, I would avoid using | - every time a command is concatenated with pipe, exit code of piped command would be exit code of the last command. So a call like command | tail would return the exit code of tail (which would almost always succeed). Even if the command has failed, it would be covered with exit code 0 from tail and make will not detect the error and will not stop.
Thus said, I tried to rewrite your approach by just creating dependencies between the targets, like so:
$ cat Makefile
APSSCHED=../../bin/apssched
EXAMPLES=../../examples
BASE=.:../../base:$(EXAMPLES)
FLAGS=-DCOT
CASES=simple-binding1 simple-binding2
# Just for reproducing
$(EXAMPLES)/%.aps: ;
# Generate output and store it in a file
%.output: $(EXAMPLES)/%.aps
# echo is only for reproducing
echo $(APSSCHED) $(FLAGS) -p $(BASE) $< > $#
# Copy actual output as expected
%.expected: %.output
cp -f $< $#
# Compare actual output with expected
.PHONY: %.compare
%.compare: %.output | %.expected
diff $| $<
# Generate and verify all outputs
.PHONY: all
all: $(addsuffix .compare,$(CASES))
# Regenerate expected output
.PHONY: build.expected
build.expected: $(addsuffix .expected,$(CASES))
.PHONY: clean.expected
clean.expected:
-rm -f *.expected
Now the make build.expected will create expected output files, while make all or make will check the actual output against expected:
$ make build.expected
echo ../../bin/apssched -DCOT -p .:../../base:../../examples ../../examples/simple-binding1.aps > simple-binding1.output
cp -f simple-binding1.output simple-binding1.expected
echo ../../bin/apssched -DCOT -p .:../../base:../../examples ../../examples/simple-binding2.aps > simple-binding2.output
cp -f simple-binding2.output simple-binding2.expected
rm simple-binding1.output simple-binding2.output
$ make
echo ../../bin/apssched -DCOT -p .:../../base:../../examples ../../examples/simple-binding1.aps > simple-binding1.output
diff simple-binding1.expected simple-binding1.output
echo ../../bin/apssched -DCOT -p .:../../base:../../examples ../../examples/simple-binding2.aps > simple-binding2.output
diff simple-binding2.expected simple-binding2.output
rm simple-binding1.output simple-binding2.output
I have few simple targets which create some files for me.
Example:
$(MAKE_INA):
#echo Building ASM compilation flags file $(notdir $(MAKE_INA))
#$(foreach i, $(sort $(ASMFLAGS) $(PFLAGS) $(ALL_INC_DIR) $(cppGetPreProcessorDefines)), $(shell echo $i >> $# ))
The target works fine, the file is being created and echo text displayed, but in that order (first the file is build then the echo is shown on cmd.exe console).
I guess that is related somehow with output buffering, but I was not able to find the way to flush the echos immediately.
Any hint? Is it even possible?
I am using Gnu Make 4.0
You are mixing up contexts here.
The first #echo line is a recipe line and is run by the shell when the target runs.
The second $(foreach) line is within the rule but is a make context line and is evaluated by make before running the recipe lines. Within that line $(shell) is also a make command and is run during the make expansion of the recipe instead of being run by the shell at recipe execution time.
To do what you want you can just use:
$(MAKE_INA):
#echo Building ASM compilation flags file $(notdir $(MAKE_INA))
#printf "%s\\n" $(sort $(ASMFLAGS) $(PFLAGS) $(ALL_INC_DIR) $(cppGetPreProcessorDefines)) >> $#
Which does the echoing at recipe execution time (so has the right order) and uses a single call to the printf built-in to output to the file instead of running N calls to echo.
Edit: For Windows cmd.exe compat you need to use echo $i >> $# & as the $(foreach) body so that cmd.exe runs multiple commands correctly.
If you did want to keep the N echo calls then you could use:
$(MAKE_INA):
#echo Building ASM compilation flags file $(notdir $(MAKE_INA))
#$(foreach i, $(sort $(ASMFLAGS) $(PFLAGS) $(ALL_INC_DIR) $(cppGetPreProcessorDefines)), echo $i >> $#; ))
Which has the $(foreach) output echo XXX >> $#; ....; echo ZZZ >> $#; as the recipe line to then execute during recipe execution.
I'm trying to create a makefile (GNU make) that does the following:
A script generates a bunch of files--filenames not known in advance.
Each one of these files is converted to a different file.
After all are converted, all of these files are combined into a single output file.
How do I create a makefile with a "bellcurve"-patterned dependency graph, where the intermediate source and target files are not known in advance?
Conceptually I'm doing the following:
combined.pdf: $(filter combined.pdf, $(wildcard *.pdf))
cat *.pdf > combined.pdf
%.pdf: %.svg
cp $^ $#
$(wildcard *.svg):
# recipe is for simple example
# actually the *.svg files are not known in advance
echo a > a.svg
echo b > b.svg
echo c > c.svg
.PHONY: clean
clean:
${RM} *.svg *.pdf *.d
Of course this doesn't work: Make evaluates the targets and sources before it runs the target that actually creates the svg. Also, there's no way to make sure all svgs are converted before they are combined.
I realized I could create dependencies and include them into the makefile, but I had trouble getting this to work too:
.PHONY: clean
include deps.d
combined.pdf: deps.d
cat *.pdf > combined.pdf
%.pdf: %.svg
cp $^ $#
deps.d:
## recipe is for simple example
## actually the *.svg files are not known in advance
echo a > a.svg
echo b > b.svg
echo c > c.svg
## we know what files exist now, so we can establish dependencies
## "a.pdf : a.svg"
echo *.svg : > deps.d
## combined.pdf: a.pdf b.pdf c.pdf
ls *.svg \
| awk '{targetfn=$$0; sub(/\.svg$$/, ".pdf", targetfn); print targetfn, ":", $$0;}' \
>> deps.d
## combined.pdf: a.pdf b.pdf c.pdf
echo combined.pdf : $$(echo *.svg | sed -e 's/\.svg/\.pdf/g') >> deps.d
clean:
${RM} *.pdf *.svg *.d
However this still isn't connecting the dependency graph properly. When I run this, make quits as follows:
Makefile:3: deps.d: No such file or directory
echo a > a.svg
echo b > b.svg
echo c > c.svg
echo *.svg : > deps.d
ls *.svg \
| awk '{targetfn=$0; sub(/\.svg$/, ".pdf", targetfn); print targetfn, ":", $0;}' \
>> deps.d
echo combined.pdf : $(echo *.svg | sed -e 's/\.svg/\.pdf/g') >> deps.d
make: Nothing to be done for `a.svg'.
I still seem to have the problem that the make doesn't know about the rules in deps.d.
Also, this still doesn't solve the problem of building all the dependencies. I thought of using a marker file like this:
%.pdf: %.svg
cp $^ $#
## if all svgs are converted, touch a target allpdfs
if [ $(ls -1 *.svg | wc -l) -eq $(ls -1 *.pdf | grep -v combined\.pdf | wc -l) ]; touch allpdfs; fi
But there's no way to inform make that "allpdfs" may be created by this rule.
I'm surprised that moving the include directive makes a difference (what version of Make are you using?), but there is a simpler way. Your use of deps.d is in effect a recursive use of Make -- Make is arranging to execute itself a second time -- so we might as well make it official:
combined.pdf: ALL_SVGS
$(MAKE) ALL_PDFS
rm -f $# # just in case it exists already
cat *.pdf > $#
.PHONY: ALL_SVGS
ALL_SVGS:
# recipe is for simple example
# actually the *.svg files are not known in advance
echo a > a.svg
echo b > b.svg
echo c > c.svg
# These variables will be empty in the first execution of Make
SVGS = $(wildcard *.svg)
PDFS = $(patsubst %.svg,%.pdf,$(SVGS))
.PHONY: ALL_PDFS
ALL_PDFS: $(PDFS))
%.pdf: %.svg
cp $^ $#
This isn't an answer exactly, because I don't know why this works, but I discovered that if I move the include directive after the target that creates the included file, everything works.
I.e. do this:
deps.d:
....
include deps.d
Because my deps.d includes enough dependency information, there's no need to have an intermediate target allpdfs file. Everything Just Works, even with make -j.
However, I don't know why this works. The include documentation isn't enlightening me.
UPDATE
I noticed the following note at the very bottom of the make manual discussing Automatic Prerequisites:
Note that the ‘.d’ files contain target definitions; you should be sure to place the include directive after the first, default goal in your makefiles or run the risk of having a random object file become the default goal. See How Make Works.
So what happened is that the first rule inside the generated deps.d became the default target, causing the mysterious premature completion of the build. So the solution is just to make sure include directives are not before your intended default target.
I was just working on this exact problem in a slightly different setting. Here is a clean solution - no need for recursion and such (and you can tweak the sed if you like):
include deps.d
combined.pdf:
cat *.pdf > combined.pdf
%.pdf: %.svg
cp $^ $#
deps.d:
echo a > a.svg
echo b > b.svg
echo c > c.svg
echo 'combined.pdf:' *.svg | sed 's/\.svg/\.pdf/g' > deps.d
Enjoy!
I want to process each source code file after it has been preprocessed:
myprocess `gcc -E file1.c`
myprocess `gcc -E file2.c`
...
myprocess `gcc -E fileN.c`
This gets tedious so how do I make this a single command?
That is, something along the line:
myprocess SOMETHINGMAGIC(gcc -E file*.c)
Thanks in advance!
You mean like
for i in file*.c ; do
myprocess `gcc -E $i`
done
If this is part of an ongoing processes (as opposed to a one time thing), use make, it is good at automating work pipelines.
In particular use suffix rules with traditional make or gmake style implicit rules.
Here is an outline for a suffix rule implementation:
.c.cpre:
$(CC) -E $(CPPFLAGS) -o $# $<
.cpre.cmy:
$(MY_PROCESS) $<
# Or whatever syntax you support..
#
# You could
# $(RM) $<
# here, but I don't recommend it
.cmy.o:
$(CC) -c $(CFLAGS) $(CPPFLAGS) -o $# $<
# $(RM) $<
No magic nedded, just
my_process $(gcc -E *.c)
Note that I used the $(command) form because backticks are deprecated.
As an aside: are you sure you want to do that? You are putting the whole output of gcc -E as command line parameters of my_process. Not sure this is what you want. Maybe you want to use a simple pipe
gcc -E file.c | my_process
so that the output of gcc becomes the input of my_process.
In the latter case something like
for c_file in *.c ; do
gcc -E $c_file | myprocess > ${c_file}.processed
done
would do the trick.
Perhaps this is what you're looking for?:
for i in file*.c ; do
echo "Header for myprocess: $i"
gcc -E $i 2>&1
done | myprocess
2>&1 assumes you want to grab stderr, too
echo ... gives myprocess a starting point for each compilation
How about:
myprocess {$(gcc -E file1.c),$(gcc -E file2.c),$(gcc -E file3.c)}
Or if you pipe it:
{$(gcc -E file1.c),$(gcc -E file2.c),$(gcc -E file3.c)} | myprocess
It's been a while since I've used bash, so, please, point out my mistakes!
ls file*.c | xargs --replace myprocess $(gcc -E {})