What is a simple way to ignore all output from GNU make's building prereqs, but keep all the rest (both stdout and stderr). I'm working with a script that randomly uses either stream. So, for example, if I have
t: main.native ${STUFF}
./${SCRIPTS}/run-tests
How can I just do make t &> output and get the output without all the rebuild info from the prerequisites prepended (eg. only outputs from run-tests in the above example)?
I'm still not quite sure what you're looking for, but you can do this:
t: main.native ${STUFF}
#echo PREREQS_DONE
./${SCRIPTS}/run-tests
>make t | sed '1,/PREREQS_DONE/d'
Or this:
t:
#$(MAKE) real_t | sed '1,/PREREQS_DONE/d'
real_t: main.native ${STUFF}
#echo PREREQS_DONE
./${SCRIPTS}/run-tests
How can I just do make t &> output and get the output without all the rebuild info from the prerequisites prepended (eg. only outputs from run-tests in the above example)?
Build targets main.native ${STUFF} first and then run tests only. E.g.
tests : main.native ${STUFF}
run_tests : tests
./${SCRIPTS}/run-tests
.PHONY : tests run_tests
And then:
make tests && make run_test &> output
Related
I am currently using a makefile for cocotb, similar to this.
That consists of a bunch of variable definitions followed by and
include $(shell cocotb-config --makefiles)/Makefile.sim
Where the final commands will be executed.
I run it in Visual studio code, the simulator output message format is (%file,%line|%column). I want them to be shown as (%file:%line:%col). I can easily do it piping the output to sed.
make MODULE=x TESTCASE=y | sed -e 's/,\([0-9]\+\)|\([0-9]\+\)):/:\1:\2):/g'
I was expecting to be able to change the make file so that the output will be edited before shown
Something like
...
.DEFAULT:
make -C $PWD -f $(shell cocotb-config --makefiles)/Makefile.sim \
(all variables/environment) (rule) \
| sed -e 's/,\([0-9]\+\)|\([0-9]\+\)):/:\1:\2):/g
Where "magic rule" would apply the specified rule and pipe its output to sed.
How can I achieve this?
Thank you.
Given your example, it seems like the .DEFAULT target would do what you want:
.DEFAULT:
$(MAKE) -f $$(cocotb-config --makefiles)/Makefile.sim $# \
| sed -e 's/,\([0-9]\+\)|\([0-9]\+\)):/:\1:\2):/g
Edit (from the OP)
The default works when the rule is given, e.g. if I invoke make sim instead of make. For this I can create a default-delegate rule.
More importantly, I have to pass all the variables to the sub make, that can be achieved by using export by itself.
Then, replacing the include command but the following snippets, works for the more common cases.
export
default-delegate: sim
.DEFAULT:
$(MAKE) -f $$(cocotb-config --makefiles)/Makefile.sim $# \
| sed -e 's/,\([0-9]\+\)|\([0-9]\+\)):/:\1:\2):/g'
I know makefile won't allow using a target specific variable as a target prerequisite.
My question is slightly different : is there a way to generate the same file differently depending on what target was called ?
For instance, let's say I want to be able to generate file_to_generate using two different methods that I call using make example_target_1 or make example_target_2
As an example, the following code gives 2 different recipes for the same file :
example_target_1 : file_to_generate-receipe1
example_target_2 : file_to_generate-receipe2
file_to_generate-receipe1:
/* some shell code here that end up generating file_to_generate */
file_to_generate-receipe2:
/* some different shell code here that also generates file_to_generate*/
issuing make example_target_1 will generate the file using one recipe while issuing make example_target_2 will do the same using the other recipe.
The issue using this is both example_target_1 and example_target_2 are done without checking if file_to_generate is up-to-date as the name of the target isn't really a file.
Is their a way to get the same behavior and still check if the file is up-to-date ?
One way that it can be achieved may be with use of target-specific variables, like so:
$ cat Makefile
target1 target2: file_to_generate
cat $<
target1: RECIPE=recipe1
target2: RECIPE=recipe2
file_to_generate:
$(if $(filter recipe1,$(RECIPE)),$(recipe1))
$(if $(filter recipe2,$(RECIPE)),$(recipe2))
define recipe1
echo recipe1
echo foo > $#
endef
define recipe2
echo recipe2
echo bar > $#
endef
Even though it works, I would strongly advise against such design. Generating a file in a non-deterministic way may easily lead to non-trivial errors. For example, using this approach you will generate a file and it will be checked if it's up to date, but there is no way for make to guess whether it was generated with recipe1 or recipe2. Therefore the next time you call a different target, the file will not be regenerated (since it already exists), even though the recipe has changed:
$ make target1
echo recipe1
recipe1
echo foo > file_to_generate
cat file_to_generate
foo
$ make target2
cat file_to_generate
foo
When called with target2 first, the file will have different contents, which will be reused in target1 as well:
$ rm file_to_generate
$ make target2
echo recipe2
recipe2
echo bar > file_to_generate
cat file_to_generate
bar
$ make target1
cat file_to_generate
bar
This may or may not be desirable, you need to be aware of such behavior.
When I execute Make, I'd like to know what shell commands / recipes are executed (and perhaps which line in which Makefile invoked these). Is there a way of doing this (print onto the stdout or write to a file) without modifying the Makefiles?
The commands will be echoed to stdout by default, unless you disabled that by prefixing them with #.
A few options you can pass to make:
-n: echos commands that would be run, but without running them. This will also echo commands that are prefixed with #.
-d: print debug output. This will output the process that make goes through in attempting to find rules that match the targets that it's trying to make. There will be a lot of useless information in here, as it will try many default build rules (like how to build .o files from .c).
There are a few ways to cut down on the noise:
-r: disable implicit rules. If you don't depend on any of the default rules , this can be used in conjunction with -d to just print the parts you (mostly) care about
--debug=b: basic debug mode. prints what it's trying to make, but not any information about implicit rules.
None of these print the line numbers of the commands, but make -n --debug=b will print both the targets being built and the commands being run, so it's almost as good. Example below.
$ cat makefile:
c: a b
cat $^ > $#
a:
echo 'foo' > $#
b: a
cat $^ > $#
echo 'bar' >> $#
$ make -n --debug=b:
Reading makefiles...
Updating goal targets....
File 'a' does not exist.
Must remake target 'a'.
echo 'foo' > a
Successfully remade target file 'a'.
File 'b' does not exist.
Must remake target 'b'.
cat a > b
echo 'bar' >> b
Successfully remade target file 'b'.
Prerequisite 'a' is newer than target 'c'.
Prerequisite 'b' is newer than target 'c'.
Must remake target 'c'.
cat a b > c
Successfully remade target file 'c'.
I was thinking about using Make for small checks for my dev setup. One thing I want is to check that a particular string exists in a file before doing some action. If I wanted to create the entire file it would be trivial
action: filename
...
filename:
echo 'blah' >> filename
But how can this logic be applied to actions, like grep? My dependency isn't that a file exists, it's that the file has correct content.
I'm asking specifically about Make and not other solutions like chef/puppet
You can run any shell commands you want in a make recipe. As many of them as you want also.
So if you need to run grep before doing something else just do that.
Just remember that every line in a recipe is run in its own shell session so they don't share state.
So this:
action: filename
...
filename:
grep -qw blah $# || echo 'blah' > $#
runs grep on filename (via the automatic variable for the current target $#) looking for whole words and quitting on the first match (-q).
If grep finds blah then it will return success and the || will short-circuit and the recipe is done. If grep fails then the || will trigger and the echo will run.
You might be tempted to do things that require the inverse logic do X only if Y is true:
filename:
grep -qw blah $# && echo blah2 > $#
but that doesn't work correctly. When grep fails the && short-circuits and make sees a recipe failure and bails the make process out with an error.
You need this instead.
filename:
! grep -qw blah $# || echo blah2 > $#
to invert the logic and ensure that the "failure" from grep is seen as success as far as make is concerned.
That all being said in this specific example if filename exists at all then that recipe won't ever run as it has no prerequisites so make will always consider it up to date. To work around that you need to give the file a prerequisite that will force it to be considered out of date. Specifically a force target.
Don't follow the advice about .PHONY for this case though. .PHONY targets should never be prerequisites of non-.PHONY targets.
Expanding on what #john wrote I got the following to work:
TEST_FILE=filename
.PHONY: ${TEST_FILE}
string=testing
filecheck=$(shell grep -qw ${string} ${TEST_FILE} || echo ${TEST_FILE})
all: ${filecheck}
${TEST_FILE}:
echo 'changing the file'
echo ${string} >> ${TEST_FILE}
Here the file on which I'm operating is a .PHONY target. I think that's ok because I'm actually not creating the file, just modifying it. This will work if the file does not exist, or exists without the needed string.
You could add a test in the target's recipe (As Etan posted before I could complete this answer...). If you do want to do this using just make logic, you could do something along the lines of:
actions: $(if $(shell grep -q $$string filename && echo y),filename,)
filename:
echo blah >> $#
If filename contains the string, then there will be an actions: filename dependency, and filename will be built when you build actions. Notice, though that this will check whether the string exists in filename at the time the makefile is parsed -- if filename is generated, or modified in this makefile, then it would not effect whether the action is run. If you want to test right before overwriting the file, then you would use a bash if statement in the recipe itself.
I have a makefile that defines several rules where the target is a foreach function.
$(foreach var,$(list), $($(var)_stuff) $($(var)_more_stuff)):
#echo Building $# from $^...
$(CC) $(FLAGS) ...
Is there any way to get make to quit when encountering an error without going through the entire list.
One workaround is to "manually" invoke exit on failure.
For example, assume we have a directory called scripts with a number of shell scripts (with filenames that end with .sh) that we want to execute.
Then a variable declaration like this:
LIST_OF_SCRIPTS ?= $(wildcard scripts/*.sh)
will give us a list of those scripts, and a target like this:
run-all-scripts
#$(foreach scriptfile,$(LIST_OF_SCRIPTS),$(scriptfile);)
will run all of those scripts, but as you note, the foreach loop will keep going whether or not one of the scripts returns an error code. Adding a || exit to the command will force the subcommand to exit on error, which Make will then treat as a failure.
E.g.,
run-all-scripts
#$(foreach scriptfile,$(LIST_OF_SCRIPTS),$(scriptfile) || exit;)
will do what you want (I believe).
Specifically, using your pseudo-code example, I think you want something like this:
$(foreach var,$(list), $($(var)_stuff) $($(var)_more_stuff)):
#echo Building $# from $^...
($(CC) $(FLAGS) ...) || exit
(where all I've changed is wrapping the (CC) $(FLAGS) ... bit in parens and appending || exit to make it fail on error).
The foreach is completely evaluated and substituted before any of the rules are executed. So the behaviour of this should be identical to as if you had hardcoded the rule without using the foreach. In other words, it's not directly relevant to the problem.
There are only a few possible explanations for what you're seeing, mostly described in the manual here:
You are running Make with -k or --keep-going
You are running Make with -i or --ignore-errors
Your targets is defined as prerequisites of the special .IGNORE target
Your recipe starts with a -
Your recipe isn't actually returning a non-zero exit status
Not sure about your example, but maybe problem is in ; - look at Makefile : show and execute:
dirs = $(shell ls)
clean:
$(foreach dir,$(dirs),echo $(dir);)
produce:
$ make clean
echo bin; echo install.sh; echo Makefile; echo README.md; echo utils;
So make check exit code only for last command: echo utils.