Prefix output for sub-make phony targets? - makefile

Problem
Given the following Makefile:
.PHONY: blah blah1 blah2
blah:
#$(MAKE) -j --no-print-directory blah1 blah2
blah1:
#echo "hello"
#caddy run
blah2:
#echo "goodbye"
#esbuild --watch
Running make blah returns the following:
❯ make blah
hello
Caddy listening on :3000
goodbye
esbuild building...
esbuild complete in 4ms
How can I prefix the name of the target before any output (stdout or stderr) from that target?
I'm looking to get this:
❯ make blah
[blah1] hello
[blah1] Caddy listening on :3000
[blah2] goodbye
[blah2] esbuild building...
[blah2] esbuild complete in 4ms
Version
GNU Make v4.4
Other solutions I've tried
I can hack this together by manually appending the output through sed on each line inside the target definition:
.PHONY: blah blah1 blah2
blah:
#$(MAKE) -j --no-print-directory blah1 blah2
blah1:
#echo "hello" | sed 's|^\(.*\)$$|[blah1] \1|'
#caddy run 2>&1 | sed 's|^\(.*\)$$|[blah1] \1|'
blah2:
#echo "goodbye" | sed 's|^\(.*\)$$|[blah2] \1|'
#esbuild --watch 2>&1 | sed 's|^\(.*\)$$|[blah2] \1|'
... but this solution gets very tedious very fast for non-contrived Makefiles.

If you want to manipulate the output of your recipe's you're going to have to pipe them to some sort of script (make doesn't offer anything like that). That being said, you can shorten it a bit:
.PHONY: blah blah1 blah2
blah:
#$(MAKE) -j --no-print-directory blah1 blah2
blah1:
#echo "[$#] hello"
#caddy run 2>&1 | awk '$$0="[$#]"$$0'
blah2:
#echo "[$#] goodbye"
#esbuild --watch 2>&1 | awk '$$0="[$#]"$$0'
Now, your question also suggests you were looking for something that is sub-make specific (as opposed to target specific) so, if that's the case, you can simply pipe the output of sub-make to your script:
.PHONY: blah blah1 blah2
blah:
#$(MAKE) -j --no-print-directory blah1 | awk '$$0="[blah1]"$$0'
#$(MAKE) -j --no-print-directory blah2 | awk '$$0="[blah2]"$$0'
##note: -j blah1 and blah2 are build sequentially here
blah1:
#echo "hello"
#caddy run 2>&1'
blah2:
#echo "goodbye"
#esbuild --watch 2>&1
--- EDIT ---
You could run two different instances in make in parallel, and pipe the output of those through the prefix script as so:
.PHONY: blah blah1 blah2 submake_blah1 submake_blah2
blah: submake_blah1 submake_blah2
submake_%:
#$(MAKE) -j --no-print-directory $* | awk '$$0="[$*]"$$0'
blah1:
#echo "hello"
#caddy run 2>&1'
blah2:
#echo "goodbye"
#esbuild --watch 2>&1
This of course involves you invoking your parent make with -j to allow these makes to run in parallel. Also, if the two submakes have common dependencies (i.e. blah1 depends on blah32, and blah2 also depends on blah32), then this would result in blah32 being built twice, and could result in race conditions.

As Henry shows it's tricky to do what you asked.
However, it could be that this is an XY problem and what you REALLY want to be able to do is just figure out which output comes from which target.
In that case you should investigate the output sync facility provided by GNU make, which collects the output of the target and generates it all in a single block, so that it's not mixed together with other targets' output. For example something like:
.PHONY: blah blah1 blah2
blah:
#$(MAKE) -j -O --no-print-directory blah1 blah2
blah1:
#echo "start hello"
#caddy run
#echo "end hello"
blah2:
#echo "start goodbye"
#esbuild --watch
#echo "end goodbye"
This won't prefix every line with the target being built but it will group all the output for each target together in the output, rather than mixing it all up.

Related

$(info) command io redirection

Can I get standard error redirected from make info command to a file? E.g. if I wanna print compilation command and redirect 2> to a file, the file gets just an empty string but the command inside the info parenthesis is printed, like bellow:
ifdef VERBOSE
verbose := #cat /tmp/compilation.out
endif
development: $(source)
# Pipe to cat is just to redirect signals from make to don't exit on error
#$(info $(CC) -o $(target) $^ $(DEBUG)) 2> /tmp/compilation.out | cat
$(verbose)
#cat /tmp/compilation.out >> $(log)
Make variables and functions are expanded by make before it invokes the shell to run the recipe. The results of expansion are passed to the shell.
2> is a shell construct, it's not interpreted by make.
The info make function is defined to write its arguments to stdout, and expand to the empty string. So, when make expands this recipe line:
$(info $(CC) -o $(target) $^ $(DEBUG)) 2> /tmp/compilation.out | cat
First it expands the arguments to info and writes the results to stdout, then it replaces the info function with the empty string, then it invokes the shell with the results; so the shell sees:
2> /tmp/compilation.out | cat
The short answer is no, there is no way to get make's info function to write output to a file. You'll have to use something the shell knows about, like echo:
#echo '$(CC) -o $(target) $^ $(DEBUG)' 2> /tmp/compilation.out | cat

Dynamic targets based on the dependency in Makefile

There are a couple of kind of similar issues but I could not fit any of the proposed concepts to my case.
Just to give a little bit of context: I have a set of Julia files which create plots as PDFs which are part of a make procedure to create scientific papers, something like:
plots = $(shell find $(PLOT_PATH)/*.jl | sed 's/\.jl/\.pdf/g')
$(PLOT_PATH)/%.pdf: $(PLOT_PATH)/%.jl $(JULIA_SYSIMAGE)
$(JL) --project $< -o $(PLOT_PATH)
$(DOCUMENT_FILENAME).pdf: FORCE $(plots) $(figures)
latexmk $(DOCUMENT_FILENAME).tex
In the current setup, each XYZ.jl file is creating a XYZ.pdf file and it works absolutely fine.
Now I am dealing with cases where it would be much easier to create multiple plots from single Julia files, so a script like this:
#!/usr/bin/env julia
using PGFPlotsX
...
...
pgfsave("whatever.pdf")
pgfsave("another.pdf")
pgfsave("yetanother.pdf")
so that one could do a grep pgfsave SCRIPT | awk... to figure out the targets. However, I could not figure out how to generate dynamic targets (plots) based on the contents of the dependency file (Julia script).
An MWE for my problem is the following: I have a couple of files (dependencies) which are generating a bunch of targets, which are defined inside those files (and can be access via awk/grep/sed/whatever). For now, let's say that these are simply *.txt files and each line is a target.
file: a.txt
foo
bar
baz
file: b.txt
naarf
fjoord
A very basic (non-working) manual Makefile to demonstrate the goal would be something like this (it does not work as it cannot figure out how to make foo etc. but it shows the pattern for *.txt which needs to be repeated):
file: Makefile
all_products := $(shell find *.txt | xargs cat)
final_product: $(all_products)
echo $< > $#
(foo bar baz): a.txt
touch $(shell cat $<)
(narf fjoord): b.txt
touch $(shell cat $<)
so in principle, I need something to "process" the dependency (*.txt) to create a list of the targets, like
$(shell cat $%): %.txt
echo $< > $#
but I cannot manage to get a reference to the dependency on the target side ($% does not work).
Any ideas? Maybe the whole approach is just a bad idea ;)
A combination of GNU make foreach, eval and call functions is probably what you need. With your example:
TXT := $(wildcard *.txt)
.PHONY: all
.DEFAULT_GOAL := all
define MY_MACRO
$(1)-targets := $$(shell cat $(1))
$$($(1)-targets): $(1)
echo $$< > $$#
all: $$($(1)-targets)
endef
$(foreach t,$(TXT),$(eval $(call MY_MACRO,$(t))))
(pay attention to the $$ in the macro definition, they are needed). And then:
$ make
make
echo a.txt > foo
echo a.txt > bar
echo a.txt > baz
echo b.txt > naarf
echo b.txt > fjoord
If you want the recipe to build all targets at once you'll need a recent enough GNU make version (4.3 or later) and its new rule with grouped targets (x y z&: w):
TXT := $(wildcard *.txt)
.PHONY: all
.DEFAULT_GOAL := all
define MY_MACRO
$(1)-targets := $$(shell cat $(1))
$$($(1)-targets)&: $(1)
touch $$($(1)-targets)
all: $$($(1)-targets)
endef
$(foreach t,$(TXT),$(eval $(call MY_MACRO,$(t))))
And then:
$ make
touch foo bar baz
touch naarf fjoord
Note that in this case we could also use a simpler and less GNU make-dependent solution. Just use empty dummy files as time stamps, for instance .a.txt.tag for a.txt, and a static pattern rule:
TXT := $(wildcard *.txt)
TAG := $(patsubst %,.%.tag,$(TXT))
.PHONY: all
all: $(TAG)
$(TAG): .%.tag: %
touch `cat $<` $#

Is there any way to make multiple targets as series of single make invocations?

I have the following Makefile:
ifneq ($(MAKECMDGOALS),clean)
-include generated.mk
endif
FOO ?= foo
all: a.txt
a.txt:
echo $(GEN_FOO) > $#
generated.mk: Makefile
echo GEN_FOO = $(FOO) > $#
.PHONY: clean
clean:
$(RM) a.txt
$(RM) generated.mk
It works OK when building single targets:
$ make clean
rm -f a.txt
rm -f generated.mk
$ make all
echo GEN_FOO = foo > generated.mk
echo foo > a.txt
However when I try to build multiple targets at once things go not so smooth:
$ make clean all
rm -f a.txt
rm -f generated.mk
echo foo > a.txt
$ make all
echo GEN_FOO = foo > generated.mk
make: Nothing to be done for 'all'.
It gets even worse if variables were provided:
$ make clean
rm -f a.txt
rm -f generated.mk
$ make FOO=bar clean all
echo GEN_FOO = bar > generated.mk
rm -f a.txt
rm -f generated.mk
echo bar > a.txt
$ make all
echo GEN_FOO = foo > generated.mk
make: Nothing to be done for 'all'.
$ make FOO=bar clean all
rm -f a.txt
rm -f generated.mk
echo foo > a.txt
Are there any ways to fix such incorrect behavior?
Make is doing exactly what you told it to do, and you haven't told us what you want it to do that's different than what you told it to do (saying fix such incorrect behavior doesn't really help us when you don't define what's incorrect about the behavior), so we can't help you very much.
You are probably getting confused about the interaction between included makefiles and comparing $(MAKECMDGOALS). Please note:
ifneq ($(MAKECMDGOALS),clean)
this will not match unless you specify exactly one target: clean. In situations where you specify multiple targets, one of which is clean, that will match because clean all is not equal to clean. So, when you run make clean all make will include the generated makefile, and will generate it if it doesn't exist.
Because generated include files are only rebuilt once, when the makefile is first parsed, it's not possible to say something like: "first run rule X (e.g., clean) then rebuild the included makefiles, then reinvoke make".
However, it's pretty much always a bad idea to invoke make with clean all. This is because if you were to ever try to add -j for parallelism, the clean and the build would be running in parallel and corrupt everything.
One semi-common option is to provide a different rule that will do both, something like this:
rebuild:
$(MAKE) clean
$(MAKE) all
then run make rebuild instead.
You can certainly force the behavior with the help of the shell. For instance, in bash you could use
for target in "clean" "all";
do
make $target;
done
and if you were going to re-do the procedure a lot you could either make it an executable script or wrap it in a shell function.

In a makefile, is there a way to redirect $(warning) or $(info) statements to file

Note that I do not want to redirect all make output to file. I only want the output from a $(warning) command to file.
someTarget:
$(warning building $# using $?) >> someLogFile.txt
My example above does not redirect the output from $(warning to someLogFile. Is there a way to do it? Maybe redirect it to a variable and then echo that to a file?
Thanks.
is there a way to redirect $(warning) or $(info) statements to file?
Here's one for GNU Make, but it's not pretty:
Makefile
LOG := log.txt
TARGET_ACQUIRED = \
$(shell echo 'NO_SUCH_TARGET:' | $(MAKE) --eval='$$(info Target acquired: $#...)' -s -f - >> $(LOG))
target_a: target_b
$(TARGET_ACQUIRED)
touch $#
target_b:
$(TARGET_ACQUIRED)
touch $#
clean:
rm -f target_* $(LOG)
With which you'll get:
$ make
touch target_b
touch target_a
$ cat log.txt
Target acquired: target_b...
Target acquired: target_a...
To understand this ruse, see the GNU make commandline options.
If you want this for the purpose of debugging a makefile, you'd probably
fare better with GNU Make's --debug options, documented at the same place.

Makefile define: recursive expansion question

In a makefile, I define a variable using the define directive. This variable will hold a configurable list of commands that I want to execute.
I would like this variable to get a list of files (.foo files, for example). These files are created during the makefile execution. For example makefile:
MY_VAR = $(wildcard *.foo)
define MY_VAR2
echo $(1) $(MY_VAR)
endef
foo: create_files
$(call MY_VAR2, ls)
rm -f *.foo
create_files:
touch foo.foo
touch bar.foo
I do not get the desired results. It appears that MY_VAR2 is evaluated upon declaration.
Is there a way to get the desired behavior?
edit:
The $(shell) command, as sateesh correctly pointed out, works for the example above. However, it does not work for the example below. The main difference in this example is that the new files are created inside MY_VAR2.
MY_VAR = $(wildcard *.foo)
TEST_VAR = $(shell ls *.foo)
define MY_VAR2
#touch foo.foo
#touch bar.foo
#echo "MY_VAR" $(1) $(MY_VAR)
#echo "TEST_VAR" $(1) $(TEST_VAR)
endef
foo:
$(call MY_VAR2, ls)
#rm -f *.foo
I can solve the above by adding rules. Is there a simpler method?
It looks to me like you are abusing make, trying to write a shell script in make.
If you write a shell script, write a shell script. You execute your commands in sequence, and you are able to know what files are present when executing each line.
touch foo.foo
touch bar.foo
your-command `ls *.foo`
On the other side, if you want to make usage of make, then have rules and dependencies, you won't even have to use define if you go the make way of thinking.
foo: create_files
your-command $(wildcard *.foo)
rm -f *.foo
create_files:
touch foo.foo
touch bar.foo
I'm presuming here that you are using here GNU Make. I think you can get the desired
result using the "shell" built-in function provided with GNU Make.
Below is the make file snippet that demonstrates how you can get the result:
MY_VAR = $(wildcard *.foo)
TEST_VAR = $(shell ls *.foo)
define MY_VAR2
#echo "MY_VAR" $(1) $(MY_VAR)
#echo "TEST_VAR" $(1) $(TEST_VAR)
endef
foo: create_files
$(call MY_VAR2, ls)
#rm -f *.foo
create_files:
#touch foo.foo
#touch bar.foo
The result of running the above make file for the target "foo" is:
MY_VAR ls
TEST_VAR ls bar.foo foo.foo
So as demonstrated usage of shell function gets the desired result.
From the GNU make documentation:
The shell
function performs the same function that backquotes (‘‘’) perform in most
shells: it does command expansion. This means that it takes as an argument a shell
command and evaluates to the output of the command. The only processing make does on
the result is to convert each newline (or carriage-return / newline pair) to a single space.
...
The commands run by calls to the shell function are run when the function calls are expanded.
...
files := $(shell echo .c)
sets files to the expansion of ‘.c’. Unless make is using a very strange shell, this has the
same result as ‘$(wildcard *.c)’ (as long as at least one ‘.c’ file exists).
So I think using a recursively expanded variable (= form) along with shell function you
can get the desired result.
4.4.3 The Function wildcard
Wildcard expansion happens automatically in rules. But wildcard expansion does not normally take place when a variable is set, or inside the arguments of a function. If you want to do wildcard expansion in such places, you need to use the wildcard function
You may work your away around the eval function.
These created files should be dependencies of another rule so that the dependency graph is correct.
References:
4.4.3 The Function wildcard
8.8 The eval function
Could you try the filter command? You can find an example here.
How about using include?
The point is that the make needs two pass of execution.
First, you need to let make create what it wants,
Then you'll let make to parse the generated rule in the second pass after
the execution of the first rule.
define MY_VAR2
echo $(1) $(MY_VAR)
$(1) $(MY_VAR)
endef
-include .listoffiles
foo: create_files
$(call MY_VAR2, ls -al)
rm -f *.foo .listoffiles
create_files: .listoffiles
.listoffiles:
touch foo.foo
touch bar.foo
#echo "MY_VAR = \$$(wildcard *.foo)" > $#
It just does what you want, I suppose.
Two pass evaluation doesn't mean you have to type make twice.
It will be done by make automagically.
% make -f test.mk
touch foo.foo
touch bar.foo
echo ls -al bar.foo foo.foo
ls -al bar.foo foo.foo
ls -al bar.foo foo.foo
-rw-rw-r-- 1 yo-hei clearusers 0 Jan 8 08:44 bar.foo
-rw-rw-r-- 1 yo-hei clearusers 0 Jan 8 08:44 foo.foo
rm -f *.foo .listoffiles
What about doing your expansion in the shell instead of in make?
MY_VAR = $$(ls *.foo)
define MY_VAR2
#echo $(1) $(MY_VAR)
endef
foo: create_files
$(call MY_VAR2, ls)
#rm -f *.foo
create_files:
#touch foo.foo
#touch bar.foo
The double-$ will allow the shell to expand the file search part. I can't help but agree with the other posts about there being a more elegant way to do this, but this is an option for you.
Works for your second example too:
MY_VAR = $(wildcard *.foo)
TEST_VAR = $$(ls *.foo)
define MY_VAR2
#touch foo.foo
#touch bar.foo
#echo "MY_VAR" $(1) $(MY_VAR)
#echo "TEST_VAR" $(1) $(TEST_VAR)
endef
foo:
$(call MY_VAR2, ls)
#rm -f *.foo

Resources