I have this simple Makefile:
a:
touch a
b: a
touch b
all: b
touch myapp
make all returns:
touch a
touch b
touch myapp
clearmake all (or clearmake -C gnu all) returns:
touch a
touch b
touch myapp
How to get rid of the unnecessary newlines?
There is not many possibility to change the output of clearmake (or clearmake -C gnu)
(-n only prints the commands, -s does not print it)
That leaves you with workarounds like:
clearmake all | grep -v '^$'
Related
How can I get make to not print flags.
Example:
prettycolors:
#echo -e "\e[32mIM GREEN\e[0m"
Current behavior:
$ make prettycolors
-e IM GREEN
Desired behavior:
$ make prettycolors
IM GREEN
Seems like shell does not accept echo flags.
You can either specify echo you want to use: (in Makefile)
ECHO=/bin/echo
prettycolors:
#$(ECHO) -e "\e[32mIM GREEN\e[0m"
or specify "smarter" shell:
$ make SHELL=/bin/bash prettycolors
Suppose I have the following situation:
.SECONDEXPANSION:
rule-%: etc/$$*
some other things
here
something-something -c "cat $< | envsubst | ..." \
> $#
And I'd like to rewrite this rule, as it is a common pattern across n items, without getting clever:
.SECONDEXPANSION:
rule-%: etc/$$*
some other things
here
make something-something-$< > $#
something-something-%:
something-something -c "cat $* | envsubst | ..."
But I can't find any guarantees that this will work if, suppose, I call the outer rule, rule-% as a submake from a different directory (make, in my experience, sometimes prints directory position into the pipeline -- which can be suppressed by executing make silently, but I would never assume a user to know that).
Will this work in all cases? Alternatively, I know I can:
etc...
make something-something arg1=... arg2=...
something-something:
something-something -c "cat ${arg1} | envsubst | ..." > ${arg2}
But I'd like to avoid this.
I don't think I understand your question, but: first you should never use raw make when invoking make recursively: you must always use the $(MAKE) variable.
Second, if you want the recursive make to not print directory info, or even to run silently, you can just add those options to the submake: it won't matter what options the person who invoked the outer make specified:
$(MAKE) -s --no-print-directory something-something-$< > $#
If you just want to keep your code DRY, you can just make use of a template, like so:
$ cat Makefile
.SECONDEXPANSION:
define something
bash -c "cat $(1) | envsubst"
endef
rule-%: etc/$$*
$(call something,$<) > $#
another-rule-%: etc/$$*
$(call something,$<) > $#
Output:
$ make rule-foo another-rule-foo rule-bar another-rule-bar
bash -c "cat etc/foo | envsubst" > rule-foo
bash -c "cat etc/foo | envsubst" > another-rule-foo
bash -c "cat etc/bar | envsubst" > rule-bar
bash -c "cat etc/bar | envsubst" > another-rule-bar
I'd like to add a quick check to a (GNU) makefile which can alert the user to the availability of -j/--jobs (parallel make). That is, something like
$ make
TIP: this will build faster if you use use "make -j"
Building ..
$ make -j
Building in parallel ..
How can I determine the number of parallel jobs when a Makefile is executed?
There is a trick here
http://blog.jgc.org/2015/03/gnu-make-insanity-finding-value-of-j.html
and a proposed change to GNU Make itself here
https://github.com/esantoro/make/commit/b0334e7f3009dc58dbc8e6e6fdec94711537fb3b
but perhaps there is something newer and/or easier.
The simplest/best solution is to upgrade your version of GNU make to 4.2 or above. Starting with that version, the MAKEFLAGS variable will provide the full -j option including the number. The NEWS file says:
The amount of parallelism can be determined by querying MAKEFLAGS, even when
the job server is enabled (previously MAKEFLAGS would always contain only
"-j", with no number, when job server was enabled).
So:
$ make --version
GNU Make 4.2.1
...
$ echo 'all:;#echo $(MAKEFLAGS)' | make -f-
$ echo 'all:;#echo $(MAKEFLAGS)' | make -f- -j
-j
$ echo 'all:;#echo $(MAKEFLAGS)' | make -f- -j10
-j10 --jobserver-auth=3,4
$ echo 'all:;#echo $(patsubst -j%,%,$(filter -j%,$(MAKEFLAGS)))' | make -f- -j10
10
You can determine the number of jobs easier and faster than that blog proposes by using make Jobserver protocol:
SHELL := /bin/bash
all:
#${MAKE} --no-print-directory job_count_test
job_count_test:
#+[[ "${MAKEFLAGS}" =~ --jobserver[^=]+=([0-9]+),([0-9]+) ]] && ( J=""; while read -t0 -u $${BASH_REMATCH[1]}; do read -N1 -u $${BASH_REMATCH[1]}; J="$${J}$${REPLY}"; done; echo "Building with $$(expr 1 + $${#J}) jobs."; echo -n $$J >&$${BASH_REMATCH[2]} ) || echo "TIP: this will build faster if you use use \"make -j$$(grep -c processor /proc/cpuinfo)\""
.PHONY: all job_count_test
And then:
$ make
TIP: this will build faster if you use use "make -j8"
$ make -j12
Building with 12 jobs.
So here's a Makefile to install foo.conf, based on a master copy called foo.conf.master. It installs it to the current directory rather than /etc, just for testing purposes:
all: foo.conf.copied
foo.conf.copied: foo.conf.master foo.conf
cp foo.conf.master foo.conf
touch $#
# Recipe to tell make that it is okay for foo.conf not to exist beforehand.
foo.conf:
So then create foo.conf.master:
$ touch foo.conf.master
$
and you're ready to test:
$ make
cp foo.conf.master foo.conf
touch foo.conf.copied
$
The point is that if I (with my "trusted" sysadmin hat on) modify foo.conf.master then make (possibly called by cron) will roll out the update:
$ touch foo.conf.master
$ make
cp foo.conf.master foo.conf
touch foo.conf.copied
$
But equally important: if I (with my "rogue" sysadmin hat on) modify the installed version then make will back out the update:
$ touch foo.conf
$ make
cp foo.conf.master foo.conf
touch foo.conf.copied
$
Woohoo.
Okay, so now the problem: obviously foo.conf isn't the only file I want do this for, so I need to change my static rules to pattern rules. Okay, that's easy: substitute foo.conf for % in targets and dependencies, substitute foo.conf for $* in the commands, and make a minor modification to the last recipe (which would otherwise become only '%:') so that it doesn't look like I'm trying to cancel a builtin pattern rule.
So clean up and create this Makefile:
all: foo.conf.copied
%.copied: %.master %
cp $*.master $*
touch $#
# Recipe to tell make that it is okay for foo.conf not to exist beforehand.
# Nop tells make that I'm not *cancelling* a pattern rule here
# (see http://stackoverflow.com/questions/34315150/make-implicit-rules-dont-work-without-command).
%: ;
But this doesn't work:
$ make
make: *** No rule to make target `foo.conf.copied', needed by `all'. Stop.
$
The error message is misleading; it is really foo.conf that it doesn't know how to make, which can be demonstrated by adding the following at the bottom of the Makefile:
foo.conf:
touch $#
But then that's a static rule again, which I don't want.
There are a couple more requirements I would also like to satisfy, which the above example doesn't demonstrate. These are:
foo.conf should be installable anywhere in the filesystem (e.g. /etc/foo/server/foo.conf)
foo.conf.master should be in a central directory, or subdirectly thereof, for all master versions, preferably without the '.master' extension (e.g. ~/poor-mans-puppet/master-files/etc/foo/foo.conf)
foo.conf.copied should be in a central directory, not in the same directory as foo.conf (e.g. ~/poor-mans-puppet/timestamp-files/etc/foo/foo.conf)
After much googling, hair pulling, I'm asking here! Any ideas please? (PS: if copying Makefiles from here, remember to change indentation back to tabs.)
Mad Scientist below suggested an elegant static rule, but I really need it to be a pattern rule. The reason is that I need to hook extra dependencies in using rules:
all: <new-dependency>
rather than hooking them in using variables:
STUFF_ALL_SHOULD_DEPEND_ON += <new-dependency>
The reason for this requirement is for consistency with how other (non-%.copied) targets are handled in my very large Makefile.
However, based on Mad Scientist's idea, I tried the following, which didn't work, but perhaps helps somebody to help me:
all: foo.conf.copied
%.copied: %.master %
$(eval FILES_FOR_WHICH_AN_EMPTY_RECIPE_ARE_NEEDED += $$*)
cp $*.master $*
touch $#
define GENERATE_STATIC_EMPTY_RULE
$(1):
endef
$(foreach X,$(FILES_FOR_WHICH_AN_EMPTY_RECIPE_ARE_NEEDED),$(eval $(call GENERATE_STATIC_EMPTY_RULE,$(X))))
Can you explain why you're using this extra ".copied" file? Why don't you just use:
%: %.master ; cp $< $#
?
Anyway, you're running afoul of make's special rules related to match-anything rules (pattern rules like % that can build everything). If you change your pattern so it's not match-anything, like %.conf: ; then it will work. However you probably don't want to assume that all files end in .conf.
Alternatively you can use static pattern rules, like this:
FILES_TO_COPY = foo.conf bar.conf biz.baz
all: $(FILES_TO_COPY:%=%.copied)
$(FILES_TO_COPY:%=%.copied): %.copied : %.master %
cp $*.master $*
touch $#
and you don't need the extra pattern rule.
In the end, I dynamically generated static rules. The following pseudo-code hopefully makes the actual Makefile easier to understand:
if flag not set # flag won't be set on first call
prepare static rules
set flag # prevent running this clause again
recurse! # make invokes make
else
include static rules
do the normal thing
endif
Here's the real Makefile:
ifeq ($(MAKELEVEL),0)
all:
for X in $(patsubst %.copied,%,$^); do \
echo "$$X.copied: $$X.master $$X"; \
echo " cp $$X.master $$X"; \
echo " touch \$$#"; \
echo "$$X: ;"; \
done > Makefile.include
$(MAKE)
# The real dependencies on all are defined below, but while creating
# Makefile.include, we don't want make to digress and start making
# the dependencies; this pattern rule will stop it from doing that.
%.copied: ;
else
include Makefile.include
endif
all: foo.conf.copied
The outer make can be silenced by use of .SILENT and the --no-print-directory option (not shown above for clarity).
Here's the output:
$ touch foo.conf.master
$ make
cp foo.conf.master foo.conf
touch foo.conf.copied
$ touch foo.conf
$ make
cp foo.conf.master foo.conf
touch foo.conf.copied
$
Criteria: Makefile is a GNU Make Makefile - I'm not interested in makepp, qmake, cmake, etc. They're all nice (especially cmake), but this is for work and at work we use GNU Make. The optimal solution is a pure Makefile solution rather than a shell script that parses make for you.
I also don't want to do a 'continue on failure' solution - if it's broken, it's broken and needs to be fixed.
The situation is this, I've got a makefile that builds several directories in parallel - if one of them fails, of course the whole build fails, but not until all the running makes run to completion (or failure). This means that the reason why make actually failed is buried somewhere arbitrarily far from the end of make's output.
Here's an example of what I've got:
all: $(SUBDIRS)
SUBDIRS = \
apple \
orange \
banana \
pineapple \
lemon \
watermelon \
grapefruit
$(SUBDIRS):
cd $# && $(MAKE) $(MFLAGS) 2>&1 | sed -e "s/^/$(notdir $(#)): /g"
If I run 'make -j 5' and 'orange' happens to fail - I'd like to see a table like this at the end
of the make process
apple - passed
orange - FAILED
banana - passed
pineapple - passed
lemon - passed
I've considered having an && echo "passed" >.result || echo "FAILED" >.result, but make still needs some sort of TRAP or __onexit() cleanup command to print at them on exit.
Any Makefile ninjas out there have a pure-makefile solution for this?
un-edit - my solution wasn't actually working the way I had hoped.. STYMIED!
When you want make to abort at the first failure, end immediately and kill all in-flight jobs instead of waiting for them to finish, you need to patch GNU Make like this
http://lists.gnu.org/archive/html/bug-make/2009-01/msg00035.html
Then you need to set a trap for every shell that make invokes (as well as set -o pipefail if you use a pipe), as described in this post http://lists.gnu.org/archive/html/help-make/2009-02/msg00011.html
In a nutshell:
target1:
trap 'kill $$(jobs -p)'; command && something || something-else
target2:
trap 'kill $$(jobs -p)'; set -o pipefail; command | sed '...'
The only way I see is self-execution with a sub-make:
all : subdirs
subdirs :
$(MAKE) -f $(lastword $(MAKEFILE_LIST)) subdirs-recursive || cat log
subdirs-recursive: $(SUBDIRS)