I have this in my Makefile:
run:
for x in *.bin ; do ./$$x ; done
such that it launches all executables one by one. I want to do this:
run:
for x in *.bin ; do ./$$x &; done
so that it starts each executable and puts it in the background. I get a syntax error for the above statement when I put the ampersand.
I dont want to invoke the make as make & since this will run processes in the background but still one by one, whereas I want individual executables to run in the background, so that at any instant I have more than one executable running.
Thank you in advance.
Try to execute via a subshell:
run:
for x in *.bin ; do (./$$x &); done
Maybe make -j is a better option. Try a Makefile that looks something like this:
BINS = $(shell echo *.bin)
.PHONY: $(BINS)
run: $(BINS)
*.bin:
./$#
And then execute with make -j <jobs> where <jobs> is number of simultaneous jobs to run.
The syntax error you're getting is a shell syntax error, rather than a problem with make syntax. The ampersand is in fact a command terminator/separator, just as semicolon is; so the way to express the for loop you want is:
run:
for x in *.bin ; do ./$$x & done
However, as others have noted, where it's practical it's usually more flexible to express things directly as make dependencies rather than complicated shell snippets and shell loops.
Try:
run:
for x in *.bin ; do (./$$x &) ; done
The ()'s run the command in a subshell.
Related
I would like to run #echo "Make complete." When the makefile finishes running. The problem is I can't figure out a way for it to do that without putting it at the end of of every option, but I wouldn't like to do that since for example the all option would echo "Make complete." multiple times. I also know I could run a script such as:
make $1
echo "Make complete."
But that solution is messy because it uses 2 files and it wouldn't work if the user didn't type an argument.
I am using GNU Make.
You can have a wrapper makefile calling the real makefile:
$(MAKECMDGOALS):
$(MAKE) -f makefile.real $#
#echo "Make complete."
There are caveats and some extra things that need doing to handle multiple targets and passing on the environment and other options, which I'm sure the SO community will add as comments to this answer.
First off, I'm using GNU Make 4.3 on Windows 10. I previously tried GNU Make 4.2.1, which gave me the exact same results as 4.3.
Anyway, I have a very simple makefile that does (or at least is intended to do) nothing more than run a simple command and print the output of that command (both stderr and stdout) to the terminal and to a text file.
$(info $$(MAKECMDGOALS) is "$(MAKECMDGOALS)". $$(SHELL) is \
"$(SHELL)". $$(MAKESHELL) is "$(MAKESHELL)". $$(COMSPEC) is "$(COMSPEC)". $$(OS) is "$(OS)".)
TEE := C:\tools\UnixTools\usr\local\wbin\tee.exe
LOG_FILE := C:\Temp\loggy__.txt
.PHONY : meep
all : meep
meep :
$(info Making meep.)
$(info Running command {dir 2>&1 | $(TEE) $(LOG_FILE)}.)
$(shell dir 2>&1 | $(TEE) $(LOG_FILE))
The last line is the one that is giving me trouble. Two things are happening that run counter to my expectations:
While the $(shell ...) call does print the output of the dir command both to my text file and the terminal, the output on the terminal is weirdly formatted. Where normally, dir prints one element per line, here I'm getting the entire output in one line, so it seems like GNU Make (or something else) somehow removes the newline characters from the output before it is shown in the terminal window.
In addition, I'm getting a The system cannot find the file specified. error message (and as usual, Windows is not nice enough to tell me which file it is that it cannot find). Running an echo %errorlevel% in the same CMD shell in which I ran GNU Make confirms that the Make call errored out (exit status is 2).
Weirdly enough, if I run the command dir 2>&1 | C:\tools\UnixTools\usr\local\wbin\tee.exe C:\Temp\loggy__.txt directly in the CMD window, everything works exactly as one would expect, without any errors whatsoever, so I'm thinking there's either something wrong with GNU Make's $(shell ...) function, or I'm using it wrong. Does anyone spot something silly in how I'm trying to use the $(shell ...) function?
I just added --debug=a to my make call to get extra debug output, and I found the following in the output:
Creating temporary batch file C:\Users\mkemp\AppData\Local\Temp\make23400-1.bat
Batch file contents:
#echo off
dir 2>&1 | C:\tools\UnixTools\usr\local\wbin\tee.exe C:\Temp\loggy__.txt
CreateProcess(C:\Users\mkemp\AppData\Local\Temp\make23400-1.bat,C:\Users\mkemp\AppData\Local\Temp\make23400-1.bat,...)
Main thread handle = 00000000000000B4
Cleaning up temporary batch file C:\Users\mkemp\AppData\Local\Temp\make23400-1.bat
Creating temporary batch file C:\Users\mkemp\AppData\Local\Temp\make23400-2.bat
Batch file contents:
#echo off
Volume in drive C is Windows Volume Serial Number is 045A-E422 Directory of C:\tools\UnixTools\usr\local\wbin (... the rest of the output)
CreateProcess(C:\Users\mkemp\AppData\Local\Temp\make23400-2.bat,C:\Users\mkemp\AppData\Local\Temp\make23400-2.bat,...)
So it appears that GNU Make's $(shell ...) function somehow interprets the output produced by the dir call as an additional command it needs to run, which is nonsense, of course.
Using $(shell) is nonsense here. make is acting exactly like you instructed it.
The proper solution is to not add the $(shell ...) function call where it makes no sense.
meep :
$(info Making meep.)
$(info Running command {dir 2>&1 | $(TEE) $(LOG_FILE)}.)
dir 2>&1 | $(TEE) $(LOG_FILE)
Of course, using $(info ...) in a recipe is probably bogus. Inside each recipe, you are running the shell; use the shell's syntax to print diagnostic messages.
meep:
#echo Making meep. >&2
#echo Running command '{dir 2>&1 | $(TEE) $(LOG_FILE)}.' >&2
dir 2>&1 | $(TEE) $(LOG_FILE)
Better yet, don't run make -s and let make itself print what commands it is running, as it does by default (if you don't sprinkle your Makefile with # before all commands to make it harder to debug).
Considering the following:
all:
mv info.h info.h.back
generate_info.sh
compile
mv info.h.back info.h
How do I force make to run the last line even if compilation failed?
I am aware of .DELETE_ON_ERROR but this deals only with removing targets on failure.
I am also aware of the option to add - before the compile command. Unfortunately this will make the whole make to return with a good error code, which is unacceptable.
You can't force make to do this. You'll have to arrange for it to be done in your shell script, yourself. Make will send every "logical line" of the shell script to the same shell command. Turn multiple physical lines into one logical line by appending backslashes to the end of the physical lines.
So, for example:
all:
mv info.h info.h.back
generate_info.sh && compile; \
r=$$?; mv info.h.back info.h; \
exit $$r
This saves the return code of the two commands in the shell variable r, then runs the mv command, then exits the shell with the result code that was saved.
The generate_info.sh && compile means that the second command (compile) will only run if the first command (generate_info.sh) succeeds.
I ran into this useful tip that if you're working on files a lot and you want them to build automatically you run:
watch make
And it re-runs make every couple seconds and things get built.
However ... it seems to swallow all the output all the time. I think it could be smarter - perhaps show a stream of output but suppress Nothing to be done for 'all' so that if nothing is built the output doesn't scroll.
A few shell script approaches come to mind using a loop and grep ... but perhaps something more elegant is out there? Has anyone seen something?
Using classic gnu make and inotifywait, without interval-based polling:
watch:
while true; do \
$(MAKE) $(WATCHMAKE); \
inotifywait -qre close_write .; \
done
This way make is triggered on every file write in the current directory tree. You can specify the target by running
make watch WATCHMAKE=foo
This one-liner should do it:
while true; do make --silent; sleep 1; done
It'll run make once every second, and it will only print output when it actually does something.
Here is a one-liner:
while true; do make -q || make; sleep 0.5; done
Using make -q || make instead of just make will only run the build if there is something to be done and will not output any messages otherwise.
You can add this as a rule to your project's Makefile:
watch:
while true; do $(MAKE) -q || $(MAKE); sleep 0.5; done
And then use make watch to invoke it.
This technique will prevent Make from filling a terminal with "make: Nothing to be done for TARGET" messages.
It also does not retain a bunch of open file descriptors like some file-watcher solutions, which can lead to ulimit errors.
How about
# In the makefile:
.PHONY: continuously
continuously:
while true; do make 1>/dev/null; sleep 3; done
?
This way you can run
make continuously
and only get output if something is wrong.
Twitter Bootstrap uses the watchr ruby gem for this.
https://github.com/twbs/bootstrap/blob/v2.3.2/Makefile
https://github.com/mynyml/watchr
Edit:
After two years the watchr project seems not to be maintained anymore. Please look for another solution among the answers. Personally, if the goal is only to have a better output, i would recommend the answer from wch here
I do it this way in my Makefile:
watch:
(while true; do make build.log; sleep 1; done) | grep -v 'make\[1\]'
build.log: ./src/*
thecompiler | tee build.log
So, it will only build when my source code is newer than my build.log, and the "grep -v" stuff removes some unnecessary make output.
This shell script uses make itself to detect changes with the -q flag, and then does a full rebuild if and only if there are changes.
#!/bin/sh
while true;
do
if ! make -q "$#";
then
echo "#-> Starting build: `date`"
make "$#";
echo "#-> Build complete."
fi
sleep 0.5;
done
It does not have any dependencies apart from make.
You can pass normal make arguments (such as -C mydir) to it as they are passed on to the make command.
As requested in the question it is silent if there is nothing to build but does not swallow output when there is.
You can keep this script handy as e.g. ~/bin/watch-make to use across multiple projects.
There are several automatic build systems that do this and more - basically when you check a change into version control they will make/build - look for Continuous Integration
Simple ones are TeamCity and Hudson
#Dobes Vandermeer -- I have a script named "mkall" that runs make in every subdirectory. I could assign that script as a cron job to run every five minutes, or one minute, or thirty seconds. Then, to see the output, I'd redirect gcc results (in each individual makefile) to a log in each subdirectory.
Could something like that work for you?
It could be pretty elaborate so as to avoid makes that do nothing. For example, the script could save the modify time of each source file and do the make when that guy changes.
You could try using something like inotify-tools. It will let you watch a directory and run a command when a file is changed or saved or any of the other events that inotify can watch for. A simple script that does a watch for save and kicks off a make when a file is saved would probably be useful.
You could change your make file to output a growl (OS X) or notify-send (Linux) notification. For me in Ubuntu, that would show a notification bubble in the upper-right corner of my screen.
Then you'd only notice the build when it fails.
You'd probably want to set watch to only cycle as fast as those notifications can display (so they don't pile up).
Bit of archaeology, but I still find this question useful. Here is a modified version of #otto's answer, using fswatch (for the mac):
TARGET ?= foo
all:
#fswatch -1 . | read i && make $(TARGET)
#make -ski TARGET=$(TARGET)
%: %.go
#go build $<
#./$#
I have a Makefile that starts by running a tool before applying the build rules (which this tool writes for me). If this tool, which is a python script, exits with a non-null status code, I want GNU Make to stop right there and not go on with building the program.
Currently, I do something like this (top level, i.e. column 1):
$(info Generating build rules...)
$(shell python collect_sources.py)
include BuildRules.mk
But this does not stop make if collect_sources.py exits with a status code of 1. This also captures the standard output of collect_sources.py but does not print it out, so I have the feeling I'm looking in the wrong direction.
If at all possible, the solution should even work when a simple MS-DOS shell is the standard system shell.
Any suggestion?
There might be a better way, but I tried the following and it works:
$(if $(shell if your_command; then echo ok; fi), , $(error your_command failed))
Here I did assume that your_command does not give any output, but it shouldn't be hard to work around such a situation.
Edit: To make it work with the default Windows shell (and probably any decent shell) you could write your_command && echo ok instead of the if within the shell function. I do not think this is possible for (older) DOS shells. For these you probably want to adapt your_command or write a wrapper script to print something on error (or success).
Ok, here's my own solution, which is unfortunately not based on the status code of the collect_sources.py script, but which Works For Me (TM) and lets me see any output that the script produces:
SHELL_OUTPUT := $(shell python collect_sources.py 2>&1)
ifeq ($(filter error: [Errno %],$(SHELL_OUTPUT)),)
$(info $(SHELL_OUTPUT))
else
$(error $(SHELL_OUTPUT))
endif
The script is written so that any error produces an output beginning with "collect_sources: error:". Additionally, if python cannot find or execute the given script, it outputs an error message containing the message "[Errno 2]" or similar. So this little piece of code just captures the output (redirecting stderr to stdout) and searches for error messages. If none is found, it simply uses $(info) to print the output, otherwise it uses $(error), which effectively makes Make stop.
Note that the indentation in the ifeq ... endif is done with spaces. If tabs are used, Make thinks you're trying to invoke a command and complains about it.
You should use a regular target to create BuildRules.mk:
BuildRules.mk: collect_sources.py
python $< >$#
include BuildRules.mk
This is the standard trick to use when automatically generating dependencies.
Fixing https://stackoverflow.com/a/226974/192373
.PHONY: BuildRules.mk
BuildRules.mk: collect_sources.py
echo Generating build rules...)
python $< >$#
$(MAKE) -f BuildRules.mk
Make sure you're not invoking make/gmake with the -k option.