In my Makefile I have:
SHELL = /usr/bin/time -f "$# total time: %E" /bin/sh
which works fine on one of my linux boxes. However, on another box it make segfaults. If I remove that line it's fine. Any ideas?
Thanks.
You need to check that
"$#" is expanded as expected
/usr/bin/time works good
look in the man time on the target machine, to see whether that version of time recognizes "%E"
Related
I'm trying to "optimize" my 'clean' target in multi-platform Makefiles so I was looking for a way to remove executable files with or without the Windows-extension .exe.
Of course, you could do
rm file file.exe
but I was looking for something like
rm file(.exe)?
I also tried
rm file{,.exe}
which doesn't work either.
I was surprised to see that what I tried did not work, so I'm mostly posting this to learn more about globing, as the version with the two explicit filenames works fine.
Make executes commands with /bin/sh by default, which has limited globbing support.
$ cat Makefile
test:
echo foo{bar,baz}
$ make
echo foo{bar,baz}
foo{bar,baz}
If you want fancy features like curly braces to work you'll need to switch the shell by setting SHELL.
$ cat Makefile
SHELL = /bin/bash
test:
echo foo{bar,baz}
$ make
echo foo{bar,baz}
foobar foobaz
I wouldn't necessarily advise doing so as it makes your Makefile less portable.
For what it's worth, GNU Automake's strategy is to set an EXEEXT variable based on the platform. Then the clean rule is:
rm -f file$(EXEEXT)
I'm trying to reproduce a situation which happens only in some machines. To reproduce it, I create a directory with 2000 files:
mkdir /tmp/test
cd /tmp/test
for f in $(seq 1 2000); do touch $f.txt; done
Then I use the following Makefile (simplified from the real use case):
FILES:=$(shell find . -name '*.txt')
%.done: %.txt
#echo "done $#"
toolong:
#$(foreach file,$(sort $(FILES)), \
if $(MAKE) $(file); \
then echo "did $(file)" >> $#; \
else echo "failed $(file)" >> $#; fi; )
Running make produces, unsurprisingly, an error:
make: execvp: /bin/sh: Argument list too long
This question presents a solution which does work. However, I need to understand exactly why this error does not happen on my colleagues' computers. I tried the following things:
Increasing stack limit (ulimit -s gives the same result on both machines, 8192, and increasing it does not change anything);
Checking getconf ARG_MAX (2097152 in both machines);
Checking MAX_ARG_STRLEN (131072 in both machines);
Using a different shell (zsh is being used in both machines; I also tried bash, dash and sh, via export SHELL=<shell> make, and also by replacing the symlink /bin/sh -> /bin/bash with a link to dash).
Finally, I tried recompiling Make from source, and realized that, even when I compile the same version of Make (4.1) in my Ubuntu test machine, I get the same behavior as I had in my Fedora, that is, the error "argument list too long".
make --version only shows a single difference between them:
Version from the apt package:
GNU Make 4.1
Built for x86_64-pc-linux-gnu
Version compiled from source (./configure && make):
GNU Make 4.1
Built for x86_64-unknown-linux-gnu
I even tried compiling make-dfsg, which results in an identical make --version, but the result is the same as with my other manually-compiled make.
By increasing the number of files on Ubuntu, I managed to identify that the actual limits in the size of the generated command line are:
Fedora or Arch Linux (both with Make 4.2.1), or Ubuntu with manually-compiled Make 4.1: 128 KB (~1200 files);
Debian Sid or Ubuntu, both with Make 4.1 installed from apt package: 2 MB (~19300 files).
I'd really like to understand (1) why this difference exists, and (2) how could I compile Make to obtain the higher limit, so that I can have the exact same behavior on both machines.
Your recipe expands as a one-line shell compound command (because of the line ending backslashes). And this line is probably way too long. Did you check what it looks like (the shell compound command) on all machines? I suggest that you wrap it in $(info...):
toolong:
#$(info $(foreach file,$(sort $(FILES)), \
if $(MAKE) $(file); \
then echo "did $(file)" >> $#; \
else echo "failed $(file)" >> $#; fi; ))
and see if the output is the same on your machine and on the others. If, for any reason, yours is longer, it could be the explanation. Else it must be a OS difference...
Note: you could have one rule per target, instead of one single rule with a huge recipe for all.
Note: your recipe (independently from its fantastic length) does nothing useful. As the files already exist and do not have pre-requisites, all your sub-make calls will just tell you that the files are up-to-date.
I am trying to time how long each file in my codebase takes to compile.
According to this SO answer, you can write a script which does times the actual compilation and then stores the results in a file
/tmp/time-build:
#!/bin/bash
{ time g++ "$#"; } 2> >(cat <(echo "g++ $#") - >> /tmp/results.txt)
You can then override CMAKE_CXX_COMPILER when calling cmake so that make uses the script to perform the compilation
cmake .. -DCMAKE_CXX_COMPILER=/tmp/time-build
make
This works as advertised, and yields results similar to the following:
real 0m1.190s
user 0m1.044s
sys 0m0.140s
g++ -Werror -Wall -Wextra ... /src/test/foo.cpp
However, to ease processing, I would like to store only the real time, and have it on the same line as the g++ command.
Question:
My command line fu is not up to the task of turning this:
{ time g++ "$#"; } 2> >(cat <(echo "g++ $#") - >> /tmp/results.txt)
Into a command which captures only the real output of time, and includes it along with the echo "g++ $#" on a single line.
I don't know what the >(cat <( and ) - parts of the above command mean, and my attempts to incorporate a grep real and echo have failed
How can I do that?
Alternately, if there is a more idiomatic way to get cmake to output timings for each file processed, that too would be ideal.
Probably the best way to get compile times for a CMake-based project is to use ninja instead of make.
To do this you have to specify ninja generator while configuring:
cmake -G Ninja <...>
Then build using ninja:
ninja
After that, look at the file called .ninja_log. First column is start time in milliseconds, second is end time, forth is target name and the last one is a hash of the command line used.
There are even some solutions for viewing and analyzing this log: ninjatracing converts .ninja_log to a format readable by Chrome's tracing tool.
cmd=(clang "$#")
{ time "${cmd[#]}"; } 2> >( tee >( real=$(grep -Po 'real\K.*'); echo "${cmd[*]} $real" >>result.txt ) )
I'm trying to use entr to recompile as soon as I change a C file with the following command:
$ echo ex8.c | entr make ex8 && ./ex8
When I run it I get the cc output but then nothing happends
$ echo ex8.c | entr make ex8 && ./ex8
cc -Wall -g ex8.c -o ex8
If I just write it manually it works great
$ make ex8 && ./ex8
How should I write it with entr for it to work?
The man page wasn't quite detailed enough, but I installed it and tried it. Note your command is actually three distinct shell commands: echo ex8.c, entr make ex8, and ./ex8. These are connected by a pipe (the first two) and the && operator (the final two). The two commands in the pipeline are both started together in parallel. The final command will not be invoked until the pipeline completes, then if the exit code is success it be run.
This means that the final command ./ex8 will not be started until after the entr make ex8 command finishes. But, entr does not exit after it runs make one time: its entire point is to continue to watch the source file and run make every time it changes. That's why the final command is never invoked: entr never exits.
There are multiple ways to fix this but the simplest way is to add a rule to your makefile that will build AND RUN the command, then call that with make; add this to your makefile:
.PHONY: run-ex8
run-ex8: ex8
./$<
Now use entr like this:
echo ex8.c | entr make run-ex8
If I run mkdir -pv, the -p works, but I do not get verbose output; however, if I run just mkdir -v, the verbose output appears as expected. Also potentially of note, the longform of -v does not seem to work at all.
From my testing:
mkdir -p a/b/c: creates a/, a/b/, and a/b/c/, gives no output to terminal (as expected)
mkdir -v d: creates d/ and outputs mkdir: created directory 'd' (as expected)
mkdir -pv e/f/g: creates e/, e/f/, and e/f/g/, gives no output to terminal (why?)
mkdir --verbose h: gives a illegal option -- - error (why?)
Update: I filed a bug report with Apple for this issue, and received the following reply:
Their answer that "-v is not applicable" does not make much sense to me, since mkdir -v works just fine, but since there are various workarounds and I am no longer even using OSX, I don't think it's worth pursuing any further to me.
Macs use BSD-based code that is (mostly) POSIX compliant but (mostly) without GNU extensions (such as double-dash long options). The man page does document -v and -p, and -p works but does seem to suppress the -v option (which is probably when it is most useful).
One of your options is to file a bug with the Darwin or BSD teams, or with Apple. That's the way it is; it is arguably not the way it should be. (GNU mkdir supports -v and prints the directories it creates when used with -p, which makes more sense, and supports the 'it is a bug' contention.)
With thanks to SnoringFrog:
Another option is to install and use the GNU mkdir command on OSX. It is part of GNU coreutils and you could install it as explained in How to replace Mac OS X utilities with GNU core utilities at Ask Different. Then, you could alias mkdir to point to gmkdir to get the expected behavior (assuming you don't use --default-names when installing the GNU tools).
You could also do this if logging is critical (since the -v flag works in the simple case):
mkdir -v path &&
mkdir -v path/to &&
mkdir -v path/to/{destination1,destination2,etc} ;
Think Different or Think Again?!