I have a binary that runs under my default shell.
The binary runs perfectly o.k. with:
./binary input.dat
However, if I put this inside a make file:
SHELL=/bin/bash
runos:
./binary input.dat
The code crashes and leaves me quite helpless.
Here is what I tested so far, everything inside my Make file and in the shell:
ulimit -a: identical.
Set the shell to bash as seen above.
diff of the environment variables in SHELL and Make with:
env | sort > vars.1
inside make
env | sort > vars.2
Then run the binary with the extra variables in Make with the following command:
env SHLVL=2 MAKELEVEL=1 MAKEFLAGS= ./binary input.dat
strace in the shell and inside make:
strace -o debug binary input.dat
The code keeps on crashing in Make, and runs in the shell. I am already thinking to dump Make for my test cases and just write shell scripts. But I am curious to know what is the difference.
The Fortran code (a mix of F77, F90 and F95) was compiled with gfortran-4.4 and the following options:
FFLAGS= -g -fbacktrace
So, the concrete question is, what can I do to make this binary run under make in Debian!?
update:
I just tested again in a CentOS machine (v5.8), The code inside Makefile does not crash (GNU Make version 3.81).
I also tested on my Debian Wheezy and openSUSE 11.4, both with GNU Make version 3.82 - It crashes!
I tested on Debian Squeeze with GNU Make version 3.81, and it does crash. So, I think it is not dependent on the GNU Make version.
error when crashing:
enter timeloop
------------------------------------------------------------------------
timestep: 1 time: 2.500E-02 days delt: 2.500E-02 days
-------------------------------------------
terminated in routine react_snia
maximum number of iterations exceeded
bye now ...
-------------------------------------------
failure in timeloop
no further time step reduction possible
try reducing min. time step, bye now ...
trying to work around 'GNU Make' using 'waf'
It has been a while since I wanted to test waf, so here is another interesting observation:
I wrote a wscript which contains a function:
import os
def run(ctx):
os.system('./binary input.dat')
And waf run runs!
If I changed the run method to:
import subprocess as sp
def run(ctx):
sp.call('./binary input.dat', shell=True)
The binary also works as expected.
So, now I am thinking GNU Make forks a new sub-shell in a way that causes may binary to fail (although, under RHEL 5.8 Make did work).
solution: compile make from sources ...
Read to find out more.
OK, so after being pretty much desperate, I did what I simply should have done before blame make for all my troubles.
I thought the problem is Debian specific. But I am guessing the version in CentOS-5.8 is a patched version, although it says it's v.3.81.
So, for those who wonder my solution was:
wget http://ftp.gnu.org/gnu/make/make-3.82.tar.gz
tar xvzf make-3.82.tar.gz
cd make-3.82
./configure
./build.sh
# copy make to the directory with the binary and input and run the local make version
./make
# everything works as expected !!!
I thought let's narrow it down -
wget http://ftp.gnu.org/gnu/make/make-3.80.tar.gz
tar xvzf make-3.80.tar.gz
cd make-3.80
./configure
./build.sh
# copy make to the directory with the binary and input and run the local make version
./make
# everything works as expected !!!
Is it the version 3.81 ?
wget http://ftp.gnu.org/gnu/make/make-3.81.tar.gz
tar xvzf make-3.81.tar.gz
cd make-3.81
./configure
./build.sh
# copy make to the directory with the binary and input and run the local make version
./make
# FAIL! Like with the make version in Debian.
Hence, I think I bumped into some very weird bug in GNU Make v.3.81.
Related
I'm trying to install a package called fminuit http://www.fis.unipr.it/~giuseppe.allodi/Fminuit/Fminuit_building.html
on ubuntu 18.04 machine using Octave. The installation step "make -f Makefile.f2c_lnx.Octave" gives me the following error
WrapIO_Matlab.c:4:10: fatal error: mex.h: No such file or directory
Any idea how to remedy this,
cheers, Damir
The build instructions provided by FMINUIT ask you to manually adapt the Makefile to your setup. I'm guessing you did one of those steps incorrectly. I'm running Octave 6.0.0 (current development sources) and worked fine:
$ wget http://www.fis.unipr.it/~giuseppe.allodi/Fminuit/fminuit-src.tar.gz
$ tar xzf fminuit-src.tar.gz
$ cd fminuit-2011.05.31/fminuit/
# modify Makefile.f2c_lnx.Octave
$ make -f Makefile.f2c_lnx.Octave
$ make -f Makefile.f2c_lnx.Octave install
The tricky part is knowing what to modify on the Makefile. For my case, these were the lines (you need to know the exact Octave version and where you installed it):
#Octave prefix directory (typically /usr or /usr/local): modify if needed
-PREFIX=/usr
+PREFIX=/usr/local
#major version number
-OCTAVE_MAJOR=2
+OCTAVE_MAJOR=6
#minor-release version number
-OCTAVE_MINOR=9.12
+OCTAVE_MINOR=0.0
OBJS= mnintr_wrkrnd.o intrac.o WrapIO_Matlab.o doflush.o
MINUIT=Minuit_.o
INSTDIR=../bin/linux_$(ARCH)/octave$(OCTAVE_MAJOR)
The fminuit Makefile will "install" inside the fminuit source directory. You may also want to adjust its INSTDIR value. You need to adjust your Octave path to use it:
>> addpath('/wherever/you/build/fmunuit/fminuit-2011.05.31/bin/linux_x86_64/octave6')
>> fminuit # you probably can figure out how to call this function
error: fminuit: Too few input arguments
In my makefile I was trying to go into a different directory and invoke jam with something like:
jam-build:
cd <somedirectory> && jam <target>
This did not work, but caused "Unknown target. Please edit 'Jamrules'." even though the exact same command on the command line works perfectly. So I know that jam can find the target.
I also tried
jam-build:
sh -c "cd <somedirectory> && jam <target>"
and variations with the same results. Also this works perfectly from the command line.
Any other command instead of "jam " works as expected (ls, ps, cat, pwd).
Update: Even creating a Makefile in <somedirectory> and running make there gives the same result.
Any ideas to why this happens would be appreciated. And, of course, things to try.
I'm running cygwin with latest gnu make, FT-jam 2.5.2.
I have some Fortran code which uses included modules, and I am wondering what environment variables actually work to set the include path.
To test this out I've been using one of the NAG example codes.
This works:
$ gfortran e04ucfe.f90 -lnag_nag -I/opt/NAG/fll6a23dfl/nag_interface_blocks
This doesn't work:
$ export CPATH=/opt/NAG/fll6a23dfl/nag_interface_blocks
$ gfortran e04ucfe.f90 -lnag_nag
e04ucfe.f90:10.37:
USE nag_library, ONLY : nag_wp
1
Fatal Error: Can't open module file 'nag_library.mod' for reading at (1): No such file or directory
However, the GCC/GFortran documentation states that:
The gfortran compiler currently does not make use of any environment
variables to control its operation above and beyond those that affect
the operation of gcc.
(see https://gcc.gnu.org/onlinedocs/gfortran/Environment-Variables.html and https://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html#Environment-Variables)
I've tried ltrace-ing the gfortran run and can see it looking at other environment variables (e.g. the regular PATH) but not CPATH.
I can work around this with this:
gfortran e04ucfe.f90 -lnag_nag `echo -I$CPATH | sed -e 's/:/ -I/'`
...but why is this necessary? CPATH works fine with gcc, including for other languages than C/C++, so why doesn't this work with gfortran?
Is there something I can successfully use to the same effect as CPATH for gcc with gfortran, to avoid having to pass multiple -I arguments?
Side note: LIBRARY_PATH works fine in a similar way, for replacing the -L/path/to/libs on the gfortran command-line.
As far as I know gfortran does not support this, which is quite annoying. But it is possible to work around it. If you name the following script gfortran and put it in a directory in your $PATH that is searched before the one with the real gfortran in it, then you will have the behavior you want, with $CPATH transparently being expanded into -I arguments:
#!/bin/bash
/path/to/gfortran $(for i in ${CPATH//:/ }; do echo -I"$i"; done) "$#"
Remember to mark it as executable. For example, if my $PATH is /home/amaurea/local/bin:/usr/local/bin:/usr/bin:/bin and gfortran lives in /usr/local/bin, I would set it up as
$ cd /home/amaurea/local/bin
$ cat <<HERE > gfortran
#!/bin/bash
/usr/bin/gfortran $(for i in ${CPATH//:/ }; do echo -I"$i"; done) "$#"
HERE
$ chmod a+x gfortran
Alternatively you can formulate it as a shell alias, but that would be less flexible and will not work in as many situations.
If you are using Makefiles, I got this to work using the subst command. This replaces the : with -I for each path in the file.
usr/bin/gfortran e04ucfe.f90 -lnag_nag -I${subst :, -I,$(CPATH)}
gcc (or other compilers) often generate huge text output and it's very difficult to see where the error is or miss warnings. I've done some search but havn't found a clean simple solution to color code the compiler output (so for instance warnings are yellow, errors are red, etc...)
Gcc 4.9 seems to have added this feature via the -fdiagnostics-color flag:
here's an alternative if you are looking for something very simple:
#!/bin/bash -e
make ${#} 2>&1 | perl -wln -M'Term::ANSIColor' -e '
m/Building|gcc|g++|\bCC\b|\bcc\b/ and print "\e[1;32m", "$_", "\e[0m"
or
m/Error/i and print "\e[1;91m", "$_", "\e[0m"
or
m/Warning/i and print "\e[1;93m", "$_", "\e[0m"
or
m/Linking|\.a\b/ and print "\e[1;36m", "$_", "\e[0m"
or
print; '
Just alias your make to this script and make sure it's executable...
Debian and Ubuntu gives the colorgcc package for that purpose.
And I usually run gcc (and make) thru emacs with M-x compile then the messages are colorized.
addenda
GCC 4.9 has a native colorization facility and GCC 6 - released end of April 2016 - (and probably GCC 5 too) is enabling it by default (when stdout is a terminal).
Ok, I'll just leave a notice about my own (python based) tool also :)
It is called Pluggable Output Processor and designed not only to colorize output of one particular program. Here is sample GCC output before:
After:
See colorgcc, a perl script that coulours the gcc output.
How to install and use colorgcc to colorize your gcc compiler output:
At least 3 answers here so far mention colorgcc, but NONE OF THEM EXPLAIN HOW TO INSTALL IT! (And it's not obvious). So, here's how to install the latest version in Ubuntu!
Go here and click "Clone or download" --> "Download Zip". I saved it into "~/Downloads/Install_Files"
Navigate to it in your file browser and right click it and go to "Extract Here." I now have a directory called "~/Downloads/Install_Files/colorgcc-master".
Copy the "colorgcc.pl" script to "/usr/bin/colorgcc" to "install" it (be sure to use the correct directory according to where you extracted it above): sudo cp ~/Downloads/Install_Files/colorgcc-master/colorgcc.pl /usr/bin/colorgcc
Make it executable: sudo chmod +x /usr/bin/colorgcc
Make the "~/bin" directory if it does not yet exist: mkdir ~/bin
*Make symbolic links that point to "/usr/bin/colorgcc" so that whenever you call gcc or g++ it automatically calls colorgcc instead:
ln -s /usr/bin/colorgcc ~/bin/g++
ln -s /usr/bin/colorgcc ~/bin/gcc
(if you ever want to uninstall colorgcc for some reason just delete these symbolic links "~/bin/g++" and "~/bin/gcc", and the Perl script: "/usr/bin/colorgcc" and you're done)
Done!
Here is a sample g++ output now when I call g++ -Wall -std=c++11 time_until_overflow_2.cpp -o time_until_overflow_2:
*Note: making these symbolic links in "~/bin" only works if "~/bin" is in your PATH variable in a location before the folder where the actual gcc and g++ executables are located. To ensure you have "~/bin" in your path you can view the PATH variable contents with: echo $PATH. If you don't see "/home/YOUR_USERNAME/bin" at the beginning of your path, add it with: export PATH=~/bin:$PATH.
References:
See here for more info. and for where I originally learned most of these steps: https://imranfanaswala.wordpress.com/2009/02/02/setting-up-colorgcc/. Thanks Imran Fanaswala!
~GS
you can use GilCC which is a Ruby tool that will convert GCC output to color in real-time. Right now you have two choices: Perl script (colorGCC) or GilCC and if you already work with Ruby you will like GilCC.
Unique to GilCC; GilCC has warning and errors counters and also shows compile time, very handy when you are trying to improve things. Because it is in Ruby it is cross platform. It is flexible and you can add more gems to customize it anyway you want.
The link to the download page is here.
https://github.com/gilmotta/GilCC
Although GCC 4.9 has -fdiagnostics-color option to enable colored outputs to terminals, I have created a tiny tool called 'crror' to get colorized compiler output.
It supports outputs from make as well. I can add colorize patterns for other tools if anyone requires.
I'm trying to setup a parallel CMake-based build for my source tree, but when I issue
$ cmake .
$ make -j2
I get:
jobserver unavailable: using -j1. Add '+' to parent make rule
as a warning. Does anyone have an idea if it is possible to fix it somehow?
In the generated Makefile, when calling into a sub-make it needs to either use $(MAKE) (not just 'make') or else precede the line with a +. That is, a rule should look like this:
mysubdir:
$(MAKE) -C mysubdir
or like this:
mysubdir:
+make -C mysubdir
If you don't do it one of those two ways, make will give you that warning.
I don't know anything about cmake, so maybe it's generating Makefiles that aren't correct. Or maybe you did something incorrectly on your end.
In my case (with CMake 3.5.2) the trivial cd build && cmake .. && make -j5 works just fine.
But, I do get the jobserver unavailable error when building custom targets (as dependencies of other targets) via the cmake --build . --target foo idiom.
Like this:
add_custom_target(buildroot
COMMAND ${CMAKE_COMMAND} --build . --target install
COMMENT "Populating buildroot..."
)
add_dependencies(deb buildroot)
add_dependencies(rpm buildroot) #... etc
— so that the user can make deb and it Just Works. CMake will regenerate makefiles if needed, run the compilation, install everything exactly as with make install, and then run my custom scripts to package up the populated buildroot into whatever shape or form I need.
Sure enough, I'd like to make -j15 deb — but that fails.
Now, as explained on the mailing list by CMake devs, the root cause lies, surprisingly (or not), within GNU Make; there is a workaround.
The root cause is that make will not pass its jobserver environment to child processes it thinks aren't make.
To illustrate, here's a process tree (ps -A f) branch:
…
\_ bash
\_ make -j15 deb
\_ make -f CMakeFiles/Makefile2 deb
\_ make -f CMakeFiles/buildroot.dir/build.make CMakeFiles/buildroot.dir/build
\_ /usr/bin/cmake --build . --target install ⦿
\_ /usr/bin/gmake install
…
At ⦿ point, make drops jobserver environment, ultimately causing single-threaded compilation.
The workaround which worked great for me, as given away in the linked email, is to prefix all custom commands with +env. Like this:
add_custom_target(buildroot
#-- this ↓↓↓ here -- https://stackoverflow.com/a/41268443/531179
COMMAND +env ${CMAKE_COMMAND} --build . --target install
COMMENT "Populating buildroot..."
)
add_dependencies(deb buildroot)
add_dependencies(rpm buildroot) #... etc
In the end, this appears in the rule for buildroot in the appropriate makefile (CMake generates a bunch of them), and causes GNU Make to behave properly and respect -j.
Hope this helps.
As pointed out by #Carlo Wood in his comment to this answer, trying to convince cmake to add + to the beginning of the command in the cmake-generated makefile is not possible.
A work-around I found is to shield underlying make command from the make flags coming from cmake. This can be done by setting environment variable MAKEFLAGS to empty string for the custom command:
COMMAND ${CMAKE_COMMAND} -E env
MAKEFLAGS=
make <your target and make options>
Hope this helps.