I have some python code which works with a C library that I compile with a Makefile. So I create some install/uninstall/installed routines to load/unload/check my library. I add the python commands to it too so everything works.
Now the problem is, not all the machines have the same python version and python might not be set to the most up to date version. I would like to execute python-{x.y} -m pip {install -e . | uninstall {libname} | show {libname}}.
My current approach was:
create a configured.dat file with the following contents:
python3.4 python3.6 python3.8
And the following Makefile routine:
CONFIGURED_FILE=configured.dat
INSTALLED_PYTHON_VERSIONS :=$(file < $(CONFIGURED_FILE))
RED := \033[0;31m
GREEN := \033[1;32m
CYAN := \033[0;36m
NO_COLOR := \033[0m
ifneq ($(VERBOSE),)
VERBOSE := ''
else
VERBOSE := '-q'
endif
EDITABLE := '-e'
.PHONY: install
install:
# make C libs
$(foreach f,$(INSTALLED_PYTHON_VERSIONS), echo -e 'installing $(CYAN)$f$(NO_COLOR)'; $f -m pip install $(VERBOSE) $(EDITABLE) .;)
On my development machine this works. On my deployment machine it doesn't. The variable INSTALLED_PYTHON_VERSIONS is empty. How? Why? The only change I found between the environments that should impact it is the make version, 3.82 on prod, and 4.3 on dev. Prod is centos 7.9.2009 and dev is ubuntu 22.04.1.
The root problem is the wanting to invoke different python versions, the superficial problem is being unable to read the file contents into a list of variables. A good answer to either would be fine by me.
You found the problem yourself: on one machine you're using GNU make 3.82 and on the other 4.3. According to the GNU make NEWS file, the $(file ...) function was added in GNU make 4.0 (released in 2013, so 9 years ago).
You can just use cat:
INSTALLED_PYTHON_VERSIONS := $(shell cat $(CONFIGURED_FILE))
I should also point out that echo -e is not portable. There are no POSIX-specified options to echo and lots of different implementations accept, or not, different options. You probably want to switch to printf which is portable and well-defined.
Related
I want to try and check an executable (in this case Python's pip) is on a specific path (the virtual environment) in the Makefile (this will be done prior to running a make command to install all the requirements, and is a safety measure to ensure they don't end up in system python by mistake).
(Also, yes, I know I can force a virtual env for pip, but this isn't just for me, so I can't guarantee that's done ...)
I've had a few attempts, but currently have this:
DIRENV := $(shell pwd)/.direnv/
PIP := $(shell which pip)
.PHONY: check-pip
check-pip:
FOUND_PIP := $(if $(findstring $(PIP),$(DIRENV)),found,)
$(info FOUND_PIP=$(FOUND_PIP))
ifeq ($(FOUND_PIP),found)
$(info Found pip on the path)
else
$(error ERROR: Cannot find pip))
endif
$(DIRENV) and $(PIP) are correct if I print them out.
There are 2 issues with this:
I can't seem to get findstring to work at all!
The ifeq runs both parts of the conditional regardless.
❯ make check-pip
FOUND_PIP=
Found pip on the path
Makefile:37: *** ERROR: Cannot find pip. Stop.
And just to clarify the make version:
❯ make --version
GNU Make 4.2.1
You have to write your check recipe using shell syntax.
.PHONY: check-pip
check-pip:
case "$(PIP)" in \
("$(DIRENV)"/*) echo "Found pip on the path" ;; \
(*) echo "Cannot find pip"; exit 1 ;; \
esac
Note, you need this if you only want to check this when the user specifically runs the check-pip target.
If you want to check it always whenever the user runs make regardless of which target they specify, then you can use makefile operations but you should not put them in a recipe, because they are run as the makefile is parsed not when a target is built. And you don't need a check-pip target at all.
I'm trying to install a package called fminuit http://www.fis.unipr.it/~giuseppe.allodi/Fminuit/Fminuit_building.html
on ubuntu 18.04 machine using Octave. The installation step "make -f Makefile.f2c_lnx.Octave" gives me the following error
WrapIO_Matlab.c:4:10: fatal error: mex.h: No such file or directory
Any idea how to remedy this,
cheers, Damir
The build instructions provided by FMINUIT ask you to manually adapt the Makefile to your setup. I'm guessing you did one of those steps incorrectly. I'm running Octave 6.0.0 (current development sources) and worked fine:
$ wget http://www.fis.unipr.it/~giuseppe.allodi/Fminuit/fminuit-src.tar.gz
$ tar xzf fminuit-src.tar.gz
$ cd fminuit-2011.05.31/fminuit/
# modify Makefile.f2c_lnx.Octave
$ make -f Makefile.f2c_lnx.Octave
$ make -f Makefile.f2c_lnx.Octave install
The tricky part is knowing what to modify on the Makefile. For my case, these were the lines (you need to know the exact Octave version and where you installed it):
#Octave prefix directory (typically /usr or /usr/local): modify if needed
-PREFIX=/usr
+PREFIX=/usr/local
#major version number
-OCTAVE_MAJOR=2
+OCTAVE_MAJOR=6
#minor-release version number
-OCTAVE_MINOR=9.12
+OCTAVE_MINOR=0.0
OBJS= mnintr_wrkrnd.o intrac.o WrapIO_Matlab.o doflush.o
MINUIT=Minuit_.o
INSTDIR=../bin/linux_$(ARCH)/octave$(OCTAVE_MAJOR)
The fminuit Makefile will "install" inside the fminuit source directory. You may also want to adjust its INSTDIR value. You need to adjust your Octave path to use it:
>> addpath('/wherever/you/build/fmunuit/fminuit-2011.05.31/bin/linux_x86_64/octave6')
>> fminuit # you probably can figure out how to call this function
error: fminuit: Too few input arguments
I have some Fortran code which uses included modules, and I am wondering what environment variables actually work to set the include path.
To test this out I've been using one of the NAG example codes.
This works:
$ gfortran e04ucfe.f90 -lnag_nag -I/opt/NAG/fll6a23dfl/nag_interface_blocks
This doesn't work:
$ export CPATH=/opt/NAG/fll6a23dfl/nag_interface_blocks
$ gfortran e04ucfe.f90 -lnag_nag
e04ucfe.f90:10.37:
USE nag_library, ONLY : nag_wp
1
Fatal Error: Can't open module file 'nag_library.mod' for reading at (1): No such file or directory
However, the GCC/GFortran documentation states that:
The gfortran compiler currently does not make use of any environment
variables to control its operation above and beyond those that affect
the operation of gcc.
(see https://gcc.gnu.org/onlinedocs/gfortran/Environment-Variables.html and https://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html#Environment-Variables)
I've tried ltrace-ing the gfortran run and can see it looking at other environment variables (e.g. the regular PATH) but not CPATH.
I can work around this with this:
gfortran e04ucfe.f90 -lnag_nag `echo -I$CPATH | sed -e 's/:/ -I/'`
...but why is this necessary? CPATH works fine with gcc, including for other languages than C/C++, so why doesn't this work with gfortran?
Is there something I can successfully use to the same effect as CPATH for gcc with gfortran, to avoid having to pass multiple -I arguments?
Side note: LIBRARY_PATH works fine in a similar way, for replacing the -L/path/to/libs on the gfortran command-line.
As far as I know gfortran does not support this, which is quite annoying. But it is possible to work around it. If you name the following script gfortran and put it in a directory in your $PATH that is searched before the one with the real gfortran in it, then you will have the behavior you want, with $CPATH transparently being expanded into -I arguments:
#!/bin/bash
/path/to/gfortran $(for i in ${CPATH//:/ }; do echo -I"$i"; done) "$#"
Remember to mark it as executable. For example, if my $PATH is /home/amaurea/local/bin:/usr/local/bin:/usr/bin:/bin and gfortran lives in /usr/local/bin, I would set it up as
$ cd /home/amaurea/local/bin
$ cat <<HERE > gfortran
#!/bin/bash
/usr/bin/gfortran $(for i in ${CPATH//:/ }; do echo -I"$i"; done) "$#"
HERE
$ chmod a+x gfortran
Alternatively you can formulate it as a shell alias, but that would be less flexible and will not work in as many situations.
If you are using Makefiles, I got this to work using the subst command. This replaces the : with -I for each path in the file.
usr/bin/gfortran e04ucfe.f90 -lnag_nag -I${subst :, -I,$(CPATH)}
I have a binary that runs under my default shell.
The binary runs perfectly o.k. with:
./binary input.dat
However, if I put this inside a make file:
SHELL=/bin/bash
runos:
./binary input.dat
The code crashes and leaves me quite helpless.
Here is what I tested so far, everything inside my Make file and in the shell:
ulimit -a: identical.
Set the shell to bash as seen above.
diff of the environment variables in SHELL and Make with:
env | sort > vars.1
inside make
env | sort > vars.2
Then run the binary with the extra variables in Make with the following command:
env SHLVL=2 MAKELEVEL=1 MAKEFLAGS= ./binary input.dat
strace in the shell and inside make:
strace -o debug binary input.dat
The code keeps on crashing in Make, and runs in the shell. I am already thinking to dump Make for my test cases and just write shell scripts. But I am curious to know what is the difference.
The Fortran code (a mix of F77, F90 and F95) was compiled with gfortran-4.4 and the following options:
FFLAGS= -g -fbacktrace
So, the concrete question is, what can I do to make this binary run under make in Debian!?
update:
I just tested again in a CentOS machine (v5.8), The code inside Makefile does not crash (GNU Make version 3.81).
I also tested on my Debian Wheezy and openSUSE 11.4, both with GNU Make version 3.82 - It crashes!
I tested on Debian Squeeze with GNU Make version 3.81, and it does crash. So, I think it is not dependent on the GNU Make version.
error when crashing:
enter timeloop
------------------------------------------------------------------------
timestep: 1 time: 2.500E-02 days delt: 2.500E-02 days
-------------------------------------------
terminated in routine react_snia
maximum number of iterations exceeded
bye now ...
-------------------------------------------
failure in timeloop
no further time step reduction possible
try reducing min. time step, bye now ...
trying to work around 'GNU Make' using 'waf'
It has been a while since I wanted to test waf, so here is another interesting observation:
I wrote a wscript which contains a function:
import os
def run(ctx):
os.system('./binary input.dat')
And waf run runs!
If I changed the run method to:
import subprocess as sp
def run(ctx):
sp.call('./binary input.dat', shell=True)
The binary also works as expected.
So, now I am thinking GNU Make forks a new sub-shell in a way that causes may binary to fail (although, under RHEL 5.8 Make did work).
solution: compile make from sources ...
Read to find out more.
OK, so after being pretty much desperate, I did what I simply should have done before blame make for all my troubles.
I thought the problem is Debian specific. But I am guessing the version in CentOS-5.8 is a patched version, although it says it's v.3.81.
So, for those who wonder my solution was:
wget http://ftp.gnu.org/gnu/make/make-3.82.tar.gz
tar xvzf make-3.82.tar.gz
cd make-3.82
./configure
./build.sh
# copy make to the directory with the binary and input and run the local make version
./make
# everything works as expected !!!
I thought let's narrow it down -
wget http://ftp.gnu.org/gnu/make/make-3.80.tar.gz
tar xvzf make-3.80.tar.gz
cd make-3.80
./configure
./build.sh
# copy make to the directory with the binary and input and run the local make version
./make
# everything works as expected !!!
Is it the version 3.81 ?
wget http://ftp.gnu.org/gnu/make/make-3.81.tar.gz
tar xvzf make-3.81.tar.gz
cd make-3.81
./configure
./build.sh
# copy make to the directory with the binary and input and run the local make version
./make
# FAIL! Like with the make version in Debian.
Hence, I think I bumped into some very weird bug in GNU Make v.3.81.
I have a application running on both Linux & Mac.
Now, for this application I need to copy some libraries from different path.
For Linux,
cp ../gccRelease/libMsSipRelease.a ../../VoipAppLinux/lib/
For Mac,
cp ../DerivedData/MsSipLib/Build/Products/Debug/libMsSipLib.dylib ../../VoipAppLinux/lib/
As you can see both paths are different.
Now, my question is can I use #ifdef Linux or #ifdef APPLE in make file. If can what is the syntax please?
If #ifdef is not possible to use in makefile than how can I solve this problem.
Thank you.
You could use ifdef APPLE, but there's a more automatic way: in a makefile, a command like $(shell uname -s) will tell Make what the OS is. Try this:
OS := $(shell uname -s)
ifeq ($(OS),Linux)
MSSIPLIB := ../gccRelease/libMsSipRelease.a
else
MSSIPLIB := ../DerivedData/MsSipLib/Build/Products/Debug/libMsSipLib.dylib
endif
# And later on
cp $(MSSIPLIB) ../../VoipAppLinux/lib/