I've got the following in my Makefile,
dosomething:
ifeq (, $(shell which python))
$(error "Python not installed, please download and install Python to create database")
else
cd myfolder; python myfile.py
endif
When I run make dosomething, it throws the error telling me to download and install python. But when I do which python in my shell, it says /usr/bin/python
Not sure what is going on here
I'm guessing the indented lines begin with a tab character? If so then the ifeq, else and endif directives will be considered to be part of the command and passed to the shell for execution.
To ensure make evaluates those directives remove the leading tabs...
dosomething:
ifeq (, $(shell which python))
$(error "Python not installed, please download and install Python to create database")
else
cd myfolder; python myfile.py
endif
Related
I have some python code which works with a C library that I compile with a Makefile. So I create some install/uninstall/installed routines to load/unload/check my library. I add the python commands to it too so everything works.
Now the problem is, not all the machines have the same python version and python might not be set to the most up to date version. I would like to execute python-{x.y} -m pip {install -e . | uninstall {libname} | show {libname}}.
My current approach was:
create a configured.dat file with the following contents:
python3.4 python3.6 python3.8
And the following Makefile routine:
CONFIGURED_FILE=configured.dat
INSTALLED_PYTHON_VERSIONS :=$(file < $(CONFIGURED_FILE))
RED := \033[0;31m
GREEN := \033[1;32m
CYAN := \033[0;36m
NO_COLOR := \033[0m
ifneq ($(VERBOSE),)
VERBOSE := ''
else
VERBOSE := '-q'
endif
EDITABLE := '-e'
.PHONY: install
install:
# make C libs
$(foreach f,$(INSTALLED_PYTHON_VERSIONS), echo -e 'installing $(CYAN)$f$(NO_COLOR)'; $f -m pip install $(VERBOSE) $(EDITABLE) .;)
On my development machine this works. On my deployment machine it doesn't. The variable INSTALLED_PYTHON_VERSIONS is empty. How? Why? The only change I found between the environments that should impact it is the make version, 3.82 on prod, and 4.3 on dev. Prod is centos 7.9.2009 and dev is ubuntu 22.04.1.
The root problem is the wanting to invoke different python versions, the superficial problem is being unable to read the file contents into a list of variables. A good answer to either would be fine by me.
You found the problem yourself: on one machine you're using GNU make 3.82 and on the other 4.3. According to the GNU make NEWS file, the $(file ...) function was added in GNU make 4.0 (released in 2013, so 9 years ago).
You can just use cat:
INSTALLED_PYTHON_VERSIONS := $(shell cat $(CONFIGURED_FILE))
I should also point out that echo -e is not portable. There are no POSIX-specified options to echo and lots of different implementations accept, or not, different options. You probably want to switch to printf which is portable and well-defined.
I want to try and check an executable (in this case Python's pip) is on a specific path (the virtual environment) in the Makefile (this will be done prior to running a make command to install all the requirements, and is a safety measure to ensure they don't end up in system python by mistake).
(Also, yes, I know I can force a virtual env for pip, but this isn't just for me, so I can't guarantee that's done ...)
I've had a few attempts, but currently have this:
DIRENV := $(shell pwd)/.direnv/
PIP := $(shell which pip)
.PHONY: check-pip
check-pip:
FOUND_PIP := $(if $(findstring $(PIP),$(DIRENV)),found,)
$(info FOUND_PIP=$(FOUND_PIP))
ifeq ($(FOUND_PIP),found)
$(info Found pip on the path)
else
$(error ERROR: Cannot find pip))
endif
$(DIRENV) and $(PIP) are correct if I print them out.
There are 2 issues with this:
I can't seem to get findstring to work at all!
The ifeq runs both parts of the conditional regardless.
❯ make check-pip
FOUND_PIP=
Found pip on the path
Makefile:37: *** ERROR: Cannot find pip. Stop.
And just to clarify the make version:
❯ make --version
GNU Make 4.2.1
You have to write your check recipe using shell syntax.
.PHONY: check-pip
check-pip:
case "$(PIP)" in \
("$(DIRENV)"/*) echo "Found pip on the path" ;; \
(*) echo "Cannot find pip"; exit 1 ;; \
esac
Note, you need this if you only want to check this when the user specifically runs the check-pip target.
If you want to check it always whenever the user runs make regardless of which target they specify, then you can use makefile operations but you should not put them in a recipe, because they are run as the makefile is parsed not when a target is built. And you don't need a check-pip target at all.
What I am trying to achieve: a make rule that would create the virtual environment for a script, activate it, and install package dependencies. (I've created a repo with files needed to recreate, for convenience).
Here is my Makefile:
venv:
#echo VENV
virtualenv $# -p python2
foo_requirements: requirements.txt venv .FORCE
#echo PIP
( . venv/bin/activate && pip install -r $< )
.PHONY: foo_requirements
FOO_CMD_SCRIPT = foo.py
FOO_CMD = . venv/bin/activate && python2 $(FOO_CMD_SCRIPT)
$(FOO_CMD_SCRIPT): foo_requirements
#--- Usage ---
all: $(FOO_CMD_SCRIPT)
$(FOO_CMD)
.FORCE:
The target all is there only for testing, in real life I would put the content in a foo.mk file, and include that from another makefile.
What I expect:
make all looks at the dependency FOO_CMD_SCRIPT for (actually a filename to a file on disk). Dependency is the foo_requirements rule (PHONY)
rule foo_requirements has file dependency requirements.txt and venv. There is .FORCE too in here, because I don't know how to check if package installation is already done. So what I think should happen is: 1. nothing for dependency requirements.txt (file exists, no rule) 2. run the rule for venv if it does not exist.
when venv rule has run and the directory is created, run the actual content of the rule: pip install.
after that, the dependencies for all should be finished, and the actual commands should run.
What actually happens:
venv gets created alright
pip never runs
the actual command never runs
Why doesn't the content of the rule foo_requirements run?
Likewise, the all rule content never runs.
Result:
$ make
VENV
virtualenv venv -p python2
created virtual environment CPython2.7.18.final.0-64 in 46ms
creator CPython2Posix(dest=/home/gauthier/tmp/test_mk/venv, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/gauthier/.local/share/virtualenv)
added seed packages: pip==20.3.4, pkg_resources==0.0.0, setuptools==44.1.1, wheel==0.34.2
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator
If you don't tell it otherwise, make will always build the first target in the makefile (along with any of its prerequisites) and then stop.
The first target in your makefile is venv and it has no prerequisites, so that target is built then make stops.
You can run make <target> to run a specific target, for example make all.
Or you can put the all target as the first one in the makefile.
Or you can add .DEFAULT_GOAL: all in your makefile.
See How make Processes a Makefile
Within a Makefile on linux, we can excute shell/bash commands with it to move directories or excute another files. However when porting the same Makefile over to macOS, all the commands are not readible (therefore path and execution are broken). Is there a universal command or workflow that can work on both?
Example of Makefile
.ONESHELL:
COMMAND ?= none
GIT_HASH ?= githash
alpine:
#cd images/alpine
#make ${COMMAND} GIT_HASH=${GIT_HASH} ALPINE_VERSION=3.6.5 TAG=3.6
so in a linux box both #CD and #MAKE are executed but not for macOS Catalina. I would like to make it universal so that both system will respect the appropriate command that follows.
Chances are that your MacOS box uses its default GNU make version (3.81). .ONESHELL was introduced with 3.82. Upgrade with Homebrew or MacPort. Anyway, better avoid make in recipes, prefer $(MAKE), and instead of cd; make you can use GNU make's -C option: $(MAKE) -C images/alpine ...
I am plagued by the Gentoo bug #580414.
In short, the default options mislead configure into not detecting standard include files because some headers contain this code:
#if defined _FORTIFY_SOURCE && _FORTIFY_SOURCE > 0
# if !defined __OPTIMIZE__ || __OPTIMIZE__ <= 0
# warning _FORTIFY_SOURCE requires compiling with optimization (-O)
, and __OPTIMIZE__ is off by default and _FORTIFY_SOURCE is on by default, and the generated warning is perceived as an error, indicating that "stdint.h", "stdlib.h" and many others are absent. Compilation eventually fails and I cannot install programs or even upgrade the gcc itself.
Can I simply put something in environment vars or in the /etc directory to turn on -O or turn off _FORTIFY_SOURCE for every invocation of gcc without editing gentoo build scripts?
Tried in /etc/portage/make.conf
EPATCH_USER_EXCLUDE='*10_all_default-fortify-source*'
CFLAGS="-O2 -O -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0"
CFLAGS_FOR_BUILD="-O2 -O -U_FORTIFY_SOURCE"
without any improvement.
There is no such environment variable. CFLAGS in make.conf won't work because the build systems usually do something like this:
$(CC) $(CFLAGS) $(MY_HARDCODED_CFLAGS)
thus overwriting your flags.
But to mangle any argument passed to gcc, you can use the following workaround.
create a directory, eg. under /usr/local/bin/
create a script which will mangle its arguments as you wish and then passes them to gcc or "/usr/bin/" + basename(argv[0]) (beware of infinite recursion)
make this script executable
create symlinks to the script in that directory with names like gcc, cc, x86_64-pc-linux-gnu-gcc
put a bunch of lines like this into /etc/portage/bashrc:
the_dir="/usr/local/bin/THE_DIR"
if [[ "${PATH}" != *"${the_dir}"* ]] ; then
export PATH="${the_dir}:${PATH}"
fi
Also to save yourself from possible problems in the future, do not forget to put a note about this change somewhere. (As should be done with any workaround anyway.)
Just documenting the commands I actually used to resolve the issue.
mv /usr/bin/i686-pc-linux-gnu-gcc /usr/bin/i686-pc-linux-gnu-gcc.OLD
cat >/usr/bin/i686-pc-linux-gnu-gcc
/usr/bin/i686-pc-linux-gnu-gcc.OLD -O "$#"
ctrl+D
chmod +x /usr/bin/i686-pc-linux-gnu-gcc
cp /usr/i686-pc-linux-gnu/gcc-bin/4.6.3/i686-pc-linux-gnu-gcc /usr/i686-pc-linux-gnu/gcc-bin/4.6.3/i686-pc-linux-gnuu-gcc
mv /usr/bin/i686-pc-linux-gnu-g++ /usr/bin/i686-pc-linux-gnuu-g++
cat >/usr/bin/i686-pc-linux-gnu-g++
/usr/bin/i686-pc-linux-gnuu-g++ -O "$#"
ctrl+D
chmod +x /usr/bin/i686-pc-linux-gnu-g++
cp /usr/i686-pc-linux-gnu/gcc-bin/4.6.3/i686-pc-linux-gnu-g++ /usr/i686-pc-linux-gnu/gcc-bin/4.6.3/i686-pc-linux-gnuu-g++
cp /etc/env.d/gcc/i686-pc-linux-gnu-4.6.3.O /etc/env.d/gcc/i686-pc-linux-gnuu-4.6.3
The credit all goes to rindeal. Recap:
identify the binaries that get invoked as compilers
rename them
in their place, create shell scripts that prepend "-O"
create a gcc profile to appease gcc-config
emerge all the unwieldy packages!