I have the following code in a Makefile, which execute one sequence of command or another based on a environmental variable.
generate :
if test -z "$$VIRTUAL_ENV"; then \
$(PYTHON) -m fades -V &>/dev/null || $(PYTHON) -m pip install --user fades; $(PYTHON) -m fades -r requirements.txt script.py;"; \
else \
python -m pip install -r requirements.txt && python script.py; \
fi
It works as expected, but I would like to do the same thing on multiple targets, to use it on other files, without having to copy this snippet of code multiple times.
My idea would be to set a variable dynamically (based on the condition that has been evaluated), containing the one command or the other, to be used over and over again, like alias in Bash.
Is that a good idea? Is it possible to set a global alias in the Makefile so it can choose between two Python interpreters based on an environmental variable?
Assuming you're using GNU make, you can do it like this:
ifdef VIRTUAL_ENV
PYCMD = python -m pip install -r requirements.txt && python
else
PYCMD = $(PYTHON) -m fades -V >/dev/null 2>&1 || $(PYTHON) -m pip install --user fades; $(PYTHON) -m fades -r requirements.txt
endif
generate:
$(PYCMD) script.py
Note I changed &>/dev/null to >/dev/null 2>&1 because the former is a bash-only feature and is not valid in POSIX sh, and make (by default) always runs /bin/sh which is (on many systems) a POSIX sh.
I don't know why you're using python in one section and $(PYTHON) in the other; it seems like you'd want to use the same in both but anyway.
Related
I'm trying to create a Python virtual environment with a Makefile and also activate it once the make command finishes to ease things for the user. Apparently, this is not possible because "a child process can not alter the parent's environment." I was wondering if there's any workaround for this. This is part of my Makefile so far:
.PHONY: create-venv venv
.DEFAULT_GOAL := all
SHELL=/bin/bash
CPUTYPE = $(shell uname -m | sed "s/\\ /_/g")
SYSTYPE = $(shell uname -s)
BUILDDIR = build/$(SYSTYPE)-$(CPUTYPE)
VENV_NAME?=venv
VENV_DIR=$(BUILDDIR)/${VENV_NAME}
VENV_BIN=$(shell pwd)/${VENV_DIR}/bin
VENV_ACTIVATE=. ${VENV_BIN}/activate
PYTHON=${VENV_BIN}/python3
create-venv:
test -d $(BUILDDIR) || mkdir -p $(BUILDDIR)
which python3 || apt install -y python3 python3-pip
test -d $(VENV_DIR) || python3 -m venv $(VENV_DIR)
venv: ${VENV_BIN}/activate
${VENV_BIN}/activate: setup.py
test -d $(VENV_DIR) || make create-venv
${PYTHON} -m pip install -r requirements.txt
touch $(VENV_BIN)/activate
source ${VENV_BIN}/activate # <- doesn't work
. ${VENV_BIN}/activate # <- doesn't work either
You can activate the environment and run a shell in the activated env:
. ${VENV_BIN}/activate && exec bash
(Please note it must be in one line to be run in one shell; exec is used to replace the shell with a new one.)
When you finish working with the environment you exit and then the Makefile is finished.
You could do something like this.
It relies on your looking at the activate script and seeing what env vars it sets, so it is totally ugly.
$(eval $(shell source $(PYTHON3_VENV)/bin/activate && echo "export PATH := $$PATH; export PYTHONHOME := $$PYTHONHOME; export VIRTUAL_ENV := $$VIRTUAL_ENV" ))
I am trying to run a custom egg through the Pterodactyl panel however, I get the error "/entrypoint.sh: line 30: syntax error: unexpected end of file"
My Docker image is as followed;
FROM ubuntu:18.04
MAINTAINER Amelia, <me#amelia.fun>
RUN apt-get update -y
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y dos2unix curl gnupg2 git-core zlib1g-dev libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev libffi-dev yarn build-essential gpg-agent zip unzip software-properties-common git default-jre python3-pip python-minimal python-pip ffmpeg libopus-dev libsodium-dev libpython2.7 libpython2.7-dev wget php7.2 php7.2-common php7.2-cli php7.2-fpm
RUN curl -sL https://deb.nodesource.com/setup_10.x -o nodesource_setup.sh
RUN bash nodesource_setup.sh
RUN apt-get install -y nodejs
RUN rm -rf nodesource_setup.sh
RUN adduser -D -h /home/container container
USER container
ENV USER=container HOME=/home/container
WORKDIR /home/container
COPY ./entrypoint.sh /entrypoint.sh
CMD ["/bin/bash", "/entrypoint.sh"]
and my entrypoint.sh is as followed;
#!/bin/bash
cd /home/container
MODIFIED_STARTUP=`eval echo $(echo ${STARTUP_PARAMETERS} | sed -e 's/{{/${/g' -e 's/}}/}/g')`
rm -rf *
git clone ${REPO_PARAMETERS}
cd */
if grep -q 'Java' AppType
then
${STARTUP_PARAMETERS}
if grep -q 'PHP' AppType
then
${STARTUP_PARAMETERS}
elif grep -q 'Python2' AppType
then
[ -f "requirements.txt" ] && pip2 install -r requirements.txt ${STARTUP_PARAMETERS} || ${STARTUP_PARAMETERS}
elif grep -q 'Python3' AppType
then
[ -f "requirements.txt" ] && pip3 install -r requirements.txt ${STARTUP_PARAMETERS} || ${STARTUP_PARAMETERS}
elif grep -q 'NodeJS' AppType
then
npm install
${STARTUP_PARAMETERS}
else
echo "Application not supported"
fi
echo "${MODIFIED_STARTUP}"
the Bash file is nowhere near 30 lines long so I'm not really sure.
The guide I used can also be found here
The immediate problem is that you have two if statements, but only one of them is closed with fi; it looks to me like the second one should be elif. But there are a number of other things that look like bad ideas to me:
cd commands in scripts should (almost) always have error tests -- for example, if cd /home/container fails for some reason, the rest of the script (including rm -rf *) will run in an unexpected location. Now, a self-destroying Docker environment may not be as big a deal as a self-destroying real system, but it's still not a good thing. I'd use something like this instead:
cd /home/container || {
echo "Error -- can't move to /home/container, something rotten in Denmark." >&2
exit 1
}
A similar comment applies to cd */.
The next line, that sets MODIFIED_STARTUP, is a mishmash of bad ideas. I'm not familiar with what's going to be in $STARTUP_PARAMETERS, but in general: Use $( ) instead of backticks (and not a weird mix of both). echo $(somecommand) is pretty much a no-op, just run the command directly. Also, variable references (and similar expansions like $( )) should almost always be in double-quotes (exception: on the right side of an assignment). And eval is generally dangerous, and should be avoided if possible. I you give me an example of what $STARTUP_PARAMETERS looks like, I could probably give a cleaned-up version of this.
The big if ... elif... etc has several conditions that do the same thing, e.g.
elif grep -q 'Python2' AppType
then
[ -f "requirements.txt" ] && pip2 install -r requirements.txt ${STARTUP_PARAMETERS} || ${STARTUP_PARAMETERS}
elif grep -q 'Python3' AppType
then
[ -f "requirements.txt" ] && pip3 install -r requirements.txt ${STARTUP_PARAMETERS} || ${STARTUP_PARAMETERS}
On the DRY principle (Don't Repeat Yourself), it'd be better to have a single test for all equivalent situations, like this:
elif grep -q 'Python2' AppType || grep -q 'Python3' AppType
then
[ -f "requirements.txt" ] && pip2 install -r requirements.txt ${STARTUP_PARAMETERS} || ${STARTUP_PARAMETERS}
or even:
elif grep -q 'Python[23]' AppType
then
[ -f "requirements.txt" ] && pip2 install -r requirements.txt ${STARTUP_PARAMETERS} || ${STARTUP_PARAMETERS}
BTW, the use of ${STARTUP_PARAMETERS} without quotes is setting off warning bells for me here, but may be inevitable -- again, I don't know its format. And the && ... || construction isn't always a safe replacement for if then else fi, since it can run both branches. In this script, if requirements.txt exists but the pip2 install command fails, it'll go ahead and run ${STARTUP_PARAMETERS} as well. Is that intentional? If not, I'd use a proper if statement instead.
I install Pip for a user (not system wide) and I would like to check that pip is installed for that user in my script that I run with sudo: sudo ./script.sh
I know to check for a command with command -v pip3 and that works when I enter it in the shell as the user.
But how can I check it in my script?
command -v pip3 exit code is 1 because I am root (because of sudo).
su -c "command -v pip3" "$SUDO_USER" has exit code 1.
sudo -u "$SUDO_USER" command -v pip3 says "command: command not found"
The simplest is
sudo -u "$SUDO_USER" -i command -v pip3
The -i option causes sudo to pass the supplied command line to the user's configured shell using its -c option, instead of trying to execute the command directly. That's necessary because command is a shell built-in; it doesn't exist as a stand-alone executable. (The -i options runs a "login" shell. There is also the -s option which runs a non-login shell. See below.)
If you want to specify a shell explicitly you could do so instead:
sudo -u "$SUDO_USER" /bin/sh -lc "command -v pip3"
Again, a login shell is forced, here by using the -l option.
As a safety feature, sudo normally resets the $PATH to a "safe" value before executing the shell (or the single command). That value will not have any of the modifications made in the /etc/profile and ~/.profile startup scripts, and without those modifications -- which add one or more user-specific directories to the path -- the shell will not find software such as pip3 which has been installed for individual users.
use following command by replacing $USER with the specific user name.
sudo -H -u $USER bash -c 'command -v pip3'
similarly, you can run any command as another user
syntax : sudo -H -u $USER bash -c 'INSERT_COMMAND_HERE'
I have access to a computing cluster (LSF) and the basic way to send stuff to the compute nodes is by doing:
bsub -I <command>
I had this in a file:
bsub -I ../configure --prefix="..." \
--solver=...\
--with-cflags=...\
&& make -j8 \
&& make install
However I just noticed that actually only the first command (configure) was running on the cluster, the remaining two were running locally. What's the best way to group the whole command and pass it to bsub?
Assuming the bsub you are referring to is the one documented here, you have two options:
Surround the entire command to be executed with single quotes (assuming you don't use a single quote anywhere in the command):
bsub -I '../configure --prefix="..."\
--solver=...\
--with-cflags=...\
&& make -j8 \
&& make install'
Feed the command to bsub's standard input, using a HERE document to avoid quoting issues:
bsub -I <<END
../configure --prefix="..." \
--solver=...\
--with-cflags=...\
&& make -j8 \
&& make install
END
Or, very similar to the second one, put the command into a file and provide the file as input.
bsub -I sh -c '../configure --prefix="..." \
--solver=...\
--with-cflags=...\
&& make -j8 \
&& make install'
Goal: when the user types 'make packages', automatically search for the package libx11-dev (required for my program to compile) and, if not found, install it. Here's a stripped-down version of my makefile:
PACKAGES = $(shell if [ -z $(dpkg -l | grep libx11-dev) ]; then sudo apt-get install libx11-dev; fi)
[other definitions and targets]
packages: $(PACKAGES)
When I type 'make packages', I'm prompted for the super-user password. If entered correctly, it then hangs indefinitely.
Is what I'm trying to do even possible from within the makefile? If so, how?
Thanks so much.
The problem is that the shell function acts like backticks in the shell: it takes the output to stdout and returns it as the value of the function. So, apt-get is not hanging, it's waiting for you to enter a response to some question. But you cannot see the question because make has taken the output.
The way you're doing this is not going to work. Why are you using shell instead of just writing it as a rule?
packages:
[ -z `dpkg -l | grep libx11-dev` ] && sudo apt-get install libx11-dev
.PHONY: packages
I figured out a better way, which avoids the problem of having unexpected arguments to the if statement:
if ! dpkg -l | grep libx11-dev -c >>/dev/null; then sudo apt-get install libx11-dev; fi
The -c flag on grep makes it return the number of lines in dpkg -l which contain the string libx11-dev, which will either be 0 (if uninstalled) or 1 (if installed), allowing
dpkg -l | grep libx11-dev -c
to be treated like an ordinary boolean variable.