I have a Makefile in a project which roughly looks like the following:
.PHONY: lint test teamcity
lint:
# commands here
test:
# commands here
teamcity: lint test
The tools I'm running in make lint and make test are able to determine whether or not they are running in CI (TeamCity) by looking at TEAMCITY_VERSION environment variable. However, we're running them in a Docker container on TeamCity agents, so the variable is not set.
I want to achieve the following:
When make teamcity is run, commands in lint and test targets must be run with TEAMCITY_VERSION set in the environment;
When make test or make lint are run, nothing special should happen (the variable must not be added, though it is not necessary to strip it if it is set in the outside environment).
How can I achieve that?
Would this work for you?
.PHONY: lint test teamcity
lint:
# commands here
test:
# commands here
teamcity:
$(MAKE) TEAMCITY_VERSION=<whatever> lint test
The second make invocation has TEAMCITY_VERSION set on the command line. So, for this second invocation the TEAMCITY_VERSION make variable will be set but also the TEAMCITY_VERSION shell environment variable for all recipes.
I ended up doing the following:
.PHONY: lint test teamcity
lint:
# commands here
test:
# commands here
teamcity: export TEAMCITY_VERSION=1
teamcity: lint test
Here TEAMCITY_VERSION is declared as a target-specific variable, and the export directive exports it into the environment for prerequisite targets.
Related
I'm editing a makefile that ran a lot of sh scripts, and i just added one that runs a terraform command.
When i use directly the .SH file (or the command manually in console, the output is OK, but if i ran it inside the makefile, the output is all together, without spacing or line breaks.
Is there any way to fix this?
PFB commands and outputs.
runcommands.sh (it ran as ./runcommands.sh init/plan/etc)
terraform $1
output:
Terraform initialized in an empty directory!
The directory has no Terraform configuration files. You may begin
working with Terraform immediately by creating Terraform configuration
files.
Makefile:
plan:
$(shell ./runcommands.sh init)
output:
Terraform initialized in an empty directory! The directory has no
Terraform configuration files. You may begin working with Terraform
immediately by creating Terraform configuration files.
The recipe:
plan:
$(shell ./runcommands.sh init)
does not do what you seem to think it does. The $(shell ...) syntax executes the command and builds a string. Because of its location in the Makefile, that string is then executed as a command. You almost certainly do not want the output of ./runcommands.sh to be executed. You want:
plan:
./runcommands.sh init
I want to set environment variables when I run my makefile commands however ideally the variables will be stored in another file.
./Makefile
_init_dev:
. ./config/dev-env
diff-dev: _init_dev
cdk diff
deploy-dev: _init_dev
cdk deploy
./config/dev-env
export HELLO_CDK_DEPLOY_ENVIRONMENT=dev
First I tried to use source to load in the vars however this failed with the message:
make: source: Command not found
I understand this is because make runs sh and not bash.
I then tried to add . in front of my config file to attempt to load the variables that way.
Neither approach has worked. When I run my cdk command I can see from the output the environment variable has not been set.
Can anyone advise on a way to get this to work?
Every recipe line in a makefile is run in its own shell. Environment variables modify the current shell, but those changes go away when the shell exits. So, it's completely impossible for one recipe to set environment variables that are then visible inside a different recipe.
If you want these variables available you must source them in each recipe line, like this:
diff-dev:
. ./config/dev-env && cdk diff
deploy-dev:
. ./config/dev-env && cdk deploy
You can put this into a variable, like:
CDK = . ./config/dev-env && cdk
diff-dev:
$(CDK) diff
deploy-dev:
$(CDK) deploy
Alternatively if your dev-env file is a simple-enough format (such as the one you show) that it works as both a makefile AND a shell script, you could include it:
include ./config/dev-env
diff-dev:
cdk diff
deploy-dev:
cdk deploy
But this will only work for very limited contents of dev-env (basically simple assignment of variables to static strings).
I have a script "set_env.py" that outputs the following uppon execution:
export MY_VAR_1=some_value
export MY_VAR_2=some_other_value
I cannot change this script, it is supplied in my current environment.
Further I have a Makefile that looks like this:
SHELL := /bin/bash
all: set_env
env | grep MY_VAR
set_env:
eval ./set_env.py
With this makefile I expected the grep to list my two variables, however it seems the environment is not set.
I suspect this is because make creates a sub-environment for each line and so the variables set on the first line will not be available in the second.
So the question is, how would I go about exporting the environment from the untouchable script in my makefile?
Actually, the output of the python is valid make.
One option then is to read the output of the python directly into the makefile.
The only fly in the ointment is that $(shell) doesn't cut the mustard.
include Environment.mk
PHONY: test
test:
env | grep MY_VAR
Environment.mk:
./set_env.py >$#-tmp
mv $#-tmp $#
How does this work?
The first thing that make tries to do is to ensure the makefile itself is up-to-date.
Since we have told it to include Environment.mk,
make must ensure that is up-to-date too.
Make finds a rule for Environment.mk
The python is run, creating Environment.mk
Environment.mk is read in, creating two make variables with the export attribute
The makefile is now up-to-date, so make proceeds on to the target (test in this case)
Make runs test's recipe, exporting any variables with the export attribute.
No recursion, but you should ensure the python always spits out make compatible syntax.
EDIT
As #raspy points out, this is not the whole story.
As it stands,
once Environment.mk has been created,
it will never be regenerated.
If set_env.py ever generates different output,
you should tell make what conditions these are by adding dependencies.
If set_env.py takes a trivial time to run,
I advise a simple .PHONY.
That way it will run every time you run make,
and Environment.mk will never be stale.
.PHONY: Environment.mk
Recursive make is your friend I'm afraid.
.PHONY: all
all:
eval $$(./set_env.py) && ${MAKE} test
.PHONY: test
test:
env | grep MY_VAR
There are a few moving parts here.
make all executes the shell command eval $(./set_env.py) && make test
The shell first does the command substitution
$(./set_env.py) is replaced by the export commands
The export commands are passed to the (shell) eval command
The environment variables are defined, but only for this shell invocation
The eval succeeds, so the execution passes to the command after the &&
Make runs recursively, but this second make has an augmented environment
I have a makefile to compile a cross-platform application. For Windows, I've decided to use MSVC. To set up the compiler toolchain (environment variables to easily access compiler for a specific target architecture), MSVC provides a batch file called vcvarsall. If I invoke it manually, it works fine and the makefile can execute too. I've tried to automatize it a bit by invoking it from the makefile, but make seems to run it in a different environment, so it fails to find the compiler.
Is there a way to force make not to execute some commands in a different environment?
Make, like all processes, runs in the environment with which it is started.
Each line of a recipe that it executes runs in a distinct subprocess, specifically,
a distinct invocation of the shell. A subprocess cannot do anything to change the environment of
its parent process or of a sibling subprocess: it can only pass a modified environment
to its own subprocesses.
targetA: prequisitesA...
command1 # New shell; inherits environment of `make`
command2 # Another new shell; inherits environment of `make`
...
targetB: prequisitesB...
command1 && command2 # New shell; inherits environment of `make`
command3 # Another new shell; inherits environment of `make`
...
targetC: prequisitesC...
command1 && \
command2 # New shell; inherits environment of `make`
command3; \
command4 # Another new shell; inherits environment of `make`
...
To make use within a command in a make-recipe of any environment settings that
are applied by running vcvarsall then either make must be run after
vcvarsall in the same environment or an inherited one that preserves those
settings, or else the same must be true of each recipe command that depends
on those settings.
You can direct Make to run all the commands of each recipe in one shell,
per recipe, by writing the pseudo-target .ONESHELL:
in the makefile. But this seems unlikely to be a fix for you.
Probably a suitable automation for your case is to write a wrapper
script that invokes vcvarsall before make.
Depending on how your makefile is structured, and your tolerance for recursive make, you might fancy a recursive solution, in the vein:
ARCH ?= amd64
.PHONY: all
all:
vcvarsall $(ARCH) && $(MAKE) prog
where:-
ARCH (architecture) defaults to amd64 if not defined on the make
commandline or in the calling environment
all is your default target phony target
prog is your real target.
In a project, I have a script called make.sh that builds the project and does some other stuff too. It is working well so far.
Then, just out of curiosity I've tried to create a Makefile that just passed its command-line parameters to this script so I could call it
make snapshot
instead of
./make.sh snapshot
this is the Makefile that I'm using right now
.PHONY: snapshot
%:
./make.sh $#
snapshot:
./make.sh snapshot
But this approach have some problems, I can't pass "build" as a parameter, because I have a "Build" directory (used by SCons), and I can't pass a second parameter to be passed to the script, like:
make upload 192.168.1.10
as make interprets it as two different targets...
Is there a way I can do this with the Makefile?
Yes, you can do it:
%:all
#true
all:
./make.sh $(MAKECMDGOALS)
but this is an abuse of Make. The idea is that Make should interpret its arguments as a set of targets, not an ordered list of general arguments. You should probably use a different tool.