I have the following Makefile:
SHELLS = $(call adjust, profile bash zsh)
adjust = $(foreach arg, $(1), $(DIR)/$(DOT)$(arg))
sys:
$(eval DIR = /etc)
$(eval DOT = )
usr:
$(eval DIR = $(wildcard ~))
$(eval DOT = .)
# Installation Recipes {{{1
shell-sys: $(SHELLS) | sys
#echo $(SHELLS)
shell-usr: $(SHELLS) | usr
#echo $(SHELLS)
$(SHELLS): $(DIR)/$(DOT)%: $(wildcard %/*)
#echo $(SHELLS)
Now I run make shell-sys and expect the following result:
$ make shell-sys
/etc/profile /etc/bash /etc/zsh
/etc/profile /etc/bash /etc/zsh
/etc/profile /etc/bash /etc/zsh
/etc/profile /etc/bash /etc/zsh
However what I get is the following:
$ make shell-sys
/profile /bash /zsh
/profile /bash /zsh
/profile /bash /zsh
/etc/profile /etc/bash /etc/zsh
Inside Static Pattern Rules $(DIR) and $(DOT) fail to expand, however in ordinary rules they seem to expand just fine. What am I doing wrong? Is there a better way to achieve what the sys and usr targets do?
Before running any recipes (shell commands),
make will read the entire Makefile.
While it does this it expands the dependency lines.
Recipes themselves are simply stored away as recursively expanded variables.
At the end of this stage make has a dependency graph.
In your case, make expands
shell-sys: $(SHELLS) | sys
$(SHELLS) becomes /profile /bash /zsh given the emptiness of $DIR and $DOT.
You can get some hint about make expanding these empty variables by giving make the --warn command-line parameter.
Thus it is as if you had written:
shell-sys: /profile /bash /zsh | sys
#echo $(SHELLS)
/profile /bash /zsh: /%:
#echo $(SHELLS)
(I doubt you have a folder on your disk called % so I'm assuming the $(wildcard ...) expands to nothing.)
Once it has a dependency graph in its little hands,
make can walk it,
executing the recipes as it goes.
You asked it to build shell-sys,
so it first tries to build /profile.
[Probably. Makefiles that rely on this implicit ordering of dependencies are broken IMHO.]
Make now builds /profile.
It knows how to do this, you told it earlier.
It expands the recipe.
#echo $(SHELLS) becomes #echo /profile /bash /zsh.
Make passes echo /profile /bash /zsh to the shell without telling you it has done this due to the # prefix.
echo dutifully prints /profile /bash /zsh.
Similarly for the targets /bash and /zsh.
Next we get to build sys.
Make expands the recipe for sys.
Each separate line in the expansion is passed to a new invocation of the shell.
The expansion however is empty,
so the shell is not called.
As a side effect DIR becomes /etc.
Finally the recipe for shell-sys.
Hopefully you can now see that that expands to #echo /etc/profile /etc/bash /etc/zsh
There's a lot not to like in your formulation,
but the biggest advice I can give is not to cause a $(eval ...) to expand at recipe execution time.
Related
Say in the working directory, I have:
$ find . | grep testfile
./testfile1
This is my Makefile:
list_files:
#echo "Show Files..."
#echo $(shell find . | grep testfile)
#touch testfile2
#echo $(shell find . | grep testfile)
#rm testfile2
With make, I got this:
$ make list_files
Show Files...
./testfile1
./testfile1
Why this happened? I expected it to be something like this:
Show Files...
./testfile1
./testfile1 ./testfile2
Then my question is:
Why all the variables/function in recipes inside a rule, are expanded likely simultaneously after target is invoked?
I have found an explanation from this answer that is pretty close to the truth:
The reason your attempt doesn't work is that make will evaluate all lines of the recipe before it starts the first line.
But there is no references provided there, I just cannot convinced myself of this working mechanism of GNU Make.
Could anyone give some clues? Thanks!
Why this happened?
Because, with one caveat that does not apply to your case, make functions such as $(shell ...) are evaluated when the makefile is parsed, not during the execution of recipes.
Why all the variables/function in recipes inside a rule, are expanded likely simultaneously after target is invoked?
They're not. They are expanded before the target's recipe runs. In fact, before make even determines whether the recipe should be run.
But there is no references provided
This is covered in the manual. See in particular section 8.14, The shell function:
The commands run by calls to the shell function are run when the function calls are expanded (see How make Reads a Makefile).
... which refers to section 3.7, How make Reads a Makefile, in particular:
GNU make does its work in two distinct phases. During the first phase it reads all the makefiles, included makefiles, etc. and internalizes all the variables and their values and implicit and explicit rules, and builds a dependency graph of all the targets and their prerequisites. During the second phase, make uses this internalized data to determine which targets need to be updated and run the recipes necessary to update them.
It also relies on section 8.1, Function Call Syntax:
A function call resembles a variable reference. It can appear anywhere a variable reference can appear, and it is expanded using the same rules as variable references.
"The same rules" of course includes the rules for when expansions are performed.
Your recipes should be written in the language of the target shell, usually /bin/sh. All make functions and variable references in each recipe will be expanded before any recipe runs, so their expansions cannot reflect the results of running any recipe during the current make run. It's particularly peculiar to try to use the $(shell ...) function to do that, because you can just use shell code directly in a recipe.
As explained in the comments, all $(shell ...) and other functions are executed (expanded) before executing any lines from the recipe. But you can delay the expansion:
list_files:
#echo "Show Files..."
#echo $$(shell find . | grep testfile)
#touch testfile2
#echo $$(shell find . | grep testfile)
#rm testfile2
This yields the expected output. Note the double $$. Make will still expand all variables/functions first before executing the recipe, and remove one of the dollar signs. The expressions will be expanded again, when executing the recipe lines.
Of course, you're better off without the $(shell) in this example. However, it's not inherently a bad idea. Sometimes you need to execute shell commands before expanding other variables, and then $(shell) is the trivial solution.
Remark: Put this Question on tex.stackexchange and was adviced to also ask here
Under Windows (using make from GnuWin32) i want to set my TEXINPUTS variable in a makefile
My Structure is as follows:
./
|-texmf_project/
|-Package.sty
|-main.tex
|-makefile
I want to be able to use Package.sty in my compilation process. The files look like this:
Package.sty contains:
\ProvidesClass{Package}[]
\NeedsTeXFormat{LaTeX2e}
\RequirePackage{xcolor}
\newcommand{\red}[1]{\textcolor{red}{#1}}
main.tex contains:
\documentclass[11pt,a4paper]{report}
\usepackage{xcolor}
\usepackage{Package}
\begin{document}
Hello World\\
\red{Hello World}
\end{document}
Now i want to set the TEXINPUTS to include what is in ./texmf-project/
Hence my Makefile:
edit:
set TEXINPUTS=./texmf-project//;
pdflatex -synctex=1 -interaction=nonstopmode main.tex
del *.log
del *.aux
However this does not seem to make the location available for compilation.
However if I put the line
set TEXINPUTS=./texmf-project//; directly into the cmd and run make afterwards it works.
I believe I am making a mistake with the set, but I am far from beeing an expert.
Any helps or hints would be greatly appreciated.
Edit: Fixed a spelling mistake
Please remember that every recipe line is executed in a separate shell, so your set is executed in a different shell than pdflatex. Either put those two commands on a single line, or concatenate lines with \ so that lines are executed in a single shell, i.e.:
edit:
set TEXINPUTS=./texmf-project//; \
pdflatex -synctex=1 -interaction=nonstopmode main.tex
Edit:
Since this is Windows, it gets more complicated. Just adding \ and even & is not enough, since make will run it in a single line (through a batch file):
> type Makefile
all:
set TEXINPUTS=./texmf-project// & \
echo %TEXINPUTS%
> make -dr
...
Must remake target 'all'.
Creating temporary batch file C:\Users\user\AppData\Local\Temp\make11444-1.bat
Batch file contents:
#echo off
set TEXINPUTS=./texmf-project// & echo %TEXINPUTS%
...
ECHO is off.
...
Now, cmd.exe uses a line parser that expands variables when the line is parsed, so it first expands %TEXINPUTS% to an empty string (since it was not yet defined) and after that it evaluates the code which sets the variable. Therefore it is crucial that those commands are on separate lines to have variable set before it is used. The easiest way (if you use quite modern make) is including a .ONESHELL directive which would place all recipe into the batch and execute all at once:
> type Makefile
.ONESHELL:
all:
set TEXINPUTS=./texmf-project//
echo %TEXINPUTS%
> make -dr
...
Must remake target 'all'.
Creating temporary batch file C:\Users\user\AppData\Local\Temp\make29908-1.bat
Batch file contents:
#echo off
set TEXINPUTS=./texmf-project//
echo %TEXINPUTS%
...
./texmf-project//
...
Alternatively, if you don't somehow calculate the value, you can just set the variable at the make level (global or target level) and export it to the process, i.e.:
> type Makefile
all: export TEXINPUTS := ./texmf-project//
all:
echo %TEXINPUTS%
> make -dr
...
Must remake target 'all'.
Creating temporary batch file C:\Users\user\AppData\Local\Temp\make25392-1.bat
Batch file contents:
#echo off
echo %TEXINPUTS%
...
./texmf-project//
...
I am trying to remove the path prefix. Here is a small example showing just the issue.
Makefile
dist_directory = ./dist
default: build
build: $(patsubst %.md, $(dist_directory)/%.html, $(wildcard *.md))
$(dist_directory)/%.html: %.md
#echo start
#echo $#
#echo ${$#//$(dist_directory)/}
#echo end
Create a file: touch stuff.md
Then build: make
The output is:
start
dist/stuff.html
end
The expected output is:
start
dist/stuff.html
/stuff.html
end
There are similar posts on Stack Exchange. However, they have not worked for me in a Makefile for some reason. I'm probably doing something wrong.
https://unix.stackexchange.com/questions/311758/remove-specific-word-in-variable
Remove a fixed prefix/suffix from a string in Bash
Remove substring matching pattern both in the beginning and the end of the variable
You have many issues here. The most fundamental one is that if you want to use shell variables you have to escape the dollar sign so that make doesn't interpret it. And, you can only use shell variable substitutions on shell variables, while $# is a make variable, so you need:
#foo='$#' ; echo $${foo//$(dist_directory)/}
The more subtle one is that make always uses /bin/sh (POSIX standard shell) when it invokes recipes, and the above syntax is specific to bash. One way around that would be to explicitly set SHELL := /bin/bash in your makefile to force make to use bash. Luckily that is not necessary because POSIX sh can also do this, as mentioned by Reda in another answer:
#foo='$#' ; echo $${###*/}
But even more, you don't need any of this because make sets the automatic variable $* to the part of the target that matches the stem (the %):
#echo $*.html
It also sets $(#F) to the filename part of the $# variable:
#echo $(#F)
ETA
If you want to do something very similar to your shell variable expansion using GNU make you can use:
#echo $(patsubst $(dist_directory)/%,%,$#)
I am trying to compile for different software directories with different optimization levels etc. I created the following makefile to do so:
OWNER = betsy molly fred
DOG = poodle mutt doberman
COLOUR = brown red yellow
ATTR = big small
LEGS = 0 3
#we want every possible combination to be excercised
OUTPUT_STUFF = $(foreach own,$(OWNER),$(foreach dog,$(DOG),$(foreach col,$(COLOUR),$(foreach attr,$(ATTR),$(foreach legs,$(LEGS),new/$(own)/$(dog)/$(col)/$(attr)/$(legs)/dogInfo.txt)))))
.PHONY: all
all: $(OUTPUT_STUFF)
define PROGRAM_template
own = $(1)
dog = $(2)
col = $(3)
attr = $(4)
legs = $(5)
BUILD_DIR = new/$(own)/$(dog)/$(col)/$(attr)/$(legs)
#for each build directory, we are going to put a file in it containing the build dir. string
$$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $$(BUILD_DIR)
#echo "$$(BUILD_DIR)" > $$(BUILD_DIR)/dogInfo.txt
endef
#call the function many times
$(foreach own,$(OWNER),$(foreach dog,$(DOG),$(foreach col,$(COLOUR),$(foreach attr,$(ATTR),$(foreach legs,$(LEGS),$(eval $(call PROGRAM_template,$(own),$(dog),$(col),$(attr),$(legs))))))))
As you can see, this simple test program loops through different combinations of owner, dog etc. The end goal is to have a directory, new, that has all owners as dirs, and in those, all dogs, etc. At the bottom is just a file with the path in it.
When I run this, the output is:
new/betsy/poodle/brown/big/0/dogInfo.txt
mkdir new/fred/doberman/yellow/small/3
mkdir: cannot create directory `new/fred/doberman/yellow/small/3': No such file or directory
make: *** [new/betsy/poodle/brown/big/0/dogInfo.txt] Error 1
So, for some reason, the target is ok, but the seemingly exact same variable is the last in my loops. Fundamentally, I don't understand what is happening that well.
Weird foreach + user-defined function behavior in Makefiles seems to answer, but I don't fully get it. In my mind, when the function is called, it fills in all instances with one $, and the escaped ones become $(BUILD_DIR). It then 'pastes' the code to the temporary file, and after it's done all the calls it evaluates the file, substituting the variables as normal.
One (ugly) solution I thought of is to make the BUILD_DIR variable different every time like so:
B_D_$(1)_$(2)_$(3)_$(4)_$(5) = ~~~
Alex is correct (although I think he means recipe, not receipt :-)). The best way to debug complex eval issues is to replace the eval function with a call to info instead. So if you have something like:
$(foreach A,$(STUFF),$(eval $(call func,$A)))
then you can rewrite this as:
$(foreach A,$(STUFF),$(info $(call func,$A)))
Now make will print out to you exactly what the eval is going to parse. It's usually pretty clear, looking at the makefile output, what the problem is. In your case you'll see something like this in the output (leaving out all the extra variable settings):
BUILD_DIR = new/betsy/poodle/brown/big/0
$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $(BUILD_DIR)
#echo "$(BUILD_DIR)" > $(BUILD_DIR)/dogInfo.txt
BUILD_DIR = new/betsy/poodle/brown/big/3
$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $(BUILD_DIR)
#echo "$(BUILD_DIR)" > $(BUILD_DIR)/dogInfo.txt
etc. Notice how you're setting the global variable BUILD_DIR every time. In make, variables have only one value (at a time). While make is reading the makefile it expands the target and prerequisite lists immediately, so whatever value BUILD_DIR has at that time will be used for targets/prerequisites, so this works for you.
But when make finishes reading the makefile, the value of BUILD_DIR will always be the last thing you set it to; in this case new/fred/doberman/yellow/small/3. Now make starts to invoke the recipes for each target, and when it does that it will expand BUILD_DIR in the recipes then, and so ALL the recipes will get that same value.
As Alex points out, you should ensure that your recipe uses only automatic variables like $#, which are set correctly for each rule. If you do that you'll notice that you don't really need to redefine the rule at all because it's actually the same recipe for all the targets. And if you notice THAT, you'll notice you don't need the whole eval or call complexity in the first place.
All you have to do is compute the names of all the targets, then write a single rule:
ALLDOGINFO = $(foreach own,$(OWNER),$(foreach dog,$(DOG),$(foreach col,$(COLOUR),$(foreach attr,$(ATTR),$(foreach legs,$(LEGS),new/$(own)/$(dog)/$(col)/$(attr)/$(legs)/dogInfo.txt)))))
$(ALLDOGINFO):
#echo "$#"
mkdir $(dir $#)
#echo "$(dir $#)" > $#
If you don't want the trailing slash you have to use $(patsubst %/,%,$(dir $#)) instead.
The problem is that when $$(BUILD_DIR) is evaluated in receipt, the loop is already complete. The solution is to rewrite the receipt:
$$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $$(#D)
#echo "$$(#D)" > $$#
I don't think your problem is necessarily with something to do with make. This command:
mkdir new/fred/doberman/yellow/small/3
will fail if one of the parent directories (for example, yellow) doesn't already exist. The error it spits out in this case is the one you're getting, so it seems likely this is the case. If you want a command that makes all parent directories of a given directory as needed, you should run mkdir -p, like this:
mkdir -p $$(BUILD_DIR)
See the mkdir man page for a full description of what -p does.
What is the correct way to get the directory where the currently executing makefile resides?
I'm currently using export ROOT=$(realpath $(dir $(lastword $(MAKEFILE_LIST)))) and am running into some problems where when running make with the exact same options will result in different values for ROOT. About 90%of the time it has the correct value, but in the remaining 10% there are a number of invalid paths.
realpath,abspath,lastword and a couple of more functions were only introduced in GNU Make 3.81 [See ref]. Now you can get the current filename in older versions using words and word:
THIS_MAKEFILE:=$(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST))
But I am not sure of a workaround for realpath without going to the shell. e.g. this works with make v3.80:
THIS_MAKEFILE_PATH:=$(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST))
THIS_DIR:=$(shell cd $(dir $(THIS_MAKEFILE_PATH));pwd)
THIS_MAKEFILE:=$(notdir $(THIS_MAKEFILE_PATH))
all:
#echo "This makefile is $(THIS_MAKEFILE) in the $(THIS_DIR) directory"
Which gives
$ make -f ../M
This makefile is M in the /home/sandipb directory
Ref: http://cvs.savannah.gnu.org/viewvc/make/NEWS?revision=2.93&root=make&view=markup
$(shell pwd) is not correct since the makefile might exist in a directory other than pwd (as allowed by make -f).
The OP's proposed
export ROOT=$(realpath $(dir $(lastword $(MAKEFILE_LIST))))
is fine, except, s/he probably wants to use firstword instead, especially if the top level makefile (potentially) includes other makefile(s) prior to assiging to ROOT.
The OPs 10% problem could be explained if there was a conditional include 10% of the time prior to the assignment, but, hey, that's a guess...
For your convenience, when GNU make starts (after it has processed any -C options)
it sets the variable CURDIR to the pathname of the current working directory. This value
is never touched by make again: in particular note that if you include files from other
directories the value of CURDIR does not change. The value has the same precedence it
would have if it were set in the makefile (by default, an environment variable CURDIR will
not override this value). Note that setting this variable has no impact on the operation of
make (it does not cause make to change its working directory, for example).
all:
echo $(CURDIR)