Expand Variables in Target of an Included Makefile - makefile

I am trying to build a framework which is supposed to apply similar operations to different designs/projects. Therefore, I have a general Makefile which defines general targets used for most of the operations. The idea is then that each design has its own main Makefile. This main Makefile includes the general Makefile for the general functionality, defines some variables for some basic configuration of the general Makefile, but can also extend or override variables from the general Makefile or define new targets or override targets when they are not applicable.
So the simplified directory structure looks something like this:
<Root Dir>
| -- targets.mk
| -- design1
| -- Makefile
| -- design2
| -- Makefile
The simplified general Makefile targets.mk looks something like this
${FF_LIST}: ${SRC_FILES}
#echo "Extract FF List for ${DESIGN_NAME}"
.PHONY: get_ff_list
get_ff_list: ${FF_LIST}
#echo "Get FF list for ${DESIGN_NAME} from ${FF_LIST}"
And the simplified design specific Makefile looks something like this:
include ../targets.mk
DESIGN_NAME = design1
FF_LIST = ./misc/ff_list.csv
With this implementation, I have the problem now, when calling the target get_ff_list within the design1 directory, that the recipe for the get_ff_list target is executed but the prerequisites are not, although the echo prints the right file.
user:/tmp/make_test/design1$ make get_ff_list
Get FF list for design1 from ./misc/ff_list.csv
It seems like that the target ${FF_LIST} is not expanded correctly. I can understand that during the time I am including the targets.mk Makefile this variable does not exist. However, my understanding of Makefile's recursive variable declaration with = should expand the variable every time the variable is used (as it is done and seems to work within the recipe itself).
I could include the targets.mk Makefile at the end after the configuration/setting the variables, like:
DESIGN_NAME = design1
FF_LIST = ./misc/ff_list.csv
include ../targets.mk
This seems to work and solve this particular issue. However, when I also want to extend or override variables/targets from the general Makefile, then it becomes a bit less obvious where to include it. Especially, if I am not the only one using the framework and other users create there own new designs.
Maybe this is even not a good way to use Makefiles to begin with. I would also be happy to get suggestions of better ways to implement this.

However, my understanding of Makefile's recursive variable declaration with = should expand the variable every time the variable is used (as it is done and seems to work within the recipe itself).
No. Read the section of the manual on How make Reads a Makefile to understand when variables are expanded immediately, and when the expansion is deferred.
The simplest way to do what you want is for the include targets.mk to come at the end of the Makefile, not at the beginning. If that's not feasible then you'll have to split the main makefile into two parts, one that sets variables and is included first, and the other that defines rules and is included last.

Related

Sub-makefiles and passing variables upward

I have a project that involves sub-directories with sub-makefiles. I'm aware that I can pass variables from a parent makefile to a sub-makefile through the environment using the export command. Is there a way to pass variables from a sub-makefile to its calling makefile? I.e. can export work in the reverse? I've attempted this with no success. I'm guessing once the sub-make finishes its shell is destroyed along with its environment variables. Is there another standard way of passing variables upward?
The short answer to your question is: no, you can't [directly] do what you want for a recursive build (see below for a non-recursive build).
Make executes a sub-make process as a recipe line like any other command. Its stdout/stderr get printed to the terminal like any other process. In general, a sub-process cannot affect the parent's environment (obviously we're not talking about environment here, but the same principle applies) -- unless you intentionally build something like that into the parent process, but then you'd be using IPC mechanisms to pull it off.
There are a number of ways I could imagine for pulling this off, all of which sound like an awful thing to do. For example you could write to a file and source it with an include directive (note: untested) inside an eval:
some_target:
${MAKE} ${MFLAGS} -f /path/to/makefile
some_other_target : some_target
$(eval include /path/to/new/file)
... though it has to be in a separate target as written above because all $(macro statements) are evaluated before the recipe begins execution, even if the macro is on a later line of the recipe.
gmake v4.x has a new feature that allows you to write out to a file directly from a makefile directive. An example from the documentation:
If the command required each argument to be on a separate line of the
input file, you might write your recipe like this:
program: $(OBJECTS)
$(file >$#.in) $(foreach O,$^,$(file >>$#.in,$O))
$(CMD) $(CMDFLAGS) #$#.in
#rm $#.in
(gnu.org)
... but you'd still need an $(eval include ...) macro in a separate recipe to consume the file contents.
I'm very leery of using $(eval include ...) in a recipe; in a parallel build, the included file can affect make variables and the timing for when the inclusion occurs could be non-deterministic w/respect to other targets being built in parallel.
You'd be much better off finding a more natural solution to your problem. I would start by taking a step back and asking yourself "what problem am I trying to solve, and how have other people solved that problem?" If you aren't finding people trying to solve that problem, there's a good chance it's because they didn't start down a path you're on.
edit You can do what you want for a non-recursive build. For example:
# makefile1
include makefile2
my_tool: ${OBJS}
# makefile2
OBJS := some.o list.o of.o objects.o
... though I caution you to be very careful with this. The build I maintain is extremely large (around 250 makefiles). Each level includes with a statement like the following:
include ${SOME_DIRECTORY}/*/makefile
The danger here is you don't want people in one tree depending on variables from another tree. There are a few spots where for the short term I've had to do something like what you want: sub-makefiles append to a variable, then that variable gets used in the parent makefile. In the long term that's going away because it's brittle/unsafe, but for the time being I've had to use it.
I suggest you read the paper Recursive Make Considered Harmful (if that link doesn't work, just google the name of the paper).
Your directory structure probably looks like this:
my_proj
|-- Makefile
|-- dir1
| `-- Makefile
`-- dir2
`-- Makefile
And what you are doing in your parent Makefile is probably this:
make -C ./dir1
make -C ./dir2
This actually spawns/forks a new child process for every make call.
You are asking for updating the environment of the parent process from its children, but that's not possible by design (1, 2).
You still could work around this by:
using a file as shared memory between two processes (see Brian's answer)
using the child's exit error code as a trigger for different actions [ugly trick]
I think the simplest solution is using standard out from a sub Makefile.
Parent Makefile
VAR := $(shell $(MAKE) -s -C child-directory)
all:
echo $(VAR)
Child Makefile
all:
#echo "MessageToTheParent"

GNU Makefile "preprocessor"?

Is there an option to output the "preprocessed" makefile, something equivalent to the GCC's -E option?
I have a project comprised of an hierarchy of dozens of modules, each with its makefile. The build is invoked from a master makefile. That master makefile contains includes, variable definitions, command line option dependent variables, etc.
So, essentially, I am looking for the processed makefile, including all substitutions.
Not that I'm aware of. The closest thing you can get to this is the output from make -qp (or similar) which will dump the make database out at you.
Part of the problem with this request is that many of the substitutions/etc. happen as targets are processed and the list of targets isn't necessarily known without actually attempting a build (at least to an extent) so it isn't necessarily possible to fully expand/etc. a makefile in-place.
The make -d output is also useful for certain incidental information related to how make has processed the makefiles but doesn't contain makefile contents directly.
Remake might also be able to provide some extra useful information.
If you are looking for the computed value of some assembled/etc. global make variable then this blog post by Eric Melski is likely to be very helpful.
tl;dr It adds a target like this to the Makefile (though there's more magic in the blog post so I suggest reading it).
print-%:
#echo '$*=$($*)'
#echo ' origin = $(origin $*)'
#echo ' flavor = $(flavor $*)'
#echo ' value = $(value $*)'
Though in personal use I replaced that first line with something more like this
#echo '$*=$(subst ','\'',$($*))'
to keep the quoting of the result correct.

calling another target inside my makefile without depending on it

I would prefer to avoid a recursive call of make. Unless someone recommends this, it seems inefficient. (if someone does recommend it, we can mark as duplicate of How can I call a specific target from my makefile?)
I've found related threads but most of the solutions involve refactoring the make command, e.g.
makefile execute another target
or depending on the target that you want to run, e.g.
How do I make a target in a makefile invoke another target in the makefile
In fact, I would be interested in how to refactor my makefile. I am newbie with makefiles so I wouldn't be surprised if I'm making things more complicated than they need to be.
# depend on all files in folder abc
myTarget1: $(shell find abc -type f)
make otherTarget
some commands...
# I have various targets that need to do the following:
.PHONY : otherTarget
otherTarget:
common commands...

make variables that depend on a target

I have a Variable in make that is dependant on a file that must be built before the variable can be set, is there a way to get this to work?
parsable_file: dependancies
commands to make parsable_file
targets=$(shell parse_cmd parsable_file)
$(targets): parsable_file
command to make targets
.phony: all
all:$(targets)
If I run $ make parsable_file && make all this will work (I get an error that parse_cmd cant find parsable_file but it works), but just make all will not work. Is there a Make idiom for this?
Set the variable in a file that you include in the main makefile and include a rule in the main makefile for how to build it (the one you already have should be fine).
I believe that will do what you want.
See Including Other Makefiles and How Makefiles Are Remade (which is linked from the first section) for more details on this concept.
Also, unless parseable_file has a usage independent from that parse_cmd call, it should be possible to do the creation and the parsing at the same time and just have the resulting makefile contain the correct value for $(targets) in one step.

multiple makefiles same variable names

im trying to write a non recursive make build system.
what im trying to accomplish is that each makefile will define its own variables and "feel" like he is the only one in the system.
for example:
Project layout:
-Makefile
-src1
----Makefile
----example1.c
-src2
----Makfile
----example2.c
the main Makefile looks like this:
include src1/Makefile
include src2/Makefile
all: $(TARGETS)
each of src1/src2 makefile looks like this:
SRC:=...
OBJ:=...
TARGETS+=bin1 #in src2 its bin2
bin1: $(OBJ)
----gcc ....
$(OBJ_DIR)/%.o : $(SRC_DIR)/%.c
----gcc ....
but this design pattern does not work as the evaluation of recipe in the targets is deferred.
this means all the variables with the same name in src1 gets the value of src2 variables(the last assignment). i tried to replace the include with $(eval include ...) but no luck.
please note that all the variables assignment is done with :=
is there a way to accomplish that each makefile can define what he wants or each variable should be unique?
thanks,
tal
It's difficult to do that. When I've done this in one or two places I've started with make snippets in each directory that can be included from the master make file, and ended up with a scripted approach which scans the make snippets and generates something else. And I had a VPATH in there as well.
There's an example of how this might be achieved in the 'recursive make considered harmful' paper which might be of help.

Resources