I'm doing something which it feels like should be pretty straightforward. I have source files in a directory called ./src which are transformed and saved to ./. For the sake of the question, I'll just say they get copied there.
Here's what the directory looks like before building:
/src/lib/foo.js
/src/lib/mod/bar.js
/src/bin/baz.js
/Makefile
Here's what should be there after building:
/src/lib/foo.js
/src/lib/mod/bar.js
/src/bin/baz.js
/lib/foo.js
/lib/mod/bar.js
/bin/baz.js
/Makefile
In my Makefile I have this:
SRC_FILES := src/lib/foo.js src/lib/mod/bar.js src/bin/baz.js
OUT_FILES := lib/foo.js lib/mod/bar.js bin/baz.js
These are generated from find and a pattern substitution, but are listed like this here for simplicity...
Now, what I was hoping would work would be:
%.js: src/%.js
cp $< $#
But when I run make lib/foo.js I get Nothing to be done for 'lib/foo.js'.
Looking at the output from debugging, it appears to be trying to find a dependency named lib/src/foo.js. This is kind of what is described in the manual.
This feels as though it should be a really easy thing! What am I doing wrong?
Additionally, I tried:
$(OUT_FILES): $(SRC_FILES)
cp $< $#
...which works, but it rebuilds every single file if any of the sources change. (Remember that my actual transformation isn't just cp, it's a relatively expensive operation, so this is undesirable)
Found something which works. Using secondary expansion, you have access to the target variable in the dependencies. I saw this earlier, but missed the all-important thing: adding an empty target called .SECONDEXPANSION
.SECONDEXPANSION:
all: $(OUT_FILES)
%.js: src/$$#
cp $< $#
Related
So I am trying to write a Makefile to remove files nested in directories with the same names. As an example, I have a directory with two files whose paths are ./1234/1234.txt and ./567/567.txt. I want to move those files up to the top directory so that they are just ./1234.txt and ./567.txt. I tried something like this but it didn't work:
variable = ./1234/1234.txt ./567/567.txt
run: $(variable)
$(variable): $$(notdir %): %
mv $< $#
It seems like something that would be a common problem but I haven't been able to find an answer.
Thanks so much for the help! :)
You have your targets the wrong way round, and trying to use functions in a rule won't work unless you use secondary expansion. You'll also need to conditionally add the dependencies because otherwise your rule will give an error if you've already moved the files.
variable = ./1234.txt ./567.txt
run: $(variable)
.SECONDEXPANSION:
$(variable): ./%.txt: $$(wildcard ./$$*/$$*.txt)
mv $< $#
For some reason make is recompiling the source files that are outside my
project tree every time. I don't normally have such files, but I noticed it while I was testing this Makefile and I would like to understand why. It behaves normally with respect to the source files inside my project tree. Here are my rules. First I have a function
src_to_obj = $(basename $(addprefix $(build_dir)/,$(1))).o
Then I populate the list of program_objects by calling foreach on the list of sources and applying the above function to each one.
$(bin_dir)/$(program_name): $(program_objects)
$(linker) $(link_flags) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) \
$^ $(OUTPUT_OPTION)
define build_template =
$$(call src_to_obj,$(1)) : $(1) $$(call dependency_file,$(1))
// make dependency and build directories
$$(compile) $$(OUTPUT_OPTION) $$<
// rename dependency
endef
$(foreach source,$(program_sources),\
$(eval $(call build_template,$(source))))
As I said, this works just fine for the source files in the project directory. For example, the Makefile lives in ~/Projects/foo and most of the source code is in ~/Projects/foo/src. But if I pull in a file A.cc from the home directory,then that file is recompiled every time I run make.
EDIT:
Following Mad-Scientists suggestion, I ran make -d and examined the output. That made the problem clear. Basically, if I feed this a source file ~/bad.cc, then the rule looks like:
build/~/bad.o : ~/bad.cc .deps/~/bad.d
...
However, my script for the rule is creating the file .deps/home/jim/bad.d. Therefore, every time I run make the dependency file is "not there" and the object file has to be remade. I am using the exact same expressions to refer to the file name in the script. However I am guessing that the shell is expanding the ~ to /home/jim. So somehow I have to escape the ~ in the recipe. Thanks to Mad Scientist for pointing out the -d option for make.
I don't have time to make a short self contained correct example right now, but if I can't solve this problem I will come up with one and post it here later.
EDIT EDIT:
So I solved the problem with the following hack:
program_sources = $(shell echo $(program_sources))
This pre-expands all names so that they are never different in the shell.
I'm trying to create a makefile which downloads some pre-requisite files to a path.
But the foreach documentation is sadly lacking in detail and examples.
I want something like:
image_files = a b
image_versions = 701.2 802.1
image_path = images
images = $(foreach ...) ??
I'd like that to result in an expansion to:
images/701.2/a
images/701.2/b
images/802.1/a
images/802.1/b
And have a phony target to download them from a URL like:
mytarget: $(images)
wget somepath $<
How do I do that?
Ok I have gotten a little further with this. But I'm still a little perplexed as to how I can get this to work.
tag = my-registry:8443/boot-server-data
versions = 557.0.0 607.0.0
images_a = $(foreach ver, $(versions), images/$(ver)/coreos_production_pxe_image.cpio.gz)
images_b = $(foreach ver, $(versions), images/$(ver)/coreos_production_pxe.vmlinuz)
all: build
.PHONY: build $(images_a) $(images_b)
build:
./make-profiles
docker build -t $(tag) .
docker push $(tag)
$(images_a):
wget http://stable.release.core-os.net/amd64-usr/$(foreach version... but depends on each image)/coreos_production
How do you do this?
In fact I only want it to download the images if they aren't there. But for some reason it downloads it every time. It's literally been years since I used Make. I normally use another build tool, but that build tool needs to be modified to make it do what I want here. So I thought I'd just whip this up in the meantime. It's prooving to be a little harder than expected.
You are pretty close, but the problem does not lie with foreach. Let's have a look at just the bit that does the downloading. When make reads the makefile it ends up with something like (after shortening the names a bit for clarity):
images/1/file.cpio.gz images/2/file.cpio.gz:
<recipe>
If, for some reason, make decides to rebuild images/1/file.cpio.gz say, at this point it will expand the recipe, and pass each line of that expansion to a separate shell.
Your job is to write a recipe that does not care whether the target is images/1/file.cpio.gz or images/2/file.cpio.gz. That's another way of saying the recipe should use macros like $# (it will expand to the target).
A sketch:
${images_a}:
wget -O $# http://stable.release.core-os.net/amd64-usr/$#
You may have to munge $# so that wget gets the right url. Just one example:
${images_a}:
wget -O $# http://stable.release.core-os.net/$(dirname $#)/deeper/$(notdir $#)
One complaint about your original makefile: the dependencies are wrong. build needs the downloads to have completed before it runs.
.PHONY: build
build: $(images_a) $(images_b)
...
The images are not phony (just ensure you don't lie to make abut their filenames) either.
The massive advantage of writing your makefile in this way is that it's parallel safe (and that's the whole point of make). When -j is in force, both wgets can proceed at the same time, halving the download time.
I'm trying to make a rule that will generate files regarding their names but regardless of the directory.
I'm starting from this makefile (see a previous question of mine):
TARGETS:=$(patsubst %_tpl,%,$(wildcard *_tpl))
.PHONY: all
all: $(TARGETS)
.SECONDEXPANSION:
$(TARGETS): %: $$(wildcard %*_tpl)
./generate $#_tpl > $#
With this, I can do, for instance, make foo.xml. It looks if a set of foo.xml*_tpl files are there, consider them as prerequisites and call the generate script to generate the target.
What I would like to do is, for example, make ../ressources/foo.xml and have make use the rule to create foo.xml but creating it in the ../ressources/ directory, without having to explicitely specify this directory in the makefile.
What I have tried for the moment is adding this to the Makefile:
../ressources/%: $(notdir %)
mv $< $#
Which works, but I would like to avoid creating the file in the current directory before moving it to the destination folder. I would also like not having to specify the possible destination folders in the makefile (but this is less important).
But first of all, does this make any sense? Or is what I want to do just conceptually wrong?
EDIT: Some precisions regarding the _tpl files and the generate script to avoid confusions:
Each target has a main template ($#_tpl) that includes others ($#-part1_tpl, $#-part2_tpl...) and the generate script only takes the main template as argument. The templates are written with Jinja2 (the subparts included with the {% include %} jinja directive).
If you always want the targets in another directory, just say so.
TARGETS:=$(patsubst %_tpl,../resources/%,$(wildcard *_tpl))
.PHONY: all
all: $(TARGETS)
.SECONDEXPANSION:
$(TARGETS): ../resources/%: $$(wildcard %*_tpl)
./generate $#_tpl > $#
I'm not sure if you should have generate $^ >$# instead; superficially, this would make more sense.
If there are multiple *_tpl files for each target (i.e. there are more tpl files than xml files), the TARGETS definition isn't really correct; but we don't have enough information to actually fix it.
On the other hand, if the target directory can change a lot, the sane way forward might be to cd into the target directory and use make -f ../path/to/Makefile -- just make sure your VPATH is set up so that the source files can be found.
Before I start, I'll mention that I'm not using GNU Make in this case for building a C/C++ project.
Makefile:
DEST_DIR = build/
SRC_DIR = src/
$(SRC_DIR)a/ : $(SOMETHING_ELSE)
$(DO_SOMETHING_TO_GENERATE_A_DIR)
$(DEST_DIR)% : $(SRC_DIR)%
cp -r $^ $#
ALL_DEPS += <SOMETHING>
... more code which appends to ALL_DEPS ...
.PHONY: all
all : $(ALL_DEPS)
I've got some files not generated via Make rules in $(SRC_DIR). (For the sake of this example, let's say there's a directory $(SRC_DIR)b/ and a file $(SRC_DIR)c .)
I want to append to ALL_DEPS all targets which represent files or directories in $(DEST_DIR) so that "make all" will run all of the available $(DEST_DIR)% rules.
I thought to do something like this:
ALL_DEPS += $(addprefix $(DEST_DIR),$(notdir $(wildcard $(SRC_DIR)*)))
But of course, that doesn't catch anything that hasn't yet been made. (i.e. it doesn't append $(DEST_DIR)a/ to the list because $(SRC_DIR)a/ doesn't yet exist when the $(wildcard ...) invocation is evaluated and the shell doesn't include it in the results returned by the $(wildcard ...) invocation.)
So, rather than a function which finds all (currently-existing) files matching a pattern, I need one which finds all targets matching a pattern. Then, I could do something like this:
ALL_DEPS += $(addprefix $(DEST_DIR),$(notdir $(targetwildcard $(SRC_DIR)*)))
If it matters any, I've got much of the GNU Make code split across multiple files and included by a "master" Makefile. The ALL_DEPS variable is appended to in any of these files which has something to add to it. This is in an attempt to keep the build process modular as opposed to dropping it all in one monster Makefile.
I'm definitely still learning GNU Make, so it's not unlikely that I'm missing something fairly obvious. If I'm just going about this all wrong, please let me know.
Thanks!
It is simply not possible to do what you're trying to do; you're trying to get make to recognise something that doesn't exist.
This is part of the reason why, in general, wildcards are bad (the other being that you can end up including stuff you didn't mean to). The right thing to do here is to explicitly create a list of source files (ls -1 | sed -e 's/\(.*\)/sources+=\1/' > dir.mk) and perform the patsubst transformation on that list.
If you have additional files that are generate as part of the build, then you can append them to that list and their rules will be found as you'd expect.