So I am trying to write a Makefile to remove files nested in directories with the same names. As an example, I have a directory with two files whose paths are ./1234/1234.txt and ./567/567.txt. I want to move those files up to the top directory so that they are just ./1234.txt and ./567.txt. I tried something like this but it didn't work:
variable = ./1234/1234.txt ./567/567.txt
run: $(variable)
$(variable): $$(notdir %): %
mv $< $#
It seems like something that would be a common problem but I haven't been able to find an answer.
Thanks so much for the help! :)
You have your targets the wrong way round, and trying to use functions in a rule won't work unless you use secondary expansion. You'll also need to conditionally add the dependencies because otherwise your rule will give an error if you've already moved the files.
variable = ./1234.txt ./567.txt
run: $(variable)
.SECONDEXPANSION:
$(variable): ./%.txt: $$(wildcard ./$$*/$$*.txt)
mv $< $#
Related
Right now I have the following in my make file to create a symbolic link to a file in my current directory:
MY_FILE := "$(SOME_PATH)/file.txt"
ln -s $(MY_FILE)
What I would like is to do some sort of glob operation to link a bunch of files (or none).
MY_FILES := "$(SOME_PATH)/*.txt"
for file in files:
ln -s $(MY_FILE)
Could anyone point me in the right direction to do this?
Thanks!
The following should do what you want:
MY_FILES := $(wildcard $(SOME_PATH)/*.txt)
LINKS := $(notdir $(MY_FILES))
.PHONY: links clean-links
links: $(LINKS)
$(LINKS): %: $(SOME_PATH)/%
ln -s $<
clean-links:
rm -f $(LINKS)
Explanation:
Make functions. wildcard and notdir are two make functions. Knowing the make functions (at least the most frequently used) really helps writing nice, elegant and efficient make files. Of course, if you have spaces or special characters in your file names you will encounter some problems because most make functions consider spaces as separators. But if you have such files and directory names you should probably use something else than make.
Static pattern rules:
$(LINKS): %: $(SOME_PATH)/%
is a static pattern rule. For each word foo.txt in $(LINKS) it instantiates one single make rule:
foo.txt: $(SOME_PATH)/foo.txt
ln -s $(SOME_PATH)/foo.txt
Knowing how to use pattern rules (static or not) is essential if you want to write compact and generic make files.
Automatic variables. $< is a make automatic variable. In recipes (the commands part of rules) it expands as the first listed pre-requisite. There are many other automatic variables and they are quite handy to write generic rules.
Phony targets. links and clean-links are phony targets because they do not correspond to real files that we want make to create or update. They are kind of short-hands for actions. make links creates all missing links and make clean-links removes them all. As make has no way to guess that these targets are not regular file names we tell it with the .PHONY special target.
Make creates "targets" from "prerequisites". In your case MY_FILES holds prerequisites to create the (link) targets. The rule for a single file might look like this:
file.txt: $(SOME_PATH)/file.txt
ln -s $<
You want multiple files, and in this case we can use this pattern rule:
%.txt: $(SOME_PATH)/%.txt
ln -s $<
You can now create links with the make commands:
make file.txt
make otherfile.txt
Finally we will look into the wildcard function to get all files, and the function notdir function to get the link names from the file names. Following rule will print all text files, and depends on all files in SOME_PATH linked to current working directory:
MY_FILES := $(notdir $(wildcard $(SOME_PATH)/*.txt))
cat: $(MY_FILES)
cat $^
%.txt: $(SOME_PATH)/%.txt
ln -s $<
I'm trying to make a rule that will generate files regarding their names but regardless of the directory.
I'm starting from this makefile (see a previous question of mine):
TARGETS:=$(patsubst %_tpl,%,$(wildcard *_tpl))
.PHONY: all
all: $(TARGETS)
.SECONDEXPANSION:
$(TARGETS): %: $$(wildcard %*_tpl)
./generate $#_tpl > $#
With this, I can do, for instance, make foo.xml. It looks if a set of foo.xml*_tpl files are there, consider them as prerequisites and call the generate script to generate the target.
What I would like to do is, for example, make ../ressources/foo.xml and have make use the rule to create foo.xml but creating it in the ../ressources/ directory, without having to explicitely specify this directory in the makefile.
What I have tried for the moment is adding this to the Makefile:
../ressources/%: $(notdir %)
mv $< $#
Which works, but I would like to avoid creating the file in the current directory before moving it to the destination folder. I would also like not having to specify the possible destination folders in the makefile (but this is less important).
But first of all, does this make any sense? Or is what I want to do just conceptually wrong?
EDIT: Some precisions regarding the _tpl files and the generate script to avoid confusions:
Each target has a main template ($#_tpl) that includes others ($#-part1_tpl, $#-part2_tpl...) and the generate script only takes the main template as argument. The templates are written with Jinja2 (the subparts included with the {% include %} jinja directive).
If you always want the targets in another directory, just say so.
TARGETS:=$(patsubst %_tpl,../resources/%,$(wildcard *_tpl))
.PHONY: all
all: $(TARGETS)
.SECONDEXPANSION:
$(TARGETS): ../resources/%: $$(wildcard %*_tpl)
./generate $#_tpl > $#
I'm not sure if you should have generate $^ >$# instead; superficially, this would make more sense.
If there are multiple *_tpl files for each target (i.e. there are more tpl files than xml files), the TARGETS definition isn't really correct; but we don't have enough information to actually fix it.
On the other hand, if the target directory can change a lot, the sane way forward might be to cd into the target directory and use make -f ../path/to/Makefile -- just make sure your VPATH is set up so that the source files can be found.
Before I start, I'll mention that I'm not using GNU Make in this case for building a C/C++ project.
Makefile:
DEST_DIR = build/
SRC_DIR = src/
$(SRC_DIR)a/ : $(SOMETHING_ELSE)
$(DO_SOMETHING_TO_GENERATE_A_DIR)
$(DEST_DIR)% : $(SRC_DIR)%
cp -r $^ $#
ALL_DEPS += <SOMETHING>
... more code which appends to ALL_DEPS ...
.PHONY: all
all : $(ALL_DEPS)
I've got some files not generated via Make rules in $(SRC_DIR). (For the sake of this example, let's say there's a directory $(SRC_DIR)b/ and a file $(SRC_DIR)c .)
I want to append to ALL_DEPS all targets which represent files or directories in $(DEST_DIR) so that "make all" will run all of the available $(DEST_DIR)% rules.
I thought to do something like this:
ALL_DEPS += $(addprefix $(DEST_DIR),$(notdir $(wildcard $(SRC_DIR)*)))
But of course, that doesn't catch anything that hasn't yet been made. (i.e. it doesn't append $(DEST_DIR)a/ to the list because $(SRC_DIR)a/ doesn't yet exist when the $(wildcard ...) invocation is evaluated and the shell doesn't include it in the results returned by the $(wildcard ...) invocation.)
So, rather than a function which finds all (currently-existing) files matching a pattern, I need one which finds all targets matching a pattern. Then, I could do something like this:
ALL_DEPS += $(addprefix $(DEST_DIR),$(notdir $(targetwildcard $(SRC_DIR)*)))
If it matters any, I've got much of the GNU Make code split across multiple files and included by a "master" Makefile. The ALL_DEPS variable is appended to in any of these files which has something to add to it. This is in an attempt to keep the build process modular as opposed to dropping it all in one monster Makefile.
I'm definitely still learning GNU Make, so it's not unlikely that I'm missing something fairly obvious. If I'm just going about this all wrong, please let me know.
Thanks!
It is simply not possible to do what you're trying to do; you're trying to get make to recognise something that doesn't exist.
This is part of the reason why, in general, wildcards are bad (the other being that you can end up including stuff you didn't mean to). The right thing to do here is to explicitly create a list of source files (ls -1 | sed -e 's/\(.*\)/sources+=\1/' > dir.mk) and perform the patsubst transformation on that list.
If you have additional files that are generate as part of the build, then you can append them to that list and their rules will be found as you'd expect.
I have a variable with a list of prerequisites in varying directories, each specified by a relative path. For example:
HTML_FILES := ../foo1/bar1.html ../foo1/bar2.html ../foo2/bar3.html foo3/bar4.html
(Note that this variable is actually generated, so the full list of folders isn't known in advance.)
For each of these, I want to generate a target file in the current directory, e.g. bar1.xml, bar2.xml, bar3.xml, bar4.xml.
How can I write a rule which will match for this? This is as close as I've come. It seems like something magic in the place of ?????? might do the trick.
build: $(XML_FILES)
$(XML_FILES): %.xml : ??????/%.html
perl $(HTML_TO_XML) $<
Use vpath.
vpath %.html $(dir $(HTML_FILES))
Now one can use simple pattern rule as follows:
$(XML_FILES): %.xml : %.html
perl $(HTML_TO_XML) $<
This should be enough to get things work, but I'm not sure how it would behave if there are some files with the same name in different directories, like ../foo1/bar.html and ../foo2/bar.html.
At work we use a common makefile that other makefiles include (via the include statement) and it has a generic "clean" target that kills some common files. I want to add on to that target in my new makefile so I can delete some specific files, but if I add a clean target in my makefile, it just overrides the old one.
I know I can just make a new target with a new name and have it call clean, and then do other stuff, but for sake of consistency I'd like to be able to just call make clean and have it do everything.
Is that possible?
I've seen this done at several shops. The most common approach is to use double-colon rules, assuming you're using something like GNU make. In your common makefile you would have something like this:
clean::
# standard cleanup, like remove all .o's:
rm -f *.o
Note that there are two colons following clean, not just one!
In your other makefile you just declare clean again, as a double-colon rule:
clean::
# custom cleanup, like remove my special generated files:
rm -f *.h.gen
When you invoke make clean, GNU make will automagically run both of these "branches" of the clean rule:
% make clean
rm -f *.o
rm -f *.h.gen
It's simple to set up and it composes quite neatly I think. Note that specifically because it is a double-colon rule, you don't get the "overriding commands" errors you normally get when you define two rules for the same target. That's sort of the point of double-colon rules.
You can write your own clean and make it a preq of the common clean.
clean: myclean
myclean:
rm whatever
Yours will run first. If for some reason you want the common clean to run first then the solution will be more complicated.
EDIT:
Here is the best solution I can see which runs the common rule before the local one:
include Makefile.common
clean:
$(MAKE) -f Makefile.common $#
rm whatever additional things
The include directive is necessary because the local makefile relies on the common one for things other than clean. The local clean rule overrides the common clean rule, but invokes the common clean rule before doing the additional work. (This overriding will cause some warnings, which is a nuisance; I don't know a good way to silence them.)
Use implicit rules:
existing-target: my-extention
my-extention:
echo running command 1
echo running command 2
Very simple make tutorial to ramp up.
When using :: you can run into issues since make complains when you mix single colon : and double colon :: rules:
a:
echo a
a::
echo aa
will result in:
. . .
*** target file `a' has both : and :: entries. Stop.
It seems like the common makefile's rule should be called something like common-clean. Then each main makefile would declare their clean rule as
clean: common-clean
and you're set.
If that isn't an option, you could take a look at double colon rules, but those introduce a whole other set of issues to consider.
Adding another possible solution I've seen for posterity... I know the OP was wary about changing the common makefile, but something like this works and involves minimal changes.
local makefile 1:
CLEAN=MyExe1 MyExe2
....
include /my/common/makefile
local makefile 2:
CLEAN=MyExe3 MyExe4
....
include /my/common/makefile
common makefile:
clean:
rm -f *.dep *.o *.a $(CLEAN)
Basically the idea is to define some variable (in this case CLEAN) in each local makefile with all the specific items you want to delete. Then the common makefile runs rm -f on all the common file types to delete, plus whatever was specifically flagged for deletion in each local makefile via the CLEAN variable. If there's nothing specific to delete, simply omit the variable declaration or leave it empty (CLEAN=)
So now if we run make clean for local makefile 1, it executes
rm -f *.dep *.o *.a MyExe1 MyExe2
And if we run make clean for local makefile 2, it executes
rm -f *.dep *.o *.a MyExe3 MyExe4
I've found a better solution:
.PHONY: my-extra-clean
clean: my-extra-clean
my-extra-clean:
rm <whatever-you-want>
include Makefile.common
The key line is clean: my-extra-clean. Ie, you can add dependencies in separate stanzas in different makefiles to add behaviour. my-extra-clean is run as a dependency of the root clean target.
For ours, we define a variable, EXTRAFILESTOCLEAN, then when the clean rule runs, it has a step to remove anything specified in the EXTRAFILESTOCLEAN variable
clean:
rm -f *.o
ifdef $(EXTRAFILESTOCLEAN)
rm -f $(EXTRAFILESTOCLEAN)
endif
That can cause unexpected problems if you set that variable to weird values, but you could guard against those by adding prefixes or other tests.
It's in the docs: https://www.gnu.org/software/make/manual/html_node/Overriding-Makefiles.html
So instead of include Makefile you use a wildcard target and forward it to the base Makefile:
# -include base.Makefile <--- not this
%:
#$(MAKE) -f base.Makefile $#