I have a list of files like this:
/path1/file1.root
/path2/file2.root
...
/pathn/filen.root
that I want to use as a target. The only prerequisite to each target is a file which is in a /path<>/log/ folder, has the same basename but different extension. So for example:
/path1/
|-- file1.root
'-- log/
'-- file1.mac
/path2/
|-- file2.root
'-- log/
'-- file1.mac
...
/pathn/
|-- filen.root
'-- log/
'-- file1.mac
I can only find all these folders directly seeking my directory structure, all the paths are different and thus not reproducible a priori.
I want to produce the following rules:
/path1/file1.root : /path1/log/file1.mac
<tab>do_something
/path2/file2.root : /path2/log/file2.mac
<tab>do_something
...
And since automatic variables are not available in prerequisites this is not so easy to achieve (or at least in my limited understanding)
I managed to do this with the following:
define function
$(1) : $$(dir $(1))log/$$(subst .root,.mac,$$(notdir $(1)))
<tab>do_something
endef
$(foreach f,$(LIST),$(eval $(call function,$(f))))
But I don't know why it takes very long (~ 4min!) for make to read the Makefile (~ 50 folders in LIST)
I also tried something with static patterns but with no luck...
Does anyone have a better solution?
Personally I would start with a list of existing mac files and construct therefrom a list of desired root files, rather than a list of existing root files. But either way we will get a list of root files.
I think the best approach is a static pattern rule. We can use Secondary Expansion to construct the prerequisite mac file:
.SECONDEXPANSION:
$(ROOTS): %.root: $$(dir $$*)log/$$(notdir $$*).mac
do something with the target $# and the prerequisite $<
A better solution might be to use a build system that allows for dynamic dependencies. Here's a make file that would work with GoodMake:
#? *.root
# Find the corresponding .mac file
mac=$(echo $1 | sed -E 's+(.*/)(.*).root+\1log/\2.mac+')
# Flag the .mac file as a dependency
$0 $mac
# Build the .root
cat "something" >$1
#? !all
$0 all-roots # List all the roots
$0 $(cat all-roots) # Make all the roots
#? all-roots
# List all desired roots by finding all .mac files
find -path '*/log/*.mac' | sed -E 's+(.*/)log/(.*).mac+\1\2.root+' >$1
Related
I'm trying to make a make file for a static page generator, and I'm using jinja and pandoc so far, and the idea is to have a file structure as such
.
|-content
|-public
|-templates
|-Makefile
VPATH=public
TARGETS=$(find content -regex ".*(htm|md)" | sed -e "s/md$/htm/g;s/^content/public/g")
all: $(TARGETS)
#echo fullbuild
public/%: content/%
content/%.md:
# Pandoc script
pandoc -i $# -o ${${#:.md=.htm}:content=public}
content/%.htm:
# Jinja Script
The problem I'm having (At least I think that's it) is that according to me the syntax is
# For a final result
target: dependency
commands
# A rule for dependency
dependency:
commands
My dependencies are in the content dir and my targets are in the public dir that may or may not exist yet, and almost all the files I generate will be htm files, and in that case if the target is public/some/route/to/file.htm the dependency will be any of this two content/some/route/to/file.(htm|md).
I can easily generate by walking the content dir, and changing extensions.
How should I write the rules properly, so
- Make knows where to "watch" for changes every time I do a make, because right now it points that every file is up to date
- How do I indicate properly the dependency of a file to it's content file.
This rule:
public/%: content/%
does nothing because pattern rules without recipes delete existing pattern rules, they don't define new pattern rules.
It's very simple, you should write two rules like this:
public/%.htm: content/%.md:
# Pandoc script
pandoc -i $< -o $#
public/%.htm: content/%.htm
# Jinja Script
Here's a hint: whenever you're writing a makefile recipe and you discover that you need to create a target which is different than exactly $#, unmodified, immediately stop what you're doing and back up: you've taken a wrong turn.
So I am trying to write a Makefile to remove files nested in directories with the same names. As an example, I have a directory with two files whose paths are ./1234/1234.txt and ./567/567.txt. I want to move those files up to the top directory so that they are just ./1234.txt and ./567.txt. I tried something like this but it didn't work:
variable = ./1234/1234.txt ./567/567.txt
run: $(variable)
$(variable): $$(notdir %): %
mv $< $#
It seems like something that would be a common problem but I haven't been able to find an answer.
Thanks so much for the help! :)
You have your targets the wrong way round, and trying to use functions in a rule won't work unless you use secondary expansion. You'll also need to conditionally add the dependencies because otherwise your rule will give an error if you've already moved the files.
variable = ./1234.txt ./567.txt
run: $(variable)
.SECONDEXPANSION:
$(variable): ./%.txt: $$(wildcard ./$$*/$$*.txt)
mv $< $#
This seems slightly related to How to write Makefile where target and source files have the same extension?. In that question the extensions are the same, but the input and output files seem to be in the same directory and filenames are being conditionally renamed.
I have a large collection of .txt files in ../src/ that need to be processed, and dumped into ./ (which is a directory called target/) as txt files of the same name. I want to use make, so that only files in ../src/ that have been changed get updated in ./. I would like to get the prototype working before I put the real code in.
My Makefile in ./ is as follows:
DIR = ../src
INPUTS = $(wildcard $(DIR)/*.txt)
OUTPUTS = $(patsubst $(DIR)/%.txt,%.txt,$(INPUTS))
all: $(OUTPUTS)
.PHONY: $(INPUTS)
check:
#echo "DIR = $(DIR)"
#echo "INPUTS = $(INPUTS)"
#echo "OUTPUTS = $(OUTPUTS)"
%.txt: $(DIR)/%.txt
sed -e "s/test/cat/g" "$<" > $#
For now, the contents of ../src/ are test1.txt and test2.txt.
As the Makefile stands now, running make test2.txt generates the file as expected.
target/ $ make test2.txt
sed -e "s/test/cat/g" "../src/test2.txt" > test2.txt
Running make check shows the INPUTS and OUTPUTS correctly.
target/ $ make check
DIR = ../src
INPUTS = ../src/test1.txt ../src/test2.txt
OUTPUTS = test1.txt test2.txt
If I run make all, it generates every file, every time. This is expected with the .PHONY $(INPUTS) line in there.
If I remove the .PHONY $(INPUTS) target, Make gets all bound up in itself trying to find the target to make ../src/test1.txt and keeps prefixing $(DIR) in front of it until it makes too long of a filename and gives up.
make: stat: ../src/../src/../src/ [repeat for a few pages] ../src/../src/test1.txt: File name too long
make: stat: ../src/../src/../src/ [repeat for a few pages] ../src/../src/../src/test1.txt: File name too long
make: *** No rule to make target `../src/../src/../src/[repeat]../src/../src/test1.txt', needed by `../src/[repeat]../src/../src/test1.txt'. Stop.
It never does get to processing test2.txt.
As I was drafting this, I had the idea to remove the ../ from the DIR,
and relocate the Makefile so it was parent to both src/ and target/. That approach seems to work, but isn't ideal. Eventually there would be a chain of these Makefiles, each pulling from one directory to another.
Is there a way to keep the Makefile in 'target/' along with the generated destination files, and base those destination files off of something in a relative path?
Replace
%.txt: $(DIR)/%.txt
with:
${CURDIR}/%.txt: $(DIR)/%.txt
This way %.txt does not match any .txt file in any directory. In other words, you limit this rule's scope to files in ${CURDIR}/ only and this prevents that endless recursion.
See ยง10.5.4 How Patterns Match for more details.
It is also good practice to avoid relative paths:
DIR = $(abspath ../src)
I have the following Makefile:
all: generate print # <-- doesn't work
date:
date > date.txt
ls:
ls -la > ls.txt
generate: ls date
print: *.txt
cat $^
clean:
rm *.txt
The targets date and ls generate a file each, the target print, prints them out.
How do I write the target all, so that it first generates the files and then prints it?
Just add the pipe symbol:
all: | generate print
From the make manual:
Order-only prerequisites can be specified by placing a pipe symbol (|) in the prerequisites list: any prerequisites to the left of the pipe symbol are normal; any prerequisites to the right are order-only:
targets : normal-prerequisites | order-only-prerequisites
The dependancy hierarchy of the rules you want is all -> print -> generate.
'print' should be dependant on 'generate'.
The Makefile you have is interesting. On first pass it generates the .txt files using { all -> generate -> ls, date } hierarchy but the print fails. On second pass if you don't do clean it works.
The 'print' rule you have is a valid rule. But doesn't allow make to know that you need to do the 'generate' actions before doing the 'print'.
You could EXplicitly make a print rule which says it is dependant on generating date.txt and ls.txt. Try change your 'all' and 'print' rules like this . . .
EDIT: SORRY! First Answer I gave doesn't work. Tested it. This works.
all: print # <- just print target here as you don't want generate to happen after print
.
print: generate ls.txt date.txt
cat $^
Make could decide to do print action first and the generate action afterwards if print is not made dependant on the files explicitly or generate.
This works but we get an error as the cat of non existant file generate doesn't work.
Taking this a bit further . . . Get rid of the generate rule. And I think the date and ls rule would be better if they explicitly detailed what file they generated, i.e.
all: print
date.txt:
date > date.txt
ls.txt:
ls -la > ls.txt
print: ls.txt date.txt
cat $^
*.txt. If you want to operate on multiple files with same extension then you can do different things e.g. put the list of files in a make variable. You can have a make rule to calculate this list of files (using shell cmd in make). But that make rule cannot be dependant on the files already existing if it is the thing that is generating them.
The make manual gives a rule very close to your original print rule - using wildcard in rule prerequisites.
http://www.gnu.org/software/make/manual/make.html#Wildcard-Examples
Make has quite a simple hierarchy of dependancies. Make will follow dependancies in order.
Getting your head around the sequence a makefile will follow can be tricky but it is actually quite simple enough so well worth while working on understanding it.
It is a common problem for makefiles for some part of make system to be missing a dependancy on another. Often build systems will get away with this but sometimes it can cause weirdness (e.g. object files being compiled after being linked in).
The make manual has a good introduction.
http://www.gnu.org/software/make/manual/make.html#Introduction
Secondary Expansion not work for me.
I choose to delay wildcard call by use another make in recipe:
all: generate
$(MAKE) print
Before I start, I'll mention that I'm not using GNU Make in this case for building a C/C++ project.
Makefile:
DEST_DIR = build/
SRC_DIR = src/
$(SRC_DIR)a/ : $(SOMETHING_ELSE)
$(DO_SOMETHING_TO_GENERATE_A_DIR)
$(DEST_DIR)% : $(SRC_DIR)%
cp -r $^ $#
ALL_DEPS += <SOMETHING>
... more code which appends to ALL_DEPS ...
.PHONY: all
all : $(ALL_DEPS)
I've got some files not generated via Make rules in $(SRC_DIR). (For the sake of this example, let's say there's a directory $(SRC_DIR)b/ and a file $(SRC_DIR)c .)
I want to append to ALL_DEPS all targets which represent files or directories in $(DEST_DIR) so that "make all" will run all of the available $(DEST_DIR)% rules.
I thought to do something like this:
ALL_DEPS += $(addprefix $(DEST_DIR),$(notdir $(wildcard $(SRC_DIR)*)))
But of course, that doesn't catch anything that hasn't yet been made. (i.e. it doesn't append $(DEST_DIR)a/ to the list because $(SRC_DIR)a/ doesn't yet exist when the $(wildcard ...) invocation is evaluated and the shell doesn't include it in the results returned by the $(wildcard ...) invocation.)
So, rather than a function which finds all (currently-existing) files matching a pattern, I need one which finds all targets matching a pattern. Then, I could do something like this:
ALL_DEPS += $(addprefix $(DEST_DIR),$(notdir $(targetwildcard $(SRC_DIR)*)))
If it matters any, I've got much of the GNU Make code split across multiple files and included by a "master" Makefile. The ALL_DEPS variable is appended to in any of these files which has something to add to it. This is in an attempt to keep the build process modular as opposed to dropping it all in one monster Makefile.
I'm definitely still learning GNU Make, so it's not unlikely that I'm missing something fairly obvious. If I'm just going about this all wrong, please let me know.
Thanks!
It is simply not possible to do what you're trying to do; you're trying to get make to recognise something that doesn't exist.
This is part of the reason why, in general, wildcards are bad (the other being that you can end up including stuff you didn't mean to). The right thing to do here is to explicitly create a list of source files (ls -1 | sed -e 's/\(.*\)/sources+=\1/' > dir.mk) and perform the patsubst transformation on that list.
If you have additional files that are generate as part of the build, then you can append them to that list and their rules will be found as you'd expect.