Removing all directories in a path - makefile

I have a makefile which lists several directories with which I need to do stuff with, e.g.
DIRS = dir1 dir2 path/to/dir3
all:
$(foreach DIR,$(DIRS), somecommand --source=$(SOURCEDIR)/$(DIR) --dest=$(DIR);)
clean:
rm -rf $(DIRS)
Currently the clean target removes dir1, dir2, and dir3, but I would like it to remove dir1, dir2 and path. Something along the lines of:
clean:
$(foreach DIR,$(DIRS), rm -rf --parents $(DIR);)
Is there an easy way to do this?

There is no such flag available to rm. You could do something like this:
clean:
$(foreach DIR,$(DIRS),rm -rf $(firstword $(subst /, ,$(DIR)));)
if you're absolutely sure that you always want to delete everything below the first directory in every path in DIRs (sounds dangerous to me, but...)

Related

How to call a bunch of subdirectories Makefile from a top level Makefile with different targets

I have a top level Makefile that define a list of subdirectories containing sub-Makefiles.
Each sub-Makefile define the same targets: all, install, clean, test, verify, style...
How do I make the top level Makefile call all my subdirectories Makefile with the same target ?
It works for the default target, like this:
SUBDIRS = dir1 dir2 dir3
all: subdirs
subdirs: $(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $#
.PHONY: subdirs $(SUBDIRS)
But how do I extend that to the other non default targets ?
If you use GNU make you could use the MAKECMDGOALS variable and the filter function:
SUBDIRS := dir1 dir2 dir3
TARGETS := all install clean test verify style
$(TARGETS): subdirs
#echo making top $#
subdirs: $(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $# $(filter $(TARGETS),$(MAKECMDGOALS))
.PHONY: subdirs $(TARGETS) $(SUBDIRS)
%::
#echo making top $#
Demo:
$ make --no-print-directory all install foobar
make -C dir1 all install
make -C dir2 all install
make -C dir3 all install
making top all
making top install
making top foobar
Note that filter is needed to avoid passing top-only goals to the sub-makes.
If, for any reason, this MAKECMDGOALS based solution does not work for you, here are some other, but a bit more complex, solutions.
Target-specific variables
Target-specific variables propagate to prerequisites. So you can define a pattern-specific variable (SUBGOAL) for each of your top-level goals and use it in the subdir's recipe:
SUBDIRS := dir1 dir2 dir3
all: SUBGOAL := all
all: subdirs
#echo making top $#
subdirs: $(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $# $(SUBGOAL)
.PHONY: subdirs $(SUBDIRS)
If you have many such top-level goals and use GNU make you can factorize a bit with the foreach and eval GNU make functions:
SUBDIRS := dir1 dir2 dir3
TARGETS := all install clean test verify style
$(foreach t,$(TARGETS),$(eval $(t): GOAL := $(t)))
$(TARGETS): subdirs
#echo making top $#
subdirs: $(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $# $(GOAL)
.PHONY: subdirs $(SUBDIRS) $(TARGETS)
One important drawback is that as you use the same phony subdirectory target (dir1) for several top phony targets (all, install...) you cannot invoke make with several top goals:
$ make --no-print-directory all install
make -C dir1 all
make -C dir2 all
make -C dir3 all
making top all
making top install
will only make all in the subdirectories. If this is a problem you can use the next solution.
One phony target (all-dir1) per top-subdirectory combination
Let's do this programmatically with a 2-variables macro and a foreach-eval-call combination (GNU make only):
SUBDIRS := dir1 dir2 dir3
TARGETS := all install clean test verify style
$(TARGETS): subdirs
#echo making top $#
TARGETS_X_SUBDIRS :=
# $(1): top target
# $(2): subdirectory target
define TARGETS_X_SUBDIRS_RULES
$(1): $(1)-$(2)
$(1)-$(2):
$(MAKE) -C $(2) $(1)
TARGETS_X_SUBDIRS += $(1)-$(2)
endef
$(foreach t,$(TARGETS),$(foreach s,$(SUBDIRS),$(eval $(call TARGETS_X_SUBDIRS_RULES,$(t),$(s)))))
.PHONY: $(TARGETS) $(TARGETS_X_SUBDIRS)
Demo:
$ make --no-print-directory all install
make -C dir1 all
make -C dir2 all
make -C dir3 all
making top all
make -C dir1 install
make -C dir2 install
make -C dir3 install
making top install
Renaud's answer is more complete, but I also found this technique that worked fine for my use case:
all:
%:
make -C dir1 $#
make -C dir2 $#
make -C dir3 $#
In this case, I don't have the subdirs list in a variable, but they are spelled out in the commands for the generic target %:. It requires to add an extra make -C in front of every subdirectory, and a # in the back, but otherwise pretty maintainable and easy to read.
But it probably would not work if there was any other targets specific to the top-level Makefile.

Makefile to generate JSON from Python scripts in separate directories

I have a Makefile which generates JSON from several different Python scripts (the scripts print to stdout) in a single directory, e.g.
/src
scriptOne.py
scriptTwo.py
scriptThree.py
Which outputs the JSON to a folder:
/templates
scriptOne.json
scriptTwo.json
scriptThree.json
I'm trying to restructure so that, for example, each script is in its own subdirectory and the Makefile creates the JSON templates in their consequent subdirectories as follows:
/src
/importantTemplates
scriptOne.py
/notSoImportantTemplates
scriptTwo.py
scriptThree.py
And the output:
/templates
/importantTemplates
scriptOne.json
/notSoImportantTemplates
scriptTwo.json
scriptThree.json
The current Makefile is as follows:
SOURCES := $(shell echo src/*.py)
TARGETS := $(patsubst src/%.py,templates/%.json,$(SOURCES))
all: $(TARGETS)
clean:
rm -f $(TARGETS)
templates/%.json: src/%.py
python2 $< > $#
I've tried changing the wildcards to include a subdirectory for each line e.g. /src/*/*.py, although I just end up with the following:
make: Nothing to be done for `all'.
You want a static pattern rule (4.12 Static Pattern Rules) for this.
SOURCES := $(wildcard src/*/*.py)
TARGETS := $(patsubst src/%.py,templates/%.json,$(SOURCES))
all: $(TARGETS)
clean:
rm -rf templates
$(TARGETS) : templates/%.json: src/%.py
mkdir -p $(#D)
python2 $< > $#
You could avoid needing mkdir -p in that rule body if you wanted to (and go with an order-only prerequisite on the directory instead) but I'm not sure the effort is worth the savings in execution cost. You could avoid the extra shell by combining the two lines mkdir -p $(#D) && python2 $< > $# if you wanted to though.
Please note that the second time you run the make, it will give you the message (if there are no new files):
make: Nothing to be done for `all'.
Try to run make clean and see if you get the same message.
Here is the Makefile which will do what you want:
SOURCES := $(wildcard src/*/*.py)
TARGETS := $(patsubst src/%.py,templates/%.json,$(SOURCES))
FOLDERS := $(sort $(dir $(TARGETS)))
all: $(TARGETS)
clean:
rm -rf $(TARGETS) $(FOLDERS)
$(FOLDERS):
mkdir -p $#
$(TARGETS): $(SOURCES) $(FOLDERS)
python2 $< > $#
The FOLDERS variable will contain the folders you need to create in the template directory. (sort will remove duplicates, so each folder will be there only once)
The $(FOLDERS) rule will create the folders.
The clean rule will remove the folders also.
If you need to add more sources, just do it like this:
SOURCES := $(wildcard src/*/*.py)
SOURCES += $(wildcard src/*.py)
...

How to avoid code repetition calling recursive Makefiles?

Is it possible to simplify a Makefile of the following form in order to avoid code repetition:
dir1/foo:
make -C dir1 foo
dir1/bar:
make -C dir1 bar
dir2/baz:
make -C dir2 baz
...
clean:
make -C dir1 clean
make -C dir2 clean
...
I imagine I could specify only:
MY_TARGETS=dir1/foo dir1/bar dir2/baz ...
And than have some general rules to derive targets, as presented in the Makefile above.
You haven't given us much information. Recursive Make can be a good solution, if your main makefile is as simple as your example (which I doubt).
You could do this:
%:
$(MAKE) -C $(dir $#) $(notdir $#)
clean:
$(MAKE) -C dir1 clean
$(MAKE) -C dir2 clean
...
If that clean recipe is too long, you can simplify it:
DIRS := dir1 dir2 ...
clean:
#for x in $(DIRS); do $(MAKE) -C $$x clean; done
Or if you don't like having that DIRS list there, you could have Make generate it, depending on whether you want to make clean in all subdirectories, or all that match the dir* pattern, or... you haven't given us enough information.

rename target files in GNU Makefile

the makefile below processes files matching the patterncontent/%.md and outputs the targets in the html directory. Source files are named with the convention of putting a leading number in front of them, like content/01.index.md or content/O2.second-page.md and so on. I would like to remove the leading 0x. number sequence in the target file. For instance, content/01.index.html would generate html/index.html.
How can I do this?
Thanks
MD_FILES = $(shell find content/ -type f -name '*.md')
HTML_FILES = $(patsubst content/%.md, html/%.html, $(MD_FILES))
all: $(HTML_FILES) html/static
html/%.html : content/%.md
mkdir -p $(#D)
python generator/generate.py $< $#
.PHONY: html/static
html/static :
rsync -rupE generator/static html/
.PHONY: clean
clean:
rm -fr html
Replace:
html/%.html : content/%.md
mkdir -p $(#D)
python generator/generate.py $< $#
with:
html/%.html : content/%.md
mkdir -p $(#D)
file='$(#F)'; python generator/generate.py $< "$(#D)/${file#*.}"
Unfortunately, I can't think of a good way of doing that in make itself. I can think of one way but it isn't as simple as that escaping and it isn't safe for files with spaces (not that that matters much here since make already can't handle those).
IMHO, it is a bad idea to use find or wildcards to list files in makefiles. This is because developers have temporary or debugging files sometimes. It is best to list files explicitly. This way, it forces the developer to think about their intent.
If you agree to list files explicitly, then in this case it is best to list the target files, rather than source files, and here is your answer:
HTML_FILES := html/index.html html/second-page.html
.SECONDEXPANSION:
$(HTML_FILES): html/%.html : $$(wildcard content/*.$$*.md)
(put recipe here, using $# and $<)

Target multiple directories

I'm trying to create a target which matches multiple directories, but no files. Is there any way to flag the target explicitly as being a directory?
For example:
output/%:
mkdir -p $#
matches files as well, which is definitely not something I want.
Basically, I have many variables which contain directory names which need to be created if they don't exist, and the best way to do that, as far as I've found is this:
all: directories other_stuff
directories: $(dir1) $(dir2) ...
$(dir1):
mkdir -p $#
$(dir2):
mkdir -p $#
...
So, I'm OK with repeating the directory variables as dependencies for the directories target, but creating targets for each directory I add seems overkill and hard to maintain.
Here is one possibility:
DIR_LIST = $(dir1) $(dir2) $(dir3)
.PHONY: all
all: directories other_stuff
.PHONY: directories # Note: you should be marking this as phony
directories: $(DIR_LIST)
$(DIR_LIST):
mkdir -p $#
See http://www.gnu.org/software/make/manual/make.html#Multiple-Targets.

Resources