Makefile processing files with same extension - makefile

This seems slightly related to How to write Makefile where target and source files have the same extension?. In that question the extensions are the same, but the input and output files seem to be in the same directory and filenames are being conditionally renamed.
I have a large collection of .txt files in ../src/ that need to be processed, and dumped into ./ (which is a directory called target/) as txt files of the same name. I want to use make, so that only files in ../src/ that have been changed get updated in ./. I would like to get the prototype working before I put the real code in.
My Makefile in ./ is as follows:
DIR = ../src
INPUTS = $(wildcard $(DIR)/*.txt)
OUTPUTS = $(patsubst $(DIR)/%.txt,%.txt,$(INPUTS))
all: $(OUTPUTS)
.PHONY: $(INPUTS)
check:
#echo "DIR = $(DIR)"
#echo "INPUTS = $(INPUTS)"
#echo "OUTPUTS = $(OUTPUTS)"
%.txt: $(DIR)/%.txt
sed -e "s/test/cat/g" "$<" > $#
For now, the contents of ../src/ are test1.txt and test2.txt.
As the Makefile stands now, running make test2.txt generates the file as expected.
target/ $ make test2.txt
sed -e "s/test/cat/g" "../src/test2.txt" > test2.txt
Running make check shows the INPUTS and OUTPUTS correctly.
target/ $ make check
DIR = ../src
INPUTS = ../src/test1.txt ../src/test2.txt
OUTPUTS = test1.txt test2.txt
If I run make all, it generates every file, every time. This is expected with the .PHONY $(INPUTS) line in there.
If I remove the .PHONY $(INPUTS) target, Make gets all bound up in itself trying to find the target to make ../src/test1.txt and keeps prefixing $(DIR) in front of it until it makes too long of a filename and gives up.
make: stat: ../src/../src/../src/ [repeat for a few pages] ../src/../src/test1.txt: File name too long
make: stat: ../src/../src/../src/ [repeat for a few pages] ../src/../src/../src/test1.txt: File name too long
make: *** No rule to make target `../src/../src/../src/[repeat]../src/../src/test1.txt', needed by `../src/[repeat]../src/../src/test1.txt'. Stop.
It never does get to processing test2.txt.
As I was drafting this, I had the idea to remove the ../ from the DIR,
and relocate the Makefile so it was parent to both src/ and target/. That approach seems to work, but isn't ideal. Eventually there would be a chain of these Makefiles, each pulling from one directory to another.
Is there a way to keep the Makefile in 'target/' along with the generated destination files, and base those destination files off of something in a relative path?

Replace
%.txt: $(DIR)/%.txt
with:
${CURDIR}/%.txt: $(DIR)/%.txt
This way %.txt does not match any .txt file in any directory. In other words, you limit this rule's scope to files in ${CURDIR}/ only and this prevents that endless recursion.
See ยง10.5.4 How Patterns Match for more details.
It is also good practice to avoid relative paths:
DIR = $(abspath ../src)

Related

Run script only on modified file using Makefile

I have few txt files in a directory. I want to run a shell script only on the files which have been modified. How can I achieve this through Makefile?
Have written the following part but it builds all the txt files in the directory. Would be great to get some pointers on this.
FILENAME:= $(wildcard dir/txts/*/*.txt)
.PHONY: build-txt
build-txt: $(FILENAME)
sh build-txts.sh $^
I'm guessing you want something like this:
files := $(wildcard dir/txts/*/*.txt)
dummies := $(addprefix .mod_,$files)
all:$(dummies)
$(dummies): .mod_% : %
sh build-txts.sh $^
touch $#
For any new text file, it will run the script, and create a .mod counterpart. For any non-new text file, it will check if the timestamp is newer than the .mod files timestamp. If it is, it runs the script, and then touches the .mod (making the .mod newer than the text). For any text file that has not been modified since the last make, the .mod file will be newer and the script will not run. Notice that the .mod files are NOT PHONY targets. They are dummy files who exist solely to mark when the text file was last modified. You can stick them in a dummy directory for easy cleaning as well.
If you need something where you don't want to rebuild the text files by default on a fresh checkout, or your script criteria isn't based on timestamps, you would need something a bit more tricky:
files := $(wildcard dir/txts/*/*.txt)
md5s:= $(addprefix .md5_,$files)
all:$(md5s)
.PHONY:$(md5s)
$(md5s):
( [ -e $# ] && md5sum -c $# ) || \
( sh build-txts.sh $# && md5sum $(#:.md5_=) > $# )
Here, you run the rule for all text files regardless, and you use bash to determine if the file is out of date. If the text file does not exist, or the md5sum is not correct, it runs the script, then updates the md5sum. Because the rules are phony, they always run for all the .md5sum files regardless of whether they already exist.
Using this method, you could submit the .md5 files to your repository, and it would only run the script on those files whose md5 sum changed after checkout.

makefile ignore removed file names

A directory with css files in it and for each file in the directory I want a separate minified .min.css file in the same directory.
So for example my.css get minified into my.min.css in the same directory.
First I tried
css/*.css:
uglifycss $# > $(subst .css,.min.css,$#)
make -B yeaaa victorie its working :D
But after doing make -B again I get a new separate file each time resulting in my.min.css my.min.min.css my.min.min.min.css
Then I tried
.PHONY: clean
css/*.css: clean
uglifycss $# > $(subst .css,.min.css,$#)
clean:
-rm css/*.min.css
DOH! after clean it is still remembering the files it deleted in the first place resulting again in a my.min.min.min.css file
How can I tell make to stop doing my.min.min.min.css?
(make: GNU Make 3.81 OSX)
First get a list of all files with the suffix .css:
ALL := $(wildcard *.css)
then remove files that have the suffix .me.css:
NEW := $(filter-out %.me.css,$(ALL))
and add the suffix to remaining files:
ME := $(patsubst %.css,%.me.css,$(NEW))
Then you add those files as prerequisites to the default target, and add you own recipe that builds those files, in this case a simple echo:
%.me.css:
echo 123 > $#
default: $(ME)
Compared to your approach, this has the benefit that you don't have to use the flag -B, as only the files that need to be built are built. Therefore invoking make is done by simply caling make without any targets or flags (assuming the makefile is named makefile or Makefile):
make

Makefile applies a rule recursively even if it shouldn't

I have a very bizzare problem with GNU make. I have the following files:
a/x.html
b/Makefile
b/c/Makefile
The contents of a/x.html are irrelevant. The contents of b/Makefile are as follows:
SRC=../a
all: x.html
%.html: ${SRC}/%.html
rsync $< $#
The contents of b/c/Makefile are the same, except for the definition of SRC:
SRC=../../a
If I run make in b/c/ the result is as expected:
rsync ../../a/x.html x.html
and x.html gets copied from a/ to b/c/.
However, if I run make in b/ the output I get is several lines of:
make: stat: ../a/../a/.. (repeated many times) ../a/x.html: File name too long
It seems that make is applying the rule for %.html recursively, but why? Is there something obvious I am missing?
To build a target that matches the pattern %.html (i.e. any target name that ends in .html), make applies the rule if it can build the dependency (target built from the original target with ../a/ prepended).
You ask to build x.html. This matches the pattern %.html, so the rule applies: make sees if it can build ../a/x.html.
../a/x.html matches the pattern %.html, so the rule applies: make sees if it can build ../a/../a/x.html.
../../a/x.html matches the pattern %.html, so the rule applies, etc.
The stem character can match any part of a path, including directory separators.
You can see what make is trying by running make -r -d (-d to show debugging output, -r to turn off built-in rules which would cause a huge amount of noise).
When you're in b/c, this stops at step 2 because ../../a/x.html exists but ../../../../a/x.html doesn't.
One way to fix this is to list the files on which you want to act. You can build that list from the list of files that already exist in ../a:
$(notdir $(wildcard ${SRC}/*.html)): %.html: ${SRC}/%.html
rsync $< $#
This has the downside that if the HTML files in ../a are themselves built by a rule in b/Makefile, then running make in b won't built them in a pristine source directory. This shouldn't be a problem though: it would be unusual to have a makefile in b build things outside b.
Another approach which doesn't have this defect is to use an absolute path.
%.html: $(abspath ${SRC})/%.html
rsync $< $#

GNU Make - build only out-of-date file in directory

Pretty new to GNU Make. This is a less complex example of something more general I have been trying to get to work.
I have many input files that have similar name format .txt, and I have a shell script that will take the input file and generate an output of the same name but with a different extension .wc. I have written the following Make file.
# name of dependencies
SRC = $(wildcard *.txt)
# get name of targets (substitute .wc for .txt)
TAR = $(SRC:.txt=.wc)
all: $(TAR)
%.wc: %.txt
sh word_count.sh $<
This runs fine, and will generate all the .wc output files. However, if I modify one of the input(dependency) files, they are all rebuilt. So the question is; what is the best way to get GNU Make to only process the modified .txt files in the directory?

Strip prefix of dependency in makefile rule target

I have a makefile which is designed to get some datasets from the web and unzip them into folders (given by the archive names). Unfortunately, the archives on the website are grouped within a folder structure. For example, if I want dataset.tar.gz, which is part of set DIR1, I call wget with: www.example.com/DIR1/dataset.tar.gz.
This is then downloaded to dataset.tar.gz, which I can then extract with tar -xzf. My makefile for this is currently:
URL = www.example.com
FILES = DIR1/something.url DIR2/another.url
FOLDERS = $(notdir $(patsubst %.url, %, $(FILES)) )
%.url: # do nothing
%.tar.gz : %.url
wget -q $(URL)/$<
% : %.tar.gz
tar -xzf $<
all: $(FOLDERS)
Unfortunately, the target for the rule %.tar.gz : %.url does not resolve to (e.g.) something.tar.gz, but to DIR1/something.tar.gz, even though it produces the expected file.
This causes problems when I rerun the makefile, as it does not detect something.tar.gz (as it is looking for DIR1/something.tar.gz), and redownloads the dataset, wasting time and bandwidth.
Is there any way of stripping prefixes/directories from makefile targets, i.e. removing DIR1 from the rule %.tar.gz : %.url, in order that the makefile correctly checks to see if it is satisfied?

Resources