I want to have a Makefile that is able to install files one by one, and also uninstall them one by one.
The install rule for a file should run only if the source file is newer than the dest file.
The uninstall rule for a file should run only if the file exists in the system.
Now I have:
SRC_FILES = ...
sys_files = ...
$(sys_files): $(DESTDIR)$(systemdir)/%: $(SRCDIR)/%
#echo ' INSTALL $#';
$(INSTALL_DATA) -T '$(SRCDIR)/$*' '$#';
uninstall_sys_files = $(addprefix uninstall-,$(sys_files))
$(unsinstall_sys_files): uninstall-%:
#echo ' RM $*';
$(RM) '$*';
The problem with the code above is that it runs the uninstall (RM) targets even if the files don't exist, (I can live with it, but I would prefer to not print RM messages for files that are not removed. And yes, I can do that with a conditional, but I'd do that only if make can't handle this with a ?"static terminal pattern rule"?).
So, is there anything like?:
$(unsinstall_sys_files):: uninstall-%: %
#echo ' RM $*';
$(RM) '$*';
which would make this rule terminal and depending on the existance of %.
Per my tests, this behaves as if the double colon was a single colon.
EDIT:
Although this would seem nice, it isn't, because then if a rule fails (because the file didn't exist), then the whole make fails, or you need to ignore errors in the uninstall part, which I'm not entirely happy with.
I'm not really sure advantage you hope to gain by marking a rule terminal... unless I'm missing something it doesn't seem to have much to do with what you want to do.
Your final problem ("a rule fails because the file didn't exist"), by which I understand you to mean the uninstall recipe fails, is simple enough to fix: just set RM = rm -f so that rm won't fail if the file doesn't exist.
It would be great if you could give an example of what you really want to do; how you expect to invoke make and how you want it to behave (resulting operations).
Make is designed to update files, not delete files. That is, its whole goal is to find files that don't exist or are out of date, and make them exist / update them.
However, you could do something like this:
RM := rm -f
sys_exists := $(wildcard $(addprefix $(DESTDIR)$(systemdir)/,$(sys_files)))
uninstall: $(patsubst $(DESTDIR)$(systemdir)/%,uninstall-%,$(sys_exists)))
uninstall-%:
#echo ' RM $*'
$(RM) '$(DESTDIR)$(systemdir)/$*'
This way uninstall would only remove files that actually existed.
Related
With GNU Make, I want to take a list of files and create symbolic links of them in another directory (with the same filenames), but also overwrite any existing link with the same filename.
existing_files = $(wildcard dir1/dir2/*.txt)
# The next line shows where I would want to put the symbolic links
symlinks = $(wildcard new_dir1/new_dir2/*.txt)
make_some_links:
# Remove previous symlinks if they share name as existing_files
ifeq ($(notdir $(existing_files)), $(notdir $(symlinks)))
$(info Removing previous symlinks)
#rm $(symlinks)
endif
# Loop to link here
As an aside, does the above condition attempt to match each element of each list, or does it succeed if only 1 of the elements match?
I can't quite figure out how to loop through in Make, including with $(foreach), so I wrote the following loop to show what I mean in something like Python with the same variable names.
for i in len($(existing_files)):
#ln -s $(existing_files)[i] $(symlinks)[i]
Here, the first element in $(list_files) is linked to the first element in $(symlinks). Any insight on how to write this loop in Make, or if there is straight-up a better way to approach this would be very helpful. Thanks.
Let's take this in stages.
First, to create a symbolic link, and remove a preexisting link of that name, if there is one:
ln -fs filename linkname
Now to make a list of the existing files:
existing_files = $(wildcard dir1/dir2/*.txt)
So far, so good. Let's suppose this gives us dir1/dir2/red.txt dir1/dir2/green.txt.
# The next line shows where I would want to put the symbolic links
symlinks = $(wildcard new_dir1/new_dir2/*.txt)
That will give you a list of the things that already exist in that directory, which is probably not what you intend. We must construct the list of links we want from the list of files we have:
filenames := $(notdir $(existing_files))
symlinks := $(addprefix new_dir1/new_dir2/, $(filenames))
Now for a rule or rules to build the symlinks. We could write two explicit rules:
new_dir1/new_dir2/red.txt: dir1/dir2/red.txt
ln -fs dir1/dir2/red.txt new_dir1/new_dir2
new_dir1/new_dir2/green.txt: dir1/dir2/green.txt
ln -fs dir1/dir2/green.txt new_dir1/new_dir2
but that is horribly redundant, and besides we don't know the file names beforehand. First we can remove some of the redundancy by defining a variable and using the automatic variale $<:
DEST_DIR := new_dir1/new_dir2
$(DEST_DIR)/red.txt: dir1/dir2/red.txt
ln -fs $< $(DEST_DIR)
$(DEST_DIR)/green.txt: dir1/dir2/green.txt
ln -fs $< $(DEST_DIR)
Now we can see how to replace these rules with a pattern rule:
$(DEST_DIR)/%.txt: dir1/dir2/%.txt
ln -fs $< $(DEST_DIR)
Now all we need is a master rule that requires the links:
.PHONY: make_some_links
make_some_links: $(symlinks)
In most cases, when you start designing shell loops in make recipes, you are missing important features of make. Make will naturally "loop" over the targets you declare to reach the goals you ask it to reach. And make offers many ways to factorize similar rules.
But in your case where the targets are symbolic links there are several issues to consider:
If you declare the links as regular targets and if some exist already but point to other files than the ones you want, make could consider them as up-to-date and skip them, instead of replacing them. You must thus find a way to force make to re-create all the links or, at least, those that do not point to the correct file. And the last modification times are not sufficient for this.
Even with the -f option ln will not replace an exiting directory (or a link to a directory), instead it will create the link inside the directory. You must thus first delete any directory (or link to a directory) that conflicts with a target link.
The ln command shall be used with the -sr options to create symbolic links relative to link location.
The target directory (new_dir1/new_dir2) shall exist before you try to create the first link.
If you don't care re-creating links that are already correct, things are easy:
declare a phony dummy target (force) and make it a prerequisite of all your links to force make re-creating them,
before creating your links delete existing links, files or directories that conflict with them,
use the proper ln options,
declare the target directory as an order-only prerequisite of the links and add a rule to create it if it is missing.
The following should do the job:
source_dir := dir1/dir2
link_dir := new_dir1/new_dir2
existing_files := $(wildcard $(source_dir)/*.txt)
symlinks := $(patsubst $(source_dir)/%,$(link_dir)/%,$(existing_files))
.PHONY: all force
all: $(symlinks)
$(link_dir)/%: $(source_dir)/% force | $(link_dir)
rm -rf $#
ln -sr $< $#
force:;
$(link_dir):
mkdir -p $#
If you want to avoid re-creating the existing correct links, things are more difficult because make needs to know which ones are correct and which ones are not, and this cannot be based on last modification times... But as I suspect that the performance impact of re-creating the existing correct links is negligible I suggest that we don't design a complex solution to a non-existing problem.
It looks to me like Makefile rules can be roughly classified into "positive" and "negative" ones: "positive" rules create missing or update outdated files, while "negative" ones remove files.
Writing prerequisites for "positive" rules is quite easy: if the target and the prerequisite are file names, make by default runs the recipe if the target is missing or outdated (a missing file in this context may be viewed as an infinitely old file).
However, consider a "negative" rule, for example for target clean. A usual way to write it seems to be something like this:
clean:
rm -f *.log *.synctex.gz *.aux *.out *.toc
This is clearly not the best way to do:
rm is executed even when there is nothing to do,
its error messages and exit status need to be suppressed with -f flag, which has other (possibly undesirable) effects, and
the fact that there were nothing to do for target clean is not reported to the user, unlike what is normal for "positive" targets.
My question is: how to write a Makefile rule that shall be processed by make only if certain files are present? (Like what would be useful for make clean.)
how to write a Makefile rule that shall be processed by make only if certain files are present? (Like what would be useful for make clean.)
You can do it like so:
filenames := a b c
files := $(strip $(foreach f,$(filenames),$(wildcard $(f))))
all: $(filenames)
$(filenames):
touch $#
clean:
ifneq ($(files),)
rm -f $(files)
endif
Example session:
$ make
touch a
touch b
touch c
$ make clean
rm -f a b c
$ make clean
make: Nothing to be done for 'clean'.
Useful perhaps for some purposes, but it strikes me as a strained refinement for make clean.
This can be easily remedied:
clean:
for file in *.log *.synctex.gz *.aux *.out *.toc; do \
if [ -e "$file" ]; then \
rm "$$file" || exit 1; \
else \
printf 'No such file: %s\n' "$file" \
fi \
done
The if statement is necessary unless your shell supports and has enabled nullglob or something similar.
If your printf supports %q you should use that instead of %s to avoid possible corruptions of your terminal when printing weird filenames.
A meta-answer is: are you sure you want to do this?
The other answers suggest to me that the cure is worse than the disease, since one involves an extension to POSIX make (ifneq), and the other uses a compound command which spreads over seven lines. Both of these are sometimes necessary expedients – I'm not criticising either answer – but both are things I avoid in a Makefile if I can. If I found myself wanting to do this in a clean rule, perhaps for the reason you mention in your comment to #MikeKinghans' answer, I'd try quite hard to change the rest of the Makefile to avoid needing this.
Reflecting on your three original points in turn:
rm is executed even when there is nothing to do: so what? The alternatives still need to, for example, expand the *.log *.synctex.gz ... so there's only miniscule efficiency gain to avoiding the rm. Make is a high-level tool which generally does not concern itself with efficiency.
its error messages and exit status need to be suppressed with -f flag: the -f flag doesn't generally suppress errors and the exit status, it merely indicates to rm that a non-existing or non-permissioned file is not to be regarded as an error.
the fact that there were nothing to do for target clean is not reported to the user: should the user really care?
The last point is the most interesting. People asking about make, on Stackoverflow and elsewhere, sometimes make things hard for themselves by trying to use it as a procedural language – make is not Python, or Fortran. Instead, it's a goal programming language (if we want to get fancy about it): you write snippets of rules to achieve sub-goals, so that the user (you, later) doesn't have to care about the details or the directory's current state, but can simply indicate a goal, and the program does whatever's necessary to get there. So whether there's is or isn't anything to do, the user ‘shouldn't’ care.
I think the short version of this answer is: it's idiomatic to keep make rules as simple (and thus as readable and robust) as possible, even at the expense of a little crudity or repetition.
I'm trying to write a Makefile that automatically calls BibTeX on files that match a specific wildcard but don't exist when I first run Make. Specifically, I have the following:
.FORCE:
all: pdf
foo=something
lat: *.tex
pdflatex $(foo).tex
pdf: lat
open $(foo).pdf &
%.aux: .FORCE
bibtex $#
bib: lat $(foo)?.aux
pdflatex $(foo).tex
pdflatex $(foo).tex
open $(foo).pdf &
What I want to happen is that the following will occur when I run make bib:
pdflatex will be called on $(foo).tex, generating files $(foo)1.aux, $(foo)2.aux, etc.
bibtex will be called on $(foo)1.aux, then $(foo)2.aux, etc.
pdflatex will be called twice on $(foo).tex
open will be called on $(foo).pdf
However, this doesn't happen: in particular, it looks as if Make evaluates the prerequisites $(foo)?.aux up-front, at a point where the files $(foo)1.aux, $(foo)2.aux, etc. don't exist. As a result, BibTeX is never called on them. If I rerun make bib, however, things work, because the files now exist after being created on the previous run.
Question: Is forcing Make to re-evaluate prerequisites for a target the right way to fix this? If so, how can I get it to re-evaluate the prerequisites for bib after running pdflatex as part of lat? If not, how can I achieve what I want please?
What I do in my Maiefile for LaTeX files is rename the targets.
That way, you can have different target names, depending on which phase has been used to create them. This is according to the spirit of make's pattern rules, which assume that files with different contents also have different extensions. So I have rules like this:
%.aux1 : %.tex
rm -f $*.aux
pdflatex -draftmode $*
mv -f $*.aux $#
%.bbl : %.aux1
cp -pf $< $*.aux
bibtex $* || : > $#
%.aux2 : %.bbl
cp -pf $*.aux1 $*.aux
pdflatex -draftmode $*
mv -f $*.aux $#
%-tex.pdf: %.aux2
cp -pf $< $*.aux
pdflatex -jobname $*-tex $*
You can't do this in a completely straightforward way, since make fundamentally assumes that one run through a target's commands will update the target. That is, there's no way in principle that you can tell make that ‘you need to run these commands twice’.
You can try to get round this with (admirably clever) tricks such as #reinerpost suggests, but a problem with that general approach is that sometimes/often a single run of BibTeX (or makeindex, or whatever) is sufficient.
After having tried various types of tricks in the past, what I generally do here is to make a command list which explicitly includes two BibTeX calls where necessary:
%.bbl: %.aux
bibtex $(#:.bbl=)
if grep -q Rerun $(#:.bbl=.log) >/dev/null; then \
bibtex $(#:.bbl=); \
fi
That command list re-runs BibTeX if the log file includes the ‘Label(s) may have changed. Rerun to get cross-references right’ message.
To be honest, what I actually do is just the single line bibtex $(#:.bbl=). When I'm writing a document, I inevitably re-run make so many times that the list of references comes out correct very quickly. This means that this target doesn't work for the ‘recreate the final version from a clean directory’ case, but that's sufficiently rare that I tend not to obsess about it.
Whenever I catch myself re-solving this problem, I now recognise that I'm trying to push water up-hill because I'm bored writing this document, so I go and do something else.
I just wanted to share an alternative solution: Using submake processes:
If so, how can I get it to re-evaluate the prerequisites for bib after running pdflatex as part of lat?
You can somewhat achieve that, by adding make lat to the recipe for bib. This will start a new make process targetting at bib. The sub-make doesn't know anything aboutits parents targets/prerequisites. (Such a concept is usually used, when some huge project is built from different smaller projekts each of which have different makefiles.)
This can be done in multiple layers (although it will be confusing):
bib: $(foo)?.aux lat check_for_bib
check_for_bib:
if grep -q Rerun $(#:.bbl=.log) >/dev/null; then make bib; fi
pdf: lat check_for_bib
open $(foo).pdf &
Note that I had to change some orders of prerequisites. The pseud-code would be something like:
latex compilation
while log suggests update:
update aux
latex compilation
Each iteration of the while loop will take place in a separate make process.
I have a list of libraries where each have 2 files (.so, .dll).
How should I create a make rule which would execute the recipe only once if both of the files are missing or if one of them is missing.
LIBS = alib blib
LIBS_SO = $(patsubst %, %.so, $(LIBS))
LIBS_DLL = $(patsubst %, %.dll, $(LIBS))
If I make this target
$(LIBS_SO) $(LIBS_DSS):
cp .....
it copies only once for all of the the possibilities.
If I make this:
all : $(LIBS_SO) $(LIBS_DSS):
$(LIBS_SO) $(LIBS_DSS):
cp .....
I copy in all cases of any missing files.
I want to copy the alib directory if both or one of the files alib.dll / alib.so is missing; the
the same with blib.
You have a consistent typo of LIBS_DSS where you (presumably) meant LIBS_DLL.
Your first 'rule' is a shorthand for:
alib.so:
cp ...
blib.so:
cp ...
alib.dll:
cp ...
blib.dll:
cp ...
So, when asked to build, make builds the first target in the file, which is alib.so. That's why it does it once.
The second version, when fixed to remove the extra colon and the typo, should work:
all: $(LIBS_SO) $(LIBS_DLL)
$(LIBS_SO) $(LIBS_DLL):
cp .....
The default rule is all; to make all, make ensures that each of the files alib.so, blib.so, alib.dll and blib.dll exists and is up to date. It should execute the commands once for each missing target.
You might conceivably run into trouble if you run a parallel make; make -j4 or something similar. It might launch four copy commands almost simultaneously to make each of the targets. But in a non-parallel build, it will ensure alib.so is up to date (and if it isn't, will do the copy). If that copy also copies alib.dll, then it won't recopy when it ensures alib.dll is up to date.
You haven't given us much information, but I think this will do what you want:
all : $(LIBS_SO) $(LIBS_DLL):
%.so %.dll:
cp $* directory ...
If both alib.so and alib.dll are missing, Make will execute this rule only once.
EDIT: Thanks to Jonathan Leffler for catching the typo.
At work we use a common makefile that other makefiles include (via the include statement) and it has a generic "clean" target that kills some common files. I want to add on to that target in my new makefile so I can delete some specific files, but if I add a clean target in my makefile, it just overrides the old one.
I know I can just make a new target with a new name and have it call clean, and then do other stuff, but for sake of consistency I'd like to be able to just call make clean and have it do everything.
Is that possible?
I've seen this done at several shops. The most common approach is to use double-colon rules, assuming you're using something like GNU make. In your common makefile you would have something like this:
clean::
# standard cleanup, like remove all .o's:
rm -f *.o
Note that there are two colons following clean, not just one!
In your other makefile you just declare clean again, as a double-colon rule:
clean::
# custom cleanup, like remove my special generated files:
rm -f *.h.gen
When you invoke make clean, GNU make will automagically run both of these "branches" of the clean rule:
% make clean
rm -f *.o
rm -f *.h.gen
It's simple to set up and it composes quite neatly I think. Note that specifically because it is a double-colon rule, you don't get the "overriding commands" errors you normally get when you define two rules for the same target. That's sort of the point of double-colon rules.
You can write your own clean and make it a preq of the common clean.
clean: myclean
myclean:
rm whatever
Yours will run first. If for some reason you want the common clean to run first then the solution will be more complicated.
EDIT:
Here is the best solution I can see which runs the common rule before the local one:
include Makefile.common
clean:
$(MAKE) -f Makefile.common $#
rm whatever additional things
The include directive is necessary because the local makefile relies on the common one for things other than clean. The local clean rule overrides the common clean rule, but invokes the common clean rule before doing the additional work. (This overriding will cause some warnings, which is a nuisance; I don't know a good way to silence them.)
Use implicit rules:
existing-target: my-extention
my-extention:
echo running command 1
echo running command 2
Very simple make tutorial to ramp up.
When using :: you can run into issues since make complains when you mix single colon : and double colon :: rules:
a:
echo a
a::
echo aa
will result in:
. . .
*** target file `a' has both : and :: entries. Stop.
It seems like the common makefile's rule should be called something like common-clean. Then each main makefile would declare their clean rule as
clean: common-clean
and you're set.
If that isn't an option, you could take a look at double colon rules, but those introduce a whole other set of issues to consider.
Adding another possible solution I've seen for posterity... I know the OP was wary about changing the common makefile, but something like this works and involves minimal changes.
local makefile 1:
CLEAN=MyExe1 MyExe2
....
include /my/common/makefile
local makefile 2:
CLEAN=MyExe3 MyExe4
....
include /my/common/makefile
common makefile:
clean:
rm -f *.dep *.o *.a $(CLEAN)
Basically the idea is to define some variable (in this case CLEAN) in each local makefile with all the specific items you want to delete. Then the common makefile runs rm -f on all the common file types to delete, plus whatever was specifically flagged for deletion in each local makefile via the CLEAN variable. If there's nothing specific to delete, simply omit the variable declaration or leave it empty (CLEAN=)
So now if we run make clean for local makefile 1, it executes
rm -f *.dep *.o *.a MyExe1 MyExe2
And if we run make clean for local makefile 2, it executes
rm -f *.dep *.o *.a MyExe3 MyExe4
I've found a better solution:
.PHONY: my-extra-clean
clean: my-extra-clean
my-extra-clean:
rm <whatever-you-want>
include Makefile.common
The key line is clean: my-extra-clean. Ie, you can add dependencies in separate stanzas in different makefiles to add behaviour. my-extra-clean is run as a dependency of the root clean target.
For ours, we define a variable, EXTRAFILESTOCLEAN, then when the clean rule runs, it has a step to remove anything specified in the EXTRAFILESTOCLEAN variable
clean:
rm -f *.o
ifdef $(EXTRAFILESTOCLEAN)
rm -f $(EXTRAFILESTOCLEAN)
endif
That can cause unexpected problems if you set that variable to weird values, but you could guard against those by adding prefixes or other tests.
It's in the docs: https://www.gnu.org/software/make/manual/html_node/Overriding-Makefiles.html
So instead of include Makefile you use a wildcard target and forward it to the base Makefile:
# -include base.Makefile <--- not this
%:
#$(MAKE) -f base.Makefile $#