I have a Makefile, attached to a project I am semi-maintaining, but did not originally write. This Makefile does, at the moment, have a nasty habit of running "rm *" on a make clean run when certain variables are not defined.
It includes, for example:
rm $(SOMEDIR)/$(SOMEPREFIX)* and rm $(SOMEPREFIX)*
There are several sets of variables like this. Is there a simple way to make sure that we're not ever going to accidentally call something like rm * or rm /*?
I could, of course, check all of the combinations of $SOMEDIR and $SOMEPREFIX individually, but it'd be nice to do something safer that doesn't require each individual variable/command to be checked.
Any ideas that don't involve me rewriting the whole thing?
Very frequently the list of generated files that you want to delete on make clean can be computed from a list of (precious) source files:
ROOTDIR = .
OBJDIR = objdir
$(SRCS) = $(wildcard *.c)
$(OBJS) = $(patsubst %.c,$(ROOTDIR)/$(OBJDIR)/%.o,$(SRCS))
To avoid catastrophic make clean consequences, prefer deleting computed file lists over unpredictable glob patterns:
clean:
rm -f $(OBJS)
is much better than:
clean:
rm -f $(ROOTDIR)/$(OBJDIR)/*
#Renaud Pacalet's is a better solution. If for some reason it doesn't fit your situation, you can give the variables a value if otherwise unset using ?=.
E.g.
ROOTDIR ?= /tmp/nonexistant
OBJDIR ?= NOPE
Or maybe just put an error check at the top to prompt the user to set the variables appropriately. (An empty string in if triggers the else case)
$(if $(ROOTDIR),,$(error Please set ROOTDIR))
$(if $(OBJDIR),,$(error Please set OBJDIR))
Safest solution I've found so far is to avoid rm -r at all costs.
Instead, use rm specifying the files individually, possibly using globs or via find -exec command. Or at least use rm -r but with literal strings as part of the parameters.
Finally, remove parent directories with rmdir.
So in other words there doesn't seem to be a bullet-proof way considering accidental/malicious changes to the Makefile variables.
As an example, instead of doing this:
clean:
rm -f -r $(VENDOR)
Better do this:
clean:
rm -f -r $(VENDOR)/*.com
rm -f -r $(VENDOR)/*.org
rm -f $(VENDOR)/modules.txt
rmdir $(VENDOR) 2> /dev/null || true
Related
I have a makefile project which generates 2 folders a build one and a deps one. I would like to be able to run make clean as many times as I want and the result would be to only delete the folders and the files in them the first time (idempotent make clean essentially).
Any subsequent time it wouldn't throw any errors, it would just do nothing.
Additionally because I accidentally deleted my home folder once by using rm -rf in a makefile, I would like to avoid using that as well.
I have tried various combinations of using rm -r, rm -f, rmdirs and/or adding a prefix (-) which will only suppress the errors.
Additionally I know I can solve this by using something like the following
if [-d "./build"]; then
rm -r ./build
fi
Do you guys have any other solutions?
Not a fan of clean targets, just tell users to delete the build/ folder (I like your hygiene!).
rm -f is the correct solution. -rf with a folder is fine. If you are feeling paranoid, just protect yourself.
.PHONY: clean
clean:
rm -rf $(or ${build-dir},$(error $$build-dir does not exist))
Here, make will expand the recipe before giving it to the shell. If the expansion of $build-dir is empty, make will expand the $(error …) and will stop, issue the error message and an error, and not even get as far as running the rm.
If you really don't want to use -f with rm, then
-rm -r ${build-dir}
will cause make to ignore any error, but is a bit noisy.
rm -r ${build-dir} || :
will attempt to run the rm. If that succeeds then your task is done and make is happy. If the rm fails, the shell will run its built-in : command which simply returns success, and make is again happy.
find paths/ -delete may be useful, although as written it will fail if paths don't exist. There are many options and it depends on exactly what you want to do.
But rm -f is by far the standard for simple removal.
Remember that you can use || [ $$? -eq 1 ] or similar to suppress specific errors - note the doubled $ for make's own quoting.
Another way of achieving idempotency is to rename things to temporary names and later either rename them into place or delete them, but I'm not sure if that's relevant here.
"because I accidentally deleted my home folder once by
using rm -rf, I'd like to avoid it"
Removing rm -rf doesn't actually make much sense, since it does exactly what you want. It's only your fear that you want to find different solution, but that would be dangerous as well :-)
I'd suggest to use rm -rf but very carefully. I was using it for years and nothing bad happened. My example of use in the Makefile:
lang_dir = lang
domain = messages
clean:
rm -rf $(lang_dir) $(domain).pot tmp_err
How could you compromise that? There's no way. Be careful but do not irrationally fear rm -rf just because it is THE INFAMOUS rm -rf :-)
If you are super unsure, you might require a manual confirmation for recursive removal by adding option -I:
rm -rfI target1 target2 ...
From the rm(1) manual page:
-I prompt once before removing more than three files, or when removing recursively.
Less intrusive than -i, while still giving protection against most mistakes
Whatever solution you use, rm -rf, if [ -d ... ]; then rm -r ...; fi, find ... -delete etc. - you'll always have the same fundamental problem: if the thing you're trying to delete contains a variable dereference, and the variable is accidentally empty, you will delete the wrong directory.
So there's rm -I (which might not work on a build server), and also rm --preserve-root, and also safe-rm.
If you're paranoid, you could have a directory called e.g. cleanups which contains links to the directories you want to clean, and have your makefile have:
clean:
rm -rf cleanups/*/*
rmdir $(readlink cleanups/*)
In the rm call above, the first star finds the links, second deletes everything in these directories, NOT including files starting with . (i.e. hidden files).
The rmdir line removes these empty directories, if they are indeed empty, assuming there are no spaces in the link names.
I have a build system setup for a library using the GNU autotools. The library has subdirectories for source and header files. It also contains a script in the top directory which auto-generates a source and header file (not the same as the config file) in the appropriate subdirectories. It is necessary that these files are generated before make is performed in the subdirectories.
What is the simplest way to have the script run before subdirectories are traversed (i.e. when user calls make, the script is ran before traversing SUBDIRS)? I have tried adding rules like all-local with no success.
EDIT:
Top-level directory:
configure.ac Makefile.am src include myscript.sh
Makefile.am:
EXTRA_DIST = myscript.sh
ACLOCAL_AMFLAGS = ${ACLOCAL_FLAGS} -I m4
SUBDIRS = src include
.PHONY: gen-script
gen-script:
./myscript.sh
src/Makefile.am:
AM_CPPFLAGS = -I$(top_srcdir)/include
lib_LTLIBRARIES = libmylib.la
libmylib_la_SOURCES = \
file1.cxx \
file2.cxx \
autogen-file.cxx
clean-local:
rm -f autogen-file.cxx
include/Makefile.am:
nobase_include_HEADERS = \
file1.h \
file2.h \
autogen-file.h
clean-local:
rm -f autogen-file.h
I think that the best solution would be to get rid of the recursive make and only have the top-level Makefile.am. Then you'd simply add a rule
include/autogen-file.h src/autogen-file.cxx: myscript.sh
${SHELL} $<
and list include/autogen-file.h in BUILT_SOURCES.
If you want to keep your recursive structure (even if this is considered harmful), you could place the rule to generate the files into the Makefile.ams in the sub-directories.
In src/Makefile.am:
autogen-file.cxx: ../myscript.sh
cd ${top_srcdir} && ${SHELL} myscript.sh
And in include/Makefile.am:
autogen-file.h: ../myscript.sh
cd ${top_srcdir} && ${SHELL} myscript.sh
By the way, how often do you need to re-build those? If the generated files only depend on the project configuration, you could simply run myscript.sh at the end of your configure.ac and delete the generated files only on make distclean.
What you can do is force the current directory to run before the rest of the subdirectories with
SUBDIRS = . src include
More seriously, though, you should not use recursive automake. Particularly if your structure is relatively simple as it seems to be. If you have interests in looking up how to do that you can see my documentation on Autotools Mythbuser.
ifeq ($(SYSTEM),WINDOWS)
# need to change something here :
RM = #(if exist "$(subst /,\,$(DIRECTORY))" rmdir /s /q $(subst/,\,$(DIRECTORY))))
else
RM := #rm -rf
endif
clean_1 :
$(RM) SOME/PATH/
clean_2 :
$(RM) SOME/PATH/
How to make a substitution path "SOME/PATH/" to "DIRECTORY"?
Is this possible without changing the structure of the code?
P.S. I can not use rmdir without checking for the file, as it generates an error.
UPDATE. I did so:
ifeq ($(SYSTEM),WINDOWS)
RM = #(if exist "$(subst /,\,$1)" rmdir /s /q $(subst/,\,$1)))
else
RM = #rm -rf $1
endif
clean_% :
$(call RM, SOME/PATH/)
It really works. Thanks to Mark Galeck.
Is there a way to do this by replacing RM variable, without changing clean-target commands?
make RM a function (consult the manual for how to do that).
You would then have
clean_%:
$(call RM, SOME/PATH/)
Yes you can also do this without changing the target names, just replace $(RM) with the function call, in your original rules.
Actually, I think the best way to handle the situation which you seem to have, where one filesystem is accessed from two OS, would be to introduce a variable for separator
/ := $(if $(filter WINDOWS, $(SYSTEM)),\,/)
Now, you can just always use SOME$/PATH and you don't have to change anything else - only put $ in front of / where you need to make paths available to Windows as well
One of the many makefiles in my project shows errors in the error console when there are no files to delete even though the -f flag is being used. Here's the offending line in the makefile
-rm -f *.o
If I remove the dash at the beginning of the line, it stops dead in its tracks - so I know that this is the line that's generating the error.
This is a big project with a dozen or so makefiles. What I don't understand is that some of the others don't have this problem.
This is the embedded programming world - I'm using WindRiver 2.6 (so the rm utility is "rm.EXE", though it seems to have the usual options).
Does anyone know why an "rm -f" call would still generate an error if there's nothing to delete?
I'm not familiar with the WindRiver tools, but here's an educated guess based on the behavior of typical Unix shells and make tools.
When you run rm -f *.o, the following happens:
the shell tries to expand *.o, leaving it as is if there are no files matching it (try issuing echo *.o in a dir containing no such files, and one with such files)
the result of the expansion is passed to rm, so if there are no matching files, rm is called with the literal string *.o as its argument
rm interprets as arguments as path names, without expanding patterns (because that's the shell's job); the -f shuts it up if a file cannot be found
From the error message, it looks like your rm is broken in that it still complains when no file called *.o can be found. You can work around this by first checking whether any file matches the pattern. I must admit that I don't know how to do that in your environment.
In Windows environment it can't expand "*.o" and sends it as is to rm command, in fact rm.exe from cygwin for example, which then fails with error exit code. It is difficult to see while running rm.exe from Windows's cmd command prompt as it's appearing to complete OK with the same arguments.
The solution is to use $(wildcard *.o) in makefile. It's important to have "*", as otherwise nothing will be expanded even if file exists.
As a result, the line in make file will look similar to this:
rm -f $(wildcard *.o) $(wildcard *.elf) $(wildcard *.lst) $(wildcard *.map) $(wildcard *.sym) $(wildcard *.lss) $(wildcard *.eep) $(wildcard *.srec) $(wildcard *.bin) $(wildcard *.hex) $(wildcard *.tmp.sh)
In Windows use del command to remove files in Makefile
clean:
del *.o *.exe
I have an existing project where I am adding gettext support. Unfortunately, due to the project's structure and limitations, I cannot go through the recommended route of changing the project over to automake. Also unfortunately, my make-fu is pretty weak and I'm having troubles implementing rules to build my output archive:
Take all .po files in the msg subdir and run msgfmt on them to produce .mo files (in my target dir)
Put the .po files in the directory structure expected by gettext: (dir)/(locale)/LC_MESSAGES/(domainname).mo
Here's what I have so far
MSGSRC=msg/*.po
MSGOBJ=$(addprefix $(TARGET_BUILD_PATH)/$(target)/,$(MSG_SRC:.po=.mo))
$(TARGET_BUILD_PATH)/$(target)/msg/%.mo: msg/%.po
msgfmt -c $< -o $#
# Put in correct place
mkdir -p $(TARGET_BUILD_PATH)/$(target)/msg/$(*F)/LC_MESSAGES
cp $# $(TARGET_BUILD_PATH)/$(target)/msg/$(*F)/LC_MESSAGES/myapp.mo
archive: $(MSGOBJ) (other objs....)
(make the archive tarball...)
The problem with the existing code is that for some reason $(*F) comes out just * instead of the locale name (the .po files are named en_US.po, etc). It also seems incorrect because the target should be the real target, not the hacky msgfmt and copy I have. The directory structure is important because the makefile is run a few times for different cross-compiles ($(target)) and the output is archived into a tarball for installation on the devices.
I assume you can use GNU make.
First of all, let make expand the wildcards. This is important for later postprocessing:
MSGSRC=$(wildcard msg/*.po)
Now you should get lists of file names in MSGSRC and MSGOBJ. Additionally, the make manual marks $(F) as obsolete, and $ (the stem of the name) should contain just the locale. So,
mkdir -p $(TARGET_BUILD_PATH)/$(target)/msg/$*/LC_MESSAGES
should do the trick just fine, the same for the cp rule.
I do it slightly different. Here are my po files:
$ find msg -type f
msg/bg_BG.po
msg/de_DE.po
Here's the Makefile:
MSGLANGS=$(notdir $(wildcard msg/*po))
MSGOBJS=$(addprefix locale/,$(MSGLANGS:.po=.UTF-8/LC_MESSAGES/appname.mo))
gettext: $(MSGOBJS)
locale/%.UTF-8/LC_MESSAGES/appname.mo: msg/%.po
mkdir -p $(dir $#)
msgfmt -c -o $# msg/$*.po
And these are the resulting mo files:
$ find locale -type f
locale/bg_BG.UTF-8/LC_MESSAGES/appname.mo
locale/de_DE.UTF-8/LC_MESSAGES/appname.mo