I have several directories which may have subdirectories, and I'm going to compress each one to a file if any file in that directory is changed.
For example, I have 2 directories dir1 dir2, and I want to compress them to comp_dir1.tar.gz and comp_dir2.tar.gz.
I wrote the following code:
comp_%.tar.gz : %/$(shell find % -name "*")
tar -czvf $# $<
But I got the error:
find: ‘%’: No such file or directory
It is obvious that I can't use "%" in the shell command.
Is there any way to solve this?
What you need in any case is a list of the directories you want to compress. Using that, you can use a GNU Make feature Remaking Makefiles to generate "dependency files" which include all files found in these directories. Here is a working example:
DIRECTORIES:=dir1 dir2
ARCHIVES:=$(addsuffix .tar.gz,$(addprefix comp_,$(DIRECTORIES)))
all: $(ARCHIVES)
comp_%.tar.gz.d: %
echo $(#:.d=) $(#): $(shell find $< -name "*") > $(#)
# include the dependency files, this will cause GNU make to attempt creating
# them using the above pattern rule.
-include $(addsuffix .d,$(ARCHIVES))
comp_%.tar.gz: %
tar czvf $# $<
.PHONY: all
Related
Hi!
I started messing with makefiles a few days ago, so I gave myself a exercise to learn as well as making my life easier.
I basically want to read the directory the makefile is in (root) and use luamin to compress the file as much as possible before I deploy it to our server. But I would like to have it as flexible as possible, so depending on where said file is in the directory it should mirror it to the server.
So if it finds a file in a sub folder called home it should create a new folder with the same name with the compressed file within. I have gotten the compression of files in the root folder working as well as creation of the directories where the files should reside.
objs = $(wildcard *.lua)
dirs = $(wildcard */)
compress: $(objs)
mkdir -p .build
luamin -f $(objs) > .build/$(objs)
mkdir .build/$(dirs)
clean:
rm -rf ./.build
deploy: .build
cp ./.build/* ~
If you use GNU make, there are several features that really help to do what you want. Warning: this works if and only if your file names do not contain spaces:
srcfiles := $(shell find . -path .build -prune -o -type f -name '*.lua' -print)
dstfiles := $(addprefix .build/,$(srcfiles))
.PHONY: compress clean deploy
compress: $(dstfiles)
$(dstfiles): .build/%: %
mkdir -p $(dir $#)
luamin -f $< > $#
clean:
rm -rf ./.build
deploy: .build
cp ./.build/* ~
Explanation:
The shell make function is used to run the find command that searches all subdirectories, except .build, for *.lua files. The result is assigned to the srcfiles make variable.
The addprefix make function is used to add the .build/ prefix to all words of the srcfiles make variable and assign the result to the dstfiles make variable.
The compress target is the first (real) target in the Makefile. It is thus the default goal that gets run when invoking just make. It is the same as invoking make compress. The compress target is declared as phony. This tells make that it is not a real file, just like clean and deploy. The compress target depends on all destination files. If one is missing or older than its corresponding source file, it must be rebuilt.
The make static pattern rule $(dstfiles): .build/%: %... declares a generic rule where each destination file (.build/./foo/bar/baz.lua) depends on the corresponding source file (./foo/bar/baz.lua). The recipe creates the destination directory (./foo/bar/), computed thanks to the dir make function. Then, it applies the luamin command. The recipe makes use of the $# and $< automatic variables.
I have this command from my bash script
find . -type f -name "*.c" -execdir bash -c "f={}; kos-cc $KOS_CFLAGS -c {} -o $PWD/\${f%.c}.o" \;
Its job it to recursively search the current directory ($PWD) for .c files, compile them with my "kos-cc" compiler then output all the .o files into the current working directory.
I want to move parts of my bash script into a makefile and this line is the last one that stumps me. I know how to make rules that compiles c files from directory A and outputs the .o files into directory B, but here directory A isn't always the same (Since I need to handle sub directories too). How do I do the equivalent task in a Makefile either with a rule or a command?
It is not trivial because you have your sources in different directories and simple make pattern rules cannot easily handle this. But using slightly more advanced make features can make it:
SRC := $(shell find . -type f -name "*.c")
OBJ := $(patsubst %.c,%.o,$(notdir $(SRC)))
.PHONY: all
all: $(OBJ)
define COMPILE_rule
$$(patsubst %.c,%.o,$$(notdir $(1))): $(1)
kos-cc $$(KOS_CFLAGS) -c -o $$# $$<
endef
$(foreach s,$(SRC),$(eval $(call COMPILE_rule,$(s))))
.PHONY: clean
clean:
rm -f $(OBJ)
Beware: you could have several source files with the same base name in different directories. And if this happens, you will end up with a fatal conflict of object file names...
EDIT (add conflicts detection):
The following detects conflicts situations and issues an error (replace error by warning or info depending on the severity level you assign to these conflicts):
SRC := $(shell find . -type f -name "*.c")
OBJ := $(patsubst %.c,%.o,$(notdir $(SRC)))
ifneq ($(words $(OBJ)),$(words $(sort $(OBJ))))
$(error object file name conflicts detected)
endif
(sort sorts and also removes duplicates and words returns the number of space-separated words in its string parameter).
the makefile below processes files matching the patterncontent/%.md and outputs the targets in the html directory. Source files are named with the convention of putting a leading number in front of them, like content/01.index.md or content/O2.second-page.md and so on. I would like to remove the leading 0x. number sequence in the target file. For instance, content/01.index.html would generate html/index.html.
How can I do this?
Thanks
MD_FILES = $(shell find content/ -type f -name '*.md')
HTML_FILES = $(patsubst content/%.md, html/%.html, $(MD_FILES))
all: $(HTML_FILES) html/static
html/%.html : content/%.md
mkdir -p $(#D)
python generator/generate.py $< $#
.PHONY: html/static
html/static :
rsync -rupE generator/static html/
.PHONY: clean
clean:
rm -fr html
Replace:
html/%.html : content/%.md
mkdir -p $(#D)
python generator/generate.py $< $#
with:
html/%.html : content/%.md
mkdir -p $(#D)
file='$(#F)'; python generator/generate.py $< "$(#D)/${file#*.}"
Unfortunately, I can't think of a good way of doing that in make itself. I can think of one way but it isn't as simple as that escaping and it isn't safe for files with spaces (not that that matters much here since make already can't handle those).
IMHO, it is a bad idea to use find or wildcards to list files in makefiles. This is because developers have temporary or debugging files sometimes. It is best to list files explicitly. This way, it forces the developer to think about their intent.
If you agree to list files explicitly, then in this case it is best to list the target files, rather than source files, and here is your answer:
HTML_FILES := html/index.html html/second-page.html
.SECONDEXPANSION:
$(HTML_FILES): html/%.html : $$(wildcard content/*.$$*.md)
(put recipe here, using $# and $<)
I have .haml files and want to convert them automatically into .html files and update the latter when .haml is changed.
The generic makefile rule is no problem:
%.html: %.haml
hamlpy $< $#
But now I need a rule or a command to do the following:
find all X.haml files in templates/
execute make X.html command, where X is the same filename (haml is replaced with html).
I can't find how to do this with GNU Make or Unix find.
If all of your *.haml files are well name (i.e. no spaces or other funny characters), you can do it with a call to find(1):
HAML_FILES = $(shell find templates/ -type f -name '*.haml')
HTML_FILES = $(HAML_FILES:.haml=.html)
all: $(HTML_FILES)
%.html : %.haml
hamlpy $< $#
You can use GNU make wildcard function to find files in a certain directory:
INDIR := templates
OUTDIR := ${CURDIR}
haml_files := $(wildcard ${INDIR}/*.haml)
html_files := $(subst ${INDIR}/,${OUTDIR}/,${haml_files:.haml=.html})
all : ${html_files}
clean :
rm -f ${html_files}
${OUTDIR}/%.html : ${INDIR}/%.haml
hamlpy $< $#
.PHONY : all clean
INDIR and OUTDIR can be customized on the command line, for example, to use the current directory for inputs and iutputs:
$ make INDIR=. OUTDIR=.
I dont know much makefile stuff I've been tending to learn bits as required.
The biggest failing of my makefiles is that I have been listing all the files manually, while this hasn't been a problem my current project is getting unwieldy. I have 4 directories each with sources files.
How can I get all the object file listing without having to list them manually.
This doesn't work, but it shows what I've been trying to do.
VPATH = Lib GameCode Moot/Moot Moot/Impl
OBJS = $(subst .cpp, .o, $(VPATH))
foobar: $(OBJS)
g++ -o $# $^
%.o: %.cpp
g++ -c $< -o $# -I Moot
clean:
rm main.o lib.o foo.o foobar
Personally, I never had any problem in listing all files manually. Listing a file to the makefile takes negligible time compared to adding filling it with useful content.
To get all files from different directories, one might suggest using wildcard function. So my_sources:=$(wildcard *.cpp dir1/*.cpp) will make the variable contain source files that match wildcard expression.
However, I find it less convenient than using usual Linux find command via shell:
# Find all sources
my_sources:=$(shell find -iname '*.cpp')
# Make targets out of them
OBJS=$(my_sources:%.cpp=%.o)
Find is more powerful than Make's builtin wildcard. You might also want to use other shell capabilities, such as pipelines, for example, to filter output of find (if Make's filter-out function is not enough). Or something like this, to avoid excessive variables:
OBJS:=$(shell find -iname '*.cpp' | sed 's/\.cpp$/.o/')
You name it!
Using VPATH or vpath will not work for your problem.. it provides a search path to find files but you still need to list the files. If you just need to compile all and any .c/.cpp files found in those directories then this should work:
foobar: $(shell ls Lib/*.cpp) $(shell ls GameCode/*.cpp) $(shell ls Moot/Moot/*.cpp) $(shell ls Moot/Impl/*cpp)
g++ -o $# $^
clean:
rm foobar $(shell ls Lib/*.o) $(shell ls GameCode/*.o) $(shell ls Moot/Moot/*.o) $(shell ls Moot/Impl/*o)
The VPATH info is not needed, the substitution of .o for .cpp can go as can the override of the implicit rule. Additionally, not the use of ls instead of find to look in, and only in, the specfified directory.