Copy list of files to specific paths with Makefile - makefile

Scratching my head for a while now, but I would like to copy an arbitrary list of files with paths to under specified path in system.
File layout:
data/a/file1.ext1
data/b/randomfile.ext2
data/c/file3.ext3
data/c/subdir/randomfile.2
Running make -f Makefile deploy DESTDIR=/path/to/somewhere copies those files to:
$(DESTDIR)/a/file1.ext1
$(DESTDIR)/b/randomfile.ext2
$(DESTDIR)/c/file3.ext3
$(DESTDIR)/c/subdir/randomfile.2
Makefile:
$FILES = \
a/file1.ext1 \
b/randomfile.ext2 \
c/file3.ext3 \
c/subdir/randomfile.2
ifneq ($(filter env_check,$(MAKECMDGOALS)),$())
ifndef DESTDIR
$(error DESTDIR not defined)
endif
endif
# lots of currently broken rules :(
# check whether target directory has certain structure
# check whether all the files listed in $(FILES) are in repository

Are you looking for something like this?
FILES := ...
DST_FILES := $(addprefix $(DESTDIR)/,$(FILES))
ifneq ($(filter env_check,$(MAKECMDGOALS)),$())
ifndef DESTDIR
$(error DESTDIR not defined)
endif
endif
all: $(DST_FILES)
$(DST_FILES) : ${DESTDIR}/% : %
#echo "$< ==> $#"
#[[ -e $< ]] || (echo "some error for $<" && false)
#mkdir -p $(dir $#)
#cp $< $#
[Edit]:
Although the version somewhat worked, I still needed to do following adjustments:
Files in source repository are stored under data directory - fixed by using $addprefix call
When file in $(DESTDIR) already existed, it was never copied - used the .FORCE target. (Another option would be --always-make commandline option).
Eventually, the working Makefile looks like that:
# File are stored under data/
FILES= \
foo/file1.ext \
bar/file2.txe \
bar/dir/file3.txt
ifneq ($(filter env_check,$(MAKECMDGOALS)),$())
ifndef DESTDIR
$(error DESTDIR not defined)
endif
endif
.PHONY: deploy help
help:
#echo "Deploy stuff"
# Check whether certain directories in the output are present
env_check:
#test -d $(DESTDIR)/WEB-INF -a -d $(DESTDIR)/META-INF || \
( echo "DESTDIR: \"$(DESTDIR)\" is not proper deployment path" && exit 1 )
DST_FILES := $(addprefix $(DESTDIR)/, $(FILES))
# We need to add our path prefix to local files and FORCE to always do the copying
$(DST_FILES) : $(addprefix $(DESTDIR), %) : $(addprefix data,%) .FORCE
#cp -pv $< $#
.FORCE:
deploy: env_check $(DST_FILES)
#echo "Deployment done..."

Related

Makefile to generate JSON from Python scripts in separate directories

I have a Makefile which generates JSON from several different Python scripts (the scripts print to stdout) in a single directory, e.g.
/src
scriptOne.py
scriptTwo.py
scriptThree.py
Which outputs the JSON to a folder:
/templates
scriptOne.json
scriptTwo.json
scriptThree.json
I'm trying to restructure so that, for example, each script is in its own subdirectory and the Makefile creates the JSON templates in their consequent subdirectories as follows:
/src
/importantTemplates
scriptOne.py
/notSoImportantTemplates
scriptTwo.py
scriptThree.py
And the output:
/templates
/importantTemplates
scriptOne.json
/notSoImportantTemplates
scriptTwo.json
scriptThree.json
The current Makefile is as follows:
SOURCES := $(shell echo src/*.py)
TARGETS := $(patsubst src/%.py,templates/%.json,$(SOURCES))
all: $(TARGETS)
clean:
rm -f $(TARGETS)
templates/%.json: src/%.py
python2 $< > $#
I've tried changing the wildcards to include a subdirectory for each line e.g. /src/*/*.py, although I just end up with the following:
make: Nothing to be done for `all'.
You want a static pattern rule (4.12 Static Pattern Rules) for this.
SOURCES := $(wildcard src/*/*.py)
TARGETS := $(patsubst src/%.py,templates/%.json,$(SOURCES))
all: $(TARGETS)
clean:
rm -rf templates
$(TARGETS) : templates/%.json: src/%.py
mkdir -p $(#D)
python2 $< > $#
You could avoid needing mkdir -p in that rule body if you wanted to (and go with an order-only prerequisite on the directory instead) but I'm not sure the effort is worth the savings in execution cost. You could avoid the extra shell by combining the two lines mkdir -p $(#D) && python2 $< > $# if you wanted to though.
Please note that the second time you run the make, it will give you the message (if there are no new files):
make: Nothing to be done for `all'.
Try to run make clean and see if you get the same message.
Here is the Makefile which will do what you want:
SOURCES := $(wildcard src/*/*.py)
TARGETS := $(patsubst src/%.py,templates/%.json,$(SOURCES))
FOLDERS := $(sort $(dir $(TARGETS)))
all: $(TARGETS)
clean:
rm -rf $(TARGETS) $(FOLDERS)
$(FOLDERS):
mkdir -p $#
$(TARGETS): $(SOURCES) $(FOLDERS)
python2 $< > $#
The FOLDERS variable will contain the folders you need to create in the template directory. (sort will remove duplicates, so each folder will be there only once)
The $(FOLDERS) rule will create the folders.
The clean rule will remove the folders also.
If you need to add more sources, just do it like this:
SOURCES := $(wildcard src/*/*.py)
SOURCES += $(wildcard src/*.py)
...

Makefile rule for copying files with multiple extensions into target directory

I would like make to copy files from the source directory into a target directory. And i would like to avoid copying unchanged files. Therefore, I am trying to utilize make function of checking for newer files with a %.:%. rule.
Now, in this instance, the source and target EXTENSION are the same. There are object files elsewhere but not for the graphical assets.
I use make to copy images.
TARGET := target
SOURCE := source
GRAPHICS := $(foreach dir,$(SOURCE), $(wildcard $(dir)/*.jpg ) $(wildcard $(dir)/**/*.jpg ) $(wildcard $(dir)/*.png ) $(wildcard $(dir)/**/*.png ) $(wildcard $(dir)/*.gif ) $(wildcard $(dir)/**/*.gif ) )
JPG = $(GRAPHICS:.jpg=.tmp)
PNG = $(GRAPHICS:.png=.tmp)
GIF = $(GRAPHICS:.gif=.tmp)
And then use the following rule to copy files into target directory:
%.tmp:%.jpg
find $< | cpio -p -d -v $(TARGET)
Questions
Is it possible to string replace the TARGET directory here
and thereby use the make newer capability?
I tried
JPG = $(GRAPHICS:$(TARGET).jpg=.tmp)
But that fails with No rule to make target. Is it only possible to compare source and object in the same directory?
Can one make a rule such that the source and object are the same extension?
%.jpg:%.jpg
The closest I can come up with is:
$(TARGET)%.jpg:%.jpg
but that never runs. Even after a clean.
Here is the solution.
This 'setup' remains the same.
TARGET := target
SOURCE := source GRAPHICS := $(foreach dir,$(SOURCE), $(wildcard $(dir)/*.jpg )
$(wildcard $(dir)/**/*.jpg ) $(wildcard $(dir)/*.png ) $(wildcard $(dir)/**/*.png )
$(wildcard $(dir)/*.gif ) $(wildcard $(dir)/**/*.gif ) )
Now add a prefix to ALL (space separated) values in the $GRAPHICS string
TARGET_GRAPHICS := $(addprefix $(TARGET)/, $(GRAPHICS) )
Note that patsubst aka -- TARGET_GRAPHICS := $(patsubst %, $(TARGET)/%, $(GRAPHICS) ) -- only worked for the first entry. Perhaps I was doing it incorrectly.
Provide a pattern match for each file type. Here do whatever you wish. In my case, copy the input into the target directory. cpio will make the needed directories. cp -p might also work as needed.
$(TARGET)/%.jpg : %.jpg
#echo "--- .jpg copying " $< " to " $# " into dir " $(<D)
#find $< | cpio -p -d -v $(TARGET)
$(TARGET)/%.png : %.png
#echo "--- .png copying " $< " to " $#
#find $< | cpio -p -d -v $(TARGET)
$(TARGET)/%.gif : %.gif
#echo "--- .gif Copying " $< " to " $#
#find $< | cpio -p -d -v $(TARGET)
So you want the copy of d1/d2/f.png into out/d1/d2/f.png to make use of make's dependency checking?
out/d1/d2/f.png: d1/d2/f.png
out/d1/d2/f.png:
cp $< $#
Adding a jpg file d3/g.jpg say,
out/d3/g.jpg: d3/g.jpg
out/d1/d2/f.png: d1/d2/f.png
out/d1/d2/f.png out/d3/g.jpg:
cp $< $#
We can express this more cleanly with a static pattern rule.
out/d3/g.jpg out/d1/d2/f.png: out/% %
cp $< $#
Nice. So adding your ${GRAPHICS} and fleshing things out a bit
outdir := out/
targets := $(addprefix ${outdir},${GRAPHICS})
.PHONY: all
all: ${targets} ; : $# Success
${targets}: ${outdir}%: %
echo $< | cpio -d -p ${outdir}
Parallel safe too, so make -j9 will exercise your 8 CPUs nicely.

Makefile: set parameters for commands executed by implicit rules (CFLAGS, LDFLAGS)

I'm very new to makefiles. I'm reading the GNU-make manual, but I'm still unclear about how to set the parameters for the compiler and the linker when they are executed by an implicit rule.
This is part of the makefile, note there is no explicit declaration of how to compile and link everything:
.PHONY: $(TARGET) build_libs
all: build_libs $(TARGET)
$(TARGET):
#echo "============> building target: $(TARGET)"
#$(MAKE) -C $(SDK_PATH)/VP_SDK/Build $(TMP_SDK_FLAGS) $(SDK_FLAGS) $(MAKECMDGOALS) USE_LINUX=yes
mv $(ARDRONE_TARGET_DIR)/ardrone_testing_tool $(TARGET)
mv $(TARGET) $(ARDRONE_TARGET_DIR)/
#echo "============> end building target: $(TARGET)"
$(MAKECMDGOALS): build_libs
#echo "============> making cmd goals"
#$(MAKE) -C $(SDK_PATH)/VP_SDK/Build $(TMP_SDK_FLAGS) $(SDK_FLAGS) $(MAKECMDGOALS) USE_LINUX=yes
#echo "============> end making cmd goals"
build_libs:
#echo "============> building libs"
#$(MAKE) -C $(SDK_PATH)/Soft/Build $(TMP_SDK_FLAGS) $(SDK_FLAGS) $(MAKECMDGOALS) USE_LINUX=yes
#echo "============> end building libs"
That makefile builds an executable from the source files and library. But I want to compile them into a shared library. Because of that I (think I) have to add the -fPIC parameter to cc and the -shared and -soname parameters to ld. I've tried with CFLAGS=-fPIC and LDFLAGS=-shared -soname foo, which didn't work. Has anybody suggestions on how to get a shared libarary? If you need more information please just ask. Thanks in advance!
UPDATE: The makefile in $(SDK_PATH)/Soft/Build:
GEN_CUSTOM_HEADER:=../Common/generated_custom.h
include custom.makefile
include config.makefile
GNUTOOLS_PATH=/usr/local/$(GNUTOOLS_VERSION)/bin
define ADD_RULE_TEMPLATE
TO_BUILD+=build_$(1)
endef
# Add rule for each target
$(foreach target,$(TARGETS),$(eval $(call ADD_RULE_TEMPLATE,$(target))))
.PHONY: linux_sample svn_update $(TO_BUILD) build_libs $(MAKECMDGOALS)
all: $(GEN_CUSTOM_HEADER) build_libs $(TO_BUILD)
$(GEN_CUSTOM_HEADER): custom.makefile
#echo "#ifndef _GENERATED_CUSTOM_CONFIGURATION_H_" > $#
#echo "#define _GENERATED_CUSTOM_CONFIGURATION_H_" >> $#
#echo >> $#
#echo "#if defined(BR2_PACKAGE_BCM4318_AP)" >> $#
#echo "# define AP" >> $#
#echo "#else" >> $#
#echo "# define STA" >> $#
#echo "#endif" >> $#
#echo "#define CURRENT_NUM_VERSION_SOFT \"$(MAJOR_VERSION).$(MINOR_VERSION).$(MODIF_VERSION)\"" >> $#
#echo "#define CURRENT_BUILD_DATE \"$(shell date +%F\ %H:%M)\"" >> $#
#echo >> $#
ifeq ("$(VIDEO_YUV)","yes")
#echo "#define USE_VIDEO_YUV" >> $#
endif
ifeq ("$(RECORD_VISION_DATA)","yes")
#echo "#define RECORD_VISION_DATA" >> $#
endif
#echo >> $#
#echo "#define WIFI_NETWORK_NAME \"$(WIFI_NETWORK_NAME)\"" >> $#
#echo "#define WIFI_BROADCAST \"$(WIFI_BROADCAST)\"" >> $#
#echo "#define WIFI_ARDRONE_IP \"$(WIFI_ARDRONE_IP)\"" >> $#
#echo >> $#
#echo "#if defined(__linux__) || defined(USE_MINGW32)" >> $#
#echo "# define WIFI_MOBILE_IP \"$(WIFI_MOBILE_IP)\"" >> $#
#echo "# define WIRED_ITFNAME \"$(WIRED_ITFNAME)\"" >> $#
#echo "#endif // ! __linux__" >> $#
#echo >> $#
#echo >> $#
#echo "#endif // ! _GENERATED_CUSTOM_CONFIGURATION_H_" >> $#
ifneq "$(MAKECMDGOALS)" ""
ifneq "$(MAKECMDGOALS)" "clean"
ifneq "$(MAKECMDGOALS)" "update"
$(MAKECMDGOALS):
#echo -e "\nCannot make what you ask me to do :-("
else
$(MAKECMDGOALS): svn_update
endif
endif
endif
$(MAKECMDGOALS): build_libs $(TO_BUILD)
checkpackages:
ifeq ($(IPHONE_MODE),yes)
sh $(shell pwd)/check_dependencies.sh iphone RELEASE_BUILD=$(RELEASE_BUILD) $(MAKECMDGOALS)
else
ifeq ($(USE_LINUX),yes)
sh $(shell pwd)/check_dependencies.sh static RELEASE_BUILD=$(RELEASE_BUILD) $(MAKECMDGOALS)
else
ifeq ($(USE_ANDROID),yes)
sh $(shell pwd)/check_dependencies.sh android_no_neon RELEASE_BUILD=$(RELEASE_BUILD) $(MAKECMDGOALS)
endif
endif
endif
define GENERIC_RULES_TEMPLATE
build_$(1):
#$(MAKE) -C $(1) $(MAKECMDGOALS)
endef
$(foreach target,$(TARGETS),$(eval $(call GENERIC_RULES_TEMPLATE,$(target))))
build_libs: checkpackages
#$(MAKE) PC_TARGET=yes USE_ARDRONE_TOOL=yes TARGET=pc_ USE_MINGW32=no -C ../Lib/Build $(MAKECMDGOALS)
#$(MAKE) PC_TARGET=yes USE_ARDRONE_TOOL=no TARGET=pc_ USE_MINGW32=no -C ../Lib/Build $(MAKECMDGOALS)
ifeq ("$(MINGW32_MODE)","yes")
ifeq ($(shell which i586-mingw32msvc-gcc 2> /dev/null),)
$(warning You need MinGW32 to compile My Ardrone lib for Windows if you want. (under Debian: apt-get install mingw32))
else
# #$(MAKE) PC_TARGET=yes TARGET=mingw32_ USE_MINGW32=yes TMP_SDK_FLAGS="USE_MINGW32=yes NO_COM=yes USE_BLUEZ=no" -C ../Lib/Build $(MAKECMDGOALS)
# #$(MAKE) PC_TARGET=yes TARGET=emb_mingw32_ USE_MINGW32=yes CONTROL_DLL=yes TMP_SDK_FLAGS="USE_MINGW32=yes NO_COM=yes USE_BLUEZ=no" -C ../Lib/Build $(MAKECMDGOALS)
endif
endif
ifeq ($(WIIMOTE_SUPPORT),yes)
# #$(MAKE) PC_TARGET=yes TARGET=pc_ TMP_SDK_FLAGS="USE_BLUEZ=yes" -C ../Lib/libcwiid $(MAKECMDGOALS)
endif
define svn_update_template
cd ../.. ; \
echo "Checking out tag $(1) of $(2) ..." ; \
if [ $(1) != head ] ; then \
svn co -r $(1) https://svn.ardrone.org/repo/ARDrone_API/$(2) ; \
else \
svn co https://svn.ardrone.org/repo/ARDrone_API/$(2) ; \
fi ; \
cd Soft/Build ;
endef
svn_update:
#-$(call svn_update_template,$(SDK_VERSION),ARDroneLib)
If custom.makefile and config.makefile are needed you can find them here: http://pastebin.com/H8PNKKhu
UPDATE 2: I just discovered this, located in $(SDK_PATH)/VP_SDK/Build: http://pastebin.com/3knnSkmy
Since you haven't showed us the makefile in $(SDK_PATH)/Soft/Build, I'm going to take a guess.
In the GNU Make manual, there is the list of variables used by implicit rules:
CFLAGS: Extra flags to give the C compiler.
LDFLAGS: Extra flags to give to compiler when they are supposed to invoke the linker 'ld'.
In addition, the catalogue of implicit rules mentions the LDLIBS variable that names libraries used in the implicit link rule. (The implicit link rule is fine when you have a single source file being linked or where one of the object files is the name of the final executable, otherwise you need to write an explicit linking rule.)
This isn't coming into focus. These appear to be generated makefiles, so they're hard to trace, there's clearly more going on than we're seeing (and I don't blame you for not posting everything), and I don't see where the build_libs rule leads, or any use of the linker, or anything that could build a static library. We could show you how to write an explicit rule to build a shared library, but without a list of sources or objects we wouldn't know what to put into that library.
This is getting desperate, but could you try the following:
1) make -C $(SDK_PATH)/Soft/Build (putting in the appropriate value for $(SDK_PATH)) and verify that this produces the static library.
2) make -n, capture the torrent of output and put it in pastebin. We'll try to see what rules it's using, which might lead to what we need.
I did it. Finally!!!
The solution: in $(SDK_PATH)/VP_SDK/Build there was a file named generic.makefile containing all the compiler and linker invocations. I had to add the -shared and -fPIC arguments to some of them. Sounds really simple, but nothing of that was documented so I first had to look for that hidden compiler/linker invocations...

Setting target variable for the name of the makefile from a subdirectory

How can I have the variable for $(MAKEFILE) be defined during target execution?
Basically I have a few make files in subdirectories that are named for a specific platform "Makefile.aix" and just Makefile in all other directories. I would like to set a variable for $(MAKEFILE) that gets defined in each subdirectory. Code would look something like this.
MAKEFILE = Makefile
SUBDIR = ./sub ./sub2
ifneq ($(wildcard Makefile),)
MAKEFILE = Makefile
else
MAKEFILE = Makefile.$(PLATFORM)
endif
all:;
#for i in $(SUBDIR);\
do (\
echo Making $$i ...;\
cd $$i;\
make -f $(MAKEFILE)\
); done
Is there just one Makefile.$(PLATFORM) in each subdirectory, or are there several, for different platforms?
In the first case, you could do something like this:
SUBDIR = ./sub ./sub2
define script
cd $(1); \
$(MAKE) -f Makefile*
endef
all:
$(foreach dir, $(SUBDIR), $(call script,$(dir)))
(The empty line inside the define is significant. It can be omitted, if you add a semicolon at the end of the line $(MAKE) ..., leading to one long command line, containing the commands for all directories, which will then be executed in one chunk.)
An alternative script would be (just a matter of personal preference which you like better):
define script
$(MAKE) -C $(1) -f $(notdir $(wildcard $(1)/Makefile*))
endef
If there are several Makefile.$(PLATFORM) files in a directory it becomes more difficult. I'll have to think about that one some more.
UPDATE: In response to your comment, something like this should work:
define script
$(MAKE) -C $(1) -f $(notdir $(wildcard $(1)/Makefile $(1)/Makefile.$(PLATFORM)))
endef
Following your logic, I'd propose update do () section with:
do (\
echo Making $$i ...;\
cd $$i;\
if [ -f Makefile.$(PLATFORM) ] \
then\
make -f Makefile.$(PLATFORM) \
else\
make -f Makefile\
fi\
); done
This is actually not a make style, but I can't suggest anything better without specific of your project
You can do most of this, including the loop over directories, using GNU make's built-in functions. Put the following in a central place, say $(TOP_DIR)/mk/subdir.mk:
makefile-for-dir = \
$(if $(wildcard $(1)/Makefile),Makefile,Makefile.$(PLATFORM))
make-recursive = \
$(foreach _d,$(1),$(MAKE) -C $(_d) -f $(call makefile-for-dir,$(_d)) && ) :
In each makefile that start recursive makes, use
include $(TOP_DIR)/mk/subdir.mk
SUBDIRS = dir1 dir2 dir3
.PHONY: all
all:
+#$(call make-recursive,$(SUBDIRS))

Create directories using make file

I want to create directories using makefile. My project directory is like this
+--Project
+--output
+--source
+Testfile.cpp
+Makefile
I want to put all the objects and output into the respective output folder. I want to create folder structure which would be like this after compiling.
+--Project
+--output
+--debug (or release)
+--objs
+Testfile.o
+Testfile (my executable file)
+--source
+Testfile.cpp
+Makefile
I tried with several options, but could not succeed. Please help me to make directories using make file. I'm posting my Makefile for your consideration.
#---------------------------------------------------------------------
# Input dirs, names, files
#---------------------------------------------------------------------
OUTPUT_ROOT := output/
TITLE_NAME := TestProj
ifdef DEBUG
TITLE_NAME += _DEBUG
else
ifdef RELEASE
TITLE_NAME += _RELEASE
endif
endif
# Include all the source files here with the directory tree
SOURCES := \
source/TestFile.cpp \
#---------------------------------------------------------------------
# configs
#---------------------------------------------------------------------
ifdef DEBUG
OUT_DIR := $(OUTPUT_ROOT)debug
CC_FLAGS := -c -Wall
else
ifdef RELEASE
OUT_DIR := $(OUTPUT_ROOT)release
CC_FLAGS := -c -Wall
else
$(error no build type defined)
endif
endif
# Put objects in the output directory.
OUT_O_DIR := $(OUT_DIR)/objs
#---------------------------------------------------------------------
# settings
#---------------------------------------------------------------------
OBJS = $(SOURCES:.cpp=.o)
DIRS = $(subst /,/,$(sort $(dir $(OBJS))))
DIR_TARGET = $(OUT_DIR)
OUTPUT_TARGET = $(OUT_DIR)/$(TITLE_NAME)
CC_FLAGS +=
LCF_FLAGS :=
LD_FLAGS :=
#---------------------------------------------------------------------
# executables
#---------------------------------------------------------------------
MD := mkdir
RM := rm
CC := g++
#---------------------------------------------------------------------
# rules
#---------------------------------------------------------------------
.PHONY: all clean title
all: title
clean:
$(RM) -rf $(OUT_DIR)
$(DIR_TARGET):
$(MD) -p $(DIRS)
.cpp.o:
#$(CC) -c $< -o $#
$(OBJS): $(OUT_O_DIR)/%.o: %.cpp
#$(CC) -c $< -o $#
title: $(DIR_TARGET) $(OBJS)
In my opinion, directories should not be considered targets of your makefile, either in technical or in design sense. You should create files and if a file creation needs a new directory then quietly create the directory within the rule for the relevant file.
If you're targeting a usual or "patterned" file, just use make's internal variable $(#D), that means "the directory the current target resides in" (cmp. with $# for the target). For example,
$(OUT_O_DIR)/%.o: %.cpp
#mkdir -p $(#D)
#$(CC) -c $< -o $#
title: $(OBJS)
Then, you're effectively doing the same: create directories for all $(OBJS), but you'll do it in a less complicated way.
The same policy (files are targets, directories never are) is used in various applications. For example, git revision control system doesn't store directories.
Note: If you're going to use it, it might be useful to introduce a convenience variable and utilize make's expansion rules.
dir_guard=#mkdir -p $(#D)
$(OUT_O_DIR)/%.o: %.cpp
$(dir_guard)
#$(CC) -c $< -o $#
$(OUT_O_DIR_DEBUG)/%.o: %.cpp
$(dir_guard)
#$(CC) -g -c $< -o $#
title: $(OBJS)
This would do it - assuming a Unix-like environment.
MKDIR_P = mkdir -p
.PHONY: directories
all: directories program
directories: ${OUT_DIR}
${OUT_DIR}:
${MKDIR_P} ${OUT_DIR}
This would have to be run in the top-level directory - or the definition of ${OUT_DIR} would have to be correct relative to where it is run. Of course, if you follow the edicts of Peter Miller's "Recursive Make Considered Harmful" paper, then you'll be running make in the top-level directory anyway.
I'm playing with this (RMCH) at the moment. It needed a bit of adaptation to the suite of software that I am using as a test ground. The suite has a dozen separate programs built with source spread across 15 directories, some of it shared. But with a bit of care, it can be done. OTOH, it might not be appropriate for a newbie.
As noted in the comments, listing the 'mkdir' command as the action for 'directories' is wrong. As also noted in the comments, there are other ways to fix the 'do not know how to make output/debug' error that results. One is to remove the dependency on the the 'directories' line. This works because 'mkdir -p' does not generate errors if all the directories it is asked to create already exist. The other is the mechanism shown, which will only attempt to create the directory if it does not exist. The 'as amended' version is what I had in mind last night - but both techniques work (and both have problems if output/debug exists but is a file rather than a directory).
Or, KISS.
DIRS=build build/bins
...
$(shell mkdir -p $(DIRS))
This will create all the directories after the Makefile is parsed.
make in, and off itself, handles directory targets just the same as file targets. So, it's easy to write rules like this:
outDir/someTarget: Makefile outDir
touch outDir/someTarget
outDir:
mkdir -p outDir
The only problem with that is, that the directories timestamp depends on what is done to the files inside. For the rules above, this leads to the following result:
$ make
mkdir -p outDir
touch outDir/someTarget
$ make
touch outDir/someTarget
$ make
touch outDir/someTarget
$ make
touch outDir/someTarget
This is most definitely not what you want. Whenever you touch the file, you also touch the directory. And since the file depends on the directory, the file consequently appears to be out of date, forcing it to be rebuilt.
However, you can easily break this loop by telling make to ignore the timestamp of the directory. This is done by declaring the directory as an order-only prerequsite:
# The pipe symbol tells make that the following prerequisites are order-only
# |
# v
outDir/someTarget: Makefile | outDir
touch outDir/someTarget
outDir:
mkdir -p outDir
This correctly yields:
$ make
mkdir -p outDir
touch outDir/someTarget
$ make
make: 'outDir/someTarget' is up to date.
TL;DR:
Write a rule to create the directory:
$(OUT_DIR):
mkdir -p $(OUT_DIR)
And have the targets for the stuff inside depend on the directory order-only:
$(OUT_DIR)/someTarget: ... | $(OUT_DIR)
All solutions including the accepted one have some issues as stated in their respective comments. The accepted answer by #jonathan-leffler is already quite good but does not take into effect that prerequisites are not necessarily to be built in order (during make -j for example). However simply moving the directories prerequisite from all to program provokes rebuilds on every run AFAICT.
The following solution does not have that problem and AFAICS works as intended.
MKDIR_P := mkdir -p
OUT_DIR := build
.PHONY: directories all clean
all: $(OUT_DIR)/program
directories: $(OUT_DIR)
$(OUT_DIR):
${MKDIR_P} $(OUT_DIR)
$(OUT_DIR)/program: | directories
touch $(OUT_DIR)/program
clean:
rm -rf $(OUT_DIR)
I've just come up with a fairly reasonable solution that lets you define the files to build and have directories be automatically created. First, define a variable ALL_TARGET_FILES that holds the file name of every file that your makefile will be build. Then use the following code:
define depend_on_dir
$(1): | $(dir $(1))
ifndef $(dir $(1))_DIRECTORY_RULE_IS_DEFINED
$(dir $(1)):
mkdir -p $$#
$(dir $(1))_DIRECTORY_RULE_IS_DEFINED := 1
endif
endef
$(foreach file,$(ALL_TARGET_FILES),$(eval $(call depend_on_dir,$(file))))
Here's how it works. I define a function depend_on_dir which takes a file name and generates a rule that makes the file depend on the directory that contains it and then defines a rule to create that directory if necessary. Then I use foreach to call this function on each file name and eval the result.
Note that you'll need a version of GNU make that supports eval, which I think is versions 3.81 and higher.
given that you're a newbie, I'd say don't try to do this yet. it's definitely possible, but will needlessly complicate your Makefile. stick to the simple ways until you're more comfortable with make.
that said, one way to build in a directory different from the source directory is VPATH; i prefer pattern rules
OS independence is critical for me, so mkdir -p is not an option. I created this series of functions that use eval to create directory targets with the prerequisite on the parent directory. This has the benefit that make -j 2 will work without issue since the dependencies are correctly determined.
# convenience function for getting parent directory, will eventually return ./
# $(call get_parent_dir,somewhere/on/earth/) -> somewhere/on/
get_parent_dir=$(dir $(patsubst %/,%,$1))
# function to create directory targets.
# All directories have order-only-prerequisites on their parent directories
# https://www.gnu.org/software/make/manual/html_node/Prerequisite-Types.html#Prerequisite-Types
TARGET_DIRS:=
define make_dirs_recursively
TARGET_DIRS+=$1
$1: | $(if $(subst ./,,$(call get_parent_dir,$1)),$(call get_parent_dir,$1))
mkdir $1
endef
# function to recursively get all directories
# $(call get_all_dirs,things/and/places/) -> things/ things/and/ things/and/places/
# $(call get_all_dirs,things/and/places) -> things/ things/and/
get_all_dirs=$(if $(subst ./,,$(dir $1)),$(call get_all_dirs,$(call get_parent_dir,$1)) $1)
# function to turn all targets into directories
# $(call get_all_target_dirs,obj/a.o obj/three/b.o) -> obj/ obj/three/
get_all_target_dirs=$(sort $(foreach target,$1,$(call get_all_dirs,$(dir $(target)))))
# create target dirs
create_dirs=$(foreach dirname,$(call get_all_target_dirs,$1),$(eval $(call make_dirs_recursively,$(dirname))))
TARGETS := w/h/a/t/e/v/e/r/things.dat w/h/a/t/things.dat
all: $(TARGETS)
# this must be placed after your .DEFAULT_GOAL, or you can manually state what it is
# https://www.gnu.org/software/make/manual/html_node/Special-Variables.html
$(call create_dirs,$(TARGETS))
# $(TARGET_DIRS) needs to be an order-only-prerequisite
w/h/a/t/e/v/e/r/things.dat: w/h/a/t/things.dat | $(TARGET_DIRS)
echo whatever happens > $#
w/h/a/t/things.dat: | $(TARGET_DIRS)
echo whatever happens > $#
For example, running the above will create:
$ make
mkdir w/
mkdir w/h/
mkdir w/h/a/
mkdir w/h/a/t/
mkdir w/h/a/t/e/
mkdir w/h/a/t/e/v/
mkdir w/h/a/t/e/v/e/
mkdir w/h/a/t/e/v/e/r/
echo whatever happens > w/h/a/t/things.dat
echo whatever happens > w/h/a/t/e/v/e/r/things.dat
See https://www.oreilly.com/library/view/managing-projects-with/0596006101/ch12.html
REQUIRED_DIRS = ...
_MKDIRS := $(shell for d in $(REQUIRED_DIRS); \
do \
[[ -d $$d ]] || mkdir -p $$d; \
done)
$(objects) : $(sources)
As I use Ubuntu, I also needed add this at the top of my Makefile:
SHELL := /bin/bash # Use bash syntax
I use the makefiles in windows environment and my simple solution is as follows,
Create a target makedir and add it as a prerequisites to where ever it is required.
# Default goal
all: gccversion makedir build finalize list sizeafter completed
The makedir target is (applicable only in windows environment)
makedir:
#IF NOT EXIST $(subst /,\,$(BUILD_DIR)) mkdir $(subst /,\,$(BUILD_DIR)) 2> NULL
#IF NOT EXIST $(subst /,\,$(OUTPUT_DIR)) mkdir $(subst /,\,$(OUTPUT_DIR)) 2> NULL
#IF NOT EXIST $(subst /,\,$(DEP_DIR)) mkdir $(subst /,\,$(DEP_DIR)) 2> NUL
#IF NOT EXIST $(subst /,\,$(OBJ_DIR)) mkdir $(subst /,\,$(OBJ_DIR)) 2> NUL
$(subst /,\,$(BUILD_DIR)) converts the directory separator / to \ and
mkdir $(subst /,\,$(BUILD_DIR)) 2> NUL redirects the error if any.
src_dir := src
obj_dir := obj
build_dir := build
dirs := $(src_dir) $(obj_dir) $(build_dir) # new variable
all: $(dirs) $(other_dependencies) # added dependency (*before* any others)
$(dirs): # rule which makes missing directories
mkdir $#
Won't clutter your terminal with "cannot create directory" error messages. If the directories exist, they don't need to be built.
Works like any other dependency, only requires one rule and one variable.

Resources