I have a Makefile with two rules. One which converts a database file into a SQL file and one which does the opposite.
all:
data.sql: data.sqlite3
sqlite3 $< .dump > $#
data.sqlite3: data.sql
sqlite3 $# < $<
This works, but I get warnings about circular dependencies.
$ rm data.sql
$ make data.sql
make: Circular data.sqlite3 <- data.sql dependency dropped.
sqlite3 data.sqlite3 .dump > data.sql
$ rm data.sqlite3
$ make data.sqlite3
make: Circular data.sql <- data.sqlite3 dependency dropped.
sqlite3 data.sqlite3 < data.sql
Is it possible to get rid of the warning?
From your own solution I gather it's never the case that
an .sql and an .sqlite3 exist at the same time. In that
case you can be more concise by making each missing .sql
or .sqlite3 as a side-effect of pretending to make a proxy target but
never actually doing it. Here's an illustration:
# These are the missing `.sql` files for which a `.sqlite3` exists...
sql_missing = $(filter-out \
$(wildcard *.sql),$(addsuffix .sql,$(basename $(wildcard *.sqlite3))))
# These are the missing `.sqlite3` files for which an `.sql` exists...
sqlite3_missing = $(filter-out \
$(wildcard *sqlite3),$(addsuffix .sqlite3,$(basename $(wildcard *.sql))))
# There are the proxy targets...
targs = $(addsuffix _missing,$(sql_missing) $(sqlite3_missing))
.PHONY: all
all: $(targs)
%.sql_missing: %.sqlite3
#echo "$< => $(basename $#).sql"
#touch $(basename $#).sql
%.sqlite3_missing: %.sql
#echo "$< => $(basename $#).sqlite3"
#touch $(basename $#).sqlite3
This caters for a population of either filetype or both in the directory,
as long as you only want to remake the missing file of each incomplete pair.
In use:
$ ls
Makefile
$ touch f1.sql
$ touch f2.sqlite3
$ make
f2.sqlite3 => f2.sql
f1.sql => f1.sqlite3
$ ls
f1.sql f1.sqlite3 f2.sql f2.sqlite3 Makefile
$ make
make: Nothing to be done for 'all'.
I think I got it. A bit verbose but it seems to work.
DBFILE := data.sqlite3
SQLFILE := data.sql
ifeq ("$(wildcard $(SQLFILE))","")
MISSING := $(SQLFILE)
endif
ifeq ("$(wildcard $(DBFILE))","")
MISSING := $(DBFILE)
endif
all: $(MISSING)
ifeq ("$(wildcard $(SQLFILE))","")
$(SQLFILE): $(DBFILE)
sqlite3 $< .dump > $#
endif
ifeq ("$(wildcard $(DBFILE))","")
$(DBFILE): $(SQLFILE)
sqlite3 $# < $<
endif
Related
Scratching my head for a while now, but I would like to copy an arbitrary list of files with paths to under specified path in system.
File layout:
data/a/file1.ext1
data/b/randomfile.ext2
data/c/file3.ext3
data/c/subdir/randomfile.2
Running make -f Makefile deploy DESTDIR=/path/to/somewhere copies those files to:
$(DESTDIR)/a/file1.ext1
$(DESTDIR)/b/randomfile.ext2
$(DESTDIR)/c/file3.ext3
$(DESTDIR)/c/subdir/randomfile.2
Makefile:
$FILES = \
a/file1.ext1 \
b/randomfile.ext2 \
c/file3.ext3 \
c/subdir/randomfile.2
ifneq ($(filter env_check,$(MAKECMDGOALS)),$())
ifndef DESTDIR
$(error DESTDIR not defined)
endif
endif
# lots of currently broken rules :(
# check whether target directory has certain structure
# check whether all the files listed in $(FILES) are in repository
Are you looking for something like this?
FILES := ...
DST_FILES := $(addprefix $(DESTDIR)/,$(FILES))
ifneq ($(filter env_check,$(MAKECMDGOALS)),$())
ifndef DESTDIR
$(error DESTDIR not defined)
endif
endif
all: $(DST_FILES)
$(DST_FILES) : ${DESTDIR}/% : %
#echo "$< ==> $#"
#[[ -e $< ]] || (echo "some error for $<" && false)
#mkdir -p $(dir $#)
#cp $< $#
[Edit]:
Although the version somewhat worked, I still needed to do following adjustments:
Files in source repository are stored under data directory - fixed by using $addprefix call
When file in $(DESTDIR) already existed, it was never copied - used the .FORCE target. (Another option would be --always-make commandline option).
Eventually, the working Makefile looks like that:
# File are stored under data/
FILES= \
foo/file1.ext \
bar/file2.txe \
bar/dir/file3.txt
ifneq ($(filter env_check,$(MAKECMDGOALS)),$())
ifndef DESTDIR
$(error DESTDIR not defined)
endif
endif
.PHONY: deploy help
help:
#echo "Deploy stuff"
# Check whether certain directories in the output are present
env_check:
#test -d $(DESTDIR)/WEB-INF -a -d $(DESTDIR)/META-INF || \
( echo "DESTDIR: \"$(DESTDIR)\" is not proper deployment path" && exit 1 )
DST_FILES := $(addprefix $(DESTDIR)/, $(FILES))
# We need to add our path prefix to local files and FORCE to always do the copying
$(DST_FILES) : $(addprefix $(DESTDIR), %) : $(addprefix data,%) .FORCE
#cp -pv $< $#
.FORCE:
deploy: env_check $(DST_FILES)
#echo "Deployment done..."
Let's consider this Makefile:
.SUFFIXES:
.DEFAULT_GOAL := all
out=foo
clean:
rm -f vars.mk
rm -f $(out)
vars.mk: vars.mk.default
#echo "Regenerating $#..."
cp $< $# # Let's assume the translation is much complex than a cp
-include vars.mk
ifeq ($(filter foo,$(FOO)),)
$(error FOO undefined)
endif
all: $(out)
$(out): vars.mk
echo "Cow says: I am not a $(FOO)." > $#
And the file vars.mk.default
FOO = foo bar
I would like to regenerate my targets if vars.mk.default is updated. Furthermore, as double check, one must check that foo exists in $(FOO).
How to force make to regenerate vars.mk if vars.mk.default is updated?
In other words, I would like this output:
$ make clean
$ sed 's/dog/cat/' vars.mk.default
$ make foo
Regenerating vars.mk...
echo "Cow says: I am not a cat" > all
$ make foo
make: Nothing to be done for 'all'.
$ sed 's/cat/dog/' vars.mk.default
$ make
Regenerating vars.mk...
echo "Cow says: I am not a dog" > all
$ rm vars.mak
$ make
Regenerating vars.mk...
echo "Cow says: I am not a dog" > all
To avoid failing if vars.mk doesn't exist, just check for it first:
ifeq ($(wildcard vars.mk),vars.mk)
ifeq ($(filter foo,$(FOO)),)
$(error FOO undefined)
endif
endif
My goal is to regenerate my targets if vars.mk.default is updated.
In this case make your targets depend on that file, but filter it out in the recipes, e.g.
foo.o : foo.cc vars.mk.default
$(COMPILE) $(filter-out vars.mk.default,$^)
In the case vars.mk does not exist, make fails on the ifeq and do not generates vars.mk.
Make is going to build vars.mk and restart, see How Makefiles Are Remade for more details.
So, to avoid that error, check first if FOO is defined with ifdef FOO.
A couple of things. First, you should put a - in front of the include to prevent a warning from popping up if the file does not exist:
-include vars.mk
This will not cause a fatal error if vars.mk is not generated, but because the vars.mk rule would fail in this case, you would get your error from there.
You can then check if $(FOO) contains foo from within a recipe:
checkForFoo: vars.mk
#[[ $(FOO) =~ .*foo.* ]] || false
all:checkForFoo
The recipe is only run after the vars.mk was generated and included, so it should only fail in the conditions you want.
I have a Makefile which generates JSON from several different Python scripts (the scripts print to stdout) in a single directory, e.g.
/src
scriptOne.py
scriptTwo.py
scriptThree.py
Which outputs the JSON to a folder:
/templates
scriptOne.json
scriptTwo.json
scriptThree.json
I'm trying to restructure so that, for example, each script is in its own subdirectory and the Makefile creates the JSON templates in their consequent subdirectories as follows:
/src
/importantTemplates
scriptOne.py
/notSoImportantTemplates
scriptTwo.py
scriptThree.py
And the output:
/templates
/importantTemplates
scriptOne.json
/notSoImportantTemplates
scriptTwo.json
scriptThree.json
The current Makefile is as follows:
SOURCES := $(shell echo src/*.py)
TARGETS := $(patsubst src/%.py,templates/%.json,$(SOURCES))
all: $(TARGETS)
clean:
rm -f $(TARGETS)
templates/%.json: src/%.py
python2 $< > $#
I've tried changing the wildcards to include a subdirectory for each line e.g. /src/*/*.py, although I just end up with the following:
make: Nothing to be done for `all'.
You want a static pattern rule (4.12 Static Pattern Rules) for this.
SOURCES := $(wildcard src/*/*.py)
TARGETS := $(patsubst src/%.py,templates/%.json,$(SOURCES))
all: $(TARGETS)
clean:
rm -rf templates
$(TARGETS) : templates/%.json: src/%.py
mkdir -p $(#D)
python2 $< > $#
You could avoid needing mkdir -p in that rule body if you wanted to (and go with an order-only prerequisite on the directory instead) but I'm not sure the effort is worth the savings in execution cost. You could avoid the extra shell by combining the two lines mkdir -p $(#D) && python2 $< > $# if you wanted to though.
Please note that the second time you run the make, it will give you the message (if there are no new files):
make: Nothing to be done for `all'.
Try to run make clean and see if you get the same message.
Here is the Makefile which will do what you want:
SOURCES := $(wildcard src/*/*.py)
TARGETS := $(patsubst src/%.py,templates/%.json,$(SOURCES))
FOLDERS := $(sort $(dir $(TARGETS)))
all: $(TARGETS)
clean:
rm -rf $(TARGETS) $(FOLDERS)
$(FOLDERS):
mkdir -p $#
$(TARGETS): $(SOURCES) $(FOLDERS)
python2 $< > $#
The FOLDERS variable will contain the folders you need to create in the template directory. (sort will remove duplicates, so each folder will be there only once)
The $(FOLDERS) rule will create the folders.
The clean rule will remove the folders also.
If you need to add more sources, just do it like this:
SOURCES := $(wildcard src/*/*.py)
SOURCES += $(wildcard src/*.py)
...
Following on from Build script to Makefile, which lives in this upstream location. I want to include the Javascript examples that are included into this generated HTML document as dependencies.
INFILES = $(shell find . -name "index.src.html")
OUTFILES = $(INFILES:.src.html=.html)
TEMP:= $(shell mktemp -u /tmp/specs.XXXXXX)
all: $(OUTFILES)
# Problem line:
%.html: %.src.html $(wildcard contacts/*js)
#echo Dependencies: $^
cd $(#D) && m4 -PEIinc index.src.html > $(TEMP)
anolis --max-depth=3 $(TEMP) $#
rm -f $(TEMP)
clean:
rm -f $(OUTFILES)
PHONY: all clean
I want $(wildcard contacts/*js) to be $(wildcard $(#D)/*js) or $(wildcard $(dirname %)/*js), but nothing I've tried works. There must be some sort of keyword to get the parent directory of the target or dependency so I can reference the javascript dependencies.
AFAIK, using $(#D) and other automatic variables inside list of prerequisites can only be achieved using secondary expansion feature of GNU Make.
Thus, your problem probably can be solved as follows:
.SECONDEXPANSION:
%.html: %.src.html $$(wildcard $$(#D)/*js)
However, I'm not sure whether it will work with pattern rules.
I have a Makefile for a C++ program that uses automatic dependency generation. The %.d recipe is taken from the GNU Make manual.
The problem is that somehow "Makefile" is being added as a target and then an implicit rule is causing it to assume it's an executable and using my src/%.cpp rule to try to compile src/Makefile.cpp. When looking at the debug info, this always happens right after the include is executed.
No need to remake target `build/Sprite.d'.
Considering target file `Makefile'.
Looking for an implicit rule for `Makefile'.
...
Trying pattern rule with stem `Makefile'.
Trying implicit prerequisite `Makefile.o'.
Looking for a rule with intermediate file `Makefile.o'.
I know include causes the given Makefiles to be rebuilt if necessary. Does it also try to rebuild the current Makefile? If so how do I stop it, and if not, then why is "Makefile" being added as a target?
Also, the include is executed, causing the .d files to be remade even if I specify a target on the command line, such as make clean. Is there any way to stop that from happening?
# $(call setsuffix,newsuffix,files)
# Replaces all the suffixes of the given list of files.
setsuffix = $(foreach file,$2,$(subst $(suffix $(file)),$1,$(file)))
# $(call twinfile,newdir,newsuffix,oldfile)
# Turns a path to one file into a path to a corresponding file in a different
# directory with a different suffix.
twinfile = $(addprefix $1,$(call setsuffix,$2,$(notdir $3)))
MAIN = main
SOURCE_DIR = src/
INCLUDE_DIR = include/
BUILD_DIR = build/
SOURCES = $(wildcard $(SOURCE_DIR)*.cpp)
OBJECTS = $(call twinfile,$(BUILD_DIR),.o,$(SOURCES))
DEPENDENCIES = $(call twinfile,$(BUILD_DIR),.d,$(SOURCES))
CXX = g++
LIBS = -lpng
CXXFLAGS = -I $(INCLUDE_DIR)
.PHONY: all
all: $(MAIN)
$(MAIN): $(OBJECTS)
$(CXX) $(LIBS) $^ -o $(MAIN)
include $(DEPENDENCIES)
%.o: $(BUILD_DIR)stamp
$(CXX) $(CXXFLAGS) -c $(call twinfile,$(SOURCE_DIR),.cpp,$#) -o $#
$(BUILD_DIR)%.d: $(SOURCE_DIR)%.cpp $(BUILD_DIR)stamp
# echo Generate dependencies for $ $#.$$$$; \
sed 's,\($*\)\.o[ :]*,$(BUILD_DIR)\1.o $# : ,g' $#; \
rm -f $#.$$$$
$(BUILD_DIR)stamp:
mkdir -p $(BUILD_DIR)
touch $#
.PHONY: clean
clean:
rm -rf $(BUILD_DIR)
.PHONY: printvars
printvars:
# echo $(SOURCES)
# echo $(OBJECTS)
# echo $(DEPENDENCIES)
Make will always try to remake the Makefile before executing the Makefile. To do so, make will look for rules which can be used to recreate the Makefile. Make will look for quite a few implicit rules and other obscure methods to (re)create the Makefile.
In your case, make somehow decided that the pattern rule %.o: $(BUILD_DIR)/stamp should be used to recreate the Makefile, which failed.
To prevent make from remaking the Makefile you can write a rule with an empty recipe:
Makefile: ;
Read the chapter Remaking Makefiles in the make manual for more explanation.
About the included Makefiles: Included Makefiles will always be included, regardless of the target. If the included makefiles are missing (or older than their prerequisites) then they will first be (re)created. That means a make clean will first generate the .d Makefiles, only to delete them again.
You can prevent the including for specific goals by wraping the include directive in a conditional:
ifneq ($(MAKECMDGOALS),clean)
include $(DEPENDENCIES)
endif
Here is your entire Makefile with some fixes. I marked the places where I changed something.
# Makefile
# $(call setsuffix,newsuffix,files)
# Replaces all the suffixes of the given list of files.
setsuffix = $(foreach file,$2,$(subst $(suffix $(file)),$1,$(file)))
# $(call twinfile,newdir,newsuffix,oldfile)
# Turns a path to one file into a path to a corresponding file in a different
# directory with a different suffix.
twinfile = $(addprefix $1/,$(call setsuffix,$2,$(notdir $3)))
MAIN = main
SOURCE_DIR = src
INCLUDE_DIR = include
BUILD_DIR = build
SOURCES = $(wildcard $(SOURCE_DIR)/*.cpp)
OBJECTS = $(call twinfile,$(BUILD_DIR),.o,$(SOURCES))
DEPENDENCIES = $(call twinfile,$(BUILD_DIR),.d,$(SOURCES))
CXX = g++
LIBS = -lpng
CXXFLAGS = -I $(INCLUDE_DIR)
.PHONY: all
all: $(MAIN)
$(MAIN): $(OBJECTS)
$(CXX) $(LIBS) $^ -o $(MAIN)
# -------> only include if goal is not clean <---------
ifneq ($(MAKECMDGOALS),clean)
include $(DEPENDENCIES)
endif
# ---------> fixed this target <--------------
$(BUILD_DIR)/%.o: $(SOURCE_DIR)/%.cpp $(BUILD_DIR)/stamp
$(CXX) $(CXXFLAGS) -c $(call twinfile,$(SOURCE_DIR),.cpp,$#) -o $#
# ---------> and this target <---------------
$(BUILD_DIR)/%.d: $(SOURCE_DIR)/%.cpp $(BUILD_DIR)/stamp
# echo Generate dependencies for $#;
#set -e; rm -f $#; \
$(CC) -M $(CPPFLAGS) $< > $#.$$$$; \
sed 's,\($*\)\.o[ :]*,$(BUILD_DIR)\1.o $# : ,g' < $#.$$$$ > $#; \
rm -f $#.$$$$
$(BUILD_DIR)/stamp:
mkdir -p $(BUILD_DIR)
touch $#
.PHONY: clean
clean:
rm -rf $(BUILD_DIR)
.PHONY: printvars
printvars:
# echo $(SOURCES)
# echo $(OBJECTS)
# echo $(DEPENDENCIES)