Suppose I have a rule to build a big target and takes too long to run.
It does not have any dependencies.
For example, it could be a task to download some content from the Internet, like a web crawler that downloads lots of pages for later processing.
I want to generate the big target only if its latest run was more than 1 day/hour/minute ago. So the task has a TTL, time to leave.
I can accomplish it via this make file below.
If one makes the "all" target, then the big_target and small_target will only be made if it has passed more than 5 seconds since the last time they were built.
Does anyone has another suggestion or a canonical way of doing this with make?
.PHONY: all update_times
all: update_times big_target small_target
update_times:
#for f in TTL_* ;\
do\
seconds=$${f##TTL_};\
if (( `date +%s` - `date +%s -r $$f` > $${seconds} ));\
then\
echo "$$f is too old";\
echo $$(( `date +%s` - `date +%s -r $$f` )) ;\
rm $$f;\
else\
echo "$$f is up to date";\
fi;\
done
TTL_%:
touch $#
big_target: TTL_5
touch big_target
small_target: big_target
touch small_target
Your suggested method has some things I don't like: most prominently, it has an implied order relationship between the update_times and big_target targets, which is generally to be avoided (relationships in makefiles really need to be explicit, not implied).
I don't think there's any need for a different target. Maybe something like this would work better?
(PS. TTL is generally "time to live", not "time to leave"):
big_target_TTL := 5
.PHONY: all
all: big_target small_target
big_target:
secs=$$(( `date +%s` - `date +%s -r $#`)); \
if [ $$secs -gt $($#_TTL) ]; \
then \
echo "$# is too old"; \
echo $$secs; \
touch $#; \
else \
echo "$# is up to date"; \
fi
small_target: big_target
touch small_target
Related
I am writing a complicated Makefile which have multiple parts like:
step1:
if [[ ${input} == "delete" ]]; then \
echo "this is a test to delete files"; \
else \
echo "error stop"; \
fi
step2:
rm -f *.txt
test:
make step1
make step2
So if I enter "make test input=delete", it will delete all the .txt files. And if I enter "make test input=none", it would not do anything. I know the simplest way is to combine step1 and step2 as:
test:
if [[ ${input} == "delete" ]]; then \
echo "this is a test to delete files"; \
rm -f *.txt; \
else \
echo "error stop"; \
fi
But my Makefile is so complicated that I have to separate into several parts. Does Makefile support similar features? If yes, what should I look for? Thanks.
You should always use $(MAKE) and never use make directly when invoking a sub-make.
If you do that, then all command-line overrides you provide will be correctly passed to the sub-makes.
BTW, you should not use the bash syntax [[ x == y ]]. If you run this makefile on a system where the default shell is limited to POSIX standard syntax this won't work. You should use POSIX syntax: [ x = y ].
As you use GNU Make you can use its conditionals, e.g.
ifeq:
test:
ifeq ($(input),delete)
echo "this is a test to delete files"
rm -f *.txt
else
echo "error stop"
endif
Or, if you want to control the prerequisites of a target:
step1:
echo "this is a test to delete files"
step2:
rm -f *.txt
# test always depends on step1
test: step1
# test also depends on step2, if input = delete
ifeq ($(input),delete)
test: step2
endif
I have created a parent makefile. as below:
SHELL = /bin/bash
HOMEDIR = $(shell pwd)
PKGNAM = PARAMETIS
override VERSION = 4.0.3
YESDIR = $(shell echo $(#:install-%=%) | tr A-Z a-z)
NODIR = $(shell echo $(#:clean-%=%) | tr A-Z a-z)
install:
$(MAKE) install-$(VERSION)
install-%:
#if [ ! -e $(YESDIR) ]; then \
echo "Library $(PKGNAM) Version=$(YESDIR) does not exist"; \
elif [ -e $(YESDIR)/Install.sh ]; then \
echo "Installing $(PKGNAM) version=$(YESDIR)" ; \
cd $(YESDIR) ;\
$(SHELL) Install.sh $(HOMEDIR) 1 ;\
elif [ -e $(YESDIR)/Makefile ]; then \
cd $(YESDIR); \
$(MAKE); \
else \
echo "Installation instruction for $(#:install-%=%) Version=$(YESDIR) does not exist"; \
fi;
clean:
#$(MAKE) clean-$(VERSION)
clean-%:
#if [! -e ${NODIR} ]; then ;\
echo "Library does not exist $(PKGNAM) version=$(NODIR)" ; \
else \
cd $(NODIR) ;\
echo "Installing $(PKGNAM) version=$(NODIR)" ; \
$(SHELL) Install.sh $(HOMEDIR) 0 ;\
fi;
This makefile calls different bash files inside each version of the libraries directories to build them, the bash files can successfully build each library if I call it from the terminal, tho when I call them from my make file using,
make install
after it executes the install.sh and build the library, I get this error that
No rule to make target 'w'. Stop.
any idea why it happens and how can I get rid of it ?
HERE is the bash file if it helps:
if (test $2 = 1) then
make --silent -f Makefile config prefix=$1/exec
make --silent -f Makefile
make --silent -f Makefile install
elif (test $2 = 0) then
make --silent -f Makefile clean
fi
Thanks
The problem was caused by calling other Makefiles using -C in the library's makefile called in the Install.sh. This sets the MAKEFLAGS to w automatically. unfortunately, the developers of the library has made mistake in calling the makefiles as below:
$(MAKE) -C $(SUBDIR) $# $(MAKEFLAGS)
when the makefile is called from another makefile, this MAKEFLAGS are set to w, but in the bash called from the terminal they are empty. because developers have forgotten to add MAKEFLAGS= before the flags, it assumes that w is another target and because it is not defined it generates the error, I menotiond.
I solved the issue by changing their makefile as below:
$(MAKE) -c $(SUBDIR) $# MAKEFLAGS=$(MAKEFLAGS)
and now everything works as expected.
I have a Makefile with multiple jobs.
Each job does a mkdir -p $(OBJDIR), to ensure the object directory exists before running gcc.
However, the mkdir -p sometimes does not ensure the directory exists before returning. I assume there is some sort of race condition going on. And then the compiler fails, as it can't write to the output file.
Is there a platform agnostic way of doing the mkdir -p so that this race condition doesn't occur?
This failure seems to only happen under OS X.
Question got sent to Stack Overflow, but really a sysadmin question.
# script named 'test'
if [ "$1" == "run" ] ; then
rm -rf XXX
NOW=`date +%s`
echo Now: $NOW
THEN=`echo "$NOW + 10" | bc`
RUNAT=`date -r $THEN +"%m%d%H%M.%S"`
echo $RUNAT
for i in {1..75}; do
./test $THEN &
sleep 0.05
done
wait
exit 0
fi
THEN=$1
# make sure things are cached
NOW=`python -c'import time; print repr(time.time())'`
NOW=`python -c'import time; print repr(time.time())'`
AMOUNT=`echo "$THEN - $NOW" | bc`
echo sleeping $AMOUNT
sleep $AMOUNT
mkdir -p XXX/YYY/ZZZ
touch XXX/YYY/ZZZ/hi
I wrote the above bash script, but it doesn't produce the predicted error. I will put the touch in the makefile, perhaps it will elucidate what is going on.
The makefile's CC statement looks like so:
$(OBJPATH)/%.o : %.cpp
mkdir -p $(dir $#)
#sleep 0.1
$(CPP) $(INCPATH) $(_FLAGS) $(CPPFLAGS) $< -o $#
The #sleep statement is there to possibly wait for the race condition to fix itself, but perhaps there is no race condition after all.
I have a Makefile that does performs a task if it hasn't happened in the last hour. It does so like this:
HOUR_FROM_NOW = $(shell perl -e '($$s,$$m,$$h,$$d,$$M)=localtime(time()+3600); printf("%02d%02d%02d%02d\n",$$M+1,$$d,$$h,$$m);')
NOW_FILE = $(shell mkdir -p .make; touch .make/now; echo .make/now )
.PHONY: externals
externals: $(PROJECTS:%=.make/proj_%)
.make/proj_%: $(NOW_FILE)
$(MAKE) -s $(*F)
touch -t $(HOUR_FROM_NOW) $#
.PHONY: $(PROJECTS)
$(PROJECTS):
# do stuff, specifically, clone git-repo if not exists, else pull latest
That part works great, except that I now get warnings:
make: Warning: File `.make/proj' has modification time 3.5e+03 s in the future
make: Nothing to be done for `externals'.
make: warning: Clock skew detected. Your build may be incomplete.
Anyone know how to suppress those warnings? (Or to do a periodic task in a makefile)
Most versions of touch I have come across can do some date time maths which allows for setting the timestamp of a file directly via the --date option.
That and the fact that variables assigned with := are only "evaluated once" makes this a bit easier to read.
HOUR_AGO := .make/hour_ago
__UGLY := $(shell mkdir -p .make && touch --date='1hour ago' $(HOUR_AGO))
# The preceding line will be executed once
.make/proj_%: .make/hour_ago | .make
$(MAKE) -s $(*F)
#touch $#
.make:
mkdir -p $#
I'm using something very similar to this to periodically refresh login tokens.
Never would have thought of it if it wasn't for Dave's answer though.
The directory is created by specifying it as a order-only-prerequisite
I suspect that the + 3600 is at fault. What happens if you remove it?
I thought and thought, and then the stupid-obvious solution hit me ...
Instead of setting timestamps in the future with HOUR_FROM_NOW, I use the real time and compare with HOUR_AGO_FILE ...
HOUR_AGO = $(shell perl -e '($$s,$$m,$$h,$$d,$$M)=localtime(time()-3600); printf("%02d%02d%02d%02d\n",$$M+1,$$d,$$h,$$m);')
HOUR_AGO_FILE = $(shell mkdir -p .make; touch -t $(HOUR_AGO) .make/hour_ago; echo .make/hour_ago )
.PHONY: externals
externals: $(PROJECTS:%=.make/proj_%)
.make/proj_%: $(HOUR_AGO_FILE)
$(MAKE) -s $(*F)
#touch $#
I'm completely stumped on how to do this in a Makefile
Let's say I have a target. Inside the target I have a loop. How do i change a variable to keep track of the iterations?
For example:
COUNTER = 0
target:
(loop){
COUNTER++
echo COUNTER
}
I know that variables in Makefiles are only expanded, and I'm not sure if they can be permanently changed, but there has to be a way to do this, right? :(
Here are some sources that are asking similar questions. It seems like those examples only change the variable temporarily:
How do I perform arithmetic in a makefile?
How to do arithmetic operation in makefile?
Doing simple math in Makefile
Maybe I have to use the eval function somehow?
Maybe I have to append onto a Makefile string a character each time and then use something in the shell to count the characters?
If the variable doesn't have to survive the rule, this should do (I'm assuming bash):
clean:
#n=0 ; \
for x in $(THINGS_TO_BE_DELETED); do \
if [ -f $$x ] ; then \
rm $$x; \
let "n+=1" ; \
fi ; \
done ; \
echo deleted $$n files;
Here is one solution: Write a simple script like this:
#!/bin/bash
count=`cat count.txt`
count=$((count + 1))
echo $count
cat $count > count.txt
Initialize the file by doing
$ echo "0" > count.txt
Then include it as a .PHONY requirement to build whatever you'd like.
This is similar to the accepted answer, but the syntax below should work with a POSIX compliant shell. Quotes should also be used inside of the test.
clean:
#n=0; \
for x in *.a *.b *.c ; do \
if [ -f "$$x" ]; then \
rm "$$x"; \
n=$$((n+1)); \
fi; \
done; \
echo deleted $$n files;
Note: tabs must be used for indentation