My project is dependent on other projects in subfolders. I want to call make recursively to build all in one run.
My first try is like this:
.PHONY: all make_recursion
all: program
program: submake1/lib submake2/lib
submake1/lib: make_recursion
submake2/lib: make_recursion
make_recursion:
make -C submake1
make -C submake2
This does not work as intended. Each run of make both submakes are executed in case a file changed in those projects. For submake2 my program is also rebuilt if submake2/lib was updated. If only submake1/lib is updated I have to run make a second time to update program.
The reason for this is that make takes the timestamp of submake1/lib before make_recursion, but does not update it after that. How can I improve this?
Related
I have a situation where I need to execute a rule in make depending on a remote file. This is an example of what I'm trying to do (the actual Makefile is a lot more complicated in ways that aren't relevant here):
URL = http://cdn.sstatic.net/stackoverflow/img/favicon.ico
stackoverflow.png: favico.ico modified | check_modified
convert favicon.ico $#
check_modified: modified
#echo Icon was modified. Downloading.
rm -f favicon.ico
wget $(URL)
.PHONY: check_modified
favico.ico: check_modified
modified:
touch -d "`wget --spider -S $(URL) 2>&1 | sed -n 's/.*Modified: //p'`" $#
The idea is:
The rule to build modified should always be run. The function of that rule is to change the modification time of the file to be the same as the modification time of what the URL points to.
After that, I want modified to behave normally in terms of dependencies. If modified is more recent than favicon.ico, I want to retrieve the new file and then let the dependency on it cause the target file to be remade.
As a wrinkle, in some applications, I have to retrieve the file manually. In those cases, I want to just have a rule that fires and tells me to manually download the file, but that doesn't otherwise affect building the target. As an example, if the source image that I'm converting were behind a site login, I'd need to manually login to the site and download it to a fixed location and then rerun make.
Everything I've tried either:
Fails to check the URL if the target is up to date, or
Always checks the URL and rebuilds the target, even if modified is not more recent than the target.
Any words of wisdom?
Your problem is probably that make builds the dependency graph before running any recipe. So, when your modified tag file is created / updated, it is too late for make to consider its last modification date and use it to decide whether other targets shall be built or not.
Your first observed behaviour (fails to check the URL if the target is up to date) probably corresponds to attempts where you did not use the .PHONY special target. As soon as modified has been created, make always considers it as up-to-date because it has no prerequisites and thus cannot be outdated.
The second behaviour (always checks the URL and rebuilds the target, even if modified is not more recent than the target) corresponds to what you posted: check_modified is a prerequisite of the .PHONY special target, which forces make to always consider it as outdated. stackoverflow.png and favico.ico, directly or not, depend on check_modified and are thus also always considered as outdated.
One option to solve your problem would be to use a recursive, two-passes, make invocation. On a first run make would build the all phony target (always), update a modified tag file, and then invoke itself again to build other targets that have the tag file as prerequisite and are outdated. As, for the second invocation, make rebuilds its dependency graph, it will take the last modification date of the tag file into account. Something like:
URL = http://cdn.sstatic.net/stackoverflow/img/favicon.ico
.PHONY: all
all:
touch -d "`wget --spider -S $(URL) 2>&1 | sed -n 's/.*Modified: //p'`" modified
$(MAKE) stackoverflow.ico
stackoverflow.ico: modified
#echo Icon was modified. Downloading.; \
rm -f favicon.ico; \
wget $(URL); \
convert favicon.ico $#
Explanations:
I replaced your conversion to png by a useless conversion to ico because the favicon.ico icon of SO is composite and its conversion to png creates two files instead of one, named stackoverflow-0.png and stackoverflow-1.png, which uselessly complicates things.
all is a true phony target and the default goal so, each time you invoke make (or make all), it is built. It first updates the modified tag file and then invokes make again to build stackoverflow.ico.
The second make invocation does nothing if stackoverflow.ico is more recent than modified, else it downloads and converts.
Your second question (get a message about required manual operations) is completely different and simpler to solve. Let's first define a message and echo it in the file's recipe:
define DIY_message
Dear user, you should first:
- do this
- and that.
Unless you know it is useless, of course.
endef
export DIY_message
the_remote_file:
#echo "$$DIY_message"
The message will be printed if make is invoked with this goal (make the_remote_file) or with a goal that somehow depends on the_remote_file and either:
the_remote_file does not exist,
or the_remote_file exists but is out-of-date with respect to its prerequisites (if you declared prerequisites for it),
or the_remote_file is a prerequisite of .PHONY.
Note: using an intermediate make variable assigned by a define-endef makes things easier for formatted multi-lines messages.
Consider the following make workflow, in which I (a) want to be sure to always work on the newest download.zip file, but (b) only run the rest of the workflow when this file changes:
my_output.file: download.zip
some_expensive_operation
download.zip:
wget -N http://some.server/download.zip
In this example, wget has got dependency information that make does not have.
If download.zip is present, the rule will never be executed and wget will never check if there is a newer file available.
I can of course make .PHONY: download.zip to have the rule executed every time, by then some_expensive_operation will be executed as well no matter if the file changed or not.
Is there any way to tell make to run the my_output.file rule above only if download.zip actually changed?
Use the force rule trick:
my_output.file: download.zip
some_expensive_operation
download.zip: FORCE
wget -N http://some.server/download.zip
FORCE: ;
(or you could declare .PHONY: FORCE if you prefer). This ensures that the recipe for download.zip is always run, but it's not marked phony itself so targets that depend on it won't be rebuilt unless it's changed.
This is for setting up the application bundle of a MacOSX app. I have a script which copies a few files and does some other things. So I want to execute the script after the build (i.e. after the linking step). I want it to be executed just every time because it is not possible to specify its dependencies.
I know there is QMAKE_POST_LINK (e.g. described here or here) but it runs only when the target does not exists, i.e. when linking needs to be done. However, I want the script to run every time, also when the target already exists.
There is also QMAKE_EXTRA_TARGETS and POST_TARGETDEPS (e.g. described here) but that forces a relink all the time but I actually only want the script to rerun and it runs the script before the linking. (Currently, that's what I'm using using anyway, because I don't see a better way. Here is my QMake source.)
There is related questions there and there. I quote my answer for first of them:
Another way to make things in given order is to use empty "super"
target:
super.depends = target_pre first target_post
QMAKE_EXTRA_TARGETS += super
Where first - is default qmake target, and target_pre and
target_post some custom targets. Now make super just do the thing.
EDIT: looks like in last versions of Qt build of dependencies is running in paralell so this solution wouldn't work.
I've scratched my head about this for a few man-days over the past several months, and I haven't yet found a "pure" solution. However, FWIW, if you don't mind the hack of forcing relink every time, here's how to do that:
(This implementation of "post-build-events" is similar to this implementation of "pre-build-events".)
Caveats:
Forces relink every time
Works only for projects that have a linking step, so, not
TEMPLATE=aux or TEMPLATE=subdirs.
FORCELINK_CPP_FILE = force_link.cpp
#This batch of statements causes the dummy file to be touched each build.
forcelink.target = $$FORCELINK_CPP_FILE
#FORCE is a nonexistent target, which will cause Make to always re-execute the recipe.
forcelink.depends = FORCE
forcelink.commands = touch $$FORCELINK_CPP_FILE
QMAKE_EXTRA_TARGETS += forcelink
#This statement ensures that touching the above file at Make time will force relinking.
SOURCES += $$FORCELINK_CPP_FILE
#QMake will complain unless the file actually exists at QMake time,
# too, so we make sure it does.
#I used to touch this on QMake build_pass runs, too,
# but it caused transient access-denied errors.
# I guess the release and debug makefiles are generated in parallel.
!build_pass : write_file($$FORCELINK_CPP_FILE)
I have a somewhat complicated Makefile which runs perl scripts and other tools and generates some 1000 files. I would like to edit/modify some of those generated files after all files are generated. So I thought I can simply add a new rule to do so like this:
(phony new rule): $LIST_OF_FILES_TO_EDIT
file_modifier ...
however, the point here is some of those generated files which I'd like to edit ($LIST_OF_FILES_TO_EDIT) are used in the same make process to generate a long list of files. So I have to wait to make sure those files are no longer needed in the make process before I can go ahead and edit them. But I don't know how to do that. Not to mention that it is really hard to find out what files are generated by the help of $LIST_OF_FILES_TO_EDIT.
If it was possible to mention in the Makefile that this rule should be only run as the last rule, then my problem would be solved. but as far as I know this is not possible. So anyone has an idea?
Some points:
List of files to edit ($LIST_OF_FILES_TO_EDIT) is determined dynamically (not known before make process)
I am not sure I have picked a good title for this question. :)
1) If you're going to modify the files like that, it might behoove you to give the targets different names, like foo_unmodified and foo_modified, so that the Make's dependency handling will take care of this.
2) If your phony new rule is the one you invoke on the command line ("make phonyNewRule"), then Make will build whatever else it's going to build before executing the file_modifier command. If you want to build targets not on that list, you could do it this way:
(phony new rule): $(LIST_OF_FILES_TO_EDIT) $(OTHER_TARGETS)
file_modifier ...
3) If your dependencies are set up correctly, you can find out which targets depend on $(LIST_OF_FILES_TO_EDIT), but it's not very tidy. You could just touch one of the files, run make, see which targets it built, repeat for all files. You could save a little time by using Make arguments: "make -n -W foo1 -W foo2 -W foo3 ... -W foo99 all". This will print the commands Make would run-- I don't know of any way to get it to tell you which targets it would rebuild.
Part of my Makefile:
CPUDEPS=./mydeps.cpu
(...)
deps: $(CPUDEPS)
$(CPUDEPS): $(CCFILES)
#echo [DEPS] CPU
$(CMDECHO)makedepend -Y -s'# CPU sources dependencies generated with "make deps"' \
-w4096 -f- -- $(CFLAGS) -- $^ 2> /dev/null > $(CPUDEPS)
(...)
sinclude $(CPUDEPS)
Problem 1: includes are done during the first phase of processing, targets during the second phase; so, if ./mydeps.cpu doesn't exist and I "make deps", I get first the error
Makefile:335: ./mydeps.cpu: No such file or directory
I hide the error using sinclude instead of include, but the problem is still there: the old file is included, not the just-generated-one. Have to run it twice to include the updated file. This is because make does a two-phase processing; is there any way to tell make to complete the target deps before parsing the includes?
Problem 2: even if the file ./mydeps.cpu doesn't exist and make deps actually creates it, I always get a "make: Nothing to do for deps". This doesn't happen with other targets. I don't understand why and how to avoid it.
Problem 1 is non-existant: before building a target, make automatically rebuilds makefiles (with implicit rules if no explicit rule is provided). So having a rule for the makefile ensures that will always be up to date, there is no need to run deps twice. Additionally, since CPUDEPS is a makefile, it will be updated automatically before any other rule is run, so dependencies will always be updated if necessary and make deps is not needed. You can probably notice this by yourself by observing the [DEPS] line being echoed if any of the CCFILES becomes more recent that the dependency file.
For Problem 2, adding anything to the recipe ensures that make doesn't complain about having nothing to do. If there is nothing else, you can use something like #echo OK to give feedback to the user, or a simple #true if you prefer totally silent makes.
What you are trying to achieve is useless: you can use the dependencies file that was created during the previous build. That's enough.
The main reasoning behind that rule is:
if you haven't changed any of your files, then the dependencies file is up-to-date, and there's nothing to build.
if you have changed anything, even very deep into your #include chain, on an existing file that were used by previous build, then the dependencies file have already caught it. You'll rebuild what is needed.
if you change something in a new file (you add that file!) then it was not used by previous build, and not listed in dependencies. But if you really want to use it, then you have to modify at least one of your other files that was used before, and you're back on the previous case.
The solution is to create the dependencies file during the normal process of the compilation, and to optionally include it (with sinclude) if it is present.