I am very new to makefile. Here i have challenge in finding time for the compiling(c code) each module.
Operating system:Linux
make : x86_64-redhat-linux-gnu
I am using "make" with -j option.only one Makefile maintained having instruction "/usr/bin/make".
And recursivley there are multiple Components are there .For each component one Config.mk maintained.
some modules doesnt have ".mk" also
I tried to echo time in each ".mk" using "{info {shell date}),but it didn't give enough results,since they all been used in very early stage of compilation..
Could anyone help in finding time for each make operation
Cheers
Murali
You can .PHONY targets that are executed as first and last targets and print out the date
all:
echo "Component foo: build started"
date
$(MAKE) build
echo "Component foo: build finished"
date
build: ....
....
Related
How to integrate an external, black-box build system like go build into Make?
I don't think this question is specific to Go, but I will use it as an example. The go build system tracks the relation between inputs and outputs (= the dependencies) internally, and avoids rebuilding the output if no inputs have changed.
I have a Makefile which contains targets based on shell scripts and targets based on invoking go build, for example:
my-exe:
go build <some-url>
intermediate: my-exe
<expensive shell script>
test: intermediate test-data
<some test>
Requirements:
Let the make goal be test.
When I touch the test-data, I don't want to run the intermediate target.
When I touch the go source code, I want to run all steps.
Options considered:
It is possible to list the go source files as dependencies of my-exe. However, my go source folder contains files for multiple targets, and I would have to somehow list the right files/folders in my Makefile. I could also overshoot and list all go source files as dependencies in the Makefile.
If I turn my-exe into a phony target, then requirement 2. is fulfilled but 1. is broken.
Renaud's solution will work. But as in my comment above, I think all you need to do is take advantage of the FORCE target trick described here: https://www.gnu.org/software/make/manual/html_node/Force-Targets.html
Change your makefile to this:
FORCE:
my-exe: FORCE
go build <some-url>
intermediate: my-exe
<expensive shell script>
test: intermediate test-data
<some test>
By adding a prerequisite that can never be satisfied to my-exe you will force it to always be built. Assuming that the go build ... command doesn't actually update the my-exe target if nothing changed and does update it when something changed, this makefile will work the way you want.
Let's abstract all this a bit with 2 targets: always and expensive. We want to always build always because it is cheap and it relies on a external build system. But we want to build expensive only if building always changed something. The solution consists in declaring always as phony, such that it is always remade, but not declaring it as a prerequisite of expensive, else it will always be remade too. We thus need a third target, relay that is not phony, that is also cheap and that will really change only if always did something.
In the following the GO make variable is used to emulate the external build system. If it is set to a non-empty string always changes, else it stays the same.
expensive: relay
#echo "$#"
#touch "$#"
relay: always
#echo "$#"
#if ! [ -f "$#" ] || [ "$<" -nt "$#" ]; then touch "$#"; fi
.PHONY: always
always:
#echo "$#"
#if ! [ -f "$#" ] || [ -n "$(GO)" ]; then touch "$#"; fi
And that's it:
$ make
always
relay
expensive
$ make
always
relay
$ make GO=1
always
relay
expensive
If we turn the external build step into a phony target, it will always be built. However, any target which depends on a phony target is also always built. GNU make documentation:
A phony target should not be a prerequisite of a real target file; if it is, its recipe will be run every time make goes to update that file.
There are two ways to use this:
Make the external build step depend on a phony target:
.PHONY: always-rebuild
my-exe: always-rebuild
go build -o my-exe <some-url> # creates my-exe
intermediate: my-exe
<expensive shell script>
test: intermediate test-data
<some test>
(GNU) make evaluates the timestamp of my-exe after the target my-exe has run. If the target did not change the timestamp, then the succeeding steps (intermediate, test) are not run.
Introduce a dummy target to remove the "phony" property:
.PHONY: external_my-exe
external_my-exe:
go build -o my-exe <some-url> # creates my-exe
my-exe: external_my-exe
# do nothing!
true
intermediate: my-exe
<expensive shell script>
test: intermediate test-data
<some test>
external_my-exe is always built (if it occurs in the dependency tree of the make goal).
my-exe is always built because it depends on a phony target.
intermediate depends on an actual file (my-exe), so it is only run if the file's timestamp changed.
TL; DR
A dependency shared by multiple targets is only invoked once.
Why?
Summary
In this tiny Makefile example, I expect deploy-everywhere to deploy staging-repo code to staging, and prod-repo code to prod.
Instead, both staging and prod get staging-repo code, which is, admittedly, suboptimal.
This seems to be happening because clone-repo is skipped / pruned when it is examined 2nd time around, as part of deploy-to-prod (see -d log output below).
Why is this happening, and can I change this behavior?
$ cat Makefile
.PHONY: deploy-everywhere deploy-to-prod deploy-to-staging clone-repo
deploy-everywhere: deploy-to-staging deploy-to-prod
#echo "deployed everywhere"
deploy-to-prod: repo="prod-repo"
deploy-to-prod: clone-repo
#echo "deployed to prod (from $(repo))"
deploy-to-staging: repo="staging-repo"
deploy-to-staging: clone-repo
#echo "deployed to staging (from $(repo))"
clone-repo:
#echo "clone-repo $(repo)"
More Details (logs, version)
Here is the output from running make:
$ make deploy-everywhere
clone-repo staging-repo
deployed to staging (from staging-repo)
deployed to prod (from prod-repo) <<<<< THIS IS A LIE, WE NEVER CLONED THE PROD REPO!!
deployed everywhere <<<< RIIIIGHT.
Extra logging:
$ make -d deploy-everywhere
...
Successfully remade target file `deploy-to-staging'.
Considering target file `deploy-to-prod'.
File `deploy-to-prod' does not exist.
Pruning file `clone-repo'. <<<< WHY? WHY?!?!?!?! WHY.
Finished prerequisites of target file `deploy-to-prod'.
Must remake target `deploy-to-prod'.
...
I am running GNU Make 3.81:
$ make --version
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program built for i386-apple-darwin11.3.0
This is by design. Targets in a makefile are more like nouns than like verbs.
You can achieve what you want several ways. Here is one:
deploy-to-prod: clone-prod-repo
#echo "deployed to prod (from prod-repo)"
deploy-to-staging: clone-staging-repo
#echo "deployed to staging (from staging-repo)"
clone-%-repo:
#echo "clone $*-repo"
EDIT: All right, the clone operation takes multiple parameters. Here is one way to do it:
deploy-to-prod: PARAM1=ProdOne
deploy-to-prod: PARAM2=ProdTwo
deploy-to-prod: clone-prod-repo
#echo "deployed to prod (from prod-repo)"
deploy-to-staging: PARAM1=StagingFirst
deploy-to-staging: PARAM2=StagingSecond
deploy-to-staging: clone-staging-repo
#echo "deployed to staging (from staging-repo)"
clone-%-repo:
#echo clone $*-repo with $(PARAM1) and $(PARAM2)
And here is another:
define clone-repo
#echo clone using $(1)
#echo and $(2) in some way
endef
deploy-to-prod:
#echo $#
$(call clone-repo, ProdOne, ProdTwo)
deploy-to-staging:
#echo $#
$(call clone-repo, StagingFirst, StagingSecond)
To add some explanation to Beta's solution: The programmatic contract of a make rule is that it is an opaque update of a permanent entity in the filesystem, thereby satisfying a "less-than" (younger-than) relation laid down in the dependency tree. Your construction OTOH hides the dependency from make by passing it as a parameter. The problem that you tried to circumvent is that there is no notion for "less-than" in make for non-filesystem-present entities (in your case, a revision control system).
The only question that I have is why you didn't simply add the clone call to each deploy target - if you really want to solve it as a "less-than" dependency (ie. cloning only if local repository non-existent or older) there is quite some make programming necessary.
EDIT after your first comment:
If I got you right, your idea of PHONY is a bit off (not totally, just a bit). PHONY is not about facilitating something across different make runs, it is a way to have a short-circtuit in your dependency tree. Given a chain foo -> bar -> phony_baz (foo depending on bar depending on phony_baz) make will, in the case that foo was the main target, first look at bar to see if it is older than phony_baz which always will be the case. This is because phony_baz itself has a make-implicit property that will always yield a "younger_than" when it is compared as a prerequisite against a target (and now comes the important part:) at the first encounter in the dependency tree. All subsequent encounters of phony_baz in other branches of the tree will do the exact opposite, terminating the evaluation right there at the phony target. This is no different from the behaviour at any other node in the dependency graph - make assumes that the contract which I noted above is fulfilled and no repetitive action is justified. This is the explanation for your clone-repo not firing a second time although it is .PHONY - your setup is effectively a directed acyclic graph where two edges join at the same node (which make is completely fine with, just worth noting), not a tree . Beta's first solution is to dynamically create two entirely unrelated nodes by parametrizing the rule, thereby returning the shape to a tree.
I know this has been asked before, but none of the solutions I've found work for me because they're anti-DRY.
I have a number of targets that depend on things that can't readily be timestamped -- such as files copied from another system. What I'd like to be able to do is list dependencies in a variable, like nobuild=this,that, and have those targets be assumed to be up-to-date. Since I have a lot of these, I don't want to ifdef around each one; what would be pseudocodibly preferable would be something like
ignorable-target: dependencies
$(call ifnobuild,$#)
.. rest of normal build steps ..
where the ifnobuild macro expanded to some sort of exit-from-this-recipe-with-success gmake instruction if ignorable-target was mentioned in the nobuild variable.
I also don't want to get into multi-line continued shell commands in order to defer the conditional to the recipe itself; I want to be able to tell make "Assume these targets are up-to-date and don't try to build them," so I can test other aspects with the local copies already obtained from the problematic recipes.
There isn't any sort of exit-recipe-with-success mechanism in gmake, is there?
[Edited to hopefully make the situation more clear.]
Here's an example. Targets remote1 and remote2 each involve using ssh to do something time-consuming on a remote system, and then copying the results locally. Target local1 is built locally, and isn't a time sink. target-under-work depends on all three of the above.
local1: local1.c Makefile
remote1: local1
scp local1 remote-host:/tmp/
ssh remote-host /tmp/local1 some-args # takes a long time
scp remote-host:/tmp/local1.out remote1
remote2: local1
scp local1 other-host:/tmp/
ssh other-host /tmp/local1 other-args # takes a long time
scp other-host:/tmp/local1.out remote2
target-under-work: local1 remote1 remote2
do-something-with remote1,remote2
Now, when I just run make target-under-work, it's going to run the recipes for remote1 and remote2. However, the local copies of those files are 'good enough' for my testing, so I don't want them run every time. Once things go into production, they will be run every time, but while I'm developing target-under-work, I just want to use the copies already built, and I can rebuild them daily (or whatever) for the necessary testing granularity.
The above is over-simplified; there are multiple steps and targets that depend on remote1 and/or remote2. I see how I can get the effect I want by making them order-only prerequisites -- but that would mean changing the dependency list of every target that has them as prerequisites, rather than making a single change to remote1 and remote2 so I can use some variable from the command line to tell their recipes 'pretend this has been built, don't actually build it if there's already a copy.'
I hope this makes my question more clear.
No, this early exit make feature does not exist.
Note that your problem is probably under-specified because you don't explain what behaviour you want when a slow target does not exist yet.
Let's assume that the slow targets listed in nobuild shall be rebuilt if and only if they don't exist. Instead of using make functions to early exit their recipe you could use make functions to "hide" their list of prerequisites. This way, if they already exist, they will not be rebuilt, even if they are outdated. The only subtlety is that you will need the second expansion to use the $# automatic variable in the lists of prerequisites. In the following example slow (your remoteX) depends on fast1 (your local1). fast2 (your target-under-work) depends on fast1 and slow:
host> cat Makefile
# Expands as empty string if $(1) exists and
# listed in $(nobuild). Else expands as $(2).
# $(1): target
# $(2): prerequisites
define HIDE_IF_NOBUILD
$(if $(wildcard $(1)),$(if $(filter $(1),$(nobuild)),,$(2)),$(2))
endef
nobuild :=
fast1:
#echo 'build $#'
#touch $#
fast2: fast1 slow
#echo 'build $#'
#touch $#
.SECONDEXPANSION:
slow: $$(call HIDE_IF_NOBUILD,$$#,fast1)
#echo 'build $#'
#touch $#
# Case 1: slow target not listed in nobuild and not existing
host> rm -f slow; touch fast1; make fast2
build slow
build fast2
# Case 2: slow target not listed in nobuild and existing and outdated
host> touch slow; sleep 2; touch fast1; make fast2
build slow
build fast2
# Case 3: slow target listed in nobuild and not existing
host> rm -f slow; touch fast1; make nobuild="slow" fast2
build slow
build fast2
# Case 4: slow target listed in nobuild and existing and outdated
host> touch slow; sleep 2; touch fast1; make nobuild="slow" fast2
build fast2
Is there a way to let make determine the number of files to be recompiled before actually compiling? The problem is this: Consider having a quite big project with hundreds of source files. It would very convenient to have a rough idea of how long compilation will take, but to know that, one needs to know the number of files to be compiled.
The general answer is no, because your build could generate files which themselves are inputs to other rules which generate more files. And so on. However if a rough answer is good enough you can try the --dry-run flag. From the GNU make documentation...
“No-op”. Causes make to print the recipes that are needed to make the targets up to date, but not actually execute them. Note that some recipes are still executed, even with this flag (see How the MAKE Variable Works). Also any recipes needed to update included makefiles are still executed (see How Makefiles Are Remade).
As you can see, despite its name even the --dry-run flag will change the state of your build.
"make -n" will do the dry run. But you can't get the list of files to be rebuilt. May be you can write shell script to store the last modified time of files and get the list of files.
I think a found a decent solution for unix. Here SRC are your source files, HDR your headers and DEP the dependency files (something like DEP:=$(OBJ:.o=.d) )
isInDepFile+=$(shell grep -q $(modifiedFile) $(depFile) 1>&2 2> /dev/null && echo $(depFile))
COMPFILES=
checkDepFiles=$(foreach depFile,$(DEP), $(eval filesToCompile+=$(isInDepFile))) $(thinOutDepFiles)
thinOutDepFiles=$(foreach fileToCompile,$(filesToCompile),$(eval DEP=$(filter-out $(fileToCompile),$(DEP))))
countFilesToCompile: $(SRC) $(HDR)
$(eval modifiedFiles=$?)
$(foreach modifiedFile,$(modifiedFiles), $(call checkDepFiles))
$(eval numOfFilesToCompile = $(words $(filesToCompile)))
$(eval numDepFiles = $(words $(DEP)))
$(eval NumSRCFiles = $(words $(SRC)))
#echo $(NumSRCFiles) sources
#echo $(numDepFiles) files to leave
#echo $(numOfFilesToCompile) files to compile
#touch $#
This first generates a list of modified files within your source and header files lists. Then for each modified file it checks all dependency files for its filename. If a dependency file contains the current file we are looking at, it is added to the list of filesToCompile. It is also removed from the list of dependency files to avoid duplication.
This can be invoked in the main building rule of your project. The advantage of that over the dry run is that it gives you a simple number to work with.
(this is similar to GNU make: Execute target but take dependency from file but slightly different).
When I try to force to build a target, make automatically thinks that this target is out of date and forces a run of all targets which depend on it.
In my case, the target is a recursive make call which does a lot of work and might just return with "nothing to be done":
.PHONY: intermediate.dat
intermediate.dat:
$(MAKE) expensive_chain_which_finally_creates_intermediate.dat
step1.dat: intermediate.dat
sleep 10
step2.dat: step1.dat
sleep 15
step3.dat: step2.dat
sleep 10
all: step3.dat
sleep 5
In this case, "make all" runs for 40 seconds although intermediate.dat might not have changed (recursive make returned "nothing to be done"). However, if the recursive make updated intermediate.dat, the target shall be out of date.
Is there really no way to do this?
Make intermediate.dat depend on a phony target instead of being phony itself.
.PHONY : always-remake
intermediate.dat : always-remake
IIRC, the last time I solved the problem, the .PHONY didn't work as intended and I used:
always-remake :
#true
instead. However, I can't recall why, so try the .PHONY first.
The problem with making intermediate.dat itself phony is that make never checks the existence/date of a phony file, which is behaviour that you want. You only need to trigger the rebuild rule, which is done by a prerequisite that is out of date; a phony prerequisite is always out of date, so it does the job.
Another trick is to use an order-only prerequisite, plus a recursive call to make. This can be useful when you want to implement some additional logic around when the intermediate.dat file needs to be created, or when you want to decouple a target from the usual exists && newer than check that make does.
For example:
Let's say that the file contains some date-sensitive material that subsequent tasks depend on, and that it expires after 12 hours. We only want to recreate the file when any of these is true:
The file is older than 12 hours
The file does not exist
To implement the make target & recipes, we'll create a .PHONY check-intermediate.dat target, which runs the time check, conditionally removes the file if it's expired, and then re-runs make for that file.
Note that the check-intermediate.dat target will be defined via the order-only prerequisite syntax (e.g. intermediate.dat: | check-intermediate.dat) which causes it to run always before the target (e.g. intermediate.dat). The time check is inexpensive, and we always want it to run and manage the file for us, but not invalidate the original make check on intermediate.dat if it already exists like a normal prerequisite .PHONY target would.
.PHONY: check-intermediate.dat
check-intermediate.dat:
if [ -e intermediate.dat ]; then find intermediate.dat -mmin +720 -exec bash -c 'rm -f "{}"; $(MAKE) intermediate.dat' \; ; fi
intermediate.dat: | check-intermediate.dat
echo run-expensive-tasks-here-to-create-intermediate.dat
step1.dat: intermediate.dat
sleep 10
step2.dat: step1.dat
sleep 15
step3.dat: step2.dat
sleep 10
all: step3.dat
sleep 5
Now, anything that depends on the intermediate.dat target will first run the check, remove the file if it's expired, and re-run make intermediate.dat before running the rest of the dependent targets. If the intermediate.dat file already exists, it will run the check, and if the check passes, it will continue without running the time expensive recreation task.
The problem with depending on a .PHONY target is that it will always run the tasks which depend on it, because of the way GNU Make handles .PHONY targets. This can cause the make run to always perform the time expensive operations even when it does not need to:
A phony target should not be a prerequisite of a real target file; if it is, its recipe will be run every time make goes to update that file. As long as a phony target is never a prerequisite of a real target, the phony target recipe will be executed only when the phony target is a specified goal (see Arguments to Specify the Goals).