In existing project I am packing files using make.
I am trying to rewrite make, so it does not have to clear everything on every my change. I have targets like this:
device1pack1.pack: images/device1/pack1/*.png packer
packer -o device1pack1.pack -i images/device1/pack1/*.png
problem is, images are not getting changed. I just put some images inside or delete them, so their modification date usually old, so make target does not getting rebuild. Only thing changing is modification date of images/device1/pack1. But that is directory. How do I make it so when I delete or put a file into it make will think this is outdated target and rebuild it?
P.S. I have a lot such targets (around 800) and about 30 folders with images
You can list the directory as a prerequisite. Then any time the directory time-last-modified changes (which will happen every time you add, remove, or rename a file in the directory) the target will be run.
Related
I have a make target that looks like this:
WXICONS_TIMESTAMP = assets/fonts/.timestamp
# compile icons
$(WXICONS_TIMESTAMP): scripts/svg2font.js artifacts/icons/*.svg
$(NODE_BIN) $<
touch $(WXICONS_TIMESTAMP)
i.e., it runs a script that compiles a bunch of .svg files in a directory into a couple font files.
If you modify any file in this directory, then the make target is triggered because the .svg file will be newer than my dumby .timestamp file. However, if you delete an .svg file nothing happens, or if you add an .svg file into the directory that has an mtime older than the .timestamp file, then nothing happens.
So how can I trigger the target when files are added, removed or modified?
This is an example of one of the only times it's reasonable to have a target depend on a directory rather than a file. The modification time of a directory is updated when either a new file is created, an existing file is deleted, or a file is renamed in that directory.
So, if you list the directory containing the .svg files as a prerequisite of the timestamp target then whenever a file is added, removed, or updated that rule will be run again.
$(WXICONS_TIMESTAMP): scripts/svg2font.js artifacts/icons/. artifacts/icons/*.svg
$(NODE_BIN) $<
touch $(WXICONS_TIMESTAMP)
I have a situation where I need to execute a rule in make depending on a remote file. This is an example of what I'm trying to do (the actual Makefile is a lot more complicated in ways that aren't relevant here):
URL = http://cdn.sstatic.net/stackoverflow/img/favicon.ico
stackoverflow.png: favico.ico modified | check_modified
convert favicon.ico $#
check_modified: modified
#echo Icon was modified. Downloading.
rm -f favicon.ico
wget $(URL)
.PHONY: check_modified
favico.ico: check_modified
modified:
touch -d "`wget --spider -S $(URL) 2>&1 | sed -n 's/.*Modified: //p'`" $#
The idea is:
The rule to build modified should always be run. The function of that rule is to change the modification time of the file to be the same as the modification time of what the URL points to.
After that, I want modified to behave normally in terms of dependencies. If modified is more recent than favicon.ico, I want to retrieve the new file and then let the dependency on it cause the target file to be remade.
As a wrinkle, in some applications, I have to retrieve the file manually. In those cases, I want to just have a rule that fires and tells me to manually download the file, but that doesn't otherwise affect building the target. As an example, if the source image that I'm converting were behind a site login, I'd need to manually login to the site and download it to a fixed location and then rerun make.
Everything I've tried either:
Fails to check the URL if the target is up to date, or
Always checks the URL and rebuilds the target, even if modified is not more recent than the target.
Any words of wisdom?
Your problem is probably that make builds the dependency graph before running any recipe. So, when your modified tag file is created / updated, it is too late for make to consider its last modification date and use it to decide whether other targets shall be built or not.
Your first observed behaviour (fails to check the URL if the target is up to date) probably corresponds to attempts where you did not use the .PHONY special target. As soon as modified has been created, make always considers it as up-to-date because it has no prerequisites and thus cannot be outdated.
The second behaviour (always checks the URL and rebuilds the target, even if modified is not more recent than the target) corresponds to what you posted: check_modified is a prerequisite of the .PHONY special target, which forces make to always consider it as outdated. stackoverflow.png and favico.ico, directly or not, depend on check_modified and are thus also always considered as outdated.
One option to solve your problem would be to use a recursive, two-passes, make invocation. On a first run make would build the all phony target (always), update a modified tag file, and then invoke itself again to build other targets that have the tag file as prerequisite and are outdated. As, for the second invocation, make rebuilds its dependency graph, it will take the last modification date of the tag file into account. Something like:
URL = http://cdn.sstatic.net/stackoverflow/img/favicon.ico
.PHONY: all
all:
touch -d "`wget --spider -S $(URL) 2>&1 | sed -n 's/.*Modified: //p'`" modified
$(MAKE) stackoverflow.ico
stackoverflow.ico: modified
#echo Icon was modified. Downloading.; \
rm -f favicon.ico; \
wget $(URL); \
convert favicon.ico $#
Explanations:
I replaced your conversion to png by a useless conversion to ico because the favicon.ico icon of SO is composite and its conversion to png creates two files instead of one, named stackoverflow-0.png and stackoverflow-1.png, which uselessly complicates things.
all is a true phony target and the default goal so, each time you invoke make (or make all), it is built. It first updates the modified tag file and then invokes make again to build stackoverflow.ico.
The second make invocation does nothing if stackoverflow.ico is more recent than modified, else it downloads and converts.
Your second question (get a message about required manual operations) is completely different and simpler to solve. Let's first define a message and echo it in the file's recipe:
define DIY_message
Dear user, you should first:
- do this
- and that.
Unless you know it is useless, of course.
endef
export DIY_message
the_remote_file:
#echo "$$DIY_message"
The message will be printed if make is invoked with this goal (make the_remote_file) or with a goal that somehow depends on the_remote_file and either:
the_remote_file does not exist,
or the_remote_file exists but is out-of-date with respect to its prerequisites (if you declared prerequisites for it),
or the_remote_file is a prerequisite of .PHONY.
Note: using an intermediate make variable assigned by a define-endef makes things easier for formatted multi-lines messages.
I am trying to port an existing code into a larger project. The larger project has a main Makefile with Makefiles in each sub-directory. I am sure the path below tells you all about how it is setup. I want to port my code to
/WORKING_DIRECTORY/Drivers/Char/example
And here is the content:
sansari#ubuntu:~/WORKING_DIRECTORY/drivers/char/examples$ ls
hello1.c Makefile
My question is 1- Should modify this local Makefile or the main one? I am setting up to modify this one, but I am not sure.
2- My other question is if I modify this local file, can I just run make from here and validate my configuration instead of running make for the entire project? I know that make only updates the files that are changed; however I feel better when I clean the build environment before each make. I have run into situations, which this alone fixed my issue.
Just as background, I did try to include the make file of my target project, the one I am trying to import here with -f command. What I did was: make -f Makefile -f ../mytarget/core/Makefile
But I ran into some issues with make not doing some of the normal things it does in the primary project. For instance, there was an include statement with a relative path to a header file, which make gave me an error about not seeing it. So now I am abandoning that strategy for the time being.
#Ahmad Masoud - Hey man, thanks. Here is the Makefile. Hey man, the link is exactly what I needed. I think it will address my other questions also. You see, I cross compiled this code, and when I flashed my phone, I get the following for
uname -r: 1|root#hltespr:/lib/modules # uname -r
uname -r
3.4.0-g7e6fbf7-dirty
And I have been wondering what "dirty" means and where it comes from. If you know please tell me. The link you sent, states that perhaps make would insert the Linux kernel version there? I ask this since, modprobe does not work when I try to load my module. Instead insmod works, and I can validate that my module is in. My main issue now is that I don't know how to execute the file to make sure it runs. I only know how to run the file using modprobe, and I can not use it. It gives me the following error:
1|root#hltespr:/ # modprobe /lib/modules/hello1.ko
modprobe /lib/modules/hello1.ko
modprobe: can't change directory to '3.4.0-g7e6fbf7-dirty': No such file or directory
Update as of 06/20/15 -I put in include /home/sansari/mytree2/tbt/makefile in my module's make file. I get the following error: makefile:3: *** missing separator. Stop.
#Ahmad - This is an update as of 062415. Thanks for the info. My goal is to get make to look into this external directory, collect all the source files and build them for me. What would you suggest? I am stuck because as it stands, I know make looks into my examples directory, but no other changes I make to that local make file in the examples directory shows up in make. For instance I tried adding ($warning ....) and #echo messages, but even they do not show up.
Update on 070215- Thanks for the previous comments and support. I feel I really should reopen this thread since I did not explain the goal in detail, and now I feel I can describe it better, and hopefully the resolution will help other. I issue the command:
TARGET=msm8974 PLATFORM=msm8974 make drivers/char/examples
But I get a message stating: Nothing to be done, while I have added a number of tasks to do. Below is my make file, and I'll elaborate on what I have added right after:
lib_tbt := ../../../m/shahin/tbt
lib_daemon := ../../../m/shahin/daemon
lib_lib := ../../../m/shahin/lib
lib_tasks := ../../../m/shahin/tasks
lib_tbt_driver := ../../../m/shahin/tbt_driver
lib_tbt_make := ../../../m/shahin/tbt/make
lib_tbt_msm_common := ../../../m/shahin/tbt/platform/msm8974/common
lib_tbt_msm8974 := ../../../m/shahin/tbt/platform/msm8974
lib_asm_generic = ../../../m/shahin/tbt/platform/msm8974/include/asm-generic
$(warning This is what is in lib_asm_generic $(lib_asm_generic))
#include $(lib_tbt_make)/macros.mk
.PHONY: all $(lib_tbt)
$(lib_tbt) $(lib_daemon) $(lib_lib) $(lib_tasks) $(lib_tbt_driver) $(lib_tbt_make) $(lib_tbt_msm_common) $(lib_tbt_msm8974) $(lib_asmgeneric) :
$(MAKE) --directory=$#
$(lib_*): $(MAKE) --directory=$#
obj-$(CONFIG_EXAMPLES) += hello1.o
_
Initially I only had the obj-$(CONFIG_EXAMPLES) += hello1.ostatement in my make file. I then proceeded to add the directory variables at the top of my makefile and added the $(lib_*): $(MAKE) --directory=$# directing make to compile what is in the directory. I believe that is what it does. Please let me know if I am mistaken. And although this same make file proceeds to create object files when I put it a different directory within my project, it won't do so when it is in a device driver directory. And I do not understand why. The other directory is the /external directory and it is at the top of the tree. But that should not matter right. What I have done was to first make sure I can compile a hello program in my device driver directory called /examples. I now want to add more source code to this section. I believe the correct term is module? I also want to know if I should copy of the source files to the /examples directory or referencing them via the path is ok. That is should I move the source code directory under /examples directory or not?
It is a LOT simpler than that if you are using a kernel that uses Kbuild.
Highly recommend reading
http://www.tldp.org/LDP/lkmpg/2.6/html/x181.html
Situation A - Your source is a sub-tree of the kernel source
You would NOT modify the top-level Makefile, just ensure that ~/WORKING_DIRECTORY/drivers/char/examples/Makefile and ~/WORKING_DIRECTORY/drivers/char/examples/KBuild are set up correctly/normally. THEN at the top-level of the kernel build directory (assuming you have a separate build directory) you would type:
foo#bar:~/build-dir$ make drivers/char/examples
The kernel top-level makefile then builds just that sub-tree. You can try it out on any part of the kernel, for example:
foo#bar:~/build-dir$ make fs
NOTE: build-dir can be the same as the kernel source directory
Situation B - You are building an external module
Then use the normal module KBuild / Makefile process.
P.S.
If you post your makefile / Kbuild then I may be able to help with the actual build processing.
This is a GNU Make dependency resolution issue.
My problem is that I copy the source files from a remote file server to a scratch disk (which speeds up the build process by 50%). If the file copy fails, I want to use the source files from the file server, else I want to read them from the scratch disk.
I have tried to use the vpath mechanism, but the problem is that, as far as I understand, make will by default start looking for the source files in the current directory and only if it fails to find the files there, look in the directories listed with vpath.
Is it possible to have make first look in the vpath directories before looking in the current directory? Or perhaps only look in the vpath directories (and explicitly and dynamically add the current directory to vpath)?
Only way I can think of is to temporary change directory so that make always will fail to find the source files.
Look at the path to the source directory on the server. Suppose it's "/server/someplace/src/". And suppose you don't have a "src" directory in the current directory (if you do, we just have to tweak this method). Just make sure that the path to the source directory on the scratch disk ends in "/src/", such as "/scratch/wednesday/src/". Then you can do this:
SCRATCH_PATH = /scratch/wednesday/
SERVER_PATH = /server/someplace/
VPATH = $(SCRATCH_PATH) $(SERVER_PATH)
%.o: src/%.cc
$(CC) blah blah blah
In the event that a Makefile itself is changed, a safe bet would be to consider all targets out of date.
Is there a clever way to add this dependency? Are there any alternatives?
Make sure the object files depend on the makefile:
$(OBJFILES) : Makefile
Where Makefile is the name of the make file.
A safe bet, but a terrible idea. Example: you're using automake and update Makefile.am to add a single source file. The correct response is to compile just the new file and link it in. In your scheme everything would be rebuilt.
Moreover, adding the dependency isn't going to do anything unless you touch the file, something like:
$(SRCS): Makefile
touch $#
This will then trip up editors that use the mtime to detect concurrent modification (emacs is one example).
If you're doing something major, just run make clean all after doing the change.
Since GNU make version 4.3 it is now possible with the use of those two special variable:
.EXTRA_PREREQS
To add new prerequisite to every target
MAKEFILE_LIST
To get the path of the make file
To have every target depend on the current make file:
Put near the top of the file (before any include since it would affect the MAKEFILE_LIST) the following line:
.EXTRA_PREREQS:= $(abspath $(lastword $(MAKEFILE_LIST)))
To have every target depend on the current make file and also the make files which were included
Put the following line at the end of your file:
.EXTRA_PREREQS+=$(foreach mk, ${MAKEFILE_LIST},$(abspath ${mk}))