I have a directory where I download some data from somewhere on which I want to perform some operations by a script. When said directory does not yet exists, I want make to automatically download the files, which I would also like to be able to initiate by running make fetch. However, I don't want the download process to be started everytime I run make for the other target.
My attempt so far:
.PHONY: fetch
fetch: data/www.example.org
foo.dat: | data/www.example.org
somescript > foo.dat
data/www.example.org:
wget --mirror --prefix-directory=data http://www.example.org
This has almost the desired effects, except for when I run make fetch it gives me:
$ make fetch
make: Nothing to be done for `fetch'.
I could, of course, simply copy the recipe of data/www.example.org to fetch, however as the recipe may get more complex in the future, I would like to avoid that kind of solution.
EDIT:
It seems kind of obvious in hindsight, but for some reason I didn't think of using variables. I think, my mind kept searching for some kind of "neater" way. But that does solve it for me, so thanks to l0b0 for pointing it out to me:
FETCH = wget --mirror --prefix-directory=data http://www.example.org
.PHONY: fetch
fetch:
$(FETCH)
foo.dat: | data/www.example.org
somescript > foo.dat
data/www.example.org:
$(FETCH)
Related
I want to force a recipe for "output.file", even though it is up-to-date.
I have already tried make --assume-old=output.file output.file, but it does not run the recipe again.
In case you are curious: use case:
I want to use this together with --dry-run to find out the command that produce a target.
I ended up hiding the file to run make --dry-run output.file, but I was hoping for something more elegant + FMI: for future debugging makefile.
I think you're misunderstanding what that option does: it does exactly the opposite of what you hoped; from the man page:
-o file, --old-file=file, --assume-old=file
Do not remake the file file even if it is older than its dependen‐
cies, and do not remake anything on account of changes in file.
Essentially the file is treated as very old and its rules are
ignored.
You want output.file to be remade, so using -o is clearly not what you want.
There is no option in GNU make to say "always rebuild this target". What you can do is tell make to pretend that some prerequisite of the target you want to be rebuilt has been updated. See this option:
-W file, --what-if=file, --new-file=file, --assume-new=file
Pretend that the target file has just been modified. When used
with the -n flag, this shows you what would happen if you were to
modify that file. Without -n, it is almost the same as running a
touch command on the given file before running make, except that
the modification time is changed only in the imagination of make.
Say for example your output.file had a prerequisite input.file. Then if you run:
make -W input.file
it will show you what rules it would run, which would include rebuilding output.file.
I'm trying to put together a Makefile that will create a folder and clone repositories from GIT
I'm having trouble putting it all together so I'm starting with a generic Makefile
My makefile:
$(shell mkdir -p myDir)
$(shell git.sh)
The shell script that I am trying to get to invoke
#!/bin/sh
REPOSRC="my bitbucket repo URL"
LOCALREPO="myDir"
# We do it this way so that we can abstract if from just git later on
LOCALREPO_VC_DIR=$LOCALREPO/.git
if [ ! -d $LOCALREPO_VC_DIR ]
then
git clone $REPOSRC $LOCALREPO
else
cd $LOCALREPO
git pull $REPOSRC
fi
# End
When I run make I'm getting the following error:
Makefile:2: *** missing separator. Stop.
Also, is this the correct way to go about this task?
What you have is not a makefile. It's really a shell script written in makefile syntax (and, as you've discovered from the errors, not correct makefile syntax).
Make is a tool that allows commands to be run to update a set of target files, or not run if any of the target files don't need to be updated, based on comparing timestamps of the target files and their prerequisite files. These dependency relationships can be chained.
That's all that make is for.
To prototypical example is compiling a program: if any of the source files have been modified then you need to recompile the object files for those sources; if object files are updated then libraries might need to be re-created; if object or library files are updated then programs might need to be re-linked.
If your problem space doesn't map, or can't be made to map, to that mechanism, then make and makefiles are not the correct tool for the job you have in mind. Based on your description of your problem, make is not the right tool for this job.
You should just write a shell script, as you've basically done here already, and move forward.
If you do want to write a makefile you should spend some time understanding the syntax of makefiles and how they work, rather than just searching on Stack Overflow and trying to put together a makefile based on the answers. For example, try reading at least the introduction of the GNU make manual.
With $(shell ...) construct you substitute shell command output into the makefile. Of course after calling mkdir or invoking git the output is not a valid makefile.
Your makefile should be like this
all:
mkdir -p myDir
./git.sh
note that indentation after all: has to be done with tabs.
And it looks like you don't need make for your task. Just shell script would be enough.
I ran into this useful tip that if you're working on files a lot and you want them to build automatically you run:
watch make
And it re-runs make every couple seconds and things get built.
However ... it seems to swallow all the output all the time. I think it could be smarter - perhaps show a stream of output but suppress Nothing to be done for 'all' so that if nothing is built the output doesn't scroll.
A few shell script approaches come to mind using a loop and grep ... but perhaps something more elegant is out there? Has anyone seen something?
Using classic gnu make and inotifywait, without interval-based polling:
watch:
while true; do \
$(MAKE) $(WATCHMAKE); \
inotifywait -qre close_write .; \
done
This way make is triggered on every file write in the current directory tree. You can specify the target by running
make watch WATCHMAKE=foo
This one-liner should do it:
while true; do make --silent; sleep 1; done
It'll run make once every second, and it will only print output when it actually does something.
Here is a one-liner:
while true; do make -q || make; sleep 0.5; done
Using make -q || make instead of just make will only run the build if there is something to be done and will not output any messages otherwise.
You can add this as a rule to your project's Makefile:
watch:
while true; do $(MAKE) -q || $(MAKE); sleep 0.5; done
And then use make watch to invoke it.
This technique will prevent Make from filling a terminal with "make: Nothing to be done for TARGET" messages.
It also does not retain a bunch of open file descriptors like some file-watcher solutions, which can lead to ulimit errors.
How about
# In the makefile:
.PHONY: continuously
continuously:
while true; do make 1>/dev/null; sleep 3; done
?
This way you can run
make continuously
and only get output if something is wrong.
Twitter Bootstrap uses the watchr ruby gem for this.
https://github.com/twbs/bootstrap/blob/v2.3.2/Makefile
https://github.com/mynyml/watchr
Edit:
After two years the watchr project seems not to be maintained anymore. Please look for another solution among the answers. Personally, if the goal is only to have a better output, i would recommend the answer from wch here
I do it this way in my Makefile:
watch:
(while true; do make build.log; sleep 1; done) | grep -v 'make\[1\]'
build.log: ./src/*
thecompiler | tee build.log
So, it will only build when my source code is newer than my build.log, and the "grep -v" stuff removes some unnecessary make output.
This shell script uses make itself to detect changes with the -q flag, and then does a full rebuild if and only if there are changes.
#!/bin/sh
while true;
do
if ! make -q "$#";
then
echo "#-> Starting build: `date`"
make "$#";
echo "#-> Build complete."
fi
sleep 0.5;
done
It does not have any dependencies apart from make.
You can pass normal make arguments (such as -C mydir) to it as they are passed on to the make command.
As requested in the question it is silent if there is nothing to build but does not swallow output when there is.
You can keep this script handy as e.g. ~/bin/watch-make to use across multiple projects.
There are several automatic build systems that do this and more - basically when you check a change into version control they will make/build - look for Continuous Integration
Simple ones are TeamCity and Hudson
#Dobes Vandermeer -- I have a script named "mkall" that runs make in every subdirectory. I could assign that script as a cron job to run every five minutes, or one minute, or thirty seconds. Then, to see the output, I'd redirect gcc results (in each individual makefile) to a log in each subdirectory.
Could something like that work for you?
It could be pretty elaborate so as to avoid makes that do nothing. For example, the script could save the modify time of each source file and do the make when that guy changes.
You could try using something like inotify-tools. It will let you watch a directory and run a command when a file is changed or saved or any of the other events that inotify can watch for. A simple script that does a watch for save and kicks off a make when a file is saved would probably be useful.
You could change your make file to output a growl (OS X) or notify-send (Linux) notification. For me in Ubuntu, that would show a notification bubble in the upper-right corner of my screen.
Then you'd only notice the build when it fails.
You'd probably want to set watch to only cycle as fast as those notifications can display (so they don't pile up).
Bit of archaeology, but I still find this question useful. Here is a modified version of #otto's answer, using fswatch (for the mac):
TARGET ?= foo
all:
#fswatch -1 . | read i && make $(TARGET)
#make -ski TARGET=$(TARGET)
%: %.go
#go build $<
#./$#
I have a relatively simple question that I cant figure out and I cant figure out the right search query to find the info I need on google so I thought I would ask the collective.
In short:
cd /var/www/config
./deploy.sh - works!
but
./var/www/config/deploy.sh
doesnt :(
deploy.sh calls another bash script and it seems that the called script cant find the libs it needs because it searches relative to where it was called from which in this case would be / instead of /var/www as it expects.
I'm trying to call this from a capistrano script therefore need to find a way to call it without having to cd first. Does anyone know a simple way to achieve this?
EDIT: Thanks for your quick suggestions, its still playing up. deploy.sh calls another bash file called sake. I have uploaded a copy here http://tinypaste.com/25fc8
Cheers guys!
Don't put a . (period) in front of your command. Just use:
$ /var/www/config/deploy.sh
You can also wrap it too so you can return to existing dir, sometimes proggies like to pick up the PWD in which to work so might be worth setting it explicitly:
( cd /var/www/config/ && ./deploy.sh )
If you want to remain where you are after the command is done:
(cd /var/www/config; ./deploy.sh)
I have a tricky issue with gmake, when I build from the parent directory, something is different and the make does not build all the .o(s) it needs and fails, but if I cd to the directory and do a make it builds them fine.
How can I get GNUmake to tell me the difference between these two runs? There must be some make variables set in the parent that break the child, but I need help figuring out how to track them down.
If running make from the parent directory fails to build foo.o, then try make foo.o. If that fails then try running make -n foo.o in both directories (to print the commands instead of executing them) to see what it's doing differently. If it succeeds, then it's not even trying to build foo.o when run from the parent directory; make -n may shed some light, and as a last resort make -d will give you a torrent of information about the decision process, why it's doing what it's doing.
Here's a handy trick to see the value of variables. Put this rule in your makefile:
show_%:
#echo $# is $($*)
Now you can run make show_FOO and it will tell you the value of the variable FOO.
Finally, are you sure you know where you build your .o files? Make is very good at using things there to build files here, but not the other way around, so it can lose track of intermediate files if you're not careful.