I have the following Makefile:
cells.csv:
echo cellA > cells.csv
echo cellB >> cells.csv
echo cellC >> cells.csv
echo cellD >> cells.csv
mkdir -p cellA
mkdir -p cellB
mkdir -p cellC
mkdir -p cellD
%/cell_gen: cells.csv
echo '$# generated' > $#
%/cell_gds: %/cell_gen
cat $(#D)/cell_gen > $#
echo $#_GDS >> $#
The idea is to generate 'cells' in two step (called [cell]_gen and [cell]_gds) while the cells list is
not known at the beginning of make.
Here: the target 'cells.csv' is human readable (just echo) , but in
the general case, I expect something complexe, itselft resulting of previous steps ...etc..: not readable.
Each step of 'cell' should be stored in the directory named [cell] .
I don't understand why in this case, if I ask for "make cellA/cell_gds" then it looks like
the steps are all executed: I get the csv file and I get cellA/cell_gds.
...but I can't explain why I don't get cellA/cell_gen ??
... Despite I can see "echo 'cellA/cell_gen generated' > cellA/cell_gen" during make execution , and i really get "cellA/cell_gen generated" instide the cellA/cell_gds
Does anybody knows why there is no file cellA/cell_gen ??
thanks !
The file cellA/cell_gen is an intermediate file; you didn't explicitly ask for it, Make deduced that it was necessary as part of a chain of pattern rules. So by default, Make will delete it once the "real" target, cellA/cell_gds, is complete.
To prevent this, just add the line
.PRECIOUS: %/cell_gen
Related
I want to run makefile with input variable. What I want is that if I write down the project name, a folder with that name will be created.
So I write read command:
CC = gcc
CFLAGS = -W -Wall
FILE := hi
src = $(wildcard *.c)
OBJ = $(src:.c=.o)
all : $(FILE)
$(FILE) : $(OBJ)
$(CC) $(CFLAGS) -o $# $^
.PHONY: clean
clean :
rm *.o $(FILE)
move :
mkdir -p ../../bin/$(FILE);
mkdir -p ../../exe/$(FILE);
mv *.o ../../bin/$(FILE);
mv $(FILE) ../../exe/$(FILE)
afterclean :
rm ../../bin/$(FILE)/*.o;
rm ../../exe/$(FILE)/$(FILE)
execute :
./../../exe/$(FILE)/$(FILE)
read :
#read -p "Enter : " enter; \
$(FILE) := enter; \
echo $FILE
What I wanna do is if I get FILE name through read I want to change FILE variable, but I can't change it. How can I do that?
Well in short, you cannot easily do that (and you should likely not want to, scroll down for rationale). If you have a closer look at your Makefile you'd notice that you're mixing make and shell syntax... and their contexts.
In your case, it literally passes the following string to shell (value of SHELL, likely defaults to /bin/sh) with -c:
read -p "Enter : " enter; \
hi := enter; \
echo ILE
Which shows the effects of the intermixed syntax. $(FILE) (value hi) and $F (unset -> empty) are make variables substituted by make before invoking shell. (while read into enter variable is not used at all and instead literal string enter is used in attempted make variable assignment inside that running shell.)
If you wanted to run a shell command and assign a value from what it has done / learned to a make variable, you would have to do so using shell function (or generate a (temporary) file you would include, but that's even messier):
FILE := $(shell read -p "Enter: " enter ; echo $${enter})
That however always asks... unless you use conditional assignment (?=) in which case you could choose already from the command line (make FILE=something, at which point we're about to close the circle). I am generally unsure what your intentions were how to tell make when to ask and when to use default value of hi.
That leads me to why this notion sounds suspect to me to start with and why suggestion made by #HolyBlackCat is a superior way of customizing invocation of make.
Also any runtime user interactions generally break automation (which is what we have make for) and also make builds non-reproducible. So, they better are to be avoided.
In other words, if you really had to, I'd say write an interactive_make_call.sh around it for this type of invocation:
#!/bin/bash
read -p "Enter : " enter
make FILE="${enter}" "$#"
Or even:
#!/bin/bash
read -p "Enter : " enter
if [[ -n "${enter}" ]] ; then
make FILE="${enter}" "$#"
else
make "$#"
fi
To fallback on the default value of FILE from the Makefile if you just press enter.
I would like to have data processing performed when launching make and one or more files in the input directory have changed. With my current Makefile processing is not triggered.
Let's say, I'm setting up an exemplary project: two directories, one with data files. Use this script:
#!/bin/bash
mkdir -p proj/input
mkdir -p proj/output
cd proj/input
echo "a2018" > 2018a.txt
echo "b2018" > 2018b.txt
echo "a2019" > 2019a.txt
echo "b2019" > 2019b.txt
My Makefile:
#proj/Makefile
RAW := $(shell find input -name "*.txt")
OUT := ./output
.PHONY: all
all: $(OUT)/*.txt
echo "Running processing of raw files"
$(OUT)/2018merged.txt: $(RAW)
./merge.sh 2018
$(OUT)/2019merged.txt: $(RAW)
./merge.sh 2019
The merge script does basic concatenation of files by year from the filename and saves the result in the file in the output directory.
#!/bin/bash
# proj/merge.sh
echo "-- merging files for $1"
cat input/$1*.txt > "./output/$1merged.txt"
I believed that providing all files in the input directory as a prerequisite will be sufficient but apparently I'm doing something wrong.
I found few questions around similar poblems and partial solutions might be there:
Processing multiple files generated from single input,
Make: How to process many input files in one invocation of a tool?,
Make dummy target that checks the age of an existing file?.
Consider this rule:
all: $(OUT)/*.txt
echo "Running processing of raw files"
This says, all depends on all the files in the $(OUT) directory that match the pattern *.txt.
Well, of course before you've run your makefile there are no files: that's the whole point of your makefile to create them. So when you first run make that pattern expands to nothing, and thus there are no prerequisites for all, and thus nothing is done.
If you want to construct the list of targets to build you have to do it based on the source files, which will always exist, or in this case where they don't match up exactly you have to list them explicitly:
YEARS = 2018 2019
all: $(patsubst %,$(OUT)/%.txt,$(YEARS))
echo "Running processing of raw files"
I would like to run a command before and after each target in a makefile.
so something like this
pre:
#echo pre
#echo running | mailx -s "Start {target}" user#foo.com
post:
#echo post
#echo post | mailx -s "Finish {target}" user#foo.com
j:
long_running_command && echo $# > $#
k: j
long_running_command2 && echo $# > $#
I would like to run pre and post for j and k. Ideally, I would like to get an email for each task that starts and stops.
One way to do it is to modify all the recipes in your makefile to invoke some command. You can put it into a variable so it doesn't look too gross:
START = mailto --subject 'Started target $#' me#my.host
END = mailto --subject 'Finished target $#' me#my.host
j:
$(START)
long_running_command && echo $# > $#
$(END)
k: j
$(START)
long_running_command2 && echo $# > $#
$(END)
The nice thing about this is you can pick and choose which targets you want it for; maybe some of them don't need it. The disadvantage is if the command fails you won't get any "end" email at all.
If you really want to do it for every single target, then you can write a shell script that mimics the shell's behavior but also sends mail, while running the command.
$ cat mailshell
#!/bin/sh
# get rid of the -c flag
shift
mailto --subject "started $*" me#my.host
/bin/sh -c "$#"
r=$?
mailto --subject "ended $* with exit code $r" me#my.host
exit $r
(note this is totally untested but you get the idea I hope). Then in your makefile, set SHELL to that shell:
SHELL := mailshell
j:
long_running_command && echo $# > $#
k: j
long_running_command2 && echo $# > $#
I guess you could still pick and choose by setting SHELL as a target-specific variable only for those targets you wanted to use the shell.
One downside of this is that if you have recipes that have multiple lines you'll get an email for each line individually. You can work around this by enabling .ONESHELL: to force the entire recipe to be passed to a single shell. However, I believe that this may require your mailshell tool to be more sophisticated.
If you only have one command per recipe, you can do this by changing the configuration for the shell the commands are run in.
Have the config file run the pre commands directly and trap EXIT to run the after commands in.
For example:
$ cat Makefile
SHELL := BASH_ENV=/dev/fd/3 3<<<'echo before; trap "echo after" EXIT' /bin/bash
default:
echo default
other:
echo first
echo second
$ make default
echo default
before
default
after
However this may not be what you want if a recipe runs several commands, as the before and after code will run each time.
$ make other
echo first
before
first
after
echo second
before
second
after
And I don't know of a way (outside of recursive Makefiles) to have different recipes use different shells, so this won't work if you only want to set before/after for several recipes.
I have a makefile that uses a source file from the internet. There are two locations where the file resides, neither of which I consider very dependable, so I also keep a local copy. So the relevant lines of my makefile look like:
src.c:
wget -nv http://location.com/$# || wget -nv http://otherplace.com/$# || cp local/$# .
src.o: src.c
$(CC) -o $# $<
Is this the "right way" to do this? What if there are multiple steps in each different way of creating the target - how do I tell make "Try A. If A fails, try B. If B fails, ..."?
The right thing to do is this:
.PHONY: phony
src.c: phony
if (wget -nv http://location.com/$# -O $#.temp) && ! diff $#.temp $# >/dev/null; then \
mv $#.temp $#; \
fi
I shortened your command to a single wget but you can put whatever you want there, including a sequence of ||s to achieve "try this, if not, try that etc". Just make sure it outputs to a temporary file (and does not hang indefinitely !) .
It is in fact important to use phony here, and not only .PHONY. Can you see why?
Also, with this method, there is no longer a need to keep another "local" copy and/or use cp. Your target src.c is your "local copy" - the latest one you were able to successfully get from the Internet.
I'm making a Makefile that moves an output file (foo.o) to a different directory (baz).
The output file moves as desired to the directory. However since make won't recompile the output file if I type make again, mv gets an error when it tries to move the non-existent empty file to the directory baz.
So this is what I have defined in my rule make all after all compilation:
-test -e "foo.o" || mv -f foo.o ../baz
Unfortunately, I'm still getting errors.
Errors in Recipes (from TFM)
To ignore errors in a recipe line, write a - at the beginning of the
line's text (after the initial tab).
So the target would be something like:
moveit:
-mv foo.o ../baz
I notice nobody has actually answered the original question itself yet, specifically how to ignore errors (all the answers are currently concerned with only calling the command if it won't cause an error).
To actually ignore errors, you can simply do:
mv -f foo.o ../baz 2>/dev/null; true
This will redirect stderr output to null, and follow the command with true (which always returns 0, causing make to believe the command succeeded regardless of what actually happened), allowing program flow to continue.
+#[ -d $(dir $#) ] || mkdir -p $(dir $#)
is what I use to silently create a folder if it does not exist. For your problem something like this should work
-#[ -e "foo.o" ] && mv -f foo.o ../baz
-test -e "foo.o" || if [ -f foo.o ]; then mv -f foo.o ../baz; fi;
That should work
Something like
test -e "foo.o" && mv -f foo.o ../baz
should work: the operator should be && instead of ||.
You can experiment with this by trying these commands:
test -e testfile && echo "going to move the file"
test -e testfile || echo "going to move the file"
I faced the same problem and as I am generating files, they always have different time. Workaround is set the same time to the files: touch -d '1 June 2018 11:02' file. In that case, gzip generates the same output and same md5sum. In my scenario, I don't need the time for the files.