Dynamic conditions in makefile - makefile

I maintain a fairly complex makefile for Arduino.
In the makefile, I have target to build *.hex file from *.cpp file. After the *.hex file is generated, I want to check whether the hex size of the file is less than the flash memory of the microcontroller.
To do that, I have added another target called verify_size which touches a *.sizeok file if the hex size is less.
Following is the relevant code
$(TARGET_HEX).sizeok: $(TARGET_HEX)
ifneq ($(strip $(HEX_MAXIMUM_SIZE)),1)
ifeq ($(shell expr `$(call avr_size,$(TARGET_HEX)) | grep Program | awk '{print $$2}'` '<' $(HEX_MAXIMUM_SIZE)), 1)
touch $#
endif
else
#$(ECHO) Maximum Hex size is not specified. Make sure the hex file that you are going to upload is less than microcontrollers flash memory
endif
verify_size: $(TARGET_HEX) $(TARGET_HEX).sizeok
The problem I am facing is that when the makefile is run for the very first time, I get a error saying that the hex file doesn't exist.
After some debugging I found that makefile first goes through the entire file before executing it. When it does this initial pass, the hex file not created yet and therefore the statements which does the parsing of hex file is not executed at all.
Is there a way by which I add dynamic conditions in makefile so that I can find the size of the hex file that was just generated?
Edit:
Based on #beta 's suggestion I changed the code to
$(OBJDIR)/%.hex: $(OBJDIR)/%.elf $(COMMON_DEPS)
$(OBJCOPY) -O ihex -R .eeprom $< $#
#$(ECHO)
$(call avr_size,$<,$#)
ifneq ($(strip $(HEX_MAXIMUM_SIZE)),)
if [ `$(SIZE) $# | awk 'FNR == 2 {print $$2}'` -le $(HEX_MAXIMUM_SIZE) ]; then touch $#.sizeok ; fi
else
#$(ECHO) Maximum Hex size is not specified. Make sure the hex file that you are going to upload is less than microcontrollers flash memory
endif
and it is working. But there is one small issue though.
In the above code, I am using a variable defined in makefile $(SIZE). But when this shell script executes, the value is not replaced. Instead it just replaces it with an empty value.
It works if I hardcode the value, but I am not able to use the value of the variable defined in makefile. Is it possible to access it?
Edit2:
I have posted a separate question for variable expansion issue.

If HEX_MAXIMUM_SIZE hasn't been set, Make should not update the sizeok file, and we shouldn't have a rule that can't actually rebuild its target. And we should update the sizeok file only when we rebuild the hex file. So instead of a rule for $(TARGET_HEX).sizeok, let's just make it a command within the $(TARGET_HEX) rule. (You haven't shown us avr_size, so I can't figure out your method for measuring the size of the hex file, so I'll just use ls and assume you aren't using pathological file names.)
$(TARGET_HEX): %.hex : %.cpps
# Commands to build the target file
if [ `ls -l $# | awk '{print $$5}'` -le $(HEX_MAXIMUM_SIZE) ]; then touch $#.sizeok ; fi
Now we can add a condition, in case HEX_MAXIMUM_SIZE hasn't been set correctly:
$(TARGET_HEX): %.hex : %.cpps
# Commands to build the target file
ifneq ($(strip $(HEX_MAXIMUM_SIZE)),1)
if [ `ls -l $# | awk '{print $$5}'` -le $(HEX_MAXIMUM_SIZE) ]; then touch $#.sizeok
else
#echo Maximum Hex size is not specified. Make sure that $# is small enough for the microcontroller\'s flash memory.
endif
EDIT:
This may take a few iterations. Replace this line:
if [ `$(SIZE) $# | awk 'FNR == 2 {print $$2}'` -le $(HEX_MAXIMUM_SIZE) ]; then touch $#.sizeok ; fi
with this:
$(SIZE) $# | awk 'FNR == 2 {print $$2}'
and tell us the result.

Related

GNUMake: Missing separator issue in makefile targer

Here is my Makefile snippet.
1. $(DIR_REL64)/%: $(SHIFT_BINDIR64)/%
2. $(copy_file)
3.ifneq ($(TEST),0)
4.ifneq($(LOG),0)
5. if [ -d $(TGT32)/tools/test/64bit ]; then cp -f $< $(TGT32)/tools/test/64bit; fi
6.else
7. if [ -d $(TGT64)/tools/test/64bit ]; then cp -f $< $(TGT64)/tools/test/64bit; fi
8.endif
9.else
10. if [ -d $(TGT64)/bin ]; then cp -f $< $(TGT64)/bin; fi
11.endif
I am getting error:
GNUmakefile:4: *** missing separator. Stop.
Anything wrong in my makefile rule ? Please help
Every line in the rule needs to be tab indented (no spaces). Your indentations appear to be mixed (though I can't tell 100% from the pasted code).
Also you should try not to mix make and bash syntax where you can avoid it. For example you can do.
There may also be some extra spaces at the ends of some of the lines (I can't tell from here) but makesure there is no white spaces at the ends.
I would re-write it line-by-line starting with just:
$(DIR_REL64)/%: $(SHIFT_BINDIR64)/%
$(copy_file)
Where $(copy_file) is tab indented. Make sure that works and then add the next bit.

Transitioning from shell to Makefile

I want to know how I can convert this shell script to a Makefile and use make all to perform all the script's operations. I have two directories pictures and thumbs, where thumbs is empty before running the script and pictures contains some .jpg files. Finally, this is the shell script that I want to convert to a Makefile:
#!/bin/bash
DIR="thumbs"
if [ "$(ls -A $DIR)" ]; then
p=$(find pictures/|grep "jpg"|cut -d"/" -f2)
for i in $p
do
m=$(ls -l pictures/$i | cut -d" " -f7)
n=$(ls -l thumbs/$i | cut -d" " -f7)
if [ "${m//':'}" -gt "${n//':'}" ] ;then
rm thumbs/$i
convert -thumbnail 100 pictures/$i thumbs/$i
fi
done
else
find pictures/ |cut -d"/" -f2 | grep "jpg"| \
awk '{system("convert-thumbnail 100 pictures/" $0 " thumbs/" $0)}'
fi
From Makefile you can call a program.
Example
$ cat a.sh
echo From Makefile
$ cat Makefile
all:
./a.sh
Test :
$ make
./a.sh
From Makefile
You can implement the behavior of your shell code much more simply as a Makefile, at least if you are willing to rely on GNU make. I interpret this to be what you're asking (modulo dependency on GNU make in particular). This is a pretty functional rough cut:
THUMBS = $(patsubst pictures/%,thumbs/%,$(wildcard pictures/*jpg*))
all: $(THUMBS)
thumbs/%: pictures/% thumbs
convert -thumbnail 100 '$<' '$#'
thumbs:
mkdir -p '$#'
.PHONY: all
Notes:
the THUMBS make variable gets set to a list of the thumbnail images you want to manage, based on expanding the shell glob pictures/*jpg* and replacing each occurrence of pictures/ with thumbs/.
The pattern is chosen to match your shell code, but perhaps you really want something more like $(wildcard pictures/*.jpg)
File names with whitespace in them are going to present a tricky problem if you need to worry about them; file names with certain other special characters too, albeit a bit less so
the patsub and wildcard functions are GNU extensions
You could also merge the definition of THUMBS into the rule for all, and avoid a separate variable
The rule for thumbs/%: pictures/% thumbs uses GNU-specific pattern rule syntax; this particular form is hard to express to POSIX make.
The thumbs directory is created if absent, but errors will occur if there is an ordinary file of that name in the way
make all (or just make) will update all out-of-date thumbnails; it does not rely on the same date comparison logic as the original script (which is a good thing)
The .PHONY rule is just to be careful. It prevents the existence of an actual file named "all" from interfering with make's operation.

Using Makefile for editing files rather than creating them

I was thinking about using Make for small checks for my dev setup. One thing I want is to check that a particular string exists in a file before doing some action. If I wanted to create the entire file it would be trivial
action: filename
...
filename:
echo 'blah' >> filename
But how can this logic be applied to actions, like grep? My dependency isn't that a file exists, it's that the file has correct content.
I'm asking specifically about Make and not other solutions like chef/puppet
You can run any shell commands you want in a make recipe. As many of them as you want also.
So if you need to run grep before doing something else just do that.
Just remember that every line in a recipe is run in its own shell session so they don't share state.
So this:
action: filename
...
filename:
grep -qw blah $# || echo 'blah' > $#
runs grep on filename (via the automatic variable for the current target $#) looking for whole words and quitting on the first match (-q).
If grep finds blah then it will return success and the || will short-circuit and the recipe is done. If grep fails then the || will trigger and the echo will run.
You might be tempted to do things that require the inverse logic do X only if Y is true:
filename:
grep -qw blah $# && echo blah2 > $#
but that doesn't work correctly. When grep fails the && short-circuits and make sees a recipe failure and bails the make process out with an error.
You need this instead.
filename:
! grep -qw blah $# || echo blah2 > $#
to invert the logic and ensure that the "failure" from grep is seen as success as far as make is concerned.
That all being said in this specific example if filename exists at all then that recipe won't ever run as it has no prerequisites so make will always consider it up to date. To work around that you need to give the file a prerequisite that will force it to be considered out of date. Specifically a force target.
Don't follow the advice about .PHONY for this case though. .PHONY targets should never be prerequisites of non-.PHONY targets.
Expanding on what #john wrote I got the following to work:
TEST_FILE=filename
.PHONY: ${TEST_FILE}
string=testing
filecheck=$(shell grep -qw ${string} ${TEST_FILE} || echo ${TEST_FILE})
all: ${filecheck}
${TEST_FILE}:
echo 'changing the file'
echo ${string} >> ${TEST_FILE}
Here the file on which I'm operating is a .PHONY target. I think that's ok because I'm actually not creating the file, just modifying it. This will work if the file does not exist, or exists without the needed string.
You could add a test in the target's recipe (As Etan posted before I could complete this answer...). If you do want to do this using just make logic, you could do something along the lines of:
actions: $(if $(shell grep -q $$string filename && echo y),filename,)
filename:
echo blah >> $#
If filename contains the string, then there will be an actions: filename dependency, and filename will be built when you build actions. Notice, though that this will check whether the string exists in filename at the time the makefile is parsed -- if filename is generated, or modified in this makefile, then it would not effect whether the action is run. If you want to test right before overwriting the file, then you would use a bash if statement in the recipe itself.

dividing outputs in make by filename

I am processing some files and want to at one point create two categories depending on the filename so I can compare the two. Is this possible in a makefile?
%.output1: %.input
ifneq (,$(findstring filename1,$(echo $<)))
mv $<.output1 $#
endif
%.output2: %.input
ifneq (,$(findstring filename2,$(echo $<)))
mv $<.output2 $#
endif
%.output_final: %.output1 %.output2
do_something
I think there is two things wrong with this code:
There is a mistake in the ifneq line.
%.output1 %.output2 will always use the same filename - it may not be possible to do this in 'make' and this may require ruffus.
You have tab-indented the ifneq line so make doesn't consider it a make directive and is considering it a shell command and attempting to pass it to the shell to execute (hence the shell error you removed in your recent edit).
Use spaces (or no indentation) on that line to have make process it correctly. That being said having done that you cannot use $< in the comparison as it will not be set at that point.
$(echo) is also not a make function. You have mixed/confused processing times. You cannot combine make and shell operations that way. (Not that you need echo there to begin with.)
If you want the comparison to happen at shell time then do not use make constructs and instead use shell constructs:
%.output1: %.input
if [ filename1 = '$<' ]; then
mv $<.output1 $#
fi
Though that is also incorrect as $< is %.input and $# is %.output1 for whatever stem matched the %. That rule should probably look more like this (though I'm having trouble understanding what you are even trying to have this rule do so I may have gotten this wrong).
%.output1: %.input
# If the stem that matched the '%' is equal to 'filename1'
if [ filename1 = '$*' ]; then
# Then copy the prerequisite/input file to the output file name.
cp $< $#
fi
I'm not sure I understand your second question point. The % in a single rule will always match the same thing but between rules it can differ.
This %.output_final: %.output1 %.output2 target will map the target file foo.output_final to the prerequisite files foo.output1 and foo.output2. But will also map any other *.output_final file to appropriately matching prerequisite files.

Makefile with variable number of targets

I am attempting to do a data pipeline with a Makefile. I have a big file that I want to split in smaller pieces to process in parallel. The number of subsets and the size of each subset is not known beforehand. For example, this is my file
$ for i in {1..100}; do echo $i >> a.txt; done
The first step in Makefile should compute the ranges,... lets make them fixed for now
ranges.txt: a.txt
or i in 0 25 50 75; do echo $$(($$i+1))'\t'$$(($$i+25)) >> $#; done
Next step should read from ranges.txt, and create a target file for each range in ranges.txt, a_1.txt, a_2.txt, a_3.txt, a_4.txt. Where a_1.txt contains lines 1 through 25, a_2.txt lines 26-50, and so on... Can this be done?
You don't say what version of make you're using, but I'll assume GNU make. There are a few ways of doing things like this; I wrote a set of blog posts about metaprogramming in GNU make (by which I mean having make generate its own rules automatically).
If it were me I'd probably use the constructed include files method for this. So, I would have your rule above for ranges.txt instead create a makefile, perhaps ranges.mk. The makefile would contain a set of targets such as a_1.txt, a_2.txt, etc. and would define target-specific variables defining the start and stop values. Then you can -include the generated ranges.mk and make will rebuild it. One thing you haven't described is when you want to recompute the ranges: does this really depend on the contents of a.txt?
Anyway, something like:
.PHONY: all
all:
ranges.mk: a.txt # really? why?
for i in 0 25 50 75; do \
echo 'a_$$i.txt : RANGE_START := $$(($$i+1))'; \
echo 'a_$$i.txt : RANGE_END := $$(($$i+25))'; \
echo 'TARGETS += a_$$i.txt'; \
done > $#
-include ranges.mk
all: $(TARGETS)
$(TARGETS) : a.txt # seems more likely
process --out $# --in $< --start $(RANGE_START) --end $(RANGE_END)
(or whatever command; you don't give any example).

Resources