Modifying file extensions using Makefiles - makefile

I'm new to Makefiles and I want to modify the extension of a set of files. The following command works on the shell:
for file in path/*.ext1; do j=`echo $file | cut -d . -f 1`;j=$j".ext2";echo mv $file $j; done
However, I'm not sure how to run this in a Makefile. I tried running
$(shell for file in path/*.ext1; do j=`echo $file | cut -d . -f 1`;j=$j".ext2";echo mv $file $j; done)
But this never did what I needed it to do. What do I need to do to make this work on the Makefile? How do I call it in a section?

The immediate answer to your question is that the $ character is special to make: it introduces a make variable. If you want to pass a $ to the shell, you'll have to write two of them: $$.
So, your shell function invocation would have to be written as:
$(shell for file in path/*.ext1; do j=`echo $$file | cut -d . -f 1`;j=$$j".ext2";echo mv $$file $$j; done)
However, this is almost certainly not a good way to do what you want. You don't really describe clearly what you want to do, however. If you just want to have a target in a makefile that can be invoked to make this change, you can use:
fixext:
for file in path/*.ext1; do \
j=`echo $$file | cut -d . -f 1`; \
j=$$j".ext2"; \
echo mv $$file $$j; \
done
Or, taking advantage of some useful shell shortcuts, you could just run:
fixext:
for file in path/*.ext1; do \
echo mv $$file $${file%.*}.ext2; \
done
Now if you run make fixext it will perform those steps.
But, a much more make-like way to do it would be to write a single rule that knows how to rename one file, then use prerequisites to have them all renamed:
TARGETS = $(patsubst %.ext1,%.ext2,$(wildcard path/*.ext1))
fixext: $(TARGETS)
%.ext2 : %.ext1
mv $< $#
Now you can even run make -j5 and do 5 of the move commands in parallel...

you can also add rename blocks at the top of your file eg to change a suffix
output := $(input:.mov=.mp4)
but this won't work inside a make command as far as I can see
check:
output := $(input:.mov=.mp4)
gives
$ input=walkthrough.mov make check
output := walkthrough.mp4
make: output: No such file or directory
make: *** [check] Error 1

Related

Trouble with cp command for directories

I'm trying to use the cp function to do copy directories:
src/1/b
src/2/d
src/3/c
src/4/a
src/5/e
then the copying should result in
tgt/a/4
tgt/b/1
tgt/c/3
tgt/d/2
tgt/e/5
I tried to use the 'basename' function as well as 'cp dir1/*dir2'. With the basename, do I make a loop to find every directory or is there a recursive builtin? Also tried the 'cp-r' recursive copy function. But nothing so far has worked.
I used tmp folder that will hols the SOURCE list of files, yo can readjust:
cat tmp
result:
src/1/b
src/2/d
src/3/c
src/4/a
src/5/e
from here, I echo out the command, but you can remove echo and it will execute, if this output seems correct:
#!/bin/bash
cat tmp |while read z
do
echo cp "$z" "tgt/$(echo "$z"|cut -d/ -f 3)/$(echo "$z"|cut -d/ -f 2)"
done
result:
cp src/1/b tgt/b/1
cp src/2/d tgt/d/2
cp src/3/c tgt/c/3
cp src/4/a tgt/a/4
cp src/5/e tgt/e/5
you can also add parameters to cp as you see fit. But first test with the echo command, then execute :)

How do I read BASH parameters correctly from a file?

I have this configuration file in my CI where I'm specifying a header file and some CMAKE flags on one line.
The configuration file looks like this (filelist):
./settings6.h -DMY_COMPILE_FLAGS="-m32 -fstrict-aliasing"
./settings7.h -DMY_FEATURE_1=ON
./settings8.h -DMY_FLAG=ON -DMY_FEATURE_2=ON -DMY_INCLUDE_DIR=/usr/include/
Now, I'm using a bash script to process this configuration file:
#!/bin/bash
SCRIPTDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
while read i; do
HEADERFILE=$(echo $i | cut -d ' ' -f 1)
CMAKEFLAGS=$(echo $i | cut -s -d ' ' -f 2-)
if [[ "$HEADERFILE" == "" ]]; then
continue
fi
CFLAGS="-Werror" cmake "my_build_dir" "$CMAKEFLAGS" -G "Ninja" -DMY_EXTRA_INCLUDE="$SCRIPTDIR/$HEADERFILE" -B"build_env_dir" > /dev/null
ninja -C "build_env_dir"
done <<ENDOFINPUT
$(grep -v '^#' $SCRIPTDIR/filelist)
ENDOFINPUT
When I have the bash script as above, the line with settings6.h gets processed properly, i.e. the MY_COMPILE_FLAGS are set to -m32 -fstrict-aliasing.
However, settings8.h is failing because the value of MY_FLAG is seen by CMAKE as ON -DMY_FEATURE_2=ON -DMY_INCLUDE_DIR=/usr/include/, so MY_FEATURE_2 and MY_INCLUDE_DIR are not processed correctly.
After googling around a bit, I thought, well, surely a quoting issue, probably I have to remove the quotes around $CMAKEFLAGS like this:
CFLAGS="-Werror" cmake "my_build_dir" $CMAKEFLAGS -G "Ninja" -DMY_EXTRA_INCLUDE="$SCRIPTDIR/$HEADERFILE" -B"build_env_dir" > /dev/null
In fact, this lets settings8.h work as expected (all three options are processed), but now, settings6.h is suddenly failing since CMAKE complains:
CMake Error: The source directory "/src/-fstrict-aliasing"" does not exist
Can someone guide me please how I read the settings correctly from my filelist so that settings6.h and settings8.h both succeed?
Here's a Makefile which refactors this into a sequence of recipes.
Cases := $(patsubst %.h,%,$(wildcard ./settings*.h))
all_done := $(patsubst %,.%.done,$(Cases))
.PHONY: all
all: $(all_done)
cases.mk: filelist.txt
sed 's%^\./%case_%;s% % := %' $< >$#
include cases.mk
.%_done: ./%.h
CFLAGS="-Werror" cmake "my_build_dir" $(case_$*) -G "Ninja" \
-DMY_EXTRA_INCLUDE="$<" -B"build_env_dir" > /dev/null
ninja -C "build_env_dir"

checking file content in Make

I am doing some data validation. Basically, I have a list of files and I want to make sure they are consistent. For a file to be consistent, they need to match the number of occurences.
grep -c ValidId file.????.txt
wc -l < output.????.txt
If they both match, I would like to create a done file so I don't have to recheck again. This will be a routine in my Makefile for data validation.
How would I go about doing this?
The pattern match % can be used in multiple dependencies in the rule. One can use that to keep track of the consistent files, so that running make and correcting errors would be checking only the not-yet-consistent files:
FILES=$(wildcard file.????.txt)
RES_FILES=$(FILES:file.%=.result.%)
all: $(RES_FILES)
.result.%.txt: file.%.txt output.%.txt
if [ `grep -c ValidId < file.$*.txt` -eq \
`wc -l < output.$*.txt` ]; then \
touch $# ; \
fi
clean:
#rm -f $(RES_FILES)

makefile: using for and ifneq

I have an objective in my makefile named "cambios" that makes a cvs commit on each file of the project (by separate) and shows the last revision.
Now, I have an auxiliar shellscript that do that, but I'd like to know how I can do it in the makefile. I've created the objective cambios2 that do the same without the auxiliar shellscript, but it has some syntax problems.
makefile:
(...)
TODO= makefile cambiosaux.sh lib/libreria.cc include/libreria.h src/principal.cc
(...)
cambios:
#./cambiosaux.sh "$(TODO)"
cambios2:
#for dir in $(TODO); do \
A = $(cvs commit -m "Incorporando cambios automáticamente." $$dir) \
ifneq ($(A),)
echo $dir ; \
echo "Última revisión:"$(echo $(A) | sed 's/.*new revision: //' | sed 's/;.*//') ; \
endif ; \
done
cambiosaux.sh :
for dir in $1
do
A=$(cvs commit -m "Incorporando cambios automáticamente." $dir)
if [ "$A" != "" ]; then
echo $dir
echo "Última revisión:"$(echo $A | sed 's/.*new revision: //' | sed 's/;.*//')
fi
done
There are some syntax problems in the objective cambios2, but I'm really new on doing makefiles and I really don't know how to solve that problems.
Thanks!
You forgot to escape dollars that are parts of Bash command command substitution, and Make tries to perform variable expansion: $(cvs commit ...), $(echo $(A) ...).
Also you can't assign a Make variable inside a recipe. A = $(cvs commit ...) is illegal, it won't be treated neither as Make assignment nor as Bash. Try to run make with --warn-undefined-variables, I guess it will say lots of interesting details.
Finally ifneq conditional is part of Make language, and it gets interpreted at the very early stage of reading Makefile. Thus you must not indent ifneq and endif with tabs. How Make reads a Makefile chapter gives a good explanation.
To conclude, I would recommend you to leave a separate sh as is and just invoke it from your Makefile. It is not good practice to mix code in two different languages.
Okay, I found the way it works:
CVS: $(TODO)
#for dir in $?; do \
echo $$dir ; \
echo "Última revisión:" $$(cvs commit -m "Incorporando cambios automáticamente." $$dir | grep "new revision" | sed 's/.*new revision: //' | sed 's/;.*//') ; \
done
cambios: CVS

How to ignore mv error?

I'm making a Makefile that moves an output file (foo.o) to a different directory (baz).
The output file moves as desired to the directory. However since make won't recompile the output file if I type make again, mv gets an error when it tries to move the non-existent empty file to the directory baz.
So this is what I have defined in my rule make all after all compilation:
-test -e "foo.o" || mv -f foo.o ../baz
Unfortunately, I'm still getting errors.
Errors in Recipes (from TFM)
To ignore errors in a recipe line, write a - at the beginning of the
line's text (after the initial tab).
So the target would be something like:
moveit:
-mv foo.o ../baz
I notice nobody has actually answered the original question itself yet, specifically how to ignore errors (all the answers are currently concerned with only calling the command if it won't cause an error).
To actually ignore errors, you can simply do:
mv -f foo.o ../baz 2>/dev/null; true
This will redirect stderr output to null, and follow the command with true (which always returns 0, causing make to believe the command succeeded regardless of what actually happened), allowing program flow to continue.
+#[ -d $(dir $#) ] || mkdir -p $(dir $#)
is what I use to silently create a folder if it does not exist. For your problem something like this should work
-#[ -e "foo.o" ] && mv -f foo.o ../baz
-test -e "foo.o" || if [ -f foo.o ]; then mv -f foo.o ../baz; fi;
That should work
Something like
test -e "foo.o" && mv -f foo.o ../baz
should work: the operator should be && instead of ||.
You can experiment with this by trying these commands:
test -e testfile && echo "going to move the file"
test -e testfile || echo "going to move the file"
I faced the same problem and as I am generating files, they always have different time. Workaround is set the same time to the files: touch -d '1 June 2018 11:02' file. In that case, gzip generates the same output and same md5sum. In my scenario, I don't need the time for the files.

Resources