Makefile: make text file and append strings in it - makefile

I'm having difficulties with makefiles.
So in a recipe, I'm making a file (with a name and a .ujc extension) in a for loop and would like to have a text file at the end which contains all the created files. Purpose is to feed it to an application.
For example, in a semi high-level example,
List= [Class1,Class2,Class3]
foreach(Class C in List) {
#do operations on C > outputs a ClassX.ujc file
# add name of file to a text file named "list_of_files"
}
At the end I should have a text file, list_of_files.txt, which contains the following string:
Class1.ujc Class2.ujc Class3.ujc
As a reference, the code I have at the moment (and which does a bit of the stuff above but does not work is) is:
pc: $(APP)
$(foreach C, $(shell echo $(CLASS) | tr ',' ' '), \
make -C BUILDENV CLASS=$(C) BUILD=just_filelist OUTPUT=filelist.txt SKIPSELF=yes && \
../classCvt/classCvt <./Applications/$(C).class> ./Applications/$(C).ujc && \
cat app_file_list.txt | xargs echo ./Applications/$(C).ujc >app_file_list.txt && \
) true
time -p ./$(APP) `cat app_file_list.txt` `cat filelist.txt`
The internal make does make a filelist which is fed to the app, but I'd also like to feed the app_file_list but its construction goes totally wrong.
Probably simple, but I'm not getting there.
Edit:
The code below does what I want:
pc: $(APP)
rm -f cat app_file_list.txt
$(foreach C, $(shell echo $(CLASS) | tr ',' ' '), \
make -C BUILDENV CLASS=$(C) BUILD=just_filelist OUTPUT=filelist.txt SKIPSELF=yes && \
../classCvt/classCvt <./Applications/$(C).class> ./Applications/$(C).ujc && \
cat app_file_list.txt | echo ./Applications/$(C).ujc >>app_file_list.txt && \
) true
time -p ./$(APP) `cat app_file_list.txt` `cat filelist.txt`
Notable mistake I made was the xargs.

(Also in the post)
The solution turned out to be not-so-difficult. I needed to remove the xargs command and do the correct operation (i.e., >> instead of >) in the 'cat app_file_list.txt | etc...' line.
The code below does what I want:
pc: $(APP)
rm -f cat app_file_list.txt
$(foreach C, $(shell echo $(CLASS) | tr ',' ' '), \
make -C BUILDENV CLASS=$(C) BUILD=just_filelist OUTPUT=filelist.txt SKIPSELF=yes && \
../classCvt/classCvt <./Applications/$(C).class> ./Applications/$(C).ujc && \
cat app_file_list.txt | echo ./Applications/$(C).ujc >>app_file_list.txt && \
) true
time -p ./$(APP) `cat app_file_list.txt` `cat filelist.txt`
Notable mistake I made was the xargs which caused strings to repeat into the .txt file.

Related

How to properly write for loop in Makefile

I went through some posts, but still didn't get how it work.
My request is:
for i in *.json
do
file = `echo $i |cut -d _ -f2`
echo ${file}
# do the rest tasks
done
How to convert above script to target of Makefile?
Here is what I tried
foo:
for i in *.json; do \
$(eval FILE = $(shell echo $$i |cut -d _ -f2)); \
echo $(FILE) ;\
done
But it doesn't work
Using $(eval) or $(shell) is ... not even wrong.
foo:
for i in *.json; do \
file=$$(echo "$$i" |cut -d _ -f2); \
echo "$$file"; \
done
Notice the quoting of the filename variables, and the absence of spaces around the = assignment operator, and the doubling of any dollar sign in order to pass it through from make to the shell.
However, the shell provides a much better mechanism for this;
foo:
for i in *.json; do \
j=$${i#*_}; \
echo "$${j%%_*}"; \
done
or perhaps
foo:
printf '%s\n' *.json \
| sed 's/[^_]*_\([^_]*\)_.*/\1/'
If you only expect a single underscore, both of these can be further simplified.
Or maybe you are just looking for
makefile_variable := $(foreach x,$(wildcard *.json),$(word 2,$(subst _, ,$x)))

Interpolate variable in command substitution in Makefile

I am trying to do variable interpolation inside a command substitution in a Makefile.
I have this code:
setup:
mkdir -p data_all ; \
for i in $(shell jq -r 'keys | #tsv' assets.json) ; do \
git_url=$(shell jq -r ".$$i" assets.json) ; \
git clone $$git_url data_all/$$i ; \
done
The code is failing, however, because $$i does not expand in the "shell" line that sets git_url.
How do I interpolate the variable $i in the "shell" line that sets git_url?
You mixed up make functions ($(shell ...)) and true shell constructs. When writing a recipe the simplest is to write it first in plain shell:
mkdir -p data_all ; \
for i in $( jq -r 'keys | #tsv' assets.json ) ; do \
git_url=$( jq -r ".$i" assets.json ) ; \
git clone $git_url data_all/$i ; \
done
And then escaping the unwanted $ expansion by make:
mkdir -p data_all ; \
for i in $$( jq -r 'keys | #tsv' assets.json ) ; do \
git_url=$$( jq -r ".$$i" assets.json ) ; \
git clone $$git_url data_all/$$i ; \
done

How do I detect a failed subprocess in a bash read statement?

In bash we can set an environment variable from a sequence of commands using read and a pipe to a subprocess. But I'm having trouble detecting errors in my processing in one edge case - a part of the subprocess pipeline producing some output before erroring.
A simplified example which takes an input file, looks for a line starting with "foo" and sets var to the first word on that line is:
set -e
set -o pipefail
set -o nounset
die() {
echo $1 > /dev/stderr
exit 1
}
read -r var rest < <( \
cat data.txt \
| grep foo \
|| die "PIPELINE" \
) || die "OUTER"
echo "var=$var"
Running this with data.txt like
blah
zap foo awesome
bang foo
will output
var=zap
Running this on a data.txt file that doesn't contain foo outputs (to stderr)
DEAD: PIPELINE
DEAD: OUTER
This is all as expected.
We can introduce another no-op stage like cat at the end of the process
...
read -r var rest < <( \
cat data.txt \
| grep foo \
| cat \
|| die "PIPELINE" \
) || die "OUTER"
...
and everything continues to work.
But if the additional stage is paste -s -d' ' and the input does not contain "foo" the output is
var=
DEAD: PIPELINE
Which seems to show that the pipeline errors, but read succeeds with an empty line. (It looks like paste -s -d' ' outputs a line of output even when its input is empty.)
Is there a simple way to detect this failure of the pipeline, and cause the main script to error out?
I guess I could check that the variable is not empty - but this is a simplified version - I'm actually using sed and paste to join multiple lines to set multiple variables, like
read -r v1 v2 v3 rest < <( \
cat data.txt \
| grep "^foo=" \
| sed -e 's/foo=//' \
| paste -s -d' ' \
|| die "PIPELINE"
) || die "OUTER"
You could use another grep to see if the output of paste contained something:
read -r var rest < <( \
cat data.txt \
| grep foo \
| paste -s -d' ' \
| grep . \
|| die "PIPELINE" \
) || die "OUTER"
In the end I went with two different solutions depending on the context.
The first was to pipe the results to a temporary file. This will process the entire file before performing the read, and thus any failures in the pipe will cause the script to fail.
cat data.txt \
| grep "^foo=" \
| sed -e 's/foo=//' \
| paste -s -d' ' \
> $TMP/result.txt
|| die "PIPELINE"
read -r var rest < $TMP/result.txt || die "OUTER"
The second was to just test that the variables were set. While this meant
there was a bunch of duplication that I wanted to avoid, it seemed the most bullet-proof solution.
read -r var rest < <( cat data.txt \
| grep "^foo=" \
| sed -e 's/foo=//' \
| paste -s -d' ' \
|| die "PIPELINE"
) || die "OUTER"
[ ! -z "$var" ] || die "VARIABLE NOT SET"

Makefile with multiple values for a parameter

Currently, I am working on a makefile that takes a parameter "CLASS=xxx" and then compiles and does stuff with that value.
In the end, it runs an application ($APP) on a bunch of files.
I enter this command:
make default CLASS=Test_UART
and the makefile processes it thusly:
pc: $(APP)
make -C BUILDENV CLASS=$(CLASS) BUILD=just_filelist OUTPUT=filelist.txt SKIPSELF=yes
../classCvt/classCvt <./Applications/$(CLASS).class> ./Applications/$(CLASS).ujc
time -p ./$(APP) ./Applications/$(CLASS).ujc `cat filelist.txt`
Hence it calls a makefile in my BUILDENV folder which does the following:
#USAGE: make -C <PATH_TO_THIS_FILES_PARENT_DIR> CLASS=<MY_JAVA_FILE_NAME_WITHOUT_JAVA_EXTENSION> OUT=<OUTPUT_FILE_NAME>
SELF := $(dir $(CURDIR)/$(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)))
CLASS ?= PLEASE_SPECIFY_CLASS_PARAM
DIR := $(PWD)#print working directory
CCVT ?= $(SELF)/../../classCvt/classCvt
TOBIN ?= $(SELF)/../../classCvt/tobin
OUTPUT ?=
### Comment: Defining CMD.
ifeq ($(BUILD), just_filelist)
CMD = echo
else
ifeq ($(BUILD), PC)
CMD = echo
else
ifeq ($(BUILD), unopt)
CMD = $(TOBIN)
else
### Comment: Optimized CMD = tobin -c ccvt
CMD = $(TOBIN) -c $(CCVT)
endif
endif
endif
ifeq ($(OUTPUT), )
OUT = &1
else
OUT = $(DIR)/$(OUTPUT)
endif
ifeq ($(SKIPSELF), yes)
MYCLASS =
else
MYCLASS = $(DIR)/Applications/$(CLASS).class
endif
all:
CLASSPATH=$(SELF)/RT/real:$(SELF)/RT/fake:$(DIR) javac $(DIR)/Applications/$(CLASS).java
find $(SELF)/RT/real -iname "*.class" -type f > $(SELF)/files
ls $(DIR)/Applications/*.class | grep -v "$(CLASS)\.class" >> $(SELF)/files || true
cat $(SELF)/files | xargs $(CMD) $(MYCLASS) >$(OUT)
rm -f $(SELF)/files
What I would like to do is give a command like:
make default CLASS=Test1,Test2,Test3
and the makefile to process it for the 3 classes and put the given classes in a .txt and the default classes in a different .txt, something like this like this:
pc: $(APP)
make -C BUILDENV default_classes BUILD=list_default_classes OUTPUT=list_default_classes.txt
# make -C BUILDENV given_classes BUILD=list_given_classes OUTPUT=list_given_classes.txt CLASS=$(CLASS) SKIPSELF=yes
../classCvt/classCvt `cat list_given_classes.txt`./Applications/$(CLASS).ujc
#here the list_given_classes should now contain the .ujc files
time -p ./$(APP) `cat list_given_classes.txt` `cat list_default_classes.txt`
and for the makefile in the BUILDENV, I expect something like:
default_classes:
CLASSPATH=$(SELF)/RT/real:$(SELF)/RT/fake:$(DIR)
find $(SELF)/RT/real -iname "*.class" -type f > $(SELF)/files
ls $(DIR)/Applications/*.class | grep -v "$(CLASS1)\.class" "$(CLASS2)\.class">> $(SELF)/files || true
cat $(SELF)/files | xargs $(CMD) >$(OUT)
rm -f $(SELF)/files
given_classes:
javac $(DIR)/Applications/$(CLASS).java
find $(SELF)/RT/real -iname "*.class" -type f > $(SELF)/files
ls $(DIR)/Applications/*.class | grep -v "$(CLASS)\.class" >> $(SELF)/files || true
cat $(SELF)/files | xargs $(CMD) $(MYCLASS) >$(OUT)
rm -f $(SELF)/files
However, I'm not sure how to do this for a CLASS parameter containing multiple classes.
I'm thinking to try and parse the Test1,Test2,Test3 value into a list of 1,2,3 and then iterating over it. But no clue if this is a good way and even on how to do it.
What do you guys suggest?
Pretty way:
pc: $(APP)
define BUILD_CLASS
pc: pc-$(CLASS_SPLIT)
pc-$(CLASS_SPLIT):
make -C BUILDENV CLASS=$(CLASS_SPLIT) BUILD=just_filelist OUTPUT=filelist.txt SKIPSELF=yes
../classCvt/classCvt <./Applications/$(CLASS_SPLIT).class> ./Applications/$(CLASS_SPLIT).ujc
time -p ./$(APP) ./Applications/$(CLASS_SPLIT).ujc `cat filelist.txt`
endef
CLASSES := $(shell echo $(CLASS) | tr ',' ' ')
$(foreach CLASS_SPLIT, $(CLASSES), $(eval $(BUILD_CLASS)))
Simple way:
pc: $(APP)
$(foreach C, $(shell echo $(CLASS) | tr ',' ' '), \
make -C BUILDENV CLASS=$(C) BUILD=just_filelist OUTPUT=filelist.txt SKIPSELF=yes && \
../classCvt/classCvt <./Applications/$(C).class> ./Applications/$(C).ujc && \
time -p ./$(APP) ./Applications/$(C).ujc `cat filelist.txt` &&) true

Problem in Makefile

I am executing the following command in Makefile:-
#ls export_mojave/marker_*.tcl > export_mojave.list
#for file in `cat export_mojave_tcl_files.list`; do \
diff $$file bk_marker > $$file.diff ; \
if ! [ -s $$file.diff ]; then\
rm -f $$file.diff ; \
else \
echo $$file >> marker.fill.tcl.diff; \
fi \
done ;
If there exists some file related to the above expression in the mentioned directory,
it will run fine but if there does not exits any files matching to above expression, It is marking an error. Is there anything exists like "catch" in Makefile?
If you need to skip error in makefiles' rule, then prefix command with '-' sign:
-#ls .... > some_file || echo Error: no file;
if [ -e some_file ] ....
Or modify to be more in make-style:
FILES := $(shell ls ...)
ifeq ($(FILES),)
$(error no files)
endif
....
target:
$(foreach file,$(FILES), ...)

Resources