I'm trying to store values in variables and trying to echo result to a file. But when it's adding two variables and echoing it to a file, extra characters are being added to the output file. This is happening in docker container, Can some one please help...
IFS=" "
#while read line
while read c s e
do
echo $c $s $e
first=$(echo "PER_${s}_${e}")
#echo -n $first
second=$(echo "/IPD_${c}")
#echo $second
echo $first$second >> /mnt/resource/step2/messages.txt
done < /mnt/resource/step2/job_control/Categories.txt
Categories.txt contains :
129490 201515 201540
I'm getting the output as :
PER__/IPD_PER_201515_201540/IPD_12949029490
But it should be like:
PER_201515_201540/IPD_129490
I can't reproduce the problem, but your code is more complicated than it needs to be.
while IFS=" " read c s e; do
first="PER_${s}_${e}"
second="/IPD_${c}"
echo "$first$second" >> /mnt/resource/step2/messages.txt
done < /mnt/resource/step2/job_control/Categories.txt
The Question
Example 1
Consider the following user-defined function:
define func =
if [ -f $(1) ]; then \
printf "'%s' is a file\n" '$(1)'; \
printf "This is a relatively long command that"; \
printf " won't fit on one line\n"; \
fi
endef
all:
$(call func,foo)
This will output the following:
$ make
if [ -f foo ]; then printf "'%s' is a file\n" 'foo'; printf "This is a rel
atively long command that"; printf " won't fit on one line\n"; fi
For readability, I would like make to print the command on multiple lines,
as written in the Makefile. How do I accomplish this?
What I've Tried
Example 2
The following works the way I want, but does not allow the parameterized
function:
filename := foo
.PHONY: foo
foo:
if [ -f $(filename) ]; then \
printf "'%s' is a file\n" '$(filename)'; \
printf "This is a relatively long command that"; \
printf " won't fit on one line\n"; \
fi
Output:
$ make foo
if [ -f foo ]; then \
printf "'%s' is a file\n" 'foo'; \
printf "This is a relatively long command that"; \
printf " won't fit on one line\n"; \
fi
Example 3
My obvious first instinct was to escape the backslashes:
define func =
if [ -f $(1) ]; then \\
printf "'%s' is a file\n" '$(1)'; \\
printf "This is a relatively long command that"; \\
printf " won't fit on one line\n"; \\
fi
endef
Output:
$ make
if [ -f foo ]; then \\
printf "'%s' is a file\n" 'foo'; \\
printf "This is a relatively long command that"; \\
printf " won't fit on one line\n"; \\
fi
/bin/sh: \: command not found
'foo' is a file
/bin/sh: line 1: \: command not found
This is a relatively long command that/bin/sh: line 2: \: command not found
won't fit on one line
/bin/sh: line 3: \: command not found
make: *** [Makefile:11: all] Error 127
Example 4
Okay, so why not try \\\?
$ make
if [ -f foo ]; then \ printf "'%s' is a file\n" 'foo'; \ printf "This is a
relatively long command that"; \ printf " won't fit on one line\n"; \ fi
/bin/sh: -c: line 1: syntax error: unexpected end of file
make: *** [Makefile:11: all] Error 1
Example 5
Interesting. Let's go for four...
$ make
if [ -f foo ]; then \\\\
printf "'%s' is a file\n" 'foo'; \\\\
printf "This is a relatively long command that"; \\\\
printf " won't fit on one line\n"; \\\\
fi
/bin/sh: \\: command not found
'foo' is a file
/bin/sh: line 1: \\: command not found
This is a relatively long command that/bin/sh: line 2: \\: command not found
won't fit on one line
/bin/sh: line 3: \\: command not found
make: *** [Makefile:11: all] Error 127
Now we're back to where we were last time.
What Works
Example 6
This is the only thing that seems to work:
define func =
if [ -f $(1) ]; then #\\
printf "'%s' is a file\n" '$(1)'; #\\
printf "This is a relatively long command that"; #\\
printf " won't fit on one line\n"; #\\
fi
endef
Output:
$ make
if [ -f foo ]; then #\\
printf "'%s' is a file\n" 'foo'; #\\
printf "This is a relatively long command that"; #\\
printf " won't fit on one line\n"; #\\
fi
But man, that looks ugly, and it feels hackish. There's got to be a better
way to do this. Or am I just going about this the wrong way in the first
place?
It seems to me that make is just confused by the magic that happens when
escaping newlines within a recipe. The lines printed to the terminal during
execution do not match what the shell sees. Should this be considered a bug?
I am using GNU Make 4.2.1 on Cygwin.
Edit
To clarify: make normally gives special treatment to trailing backslashes within a recipe. They do not indicate line continuation, as they do elsewhere. Instead, they indicate that multiple recipe lines are to be treated as a single command, and they are passed to the shell intact.
When not in a recipe, but defining a variable, this special treatment does not apply. The lines are simply joined, as in Example 1. This is expected.
I would expect that a double backslash would be translated to a single literal backslash in the variable, but instead both backslashes are retained. When the variable is expanded in the recipe, I would expect make to behave as if the recipe had \\ at the end of every line. If this were the case, each line would be executed separately. But as you can see from Examples 3 and 6, the lines are executed together.
The point is, it is possible to get magic backslash parsing from the expansion of a variable. The problem is the mechanics of this behavior are inconsistent and confusing.
Urk! This is due to make's noddy parser.
Recipes are stored as-is.
They are expanded as-and-when make is about to call the shell.
Once the entire recipe has been expanded,
the first line is passed to a shell.
If that command succeeds, then the second is run.
Wash, rinse, repeat.
Backslashes at the end of line are preserved,
with the effect that the following line is passed at the same time as the first.
In recursive variable definition however,
backslashes at the end of line are removed as the definition is read
define oneline =
aa \
bb \
cc
endef
$(error [$(value oneline)])
which gives
$ make
Makefile:9: *** [ aa bb cc]. Stop.
What we are aiming for
We need to massage make's syntax so that a variable expands to exactly this text:
target:
if [ -f foo ]; then \
printf "'%s' is a file\n" 'foo'; \
printf "This is a relatively long command that"; \
printf " won't fit on one line\n"; \
fi
which we then simply feed to make via something like
$(eval ${var})
Backslashes
Just replace each newline with a space-backslash-newline-tab quad using $(subst).
A function to do that:
empty :=
tab := ${empty} # Trailing tab
define \n :=
endef
stanza = $(subst ${\n}, \${\n}${tab})
To check it works:
define func =
if [ -f $1 ]; then
printf "'%s' is a file\n" '$1';
printf "This is a relatively long command that";
printf " won't fit on one line\n";
fi
endef
$(error [$(call stanza,$(call func,foo))])
giving:
Makefile:23: *** [ if [ -f foo ]; then \
printf "'%s' is a file\n" 'foo'; \
printf "This is a relatively long command that"; \
printf " won't fit on one line\n"; \
fi]. Stop.
Note that the definition of func now has no annotation at the end of its lines.
Putting it all together
define \n :=
endef
empty :=
tab := ${empty} # Trailing tab
stanza = $(subst ${\n}, \${\n}${tab},$1)
define func =
if [ -f $1 ]; then
printf "'%s' is a file\n" '$1';
printf "This is a relatively long command that";
printf " won't fit on one line\n";
fi
endef
define rule =
target:
echo vvv
$(call stanza,$(call func,foo))
echo ^^^
endef
$(eval ${rule})
Leading to
$ touch foo; make -R --warn
echo vvv
vvv
if [ -f foo ]; then \
printf "'%s' is a file\n" 'foo'; \
printf "This is a relatively long command that"; \
printf " won't fit on one line\n"; \
fi
'foo' is a file
This is a relatively long command that won't fit on one line
echo ^^^
^^^
An interesting academic exercise.
Still can't put literal backslashes in either :)
define newline :=
$(strip)
$(strip)
endef
define func =
if [ -f $(1) ]; then \$(newline)\
printf "'%s' is a file\n" '$(1)'; \$(newline)\
printf "This is a relatively long command that"; \$(newline)\
printf " won't fit on one line\n"; \$(newline)\
fi
endef
func2 = if [ -f $(1) ]; then \$(newline) printf "'%s' is a file\n" '$(1)'; \$(newline) printf "This is a relatively long command that"; \$(newline) printf " won't fit on one line\n"; \$(newline)fi
all:
$(call func,foo)
#echo --------------
$(call func2,foo)
The first one seems to be space-stripped. The second looks nice at the output but.. oh well, seems like being stuck between a rock and a hard place :/
I found a nicer (albeit still hacky) solution that seems to work:
define func =
if [ -f $(1) ]; then $\\
printf "'%s' is a file\n" '$(1)'; $\\
printf "This is a relatively long command that"; $\\
printf " won't fit on one line\n"; $\\
fi
endef
all:
#echo vvvvvvvv
$(call func,foo)
#echo ^^^^^^^^
Output:
$ make
vvvvvvvv
if [ -f foo ]; then \
printf "'%s' is a file\n" 'foo'; \
printf "This is a relatively long command that"; \
printf " won't fit on one line\n"; \
fi
'foo' is a file
This is a relatively long command that won't fit on one line
^^^^^^^^
I think this is how it works:
During the first scan for line continuations, the $ is ignored, so the
parser sees \\, assumes the backslash is escaped, and leaves the lines
intact.
When expanding the function, $\ is recognized as a variable. Assuming
you haven't actually assigned a variable named \, this expands to
nothing.
Now we are left with a single backslash at the end of the line, which is
treated as if it were literally typed into the recipe.
I have a couple of files with certain keywors (one is MAT1).
For this key word i like to read the ID corresponding to it, put this together with the filename into an array.
I tried the following (I am not very familiar with bash programming):
#!/bin/bash
Num=0
arr=( $(find . -name '*.mat' | sort) )
for i in "${arr[#]}"
do
file=$(basename "${i}")
while read -r line
do
name="$line"
IFS=' ' read -r -a array <<< "$line"
for index in "${!array[#]}"
do
if [ ${array[index]} == "MAT1" ]
then
out[$num] = "${array[index+1]} $file "
let num++
#printf "%-32s %8i\n" "$file" "${array[index+1]}"
fi
done
done < "$i"
done
With this get the message
make_mat_list.bsh: line 21: out[0]: command not found
make_mat_list.bsh: line 21: out[1]: command not found
What is wrong here?
bash is white-space sensitive, your line below cannot have spaces.
out[$num] = "${array[index+1]} $file "
As for the reason for the error, the shell treats that particular line as the first word being a command out[$num] i.e. out[1]..etc and rest of it as arguments to it = and "${array[index+1]} $file ", which does not make any sense. Remove the spaces and do jsut
out[$num]="${array[index+1]} $file"
I'm new to bash scripting and I'm asking for a little help !
I've got a little scipt in bash that is not making what I want (but almost) and the behavior of my echo command seems strange to me, look at it :
TST='test'
TEST="${ADDR[3]}"_"$TST"
echo $TEST
#result : _test
echo ${ADDR[3]}
#result : 5
How can you explain these results ? Thanks in advance :)
My ADDR var is defined like this :
#parsing the read line, split on whitespace
IFS=' ' read -ra ADDR <<< "$line"
Here is my complete script :
#!/bin/bash
NUMBER=2
{ read ;
while IFS= read -r line; do
echo "$NUMBER : $line"
IFS=' ' read -ra ADDR <<< "$line"
#If the countdown is set to 0, launch the task ans set it to init value
if [ ${ADDR[0]} == '0' ]; then
#task launching
echo `./${ADDR[1]}.sh ${ADDR[2]} &`
TST='test'
TEST=${ADDR[3]}_$TST
echo $TEST
VAR=$(echo -E "${ADDR[3]}" | tr -d '\n')
#countdown set to init value
sed -i "$NUMBER c $VAR ${ADDR[1]} ${ADDR[2]} ${ADDR[3]}" listing.txt
else
sed -i "$NUMBER c $((ADDR-1)) ${ADDR[1]} ${ADDR[2]} ${ADDR[3]}" listing.txt
fi
((NUMBER++))
done } < listing.txt
Answer: the following is fine,
TEST="${ADDR[3]}"_"$TST"
Although I would recommend.
TEST="${ADDR[3]}_${TST}"
What you need to do is dump ${ADDR[3]} before this statement and confirm that ADDR holds the expected values. You may as well dump the entire array with indexes and confirm all entries
for ((i=0; i<${#ADDR[#]}; i++)); do
printf "ADDR[%3d] %s\n" "$i" "${ADDR[$i]}"
done
This will help isolate the issue. Sorry for the earlier answer. Lesson [sleep 1st: answer 2nd]
Why does read behave differently with the same input from a pipe and a heredoc:
printf "" | while read line; do echo "line=$line"; done # outputs nothing
while read line; do echo "line=$line"; done <<< "" # outputs 'line='
How can I disable output in the second case?
The here document has an implicit newline (\n) at the end; printf "" outputs nothing whatsoever. I don't know offhand of a way to get rid of the implicit newline.
If you can discard all empty lines...
while read line; do if test -n "$line"; then echo "line=$line"; fi; done <<< ""
How about using $'\c':
man bash | less -p '\\c * suppress trailing newline'
str=""
while read line; do echo "line=$line"; done <<<$'\c'"${str}"
str="abc"
while read line; do echo "line=$line"; done <<<$'\c'"${str}"