Why does read behave differently with the same input from a pipe and a heredoc:
printf "" | while read line; do echo "line=$line"; done # outputs nothing
while read line; do echo "line=$line"; done <<< "" # outputs 'line='
How can I disable output in the second case?
The here document has an implicit newline (\n) at the end; printf "" outputs nothing whatsoever. I don't know offhand of a way to get rid of the implicit newline.
If you can discard all empty lines...
while read line; do if test -n "$line"; then echo "line=$line"; fi; done <<< ""
How about using $'\c':
man bash | less -p '\\c * suppress trailing newline'
str=""
while read line; do echo "line=$line"; done <<<$'\c'"${str}"
str="abc"
while read line; do echo "line=$line"; done <<<$'\c'"${str}"
Related
I have problem with url formatting in bash script. In below code url request:
text="$(lynx --dump https://address/"${array[${i}]}")"
returns HTTP Error 400. The request URL is invalid. I assume that on
"${array[${i}]}"
is something wrong in url part. But I can't figure out what is right format.
#!/bin/bash
saveIFS="$IFS"
IFS=$'\n'
array=($(<words))
IFS="$saveIFS"
elements=${#array[#]}
for (( i=0;i<$elements;i++))
do
text="$(lynx --dump https://address/"${array[${i}]}")"
echo "$text" >> "outputfilename"
fi
done
I also tried:
text="$(lynx --dump https://address/${array[${i}]})"
Try
#!/bin/bash
IFS=$'\n' read -rd '' -a array <words
elements=${#array[#]}
for (( i=0;i<$elements;i++))
do
text="$(lynx --dump https://address/"${array[${i}]}")"
echo "$text" >> "outputfilename"
done
The array variable wasn't being set with array=($(<words))
You can use read or readarray, but this example is with read
Incidentally, putting IFS=$'\n' before read without a command separator ; sets $IFS only for the read command, removing the need to save and re-set $IFS
You don't need an array at all; the following will work in any POSIX-compatible shell, assuming you have one URL component per line:
while IFS= read -r line; do
text=$(lynx --dump https://address/"$line")
echo "$text"
done < words >> output filename
My two cents...
I prefer use printf -v for this, and this could be build like a filter:
catWeb() {
while IFS= read -r word;do
printf -v url "https://address/%s" "$word"
lynx --dump "$url"
done
}
catWeb <words >outputfilename
I was reading windows file. Lines ended with CR LF. So address contains
\r
character. I can remove it:
array[${i}]=${array[${i}]%$'\r'}
Or I can reformat input file so lines end only with LF.
Main structure of working script reading from CR LF file is
#!/bin/bash
IFS=$'\n' read -rd '' -a array <words
elements=${#array[#]}
for (( i=0;i<$elements;i++))
do
array[${i}]=${array[${i}]%$'\r'}
text="$(lynx --dump https://adrress/"${array[${i}]}")"
if [ ${#text} -gt 1 ]
then
echo "$text" >> "filename"
else
echo "${array[${i}]}" >> "filename2"
fi
done
I'm new to bash scripting and I'm asking for a little help !
I've got a little scipt in bash that is not making what I want (but almost) and the behavior of my echo command seems strange to me, look at it :
TST='test'
TEST="${ADDR[3]}"_"$TST"
echo $TEST
#result : _test
echo ${ADDR[3]}
#result : 5
How can you explain these results ? Thanks in advance :)
My ADDR var is defined like this :
#parsing the read line, split on whitespace
IFS=' ' read -ra ADDR <<< "$line"
Here is my complete script :
#!/bin/bash
NUMBER=2
{ read ;
while IFS= read -r line; do
echo "$NUMBER : $line"
IFS=' ' read -ra ADDR <<< "$line"
#If the countdown is set to 0, launch the task ans set it to init value
if [ ${ADDR[0]} == '0' ]; then
#task launching
echo `./${ADDR[1]}.sh ${ADDR[2]} &`
TST='test'
TEST=${ADDR[3]}_$TST
echo $TEST
VAR=$(echo -E "${ADDR[3]}" | tr -d '\n')
#countdown set to init value
sed -i "$NUMBER c $VAR ${ADDR[1]} ${ADDR[2]} ${ADDR[3]}" listing.txt
else
sed -i "$NUMBER c $((ADDR-1)) ${ADDR[1]} ${ADDR[2]} ${ADDR[3]}" listing.txt
fi
((NUMBER++))
done } < listing.txt
Answer: the following is fine,
TEST="${ADDR[3]}"_"$TST"
Although I would recommend.
TEST="${ADDR[3]}_${TST}"
What you need to do is dump ${ADDR[3]} before this statement and confirm that ADDR holds the expected values. You may as well dump the entire array with indexes and confirm all entries
for ((i=0; i<${#ADDR[#]}; i++)); do
printf "ADDR[%3d] %s\n" "$i" "${ADDR[$i]}"
done
This will help isolate the issue. Sorry for the earlier answer. Lesson [sleep 1st: answer 2nd]
I'm trying to parse a csv file I made with Google Spreadsheet. It's very simple for testing purposes, and is basically:
1,2
3,4
5,6
The problem is that the csv doesn't end in a newline character so when I cat the file in BASH, I get
MacBook-Pro:Desktop kkSlider$ cat test.csv
1,2
3,4
5,6MacBook-Pro:Desktop kkSlider$
I just want to read line by line in a BASH script using a while loop that every guide suggests, and my script looks like this:
while IFS=',' read -r last first
do
echo "$last $first"
done < test.csv
The output is:
MacBook-Pro:Desktop kkSlider$ ./test.sh
1 2
3 4
Any ideas on how I could have it read that last line and echo it?
Thanks in advance.
You can force the input to your loop to end with a newline thus:
#!/bin/bash
(cat test.csv ; echo) | while IFS=',' read -r last first
do
echo "$last $first"
done
Unfortunately, this may result in an empty line at the end of your output if the input already has a newline at the end. You can fix that with a little addition:
!/bin/bash
(cat test.csv ; echo) | while IFS=',' read -r last first
do
if [[ $last != "" ]] ; then
echo "$last $first"
fi
done
Another method relies on the fact that the values are being placed into the variables by the read but they're just not being output because of the while statement:
#!/bin/bash
while IFS=',' read -r last first
do
echo "$last $first"
done <test.csv
if [[ $last != "" ]] ; then
echo "$last $first"
fi
That one works without creating another subshell to modify the input to the while statement.
Of course, I'm assuming here that you want to do more inside the loop that just output the values with a space rather than a comma. If that's all you wanted to do, there are other tools better suited than a bash read loop, such as:
tr "," " " <test.csv
cat file |sed -e '${/^$/!s/$/\n/;}'| while IFS=',' read -r last first; do echo "$last $first"; done
If the last (unterminated) line needs to be processed differently from the rest, #paxdiablo's version with the extra if statement is the way to go; but if it's going to be handled like all the others, it's cleaner to process it in the main loop.
You can roll the "if there was an unterminated last line" into the main loop condition like this:
while IFS=',' read -r last first || [ -n "$last" ]
do
echo "$last $first"
done < test.csv
I need to to analyze (with grep) and print (with some formatting) the content of an
app's log.
This log contains text data in variable length lines. What I need is, after some grepping, loop each line of this output and print it with a maximum fixed length of 50 characters. If a line is longer than 50 chars, it should print a newline and then continue with the rest in the following line and so on until the line is completed.
I tried to use printf to do this, but it's not working and I don't know why. It just outputs the lines in same fashion of echo, without any consideration about printf formatting, though the \t character (tab) works.
function printContext
{
str="$1"
log="$2"
tmp="/tmp/deluge/$$"
rm -f $tmp
echo ""
echo -e "\tLog entries for $str :"
ln=$(grep -F "$str" "$log" &> "$tmp" ; cat "$tmp" | wc -l)
if [ $ln -gt 0 ];
then
while read line
do
printf "\t%50s\n" "$line"
done < $tmp
fi
}
What's wrong? I Know that I can make a substring routine to accomplish this task, but printf should be handy for stuff like this.
Instead of:
printf "\t%50s\n" "$line"
use
printf "\t%.50s\n" "$line"
to truncate your line to 50 characters only.
I'm not sure about printf but seeing as how perl is installed everywhere, how about a simple 1 liner?
echo $ln | perl -ne ' while( m/.{1,50}/g ){ print "$&\n" } '
Here's a clunky bash-only way to break the string into 50-character chunks
i=0
chars=50
while [[ -n "${y:$((chars*i)):$chars}" ]]; do
printf "\t%s\n" "${y:$((chars*i)):$chars}"
((i++))
done
I am having a bash script which is something like following,
cat filename | while read line
do
read input;
echo $input;
done
but this is clearly not giving me the right output as when I do read in the while loop it tries to read from the file filename because of the possible I/O redirection.
Any other way of doing the same?
Read from the controlling terminal device:
read input </dev/tty
more info: http://compgroups.net/comp.unix.shell/Fixing-stdin-inside-a-redirected-loop
You can redirect the regular stdin through unit 3 to keep the get it inside the pipeline:
{ cat notify-finished | while read line; do
read -u 3 input
echo "$input"
done; } 3<&0
BTW, if you really are using cat this way, replace it with a redirect and things become even easier:
while read line; do
read -u 3 input
echo "$input"
done 3<&0 <notify-finished
Or, you can swap stdin and unit 3 in that version -- read the file with unit 3, and just leave stdin alone:
while read line <&3; do
# read & use stdin normally inside the loop
read input
echo "$input"
done 3<notify-finished
Try to change the loop like this:
for line in $(cat filename); do
read input
echo $input;
done
Unit test:
for line in $(cat /etc/passwd); do
read input
echo $input;
echo "[$line]"
done
I have found this parameter -u with read.
"-u 1" means "read from stdout"
while read -r newline; do
((i++))
read -u 1 -p "Doing $i""th file, called $newline. Write your answer and press Enter!"
echo "Processing $newline with $REPLY" # united input from two different read commands.
done <<< $(ls)
It looks like you read twice, the read inside the while loop is not needed. Also, you don't need to invoke the cat command:
while read input
do
echo $input
done < filename
echo "Enter the Programs you want to run:"
> ${PROGRAM_LIST}
while read PROGRAM_ENTRY
do
if [ ! -s ${PROGRAM_ENTRY} ]
then
echo ${PROGRAM_ENTRY} >> ${PROGRAM_LIST}
else
break
fi
done