I'm a newbie here. I've really tried to google it but failed.
I've got a simple script that reads from file line by line and then prints it. "b.txt" has the first line of "02083192846".
#!/bin/bash
while IFS= read -r line; do
printf 'downloading %s .html\n' $line
done < "$1"
however, the output is screwed up:
User#User-pk ~/test
$ ./test.sh b.txt
.htmlading 02083192846
Once this is fixed, my next question is how to use this line as part of the file name that some command will store into. i.e. the filename should be "02083192846.html". I've tried setting using it as ${line}.html but it doesn't work. For example grep "foo" -f ${line}.html doesn't work. But curl http://foo -o $line.html does work still!
What I would do :
#!/bin/bash
while IFS= read -r line; do
printf 'downloading %s .html\n' "${line//$'\r'/}"
# ________
done < "$1"
# ^
# remove \r with bash parameter expansion
Related
Im working on a bash script that recive a list of processes and do a bunch of things with them, however when I want to analyze them with a loop this error happens.
Here is some code:
#! /bin/bash
ls /proc | grep ^9 > processes.txt
cat processes.txt
for line in $processes.txt
do
echo "$line"
done
PD: Im preatty new to bash
$ does parameter expansion; it does not expand a file name to the contents of the file.
Use a while read loop instead.
while IFS= read -r line; do
echo "$line"
done < processes.txt
Just noticed something strange which I can't quite explain:
When I split my $PATH variable using read -a everything works fine
IFS=: read -r -a lines <<< "$PATH"
for line in "${lines[#]}"; do echo "$line"; done
But when I try to do the same using while ... read loop, only the first line is printed
while IFS=: read -r line; do echo "$line"; done <<< "$PATH"
You can make this work; switch from using IFS=: to using -d:, and append a : to the end of your input stream:
while IFS= read -r -d: line; do echo "$line"; done <<< "$PATH:"
The difference is that IFS is used to find boundaries between words, but read -r line reads into exactly one variable, line, so it's not looking for multiple words at all. By contrast, -d tells each invocation of read which character to stop at; by default that's a newline, but you can replace it with any other single character. (If that character isn't found, read exits with a nonzero status; that's why the standard/idiomatic while read loop idiom skips the last line of your file if it isn't correctly terminated by a newline, and why we use $PATH: as our input here).
If you ran IFS=: read -r first second rest, on the other hand, it would put your first PATH entry into $first, the second one into $second, and the remainder of the line into $rest; whereas with IFS: read -r line, it's as if you only had a single item, $rest.
Your while loop processes 1 line, it is not a loop. So the complete path is stored in the field line.
When you had given more fields, the path would be divided to those fields (and the last field gets the remainder):
while IFS=: read -r line field2 field3 otherfields; do echo "$line"; done <<< "$PATH"
When you want to avoid an array, you can use
while read -r line; do echo "$line"; done <<< "${PATH//:/$'\n'}"
It works fine.
Splitting into an array gives an open-ended number of elements, so does what you expect.
Splitting into a single variable does the same thing, but when it runs out of supplied variable names into which to put the data, it's stops splitting and puts the rest into the last one.
Try this:
$: IFS=: read -r a b c <<< "$PATH"
$: printf "[%s]\n" "$a" "$b" "$c"
You'll get the first PATH element in $a, the second in $b, and the rest ALL in $c.
Does that make it clearer?
c.f. this guide
Why does splitting my $PATH with read -r -a line work but not with while read -r line?
Because read -r line reads the whole line and then after reading the single whole line then the line is spitt on IFS. Because you provided only one variable to read, all the line is in that one variable. You could like split the line on the first element and rest of elements:
IFS=: read -r part1 rest_of_parts <<<"$line"
See read 1p read the If there are fewer vars than fields, part. Note that still IFS=: read -r -a lines <<< "$PATH" will fail when PATH contains a newline, like so:
$ export PATH=/usr/bin # reset PATH to something short
$ cd /tmp/
$ mkdir temp$'\n'dir # create a directory with a newline in the name
$ ls -d tem*
'temp'$'\n''dir'/
$ cd temp$'\n'dir
$ printf "%s\n" '#!/bin/bash' 'echo hello world' > script.sh
$ chmod +x ./script.sh # add a script in that directory
$ export PATH="$PATH:$PWD" # add that directory to path
$ ./script.sh # yes. yes, it works
hello world
$ IFS=: read -r -a lines <<< "$PATH"
$ declare -p lines
declare -a lines=([0]="/usr/bin" [1]="/tmp/temp")
# ^^^^ newline and 'dir' is missing
# That is because `read` reads _one line_ and one line only
# _after_ reading that one line that _one line_ is split on IFS
# so any more lines are ignored.
You could use a bash extension to read -d that makes read not read the whole line, but up until a character (but I needed to ignore read exit status, dunno why):
$ while IFS= read -r -d':' line || [[ -n "$line" ]]; do declare -p line; done < <(printf "%s" "$PATH")
declare -- line="/usr/bin"
declare -- line="/tmp/temp
dir"
Note that <<< adds a trailing newline, so using that will result in the last element of PATH having a newline - as a workaround, in bash you may use process substitution < <(printf "%s" "$PATH").
The real safe solution if using bash is just using mapfile/readarray:
$ mapfile -d: -t lines < <(printf "%s" "$PATH")
$ declare -p lines
declare -a lines=([0]="/usr/bin" [1]=$'/tmp/temp\ndir')
I have a script which contains the file location and i am running a command to fix the files.
But i am not able to write the output to a file.
#!/bin/ksh
while read line
do
`shnfix "$line"` >> output.txt
done < filename.txt
The output generated after adding set -x in the beginning of the script.
+ < filename.txt
+ read line
+ shnfix /x01/naveen_wav/file1.wav
+ >> output.txt
Fixing [/x01/naveen_wav/file1.wav] (3:49.42) --> [file1-fixed.wav] : 100% OK
Padded last file with 1194 zero-bytes.
+ read line
+ shnfix /x01/naveen_wav/file2.wav
+ >> output.txt
Fixing [/x01/naveen_wav/file2.wav] (4:30.35) --> [file2-fixed.wav] : 100% OK
Padded last file with 644 zero-bytes.
+ read line
A more efficient version (I/O wise) of #gile's code:
#!/bin/ksh
filename="/path/to/filename.txt"
while IFS= read -r line
do
shnfix "$line"
done < filename.txt > output.txt
The output should be inside `
`shnfix $line >> output.txt`
So the script could be like this:
#!/bin/ksh
filename="/path/to/filename.txt"
while IFS= read -r line
do
# display line or do somthing on $line
echo "$line"
`shnfix $line >> output.txt`
done <"$fileName"
Just remove the backticks, they are at least confusing.
Written as it is the shell will exeute the command in a subshell and try to execute the result, adding the redirection to it.
I assume you don't want to execute the output, but you want to redirect the output.
If the output starts with, for example line, that is a correct unix command, which does not create output you don't see an error, but neither the output.
I get a test.ksh[4]: line1: not found [No such file or directory] where 'line1' is the first line in my test file.
Or keep it in a block and redirect all output from it. Makes the intention more clear and it is easier to add commnds.
#!/bin/ksh
filename="/path/to/filename.txt"
{
while IFS= read -r line
do
shnfix "$line"
done < "$filename"
} > output.txt
http://www.shellcheck.net (a big troubleshooter) will give similar hints
I'm trying to use the read builtin in bash to read one character at a time. This actually works flawlessly when I use the -N 1 argument to read, but I had some OSX users report to me that their bash does not have that option.
So now I'm using something along the lines of:
$ while IFS= read -r -d'' -s -n 1 char; do echo -n "${char}"; done < filename
This echo's back every character in filename one at a time except, mysteriously, hyphens (-). E.g. if I have
$ cat blah
uh-oh
The result is:
$ while IFS= read -r -d'' -s -n 1 char; do echo -n "${char}"; done < blah
uhoh
Nothing in the documentation says anything that would indicate this. If I replace ${char} in the echo with ${#char} it prints 0 where it should have read the hyphen. It just gets completely eaten.
If I drop the -d'' it instead eats newlines, but does not eat hyphen, so that at least makes sense since newline is the default delimiter. It almost seems like a bug that -d'' is treating hyphen as a delimiter.
FWIW I have
$ bash --version
GNU bash, version 4.3.42(4)-release (x86_64-unknown-cygwin)
but this was first reported to me by an OSX user.
You actually did not set the delimiter option for read correctly.
Notice the extra space:
while IFS= read -r -d '' -s -n 1 char; do echo -n "${char}"; done < filename
This works fine.
In your code the delimiter chars where set to -s
I'm working on a script and it isn't clear how read -r line knows which variable to get the data from.. I want to read line by line from the FILE variable.
Here is the script I'm working on:
#!/bin/bash
cd "/"
FILE="$(< /home/FileSystemCorruptionTest/*.chk)"
while read -r line
do
echo "$line" > /home/FileSystemCorruptionTest/`date +%Y_%m_%d_%H_%M_%S`_1.log
done
echo "" > /home/FileSystemCorruptionTest/Done
Since it looks like you want to combine multiple files, I guess that I would regard this as a legitimate usage of cat:
cat /home/FileSystemCorruptionTest/*.chk | while read -r line
do
echo "$line"
done > /home/FileSystemCorruptionTest/`date +%Y_%m_%d_%H_%M_%S`_1.log
Note that I moved the redirect out of the loop, to prevent overwriting the file once per line.
Also note that your example could easily be written as:
cat /home/FileSystemCorruptionTest/*.chk > /home/FileSystemCorruptionTest/`date +%Y_%m_%d_%H_%M_%S`_1.log
If you only actually have one file (and want to store it inside a variable), then you can use <<< after the loop:
while read -r line
do
echo "$line"
done <<<"$FILE" > /home/FileSystemCorruptionTest/`date +%Y_%m_%d_%H_%M_%S`_1.log
<<< "$FILE" has the same effect as using echo "$FILE" | before the loop but it doesn't create any subshells.
What you are requesting:
echo "${FILE}" | while read -r line …
But I think Tom's solution is better.