I'm working on a script and it isn't clear how read -r line knows which variable to get the data from.. I want to read line by line from the FILE variable.
Here is the script I'm working on:
#!/bin/bash
cd "/"
FILE="$(< /home/FileSystemCorruptionTest/*.chk)"
while read -r line
do
echo "$line" > /home/FileSystemCorruptionTest/`date +%Y_%m_%d_%H_%M_%S`_1.log
done
echo "" > /home/FileSystemCorruptionTest/Done
Since it looks like you want to combine multiple files, I guess that I would regard this as a legitimate usage of cat:
cat /home/FileSystemCorruptionTest/*.chk | while read -r line
do
echo "$line"
done > /home/FileSystemCorruptionTest/`date +%Y_%m_%d_%H_%M_%S`_1.log
Note that I moved the redirect out of the loop, to prevent overwriting the file once per line.
Also note that your example could easily be written as:
cat /home/FileSystemCorruptionTest/*.chk > /home/FileSystemCorruptionTest/`date +%Y_%m_%d_%H_%M_%S`_1.log
If you only actually have one file (and want to store it inside a variable), then you can use <<< after the loop:
while read -r line
do
echo "$line"
done <<<"$FILE" > /home/FileSystemCorruptionTest/`date +%Y_%m_%d_%H_%M_%S`_1.log
<<< "$FILE" has the same effect as using echo "$FILE" | before the loop but it doesn't create any subshells.
What you are requesting:
echo "${FILE}" | while read -r line …
But I think Tom's solution is better.
Related
This question already has answers here:
Looping through the content of a file in Bash
(16 answers)
Closed 2 years ago.
I have the following .txt file:
Marco
Paolo
Antonio
I want to read it line-by-line, and for each line I want to assign a .txt line value to a variable. Supposing my variable is $name, the flow is:
Read first line from file
Assign $name = "Marco"
Do some tasks with $name
Read second line from file
Assign $name = "Paolo"
The following reads a file passed as an argument line by line:
while IFS= read -r line; do
echo "Text read from file: $line"
done < my_filename.txt
This is the standard form for reading lines from a file in a loop. Explanation:
IFS= (or IFS='') prevents leading/trailing whitespace from being trimmed.
-r prevents backslash escapes from being interpreted.
Or you can put it in a bash file helper script, example contents:
#!/bin/bash
while IFS= read -r line; do
echo "Text read from file: $line"
done < "$1"
If the above is saved to a script with filename readfile, it can be run as follows:
chmod +x readfile
./readfile filename.txt
If the file isn’t a standard POSIX text file (= not terminated by a newline character), the loop can be modified to handle trailing partial lines:
while IFS= read -r line || [[ -n "$line" ]]; do
echo "Text read from file: $line"
done < "$1"
Here, || [[ -n $line ]] prevents the last line from being ignored if it doesn't end with a \n (since read returns a non-zero exit code when it encounters EOF).
If the commands inside the loop also read from standard input, the file descriptor used by read can be chanced to something else (avoid the standard file descriptors), e.g.:
while IFS= read -r -u3 line; do
echo "Text read from file: $line"
done 3< "$1"
(Non-Bash shells might not know read -u3; use read <&3 instead.)
I encourage you to use the -r flag for read which stands for:
-r Do not treat a backslash character in any special way. Consider each
backslash to be part of the input line.
I am citing from man 1 read.
Another thing is to take a filename as an argument.
Here is updated code:
#!/usr/bin/bash
filename="$1"
while read -r line; do
name="$line"
echo "Name read from file - $name"
done < "$filename"
Using the following Bash template should allow you to read one value at a time from a file and process it.
while read name; do
# Do what you want to $name
done < filename
#! /bin/bash
cat filename | while read LINE; do
echo $LINE
done
Use:
filename=$1
IFS=$'\n'
for next in `cat $filename`; do
echo "$next read from $filename"
done
exit 0
If you have set IFS differently you will get odd results.
Many people have posted a solution that's over-optimized. I don't think it is incorrect, but I humbly think that a less optimized solution will be desirable to permit everyone to easily understand how is this working. Here is my proposal:
#!/bin/bash
#
# This program reads lines from a file.
#
end_of_file=0
while [[ $end_of_file == 0 ]]; do
read -r line
# the last exit status is the
# flag of the end of file
end_of_file=$?
echo $line
done < "$1"
If you need to process both the input file and user input (or anything else from stdin), then use the following solution:
#!/bin/bash
exec 3<"$1"
while IFS='' read -r -u 3 line || [[ -n "$line" ]]; do
read -p "> $line (Press Enter to continue)"
done
Based on the accepted answer and on the bash-hackers redirection tutorial.
Here, we open the file descriptor 3 for the file passed as the script argument and tell read to use this descriptor as input (-u 3). Thus, we leave the default input descriptor (0) attached to a terminal or another input source, able to read user input.
For proper error handling:
#!/bin/bash
set -Ee
trap "echo error" EXIT
test -e ${FILENAME} || exit
while read -r line
do
echo ${line}
done < ${FILENAME}
Use IFS (internal field separator) tool in bash, defines the character using to separate lines into tokens, by default includes <tab> /<space> /<newLine>
step 1: Load the file data and insert into list:
# declaring array list and index iterator
declare -a array=()
i=0
# reading file in row mode, insert each line into array
while IFS= read -r line; do
array[i]=$line
let "i++"
# reading from file path
done < "<yourFullFilePath>"
step 2: now iterate and print the output:
for line in "${array[#]}"
do
echo "$line"
done
echo specific index in array: Accessing to a variable in array:
echo "${array[0]}"
The following will just print out the content of the file:
cat $Path/FileName.txt
while read line;
do
echo $line
done
Im working on a bash script that recive a list of processes and do a bunch of things with them, however when I want to analyze them with a loop this error happens.
Here is some code:
#! /bin/bash
ls /proc | grep ^9 > processes.txt
cat processes.txt
for line in $processes.txt
do
echo "$line"
done
PD: Im preatty new to bash
$ does parameter expansion; it does not expand a file name to the contents of the file.
Use a while read loop instead.
while IFS= read -r line; do
echo "$line"
done < processes.txt
This question already has answers here:
Looping through the content of a file in Bash
(16 answers)
Closed 2 years ago.
I have the following .txt file:
Marco
Paolo
Antonio
I want to read it line-by-line, and for each line I want to assign a .txt line value to a variable. Supposing my variable is $name, the flow is:
Read first line from file
Assign $name = "Marco"
Do some tasks with $name
Read second line from file
Assign $name = "Paolo"
The following reads a file passed as an argument line by line:
while IFS= read -r line; do
echo "Text read from file: $line"
done < my_filename.txt
This is the standard form for reading lines from a file in a loop. Explanation:
IFS= (or IFS='') prevents leading/trailing whitespace from being trimmed.
-r prevents backslash escapes from being interpreted.
Or you can put it in a bash file helper script, example contents:
#!/bin/bash
while IFS= read -r line; do
echo "Text read from file: $line"
done < "$1"
If the above is saved to a script with filename readfile, it can be run as follows:
chmod +x readfile
./readfile filename.txt
If the file isn’t a standard POSIX text file (= not terminated by a newline character), the loop can be modified to handle trailing partial lines:
while IFS= read -r line || [[ -n "$line" ]]; do
echo "Text read from file: $line"
done < "$1"
Here, || [[ -n $line ]] prevents the last line from being ignored if it doesn't end with a \n (since read returns a non-zero exit code when it encounters EOF).
If the commands inside the loop also read from standard input, the file descriptor used by read can be chanced to something else (avoid the standard file descriptors), e.g.:
while IFS= read -r -u3 line; do
echo "Text read from file: $line"
done 3< "$1"
(Non-Bash shells might not know read -u3; use read <&3 instead.)
I encourage you to use the -r flag for read which stands for:
-r Do not treat a backslash character in any special way. Consider each
backslash to be part of the input line.
I am citing from man 1 read.
Another thing is to take a filename as an argument.
Here is updated code:
#!/usr/bin/bash
filename="$1"
while read -r line; do
name="$line"
echo "Name read from file - $name"
done < "$filename"
Using the following Bash template should allow you to read one value at a time from a file and process it.
while read name; do
# Do what you want to $name
done < filename
#! /bin/bash
cat filename | while read LINE; do
echo $LINE
done
Use:
filename=$1
IFS=$'\n'
for next in `cat $filename`; do
echo "$next read from $filename"
done
exit 0
If you have set IFS differently you will get odd results.
Many people have posted a solution that's over-optimized. I don't think it is incorrect, but I humbly think that a less optimized solution will be desirable to permit everyone to easily understand how is this working. Here is my proposal:
#!/bin/bash
#
# This program reads lines from a file.
#
end_of_file=0
while [[ $end_of_file == 0 ]]; do
read -r line
# the last exit status is the
# flag of the end of file
end_of_file=$?
echo $line
done < "$1"
If you need to process both the input file and user input (or anything else from stdin), then use the following solution:
#!/bin/bash
exec 3<"$1"
while IFS='' read -r -u 3 line || [[ -n "$line" ]]; do
read -p "> $line (Press Enter to continue)"
done
Based on the accepted answer and on the bash-hackers redirection tutorial.
Here, we open the file descriptor 3 for the file passed as the script argument and tell read to use this descriptor as input (-u 3). Thus, we leave the default input descriptor (0) attached to a terminal or another input source, able to read user input.
For proper error handling:
#!/bin/bash
set -Ee
trap "echo error" EXIT
test -e ${FILENAME} || exit
while read -r line
do
echo ${line}
done < ${FILENAME}
Use IFS (internal field separator) tool in bash, defines the character using to separate lines into tokens, by default includes <tab> /<space> /<newLine>
step 1: Load the file data and insert into list:
# declaring array list and index iterator
declare -a array=()
i=0
# reading file in row mode, insert each line into array
while IFS= read -r line; do
array[i]=$line
let "i++"
# reading from file path
done < "<yourFullFilePath>"
step 2: now iterate and print the output:
for line in "${array[#]}"
do
echo "$line"
done
echo specific index in array: Accessing to a variable in array:
echo "${array[0]}"
The following will just print out the content of the file:
cat $Path/FileName.txt
while read line;
do
echo $line
done
I have a text file, it contains a single word on each line.
I need a loop in bash to read each line, then perform a command each time it reads a line, using the input from that line as part of the command.
I am just not sure of the proper syntax to do this in bash. If anyone can help, it would be great. I need to use the line from the test file obtained as a paramter to call another function. The loop should stop when there are no more lines in the text file.
Psuedo code:
Read testfile.txt.
For each in testfile.txt
{
some_function linefromtestfile
}
How about:
while read line
do
echo $line
// or some_function "$line"
done < testfile.txt
As an alternative, using a file descriptor (#4 in this case):
file='testfile.txt'
exec 4<$file
while read -r -u4 t ; do
echo "$t"
done
Don't use cat! In a loop cat is almost always wrong, i.e.
cat testfile.txt | while read -r line
do
# do something with "$line" here
done
and people might start to throw an UUoCA at you.
while read line
do
nikto -Tuning x 1 6 -h $line -Format html -o NiktoSubdomainScans.html
done < testfile.txt
Tried this to automate nikto scan of list of domains after changing from cat approach. Still just read the first line and ignored everything else.
This question already has answers here:
Looping through the content of a file in Bash
(16 answers)
Closed 2 years ago.
I have the following .txt file:
Marco
Paolo
Antonio
I want to read it line-by-line, and for each line I want to assign a .txt line value to a variable. Supposing my variable is $name, the flow is:
Read first line from file
Assign $name = "Marco"
Do some tasks with $name
Read second line from file
Assign $name = "Paolo"
The following reads a file passed as an argument line by line:
while IFS= read -r line; do
echo "Text read from file: $line"
done < my_filename.txt
This is the standard form for reading lines from a file in a loop. Explanation:
IFS= (or IFS='') prevents leading/trailing whitespace from being trimmed.
-r prevents backslash escapes from being interpreted.
Or you can put it in a bash file helper script, example contents:
#!/bin/bash
while IFS= read -r line; do
echo "Text read from file: $line"
done < "$1"
If the above is saved to a script with filename readfile, it can be run as follows:
chmod +x readfile
./readfile filename.txt
If the file isn’t a standard POSIX text file (= not terminated by a newline character), the loop can be modified to handle trailing partial lines:
while IFS= read -r line || [[ -n "$line" ]]; do
echo "Text read from file: $line"
done < "$1"
Here, || [[ -n $line ]] prevents the last line from being ignored if it doesn't end with a \n (since read returns a non-zero exit code when it encounters EOF).
If the commands inside the loop also read from standard input, the file descriptor used by read can be chanced to something else (avoid the standard file descriptors), e.g.:
while IFS= read -r -u3 line; do
echo "Text read from file: $line"
done 3< "$1"
(Non-Bash shells might not know read -u3; use read <&3 instead.)
I encourage you to use the -r flag for read which stands for:
-r Do not treat a backslash character in any special way. Consider each
backslash to be part of the input line.
I am citing from man 1 read.
Another thing is to take a filename as an argument.
Here is updated code:
#!/usr/bin/bash
filename="$1"
while read -r line; do
name="$line"
echo "Name read from file - $name"
done < "$filename"
Using the following Bash template should allow you to read one value at a time from a file and process it.
while read name; do
# Do what you want to $name
done < filename
#! /bin/bash
cat filename | while read LINE; do
echo $LINE
done
Use:
filename=$1
IFS=$'\n'
for next in `cat $filename`; do
echo "$next read from $filename"
done
exit 0
If you have set IFS differently you will get odd results.
Many people have posted a solution that's over-optimized. I don't think it is incorrect, but I humbly think that a less optimized solution will be desirable to permit everyone to easily understand how is this working. Here is my proposal:
#!/bin/bash
#
# This program reads lines from a file.
#
end_of_file=0
while [[ $end_of_file == 0 ]]; do
read -r line
# the last exit status is the
# flag of the end of file
end_of_file=$?
echo $line
done < "$1"
If you need to process both the input file and user input (or anything else from stdin), then use the following solution:
#!/bin/bash
exec 3<"$1"
while IFS='' read -r -u 3 line || [[ -n "$line" ]]; do
read -p "> $line (Press Enter to continue)"
done
Based on the accepted answer and on the bash-hackers redirection tutorial.
Here, we open the file descriptor 3 for the file passed as the script argument and tell read to use this descriptor as input (-u 3). Thus, we leave the default input descriptor (0) attached to a terminal or another input source, able to read user input.
For proper error handling:
#!/bin/bash
set -Ee
trap "echo error" EXIT
test -e ${FILENAME} || exit
while read -r line
do
echo ${line}
done < ${FILENAME}
Use IFS (internal field separator) tool in bash, defines the character using to separate lines into tokens, by default includes <tab> /<space> /<newLine>
step 1: Load the file data and insert into list:
# declaring array list and index iterator
declare -a array=()
i=0
# reading file in row mode, insert each line into array
while IFS= read -r line; do
array[i]=$line
let "i++"
# reading from file path
done < "<yourFullFilePath>"
step 2: now iterate and print the output:
for line in "${array[#]}"
do
echo "$line"
done
echo specific index in array: Accessing to a variable in array:
echo "${array[0]}"
The following will just print out the content of the file:
cat $Path/FileName.txt
while read line;
do
echo $line
done