I have this test script:
#!/bin/bash
echo "Read a variable"
#open file
exec 6<test.txt
read EXAMPLE <&6
#close file again
exec 6<&-
echo $EXAMPLE
The file test.txt has only one line:
EXAMPLE=1
The output is:
bash-3.2$ ./Read_Variables.sh
Read the variable
EXAMPLE=1
I need just to use the value of $EXAMPLE, in this case 1. So how can I avoid getting the EXAMPLE= part in the output?
Thanks
If the file containing your variables is using bash syntax throughout (e.g. X=Y), another option is to use source:
#!/bin/bash
echo "Read a variable"
source test.txt
echo $EXAMPLE
As an alternative to sourcing the entire file, you can try the following:
while read line; do
[[ $line =~ EXAMPLE= ]] && declare "$line" && break
done < test.txt
which will scan the file until it finds the first line that looks like an assignment to EXAMPLE, then use the declare builtin to perform the assignment. It's probably a little slower, but it's more selective about what is actually executed.
I think the most proper way to do this is by sourcing the file which contains the variable (if it has bash syntax), but if I were to do that, I'd source it in a subshell, so that if there are ever other variables declared there, they won't override any important variables in current shell:
(. test.txt && echo $EXAMPLE)
You could read the line in as an array (notice the -a option) which can then be indexed into:
# ...
IFS='=' read -a EXAMPLE <&6
echo ${EXAMPLE[0]} # EXAMPLE
echo ${EXAMPLE[1]} # 1
# ...
This call to read splits the input line on the IFS and puts the remaining parts into an indexed array.
See help read for more information about read options and behaviour.
You could also manipulate the EXAMPLE variable directly:
# ...
read EXAMPLE <&6
echo ${EXAMPLE##*=} # 1
# ...
If all you need is to "import" other Bash declarations from a file you should just use:
source file
Related
This question already has answers here:
Looping through the content of a file in Bash
(16 answers)
Closed 2 years ago.
I have the following .txt file:
Marco
Paolo
Antonio
I want to read it line-by-line, and for each line I want to assign a .txt line value to a variable. Supposing my variable is $name, the flow is:
Read first line from file
Assign $name = "Marco"
Do some tasks with $name
Read second line from file
Assign $name = "Paolo"
The following reads a file passed as an argument line by line:
while IFS= read -r line; do
echo "Text read from file: $line"
done < my_filename.txt
This is the standard form for reading lines from a file in a loop. Explanation:
IFS= (or IFS='') prevents leading/trailing whitespace from being trimmed.
-r prevents backslash escapes from being interpreted.
Or you can put it in a bash file helper script, example contents:
#!/bin/bash
while IFS= read -r line; do
echo "Text read from file: $line"
done < "$1"
If the above is saved to a script with filename readfile, it can be run as follows:
chmod +x readfile
./readfile filename.txt
If the file isn’t a standard POSIX text file (= not terminated by a newline character), the loop can be modified to handle trailing partial lines:
while IFS= read -r line || [[ -n "$line" ]]; do
echo "Text read from file: $line"
done < "$1"
Here, || [[ -n $line ]] prevents the last line from being ignored if it doesn't end with a \n (since read returns a non-zero exit code when it encounters EOF).
If the commands inside the loop also read from standard input, the file descriptor used by read can be chanced to something else (avoid the standard file descriptors), e.g.:
while IFS= read -r -u3 line; do
echo "Text read from file: $line"
done 3< "$1"
(Non-Bash shells might not know read -u3; use read <&3 instead.)
I encourage you to use the -r flag for read which stands for:
-r Do not treat a backslash character in any special way. Consider each
backslash to be part of the input line.
I am citing from man 1 read.
Another thing is to take a filename as an argument.
Here is updated code:
#!/usr/bin/bash
filename="$1"
while read -r line; do
name="$line"
echo "Name read from file - $name"
done < "$filename"
Using the following Bash template should allow you to read one value at a time from a file and process it.
while read name; do
# Do what you want to $name
done < filename
#! /bin/bash
cat filename | while read LINE; do
echo $LINE
done
Use:
filename=$1
IFS=$'\n'
for next in `cat $filename`; do
echo "$next read from $filename"
done
exit 0
If you have set IFS differently you will get odd results.
Many people have posted a solution that's over-optimized. I don't think it is incorrect, but I humbly think that a less optimized solution will be desirable to permit everyone to easily understand how is this working. Here is my proposal:
#!/bin/bash
#
# This program reads lines from a file.
#
end_of_file=0
while [[ $end_of_file == 0 ]]; do
read -r line
# the last exit status is the
# flag of the end of file
end_of_file=$?
echo $line
done < "$1"
If you need to process both the input file and user input (or anything else from stdin), then use the following solution:
#!/bin/bash
exec 3<"$1"
while IFS='' read -r -u 3 line || [[ -n "$line" ]]; do
read -p "> $line (Press Enter to continue)"
done
Based on the accepted answer and on the bash-hackers redirection tutorial.
Here, we open the file descriptor 3 for the file passed as the script argument and tell read to use this descriptor as input (-u 3). Thus, we leave the default input descriptor (0) attached to a terminal or another input source, able to read user input.
For proper error handling:
#!/bin/bash
set -Ee
trap "echo error" EXIT
test -e ${FILENAME} || exit
while read -r line
do
echo ${line}
done < ${FILENAME}
Use IFS (internal field separator) tool in bash, defines the character using to separate lines into tokens, by default includes <tab> /<space> /<newLine>
step 1: Load the file data and insert into list:
# declaring array list and index iterator
declare -a array=()
i=0
# reading file in row mode, insert each line into array
while IFS= read -r line; do
array[i]=$line
let "i++"
# reading from file path
done < "<yourFullFilePath>"
step 2: now iterate and print the output:
for line in "${array[#]}"
do
echo "$line"
done
echo specific index in array: Accessing to a variable in array:
echo "${array[0]}"
The following will just print out the content of the file:
cat $Path/FileName.txt
while read line;
do
echo $line
done
To get started, here's the script I'm running to get the offending string:
# sed finds all sourced file paths from inputted file.
#
# while reads each match output from sed to $SOURCEFILE variable.
# Each should be a file path, or a variable that represents a file path.
# Any variables found should be expanded to the full path.
#
# echo and calls are used for demonstractive purposes only
# I intend to do something else with the path once it's expanded.
PATH_SOME_SCRIPT="/path/to/bash/script"
while read -r SOURCEFILE; do
echo "$SOURCEFILE"
"$SOURCEFILE"
$SOURCEFILE
done < <(cat $PATH_SOME_SCRIPT | sed -n -e "s/^\(source\|\.\|\$include\) //p")
You may also wish to use the following to test this out as mock data:
[ /path/to/bash/script ]
#!/bin/bash
source "$HOME/bash_file"
source "$GLOBAL_VAR_SCRIPT_PATH"
echo "No cow powers here"
For the tl;dr crew, basically the while loop spits out the following on the mock data:
"$HOME/bash_file"
bash: "$HOME/bash_file": no such file or directory
bash: "$HOME/bash_file": no such file or directory
"$GLOBAL_VAR_SCRIPT_PATH"
"$GLOBAL_VAR_SCRIPT_PATH": command not found
"$GLOBAL_VAR_SCRIPT_PATH": command not found
My question is, can you get the variable to expand correctly, e.g., print "/home//bash_file" and "/expanded/variable/path"? I should also state that although eval works I do not intend to use it because of its potential insecurities.
Protip that any variable value used in cat | sed would be available globally, including to the calling script, so it's not because the script cannot call the variable value.
FIRST SOLUTION ATTEMPT
Using anubhava's envsubst solution:
SOMEVARIABLE="/home/nick/.some_path"
while read -r SOURCEFILE; do
echo "$SOURCEFILE"
envsubst <<< "$SOURCEFILE";
done < <(echo -e "\"\$SOMEVARIABLE\"\n\"$HOME/.another_file\"")
This outputs the following:
"$SOMEVARIABLE"
""
"/home/nick/.another_file"
"/home/nick/.another_file"
Unfortunately, it does not expand the variable! Oh dear :(
SECOND SOLUTION ATTEMPT
Based upon the first attempt:
export SOMEVARIABLE="/home/nick/.some_path"
while read -r SOURCEFILE; do
echo "$SOURCEFILE"
envsubst <<< "$SOURCEFILE";
done < <(echo -e "\"\$SOMEVARIABLE\"\n\"$HOME/.another_file\"")
unset SOMEVARIABLE
which produces the results we wanted without eval and without messing with global variables (for too long anyway), hoorah!
Good runner-ups were further suggested using eval (although potentially unsafe) which can be found in this answer and here (link courtesy of anubhava's extended comments).
My question is, can you get the variable to expand correctly, e.g., print "/home//bash_file" and "/expanded/variable/path"?
Yes you can use envsubst program, that substitutes the values of environment variables:
while read -r sourceFile; do
envsubst <<< "$sourceFile"
done < <(sed -n "s/^\(source\|\.\|\$include\) //p" "$PATH_SOME_SCRIPT")
I think you are asking how to recursively expand variables in bash. Try
expanded=$(eval echo $SOURCEFILE)
inside your loop. eval runs the expanded command you give it. Since $SOURCEFILE isn't in quotes, it will be expanded to, e.g., $HOME/whatever. Then the eval will expand the $HOME before passing it to echo. echo will print the result, and expanded=$(...) will put the printed result in $expanded.
Script doesn't work when I want to use standard input when there are no arguments (files) passed. Is there any way how to use stdin instead of a file in this code?
I tried this:
if [ ! -n $1 ] # check if argument exists
then
$1=$(</dev/stdin) # if not use stdin as an argument
fi
var="$1"
while read line
do
... # find the longest line
done <"$var"
For a general case of wanting to read a value from stdin when a parameter is missing, this will work.
$ echo param | script.sh
$ script.sh param
script.sh
#!/bin/bash
set -- "${1:-$(</dev/stdin)}" "${#:2}"
echo $1
Just substitute bash's specially interpreted /dev/stdin as the filename:
VAR=$1
while read blah; do
...
done < "${VAR:-/dev/stdin}"
(Note that bash will actually use that special file /dev/stdin if built for an OS that offers it, but since bash 2.04 will work around that file's absence on systems that do not support it.)
pilcrow's answer provides an elegant solution; this is an explanation of why the OP's approach didn't work.
The main problem with the OP's approach was the attempt to assign to positional parameter $1 with $1=..., which won't work.
The LHS is expanded by the shell to the value of $1, and the result is interpreted as the name of the variable to assign to - clearly, not the intent.
The only way to assign to $1 in bash is via the set builtin.
The caveat is that set invariably sets all positional parameters, so you have to include the other ones as well, if any.
set -- "${1:-/dev/stdin}" "${#:2}" # "${#:2}" expands to all remaining parameters
(If you expect only at most 1 argument, set -- "${1:-/dev/stdin}" will do.)
The above also corrects a secondary problem with the OP's approach: the attempt to store the contents rather than the filename of stdin in $1, since < is used.
${1:-/dev/stdin} is an application of bash parameter expansion that says: return the value of $1, unless $1 is undefined (no argument was passed) or its value is the empty string (""or '' was passed). The variation ${1-/dev/stdin} (no :) would only return /dev/stdin if $1 is undefined (if it contains any value, even the empty string, it would be returned).
If we put it all together:
# Default to filename '/dev/stdin' (stdin), if none was specified.
set -- "${1:-/dev/stdin}" "${#:2}"
while read -r line; do
... # find the longest line
done < "$1"
But, of course, the much simpler approach would be to use ${1:-/dev/stdin} as the filename directly:
while read -r line; do
... # find the longest line
done < "${1:-/dev/stdin}"
or, via an intermediate variable:
filename=${1:-/dev/stdin}
while read -r line; do
... # find the longest line
done < "$filename"
Variables are assigned a value by Var=Value and that variable is used by e.g. echo $Var. In your case, that would amount to
1=$(</dev/stdin)
when assigning the standard input. However, I do not think that variable names are allowed to start with a digit character. See the question bash read from file or stdin for ways to solve this.
Here is my version of script:
#!/bin/bash
file=${1--} # POSIX-compliant; ${1:--} can be used either.
while IFS= read -r line; do
printf '%s\n' "$line"
done < <(cat -- "$file")
If file is not present in the argument, read the from standard input.
See more examples: How to read from file or stdin in bash? at stackoverflow SE
This question already has answers here:
Looping through the content of a file in Bash
(16 answers)
Closed 2 years ago.
I have the following .txt file:
Marco
Paolo
Antonio
I want to read it line-by-line, and for each line I want to assign a .txt line value to a variable. Supposing my variable is $name, the flow is:
Read first line from file
Assign $name = "Marco"
Do some tasks with $name
Read second line from file
Assign $name = "Paolo"
The following reads a file passed as an argument line by line:
while IFS= read -r line; do
echo "Text read from file: $line"
done < my_filename.txt
This is the standard form for reading lines from a file in a loop. Explanation:
IFS= (or IFS='') prevents leading/trailing whitespace from being trimmed.
-r prevents backslash escapes from being interpreted.
Or you can put it in a bash file helper script, example contents:
#!/bin/bash
while IFS= read -r line; do
echo "Text read from file: $line"
done < "$1"
If the above is saved to a script with filename readfile, it can be run as follows:
chmod +x readfile
./readfile filename.txt
If the file isn’t a standard POSIX text file (= not terminated by a newline character), the loop can be modified to handle trailing partial lines:
while IFS= read -r line || [[ -n "$line" ]]; do
echo "Text read from file: $line"
done < "$1"
Here, || [[ -n $line ]] prevents the last line from being ignored if it doesn't end with a \n (since read returns a non-zero exit code when it encounters EOF).
If the commands inside the loop also read from standard input, the file descriptor used by read can be chanced to something else (avoid the standard file descriptors), e.g.:
while IFS= read -r -u3 line; do
echo "Text read from file: $line"
done 3< "$1"
(Non-Bash shells might not know read -u3; use read <&3 instead.)
I encourage you to use the -r flag for read which stands for:
-r Do not treat a backslash character in any special way. Consider each
backslash to be part of the input line.
I am citing from man 1 read.
Another thing is to take a filename as an argument.
Here is updated code:
#!/usr/bin/bash
filename="$1"
while read -r line; do
name="$line"
echo "Name read from file - $name"
done < "$filename"
Using the following Bash template should allow you to read one value at a time from a file and process it.
while read name; do
# Do what you want to $name
done < filename
#! /bin/bash
cat filename | while read LINE; do
echo $LINE
done
Use:
filename=$1
IFS=$'\n'
for next in `cat $filename`; do
echo "$next read from $filename"
done
exit 0
If you have set IFS differently you will get odd results.
Many people have posted a solution that's over-optimized. I don't think it is incorrect, but I humbly think that a less optimized solution will be desirable to permit everyone to easily understand how is this working. Here is my proposal:
#!/bin/bash
#
# This program reads lines from a file.
#
end_of_file=0
while [[ $end_of_file == 0 ]]; do
read -r line
# the last exit status is the
# flag of the end of file
end_of_file=$?
echo $line
done < "$1"
If you need to process both the input file and user input (or anything else from stdin), then use the following solution:
#!/bin/bash
exec 3<"$1"
while IFS='' read -r -u 3 line || [[ -n "$line" ]]; do
read -p "> $line (Press Enter to continue)"
done
Based on the accepted answer and on the bash-hackers redirection tutorial.
Here, we open the file descriptor 3 for the file passed as the script argument and tell read to use this descriptor as input (-u 3). Thus, we leave the default input descriptor (0) attached to a terminal or another input source, able to read user input.
For proper error handling:
#!/bin/bash
set -Ee
trap "echo error" EXIT
test -e ${FILENAME} || exit
while read -r line
do
echo ${line}
done < ${FILENAME}
Use IFS (internal field separator) tool in bash, defines the character using to separate lines into tokens, by default includes <tab> /<space> /<newLine>
step 1: Load the file data and insert into list:
# declaring array list and index iterator
declare -a array=()
i=0
# reading file in row mode, insert each line into array
while IFS= read -r line; do
array[i]=$line
let "i++"
# reading from file path
done < "<yourFullFilePath>"
step 2: now iterate and print the output:
for line in "${array[#]}"
do
echo "$line"
done
echo specific index in array: Accessing to a variable in array:
echo "${array[0]}"
The following will just print out the content of the file:
cat $Path/FileName.txt
while read line;
do
echo $line
done
I have a file with 2 lines and I want to read them into 2 variables respectively. How do I accomplish this in shellscript(bash)?
You can open file descriptors in a shell to read the variables:
#!/bin/bash
# open file
exec 6<tst.txt
read foo <&6
read bar <&6
# close file again
exec 6<&-
echo $foo $bar
EDIT:
As a quick explanation, this is using IO redirection. Normally the file descriptors are handled as follows:
0 stdin (input)
1 stdout (output)
2 stderr (error)
However, there's nothing preventing from using other file descriptors (up to 9), so we're opening the "tst.txt" file in file descriptor 6, and read from it using IO redirection.
So, exec 6<tst.txt opens file descriptor 6 and redirects tst.txt into it, whereas exec 6<&- closes it again.
I'm unfortunately not on linux right now to test, but this would be close.
#!/bin/bash
file="/path/to/file"
# Store the previous IFS so we don't break anything else in the script.
prevIFS='$IFS'
# You need the line break to capture a newline.
IFS='
'
read var1 var2 < $file
echo "Var1: $var1"
echo "Var2: $var2"
# Set IFS back to normal
IFS='$prevIFS'
The simplest answer would be using sed command. Assuming that your file name is file.txt
var1=($(sed '1q;d' file.txt))
var2=($(sed '2q;d' file.txt))
Where 1q and 2q defines the line number.
All the values in Line 1 will be assigned to var1 and similarly to var2.
try this
#!/bin/bash
I=0
while read; do
VAR[$I]=$REPLY
((I++))
done < file
echo ${VAR[0]}
echo ${VAR[1]}
this will work with a file with more than 2 lines
Can you reconfigure the input file (with variables) to work as shell code? i.e.
$ cat varFile
var1=xyz
var2=abc
$ cat myShellScript.sh
#/bin/whatever (bash)?
# source the variable file
. /path/to/varFile
echo $var1
echo $var2
This is a standard concept in shell scripting and makes it much easier to manage configuration issues where you need to control your (unix/linux) environment based on which physical hardware you are running your system. If this is part of you concern, please let me know and I'll update the sample code to extend on this technique.
I hope this helps.