This question already has answers here:
While loop stops reading after the first line in Bash
(5 answers)
Closed 2 years ago.
I am trying to execute a simple script to capture multiple server's details using svmatch on server names input from a file.
#!/bin/sh
while read line; do
svmatch $line
done < ~/svr_input;
The svmatch command works with no problem when executed as a stand along command.
Redirect your inner command's stdin from /dev/null:
svmatch $line </dev/null
Otherwise, svmatch is able to consume stdin (which, of course, is the list of remaining lines).
The other approach is to use a file descriptor other than the default of stdin:
#!/bin/sh
while IFS= read -r line <&3; do
svmatch "$line"
done 3<svr_input
...if using bash rather than /bin/sh, you have some other options as well; for instance, bash 4.1 or newer can allocate a free file descriptor, rather than requiring a specific FD number to be hardcoded:
#!/bin/bash
while IFS= read -r -u "$fd_num" line; do
do-something-with "$line"
done {fd_num}<svr_input
Related
This question already has answers here:
Looping through the content of a file in Bash
(16 answers)
Closed 1 year ago.
Absolute beginner here.
I have a list of passwords in a file named "pw.prn".
Welcome99
ABCDEFGH
12545678
lakers2021
gododgers
I wish to run each line in the file against a hashing program (mkpasswd -m sha-512) and output the password concatenated with a semi-colon then results of the hashing process to a new file:
So the output would look like this:
Welcome99:$6$xDb6xNDzqtnwzVLz$LMA3CNodueIyZavW3CIGDdcl19cekNNG8EB5Hc/vMzZGUSRhbueNCkYRlyaGKAb/VjW0cBiCHdJLt4iL08gBn/
ABCDEFGH:$6$mANzCeK.SUgSD$ID/E6NYPp4cddHCevI.yua3HotbA/a7fZ7xjSk7dUI6fayuTMsO9SCdSA7MFcgh8SUcmNqrqqE4IxAoIEcmFb0
12345678:$6$CwjNF9B1Q8bkwohy$N4eZcj6YPxxbA1MYz0k9t96nCcj9VsZmzrvgqTd9tp2yXbzAdb3hWyjBq6nquMwFbKMJw9ZXs3Uqj.gfnozUS0
lakers2021:$6$fENvTJijoQgyjWMo$W37vZ364wQugW.W7k9Gl8OfJLl8DfR3tpFO/O4oPTCazJgNkJfNE4WiP4z8qSM8H1.ZJrMUWVAYdYOxt0GSHG1
gododgers:$6$1JdXTdpguO0$ZwFoDtZZ2byDemiLv5JAuea6ucAdtYQUTC4EppX2PMzSLaYtMm/ENpBZZAy70Ceuu6yAjXYtggrSOINTRWoBi0
Unfortunately, I have no "code that I have tried" as I do not even know where to being. Is this a for loop? While? I have tried searching using bash script with interactive answers from a file but have not been able to piece anything together.
I hope that my example provides enough information to understand what I am looking for.
I am running this on Ubuntu Linux
Thank You
You could use a while + read loop. See How can I read a file (data stream, variable) line-by-line (and/or field-by-field)
Something like.
#!/usr/bin/env bash
while IFS= read -r line; do
printf '%s:%s\n' "$line" "$(mkpasswd -m sha-512 "$line")"
done < pw.prn
In bash script how do I reference a file as the input for an interactive prompt
Use a variable (positional parameter "$1")
#!/usr/bin/env bash
while IFS= read -r line; do
printf '%s:%s\n' "$line" "$(mkpasswd -m sha-512 "$line")"
done < "$1"
Then
./myscript pw.prn
Assuming the name of the script is myscript and the file in question is pw.prn
This question already has answers here:
Shell script while read loop executes only once
(6 answers)
While loop stops reading after the first line in Bash
(5 answers)
Bash script stops execution of ffmpeg in while loop - why?
(3 answers)
Closed 5 years ago.
I have a while loop that should iterate over a text file but stops at the first line and I can't figure out why. My code is below.
while read hadoop_accounts; do
if ! grep "no lock no remove"; then
echo "${hadoop_accounts%%:*}"
echo "${hadoop_accounts%%:*}" >> missing_no_lock_no_remove.txt
fi
done < hadoop_accounts.txt
When grep is run with no explicit redirection or file to read, it reads stdin. All of stdin. The same stdin your while read loop is reading from.
Thus, with all your stdin consumed by grep, there's nothing left for the next read command to consume.
The easy approach (and much better for performance) is to do the substring check internal to the shell, and not bother starting up a new copy of grep per line processed at all:
while IFS= read -r hadoop_accounts; do
if ! [[ $hadoop_accounts = *"no lock no remove"* ]]; then
echo "${hadoop_accounts%%:*}"
echo "${hadoop_accounts%%:*}" >&3
fi
done < hadoop_accounts.txt 3>> missing_no_lock_no_remove.txt
Note also that we're only opening the output file once, not re-opening it every single time we want to write a single line.
If you really want to call grep over and over and over with only a single line of input each time, though, you can do that:
while IFS= read -r hadoop_accounts; do
if ! grep "no lock no remove" <<<"$hadoop_accounts"; then
echo "${hadoop_accounts%%:*}"
echo "${hadoop_accounts%%:*}" >&3
fi
done < hadoop_accounts.txt 3>> missing_no_lock_no_remove.txt
Or, even better than either of the above, you can just run grep a single time over the entire input file and read its output in the loop:
while IFS= read -r hadoop_accounts; do
echo "${hadoop_accounts%%:*}"
echo "${hadoop_accounts%%:*}" >&3
done < <(grep -v 'no lock no remove' <hadoop_accounts.txt) 3>>missing_flags.txt
This question already has an answer here:
Why redirect stdin inside a while read loop in bash?
(1 answer)
Closed 5 years ago.
I've finally run into the surprising stdin behavior when piping into a while read loop.
Consider the following:
find . | while read file;
do
echo "==[$file]==";
cat;
done
In this instance, catis just a stand-in for any command that receives input from STDIN. It's surprising (to me at least) that cat's STDIN is actually coming from find, so it gobbles up the rest of the find output.
Suppose one wanted to interact directly from the tty with the command in cat's place. E.g. Suppose instead of cat you wanted to run a script which might ask questions you wanted to respond to interactively ("<file> exists: Overwrite? [y/n]").
Is there a way to force the inner command's STDIN to be the tty?
I've found a lot of similar questions, including this: Why redirect stdin inside a while read loop in bash?
But I couldn't understand the answer well enough to get it to work.
(edit: in light of clarifications to that other question, I'm now considering this a duplicate of that question.)
I'm replacing cat with something a little less problematic in the examples below:
read_a_line() { local line; read -r line; echo "Read line: $line"; }
That way it only reads one line of input per loop invocation, rather than reading all the way to EOF. Otherwise, though, I'm trying to keep changes minimal to focus on the immediate problem.
See BashFAQ #24 for a discussion of why it's preferable to redirect from a process substitution into your loop rather than to pipe to a loop.
First, you can simply redirect from /dev/tty
find . | while read file;
do
echo "==[$file]=="
read_a_line </dev/tty
done
Second, you can copy stdin to a different file descriptor, and reuse it later:
exec 3<&0 # make FD 3 a copy of FD 0
find . | while read file; do
echo "==[$file]=="
read_a_line <&3
done
exec 3<&- # close FD 3 now that we're done with it
Third, you can try to do both -- attempting to make FD 3 (or any other FD of your choice above 2) be open to /dev/tty, but making it a backup of your original stdin if that fails.
exec 3</dev/tty || exec 3<&0
find . | while read file; do
echo "==[$file]=="
read_a_line <&3
done
exec 3<&-
This example could help:
{
while IFS= read -r -d '' file
do
read -u3 -p "what to do with: [$file]?> " action
printf "got [$action] for the [$file]\n\n"
done < <(find . -print0)
} 3<&0
for the whole script's the stdin is reditected to fd3
and internal while is redirected from the find
the read reads from the fd3 - e.g. from the terminal
This question already has an answer here:
Read stdin for user input when a file is already redirected to stdin [duplicate]
(1 answer)
Closed 8 years ago.
I need to read lines from a file. Usually send the file in via stdin, but if I also need to do user input, I can't!
Here's a (contrived) example:
#!/usr/bin/env bash
cat <<HERE >/tmp/mytemp.txt
four of clubs
two of hearts
queen of spades
HERE
while read line ; do
echo "CARD: $line"
read -p 'Is that your card? ' answer
echo "YOUR ANSWER: $answer"
echo
done </tmp/mytemp.txt
This doesn't work. Instead you get this:
$ ~/bin/sample_script.sh
LINE: four of clubs
MY ANSWER: two of hearts
LINE: queen of spades
MY ANSWER:
$
How can I do both in the same loop?
Use two different file descriptors.
while IFS= read -r -u 3 from_file; do
read -r from_user
# ...your logic here...
done 3< filename
Or, to not depend on any bash extensions:
while IFS= read -r from_file <&3; do
read -r from_user
done 3< filename
(The explicit -r and clearing of IFS are necessary to read contents without trimming trailing whitespace, expanding backslash escapes, etc; their use is a good habit to be in by default unless you explicitly know that you want these behaviors).
This question already has answers here:
rewinding stdin in a bash script
(4 answers)
Closed 7 years ago.
I have a shell script where I pass a txt file to the script as follows:
./run.sh < list.txt
Within the script, I am doing a "while read LIST do ... end"
It all works well, and the script executes using the list.
However, now I want to have a second while read LIST do ... end in the same shell script. I want it to read again from the original list I'm passing it on execution, but it doesn't work. It reads the list.txt file for the first loop, but not the second.
What do I do to make the script read list.txt each time I'm asking for it?
You can't read stdin twice. Try passing list.txt on the command-line rather than redirecting it.
./run.sh list.txt
Then in your script:
while read LINE; do
...
done < "$1"
while read LINE; do
...
done < "$1"
Alternatively, save the contents of stdin off the first time you read through it. For instance:
# First loop, save stdin in an array.
LINES=()
while read LINE; do
LINES+=("$LINE")
...
done
# Second loop, iterate over the array.
for LINE in "${LINES[#]}"; do
...
done