Bash read line stops in the middle of the file [duplicate] - bash

Question

This question already has answers here:
While loop stops reading after the first line in Bash
(5 answers)
Closed 2 years ago.
I've created a bash script to connect to a number of servers and execute a program. The ips and quantities per IP should be read from a config file that is structured like this:
127.0.0.1 10
127.0.0.1 1
127.0.0.1 3
etc
j=$((0))
while IFS=' ' read -r ip quantity; do
echo "${ip} x ${quantity}";
for (( i = 1; i <= quantity; i++ ))
do
echo "ssh root#${ip} cd test/libhotstuff && ./examples/hotstuff-app --conf ./hotstuff.gen-sec${j}.conf > log${j} 2>&1"
ssh root#"${ip}" "cd test/libhotstuff && ./examples/hotstuff-app --conf ./hotstuff.gen-sec${j}.conf > log${j} 2>&1" &
j=$((j+1))
done
sleep 1
done < ips
I noticed that this while loop breaks if the execution takes too long. If I put sleep for 1s here it will stop after the first execution. If I remove it, but the inner loop takes too long a subset of the lines will not be read.
What is the problem here?

Answer 1

Here's a version that starts your background processes with a 1 second delay between each, waits 6 minutes before killing them one by one, with a 1 second delay between each, to give them approximately the same running time.
You should also add some options to ssh to prevent it from interfering with stdin and terminate your loop prematurely while running.
-n
Prevents reading from stdin
-oBatchMode=yes
Passphrase/password querying will be disabled
-oStrictHostKeyChecking=no
Connect to host even if the host key has changed
#!/bin/bash
sshopts=(-n -oBatchMode=yes -oStrictHostKeyChecking=no)
j=0
pids=()
while IFS=$' \t\n' read -r ip quantity; do
echo "${ip} x ${quantity}";
for (( i = 0; i < quantity; ++i ))
do
remotecmd="cd test/libhotstuff && ./examples/hotstuff-app --conf ./hotstuff.gen-sec${j}.conf > log${j} 2>&1"
localcmd=(ssh ${sshopts[#]} root#${ip} "$remotecmd")
echo "${localcmd[#]}"
"${localcmd[#]}" &
# store the background pid
pids+=($!)
(( ++j ))
sleep 1
done
done < ips
seconds=360
echo "running ${pids[#]} in the background $seconds seconds"
sleep $seconds
echo "telling the background processes to terminate"
for pid in ${pids[#]}
do
echo killing $pid
kill $pid
sleep 1
done
echo "waiting for all the background processes to terminate"
wait
echo Done

Answer 2

Here is a version that offloads the loop and parallel processes to the remote shell script. Generate a remote shell script from a HereDocument with quantity, and wait for all the background processes to terminate before exiting.
#!/usr/bin/env sh
while IFS=$' \t\n\r' read -r ip quantity || [ -n "$quantity" ]
do
{
# When satisfied by the output:
# Ucomment the line below and delete its following line with the echo and cat
# ssh "root#$ip" <<EOF
echo ssh "root#$ip"; cat <<EOF
if cd test/libhotstuff
then
i=$quantity
until
i=\$((i - 1))
[ \$i -lt 0 ]
do
./examples/hotstuff-app \\
--conf "./hotstuff.gen-sec\$i.conf" >"log\$i" 2>&1 &
done
wait
fi
EOF
} &
done <ips
# Wait for all child processes to terminate
wait
echo "All child ssh done!"
Another way replacing the dynamic HereDocument by an inline shell script called with a quantity argument:
#!/usr/bin/env sh
while IFS=$' \t\n\r' read -r ip quantity || [ -n "$quantity" ]; do
echo ssh "root#$ip" sh -c '
if cd test/libhotstuff
then
i=0
while [ $i -lt "$1" ]; do
./examples/hotstuff-app --conf "./hotstuff.gen-sec$i.conf" >"log$i" 2>&1 &
i=$((i + 1))
done
wait
fi
' _ "$quantity" &
done <ips
# Wait for all child processes to terminate
wait
echo "All child ssh done!"