While loop in bash script execute only once [duplicate] - bash

This question already has answers here:
bash cycle breaks when calling ssh command in the loop
(2 answers)
Closed 3 months ago.
I have a given shell script:
#!/bin/bash
USERNAME=xenobot
SSH_KEY_LOCATION="~/.ssh/id_rsa"
FILE="server.info"
while read -r server_ip package_type; do
if [[ $package_type == "deb" ]]; then
echo "For $server_ip package type is $package_type"
DPKG_FILE_PATH=$(ls ./src/dpkg/dpkg-*)
DPKG_FILE=$(basename $DPKG_FILE_PATH)
echo "$DPKG_FILE located at $DPKG_FILE_PATH will be transfered via SSH to server"
scp ./src/dpkg/${DPKG_FILE} $USERNAME#$server_ip:/tmp
ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -l ${USERNAME} ${server_ip} "cd /tmp; sudo dpkg -i *.deb"
elif [[ $package_type == "rpm" ]]; then
echo "For $server_ip package type is $package_type"
RPM_FILE_PATH=$(ls ./src/rpm/rpm-*)
RPM_FILE=$(basename $RPM_FILE_PATH)
echo "$RPM_FILE located at $RPM_FILE_PATH will be transfered via SSH to server"
scp -o 'StrictHostKeyChecking no' ./src/rpm/${RPM_FILE} $USERNAME#$server_ip:/tmp
echo "$RPM_FILE has been successfuly transfered to server!"
ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -l ${USERNAME} ${server_ip} "cd /tmp; sudo rpm -ivh --force $RPM_FILE"
fi
done <"$FILE"
And I have following file server.info, from which previous shell script reads:
213.136.80.123 rpm 213.136.91.102 rpm
The problem with it, as it is execute only once from first line, and just stops there without executing the second line
I have tried just to print output from this server.info using the same while read -r and it seems that outputs just fine like this
#!/bin/bash
USERNAME=xenobot
SSH_KEY_LOCATION="~/.ssh/id_rsa"
FILE="server.info"
while read -r server_ip package_type; do
if [[ $package_type == "deb" ]]; then
echo "For $server_ip package type is $package_type"
elif [[ $package_type == "rpm" ]]; then
echo "For $server_ip package type is $package_type"
fi
done <"$FILE"
Output:
213.136.80.123 rpm 213.136.91.102 rpm
So it seems like it outputs two separate lines, however I do not know why it runs script only once in bigger example

Try using a different file descriptor for your list other than 0, 1, or 2 (in this case it's set to 3).
while read -r server_ip package_type <&3; do
...
done 3<"$FILE"
If that works, then what's going on here is that ssh -- being an interactive program -- is sucking up your STDIN input.

Related

How to launch a script for a list of ip addresses (bash script)

i have a problem with my bash shell script. This is my code:
#!/bin/bash/
while read line
do
echo -e "$line
"
sleep 5;
done < Ip.txt
sshpass -p 'Password' ssh -o StrictHostKeyChecking=no root#$ip -t "cd /exemple/ && sh restart-launcher.sh;exec \$SHELL -l"
My script allows to launch for each ip (Ip.txt) in the folder "exemple" a script(restart-launcher.sh) but when I launch it , it only lists the ip without taking this part into account:
sshpass -p 'Password' ssh -o StrictHostKeyChecking=no root#$ip -t "cd /folders/ && sh restart-launcher.sh;exec \$SHELL -l"
How do I create a bash script that works in Linux?
#!/bin/bash/
while read -r line; do
echo -e "$line\n"
sshpass -p 'Password' ssh -o StrictHostKeyChecking=no root#$line -t \
"cd /exemple/ && sh restart-launcher.sh;exec \$SHELL -l"
sleep 5
done < Ip.txt
Now, we could have a discussion about using sshpass (or rather, why you shouldn't), but it feels out of scope.
So, all commands you wish to be looped over need to be inside the loop.
As you read sets the variable $line, that is what you need to use. In your example, you used the variable $ip which you haven't set anywhere.
If we assume that "Ip.txt" contains a list of only IPs perhaps what you mean to do is use sshpass inside the while-loop so it runs for each IP in the .txt file.
#!/bin/bash/
while read line
do
echo -e "$line
"
sshpass -p 'Password' ssh -o StrictHostKeyChecking=no root#$line -t "cd /exemple/ && sh restart-launcher.sh;exec \$SHELL -l"
sleep 5;
done < Ip.txt
sshpass has been moved into the while loop and $ip replaced with $line

Passing variables to SSH [duplicate]

This question already has answers here:
Passing external shell script variable via ssh
(2 answers)
Variable issues in SSH
(1 answer)
Closed 1 year ago.
The following code loops through states in a array and passes a state to a server via ssh -
STATES="NY CO"
arr_states=(${STATES//' /'/ })
for i in "${arr_states[#]}"; do
state=$i
ssh -o SendEnv=state jenkins#server sh -s << 'EOF'
sudo su
cd /home/jenkins/report
psql -d db -c "$(sed 's/state_name/'"$state"'/' county.sql)" -U user
echo $state
EOF
done
The output of echo $state in the above is an empty string even if I pass it NY.
When I change the 'EOF' to EOF, the output of echo $state is the string I passed (NY). But then it says, the file county.sql does not exist.
How do I get it to recognize both the variable I pass and the file on the remote I am trying to run.
As an approach that doesn't require you to do any manual escaping of your code (which frequently becomes a maintenance nightmare, since it means that code needs to be changed whenever you modify where it's expected to run) -- consider defining a function, and using declare -f to ask the shell to generate code that will output that function for you.
The same can be done with variables, using declare -p. Thus, passing both a function with the remote code, and the variables that remote code needs to operate that way:
#!/usr/bin/env bash
# This is run on the remote server _as root_ (behind sudo su)
remotePostEscalationFunc() {
cd /home/jenkins/report || return
if psql -d db -U user -c "$(sed -e "s/state_name/${state}/" county.sql)"; then
echo "Success processing $state" >&2
else
rc=$?
echo "Failure processing $state" >&2
return "$rc"
fi
}
# This is run on the remote server as the jenkins user (before sudo).
remoteFunc() {
sudo su -c "$(declare -p state); $(declare -f remotePostEscalationFunc); remotePostEscalationFunc"
}
# Everything below here is run locally.
arr_states=( NY CO )
for state in "${arr_states[#]}"; do
ssh jenkins#server 'bash -s' <<EOF
$(declare -f remoteFunc remotePostEscalationFunc); $(declare -p state); remoteFunc
EOF
done
You were almost right with the change from 'EOF' to EOF. You are just missing a backslash (\) before $(sed. So the following should work:
arr_states=(${STATES//' /'/ })
for i in "${arr_states[#]}"; do
state=$i
ssh -o SendEnv=state jenkins#server sh -s << EOF
sudo su
cd /home/jenkins/report
psql -d db -c "\$(sed 's/state_name/'"$state"'/' county.sql)" -U user
echo $state
EOF
done

sshpass - connect to multiple servers from txt file? [duplicate]

This question already has answers here:
While loop stops reading after the first line in Bash
(5 answers)
Closed 6 years ago.
I use this bash-code to upload files to a remote server, for normal files this works fine:
for i in `find devel/ -newer $UPLOAD_FILE`
do
echo "Upload:" $i
if [ -d $i ]
then
echo "Creating directory" $i
ssh $USER#$SERVER "cd ${REMOTE_PATH}; mkdir -p $i"
continue
fi
if scp -Cp $i $USER#$SERVER:$REMOTE_PATH/$i
then
echo "$i OK"
else
echo "$i NOK"
rm ${UPLOAD_FILE}_tmp
fi
done
The only problem is that for files with a space in the name, the for-loop fails, so I replaced the first line like this:
find devel/ -newer $UPLOAD_FILE | while read i
do
echo "Upload:" $i
if [ -d $i ]
then
echo "Creating directory" $i
ssh $USER#$SERVER "cd ${REMOTE_PATH}; mkdir -p $i"
continue
fi
if scp -Cp $i $USER#$SERVER:$REMOTE_PATH/$i
then
echo "$i OK"
else
echo "$i NOK"
rm ${UPLOAD_FILE}_tmp
fi
done
For some strange reason, the ssh-command breaks out of the while-loop, therefore the first missing directory is created fine, but all subsequent missing files/directories are ignored.
I guess this has something to do with ssh writing something to stdout which confuses the "read" command. Commenting out the ssh-command makes the loop work as it should.
Does anybody know why this happens and how one can prevent ssh from breaking the while-loop?
The problem is that ssh reads from standard input, therefore it eats all your remaining lines. You can just connect its standard input to nowhere:
ssh $USER#$SERVER "cd ${REMOTE_PATH}; mkdir -p $i" < /dev/null
You can also use ssh -n instead of the redirection.
Another approach is to loop over a FD other than stdin:
while IFS= read -u 3 -r -d '' filename; do
if [[ -d $filename ]]; then
printf -v cmd_str 'cd %q; mkdir -p %q' "$REMOTE_PATH" "$filename"
ssh "$USER#$SERVER" "$cmd_str"
else
printf -v remote_path_str '%q#%q:%q/%q' "$USER" "$SERVER" "$REMOTE_PATH" "$filename"
scp -Cp "$filename" "$remote_path_str"
fi
done 3< <(find devel/ -newer "$UPLOAD_FILE" -print0)
The -u 3 and 3< operators are critical here, using FD 3 rather than the default FD 0 (stdin).
The approach given here -- using -print0, a cleared IFS value, and the like -- is also less buggy than the original code and the existing answer, which can't handle interesting filenames correctly. (Glenn Jackman's answer is close, but even that can't deal with filenames with newlines or filenames with trailing whitespace).
The use of printf %q is critical to generate commands which can't be used to attack the remote machine. Consider what would happen with a file named devel/$(rm -rf /)/hello with code which didn't have this paranoia.

How to check connection to a list of servers in bash?

Im trying to check connections for a list of servers. I want to loop through the list, check if a connection works and if yes, do some stuff, if not, echo out a problem message.
My problem is:
the script stops at the first node without echoing the $?.
So, whats wrong with my for-loop?
These vars are included from a config file:
$nodes is a list of server IPs like 1.1.1.1,2.2.2.2,10.10.10.10
$user is one string
for node in $(echo $nodes | sed "s/,/ /g")
do
echo "Checking Node: $node"
ssh -q -o ConnectTimeout=3 $user#$node echo ok
echo $?
if [[ $? != 0 ]]
then
echo "Problem in logging into $node"
else
# do some stuff here
fi
done
EDIT #1:
for node in $(echo $nodes | sed "s/,/ /g")
do
echo "Checking Node: $node"
ssh -q -t -o ConnectTimeout=3 $user#$node "echo ok"
retcode=$?
echo $retcode
if [[ "$retcode" -ne 0 ]]
then
echo "Problem in logging into $node"
else
echo "OK"
fi
done
It is because ssh first asks you to validate The authority of the host and If you accept the authority it will ask for password. That is why your command does not return to shell and waits for input.
If your intention is just validating ssh connection, then you may consider to use
telnet <your_host> <port> < /dev/null
But if your intend is to run some commands you need a trust relationship between hosts. In that case you can use:
Execute this commands:
ssh-keygen -t rsa
then
ssh-copy-id -i root#ip_address
Now you can connect with
ssh <user>#<host>
Furher information
You can add -tto make virtual terminal and add quotes on command:
ssh -q -t -o ConnectTimeout=3 ${user}#${node} "echo ok"
Also use -ne instead of != which is for compare strings
if [[ "$?" -ne 0 ]]
Also echo $? mess the return code. You should use something like:
ssh -q -t -o ConnectTimeout=3 ${user}#${node} "echo ok"
retcode=$?
echo $retcode
if [[ "$retcode" -ne 0 ]]
You can rewrite ssh command like this to avoid problems with ssh host keys
ssh -q -t -o StrictHostKeyChecking=no -o ConnectTimeout=3 ${user}#${node} "echo ok"

Variable Not Picking up when in Quotes

I'm trying to rsync a DIR from one Server to 100s of Servers using script (Bottom)
But, When i put single or double quotes around ${host} variable, Host names are not picked properly or not resolved.
Error is like below
server1.example.com
Host key verification failed.
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(600) [sender=3.0.6]
and when I run only with rync command like below, It works. But, Output doesn't contain hostname which is important for me to correlate the output with associated hostname.
hostname -f && rsync -arpn --stats /usr/xyz ${host}:/usr/java
Can you please review and suggest me how to make the script work even with quotes around Host variable. ?
So, that , Output will contain hostname and output of rsync together.
==============================================
#!/bin/bash
tmpdir=${TMPDIR:-/home/user}/output.$$
mkdir -p $tmpdir
count=0
while IFS= read -r host; do
ssh -n -o BatchMode=yes ${host} '\
hostname -f && \
rsync -arpn --stats /usr/xyz '${host}':/usr/java && \
ls -ltr /usr/xyz'
> ${tmpdir}/${host} 2>&1 &
count=`expr $count + 1`
done < /home/user/servers/non_java7_nodes.list
while [ $count -gt 0 ]; do
wait $pids
count=`expr $count - 1`
done
echo "Output for hosts are in $tmpdir"
exit 0
UPDATE:
Based on observation with (set -x), Host name is being resolved on remote (self) it self, it supposed to be resolved on initiating host. I think Once we know how to make host name resolved with in initiating host even when quotes are in place.
As far as I can tell, what you're looking for is something like:
#!/bin/bash
tmpdir=${TMPDIR:-/home/user}/output.$$
mkdir -p "$tmpdir"
host_for_pid=( )
while IFS= read -r host <&3; do
{
ssh -n -o BatchMode=yes "$host" 'hostname -f' && \
rsync -arpn --stats /usr/xyz "$host:/usr/java" && \
ssh -n -o BatchMode=yes "$host" 'ls -ltr /usr/java'
} </dev/null >"${tmpdir}/${host}" 2>&1 & pids[$!]=$host
done 3< /home/user/servers/non_java7_nodes.list
for pid in "${!host_for_pid[#]}"; do
if wait "$pid"; then
:
else
echo "ERROR: Process for host ${host_for_pid[$pid]} had exit status $?" >&2
fi
done
echo "Output for hosts are in $tmpdir"
Note that the rsync is no longer inside the ssh command, so it's run locally, not remotely.

Resources