I want to search different packages in multiple servers in Solaris 10. One file contains the package information, another file contains the server information.
I have tried this:
bash-3.00# cat pk
"VRTSvcs|VRTSvxfen"
"SUNWfmd|SUNWfsmgtr"
bash-3.00# cat ser
mokshi
niki
This is my script:
bash-3.00# cat tt
#!/usr/bin/bash
>output
for j in `cat ser`
do
for ip in `cat pk`
do
M=`ssh $j "pkginfo |egrep $ip |cut -d \" \" -f7 "`;
echo "$j " >>output
echo "$M" >>output
done
done
The expected output is
cat output
bash-3.00# cat output
moksha
VRTSvcs
VRTSvcsag
VRTSvcsea
VRTSvxfen
niki
SUNWfmd
SUNWfmdr
SUNWfsmgtr
but when I run the script it is running twice like this:
bash-3.00# bash -x tt
++ cat ser
+ for j in '`cat ser`
'
++ cat pk
+ for ip in '`cat pk`'
++ ssh mokshi 'pkginfo |egrep "VRTSvcs|VRTSvxfen" |cut -d " " -f7 '
Password:
+ M='VRTSvcs
VRTSvcsag
VRTSvcsea
VRTSvxfen'
+ echo 'mokshi '
+ echo 'VRTSvcs
VRTSvcsag
VRTSvcsea
VRTSvxfen'
+ for ip in '`cat pk`'
++ ssh mokshi 'pkginfo |egrep "SUNWfmd|SUNWfsmgtr" |cut -d " " -f7 '
Password:
+ M='SUNWfmd
SUNWfmdr
SUNWfsmgtr'
+ echo 'mokshi '
+ echo 'SUNWfmd
SUNWfmdr
SUNWfsmgtr'
+ for j in '`cat ser`'
++ cat pk
+ for ip in '`cat pk`'
++ ssh niki 'pkginfo |egrep "VRTSvcs|VRTSvxfen" |cut -d " " -f7 '
Password:
+ M='VRTSvcs
VRTSvcsag
VRTSvcsea
VRTSvxfen'
+ echo 'niki '
+ echo 'VRTSvcs
VRTSvcsag
VRTSvcsea
VRTSvxfen'
+ for ip in '`cat pk`'
++ ssh niki 'pkginfo |egrep "SUNWfmd|SUNWfsmgtr" |cut -d " " -f7 '
Password:
+ M='SUNWfmd
SUNWfmdr
SUNWfsmgtr'
+ echo 'niki '
+ echo 'SUNWfmd
SUNWfmdr
SUNWfsmgtr'
And I am getting output like this:
bash-3.00# cat output
moksha
VRTSvcs
VRTSvcsag
VRTSvcsea
VRTSvxfen
moksha
SUNWfmd
SUNWfmdr
SUNWfsmgtr
niki
VRTSvcs
VRTSvcsag
VRTSvcsea
VRTSvxfen
niki
SUNWfmd
SUNWfmdr
SUNWfsmgtr
The main goal of the script is that its take one server from server file and it has to search first line in package file. The it has to take second server name from server name and it searches for second line in package file.
please help me where I made a mistake.
You could do this - assuming you have an equal number of lines in your server and package files:
#!/usr/bin/env bash
while read -u 3 -r server && read -u 4 -r pkg; do
m=$(ssh -n "$server" "pkginfo | egrep '$pkg' | cut -d' ' -f7")
echo "$server"
echo "$m"
done 3<ser 4<pk >> output
3<ser connects fd 3 to file ser and 4<pk connects fd 4 to file pk
read -u 3 reads from file descriptor 3, read -u 4 reads from file descriptor 4
$() is better than back ticks (see the post below to find why)
cut -d' ' -> guess your delimiter is a space
ssh -n is needed so that ssh ignored stdin and doesn't interfere with read
it is better to put >> output at the end for more efficient I/O
See also:
Why you don't read lines with "for"
'read -r' doesn't read beyond first line in a loop that does ssh
What's the difference between $(command) and `command` in shell programming?
Related
I'm trying to use Wireguard road-warrior script from here https://github.com/Nyr/wireguard-install
But i cant make it run in noninteractive mode with predefined parameters. I've read some other similar topics here about how to provide answers to bash "read", but suggestions from there doesn't work.
I've tried this:
# bash wireguard-install.sh < params
# printf '%s\n' 51822 clientcustom 2 y | bash wireguard-install.sh
# echo "51822 clientcustom 2 y" | bash wireguard-install.sh
in every case installer just uses default values. What am i doing wrong?
https://github.com/Nyr/wireguard-install/blob/master/wireguard-install.sh#L15
Indeed, this is a bit problematic. Typically, not caring, you would just:
( sleep 1; printf '%s\n' 51822 clientcustom 2 y ) | ...
A real robust solution, you would parse the output of the process to know when to write response, either with expect or Bash or something better.
coproc bash wireguard-install.sh
while IFS= read -r -u "${COPROC[0]}" line; do
case "$line" in
"IPv4 address [1]:"*) echo something >&"${COPROC[1]}"; ;;
"other prompt"*) echo other stuff >&"${COPROC[1]}"; ;;
"etc...") echo ... ;;
esac
done
Some alternatives:
1. you could simply drop problematic line while running.
bash <(sed /read\ -N\ 999999/d wireguard-install.sh) < <(
printf %s\\n 51822 clientcustom 2 y)
This will drop line used to
# Discard stdin. Needed when running from an one-liner which includes a newline
read -N 999999 -t 0.001
2. Instead of passing variables, you could edit script:
$ sed 's/^[ \o11]*read -p/ /p;d' wireguard-install.sh | uniq
"DNS server [1]: " dns
"IPv4 address [1]: " ip_number
"Public IPv4 address / hostname [$get_public_ip]: " public_ip
"Public IPv4 address / hostname: " public_ip
"IPv6 address [1]: " ip6_number
"Port [51820]: " port
"Name [client]: " unsanitized_client
"Should automatic updates be enabled for it? [Y/n]: " boringtun_updates
"Option: " option
"Name: " unsanitized_client
"Client: " client_number
"Confirm $client removal? [y/N]: " remove
"Confirm WireGuard removal? [y/N]: " remove
So you could prepare a sed string:
printf -v sedscr 's/read -p.* \(%s\) *$/\\1="%s"/;' port 51822 \
unsanitized_client clientcustom client_number 2 remove y
Then ensure all's ok:
sed -e "/read -N 99999/d;$sedscr" <wireguard-install.sh |
diff -u wireguard-install.sh -
you will see some lines like
# Discard stdin. Needed when running from an one-liner which includes a newline
-read -N 999999 -t 0.001
# Detect OpenVZ 6
if [[ $(uname -r | cut -d "." -f 1) -eq 2 ]]; then
## -235,15 +234,15 ##
fi
echo
echo "What port should WireGuard listen to?"
- read -p "Port [51820]: " port
+ port="51822"
until [[ -z "$port" || "$port" =~ ^[0-9]+$ && "$port" -le 65535 ]]; do
And finally
bash <(
printf -v sedscr 's/read -p.* \(%s\) *$/\\1="%s"/;' port 51822 \
unsanitized_client clientcustom client_number 2 remove y
sed -e "/read -N 99999/d;$sedscr" <wireguard-install.sh
)
#!/bin/bash
for ip in 'seq 1 254'; do
ping -c 1 $0.$ip | grep "64 bytes" | cut -d " " -f 4 | tr -d ":" &
done
the file is ipsweep.sh
when i run it show me this
root#kali:ping: ./ipsweep.sh.seq: Name or service unkown
First issue:
Replace 'seq 1 254' with $(seq 1 254).
If you replace both ' with backticks it works too, but it's old syntax.
Second issue:
Replace $0 with $1 if you want to provide this part (192.168.1, e.g.) on the command line. $0 contains name of your script.
Need some help on these commands that we're not working during the ssh session but works fine when manually run on an ubuntu machine.
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done
somehow the code above does not work, it expected to have an output like this:
var
123
asd
123
dsa
123
123
123
instead I get like this:
123
123
123
123
<blank space>
<blank space>
<blank space>
<blank space>
Here is the full code:
ssh -o "StrictHostKeyChecking no" -i $PEM_PATH $SERVER_USER#$SERVER_IP << EOF
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done
EOF
Any help are really appreciated, thanks!
The problem is that the local shell is interpolating your $ expressions prior to sending them to the ssh session. You can prevent this by single quoting your heredoc tag: 'EOF'.
ssh -o "StrictHostKeyChecking no" -i $PEM_PATH $SERVER_USER#$SERVER_IP <<'EOF'
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done
EOF
With the above code, $(echo "/var/asd/dsa/123" | tr "/" " ") ; is sent to the server instead of being replaced locally.
There are several issues with your approach:
Your here doc gets interpolated locally. The actual command you are running is: for w in var asd dsa 123 ; do echo >&2; echo "123"; done
You can see that by using set -x to enable tracing locally, and run ssh with cat instead of the login shell:
ssh pi cat << EOF
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done
EOF
+ ssh pi cat
++ echo /var/asd/dsa/123
++ tr / ' '
for w in var asd dsa 123 ; do echo >&2; echo "123"; done
As you can see echo /var/asd/dsa/123 and tr / ' ' are executed locally, and $w is interpolated to the empty string.
You probably want to use single quotes around 'EOF':
ssh pi cat << 'EOF'
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done
EOF
+ ssh pi cat
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done
Now, the command gets there correctly.
Note: if you wish that $(echo "/var/asd/dsa/123" | tr "/" " ") is run locally, then you at least need to escape $w.
ssh pi << EOF
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo \$w >&2; echo "123"; done
EOF
Buffering will mess up the order. That's why your empty lines (the ones printed by echo >&2) are at the end of your output. You can use tee to "force" line buffering:
ssh -T pi << 'EOF'
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done
EOF
+ ssh -T pi
Linux mserv 3.10.25+ #622 PREEMPT Fri Jan 3 18:41:00 GMT 2014 armv6l
...
123
123
123
123
var
asd
dsa
123
Versus:
ssh pi << 'EOF'
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done 2>&1 | tee
EOF
+ ssh pi
Linux mserv 3.10.25+ #622 PREEMPT Fri Jan 3 18:41:00 GMT 2014 armv6l
...
var
123
asd
123
dsa
123
123
123
(in the first case STDIN is printed first, then STDERR, in the second, lines are alternated)
Piping commands to ssh and forcing a login shell
If you run commands by piping them to stdin of ssh, it will run a login shell and print the MOTD (depending on the server configuration) and run a login shell (which has several side effects).
In my example, Linux mserv 3.10.25+ #622 PREEMPT Fri Jan 3 18:41:00 GMT 2014 armv6l is part of the MOTD. However, there is no need for piping, ssh will run last argument via the user shell, without the need for piping it to STDIN. This might be limited to MAX_ARG_STRLEN (which is ~128K), but if you have a script that size, you probably should scp-it to the host first.
$ ssh pi '
for w in $(echo "/var/asd/dsa/123" | tr "/" " ")
do
echo $w >&2
echo "123"
done 2>&1 | tee
'
var
123
asd
123
dsa
123
123
123
Note: This might be limited to MAX_ARG_STRLEN (which is ~128K), but if you have a script that size, you probably should scp-it to the host first.
Note: pi is a test server I use, replace it with your appropriate connection info.
Context
Got a daft script that checks a process is running on a group of hosts, like a watchdog, as I say it's a daft script so bear in mind it isn't 'perfect' by scripting standards
Problem
I've ran bash -x and can see that the script finishes its first check without actually redirecting the output of the command to the file which is very frustrating, it means each host is actually being evaluated to the last hosts output
Code
#!/bin/bash
FILE='OUTPUT'
for host in $(cat /etc/hosts | grep webserver.[2][1-2][0-2][0-9] | awk {' print $2 ' })
do ssh -n -f $host -i <sshkey> 'ps ax | grep myprocess | wc -l' > $FILE 2> /dev/null
cat $FILE
if grep '1' $FILE ; then
echo "Process is NOT running on $host"
cat $FILE
else
cat $FILE
echo "ALL OK on $host"
fi
cat $FILE
done
Script traceback
++ cat /etc/hosts
++ awk '{ print $2 }'
++ grep 'webserver.[2][1-2][0-2][0-9]'
+ for host in '$(cat /etc/hosts | grep webserver.[2][1-2][0-2][0-9] | awk {'\'' print $2 '\''})'
+ ssh -n -f webserver.2100 -i <omitted> 'ps ax | grep myprocess | wc -l'
+ cat OUTPUT
+ grep 1 OUTPUT
+ cat OUTPUT
+ echo 'ALL OK on webserver.2100'
ALL OK on webserver.2100
+ cat OUTPUT
+ printf 'webserver.2100 checked \n'
webserver.2100 checked
+ for host in '$(cat /etc/hosts | grep webserver.[2][1-2][0-2][0-9] | awk {'\'' print $2 '\''})'
+ ssh -n -f webserver.2101 -i <omitted> 'ps ax | grep myprocess | wc -l'
+ cat OUTPUT
2
+ grep 1 OUTPUT
+ cat OUTPUT
2
+ echo 'ALL OK on webserver.2101'
ALL OK on webserver.2101
+ cat OUTPUT
2
+ printf 'webserver.2101 checked \n'
webserver.2101 checked
Issue
As you can see, it's registering nothing for the first host, then after it is done, it's piping the data into the file, then the second host is being evaluated for the previous hosts data...
I suspect its to do with redirection, but in my eyes this should work, it doesn't so it's frustrating.
I think you're assuming that ps ax | grep myprocess will always return at least one line (the grep process). I'm not sure that's true. I'd rewrite that like this:
awk '/webserver.[2][1-2][0-2][0-9]/ {print $2}' /etc/hosts | while IFS= read -r host; do
output=$( ssh -n -f "$host" -i "$sshkey" 'ps ax | grep "[m]yprocess"' )
if [[ -z "$output" ]]; then
echo "Process is NOT running on $host"
else
echo "ALL OK on $host"
fi
done
This trick ps ax | grep "[m]yprocess" effectively removes the grep process from the ps output:
the string "myprocess" matches the regular expression "[m]yprocess" (that's the running "myprocess" process), but
the string "[m]yprocess" does not match the regular expression "[m]yprocess" (that's the running "grep" process)
I have a file server.txt containing different hostnames like:
hostname1.com
hostname2.com
My shell script servers.sh is written to get the /etc/passwd and /etc/group files from the list in servers.txt file.
I wish to add the hostname from which the entries came from in my final output file. My script looks something like below:
while read HOST ;
do
sshpass -p $password ssh -n $username#$HOST 'cat /etc/passwd'>>users.txt
sshpass -p $password ssh -n $username#$HOST 'cat /etc/group'>>groups.txt
done < servers.txt
echo -e "UserName;UID;GID;HomeDir;Shell" > final_users.csv
cut -d: -f1,3,4,6,7 users.txt | tr ':' ';'>> final_users.csv
echo -e "GroupName;GID;Members" > final_groups.csv
awk -F '[:,]' '{for(i=4;i<=NF;i++)print$1";"$3";"$i}' groups.txt >> final_groups.csv
The goal is to add another column in both final_users.csv and final_groups.csv like hostname so I can know which servers each entry came from.
Try like this:
while read HOST ;
do
sshpass -p $password ssh -n $username#$HOST 'cat /etc/passwd'>>users.txt
sshpass -p $password ssh -n $username#$HOST 'cat /etc/group'>>groups.txt
done < servers.txt
echo -e "UserName;UID;GID;HomeDir;Shell;Hostname" > final_users.csv
echo "`cut -d: -f1,3,4,6,7 users.txt | tr ':' ';'`;$HOST">> final_users.csv
echo -e "GroupName;GID;Members;Hostname" > final_groups.csv
echo "`awk -F '[:,]' '{for(i=4;i<=NF;i++)print$1";"$3";"$i}' groups.txt`;$HOST" >> final_groups.csv
Think I got it:
while read HOST ;
do
sshpass -p $password ssh -n $username#$HOST 'cat /etc/passwd'>>users_1.txt
while IFS= read -r line; do echo "$line:"$HOST; done < users_1.txt >> users_2.txt
sshpass -p $password ssh -n $username#$HOST 'cat /etc/group'>>groups_1.txt
while IFS= read -r line; do echo "$line:"$HOST; done < groups_1.txt >> groups_2.txt
done < servers.txt
echo -e "UserName;UID;GID;HomeDir;Shell;Hostname" > final_users.csv
cut -d: -f1,3,4,6,7,8 users_2.txt | tr ':' ';'>> final_users.csv
echo -e "GroupName;GID;Members;Hostname" > final_groups.csv
awk -F'[:,]' -v OFS=';' '{for(i=4;i<NF;i++) print $1, $3, $i, $NF}' groups_2.txt >> final_groups.csv