I am attempting to write a script that greps for something in a number of servers and appends the output of all of them into a single file. The servers are password protected. I use expect to enter the servers and pass the grep command but I am hoping to get the output of each of them to end up in a single file.
Here is an overview of what I want to do:
spawn ssh xxx#server1
expect "password: "
send "PASSWORD\r"
expect "$ "
send "grep <something> /some/log/file >> file.txt"
expect "$ "
send "exit\r"
... then continue doing this in dozens more servers with the output of the grep command appending to file.txt each time. I don't mind where the file.txt actually is. It can be on my local computer or any of the servers.
The best I've come up with would be to put each of these in a file on the server the grep is being done on and then scp all those files to local and appending them all. This seems incredibly wasteful though, so I am looking for a way to send the output to a server or to local from a server.
It would be both easier to automate and more secure if you used public key authentication instead of password authentication to get to the servers. Then you could simply loop over them like this:
for host in server1 server2 server3 ...; do
ssh -n "$host" 'grep <something> /some/log/file'
done >file.txt
Since you have password access, you can easily put a public key in .ssh/authorized_keys to enable key access first. You can do it with your expect script:
spawn ssh xxx#server1
expect "password: "
send "PASSWORD\r"
expect "$ "
send "mkdir -p .ssh\r"
expect "$ "
send "cat >>.ssh/authorized_keys <<EOF"
send "(public key goes here)\r"
send "EOF\r"
expect "$ "
send "chmod 0700 .ssh\r"
expect "$ "
send "chmod 0600 .ssh/authorized_keys\r"
expect "$ "
send "exit\r"
If for some reason you must use a solution with password-entry, you can append to a file with expect with something like:
log_user 0 # to not see the output on screen
set fh [open foo.log a] # open the file for appending
set servers {user#server user#server2 […]}
foreach s $servers {
spawn ssh user#server
[…]
send "command"
expect "$ " { puts $fh "$expect_out(buffer)"}
}
close $fh
Related
I am trying to send commands from a file to a device over expect. I tried sending them one at a time from my local machine but all of my file paths were relative to local, not relative to the remote device. My solution was to try to upload the file to the device and load the commands from there. When I try to load the file I keep getting a permissions issue even though if I cat the file from the device I don't have a problem reading it. The file has 1 command per line.
devicepath=rsync://root#localhost:$PORT_RSYNC/root/var/root/file.txt
/usr/bin/rsync -Pavr $1 $devicepath
expect <<- expect_feed
set send_slow {1 .001}
spawn ssh -o NoHostAuthenticationForLocalhost=yes -p $PORT_SSH root#localhost
expect -re "password:"
send -s "password\r"
expect -re $PROMPT_ROOT
send -s "chmod 777 /var/root/file.txt\r"
expect -re $PROMPT_ROOT
set f [cat /var/root/file.txt]
set cmds [split [read $f] "\n"]
close $f
foreach line $cmds {
send -s "$line\r"
expect -re $PROMPT_ROOT
expect_feed
This yields:
root# cat: /var/root/file.txt: Permission denied
I also tried
set f [open /var/root/file.txt]
...but it gave the same error.
If the file you send over contains shell commands, treat it as such and simply source it on the remote host
devicepath=rsync://root#localhost:$PORT_RSYNC/root/var/root/file.txt
/usr/bin/rsync -Pavr "$1" "$devicepath"
export PROMPT_ROOT PORT_SSH
expect << 'EXPECT_FEED'
set send_slow {1 .001}
spawn ssh -o NoHostAuthenticationForLocalhost=yes -p $env(PORT_SSH) root#localhost
expect -re "password:"
send -s "password\r"
expect -re $env(PROMPT_ROOT)
send -s ". /var/root/file.txt\r" ;# <<<<
expect -re $env(PROMPT_ROOT)
send "exit\r"
expect eof
EXPECT_FEED
I prefer to use quoted heredocs: shell variables can be passed to expect via the environment.
I'm assuming root's shell is a POSIX-type shell where . is the "source" command.
Thanks for the great suggestions. It is working like this.
It would probably work just as well without send slow now that it is sending one line at a time and waiting for a response.
The last command in the file is 'quit' in order to return to the root prompt, I suppose that could have been hard coded
cmdpath=$1
export cmdpath
expect << 'EXPECT_FEED'
set send_slow {1 .001}
set commandfile [open $env(cmdpath)]
spawn ssh -o NoHostAuthenticationForLocalhost=yes -p $env(PORT_SSH) root#localhost
expect -re "password:"
send -s "password\r"
expect -re $env(PROMPT_ROOT)
send -s ">the name of the process accepting commands<\r"
while { [gets $commandfile line] != -1 } {
expect -re $env(PROMPT)
send -s "$line\r" }
expect -re $env(PROMPT_ROOT)
send "exit\r"
expect eof
EXPECT_FEED
The use case of this script is I have various servers with different ssh keys. I am trying to write a script so when called will log into the specified server. An example of usage would be:
./ServerLogin.sh Server1
I feel that I am fairly close, but The last part of expect interact is tripping me up. This is a simplified version:
#!/bin/bash
ServerName="$1"
case $ServerName in
"Server1") IP="1.2.3.4" ; keyPath="/path/to/key.pem" ; password="password" ; break ;;
*) echo "Server not recognized" ; exit ;;
esac
/usr/bin/expect << EOD
spawn ssh -i $keyPath user#$IP
expect "*.pem': "
send "$password\r"
interact
EOD
The result of this is it logs in and immediately closes. I want for the session to remain interactable.
Any ideas?
The problem is in expect << EOF. With expect << EOF, expect's stdin is the here-doc rather than a tty. But the interact command only works when expect's stdin is a tty. Your answer is one solution. Another solution is to use expect -c if you prefer not using a tmp file.
expect -c "
spawn ssh -i $keyPath user#$IP
expect \"*.pem': \"
send \"$password\r\"
interact
"
After toying around with it some more, I found a working solution. Basically create the expect script and run it. Why it works like this and not in the original question is beyond me. But it works and I will use this for the time being. Thanks everyone for the help!
Working Solution:
#!/bin/bash
ServerName="$1"
case $ServerName in
"Server1") IP="1.2.3.4" ; keyPath="/path/to/key.pem" ; password="password" ; break ;;
*) echo "Server not recognized" ; exit ;;
esac
function WriteExp {
echo "#!/usr/bin/expect"
echo "spawn ssh -i $keyPath ubuntu#$IP"
echo "expect \"*.pem': \""
echo "send \"$password\\r\""
echo "interact"
}
WriteExp > $ServerName.exp
chmod 755 $ServerName.exp
/usr/bin/expect $ServerName.exp
# Cleanup the evidence
rm $ServerName.exp
Using just tcl/expect, instead of a hybrid of shell and it, makes for much cleaner code, without any of the potential issues posed by shell variable interpolation:
#!/usr/bin/expect -f
switch -- [lindex $argv 0] {
Server1 {
set IP 1.2.3.4
set keyPath /path/to/key.pem
set password "password"
}
default {
puts stderr "Server not recognized"
exit 1
}
}
spawn ssh -i $keyPath user#$IP
expect "*.pem': "
send "$password\r"
interact
I have written below Shell script which is intended to get Model name from remote host by doing SSH and executing the command.
#!/bin/bash
> output.csv
IFS=","
echo "IP,Model Name" >> output.csv
while read ip
do
#echo "Current IP is: $ip"
model=expect -c 'spawn ssh username#'"$ip"' "show version | in cisco"; expect -re "The.*(yes/no)?"; send "yes\r"; expect -re ".*UNAUTH.*password:"; send "password\r";' | grep cisco
echo "$ip,$model" >> output.csv
done < Check_SSH.csv
When I execute below command manually, then it gives expected model name as output.
Command:
expect -c 'spawn ssh username#'"$ip"' "show version | in cisco"; expect -re "The.*(yes/no)?"; send "yes\r"; expect -re ".*UNAUTH.*password:"; send "password\r";' | grep cisco
But when its put into script like above it doesn't produce any output.
Also, there are MOTD (Message of the day) configured on most of the servers and "The authenticity of host..." message to adding server into .ssh/known_hosts, So I tried to handle them in script but Expect is failing to handle the situation when MOTD doesn't appear or when remote is already present in .ssh/known_hosts.
Any help is highly appreciated to get this script running.
Expected output:
IP,Model Name
8.8.8.8,C9407R
8.8.8.1,C9407R
8.8.8.2,C9407R
8.8.8.3,C9407R
First, you're missing the Command Substitution syntax to execute the expect code:
model=$(expect -c ...)
# ....^^.............^
Next, to optionally expect patterns, you need the expect {patt1 action1 patt2 action2 ...} form:
expect -c '
spawn ssh username#'"$ip"' "show version | in cisco"
expect {
-re "The.*(yes/no)?" {send "yes\r"; exp_continue}
-re ".*UNAUTH.*password:" {send "password\r"; exp_continue}
eof
}
'
That way, expect can match any of the patterns. The exp_continue command "loops" within the same expect command so you can match more than one of them. The eof pattern matches when ssh connection closes after the "show version ..." command has finished.
Newlines for readability.
Putting this together:
model=$(
expect -c '
spawn ssh username#'"$ip"' "show version | in cisco"
expect {
-re "The.*(yes/no)?" {send "yes\r"; exp_continue}
-re ".*UNAUTH.*password:" {send "password\r"; exp_continue}
eof
}
' | grep -i cisco
)
I have a feeling that there's more you need to do in the grep part, but you didn't show the output of just the expect command.
update:
use spawn -noecho ssh ... so expect will not print the spawn command.
then, you'll get whatever output ssh needs to show for the login process, and then the "show" command output:
if you're expecting exactly 1 line of output, you might want to change grep to tail -n 1.
otherwise, show the output you get and we can help you filter out the noise.
update 2: filtering out the noise
I'm going to assume that the regex pattern cisco (.*) processor is what you need to match:
model=$(
expect -c '
log_user 0
spawn ssh username#'"$ip"' "show version | in cisco"
expect {
-re "The.*(yes/no)?" {send "yes\r"; exp_continue}
-re ".*UNAUTH.*password:" {send "password\r"; exp_continue}
-re "cisco (.*) processor" {puts $expect_out(1,string)}
}
expect eof
'
)
log_user 0 turns off the spawned process's ability to write to stdout. Expect can still capture its output though.
I have a list of 400 servers and I like to check unix account existence with expect to loop it
I wrote a bash script that uses expect command but it returns me error message that I don't understand the meaning
#!/bin/bash
fic_serv="test.txt"
echo "Passwd"
stty -echo
read -s passwd
stty echo
suffix="suffix"
account="acc"
for server in `cat $fic_serv`
do
prompt="[$acc#$server ~]$ "
expect -c "
spawn ssh -o StrictHostKeyChecking=no $account#$server.$suffix
expect "Password: "
send "$passwd\r"
expect $prompt
send "logout\r"
"
done
[acc#serv ~]$ couldn't read file "
send "passwd\r"
expect [acc#server ~]$
send "logout\r"
": no such file or directory
(I modified the value)
You should use while, not for, to parse files in Bash. Use a "redirect" to treat a file as standard input and read one line at a time.
while read server; do
...
done < $fic_serv
Your major problem is Expect interprets your "s as "end of script". Escape them, as in \", or use {}, as in:
expect -c "
spawn ssh -o StrictHostKeyChecking=no $account#$server.$suffix
expect {Password: }
send {$passwd\r}
expect $prompt
send {logout\r}
"
If you have 400 servers to manage, I strongly recommend you use ansible.
You could just put the list of hosts into a file, let's call it inventory, and run the following command:
ansible -i inventory -m shell -a "id acc" all
Using here-docs in the shell to embed code for another language is usually better than quoting hell, and sharing variables through the environment is easier and safer than parameter expansion:
export account passwd
while IFS= read -r server; do
export prompt="[$acc#$server ~]$ "
export host="$server.$suffix"
expect << 'END_EXPECT'
spawn ssh -o StrictHostKeyChecking=no $env(account)#$env(host)
expect "Password: "
send "$env(passwd)\r"
expect $env(prompt)
send "logout\r"
expect eof
END_EXPECT
done < "$fic_serv"
As shown, I like to indent the heredoc to make it more obvious.
And depending on the error message or login prompt, there can be more logic to indicate that the account name and/or password are incorrect.
So far I have been able to create a small script using ssh combined with expect to pass a single command through to the dd-wrt router that I am working with. Now that this has been accomplished I wish to pass the same command several times through the ssh log-in instead of just one from a text file, if it is possible.
The other way to accomplish this would be to create a loop and pass the command over, and over again. I would have to use a variable though because the data for the command in the text file changes.
Here is what I have so far
#!/bin/expect -f
set password password
spawn ssh -l root x.x.x.x -p "command"
expect "*password:*"
send -- "$password\r"
send -- "\r"
From what I can see creating a loop would be the easiest way, but I may be wrong. NOTE that the "command & variables" that I want to pass through are in a separate text file, and that it needs to read/take each line and insert each one into the loop. Unless there is a way to send them through all at once.
#!/bin/expect -f
set password password
spawn ssh -l root x.x.x.x -p "command Variable" <-- Command to be passed through
expect "*password:*"
send -- "$password\r"
send -- "\r"
It is the same command every time in the text file, only the variable changes.
test.txt
command xxxxxxx
command xxxxxxx
command xxxxxxx
command xxxxxxx
Thank-you
I think you should do something like this.
start.sh
#!/bin/bash
password="your_password"
cat test.txt|while read line
do
for i in $line
do
ssh.exp $i $password
done
done
ssh.exp
#!/usr/bin/expect
set command [lrange $argv 0 0]
set password [lrange $argv 1 1]
spawn ssh -l root x.x.x.x -p "$command"
expect "*password:*"
send -- "$password\r"
send -- "\r"
And test.txt with list of your commands. Each on the different line.