I've been struggling with getting the output from a remote server to a local variable or a local file.
My attempt:
#!/bin/bash
my_pass=!!psw!!
server=10.10.10.10
/usr/bin/expect << ENDOFEXPECT
exp_internal 1 ;# expect internal debugging. remove when not needed
set PROMPT ":~ ?# ?"
set timeout 30
spawn bash -c "ssh root#$server"
expect "assword:"
send "$my_pass\r"
expect -re "$PROMPT"
send -- "df -kh /\r"
expect -re "df\[^\n]+\n.+\n(.+\r\n.+)\r\n"
set command_output $expect_out(1,string)
send_user "$command_output\r"
interact
ENDOFEXPECT
echo "====================="
echo " >> $command_output"
Output:
spawn bash -c ssh root#10.10.10.10
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {154725}
expect: does "" (spawn_id exp4) match glob pattern "assword:"? no
Password:
expect: does "\rPassword: " (spawn_id exp4) match glob pattern "assword:"? yes
expect: set expect_out(0,string) "assword:"
expect: set expect_out(spawn_id) "exp4"
expect: set expect_out(buffer) "\rPassword:"
send: sending "!!psw!!\r" to { exp4 }
Gate keeper glob pattern for '' is ''. Not usable, disabling the performance booster.
expect: does " " (spawn_id exp4) match regular expression ""? (No Gate, RE only) gate=yes re=yes
expect: set expect_out(0,string) ""
expect: set expect_out(spawn_id) "exp4"
expect: set expect_out(buffer) ""
send: sending "df -kh /\r" to { exp4 }
Gate keeper glob pattern for 'df[^
]+
.+
(.+
.+)
' is ''. Not usable, disabling the performance booster.
expect: does " " (spawn_id exp4) match regular expression "df[^\n]+\n.+\n(.+\r\n.+)\r\n"? (No Gate, RE only) gate=yes re=no
expect: does " \r\n" (spawn_id exp4) match regular expression "df[^\n]+\n.+\n(.+\r\n.+)\r\n"? (No Gate, RE only) gate=yes re=no
Last login: Fri Dec 2 23:58:09 2022 from 10.10.10.1
Welcome to server image 2.2
expect: does " \r\nLast login: Fri Dec 2 23:58:09 2022 from 10.10.10.1\r\r\n\r\nWelcome to server image 2.2\r\n\r\n" (spawn_id exp4) match regular expression "df[^\n]+\n.+\n(.+\r\n.+)\r\n"? (No Gate, RE only) gate=yes re=no
REMY_SERVER:~ #
expect: does " \r\nLast login: Fri Dec 2 23:58:09 2022 from 10.10.10.1\r\r\n\r\nWelcome to server image 2.2\r\n\r\n\u001b[?1034h\u001b[1m\u001b[31mREMY_SERVER:~ # \u001b(B\u001b[m" (spawn_id exp4) match regular expression "df[^\n]+\n.+\n(.+\r\n.+)\r\n"? (No Gate, RE only) gate=yes re=no
expect: timed out
interact: received eof from spawn_id exp0
=====================
>>
Expected:
What I ultimately want is to get the output of df -kh into a local variable or even better, append it directly to a local file (on the local machine, not the server on which the command is executed) so that it contains something like:
$ cat ./result.txt
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 20G 18G 1,7G 92% /
Method 1: The proper way is to not use expect and use key pair access :
Step #1
Setup a SSH key pair (google it) and then copy the SSH key to the remote server. To do this I'd recommend using ssh-copy-id.
Step #2
Now with the ability to SSH to a server in place using a key, your above problem turns into this:
$ ssh root#10.10.10.10 "df -kh"
You can get fancy and use here documents (heredocs aka. here-docs) to further enhance this technique.
$ ssh root#10.10.10.10 <<EOF
> df -kh
> EOF
or put the commands in a file and pass them to ssh:
$ ssh root#10.10.10.10 < my.cmds
Method 2: Expect
See the following, expains how to use it properly and a tool to create expect scripts
https://hostadvice.com/how-to/how-to-automate-tasks-in-ssh/
First, your PROMPT regex is not matching. I see the output has some colour codes in it:
expect: does " \r\nLast login: ...REMY_SERVER:~ # \u001b(B\u001b[m" (spawn_id exp4) match regular expression ...
It's good to anchor prompt regexes, and to enclose them in braces. Try
set PROMPT { # \S*$}
Or, assuming the login shell is bash, set a new prompt that's easier to match:
send "$my_pass\r"
expect "Welcome to server"
send -- "PS1='>'\r"
set PROMPT {>$}
expect -re $PROMPT
Next, the relevant code for the question.
send -- "df -kh /\r"
expect -re "df\[^\n]+\n.+\n(.+\r\n.+)\r\n"
set command_output $expect_out(1,string)
send_user "$command_output\r"
I'd adjust your regex a touch:
set cmd "df -kh /"
send -- "$cmd\r"
expect -re "$cmd\r\n(.+)\r\n.*$PROMPT"
Then you're capturing and "echoing" the result correctly
set command_output $expect_out(1,string)
send_user "$command_output\n"
# use a newline here ......^
And to append it to a local file:
set fh [open ./results.txt a]
puts $fh $command_output
close $fh
Related
I'm trying to create a script that retreives a secret from a keepass database.
The script uses Expect to get secret via the keepass cli.
Here after my script:
#!/bin/bash
set +x
entry="$1"
keepass_password="azerty"
keepass_db="canadaplatfomsecretsdb.kdbx"
keepass_entry_dir="canadaplatfomsecretsdb/k8s-enabling"
kubesecretname="artifactoryregcred"
kubenamespace="dev"
echo -e "\n"
echo "Connecting to keepass Database..."
function get_creds {
expect -d <<EOF
set timeout 10
match_max 100000000
spawn kpcli
expect "kpcli:/>"
send "open $keepass_db\n"
expect "password:"
send "$keepass_password\n"
expect ">"
send "cd $keepass_entry_dir\n"
expect "k8s-enabling>"
send "show -f $entry\n"
expect ">"
EOF
}
credentials=$(get_creds)
Here after the output form the gitlab-ci job logs
$ bash getcredsfromkeepass.sh ${entry}
Connecting to keepass Database...
expect version 5.45.4
argv[0] = expect argv[1] = -d
set argc 0
set argv0 "expect"
set argv ""
executing commands from command file
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {3855}
expect: does "" (spawn_id exp3) match glob pattern "kpcli:/>"? no
expect: does "No usable Term::ReadLine::* modules found.\r\nThis list was tried:\r\n * Term::ReadLine::Gnu\r\n * Term::ReadLine::Perl\r\n * Term::ReadLine::Perl5\r\nFor more information, read the documentation: perldoc kpcli\r\n" (spawn_id exp3) match glob pattern "kpcli:/>"? no
expect: read eof
expect: set expect_out(spawn_id) "exp3"
What is causing the problem and how can it be fixed?
I need to get a dataset from a telnet shell, for this I have looked at expect https://www2.lib.uchicago.edu/keith/tcl-course/topics/expect.html
This is my expect script:
/usr/bin/expect -d << EOD
spawn telnet myhost
log_user 1
set timeout 10
expect "Login:"
send "user\n\r"
expect "Password:"
send "password\n\r"
expect "*$*"
send "AT\n\r"
expect "OK"
send "AT&CSR\n"
expect "OK"
send "AT!G=A6\n"
expect "OK"
send "AT^MI=0\n"
expect "OK"
send "AT^SX=0\n"
expect "OK"
send "AT^SR=0,1"
expect "*$*"
send "exit\r"
Now I can connect to the remote host, send command, but cannot read the results, here a shell result:
> expect: does " " (spawn_id exp4) match glob pattern "*"? yes expect:
> set expect_out(0,string) " " expect: set expect_out(spawn_id) "exp4"
> expect: set expect_out(buffer) " " send: sending "AT\n\r" to { exp4 }
>
> expect: does "" (spawn_id exp4) match glob pattern "OK"? no
> ********
>
> OK
>
> expect: does "********\r\n\r\nOK\r\n" (spawn_id exp4) match glob
> pattern "OK"? yes expect: set expect_out(0,string) "OK" expect: set
> expect_out(spawn_id) "exp4" expect: set expect_out(buffer)
> "********\r\n\r\nOK" send: sending "AT&CSR\n" to { exp4 }
>
> expect: does "\r\n" (spawn_id exp4) match glob pattern "OK"? no AT
>
> expect: does "\r\nAT\r\n" (spawn_id exp4) match glob pattern "OK"? no
>
> OK
>
> expect: does "\r\nAT\r\n\r\nOK\r\n" (spawn_id exp4) match glob pattern
> "OK"? yes expect: set expect_out(0,string) "OK" expect: set
> expect_out(spawn_id) "exp4" expect: set expect_out(buffer)
> "\r\nAT\r\n\r\nOK" send: sending "AT!G=A6\n" to { exp4 }
>
> expect: does "\r\n" (spawn_id exp4) match glob pattern "OK"? no AT&CSR
> expect: does "\r\nAT&CSR" (spawn_id exp4) match glob pattern "OK"? no
> AT!G=A6 expect: does "\r\nAT&CSRAT!G=A6" (spawn_id exp4) match glob
> pattern "OK"? no expect: timed out send: sending "AT^MI=0\n" to { exp4
> }
>
> expect: does "\r\nAT&CSRAT!G=A6" (spawn_id exp4) match glob pattern
> "OK"? no AT^MI=0
If the result of the command doesn't match the expected it goes to next command, is there a way to catch the response result?
EDIT:
I tried to simplify thing and added interact, without no luck:
/usr/bin/expect -d << EOD
> spawn telnet myhost
> log_user 1
> interact"
> EOD
expect version 5.45.4
argv[0] = /usr/bin/expect argv[1] = -d
set argc 0
set argv0 "/usr/bin/expect"
set argv ""
executing commands from command file
spawn telnet myhost
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {11306}
invalid command name "interact""
while executing
"interact""
The way you send your AT commands is wrong.
According to ETSI specification 127.007, by default AT commands are terminated by \r (IRA 13 - ASCII carriage return) character[1]. For further details, see paragraph "4.1 - Command line".
In order to fix your issue, just send AT\r instead of AT\n\r, AT&CSR\r instead of AT&CSR\n and so on (the latter is a mistake you repeated for every following AT command). Otherwise the AT parser won't recognize your commands correctly.
[1] Command line termination character can be customized by means of ATS3 command, with the following syntax:
ATS3=<value>
Where <value> is the decimal ASCII value of the new command line termination character (default is 13).
How can I control bash with Expect? My Expect script looks like this:
#!/usr/bin/expect
# ENABLE DEBUGGING
exp_internal 1
set timeout 10
log_user 0
spawn bash -i
sleep 5
send "ls -1 db*\r"
expect {
-re "^db.*$" {
puts $expect_out(0,string)
}
timeout {
send_error "Script has reached the 'timeout' branch\n"
}
}
But I always get this output which is caused by time out:
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {4740}
send: sending "ls -1 db*\r" to { exp4 }
Gate keeper glob pattern for '^db.*$' is 'db*'. Activating booster.
expect: does "" (spawn_id exp4) match regular expression "^db.*$"? Gate "db*"? gate=no
expect: does "Agent pid 6228\r\nIdentity added: /home/wakatana/.ssh/id_rsa (/home/wakatana/.ssh/id_rsa)\r\n\u001b[?1034h\u001b]0;~/scripts\u0007\r\r\n\u001b[32mwakatana#ANTARES \u001b[33m~/scripts\u001b[0m\r\r\n$ ls -1 db*\r\n" (spawn_id exp4) match regular expression "^db.*$"? Gate "db*"? gate=yes re=no
expect: does "Agent pid 6228\r\nIdentity added: /home/wakatana/.ssh/id_rsa (/home/wakatana/.ssh/id_rsa)\r\n\u001b[?1034h\u001b]0;~/scripts\u0007\r\r\n\u001b[32mwakatana#ANTARES \u001b[33m~/scripts\u001b[0m\r\r\n$ ls -1 db*\r\ndbupgrade.log\r\n" (spawn_id exp4) match regular expression "^db.*$"? Gate "db*"? gate=yes re=no
expect: does "Agent pid 6228\r\nIdentity added: /home/wakatana/.ssh/id_rsa (/home/wakatana/.ssh/id_rsa)\r\n\u001b[?1034h\u001b]0;~/scripts\u0007\r\r\n\u001b[32mwakatana#ANTARES \u001b[33m~/scripts\u001b[0m\r\r\n$ ls -1 db*\r\ndbupgrade.log\r\n\u001b]0;~/scripts\u0007\r\r\n\u001b[32mwakatana#ANTARES \u001b[33m~/scripts\u001b[0m\r\r\n$ " (spawn_id exp4) match regular expression "^db.*$"? Gate "db*"? gate=yes re=no
expect: timed out
Script has reached the 'timeout' branch
File that I am trying to ls exists:
$ ls -1 db*
dbupgrade.log
PS: This was inspired by this question
In Expect, the ^ and $ mean very differently:
Note that in many editors, the ^ and $ match the beginning and
end of lines respectively. However, because expect is not line
oriented, these characters match the beginning and end of the
data (as opposed to lines) currently in the expect matching
buffer.
You can do like this:
[STEP 101] # cat foo.exp
spawn bash --noprofile --norc
sleep 1
send "ls -1 db*\r"
expect {
-re {[\r\n]+(db.*?)[\r\n]+} {
send_user "\n>>> $expect_out(1,string) <<<\n"
}
}
[STEP 102] # ls -1 dbupgrade.log
dbupgrade.log
[STEP 103] # expect foo.exp
spawn bash --noprofile --norc
bash-4.3# ls -1 db*
dbupgrade.log
>>> dbupgrade.log <<<
[STEP 104] #
I have the simplest script ever:
#!/usr/bin/expect
expect "hello"
send -- "ll \r"
When I run it, I manually type "hello" word and that ll command never runs ...
Afterwards I put exp_internal 1 and then this comes up
[root#localhost tmp]# ./file1.exp
expect: does "" (spawn_id exp0) match glob pattern "hello"? no
hello
expect: does "hello\n" (spawn_id exp0) match glob pattern "hello"? yes
expect: set expect_out(0,string) "hello"
expect: set expect_out(spawn_id) "exp0"
expect: set expect_out(buffer) "hello"
}end: sending "ll \r" to { exp0 ll
[root#localhost tmp]#
Can anyone explains me why the command is not being ran as command ?
The following expect / code pair only works sometimes:
Simple cpp echo program:
#include <iostream>
#include <string>
using namespace std;
int main()
{
string buffer;
while (1)
{
getline(cin, buffer, '\n');
if (cin.eof())
break;
cout<<buffer<<endl;
}
return 0;
}
Expect script
#!/usr/bin/expect
spawn ./echo
exp_internal 1
set timeout 1
send "a\n"
expect {
-re {^a\r\n$}
}
Success match:
spawn ./echo
send: sending "a\n" to { exp6 }
Gate keeper glob pattern for '^a\r\n$' is 'a
'. Activating booster.
expect: does "" (spawn_id exp6) match regular expression "^a\r\n$"? Gate "a\r\n"? gate=no
a
expect: does "a\r\n" (spawn_id exp6) match regular expression "^a\r\n$"? Gate "a\r\n"? gate=yes re=yes
expect: set expect_out(0,string) "a\r\n"
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) "a\r\n"
Failure match:
spawn ./echo
send: sending "a\n" to { exp6 }
Gate keeper glob pattern for '^a\r\n$' is 'a
'. Activating booster.
expect: does "" (spawn_id exp6) match regular expression "^a\r\n$"? Gate "a\r\n" gate=no
a
a
expect: does "a\r\na\r\n" (spawn_id exp6) match regular expression "^a\r\n$"? Gate "a\r\n"? gate=yes re=no
expect: timed out
I don't understand why on some runs I get the double line "a", and some runs I get the single one? I would have assumed the expect hooks up spawned process such that send will pipe everything to stdin and expect gets everything from stdout.
Any clarifications would be appreciated.
It is because your tty have echo enabled. In your first case you send a\n and your program read it and return a\n but expect did read before std::cout is flushed so the buffer contain only a\n. In the other case it read and send didn't finished so it read nothing and the second run it does read and your std::cout was flushed before. So you read two times. You could modify your program to output something else (like with a prefix) or add a dummy expect "<non existing string>" as last instruction so you force to drain stdout.
$ ./expect.sh
spawn ./echo
send: sending "a\n" to { exp6 }
Gate keeper glob pattern for '^a\r\n$' is 'a
'. Activating booster.
expect: does "" (spawn_id exp6) match regular expression "^a\r\n$"? Gate "a\r\n"? gate=no
a
expect: does "a\r\n" (spawn_id exp6) match regular expression "^a\r\n$"? Gate "a\r\n"? gate=yes re=yes
expect: set expect_out(0,string) "a\r\n"
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) "a\r\n"
expect: does "" (spawn_id exp6) match glob pattern "<non existing string>"? no
a
expect: does "a\r\n" (spawn_id exp6) match glob pattern "<non existing string>"? no
expect: timed out
To fix the root cause you can first disabling echo with stty -echo and then spawn with option -nottyinit used like this spawn -nottyinit ./echo because
-nottyinit skips the "sane" initialization
An alternative way is to set set stty_init -echo then spawn
https://github.com/aeruder/expect/blob/ce11b8121e6900888d137fec0b990bd1484cbb7c/pty_unicos.c#L147-L155 here is where tty is setup. stty_init is applied after tty sane. So it might be a preferred method.