Using expect to confirm a password - bash

What I'm trying to do is write an expect script that backs up all databases on a server, then create an encrypted zip file (I couldn't get my tar/openssl command to run properly in expect) containing the sql file.
This is just an exercise to learn about expect, not a real backup solution.
I'm obviously lacking some understanding here. What I want to do is:
Back up all databases to a file (done)
Run the zip command to create an encrypted zip file (done... sorta)
Respond to "Enter password: "
Then response to the confirmation "Verify password: "
#!/usr/bin/expect -f
exp_internal 1
set backupdir "/mnt/db-backups/"
set now [clock seconds]
set date [clock format $now -format {%Y-%m-%d}]
set filename $date
append filename "_dbbackups.sql"
exec mysqldump -u root --all-databases --events > $backupdir$filename
spawn zip -e $backupdir$filename.enc.zip $backupdir$filename
expect {
"Enter password: " { send "monkey"
exp_continue
}
"Verify password: " {send "monkey"
exp_continue
}
}
And the output is
$expect encrypt.sh
spawn zip -e /mnt/db-backups/2013-12-11_dbbackups.sql.enc.zip /mnt/db-backups/2013-12-11_dbbackups.sql
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {15733}
expect: does "" (spawn_id exp6) match glob pattern "Enter password: "? no
"Verify password: "? no
Enter password:
expect: does "Enter password: " (spawn_id exp6) match glob pattern "Enter password: "? yes
expect: set expect_out(0,string) "Enter password: "
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) "Enter password: "
send: sending "monkey" to { exp6 }
expect: continuing expect
expect: does "" (spawn_id exp6) match glob pattern "Enter password: "? no
"Verify password: "? no
monkey
expect: does "monkey" (spawn_id exp6) match glob pattern "Enter password: "? no
"Verify password: "? no
-- then I exited --
It's a pretty simple script... but I'm sucking at it.

The problem is simple:
You have to press return. So just change the command
send "monkey"
to
send "monkey\r"
(both of them)

DRY:
spawn zip -e $backupdir$filename.enc.zip $backupdir$filename
expect {
"* password: " {
send "monkey\r"
exp_continue
}
eof
}

[...] write an expect script [...]
Not how I'd do it.
zip -P password [...]
Knowing nothing about your Zip version ("zip -v"), it's hard to say
if you simply didn't look at the documentation ("zip -h2", "man zip").
Putting a password into a script is a sufficient security hazard that
"-P" was left undocumented until Zip 3.0 (when the developers got tired
of answering the how-do-I questions). But putting a password into a
script once can hardly be worse than putting a password into a script
twice.

Expect is a really ugly and troublesome solution for this.
For the mysqldump part, add your database connection information and the password in ~/.my.cnf following this format:
[client]
database=dbname
user=dbuser
password=dbpass
host=dbhost
If you work with multiple databases, then you'll have to create one file per database, for example ~/.my.cnf.dbname, and call mysqldump like this:
mysqldump --defaults-file=~/.my.cnf.$dbname $dbname | gzip >"$target"
Make sure to do chmod 600 on this file to make it as secure as possible.
And when you encrypt with openssl, you can put the password on the command line like this:
... | openssl des3 -pass pass:monkey >"$target"
You can put this after gzip in the earlier command. Of course, use a different cipher instead of des3 if you want more security. Also, instead of passing the password on the command line, you might want to use file:pathname option instead to get the password from the first line of the file instead.

Related

Extract EXPECT result to local variable/file

I've been struggling with getting the output from a remote server to a local variable or a local file.
My attempt:
#!/bin/bash
my_pass=!!psw!!
server=10.10.10.10
/usr/bin/expect << ENDOFEXPECT
exp_internal 1 ;# expect internal debugging. remove when not needed
set PROMPT ":~ ?# ?"
set timeout 30
spawn bash -c "ssh root#$server"
expect "assword:"
send "$my_pass\r"
expect -re "$PROMPT"
send -- "df -kh /\r"
expect -re "df\[^\n]+\n.+\n(.+\r\n.+)\r\n"
set command_output $expect_out(1,string)
send_user "$command_output\r"
interact
ENDOFEXPECT
echo "====================="
echo " >> $command_output"
Output:
spawn bash -c ssh root#10.10.10.10
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {154725}
expect: does "" (spawn_id exp4) match glob pattern "assword:"? no
Password:
expect: does "\rPassword: " (spawn_id exp4) match glob pattern "assword:"? yes
expect: set expect_out(0,string) "assword:"
expect: set expect_out(spawn_id) "exp4"
expect: set expect_out(buffer) "\rPassword:"
send: sending "!!psw!!\r" to { exp4 }
Gate keeper glob pattern for '' is ''. Not usable, disabling the performance booster.
expect: does " " (spawn_id exp4) match regular expression ""? (No Gate, RE only) gate=yes re=yes
expect: set expect_out(0,string) ""
expect: set expect_out(spawn_id) "exp4"
expect: set expect_out(buffer) ""
send: sending "df -kh /\r" to { exp4 }
Gate keeper glob pattern for 'df[^
]+
.+
(.+
.+)
' is ''. Not usable, disabling the performance booster.
expect: does " " (spawn_id exp4) match regular expression "df[^\n]+\n.+\n(.+\r\n.+)\r\n"? (No Gate, RE only) gate=yes re=no
expect: does " \r\n" (spawn_id exp4) match regular expression "df[^\n]+\n.+\n(.+\r\n.+)\r\n"? (No Gate, RE only) gate=yes re=no
Last login: Fri Dec 2 23:58:09 2022 from 10.10.10.1
Welcome to server image 2.2
expect: does " \r\nLast login: Fri Dec 2 23:58:09 2022 from 10.10.10.1\r\r\n\r\nWelcome to server image 2.2\r\n\r\n" (spawn_id exp4) match regular expression "df[^\n]+\n.+\n(.+\r\n.+)\r\n"? (No Gate, RE only) gate=yes re=no
REMY_SERVER:~ #
expect: does " \r\nLast login: Fri Dec 2 23:58:09 2022 from 10.10.10.1\r\r\n\r\nWelcome to server image 2.2\r\n\r\n\u001b[?1034h\u001b[1m\u001b[31mREMY_SERVER:~ # \u001b(B\u001b[m" (spawn_id exp4) match regular expression "df[^\n]+\n.+\n(.+\r\n.+)\r\n"? (No Gate, RE only) gate=yes re=no
expect: timed out
interact: received eof from spawn_id exp0
=====================
>>
Expected:
What I ultimately want is to get the output of df -kh into a local variable or even better, append it directly to a local file (on the local machine, not the server on which the command is executed) so that it contains something like:
$ cat ./result.txt
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 20G 18G 1,7G 92% /
Method 1: The proper way is to not use expect and use key pair access :
Step #1
Setup a SSH key pair (google it) and then copy the SSH key to the remote server. To do this I'd recommend using ssh-copy-id.
Step #2
Now with the ability to SSH to a server in place using a key, your above problem turns into this:
$ ssh root#10.10.10.10 "df -kh"
You can get fancy and use here documents (heredocs aka. here-docs) to further enhance this technique.
$ ssh root#10.10.10.10 <<EOF
> df -kh
> EOF
or put the commands in a file and pass them to ssh:
$ ssh root#10.10.10.10 < my.cmds
Method 2: Expect
See the following, expains how to use it properly and a tool to create expect scripts
https://hostadvice.com/how-to/how-to-automate-tasks-in-ssh/
First, your PROMPT regex is not matching. I see the output has some colour codes in it:
expect: does " \r\nLast login: ...REMY_SERVER:~ # \u001b(B\u001b[m" (spawn_id exp4) match regular expression ...
It's good to anchor prompt regexes, and to enclose them in braces. Try
set PROMPT { # \S*$}
Or, assuming the login shell is bash, set a new prompt that's easier to match:
send "$my_pass\r"
expect "Welcome to server"
send -- "PS1='>'\r"
set PROMPT {>$}
expect -re $PROMPT
Next, the relevant code for the question.
send -- "df -kh /\r"
expect -re "df\[^\n]+\n.+\n(.+\r\n.+)\r\n"
set command_output $expect_out(1,string)
send_user "$command_output\r"
I'd adjust your regex a touch:
set cmd "df -kh /"
send -- "$cmd\r"
expect -re "$cmd\r\n(.+)\r\n.*$PROMPT"
Then you're capturing and "echoing" the result correctly
set command_output $expect_out(1,string)
send_user "$command_output\n"
# use a newline here ......^
And to append it to a local file:
set fh [open ./results.txt a]
puts $fh $command_output
close $fh

Expect script to change default password

I'm trying to build a simple Expect script to change the default password. Our admin forces us to change passwords immediately after deploying a new system in our cloud infrastructure. I want to build an Expect script because I have to do this across a few dozen systems.
The prompts look like this:
❯ ssh root#mysystem.company.com
root#mysystem.company.com's password:
You are required to change your password immediately (administrator enforced)
Last login: Tue Aug 23 12:14:58 2022 from 10.123.45.67
WARNING: Your password has expired.
You must change your password now and login again!
Changing password for user root.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
Connection to mysystem.company.com closed.
My expect script looks like this:
#! /usr/local/bin/expect -f
set timeout 60
set user [lindex $argv 0]
set host [lindex $argv 1]
set old [lindex $argv 2]
set new [lindex $argv 3]
spawn -noecho ssh -q -o StrictHostKeychecking=no "$user\#$host"
expect "password:"
send "$old\r"
expect "New password"
send "$new\r"
expect "Retype new password:"
send "$new\r"
That 1st expect statement is working, but I cannot figure out why the 2nd expect statement isn't working:
❯ expect spawn root mysystem.company.com oldPassword newPassword
root#mysystem.company.com's password:
You are required to change your password immediately (administrator enforced)
Last login: Tue Aug 23 13:14:12 2022 from 10.123.45.67
WARNING: Your password has expired.
You must change your password now and login again!
Changing password for user root.
New password: %
It immediately exits from the script once it reaches New password:. I've tried various things but not figuring this out.
Please help!
It was suggested to re-run with -d (debug) enabled. The output appears to show its working as expected (no pun intended), but the password wasn't actually changed. When I attempt to login, I'm prompted through the same steps. Maybe someone else can help interpret this output:
❯ expect -d spawn root mysystem.company.com lgmPjgY2xTJH 73a40HpVCgRey9fP
expect version 5.45
argv[0] = expect argv[1] = -d argv[2] = spawn argv[3] = root argv[4] = mysystem.company.com argv[5] = lgmPjgY2xTJH argv[6] = 73a40HpVCgRey9fP
set argc 4
set argv0 "spawn"
set argv "root mysystem.company.com lgmPjgY2xTJH 73a40HpVCgRey9fP"
executing commands from command file spawn
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {10930}
expect: does "" (spawn_id exp6) match glob pattern "password:"? no
expect: does "\r" (spawn_id exp6) match glob pattern "password:"? no
root#mysystem.company.com's password:
expect: does "\rroot#mysystem.company.com's password: " (spawn_id exp6) match glob pattern "password:"? yes
expect: set expect_out(0,string) "password:"
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) "\rroot#mysystem.company.com's password:"
send: sending "lgmPjgY2xTJH\r" to { exp6 }
expect: does " " (spawn_id exp6) match glob pattern "New password"? no
expect: does " \r\n" (spawn_id exp6) match glob pattern "New password"? no
You are required to change your password immediately (administrator enforced)
Last login: Wed Aug 24 20:01:38 2022 from 10.123.45.678
WARNING: Your password has expired.
You must change your password now and login again!
expect: does " \r\nYou are required to change your password immediately (administrator enforced)\r\nLast login: Wed Aug 24 20:01:38 2022 from 10.123.45.678\r\r\nWARNING: Your password has expired.\r\nYou must change your password now and login again!\r\n" (spawn_id exp6) match glob pattern "New password"? no
Changing password for user root.
expect: does " \r\nYou are required to change your password immediately (administrator enforced)\r\nLast login: Wed Aug 24 20:01:38 2022 from 10.123.45.678\r\r\nWARNING: Your password has expired.\r\nYou must change your password now and login again!\r\nChanging password for user root.\r\n" (spawn_id exp6) match glob pattern "New password"? no
New password:
expect: does " \r\nYou are required to change your password immediately (administrator enforced)\r\nLast login: Wed Aug 24 20:01:38 2022 from 10.123.45.678\r\r\nWARNING: Your password has expired.\r\nYou must change your password now and login again!\r\nChanging password for user root.\r\nNew password: " (spawn_id exp6) match glob pattern "New password"? yes
expect: set expect_out(0,string) "New password"
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) " \r\nYou are required to change your password immediately (administrator enforced)\r\nLast login: Wed Aug 24 20:01:38 2022 from 10.123.45.678\r\r\nWARNING: Your password has expired.\r\nYou must change your password now and login again!\r\nChanging password for user root.\r\nNew password"
send: sending "73a40HpVCgRey9fP\r" to { exp6 }
expect: does ": " (spawn_id exp6) match glob pattern "Retype new password:"? no
expect: does ": \r\n" (spawn_id exp6) match glob pattern "Retype new password:"? no
Retype new password:
expect: does ": \r\nRetype new password: " (spawn_id exp6) match glob pattern "Retype new password:"? yes
expect: set expect_out(0,string) "Retype new password:"
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) ": \r\nRetype new password:"
send: sending "73a40HpVCgRey9fP\r" to { exp6 }
❯ echo $?
0
According to the expect -d output, the script stopped after sending the password for the Retype new password: prompt.
So just add expect eof at the end of the script.
This is the most common mistake when using Expect. You must wait for the spawned program to finish.
(credit goes to #sexpect)

Bash script using Expect working fine locally but fails in a gitlab-ci job

I'm trying to create a script that retreives a secret from a keepass database.
The script uses Expect to get secret via the keepass cli.
Here after my script:
#!/bin/bash
set +x
entry="$1"
keepass_password="azerty"
keepass_db="canadaplatfomsecretsdb.kdbx"
keepass_entry_dir="canadaplatfomsecretsdb/k8s-enabling"
kubesecretname="artifactoryregcred"
kubenamespace="dev"
echo -e "\n"
echo "Connecting to keepass Database..."
function get_creds {
expect -d <<EOF
set timeout 10
match_max 100000000
spawn kpcli
expect "kpcli:/>"
send "open $keepass_db\n"
expect "password:"
send "$keepass_password\n"
expect ">"
send "cd $keepass_entry_dir\n"
expect "k8s-enabling>"
send "show -f $entry\n"
expect ">"
EOF
}
credentials=$(get_creds)
Here after the output form the gitlab-ci job logs
$ bash getcredsfromkeepass.sh ${entry}
Connecting to keepass Database...
expect version 5.45.4
argv[0] = expect argv[1] = -d
set argc 0
set argv0 "expect"
set argv ""
executing commands from command file
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {3855}
expect: does "" (spawn_id exp3) match glob pattern "kpcli:/>"? no
expect: does "No usable Term::ReadLine::* modules found.\r\nThis list was tried:\r\n * Term::ReadLine::Gnu\r\n * Term::ReadLine::Perl\r\n * Term::ReadLine::Perl5\r\nFor more information, read the documentation: perldoc kpcli\r\n" (spawn_id exp3) match glob pattern "kpcli:/>"? no
expect: read eof
expect: set expect_out(spawn_id) "exp3"
What is causing the problem and how can it be fixed?

Expect call shell script

I am trying to call a shell script and store the result in an expect variable. get_pw.sh accepts 2 args and decrypts the file using the provided md5hash. If I execute ./get_pw.sh file.test md5hash from the bash prompt it returns the password string as expected. When called from expect, the password does not get returned. The expect debug shows:
expect: does "" (spawn_id exp0) match regular expression "[^\s]"?
So it looks like the script is not returning the password string when called from expect. Relevant code:
#!/usr/bin/expect
send "./get_pw.sh file.test md5hash \r"
expect -re {[^\s]} {
set password $expect_out(0,string)
}
puts "The password is: $password"
You need to spawn a command first before you can send input and expect output from it.
To set an expect variable to the output of a command, use
set varname [exec command]
If you must do this with expect,
log_user 0
spawn -noecho get_pw.sh file hash
expect eof
set passwd [string trimright $expect_out(buffer) "\r\n"]
puts $passwd
Jens's answer looks pretty good by now ...

The role of puts in expect, script hangs after puts

I'm new to expect and learning from somebody's script. There is a block like this after the ssh command,
expect {
"Password:" {puts stderr "Wrong password."; exit 1}
"Last login:" {puts "Login Successful."}
timeout abort
}
Well, a problem of this in my case is that if this is the first login to the machine, there won't be a "Last login" showing up. So what I did was to add something and thought this might work
expect {
"Password:" {puts stderr "Wrong password."; exit 1}
"Last login:" {puts "Login Successful."}
"# " {puts "Login Successful."}
timeout abort
}
Now it can print the message, however after adding this the script just hang there after login succeeded.
Turning on -d option while running it, looks like it's trying to match "", not sure how does puts results in this? Or what mistake am I making here?
EDIT:
The last few lines of -d output is
expect: does " *********# " (spawn_id exp6) match glob pattern "Password:"? no
"Last login:"? no
"# "? yes
expect: set expect_out(0,string) "# "
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) " *********# "
Login Successful too.
expect: does "" (spawn_id exp6) match glob pattern "# "? no
I replaced something with private info with ***. The next line in my script is
expect "# "

Resources