How to prevent pexpect from echoing the password? - python-2.x

By default, pexpect.spawn() would not output anything. But when I specify logfile=sys.stdout it'll also echo the password (e.g. for ssh). So how can I see the real time interaction with the spawned process without the password being echoed (just like the Expect (the Tcl extension) does)?
pexepct example:
# cat expect.py
import pexpect, sys
logfile = sys.stdout if len(sys.argv) == 2 else None
ssh = pexpect.spawn('ssh foo#localhost', logfile=logfile)
ssh.delaybeforesend = 1
ssh.expect('assword:')
ssh.sendline('123456')
ssh.expect('\r\n\\$')
ssh.sendline('exit')
ssh.expect(pexpect.EOF)
ssh.wait()
# python expect.py <-- no output
# python expect.py stdout
foo#localhost's password: 123456 <-- the password is visible
Last login: Tue Mar 22 10:32:49 2016 from localhost
$ exit
exit
Connection to localhost closed.
#
Expect example:
# cat ssh.exp
spawn ssh foo#localhost
expect assword:
send "123456\r"
expect {\$}
send "exit\r"
expect eof
wait
# expect ssh.exp
spawn ssh foo#localhost
foo#localhost's password: <-- the password is invisible
Last login: Tue Mar 22 10:45:03 2016 from localhost
$ exit
Connection to localhost closed.
#

Just to make the question answered. The credit goes to Thomas K. See his comments under the question for more details.
[STEP 101] # cat foo.py
#!/usr/bin/env python3
import pexpect, sys
spawn = pexpect.spawnu if sys.version_info[0] >= 3 else pexpect.spawn
ssh = spawn('ssh -t foo#localhost bash --noprofile --norc')
ssh.logfile_read = sys.stdout
ssh.expect('assword:')
ssh.sendline('123456')
ssh.expect('bash-[.0-9]+[$#]')
ssh.sendline('exit')
ssh.expect(pexpect.EOF)
ssh.wait()
[STEP 102] #
[STEP 103] # python2 foo.py
foo#localhost's password:
bash-5.1$ exit
exit
Connection to localhost closed.
[STEP 104] #
[STEP 105] # python3 foo.py
foo#localhost's password:
bash-5.1$ exit
exit
Connection to localhost closed.
[STEP 106] #

Related

Bash script using Expect working fine locally but fails in a gitlab-ci job

I'm trying to create a script that retreives a secret from a keepass database.
The script uses Expect to get secret via the keepass cli.
Here after my script:
#!/bin/bash
set +x
entry="$1"
keepass_password="azerty"
keepass_db="canadaplatfomsecretsdb.kdbx"
keepass_entry_dir="canadaplatfomsecretsdb/k8s-enabling"
kubesecretname="artifactoryregcred"
kubenamespace="dev"
echo -e "\n"
echo "Connecting to keepass Database..."
function get_creds {
expect -d <<EOF
set timeout 10
match_max 100000000
spawn kpcli
expect "kpcli:/>"
send "open $keepass_db\n"
expect "password:"
send "$keepass_password\n"
expect ">"
send "cd $keepass_entry_dir\n"
expect "k8s-enabling>"
send "show -f $entry\n"
expect ">"
EOF
}
credentials=$(get_creds)
Here after the output form the gitlab-ci job logs
$ bash getcredsfromkeepass.sh ${entry}
Connecting to keepass Database...
expect version 5.45.4
argv[0] = expect argv[1] = -d
set argc 0
set argv0 "expect"
set argv ""
executing commands from command file
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {3855}
expect: does "" (spawn_id exp3) match glob pattern "kpcli:/>"? no
expect: does "No usable Term::ReadLine::* modules found.\r\nThis list was tried:\r\n * Term::ReadLine::Gnu\r\n * Term::ReadLine::Perl\r\n * Term::ReadLine::Perl5\r\nFor more information, read the documentation: perldoc kpcli\r\n" (spawn_id exp3) match glob pattern "kpcli:/>"? no
expect: read eof
expect: set expect_out(spawn_id) "exp3"
What is causing the problem and how can it be fixed?

expect redirect outputs giberish

I have backup.sh
#!/bin/bash
expect backup.exp
and backup.exp
#!/usr/bin/expect -f
#exp_internal 1
#log_user 0
set timeout 29
puts "----------------- [exec date +%Y.%m.%d\ %H:%M:%S] ------ exp start -----"
spawn -noecho ssh andrej#10.11.22.17
expect {
timeout { send_user "\n--- failed to get expected string---\n"; exit 1 }
eof { send_user "\nSSH failure\n"; exit 1 }
" andrej $ "
}
send "sudo bash\r"
send "ls -lh\r"
send exit\r
send exit\r
interact
puts "----------------- [exec date +%Y.%m.%d\ %H:%M:%S] ------ exp ende ------"
when I run
bash backup.sh >> backup.templog 2>&1
I get
----------------- 2015.09.02 18:48:29 ------ exp start -----
Last login: Wed Sep 2 18:48:24 2015 from 10.11.22.16
^[]0;andrej#centos7c:~/andrej^G^[[?1034h^[[01;32mandrej#centos7c ^[[01;34m18:48 andrej $ ^[[00msudo bash
^[]0;root#centos7c:/home/andrej/andrej^G^[[?1034h^[[01;31mroot#centos7c ^[[01;34m18:48 andrej $ ^[[00mls -lh
total 16K
-rw-rw-r--. 1 andrej andrej 13K Jun 30 19:04 iptables.sh
drwxrwxr-x. 2 andrej andrej 6 Jun 17 22:46 ^[[0m^[[01;34mmount^[[0m
^[]0;root#centos7c:/home/andrej/andrej^G^[[01;31mroot#centos7c ^[[01;34m18:48 andrej $ ^[[00mexit
exit
^[]0;andrej#centos7c:~/andrej^G^[[01;32mandrej#centos7c ^[[01;34m18:48 andrej $ ^[[00mexit
logout
Connection to 10.11.22.17 closed.
----------------- 2015.09.02 18:48:39 ------ exp ende ------
but when I do
bash backup.sh
, I get nice output that I need :
----------------- 2015.09.02 18:52:09 ------ exp start -----
Last login: Wed Sep 2 18:48:39 2015 from 10.11.22.16
andrej#centos7c 18:52 andrej $ sudo bash
root#centos7c 18:52 andrej $ ls -lh
total 16K
-rw-rw-r--. 1 andrej andrej 13K Jun 30 19:04 iptables.sh
drwxrwxr-x. 2 andrej andrej 6 Jun 17 22:46 mount
root#centos7c 18:52 andrej $ exit
exit
andrej#centos7c 18:52 andrej $ exit
logout
Connection to 10.11.22.17 closed.
----------------- 2015.09.02 18:52:20 ------ exp ende ------
How can I get rid of extra giberish when outputing to the log file when executing the script? I am using centos 7.1 in both server and client.
If you have any sugesstion please help me out. Thank you very much.
rewriting a but to address a couple of issues:
expect has 2 built-in methods to get the current datetime: timestamp (expect-specific) and clock (from Tcl)
I turn off output (log_user 0)
when it's important, I change the prompt and use an anchored regular expression to match it
I only output the results of the ls -lh command, which I suspect is your goal.
#!/usr/bin/expect -f
#exp_internal 1
set timeout 29
proc separator {msg} {
puts [format "----------------- %s ------ %s -----" [timestamp -format "%Y.%m.%d %T"] $msg]
}
separator "exp start"
log_user 0
spawn -noecho ssh andrej#10.11.22.17
expect {
timeout { send_user "\n--- failed to get expected string---\n"; exit 1 }
eof { send_user "\nSSH failure\n"; exit 1 }
" $ "
}
send "sudo bash\r"
expect " $ "
send "PS1='$'\r"
expect -re {\$$}
send "ls -lh\r"
expect -re {(.+)\r\n\$$}
puts $expect_out(1,string)
send "exit\r"
expect " $ "
send "exit\r"
expect eof
separator "exp ende"

Changing an AIX password via script?

I am trying to change a password of a user via script. I cannot use sudo as there is a feature that requires the user to change the password again if another user changes their password.
AIX is running on the system.
unfortunately, chpasswd is unavailable.
I have expected installed, but I am having trouble with that also.
here is what I thought would work
echo "oldpassword\nnewpasswd123\nnewpasswd123" | passwd user
However once run the script I am prompted with please enter user's old password
shouldn't they all be echoed in?
I am a beginner with shell scripting and this has been baffled.
You can try:
echo "USERNAME:NEWPASSWORD" | chpasswd
Use GNU passwd stdin flag.
From the man page:
--stdin
This option is used to indicate that passwd should read the new password from standard input, which can be a pipe.
NOTE: Only for root user.
Example
$ adduser foo
$ echo "NewPass" |passwd foo --stdin
Changing password for user foo.
passwd: all authentication tokens updated successfully.
Alternatively you can use expect, this simple code will do the trick:
#!/usr/bin/expect
spawn passwd foo
expect "password:"
send "Xcv15kl\r"
expect "Retype new password:"
send "Xcv15kl\r"
interact
Results
$ ./passwd.xp
spawn passwd foo
Changing password for user foo.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
In addition to the other suggestions, you can also achieve this using a HEREDOC.
In your immediate case, this might look like:
$ /usr/bin/passwd root <<EOF
test
test
EOF
You need echo -e for the newline characters to take affect
you wrote
echo "oldpassword\nnewpasswd123\nnewpasswd123" | passwd user
you should try
echo -e "oldpassword\nnewpasswd123\nnewpasswd123" | passwd user
more than likely, you will not need the oldpassword\n portion of that command, you should just need the two new passwords. Don't forget to use single quotes around exclamation points!
echo -e "new"'!'"passwd123\nnew"'!'"passwd123" | passwd user
You can try :
echo -e "newpasswd123\nnnewpasswd123" | passwd user
Just this
passwd <<EOF
oldpassword
newpassword
newpassword
EOF
Actual output from ubuntu machine (sorry no AIX available to me):
user#host:~$ passwd <<EOF
oldpassword
newpassword
newpassword
EOF
Changing password for user.
(current) UNIX password: Enter new UNIX password: Retype new UNIX password:
passwd: password updated successfully
user#host:~$
This is from : Script to change password on linux servers over ssh
The script below will need to be saved as a file (eg ./passwdWrapper) and made executable (chmod u+x ./passwdWrapper)
#!/usr/bin/expect -f
#wrapper to make passwd(1) be non-interactive
#username is passed as 1st arg, passwd as 2nd
set username [lindex $argv 0]
set password [lindex $argv 1]
set serverid [lindex $argv 2]
set newpassword [lindex $argv 3]
spawn ssh $serverid passwd
expect "assword:"
send "$password\r"
expect "UNIX password:"
send "$password\r"
expect "password:"
send "$newpassword\r"
expect "password:"
send "$newpassword\r"
expect eof
Then you can run ./passwdWrapper $user $password $server $newpassword which will actually change the password.
Note: This requires that you install expect on the machine from which you will be running the command. (sudo apt-get install expect) The script works on CentOS 5/6 and Ubuntu 14.04, but if the prompts in passwd change, you may have to tweak the expect lines.
Here is the script...
#!/bin/bash
echo "Please enter username:"
read username
echo "Please enter the new password:"
read -s password1
echo "Please repeat the new password:"
read -s password2
# Check both passwords match
if [ $password1 != $password2 ]; then
echo "Passwords do not match"
exit
fi
# Does User exist?
id $username &> /dev/null
if [ $? -eq 0 ]; then
echo "$username exists... changing password."
else
echo "$username does not exist - Password could not be updated for $username"; exit
fi
# Change password
echo -e "$password1\n$password1" | passwd $username
Refer the link below as well...
http://www.putorius.net/2013/04/bash-script-to-change-users-password.html
You can try
LINUX
echo password | passwd username --stdin
UNIX
echo username:password | chpasswd -c
If you dont use "-c" argument, you need to change password next time.
If you can use ansible, and set the sudo rights in it, then you can easily use this script. If you're wanting to script something like this, it means you need to do it on more than one system. Therefore, you should try to automate that as well.
For me this worked in a vagrant VM:
sudo /usr/bin/passwd root <<EOF
12345678
12345678
EOF
printf "oldpassword/nnewpassword/nnewpassword" | passwd user
#!/usr/bin/python
import random
import string
import smtplib
import sys
import os
from subprocess import call
import socket
user = sys.argv[1]
receivers = ["%s#domain.com" %user]
'''This will generate a 30 character random password'''
def genrandpwd():
return ''.join(random.SystemRandom().choice(string.ascii_lowercase + string.digits + string.ascii_uppercase + string.punctuation) for _ in range(30))
def change_passwd(user, password):
p = os.popen("/usr/bin/passwd %s" %user, "w")
p.write(password)
p.write("\n")
p.write(password)
p.close()
def chage(user):
agepasswd = call(["/usr/bin/chage", "-d", "0", "%s" %user])
def mailpwd(user, password):
sender = "admin#%s" %socket.gethostname()
subj = "!!!IMPORTANT!!!, Unix password changed for user %s" %user
text = "The password for the %s user has changed, the new password is:\n\n %s \n\n Note: The system will force to change the password upon initial login. Please use the password provided in the mail as your current password and type the password of your choice as the New password" %(user, password)
message = message = 'Subject: %s\n\n%s' % (subj, text)
smtpObj = smtplib.SMTP('mailrelay-server.domain.com')
smtpObj.sendmail(sender, receivers, message)
smtpObj.quit()
def main():
newpwd = genrandpwd()
change_passwd(user, newpwd)
chage(user)
mailpwd(user, newpwd)
if __name__ == "__main__":
main()

How to decrease TCP connect() system call timeout?

In command below I enable file /dev/tcp/10.10.10.1/80 both for reading and writing and associate it with file descriptor 3:
$ time exec 3<>/dev/tcp/10.10.10.1/80
bash: connect: Operation timed out
bash: /dev/tcp/10.10.10.1/80: Operation timed out
real 1m15.151s
user 0m0.000s
sys 0m0.000s
This automatically tries to perform TCP three-way handshake. If 10.10.10.1 is not reachable as in example above, then connect system call tries to connect for 75 seconds. Is this 75 second timeout determined by bash? Or is this system default? Last but not least, is there a way to decrease this timeout value?
It's not possible in Bash without modifying the source as already mentioned, although here is the workaround by using timeout command, e.g.:
$ timeout 1 bash -c "</dev/tcp/stackoverflow.com/80" && echo Port open. || echo Port closed.
Port open.
$ timeout 1 bash -c "</dev/tcp/stackoverflow.com/81" && echo Port open. || echo Port closed.
Port closed.
Using this syntax, the timeout command will kill the process after the given time.
See: timeout --help for more options.
It is determined by TCP. It can be decreased on a per-socket basis by application code.
NB The timeout only takes effect if there is no response at all. If there is a connection refusal, the error occurs immediately.
No: there is no way of changing timeout by using /dev/tcp/
Yes, you could change default timeout for TCP connection in any programming language.
But, bash is not a programming language!
You could have a look into source code (see: Bash Homepage), you may find lib/sh/netopen.c file where you could read in _netopen4 function:
s = socket(AF_INET, (typ == 't') ? SOCK_STREAM : SOCK_DGRAM, 0);
You could read this file carefully, there are no consideration of connection timeout.
Without patching bash sources, there is no way of changing connection timeout by a bash script.
Simple HTTP client using netcat (near pure bash)
There is a little sample HTTP client written in pure bash, but using netcat:
#!/bin/bash
tmpfile=$(mktemp -p $HOME .netbash-XXXXXX)
exec 7> >(nc -w 3 -q 0 stackoverflow.com 80 >$tmpfile)
exec 6<$tmpfile
rm $tmpfile
printf >&7 "GET %s HTTP/1.0\r\nHost: stackoverflow.com\r\n\r\n" \
/questions/24317341/how-to-decrease-tcp-connect-system-call-timeout
timeout=100;
while ! read -t .001 -u 6 status ; do read -t .001 foo;done
echo STATUS: $status
[ "$status" ] && [ -z "${status//HTTP*200 OK*}" ] || exit 1
echo HEADER:
while read -u 6 -a head && [ "${head//$'\r'}" ]; do
printf "%-20s : %s\n" ${head%:} "${head[*]:1}"
done
echo TITLE:
sed '/<title>/s/<[^>]*>//gp;d' <&6
exec 7>&-
exec 6<&-
This could render:
STATUS: HTTP/1.1 200 OK
HEADER:
Cache-Control : private
Content-Type : text/html; charset=utf-8
X-Frame-Options : SAMEORIGIN
X-Request-Guid : 46d55dc9-f7fe-425f-a560-fc49d885a5e5
Content-Length : 91642
Accept-Ranges : bytes
Date : Wed, 19 Oct 2016 13:24:35 GMT
Via : 1.1 varnish
Age : 0
Connection : close
X-Served-By : cache-fra1243-FRA
X-Cache : MISS
X-Cache-Hits : 0
X-Timer : S1476883475.343528,VS0,VE100
X-DNS-Prefetch-Control : off
Set-Cookie : prov=ff1129e3-7de5-9375-58ee-5f739eb73449; domain=.stackoverflow.com; expires=Fri, 01-Jan-2055 00:00:00 GMT; path=/; HttpOnly
TITLE:
bash - How to decrease TCP connect() system call timeout? - Stack Overflow
Some explanations:
We create first a temporary file (under private directory for security reason), bind and delete before using them.
$ tmpfile=$(mktemp -p $HOME .netbash-XXXXXX)
$ exec 7> >(nc -w 3 -q 0 stackoverflow.com 80 >$tmpfile)
$ exec 6<$tmpfile
$ rm $tmpfile
$ ls $tmpfile
ls: cannot access /home/user/.netbash-rKvpZW: No such file or directory
$ ls -l /proc/self/fd
lrwx------ 1 user user 64 Oct 19 15:20 0 -> /dev/pts/1
lrwx------ 1 user user 64 Oct 19 15:20 1 -> /dev/pts/1
lrwx------ 1 user user 64 Oct 19 15:20 2 -> /dev/pts/1
lr-x------ 1 user user 64 Oct 19 15:20 3 -> /proc/30237/fd
lr-x------ 1 user user 64 Oct 19 15:20 6 -> /home/user/.netbash-rKvpZW (deleted)
l-wx------ 1 user user 64 Oct 19 15:20 7 -> pipe:[2097453]
$ echo GET / HTTP/1.0$'\r\n\r' >&7
$ read -u 6 foo
$ echo $foo
HTTP/1.1 500 Domain Not Found
$ exec 7>&-
$ exec 6>&-

pexpect - run script.sh over ssh

I'm having trouble programmatically running a local script over ssh.
I'm unsure if this is a problem with the shell variable substitution on the local host.
When manually running,
ssh monit#server1 'bash -s' < /u02/splunk/splunk/etc/apps/Splunk_TA_nix/bin/cpu.sh
I get the expected output,
CPU pctUser pctNice pctSystem pctIowait pctIdle
all 11.21 0.00 1.50 0.31 86.98
0 0.00 0.00 0.00 0.00 100.00
1 3.00 0.00 1.00 0.00 96.00
....
but I get
bash: /u02/splunk/splunk/etc/apps/Splunk_TA_nix/bin/cpu.sh: No such file or directory
when running the following code,
splunk_bin_dir = '/u02/splunk/splunk/etc/apps/Splunk_TA_nix/bin'
hostname = 'server1'
username = 'monit'
password = 'monit#_'
command = "/usr/bin/ssh %(username)s#%(hostname)s 'bash -s' < %(splunk_bin_dir)s/cpu.sh" % locals()
print command
ssh_new_conn = 'Are you sure you want to continue connecting'
p = pexpect.spawn(command, timeout=360)
# Handles the 3 possible connection outcomes:
# a) Ssh to the remote host for the first time, triggering 'Are you sure you want to continue connecting'
# b) ask you for password
# c) No password is needed at all, because you already have the key.
i = p.expect([ssh_new_conn,'[pP]assword:',pexpect.EOF])
print ' Initial pexpect command output: ', i
if i == 0:
# send 'yes'
p.sendline('yes')
i = p.expect(['[pP]assword:',pexpect.EOF])
print 'sent yes. pexpect command output', i
if i == 0:
# send the password
p.sendline(password)
p.expect(pexpect.EOF)
elif i == 1:
# send the password
p.sendline(password)
p.expect(pexpect.EOF)
elif i == 2:
print "pexpect faced key or connection timeout"
pass
print p.before
These are the printed outputs,
/usr/bin/ssh monit#server1 'bash -s' < /u02/splunk/splunk/etc/apps/Splunk_TA_nix/bin/cpu.sh
Initial pexpect command output: 1
bash: /u02/splunk/splunk/etc/apps/Splunk_TA_nix/bin/cpu.sh: No such file or directory
pexpect is bumping into the [pP]assword line so I guess the password is being correctly passed,
Here the note from pexpect manual:
Remember that Pexpect does NOT interpret shell meta characters such as
redirect, pipe, or wild cards (>, |, or *). This is a common mistake.
If you want to run a command and pipe it through another command then
you must also start a shell.
This is the working line
command = """/bin/bash -c "/usr/bin/ssh %(username)s#%(hostname)s 'bash -s' < %(splunk_bin_dir)s/cpu.sh" """ % locals()

Resources