My expect script fails to proceed when r-syncing the VirtualBox directories.
Here is the script:
#!/usr/bin/expect -f
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/james/Documents/ username#server:/home/username/BAK/Documents/
expect "password:"
send "thepassword\r"
interact
sleep 1
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/james/AndroidDev/ username#server:/home/username/BAK/AndroidDev/
expect "password:"
send "thepassword\r"
interact
sleep 1
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/james/Android/ username#server:/home/username/BAK/Android/
expect "password:"
send "thepassword\r"
interact
sleep 1
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/james/Videos/ username#server:/home/username/BAK/Videos/ ;
expect "password:"
send "thepassword\r"
interact
sleep 1
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/james/VirtualBox\ VMs/ username#server:/home/username/BAK/"VirtualBox\ VMs/" ;
expect "password:"
send "thepassword\r"
interact
sleep 1
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/james/Pictures/ username#server:/home/username/BAK/Pictures ;
expect "password:"
send "thepassword\r"
interact
sleep 1
spawn scp -v /home/james/.vimrc username#server:/home/username/BAK/ ;
expect "password:"
send "thepassword\r"
interact
sleep 1
spawn scp -v /home/james/.bashrc username#server:/home/username/BAK/ ;
expect "password:"
send "thepassword\r"
interact
sleep 1
spawn scp -v /home/james/.profile username#server:/home/username/BAK/ ;
expect "password:"
send "thepassword\r"
interact
sleep 1
spawn scp -v /home/james/.xsession* username#server:/home/username/BAK/ ;
expect "password:"
send "thepassword\r"
interact
sleep 1
If I change spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/rschostag/VirtualBox\ VMs/ remote#server:/home/directory/BAK/VirtualBox to spawn scp -vr /home/rschostag/VirtualBox\ VMs/ remote#server:/home/directory/BAK/VirtualBox, it works. However, I do not want to do full backup every time.
What is the best way to incrementally backup this directory or otherwise make the rsync command work properly?
Thank you!
As requested, please find the output of expect -d script as follows:
$ sudo expect -d rsync_backup
expect version 5.45
argv[0] = expect argv[1] = -d argv[2] = rsync_backup
set argc 0
set argv0 "rsync_backup"
set argv ""
executing commands from command file rsync_backup
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/james/Documents/ username#server:/home/username/BAK/Documents/
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {5064}
expect: does "" (spawn_id exp6) match glob pattern "password:"? no
username#server password:
expect: does "username#server password: " (spawn_id exp6) match glob pattern "password:"? yes
expect: set expect_out(0,string) "password:"
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) "username#server password:"
send: sending "thepassword\r" to { exp6 }
tty_raw_noecho: was raw = 0 echo = 1
spawn id exp6 sent <\r\n>
spawn id exp6 sent <sending incremental file list\r\n>
sending incremental file list
spawn id exp6 sent <Scripts/rsync.log\r\n>
Scripts/rsync.log
spawn id exp6 sent <\r 2.29K 0% 0.00kB/s 0:00:00 >
2.29K 0% 0.00kB/s 0:00:00 spawn id exp6 sent <\r 5.26M 100% 78.42MB/s 0:00:00 (xfr#1, ir-chk=1561/3024)\r\n>
5.26M 100% 78.42MB/s 0:00:00 (xfr#1, ir-chk=1561/3024)
spawn id exp6 sent <projects/JOBS/jobsearch_production/\r\n>
projects/JOBS/jobsearch_production/
spawn id exp6 sent <projects/JOBS/jobsearch_production/db.sqlite3\r\n>
projects/JOBS/jobsearch_production/db.sqlite3
spawn id exp6 sent <\r 5.41K 0% 0.00kB/s 0:00:00 >
5.41K 0% 0.00kB/s 0:00:00 spawn id exp6 sent <\r 7.35M 100% 73.71MB/s 0:00:00 (xfr#2, ir-chk=1025/6252)\r\n>
7.35M 100% 73.71MB/s 0:00:00 (xfr#2, ir-chk=1025/6252)
spawn id exp6 sent <\r\nNumber of files: 10,529 (reg: 8,419, dir: 2,096, link: 14)>
Number of files: 10,529 (reg: 8,419, dir: 2,096, link: 14)spawn id exp6 sent <\r\nNumber of created files: 0>
Number of created files: 0spawn id exp6 sent <\r\nNumber of regular files transferred: 2>
Number of regular files transferred: 2spawn id exp6 sent <\r\nTotal file size: 1.73G bytes>
Total file size: 1.73G bytesspawn id exp6 sent <\r\nTotal transferred file size: 12.61M bytes>
Total transferred file size: 12.61M bytesspawn id exp6 sent <\r\nLiteral data: 62.70K bytes>
Literal data: 62.70K bytesspawn id exp6 sent <\r\nMatched data: 12.55M bytes>
Matched data: 12.55M bytesspawn id exp6 sent <\r\nFile list size: 0>
File list size: 0spawn id exp6 sent <\r\nFile list generation time: 0.001 seconds>
File list generation time: 0.001 secondsspawn id exp6 sent <\r\nFile list transfer time: 0.000 seconds>
File list transfer time: 0.000 secondsspawn id exp6 sent <\r\nTotal bytes sent: 333.42K>
Total bytes sent: 333.42Kspawn id exp6 sent <\r\nTotal bytes received: 40.66K>
Total bytes received: 40.66Kspawn id exp6 sent <\r\n\r\n>
spawn id exp6 sent <sent 333.42K bytes received 40.66K bytes 32.53K bytes/sec\r\ntotal size is 1.73G speedup is 4,616.35>
sent 333.42K bytes received 40.66K bytes 32.53K bytes/sec
total size is 1.73G speedup is 4,616.35spawn id exp6 sent <\r\n>
interact: received eof from spawn_id exp6
tty_set: raw = 0, echo = 1
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/james/AndroidDev/ username#server:/home/username/BAK/AndroidDev/
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {5069}
expect: does "" (spawn_id exp7) match glob pattern "password:"? no
username#server password:
expect: does "username#server password: " (spawn_id exp7) match glob pattern "password:"? yes
expect: set expect_out(0,string) "password:"
expect: set expect_out(spawn_id) "exp7"
expect: set expect_out(buffer) "username#server password:"
send: sending "thepassword\r" to { exp7 }
tty_raw_noecho: was raw = 0 echo = 1
spawn id exp7 sent <\r\n>
spawn id exp7 sent <sending incremental file list>
sending incremental file listspawn id exp7 sent <\r\n>
spawn id exp7 sent <\r\n>
spawn id exp7 sent <Number of files: 2,758 (reg: 2,273, dir: 485)>
Number of files: 2,758 (reg: 2,273, dir: 485)spawn id exp7 sent <\r\n>
spawn id exp7 sent <Number of created files: 0>
Number of created files: 0spawn id exp7 sent <\r\nNumber of regular files transferred: 0\r\nTotal file size: 1.00G bytes>
Number of regular files transferred: 0
Total file size: 1.00G bytesspawn id exp7 sent <\r\nTotal transferred file size: 0 bytes>
Total transferred file size: 0 bytesspawn id exp7 sent <\r\nLiteral data: 0 bytes\r\nMatched data: 0 bytes>
Literal data: 0 bytes
Matched data: 0 bytesspawn id exp7 sent <\r\nFile list size: 0\r\nFile list generation time: 0.001 seconds>
File list size: 0
File list generation time: 0.001 secondsspawn id exp7 sent <\r\nFile list transfer time: 0.000 seconds>
File list transfer time: 0.000 secondsspawn id exp7 sent <\r\nTotal bytes sent: 74.25K\r\n>
Total bytes sent: 74.25K
spawn id exp7 sent <Total bytes received: 2.43K\r\n\r\nsent 74.25K bytes received 2.43K bytes 30.67K bytes/sec>
Total bytes received: 2.43K
sent 74.25K bytes received 2.43K bytes 30.67K bytes/secspawn id exp7 sent <\r\ntotal size is 1.00G speedup is 13,102.55>
total size is 1.00G speedup is 13,102.55spawn id exp7 sent <\r\n>
interact: received eof from spawn_id exp7
tty_set: raw = 0, echo = 1
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/james/Android/ username#server:/home/username/BAK/Android/
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {5074}
expect: does "" (spawn_id exp8) match glob pattern "password:"? no
username#server password:
expect: does "username#server password: " (spawn_id exp8) match glob pattern "password:"? yes
expect: set expect_out(0,string) "password:"
expect: set expect_out(spawn_id) "exp8"
expect: set expect_out(buffer) "username#server password:"
send: sending "thepassword\r" to { exp8 }
tty_raw_noecho: was raw = 0 echo = 1
spawn id exp8 sent <\r\n>
spawn id exp8 sent <sending incremental file list\r\n>
sending incremental file list
spawn id exp8 sent <\r\nNumber of files: 3 (dir: 3)>
Number of files: 3 (dir: 3)spawn id exp8 sent <\r\n>
spawn id exp8 sent <Number of created files: 0>
Number of created files: 0spawn id exp8 sent <\r\nNumber of regular files transferred: 0>
Number of regular files transferred: 0spawn id exp8 sent <\r\nTotal file size: 0 bytes>
Total file size: 0 bytesspawn id exp8 sent <\r\nTotal transferred file size: 0 bytes>
Total transferred file size: 0 bytesspawn id exp8 sent <\r\n>
spawn id exp8 sent <Literal data: 0 bytes\r\nMatched data: 0 bytes\r\nFile list size: 0>
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0spawn id exp8 sent <\r\n>
spawn id exp8 sent <File list generation time: 0.001 seconds\r\nFile list transfer time: 0.000 seconds\r\nTotal bytes sent: 83>
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 83spawn id exp8 sent <\r\nTotal bytes received: 21>
Total bytes received: 21spawn id exp8 sent <\r\n\r\n>
spawn id exp8 sent <sent 83 bytes received 21 bytes 69.33 bytes/sec\r\ntotal size is 0 speedup is 0.00>
sent 83 bytes received 21 bytes 69.33 bytes/sec
total size is 0 speedup is 0.00spawn id exp8 sent <\r\n>
interact: received eof from spawn_id exp8
tty_set: raw = 0, echo = 1
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/james/Videos/ username#server:/home/username/BAK/Videos/
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {5079}
expect: does "" (spawn_id exp9) match glob pattern "password:"? no
username#server password:
expect: does "username#server password: " (spawn_id exp9) match glob pattern "password:"? yes
expect: set expect_out(0,string) "password:"
expect: set expect_out(spawn_id) "exp9"
expect: set expect_out(buffer) "username#server password:"
send: sending "thepassword\r" to { exp9 }
tty_raw_noecho: was raw = 0 echo = 1
spawn id exp9 sent <\r\n>
spawn id exp9 sent <sending incremental file list\r\n>
sending incremental file list
spawn id exp9 sent <\r\nNumber of files: 30 (reg: 29, dir: 1)\r\nNumber of created files: 0>
Number of files: 30 (reg: 29, dir: 1)
Number of created files: 0spawn id exp9 sent <\r\nNumber of regular files transferred: 0>
Number of regular files transferred: 0spawn id exp9 sent <\r\nTotal file size: 1.35G bytes>
Total file size: 1.35G bytesspawn id exp9 sent <\r\nTotal transferred file size: 0 bytes>
Total transferred file size: 0 bytesspawn id exp9 sent <\r\nLiteral data: 0 bytes>
Literal data: 0 bytesspawn id exp9 sent <\r\nMatched data: 0 bytes\r\nFile list size: 0>
Matched data: 0 bytes
File list size: 0spawn id exp9 sent <\r\nFile list generation time: 0.001 seconds>
File list generation time: 0.001 secondsspawn id exp9 sent <\r\nFile list transfer time: 0.000 seconds>
File list transfer time: 0.000 secondsspawn id exp9 sent <\r\nTotal bytes sent: 587\r\n>
Total bytes sent: 587
spawn id exp9 sent <Total bytes received: 11\r\n\r\nsent 587 bytes received 11 bytes 398.67 bytes/sec\r\n>
Total bytes received: 11
sent 587 bytes received 11 bytes 398.67 bytes/sec
spawn id exp9 sent <total size is 1.35G speedup is 2,263,533.89\r\n>
total size is 1.35G speedup is 2,263,533.89
interact: received eof from spawn_id exp9
tty_set: raw = 0, echo = 1
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/james/VirtualBox VMs/ username#server:/home/username/BAK/"VirtualBox VMs/"
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {5084}
expect: does "" (spawn_id exp10) match glob pattern "password:"? no
username#server password:
expect: does "username#server password: " (spawn_id exp10) match glob pattern "password:"? yes
expect: set expect_out(0,string) "password:"
expect: set expect_out(spawn_id) "exp10"
expect: set expect_out(buffer) "username#server password:"
send: sending "thepassword\r" to { exp10 }
tty_raw_noecho: was raw = 0 echo = 1
spawn id exp10 sent <\r\n>
spawn id exp10 sent <sending incremental file list\r\n>
sending incremental file list
This is where it hangs and does not provide an error message that I can identify.
I moved the remote VirtualBox directory and started the backup again, and it completed finally after running the backup for a few days. The total size of the backup was 71 GB. I ran it with expect -d which prints a very verbose output in the terminal, so that was helpful alone, because the script without -d may have looked like it was hanging only because it was not printing anything in the output. So, there was never anything wrong, I believe, other than no progress indication, which made me believe that it may not be working.
Thank you for advising me about the expect -d option.
#glennjackman answered my question in a comment but did not answer the question formally.
Here is the new script that works:
#!/usr/bin/expect -f
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/ryan/Documents/ username#server:/home/username/BAK/Documents/
expect "password:"
send "thepassword\r"
expect eof
sleep 1
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/ryan/AndroidDev/ username#server:/home/username/BAK/AndroidDev/
expect "password:"
send "thepassword\r"
expect eof
sleep 1
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/ryan/Android/ username#server:/home/username/BAK/Android/
expect "password:"
send "thepassword\r"
expect eof
sleep 1
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/ryan/Videos/ username#server:/home/username/BAK/Videos/ ;
expect "password:"
send "thepassword\r"
expect eof
sleep 1
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/ryan/VirtualBox\ VMs/ username#server:/home/username/BAK/"VirtualBox\ VMs/" ;
expect "password:"
send "thepassword\r"
expect eof
sleep 1
spawn rsync -vruEtl --progress -h --log-file=rsync.log --fuzzy --stats /home/ryan/Pictures/ username#server:/home/username/BAK/Pictures ;
expect "password:"
send "thepassword\r"
expect eof
sleep 1
spawn scp -v /home/ryan/.vimrc username#server:/home/username/BAK/ ;
expect "password:"
send "thepassword\r"
expect eof
sleep 1
spawn scp -v /home/ryan/.bashrc username#server:/home/username/BAK/ ;
expect "password:"
send "thepassword\r"
expect eof
sleep 1
spawn scp -v /home/ryan/.profile username#server:/home/username/BAK/ ;
expect "password:"
send "thepassword\r"
expect eof
sleep 1
spawn scp -v /home/ryan/.xsession* username#server:/home/username/BAK/ ;
expect "password:"
send "thepassword\r"
expect eof
sleep 1
Related
I'm trying to create a script that retreives a secret from a keepass database.
The script uses Expect to get secret via the keepass cli.
Here after my script:
#!/bin/bash
set +x
entry="$1"
keepass_password="azerty"
keepass_db="canadaplatfomsecretsdb.kdbx"
keepass_entry_dir="canadaplatfomsecretsdb/k8s-enabling"
kubesecretname="artifactoryregcred"
kubenamespace="dev"
echo -e "\n"
echo "Connecting to keepass Database..."
function get_creds {
expect -d <<EOF
set timeout 10
match_max 100000000
spawn kpcli
expect "kpcli:/>"
send "open $keepass_db\n"
expect "password:"
send "$keepass_password\n"
expect ">"
send "cd $keepass_entry_dir\n"
expect "k8s-enabling>"
send "show -f $entry\n"
expect ">"
EOF
}
credentials=$(get_creds)
Here after the output form the gitlab-ci job logs
$ bash getcredsfromkeepass.sh ${entry}
Connecting to keepass Database...
expect version 5.45.4
argv[0] = expect argv[1] = -d
set argc 0
set argv0 "expect"
set argv ""
executing commands from command file
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {3855}
expect: does "" (spawn_id exp3) match glob pattern "kpcli:/>"? no
expect: does "No usable Term::ReadLine::* modules found.\r\nThis list was tried:\r\n * Term::ReadLine::Gnu\r\n * Term::ReadLine::Perl\r\n * Term::ReadLine::Perl5\r\nFor more information, read the documentation: perldoc kpcli\r\n" (spawn_id exp3) match glob pattern "kpcli:/>"? no
expect: read eof
expect: set expect_out(spawn_id) "exp3"
What is causing the problem and how can it be fixed?
#!/usr/bin/expect
spawn ssh admin#10.10.10.10 -o StrictHostKeyChecking=no
expect "5555:"
send "$password\n"
expect "% "
send "exit\n"
No matter what value wrote on expect "5555:" this command still working and I have ssh connection.
You should set a timeout and an action upon the timeout:
#!/usr/bin/expect
set timeout 5
spawn ssh admin#10.10.10.10 -o StrictHostKeyChecking=no
expect {
"5555:" { }
timeout { exit 2 }
}
send "$password\n"
expect "% "
send "exit\n"
Some extra info: the default timeout is 10 seconds. While expecting something, if the pattern has not been seen after $timeout seconds, the expect command without a "timeout" pattern simply returns and the script continues.
I'm trying to write a shell function that spawns a ssh process and authentificates with a password. Then, I'd like to use the spawned process to do further stuff with expect.
Here's the function I have so far:
ssh_cmd() {
ssh $USER#$HOST 2>&1 | expect -c "
log_user 0
set timeout 5
expect password: {
send \"$PASS\n\"
sleep 2
log_user 1
}
"
}
And then I'd like to use given function in other places to interact with the ssh process like this:
ssh_cmd | expect -c "
expect '#' {
send pwd\n
send exit\n
}
expect eof
"
However, running the ssh_cmd function with -d option for expect I get the following result:
expect version 5.45.4
expect: does "" (spawn_id exp0) match glob pattern "password:"? no
ubnt#ui1's password: expect: timed out
From what I understand, the output of ssh does not get piped correctly. I know the common way to do this would be to use spawn, but that would mean the process would get killed after expect exits and I could not have a generic function that authentificates ssh sessions and keeps the process alive for further usage.
What you're designing won't work. Expect needs the process to be spawned from within the expect interpreter using the spawn command. Passing the command's stdout into expect is insufficient.
You could try this:
ssh_cmd() {
# a default bit of code if user does not provide one.
# you probably want some checking and emit an error message.
local user_code=${1:-set timeout 1; send "exit\r"; expect eof}
expect -c "
log_user 0
set timeout 5
spawn ssh -l $USER $HOST
expect password: {
send \"$PASS\n\"
sleep 2
log_user 1
}
$user_code
"
}
and then invoke it like:
ssh_cmd '
expect "#" {
send pwd\r
send exit\r
}
expect eof
'
Note that single quotes have no special meaning in expect, they are just plain characters: you probably don't want to expect the prompt to be the 3 character pattern '#'
The problem is the command send "xxxxx" just does not send. The command seem to be completely ignored.
In the script below I am able to log in to the server but no more send commands are transmitted to the server.
At the end there is an example of the output.
The timeout traps were for debugging.
#!/usr/bin/expect -f
## getver_t.sh - Use telnet to get ver from Cisco devices
## Could be asked for login ID or just pasword
set pass {Password1 Pass2}
set index 0
set timeout 5
set host [lindex $argv 0 ]
spawn ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null admin#$host
expect {
"assword:" {
send_user "\n--- sending [lindex $pass $index] ---\n"
send "[lindex $pass $index]\r"
incr index
exp_continue
}
"\>" {
send_user "\n--- I see a prompt ---\n"
send -- "ter len 0\r"
sleep 3
send -- "show inv\r"
sleep 3
send -- "show ver\r"
send_user "\n--- Time to end ---\n"
sleep 3
send -- "exit\r"
sleep 3
exit
}
timeout {
send -- "\r"
set timeout 5
send_user "\n--- TIMED OUT 1 ---\n"
send -- "ter len 0\r"
sleep 3
send -- "show inv\r"
sleep 3
send -- "show ver\r"
sleep 3
send -- "exit\r"
sleep 3
exit
}
}
send_user "\n--- out side land ---\n"
send "\r"
expect {
timeout {
send_user "\n--- TIMED OUT 2 ---\n"
}
}
This is what I see running the script.
$ ./getver_s.sh server1
spawn ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null admin#server1
Warning: Permanently added 'server1,10.1.1.3' (RSA) to the list of known hosts.
Password:
--- sending Password1 ---
Password:
--- sending Pass2 ---
------------------------------------------------------------
Welcome to server1
------------------------------------------------------------
server1>
--- I see a prompt ---
--- Time to end ---
$
I see the "--- I see a prompt ---" and "--- Time to end ---" messages
but the send commands between the send_user commands are not sent to the server.
I have rewritten this script changed the logic and run it off different versions of servers (Debian7 & 8) but the result is always the same.
Why are those send commands not being sent?
Added expect -d output
Login Banner
-------------- CUT ------------------
Password:"
--- sending Password1 ---
send: sending "Password1\r" to { exp7 }
expect: continuing expect
expect: does " " (spawn_id exp7) match glob pattern "assword:"? no
">"? no
expect: does " \r\n" (spawn_id exp7) match glob pattern "assword:"? no
">"? no
Password:
expect: does " \r\nPassword: " (spawn_id exp7) match glob pattern "assword:"? yes
expect: set expect_out(0,string) "assword:"
expect: set expect_out(spawn_id) "exp7"
expect: set expect_out(buffer) " \r\nPassword:"
--- sending Pass2 ---
send: sending "Pass2\r" to { exp7 }
expect: continuing expect
expect: does " " (spawn_id exp7) match glob pattern "assword:"? no
">"? no
expect: does " \r\n" (spawn_id exp7) match glob pattern "assword:"? no
">"? no
-------------- CUT ------------------
motd
-------------- CUT ------------------
-\r\nServer1>"
--- I see a prompt ---
send: sending "ter len 0\r" to { exp7 }
send: sending "show inv\r" to { exp7 }
send: sending "show ver\r" to { exp7 }
send: sending "exit\r" to { exp7 }
Server1$
I need some help with an expect script and rsync.
When I run rsync in command line like this it works fine:
rsync -avz root#10.33.122.22:'$(find /cluster/storage/nobackup/coremw/var/log/saflog/FaultManagementLog/alarm/ -ctime -1)' /home/imstest/shared/Generate/ReportingT3/tmp/
But when I use my expect script that looks like this I get an error:
#!/usr/bin/expect -f
set host_ip [lindex $argv 0 ]
set user [lindex $argv 1 ]
set password [lindex $argv 2]
set scp_remote_directory [lindex $argv 3 ]
#set scp_remote_filename [lindex $argv 4]
set local_directory [lindex $argv 4]
set force_conservative 1 ;# set to 1 to force conservative mode even if
;# script wasn't run conservatively originally
if {$force_conservative} {
set send_slow {1 .1}
proc send {ignore arg} {
sleep .1
exp_send -s -- $arg
}
}
set timeout -1
#spawn scp $user#$host_ip:$scp_remote_directory/$scp_remote_filename $local_directory/
spawn rsync -azv $user#$host_ip:\'\$(find $scp_remote_directory -ctime -1)\' $local_directory/
#spawn scp $user#$host_ip:$scp_remote_directory/CscfHealthCheckReport_$TagID_$phase* $local_directory/
match_max 100000
expect {
-exact "Are you sure you want to continue connecting (yes/no)? " {
send -- "yes\r"
exp_continue
}
-exact "${user}#${host_ip}'s password: " {
send -- "$password\r"
}
-exact "Password: " {
send -- "$password\r"
}
}
expect eof
The script name is td_general_scp.exp and i run it like this:
./td_general_scp.exp 10.33.122.22 root rootroot /cluster/storage/no-backup/coremw/var/log/saflog/FaultManagementLog/alarm/Fm*.log /home/imstest/shared/Generate/ReportingT3/tmp
But then i get this error:
spawn rsync -azv root#10.33.122.22:'$(find /cluster/storage/no-backup/coremw/var/log/saflog/FaultManagementLog/alarm/Fm*.log -ctime -1)' /home/imstest/shared/Generate/ReportingT3/tmp/
Unexpected local arg: /cluster/storage/no-backup/coremw/var/log/saflog/FaultManagementLog/alarm/Fm*.log
If arg is a remote file/dir, prefix it with a colon (:).
rsync error: syntax or usage error (code 1) at main.c(1362) [Receiver=3.1.0]
expect: spawn id exp6 not open
while executing
"expect eof"
(file "./td_general_scp.exp" line 36)
But the spawn printout looks exactly the same as when i run it in command line?
What am i doing wrong?
Thanks for the help!
I addded the comments from #Shubhangi Pardeshi but now i get this error:
spawn rsync -azv root#10.33.122.22:'$(find /cluster/storage/no-backup/coremw/var/log/saflog/FaultManagementLog/alarm/Fm*.log -ctime -1 )' /home/imstest/shared/Generate/ReportingT3/tmp/
Password:
receiving incremental file list
rsync: change_dir "/root//$(find /cluster/storage/no-backup/coremw/var/log/saflog/FaultManagementLog/alarm" failed: No such file or directory (2)
The scripts looks like this now:
#!/usr/bin/expect -f
set host_ip [lindex $argv 0 ]
set user [lindex $argv 1 ]
set password [lindex $argv 2]
set scp_remote_directory [lindex $argv 3 ]
set local_directory [lindex $argv 4]
set force_conservative 1 ;# set to 1 to force conservative mode even if
;# script wasn't run conservatively originally
if {$force_conservative} {
set send_slow {1 .1}
proc send {ignore arg} {
sleep .1
exp_send -s -- $arg
}
}
set timeout -1
#spawn scp $user#$host_ip:$scp_remote_directory/$scp_remote_filename $local_directory/
spawn rsync -azv $user#$host_ip:\'\$(find\ $scp_remote_directory\ -ctime\ -1\ )\' $local_directory/
#spawn scp $user#$host_ip:$scp_remote_directory/CscfHealthCheckReport_$TagID_$phase* $local_directory/
match_max 100000
expect {
-exact "Are you sure you want to continue connecting (yes/no)? " {
send -- "yes\r"
exp_continue
}
-exact "${user}#${host_ip}'s password: " {
send -- "$password\r"
}
-exact "Password: " {
send -- "$password\r"
}
}
expect eof
Here is the printout i gert when using exp_internal 1 after channging to #Shubhangi Pardeshi suggestion.
spawn rsync -azv root#10.33.122.22:'$(find /cluster/storage/no-backup/coremw/var/log/saflog/FaultManagementLog/alarm/Fm*.log -ctime -1 )' /home/imstest/shared/Generate/ReportingT3/tmp/
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {115454}
expect: does "" (spawn_id exp6) match exact string "Are you sure you want to continue connecting (yes/no)? "? no
"root#10.33.122.22's password: "? no
"Password: "? no
Password:
expect: does "Password: " (spawn_id exp6) match exact string "Are you sure you want to continue connecting (yes/no)? "? no
"root#10.33.122.22's password: "? no
"Password: "? yes
expect: set expect_out(0,string) "Password: "
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) "Password: "
send: sending "rootroot\r" to { exp6 }
receiving incremental file list
rsync: change_dir "/root//$(find /cluster/storage/no-backup/coremw/var/log/saflog/FaultManagementLog/alarm" failed: No such file or directory (2)
sent 8 bytes received 159 bytes 111.33 bytes/sec
total size is 0 speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1655) [Receiver=3.1.0]
rsync: [Receiver] write error: Broken pipe (32)
expect: read eof
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) "\r\nreceiving incremental file list\r\nrsync: change_dir "/root//$(find /cluster/storage/no-backup/coremw/var/log/saflog/FaultManagementLog/alarm" failed: No such file or directory (2)\r\n\r\nsent 8 bytes received 159 bytes 111.33 bytes/sec\r\ntotal size is 0 speedup is 0.00\r\nrsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1655) [Receiver=3.1.0]\r\nrsync: [Receiver] write error: Broken pipe (32)\r\n"
imstest#TCA-Hubba3:~/shared/Generate/ReportingT3/bin$
I missed to chang to ` as the suggestion but now i have the exact row as suggested:
spawn rsync -azv $user#$host_ip:\`\$(find\ $scp_remote_directory\ -ctime\ -1\ )\` $local_directory/
But now I get a permission denied output? I logged in as root it can't be a permission problem. The error output looks like this.
spawn rsync -zv root#10.33.122.22:`$(find /cluster/storage/no-backup/coremw/var/log/saflog/FaultManagementLog/alarm/Fm*.log -ctime -1 )` /home/imstest/shared/Generate/ReportingT3/tmp/
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {118075}
expect: does "" (spawn_id exp6) match exact string "Are you sure you want to continue connecting (yes/no)? "? no
"root#10.33.122.22's password: "? no
"Password: "? no
Password:
expect: does "Password: " (spawn_id exp6) match exact string "Are you sure you want to continue connecting (yes/no)? "? no
"root#10.33.122.22's password: "? no
"Password: "? yes
expect: set expect_out(0,string) "Password: "
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) "Password: "
send: sending "rootroot\r" to { exp6 }
bash: /cluster/storage/no-backup/coremw/var/log/saflog/FaultManagementLog/alarm/FmAlarmLog_20160422_152507.log: Permission denied
sent 8 bytes received 5 bytes 8.67 bytes/sec
total size is 0 speedup is 0.00
expect: read eof
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) "\r\nbash: /cluster/storage/no-backup/coremw/var/log/saflog/FaultManagementLog/alarm/FmAlarmLog_20160422_152507.log: Permission denied\r\n\r\nsent 8 bytes received 5 bytes 8.67 bytes/sec\r\ntotal size is 0 speedup is 0.00\r\n"
imstest#TCA-Hubba3:~/shared/Generate/ReportingT3/bin$
Below two changes in rsync command in expect script will resolve issue
Escape space in find command- $(find $scp_remote_directory -ctime -1)
replace ' by `
Hence, spawn statement in expect script would look like
spawn rsync -azv $user#$host_ip:\`\$(find\ $scp_remote_directory\ -ctime\ -1\ )\` $local_directory/