How can we copy the files recursively to the remote server using expect script or any other script?.
Constraints.
1. We couldn't limit the number of file will be copied.
2. File size may be 1mb or upto 10mb.
I was tried with the following script. But it's only transfer upto 4 or 5 files only. (I need to transfer files, nearly 200 or 300 above)
spawn scp -r /home/test root#example.com:/home/test
sleep 2
expect "password"
send "XXXXXX"
sleep 2
Before the spawn command, add the line
set timeout -1
and replace the 2nd sleep command with
expect eof
Don't forget to add \r when you send your password: send "password\r"
I'd recommend you set up SSH keys -- then you won't be prompted for a password and you don't need the expect script at all.
Related
I'm downloading huge files and have a python downloading script. Since the data are to huge to be stored in their original format, I only download a couple of years, process them, then delete the original before I download the next few years. For every set of years I have a separate download script and I just build a quick bash script to run them
#!/bin/bash
python download_1901-1909.py
## processing, delete
python download_1910-1919.py
## processing, delete
python download_1920-1929.py
## processing, delete
[...]
For the download scipt, I have to type in a password. I know it's is possible to do some automatic password passing with spawn, but I couldn't make it work. Trying the spawn thing does this
#!/usr/bin/expect -f
spawn python download_1901-1909.py
expect "Password:"
send "mypassword"
interact
yields
sh test.sh
test.sh: line 2: spawn: command not found
couldn't read file "Password:": no such file or directory
test.sh: line 4: send: command not found
test.sh: line 5: interact: command not found
How can I pass a password to a python download script?
This answer is trying to solve your question of finishing what you need, not the question of executing a #!/usr/bin/expect, which may be what you are asking.
You may try to use the -c param of expect inline.
For example,
#!/bin/bash
expect -c "spawn python download-1901_1909.py; expect \"assword\"; send \"mypassword\r\n\"; interact"
expect -c "spawn python download-1910_1919.py; expect \"assword\"; send \"mypassword\r\n\"; interact"
# and the remaining, similarly
Note:
Your send part may need to send the \r\n to "simulate" the Enter key of finishing enter the password.
You may use sed within a for to simply the lines.
"assword" is to prevent the case of first letter of that word, "Password" vs "password", for some portability.
I'd like to automate getting diagnostic files from a controller that responds to ssh commands, like e.g.
ssh diag#controller tarred > diags.tgz
Unfortunately, I have to type a password to make the above command go through.
What have I considered to get around that:
using ssh keys: not possible, since I can't login to the controller, it just expects commands and doesn't offer a shell
using ssh-pass package: I don't have admin rights on the machine and can't install packages
using "expect": works to some extent, but the resulting file is corrupted.
Here's the "expect" script I've used:
#!/usr/bin/expect -f
log_user 0
set timeout 300
spawn ssh diag#controller tarred
expect "?assword:"
send "unrealpassword\r"
expect \r\n
log_user 1
expect eof
The script makes sure that only the required output gets stored with the "log_user" commands until eof is encountered.
I've piped this script to a file and that file is corrupted, i.e. it's either too short (because of a timeout?) or too long (?).
Any idea about what goes wrong here.?
This Expect script is a part of my UNIX Bash script
expect -c "
spawn scp yoko#sd.lindeneau.com:\"encryptor *.enc\" .
expect password: { send \"$PASS\r\" }
expect 100%
sleep 1
exit
"
I am trying to copy both 'encryptor' and '*.enc' with this one SCP command. Console tells me it cannot find ' *.enc" '
The syntax for multiple files:
$ scp your_username#remotehost.edu:~/\{foo.txt,bar.txt\} .
I would guess in your case (untested)
scp yoko#sd.lindeneau.com:\\{encryptor, \*.enc\\} .
Not sure it helps but I was looking for a similar objective.
Means: Copying some selected files based on their filenames / extensions but located in different subfolders (same level of subfolders).
this works e.g. copying .csv files from Server:/CommonPath/SpecificPath/
scp -r Server:/CommonPath/\*/\*csv YourRecordingLocation
I even tested more complex "perl like" regular expressions.
Not sure the -r option is still useful.
#!/bin/bash
expect -c "
set timeout 60; # 1 min
spawn scp yoko#sd.lindeneau.com:{encryptor *.enc} .
expect \"password: $\"
send \"mypassword\r\"
expect eof
"
You can increase the timeout if your file copy takes more time to complete. I have used expect eof which will wait till the closure of the scp command. i.e. we are waiting for the End Of File (EOF) of the scp after sending the password.
Until very recently I have been using the following bash script to upload any edited files to my server.
./updatesite.sh
#!/usr/bin/expect -f
spawn rsync -av -e ssh "/(...)/webs" xusernamex#xdomainx.com:/home/webs
expect "password:"
send "xpasswordx\r"
expect "*\r"
expect "\r"
Usually it has worked fine. For some reason it has abruptly stopped working several weeks ago. Here is the output it now provides:
xuserx#xdomainx.com's password:
building file list ... done
As you can see, No files are actually uploaded. But if I paste that exact same command directly into my terminal window without the "spawn," its behavior changes and it uploads the files as usual.
Here is an example:
Squid:~ John$ rsync -av -e ssh "/(...)/webs" xuserx#xdomainx.com:/home/xuserx
xuserx#xdomainx.com's password:
building file list ... done
webs/somefile.txt
sent 878 bytes received 42 bytes 204.44 bytes/sec
total size is 96409 speedup is 104.79
Squid:~ John$
Do you know what could be causing this?
#!/usr/bin/expect -f
# exp_internal 1 ;# uncomment to turn on expect debugging
set timeout -1
spawn rsync -av -e ssh "/(...)/webs" xusernamex#xdomainx.com:/home/webs
expect "password:"
send "xpasswordx\r"
expect eof
It may be a timeout issue, so set the timeout to infinite. Since you don't have to interact with rsync in any way except for the password, just wait for it to finish (expect eof).
I am trying to transfer a file from one server to a remote server with the help of a TCL script.
But my script stops after the message "200 Port set okay" and continues to rum from the below telnet session.
I have checked the destination location, my file is not transferred.
Please suggest what can I do or where I am wrong
#!/usr/bin/tclsh
#!/usr/bin/expect
package require Expect
set p "mm155_005.006.010.200_bt.fw"
#**************************************************************\
FILE TRANSFER TO REMOTE SERVER \
***************************************************************
spawn ftp 10.87.121.26
expect "User (10.87.121.26:(none)):"
send "user\r"
expect "Password:"
send "pswd\r"
expect "ftp>"
send "cd FW\r"
expect "ftp>"
send "ha\r"
expect "ftp>"
send "bi\r"
expect "ftp>"
send "mput \"$p\"\r"
expect "mput $p? "
send "yes\r"
expect "ftp>"
send "ls\r"
#**************************************************************\
RUNNING THE TRANSFERED FILE \
***************************************************************
spawn telnet 10.87.121.26
expect "Login: "
send "user\r"
expect "password: "
send "pswd\r"
expect "*? > "
send "cd FW\r"
expect "*? > "
send "burnboot 30 5.6(10.200)\r"
Output
spawn ftp 10.87.121.26
Connected to 10.87.121.26.
220 VxWorks FTP server (VxWorks VxWorks5.4.2) ready.
Name (10.87.121.26:vkumar): user
331 Password required
Password:
230 User logged in
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> cd FW
250 Changed directory to "C:/FW"
ftp> ha
Hash mark printing on (1024 bytes/hash mark).
ftp> bi
200 Type set to I, binary mode
ftp> mput "mm155_005.006.010.200_bt.fw"
mput mm155_005.006.010.200_bt.fw? yes
200 Port set okay \ I am unable to see hash progress bar after this line
spawn telnet 10.87.121.26
Trying 10.87.121.26...
Connected to 10.87.121.26.
Escape character is '^]'.
Login: user
password:
node84.7.PXM.a > cd FW
node84.7.PXM.a > bash-2.05b$
Instead of rolling your own solution in Expect (I also did this about 9 years ago), use the FTP module from tcllib -- it's already battle-hardened.
http://tcllib.sourceforge.net/doc/ftp.html
The script as reported is unlikely to produce exactly that output; there is nothing from the ls done after the mput. However, if the mput is hanging the most likely problem is that there is a firewall issue; FTP uses multiple sockets to do file transfers (which is why FTP is such a pain when it comes to overall firewall management). In particular, it has a command channel (the socket which you communicate with the FTP server over) and a separate data channel per file (and also with the output of some remote commands, such as ls); that's what that Port set okay is about. This is not firewall-friendly, and it's easy to misconfigure firewalls in this area (especially when there is NAT also in place).
You might (i.e., try this first) want to use passive mode instead, as that reduces the complexity at the firewall level. Try issuing a passive before the mput (just as you currently issue a binary).
Here's a bash script I wrote with a similar function;
You should be able to adapt it to your needs.
Adding a few lines to SSH in and run the script should be quite trivial.
As a side note; why are you using TCL for this?
#!/bin/bash
fileName=`ls /home/user/downloads -t1 | head -n1`
latestFile="/home/user/downloads/$fileName"
echo $latestFile
hostname="HOSTNAME"
username="USER"
password="PASS"
ftp -inv $hostname << EOF
quote USER $username
quote PASS $password
cd transfer/jirabackup
binary
put $latestFile $fileName
quit