please advice why
spawn scp $FILE1 $FILE2 $LOGIN#$IP:/tmp
in my expect script copy only FILE1 and not copy FILE2 ?
I try to to transfer both files by scp as
scp file1.csv file2.crt 192.8.200.1:/tmp
without expect and they transferred successfully to /tmp
so why VIA expect the only file that copied is FILE1 ??
what wrong in my syntax ?
example of my expect script
#!/usr/bin/expect -f
set FILE1 file1.csv
set FILE2 file2.crt
set multiPrompt {[#>$]}
spawn scp $FILE1 $FILE2 $LOGIN#$IP:/tmp
expect {
")?" { send "yes\r" ; exp_continue }
word: {send $PASS\r}
}
I also try this:
spawn scp "$FILE1 $FILE2" $LOGIN#$IP:/tmp
OR
spawn scp '$FILE1 $FILE2' $LOGIN#$IP:/tmp
but I get the same problem
PLEASE HELP!!!!!!!!
You can workaround it by using array and foreach in expect script, like:
#!/usr/bin/expect
set files { file1.csv file2.crt }
foreach file $files {
puts "Let's scp $file"
send "scp $file $LOGIN#$IP:/tmp"
}
Related
I have a problem with my bash script.
It connects to my sftp server.
Gets the list of files to download.
Should download the files. But it doesnt do this. I can see the commands. if I write the commands manually, it works.
You can see the script and the log files here:
#!/bin/bash
LOCALDIR="/data/IMPORT/$(date +%Y%m%d)"
REMOTEDIR="/EXPORT"
FILELIST="$LOCALDIR/filelist.txt"
FILELIST2="$LOCALDIR/filelist2.txt"
SFTP="sftp -P 1234 -i /var/xxxxxx.pem -oStrictHostKeyChecking=no user#xxxxx.xxxx"
PASSPHRASE="xxxxxxxxxxxxxxxx"
mkdir -p $LOCALDIR
rm $FILELIST
rm $FILELIST2
allfilenames=()
function readFileList {
expect -c "
spawn $SFTP
expect \"assphrase\"
send \"$PASSPHRASE\r\"
expect \"sftp>\"
send \"lcd $LOCALDIR\r\"
send \"ls -l $REMOTEDIR/*\r\"
expect \"sftp>\"
send \"exit\r\"
interact " > $FILELIST
}
function getFiles {
myfilenames=("$#")
expect -c "
spawn $SFTP
expect \"assphrase\"
send \"$PASSPHRASE\r\"
expect \"sftp>\"
send \"lcd $LOCALDIR\r\"
expect \"sftp>\"
send \"cd $REMOTEDIR\r\"
expect \"sftp>\"
" >> $FILELIST2
for filepath in "${myfilenames[#]}"
do
file="$(basename -- $filepath)"
expect -c "
send \"get -P $file\r\n\"
sleep 3
expect \"sftp>\"
" >> $FILELIST2
done
expect -c "
send \"exit\r\"
" >> $FILELIST2
}
readFileList
c=0
if [[ -f "$FILELIST" ]]; then
while read line; do
filename=$(echo $line | awk '{ print $9 }')
if [[ "$filename" =~ ^$REMOTEDIR ]] ; then
allfilenames+=($filename)
fi
done < $FILELIST
fi
getFiles "${allfilenames[#]}"
filelist.txt looks like:
spawn sftp -P 1234 -i /var/xxxxxx.pem -oStrictHostKeyChecking=no user#xxxxx.xxxx
Enter passphrase for key '/var/xxxxxx.pem':
Connected to xxxxx.xxxx.
sftp> lcd /data/IMPORT/20200401
sftp> ls -l /EXPORT/*
-rw-r--r-- 0 1000472 1000472 3681 Mar 31 22:31 /EXPORT/file1.txt
-rw-r--r-- 0 1000472 1000472 14537 Mar 31 22:34 /EXPORT/file2.txt
-rw-r--r-- 0 1000472 1000472 5932 Mar 31 22:34 /EXPORT/file3.txt
sftp> exit
filelist2.txt looks like:
spawn sftp -P 1234 -i /var/xxxxxx.pem -oStrictHostKeyChecking=no user#xxxxx.xxxx
Enter passphrase for key '/var/xxxxxx.pem':
Connected to xxxxx.xxxx.
sftp> lcd /data/IMPORT/20200401
sftp> cd /EXPORT
sftp> get -P file1.txt
get -P file2.txt
get -P file3.txt
exit
expect -c 'spawn sftp ...'
expect -c 'send "get file\r" '
Here when the first expect -c completes, the SFTP connection will be closed so the second expect -c would not work. You have to use one single expect -c for one SFTP session.
It's like when you manually sftp to the server, you cannot temporarily go back to Bash and come back to sftp later.
I have an array in my bash script:
my_file=("a.txt" "b.txt" "c.txt" "d.txt" "e.txt")
Now in the same file I want to use Expect and make a looping to get some files in sftp. This is my code
/usr/bin/expect <<EOF
set timeout -1
array set param ${!my_file[#]}
spawn sftp $sftp_option $user#$host
expect "Password:"
send "$pswd\r"
expect "sftp>"
for arg in $param; do
send "mget $arg*\r"
expect "sftp>"
done
send "bye\r"
EOF
With that code I can't make a loop with that array above. And I got error like this:
wrong # args: should be "array set arrayName list"
while executing
"array set param 0 1"
Full Code:
#!/bin/bash
export PATH=$PATH:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin;
homeDir=/home/jlo_apps/daemon/jlo_ex/collection/wssh;
filePID=$homeDir/rsync_rfs_bbni.pid;
trap "rm $filePID; exit" SIGHUP SIGINT SIGTERM;
if [ -e $filePID ]; then
datenow=`date`;
echo "[ Rsync ] There are rsync processes still running...";
echo "========================= $datenow =========================";
exit 1;
else
echo $$ > $filePID;
fi
. $homeDir/get_rfs_bni.conf;
strconn="$dbuser/$dbpass#$dbhost:$dbport/$dbsid";
profile=`$sqlldr_path -s $strconn <<< "select host|| '|' ||username|| '|' ||password|| '|' ||port|| '|' ||outdir from p_epdp where bank='BBNI';" | tail -2 | head -1`;
mapfile mid < <($sqlldr_path -s $strconn <<< "select distinct(substr(mid, 0, 13)) from p_mid where kode_bank = 'BBNI';" | tail -3 | head -2);
host=$(echo $profile | cut -d"|" -f1);
user=$(echo $profile | cut -d"|" -f2);
pswd=$(echo $profile | cut -d"|" -f3);
port=$(echo $profile | cut -d"|" -f4);
outdir=$(echo $profile | cut -d"|" -f5);
/usr/bin/expect <<EOF
set timeout -1
spawn sftp $sftp_option $user#$host
expect "Password:"
send "$pswd\r"
expect "sftp>"
send "cd $outdir\r"
expect "sftp>"
send "lcd $rfshome\r"
expect "sftp>"
foreach arg {${mid[#]}} {
send "mget $arg*\r"
expect "sftp>"
}
send "bye\r"
EOF
rm $filePID;
sleep 10;
datenow=`date`;
echo "========================= $datenow =========================";
Is there any solution for this problem without separate file between bash and Expect?
The correct syntax to interpolate the array's values would be
array set param ${my_file[#]}
without a !, but this produces
array set param a.txt b.txt c.txt d.txt e.txt
However, the Expect syntax for creating an array looks like
array set param {one one.txt two two.txt}
with alternating keys and values (more like an associative array than just a list of values). But then you still can't use a shell for loop inside the Expect script; the language uses a completely different syntax (Expect is based on TCL, not shell script).
Probably you are looking for something like
/usr/bin/expect <<EOF
set timeout -1
spawn sftp $sftp_option $user#$host
expect "Password:"
send "$pswd\r"
expect "sftp>"
foreach arg {${my_file[#]}} {
send "mget $arg*\r"
expect "sftp>"
}
send "bye\r"
EOF
where I cribbed the proper TCL loop syntax from Pass bash array to expect script
It's a bit awkward looking, but you can do this:
$ my_file=("a.txt" "b.txt" "c.txt" "d.txt" "e.txt")
$ expect -f - -- "${my_file[#]}" <<'END'
foreach arg $argv {puts $arg}
END
a.txt
b.txt
c.txt
d.txt
e.txt
The -f - tells expect to use stdin for the script file.
Then -- ends the command line options, and the remaining arguments go into the argv list.
Don't forget to use a quoted heredoc, like I demonstrate. Otherwise the shell will expand the expect variables.
I want to make my script password protected. If I use this code it works:
ACTUAL="sam123"
read -s -p "Password: " enteredpass
I also want to protect the script from being read with cat and vi. I tried to use vim -x <script> to encrypt it but then it won't allow me to run it.
I am using a generic user and haven't gotten anywhere.
You can't do this securely without your sysadmin's help, but you can do something sorta-kinda-maybe-not-really-adequate without it.
So, let's say you create your script like so:
cat >myscript <<EOF
echo "Doing something super secret here"
EOF
...but you don't want anyone who doesn't know the password to run it, even if they're using a shared account. You can do this by encrypting it:
gpg -ac <myscript >myscript.asc
...and then embedding that plaintext into a script:
#!/usr/bin/env bash
{ gpg -d | bash -s "$#"; } <<'EOF'
-----BEGIN PGP MESSAGE-----
jA0EBwMCBogTuO9LcuZg0lsB2wqrsPU8Bw2DRzAZr+hiecYTOe//ajXfcjPI4G6c
P3anEYb0N4ng6gsOhKqOYpZU9JzVVkxeL73CD1GSpcQS46YlKWJI8FKcPckR6BE+
7vqkcPWwcS7oy4H2
=gmFu
-----END PGP MESSAGE-----
EOF
That said, other users in the shared account can still collect your password if they connect to and trace your process while it's running -- running strace on the copy of bash -s will show the text being fed into its stdin. In general, you shouldn't rely on shared accounts for anything that needs to remain confidential.
Late answer for posterity, how about using openssl? here's my scriptencrypt.sh
It generates a new .sh file that requires a password
#!/bin/bash
if [ -z "$1" ]; then echo "usage: $(basename $0) script"; exit 1; fi
script=$(cat "$1")
checksum="$(echo "$script" | md5sum | awk '{ print $1 }')"
extension=$([[ "$(basename $1)" =~ .\.. ]] && echo ".${1##*.}" || echo "")
cat << EOF > "${1%.*}.enc${extension}"
#!/bin/bash
read -r -d '' encrypted_script << EOF2
$(openssl aes-256-cbc -a -salt -in /dev/stdin -out /dev/stdout <<< "${script}")
EOF2
read -s -p "Enter script password: " password
echo
unencrypted_script=\$(openssl aes-256-cbc -d -a -salt -in /dev/stdin -out /dev/stdout <<< "\${encrypted_script}" -pass pass:"\${password}" 2>/dev/null | tr -d '\000')
clear
checksum="\$(echo "\$unencrypted_script" | md5sum | awk '{ print \$1 }')"
if [ "\${checksum}" = "${checksum}" ]; then
eval "\${unencrypted_script}"
exit 0
else
echo "Wrong password inserted"
exit 1
fi
EOF
I have an expect script that grabs a bunch of configs and outputs them to separate files. The files though have a bunch of ^H's though I'd like to clean out. I used the CtrlV+CtrlH to put it in the expect script, but when it runs it just does it. The sed works fine in bash, but fails in expect. How can I delete the ^H from the files?
foreach host $ip {
set output [ open "$host" w ]
set timeout 2
spawn ssh -oKexAlgorithms=+diffie-hellman-group1-sha1 -o "StrictHostKeyChecking no" -oHostKeyAlgorithms=+ssh-dss $username#$host
expect {
eof {wait; spawn telnet $host;
expect "ame:";
send "$username\r"}
}
expect "word:"
send "$password\r"
expect "#"
send "$command\r"
sleep 2
expect {
"ore--" { send -- " "; puts $output $expect_out(buffer); exp_continue}
"#" {send -- "exit\r"}
}
puts $output $expect_out(buffer)
close $output
close
}
foreach host $ip {
spawn bash -c "sed -i 's/^H//gi; s/--More--//g' ./$host"
}
close
Here's what it looks like when it runs:
spawn bash -c sed -i 's//gi; s/--More--//g' ./192.168.50.1
spawn bash -c sed -i 's//gi; s/--More--//g' ./192.168.51.1
spawn bash -c sed -i 's//gi; s/--More--//g' ./192.168.52.1
Assuming an ASCII-based locale, this should work:
tr -d '\010' < file > file.new
\010 is octal 8, which is the ASCII equivalent of backspace.
For calling out to sed, since you don't need to interact with the process, you can use exec instead:
foreach host $ip {
exec sed -i {s/^H//gi; s/--More--//g} ./$host
}
Using Tcl's version of single quotes, which are {}
I have a CSV which has three columns: object-ID, image-url1, image-url2. I'd like to be able to run a bash script that does the following for each row in the CSV:
create a new folder using 'object-ID' as the folder name
download both images into that folder
repeat for each row
I've got this code but it needs some help!
IFS=$'\n';
for file in `cat <filename.csv>`; do
echo "Creating folder $object-ID";
mkdir $object-ID
echo "Downloading image 1";
wget $image-url1
echo "Downloading image 2";
wget $image-url2
done
Try this:
while IFS=, read objid url1 url2;
do
echo "Creating folder $objid"
mkdir -p "$objid"
# Run in a subshell
(
cd "$objid"
echo "Downloading image 1"
wget "$url1"
echo "Downloading image 2"
wget "$url2";
)
done < myfile.csv
It assumes your CSV uses comma (,) as a separator. This can be adjusted by changing the IFS=, part in the while loop.
Also, if $objid contains forward slashes (/) in it, mkdir -p will treat it as a path with subdirectories and create all of them. If that's undesirable you can replace / in $objid prior to mkdir like so:
objid="${objid//\//_}"
With read :
while IFS=',' read id image_one image_two; do
[ ! -d "${id}" ] && mkdir "${id}"
for img in ${image_one} ${image_two}; do
printf "Downloading %s" "${img}"
wget -P "${id}" "${img}"
echo "---"
done
done < file.csv
For each line : creates directory based on id value if directory doesn't exist and retrieves images in created dir (with -P option of the wget).
With awk:
awk -F "," '{
print "mkdir",$1"; echo wget -P",$1,$2"; echo wget -P",$1,$3
}' filename.csv | bash