I want to put in for-loop in the ftp script. My code is as follows.
ftp -n ftp.server.com <<EOF
user name passwd
bin
prompt
for DATA in d f g l m n o p q; do
mkdir /directory/$DATA
cd /directory/$DATA
mput *.$DATA
done
hash
bye
EOF
It's not working and error is
Interactive mode off.
We only support non-print format, sorry.
?Invalid command
?Invalid command
?Invalid command
Create directory operation failed.
local: mput: No such file or directory
(local-files) ?Invalid command
Hash mark printing on (1024 bytes/hash mark).
Of course I can put this ftp script in for loop, inversely. Is there anything good advice?
You cannot use bash loop inside ftp prompt. But you can generate the stdin for ftp via a bash loop.
{
echo username passwd
echo bin
echo prompt
for DATA in d f g l m n o p q; do
echo mkdir /directory/$DATA
echo cd /directory/$DATA
echo mput *.$DATA
done
echo hash
echo bye
} | ftp -n ftp.server.com
My solution to upload all files to remote server which are created/modified today.
#!/bin/bash
HOST='hostname'
USER='username'
PASSWD='password'
# Local directory where the files are stored.
cd "/local/directory/from where to upload files/"
# To get all the files added today only.
TODAYSFILES=`find -maxdepth 1 -type f -mtime -1`
# remote server directory to upload backup
REMOTEDIR="/directory on remote ftp computer/"
for FILENAME in ${TODAYSFILES[#]}; do
ftp -n -v $HOST << EOT
ascii
user $USER $PASSWD
prompt
cd $REMOTEDIR
put $FILENAME
bye
EOT
done
Related
I'm connected to a remote machine via SSH as part of a bash script. After navigating to the directory, I run ls which confirms matching files are found. However, I then try to loop through the files and run other commands on them, and the variable is now empty.
Code:
echo "DOING STUFF!"
cd /mnt/slowdata/ls8_processing
ls
for f in *.tar.gz
do
echo $f
done
Output:
DOING STUFF!
LC080330242019031901T1-SC20190606111327.tar.gz
LC080330242019042001T1-SC20190606111203.tar.gz
LC080330242019052201T1-SC20190606111130.tar.gz
LC080330252019030301T2-SC20190606111021.tar.gz
LC080330252019031901T1-SC20190606120750.tar.gz
LC080340232019031001T1-SC20190606111056.tar.gz
LC080340232019041101T1-SC20190606111215.tar.gz
LC080340242019031001T1-SC20190606111201.tar.gz
LC080340242019041101T1-SC20190606111250.tar.gz
LC080340242019052901T1-SC20190606111331.tar.gz
As can be seen via the output, the $f is picking something up, as there are the correct number of blank lines. However I wish to untar each file which I cannot do.
TIA.
You have to remove special meaning of $ to pass it to the remote host as '$' else the variable will be expanded before you send the command to the remote host.
Keep in mind the for cycle will run regardless of whether the cd was successful.
ssh server1 << EOF
cd /mnt/slowdata/ls8_processing
ls
for f in *.tar.gz
do
echo \$f
done
EOF
My example show the difference:
script.sh
#!/bin/bash
f=123
ssh -i .ssh/keyauth.pem root#server1 << EOF
for f in ./*.log
do
echo "\$f"
echo "$f"
done
EOF
Output
[edvin#server2 ~]$ ./script.sh
./sepap-install.log
123
./sepfl-upgrade.log
123
./sep-install.log
123
./sepjlu-install.log
123
./sepui-install.log
123
Currently i have a shell script which will sftp all the .txt files in the remote path to local and delete all the .txt files from the remote path.
But i need to change the logic such that i need to delete the file which successfully fetched from the remote path not all the file.
Current code:
sftp user#server << END_SCRIPT
cd /test
mget *.txt
rm *.txt
quit
END_SCRIPT
You can do it in two steps:
# get the files
sftp user#server << END_SCRIPT
cd /test
mget *.txt
quit
END_SCRIPT
# remove the files
{
echo "cd /test"
# assuming that all files in the current dir are fetched through sftp
for file in *.txt; printf 'rm "%s"\n' "$file"; fi
quit
} | sftp user#server
I am making weather model charts with the Grads scripting language, and I am using a bash script so I can use a while loop to download model data (in grib2 format) and call the grads scripts for each frame on the model run. Right now, I have a loop that runs through all the scripts for a given forecast hour and uploads the image output via FTP. After this for loop completes, the grib2 data for the next hour is downloaded, and the loop runs again.
for ((i=0;i<${#SCRIPTS[#]};i++)); do
#define filename
FILENAME="${FILENAMES[i]}${FORECASTHOUR}hrfcst.png"
#run grads script
/home/mint/opengrads/Contents/opengrads -lbc "run /home/mint/opengrads/Contents/plotscripts/${SCRIPTS[i]} $CTLFILE $INIT_STRINGDATE $INIT_INTDATE $INITHOUR $FILENAME $h $MODEL $MODELFORTITLE 500"
#run ftp script
#sh /home/mint/opengrads/Contents/bashscripts/ftpsample.sh $INIT_INTDATE $INITHOUR $FILENAME $MODEL
done
This is inelegant because I open and close an FTP session each time I send a single image. I would much rather write the names of the filenames for a given forecast hour to a .txt file (ex: have a "echo ${FILENAME} >> FILEOFFILENAMES.txt" in the loop) and have my FTP script read and send all those files in a single session. Is this possible?
It's possible. You can add this to your shell script to generate the ftp script and then have it run after you've generated the files:
echo open $HOST > ftp.txt
echo user $USER $PASS >> ftp.txt
find . -type f -name '*hrfcst.png' -printf "put destination/%f %f\n" >> ftp.txt
echo bye >> ftp.txt
ftp < ftp.txt
The above code will generate file ftp.txt with commands and pass that to ftp. The generated ftp.txt will look like:
open host
user user pass
put destination/forecast1.hrfcst.png forecast1.hrfcst.png
put destination/forecast2.hrfcst.png forecast2.hrfcst.png
put destination/forecast3.hrfcst.png forecast3.hrfcst.png
...
bye
The following script will upload all files added today from local directory to remote ftp directory.
#!/bin/bash
HOST='hostname'
USER='username'
PASSWD='password'
# Local directory where the files are stored.
cd "/local/directory/from where to upload files/"
# To get all the files added today only.
TODAYSFILES=`find -maxdepth 1 -type f -mtime -1`
# remote server directory to upload backup
REMOTEDIR="/directory on remote ftp computer/"
for FILENAME in ${TODAYSFILES[#]}; do
ftp -n -v $HOST << EOT
ascii
user $USER $PASSWD
prompt
cd $REMOTEDIR
put $FILENAME
bye
EOT
done
I'm writing a bash script to send files from a linux server to a remote Windows FTP server.
I would like to check using FTP if the folder where the file will be stored exists before attempting to create it.
Please note that I cannot use SSH nor SCP and I cannot install new scripts on the linux server. Also, for performance issues, I would prefer if checking and creating the folders is done using only one FTP connection.
Here's the function to send the file:
sendFile() {
ftp -n $FTP_HOST <<! >> ${LOCAL_LOG}
quote USER ${FTP_USER}
quote PASS ${FTP_PASS}
binary
$(ftp_mkdir_loop "$FTP_PATH")
put ${FILE_PATH} ${FTP_PATH}/${FILENAME}
bye
!
}
And here's what ftp_mkdir_loop looks like:
ftp_mkdir_loop() {
local r
local a
r="$#"
while [[ "$r" != "$a" ]]; do
a=${r%%/*}
echo "mkdir $a"
echo "cd $a"
r=${r#*/}
done
}
The ftp_mkdir_loop function helps in creating all the folders in $FTP_PATH (Since I cannot do mkdir -p $FTP_PATH through FTP).
Overall my script works but is not "clean"; this is what I'm getting in my log file after the execution of the script (yes, $FTP_PATH is composed of 5 existing directories):
(directory-name) Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
To solve this, do as follows:
To ensure that you only use one FTP connection, you create the input (FTP commands) as an output of a shell script
E.g.
$ cat a.sh
cd /home/test1
mkdir /home/test1/test2
$ ./a.sh | ftp $Your_login_and_server > /your/log 2>&1
To allow the FTP to test if a directory exists, you use the fact that "DIR" command has an option to write to file
# ...continuing a.sh
# In a loop, $CURRENT_DIR is the next subdirectory to check-or-create
echo "DIR $CURRENT_DIR $local_output_file"
sleep 5 # to leave time for the file to be created
if (! -s $local_output_file)
then
echo "mkdir $CURRENT_DIR"
endif
Please note that "-s" test is not necessarily correct - I don't have acccess to ftp now and don't know what the exact output of running DIR on non-existing directory will be - cold be empty file, could be a specific error. If error, you can grep the error text in $local_output_file
Now, wrap the step #2 into a loop over your individual subdirectories in a.sh
#!/bin/bash
FTP_HOST=prep.ai.mit.edu
FTP_USER=anonymous
FTP_PASS=foobar#example.com
DIRECTORY=/foo # /foo does not exist, /pub exists
LOCAL_LOG=/tmp/foo.log
ERROR="Failed to change directory"
ftp -n $FTP_HOST << EOF | tee -a ${LOCAL_LOG} | grep -q "${ERROR}"
quote USER ${FTP_USER}
quote pass ${FTP_PASS}
cd ${DIRECTORY}
EOF
if [[ "${PIPESTATUS[2]}" -eq 1 ]]; then
echo ${DIRECTORY} exists
else
echo ${DIRECTORY} does not exist
fi
Output:
/foo does not exist
If you want to suppress only the messages in ${LOCAL_LOG}:
ftp -n $FTP_HOST <<! | grep -v "Cannot create a file" >> ${LOCAL_LOG}
I have a bash script that backs up my iOS files over FTP and I'm getting a few problems, I'm just wondering if anyone could help me out?
Here's my script:
#!/bin/bash
mkdir zipfolder
cp /var/mobile/Library/SMS/sms.db /var/root/zipfolder/
cp /var/mobile/Library/Notes/notes.sqlite /var/root/zipfolder/
cp /var/mobile/Library/Safari/Bookmarks.db /var/root/zipfolder/
cp /var/mobile/Library/Safari/History.plist /var/root/zipfolder/
cd var/root
zip -r zippyy.zip zipfolder
HOST=HOSTNAME
USER=USERNAME
PASS=PASSWORD
ftp -inv $HOST << EOF
user $USER $PASS
cd sms
LIST=$(ls | grep zippyy*.zip)
FILECOUNT=0
for FILE in $LIST
do
if [ -f $FILE ];
then
FILECOUNT+=1
done
FILECOUNT+=1
NEXTDB="zippyy$FILECOUNT.db"
mv zippyy.zip $NEXTDB
ftp -inv $HOST << EOF
put $NEXTDB
bye
EOF
rm -f zippyy.zip
rmdir zipfolder
I get the following errors:
?Invalid command
?Invalid command
We only support non-print format, sorry.
?Invalid command
?Invalid command
?Invalid command
?Invalid command
?Invalid command
?Invalid command
?Invalid command
?Invalid command
?Invalid command
(local-file) (remote-file)
rmdir: failed to remove 'zipfolder': Not a directory
Answer #3 for formatting
Try something like this (totally untested!)
#!/bin/bash
ROOTFOLDER="/var/root"
ZIPNAME="zipfolder"
ZIPFOLDER=$ROOTFOLDER/$ZIPNAME
LIBFOLDER="/var/mobile/Library"
ZIPFILE="zippyy.zip"
mkdir -p $ZIPFOLDER
cp $LIBFOLDER/SMS/sms.db $ZIPFOLDER/
cp $LIBFOLDER/Notes/notes.sqlite $ZIPFOLDER/
cp $LIBFOLDER/Safari/Bookmarks.db $ZIPFOLDER/
cp $LIBFOLDER/Safari/History.plist $ZIPFOLDER/
cd $ROOTFOLDER
zip -r $ZIPFILE $ZIPNAME
HOST=HOSTNAME
USER=USERNAME
PASS=PASSWORD
ftp -inv $HOST << EOF
user $USER $PASS
cd sms
dir . remote_dir.txt
bye
EOF
FILECOUNT=$(grep zippyy remote_dir.txt | wc -l)
NEXTDB="zippyy${FILECOUNT}.db"
mv $ZIPFILE $NEXTDB
ftp -inv $HOST << EOF
user $USER $PASS
put $NEXTDB
bye
EOF
Why are you using cp -i in a script? The -i switch makes the copy "interactive" and so is expecting input from the user, which it wont get because of the script.
Also, can you format your script using the "Code sample" format rather than bullet points! ;-)
New answer for formatting...
It's not entirely clear to me what you're trying to do. It looks like you're trying to find out how many existing backups there are on the ftp server and rename the new backup to go at the end of the list.
You cant execute code on an ftp server (massive security hole!) so the best way to do accomplish this would probably be to get the remote directory listing and process it locally. Try using something like:
ftp -inv $HOST << EOF
user $USER $PASS
cd sms
dir . remote_dir.txt
bye
EOF
{process remote_dir.txt now to get new backup name}
ftp -inv $HOST << EOF
user $USER $PASS
put $NEXTDB
bye
EOF