i'm trying to create a simple script that will copy files from server1 to server 2 or from server 2 to server1(depends where i run the script from)
I created a script that should recognize on which server I am, take the source folder and destination folder and execute.
for example
sh script.sh /home/test /destest
should cop y files from test folder to the other server to destest folder
but something is not working for me, I keep getting
No such file or directoryscp:
any ideas?
#!/bin/bash
SRC1=$1
DEST=$3
BOX=$(hostname)
if [ $BOX=server1 ]; then
sudo scp $SRC1 server2:\ $DEST
else
sudo scp -v $SRC1/* server1:\ $DEST
fi
Don't put a space after server1: and server2:.
You need a space around = in the if test.
You should almost always quote variables, in case the value contains whitespace, unless you actually want to split it into separate arguments.
#!/bin/bash
SRC1=$1
DEST=$3
BOX=$(hostname)
if [ "$BOX" = server1 ]; then
sudo scp "$SRC1" "server2:$DEST"
else
sudo scp -v "$SRC1"/* "server1:$DEST"
fi
This is my fixed script that is now working :)
#!/bin/bash
BOX=$(hostname)
if [ "$BOX" = server1 ]; then
sudo scp "$1" user#server2:\ "$2"
else
sudo scp "$1"/* user#server1:\ "$2"
fi
Related
I'm writing a bash script to send files from a linux server to a remote Windows FTP server.
I would like to check using FTP if the folder where the file will be stored exists before attempting to create it.
Please note that I cannot use SSH nor SCP and I cannot install new scripts on the linux server. Also, for performance issues, I would prefer if checking and creating the folders is done using only one FTP connection.
Here's the function to send the file:
sendFile() {
ftp -n $FTP_HOST <<! >> ${LOCAL_LOG}
quote USER ${FTP_USER}
quote PASS ${FTP_PASS}
binary
$(ftp_mkdir_loop "$FTP_PATH")
put ${FILE_PATH} ${FTP_PATH}/${FILENAME}
bye
!
}
And here's what ftp_mkdir_loop looks like:
ftp_mkdir_loop() {
local r
local a
r="$#"
while [[ "$r" != "$a" ]]; do
a=${r%%/*}
echo "mkdir $a"
echo "cd $a"
r=${r#*/}
done
}
The ftp_mkdir_loop function helps in creating all the folders in $FTP_PATH (Since I cannot do mkdir -p $FTP_PATH through FTP).
Overall my script works but is not "clean"; this is what I'm getting in my log file after the execution of the script (yes, $FTP_PATH is composed of 5 existing directories):
(directory-name) Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
To solve this, do as follows:
To ensure that you only use one FTP connection, you create the input (FTP commands) as an output of a shell script
E.g.
$ cat a.sh
cd /home/test1
mkdir /home/test1/test2
$ ./a.sh | ftp $Your_login_and_server > /your/log 2>&1
To allow the FTP to test if a directory exists, you use the fact that "DIR" command has an option to write to file
# ...continuing a.sh
# In a loop, $CURRENT_DIR is the next subdirectory to check-or-create
echo "DIR $CURRENT_DIR $local_output_file"
sleep 5 # to leave time for the file to be created
if (! -s $local_output_file)
then
echo "mkdir $CURRENT_DIR"
endif
Please note that "-s" test is not necessarily correct - I don't have acccess to ftp now and don't know what the exact output of running DIR on non-existing directory will be - cold be empty file, could be a specific error. If error, you can grep the error text in $local_output_file
Now, wrap the step #2 into a loop over your individual subdirectories in a.sh
#!/bin/bash
FTP_HOST=prep.ai.mit.edu
FTP_USER=anonymous
FTP_PASS=foobar#example.com
DIRECTORY=/foo # /foo does not exist, /pub exists
LOCAL_LOG=/tmp/foo.log
ERROR="Failed to change directory"
ftp -n $FTP_HOST << EOF | tee -a ${LOCAL_LOG} | grep -q "${ERROR}"
quote USER ${FTP_USER}
quote pass ${FTP_PASS}
cd ${DIRECTORY}
EOF
if [[ "${PIPESTATUS[2]}" -eq 1 ]]; then
echo ${DIRECTORY} exists
else
echo ${DIRECTORY} does not exist
fi
Output:
/foo does not exist
If you want to suppress only the messages in ${LOCAL_LOG}:
ftp -n $FTP_HOST <<! | grep -v "Cannot create a file" >> ${LOCAL_LOG}
Can an if statement in a bash script confirm its current folder based on wildcards?
I am making a script to quickly place a drupal site in maintenance mode, ask if you want to keep the .htaccess file, keep the sites folder and the .htaccess file (if requested), update modules and the database then take the site out of maintenance mode. I have accomplished the above, but for ease of use, I would like to have one master script for all site folders, stored in a central location.
Here is my script that works as designed:
#/bin/bash
CWD=$(pwd)
cd $CWD
echo $CWD
if [ $CWD = "/var/www/vhosts/specific.site.folder" ]; then
echo "Updating drupal core files"
read -r -p "do you need to keep the .htaccess file? [y/N]" response
if [ $response = y ]
then
/usr/local/bin/drush vset maintenance_mode 1
mv ./.htaccess ../.htaccess
mv ./sites ../sites
rm -rf *
cp -R /sources/drupal/* ./
mv ../sites ./sites
mv ../.htaccess ./.htaccess
chown -R httpd:httpd *
/usr/local/bin/drush up && /usr/local/bin/drush updb
/usr/local/bin/drush vset maintenance_mode 0
elif [ $response = n ]
then
/usr/local/bin/drush vset maintenance_mode 1
mv ./sites ../sites
rm -rf *
cp -R /sources/drupal/* ./
mv ../sites ./sites
chown -R httpd:httpd *
/usr/local/bin/drush up && /usr/local/bin/drush updb
/usr/local/bin/drush vset maintenance_mode 0
else
echo "Response must be either y or n"
fi
else
echo "not in a web directory, exiting"
fi
What I would like is for the same script, but something like:
if [ $CWD = "/var/www/vhosts/*" ]; then
This did not work. I would like this to work so that the same script can run for any site, but keep other directories safe, including subdirectories on the site folder itself.
If there are better ways to do anything I have there, please suggest them.
the servers that this is intended to run on are based on centos 6, but it may need to be used on a debian based machine at some point, if anything needs changing please also let me know.
First, you need to use the [[ command; [ does not do pattern matching. Second, the asterisk needs to be unquoted to be treated as a pattern metacharacter, rather than a literal asterisk.
if [[ $CWD = "/var/www/vhosts/"* ]]; then
i have a long string in each line, one line like,
1000 AS34_59329 RICwdsRSYHSD11-2-IPAAPEK-93 /ifshk5/BC_IP/PROJECT/T11073/T11073_RICekkR/Fq/AS34_59329/111220_I631_FCC0E5EACXX_L4_RICwdsRSYHSD11-2-IPAAPEK-93_1.fq.gz /ifshk5/BC_IP/PROJECT/T11073/T11073_RICekkR/Fq/AS34_59329/111220_I631_FCC0E5EACXX_L4_RICwdsRSYHSD11-2-IPAAPEK-93_2.fq.gz /ifshk5/BC_IP/PROJECT/T11073/T11073_RICekkR/Fq/AS34_59329/clean_111220_I631_FCC0E5EACXX_L4_RICwdsRSYHSD11-2-IPAAPEK-93_1.fq.gz.total.info 11.824 0.981393 43.8283 95.7401 OK
this line contains three file locations(bold parts), i need to scp those files to another location like /sample . and also create sub-directory to put files, like this line files put into AS34_59329. so need create /sample/AS34_59329
Maybe many lines' sub-directory name is the same, so it need to judge whether the sub-directory has already create.
how to auto create the sub-directory?
#! /bin/bash
while read myline
do
for i in $myline
do
if [ -f $i]; then
scp -r $i xxxx#192.168.174.33:/sample
fi
done
done < data.list
It looks like you have ssh keys, so if you ssh then remote commands will work for you
if [ -f $i]; then
ssh xxxx#192.168.174.33 '[ -d /sample ] && echo "OK" || mkdir /sample'
scp -r $i xxxx#192.168.174.33:/sample
fi
This will only work if you have privilege on the remote box to create /sample.
I am trying to upload multiple files from one folder to a ftp site and wrote this script:
#!/bin/bash
for i in '/dir/*'
do
if [-f /dir/$i]; then
HOST='x.x.x.x'
USER='username'
PASSWD='password'
DIR=archives
File=$i
ftp -n $HOST << END_SCRIPT
quote USER $USER
quote PASS $PASSWD
ascii
put $FILE
quit
END_SCRIPT
fi
It is giving me following error when I try to execute:
username#host:~/Documents/Python$ ./script.sh
./script.sh: line 22: syntax error: unexpected end of file
I can't seem to get this to work. Any help is much appreciated.
Thanks,
Mayank
It's complaining because your for loop does not have a done marker to indicate the end of the loop. You also need more spaces in your if:
if [ -f "$i" ]; then
Recall that [ is actually a command, and it won't be recognized if it doesn't appear as such.
And... if you single quote your glob (at the for) like that, it won't be expanded. No quotes there, but double quotes when using $i. You probably also don't want to include the /dir/ part when you use $i as it's included in your glob.
If I'm not mistaken, ncftp can take wildcard arguments:
ncftpput -u username -p password x.x.x.x archives /dir/*
If you don't already have it installed, it's likely available in the standard repo for your OS.
First, the literal, fixing-your-script answer:
#!/bin/bash
# no reason to set variables that don't change inside the loop
host='x.x.x.x'
user='username'
password='password'
dir=archives
for i in /dir/*; do # no quotes if you want the wildcard to be expanded!
if [ -f "$i" ]; then # need double quotes and whitespace here!
file=$i
ftp -n "$host" <<END_SCRIPT
quote USER $user
quote PASS $password
ascii
put $file $dir/$file
quit
END_SCRIPT
fi
done
Next, the easy way:
lftp -e 'mput -a *.i' -u "$user,$password" "ftp://$host/"
(yes, lftp expands the wildcard internally, rather than expecting this to be done by the outer shell).
First of all my apologies in not making myself clear in the question. My actual task was to copy a file from local folder to a SFTP site and then move the file to an archive folder. Since the SFTP is hosted by a vendor I cannot use the key sharing (vendor limitation. Also, SCP will require password entering if used in a shell script so I have to use SSHPASS. SSHPASS is in the Ubuntu repo however for CentOS it needs to be installed from here
Current thread and How to run the sftp command with a password from Bash script? did gave me better understanding on how to write the script and I will share my solution here:
#!/bin/bash
#!/usr/bin
for i in /dir/*; do
if [ -f "$i" ]; then
file=$i
export SSHPASS=password
sshpass -e sftp -oBatchMode=no -b - user#ftp.com << !
cd foldername/foldername
put $file
bye
!
mv $file /somedir/test
fi
done
Thanks everyone for all the responses!
--Mayank
how to find a file exists in particular dir through SSH
for example :
host1 and dir /home/tree/TEST
Host2:- ssh host1 - find the TEST file exists or not using bash
ssh will return the exit code of the command you ask it to execute:
if ssh host1 stat /home/tree/TEST \> /dev/null 2\>\&1
then
echo File exists
else
echo Not found
fi
You'll need to have key authentication setup of course, so you avoid the password prompt.
This is what I ended up doing after reading and trying out the stuff here:
FileExists=`ssh host "test -e /home/tree/TEST && echo 1 || echo 0"`
if [ ${FileExists} = 0 ]
#do something because the file doesn't exist
fi
More info about test: http://linux.die.net/man/1/test
An extension to Erik's accepted answer.
Here is my bash script for waiting on an external process to upload a file. This will block current script execution indefinitely until the file exists.
Requires key-based SSH access although this could be easily modified to a curl version for checks over HTTP.
This is useful for uploads via external systems that use temporary file names:
rsync
transmission (torrent)
Script below:
#!/bin/bash
set -vx
#AUTH="user#server"
AUTH="${1}"
#FILE="/tmp/test.txt"
FILE="${2}"
while (sleep 60); do
if ssh ${AUTH} stat "${FILE}" > /dev/null 2>&1; then
echo "File found";
exit 0;
fi;
done;
No need for echo. Can't get much simpler than this :)
ssh host "test -e /path/to/file"
if [ $? -eq 0 ]; then
# your file exists
fi