I'm trying to use SFTP to copy some files from one server to another, this task should run everyweek. The script I use :
HOST='sftp://my.server.com'
USER='user1'
PASSWORD='passwd'
DIR=$HOSTNAME
REMOTE_DIR='/home/remote'
LOCAL_DIR='/home/local'
# LFTP via SFTP connexion
lftp -u "$USER","$PASSWORD" $HOST <<EOF
# changing directory
cd "$REMOTE_DIR"
$(if [ ! -d "$DIR" ]; then
mkdir $DIR
fi)
put -O "$REMOTE_DIR"/$DIR "$LOCAL_DIR"/uploaded.txt
EOF
My issue is that put is executed without taking in consideration the result of if statment.
PS : The error message I got is the following :
put: Access failed: No such file (/home/backups/myhost/upload.txt)
LFTP has no if statement!
What you are doing here?
lftp -u "$USER","$PASSWORD" $HOST <<EOF
cd "$REMOTE_DIR"
$(if [ ! -d "$DIR" ]; then
mkdir $DIR
fi)
put -O "$REMOTE_DIR"/$DIR "$LOCAL_DIR"/uploaded.txt
EOF
You call a sub command in a here document. The sub command is executed locally before lftp is started and its output is pasted in the here document, which gets passed to lftp. This works just, because mkdir has no output. You do not call mkdir on the ftp server. You call the mkdir of your local shell. Effectively it is the same as if you put the if statement before the lftp execution.
if [ ! -d "$DIR" ]; then
mkdir $DIR
fi
lftp -u "$USER","$PASSWORD" $HOST <<EOF
cd "$REMOTE_DIR"
put -O "$REMOTE_DIR"/$DIR "$LOCAL_DIR"/uploaded.txt
EOF
What you are trying to do, does not work. You have to think about a different solution.
Right now I have no FTP server to test it, but it might be possible to use the -f option of LFTP's mkdir. I assume that it may work like the -f option of the Unix rm command. Try this:
lftp -u "$USER","$PASSWORD" $HOST <<EOF
cd "$REMOTE_DIR"
mkdir -f "$DIR"
put -O "$REMOTE_DIR"/$DIR "$LOCAL_DIR"/uploaded.txt
EOF
Update: It works as supposed. The creation of a directory, which exist already, throws no error, if you use the option -f:
lftp anonymous#localhost:/pub> mkdir -f dir
mkdir ok, `dir' created
lftp anonymous#localhost:/pub> mkdir -f dir
lftp anonymous#localhost:/pub> ls
drwx------ 2 116 122 4096 Aug 10 12:04 dir
Maybe you lftp client is outdated. I tested it with Debian 9.
Related
I want to execute cd and scp commands on a remote server which have to be logged in with a different sudo user. Below code snippet asks for the password(echos on screen) for my user but hangs there. It doesn't execute cd
#!/bin/bash
server=myserver.com
ssh $server 'sudo -S -u <user> -i; cd dir1/dir2/; scp file1 user#local-sever'
The issue is that you have a semi colon before cd and so sudo has no command to execute. Remove the ; and it should work:
ssh $server 'sudo -S -u <user> -i scp dir1/dir2/file1 user#local-sever'
There are several ways to address this, but most boil down to wrapping up the commands into a set of instructions. Raman's solution is good since it handles the issue by using full paths, but sometimes that isn't an option. Here's another take -
Assuming your command list can afford the quotes, I like here-strings.
ssh -t sa-nextgen-jenkins.eng.rr.com <<< "
echo 'set -x; cd /tmp; whoami; touch foo; ls -l foo; rm -f foo;'|sudo -iSu user
"
If you need the quotes, try a here-doc.
ssh -t sa-nextgen-jenkins.eng.rr.com <<END
echo 'set -x; echo "$RANDOM"; cd /tmp; whoami; touch foo; ls -l foo; rm -f foo;'|sudo -iSu $user
END
You can also write a small script that has arbitrarily complex commands and scp it over, then use a remote ssh call to execute it as the relevant user.
I have the following code on cron job, it runs but the code does not really do what it supposed to. It does not create the directory plus is does not do anything in the code. Please help check if the way I pointed to the directory is wrong.
#!/bin/bash
NAMEDATE=`date +%F_%H-%M`_`whoami`
NAMEDATE2=`date `
mkdir ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE -m 0755
mysqldump -u u3811*****_boss -p"*******" u3811*****_data | gzip ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE/db.sql.gz
echo "This is the database backup for website.com on $NAMEDATE2" |
mailx -a ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE/db.sql.gz -s "website.com Database attached" -- mail#gmail.com
chmod -R 0644 ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE/*
exit 0
Your NAMEDATE variable needs to be modified a bit, as shown below, for more information about variables in bash see this link
NAMEDATE=$(date +%F_%H-%M"_"$(whoami))
When you issue the mkdir command you will need to pass the -p option to create the complete directory structure if it doesn't exists.
mkdir -p ~/home/u3811numbers/domains/website.com/public_html/cron/backup/files/$NAMEDATE -m 0755
Also, the ~ character on Linux based distributions is used as a shortcut for the home directory of the user that invokes it so, in the line below the result is /home//home/u3811*****/domains/website.com/public_html/cron/backup/files/2020-09-04_23-13_ you can read more about it in here
In you last command before the exit, you might need to pass a wildcard (*) to avoid removing the executable bit on the directory, see below
chmod -R 0644 ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE/
The final version of your script will look something like this.
#!/bin/bash
NAMEDATE=$(date +%F_%H-%M"_"$(whoami))
NAMEDATE2=date
mkdir -p ~/home/u3811******/domains/website.com/public_html/cron/backup/files/$NAMEDATE -m 0755
mysqldump -u u3811*****_boss -p"******" u3811*****_data | gzip > ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE/db.sql.gz
echo "This is the database backup for website.com on $NAMEDATE2" | mailx -a ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE/db.sql.gz -s "website.com Database attached" -- mail#gmail.com
chmod -R 0644 ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE/*
To debug a bash script you can always pass the -x flag for more information take a look at this article
First I want to say that I don't really know what I should look for, here in Stack Overflow and what could be a good query for my problem.
In simple words I want to create a new directory and than do some file operations in it. But with the script that I have crafted I got always a file instead of a directory. It seems to be absolutely regardless how I stick the code together there is always the same result. I hope tat masses can help me with their knowledge.
Here is the script:
#!/bin/bash
DLURL=http://drubuntu.googlecode.com/git'
d7dir=/var/www/d7/'
dfsettings=/var/www/d7/sites/default/default.settings.php
settings=/var/www/d7/sites/default/settings.php
#settiing up drush
drush -y dl drush --destination=/usr/share;
#Download and set up drupal
cd /var/www/;
drush -y dl drupal;
mkdir "$d7dir"; #this is the line that always produces a file instead a directory
# regardless if it is replaced by the variable or entered as
# /var/www/d7
cd /var/www/drup*;
cp .htaccess .gitignore "$d7dir";
cp -r * "$d7dir";
cd "$d7dir";
rm -r /var/www/drup*;
mkdir "$d7dir"sites/default/files;
chmod 777 "$d7dir"sites/default/files;
cp "$dfsettings" "$settings";
chmod 777 "$settings";
chown $username:www-data /var/www/d7/.htaccess;
wget -O $d7dir"setupsite $DLURL/scripts/setupsite.sh; > /dev/null 2>&1
chmod +x /var/www/setupsite;
echo "Login Details following...";
read -sn 1 -p "Press any key to continue...";
bash "$d7dir"setupsite;
chown -Rh $username:www-data /var/www;
chmod 644 $d7dir".htaccess;
chmod 644"$settings";
chmod 644"$dfsettings";
exit
I hope someone got the reason for that.
There are many way to debug a shell-scripting.
Add set -x in your beginning script
Get the return value.
mkdir 'the-directory'
ret=$?
if test $ret -eq 0; then
echo 'Create success.'
else
echo 'Failed to create.'
fi
Set to verbose mode $ mkdir -v 'the-directory'
Try this command $ type mkdir, to checking mkdir command.
First of all, I'm saying that it doesn't work properly with the crontab because when I run the script manually it works fine.
The problem is that when I run the backup script with the cronjob and... it's coming to tar up the mysql dump, the tar archive has only 16 bytes size (and its empty, so it looks like there were no files to pack into the archive), the strange thing about that is that when I run the script manually, it runs almost 5~ minutes, and the tar package size is ~1.8GB.
Here is my bash code:
#!/usr/local/bin/bash
# Configuration
BACKUPD="/backup/mysql"
MySQLuser='root'
MySQLpass='xxxx'
# End configuration
ROK=`date +%Y`
MIESIAC=`date +%m`
DZIEN=`date +%d`
GIM=`date +%H-%M`
if [ -d $BACKUPD/$ROK/$MIESIAC/$DZIEN ]
then
echo
else
mkdir -p $BACKUPD/$ROK/$MIESIAC/$DZIEN
fi
for db in $(echo "SHOW DATABASES;" | mysql --user=$MySQLuser --password=$MySQLpass | grep -v -e "Database" -e "information_schema")
do
mysqldump --skip-lock-tables --ignore-table=log.log --user="$MySQLuser" --password="$MySQLpass" $db >$BACKUPD/$ROK/$MIESIAC/$DZIEN/$db.sql
done
cd $BACKUPD/$ROK/$MIESIAC/$DZIEN && tar jcPf $BACKUPD/$ROK/$MIESIAC/$DZIEN/mysql-$GIM.tar.bz2 *.sql && rm -rf *.sql
Where is the problem? Did anyone experienced a problem like this before?
Regards.
Can you try with full path name for mysqldump and mysql inside your script.
So:
if which mysql is equal to /usr/local/mysql/bin/mysql
and
if which mysqldump is equal to /usr/local/mysql/bin/mysqldump
Modify your script to:
for db in $(echo "SHOW DATABASES;" | /usr/local/mysql/bin/mysql --user=$MySQLuser --password=$MySQLpass | grep -v -e "Database" -e "information_schema")
do
/usr/local/mysql/bin/mysqldump --skip-lock-tables --ignore-table=log.log --user="$MySQLuser" --password="$MySQLpass" $db >$BACKUPD/$ROK/$MIESIAC/$DZIEN/$db.sql
done
My guess is that the last line is your problem. The shell glob (*.sql) in:
cd $BACKUPD/$ROK/$MIESIAC/$DZIEN && tar jcPf $BACKUPD/$ROK/$MIESIAC/$DZIEN/mysql-$GIM.tar.bz2 *.sql && rm -rf *.sql
is expanded in the current directory and not after the cd as you might expect. Try the following instead, it is safer.
old_dir=`pwd`
cd "$BACKUPD/$ROK/$MIESIAC/$DZIEN"
tar jcPf mysql-$GIM.tar.bz2 *.sql
rm -fr *.sql
cd "$old_dir"
There still might not be any .sql files to tar ball. I don't have mysql installed but I suspect that the for loop is messed up as well. Try something like the following instead:
mysqlshow | \
xargs mysqldump --databases | \
bzip2 > $BACKUPD/$ROK/$MIESIAC/$DZIEN/mysql-$GIM.bz2
You will probably beed to insert other arguments for the mysqlshow and mysqldump commands. Of course this won't create a tarball but it will give you a compressed backup.
I have this simple bash script that gets a copy from my dev server:
#!/bin/sh
DATE=`date +%Y-%m-%d_%H%M.%S`
BASEDIR="/var/www/db"
RELEASEDIR="$DATE";
RELEASEDIRFULL="$BASEDIR/releases/$RELEASEDIR"
mkdir -p "$RELEASEDIRFULL"
echo "Chdir to \"$RELEASEDIRFULL\""
cd "$RELEASEDIRFULL"
echo "Getting copy from dev"
ssh dev.example.tld "cd /tmp; cd /sites/db; tar -zcvp --exclude data --exclude scripts -f - *" | tar zxvpf -
ln -s /var/www/db/data data
ln -s /var/www/db/scripts scripts
cd $BASEDIR
rm htdocs; ln -s releases/$RELEASEDIR htdocs
Recently it stopped working properly with no apparent reason. It gets to the ssh line, executes it fine (files appear on live server) but does not proceed with ln commands. If I comment the ssh line out, ln lines will get executed properly.
UPDATE: I noticed that when I'm logged on as www-data and start the script, it completes as expected, without errors.
No time to check up the man page, but looks like your tar input is - * - all files + stdin? Are you meaning -- for suspension of further argument processing (if tar supports that)