inotifywait adding multiple files at ones - bash

this shell script should add everything put in the folder to transmission. With one folder it works fine, but when i add more then one folder at the same moment it ignores the second one.
while true;
do
file=$(inotifywait -e moved_to --format %f /srv/watchfolderfilme)
file="/srv/watchfolderfilme/$file"
transmission-create -o $file.torrent -s 16384 -t http://0.0.0.0:6969/announce $file
mv $file /srv/downloads
chmod 0777 $file.torrent
cp $file.torrent /srv/newtorrentfiles
mv $file.torrent /srv/watchfoldertorrents
done

Rethough my solution and found a better one that works fine for multiple adds
inotifywait -m /srv/watchfolderfilme -e create -e moved_to |
while read path action file; do
# echo "The file '$file' appeared in directory '$path' via '$action'"
chmod 0777 $path$file
transmission-create -o /srv/newtorrentfiles/$file.torrent -s 16384 -t http://0.0.0.0:6969/announce $path$file
mv $path$file /srv/downloads
chmod 0777 /srv/newtorrentfiles/$file.torrent
cp /srv/newtorrentfiles/$file.torrent /srv/watchfoldertorrents
done

Related

Adding shell if statement inside lftp

I'm trying to use SFTP to copy some files from one server to another, this task should run everyweek. The script I use :
HOST='sftp://my.server.com'
USER='user1'
PASSWORD='passwd'
DIR=$HOSTNAME
REMOTE_DIR='/home/remote'
LOCAL_DIR='/home/local'
# LFTP via SFTP connexion
lftp -u "$USER","$PASSWORD" $HOST <<EOF
# changing directory
cd "$REMOTE_DIR"
$(if [ ! -d "$DIR" ]; then
mkdir $DIR
fi)
put -O "$REMOTE_DIR"/$DIR "$LOCAL_DIR"/uploaded.txt
EOF
My issue is that put is executed without taking in consideration the result of if statment.
PS : The error message I got is the following :
put: Access failed: No such file (/home/backups/myhost/upload.txt)
LFTP has no if statement!
What you are doing here?
lftp -u "$USER","$PASSWORD" $HOST <<EOF
cd "$REMOTE_DIR"
$(if [ ! -d "$DIR" ]; then
mkdir $DIR
fi)
put -O "$REMOTE_DIR"/$DIR "$LOCAL_DIR"/uploaded.txt
EOF
You call a sub command in a here document. The sub command is executed locally before lftp is started and its output is pasted in the here document, which gets passed to lftp. This works just, because mkdir has no output. You do not call mkdir on the ftp server. You call the mkdir of your local shell. Effectively it is the same as if you put the if statement before the lftp execution.
if [ ! -d "$DIR" ]; then
mkdir $DIR
fi
lftp -u "$USER","$PASSWORD" $HOST <<EOF
cd "$REMOTE_DIR"
put -O "$REMOTE_DIR"/$DIR "$LOCAL_DIR"/uploaded.txt
EOF
What you are trying to do, does not work. You have to think about a different solution.
Right now I have no FTP server to test it, but it might be possible to use the -f option of LFTP's mkdir. I assume that it may work like the -f option of the Unix rm command. Try this:
lftp -u "$USER","$PASSWORD" $HOST <<EOF
cd "$REMOTE_DIR"
mkdir -f "$DIR"
put -O "$REMOTE_DIR"/$DIR "$LOCAL_DIR"/uploaded.txt
EOF
Update: It works as supposed. The creation of a directory, which exist already, throws no error, if you use the option -f:
lftp anonymous#localhost:/pub> mkdir -f dir
mkdir ok, `dir' created
lftp anonymous#localhost:/pub> mkdir -f dir
lftp anonymous#localhost:/pub> ls
drwx------ 2 116 122 4096 Aug 10 12:04 dir
Maybe you lftp client is outdated. I tested it with Debian 9.

Script for directory mirroring with inotifywait and ssh

I have a script that try to mirror a specific directory from a local server to a remote one. It looks like that:
inotifywait -mr --format '%w%f' -e close_write -e moved_to -e delete /mydir | \
while read FILECHANGE
do
if [ -f $FILECHANGE ]
then
rsync --bwlimit=4096 --progress --relative -vrae 'ssh -p 22' $FILECHANGE $REMOTEHOST:/
else
ssh -p 22 $REMOTEHOST "rm $FILECHANGE"
fi
done
In case of multiple create of files, as for example a touch command:
touch 1 2 3
The 3 files are well transfered.
But if I delete several files at once:
rm -f 1 2 3
Only the first 1 is deleted.
If I replace the ssh command by just an echo $FILECHANGE, the 3 files are well displayed in the console. So it seems the problem come from the ssh command, but I can't explain why and solve it.
Anyone as an idea?
Well, I found the issue: it seems that the ssh command was eating the output of the inotifywait command when run. So, to prevent that, I add the 0<&- redirection after the ssh, to close the stdin.
inotifywait -mr --format '%w%f' -e close_write -e moved_to -e delete /mydir | \
while read FILECHANGE
do
if [ -f $FILECHANGE ]
then
rsync --bwlimit=4096 --progress --relative -vrae 'ssh -p 22' $FILECHANGE $REMOTEHOST:/
else
ssh -p 22 $REMOTEHOST "rm $FILECHANGE" 0<&-
fi
done
Now it works.

mkdir always creates a file instead a directory

First I want to say that I don't really know what I should look for, here in Stack Overflow and what could be a good query for my problem.
In simple words I want to create a new directory and than do some file operations in it. But with the script that I have crafted I got always a file instead of a directory. It seems to be absolutely regardless how I stick the code together there is always the same result. I hope tat masses can help me with their knowledge.
Here is the script:
#!/bin/bash
DLURL=http://drubuntu.googlecode.com/git'
d7dir=/var/www/d7/'
dfsettings=/var/www/d7/sites/default/default.settings.php
settings=/var/www/d7/sites/default/settings.php
#settiing up drush
drush -y dl drush --destination=/usr/share;
#Download and set up drupal
cd /var/www/;
drush -y dl drupal;
mkdir "$d7dir"; #this is the line that always produces a file instead a directory
# regardless if it is replaced by the variable or entered as
# /var/www/d7
cd /var/www/drup*;
cp .htaccess .gitignore "$d7dir";
cp -r * "$d7dir";
cd "$d7dir";
rm -r /var/www/drup*;
mkdir "$d7dir"sites/default/files;
chmod 777 "$d7dir"sites/default/files;
cp "$dfsettings" "$settings";
chmod 777 "$settings";
chown $username:www-data /var/www/d7/.htaccess;
wget -O $d7dir"setupsite $DLURL/scripts/setupsite.sh; > /dev/null 2>&1
chmod +x /var/www/setupsite;
echo "Login Details following...";
read -sn 1 -p "Press any key to continue...";
bash "$d7dir"setupsite;
chown -Rh $username:www-data /var/www;
chmod 644 $d7dir".htaccess;
chmod 644"$settings";
chmod 644"$dfsettings";
exit
I hope someone got the reason for that.
There are many way to debug a shell-scripting.
Add set -x in your beginning script
Get the return value.
mkdir 'the-directory'
ret=$?
if test $ret -eq 0; then
echo 'Create success.'
else
echo 'Failed to create.'
fi
Set to verbose mode $ mkdir -v 'the-directory'
Try this command $ type mkdir, to checking mkdir command.

TAR doesn't work properly with the crontab

First of all, I'm saying that it doesn't work properly with the crontab because when I run the script manually it works fine.
The problem is that when I run the backup script with the cronjob and... it's coming to tar up the mysql dump, the tar archive has only 16 bytes size (and its empty, so it looks like there were no files to pack into the archive), the strange thing about that is that when I run the script manually, it runs almost 5~ minutes, and the tar package size is ~1.8GB.
Here is my bash code:
#!/usr/local/bin/bash
# Configuration
BACKUPD="/backup/mysql"
MySQLuser='root'
MySQLpass='xxxx'
# End configuration
ROK=`date +%Y`
MIESIAC=`date +%m`
DZIEN=`date +%d`
GIM=`date +%H-%M`
if [ -d $BACKUPD/$ROK/$MIESIAC/$DZIEN ]
then
echo
else
mkdir -p $BACKUPD/$ROK/$MIESIAC/$DZIEN
fi
for db in $(echo "SHOW DATABASES;" | mysql --user=$MySQLuser --password=$MySQLpass | grep -v -e "Database" -e "information_schema")
do
mysqldump --skip-lock-tables --ignore-table=log.log --user="$MySQLuser" --password="$MySQLpass" $db >$BACKUPD/$ROK/$MIESIAC/$DZIEN/$db.sql
done
cd $BACKUPD/$ROK/$MIESIAC/$DZIEN && tar jcPf $BACKUPD/$ROK/$MIESIAC/$DZIEN/mysql-$GIM.tar.bz2 *.sql && rm -rf *.sql
Where is the problem? Did anyone experienced a problem like this before?
Regards.
Can you try with full path name for mysqldump and mysql inside your script.
So:
if which mysql is equal to /usr/local/mysql/bin/mysql
and
if which mysqldump is equal to /usr/local/mysql/bin/mysqldump
Modify your script to:
for db in $(echo "SHOW DATABASES;" | /usr/local/mysql/bin/mysql --user=$MySQLuser --password=$MySQLpass | grep -v -e "Database" -e "information_schema")
do
/usr/local/mysql/bin/mysqldump --skip-lock-tables --ignore-table=log.log --user="$MySQLuser" --password="$MySQLpass" $db >$BACKUPD/$ROK/$MIESIAC/$DZIEN/$db.sql
done
My guess is that the last line is your problem. The shell glob (*.sql) in:
cd $BACKUPD/$ROK/$MIESIAC/$DZIEN && tar jcPf $BACKUPD/$ROK/$MIESIAC/$DZIEN/mysql-$GIM.tar.bz2 *.sql && rm -rf *.sql
is expanded in the current directory and not after the cd as you might expect. Try the following instead, it is safer.
old_dir=`pwd`
cd "$BACKUPD/$ROK/$MIESIAC/$DZIEN"
tar jcPf mysql-$GIM.tar.bz2 *.sql
rm -fr *.sql
cd "$old_dir"
There still might not be any .sql files to tar ball. I don't have mysql installed but I suspect that the for loop is messed up as well. Try something like the following instead:
mysqlshow | \
xargs mysqldump --databases | \
bzip2 > $BACKUPD/$ROK/$MIESIAC/$DZIEN/mysql-$GIM.bz2
You will probably beed to insert other arguments for the mysqlshow and mysqldump commands. Of course this won't create a tarball but it will give you a compressed backup.

Downloading and automatically installing a tgz file

#!/bin/bash
mkdir /tmp
curl -O http://www.mucommander.com/download/nightly/mucommander-current.app.tar.gz /tmp/mucommander.tgz
tar -xvzf /tmp/mucommander.tgz */mucommander.app/*
cp -r /tmp/mucommander.app /Applications
rm -r /tmp
I'm trying to create a shell script to download and extract muCommander to my applications directory on a Mac.
I tried cd into the tmp dir, but then the script stops when I do that.
I can extract all using the -C argument, but the current tgz path is muCommander-0_9_0/mucommander.app, which could change on later builds, so I'm trying to keep it generic.
Can anyone give me pointers where I'm going wrong?
Thanks in advance.
Strip the first path component when you untar the archive, from tar(1):
--strip-components count
(x mode only) Remove the specified number of leading path ele-
ments. Pathnames with fewer elements will be silently skipped.
Note that the pathname is edited after checking inclusion/exclu-
sion patterns but before security checks.
Update
Here is a working bash example of how to, fairly generically, copy the contents of the tgz file to /Applications.
shopt -s nocaseglob
TMPDIR=/tmp
APP=mucommander
TMPAPPDIR=$TMPDIR/$APP
mkdir -p $TMPAPPDIR
curl -o $TMPDIR/$APP.tgz http://www.mucommander.com/download/nightly/mucommander-current.app.tar.gz
tar --strip-components=1 -xvzf $APP.tgz -C $TMPAPPDIR
mv $TMPAPPDIR/${APP}* /Applications
# rm -rf $TMPAPPDIR $TMPDIR/$APP
The rm command is commented out for now, verify that it does no harm before you use it.
The following will update your muCommander.
#for the safety, remove old temporary extraction from the /tmp
rm -rf /tmp/muCommander.app
#kill the running mucommander - you dont want replace the runnung app
ps -ef | grep ' /Applications/muCommander.app/' | grep -v grep | awk '{print $2}' | xargs kill
#download, extract, remove old, move new, open
#each command run only when the previous ended with success
curl http://www.mucommander.com/download/nightly/mucommander-current.app.tar.gz |\
tar -xzf - -C /tmp --strip-components=1 '*/muCommander.app' && \
rm -rf /Applications/muCommander.app && \
mv /tmp/muCommander.app /Applications && \
open /Applications/muCommander.app
Beware, after the '\' must following new line, and not any spaces...

Resources