shell script not deleting files - bash

I am using some shell script coding , here it is
DBHOSTNAME=********
DBUSERNAME=*****
DBPASSWORD=******
DBNAME=******
BACKUPFOLDER=$HOME/*******
DELETEFILES=Y
DAILYBACKUP=Y
NUMDAILYBACKUPS=2
WEEKLYBACKUP=Y
NUMWEEKLYBACKUPS=2
MONTHLYBACKUP=Y
NUMMONTHLYBACKUPS=2
TODATE=$(date +%d)
TOMORROW=`date +%d -d "1 day"`
TODAY=$(date +%a)
MONTH=$(date +%B)
WEEK=$(date +%U)
if [ $TODATE -gt $TOMORROW ] && [ "$MONTHLYBACKUP" == "Y" ]
then
/usr/bin/mysqldump -h $DBHOSTNAME -u $DBUSERNAME -p$DBPASSWORD $DBNAME | gzip > $BACKUPFOLDER/$DBNAME'_'`date '+%m-%d-%Y'`'_'$MONTH.sql.gz
else
if [ "$TODAY" == "Sat" ] && [ "$WEEKLYBACKUP" == "Y" ]
then
/usr/bin/mysqldump -h $DBHOSTNAME -u $DBUSERNAME -p$DBPASSWORD $DBNAME | gzip > $BACKUPFOLDER/$DBNAME'_'`date '+%m-%d-%Y'`'_'Week$WEEK.sql.gz
else
if [ "$DAILYBACKUP" == "Y" ]
then
/usr/bin/mysqldump -h $DBHOSTNAME -u $DBUSERNAME -p$DBPASSWORD $DBNAME | gzip > $BACKUPFOLDER/$DBNAME'_'`date '+%m-%d-%Y'`'_'$TODAY.sql.gz
fi
fi
fi
if [ $DELETEFILES == Y ]
then
NUMWEEKLY=$[$NUMWEEKLYBACKUPS*7]
NUMMONTHLY=$[$NUMMONTHLYBACKUPS*31]
find $BACKUPFOLDER/*Sun.sql.gz -type f -mtime +$NUMDAILYBACKUPS -delete 2> /dev/null
find $BACKUPFOLDER/*Mon.sql.gz -type f -mtime +$NUMDAILYBACKUPS -delete 2> /dev/null
find $BACKUPFOLDER/*Tue.sql.gz -type f -mtime +$NUMDAILYBACKUPS -delete 2> /dev/null
find $BACKUPFOLDER/*Wed.sql.gz -type f -mtime +$NUMDAILYBACKUPS -delete 2> /dev/null
find $BACKUPFOLDER/*Thu.sql.gz -type f -mtime +$NUMDAILYBACKUPS -delete 2> /dev/null
find $BACKUPFOLDER/*Fri.sql.gz -type f -mtime +$NUMDAILYBACKUPS -delete 2> /dev/null
find $BACKUPFOLDER/*Sat.sql.gz -type f -mtime +$NUMDAILYBACKUPS -delete 2> /dev/null
find $BACKUPFOLDER/*Week*.sql.gz -type f -mtime +$NUMWEEKLY -delete 2> /dev/null
find $BACKUPFOLDER/*January.sql.gz -type f -mtime +$NUMMONTHLY -delete 2> /dev/null
find $BACKUPFOLDER/*February.sql.gz -type f -mtime +$NUMMONTHLY -delete 2> /dev/null
find $BACKUPFOLDER/*March.sql.gz -type f -mtime +$NUMMONTHLY -delete 2> /dev/null
find $BACKUPFOLDER/*April.sql.gz -type f -mtime +$NUMMONTHLY -delete 2> /dev/null
find $BACKUPFOLDER/*May.sql.gz -type f -mtime +$NUMMONTHLY -delete 2> /dev/null
find $BACKUPFOLDER/*June.sql.gz -type f -mtime +$NUMMONTHLY -delete 2> /dev/null
find $BACKUPFOLDER/*July.sql.gz -type f -mtime +$NUMMONTHLY -delete 2> /dev/null
find $BACKUPFOLDER/*August.sql.gz -type f -mtime +$NUMMONTHLY -delete 2> /dev/null
find $BACKUPFOLDER/*September.sql.gz -type f -mtime +$NUMMONTHLY -delete 2> /dev/null
find $BACKUPFOLDER/*October.sql.gz -type f -mtime +$NUMMONTHLY -delete 2> /dev/null
find $BACKUPFOLDER/*November.sql.gz -type f -mtime +$NUMMONTHLY -delete 2> /dev/null
find $BACKUPFOLDER/*December.sql.gz -type f -mtime +$NUMMONTHLY -delete 2> /dev/null
fi
it is working fine, but doesnt delete old files, i taken this from following address http://www.htpcbeginner.com/automatic-mysql-database-backup-on-godaddy/3/
what i need is backup of 3 days
like
Today files in folder
03/01/2017
04/01/2017
05/01/2017
and tommorow it should be
04/01/2017
05/01/2017
06/01/2017
it should delete 3jan file

Besides what the commenters have said, I think a key issue is your find command. You have, for one example:
find $BACKUPFOLDER/*Sun.sql.gz -type f -mtime +$NUMDAILYBACKUPS -delete 2> /dev/null
But that should be
find "$BACKUPFOLDER" -name '*Sun.sql.gz' -type f -mtime "+$NUMDAILYBACKUPS" -delete
When using find, you list the paths ("$BACKUPFOLDER") and the filenames (-name '*Sun.sql.gz') separately. You also single-quote the filenames to prevent the shell from expanding them.
While debugging, don't use 2> /dev/null. That discards error messages you might otherwise find helpful in solving your problem :) .

Related

find printf what is correct format in crontab?

i want to save a list of files and i want to do in cron but i dont know how to convert this command
echo $(find /tmp -type f -printf "%p||%s||||||") > /share/Public/serwer/test33.data
in terminal this command works ok but in cron is smothing wrong, file is empty, why?
i tried add slashes before variables %s %p
i tried many many other combinations:
echo $(find /tmp -type f -printf "%p||%s||||||") > /share/Public/serwer/test33.data
echo $(find /tmp -type f -printf "%p||%s||||||") > /share/Public/serwer/test32.data
echo $(find /tmp -type f) > $TMP_DIR/test31.data
echo $(find $BACKUP_DIR -type f -printf \%p) > $TMP_DIR/test30.data
echo $(find $BACKUP_DIR -type f -printf %p) > $TMP_DIR/test28.data
echo $(find $BACKUP_DIR -type f) > $TMP_DIR/test27.data
echo $(find $BACKUP_DIR -type f -printf "ab") > $TMP_DIR/test26.data
echo $(find $BACKUP_DIR -type f -printf "||") > $TMP_DIR/test25.data
echo $(find $BACKUP_DIR -type f -printf "%p||%s\r\n") > $TMP_DIR/test01.data
echo $(find $BACKUP_DIR -type f -printf "\%p||\%s\r\n") > $TMP_DIR/test02.data
echo $(find $BACKUP_DIR -type f -printf "\%p") > $TMP_DIR/test03.data
echo $(find $BACKUP_DIR -type f -printf "\%s") > $TMP_DIR/test04.data
echo $(find $BACKUP_DIR -type f -printf "\\%s") > $TMP_DIR/test05.data
echo $(find $BACKUP_DIR -type f -printf "\\%p") > $TMP_DIR/test06.data
echo $(find $BACKUP_DIR -type f -printf "\r\n") > $TMP_DIR/test07.data
echo $(find $BACKUP_DIR -type f -printf) > $TMP_DIR/test10.data
echo $(find $BACKUP_DIR -type f -printf) > $TMP_DIR/test11.data
echo $(find $BACKUP_DIR -type f -printf "%p") > $TMP_DIR/test12.data
echo $(find $BACKUP_DIR -type f -printf "\%p") > $TMP_DIR/test13.data
echo `find $BACKUP_DIR -type f -printf "%p"` > $TMP_DIR/test14.data
echo `find $BACKUP_DIR -type f -printf "\\\\%p"` > $TMP_DIR/test15.data
echo `find $BACKUP_DIR -type f -printf \%p` > $TMP_DIR/test16.data
echo `find $BACKUP_DIR -type f -printf '\%p'` > $TMP_DIR/test17.data
echo `find $BACKUP_DIR -type f -printf '\\%p'` > $TMP_DIR/test18.data
echo `find $BACKUP_DIR -type f -printf '\\\%p'` > $TMP_DIR/test19.data
echo `find $BACKUP_DIR -type f -printf '\\\\%p'` > $TMP_DIR/test20.data
echo `find $BACKUP_DIR -type f -printf \%'s` > $TMP_DIR/test21.data
echo `find $BACKUP_DIR -type f -printf \%'p` > $TMP_DIR/test22.data
echo `find $BACKUP_DIR -type f -printf \%'p'` > $TMP_DIR/test23.data
echo `find $BACKUP_DIR -type f -printf \%'s'` > $TMP_DIR/test24.data
nothing work
This crontab entry should work :
* * * * * find /tmp -type f -printf "\%p||\%s||||||\n" > /share/Public/serwer/test33.data 2>/tmp/crontab.err
i added full path to command find, now it works correctly
/share/CACHEDEV1_DATA/.qpkg/Qapache/bin/find $BACKUP_DIR -type f -printf "%p||%s\r"
thanks for help
Thanks Philippe i do that and i have smothing like this:
find: unrecognized: -printf
BusyBox v1.24.1 (2021-09-23 02:31:15 CST) multi-call binary.
Usage: find [-HL] [PATH]... [OPTIONS] [ACTIONS]
Search for files and perform actions on them.
First failed action stops processing of current file.
Defaults: PATH is current directory, action is '-print'
-L,-follow Follow symlinks
-H ...on command line only
-xdev Don't descend directories on other filesystems
-maxdepth N Descend at most N levels. -maxdepth 0 applies
actions to command line arguments only
-mindepth N Don't act on first N levels
-depth Act on directory *after* traversing it
Actions:
( ACTIONS ) Group actions for -o / -a
! ACT Invert ACT's success/failure
ACT1 [-a] ACT2 If ACT1 fails, stop, else do ACT2
ACT1 -o ACT2 If ACT1 succeeds, stop, else do ACT2
Note: -a has higher priority than -o
-name PATTERN Match file name (w/o directory name) to PATTERN
-iname PATTERN Case insensitive -name
-path PATTERN Match path to PATTERN
-ipath PATTERN Case insensitive -path
-regex PATTERN Match path to regex PATTERN
-type X File type is X (one of: f,d,l,b,c,...)
-perm MASK At least one mask bit (+MASK), all bits (-MASK),
or exactly MASK bits are set in file's mode
-mtime DAYS mtime is greater than (+N), less than (-N),
or exactly N days in the past
-mmin MINS mtime is greater than (+N), less than (-N),
or exactly N minutes in the past
-newer FILE mtime is more recent than FILE's
-inum N File has inode number N
-user NAME/ID File is owned by given user
-group NAME/ID File is owned by given group
-size N[bck] File size is N (c:bytes,k:kbytes,b:512 bytes(def.))
+/-N: file size is bigger/smaller than N
-links N Number of links is greater than (+N), less than (-N),
or exactly N
-prune If current file is directory, don't descend into it
If none of the following actions is specified, -print is assumed
-print Print file name
-print0 Print file name, NUL terminated
-exec CMD ARG ; Run CMD with all instances of {} replaced by
file name. Fails if CMD exits with nonzero
-exec CMD ARG + Run CMD with {} replaced by list of file names
-delete Delete current file/directory. Turns on -depth option

How can I properly execute a command with lists using `sh -c`?

I have this command to recursively find directories that contain mustExist.js but not cannotExist.js:
comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \; | sort -u)
It works fine.
Now, I must pass it as a string to a node.js automation script. The script picks up the string and runs it as sh -c <string>. I cannot change that part.
So I pass this string:
'comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \\; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \\; | sort -u)'
However, I always encounter this error:
Warning: Command failed: /bin/sh -c comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \; | sort -u)
/bin/sh: 1: Syntax error: "(" unexpected
When I pass this string:
'"comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \\; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \\; | sort -u)"'
I get:
Warning: Command failed: /bin/sh -c "comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \; | sort -u)"
/bin/sh: 1: comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \; | sort -u): not found
When I manually try using sh to immitate the automation script:
sh -c comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \; | sort -u)
I get a different error:
comm: missing operand
Or with quotes:
sh -c "comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \; | sort -u)"
I get:
/bin/sh: 1: Syntax error: "(" unexpected
Other quotes:
sh -c 'comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \; | sort -u)'
/bin/sh: 1: Syntax error: "(" unexpected
Is it possible to do this somehow?
There is a much simpler and more efficient command you can use:
find . -type d -exec sh -c 'test -e "$1"/mustExist.js && ! test -e "$1"/cannotExist.js' _ {} \; -print
This iterates over the directories, and checks each one for the required file as well as making sure the forbidden file is not present.
An example, where only foo/bar3 contains yes.js without also containing no.js.
$ mkdir -p foo/bar1 foo/bar2 foo/bar3
$ touch foo/bar1/no.js foo/bar1/yes.js foo/bar2/no.js foo/bar3/yes.js
$ find foo -type d -exec sh -c 'test -e "$1"/yes.js && ! test -e "$1"/no.js' _ {} \; -print
foo/bar3
Passing this to your script requires some creative quoting, though:
somescript "find foo -type d -exec sh -c 'test -e \"\$1\"/yes.js && ! test -e \"\$1\"/no.js' _ {} \; -print"
If you are using bash, you can simplify it a little:
somescript $'find foo -type d -exec sh -c \'test -e "$1"/yes.js && ! test -e "$1"/no.js\' _ {} \; -print'
If you are willing to use the obsolete and possibly unsupported operator -a, you can reduce this to a single invocation of test.
find . -type d -exec test -e {}/mustExist.js -a ! -e {}/cannotExist.js \; -print
This is also a little simpler to pass to your script, since it does not itself contain any quotes:
somescript 'find . -type d -exec test -e {}/mustExist.js -a ! -e {}/cannotExist.js \; -print'
You can simplify it using multiple -exec primaries as well:
somescript 'find . -type d -exec test -e {}/mustExist.js \; ! -exec test -e {}/cannotExist.js \; -print'
which is a little less efficient (it runs test twice instead of once) but is more portable, while being easier to quote than the version that passes the string to sh -c.

ls command and size of files in shell script

count=0; #count for counting
IFS='
'
for x in `ls -l $input`; #for loop using ls command
do
a=$(ls -ls | awk '{print $6}') #print[6] is sizes of file
echo $a
b=`echo $a | awk '{split($0,numbers," "); print numbers[1]}'`
echo $b
if [ $b -eq 0 ] # b is only size of a file
then
count=`expr $count + 1` #if b is zero , the count will increase one by one
fi
echo $count
done
I want to find 0 size files . I do that using find command. The second thing is I want to count number of has 0 size of files using ls command and awk. But It doesn't true code . What is my mistake ?
The -s test is true if a file has non-zero size. If that test fails for file, increment your empty-file count.
empty_files=0
for f in "$input"/*; do
[ -s "$f" ] || : $(( empty_files++ ))
done
Your main mistake is that you're parsing ls!
If you want to find (regular) files that are empty, and if you have a version of find that supports the -empty predicate, use it:
find . -type f -empty
Note that this will recurse in subfolders too; if you don't want that, use:
find . -maxdepth 1 -type f -empty
(assuming that your find also supports -maxdepth).
If you only want to count how many empty (regular) files you have:
find . -maxdepth 1 -type f -empty -printf x | wc -m
and if you want to perform both operations at the same time, i.e., print out the name or save them in an array for future use, and count them:
empty_files=()
while IFS= read -r -d '' f; do
empty_files+=( "$f" )
done < <(find . -maxdepth 1 -type f -empty -print0)
printf 'There are %d empty files:\n' "${#empty_files[#]}"
printf ' %s\n' "${empty_files[#]}"
With Bash≥4.4, you could use mapfile instead of the while-read loop:
mapfile -t -d '' empty_files < <(find . -maxdepth 1 -type f -empty -print0)
printf 'There are %d empty files:\n' "${#empty_files[#]}"
printf ' %s\n' "${empty_files[#]}"
For a POSIX-compliant way, use test with the -s option:
find . -type f \! -exec test -s {} \; -print
and if you don't want to recurse into subdirectories, you'll have to -prune them:
find . \! -name . -prune -type f \! -exec test -s {} \; -print
and if you want to count them:
find . \! -name . -prune -type f \! -exec test -s {} \; -exec printf x | wc -m
and here, if you want to perform both operations (count them and save them in an array for later use), use the previous while-read loop (or mapfile if you live in the future) with this find:
find . \! -name . -prune -type f \! -exec test -s {} \; -exec printf '%s\0' {} \;
Also see chepner's answer for a pure shell solution (needs minor tweaking to be POSIX compliant).
Regarding your comment
I want to count and delete [empty files]. How can I do that at the same time?
If you have GNU find (or a find that supports all the goodies):
find . -maxdepth 1 -type f -empty -printf x -delete | wc -m
if not,
find . \! -name . -prune -type f \! -exec test -s {} \; -printf x -exec rm {} \; | wc -m
Make sure that the -delete (or -exec rm {} \;) predicate is at the end! do not exchange the order of the predicates!

Deleting files in the tree

Can you help me to find trouble with my code? It has to delete every files in the tree
function option_c {
for i in `find "$TEST_DIR" -type f | grep -E "(stdout|stderr|status)-(captured|delta)"` ; do
if [ -w $i ] ; then
rm $i
fi
exit 0
done
}
Thanks for your help.
Take out the exit, it terminates your script after the first file.
You could do all of this with find itself, though.
As tripleee mentions, you can do all in find:
find "$TEST_DIR" -type f -perm +0200 -regextype posix-awk -regex ".*st(dout|derr|atus)-(captured|delta).*" -delete
or
find "$TEST_DIR" -type f -perm +0200 -regex ".*st\(dout\|derr\|atus\)-\(captured\|delta\).*" -delete
find "$TEST_DIR" -type f -perm +0200 -exec egrep "(stdout|stderr|status)-(captured|delta)" {} ";" -delete
(at least gnu-find, which is all I have, has the -delete switch and the regex-options).

modify shell script to delete folders as well as files

My shell script:
#!/bin/bash
if [ $# -lt 2 ]
then
echo "$0 : Not enough argument supplied. 2 Arguments needed."
echo "Argument 1: -d for debug (lists files it will remove) or -e for execution."
echo "Followed by some path to remove files from. (path of where to look) "
exit 1
fi
if test $1 == '-d'
then
find $2 -mmin +60 -type f -exec ls -l {} \;
elif test $1 == '-e'
then
find $2 -mmin +60 -type f -exec rm -rf {} \;
fi
Basically this will find files in a given directory provided as second argument and either list (-d for argument 1) or remove (-e for argument 1) files modified >60 minutes ago.
How can I rework this to also remove folders ?
Remove -type f
changing ls -l to ls -ld
Change 1 will list everything and not just files. This includes links as well. If you are not fine with listing/deleting anything other than files and directories then you need to separately list/delete files and directories as:
if test $1 == '-d'
then
find $2 -mmin +60 -type f -exec ls -ld {} \;
find $2 -mmin +60 -type d -exec ls -ld {} \;
elif test $1 == '-e'
then
find $2 -mmin +60 -type f -exec rm -rf {} \;
find $2 -mmin +60 -type d -exec rm -rf {} \;
fi
Change 2 is needed as ls -l on a directory will list the files in the directories.
#!/bin/bash
if [ $# -lt 2 ]
then
echo "$0 : Not enough argument supplied. 2 Arguments needed."
echo "Argument 1: -d for debug (lists files it will remove) or -e for execution."
echo "Followed by some path to remove files from. (path of where to look) "
exit 1
fi
if test $1 == '-d'
then
find $2 -mmin +60 -type d -exec ls -l {} \;
find $2 -mmin +60 -type f -exec ls -l {} \;
elif test $1 == '-e'
then
find $2 -mmin +60 -type d -exec rm -rf {} \;
find $2 -mmin +60 -type f -exec rm -rf {} \;
fi
That should work for you.

Resources