Sorry if this is a simple question, but I'm new to Bash scripting and my Google skills are failing me. I am trying to write a script that ultimately will examine the last modified time for every file in a directory, and if a file was last modified > 3 days ago, then compress the file with gzip. I can currently print out the last modified time with my script here:
1 !#/bin/bash
2 echo "Enter directory"
3
4 read directory
5
6 for entry in "$directory"/*
7 do
8 stat -f "%Sm" "$entry"
9 done
This prints out the times just fine:
Randalls-MacBook-Pro:bash_scripts randallbanks$ ./print_last_modified.sh
./print_last_modified.sh: line 1: !#/bin/bash: No such file or directory
Enter directory
..
Apr 6 13:12:21 2015
Apr 19 18:50:26 2015
Apr 14 11:29:06 2015
Apr 7 12:26:37 2015
Apr 15 16:05:17 2015
Apr 6 16:28:06 2015
Apr 6 12:28:40 2015
Apr 6 12:28:40 2015
Apr 6 12:28:40 2015
Apr 6 12:28:40 2015
Apr 23 17:03:03 2015
But what I'm having trouble figuring out is how to store or parse these in such a way that I can check if their last modified time was > 3 days ago. How can I go about this?
Edit: bonus question: What is up with it apparently not recognizing my shebang in the header?
To gzip files that were last modified 3 or more days ago, use:
find "$directory" -mtime +2 -type f -exec gzip {} \;
If your find is modern enough, use:
find "$directory" -mtime +2 -type f -exec gzip {} +
How it works
find "$directory"
This starts find looking for files in the directory tree under $directory.
-mtime +2
This looks for files that were modified more than 2 days old. (More than 2 means three or more). See man find for details on how find computes age rounded to days.
-type f
This restricts find to looking for regular files.
-exec gzip {} \;
This tells find to gzip the files that it finds.
Your shebang is a bangshe. It should be #!; it is !#.
The find command on OS X (and BSDs, generally) can do what you're looking for. The -newermt checks the modification time against a time specified in a string, including such strings as "3 days ago".
So:
find . -not -newermt "3 days ago" -exec gzip {} \;
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 9 years ago.
Improve this question
How do I use the UNIX command find to search for files created on a specific date?
As pointed out by Max, you can't, but checking files modified or accessed is not all that hard. I wrote a tutorial about this, as late as today. The essence of which is to use -newerXY and ! -newerXY:
Example: To find all files modified on the 7th of June, 2007:
$ find . -type f -newermt 2007-06-07 ! -newermt 2007-06-08
To find all files accessed on the 29th of september, 2008:
$ find . -type f -newerat 2008-09-29 ! -newerat 2008-09-30
Or, files which had their permission changed on the same day:
$ find . -type f -newerct 2008-09-29 ! -newerct 2008-09-30
If you don't change permissions on the file, 'c' would normally correspond to the creation date, though.
Use this command to search for files and folders on /home/ add a time period of time according to your needs:
find /home/ -ctime time_period
Examples of time_period:
More than 30 days ago: -ctime +30
Less than 30 days ago: -ctime -30
Exactly 30 days ago: -ctime 30
It's two steps but I like to do it this way:
First create a file with a particular date/time. In this case, the file is 2008-10-01 at midnight
touch -t 0810010000 /tmp/t
Now we can find all files that are newer or older than the above file (going by file modified date). You can also use -anewer for accessed and -cnewer file status changed.
find / -newer /tmp/t
find / -not -newer /tmp/t
You could also look at files between certain dates by creating two files with touch
touch -t 0810010000 /tmp/t1
touch -t 0810011000 /tmp/t2
This will find files between the two dates & times
find / -newer /tmp/t1 -and -not -newer /tmp/t2
You could do this:
find ./ -type f -ls |grep '10 Sep'
Example:
[root#pbx etc]# find /var/ -type f -ls | grep "Dec 24"
791235 4 -rw-r--r-- 1 root root 29 Dec 24 03:24 /var/lib/prelink/full
798227 288 -rw-r--r-- 1 root root 292323 Dec 24 23:53 /var/log/sa/sar24
797244 320 -rw-r--r-- 1 root root 321300 Dec 24 23:50 /var/log/sa/sa24
You can't. The -c switch tells you when the permissions were last changed, -a tests the most recent access time, and -m tests the modification time. The filesystem used by most flavors of Linux (ext3) doesn't support a "creation time" record. Sorry!
#Max: is right about the creation time.
However, if you want to calculate the elapsed days argument for one of the -atime, -ctime, -mtime parameters, you can use the following expression
ELAPSED_DAYS=$(( ( $(date +%s) - $(date -d '2008-09-24' +%s) ) / 60 / 60 / 24 - 1 ))
Replace "2008-09-24" with whatever date you want and ELAPSED_DAYS will be set to the number of days between then and today. (Update: subtract one from the result to align with find's date rounding.)
So, to find any file modified on September 24th, 2008, the command would be:
find . -type f -mtime $(( ( $(date +%s) - $(date -d '2008-09-24' +%s) ) / 60 / 60 / 24 - 1 ))
This will work if your version of find doesn't support the -newerXY predicates mentioned in #Arve:'s answer.
With the -atime, -ctime, and -mtime switches to find, you can get close to what you want to achieve.
cp `ls -ltr | grep 'Jun 14' | perl -wne 's/^.*\s+(\S+)$/$1/; print $1 . "\n";'` /some_destination_dir
I found this scriplet in a script that deletes all files older than 14 days:
CNT=0
for i in $(find -type f -ctime +14); do
((CNT = CNT + 1))
echo -n "." >> $PROGRESS
rm -f $i
done
echo deleted $CNT files, done at $(date "+%H:%M:%S") >> $LOG
I think a little additional "man find" and looking for the -ctime / -atime etc. parameters will help you here.
I am using below command to pick the files 23 hrs after they were first created, but it is not picking can you tell me where i am going wrong
find /test/files -maxdepth 1 -type f -mtime +0.9
mtime +1 means 24 hours later
so, used +0.9 so it should pick 23 hours , but it is not picking.
I am afraid fractions will not work in find. What you can do is to create file with timestamp 23 hours ago and with find get older files:
touch -d '23 hours ago' /tmp/tmp_file
find /test/files -maxdepth 1 -type f ! -newer /tmp/tmp_file
I'm trying to construct a reliable shell script to remove older files based on Xn of days using find. However, the script seems to work intermittently. Is there a better way? I list the files first to make sure I capture them, then use -exec rm{} to delete them.
I execute the script like so:
/home/scripts/rmfiles.sh /u05/backup/export/test dmp 1
#!/usr/bin/ksh
if [ $# != 3 ]; then
echo "Usage: rmfiles.sh <directory> <log|dmp|par> <numberofdays>" 2>&1
exit 1
fi
# Declare variables
HOURDATE=`date '+%Y%m%d%H%M'`;
CLEANDIR=$1;
DELETELOG=/tmp/cleanup.log;
echo "Listing files to remove..." > $DELETELOG 2>&1
/usr/bin/find $CLEANDIR -name "*.$2" -mtime +$3 -exec ls -ltr {} \; > $DELETELOG 2>&1
echo "Removing files --> $HOURDATE" > $DELETELOG 2>&1
#/usr/bin/find $CLEANDIR -name "*.$2" -mtime +$3 -exec rm {} \; > $DELETELOG 2>&1
My sample directory clearly has files older than one day as of today, but find is not picking it up when it was before during some previous testing.
Thu Sep 26 08:54:57 PDT 2013
total 161313630
-rw------- 1 oracle dba 10737418240 Sep 24 14:17 testexp01.dmp
-rw------- 1 oracle dba 10737418240 Sep 24 14:20 testexp02.dmp
-rw------- 1 oracle dba 10737418240 Sep 24 14:30 testexp03.dmp
-rw------- 1 oracle dba 508 Sep 24 15:41 EXPORT-20130924.log
-rw------- 1 oracle dba 509 Sep 25 06:00 EXPORT-20130925.log
-rw------- 1 oracle dba 508 Sep 26 08:30 EXPORT-20130926.log
Apart from a couple of small issues, the script looks good in general. My guess is that you want to add -daystart to the list of options so the base for the -mtime test is measured "from the beginning of today rather than from 24 hours ago. This option only affects tests which appear later on the command line."
If you have GNU find, then try find -D tree,search,stat,rates to see what is going on.
Some comments:
Always quote variables to make sure odd spaces don't have an effect: /usr/bin/find "$CLEANDIR" -name "*.$2" -mtime "+$3" .... Same with CLEANDIR="$1"
Don't terminate lines with ;, it's bad style.
You can replace -exec ls -ltr {} \; with -ls or -print. That way, you don't have to run the find command twice.
You should quote {} since some shells interpret them as special characters.
man find
-mtime mentions the read the comment at -atime
"When find figures out how many 24-hour periods ago the file was last accessed, any fractional part is ignored, so to match -atime +1, a file has to have been accessed at least two days ago." so this is also true for -mtime.
I am struggling with listing files modified in a specific month (for example, in February).
Here are several unsuccessful attempts:
I tried creating temporary files and setting their timestamp to the first time in the next month and the first time in the target month and use -newer in find, like this:
find -newer "$from" ! -newer "$to"
This lists files modified in the time interval ($from, $to], but I would like the time interval [$from, $to) (otherwise, there would be false positives on files created on the first second in the next month). Listing files modified in February is another problem, since this would require to set one of the timestamps to the greatest one still in February, but the number of days in February varies depending on whether it is a leap year or not, which requires extra checking.
If I use ls, I encounter a lot of complication when parsing, because of the possibility that user names or groups contain whitespace.
Is there an easy way and relatively portable way for doing this (so it works for any month, regardless of file names, etc.)?
date allows you to easily generate timestamps for purposes like that:
date -d "01-Mar-2011 -1 sec" # last second of Feb-2011
Fortunately, the same syntax is possible in find:
month="Mar-2010"
find . -newermt "01-$month -1 sec" -and -not -newermt "01-$month +1 month -1 sec"
will find all files modified in March 2010.
See the option -newerXY in find's man page.
Well, I can create files that have the minimum timestamp and the maximum timestamp in February, and files that are just beyond February in each direction.
$ touch -t 201102010000.01 from
$ touch -t 201102282359.59 to
$ touch -t 201103010000.01 march
$ touch -t 201101312359.59 january
$ ls -l
total 0
-rw-r--r-- 1 mike None 0 Feb 1 00:00 from
-rw-r--r-- 1 mike None 0 Jan 31 23:59 january
-rw-r--r-- 1 mike None 0 Mar 1 00:00 march
-rw-r--r-- 1 mike None 0 Feb 28 23:59 to
Then using GNU 'find' like this seems to show just the files whose timestamp is in February.
$ find -newermt '2011-02-01' ! -newermt '2011-03-01' -print
./from
./to
I don't know how portable these arguments are to other versions of 'find'.
Adding to Pumbaa80's answer:
In my pre-production environment, find does not support -newermt.
What I did instead was:
Get a list of all possible files (via find, ls etc.)
Generate the timestamps of the last second of last month and this month
LAST_MONTH=$(date -d "01-Jun-2015" -1 sec +%s)
THIS_MONTH=$(date -d "31-Jul-2015" +%s)
Iterate over the list from point 1 and compare the timestamp of each file with the timestamps from point 2
for file in $LIST_OF_FILES
do
TIMESTAMP=$(stat -c"%Y" $file)
if (( $LAST_MONTH < $TIMESTAMP ))
then
if (( $TIMESTAMP < $THIS_MONTH ))
then
echo "Your code here"
fi
fi
done
A workaround could be to use the -printf option to find:
find -printf "%Cm %p\\n"| egrep ^02 |cut -b4-
I don't think find can filter the -printf result itself, nor can it filter on date elements.
edit or if you really want the ls-like output:
find -printf "%Cm " -ls | egrep ^02 |cut -b4-
I'm writing a bash script that needs to delete old files.
It's currently implemented using :
find $LOCATION -name $REQUIRED_FILES -type f -mtime +1 -delete
This will delete of the files older than 1 day.
However, what if I need a finer resolution that 1 day, say like 6 hours old? Is there a nice clean way to do it, like there is using find and -mtime?
Does your find have the -mmin option? That can let you test the number of mins since last modification:
find $LOCATION -name $REQUIRED_FILES -type f -mmin +360 -delete
Or maybe look at using tmpwatch to do the same job. phjr also recommended tmpreaper in the comments.
Here is the approach that worked for me (and I don't see it being used above)
$ find /path/to/the/folder -name '*.*' -mmin +59 -delete > /dev/null
deleting all the files older than 59 minutes while leaving the folders intact.
You could to this trick: create a file 1 hour ago, and use the -newer file argument.
(Or use touch -t to create such a file).
-mmin is for minutes.
Try looking at the man page.
man find
for more types.
For SunOS 5.10
Example 6 Selecting a File Using 24-hour Mode
The descriptions of -atime, -ctime, and -mtime use the ter-
minology n ``24-hour periods''. For example, a file accessed
at 23:59 is selected by:
example% find . -atime -1 -print
at 00:01 the next day (less than 24 hours later, not more
than one day ago). The midnight boundary between days has no
effect on the 24-hour calculation.
If you do not have "-mmin" in your version of "find", then "-mtime -0.041667" gets pretty close to "within the last hour", so in your case, use:
-mtime +(X * 0.041667)
so, if X means 6 hours, then:
find . -mtime +0.25 -ls
works because 24 hours * 0.25 = 6 hours
If one's find does not have -mmin and if one also is stuck with a find that accepts only integer values for -mtime, then all is not necessarily lost if one considers that "older than" is similar to "not newer than".
If we were able to create a file that that has an mtime of our cut-off time, we can ask find to locate the files that are "not newer than" our reference file.
To create a file that has the correct time stamp is a bit involved because a system that doesn't have an adequate find probably also has a less-than-capable date command that could do things like: date +%Y%m%d%H%M%S -d "6 hours ago".
Fortunately, other old tools can manage this, albeit in a more unwieldy way.
To begin finding a way to delete files that are over six hours old, we first have to find the time that is six hours ago. Consider that six hours is 21600 seconds:
$ date && perl -e '#d=localtime time()-21600; \
printf "%4d%02d%02d%02d%02d.%02d\n", $d[5]+1900,$d[4]+1,$d[3],$d[2],$d[1],$d[0]'
> Thu Apr 16 04:50:57 CDT 2020
202004152250.57
Since the perl statement produces the date/time information we need, use it to create a reference file that is exactly six hours old:
$ date && touch -t `perl -e '#d=localtime time()-21600; \
printf "%4d%02d%02d%02d%02d.%02d\n", \
$d[5]+1900,$d[4]+1,$d[3],$d[2],$d[1],$d[0]'` ref_file && ls -l ref_file
Thu Apr 16 04:53:54 CDT 2020
-rw-rw-rw- 1 root sys 0 Apr 15 22:53 ref_file
Now that we have a reference file exactly six hours old, the "old UNIX" solution for "delete all files older than six hours" becomes something along the lines of:
$ find . -type f ! -newer ref_file -a ! -name ref_file -exec rm -f "{}" \;
It might also be a good idea to clean up our reference file...
$ rm -f ref_file
Here is what one can do for going on the way #iconoclast was wondering about in their comment on another answer.
use crontab for user or an /etc/crontab to create file /tmp/hour:
# m h dom mon dow user command
0 * * * * root /usr/bin/touch /tmp/hour > /dev/null 2>&1
and then use this to run your command:
find /tmp/ -daystart -maxdepth 1 -not -newer /tmp/hour -type f -name "for_one_hour_files*" -exec do_something {} \;
find $PATH -name $log_prefix"*"$log_ext -mmin +$num_mins -exec rm -f {} \;