How to delete files older than X hours - bash

I'm writing a bash script that needs to delete old files.
It's currently implemented using :
find $LOCATION -name $REQUIRED_FILES -type f -mtime +1 -delete
This will delete of the files older than 1 day.
However, what if I need a finer resolution that 1 day, say like 6 hours old? Is there a nice clean way to do it, like there is using find and -mtime?

Does your find have the -mmin option? That can let you test the number of mins since last modification:
find $LOCATION -name $REQUIRED_FILES -type f -mmin +360 -delete
Or maybe look at using tmpwatch to do the same job. phjr also recommended tmpreaper in the comments.

Here is the approach that worked for me (and I don't see it being used above)
$ find /path/to/the/folder -name '*.*' -mmin +59 -delete > /dev/null
deleting all the files older than 59 minutes while leaving the folders intact.

You could to this trick: create a file 1 hour ago, and use the -newer file argument.
(Or use touch -t to create such a file).

-mmin is for minutes.
Try looking at the man page.
man find
for more types.

For SunOS 5.10
Example 6 Selecting a File Using 24-hour Mode
The descriptions of -atime, -ctime, and -mtime use the ter-
minology n ``24-hour periods''. For example, a file accessed
at 23:59 is selected by:
example% find . -atime -1 -print
at 00:01 the next day (less than 24 hours later, not more
than one day ago). The midnight boundary between days has no
effect on the 24-hour calculation.

If you do not have "-mmin" in your version of "find", then "-mtime -0.041667" gets pretty close to "within the last hour", so in your case, use:
-mtime +(X * 0.041667)
so, if X means 6 hours, then:
find . -mtime +0.25 -ls
works because 24 hours * 0.25 = 6 hours

If one's find does not have -mmin and if one also is stuck with a find that accepts only integer values for -mtime, then all is not necessarily lost if one considers that "older than" is similar to "not newer than".
If we were able to create a file that that has an mtime of our cut-off time, we can ask find to locate the files that are "not newer than" our reference file.
To create a file that has the correct time stamp is a bit involved because a system that doesn't have an adequate find probably also has a less-than-capable date command that could do things like: date +%Y%m%d%H%M%S -d "6 hours ago".
Fortunately, other old tools can manage this, albeit in a more unwieldy way.
To begin finding a way to delete files that are over six hours old, we first have to find the time that is six hours ago. Consider that six hours is 21600 seconds:
$ date && perl -e '#d=localtime time()-21600; \
printf "%4d%02d%02d%02d%02d.%02d\n", $d[5]+1900,$d[4]+1,$d[3],$d[2],$d[1],$d[0]'
> Thu Apr 16 04:50:57 CDT 2020
202004152250.57
Since the perl statement produces the date/time information we need, use it to create a reference file that is exactly six hours old:
$ date && touch -t `perl -e '#d=localtime time()-21600; \
printf "%4d%02d%02d%02d%02d.%02d\n", \
$d[5]+1900,$d[4]+1,$d[3],$d[2],$d[1],$d[0]'` ref_file && ls -l ref_file
Thu Apr 16 04:53:54 CDT 2020
-rw-rw-rw- 1 root sys 0 Apr 15 22:53 ref_file
Now that we have a reference file exactly six hours old, the "old UNIX" solution for "delete all files older than six hours" becomes something along the lines of:
$ find . -type f ! -newer ref_file -a ! -name ref_file -exec rm -f "{}" \;
It might also be a good idea to clean up our reference file...
$ rm -f ref_file

Here is what one can do for going on the way #iconoclast was wondering about in their comment on another answer.
use crontab for user or an /etc/crontab to create file /tmp/hour:
# m h dom mon dow user command
0 * * * * root /usr/bin/touch /tmp/hour > /dev/null 2>&1
and then use this to run your command:
find /tmp/ -daystart -maxdepth 1 -not -newer /tmp/hour -type f -name "for_one_hour_files*" -exec do_something {} \;

find $PATH -name $log_prefix"*"$log_ext -mmin +$num_mins -exec rm -f {} \;

Related

Bash: files older than a certain date [duplicate]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 9 years ago.
Improve this question
How do I use the UNIX command find to search for files created on a specific date?
As pointed out by Max, you can't, but checking files modified or accessed is not all that hard. I wrote a tutorial about this, as late as today. The essence of which is to use -newerXY and ! -newerXY:
Example: To find all files modified on the 7th of June, 2007:
$ find . -type f -newermt 2007-06-07 ! -newermt 2007-06-08
To find all files accessed on the 29th of september, 2008:
$ find . -type f -newerat 2008-09-29 ! -newerat 2008-09-30
Or, files which had their permission changed on the same day:
$ find . -type f -newerct 2008-09-29 ! -newerct 2008-09-30
If you don't change permissions on the file, 'c' would normally correspond to the creation date, though.
Use this command to search for files and folders on /home/ add a time period of time according to your needs:
find /home/ -ctime time_period
Examples of time_period:
More than 30 days ago: -ctime +30
Less than 30 days ago: -ctime -30
Exactly 30 days ago: -ctime 30
It's two steps but I like to do it this way:
First create a file with a particular date/time. In this case, the file is 2008-10-01 at midnight
touch -t 0810010000 /tmp/t
Now we can find all files that are newer or older than the above file (going by file modified date). You can also use -anewer for accessed and -cnewer file status changed.
find / -newer /tmp/t
find / -not -newer /tmp/t
You could also look at files between certain dates by creating two files with touch
touch -t 0810010000 /tmp/t1
touch -t 0810011000 /tmp/t2
This will find files between the two dates & times
find / -newer /tmp/t1 -and -not -newer /tmp/t2
You could do this:
find ./ -type f -ls |grep '10 Sep'
Example:
[root#pbx etc]# find /var/ -type f -ls | grep "Dec 24"
791235 4 -rw-r--r-- 1 root root 29 Dec 24 03:24 /var/lib/prelink/full
798227 288 -rw-r--r-- 1 root root 292323 Dec 24 23:53 /var/log/sa/sar24
797244 320 -rw-r--r-- 1 root root 321300 Dec 24 23:50 /var/log/sa/sa24
You can't. The -c switch tells you when the permissions were last changed, -a tests the most recent access time, and -m tests the modification time. The filesystem used by most flavors of Linux (ext3) doesn't support a "creation time" record. Sorry!
#Max: is right about the creation time.
However, if you want to calculate the elapsed days argument for one of the -atime, -ctime, -mtime parameters, you can use the following expression
ELAPSED_DAYS=$(( ( $(date +%s) - $(date -d '2008-09-24' +%s) ) / 60 / 60 / 24 - 1 ))
Replace "2008-09-24" with whatever date you want and ELAPSED_DAYS will be set to the number of days between then and today. (Update: subtract one from the result to align with find's date rounding.)
So, to find any file modified on September 24th, 2008, the command would be:
find . -type f -mtime $(( ( $(date +%s) - $(date -d '2008-09-24' +%s) ) / 60 / 60 / 24 - 1 ))
This will work if your version of find doesn't support the -newerXY predicates mentioned in #Arve:'s answer.
With the -atime, -ctime, and -mtime switches to find, you can get close to what you want to achieve.
cp `ls -ltr | grep 'Jun 14' | perl -wne 's/^.*\s+(\S+)$/$1/; print $1 . "\n";'` /some_destination_dir
I found this scriplet in a script that deletes all files older than 14 days:
CNT=0
for i in $(find -type f -ctime +14); do
((CNT = CNT + 1))
echo -n "." >> $PROGRESS
rm -f $i
done
echo deleted $CNT files, done at $(date "+%H:%M:%S") >> $LOG
I think a little additional "man find" and looking for the -ctime / -atime etc. parameters will help you here.

Check from files in directory which is the most recent in Bash Shell Script

I am making a bash script to run in a directory with files generated everyday and copy the most recent file to another directory.
I am using this now
for [FILE in directory]
do
if [ls -Art | tail -n 1]
something...
else
something...
fi
done
I know this is not alright. I would like to compare the date modified of the files with the current date and if it was equal, copy that file then.
How would that work or is there an easier method to do it?
We could use find:
find . -maxdepth 1 -daystart -type f -mtime -1 -exec cp -f {} dest \;
Explanation:
-maxdepth 1 limits the search to the current directory.
-daystart sets the reference time of -mtime to the beginning of today.
-type f limits the search to files.
-mtime -1 limits the search to files that have been modified less than 1 day from reference time.
-exec cp -f {} dest \; copies the found files to directory dest.
Note that -daystart -mtime -1 means anytime after today 00:00 (included), but also tomorrow or any time in the future. So if you have files with last modification time in year 2042 they will be copied too. Use -mtime 0 if you prefer coping files that have been modified between today at 00:00 (excluded) and tomorrow at 00:00 (included).
Note also that all this could be impacted by irregularities like daylight saving time or leap seconds (not tested).
The newest file is different from file(s) modified today.
Using ls is actually a pretty simple and portable approach. The stdout output format is defined by POSIX (if not printing to a terminal), and ls -A is also in newer POSIX standards.
It should look more like this though:
newest=$(ls -At | head -n 1)
You could add -1, but it AFAIK it shouldn’t be required, as it’s not printing to a terminal.
If you don’t want to use ls, you can use this on linux:
find . -mindepth 1 -maxdepth 1 -type f -exec stat -c ‘%Y:%n’ {} + |
sort -n |
tail -n 1 |
cut -d : -f 2-
Note using 2- not 2 with cut, in case a filename contains :.
Also, the resulting file name will be a relative path (./file), or an empty string if no files exist.

Move all the Files generated 23 hrs. after they were first created

I am using below command to pick the files 23 hrs after they were first created, but it is not picking can you tell me where i am going wrong
find /test/files -maxdepth 1 -type f -mtime +0.9
mtime +1 means 24 hours later
so, used +0.9 so it should pick 23 hours , but it is not picking.
I am afraid fractions will not work in find. What you can do is to create file with timestamp 23 hours ago and with find get older files:
touch -d '23 hours ago' /tmp/tmp_file
find /test/files -maxdepth 1 -type f ! -newer /tmp/tmp_file

Unix Count Multiple Folders Needed

I have a directory on unix server.
cd /home/client/files
It has multiple client folders like these below.
cd /home/client/files/ibm
cd /home/client/files/aol
cd /home/client/files/citi
All of them send us a file starting with either lower or upper case like below:
pre-ibm-03222017
PRE-aol-170322
Once we recieve the files, we process them and convert pre to pro as below:
pro-ibm-03222017
PRO-aol-170322
I want to count the files processed each day. Here is what I am looking for:
If I can just get the total count per client, that would be perfect. If not, then the total count overall.
Keep in mind it has all files as below:
cd /home/client/files/ibm
pre-ibm-03222017
pro-ibm-03222017
cd /home/client/files/aol
PRE-aol-170322
PRO-aol-170322
And I want to COUNT ONLY the PRO/pro that will either be lower or upper case. One folder can get more than 1 file per day.
I am using the below command:
find /home/client/files -type f -mtime -1 -exec ls -1 {} \;| wc -l
But it is giving me the total count of pre and pro files and also it is counting files for last 24 hours....and not the current day only.
For Example. It is currently 09:00 PM. The above command include files received yesterday between 09:00 PM and 12:00 AM as well. I don't wan't those. In other words if I run it at 01:00 AM....it should have all files for 1 hour only and not last 24 hours.
Thanks
---- Update -----
This works great for me.
touch -t 201703230000 first
touch -t 201703232359 last
find /home/client/files/ -newer first ! -newer last | grep -i pro | wc -l
Now, I was just wondering if I can pass the above as parameter.
For example, instead of using touch -t date and alias.....I want to type shortcuts and dates only to get the output. I have made the following aliases:
alias reset='touch -t `date +%m%d0000` /tmp/$$'
alias count='find /home/client/files/ -type f -newer /tmp/$$ -exec ls -1 {} \; | grep -i pro | wc -l'
This way as soon as I logon to the server, I type reset and then I type count and I get my daily number.
I was wondering if I can do something similar for any duration of days by setting date1 and date2 as aliases. If not, then perhaps a short script that would ask for parameters.
What about this?
touch -t `date +%m%d0000` /tmp/$$
find /home/client/files -type f -newer /tmp/$$ -exec ls -1 {} \; | grep -i pro | wc -l
rm /tmp/$$
Other options for finding a file created today can be found in this question:
How do I find all the files that were created today
Actually, a better way to do this is to just use this:
find /home/client/files -type f -m 0 | grep -i pro | wc -l
You can replace -m 0 with -m 5 to find files 5 days old.
For the same day issue u can use -daystart (GNU find)
The regex define a contains of /pre
find /home/client/files -regex '.*\/[pP][rR][eE].*' -type f -daystart -mtime -1 -exec ls -1 {} \;| wc -l

Bash script to remove files older than 15 months

In a bash script, if I want to remove files older than 15 days in a directory, I can run:
find "$DIR" -type f -mtime +15 -exec rm {} \;
Can someone help me with a bash script to remove files older than 15 months in a directory?
Is this the same as ctime in Bash?
According to the man page:
-mtime n[smhdw]
If no units are specified, this primary evaluates to true if the difference between the file last modification time and the time find was started, rounded up to the next
full 24-hour period, is n 24-hour periods.
If units are specified, this primary evaluates to true if the difference between the file last modification time and the time find was started is exactly n units. Please
refer to the -atime primary description for information on supported time units.
Then, at -atime:
-atime n[smhdw]
If no units are specified, this primary evaluates to true if the difference between the file last access time and the time find was started, rounded up to the next full
24-hour period, is n 24-hour periods.
If units are specified, this primary evaluates to true if the difference between the file last access time and the time find was started is exactly n units. Possible
time units are as follows:
s second
m minute (60 seconds)
h hour (60 minutes)
d day (24 hours)
w week (7 days)
Any number of units may be combined in one -atime argument, for example, ``-atime -1h30m''. Units are probably only useful when used in conjunction with the + or - modi-
fier.
So we have weeks. 15 months * 4 weeks/month = 60 weeks.
find $DIR -type f -mtime +60w -exec rm {} \;
use 450 (=15*30) as -mtime parameter.
find $DIR -type f -mtime +450 -exec rm {} \;
http://linux.die.net/man/8/tmpwatch designed for exactly this application. Typically used with cron.
A funny possibility: you can touch a tmp file with timestamp 15 months ago and use it with the (negated) -newer flag of find:
a=$(mktemp)
touch -d '15 months ago' -- "$a"
find "$DIR" -type f \! -newer "$a" -exec rm {} +
rm -- "$a"
This, of course, assumes that your touch and find have these capabilities.
If there's a chance that mktemp creates the file in subdirectories of your directory $DIR, it will get very messy as the file refered to by "$a" could be deleted before the end of the process. In this case, to be 100% sure, use the (negated) -samefile test:
find "$DIR" -type f \! -newer "$a" \! -samefile "$a" -exec rm {} +
You can of course use the -delete command of find if your find supports it. That would give:
a=$(mktemp)
touch -d '15 months ago' -- "$a"
find "$DIR" -type f \! -newer "$a" \! -samefile "$a" -delete
rm -- "$a"

Resources