A cron action saves database files on an hourly basis and assigns a file name based on year, month, day and hour
/$(date +\%m)/$(date +\%y\%m\%d\%H)_thedb.sql
This leads to archive bloat and the goal is to keep the last file of the day (i.e. delete all those lesser than 15050923* ) in a separate cron action.
What is an effective way of achieving this?
Before you start with complex bash string substitutions, I suggest you try to go after the file date. find can help you with that.
For example, to delete all files in a directory that are older than 5 days, you could try something like this:
find <DIR> -mtime +5 -exec rm {} \;
Now if there are subdirectories in <DIR>, you might also want to include the options -type f to limit the finding to files, and -maxdepth 1 to not search subdirectories.
If you have a file and want to delete everything older than that, you could slightly modify this:
find <DIR> -not -newer <FILE> -not -name <FILE> -exec rm {} \;
I simply don't know why there is no -older search term in find, it seems so obvious.
Warning: I strongly recommend to first leave out -exec and everything after it to check whether the files it finds can all be deleted.
Related
I would like to create a bash script that compresses the files in a folder, for example:
/home/<username>/Desktop/Folder
And for that, if I'm not mistaken, you could do something like this:
zip -r Folder_2021-Jan.zip /home/<username>/Desktop/Folder
But there is one condition: the files to be compressed must be older than 30 days.
I have no idea how to add that condition to the script. I've searched but haven't found anything similar.
Use find with -mtime to check for files older than 30 days
find /home/<username>/Desktop/Folder -maxdepth 1 -mtime +30 -type f -exec zip Folder_2021-Jan.zip '{}' +
Search for only the directory /home//Desktop/Folder and no child directories for files only (type -f) and then execute zip on as many returned entries as possible, using + and {} as a place holder for entries.
I have a cron job, every 5 minutes, backing up my MYSQL to files ending in .sql.gz. But this is hundreds of files a day. So I searched the internet and found this bash script which I expected to just work on the files in the /backup folder specified and only on .sql.gz files. but I soon found that it deleted everything in my root folder. :-) I was able to FTP the files back and have my site back up in half an hour, but I still need the script to work as intended. I'm new to bash scripting so I'm asking what did I do wrong in editing the script I found on the internet to my needs? What would work?
Here is the rogue script. DO NOT run this as is. its broken, thats why im here:
find /home/user/backups/*.gz * -mmin +60 -exec rm {} \;
Im suspecting its that last backslash should be /home/user/backups/
And also I should remove the * before -min
so what I need should be:
find /home/user/backups/*.gz -mmin +60 -exec rm {} /home/user/backups/;
Am I correct? Or still missing something?
BTW Im running this on Dreamhost shared hosting CRON. Their support don't want to help with BASH questions really, I tried.
The filename arguments to find should be the directories to start the recursive search. Then use -name and other options to filter down to the files that match the criteria you want.
find /home/user/backups -type f -name '*.sql.gz' -mmin +60 -exec rm {} +
-type f means only select ordinary files
-name '*.sql.gz' means only filenames ending in .sql.gz
-mmin +60 means files more than 60 minutes old
And using + instead of \; at the end of -exec means that it should just run the command once with all the selected filenames, rather than separately for each filename; this is a minor efficiency improvement.
I'm a beginner with this stuff and seem to be running into an issue.
Basically, I have many files with names containing a keyword (let's call it "Category1") within a directory. For example:
ABC-Category1-XYZ.txt
I'm trying to move them from a directory into another directory with the same name as the keyword.
I started with this:
find /path_A -name "*Category1*" -exec mv {} /path_A/Category1 \;
It spit out something like this:
mv: rename /path_A/Category1 to /path_A/Category1/Category1: Invalid
Argument
So I did some fiddling and hypothesized that the problem was caused by the command trying to move the directory Category1 into itself(maybe). I decided to exclude directories from the search so it would only attempt to move files. I came up with this:
find /path_A -name "*Category1*" \(! -type d \) -exec mv {} /path_A/Category1 \;
This did move the files from their original location to where I wanted them, but it still gave me something like:
mv: /path_A/Category1/ABC-Category1-XYZ.txt and
/path_A/Category1/ABC-Category1-XYZ.txt are identical
I'm no expert, so I could be wrong... but I believe the command is trying to find and move the files from their original directory, then find them again. The directory Category1 is a subdirectory of the starting point, /path_A, So i believe it is finding the files it just moved in the directory Category1 and attempting to move them again.
Can anyone help me fix this issue?
You are creating new files that find tries to process. Safest approach is to move them somewhere else not in the path_A you are searching with find.
Or you can use prune to ignore that directory if you don't have any other directory matching:
find /path_A -name '*Category1*' -prune -type f -exec mv {} /path_A/Category1/ \;
Although another post has been accepted, let me post a proper answer.
Would you please try:
find /path_A -name 'Category1' -prune -o -type f -name '*Category1*' -exec mv -- {} /path_A/Category1/ \;
The option -prune is rather a command than a condition. It tells find to
ignore the directory tree specified by the conditions before -prune.
In this case it excludes the directory Category1 from the search.
The following -o is logical OR and may be interpreted something like instead or else. The order of the options makes difference.
Please be noticed the 1st category1 is the directory name to exclude and the 2nd *Category1* is the filenames to find.
If you are not sure which files are the result of find, try to execute:
find /path_A -name 'Category1' -prune -o -type f -name '*Category1*' -print
then tweak the options to see the change of output.
I have a set of snapshots. Each snap shot resides in a accountname folder, and each snap shot is named with a date format as: YYYY-MM-DD-accountname
How can I retrieve the name of the "snap shot folder" where it is older than 2 days old? (The 2017-05-* directories)
Folder structure such as:
/home/snapshots
/home/snapshots/account1
/home/snapshots/account1/2017-05-01-account1
/home/snapshots/account1/2017-05-02-account1
/home/snapshots/account1/2017-05-03-account1
/home/snapshots/account1/2017-05-04-account1
/home/snapshots/account1/2017-05-05-account1
/home/snapshots/account1/2017-05-06-account1
/home/snapshots/account2
/home/snapshots/account2/2017-05-01-account1
/home/snapshots/account2/2017-05-02-account1
/home/snapshots/account2/2017-05-03-account1
/home/snapshots/account2/2017-05-04-account1
/home/snapshots/account2/2017-05-05-account1
/home/snapshots/account2/2017-05-06-account1
For instance... I want to list /home/snapshots/account1/2017-05-01 through /home/snapshots/account1/2017-05-04, given that today is 05/06/2017 (US), and vice-versa for account2
I thought find /home/snapshots/ -type d -mtime +2 -exec -ls -la {} \; may do the trick, but that returned me all folders older directories older than 2 days... and adding maxdepth 1 returned nothing...
Continuing from my comment above, the reason you are having problems is you want to search within /home and then select and delete the snapshots directories found if they are more than two days old. With -execdir, it would be
find /home -type d -name "snapshots" -mtime +2 -execdir rm -r '{}' +
Let me know if you have problems. (also, there is no need to use ls -la within find, the -printf option provide you complete output format control without spawning a multiple separate subshells for each ls call, see man find)
note: you should quote '{}' to protect against filenames with whitespace, etc.
Edit
Sorry I misread your question, Obviously if you only want to delete the account* subdirectories of each snapshots directory, then the search path of /home/snapshots is fine and you then include the account*/*account* designator as #BroSlow correctly caught below.
I have files with name as
filename_201702200800.tar.bz2
filename_201702201800.tar.bz2
and so on
I am looking to remove files which are 5 days and older.
The date in the files is of the format %Y%m%d%H%M.
As the creation time corresponds to the names, just use find:
find /path/to/files -type f -ctime +5 -exec rm {} +
From man page:
-exec command {} +
This variant of the -exec action runs the specified command on the selected files, but the command line is built by
appending each selected file name at the end; the total number of
invocations of the command will be much less than the number of
matched files. The command line is built in much the same way that
xargs builds its command lines. Only one instance of ‘{}’ is
allowed within the command. The command is executed in the starting
directory.
Not enough rep to comment.
Can you use the mtime of the files?
find /path/to/files -type f -mtime +5 -delete
Otherwise, you could calculate the dates to find with:
date -d "5 days ago" +%Y%m%d%H%M