delete file - specific date - shell

How to Delete file created in a specific date??
ls -ltr | grep "Nov 22" | rm -- why this is not wrking??

There are three problems with your code:
rm takes its arguments on its command line, but you're not passing any file name on the command line. You are passing data on the standard input, but rm doesn't read that¹. There are ways around this.
The output of ls -ltr | grep "Nov 22" doesn't just consist of file names, it consists of mangled file names plus a bunch of other information such as the time.
The grep filter won't just catch files modified on November 22; it will also catch files whose name contains Nov 22, amongst others. It also won't catch the files you want in a locale that displays dates differently.
The find command lets you search files according to criteria such as their name matching a certain pattern or their date being in a certain range. For example, the following command will list the files in the current directory and its subdirectories that were modified today (going by GMT calendar date). Replace echo by rm -- once you've checked you have the right files.
find . -type f -mtime -2 -exec echo {} +
With GNU find, such as found on Linux and Cygwin, there are a few options that might do a better job:
-maxdepth 1 (must be specified before other criteria) limits the search to the specified directory (i.e. it doesn't recurse).
-mmin -43 matches files modified at most 42 minutes ago.
-newermt "Nov 22" matches files modified on or after November 22 (local time).
Thus:
find . -maxdepth 1 -type f -newermt "Nov 22" \! -newermt "Nov 23" -exec echo {} +
or, further abbreviated:
find -maxdepth 1 -type f -newermt "Nov 22" \! -newermt "Nov 23" -delete
With zsh, the m glob qualifier limits a pattern to files modified within a certain relative date range. For example, *(m1) expands to the files modified within the last 24 hours; *(m-3) expands to the files modified within the last 48 hours (first the number of days is rounded up to an integer, then - denotes a strict inequality); *(mm-6) expands to the files modified within the last 5 minutes, and so on.
¹ rm -i (and plain rm for read-only files) uses it to read a confirmation y before deletion.

If you need to try find for any given day,
this might help
touch -d "2010-11-21 23:59:59" /tmp/date.start;
touch -d "2010-11-23 00:00:00" /tmp/date.end;
find ./ -type f -newer /tmp/date.start ! -newer /tmp/date.end -exec rm {} \;

If your find supports it, as GNU find does, you can use:
find -type f -newermt "Nov 21" ! -newermt "Nov 22" -delete
which will find files that were modified on November 21.

You would be better suited to use the find command:
find . -type f -mtime 1 -exec echo rm {} +
This will delete all files one day old in the current directory and recursing down into its sub-directories. You can use '0' if you want to delete files created today. Once you are satisfied with the output, remove the echo and the files will truly be deleted.

for i in `ls -ltr | grep "NOV 23" | awk '{print $9}'`
do
rm -rf $i
done

Mb better
#!/bin/bash
for i in $(ls -ltr | grep "NOV 23" | awk '{print $9}')
do
rm -rf $i
done
then in the previous comment

Related

find the last created subdirectory in a directory of 500k subdirs

I have a folder with some 500k subfolders - and I would like to find the last directory which was added to this folder. I am having to do this due to a power failure issue :(
I dont excatly know when the power failed, so using this:
find . -type d -mmin -360 -print
which I beleive is the last 360 minutes? However, gives me results which I am not exactly sure of.
Shortly speaking, I would like to get the last directory which was created within this folder.
Any pointers would be great.
Suggesting :
find . -type d -printf "%C# %p\n" |sort -n|tail -n1|awk '{print $2}'
Explanation:
find . -type d -printf "%C# %p\n"
find . start searching from current directory recursively
-type d search only directory files
-printf "%C# %p\n" for each directory print its last change time in secs from Unix epoch time including sec fraction, followed by file name with path.
For example: 1648051886.4404644000 /tmp/mc-dudi
|sort -n|tail -n1
Sort the result from find as numbers, and print the last row.
awk '{print $2}'
From last row, print only second field
You might try this: it shows your last modification date/time in a sortable manner, and by sorting it, the last entry should be the most recent one:
find ./ -exec ls -dils --time-style=long-iso {} \; | sort -k8,9
Edit: and specific for directories:
find ./ -type d -exec ls -dils --time-style=long-iso {} \; | sort -k8,9
Assuming you're using a file system that tracks file creation ('birth' is the usual terminology) times, and GNU versions of the programs used below:
find . -type d -exec stat --printf '%W\t%n\0' \{\} + | sort -z -k1,1nr | head -1 -z | cut -f 2-
This will find all subdirectories of the current working directory, and for each one, print its birth time (The %W format field for stat(1)) and name (The %n format). Those entries are then sorted based on the timestamp, newest first, and the first line is returned minus the timestamp.
Unfortunately, GNU find's -printf doesn't support birth times, so it calls out to stat(1) to get those, using the multi-argument version of -exec to minimize the number of instances of the program that need to be run. The rest is straightforward sorting of a column, using 0-byte terminators instead of newlines to robustly handle filenames with newlines or other funky characters in them.
Mantaining a symbolic link to the last known subdirectory could avoid listing all of them to find the latest one.
ls -dl $(readlink ~/tmp/last_dir)
drwxr-xr-x 2 lmc users 4096 Jan 13 13:20 /home/lmc/Documents/some_dir
Find newer ones
ls -ldt $(find -L . -newer ~/tmp/last_dir -type d ! -path .)
drwxr-xr-x 2 lmc users 6 Mar 1 00:00 ./dir2
drwxr-xr-x 2 lmc users 6 Feb 1 00:00 ./dir1
Or
ls -ldt $(find -L . -newer ~/tmp/last_dir -type d ! -path .) | head -n 1
drwxr-xr-x 2 lmc users 6 Mar 1 00:00 ./dir2
Don't use the chosen answer if you really want to find the last created sub-directory
According to the question:
Directories should be sorted by creation time instead of modification time.
find --mindepth 1 is necessary because we want to search only sub-directories.
Here are 2 solutions that both fulfill the 2 requirements:
GNU
find . -mindepth 1 -type d -exec stat -c '%W %n' '{}' '+' |
sort -nr | head -n1
BSD
find . -mindepth 1 -type d -exec stat -f '%B %N' '{}' '+' |
sort -nr | head -n1

Check from files in directory which is the most recent in Bash Shell Script

I am making a bash script to run in a directory with files generated everyday and copy the most recent file to another directory.
I am using this now
for [FILE in directory]
do
if [ls -Art | tail -n 1]
something...
else
something...
fi
done
I know this is not alright. I would like to compare the date modified of the files with the current date and if it was equal, copy that file then.
How would that work or is there an easier method to do it?
We could use find:
find . -maxdepth 1 -daystart -type f -mtime -1 -exec cp -f {} dest \;
Explanation:
-maxdepth 1 limits the search to the current directory.
-daystart sets the reference time of -mtime to the beginning of today.
-type f limits the search to files.
-mtime -1 limits the search to files that have been modified less than 1 day from reference time.
-exec cp -f {} dest \; copies the found files to directory dest.
Note that -daystart -mtime -1 means anytime after today 00:00 (included), but also tomorrow or any time in the future. So if you have files with last modification time in year 2042 they will be copied too. Use -mtime 0 if you prefer coping files that have been modified between today at 00:00 (excluded) and tomorrow at 00:00 (included).
Note also that all this could be impacted by irregularities like daylight saving time or leap seconds (not tested).
The newest file is different from file(s) modified today.
Using ls is actually a pretty simple and portable approach. The stdout output format is defined by POSIX (if not printing to a terminal), and ls -A is also in newer POSIX standards.
It should look more like this though:
newest=$(ls -At | head -n 1)
You could add -1, but it AFAIK it shouldn’t be required, as it’s not printing to a terminal.
If you don’t want to use ls, you can use this on linux:
find . -mindepth 1 -maxdepth 1 -type f -exec stat -c ‘%Y:%n’ {} + |
sort -n |
tail -n 1 |
cut -d : -f 2-
Note using 2- not 2 with cut, in case a filename contains :.
Also, the resulting file name will be a relative path (./file), or an empty string if no files exist.

Shell script to remove and compress files from 2 paths

Need to delete log files older than 60 days and compress it if files are greater than 30 days and lesser than 60 days. I have to remove and compress files from 2 paths as mentioned in PURGE_DIR_PATH.
Also have to take the output of find command and redirect it to log file. Basically need to create an entry in the log file whenever a file is deleted. How can i achieve this?
I have to also validate if the directory path is valid or not and put a message in log file if the directory is valid or not
I have written a shell script but doesn't cover all the scenarios. This is my first shell script and need some help. How do i keep just one variable log_retention and
use it to compress files as the condition would be >30 days and <60 days? how do I validate if directories is valid or not? is my IF condition checking that?
Please let me know.
#!/bin/bash
LOG_RETENTION=60
WEB_HOME="/web/local/artifacts"
ENG_DIR="$(dirname $0)"
PURGE_DIR_PATH="$(WEB_HOME)/backup/csvs $(WEB_HOME)/home/archives"
if[[ -d /PURGE_DIR_PATH]] then echo "/PURGE_DIR_PATH exists on your filesystem." fi
for dir_name in ${PURGE_DIR_PATH}
do
echo $PURGE_DIR_PATH
find ${dir_name} -type f -name "*.csv" -mtime +${LOG_RETENTION} -exec ls -l {} \;
find ${dir_name} -type f -name "*.csv" -mtime +${LOG_RETENTION} -exec rm {} \;
done
Off the top of my head -
#!/bin/bash
CSV_DELETE=60
CSV_COMPRESS=30
WEB_HOME="/web/local/artifacts"
PURGE_DIR_PATH=( "$(WEB_HOME)/backup/csvs" "$(WEB_HOME)/home/archives" ) # array, not single string
# eliminate the oldest
find "${PURGE_DIR_PATH[#]}" -type f -name "*.csv" -mtime +${CSV_DELETE} |
xargs -P 100 rm -f # run 100 in bg parallel
# compress the old-enough after the oldest are gone
find "${PURGE_DIR_PATH[#]}" -type f -name "*.csv" -mtime +${CSV_COMPRESS} |
xargs -P 100 gzip # run 100 in bg parallel
Shouldn't need loops.

I want to grep a pattern inside a file n list that file based on current date

ls -l | grep "Feb 22" | grep -l "good" *
This is the command i am using . i have 4 files among which one file contains the world good . I want to list that file . And that file creation is the current date . based on both the criteria i want to list that file
Try this :
find . -type f -newermt 2018-02-21 ! -newermt 2018-02-22 -exec grep -l good {} \;
or
find . -type f -newermt 2018-02-21 ! -newermt 2018-02-22 | xargs grep -l good
And please, don't parse ls output
Hi Try with below command. How it works? Here find command with parameter -mtime -1 will search for files with current date in current directory as well as its sub directories. Each file found will be pass to grep command one at a time. grep command will check for the string in that file (means each file passes to it)
find . -mtime -1 -type f | xargs grep -i "good"
In the above command it will list all the file with current date. To list a files of specific kind you below command. Here I am listing only txt files.
find . -name "*.txt" -mtime -1 -type f | xargs grep -i "good"
find . is for running it from current directory (dot means current directory). To run it from a specific directory path modify like below:-
find /yourpath/ -name "*.txt" -mtime -1 -type f | xargs grep -i "good"
Also grep -i means "ignore case". For a specific case just use grep "good"

Unix Count Multiple Folders Needed

I have a directory on unix server.
cd /home/client/files
It has multiple client folders like these below.
cd /home/client/files/ibm
cd /home/client/files/aol
cd /home/client/files/citi
All of them send us a file starting with either lower or upper case like below:
pre-ibm-03222017
PRE-aol-170322
Once we recieve the files, we process them and convert pre to pro as below:
pro-ibm-03222017
PRO-aol-170322
I want to count the files processed each day. Here is what I am looking for:
If I can just get the total count per client, that would be perfect. If not, then the total count overall.
Keep in mind it has all files as below:
cd /home/client/files/ibm
pre-ibm-03222017
pro-ibm-03222017
cd /home/client/files/aol
PRE-aol-170322
PRO-aol-170322
And I want to COUNT ONLY the PRO/pro that will either be lower or upper case. One folder can get more than 1 file per day.
I am using the below command:
find /home/client/files -type f -mtime -1 -exec ls -1 {} \;| wc -l
But it is giving me the total count of pre and pro files and also it is counting files for last 24 hours....and not the current day only.
For Example. It is currently 09:00 PM. The above command include files received yesterday between 09:00 PM and 12:00 AM as well. I don't wan't those. In other words if I run it at 01:00 AM....it should have all files for 1 hour only and not last 24 hours.
Thanks
---- Update -----
This works great for me.
touch -t 201703230000 first
touch -t 201703232359 last
find /home/client/files/ -newer first ! -newer last | grep -i pro | wc -l
Now, I was just wondering if I can pass the above as parameter.
For example, instead of using touch -t date and alias.....I want to type shortcuts and dates only to get the output. I have made the following aliases:
alias reset='touch -t `date +%m%d0000` /tmp/$$'
alias count='find /home/client/files/ -type f -newer /tmp/$$ -exec ls -1 {} \; | grep -i pro | wc -l'
This way as soon as I logon to the server, I type reset and then I type count and I get my daily number.
I was wondering if I can do something similar for any duration of days by setting date1 and date2 as aliases. If not, then perhaps a short script that would ask for parameters.
What about this?
touch -t `date +%m%d0000` /tmp/$$
find /home/client/files -type f -newer /tmp/$$ -exec ls -1 {} \; | grep -i pro | wc -l
rm /tmp/$$
Other options for finding a file created today can be found in this question:
How do I find all the files that were created today
Actually, a better way to do this is to just use this:
find /home/client/files -type f -m 0 | grep -i pro | wc -l
You can replace -m 0 with -m 5 to find files 5 days old.
For the same day issue u can use -daystart (GNU find)
The regex define a contains of /pre
find /home/client/files -regex '.*\/[pP][rR][eE].*' -type f -daystart -mtime -1 -exec ls -1 {} \;| wc -l

Resources