Files are created at regular interval with the name which do not contain any timestamp.
How can I grep/filter any text from these files created in last 24 hours or with creation date of the files?
/data/logs/file*.log
You can use find to get the files in /data/logs/ on the form file*log that were modified on the last 24 hours:
find /data/logs/ -mtime -1 -name file*log
Then, just grep through exec:
find /data/logs -mtime -1 -name file*log -exec grep "whatever" {} \;
If you also want to show the filename, use -H as suggested by Mark Setchell --> ... -exec grep -H "whatever" {} \;.
Credits to Scripts: find the files have been changed in last 24 hours.
Related
I want to grep files which created today in the current directory. So how many ways to do that? What's the best way to do that?
grep --color 'content' ./directory
This should do the trick for you:
find ./directory -maxdepth 1 -type f -daystart -ctime 0 -print | xargs grep --color 'content'
In the above command, we are using find to find all the files (-type f) in directory, that were made today (-daystart -ctime 0) and then -print the full files paths to standard output. We then send the output to xargs. Using xargs we are able to execute each line of the output through the grep command. This is much simpler than having to create a for loop and iterate over each line of the output.
If I understand you want to grep "content" within all file in ./directory modified today, then you can use a combination of find and xargs. For example to find the files in ./directory modified today, you can give the -mtime 0 option which find files modified 0 24 hour periods ago (e.g. today). To handle strange filenames, use the -print0 option to have find output nul-terminated filenames. Your find command could be:
find . -maxdepth 1 -type f -mtime 0 -print0
One the list of files is generated, you can pass the result to xargs -0 which will process the list of filenames as being nul-terminated and using your grep command, you would have:
xargs -0 grep --color 'content'
To put it altogether, simply pipe the result of find to xargs, e.g.
find . -maxdepth 1 -type f -mtime 0 -print0 |
xargs -0 grep --color 'content'
Give that a go and let me know if it does what you need or if you have further questions.
Edit Per Comment
If you want more exact control of the hour, or minute or second from which you want to select your files, you can use the -newermt option for find to file all files newer than the date you give as the option, e.g. -newermt "2021-07-02 02:10:00" would select today's file created after 2:10:00 (all files after 2:10:00 am this morning)
Modifying the test above and replacing -mtime 0 with -newermt "2021-07-02 02:10:00" you would have:
find . -maxdepth 1 -type f -newermt "2021-07-02 02:10:00"` -print 0 |
xargs -0 grep --color 'content'
(adjust the time to your exact starting time you want to begin selecting files from)
Give that a go also. It is quite a bit more flexible as you can specify any time within the day to begin selecting files from based on the files modification time.
I have a directory on unix server.
cd /home/client/files
It has multiple client folders like these below.
cd /home/client/files/ibm
cd /home/client/files/aol
cd /home/client/files/citi
All of them send us a file starting with either lower or upper case like below:
pre-ibm-03222017
PRE-aol-170322
Once we recieve the files, we process them and convert pre to pro as below:
pro-ibm-03222017
PRO-aol-170322
I want to count the files processed each day. Here is what I am looking for:
If I can just get the total count per client, that would be perfect. If not, then the total count overall.
Keep in mind it has all files as below:
cd /home/client/files/ibm
pre-ibm-03222017
pro-ibm-03222017
cd /home/client/files/aol
PRE-aol-170322
PRO-aol-170322
And I want to COUNT ONLY the PRO/pro that will either be lower or upper case. One folder can get more than 1 file per day.
I am using the below command:
find /home/client/files -type f -mtime -1 -exec ls -1 {} \;| wc -l
But it is giving me the total count of pre and pro files and also it is counting files for last 24 hours....and not the current day only.
For Example. It is currently 09:00 PM. The above command include files received yesterday between 09:00 PM and 12:00 AM as well. I don't wan't those. In other words if I run it at 01:00 AM....it should have all files for 1 hour only and not last 24 hours.
Thanks
---- Update -----
This works great for me.
touch -t 201703230000 first
touch -t 201703232359 last
find /home/client/files/ -newer first ! -newer last | grep -i pro | wc -l
Now, I was just wondering if I can pass the above as parameter.
For example, instead of using touch -t date and alias.....I want to type shortcuts and dates only to get the output. I have made the following aliases:
alias reset='touch -t `date +%m%d0000` /tmp/$$'
alias count='find /home/client/files/ -type f -newer /tmp/$$ -exec ls -1 {} \; | grep -i pro | wc -l'
This way as soon as I logon to the server, I type reset and then I type count and I get my daily number.
I was wondering if I can do something similar for any duration of days by setting date1 and date2 as aliases. If not, then perhaps a short script that would ask for parameters.
What about this?
touch -t `date +%m%d0000` /tmp/$$
find /home/client/files -type f -newer /tmp/$$ -exec ls -1 {} \; | grep -i pro | wc -l
rm /tmp/$$
Other options for finding a file created today can be found in this question:
How do I find all the files that were created today
Actually, a better way to do this is to just use this:
find /home/client/files -type f -m 0 | grep -i pro | wc -l
You can replace -m 0 with -m 5 to find files 5 days old.
For the same day issue u can use -daystart (GNU find)
The regex define a contains of /pre
find /home/client/files -regex '.*\/[pP][rR][eE].*' -type f -daystart -mtime -1 -exec ls -1 {} \;| wc -l
I have files with name as
filename_201702200800.tar.bz2
filename_201702201800.tar.bz2
and so on
I am looking to remove files which are 5 days and older.
The date in the files is of the format %Y%m%d%H%M.
As the creation time corresponds to the names, just use find:
find /path/to/files -type f -ctime +5 -exec rm {} +
From man page:
-exec command {} +
This variant of the -exec action runs the specified command on the selected files, but the command line is built by
appending each selected file name at the end; the total number of
invocations of the command will be much less than the number of
matched files. The command line is built in much the same way that
xargs builds its command lines. Only one instance of ‘{}’ is
allowed within the command. The command is executed in the starting
directory.
Not enough rep to comment.
Can you use the mtime of the files?
find /path/to/files -type f -mtime +5 -delete
Otherwise, you could calculate the dates to find with:
date -d "5 days ago" +%Y%m%d%H%M
We have an AIX system, which gets files on a daily basis, so we manually delete the previous days files manually. Is it possible to write a script which will take the files 15 or 20 days before today and delete the files from the folder?
Or you can use native AIX find command:
find /dir/to/files -type f -mtime +15 -exec rm {} \;
where:
-type f - Find only files, not directories
-mtime +15 - Find files, that modification time more then 15 days
-exec rm {} \; - Run command rm on each matched file
You can run this command with -exec ls -l {} \; for testing, that found files correspond to your criteria.
If you can/may install GNU!find, them it's simple, e.g.:
#!/bin/sh
cd /var/log/apache
gfind . -name '*log*Z' -mtime +30 -delete
this script is run by cron; a line from crontab:
02 23 1 * * /root/cmd/httpd.logdelete >/dev/null 2>&1
Edit:
-mdays + means files of which last modification date is earlier than now-
-delete means deleting the files that match the criteria
How to Delete file created in a specific date??
ls -ltr | grep "Nov 22" | rm -- why this is not wrking??
There are three problems with your code:
rm takes its arguments on its command line, but you're not passing any file name on the command line. You are passing data on the standard input, but rm doesn't read that¹. There are ways around this.
The output of ls -ltr | grep "Nov 22" doesn't just consist of file names, it consists of mangled file names plus a bunch of other information such as the time.
The grep filter won't just catch files modified on November 22; it will also catch files whose name contains Nov 22, amongst others. It also won't catch the files you want in a locale that displays dates differently.
The find command lets you search files according to criteria such as their name matching a certain pattern or their date being in a certain range. For example, the following command will list the files in the current directory and its subdirectories that were modified today (going by GMT calendar date). Replace echo by rm -- once you've checked you have the right files.
find . -type f -mtime -2 -exec echo {} +
With GNU find, such as found on Linux and Cygwin, there are a few options that might do a better job:
-maxdepth 1 (must be specified before other criteria) limits the search to the specified directory (i.e. it doesn't recurse).
-mmin -43 matches files modified at most 42 minutes ago.
-newermt "Nov 22" matches files modified on or after November 22 (local time).
Thus:
find . -maxdepth 1 -type f -newermt "Nov 22" \! -newermt "Nov 23" -exec echo {} +
or, further abbreviated:
find -maxdepth 1 -type f -newermt "Nov 22" \! -newermt "Nov 23" -delete
With zsh, the m glob qualifier limits a pattern to files modified within a certain relative date range. For example, *(m1) expands to the files modified within the last 24 hours; *(m-3) expands to the files modified within the last 48 hours (first the number of days is rounded up to an integer, then - denotes a strict inequality); *(mm-6) expands to the files modified within the last 5 minutes, and so on.
¹ rm -i (and plain rm for read-only files) uses it to read a confirmation y before deletion.
If you need to try find for any given day,
this might help
touch -d "2010-11-21 23:59:59" /tmp/date.start;
touch -d "2010-11-23 00:00:00" /tmp/date.end;
find ./ -type f -newer /tmp/date.start ! -newer /tmp/date.end -exec rm {} \;
If your find supports it, as GNU find does, you can use:
find -type f -newermt "Nov 21" ! -newermt "Nov 22" -delete
which will find files that were modified on November 21.
You would be better suited to use the find command:
find . -type f -mtime 1 -exec echo rm {} +
This will delete all files one day old in the current directory and recursing down into its sub-directories. You can use '0' if you want to delete files created today. Once you are satisfied with the output, remove the echo and the files will truly be deleted.
for i in `ls -ltr | grep "NOV 23" | awk '{print $9}'`
do
rm -rf $i
done
Mb better
#!/bin/bash
for i in $(ls -ltr | grep "NOV 23" | awk '{print $9}')
do
rm -rf $i
done
then in the previous comment