I need to compare the size of the file generated today with the one of yesterday (everyday a new one is generated - huge in size)
fileA_20150716.log
fileB_20150717.log
fileC_20150718.log
fileD_20150719.log
What would be the best approach for this (new to coding)?
I found this code but I think it is not good for leap years, etc.
prev_date=`TZ=bb24 date +%Y%m%d`
echo $prev_date
You could start from this Bash script and expand it to your needs:
#!/bin/bash
A=(*.log)
for ((i = 0; i < ${#A[#]} - 1; i++)); do
diff "${A[i]}" "${A[i + 1]}"
done
It does not use dates but rather compares adjacent pairs of log files. You can replace the diff command with something more suitable for you.
I need to compare the size of the file generated today with the one of yesterday
Instead of constructing the filename for yesterday's file from today's date (which is tricky, as you found out), why not simply look for a file with the correct name (pattern) and a modification time of yesterday? That's much easier:
TODAYSSIZE=`find . -name "file*\.log" -daystart -mtime 0 -printf "%s"`
YESTERSIZE=`find . -name "file*\.log" -daystart -mtime 1 -printf "%s"`
Then do with the values whatever you need to do.
Tweak search path (.), file name pattern (file*\.log) and the actual size format displayed (%s) to your requirements.
This is assuming you have GNU find available; the find shipped with AIX doesn't do -printf. You can still use it to get the filenames, though:
TODAYSFILE=`find . -name "file*\.log" -daystart -mtime 0`
YESTERFILE=`find . -name "file*\.log" -daystart -mtime 1`
Then retrieve the file size any way you like (ls -s $TODAYSFILE or whatever).
Note that find works recursively, i.e. it would find logfiles in subdirectories as well. GNU find can be told to -maxdepth 1 to avoid this, AIX find cannot.
#!/usr/bin/bash
a=$(date +%Y%m%d)
b=$(date -d yesterday +%Y%m%d)
for f in *$a.log
do
wc $f ; wc ${f/$a/$b}
done
Related
I am making a bash script to run in a directory with files generated everyday and copy the most recent file to another directory.
I am using this now
for [FILE in directory]
do
if [ls -Art | tail -n 1]
something...
else
something...
fi
done
I know this is not alright. I would like to compare the date modified of the files with the current date and if it was equal, copy that file then.
How would that work or is there an easier method to do it?
We could use find:
find . -maxdepth 1 -daystart -type f -mtime -1 -exec cp -f {} dest \;
Explanation:
-maxdepth 1 limits the search to the current directory.
-daystart sets the reference time of -mtime to the beginning of today.
-type f limits the search to files.
-mtime -1 limits the search to files that have been modified less than 1 day from reference time.
-exec cp -f {} dest \; copies the found files to directory dest.
Note that -daystart -mtime -1 means anytime after today 00:00 (included), but also tomorrow or any time in the future. So if you have files with last modification time in year 2042 they will be copied too. Use -mtime 0 if you prefer coping files that have been modified between today at 00:00 (excluded) and tomorrow at 00:00 (included).
Note also that all this could be impacted by irregularities like daylight saving time or leap seconds (not tested).
The newest file is different from file(s) modified today.
Using ls is actually a pretty simple and portable approach. The stdout output format is defined by POSIX (if not printing to a terminal), and ls -A is also in newer POSIX standards.
It should look more like this though:
newest=$(ls -At | head -n 1)
You could add -1, but it AFAIK it shouldn’t be required, as it’s not printing to a terminal.
If you don’t want to use ls, you can use this on linux:
find . -mindepth 1 -maxdepth 1 -type f -exec stat -c ‘%Y:%n’ {} + |
sort -n |
tail -n 1 |
cut -d : -f 2-
Note using 2- not 2 with cut, in case a filename contains :.
Also, the resulting file name will be a relative path (./file), or an empty string if no files exist.
I know I can use the find command with options like -mtime and -ctime, but those expect a number to be set in the command.
In my case I don't care what the time is, I just want to find any files where the -ctime and -mtime are equal to each other. (Im on a Mac so technically its -mtime and -Btime)
Im having a harder time than I expected finding how to do this.
Edit: I’m trying to do this in macOS and the file system is APFS
As you can see here, creation time is not really stored on Unix-like systems. Some filesystems may support this feature and you can check the output of stat file command, for me the last line of this output is Birth: -. So in case you do have creation times, you could get files never modified by this:
find . -type f -print0 | xargs -0 stat -c "%n %W %Y" |
awk '$NF==$(NF-1) {$(NF-1)=$NF=""; print}'
%W will print birth time (probably 0 if not supported) and %Y the last modification time. The last awk command above prints only filenames where these times are matching.
for macOS:
find . -type f -print0 | xargs -0 stat -f "%N %B %m" |
awk '$NF==$(NF-1) {$(NF-1)=$NF=""; print}'
see also macOS stat man page
I think this is not possible with just find, but you may filter these files using external tools, e.g. shell:
find . -type f -printf '%T# %C# %f\n' | while read mtime ctime fname; do
[ "$mtime" == "$ctime" ] && echo "$fname"
done
I'm trying to find the particular below files in the directory using find command pattern in shell script .
The below files will create in the directory "/data/output" in the below format every time.
PO_ABCLOAD0626201807383269.txt
PO_DEF 0626201811383639.txt
So I need to find the above txt files starting from "PO_ABCLOAD" and "PO_DEF" is created or not.if not create for four hours then I need to write in logs.
I written script but I am stuck up to find the file "PO_ABCLOAD" and "PO_DEF format text file in the below script.
Please help on this.
What changes i need to add in the find command.
My script is:
file_path=/data/output
PO_count='find ${file_path}/PO/*.txt -mtime +4 -exec ls -ltr {} + | wc -l'
if [ $PO_count == 0 ]
then
find ${file_path}/PO/*.xml -mtime +4 -exec ls -ltr {} + >
/logs/test/PO_list.txt
fi
Thanks in advance
Welcome to the forum. To search for files which match the names you are looking for you could try the -iname or -name predicates. However, there are other issues with your script.
Modification times
Firstly, I think that find's -mtime test works in a different way than you expect. From the manual:
-mtime n
File's data was last modified n*24 hours ago.
So if, for example, you run
find . -mtime +4
you are searching for files which are more than four days old. To search for files that are more than four hours old, I think you need to use the -mmin option instead; this will search for files which were modified a certain number of minutes ago.
Command substitution syntax
Secondly, using ' for command substitution in Bash will not work: you need to use backticks instead - as in
PO_COUNT=`find ...`
instead of
PO_COUNT='find ...'
Alternatively - even better (as codeforester pointed out in a comment) - use $(...) - as in
PO_COUNT=$(find ...)
Redundant options
Thirdly, using -exec ls -ltr {} + is redundant in this context - since all you are doing is determining the number of lines in the output.
So the relevant line in your script might become something like
PO_COUNT=$(find $FILE_PATH/PO/ -mmin +240 -a -name 'PO_*' | wc -l)
or
PO_COUNT=$(find $FILE_PATH/PO/PO_* -mmin +240 | wc -l)
If you wanted tighter matching of filenames, try (as per codeforester's suggestion) something like
PO_COUNT=$(find $file_path/PO/PO_* -mmin +240 -a \( -name 'PO_DEF*' -o -name 'PO_ABCLOAD*' \) | wc -l)
Alternative file-name matching in Bash
One last thing ...
If using bash, you can use brace expansion to match filenames, as in
PO_COUNT=$(find $file_path/PO/PO_{ABCLOAD,DEF}* -mmin +240 | wc -l)
Although this is slightly more concise, I don't think it is compatible with all shells.
I have to write a shell script that should copy the latest file in to a target directory. I use following shell command.
find . -type f -daystart -mtime -$dateoffset
It gives me the latest file set. But i need to get the latest file from that list and copy it to a target directory.
Thanks.
I can't think of a way to do this in Bourne shell, since you need to use a tool that actually reads datestamps and sorts them, and Bourne shell doesn't do that.
But here's a solution in PHP:
<?php
$fdate=array();
foreach(glob("*") as $filename)
$fdate[filemtime($filename)]=$filename;
krsort($fdate);
print "Newest item: " . reset($fdate) . "\n";'
?>
And if you hapen to be using bash instead of Bourne, he's a round-about way of getting what you want using an associative array:
#!/usr/local/bin/bash
declare -A fdate
highest=0
for file in *; do
timestamp=$(stat -f '%m' "$file")
fdate[$timestamp]="$file"
if [ "$timestamp" -gt "$highest" ]; then
highest=$timestamp
fi
done
printf "Newest file: %s\n" "${fdate[$highest]}"
Note that I'm using FreeBSD, so this solution will also work in OSX, but if you happen to be using Linux, you'll need to figure out how your implementation of the stat command differs from mine. (Hint: you may be able to use stat -c '%y', but man stat to be sure. Solaris, HP/UX, OSF/1, etc do not seem to include a stat binary that can be called from your shell.)
Update: #ghoti's neat solution is recommended over this one. The following has been proved nonrobust. It is left here only because, as a partial answer, it might point the way toward a better one-line solution.
ls -1dt $(find . -type f -daystart -mtime -$dateoffset) | head -n1
To copy the file to $TARGET_DIR,
A=$(ls -1dt $(find . -type f -daystart -mtime -$dateoffset) | head -n1)
if [ -n "$A" ] cp -u "$A" "$TARGET_DIR/$(basename $A)"
find . -name "*" -type f -daystart -mtime -$dateoffset | xargs -i mv {} /where/to/put/files
or
mv `find . -name "*" -type f -daystart -mtime -$dateoffset` /where/to/put/files
If you use ls command with -lt option then it will give you newest file at top.
So using this you can easily extract latest file name
You can use something like this:
find . -type f -name "*" -mtime +x_NUMBER_OF_DAYS|ls -lrt|awk -F' ' '{print $(COLUMN_NUMBER_IN_WHICH_FILE_NAME_APPEARS)}'|tail -1
This will give you the latest file till a given date.
As somebody suggested above daystart is only present in GNU flavor of find while -mtime is a more generic command.
P.S.: This again suffers from parsing problem if file name has space in it.But till we come-up with something more creative you can use this!
I'm looking to combine find . -mtime 0
and ls -lt
To find all files modified in the last day in the current working directory, sorted by last modification date.
It sounds like you want command substitution which is done with $(command). It takes the output of a command and allows you to use it as command line arguments for another command:
ls -lt $(find . -mtime 0)