I'm trying to construct a reliable shell script to remove older files based on Xn of days using find. However, the script seems to work intermittently. Is there a better way? I list the files first to make sure I capture them, then use -exec rm{} to delete them.
I execute the script like so:
/home/scripts/rmfiles.sh /u05/backup/export/test dmp 1
#!/usr/bin/ksh
if [ $# != 3 ]; then
echo "Usage: rmfiles.sh <directory> <log|dmp|par> <numberofdays>" 2>&1
exit 1
fi
# Declare variables
HOURDATE=`date '+%Y%m%d%H%M'`;
CLEANDIR=$1;
DELETELOG=/tmp/cleanup.log;
echo "Listing files to remove..." > $DELETELOG 2>&1
/usr/bin/find $CLEANDIR -name "*.$2" -mtime +$3 -exec ls -ltr {} \; > $DELETELOG 2>&1
echo "Removing files --> $HOURDATE" > $DELETELOG 2>&1
#/usr/bin/find $CLEANDIR -name "*.$2" -mtime +$3 -exec rm {} \; > $DELETELOG 2>&1
My sample directory clearly has files older than one day as of today, but find is not picking it up when it was before during some previous testing.
Thu Sep 26 08:54:57 PDT 2013
total 161313630
-rw------- 1 oracle dba 10737418240 Sep 24 14:17 testexp01.dmp
-rw------- 1 oracle dba 10737418240 Sep 24 14:20 testexp02.dmp
-rw------- 1 oracle dba 10737418240 Sep 24 14:30 testexp03.dmp
-rw------- 1 oracle dba 508 Sep 24 15:41 EXPORT-20130924.log
-rw------- 1 oracle dba 509 Sep 25 06:00 EXPORT-20130925.log
-rw------- 1 oracle dba 508 Sep 26 08:30 EXPORT-20130926.log
Apart from a couple of small issues, the script looks good in general. My guess is that you want to add -daystart to the list of options so the base for the -mtime test is measured "from the beginning of today rather than from 24 hours ago. This option only affects tests which appear later on the command line."
If you have GNU find, then try find -D tree,search,stat,rates to see what is going on.
Some comments:
Always quote variables to make sure odd spaces don't have an effect: /usr/bin/find "$CLEANDIR" -name "*.$2" -mtime "+$3" .... Same with CLEANDIR="$1"
Don't terminate lines with ;, it's bad style.
You can replace -exec ls -ltr {} \; with -ls or -print. That way, you don't have to run the find command twice.
You should quote {} since some shells interpret them as special characters.
man find
-mtime mentions the read the comment at -atime
"When find figures out how many 24-hour periods ago the file was last accessed, any fractional part is ignored, so to match -atime +1, a file has to have been accessed at least two days ago." so this is also true for -mtime.
Related
I'm trying to compose a command/function to list any directory displaying the files with the following rules, from top level to deepest level:
hidden files
directories
alphabetically
Another feature I want to grant is to be able to use the command/function for a passed directory. Example:
Goal
Basically I'd like to input something like:
user#machine:cuur-path $ my_listing_magic /some/crazy/directory
and get an output looking like:
drwxrwxr-x 6 User Group 4,0K abr 8 12:12 .aaa_dir/
drwxrwxr-x 3 User Group 4,0K abr 8 12:12 .bbb_dir/
drwxrwxr-x 3 User Group 4,0K out 14 2020 .ccc_dir/
-rw-rw-r-- 1 User Group 4,1K abr 12 17:44 .a_file
-rw-r--r-- 1 User Group 25 dez 4 2017 .b_file
-rw-rw-r-- 1 User Group 21 mai 20 15:50 .c_file
drwx------ 44 User Group 4,0K mai 27 16:44 ddd_dir/
drwx------ 3 User Group 4,0K abr 5 2018 eee_dir/
drwx------ 3 User Group 4,0K abr 5 2018 fff_dir/
-rw-r--r-- 1 User Group 4,8K jun 1 18:52 d_file
-rw-r--r-- 1 User Group 1,2K jun 1 19:42 e_file
-rw------- 1 User Group 106K jun 1 19:33 f_file
(note that the directories . and .. are not there.)
Failed Atempts
Just use ls options
The closest I've gotten is ls -alhvF --group-directories-first. It is almost there, but doesn't respect the desired order and keeps both . and ... I coulld strip those out a posteriori, but since the ordering is not correct, there is no point.
ls -lv namely the -v is nice as it sorts all the .files/dirs up alphabetical.
ls -l --goups-directories-first is also cool, does what is expected. To bad I can't make it 'act before' the -v.
Divide and Conquer
I though of splitting up the problem into two: I could first list the hidden files and then the not hidden: list_hidden; list_non_hiden. Then for simplicity of usage I could create an alias or function to call them in proper order.
The non-hidden is quite trivial! But listing solely the hidden files is becoming a bit more troublesome. I tried something like ls -ld .!(|.), but this has the drawback of using -d - so it cannot be used to list some other directory. At least not without having the full path of the file there...
Thanks!
The ls utility does not support what you want. So implement it yourself. List the files, then sort them, then ls them, yourself. A script that uses zero-terminated stream and GNU coreutils and findutils could look like the following:
{
printf "%s\0" . ..;
cmd=(
find "${1:-.}" -mindepth 1 -maxdepth 1 '('
-name '.*' '('
-type d -printf "1 %p\0"
')' -o
-printf "2 %p\0"
')' -o '('
'!' -name '.*' '('
-type d -printf "3 %p\0"
')' -o
-printf "4 %p\0"
')'
)
"${cmd[#]}" |
sort -z |
cut -z -d' ' -f2-;
} | xargs -0 ls -ald
You can patch some open-source ls utility with your own behavior - it would be nice and I am in favor of implementing it inside GNU coreutils ls as yet another option like --sort-hidden-directories-first to sort them in front of hidden files.
You can also write your own ls utility in C (or you could reimplement ls in shell, but I suspect It would be harder then just writing it in C).
Because I liked the idea for a long time and wanted to have hidden files sorted before anything else, I created this l utility that uses the method described above.
I loaded a find command into autosys. It zips anythin older than 15 days old.
The script only has two lines.
#!/bin/bash
find /casper/dir/usa.* -type f -mtime +15 -exec gzip --verbose {} \;
Problem is that is finished successfully immediately, but continues to run and writes everything to the error file.
casperrd#usa04 1026$ ls -ltr /unity_apps/casperrd/logs/autosys/*CASPER_JOB_AUTOSYS*
-rw-r--r-- 1 capsergrp casper 0 Aug 21 16:35 /unity_apps/casperrd/logs/autosys/CAPSER_JOB_AUTOSYS_20180821_20:35:20.out
-rw-r--r-- 1 capsergrp casper 662 Aug 21 16:43 /unity_apps/casperrd/logs/autosys/CAPSER_JOB_AUTOSYS_20180821_20:35:20.err
casperrd#usa04 1027$
I need to show what files are zipped, and want it to write to the .out file, not .err file
cat /unity_apps/casperrd/logs/autosys/CAPSER_JOB_AUTOSYS_20180821_20:35:20.err
/casper/log/casperjob.20180622.txt: 94.2% -- replaced with /casper/log/casperjob.20180622.txt.gz
/casper/log/casperjob.20180625.csv: 74.6% -- replaced with /casper/log/casperjob.20180625.csv.gz
/casper/log/casperjob.20180625.txt: 94.2% -- replaced with /casper/log/casperjob.20180625.txt.gz
/casper/log/casperjob.20180626.csv:
Sorry if this is a simple question, but I'm new to Bash scripting and my Google skills are failing me. I am trying to write a script that ultimately will examine the last modified time for every file in a directory, and if a file was last modified > 3 days ago, then compress the file with gzip. I can currently print out the last modified time with my script here:
1 !#/bin/bash
2 echo "Enter directory"
3
4 read directory
5
6 for entry in "$directory"/*
7 do
8 stat -f "%Sm" "$entry"
9 done
This prints out the times just fine:
Randalls-MacBook-Pro:bash_scripts randallbanks$ ./print_last_modified.sh
./print_last_modified.sh: line 1: !#/bin/bash: No such file or directory
Enter directory
..
Apr 6 13:12:21 2015
Apr 19 18:50:26 2015
Apr 14 11:29:06 2015
Apr 7 12:26:37 2015
Apr 15 16:05:17 2015
Apr 6 16:28:06 2015
Apr 6 12:28:40 2015
Apr 6 12:28:40 2015
Apr 6 12:28:40 2015
Apr 6 12:28:40 2015
Apr 23 17:03:03 2015
But what I'm having trouble figuring out is how to store or parse these in such a way that I can check if their last modified time was > 3 days ago. How can I go about this?
Edit: bonus question: What is up with it apparently not recognizing my shebang in the header?
To gzip files that were last modified 3 or more days ago, use:
find "$directory" -mtime +2 -type f -exec gzip {} \;
If your find is modern enough, use:
find "$directory" -mtime +2 -type f -exec gzip {} +
How it works
find "$directory"
This starts find looking for files in the directory tree under $directory.
-mtime +2
This looks for files that were modified more than 2 days old. (More than 2 means three or more). See man find for details on how find computes age rounded to days.
-type f
This restricts find to looking for regular files.
-exec gzip {} \;
This tells find to gzip the files that it finds.
Your shebang is a bangshe. It should be #!; it is !#.
The find command on OS X (and BSDs, generally) can do what you're looking for. The -newermt checks the modification time against a time specified in a string, including such strings as "3 days ago".
So:
find . -not -newermt "3 days ago" -exec gzip {} \;
Could I please know how I can print contents of the file with same extension (for example, .coords) in multiple directories to a text file using a shell script, confining to specific date and time of the directory created.
Would be thankful to your replies.
EDIT:
ls -ltr
total 16
drwxrwxr-x 2 prakkisr prakkisr 4096 Jul 28 13:23 A
drwxrwxr-x 2 prakkisr prakkisr 4096 Jul 29 09:56 B
drwxrwxr-x 2 prakkisr prakkisr 4096 Jul 31 12:15 C
drwxrwxr-x 2 prakkisr prakkisr 4096 Jul 31 14:34 D
All the folders A,B,C,D have a file which ends with .coords (a.coords in A folder, b.coords in B folder etc..)
Firstly, I want only those folders generated on Jul 31 (i.e C and D folder) to be accessed and want to print the contents of c.coords and d.coords files in those folder into a text file. ( this is according to date)
Secondly, Is it possible to print it according to time. Like suppose, I want only those ".coords" file from the folder (in this case 'D' folder), which are generated after time 14:00 today and get printed into another file. (this is according to date as well as time)
The following command will print the contents of all *.coords files that are in directories with a modification date within the last day:
find . -type d -mtime 0 -exec sh -c 'cat {}/*.coords 2>/dev/null' \;
If you wanted to see the names of the *.coords files rather than their content, then use:
find . -type d -mtime 0 -exec sh -c 'ls {}/*.coords 2>/dev/null' \;
The age of the directory can be specified in many other ways. For example:
To specify the directories age in minutes, use -mmin in place of -mtime.
To specify the directories creation date, rather than its last modification date, use -cmin or -ctime.
If your file system supports it, it is also possible to select directories based on their last access time. Use -amin or -atime.
It is also possible to select directories based some range in times by prepending the age with a + or - sign. To select directories with a creation date more recent than two days, use -ctime -2. By including two such specifiers, you can select from a range of dates.
See man find for full details.
Variation
Suppose that we want to search based on the date of the file, rather than the date of the directory in which the file resides. In this case, a simpler command may be used to print the contents of the matching files:
find . -name '*.coords' -mtime 0 -exec cat {} \;
Suppose that we want to both print the file's name and its contents. Then, we include to actions to the find command:
find . -name '*.coords' -mtime 0 -print -exec cat {} \;
Note the use of quotation marks around *.coords. This assures that the command will work in case that the current directory happens to have .coords file in it.
I have a directory that contains sub-directories and other files and would like to update the date/timestamps recursively with the date/timestamp of another file/directory.
I'm aware that:
touch -r file directory
changes the date/timestamp for the file or directory with the others, but nothing within it. There's also the find version which is:
find . -exec touch -mt 201309300223.25 {} +\;
which would work fine if i could specify the actual file/directory and use anothers date/timestamp. Is there a simple way to do this? even better, is there a way to avoid changing/updating timestamps when doing a 'cp'?
even better, is there a way to avoid changing/updating timestamps when doing a 'cp'?
Yes, use cp with the -p option:
-p
same as --preserve=mode,ownership,timestamps
--preserve
preserve the specified attributes (default:
mode,ownership,timestamps), if possible additional attributes:
context, links, xattr, all
Example
$ ls -ltr
-rwxrwxr-x 1 me me 368 Apr 24 10:50 old_file
$ cp old_file not_maintains <----- does not preserve time
$ cp -p old_file do_maintains <----- does preserve time
$ ls -ltr
total 28
-rwxrwxr-x 1 me me 368 Apr 24 10:50 old_file
-rwxrwxr-x 1 me me 368 Apr 24 10:50 do_maintains <----- does preserve time
-rwxrwxr-x 1 me me 368 Sep 30 11:33 not_maintains <----- does not preserve time
To recursively touch files on a directory based on the symmetric file on another path, you can try something like the following:
find /your/path/ -exec touch -r $(echo {} | sed "s#/your/path#/your/original/path#g") {} \;
It is not working for me, but I guess it is a matter of try/test a little bit more.
In addition to 'cp -p', you can (re)create an old timestamp using 'touch -t'. See the man page of 'touch' for more details.
touch -t 200510071138 old_file.dat