I want to copy all the files in a directory that were modified this month. I can list those files like this:
ls -l * | grep Jul
And then to copy them I was trying to pipe the result into cp via xargs but had no success (I think) because I couldn't figure out how to parse the ls -l output to just grab the filename for cp.
I'm sure there are many ways of doing this; I'll give the correct answer out to the person who can show me how to parse ls -l in this manner (or talk me down from that position) though I'd be interested in seeing other methods as well.
Thanks!
Of course, just doing grep Jul is bad because you might have files with Jul in their name.
Actually, find is probably the right tool for your job. Something like this:
find $DIR -maxdepth 1 -type f -mtime -30 -exec cp {} $DEST/ \;
where $DIR is the directory where your files are (e.g. '.') and $DEST is the target directory.
The -maxdepth 1 flag means it doesn't look inside sub-directories for files (isn't recursive)
The -type f flag means it looks only at regular files (e.g. not directories)
The -mtime -30 means it looks at files with modification time newer than 30 days (+30 would be older than 30 days)
the -exec flag means it executes the following command on each file, where {} is replaced with the file name and \; is the end of the command
interested in seeing how this might be done with zsh
ls -lt *.*(mM0)
last month
ls -lt *.*(mM1)
or for precise date date ranges
autoload -U age
ls -tl *.*(e#age 2014/06/07 now#)
ls -tl *.*(e#age 2014/06/01 2014/06/20#)
Related
I am making a bash script to run in a directory with files generated everyday and copy the most recent file to another directory.
I am using this now
for [FILE in directory]
do
if [ls -Art | tail -n 1]
something...
else
something...
fi
done
I know this is not alright. I would like to compare the date modified of the files with the current date and if it was equal, copy that file then.
How would that work or is there an easier method to do it?
We could use find:
find . -maxdepth 1 -daystart -type f -mtime -1 -exec cp -f {} dest \;
Explanation:
-maxdepth 1 limits the search to the current directory.
-daystart sets the reference time of -mtime to the beginning of today.
-type f limits the search to files.
-mtime -1 limits the search to files that have been modified less than 1 day from reference time.
-exec cp -f {} dest \; copies the found files to directory dest.
Note that -daystart -mtime -1 means anytime after today 00:00 (included), but also tomorrow or any time in the future. So if you have files with last modification time in year 2042 they will be copied too. Use -mtime 0 if you prefer coping files that have been modified between today at 00:00 (excluded) and tomorrow at 00:00 (included).
Note also that all this could be impacted by irregularities like daylight saving time or leap seconds (not tested).
The newest file is different from file(s) modified today.
Using ls is actually a pretty simple and portable approach. The stdout output format is defined by POSIX (if not printing to a terminal), and ls -A is also in newer POSIX standards.
It should look more like this though:
newest=$(ls -At | head -n 1)
You could add -1, but it AFAIK it shouldn’t be required, as it’s not printing to a terminal.
If you don’t want to use ls, you can use this on linux:
find . -mindepth 1 -maxdepth 1 -type f -exec stat -c ‘%Y:%n’ {} + |
sort -n |
tail -n 1 |
cut -d : -f 2-
Note using 2- not 2 with cut, in case a filename contains :.
Also, the resulting file name will be a relative path (./file), or an empty string if no files exist.
Need to delete log files older than 60 days and compress it if files are greater than 30 days and lesser than 60 days. I have to remove and compress files from 2 paths as mentioned in PURGE_DIR_PATH.
Also have to take the output of find command and redirect it to log file. Basically need to create an entry in the log file whenever a file is deleted. How can i achieve this?
I have to also validate if the directory path is valid or not and put a message in log file if the directory is valid or not
I have written a shell script but doesn't cover all the scenarios. This is my first shell script and need some help. How do i keep just one variable log_retention and
use it to compress files as the condition would be >30 days and <60 days? how do I validate if directories is valid or not? is my IF condition checking that?
Please let me know.
#!/bin/bash
LOG_RETENTION=60
WEB_HOME="/web/local/artifacts"
ENG_DIR="$(dirname $0)"
PURGE_DIR_PATH="$(WEB_HOME)/backup/csvs $(WEB_HOME)/home/archives"
if[[ -d /PURGE_DIR_PATH]] then echo "/PURGE_DIR_PATH exists on your filesystem." fi
for dir_name in ${PURGE_DIR_PATH}
do
echo $PURGE_DIR_PATH
find ${dir_name} -type f -name "*.csv" -mtime +${LOG_RETENTION} -exec ls -l {} \;
find ${dir_name} -type f -name "*.csv" -mtime +${LOG_RETENTION} -exec rm {} \;
done
Off the top of my head -
#!/bin/bash
CSV_DELETE=60
CSV_COMPRESS=30
WEB_HOME="/web/local/artifacts"
PURGE_DIR_PATH=( "$(WEB_HOME)/backup/csvs" "$(WEB_HOME)/home/archives" ) # array, not single string
# eliminate the oldest
find "${PURGE_DIR_PATH[#]}" -type f -name "*.csv" -mtime +${CSV_DELETE} |
xargs -P 100 rm -f # run 100 in bg parallel
# compress the old-enough after the oldest are gone
find "${PURGE_DIR_PATH[#]}" -type f -name "*.csv" -mtime +${CSV_COMPRESS} |
xargs -P 100 gzip # run 100 in bg parallel
Shouldn't need loops.
ls -d "$PWD"/* > a.txt
While I'm trying to run this script this script getting timeout.
Use find instead of ls to avoid delay for sorting:
find "$PWD" -depth 1 -print >files.txt
Even better, change your output format to be NUL-delimited, and then your output will be unambiguous even with files having surprising names (touch $'foo\nbar' if you want to create one of those to play with):
find "$PWD" -depth 1 -print0 >files.nsv
I have a folder /var/backup where a cronjob saves a backup of a database/filesystem. It contains a latest.gz.zip and lots of older dumps which are names timestamp.gz.zip.
The folder ist getting bigger and bigger and I would like to create a bash script that does the following:
Keep latest.gz.zip
Keep the youngest 10 files
Delete all other files
Unfortunately, I'm not a good bash scripter so I have no idea where to start. Thanks for your help.
In zsh you can do most of it with expansion flags:
files=(*(.Om))
rm $files[1,-9]
Be careful with this command, you can check what matches were made with:
print -rl -- $files[1,-9]
You should learn to use the find command, possibly with xargs, that is something similar to
find /var/backup -type f -name 'foo' -mtime -20 -delete
or if your find doesn't have -delete:
find /var/backup -type f -name 'foo' -mtime -20 -print0 | xargs -0 rm -f
Of course you'll need to improve a lot, this is just to give ideas.
Would like to create a shell script that will read the contents of a folder that contains many folders on a server- it should output a list of these folders with their size and date modified if possible.
If you want to do it recursively (it's not clear to me from the question whether or not you do), you can do:
$ find /path/to/dir -type d -exec stat -c '%n: %s: %y' {} \;
(If you have a find which supports the feature, you can replace '\;' with '+')
Note that the %s gives the size of the directory, which is not the number of files in the directory, nor is it the disk usage of the files in the directory.
ls -l /path/to/folder | grep ^d
Try this find command to list sub-directories and their size (since stat command doesn't run same way on mac and Linux):
#!/bin/bash
find /your/base/dir -type d -exec du -k {} \; > sizes.txt