Purpose of the script :
1.This script will delete Files older than 4 months.
2.Files older than 3 days will be compressed.
A script has been written such as :
#!/bin/bash
exec >> /dir5/dir6/cleanup-logfiles.log 2>&1
# customer list job
cd /dir1/dir2/dir3/dir4/tmp
find -type f -mtime +120 -exec rm -v '{}' \;
find -type f -mtime +3 -name '*.csv' -exec gzip -v '{}' \;
Can anyone please explain usage of both the above commands (and how do they serve the purpose ?
And this script has been placed at /etc/. what could be the reason ?
exec without a command parameter redirects all output (stdout + stderr [2>&1]) from the current shell (i.e. this script) to /dir5/dir6/cleanup-logfiles.log
cd changes directory ;)
the find commands will find all files (-type f) whose modified time (-mtime) is older than 120, respectively 3 days and: delete them (-exec rm -v '{}' \;) or gzip them (-exec gzip -v '{}' \;). gzipping only happens when the file has a csv extension (-name '*.csv')
{} is a placeholder for the currently found file
the script is probably run through cron (/etc/cron.{d,daily,hourly,weekly,monthly} or /etc/crontab)
Related
Need to delete log files older than 60 days and compress it if files are greater than 30 days and lesser than 60 days. I have to remove and compress files from 2 paths as mentioned in PURGE_DIR_PATH.
Also have to take the output of find command and redirect it to log file. Basically need to create an entry in the log file whenever a file is deleted. How can i achieve this?
I have to also validate if the directory path is valid or not and put a message in log file if the directory is valid or not
I have written a shell script but doesn't cover all the scenarios. This is my first shell script and need some help. How do i keep just one variable log_retention and
use it to compress files as the condition would be >30 days and <60 days? how do I validate if directories is valid or not? is my IF condition checking that?
Please let me know.
#!/bin/bash
LOG_RETENTION=60
WEB_HOME="/web/local/artifacts"
ENG_DIR="$(dirname $0)"
PURGE_DIR_PATH="$(WEB_HOME)/backup/csvs $(WEB_HOME)/home/archives"
if[[ -d /PURGE_DIR_PATH]] then echo "/PURGE_DIR_PATH exists on your filesystem." fi
for dir_name in ${PURGE_DIR_PATH}
do
echo $PURGE_DIR_PATH
find ${dir_name} -type f -name "*.csv" -mtime +${LOG_RETENTION} -exec ls -l {} \;
find ${dir_name} -type f -name "*.csv" -mtime +${LOG_RETENTION} -exec rm {} \;
done
Off the top of my head -
#!/bin/bash
CSV_DELETE=60
CSV_COMPRESS=30
WEB_HOME="/web/local/artifacts"
PURGE_DIR_PATH=( "$(WEB_HOME)/backup/csvs" "$(WEB_HOME)/home/archives" ) # array, not single string
# eliminate the oldest
find "${PURGE_DIR_PATH[#]}" -type f -name "*.csv" -mtime +${CSV_DELETE} |
xargs -P 100 rm -f # run 100 in bg parallel
# compress the old-enough after the oldest are gone
find "${PURGE_DIR_PATH[#]}" -type f -name "*.csv" -mtime +${CSV_COMPRESS} |
xargs -P 100 gzip # run 100 in bg parallel
Shouldn't need loops.
I am searching specific directory and subdirectories for new files, I will like to copy the files. I am using this:
find /home/foo/hint/ -type f -mtime -2 -exec cp '{}' ~/new/ \;
It is copying the files successfully, but some files have same name in different subdirectories of /home/foo/hint/.
I will like to copy the files with its base directory to the ~/new/ directory.
test#serv> find /home/foo/hint/ -type f -mtime -2 -exec ls '{}' \;
/home/foo/hint/do/pass/file.txt
/home/foo/hint/fit/file.txt
test#serv>
~/new/ should look like this after copy:
test#serv> ls -R ~/new/
/home/test/new/pass/:
file.txt
/home/test/new/fit/:
file.txt
test#serv>
platform: Solaris 10.
Since you can't use rsync or fancy GNU options, you need to roll your own using the shell.
The find command lets you run a full shell in your -exec, so you should be good to go with a one-liner to handle the names.
If I understand correctly, you only want the parent directory, not the full tree, copied to the target. The following might do:
#!/usr/bin/env bash
findopts=(
-type f
-mtime -2
-exec bash -c 'd="${0%/*}"; d="${d##*/}"; mkdir -p "$1/$d"; cp -v "$0" "$1/$d/"' {} ./new \;
)
find /home/foo/hint/ "${findopts[#]}"
Results:
$ find ./hint -type f -print
./hint/foo/slurm/file.txt
./hint/foo/file.txt
./hint/bar/file.txt
$ ./doit
./hint/foo/slurm/file.txt -> ./new/slurm/file.txt
./hint/foo/file.txt -> ./new/foo/file.txt
./hint/bar/file.txt -> ./new/bar/file.txt
I've put the options to find into a bash array for easier reading and management. The script for the -exec option is still a little unwieldy, so here's a breakdown of what it does for each file. Bearing in mind that in this format, options are numbered from zero, the {} becomes $0 and the target directory becomes $1...
d="${0%/*}" # Store the source directory in a variable, then
d="${d##*/}" # strip everything up to the last slash, leaving the parent.
mkdir -p "$1/$d" # create the target directory if it doesn't already exist,
cp "$0" "$1/$d/" # then copy the file to it.
I used cp -v for verbose output as shown in "Results" above, but IIRC it's also not supported by Solaris, and can be safely ignored.
The --parents flag should do the trick:
find /home/foo/hint/ -type f -mtime -2 -exec cp --parents '{}' ~/new/ \;
Try testing with rsync -R, for example:
find /your/path -type f -mtime -2 -exec rsync -R '{}' ~/new/ \;
From the rsync man:
-R, --relative
Use relative paths. This means that the full path names specified on the
command line are sent to the server rather than just the last parts of the
filenames.
The problem with the answers by #Mureinik and #nbari might be that the absolute path of new files will spawn in the target directory. In this case you might want to switch to the base directory before the command and go back to your current directory afterwards:
path_current=$PWD; cd /home/foo/hint/; find . -type f -mtime -2 -exec cp --parents '{}' ~/new/ \; ; cd $path_current
or
path_current=$PWD; cd /home/foo/hint/; find . -type f -mtime -2 -exec rsync -R '{}' ~/new/ \; ; cd $path_current
Both ways work for me at a Linux platform. Let’s hope that Solaris 10 knows about rsync’s -R ! ;)
I found a way around it:
cd ~/new/
find /home/foo/hint/ -type f -mtime -2 -exec nawk -v f={} '{n=split(FILENAME, a, "/");j= a[n-1];system("mkdir -p "j"");system("cp "f" "j""); exit}' {} \;
We have an AIX system, which gets files on a daily basis, so we manually delete the previous days files manually. Is it possible to write a script which will take the files 15 or 20 days before today and delete the files from the folder?
Or you can use native AIX find command:
find /dir/to/files -type f -mtime +15 -exec rm {} \;
where:
-type f - Find only files, not directories
-mtime +15 - Find files, that modification time more then 15 days
-exec rm {} \; - Run command rm on each matched file
You can run this command with -exec ls -l {} \; for testing, that found files correspond to your criteria.
If you can/may install GNU!find, them it's simple, e.g.:
#!/bin/sh
cd /var/log/apache
gfind . -name '*log*Z' -mtime +30 -delete
this script is run by cron; a line from crontab:
02 23 1 * * /root/cmd/httpd.logdelete >/dev/null 2>&1
Edit:
-mdays + means files of which last modification date is earlier than now-
-delete means deleting the files that match the criteria
In my script I have the following 3 commands
Basically what it is trying to do is:
create a symlink to a certain bunch of files based on their filenames, in a temp directory.
change the name of the symlink to match the current date
move the symlinks from a temp directory to their proper location
-
find . -type f -name "*${regex}-*" -exec ln -s {} "${DataTempPath}/"{} \;
find "$DataTempPath" -type l | sed -e "p;s/A[0-9]*/A${today}/" | xargs -n2 mv
mv $DataTempPath/* $DataSetPath
This will be inserted as a cron job to run every 15 mins, which is not a problem when the source directory contains valid data.
However when it doesn't contain any files I get errors on the second find command and the mv command
What I want I guess is a way of not executing the last two lines of the script if the first one does not create any new links
GNU xargs supports a --no-run-if-empty parameter that, to quote the documentation "If the standard input is completely empty, do not run the command. By default, the command is run once even if there is no input".
This should help avoid the xargs error (assuming you are running GNU xargs)
check the status of the command:
find . -type f -name "*${regex}-*" -exec ln -s {} "${DataTempPath}/"{} \;
if [[ $? == 0 ]]; then
find "$DataTempPath" -type l | sed -e "p;s/A[0-9]*/A${today}/" | xargs -n2 mv
mv $DataTempPath/* $DataSetPath
fi
I cannot delete folder created by php mkdir
for I in `echo $*`
do
find $I -type f -name "sess_*" -exec rm -f {} \;
find $I -type f -name "*.bak" -exec rm -f {} \;
find $I -type f -name "Thumbs.db" -exec rm -f {} \;
find $I -type f -name "error.log" -exec sh -c 'echo -n > "{}"' -f {} \;
find $I -type f -path "*/cache/*" -name "*.*" -exec rm -f {} \;
find $I -path "*/uploads/*" -exec rm -rdf {} \;
done
I want to delete under /uploads/ all files and folders please help me thanks...
You should consider changing your find command to use the -o pragma to join your conditions together as the final exec is basically the same. This will avoid recursing the file system repeatedly.
The other answers address your concern about php mkdir. I'll just add that it has nothing to do with the fact it was created with php mkdir rather than any other code or command. It is due to the ownership and permissions.
I think this is most likely because php is running in apache or another http server under a different user than you are invoking the bash script. Or perhaps the files uploaded in uploads/ are owned by the http server's user and not the user invoking it.
Make sure that you run the bash script under the same user as your http server.
To find out which user owns which file do:
ls -l
If you run you bash script as root, you should be able to delete it anyway, but that is not recommended.
Update
To run it as root for nautilus script use the following as your nautilus script:
gksudo runmydeletescript
Then put all the other code into another file with the same path as whatever you have put for runmydeletescript and run chmod +x on it. This is extremely dangerous!
You should probably add -depth to the command to delete sub-directories of upload before the directory itself.
I worry about the -path but I'm not familiar with it.
Also consider using + instead of \; to reduce the number of commands executed.