I'm working on a script that takes a date in the format YYYMMDD as an input:
#!/bin/bash
check_dir () {
d1=$2
d2=$((d1+1))
f1=`mktemp`
f2=`mktemp`
touch -d $d1 $f1
touch -d $d2 $f2
n=$(find $1 -mindepth 1 \( -name "*$d1*" \) -o \( -newer $f1 ! -newer $f2 \) | wc -l)
if [ $n != $3 ]; then echo $1 "=" $n ; fi
rm -f $f1 $f2
}
In this script d2=$((d1+1)) calculates the next day's date from the date provide i.e., d1. But, what if the date is 20151231, then it will not be able to handle this situation. Could anyone please help me how to handle this exception?
gdate does not work on my platform!!
With the date version from GNU coreutils (i.e. used in Linux):
$ date -d '20151231 +1 day' +'%Y%m%d'
20160101
Related
I am currently stuck with a problem in my Bash script and seem to run even deeper in the dark with every attempt of trying to fix it.
Background:
We have a folder which is getting filled with numbered crash folders, which get filled with crash files. Someone is exporting a list of these folders on a daily basis. During that export, the numbered crash folders get an attribute "user.exported=1".
Some of them do not get exported, so they will not have the attribute and these should be deleted only if they are older than 30 days.
My problem:
I am setting up a bash script, which is being run via Cron in the end to check on a regular basis for folders, which have the attribute "user.exported=1" and are older than 14 days and deletes them via rm -rfv FOLDER >> deleted.log
We however also have folders which do not have or get the attribute "user.exported=1" which then need to be deleted after they are older than 30 days. I created an IF ELIF FI comparison to check for that but that is where I got stuck.
My Code:
#!/bin/bash
# Variable definition
LOGFILE="/home/crash/deleted.log"
DATE=`date '+%d/%m/%Y'`
TIME=`date '+%T'`
FIND=`find /home/crash -maxdepth 1 -mindepth 1 -type d`
# Code execution
printf "\n$DATE-$TIME\n" >> "$LOGFILE"
for d in $FIND; do
# Check if crash folders are older than 14 days and have been exported
if [[ "$(( $(date +"%s") - $(stat -c "%Y" $d) ))" -gt "1209600" ]] && [[ "$(getfattr -d --absolute-names -n user.exported --only-values $d)" == "1" ]]; then
#echo "$d is older than 14 days and exported"
"rm -rfv $d" >> "$LOGFILE"
# Check if crash folders are older than 30 days and delete regardless
elif [[ "$(( $(date +"%s") - $(stat -c "%Y" $d) ))" -gt "1814400" ]] && [[ "$(getfattr -d --absolute-names -n user.exported $d)" == FALSE ]]; then
#echo "$d is older than 30 days"
"rm -rfv $d" >> "$LOGFILE"
fi
done
The IF part is working fine and it deleted the folders with the attribute "user.exported=1" but the ELIF part does not seem to work, as I only get an output in my bash such as:
/home/crash/1234: user.exported: No such attribut
./crash_remove.sh: Line 20: rm -rfv /home/crash/1234: File or Directory not found
When I look into the crash folder after the script ran, the folder and its content is still there.
I definitely have an error in my script but cannot see it. Please could anyone help me out with this?
Thanks in advance
Only quote the expansions, not the whole command.
Instead of:
"rm -rfv $d"
do:
rm -rfv "$d"
If you quote it all, bash tries to run a command named literally rm<space>-rfv<space><expansion of d>.
Do not use backticks `. Use $(...) instead. Bash hackers wiki obsolete deprecated syntax.
Do not for i in $(cat) or var=$(...); for i in $var. Use a while IFS= read -r loop. How to read a file line by line in bash.
Instead of if [[ "$(( $(date +"%s") - $(stat -c "%Y" $d) ))" -gt "1814400" ]] just do the comparison in the arithmetic expansion, like: if (( ( $(date +"%s") - $(stat -c "%Y" $d) ) > 1814400 )).
I think you could just do it all in find, like::
find /home/crash -maxdepth 1 -mindepth 1 -type d '(' \
-mtime 14 \
-exec sh -c '[ "$(getfattr -d --absolute-names -n user.exported --only-values "$1")" = "1" ]' -- {} \; \
-exec echo rm -vrf {} + \
')' -o '(' \
-mtime 30 \
-exec sh -c '[ "$(getfattr -d --absolute-names -n user.exported "$1")" = FALSE ]' -- {} \; \
-exec echo rm -vrf {} + \
')' >> "$LOGFILE"
i made a script, which lists everysingle file in the current directory and subdirectory and gives me the md5sum from the head and tail (with offset) of this file and saves it into a .txt file.
i made this with pipes, so i wasn't able to redirect a variable, which has been declared before by userinput. So i changed my script to a for loop.
Problem now: it doesn't list all the files, but only one. And it seems to do this randomely. Why doesn't it list all the files like before?
I even tryed **.* and ./* and so on. I use a macbookpro mac os 10.13.6. I onced installed something so i could use linux commands aswell for example like tree etc...
any help is appreciated! I have no clue whatelse i can do.
old code in which the variable wasn't redirected:
#!/bin/bash
echo Wie heißt die Festplatte?
read varname
echo Los gehts!
before=$(date +%s)
find . \( ! -regex '.*/\..*' \) -type f -exec bash -c 'h=`tail -n +50000 "{}" | head -c 1000 | md5`;\
t=`tail -c 51000 "{}" | head -c 1000 | md5`;\
echo "$varname {} ; $h ; $t"' \;> /Users/Tobias/Desktop/$varname.txt
after=$(date +%s)
echo Das hat: $(((after - $before)/60)) Minuten bzw $(((after - $before))) Sekunden gedauert
new code in which it doesn't list all the files but only one :
#!/bin/bash
echo Wie heißt die Festplatte?
read varname
echo Los gehts!
before=$(date +%s)
for i in $( find . \( ! -regex '.*/\..*' \) -type f ); do
h=$(tail -n +50000 $i | head -c 1000 | md5)
t=$(tail -c 51000 $i | head -c 1000 | md5)
echo "$varname; $i ; $h ; $t" > /Users/Tobias/Desktop/$varname.txt
done
after=$(date +%s)
echo Das hat: $(((after - $before)/60)) Minuten bzw $(((after - $before))) Sekunden gedauert
You are overwriting the file in each iteration of the loop. Use the append mode instead:
echo "$varname; $i ; $h ; $t" >> /Users/Tobias/Desktop/"$varname".txt
# ~~
or redirect the output of the whole loop:
echo "$varname; $i ; $h ; $t"
done > /Users/Tobias/Desktop/"$varname".txt
Redirect the output of the entire loop, not each echo statement, which overwrites the file each time.
for i in $( find . \( ! -regex '.*/\..*' \) -type f ); do
h=$(tail -n +50000 $i | head -c 1000 | md5)
t=$(tail -c 51000 $i | head -c 1000 | md5)
echo "$varname; $i ; $h ; $t"
done > /Users/Tobias/Desktop/$varname.txt
I'm trying to use the newer command as follows:
touch $HOME/mark.start -d "$d1"
touch $HOME/mark.end -d "$d2"
SF=$HOME/mark.start
EF=$HOME/mark.end
find . -newer $SF ! -newer $EF
But this gives me an output like this:
.
./File5
and counts it as 2 files, however that directory only has 1 file i.e., File5. Why is this happening and how to solve it?
UPDATE:
I'm actually trying to run the following script:
#!/bin/bash
check_dir () {
d1=$2
d2=$((d1+1))
f1=`mktemp`
f2=`mktemp`
touch -d $d1 $f1
touch -d $d2 $f2
n=$(find $1 \( -name "*$d1*" \) -o \( -newer $f1 ! -newer $f2 \) | wc -l)
if [ $n != $3 ]; then echo $1 "=" $n ; fi
rm -f $f1 $f2
}
That checks if the directory has file that either has a particular date in the format YYYMMDD or if its last modification time was last 1 day.
check_dir ./dir1 20151215 4
check_dir ./dir2 20151215 3
where in dir1 there should be 4 such files and if it is not true then it will print the actual number of files that is there.
So, when the directory only has file with dates in their name, then it checks them fine, but when it checks with newer, it always gives 1 file extra (which is not even there in the directory). Why is this happening???
The question asks why there's an extra . in the results from find, even when no file or directory by that name exists. The answer is simple: . always exists, even when it's hidden. Use ls -a to show hidden contents, and you'll see that it's present.
Your existing find command doesn't exempt the target directory itself -- . -- from being a legitimate result, which is why you're getting more results than you expect.
Add the following filter:
-mindepth 1 # only include content **under** the file or directory specified
...or, if you only want to count files, use...
-type f # only include regular files
Assuming GNU find, by the way, this all can be made far more efficient:
check_dir() {
local d1 d2 # otherwise these variables leak into global scope
d1=$2
d2=$(gdate -d "+ 1 day $d1" '+%Y%m%d') # assuming GNU date is installed as gdate
n=$(find "$1" -mindepth 1 \
-name "*${d1}*" -o \
'(' -newermt "$d1" '!' -newermt "$d2" ')' \
-printf '\n' | wc -l)
if (( n != $3 )); then
echo "$1 = $n"
fi
}
I have a Base Directory that has 4 directories : Dir1 Dir2 Dir3 Dir4. Each of these directories have files in the format: "Sometext_YYYMMMDD". I'm writing a UNIX script to search through the files in all these directories that have a particular string say "20151215", and then printing it on the console.
find . -name "*20151215" -print
Example of files: File1_20151215 (this will be printed);
File2_20151214 (this will not be printed)
I want to write a script that runs through these directories and checks if Dir1 contains 4 files with string "20151215", Dir2 contains 3 files with string "20151215" and Dir3 & Dir4 contains 4 files with string "20151215". If the directories don't contain that number of files with that string, then I want to print those directories.
How do I do that? Please help!
UPDATE: I have an addition to this: There are also some files that are not in the format "Sometext_YYYMMMDD" So, for those I used something like:
find . -name "FILENAME*" -mtime -1 -exec ls -ltr '{}' \;
to extract the timestamp when that file was created. But, I want to know how do I add it to the script so that if the timestamp is 15 Dec 2015, then this file should also be counted in the search?
You got the find part, but now you need to count how many files match the pattern. Since find prints one line per match to the output, you can use "wc -l" to count how many lines there are. Assign that to a variable, that you can use in a comparison, and you're 90% of the way there. E.g.
d1=$(find ./dir1 -name '*20151215*' | wc -l)
if [ $d1 != 4 ]; then echo "dir1" ; fi
For extra credit, you can imagine turning this into a function with inputs of
Directory to search
Filename pattern to match on
How many matches to expect
Which would look like:
check_dir () {
d1=$(find $1 -name "*$2*" | wc -l)
if [ $d1 != $3 ]; then echo $1 ; fi
}
check_dir ./dir1 20151215 4
check_dir ./dir2 20151215 3
Update: with the new requirement to find files based either on the name of the file or the last modification (creation isn't possible), here's two approaches:
The first uses a fairly modern feature of find that isn't available in all versions, newermt:
check_dir () {
d1=$2
d2=$((d1+1))
n=$(find $1 \( -name "*$d1*" \) -o \( -newermt $d1 ! -newermt $d2 \) | wc -l)
if [ $n != $3 ]; then echo $1 ; fi
}
check_dir ./dir1 20151215 4
check_dir ./dir2 20151215 3
Which looks a little confusing, but break it down into small steps and it makes sense:
d1=$2 # So d1=20151215
d2=$((d1+1)) # d2=20151216 (lucky you're specifying the date format this way!)
The find command now has two predicates, to match based on the filename or the modification time:
\( -name "*$2*" \) # Matches filenames that contain 20151215
-o # Or
\( -newermt $d1 ! -newermt $d2 \)
The modification time is greater than midnight on the first day, and not greater than midnight on the next day
The second approach uses a couple of temp files, and sets the timestamps on them using the -d option of the touch command
#!/bin/bash
check_dir () {
d1=$2
d2=$((d1+1))
f1=`mktemp`
f2=`mktemp`
touch -d $d1 $f1
touch -d $d2 $f2
n=$(find $1 \( -name "*$d1*" \) -o \( -newer $f1 ! -newer $f2 \) | wc -l)
if [ $n != $3 ]; then echo $1 "=" $n ; fi
rm -f $f1 $f2
}
Again, it's lucky that the date is in YYYYMMDD since that works with the -d option of the touch command. If not, you would need to do some string manipulation to get the date into the correct format for "touch -t".
Need to get the files between two given dates via a KornShell (ksh) script. If there are multiple files on one day get the latest of the files for that day.
I haven't tried it out, but there's a mailing list post about finding files between two dates. The relevant part:
Touch 2 files, start_date and
stop_date, like this: $ touch -t
200603290000.00 start_date $ touch -t 200603290030.00 stop_date
Ok, start_date is 03/29/06 midnight,
stop_date is 03/29/06 30 minutes after
midnight. You might want to do a ls
-al to check.
On to find, you can find -newer and
then ! -newer, like this: $ find /dir
-newer start_date ! -newer stop_date -print
Combine that with ls -l, you get: $
find /dir -newer start_date ! -newer
stop_date -print0 | xargs -0 ls -l
(Or you can try -exec to execute ls
-l. I am not sure of the format, so you have to muck around a little bit)
in bash shell, just an example, you can use the -nt test operator (korn shell comes with it also, if i am not wrong).
printf "Enter start date( YYYYMMDD ):"
read startdate
printf "Enter end date( YYYYMMDD ):"
read enddate
touch -t "${startdate}0000.00" sdummy
touch -t "${enddate}0000.00" edummy
for fi in *
do
if [ $fi -nt "sdummy" -a ! $fi -nt "edummy" ] ;then
echo "-->" $fi
fi
done
In a nut shell for ksh :
!/usr/bin/ksh
# main from_date to_date path
# date format: YYMMDDhhmmss
ls -l --time-style "+%y%m%d%H%M%S" $3 | awk '{ print $6 " " $7 }' | while read t n
do
if (( t > $1 )) && (( t < $2 )); then
echo $t $n
fi
done