I am looking for a multiple grep with NOT and AND conditions. I have a directory with with txt files and some csv files which have the date included in the filename. I want to delete the csv files that do not include today’s date. The directory does include csv files with previous dates. So I am trying the code below in bash
#!/bin/bash
now=$(date "+%m-%d-%Y")
dir="/var/tmp/"
for f in "$dir"/*; do
ls -1 | grep -v *$now* | grep *csv* | xargs rm -f
done
This is not deleting anything. If I take out the grep csv operator then it deletes the text files. Only the CSV files have dates on them, the text files don’t. Please advise.
I suggest using the find utility for this:
find /var/tmp -maxdepth 1 -name '*.csv' ! -name "*$now*" -delete
If you want to do it with grep,
ls -1 /var/tmp/*.csv | grep -v "$now" | xargs rm -f
should also work.
EDIT: -delete in the find command instead of -exec rm '{}' \;. Thanks #amphetamachine.
Related
I am writing a script to search all files from a directory/root based on a keyword input.
echo "Enter keyword"
read key
grep -r -l . -e "$key"
If I search for "hei", which I know is located in a file called mem.c, it prints /folder/mem.c, and not only the filename. Do I have some wrong arguments or is it supposed to be like that?
Additional question, is there a way to store these filenames, and copy them into another directory if there is a match in the keyword? Or maybe it is possible to loop through the files found with grep?
This is what grep -l does: print the path of matching files. If you want to print only the base name of the files you can pipe the result to some command that does that. Example:
grep -rlZ "$key" . | xargs -0 -n1 basename
Storing the file names in a file is just a matter of redirection:
grep -rlZ "$key" . | xargs -0 -n1 basename > mylist.txt
To copy the matching files somewhere you will need their path, not just their base name. Assuming you want to copy them all in the same destdir directory and you have no name conflicts, you could use find:
find . -type f -exec grep -l "$key" {} \; -exec cp -f {} destdir \;
I can't find posts that help with this exact problem:
On Mac Terminal I want to read a txt file (example.txt) containing file names such as:
20130815 144129 865 000000 0172 0780.bmp
20130815 144221 511 000003 1068 0408.bmp
....100 more
And I want to search for them in a certain folder/subfolders (example_folder). After each find, the file should be copied to a new folder x (new_destination).
Your help would be much appreciated!
Chers,
Mo
You could use a piped command with a combination of ls, grep, xargs and cp.
So basically you start with getting the list of files
ls
then you filter them with egrep -e, grep -e or whatever flavor of grep Mac uses for their terminal. If you want to find all files ending with text you can use the regex .txt$ (which means ends with '.txt')
ls | egrep -e "yourRegexExpression"
After that you get an input stream, but cp doesn't work with input streams and only takes a bunch of arguments, that's why we use xargs to convert it to arguments. The final step is to add the flag -t to the argument to signify that the next argument is the target directory.
ls | egrep -e "yourRegexExpression" | xargs cp -t DIRECTORY
I hope this helps!
Edit
Sorry I didn't read the question well enough, I updated to be match your problem. Here you can see that the egrep command compiles a rather large regex string with all the file names in this way (filename1|filename2|...|fileN). The $() evaluates the command inside and uses the tr to translate newLines to "|" for the regex.
ls | egrep -e "("$(cat yourtextfile.txt | tr "\n" "|")")" | xargs cp -t DIRECTORY
You could do something like:
$ for i in `cat example.txt`
find /search/path -type f -name "$i" -exec cp "{}" /new/path \;
This is how it works, for every line within example.txt:
for i in `cat example.txt`
it will try to find a file matching the line $i in the defined path:
find /search/path -type f -name "$i"
And if found it will copy it to the desired location:
-exec cp "{}" /new/path \;
I have a directory that contains files and other directories. And I have one specific file where I know that there are duplicates of somewhere in the given directory tree.
How can I find these duplicates using Bash on macOS?
Basically, I'm looking for something like this (pseudo-code):
$ find-duplicates --of foo.txt --in ~/some/dir --recursive
I have seen that there are tools such as fdupes, but I'm neither interested in any duplicate files (only duplicates of a specific file) nor am I interested in duplicates anywhere on disk (only within the given directory or its subdirectories).
How do I do this?
For a solution compatible with macOS built-in shell utilities, try this instead:
find DIR -type f -print0 | xargs -0 md5 -r | grep "$(md5 -q FILE)"
where:
DIR is the directory you are interested in;
FILE is the file (path) you are searching for duplicates of.
If you only need the duplicated files paths, then pipe thru this as well:
cut -d' ' -f2
If you're looking for a specific filename, you could do:
find ~/some/dir -name foo.txt
which would return a list of all files with the name foo.txt in the directory. If you're looking if there are multiple files in the directory with the same name, you could do:
find ~/some/dir -exec basename {} \; | sort | uniq -d
This will give you a list of files with duplicate names (you can then use find again to figure out where those live).
---- EDIT -----
If you're looking for identical files (with the same md5 sum), you could also do:
find . -type f -exec md5sum {} \; | sort | uniq -d --check-chars=32
--- EDIT 2 ----
If your md5sum doesn't output the filename, you can use:
find . -type f -exec echo -n "{} " \; -exec md5sum {} \; | awk {'print $2 $1'} | sort | uniq -d --check-chars=32
--- EDIT 3 ----
if you're looking for a file with a specific md5 sums:
sum=`md5sum foo.txt | cut -f1 -d " "`
find ~/some/dir -type f -exec md5sum {} \; | grep $sum
I have a structure like that:
/usr/local/a/1.txt
/usr/local/a/2.txt
/usr/local/a/3.txt
/usr/local/b/4.txt
/usr/local/b/3.txt
/usr/local/c/1.txt
/usr/local/c/7.txt
/usr/local/c/6.txt
/usr/local/c/12.txt
...
I want to delete all the files *.txt in subfolders except the last three files with the greatest modification date, but here I am in current directory
ls -tr *.txt | head -n-3 |xargs rm -f
I need to combine that with the code:
find /usr/local/**/* -type f
Should I use the maxdepth option?
Thanks for helping,
aola
Added maxdepth options to find for one level, sorting files by last modification time, tail to ignore the oldest modified 3 files and xargs with -r to remove the files only if they are found.
for folder in $(find /usr/local/ -type d)
do
find $folder -maxdepth 1 -type f -name "*.txt" | xargs -r ls -1tr | tail -n+3 | xargs -r rm -f
done
Run the above command once without rm to ensure that the previous commands pick the proper files for deletion.
You've almost got the solution: use find to get the files,ls to sort them by modification date and tail to omit three most recently modified ones:
find /usr/lib -type f | xargs ls -t | tail -n +4 | xargs rm
If you would like to remove only the files at a specified depth add -mindepth 4 -maxdepth 4 to find parameters.
You can use find's -printf option, to print the modification time in front of the file name and then sort and strip the date off. This avoids using ls at all.
find /usr/local -type f -name '*.txt' -printf '%T#|%p\n' | sort -r | cut -d '|' -f 2 | head -n-3 | xargs rm -f
The other Answers using xargs ls -t can lead to incorrect results, when there are more results than xargs can put in a single ls -t command.
but for each subfolder, so when I have
/usr/local/a/1.txt
/usr/local/a/2.txt
/usr/local/a/3.txt
/usr/local/a/4.txt
/usr/local/b/4.txt
/usr/local/b/3.txt
/usr/local/c/1.txt
/usr/local/c/7.txt
/usr/local/c/6.txt
/usr/local/c/12.txt
I want to to use the code for each subfolder separately
head -n-3 |xargs rm -f
so I bet if I have it sorted by date then the files to delete:
/usr/local/a/4.txt
/usr/local/c/12.txt
I want to leave in any subfolder three newest files
Is it possible to get the modification date and time of a folder?
I know you can use stat -f "%m" folder, but it doesn't reflect sub-files/folders changes.
Things that doesn't work:
ls -l folder - doesn't reflect changes inside the folder
stat -f "%m" folder - same as above
date -r folder - same again
find foo bar baz -printf - the printf option doesn't exist on my version of find
Versions of things:
OS: Mac OS X 10.7.1
Bash: GNU bash, version 3.2.48(1)-release (x86_64-apple-darwin11)
Solution:
find . -exec stat -f "%m" \{} \; | sort -n -r | head -1
Explanation:
the find command traverses the current directory (.) and for each file encountered executes (-exec) the command stat -f "%m". stat -f "%m" prints the last modification unix timestamp of the file.
sort -n -r sorts the output of the find command numerically (-n) in reverse order (-r). This will list the latest modification timestamp first.
head -1 then extracts the first line of the output from sort. This is the latest modification unix timestamp of all the files.
You could try 'date -r folder' to give you a date last modified
You could always get it from ls :
ls -ld mydir | awk -F' ' '{ print $6 " "$7 }'
if you need to clear cache after build . then you can check the age of the last change and delete it like this
sh("find ~/Library/Developer/Xcode/DerivedData/ -type d -maxdepth 1 -mmin +360 -name 'Cache-folder-*' -print0 | xargs -0 -I {} /bin/rm -rf '{}' || echo 'There is no such folder! Start script execution' ; exit 0")
sh("find ~/Library/Developer/Xcode/DerivedData/ -type d -maxdepth 1 -mtime 0 -name 'Cache-folder-*' -ls -exec rm -r {} \\;")