I have a txt file which has list of Prefix Id's , I need to take that and match it against the source directory and get the matching file names and build a list.
Im currently using the below code to achieve this , but im getting only the file for last file name in the list
set -A flist $(find $1 -name "*.csv" | grep "`cat ${tempfile}.txt`"
There are some 10 matching files but this command returns only the last file , any help would be appreciated
$ cat prefixes.txt
foo_[^/]*$
bar_[^/]*$
$ find
.
./blah
./foo_1.txt
./bar_3.txt
./bar_4.txt
./quux_6.txt
./quux_5.txt
./prefixes.txt
./foo_2.txt
$ foo=( $(find . -name "*.txt*" | grep -f prefixes.txt) )
$ set | grep ^foo=
foo=([0]="./foo_1.txt" [1]="./bar_3.txt" [2]="./bar_4.txt" [3]="./foo_2.txt")
Note though that this will break on filenames with whitespace and there is probably a better solution to your actual problem.
set -A aRray `find . -name "*" | egrep -f RegExPrefix.txt | sed "s/.*/'&'/"`
will depend on your shell. Here is tested with a ksh on AIX (so not a GNU sed and set -A create an array)
Related
I can't find posts that help with this exact problem:
On Mac Terminal I want to read a txt file (example.txt) containing file names such as:
20130815 144129 865 000000 0172 0780.bmp
20130815 144221 511 000003 1068 0408.bmp
....100 more
And I want to search for them in a certain folder/subfolders (example_folder). After each find, the file should be copied to a new folder x (new_destination).
Your help would be much appreciated!
Chers,
Mo
You could use a piped command with a combination of ls, grep, xargs and cp.
So basically you start with getting the list of files
ls
then you filter them with egrep -e, grep -e or whatever flavor of grep Mac uses for their terminal. If you want to find all files ending with text you can use the regex .txt$ (which means ends with '.txt')
ls | egrep -e "yourRegexExpression"
After that you get an input stream, but cp doesn't work with input streams and only takes a bunch of arguments, that's why we use xargs to convert it to arguments. The final step is to add the flag -t to the argument to signify that the next argument is the target directory.
ls | egrep -e "yourRegexExpression" | xargs cp -t DIRECTORY
I hope this helps!
Edit
Sorry I didn't read the question well enough, I updated to be match your problem. Here you can see that the egrep command compiles a rather large regex string with all the file names in this way (filename1|filename2|...|fileN). The $() evaluates the command inside and uses the tr to translate newLines to "|" for the regex.
ls | egrep -e "("$(cat yourtextfile.txt | tr "\n" "|")")" | xargs cp -t DIRECTORY
You could do something like:
$ for i in `cat example.txt`
find /search/path -type f -name "$i" -exec cp "{}" /new/path \;
This is how it works, for every line within example.txt:
for i in `cat example.txt`
it will try to find a file matching the line $i in the defined path:
find /search/path -type f -name "$i"
And if found it will copy it to the desired location:
-exec cp "{}" /new/path \;
Task (12) from here.
Using grep I could solve this task as following:
grep --no-message -l [[:alnum:]] /etc/* | grep [[:digit:]]
Similar results were obtained:
ls -o /etc/ | grep ^- | awk '{print $8}' | grep [[:digit:]]
But I want to read all files under each directory, recursively. And this is what I could do:
grep --no-message -lR [[:alnum:]] /etc/* | grep [[:digit:]]
Also how this task can be solved today? Is this a correct way? What kind of additional solutions you can offer to solve this task?
You shouldn't be using to grep just to get file names. i.e. instead of what I think you are trying to do with
grep --no-message -l [[:alnum:]] /etc/*
You can just do
echo /etc/*
Since the goal is to list files with numbers, you can use a glob like:
echo /etc/*[[:digit:]]*
If you want to do this recursively you can do it in bash like:
shopt -s globstar
echo /etc/**/*[[:digit:]]*
Or you can use find:
find /etc -name '*[[:digit:]]*'
You can use find:
find /etc -type f -name \*[[:digit:]]\*
Please note that this command lists only files with digits in names, not the full paths. For the latter use a different filter:
find /etc -type f -path \*[[:digit:]]\*
Grisha and Andrey have given perfect answers. However, if you are looking for a solution that does use grep, here it is.
while read -r filepath; do
# grep for digits in the basename so that we choose only those
# files that have a digit in them, not in any parent directory name
if grep -q '[[:digit:]]' <<< "$(basename $filepath)"; then
echo "$filepath"
fi
done < <(find /etc -type f -print)
It's not efficient at all.
If the task is to have a pure grep solution, then here it is:
grep -laR . /etc | grep '^.*/[^/]*[[:digit:]][^/]*$'
The regular expression:
^...$ cover the full line;
.*/[^/]*...[^/]* look only for the file name.
I want to find all files within the current directory that contain a given string, then print just the 4th line of each file.
grep --null -l "$yourstring" * | # List all the files containing your string
xargs -0 sed -n '4p;q' # Print the fourth line of said files.
Different editions of grep have slightly different incantations of --null, but it's usually there in some form. Read your manpage for details.
Update: I believe one of the null file list incantations of grep is a reasonable solution that will cover the vast majority of real-world use cases, but to be entirely portable, if your version of grep does not support any null output it is not perfectly safe to use it with xargs, so you must resort to find.
find . -maxdepth 1 -type f -exec grep -q "$yourstring" {} \; -exec sed -n '4p;q' {} +
Because find arguments can almost all be used as predicates, the -exec grep -q… part filters the files that are eventually fed to sed down to only those that contain the required string.
From other user:
grep -Frl string . | xargs -n 1 sed -n 4p
Give a try to the below GNU find command,
find . -maxdepth 1 -type f -exec grep -l 'yourstring' {} \; | xargs -I {} awk 'NR==4{print; exit}' {}
It finds all the files in the current directory which contains specific string, and prints the line number 4 present in each file.
This for loop should work:
while read -d '' -r file; do
echo -n "$file: "
sed '4q;d' "$file"
done < <(grep --null -l "some-text" *.txt)
Basically I have a directory and sub-directories that needs to be scanned to find .csv files. From there I want to copy all lines containing "foo" from the csv's found to new files (in the same directory as the original) but with the name reflecting the file it was found in.
So far I have
find -type f -name "*.csv" | xargs egrep -i "foo" > foo.csv
which yields one backup file (foo.csv) with everything in it, and the location it was found in is part of the data. Both of which I don't want.
What I want:
For example if I have:
csv1.csv
csv2.csv
and they both have lines containing "foo", I would like those lines copied to:
csv1_foo.csv
csv2_foo.csv
and I don't anything extra entered in the backups, other than the full line containing "foo" from the original file. I.e. I don't want the original file name in the backup data, which is what my current code does.
Also, I suppose I should note that I'm using egrep, but my example doesn't use regex. I will be using regex in my search when I apply it to my specific scenario, so this probably needs to be taken into account when naming the new file. If that seems too difficult, an answer that doesn't account for regex would be fine.
Thanks ahead of time!
try this if helps it anyway.
find -type f -name "*.csv" | xargs -I {} sh -c 'filen=`echo {} | sed 's/.csv//' | sed "s/.\///"` && egrep -i "foo" {} > ${filen}_foo.log'
You can try this:
$ find . -type f -exec grep -H foo '{}' \; | perl -ne '`echo $2 >> $1_foo` if /(.*):(.*)/'
It uses:
find to iterate over files
grep to print file path:line tuples (-H switch)
perl to echo those line to the output files (using backslashes, but it could be done prettier).
You can also try:
find -type f -name "*.csv" -a ! -name "*_foo.csv" | while read f; do
grep foo "$f" > "${f%.csv}_foo.csv"
done
Not 100% on how to go about this.
Basically my app creates two file outputs.
file
file.ext
when searching through the directory it always returns both files. I wish only to work with
file
how can I do a grep for just the file? Or is grep even the correct command?
!grep *.ext
pseudo code ^
Why not just:
grep foobar file
or if you want to search your directory
find . -name 'file' | xargs -r grep boofar
You can try using grep 'file$', i.e. select lines ending with file and nothing after that. Alternatively you can use grep -v to invert results.
# (echo "file"; echo "file.txt") | grep file$
file
# (echo "file"; echo "file.txt") | grep -v .ext
file
I have no idea what you are trying to do with grep, but if you want to check whether a file exists, you do simply this:
if [ -r /some/path/file ]; then
echo "File is readable."
fi
Grep has a -v flag that inverses the meaning (looks for lines that do not contain the pattern):
ls /your/directory | grep -v '\.ext$'
This will exclude every filename ending with .ext.
Try
grep -v option
-v do the inverse match
For details do $ man grep
To generate a list of all files that do not match a pattern, use -not in find. So you want:
find . -not -name '*.ext'