have a folder structure as shown below
./all_files
-rwxrwxrwx reference_file.txt
drwxrwxrwx file1.txt
drwxrwxrwx file2.txt
drwxrwxrwx file3.txt
reference_file.txt has filenames as shown below
file1.txt
file2.txt
data in file1.txt and file2.txt are as shown below:
step_1
step_2
step_3
Now, I have to take particular step say step2 from each file
Note1: file name must present in reference_file.txt
Note2: step2 is not line no:2 always.
Note3: search should perform recursively.
I have used below scripts:
which is not giving what i expected. All blank and unwanted lines displayed
#!/bin/sh
while read f; do
find . -type f -name "${f}" | xargs grep -l -F 'step_2'
done <reference_file.txt
It is searching in only one folder but not searching recursively
#!/bin/sh
for i in reference_file.txt
do
find . -type f -name "${f}" | xargs grep -l -F 'step_2'
done
Please help me on this.
With removing -l option it works for me.
for f incat reference_file.txt; do find . -type f -name $f | xargs grep 'step_2' ; done
I can see it works recursively as well. See the output below when I run it from outside of dir1 (directory structure created by me)
./dir1/file1:step_2:
./dir1/file2:step_2:
Please share more information like directory structure and file names if issue in recursion still exists.
Related
have a folder structure as shown below ./all_files
-rwxrwxrwx reference_file.txt
drwxrwxrwx file1.txt
drwxrwxrwx file2.txt
drwxrwxrwx file3.txt
reference_file.txt has filenames as shown below
$cat reference_file.txt
file1.txt
file2.txt
data in file1.txt and file2.txt are as shown below:
$cat file1.txt
step_1
step_2
step_3
Now, I have to take particular step say step2 from each file
Note1: file name must present in reference_file.txt
Note2: step2 is not line no:2 always.
Note3: search should perform recursively.
I have used below script:
#!/bin/sh
for i in cat reference_file.txt;
do
find . -type f -name $i | grep -v 'FS*' | xargs grep -F 'step_2'
done<reference_file.txt
after using above code i got no output.
# bash -x script.sh
+ for i in cat reference_file.txt
+ find . -type f -name **cat**
+ xargs grep -F 'step_2'
+ for i in cat **reference_file.txt**
+ find . -type f -name reference_file.txt
+ xargs grep -F 'step_2'
Added New requirement:
target=step_XX_2 where XX can be anything and should be skipped for search.. so that desire ouput will be.. step_ab_2 step_cd_2 step_ef_2
I think this is what you are trying to achieve. Please let me know:
EDIT: my previous version did not search recursively.
Further edits: Note that using process substitution for find means that this script MUST be run under bash and not sh.
Further edit for change in specification: note the change to target and the -E option to grep instead of -F.
#!/bin/bash
target='step_.*?_?2'
while read -r name
do
# EDIT: exclude certain directories
if [[ $name == "old1" || $name == "old2" ]]
then
# do the next iteration of the loop
continue
fi
while read -r fname
do
if [[ $fname != FS* ]]
then
# Display the filename (grep -H is not in POSIX)
if out=$(grep -E "$target" "$fname")
then
echo "$fname: $out"
fi
fi
done < <(find . -type f -name "$name")
done < reference_file.txt
Note that your trace (bash -x) uses bash but your #! line uses sh. They are different - you should be consistent with the shell you are using.
So, I have dropped the xargs, that reads strings standard input and executes a program using the strings as argument. Since we already have the argument strings for grep we don't need it.
Your grep -v 'FS*' probably doesn't do what you expect. The regular expression FS* means "F followed by zero or more S's". Not the same as a shell pattern matching (globbing). In my solution I have used FS* because I am using the shell, not grep.
I believe this question is duplicate of this
What you need is
#!/bin/sh
for i in `cat reference_file.txt`
do find . -type f -name $i | grep -v 'FS*' | xargs grep -F 'step_2'
done
See the backticks and Do Not read the file reference_file.txt twice.
I have ./all_files/reference_file.txt which has data as shown below.
reference_file.txt contains filenames as shown
file1.txt
file2.txt
file3.txt
file4.txt
file5.txt
data in all files are as shown below:
step_1
step_2
step_3
step_4
Now, I have to take particular step say step2 from each file
Note1: file name must present in reference_file.txt
Note2: step2 is not line no:2 always.
Note3: search should perform recursively.
I have used below code
#!/bin/sh
while read f; do
if [ -f "$f" ]; then
find . -type f -name "*.txt" | xargs grep -l -F 'step_2' "$f"
fi
done <reference_file.txt
please help me on this
Two changes to your code:
Remove the if because find is supposed to find the file and in
that case if might return false if the file is not in current
directory.
find command should be passed $f as argument.
Below is the updated example:
while read f; do
find . -type f -name "${f}" | xargs grep -l -F 'step_2'
done <reference_file.txt
Say I've file1.txt with
ptext1
ptext2
ptext3
ptext4
These are the partial file names (library names) which I'm trying to find from a directory. Something like
cat file1.txt | xargs find . -name "*$0*"
or say,
cat file.txt | awk '{system("find . -name " *$0*)}'
None of them are working.
Please suggest.
I'm sure there is a more elegant way, but you could always loop over and run find on each:
Update to reflect suggestions in comments
while read -r filename; do
find . -type f -name "*$filename*"
done < file1.txt
One way with xargs
xargs -I{} find . -name "*"{}"*" < file
Bash newbie here trying to insert the name of a folder into certain files inside that folder.
The problem is that these files are in subfolders of subfolders of the main directory, and the names of each level are different in each case.
For example, the main folder interviews may contain John Doe and under John Doe is a directory Images with a file Screenshot.jpg. But there might also be John Smith with a folder Etc in which is 12_Screenshot 2.jpg.
I want to rename all these files containing Screenshot inserting John Doe or John Smith before the filename.
I tried adapting a couple of scripts I found and ran them from the interviews directory:
for i in `ls -l | egrep '^d'| awk '{print $10}'`; do find . -type f -name "*Screenshot*" -exec sh -c 'mv "$0" "${i}${0}"' '{}' \; done
after which the terminal gives the caret prompt, as if I'm missing something. I also tried
find -regex '\./*' -type d -exec mv -- {}/*/*Screenshot* {}/{}.jpg \; -empty -delete
which returns find: illegal option -- r
The fact that the second one theoretically moves the file up to the parent folder is not a problem since I'll have to do this eventually anyways.
The following script will work as desired :
dir=$1
find $dir -name "*Screenshot*" -type f | while read file
do
base=$(basename $file)
dirpath=$(dirname $file)
extr=$(echo $file | awk -F/ '{print $(NF-2)}') #extracts the grandparent directory
mv $file $dirpath/$extr-$base
done
As #loneswap mentioned, this must be invoked as a script. So if your main directory is mainDir, then you would invoke it as so...
./script mainDir
For each directory in current working directory, recursively find files containing string "screenshot" (case insensitive due to OSX). Split the found path into parent part (always present at least in form './') and file name, produce two lines one original file path, second one original folder + modified target file name. Execute mv command via xargs using two arguments (separate by newline to allow whitespaces in paths):
for i in `ls -l | sed -n '/^d\([^[:space:]]\+[[:space:]]\+\)\+\([^[:space:]]\+\)$/s//\2/p'`; do
find "$i" -type f -iname "*Screenshot*" \
| sed -n '\!^\(\([^/]\+/\)\+\)\([^/]\+\)$!s!!\1\3\n\1'$i'\3!p' \
| xargs -d '\n' -n 2 mv;
done
Drawback: xargs on OSX does not know --no-run-if-empty, so for directories that do not contain files with "screenshot" string empty mv is invoked. Proper option needs to be added (don't have access to OSX man pages) or xargs ... 2>&/dev/null to ignore all errors...
Basically I have a directory and sub-directories that needs to be scanned to find .csv files. From there I want to copy all lines containing "foo" from the csv's found to new files (in the same directory as the original) but with the name reflecting the file it was found in.
So far I have
find -type f -name "*.csv" | xargs egrep -i "foo" > foo.csv
which yields one backup file (foo.csv) with everything in it, and the location it was found in is part of the data. Both of which I don't want.
What I want:
For example if I have:
csv1.csv
csv2.csv
and they both have lines containing "foo", I would like those lines copied to:
csv1_foo.csv
csv2_foo.csv
and I don't anything extra entered in the backups, other than the full line containing "foo" from the original file. I.e. I don't want the original file name in the backup data, which is what my current code does.
Also, I suppose I should note that I'm using egrep, but my example doesn't use regex. I will be using regex in my search when I apply it to my specific scenario, so this probably needs to be taken into account when naming the new file. If that seems too difficult, an answer that doesn't account for regex would be fine.
Thanks ahead of time!
try this if helps it anyway.
find -type f -name "*.csv" | xargs -I {} sh -c 'filen=`echo {} | sed 's/.csv//' | sed "s/.\///"` && egrep -i "foo" {} > ${filen}_foo.log'
You can try this:
$ find . -type f -exec grep -H foo '{}' \; | perl -ne '`echo $2 >> $1_foo` if /(.*):(.*)/'
It uses:
find to iterate over files
grep to print file path:line tuples (-H switch)
perl to echo those line to the output files (using backslashes, but it could be done prettier).
You can also try:
find -type f -name "*.csv" -a ! -name "*_foo.csv" | while read f; do
grep foo "$f" > "${f%.csv}_foo.csv"
done