Many times I need to search from a directory and below for a pattern in all files with a specific type. For example, I need to ask grep not to look into files other than *.h, *.cpp or *.c. But if I enter:
grep -r pattern .
it looks into all files. If I enter:
grep -r pattern *.c
it tries *.c files in the current folder (no file in my case) and files in *.c folders (no folder in my case). I want to ask it too look into all folders but only into file with the given type. I think grep is not enough to be used for this purpose. So, I get help from find too, like this:
grep pattern `find . -name '*c'`
First, let me know whether I'm right about getting help from find. Can grep be enough? Second, I prefer to write an alias for bash to be used like this:
mygrep pattern c
to be translated to the same command avoiding usage of ` and ' and be simpler. I tried:
alias mygrep="grep $1 `find . -name '*$2'`"
But it doesn't work and issues an error:
grep: c: No such file or directory
I tried to change it, but I couldn't succeed to a successful alias.
Any idea?
This would be better done as a function than an alias, and using -exec instead of passing the output of find to grep. That output would be subject to word splitting and globbing, so could produce surprising results as is. Instead try:
mygrep () {
find . -name "*$2" -exec grep "$1" {} +
}
Related
So, I have this simple script which converts videos in a folder into a format which the R4DS can play.
#!/bin/bash
scr='/home/user/dpgv4/dpgv4.py';mkdir -p 'DPG_DS'
find '../Exports' -name "*1080pnornmain.mp4" -exec python3 "$scr" {} \;
The problem is, some of the videos are invalid and won't play, and I've moved those videos to a different directory inside the Exports folder. What I want to do is check to make sure the files are in a folder called new before running the python script on them, preferably within the find command. The path should look something like this:
../Exports/(anything here)/new/*1080pnornmain.mp4
Please note that (anything here) text does not indicate a single directory, it could be something like foo/bar, foo/b/ar, f/o/o/b/a/r, etc.
You cannot use -name because the search is on the path now. My first solution was:
find ./Exports -path '**/new/*1080pnornmain.mp4' -exec python3 "$scr" {} \;
But, as #dan pointed out in the comments, it is wrong because it uses the globstar wildcard (**) unnecessarily:
This checks if /new/ is somewhere in the preceding path, it doesn't have to be a direct parent.
So, the star is not enough here. Another possibility, using find only, could be this one:
find ./Exports -regex '.*/new/[^\/]*1080pnornmain.mp4' -exec python3 "$scr" {} \;
This regex matches:
any number of nested folders before new with .*/new
any character (except / to leave out further subpaths) + your filename with [^\/]*1080pnornmain.mp4
Performances could degrade given that it uses regular expressions.
Generally, instead of using the -exec option of the find command, you should opt to passing each line of find output to xargs because of the more efficient thread spawning, like:
find ./Exports -regex '.*/new/[^\/]*1080pnornmain.mp4' | xargs -0 -I '{}' python3 "$scr" '{}'
I have a directory with this structure:
root
|-dir1
| |-pred_20181231.csv
|
|-dir2
| |-pred_20181234.csv
...
|-dir84
|-pred_2018123256.csv
I want to run a command that will rename all the pred_XXX.csv files to pred.csv.
How can I easily achieve that?
I have looked into the rename facility but I do not understand the perl expression syntax.
EDIT: I tried with this code: rename -n 's/\training_*.csv$/\training_history.csv/' *.csv but it did not work
Try with this command:
find root -type f -name "*.csv" -exec perl-rename 's/_\d+(\.csv)/$1/g' '{}' \;
Options used:
-type f to specify file or directory.
-name "*.csv" to only match files with extension csv
-exec\-execdir to execute a command, in this case, perl-rename
's/_\d+(\.csv)/$1/g' search a string like _20181234.csv and replace it with .csv, $1 means first group found.
NOTE
Depending in your S.O. you could use just rename instead of perl-rename.
Use some shell looping:
for file in **/*.csv
do
echo mv "$(dirname "$file")/$(basename "$file")" "$(dirname "$file")/pred.csv"
done
On modern shells ** is a wildcard that matches multiple directories in a hierarchy, an alternative to find, which is a fine solution too. I'm not sure if this should instead be /**/*.csv or /root/**/*.csv based on tree you provided, so I've put echo before the 'mv' to see what it's about to do. After making sure this is going to do what you expect it to do, remove the echo.
I've never come to SO asking "Do my homework" but I really don't know where to start with this one.
I have a load of documents which are dumped in a directory after being auto-signed using JSignPdf (--output-directory option seemingly has no ability to output to same as input):
/some/dir/Signed/PDF1_signed.pdf
/some/dir/Signed/PDF2_signed.pdf
/some/dir/Signed/PDF2_signed.pdf
I'd like to then find their source/unsigned counterparts:
/some/dir/with/docs/PDF1.pdf
/some/dir/where/is/PDF2.pdf
/some/dir/why/this/PDF3.pdf
...and move the signed PDFs into the respective directories.
I use the command, to find all the PDFs in the variety of directories:
find . -name '*.pdf' -exec sh -c 'exec java -jar jsignpdf-1.4.3/JSignPdf.jar ... ' sh {} +
...and I've tried things like making find output a variable and then using IF THEN to match with no success. Would I need find output to be made into multiple variables? I'm so lost :(
I'd like to accomplish this in some shell, but if there are Perl junkies out there or anything else, I am more than happy for another portable solution.
I've tried to break it down, but still don't understand how to accomplish it...
find files matching VarName without _signed
move _signed file with matching name to the directory of found file
Thanks for any help/guidance.
Use a while loop to read each file found by find and move it to the correct place:
find /some/dir -name "*.pdf" ! -name "*_signed.pdf" -print0 | while IFS= read -d '' -r file
do
f="${file##*/}"
mv "/some/dir/Signed/${f%.*}_signed.pdf" "${file%/*}"
done
I have a similar problem I've been working on. Since the path manipulation required to convert /some/dir/where/is/PDF2.pdf to /some/dir/Signed/PDF2_signed.pdf is fairly simple but more involved than can be done in a simple one-liner, I've been using find to locate the first set, and using a simple loop to process them one at a time. You did mention homework, so I'll try not to give you too much code.
find /dir/containing/unsigned -name '*.pdf' -print0 | while IFS= read -d path; do
fetch_signed_version "$path"
done
where fetch_signed_version is a shell function you write that, given a path such as /some/dir/where/is/PDF2.pdf, extracts the directory (/some/dir/where/is), computes the signed PDF's name (PDF2_signed.pdf), then executes the necessary move (mv /some/dir/Signed/$signed_pdf /some/dir/where/is)
fetch_signed_version is actually pretty simple:
fetch_signed_version () {
dir=${1%/*}
fname=${1##*/}
signed_name=${fname%.pdf}_signed.pdf
mv "/some/dir/Signed/$signed_name" "$dir"
}
I have a folder called "week1", and in that folder there are about ten other folders that all contain multiple files, including one called "submit.pdf". I would like to be able to copy all of the "submit.pdf" files into one folder, ideally using Terminal to expedite the process. I've tried cp week1/*/submit.pdf week1/ as well as cp week1/*/*.pdf week1/, but it had only been ending up copying one file. I just realized that it has been writing over each file every time which is why I'm stuck with one...is there anyway I can prevent that from happening?
You don't indicate your OS, but if you're using Gnu cp, you can use cp week1/*/submit.pdf --backup=t week/ to have it (arbitrarily) number files that already exist; but, that won't give you any real way to identify which-is-which.
You could, perhaps, do something like this:
for file in week1/*/submit.pdf; do cp "$file" "${file//\//-}"; done
… which will produce files named something like "week1-subdir-submit.pdf"
For what it's worth, the "${var/s/r}" notation means to take var, but before inserting its value, search for s (\/, meaning /, escaped because of the other special / in that expression), and replace it with r (-), to make the unique filenames.
Edit: There's actually one more / in there, to make it match multiple times, making the syntax:
"${ var / / \/ / - }"
take "var" replace every instance of / with -
find to the rescue! Rule of thumb: If you can list the files you want with find, you can copy them. So try first this:
$ cd your_folder
$ find . -type f -iname 'submit.pdf'
Some notes:
find . means "start finding from the current directory"
-type -f means "only find regular files" (i.e., not directories)
-iname 'submit.pdf' "... with case-insensitive name 'submit.dpf'". You don't need to use 'quotation', but if you want to search using wildcards, you need to. E.g.:
~ foo$ find /usr/lib -iname '*.So*'
/usr/lib/pam/pam_deny.so.2
/usr/lib/pam/pam_env.so.2
/usr/lib/pam/pam_group.so.2
...
If you want to search case-sensitive, just use -name instead of -iname.
When this works, you can copy each file by using the -exec command. exec works by letting you specify a command to use on hits. It will run the command for each file find finds, and put the name of the file in {}. You end the sequence of commands by specifying \;.
So to echo all the files, do this:
$ find . -type f -iname submit.pdf -exec echo Found file {} \;
To copy them one by one:
$ find . -type f -iname submit.pdf -exec cp {} /destination/folder \;
Hope this helps!
I have a problem related to searching a pattern among several files.
I want to search "Logger." pattern in jsp files,so i used the command
grep -ir Logger. * | find . -name *.jsp
Now the problem i am facing is that this command is listing all the jsp files and its not searching the pattern "Logger." in jsp files and listing them.
I just want the jsp files in which "Logger." instance is present.
start like this
you want to search in jsp files.
find . -name "*.jsp"
the above will output all the jsp files recursively from current directory. like below
1/2/ahbd.jsp
befwej/dg/wefwefw/wefwefwe/ijn.jsp
And now you want to find the string in just these files.
grep -ir Logger. (output of find)
so the actual complete command becomes:
find . -name "*.jsp"|xargs grep -ir 'Logger.'
magic here is done by xargs
it gives the output of find as an input for grep line by line.
if you remove xargs,then only the first line that is 1/2/ahbd.jsp will be searched for the string.
there are several other ways to do this.But i feel more comfortable using this regularly
To recursively find all *.jsp files containing the string Logger. you can do:
find . -type f -name '*.jsp' -exec grep -l "Logger\." {} \;
grep -l means to print only the file name if the file contains the string.
The -exec switch of find will execute the given command for each file matching the other criteria (-type f and -name '*.jsp'). The string {} is substituted by the filename. Some versions of find also support + instead of {} to feed several file names to the command (like xargs does) and not only one at once, e.g.:
find . -type f -name '*.jsp' -exec grep -l "Logger\." + \;
You can just use grep for that, here's a command that should give you the results:
grep -ir "Logger\." * | grep ".jsp"
Problem is, grep will bail when you use ".jsp" instead or "" if you don't have at least one .jsp file into your root directory. So we have to tell him to look every file.
Since you give grep the -r (recursive) argument, it will walk the subdirectories to find the pattern "Logger.", then the second grep will only display the .jsp files. Note that -i tells grep not to care about the letter case, which is may be not what you want.
edit: following John's answer: we have to escape the . to prevent it to be taken as a regexp.
re-edit: actually, I think that using find is better, since it will filter the jsp files directly instead of grepping all the files:
find . -name "*.jsp" -exec grep -i "Logger\." {} \;
(you don't need the -r anymore since find takes care of recursion.
If you have bash 4+
shopt -s globstar
shopt -s nullglob
for file in **/*.jsp
do
if grep -q "Logger." "$file" ;then
echo "found in $file"
fi
# or just grep -l "Logger." "$file"
done