I am having 35K + odd files in multiple directories and subdirectories. There are 1000 odd files (.c .h and other file names) with a unique string "FOO" in the FILE CONTENT. I am trying to copy those files alone (with the directory structure preserved) to a different directory 'MEOW'. Can some one look in to my bash execution and correct my mistake
for Eachfile in `find . -iname "*FOO*" `
do
cp $Eachfile MEOW
done
getting the following error
./baash.sh: line 2: syntax error near unexpected token `$'do\r''
'/baash.sh: line 2: `do
To find all files under the current directory with the string "FOO" in them and copy them to directory MEOW, use:
grep --null -lr FOO . | xargs -0 cp -t MEOW
Unlike find, grep looks in the contents of a file. The -l option to grep tells it to list file names only. The -r option tells it to look recursively in subdirectories. Because file names can have spaces and other odd characters in them, we give the --null option to grep so that, in the list it produces, the file names are safely separated by null characters. xargs -0 reads from that list of null-separated file names and provides them as argument to the command cp -t MEOW which copies them to the target directory MEOW.
In case you only want to search for the string FOO in .c and .h file then
find ./ -name "*\.c" -o -name "*\.h" -exec grep -l FOO {} \; | xargs cp -t MEOW/
For me its working even without --null option to xrags, if doesn't for you.. then append -0 in xargs part as follow:
xargs -0 cp -t MEOW/
for file in $(find -name '*.c' -o -name '*.h' -exec grep -l 'FOO' {} \;); do
dname=$(dirname "$file")
dest="MEOW/$dname"
mkdir -p $dest
cp "$file" "$dest"
done
Related
Hi guys i was building a script to order my files related to my studies file, but i don't understand why the prompt give me this error
error 1.1
mv: cannot stat 'filefilefilefilefilefilefilefilefilefilefilefile.pdf'$'\n': File name too long
that's mean i have to rename all long files? exists an other way to prevent this error?
the example below it's the script that has generated the error
Script 1 - move all greped files that contain business inside their name file and move them inside auto_folder_business
mkdir -p /mnt/c/Users/alber/Desktop/testfileorder/auto_folder_business
ls /mnt/c/Users/alber/Desktop/testfileorder | egrep -i 'business.' | xargs -0 -I '{}' mv '{}' /mnt/c/Users/alber/Desktop/testfileorder/auto_folder_business
In the example above I had also this other error
error 1.2
xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option
that i solved inserting -0 option,
despite this i tried to generalize this process writing this snippet
script 2 - move all greped files that contain the inserted keyword inside their name file and move them inside auto_folder_business
#!/bin/sh
read -p "file to order: --> " fetching_keyword
mypath=/mnt/c/Users/alber/Desktop/testfileorder/auto_folder_$fetching_keyword/
echo $mypath
mkdir -p $mypath
ls /mnt/c/Users/alber/Desktop/testfileorder |
egrep -i "$fetching_keyword" |
xargs -0 -I {} mv -n {} $mypath
also here I have an other error I think they are related
error 2
mv: cannot stat 'Statino (1).pdf'$'\n''Statino (2).pdf'$'\n''Statino (3).pdf'$'\n''Statino (4).pdf'$'\n''Statino.pdf'$'\n''auto_folder_statino'$'\n': No such file or directory
xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option
I'm not understanding what I'm doing wrong...
xargs -0 expects to read null-terminated lines from standard input. ls and egrep are both writing newline-terminated lines, so their entire output is being read as a single input here.
The quickest fix is to utilize find -print0.
find /mnt/c/Users/alber/Desktop/testfileorder -iname "*${fetching_keyword}*" -print0 | \
xargs -0 -I {} mv -n {} "$mypath"
...but in this specific case, you might just want to use -exec at that point.
find /mnt/c/Users/alber/Desktop/testfileorder -iname "*${fetching_keyword}*" \
-exec mv -n {} "${mypath}" \;
I am writing a script to search all files from a directory/root based on a keyword input.
echo "Enter keyword"
read key
grep -r -l . -e "$key"
If I search for "hei", which I know is located in a file called mem.c, it prints /folder/mem.c, and not only the filename. Do I have some wrong arguments or is it supposed to be like that?
Additional question, is there a way to store these filenames, and copy them into another directory if there is a match in the keyword? Or maybe it is possible to loop through the files found with grep?
This is what grep -l does: print the path of matching files. If you want to print only the base name of the files you can pipe the result to some command that does that. Example:
grep -rlZ "$key" . | xargs -0 -n1 basename
Storing the file names in a file is just a matter of redirection:
grep -rlZ "$key" . | xargs -0 -n1 basename > mylist.txt
To copy the matching files somewhere you will need their path, not just their base name. Assuming you want to copy them all in the same destdir directory and you have no name conflicts, you could use find:
find . -type f -exec grep -l "$key" {} \; -exec cp -f {} destdir \;
I am using find to list files within multiple directories with a specific extension. I tried
find /path/to/encompassing/directory/ -d -name "*modified.tif" | xargs cp Destination_Directory/
but it didn't work. Using
find /path/ -d -name "*modified.tif" -type f -exec cp {} Destination_Directory \;
works but I don't understand why xargs isn't working.
If you write
find -name '*modified.tif' | xargs cp directory
then that's the same as writing
cp directory file1modified.tif file2modified.tif
(or whatever filenames matched), which is the wrong way around, because xargs by default appends arguments.
find -name '*modified.tif' -exec cp {} directory \;
is the same as
cp file1modified.tif directory
cp file2modified.tif directory
which is what you want.
You can achieve the same with xargs by using
xargs -I{} cp {} directory
to specify where in the command you want to use the argument, but that implies that only one file at a time will be copied (because -I implies -L1).
To avoid calling cp once per file, you can use the -t option for cp so the files to be copied can be appended to the end of the command (requires GNU cp):
find -name '*modified.tif' | xargs cp -t directory
which is equivalent to
cp -t directory file1modified.tif file2modified.tif
or better, taking care of blanks in filenames,
find -name '*modified.tif' -print0 | xargs -0 cp -t directory
Alternatively, without xargs:
find -name '*modified.tif' -exec cp -t directory {} +
where -exec {} + makes sure to invoke cp as few times as possible.
xargs passes each word from its standard input as the last argument to cp, not the first. As a result, you are trying to run the series of commands
cp Destination_Directory/ foo
cp Destination_Directory/ bar
# etc
If you are using GNU cp, you can fix this simply by using the -t option to specify that Destination_Directory is the target, rather than a source.
... | xargs cp -t Destination_Directory
# cp -t Destination_Directory foo
# cp -t Destination_Directory bar
# etc
You might be able to use the -I option in xargs to make it use the incoming file name as the first argument:
... | xargs -I '{}' cp '{}' Destination_Directory
however, this makes a lot of assumptions about the names find will produce. (No leading or trailing whitespace, and no newlines in the file names.) (For that matter, xargs without -I is treating each whitespace-delimited word from its input as a separate argument for a call to cp.) In general, you should not try to use the output of find programmatically. Stick with its -exec primary instead.
Your code
find /path/ -d -name "*modified.tif" -type f -exec cp {} Destination_Directory \;
is the right way to go. No shell is involved, so each file name is passed as is as the first argument to cp.
I don't use xargs, but I think it should work like this :
cp `find /path/to/encompassing/directory/ -d -name "*modified.tif"` Destination_Directory/
No need for a pipe then.
So I want to grab all certain files then duplicate them in the same folder/location but with a different extension. So far I have this from another question Copy all files with a certain extension from all subdirectories:
find . -name \*.js -exec cp {} newDir \;
I want to duplicate all those .js files into .ts e.g. duplicate functions.js to functions.ts wherever it may be.
more examples:
a/functions.js
b/test.js
c/another.js
index.js
to
a/functions.ts
b/test.ts
c/another.ts
index.ts
find . -name \*.js | while read jsfile; do cp "${jsfile}" "${jsfile%.js}.ts"; done
find . -name \*.js list all .js files
using read command to read each line from the output of fine command.
${jsfile%.js} means to remove the suffix .js from variable jsfile, for example, a/functions.js will become to a/functions
Here is how to assign variables using find and xargs and open up all sort of command-line options,
$ find . -name '*.js' | xargs -I {} bash -c 'p="{}"; cp $p newDir/$(basename ${p%.js}.ts)'
Use xargs -I {} to get the output of find as input to xargs. Use bash -c to execute a command.
Here is a demo:
$ mkdir -p a b c d newDir
$ touch a/1.js b/2.js c/three.js d/something.js
$ find . -name '*.js' | xargs -I {} bash -c 'p="{}"; cp $p newDir/$(basename ${p%.js}.ts)'
$ ls newDir/
1.ts 2.ts something.ts three.ts
EDIT (Question changed after hours of initial post). To keep a duplicate in the same directory use the same cp command and remove newDir and basename:
$ find . -name '*.js' | xargs -I {} bash -c 'p="{}"; cp $p ${p%.js}.ts'
I want to make a search for all .fits files that contain a certain text in their name and then copy them to a directory.
I can use a command called fetchKeys to list the files that contain say 'foo'
The command looks like this : fetchKeys -t 'foo' -F | grep .fits
This returns a list of .fits files that contain 'foo'. Great! Now I want to copy all of these to a directory /path/to/dir. There are too many files to do individually , I need to copy them all using one command.
I'm thinking something like:
fetchKeys -t 'foo' -F | grep .fits > /path/to/dir
or
cp fetchKeys -t 'foo' -F | grep .fits /path/to/dir
but of course neither of these works. Any other ideas?
If this is on Linux/Unix, can you use the find command? That seems very much like fetchkeys.
$ find . -name "*foo*.fit" -type f -print0 | while read -r -d $'\0' file
do
basename=$(basename $file)
cp "$file" "$fits_dir/$basename"
done
The find command will find all files that match *foo*.fits in their name. The -type f says they have to be files and not directories. The -print0 means print out the files found, but separate them with the NUL character. Normally, the find command will simply return a file on each line, but what if the file name contains spaces, tabs, new lines, or even other strange characters?
The -print0 will separate out files with nulls (\0), and the read -d $'\0' file means to read in each file separating by these null characters. If your files don't contain whitespace or strange characters, you could do this:
$ find . -name "*foo*.fit" -type f | while read file
do
basename=$(basename $file)
cp "$file" "$fits_dir/$basename"
done
Basically, you read each file found with your find command into the shell variable file. Then, you can use that to copy that file into your $fits_dir or where ever you want.
Again, maybe there's a reason to use fetchKeys, and it is possible to replace that find with fetchKeys, but I don't know that fetchKeys command.
Copy all files with the name containing foo to a certain directory:
find . -name "*foo*.fit" -type f -exec cp {} "/path/to/dir/" \;
Copy all files themselves containing foo to a certain directory (solution without xargs):
for f in `find . -type f -exec grep -l foo {} \;`; do cp "$f" /path/to/dir/; done
The find command has very useful arguments -exec, -print, -delete. They are very robust and eliminate the need to manually process the file names. The syntax for -exec is: -exec (what to do) \;. The name of the file currently processed will be substituted instead of the placeholder {}.
Other commands that are very useful for such tasks are sed and awk.
The xargs tool can execute a command for every line what it gets from stdin. This time, we execute a cp command:
fetchkeys -t 'foo' -F | grep .fits | xargs -P 1 -n 500 --replace='{}' cp -vfa '{}' /path/to/dir
xargs is a very useful tool, although its parametrization is not really trivial. This command reads in 500 .fits files, and calls a single cp command for every group. I didn't tested it to deep, if it doesn't go, I'm waiting your comment.