I have a folder with a bunch of log files. Each set of log files is in a folder detailing the time and date that the program was run. Inside these log folders, I've got some video files that I want to extract. All I want is the video files, nothing else. I tried using this command to only copy the video files, but it didn't work because a directory didn't exist.
.rmv is the file extension of the files I want.
$ find . -regex ".*\.rmv" -type f -exec cp '{}' /copy/to/here/'{}'
If I have a folder structure such as:
|--root
|
|--folder1
| |
| |--file.rmv
|
|--folder2
|
|--file2.rmv
How can I get it to copy to copy/to/here with it copying the structure of folder1 and folder2 in the destination directory?
cp has argument --parents
so the shortest way to do what you want is:
find root -name '*.rmv' -type f -exec cp --parents "{}" /copy/to/here \;
I would just use rsync.
The {} represents the full path of the found file, so your cp command evaluate to this sort of thing:
cp /root/folder1/file.rmv /copy/to/here/root/folder1/file.rmv
If you just drop the second {} it will instead be
cp /root/folder1/file.rmv /copy/to/here
the copy-file-to-directory form of cp, which should do the trick.
Also, instead of -regex, yor could just use the -name operand:
find root -name '*.rmv' -type f -exec cp {} /copy/to/here \;
Assuming src is your root and dst is your /copy/to/here
#!/bin/sh
find . -name *.rmv | while read f
do
path=$(dirname "$f" | sed -re 's/src(\/)?/dst\1/')
echo "$f -> $path"
mkdir -p "$path"
cp "$f" "$path"
done
putting this in cp.sh and running ./cp.sh from the directory over root
Output:
./src/folder1/file.rmv -> ./dst/folder1
./src/My File.rmv -> ./dst
./src/folder2/file2.rmv -> ./dst/folder2
EDIT: improved script version (thanks for the comment)
Related
I'm looking to extract informations from subfolders.
I have a folder containing several folders containing several folders with text file information.
I've done something like this, but it works only when text files have different names (otherwise files with same names are erased by the most recent ones):
mkdir target_directory
pwd=`pwd`
find $pwd . -name \*.txt -exec cp {} target_directory \;
cd target_directory
cat *.txt > all-info
rm *.txt
I was thinking to had directory to the name of extracted files. How can I do that?
Maybe there is a smarter way?
Thank you!
If your goal is to concatenate all *.txt files in target_directory/all-info then just use cat {} in the exec action of your find command and redirect the output:
$ mkdir -p target_directory
$ find . -type f -name '*.txt' -exec cat {} \; > target_directory/all-info
This should do the trick:
mkdir -p target_directory
find . -name "*.txt" -exec cat {} >> target_directory/all-info \;
Man-pages of cp mention:
-n, --no-clobber
do not overwrite an existing file (overrides a previous -i option)
So I think your solution (only the find part) should be:
find $pwd . -name \*.txt -exec cp -n {} target_directory \;
So I want to grab all certain files then duplicate them in the same folder/location but with a different extension. So far I have this from another question Copy all files with a certain extension from all subdirectories:
find . -name \*.js -exec cp {} newDir \;
I want to duplicate all those .js files into .ts e.g. duplicate functions.js to functions.ts wherever it may be.
more examples:
a/functions.js
b/test.js
c/another.js
index.js
to
a/functions.ts
b/test.ts
c/another.ts
index.ts
find . -name \*.js | while read jsfile; do cp "${jsfile}" "${jsfile%.js}.ts"; done
find . -name \*.js list all .js files
using read command to read each line from the output of fine command.
${jsfile%.js} means to remove the suffix .js from variable jsfile, for example, a/functions.js will become to a/functions
Here is how to assign variables using find and xargs and open up all sort of command-line options,
$ find . -name '*.js' | xargs -I {} bash -c 'p="{}"; cp $p newDir/$(basename ${p%.js}.ts)'
Use xargs -I {} to get the output of find as input to xargs. Use bash -c to execute a command.
Here is a demo:
$ mkdir -p a b c d newDir
$ touch a/1.js b/2.js c/three.js d/something.js
$ find . -name '*.js' | xargs -I {} bash -c 'p="{}"; cp $p newDir/$(basename ${p%.js}.ts)'
$ ls newDir/
1.ts 2.ts something.ts three.ts
EDIT (Question changed after hours of initial post). To keep a duplicate in the same directory use the same cp command and remove newDir and basename:
$ find . -name '*.js' | xargs -I {} bash -c 'p="{}"; cp $p ${p%.js}.ts'
I am searching specific directory and subdirectories for new files, I will like to copy the files. I am using this:
find /home/foo/hint/ -type f -mtime -2 -exec cp '{}' ~/new/ \;
It is copying the files successfully, but some files have same name in different subdirectories of /home/foo/hint/.
I will like to copy the files with its base directory to the ~/new/ directory.
test#serv> find /home/foo/hint/ -type f -mtime -2 -exec ls '{}' \;
/home/foo/hint/do/pass/file.txt
/home/foo/hint/fit/file.txt
test#serv>
~/new/ should look like this after copy:
test#serv> ls -R ~/new/
/home/test/new/pass/:
file.txt
/home/test/new/fit/:
file.txt
test#serv>
platform: Solaris 10.
Since you can't use rsync or fancy GNU options, you need to roll your own using the shell.
The find command lets you run a full shell in your -exec, so you should be good to go with a one-liner to handle the names.
If I understand correctly, you only want the parent directory, not the full tree, copied to the target. The following might do:
#!/usr/bin/env bash
findopts=(
-type f
-mtime -2
-exec bash -c 'd="${0%/*}"; d="${d##*/}"; mkdir -p "$1/$d"; cp -v "$0" "$1/$d/"' {} ./new \;
)
find /home/foo/hint/ "${findopts[#]}"
Results:
$ find ./hint -type f -print
./hint/foo/slurm/file.txt
./hint/foo/file.txt
./hint/bar/file.txt
$ ./doit
./hint/foo/slurm/file.txt -> ./new/slurm/file.txt
./hint/foo/file.txt -> ./new/foo/file.txt
./hint/bar/file.txt -> ./new/bar/file.txt
I've put the options to find into a bash array for easier reading and management. The script for the -exec option is still a little unwieldy, so here's a breakdown of what it does for each file. Bearing in mind that in this format, options are numbered from zero, the {} becomes $0 and the target directory becomes $1...
d="${0%/*}" # Store the source directory in a variable, then
d="${d##*/}" # strip everything up to the last slash, leaving the parent.
mkdir -p "$1/$d" # create the target directory if it doesn't already exist,
cp "$0" "$1/$d/" # then copy the file to it.
I used cp -v for verbose output as shown in "Results" above, but IIRC it's also not supported by Solaris, and can be safely ignored.
The --parents flag should do the trick:
find /home/foo/hint/ -type f -mtime -2 -exec cp --parents '{}' ~/new/ \;
Try testing with rsync -R, for example:
find /your/path -type f -mtime -2 -exec rsync -R '{}' ~/new/ \;
From the rsync man:
-R, --relative
Use relative paths. This means that the full path names specified on the
command line are sent to the server rather than just the last parts of the
filenames.
The problem with the answers by #Mureinik and #nbari might be that the absolute path of new files will spawn in the target directory. In this case you might want to switch to the base directory before the command and go back to your current directory afterwards:
path_current=$PWD; cd /home/foo/hint/; find . -type f -mtime -2 -exec cp --parents '{}' ~/new/ \; ; cd $path_current
or
path_current=$PWD; cd /home/foo/hint/; find . -type f -mtime -2 -exec rsync -R '{}' ~/new/ \; ; cd $path_current
Both ways work for me at a Linux platform. Let’s hope that Solaris 10 knows about rsync’s -R ! ;)
I found a way around it:
cd ~/new/
find /home/foo/hint/ -type f -mtime -2 -exec nawk -v f={} '{n=split(FILENAME, a, "/");j= a[n-1];system("mkdir -p "j"");system("cp "f" "j""); exit}' {} \;
I use the find command to copy some files from one destination to another. If I do
$ mkdir dir1 temp
$ touch dir1/dir1-file1
$ find . -iname "*file*" -exec cp {} temp/ \;
everything works fine as expected, but if I do
$ mkdir SR0a temp
$ touch SR0a/SR0a-file1
$ find . -iname "*file*" -exec cp {} temp/ \;
> cp: `./temp/SR0a-file1' and `temp/SR0a-file1' are the same file
I get an error message. I do not understand this behavior. Why do I get an error by simply changing names?
That is because find searchs in SR0a/ folder at first, and then in temp/, and since you have copied into it the file, find founds it again in temp/ folder. It seems that find uses crafty sorting so it just should be take into account on use of find:
$ mkdir temp dir1 SR0a DIR TEMP
$ find .
.
./TEMP
./SR0a
./temp
./dir1
./DIR
So in case the dir1/ find founds the it at first, and this don't make such problems, let see the search sequence:
temp/
dir1/
When you search with SR0a the sequence is:
SR0a/
temp/
so found file is being copied into temp before searching it.
To fix it, either move temp/ folder outside the current one:
$ mkdir SR0a ../temp
$ touch SR0a/SR0a-file1
$ find . -iname "*file*" -exec cp {} ../temp/ \;
or use pipe to separate find and copy procedures:
$ find . -iname "*file*" | while read -r i; do cp "$i" temp/; done
This find should work:
find . -path ./temp -prune -o -iname "*file*" -type f -exec cp '{}' temp/ \;
-path ./misc -prune -o is used to skip ./temp directory while copying files to temp folder.
Your find command is also finding ./temp/*file* files and trying to copy them also into ./temp folder.
It is caused by the find that is trying to copied to it self.
Pipe output using while to separate with find command
Use cp with the option: -frpvT for match with file/dir target path
Print the realpath of the ouput file, see if the file path are the same.
find . -iname "*file*" | while read -r f; do echo cp -frpvT "$(realpath $f)" "/temp/$f"; done
If so, then correct the file path, when it is done then you can remove the echo from the command.
I have a directory which contains multiple sub-directories with mov and jpg files.
/dir/
/subdir-a/ # contains a-1.jpg, a-2.jpg, a-1.mov
/subdir-b/ # contains b-1.mov
/subdir-c/ # contains c-1.jpg
/subdir-d/ # contains d-1.mov
... # more directories with the same pattern
I need to find a way using command-line tools (on Mac OSX, ideally) to move all the mov files to a new location. However, one requirement is to keep directory structure i.e.:
/dir/
/subdir-a/ # contains a-1.mov
/subdir-b/ # contains b-1.mov
# NOTE: subdir-c isn't copied because it doesn't have mov files
/subdir-d/ # contains d-1.mov
...
I am familiar with find, grep, and xargs but wasn't sure how to solve this issue. Thank you very much beforehand!
It depends slightly on your O/S and, more particularly, on the facilities in your version of tar and whether you have the command cpio. It also depends a bit on whether you have newlines (in particular) in your file names; most people don't.
Option #1
cd /old-dir
find . -name '*.mov' -print | cpio -pvdumB /new-dir
Option #2
find . -name '*.mov' -print | tar -c -f - -T - |
(cd /new-dir; tar -xf -)
The cpio command has a pass-through (copy) mode which does exactly what you want given a list of file names, one per line, on its standard input.
Some versions of the tar command have an option to read the list of file names, one per line, from standard input; on MacOS X, that option is -T - (where the lone - means 'standard input'). For the first tar command, the option -f - means (in the context of writing an archive with -c, write to standard output); in the second tar command, the -x option means that the -f - means 'read from standard input'.
There may be other options; look at the manual page or help output of tar rather carefully.
This process copies the files rather than moving them. The second half of the operation would be:
find . -name '*.mov' -exec rm -f {} +
ASSERT: No files have newline characters in them. Spaces, however, are AOK.
# TEST FIRST: CREATION OF FOLDERS
find . -type f -iname \*.mov -printf '%h\n' | sort | uniq | xargs -n 1 -d '\n' -I '{}' echo mkdir -vp "/TARGET_FOLDER_ROOT/{}"
# EXECUTE CREATION OF EMPTY TARGET FOLDERS
find . -type f -iname \*.mov -printf '%h\n' | sort | uniq | xargs -n 1 -d '\n' -I '{}' mkdir -vp "/TARGET_FOLDER_ROOT/{}"
# TEST FIRST: REVIEW FILES TO BE MOVED
find . -type f -iname \*.mov -exec echo mv {} /TARGET_FOLDER_ROOT/{} \;
# EXECUTE MOVE FILES
find . -type f -iname \*.mov -exec mv {} /TARGET_FOLDER_ROOT/{} \;
Being large files, if they are on the same file system you don't want to copy them, but just to replicate their directory structure while moving.
You can use this function:
# moves a file (or folder) preserving its folder structure (relative to source path)
# usage: move_keep_path source destination
move_keep_path () {
# create directories up to one level up
mkdir -p "`dirname "$2"`"
mv "$1" "$2"
}
Or, adding support to merging existing directories:
# moves a file (or folder) preserving its folder structure (relative to source path)
# usage: move_keep_path source destination
move_keep_path () {
# create directories up to one level up
mkdir -p "`dirname "$2"`"
if [[ -d "$1" && -d "$2" ]]; then
# merge existing folder
find "$1" -depth 1 | while read file; do
# call recursively for all files inside
mv_merge "$file" "$2/`basename "$file"`"
done
# remove after merge
rmdir "$1"
else
# either file or non-existing folder
mv "$1" "$2"
fi
}
It is easier to just copy the files like:
cp --parents some/folder/*/*.mov new_folder/
from the parent directory of "dir execute this:
find ./dir -name "*.mov" | xargs tar cif mov.tar
Then cd to the directory you want to move the files to and execute this:
tar xvf /path/to/parent/directory/of"dir"/mov.tar
This should work if you want to move all mov files to a directory called new location -
find ./dir -iname '*.mov' -exec mv '{}' ./newlocation \;
However, if you wish to move the mov files along with their sub-dirs then you can do something like this -
Step 1: Copy entire structure of /dir to a new location using cp
cp -iprv dir/ newdir
Step 2: Find jpg files from newdir and delete them.
find ./newdir -iname "*.jpg" -delete
Test:
[jaypal:~/Temp] ls -R a
a.mov aa b.mov
a/aa:
aaa c.mov d.mov
a/aa/aaa:
e.mov f.mov
[jaypal:~/Temp] mkdir d
[jaypal:~/Temp] find ./a -iname '*.mov' -exec mv '{}' ./d \;
[jaypal:~/Temp] ls -R d
a.mov b.mov c.mov d.mov e.mov f.mov
I amended the function of #djjeck, because it didn't work as I needed. The function below moves a source file to a destination directory also creating the needed levels of hierarchy in the source file path (see the example below):
# moves a file, creates needed levels of hierarchy in destination
# usage: move_with_hierarchy source_file destination top_level_directory
move_with_hierarchy () {
path_tail=$(dirname $(realpath --relative-to="$3" "$1"))
cd "$2"
mkdir -p $path_tail
cd - > /dev/null
mv "$1" "${2}/${path_tail}"
}
example:
$ ls /home/sergei/tmp/dir1/dir2/bla.txt
/home/sergei/tmp/dir1/dir2/bla.txt
$ rm -rf tmp2
$ mkdir tmp2
$ move_with_hierarchy /home/sergei/tmp/dir1/dir2/bla.txt /home/sergei/tmp2 /home/sergei/tmp
$ tree ~/tmp2
/home/sergei/tmp2
└── dir1
└── dir2
└── bla.txt
2 directories, 1 file