I have two directories structured as follows:
dir1/a/file1
dir1/a/b/file2
dir1/a/c/d/file3
and
dir2/a/file4
dir2/a/b/file5
dir2/a/c/d/file6
I want to copy all the files in the subdirectories under dir1 to dir2, but keep the files that are currently in dir2, in other words I want to resulting structure to look like:
dir2/a/file1
dir2/a/file4
dir2/a/b/file2
dir2/a/b/file5
dir2/a/c/d/file3
dir2/a/c/d/file6
Is there a simple way to do this using bash?
You could start with
cd dir1
cp -rpuv * ../dir2/
Before:
$ find dir2/
dir2/
dir2/a
dir2/a/file4
dir2/a/c
dir2/a/c/d
dir2/a/c/d/file6
dir2/a/b
dir2/a/b/file5
After:
$ find dir2/
dir2/
dir2/a
dir2/a/file1
dir2/a/file4
dir2/a/c
dir2/a/c/d
dir2/a/c/d/file3
dir2/a/c/d/file6
dir2/a/b
dir2/a/b/file2
dir2/a/b/file5
Note that -p preserves permissions, -v make copy verbose and -u only updates files (doing what the question suggests: keep the files already in dir2)
Related
I need a way to recursively iterate through a directory and copy contents over to another directory following the same directory structure.
For instance, I have the below:
src1/
dir1/
sub-dir1.1/
file1.1
dir2/
sub-dir2.1/
file1
Now, I have another directory elsewhere, which has a similar structure as the above but with less/more files + directories.
src2/
dir1/
sub-dir1.1/
file1.1
file1.2
dir2/
dir3/
file4
I need to copy all contents over from src1 to src2 including all sub-directories under dir1 and dir2. Is there a way to do rather than drilling down to each file under N subdirectories and copying files?
I tried using a "cp -r" but it gives me a "cp: will not overwrite just-created"
Thanks for any insight into this.
The rsync command is well suited to this.
Something like this might achieve what you want:
rsync -av src1/ src2/
Try using the -n flag to begin with to run it in "dry-run" or test mode. This shows you what files it will copy without actually making any changes.
Try rsync -aing the two folders
I have directory with content (example)
/dir1/a/b/c/file1
/dir1/a/b/c/file2
/dir1/a/d/file3
/dir1/a/e/file4
/dir1/f/dir3/
/dir1/f/dir4/
...
I have list of files and directories, which can be removed - for example file1,file3 and dir3
I would like to move(move, not copy nor tar them - files are large and i need to do it in short time) them to another directory /dir2 (on the same filesystem), but - preserving subdirectories:
/dir1/a/b/c/file1 -> /dir2/a/b/c/file1
/dir1/a/d/file3 -> /dir2/a/d/file3
/dir1/f/dir3/ -> /dir2/f/dir3/
Is there any better way than for each file and directory(for directories skipping last part) create directory in dir2(using mkdir -p/install -d) and then moving it into?
one of simplest solutions is using rsync, with list of files in
--include-from, and with --remove-source-files. But - it copy files, and then remove then - i need to avoid copying - for large files it
take too much time.
If you are comfortable with rsync, you can use it just to list the files and then process that list with this short shell script:
cd dir1
rsync --files-from list --list-only --no-implied-dirs . / |
while read mode size date time path
do
dest=$dir2/`dirname $path` # $dir2 must be an absolute path
mkdirhier $dest
mv $path $dest
done
I have tried this code with example you mentioned above and it worked okay.. Please test it before you use it. In second line, you have to put all file names in a plain text file and provide it's path. My file contents are shown below
#!/bin/ksh
c_file="Path_to_the_file_containing_list_for_movement"
while IFS= read v_line
do
v_fullfilepath=$(find $1 -name "$v_line")
v_dirname=$(dirname $v_fullfilepath)
v_target_path=${v_dirname/$1\//$2/}
mkdir -p "$v_target_path"
mv $v_fullfilepath $v_target_path
#echo $v_line " " $v_fullfilepath " " $v_dirname " " $v_target_path
done <"$c_file"
This was my file contents,
file1
file3
dir3
Supposed I have directory structure like
src/
src/a/
src/a/1.ocf
src/a/1.pdf
src/a/1.txt
src/b/
src/b/2.ocf
src/b/2.pdf
src/b/2.xls
src/c/
src/c/3.doc
src/c/3.ocf
src/c/3.txt
src/d/
Then, I just want to synchronize only files with extension *.txt. So, I tried to use command like:
#rsync -avvH --include="*/" --include="*.txt" --exclude="*" src/ dst/
sending incremental file list
./
a/
a/1.txt
b/
c/
c/3.txt
d/
Unfortunately, this command not only synchronize *.txt file but also all directory. I don't want directory 'b' and 'd' be synchronized because it not contain file *.txt
Is there simple way to do that?
The option you're looking for is -m to prune empty directories:
rsync -avvHm --include="*/" --include="*.txt" --exclude="*" src/ dst/
I have a question concerning the "cp"-command:
I have to copy folders to a directory. For me two different scenarios exist. In the first scenario folders with the same name don't exist. That's simple so far:
ls test/folder1/
file1 file2
ls test/destination/
cp -r -v test/folder1/ test/destination/
»test/folder1/“ -> »test/destination/folder1“
»test/folder1/file1“ -> »test/destination/folder1/file1“
»test/folder1/file2“ -> »test/destination/folder1/file2“
In the second scenario a folder with the same name exists. These files have different names than the ones in the source folder. What I actually want is that the folder in the destination directory will be completely replaced by the source folder (if files with the same name exist they should be overriden; if files don't exist in the source directory they should be deleted).
ls test/folder1/
file1 file2
ls test/destination/
folder1
ls test/destination/folder1/
file3 file4
cp -r -v test/folder1/ test/destination/
»test/folder1/file1“ -> »test/destination/folder1/file1“
»test/folder1/file2“ -> »test/destination/folder1/file2“
ls test/destination/folder1/
file1 file2 file3 file4
It would probably be possible with something like
if [ -d destination/$foldername ]
then
rm -r /destination/$foldername
cp-r -v test/$foldername/ test/destination/$foldername
else
cp-r -v test/$foldername/ test/destination/$foldername
fi
but I was wondering if there is a better solution to that.
Thanks already!
In you second case you want to use:
cp -av test/$foldername/* test/destination/$foldername
That will copy the contents of test/$foldername to test/destination/$foldername overwriting any files of the same name that exist in test/destination/$foldername. Additionally, to preserve any (heaven forbid) spaces in $foldername you can quote the path/filename, but NOT the *:
cp -av "test/$foldername/"* "test/destination/$foldername"
I found a solution for my question in Windows but I'm using Ubuntu: How to copy a directory structure but only include certain files using Windows batch files?
As the title says, how can I recursively copy a directory structure but only include some files? For example, given the following directory structure:
folder1
folder2
folder3
data.zip
info.txt
abc.xyz
folder4
folder5
data.zip
somefile.exe
someotherfile.dll
The files data.zip and info.txt can appear everywhere in the directory structure. How can I copy the full directory structure, but only include files named data.zip and info.txt (all other files should be ignored)?
The resulting directory structure should look like this:
copy_of_folder1
folder2
folder3
data.zip
info.txt
folder4
folder5
data.zip
Could you tell me a solution for Ubuntu?
$ rsync --recursive --include="data.zip" --include="*.txt" --filter="-! */" dir_1 copy_of_dir_1
To exclude dir3 regardless of where it is in the tree (even if it contains files that would match the --includes):
--exclude 'dir3/' (before `--filter`)
To exclude dir3 only at at specific location in the tree, specify an absolute path, starting from your source dir:
--exclude '/dir1/dir2/dir3/' (before `--filter`)
To exclude dir3 only when it's in dir2, but regardless of where dir2 is:
--exclude 'dir2/dir3/' (before `--filter`)
Wildcards can also be used in the path elements where * means a directory with any name and ** means multiple nested directories.
To specify only files and dirs to include, run two rsyncs, one for the files and one for the dirs. The problem with getting it done in a single rsync is that when you don't include a dir, rsync won't enter the dir and so won't discover any files in that branch that may be matching your include filter. So, you start by copying the files you want while not creating any dirs that would be empty. Then copy any dirs that you want.
$ rsync --recursive --prune-empty-dirs --include="*.txt" --filter="-! */" dir_1 copy_of_dir_1
$ rsync --recursive --include '/dir1/dir2/' --include '/dir3/dir4/' --filter="-! */" dir_1 copy_of_dir_1
You can combine these if you don't mind that your specified dirs don't get copied if they're empty:
$ rsync --recursive --prune-empty-dirs --include="*.txt" --include '/dir1/dir2/' --include '/dir3/dir4/' --filter="-! */" dir_1 copy_of_dir_1
The --filter="-! */" is necessary because rsync includes all files and folders that match none of the filters (imagine it as an invisible --include filter at the end of the list of filters). rsync checks each item to be copied against the list of filters and includes or excludes the item depending on the first match it finds. If there's no match, it hits that invisible --include and goes on to include the item. We wanted to change this default to --exclude, so we added an exclude filter (the - in -! */), then we negate the match (!) and match all dirs (*/). Since this is a negated match, the result is that we allow rsync to enter all the directories (which, as I mentioned earlier, allows rsync to find the files we want).
We use --filter instead of --exclude for the final filter because --exclude does not allow specifying negated matches with the ! operator.
I don't have a beautiful one liner, but since nobody else has answered you can always:
find . -name 'file_name.extension' -print | cpio -pavd /path/to/receiving/folder
For each specific file after copying the directories.
(Make sure you're in the original folder first, of course! :) )
Here is a one-liner using rsync:
rsync -a -f"+ info.txt" -f"+ data.zip" -f'-! */' folder1/ copy_of_folder1/
If you already have a file list, and want a more scalable solution
cat file.list | xargs -i rsync -a -f"+ {}" -f'-! */' folder1/ copy_of_folder1/
cp -pr folder1 copy_of_folder1; find copy_of_folder1 -type f ! \( -name data.zip -o -name info.txt \) -exec rm -f {} \;
first time : copy entirely folder1 to copy_of_folder1
second time : erase all files differents from data.zip and
info.txt
At the end, you have your complete structure with only the file data.zip and info.txt