I am using a NAS to backup my file server. The NAS exports /share/Backup via NFS, which is mounted on the fileserver as /mount/qnap. I want to keep track which files are rsynced but exclude the Backup-Dir, which contains many small files.Therefore I am running two instances of rsync, one with -v and another one without. The following command works as it should, after executing it the directory structure on /mount/qnap is identical to /mount/btrfs-raid.
rsync --delete -av --exclude Backup /mnt/btrfs-raid/ /mnt/qnap/
Rsyncing the Backup folder with the command
rsync --delete -av /mnt/btrfs-raid/Backup /mnt/qnap/Backup
produces the following directory structure on the NAS:
/mnt/qnap/Backup/Backup/..Subdirectories
To get the result I want I have to delete the last "Backup" from the target directory path:
rsync --delete -av /mnt/btrfs-raid/Backup /mnt/qnap/
Why does the second example not work like the first one?
Thanks
Stefan
Trailing slashes in paths are important for rsync. See the documentation.
rsync -avz foo:src/bar /data/tmp
This would recursively transfer all files from the directory src/bar on the machine foo into the /data/tmp/bar directory on the local machine. The files are transferred in "archive" mode, which ensures that symbolic links, devices, attributes, permissions, ownerships, etc. are preserved in the transfer. Additionally, compression will be used to reduce the size of data portions of the transfer.
rsync -avz foo:src/bar/ /data/tmp
A trailing slash on the source changes this behavior to avoid creating an additional directory level at the destination. You can think of a trailing / on a source as meaning "copy the contents of this directory" as opposed to "copy the directory by name", but in both cases the attributes of the containing directory are transferred to the containing directory on the destination. In other words, each of the following commands copies the files in the same way, including their setting of the attributes of /dest/foo:
Related
Good morning,
I want to transfer specific files in a directory to a remote machine while keeping the architecture of the subdirectories. Moreover, I only want to transfer files that have a peculiar extension (e.g. ".txt").
I have tried the following:
rsync -aP --include *.txt ./sourceDirectory user#hostIP:destDirectory
but it copies to the destination (destDirectory) all the files, and not only those which match the .txt pattern.
Could anybody help me with such riddle?
P.S.: obviously, the directory sourceDirectory contains subdirectories where are located my .txt files.
I used to have the same problem when rsync videos to home NAS. If you read the manual of rsync, the --include flag really means "Do not exclude". It actually has to work together with a --exclude flag.
The follow command will do your job:
rsync -aP --include="*/" --include="*.txt" --exclude="*" sourceDirectory destDirectory
The command literally means exclude everything, except for subfolders and txt file.
I'm loging to a remote server which has dir structure as follows-
cd /exp/gc/data/tmp/
ABCD
GED
TROOP
These directories in turn may or may not have subdirectories.My requirement is to copy files from remote server to hadoop(final destination) one at a time keeping the directory structure same as source.
eg:
source:
cd /exp/gc/data/tmp/ABCD
now ABCD has 3 files and two subdirectories.So my target location(hadoop) should also have same directory structure i.e. ABCD must have 3 files and two subdir.
Using rsync iam able to replicate the directory structure but directories with spaces or special character in the name are not getting copied.
Code snippet:
result = $(ssh username#hostname "find /path/ -type f")
for file in $result
do
rsync -arsv username#hostname /path-source/ /target-path-tmp/
hadoop fs -copyFromLocal /target-path-tmp/ /hadoop -location/
rm -rf /target-path-tmp/*
I think what you are looking for is rsync, try it as follow:
rsync -avz foo:src/bar /data/tmp
This would recursively transfer all files from the directory src/bar on the machine foo into the /data/tmp/bar directory on the local machine. The files are transferred in archive permissions, ownerships, etc. are preserved in the transfer. Additionally, compression will be used to reduce the size of data portions of the transfer.
example
I'd like to be able to have rsync copy files I have listed in my bash script and then delete any files that do not exist in the source directory (if i change the file list in the script). Expanded the command looks like this:
rsync -axSR --delete src_dir1/file1 src_dir2/file2 dst_dir/
However, rsync is not deleting extraneous files. Is there a way to do this?
With this syntax, it could be because you are using multiple sources, and those sources are actual files.
Why not just sync the parent directory of src_dir to dst_dir?
Using rsync, my source directory has a number of files and directories. My destination already has been synced, so it mirrors those files and directories.
However, I have manually created a symlink in my destination that does not exist in my source.
I need to use the --delete operation in rsync. Is there a way to get rsync to not remove the symlink?
There is no option to achieve this the way you suggested BUT the simplest solution to the described problem is just to add the filenames of the symlinks to the rsync exclude pattern
like: --exclude="folder/symlinkname1" --exclude="folder/symlinkname2"
if there are many symlinks you can keep a list of them in a exclude pattern file
this file may be autogenerated by a little script or a bash one liner...
Does your destination existing symlink point to a directory or file? If a directory, you may be able to use --keep-dirlinks.
I am using RSync to copy tar balls to an external hard drive on a Windows XP machine.
My files are tar.gz files (perms 600) in a directory (perms 711).
However, when I do a dry-run, only the folders are returned, the files are ignored.
I use RSync a lot, so I presume there is no issue with my installation.
I have tried changing permissions of the files but this makes no difference
The owner of the files is root, which is also the user which the script logs in as
I am not using Rsync's CVS option
The command I am using is:
rsync^
-azvr^
--stats^
--progress^
-e 'ssh -p 222' root#servername:/home/directory/ ./
Is there something I am missing to get my files copied over?
I can think of only a single possibility: My experience with rsync is that it creates the directory structure before copying files in. Rsync may be terminating prematurely, but after this directory step has been completed.
Update0
You mentioned that you were running dry run. Rsync by default only shows the directory names when the directory and all its contents are not present on the receiver.
After a lot of experimentation, I'm only able to reproduce the behaviour you describe if the directories on the source have later modification dates than on the receiver. In this instance, the modification times are adjusted on the receiver.
I had this problem too, and it turns out that backing up to a windows drive from linux doesn't seem to copy the temp files in place, after they are transferred over.
Try adding the --inplace flag, when rsyncing to windows drives.