I'd like to be able to have rsync copy files I have listed in my bash script and then delete any files that do not exist in the source directory (if i change the file list in the script). Expanded the command looks like this:
rsync -axSR --delete src_dir1/file1 src_dir2/file2 dst_dir/
However, rsync is not deleting extraneous files. Is there a way to do this?
With this syntax, it could be because you are using multiple sources, and those sources are actual files.
Why not just sync the parent directory of src_dir to dst_dir?
Related
I'm not allowed to use rsync on the cluster I'm working on so I need to use cp. I want to copy a large directory including all files and subfolders etc. but without any folders that have the name "outdir".
I tried cp -r -v ./!(outdir) ../target-directory/
but it still copies all folders and contents in deeper directories with the name outdir. It only included the outdir folders in the highest directory.
I also tried cp -r ./*/!(outdir) ../target-directory/ but that one copied all files into the folder without keeping any hirarchy or folders etc.
I also tried certain find commands but it didn't work, but maybe I was just doing something stupid. I'm a beginner with bash so if you could explain your answer and what the flags etc. do that would really be helpfull, I've been trying forever now, on what I think shouldn't be that hard to do.
Instead of cp, you can use tar with option --exclude to control what you want copied or not.
The full command is:
tar --exclude="outdir" -cvpf - . | (cd TARGET_DIRECTORY; tar -xpf -)
So any path that contains the "outdir" pattern will be excluded.
Without the --exclude option, it will copy the entire structure of your current directory under TARGET_DIRECTORY.
You can replace the . in the first tar by your desired source directory.
Good morning,
I want to transfer specific files in a directory to a remote machine while keeping the architecture of the subdirectories. Moreover, I only want to transfer files that have a peculiar extension (e.g. ".txt").
I have tried the following:
rsync -aP --include *.txt ./sourceDirectory user#hostIP:destDirectory
but it copies to the destination (destDirectory) all the files, and not only those which match the .txt pattern.
Could anybody help me with such riddle?
P.S.: obviously, the directory sourceDirectory contains subdirectories where are located my .txt files.
I used to have the same problem when rsync videos to home NAS. If you read the manual of rsync, the --include flag really means "Do not exclude". It actually has to work together with a --exclude flag.
The follow command will do your job:
rsync -aP --include="*/" --include="*.txt" --exclude="*" sourceDirectory destDirectory
The command literally means exclude everything, except for subfolders and txt file.
I have folders & subfolders organized by date in the name containing files .jpgs and .arw. I want to keep the same folder structure but move the ARW files to a copy of the folder structure AND delete them out of the source directory. Also if the folder does not contain any .ARW files it would not be copied.
I was working on something like this that I could run in the terminal screen:
rsync -av --exclude=.*/ --include=‘*/‘ --include=‘*.arw’ --exclude=‘*’ /Users/adam/test1/ /Users/Adam/test2/
But in my testing it's copying both the .jpg and .arw file to test 2. My assumption would be that I would add --remove-source-files once I got the code working with only moving the ARW file.
Thanks for your help!
The right command is:
rsync -avm --remove-source-files --include='*.arw' --include='*/' --exclude='*' /Users/adam/test1 /Users/adam/test2
The option 'm' prevents the copy of the empty directories.
You can run rsync with the '--dry-run' option to see which files would be copied without actually execute it:
rsync -avm --dry-run --remove-source-files --include='*.arw' --include='*/' --exclude='*' /Users/adam/test1 /Users/adam/test2
It appears rsync has an issue with .ARW file type. Changed requirement to move JPEG files instead to the new directory.
rsync -am --remove-source-files --include=‘.jpeg’ --include='.jpg' --include='/' --exclude='' /Users/adam/test\ 1/ /Users/Adam/test2/
Thanks!
I am using a NAS to backup my file server. The NAS exports /share/Backup via NFS, which is mounted on the fileserver as /mount/qnap. I want to keep track which files are rsynced but exclude the Backup-Dir, which contains many small files.Therefore I am running two instances of rsync, one with -v and another one without. The following command works as it should, after executing it the directory structure on /mount/qnap is identical to /mount/btrfs-raid.
rsync --delete -av --exclude Backup /mnt/btrfs-raid/ /mnt/qnap/
Rsyncing the Backup folder with the command
rsync --delete -av /mnt/btrfs-raid/Backup /mnt/qnap/Backup
produces the following directory structure on the NAS:
/mnt/qnap/Backup/Backup/..Subdirectories
To get the result I want I have to delete the last "Backup" from the target directory path:
rsync --delete -av /mnt/btrfs-raid/Backup /mnt/qnap/
Why does the second example not work like the first one?
Thanks
Stefan
Trailing slashes in paths are important for rsync. See the documentation.
rsync -avz foo:src/bar /data/tmp
This would recursively transfer all files from the directory src/bar on the machine foo into the /data/tmp/bar directory on the local machine. The files are transferred in "archive" mode, which ensures that symbolic links, devices, attributes, permissions, ownerships, etc. are preserved in the transfer. Additionally, compression will be used to reduce the size of data portions of the transfer.
rsync -avz foo:src/bar/ /data/tmp
A trailing slash on the source changes this behavior to avoid creating an additional directory level at the destination. You can think of a trailing / on a source as meaning "copy the contents of this directory" as opposed to "copy the directory by name", but in both cases the attributes of the containing directory are transferred to the containing directory on the destination. In other words, each of the following commands copies the files in the same way, including their setting of the attributes of /dest/foo:
Using rsync, my source directory has a number of files and directories. My destination already has been synced, so it mirrors those files and directories.
However, I have manually created a symlink in my destination that does not exist in my source.
I need to use the --delete operation in rsync. Is there a way to get rsync to not remove the symlink?
There is no option to achieve this the way you suggested BUT the simplest solution to the described problem is just to add the filenames of the symlinks to the rsync exclude pattern
like: --exclude="folder/symlinkname1" --exclude="folder/symlinkname2"
if there are many symlinks you can keep a list of them in a exclude pattern file
this file may be autogenerated by a little script or a bash one liner...
Does your destination existing symlink point to a directory or file? If a directory, you may be able to use --keep-dirlinks.