Using rsync, my source directory has a number of files and directories. My destination already has been synced, so it mirrors those files and directories.
However, I have manually created a symlink in my destination that does not exist in my source.
I need to use the --delete operation in rsync. Is there a way to get rsync to not remove the symlink?
There is no option to achieve this the way you suggested BUT the simplest solution to the described problem is just to add the filenames of the symlinks to the rsync exclude pattern
like: --exclude="folder/symlinkname1" --exclude="folder/symlinkname2"
if there are many symlinks you can keep a list of them in a exclude pattern file
this file may be autogenerated by a little script or a bash one liner...
Does your destination existing symlink point to a directory or file? If a directory, you may be able to use --keep-dirlinks.
Related
I'm not allowed to use rsync on the cluster I'm working on so I need to use cp. I want to copy a large directory including all files and subfolders etc. but without any folders that have the name "outdir".
I tried cp -r -v ./!(outdir) ../target-directory/
but it still copies all folders and contents in deeper directories with the name outdir. It only included the outdir folders in the highest directory.
I also tried cp -r ./*/!(outdir) ../target-directory/ but that one copied all files into the folder without keeping any hirarchy or folders etc.
I also tried certain find commands but it didn't work, but maybe I was just doing something stupid. I'm a beginner with bash so if you could explain your answer and what the flags etc. do that would really be helpfull, I've been trying forever now, on what I think shouldn't be that hard to do.
Instead of cp, you can use tar with option --exclude to control what you want copied or not.
The full command is:
tar --exclude="outdir" -cvpf - . | (cd TARGET_DIRECTORY; tar -xpf -)
So any path that contains the "outdir" pattern will be excluded.
Without the --exclude option, it will copy the entire structure of your current directory under TARGET_DIRECTORY.
You can replace the . in the first tar by your desired source directory.
I need to move files from /var/www/html/internal/packages.confluent.io/rpm/5.5 to /var/www/html/internal
I am using the below script to perform the move. I also trtied it with command in stead of shell
The below script moves the files from under the 5.5 directory to /var/www/html/internal/5.5. I need them to be under /var/www/html/internal
- name: Move confluent packages to /var/www/html/internal
command: mv /var/www/html/internal/packages.confluent.io/rpm/5.5 /var/www/html/internal/
when: internal_stat.stat.exists
How should I formulate the command to make sure the files and subdirectories are moved under /var/www/html/internal and not /var/www/html/internal/5.5 ?
Thanks
I was doing
mv /var/www/html/internal/packages.confluent.io/rpm/5.5/*.* /var/www/html/internal/
that was not copying the subdirectories only files. It seems I have to do
mv /var/www/html/internal/packages.confluent.io/rpm/5.5/* /var/www/html/internal/.
Initially I had thought that wouldn't copy the files, only directories, but this seems to work for both files and directories. This does not move the . files, I will research that later
I am using a NAS to backup my file server. The NAS exports /share/Backup via NFS, which is mounted on the fileserver as /mount/qnap. I want to keep track which files are rsynced but exclude the Backup-Dir, which contains many small files.Therefore I am running two instances of rsync, one with -v and another one without. The following command works as it should, after executing it the directory structure on /mount/qnap is identical to /mount/btrfs-raid.
rsync --delete -av --exclude Backup /mnt/btrfs-raid/ /mnt/qnap/
Rsyncing the Backup folder with the command
rsync --delete -av /mnt/btrfs-raid/Backup /mnt/qnap/Backup
produces the following directory structure on the NAS:
/mnt/qnap/Backup/Backup/..Subdirectories
To get the result I want I have to delete the last "Backup" from the target directory path:
rsync --delete -av /mnt/btrfs-raid/Backup /mnt/qnap/
Why does the second example not work like the first one?
Thanks
Stefan
Trailing slashes in paths are important for rsync. See the documentation.
rsync -avz foo:src/bar /data/tmp
This would recursively transfer all files from the directory src/bar on the machine foo into the /data/tmp/bar directory on the local machine. The files are transferred in "archive" mode, which ensures that symbolic links, devices, attributes, permissions, ownerships, etc. are preserved in the transfer. Additionally, compression will be used to reduce the size of data portions of the transfer.
rsync -avz foo:src/bar/ /data/tmp
A trailing slash on the source changes this behavior to avoid creating an additional directory level at the destination. You can think of a trailing / on a source as meaning "copy the contents of this directory" as opposed to "copy the directory by name", but in both cases the attributes of the containing directory are transferred to the containing directory on the destination. In other words, each of the following commands copies the files in the same way, including their setting of the attributes of /dest/foo:
I currently have rsync working well. It copies all my files from one directory to another directory. The only thing is it is physically copying the files.
I have a lot of large files that I don't want to have a duplicate of all the files. I just want to create a symbolic link in the new directory so that I can serve the data on a webpage. The source directory has some scripts and files I don't want the public to see. I'm moving the safe data to the web root (destination).
What I would like rsync to do is any new files in the source directory would create links into the destination. That way I am not using up my hard drive space like I currently am doing. What I have works perfect except for doing the symbolic link aspect to it. Is there a way to have rsync track and create symbolic links?
rsync -aP --exclude="file.sql" --exclude="*~" --exclude=".*" --exclude="*.sh" . ${destination}
It's not a symlink, but you might be able to work with --link-dest=DIR. It creates a hard link which will create a new name for the same file. This will behave similarly to a softlink as long as:
Both files are on the same filesystem
You don't plan to delete the original and not the copy (the symlink would break but a hard-link won't)
You don't have anything explicitly checking to see if it's a softlink
You could use cp -aR -s (Linux or FreeBSD) or cp --archive --recursive --symbolic-link (Linux) to create symbolic links to the source files in the destination directory instead of copies. Note that -s is non-standard.
Can lndir be useful to you. According to manual it creates a shadow directory of symbolic links to another directory tree.
I think master_delivery is probably the best tool for this. With the already introduced --link-dest option of rsync, files which are not the same will be copied. If you don't mind the situation where copies and hardlinks are mixed, you can use rsync, but if you want to eliminate duplicates completely, use master_delivery.
Usage is:
gem install master_delivery
master_delivery -m <path_to_master> -d <path_to_delivery_root>
I'd like to be able to have rsync copy files I have listed in my bash script and then delete any files that do not exist in the source directory (if i change the file list in the script). Expanded the command looks like this:
rsync -axSR --delete src_dir1/file1 src_dir2/file2 dst_dir/
However, rsync is not deleting extraneous files. Is there a way to do this?
With this syntax, it could be because you are using multiple sources, and those sources are actual files.
Why not just sync the parent directory of src_dir to dst_dir?