Rsync photos from internal to external hard drive [duplicate] - bash

This question already has answers here:
When to wrap quotes around a shell variable?
(5 answers)
Closed 4 years ago.
I have ~18gb worth of photos/videos on my internal that I'm trying to move to my external via Rsync. I initially tried:
sudo rsync -avh --progress --delete /Non-system files/Photo.photoslibrary/
/#2/Text_photos/
But no dice, "No such file" error. I thought that maybe .photoslibrary had to be in a folder so I moved it to "photos" folder in Non-system files and subsequently tried:
sudo rsync -avh --progress --delete /Non-system files/photos/
Photo.photoslibrary/ /#2/Text_photos/
I'm still getting the same error of:
rsync: link_stat "/Non-system" failed: No such file or directory (2)
rsync: link_stat "/Users/jcb_carrillo/files/photos/Photos" failed: No such file
or directory (2)
So two things I've noticed but don't know why it's reacting the way it is.
1. It's not reading the space in my partition "Non-system files" it only reads "Non-system".
And 2. Once the first error of "No such file or directory" happens it for some reason moves to "/Users/jcb_carrillo/files/photos/Photos" which is in an entirely other partition that as you can see in my command, was never specified to go there.
I think there's something wrong with my syntax. Any idea?

Looks like you might need to escape the blank space for the directory name "Non-system file"
sudo rsync -avh --progress --delete /Non-system\ files/Photo.photoslibrary/
/#2/Text_photos/

Related

rsync not working with variable containing a white space

I cant get my rsync script to work if source or destination variable have a space in the folder name
#!/bin/bash
HOST="root#192.168.1.100"
source="/path/My Files/"
dest="/path/My Files2/"
rsync -avhP --delete "$HOST":"$source" "$dest"
the error i get is
receiving incremental file list
rsync: [sender] link_stat "/path/My" failed: No such file or directory (2)
rsync: [sender] change_dir "/root/Files" failed: No such file or directory (2)
Any help how to write this would be great
Add option -s with current rsync version.
From man rsync:
-s, --protect-args: This option sends all filenames and most options to the remote rsync without allowing the remote shell to interpret them. This means that spaces are not split
in names, and any non-wildcard special characters are not translated (such as ~, $, ;, &, etc.). Wildcards are expanded on the remote host by rsync (instead of
the shell doing it).

Getting errors when using rsync in a script, but not when using the line directly in the command line

I'm trying to set up rsync to run backups from my Unraid shares to external drives mounted to the Unraid server.
I have different folders in different shares that I want backed up to different folders on different external drives.
I.e:
/share1/folder1/ should be synced to /externalDisk1/folder1/
/share1/folder2/ should be synced to /externalDisk2/folder1/
etc.
I have eight of these jobs, all together.
The idea is to run these as a cron job every night, and so I wrote them all down in a bash script.
Running the rsync lines one by one, or even as "job1; job2.." in the command line works great, however, calling them from "backup.sh" returns errors.
The script (with only one rsync line as an example):
#!/bin/sh
rsync -rtvh --progress /mnt/user/share1/folder1/ /mnt/disks/externalDrive1/Folder1/
returns
sending incremental file list
rsync: [Receiver] mkdir "/mnt/disks/externalDisk1/Folder1/\#015" failed: No such file or directory (2)
rsync error: error in file IO (code 11) at main.c(784) [Receiver=3.2.3]
rsync: [sender] write error: Broken pipe (32)
rsync error: error in file IO (code 11) at io.c(823) [sender=3.2.3]
Any suggestions on what's going on and how to fix this?
Running the script through the CA User Scripts plug-in makes it run perfectly. It's a work-around, if nothing else :)

cp and rsync cannot find system directory

I am working on a script to copy a custom user profile over to the default. Part of the script uses rsync to copy the contents of the customized profile, named "profile" to the default profile located in: /System/Library/User Template/English.lproj. Each time I run the command interactively, it fails indicating "No such file or directory." I can browse to the directory in the Finder. I can navigate to the directory via the terminal. Why can't rsync find it? Here is the command:
rsync -av /Users/profile/* /System/Library/User\ Template/English.lproj
I tried a similar approach using cp -R instead of rsync and got a whole bunch of file not found error messages. Using the cd command to that same path also fails. I can step through each individual directory and arrive at English.lproj but I can't do it in one command. Any ideas where I am going wrong with my command?
Thanks
Jason

Dealing with spaces in rsync path arguments

So I'm currently trying to write a few small scripts that allow me to manage my iTunes library of which I have clones on multiple OS X machines.
The basic idea is that I have a NAS holding a copy of the library that is used as an intermediate "master copy" since the machines holding the actually used copies aren't available all the time. If I want to update my old copy on machine B with the newer version from machine A, I'd then update the NAS copy based on machine A's current state, then update machine B from the updated NAS copy possibly at a later time.
The script files are located on the NAS within the same folder that also houses the iTunes directory. Since I'm mounting the NAS as a volume via AFP, I simply open a Finder window with the directory containing the scripts and drag'n'drop the script I want to use to a Terminal window for easy execution.
This is my attempt at the "update NAS from local copy" script:
rsync -avz --compress-level 1 --exclude 'Mobile Applications/*.ipa' --delete --delay-updates -n "$(echo $HOME | sed 's/ /\\ /g')/Music/iTunes" "$(dirname $0 | sed 's/ /\\ /g')"
(-n option of course only for testing the script)
Since there will be spaces in the paths I supply rsync with, I already figured out that I'd need to escape those somehow. I also know that the standard way to do that on OS X is to prepend all the spaces with a backslash, at least when manually typing paths in Terminal. But the code above still won't work – rsync complains that it cannot change to the directory I supplied, although the path it spits out in the error message seems to be perfectly fine and can be cd'd to, if you remove the double quotes around it first:
[...]
building file list ... rsync: change_dir "/Volumes/Macintosh\ HD/Users/Julian/Music" failed: No such file or directory (2)
done
[...]
If I remove the surrounding double quotes in the script itself, rsync seems to not honor the escaping backslashes at all and still treat the space following the backslash as a path separator:
[...]
building file list ... rsync: link_stat "/Volumes/Macintosh\" failed: No such file or directory (2)
rsync: change_dir "/Volumes/Macintosh HD/Users/Julian/HD/Users/Julian/Music" failed: No such file or directory (2)
done
[...]
And no, I can't work around the issue by shortening /Volumes/Macintosh\ HD/Users/Julian/Music to /Users/Julian/Music since this machine has multiple HDDs and / is not the same disk/partition as /Volumes/Macintosh\ HD. So I need to find a solution for this specific problem.
I'm seriously lost now.
Can anyone please explain to me what I need to change in order to have rsync recognize the paths correctly?
After messing around quite a bit more and finding this question, I managed to develop a working solution:
localpath=$HOME/Music/iTunes
remotepath=$(dirname $0)/
rsync -avz \
--compress-level 1 \
--exclude 'Mobile Applications/*.ipa' \
--delete \
--delay-updates \
-n \
"$localpath" \
"$remotepath"

Why does my rsync script do a full backup everytime?

I have the following rsync script which I created to do incremental backups:
rsync -arv --exclude-from '/usr/bin/exclude-list.txt' --delete /Volumes/DOCS/ /Volumes/BKUP1/DOCS/
&& rsync -arv --delete /Volumes/Webserver/ /Volumes/BKUP1/Webserver/
My exclude list is
/Volumes/Webserver/.Spotlight-V100
/Volumes/Webserver/.Trashes
/Volumes/Webserver/.fseventsd
Everytime I run this backup. It seems to go through and copy all the files everytime, despite the fact that rsync is supposed to be an incremental backup solution.
E.G. First run:
....
sites/website/sites/all/libraries/tinymce/jscripts/tiny_mce/plugins/style/js/.svn/prop-base/
sites/website/sites/all/libraries/tinymce/jscripts/tiny_mce/plugins/style/js/.svn/props/
sites/website/sites/all/libraries/tinymce/jscripts/tiny_mce/plugins/style/js/.svn/text-base/
....
Second run:
....
sites/website/sites/all/libraries/tinymce/jscripts/tiny_mce/plugins/style/js/.svn/prop-base/
sites/website/sites/all/libraries/tinymce/jscripts/tiny_mce/plugins/style/js/.svn/props/
sites/website/sites/all/libraries/tinymce/jscripts/tiny_mce/plugins/style/js/.svn/text-base/
....
etc...
The same files are copied across again. Also I am constantly encountering the following permission denied errors, despite the fact they are ignored in my excude-from argument:
building file list ... rsync: opendir "/Volumes/Webserver/.Spotlight-V100" failed: Permission denied (13)
rsync: opendir "/Volumes/Webserver/.Trashes" failed: Permission denied (13)
rsync: opendir "/Volumes/Webserver/.fseventsd" failed: Permission denied (13)
Any ideas? I am hoping I can tweak this script so it will only copy across modified / new files and show me what files these are in the verbose output.
Many thanks in advanced.
I ran in to this myself. The best I could come up with, as silly as this sounds, is that the files' timestamps aren't being preserved. Then when you do it again it thinks "Hey! These timestamps don't match - better sync 'em!" If you use the -t option, It will send the timestamps along, then the files will be seen as the same
Or you can use the "size only" option, which does what it sounds, if you're sure there are no files you've modified but are the same size.
Are you copying to a FAT32 drive from a differently formatted drive? My understanding is that FAT32 keeps a 16-bit time-stamp which only permits a resolution of about two seconds, which is far less precise than other drive formats. By default, rsync requires timestamps to match exactly, so virtually every file will fail this test and be recopied.
To fix this, you need to have rsync to pass files that are time-stamped +/-1 second (2 second total range) from the source. You do this by adding
--modify-window=1
to the rsync command.

Resources