Rsync on Windows: wrong permissions for created directories - windows

I'm trying to push changes to my server through ssh on windows (cygwin) using rsync.
The command I am using is:
rsync -rvz -e ssh /cygdrive/c/myfolder/ rsyncuser#192.168.1.110:/srv/www/prj112/myfolder/
/srv/www/prj112/myfolder/ is owned by rsyncuser. My problem is that eventhough with rsync the sub directories are create as they publish, each directory is assigned default permission of d--------- so rsync fails to copy any files inside it.
How do I fix this?

The option to ignore NTFS permissions has changed in Cygwin version 1.7. This might be what's causing the problem.
Try adding the 'noacl' flag to your Cygwin mounts in C:\cygwin\etc\fstab, for example:
none /cygdrive cygdrive user,noacl,posix=0 0 0
You can pass custom permissions via rsync using the 'chmod' option:
rsync -rvz --chmod=ugo=rwX -e ssh source destination

Your problem stems from the fact that the Unix permissions on that directory really are 0. All of the access information is stored in separate ACLs, which rsync does not copy. Thus, it sets the permissions on the remote copy to 0, and, obviously, is unable to write to that directory afterwards.
You can run
chmod -R 775
on that directory, which should fix your rsync problem.
After a look at the manpage I can tell that the chmod param is available in rsync since version ~2.6.8. But you have to use --chmod=ugo=rwX in combination with rsync -av
You should also try this command:
rsync -av <SOURCE_DIR> rsyncuser#192.168.1.110:/srv/www/prj112/myfolder
It would work on Linux at least. And note that rsync does not need to mention ssh--at least on Linux.
But if all fails and just to give an option you may take a look at this ready packed-up tool cwRsync

if you deploy a site from windows (for ex. octopress use rsync) it's possible set permission to 775 adding multiple chmod command:
rsync -avz --chmod=ug=rwx --chmod=o=rx -e ssh

To rsync from Windows to Unix/Linux you should provide a command like
SET BACKUP_SERVER=my.backup.server
SET SSH_USER=theUnixUserName
SET STORAGEPATH=/home/%SSH_USER%/Backup/
SET STORAGEURI=%BACKUP_SERVER%:%STORAGEPATH%
SET SSH_ID=/cygdrive/c/Users/theWindowsUserName/Documents/keyfiles/id_dsa
SET EXCLUDEFILE=backup_excludes.txt
SET BACKUPLOGFILE=/cygdrive/c/Users/theWindowsUserName/Backuplogs/backup-%DATE%-%TIME::=-%.log
The ssh command then is
SET BACKUP=rsync -azvu --chmod=Du=rwx,Dgo=rx,Fu=rw,Fgo=r --rsh="ssh -l %SSH_USER% -i '%SSH_ID%'" --exclude-from=%EXCLUDEFILE% --delete --delete-excluded --log-file="%BACKUPLOGFILE%"
with backup_excludes.txt containing lines of ignored elements like
.git
.svn
.o
\Debug
\Release
Then you would use this in a script with
%BACKUP% /cygdrive/c/mySensibleData %STORAGEURI%
%BACKUP% /cygdrive/c/myOtherSensibleData %STORAGEURI%
%BACKUP% /cygdrive/c/myOtherSensibleData2 %STORAGEURI%
and so on. This will backup your directories mySensibleData, myOtherSensibleData and myOtherSensibleData2 with the permissions 755 for directories and 644 for files. You also get backup logs in your %BACKUPLOGFILE% for each backup.

Cygwin rsync will report permission denied when some process has the target file open. Download and run Process Explorer and find out if anything else is locking the file or simply try renaming the file and see if you get the Windows error about some other process having the file open.

Also, you can try to create a (global) environment variable CYGWIN and set its value to nontsec

Related

Rsync including all files

After some reading and trying rsync copy over only certain types of files using include option I can't get seem to get it to work.
I run the following command:
rsync -zarv -vvv -e ssh --prune-empty-dirs --delete --include="*/" --include="*.csv"
--include="*.hdf5" --include="*.pickle" --include="*.tar.gz" --include="*.bin"
--include="*.zip" --include="*.npz" --exclude="*" . user#host.com:/rsync
But at the target it backups any file I have in the directory and subdirectories. delete-before and delete-after does not delete files like .txt or .py. I have also tried the --exclude="*" before the extension includes but I am running 2.6.9 so it should be after as far as I have understood it.
Deleting files on the host machine will just sync them again for whatever reason I don't know.
Your command looks fine, although try using --delete-excluded instead of --delete.
--delete-excluded - also delete excluded files from destination dirs
It should eliminate any files that are --excluded and not --included on the destination.
Sorry to have bothered. This was a bash issue and not a command issue.
As I was using the command:
exec $COMMAND
instead of
eval $COMMAND
This made god knows what for error but executing it manually (after print) and correctly in bash made it work. Deleting items still seems flaky but that I can try some.

rsync backup to external hard disk exFat fails

I tried to back up data from my macbook to an external hard drive - formatted with exFat (bacause of the Windows and Linux/Mac compatibility).
With Automator I will create a little Program, to backup my data easily. It works fine on the local drive and from local drive to an SD-Card. But it do not work from local drive to an external hard drive. What's wrong?
SOURCE=/Users/username/Pictures/test
TARGET=/Volumes/Backup/
LOG=~/Documents/AutomatorLogs/BackupSync.log
rsync -aze "ssh" --delete --exclude=".*" "$SOURCE" "$TARGET" > "$LOG"
I got this Error:
rsync: recv_generator: mkdir "/Volumes/Backup/test" failed: Permission
denied (13)
I know this is older but I just ran into this and I wanted to make sure this info was included. I know the OP is a little different, but I'm using a macbook and ran into the error I describe so I don't know how even with changing the disk name it worked.
rsync can't use -a when the target is an exfat drive. It will make the directories and seem to be backing up but no files are actually created. You need to use:
rsync -rltDv [SRC] [DESTINATION]
where:
-v, --verbose increase verbosity
-r, --recursive recurse into directories
-l, --links copy symlinks as symlinks
--devices preserve device files (super-user only)
--specials preserve special files
-D same as --devices --specials
-t, --times preserve times
The reason is because rsync doesn't handle permissions on exfat. You will see an rsync error (in syslog or if you control-c out):
chgrp [file] failed: Function not implemented (38)
It looks like the user that you're running the command as doesn't have permission to make a new directory in the /Volumes/Backup/ directory.
To solve this, you will probably need to change the permissions on that directory so that your script will be able to write to it and create the new directory it uses to make the backup.
Here are some links about permissions:
http://linuxcommand.org/lts0070.php
http://www.perlfect.com/articles/chmod.shtml
I think, I've got it:
It is related to the name of the external hard disk.
With the name "Backup" for my external hard drive, it does not work.
If I changed the name to anything else, it works.
(I tested some other exFat formatted external hard drives with other names and it worked. So I changed the name of this external drive to anything else and now it works. Crazy...)

rsync (cwrsync) in Gitbash - The source and destination cannot both be remote. [windows 7]

I am trying to sync a local path to another drive using Git Bash in Windows 7. I downloaded cwRsync and copied the files over to my Git bash bin directory. I can run rsync but the following command gives me problems
rsync -av /c/00-dev/05-batch /f/RSYNC/
The source and destination cannot both be remote.
rsync error: syntax or usage error (code 1) at main.c(1135) [receiver=3.0.6]
Both the paths are correct but it won't sync.
Workaround: cd to /c and rsync using relative paths.
So this works:
cd /c
touch temp/test.txt
rsync temp/test.txt temp/test2.txt
while this doesn't, even if you cd to /c:
$ rsync /c/temp/test.txt /c/temp/test2.txt
The source and destination cannot both be remote.
Tested with the Git Bash that came with git 2.9.0.windows.1.
This works using cwRsync from Git Bash.
export MSYS_NO_PATHCONV=1
rsync -av /cygdrive/c/00-dev/05-batch /cygdrive/f/RSYNC/
cwRsync.exe needs /cygdrive/<drive-letter> for absolute pathing, so the simple rsync -av /cygdrive/c/00-dev/05-batch /cygdrive/f/RSYNC/ works from the cmd.
However, Git Bash converts paths like .../c/... to weird things like ...C:\.... The line: export MSYS_NO_PATHCONV=1 prevents that conversion. (See more: How to stop MinGW and MSYS from mangling path names given at the command line).
Use unset MSYS_NO_PATHCONV in the script to restore standard Git Bash behavior.
In Windows I tried cwRsync to sync data between folders located on Z:(network disk) and D:(usb stick) excluding a subfolder 'Data':
rsync -r -v --size-only --dry-run --exclude=Data --chmod=ugo=rwX /cygdrive/z/"Scan folder"/"Sub folder" /cygdrive/d/"Scan folder"/"Sub folder"
-r means recursively, -v verbose output, --dry-run is just for testing (data doesn't transfer), --exclude excludes subfolder or file(s), --chmod=ugo=rwX sets proper permissions (read more about permissions here with comments)
For cyrillic names on Windows use 'chcp 65001'.

cp and rsync cannot find system directory

I am working on a script to copy a custom user profile over to the default. Part of the script uses rsync to copy the contents of the customized profile, named "profile" to the default profile located in: /System/Library/User Template/English.lproj. Each time I run the command interactively, it fails indicating "No such file or directory." I can browse to the directory in the Finder. I can navigate to the directory via the terminal. Why can't rsync find it? Here is the command:
rsync -av /Users/profile/* /System/Library/User\ Template/English.lproj
I tried a similar approach using cp -R instead of rsync and got a whole bunch of file not found error messages. Using the cd command to that same path also fails. I can step through each individual directory and arrive at English.lproj but I can't do it in one command. Any ideas where I am going wrong with my command?
Thanks
Jason

SCP says file has downloaded, but the file does not appear

I am using ssh to work on a remote server, however when I try to download a file using scp in this format:
scp name#website.com:somefile.zip ~/Desktop
It asks me for my password, and shows this:
somefile.zip 100% 6491 6.3KB/s 00:00
however, this file never appears on my desktop. Any help
I think that you are logging into the remote machine using ssh and then running the command on the remote machine. You should actually be running the command without logging into your remote server first.
You need to specify the file path
scp name#website.com:/path/to/somefile.zip ~/Desktop
~/Desktop should actually be a directory, not a file. I suggest that you do the following:
Remove the ~/Desktop file with rm ~/Desktop (or move it with mv if you want to keep its contents).
Create the directory with mkdir ~/Desktop.
Try again to scp the zip file.
BTW, when I need to copy files into directories, I usually put a slash after the directory to avoid such problems (in case I make a mistake), e.g. scp server:file ~/Desktop/; if the directory doesn't exist, I get an error instead of unwanted file creation.
You are doing this from a command line, and you have a working directory for that command line (on your local machine), this is the directory that your file will be downloaded to. The final argument in your command is only what you want the name of the file to be. So, first, change directory to where you want the file to land. I'm doing this from git bash on a Windows machine, so it looks like this:
cd C:\Users\myUserName\Downloads
Now that I have my working directory where I want the file to go:
scp -i 'c:\Users\myUserName\.ssh\AWSkeyfile.pem' ec2-user#xx.xxx.xxx.xxx:/home/ec2-user/IwantThisFile.tar IgotThisFile.tar
Or, in your case, (that is with the VERY strong password you must be using):
cd ~/Desktop
scp name#website.com:/path/to/somefile.zip somefile.zip

Resources