I am trying to save some typing with this bash script:
#!/usr/bin/env bash
n=$(($#-1))
files=${#:1:$n}
dest=${!#}
echo "$n files"
echo "${files[#]}"
echo "$dest"
rsync -av "${files[#]}" kyopti:$dest
Notice the "${files[#]}" part, I did this because it's recommended on Greg's bash tutorial. This generates an error:
rsync: link_stat "/tmp/scplot.pdf ssknplot.pdf" failed: No such file or directory (2)
But if I remove the quotes, everything works fine. I am confused, isn't this supposed to be the best practice, to add the quotes?
files is not an array. It's just a single string consisting of a space-separated list of the positional arguments. You want
files=( "${#:1:$n}" )
Then your quoted use of files in the rsync command will work as expected.
Related
This question already has answers here:
Bash: need help passing a a variable to rsync
(3 answers)
Closed 4 years ago.
I have the following for-loop which loops through all the given sources that need to be copied.
for i in "${sources[#]}"; do
exclude="--exclude 'exclude_folder/exclude_file'"
rsync -az $exclude $i $destination
done
However, the exclude option is not working.
for i in "${sources[#]}"; do
exclude="--exclude 'exclude_folder/exclude_file'"
rsync -az "$exclude" "$i" "$destination"
done
If I use the code above, rsync will exit and give an error that it is an unknown option.
If I simply use the following code, it works, but I want to use a variable for the exclude option.
for i in "${sources[#]}"; do
rsync -az --exclude 'exclude_folder/exclude_file' $i $destination
done
I would use eval.
Your code:
for i in "${sources[#]}"; do
exclude="--exclude 'exclude_folder/exclude_file'"
rsync -az "$exclude" "$i" "$destination"
done
would be then (I'm trying to be as close to your logic as possible):
for i in "${sources[#]}"; do
exclude="--exclude 'exclude_folder/exclude_file'"
rsync_command="rsync -az $exclude $i $destination"
eval rsync_command
done
From the eval man pages:
eval
Evaluate several commands/arguments
Syntax
eval [arguments]
The arguments are concatenated together into a single command, which
is then read and executed, and its exit status returned as the exit
status of eval. If there are no arguments or only empty arguments, the
return status is zero.
eval is a POSIX `special' builtin
EDIT
Gordon Davisson is right about the bugs/insecurities in eval. If any other solution is available then it is better to use it. Here the bash arrays are better. The array answer is superior answer.
Please see the answer at Bash: need help passing a a variable to rsync
Example list directories to exclude (also wildcharts) :
#!/bin/sh
export PATH=/usr/local/bin:/usr/bin:/bin
LIST="rootfs usr data data2"
for d in $LIST; do
rsync -az --exclude /$d/ .....
done
I want to use rsync for incremental backups. However it fails when I try it like this:
SRC="/"
TRG="/backup/"
LNK="/oldbackup/"
OPT="-a --exclude={/dev,/proc,/sys,/tmp,/run,/mnt,/media,/lost+found} --link-dest=$LNK"
rsync $OPT $SRC $TRG
But works like this:
SRC="/"
TRG="/backup/"
LNK="/oldbackup/"
rsync -a --exclude={/dev,/proc,/sys,/tmp,/run,/mnt,/media,/lost+found} --link-dest=$LNK $SRC $TRG
What did I do wrong?
Check the order of expansions in man bash: brace expansion happens first, variable expansion happens later. Therefore, braces in a variable are not expanded.
You can use an array to capture the values, use expansion already in the assignment:
opts=( -a --exclude={/dev,/proc,/sys,/tmp,/run,/mnt,/media,/lost+found} --link-dest="$LNK" )
rsync "${opts[#]}" "$SRC" "$TRG"
I have an rsync command in my csh script like this:
#! /bin/csh -f
set source_dir = "blahDir/blahBlahDir"
set dest_dir = "foo/anotherFoo"
rsync -av --exclude=*.csv ${source_dir} ${dest_dir}
When I run this I get the following error:
rsync: No match.
If I remove the --exclude option it works. I wrote the equivalent script in bash and that works as expected
#/bin/bash -f
source_dir="blahDir/blahBlahDir"
dest_dir="foo/anotherFoo"
rsync -av --exclude=*.csv ${source_dir} ${dest_dir}
The problem is that this has to be done in csh only. Any ideas on how I can get his to work?
It's because csh is trying to expand --exclude=*.csv into a filename, and complaining because it cannot find a file matching that pattern.
You can get around this by enclosing the option in quotes:
rsynv -rv '--exclude=*.csv' ...
or escaping the asterisk:
rsynv -rv --exclude=\*.csv ...
This is a consequence of the way csh and bash differ in their default treatment of arguments with wildcards that don't match a file. csh will complain while bash will simply leave it alone.
You may think bash has chosen the better way but that's not necessarily so, as shown in the following transcript where you have a file matching the argument:
pax> touch -- '--file=xyzzy.csv' ; ls -- *.csv
--file=xyzzy.csv
pax> echo --file=*.csv
--file=xyzzy.csv
You can see there that the bash shell expands the argument rather than giving it to the program as is. Both sides have their pros and cons.
I have been working on a backup script that uses rsync to do an incremental backup.
I have tested the following rsync command manually, and it runs and completes a backup without error:
rsync -aAXv --delete --progress --link-dest=/backup/Uyuk/Uyuk-backup-part1/2014-02-24/ /mnt/backup/ /backup/Uyuk/Uyuk-backup-part1/2014-02-25/
however when I run that same command in my backup script it gives me the following error:
rsync: -aAXv --delete --progress --link-dest=/backup/Uyuk/Uyuk-backup-part1/2014-02-24/ /mnt/backup/ /backup/Uyuk/Uyuk-backup-part1/2014-02-25/: unknown option
rsync error: syntax or usage error (code 1) at main.c(1422) [client=3.0.6]
I ran bash -x on my script to figure out exactly what is sent to the console and here is what was printed:
+ rsync '-aAXv --delete --progress --link-dest=/backup/Uyuk/Uyuk-backup-part1/2014-02-24/ /mnt/backup/ /backup/Uyuk/Uyuk-backup-part1/2014-02-25/'
Does anyone see what is wrong? I cant find anything that would cause the syntax error.
EDIT:
Here is the actual code I have in the script, and this is a pretty large script so yes some variables are not defined here, but you get the idea.
mkdir -p "/backup/$HOST/$NAME/$TODAY"
#source directory
SRC="$MNT"
#link directory
LNK="/backup/$HOST/$NAME/$LAST/"
#target directory
TRG="/backup/$HOST/$NAME/$TODAY/"
#rsync options
OPT1="-aAXv --delete --progress --link-dest=$LNK"
#run the rsync command
echo "rsync $OPT1 $SRC $TRG"
rsync "$OPT1 $SRC $TRG" > /var/log/backup/backup.rsync.log 2>&1
You are passing your option list as a single argument, when it needs to be passed as a list of arguments. In general, you should use an array in bash to hold your arguments, in case any of them contain whitespace. Try the following:
mkdir -p "/backup/$HOST/$NAME/$TODAY"
#source directory
SRC="$MNT"
#link directory
LNK="/backup/$HOST/$NAME/$LAST/"
#target directory
TRG="/backup/$HOST/$NAME/$TODAY/"
#rsync options
OPTS=( "-aAXv" "--delete" "--progress" "--link-dest=$LNK" )
#run the rsync command
echo "rsync $OPT1 $SRC $TRG"
rsync "${OPTS[#]}" "$SRC" "$TRG" > /var/log/backup/backup.rsync.log 2>&1
An array expansion ${OPTS[#]}, when quoted, is treated specially as a sequence of arguments, each of which is quoted individually to preserve any whitespace or special characters in the individual elements. If arr=("a b" c d), then echo "${arr[#]}" is the same as
echo "a b" "c" "d"
rather than
echo "a b c d"
This will not work in a shell that doesn't support arrays, but then, arrays were invented because there wasn't a safe way (that is, without using eval) to handle this use case without them.
This:
rsync "$OPT1 $SRC $TRG"
passes all your intended arguments lumped together as one argument, which rsync doesn't know how to deal with.
Try this instead:
rsync ${OPT1} ${SRC} ${TRG}
The approach suggested by #chepner didn't work on my Mac OS X (10.9.4), but eval did.
eval rsync "$OPT1 $SRC $TRG"
I'm trying to get the contents of a directory using shell script.
My script is:
for entry in `ls`; do
echo $entry
done
However, my current directory contains many files with whitespaces in their names. In that case, this script fails.
What is the correct way to loop over the contents of a directory in shell scripting?
PS: I use bash.
for entry in *
do
echo "$entry"
done
don't parse directory contents using ls in a for loop. you will encounter white space problems. use shell expansion instead
for file in *
do
if [ -f "$file" ];then
echo "$file"
fi
done