Periodically, I like to put files in the /tmp directory of my webserver to share out. What is annoying is that I must set the permissions whenever I scp the files. Following the advice from another question I've written a script which copies the file over, sets the permissions and then prints the URL:
#!/bin/bash
scp "$1" SERVER:"/var/www/tmp/$1"
ssh SERVER chmod 644 "/var/www/tmp/$1"
echo "URL is: http://SERVER/tmp/$1"
When I replace SERVER with my actual host, everything works as expected...until I execute the script with an argument including spaces. Although I suspect the solution might be to use $# I've not yet figured out how to get a spaced filename to work.
It turns out that what is needed is to escape the path which will be sent to the remote server. Bash thinks the quotes in SERVER:"/var/www/tmp/$1" are related to the $1 and removes them from the final output. If I try to run:
tmp-scp.sh Screen\ shot\ 2010-02-18\ at\ 9.38.35\ AM.png
Echoing we see it is trying to execute:
scp SERVER:/var/www/tmp/Screen shot 2010-02-18 at 9.38.35 AM.png
If instead the quotes are escaped literals then the scp command looks more like you'd expect:
scp SERVER:"/var/www/tmp/Screen shot 2010-02-18 at 9.38.35 AM.png"
With the addition of some code to truncate the path the final script becomes:
#!/bin/bash
# strip path
filename=${1##*/}
fullpath="$1"
scp "$fullpath" SERVER:\"/var/www/tmp/"$filename"\"
echo SERVER:\"/var/www/tmp/"$filename"\"
ssh SERVER chmod 644 \"/var/www/tmp/"$filename"\"
echo "URL is: http://SERVER/tmp/$filename"
The script looks right. My guess is that you need to quote the filename when you pass it into your script:
scp-chmod.sh "filename with spaces"
Or escape the spaces:
scp-chmod.sh filename\ with\ spaces
the easier way without worrying about spaces in file names, (besides quoting) is to rename your files to get rid of spaces before transferring. Or when you create the files, don't use spaces. You can make this your "best practice" whenever you name your files.
Related
I need to run a bash script to copy a list of files to my phone.
This was a test run that worked:
cp f.jpg /home/ariela/phone/Internal\ storage/Pictures/Test
I want the directory to be a variable. So I tried this:
DEST="/home/ariela/phone/Internal\ storage/Pictures/Test"
cp f.jpg $DEST
And it did not work because it recognizes storage/Pictures/Test as second parameter and not part of the path.
What am I doing wrong?
Inside quotes no need to escape the space but make sure to quote it in cp command.
You can use:
phonedir='/home/ariela/phone/Internal storage/Pictures/Test'
cp f.jpg "$phonedir"
Originally I would like to sync directory (with all files and subdirectories) given in parameter in bash script.
I found this post: How can I recursively copy a directory into another and replace only the files that have not changed? which explains how to use rsync in similar case.
My bash script is quite simple and listed below:
#!/bin/bash
echo -e "Type the project to be deployed: \c "
read project
echo -e "* Deploying: $project *"
echo -e "Sync: /var/repo/released/$project"
echo -e " /var/www/released/$project"
rsync -pr /var/repo/released/$project /var/www/released/$project
As a result it copies everything within /released (there are many directories in there, let's say -projects-).
I would like to copy (sync) only project given in parameter.
Could you please advice how to do this?
When you call the script without an argument (which most likely is what you're doing since you interactively read the project name into the variable $project), the positional parameter $1 remains empty. Therefore the script will rsync the entire content of /var/repo/released/.
You need to replace $1 with $project in your script. Also, I'd recommend to put double quotes around the paths to avoid problems due to spaces in a directory name.
rsync -pr "/var/repo/released/$project" "/var/www/released/$project"
I have been trying to put the following into a bash script:
scp /cygdrive/c/Program\ Files\ \(x86\)/Spiceworks/data/configurations/LA_$(date +%F)-firewall-5520 "sf-mlawler#10.21.1.212:/home/sf-mlawler/SBOX_Automation/SBOX_Dumps/08\ -\ Security/Firewalls"
...which works as expected via command line.
When I try to place it in a script, I'm obviously going to have to double escape the spaces, anything else? I'm unable to have success with the multitude of variations I've tried.
I would think:
scp /cygdrive/c/Program\\ Files\\ \(x86\)/Spiceworks/data/configurations/LA_\$(date\ +%F)-firewall-5520 "sf-mlawler#10.21.1.212:/home/sf-mlawler/SBOX_Automation/SBOX_Dumps/08\\ \\-\\ Security/Firewalls"
...would work, but it doesn't. Any suggestions, before I go grey and bald? Thanks
I wont waste the space with all variations I've tried, but I will say I have tried escaping almost everything and basically nothing and many in between with no success. When I receive a "No such file or Directory" I obviously escape until the file actually resolves, but even when I do not get a path error, the command is not successfully completing.
I do understand this is quite a specific case, but I imagine it will help others in the future: escaping spaces, in a bash script, using embedded expect (I have tested with a #!/bin/bash shebang and the embedded expect using expect -c ' .... ' as well as #!/usr/bin/expect using a simple spawn scp command with no change.
EDIT (Based on responses):
I should have mentioned I have tried quoting it...
Quoting the first (host) part gives me the error
can't read "(date +%F)": no such variable
while executing
"spawn scp "/cygdrive/c/Program Files (x86)/Spiceworks/data/configurations/LA_$(date +%F)-firewall-5520" "sf-mlawler#10.21.1.212:/home/sf-mlawler/SBOX_..."
...it is not a variable, it is a function giving me the current date in the format year-month-day
Quoting the destination section without escaping anything gives me the error
scp: ambiguous target
The same command that worked on your bash shell should work the same in your bash script. You shouldn't and cannot escape it twice.
If it doesn't work, make sure you're testing the script from the same terminal, ensure that the script contents and your command is identical, then paste the output from your terminal when you do:
$ scp /cygdrive/c/Program\ Files\ \(x86\)/Spiceworks/data/configurations/LA_$(date +%F)-firewall-5520 "sf-mlawler#10.21.1.212:/home/sf-mlawler/SBOX_Automation/SBOX_Dumps/08\ -\ Security/Firewalls"
(successful scp output here)
$ cat testscript
(testscript contents here, should be identical to above scp command)
$ bash testscript
(unsuccessful script/scp output here)
Just stick it in quotes and stop hitting yourself, as #that_other_guy said, you cannot, and shouldn't try and escape twice, wherever you got that idea, disregard it as a source of information. (my guess is it came from "thin air")
scp "/cygdrive/c/Program Files (x86)/Spiceworks/data/configurations/LA_$(date +%F)-firewall-5520" "sf-mlawler#10.21.1.212:/home/sf-mlawler/SBOX_Automation/SBOX_Dumps/08 - Security/Firewalls"
You could even give yourself some helpers:
export PROGFILESx86="/cygdrive/c/Program Files (x86)"
export SPICEWORKS_CONFIGS="${PROGFILESx86}/Spiceworks/data/configurations"
(add them to .bashrc, exec it.) and then do:
scp "${SPICEWORKS_CONFIGS}/LA_$(date +%F)-firewall-5520" "sf-mlawler#10.21.1.212:/home/sf-mlawler/SBOX_Automation/SBOX_Dumps/08 - Security/Firewalls"
I'd also be concerned the the files you are attempting to scp actually exist, wrap it up in a condition to verify they're there, it only makes sense when you're auto-generating a filename.
firewall_config="${SPICEWORKS_CONFIGS}/LA_$(date +%F)-firewall-5520"
if [ -e "${firewall_config}" ]
then
scp "${firewall_config}" "sf-mlawler#10.21.1.212:/home/sf-mlawler/SBOX_Automation/SBOX_Dumps/08 - Security/Firewalls"
else
echo "File doesn't exist: ${firewall_config}"
fi
Update:
Since you've updated your question, it's obviously the date call that's giving you the biggest problem, again, indirection saves you doing bizarre escape-char-yoga, just get the result into a var and use that.
fw_date=$(date +%F)
firewall_config="${SPICEWORKS_CONFIGS}/LA_${fw_date}-firewall-5520"
Been searching for hours and still cant figure this out :(
Anyway, im creating a script that will automatically rsync about 40 different directories to 40 other directories on another server. if you want to see the entire script you can view it here: http://pastebin.com/Nt3KKvx9
But the important bit is the for loop where i run rsync
for ((i=0; i<${#websys[#]}; i++))
do
localpath="/nusdata/staff/NUS/NUS/Systems/${kiskasys[$i]}"
remotepath="/home/www/html/nusalaska.com/html/systems/${websys[$i]}"
rsync -rlptnvz -s "$localpath" -e "ssh -p 50014" "nusak#webserver:$remotepath/"
done
The problem is that the array "kiskasys" has many directory names that have spaces in them (Example: '101 greenbrook').
I have tried making the array variables have single quotes around them, double quotes around them, escaped spaces like '\ ', and combinations of all three. I have also tried putting the $localpath in quotes, not in quotes, etc. etc.
I guess im just confused on how the -s (--protect-args) deals with the spaces and how I can get it to work in my situation.
The error output always looks something like the following:
rsync: change_dir "/nusdata/staff/NUS/NUS/101" failed: No such file or directory (2)
or
rsync: change_dir "/nusdata/staff/NUS/NUS/'101 greenbrook'" failed: No such file or directory (2)
Any help is appreciated!
Thanks
Found My Problem
In copying my code from my script to this page I accidently copied it wrong... however in copying it wrong, what i posted above was perfectly good code that works fine haha.
so what i posted above was the solution to my own problem.
The original had single quotes in the localpath variable like so:
localpath="'/nusdata/staff/NUS/NUS/Systems/${kiskasys[$i]}'"
and the single quotes was the problem. And for everyone's benefit here is an example output
echo ${kiskasys[1]}
#output would look like this:
101 greenbrook
Basically there are no special escape characters etc.
For what it's worth, I'm not able to replicate your problem. If I set localpath and remotepath to a directory with spaces:
localpath="/home/lars/tmp/so/dir1/a directory"
remotepath="/home/lars/tmp/so/dir2/a directory"
And then run your rsync command (modified slightly for my environment):
rsync -rlptvz -s "$localpath/" -e "ssh" "localhost:$remotepath/"
It Just Works. Here's the content of dir1:
dir1/a directory/file1
dir1/a directory/file3
dir1/a directory/file2
And after running rsync, dir2 looks like this:
dir2/a directory/file1
dir2/a directory/file3
dir2/a directory/file2
Can you show a specific command line that results in the errors you're seeing above? Maybe run the script with the -x flag and show exactly how localpath and remotepath are set.
Using a handy script that I found for FTP-ing, I modified it to my own use, and it works, but too well, or too literally.
#!/bin/sh
HOST='10.0.1.110'
USER='myName'
PASSWD='myPass'
FILE='*.sql' # WILDCARD NOT WORKING - Takes literal string of '*.sql'
# Stripped unrelated code
ftp -n $HOST <<END_SCRIPT
quote USER $USER
quote PASS $PASSWD
prompt
binary
cd Desktop/SweepToDiskBK
mput /home/myAcct/SQLbackups/"$FILE" "$FILE"
quit
END_SCRIPT
exit
That is, the file(s) that gets 'put', is named *.sql, and replaces any previous versions of it, instead of file1.sql, file2.sql, etc. In the original script, they were doing a put, instead of an mput, and with a single file names text.txt. I've also tried changing the single quotes after FILE, to double quotes, and got the same result. Can someone let me in on the 'trick' to using variables for CLI FTP-ing?
Thanks in advance,
LO
I'd try this:
FILE=*.sql
without any quotes enabling wildcard expansion
and:
mput /home/myAcct/SQLbackups/"$FILE"
with just one $FILE
But i think you can just do mput /home/myAcct/SQLbackups/$FILE to mput *.sql? Or if you need to put several files with put, then you need to do it one by one with some sort of a loop.
To check what the script does, change ftp -n $HOST to cat. Try these very commands in the plain ftp session. Do they work?