Using perl system() query with spaces in path variable - bash

I seem to be stuck here. I want to send a system request from my script to another server via SSH, checking if a folder there exists. A folder path is passed from another script, stored in a variable and might have a space character in it. Since I couldn't replace the space with another character, to avoid a "not found" on folder like "foo bar", I need to pass something like
ls '/folderpath/foo bar' to other server's shell.
Sample code looks like this:
$cmd = 'ssh -i id_pub $ssh_addr ls $remote_dir';
if (system($cmd) == 0) {
do something
}
I've exhausted all possible options - tired to escape the possible space with \ before passing it to the command, tried to pass it with ' ', " ", inside and adding both before passing it into $cmd.
But I always end up with something like this:
ls \folderpath\foo\\ bar or ls \' \folderpath\foo bar\'
but not ls '\folderpath\foo bar'
I'm not that good with Perl, possible someone more experienced can recommend a workaround?

String::ShellQuote's shell_quote is useful in building shell commands.
my $remote_cmd = shell_quote("ls", "--", $remote_dir);
my $local_cmd = shell_quote("ssh", "-i", "id_pub", $ssh_addr, $remote_cmd);
system($local_cmd);
Of course, you can avoid the shell on the local side as follows:
use String::ShellQuote qw( shell_quote );
my $remote_cmd = shell_quote("ls", "--", $remote_dir);
system("ssh", "-i", "id_pub", $ssh_addr, $remote_cmd);

Running a local shell and using it to escape your command to be safe for the remote shell would look like this:
system('env', "ssh_addr=$ssh_addr", "remote_dir=$remote_dir", 'bash', '-c',
'printf -v remote_cmd "%q " ls -- "$remote_dir"; ssh "$ssh_addr" "$remote_cmd"');
Unlike just using "'$remote_cmd'", the above works with all possible values, including intentionally malicious ones, so long as your remote shell is also bash.
Thanks to #ikegami's answer for demonstrating the use of the end-of-options sigil -- to ensure that even a remote_dir value that starts with dashes is parsed as a positional argument by ls

OK you have several possibilities for shell expansion with the way you are doing this.
Firstly is using system() with a string. This will break all your paths on the space characters. you can solve this by using system as a list
system('ssh', '-i', 'id_pub', $ssh_addr, 'ls', $remote_dir)
Now we still have a problem as ssh will run the remote code on the remote server in a shell with shell expansion which will break the path on spaces again
So you need to put $remote_dir inside ' characters to stop the remote shell from breaking up the path: giving
system('ssh', '-i', 'id_pub', $ssh_addr, 'ls', "'$remote_dir'")
Hope this helps/works
Note that as the commenters below have said this makes the assumption that $remote_dir has no ' characters in it. You need to be either escaping or parsing $remote_dir to ensure that you don't get a path that looks like /file.txt'; rm -rf / # which will attempt to remove every file on the remote system

Let Net::OpenSSH take care of everything for you:
my $ssh = Net::OpenSSH->new($ssh_addr);
$ssh->error and die "unable to connect to remote host: " . $ssh->error;
if ($ssh->test('test', '-d', $remote_dir)) {
# do something here!
}
Oh, it seems you are on a Windows machine! You can use Net::SSH::Any there in a similar fashion.

Related

zsh command not found error when setting an alias

Currently trying to move all of my aliases from .bash_profile to .zshrc. However, found a problem with one of the longer aliases I use for substituting root to ubuntu when passing a command to access AWS instances.
AWS (){
cd /Users/user/aws_keys
cmd=$(echo $# | sed "s/root/ubuntu/g")
$cmd[#]
}
The error I get is AWS:5: command not found ssh -i keypair.pem ubuntu#ec1.compute.amazonaws.com
I would really appreciate any suggestions!
The basic problem is that the cmd=$(echo ... line is mashing all the arguments together into a space-delimited string, and you're depending on word-splitting to split it up into a command and its arguments. But word-splitting is usually more of a problem than anything else, so zsh doesn't do it by default. This means that rather than trying to run the command named ssh with arguments -i, keypair.pem, etc, it's treating the entire string as the command name.
The simple solution is to avoid mashing the arguments together, so you don't need word-splitting to separate them out again. You can use a modifier to the parameter expansion to replace "root" with "ubuntu". BTW, I also strongly recommend checking for error when using cd, and not proceeding if it gets an error.
So something like this:
AWS (){
cd /Users/user/aws_keys || return $?
"${#//root/ubuntu}"
}
This syntax will work in bash as well as zsh (the double-quotes prevent unexpected word-splitting in bash, and aren't really needed in zsh).
BTW, I'm also a bit nervous about just blindly replacing "root" with "ubuntu" in the arguments; what if it occurs somewhere other than the username, like as part of a filename or hostname?

Archive files at remover server where file name has "SPACE" and special character

I need to archive files with sys time on remote server but name of the file contains "SPACE" and special character. So below commands are not working.
FileName="BBB ABC#textfile.xml"
ts=`date +"%m%d%Y%H%M%S"`
ssh remoteid#remoteserver "'mv /upload/hotfolders/in/"$FileName"
/upload/hotfolders/Archive/${FileName}_${ts}'"
But above command is failing with below error.
bash: mv /upload/hotfolders/in/BBB ABC#textfile.xml /upload/hotfolders/Archive/BBB ABC#textfile.xml_01282019050200: No
such file or directory
In the original provided code:
ssh remoteid#remoteserver 'cd /upload/hotfolders/; mv "$FileName"
/upload/hotfolders/Archive/"${FileName}_${ts}"'
the outermost ' are used on the local filesystem to keep all the commands as a single argument to ssh. However, this means that $FileName, etc are not expanded locally! Instead, the unexpanded strings are passed verbatim to the remoteserver, where a shell is started to run the command. $FileName, etc, are then expanded there. Because they are not defined there (probably), the expansion fails to produce anything useful.
In the amended version:
ssh remoteid#remoteserver "'mv /upload/hotfolders/in/"$FileName"
/upload/hotfolders/Archive/${FileName}_${ts}'"
there is a different problem. Here, the two sets of outermost " allow the local system to expand the variables (although it may not be obvious that the first $FileName is not actually inside "). However, as the command that is passed is now wrapped in ', the remote server will treat the entire string as a single word.
If we assume that FileName and ts will not contain shell-special characters (such as ') then the fix is to wrap the command sequence in " (so that it expands locally), and only wrap the variables in ' (so that the remote server treats the now-expanded strings as single words):
ssh remoteid#remoteserver "cd /upload/hotfolders/; mv '$FileName'
/upload/hotfolders/Archive/'${Filename}_${ts}'"

Getting the ruby exec syntax right

I want to execute a variable shell command through ruby. I am a ruby beginner. I had the command working without variables, but when I tried to set variables the exec command doesn't work and the script stops running.
All 3 variables hold Strings.
The print is working fine, but exec not.
print "#{jobserv}" + options[:machines] + " -c touch #{noautooffPath} \n"
exec('"#{jobserv}" + options[:machines] + " -c touch #{noautooffPath}"')
What do i have to do to make the exec right or to get more hints by the system on how to correct this?
jobserv hold a path to an executable file plus options, options[:machines] holds a parameter I give to the last option flag of jobserv.
First, check you want exec. Several other options, like system, are more common choices for running a shell command. (Specifically, if exec is successful, the Ruby script will stop executing and be replaced by the command it executes; the rest of the script will never be run.)
You seem to have an extra layer of single quotes in your exec call. You probably want something closer to this:
exec("#{jobserv}" + options[:machines] + " -c touch #{noautooffPath}")
(assuming the print is showing the value you want)
In general, it's safer to avoid the single-string shell form, and pass all the argument individually. That might look more like this:
exec("#{jobserv}#{options[:machines]}", "-c", "touch", noautooffPath)
or:
exec(jobserv, options[:machines], "-c", "touch", noautooffPath)
(depending on whether jobserv and options[:machines] are expected to combine. And it'd look different again if any of those are expected to themselves contain a full multi-argument command invocation.)

How to write a bash script alias with whitespace on macOS

UPDATE:
I made progress, but I'm still unsure what I'm not understanding. I've reduced my script down to a single line, and I still run into the same issue.
Any insight on the following?
NOTE: if I enter "/Users/Da.../Docker" or as it is below using a backspace for the whitespace, the result is the same.
for myvm in $(find /Users/David/Documents/Virtual\ Machines/Docker -name *.vmx); do echo -e "VM name: $myvm"; done
VM name: /Users/David/Documents/Virtual
VM name: Machines/Docker/RHEL7-DockerHost-Node04.vmwarevm/RHEL7-DockerHost-Node04.vmx
...
VM name: /Users/David/Documents/Virtual
VM name: Machines/Docker/RHEL7-DockerHost.vmwarevm/RHEL7-DockerHost.vmx
VM name: /Users/David/Documents/Virtual
VM name: Machines/Docker/RHEL7-DockerHost-Node01.vmwarevm/RHEL7-DockerHost-Node01.vmx
What am I missing?
--------------------------
I've found similar questions, but the answers aren't working for me.
I'm writing a script that will allow me to start and stop multiple VMs within VMware Fusion without having to click each one using the vmrun command, found at "/Applications/VMware Fusion.app/Contents/Library/vmrun".
Problem
When I try to alias vmrun:
alias myvmcli='/Applications/VMware\ Fusion.app/Contents/Library/vmrun'
...I get the following error from bash:
line 3: /Applications/VMware\: No such file or directory
Discussion
It obviously is missing the whitespace, but I thought using the backspace character would indicate that whitespace should be used.
I've tried single quotes and double quotes in the alias line to no avail.
If I run alias from the command line, it works fine.
Solution/Thoughts?
I'm sure that creating a link can solve my problem, but I really want to know why this isn't working.
You can avoid both functions and aliases by adding the directory to your path:
PATH="/Applications/VMware Fusion.app/Contents/Library:$PATH"
Then vmrun by itself will work.
You need to quote twice:
alias myvmcli='"/Applications/VMware Fusion.app/Contents/Library/vmrun"'
or escape:
alias myvmcli=\''/Applications/VMware Fusion.app/Contents/Library/vmrun'\'
or
alias myvmcli="\"/Applications/VMware Fusion.app/Contents/Library/vmrun\""
Be careful of the way the 2nd and 3rd are escaped. They differ.
You need the double quotation because you need to quote once because the alias command requires it (alias asd="echo asd"), and then once again because the command inside the alias requires it.
EDIT:
Even though I answered this, the alias you posted, works with me just fine. Could be due to differences in bash version though:
[ 0][s: ~ ]$ cd /tmp
[ 0][s: tmp ]$ alias asd='asd\ zxc/test'
[ 0][s: tmp ]$ asd
woooo!
A backslash \ is not required inside quotes, it is retained. You would only need the backslash if you didn't use quotes, but generally quotes are good.
$ echo 'Hello\ world'
Hello\ world
$ echo "Hello\ world"
Hello\ world
$ echo Hello\ world
Hello world
$ echo "Hello world"
Hello world
Using an alias inside a script is not good, aliases are really designed as productivity aids on the command-line.
In this case just assign it to a variable, you don't need an alias or a function.
cmd='/Applications/VMware Fusion.app/Contents/Library/vmrun'
Then, when you want to run it:
"$cmd"
Note the double quotes, required because of the embedded space.
Tested on OS X with vmware.
As others have said, something more complex than this would require a function.

zsh run a command stored in a variable?

In a shell script (in .zshrc) I am trying to execute a command that is stored as a string in another variable. Various sources on the web say this is possible, but I'm not getting the behavior i expect. Maybe it's the ~ at the beginning of the command, or maybe it's the use of sudo, I'm not sure. Any ideas? Thanks
function update_install()
{
# builds up a command as a string...
local install_cmd="$(make_install_command $#)"
# At this point the command as a string looks like: "sudo ~some_server/bin/do_install arg1 arg2"
print "----------------------------------------------------------------------------"
print "Will update install"
print "With command: ${install_cmd}"
print "----------------------------------------------------------------------------"
echo "trying backticks"
`${install_cmd}`
echo "Trying \$()"
$(${install_cmd})
echo "Trying \$="
$=install_cmd
}
Output:
Will update install
With command: sudo ~some_server/bin/do_install arg1 arg2
trying backticks
update_install:9: no such file or directory: sudo ~some_server/bin/do_install arg1 arg2
Trying $()
update_install:11: no such file or directory: sudo ~some_server/bin/do_install arg1 arg2
Trying $=
sudo ~some_server/bin/do_install arg1 arg2: command not found
Use eval:
eval ${install_cmd}
As explained in ยง3.1 "Why does $var where var="foo bar" not do what I expect?" of the Z-Shell FAQ, you can use the shwordsplit shell option to tell zsh that you want it to split variables up by spaces and treat them as multiple words. That same page also discusses alternatives that you might want to consider.
I believe you have two problems here - the first is that your install_cmd is being interpreted as a single string, instead of a command (sudo) with 3 arguments.
Your final attempt $=install_cmd actually does solve that problem correctly (though I'd write it as ${=install_cmd} instead), but then you hit your second problem: ~some_server/bin/do_install is not a known command. This is because sudo doesn't interpret the ~ like you intend, for safety reasons; it would need to evaluate its arguments using the shell (or do some special-casing for ~, which is really none of sudo's business), which opens up a whole can of worms that, understandably, sudo does its best to avoid.
That's also why it worked to do eval ${install_cmd} - because that's literally treating the whole string as a thing to be evaluated, possibly containing multiple commands (e.g. if install_cmd contained echo foo; sudo rm -rf / it would be happy to wipe your system).
You have to be the one to decide whether you want install_cmd to allow full shell semantics, including variable interpolation, path expansion, multiple commands, etc. or whether it should just expand the words out and run them as a single command.
Previous answers are correct, what worked for me was the below command in zsh
totalin=$(eval echo testing |wc -l)
echo "Total: $totalin"

Resources