How to escape quotes and pipe?
#!/bin/bash
set -x
MYCMD="VBoxManage showvminfo --machinereadable $1 \| grep \'VMState=\"poweroff\"\'"
echo "`$MYCMD`"
Executed command :
++ VBoxManage showvminfo --machinereadable d667 '|' grep '\'\''VMState="poweroff"\'\'''
And finally getting this error:
Syntax error: Invalid parameter '|'
You don't; you would need to use eval to embed an arbitrary pipeline in a regular string parameter.
MYCMD="VBoxManage showvminfo --machinereadable \"$1\" | grep 'VMState=\"poweroff\"'"
eval "$MYCMD"
However, this is not recommended unless you are certain that the value of $1 will not cause problems. (If you need an explanation of what those risks might be, then you should not be using eval.)
Instead, define a shell function:
mycmd () {
VBoxManage showvminfo --machinereadable "$1" | grep 'VMState="poweroff"'
}
mycmd "$1"
One really simple way to do it is by using an array or positional parameters.
Array-based solution :
# Build command
declare -a CMD_AND_ARGS=(command and args with normal quoting)
# Append arguments
CMD_AND_ARGS+=(more arguments quoted the normal way)
# Execute command
"${CMD_AND_ARGS[#]}"
Positional parameter-based solution:
# Create command
set -- command and args with normal quoting
# Append arguments
set -- "$#" more arguments quoted the normal way
# Execute command
"$#"
The nice thing about both solutions is you do not need to put quotes inside quotes, because expanding positional parameters or an array surrounded by double quotes does not cause word-splitting and expansion to be performed again.
Examples:
declare -a CMD=()
CMD=(ls "/dir/with spaces/in its name")
"$CMD"
set -- ls "/dir/with spaces/in its name"
"$#"
Note that in both cases, you get to build your command incrementally, for instance having conditional expressions (e.g. if/case) choosing to add different arguments depending on the flow of your script.
If you want to pipe a command to another, you will have to build each command separately (e.g. two arrays), as the | symbol cannot be used inside an array declaration unquoted, and once quoted will be treated as a string argument and will not cause piping to occur.
Related
I have the following case where exec and eval will handle variables passed as arguments differently.
Here, eval seems to output something which is intended.
But is there any alternative to using that?
$ cat arg.sh
#!/bin/bash
eval ./argtest $*
$ ./arg.sh "arg1 'subarg1 subarg2'"
Args: 2
Arg1: arg1
Arg2: subarg1 subarg2
But at the same time if I use exec instead of eval call, the single quotes are not getting honored.
$ ./arg.sh "arg1 'subarg1 subarg2'"
Args: 3
Arg1: arg1
Arg2: 'subarg1
Arg3: subarg2'
You should do:
#!/bin/bash
./argtest "$#"
To properly pass unchanged arguments.
Then do:
$ ./arg.sh arg1 'subarg1 subarg2'
As you would do with any other command.
Research when to use quoting in shell, how is $# positional arguments expansions handled specially in quotes, research how does $* and $# differ and research word splitting. Also research what is variable expansion and in which contexts it happens and how does single quotes differ from double quotes. And because exec is mentioned see bashfaq Eval command and security issues. Remember to check your scripts with https://shellcheck.net .
Is there any alternative to using eval in a shell script to achieve variable expansion
Yes - use envsubst for variable expansion, it's a tool just for that.
#!/bin/bash
arg=$(VARIABLE=something envsubst '$VARIABLE' <<<"$1")
./argtest "$arg"
$ bash -x ./arg.sh 'string with **not-expanded** $VARIABLE'
+ ./argtest 'string with **not-expanded** something'
Is there any alternative to using eval in a shell script to achieve *single quotes parsing
Yes - you would potentially write your own parser, probably in awk, that would split the string and then reload. A very very crude example:
#!/bin/bash
readfile -t args < <(sed "s/ *'\([^']*\)' */\n\1\n/; s/\n$//" <<<"$*")
./argtest "${args[#]}"
$ bash -x ./arg.sh "arg1 'subarg1 subarg2'"
+ ./argtest 'arg1' 'subarg1 subarg2'
Using $*, the shell applies word splitting to the parameters and passes the effect after word splitting to eval, repsepcitvely exec. What happens after, differs between them:
exec simply replaces the current process by a new one, based on the first parameter it gets. Than in passes the remaining parameters unmodified to this process.
eval on the other hand catenates the parameters together to a single string (using one space as a separator between those strings), then treats this resulting string as a new command where the usual expansion and word splitting mechanism of bash are applied, and finally runs this command.
The mechanism is completely different, which is not surprising, since these commands serve a different purpose.
What is the correct way to call some command stored in variable?
Are there any differences between 1 and 2?
#!/bin/sh
cmd="ls -la $APPROOTDIR | grep exception"
#1
$cmd
#2
eval "$cmd"
Unix shells operate a series of transformations on each line of input before executing them. For most shells it looks something like this (taken from the Bash man page):
initial word splitting
brace expansion
tilde expansion
parameter, variable and arithmetic expansion
command substitution
secondary word splitting
path expansion (aka globbing)
quote removal
Using $cmd directly gets it replaced by your command during the parameter expansion phase, and it then undergoes all following transformations.
Using eval "$cmd" does nothing until the quote removal phase, where $cmd is returned as is, and passed as a parameter to eval, whose function is to run the whole chain again before executing.
So basically, they're the same in most cases and differ when your command makes use of the transformation steps up to parameter expansion. For example, using brace expansion:
$ cmd="echo foo{bar,baz}"
$ $cmd
foo{bar,baz}
$ eval "$cmd"
foobar foobaz
If you just do eval $cmd when we do cmd="ls -l" (interactively and in a script), you get the desired result. In your case, you have a pipe with a grep without a pattern, so the grep part will fail with an error message. Just $cmd will generate a "command not found" (or some such) message.
So try use to eval (near "The args are read and concatenated together") and use a finished command, not one that generates an error message.
$cmd would just replace the variable with it's value to be executed on command line.
eval "$cmd" does variable expansion & command substitution before executing the resulting value on command line
The 2nd method is helpful when you wanna run commands that aren't flexible eg.
for i in {$a..$b} format loop won't work because it doesn't allow variables. In this case, a pipe to bash or eval is a workaround.
Tested on Mac OSX 10.6.8, Bash 3.2.48
I think you should put
`
(backtick) symbols around your variable.
I'm trying to make a little script as a wrapper around this command:
$ egrep target /usr/lusers/me/test/*test.txt
/usr/lusers/me/test/1test.txt:target
That directory has files called 1test.txt and 2test.txt, one of which contains some text I want to find.
Here is my whole script, called mygrep.sh:
set -v
set -x
egrep "$1" '/usr/lusers/me/test/*test.txt'
Here's the output:
$ ./mygrep.sh target
set -x
egrep "$1" '/usr/lusers/me/test/*test.txt'
++ egrep targ '/usr/lusers/me/test/*test.txt'
egrep: /usr/lusers/me/test/*test.txt: No such file or directory
Note the 's around the file path in the set -x output, and that the command fails.
Now compare this variation of the script:
set -v
set -x
egrep "$1" '/usr/lusers/me/test/1test.txt'
Note that the only difference is the asterisk vs the literal file name.
Output:
$ ./mygrep.sh target
set -x
egrep "$1" '/usr/lusers/me/test/1test.txt'
++ egrep target /usr/lusers/me/test/1test.txt
target
No single quotes after expansion, and the command works.
So why are those single quotes added when there's an asterisk, and why is the command failing in that case?
The output resulting from set -x is for debugging purposes. No quotes are added to the argument; they are just for display purposes.
The correct command is egrep "$1" /usr/lusers/me/test/*.test.txt, because the shell must expand the pattern (if possible) before passing the results to egrep. You don't have an actual file named *.test.txt.
The globbing character must be outside (single or double) quotes, as quotes disable globbing.
Use this instead :
egrep "$1" '/usr/lusers/me/test/'*'test.txt'
Or this :
egrep "$1" "/usr/lusers/me/test/"*"test.txt"
Or, since there is nothing inside this specific pattern that would cause word splitting to occur (but that would not be a generally safe way to do it if the path is not known safe in advance) :
egrep "$1" /usr/lusers/me/test/*test.txt
What is the correct way to call some command stored in variable?
Are there any differences between 1 and 2?
#!/bin/sh
cmd="ls -la $APPROOTDIR | grep exception"
#1
$cmd
#2
eval "$cmd"
Unix shells operate a series of transformations on each line of input before executing them. For most shells it looks something like this (taken from the Bash man page):
initial word splitting
brace expansion
tilde expansion
parameter, variable and arithmetic expansion
command substitution
secondary word splitting
path expansion (aka globbing)
quote removal
Using $cmd directly gets it replaced by your command during the parameter expansion phase, and it then undergoes all following transformations.
Using eval "$cmd" does nothing until the quote removal phase, where $cmd is returned as is, and passed as a parameter to eval, whose function is to run the whole chain again before executing.
So basically, they're the same in most cases and differ when your command makes use of the transformation steps up to parameter expansion. For example, using brace expansion:
$ cmd="echo foo{bar,baz}"
$ $cmd
foo{bar,baz}
$ eval "$cmd"
foobar foobaz
If you just do eval $cmd when we do cmd="ls -l" (interactively and in a script), you get the desired result. In your case, you have a pipe with a grep without a pattern, so the grep part will fail with an error message. Just $cmd will generate a "command not found" (or some such) message.
So try use to eval (near "The args are read and concatenated together") and use a finished command, not one that generates an error message.
$cmd would just replace the variable with it's value to be executed on command line.
eval "$cmd" does variable expansion & command substitution before executing the resulting value on command line
The 2nd method is helpful when you wanna run commands that aren't flexible eg.
for i in {$a..$b} format loop won't work because it doesn't allow variables. In this case, a pipe to bash or eval is a workaround.
Tested on Mac OSX 10.6.8, Bash 3.2.48
I think you should put
`
(backtick) symbols around your variable.
I am trying to write a bash script so that I will use to replace my egrep command. I want to be able to take the exact same input that is given to my script and feed it to egrep.
i.e.
#!/bin/bash
PARAMS=$#
`egrep "$PARAMS"`
But I have noticed that if I echo what I am executing, that the quotes have been removed as follows:
./customEgrep -nr "grep my ish" *
returns
egrep -nr grep my ish (file list from the expanded *)
Is there a way that I can take the input literally so I can use it directly with egrep?
You want this:
egrep "$#"
The quotes you type are not passed to the script; they're used to determine word boundaries. Using "$#" preserves those word boundaries, so egrep will get the same arguments as it would if you ran it directly. But you still won't see quotation marks if you echo the arguments.
" is a special char. you need to use escape character in order to retrieve "
use
./customEgrep -nr "\"grep my ish\"" *
If you don't need to do any parameter expansion in the argument, you can use
single quotes to avoid the need to escape the double quotes:
./customerEgrep -nr '"grep my ish"' *
$# is special when quoted. Try:
value=$( egrep "$#" )
It's not clear to me why you are using bacticks and ignoring the result, so I've used the $() syntax and assigned the value.
If for some reason you want to save the parameters to use later, you can also do things like:
for i; do args="$args '$i'"; done # Save the arguments
eval grep $args # Pass the arguments to grep without resetting $1,$2,...
eval set $args # Restore the arguments
grep "$#" # Use the restored arguments