while writing some script i found the following issue ,
suppose
set x = "param[xyz]";
echo $x
the output was echo:No match
But when i do ,
echo "$x"
i got the output: param[xyz]
so echo is doing a two way substitution ,
Initially echo $x was converted to echo param[xyz] and then it tried to look for the param[xyz] value .
But Ideally it should have just printed the variable whatever provided to it .
Does this behavior is a valid use case?
echo does no substitution at all, it's the shell that does it. It depends on the shell you use, but it seems that you are using a shell of the family of c-shells. Shell expands variables in the command line, so the first step is to generate:
[csh] echo param[xyz]
and then the shell does file match expansion, but as there is no file that correspond to the pattern the shell answers that there is no match. The message is somehow misleading as the shell reminds you what "command" was concerned not that the command failed by itself.
In the second try, enclosing the variable inside " prevent the shell to do other expansion and the shell launch the command with the argument obtained after the first expansion.
There exists another prevention if you use ', the shell won't ever expand variables:
[csh] echo '$x'
$x
Please refer to shell documentation and especially about expansion.
Another experience to convince you is to try with an non existing command:
[csh] weirdo z*
weirdo: No match
which is different than an non existing command:
[csh] weirdo
weirdo: Command not found.
If you'd use other shell behavior would be different:
[bash] echo z*
z*
because that shell produces as the argument the string itself in the case file matching is not working.
With:
[zsh] echo z*
zsh: no matches found: z*
the behavior is much more similar to c-shells but the message is much more clear, the shell failed at matching.
Related
I have googled and found nothing. I am reading a shell script file and there there is this line
echo "docker run command"
echo ${command_}
What does this mean? Please notice that
there is an underscore after "command"
command_ seems not to be defined elsewhere
What is ${command_} in a shell script
${command_} means to expand the variable named command_ to it's value.
What does this mean?
echo "docker run command" - means to execute the command echo (possibly a builtin) with one argument docker run command.
echo ${command_} - means to execute the command echo (possibly a builtin) with the result of expansion of the variable command_ will undergo word splitting expansion and then the result will be passed as arguments to the echo command.
there is an underscore after "command"
It doesn't matter - underscore is nothing special.
command_ seems not to be defined elsewhere
That means that ${command_} expansion will expand to an empty string, and the second echo will run with no arguments.
echo command is used to print the arguments passed. But if argument has $ to it, it means it evaluates its value and prints the same.
Here, you shell script would be having a variable named "command_". It'll just find its value, if defined, and print it. If its not defined, nothing will be printed.
NOTE: The goal of this question is to find a suitable character sequence for an effective placeholder substitution in a bash script, not finding if a command is evaluated or not.
I have a skeleton named my_script.skelof a bash script in which I have to put a placeholder. I want to find a placeholder sequence that can be safely substituted (I mean that there aren't any clashes with other bash commands or other pathological substitutions, see A non-safe example for an example).
I've figured out on my own that enveloping placeholder_name within #~ and ~# seems a good solution, but I'm not sure that this solution is safe in every possible case.
A non-safe example
One can decide to use /placeholder_name/. So my_dumb_script.skel is:
#!/bin/bash
AN_INNOCENT_PATH="/a/simple/placeholder_name/path"
echo /placeholder_name/
The goal is to replace only /placeholder_name/ in the echo command. If now I use sed on the placeholder:
sed 's%/placeholder_name/%foobar%g' my_dumb_script.skel > output.bash
The output could be unexpected:
#!/bin/bash
AN_INNOCENT_PATH="/a/simplefoobarpath"
echo foobar
In this case we've obtained an unwanted substitution inside AN_INNOCENT_PATH, since it's easy to pattern-match on placeholders that contains the / character. I know that it seems very dumb, but you cannot know how people will use your code in the future (and someone could create a folder named placeholder_name).
A safe example that uses #~
In this case my_better_script.skel is the following one:
#!/bin/bash
AN_INNOCENT_PATH="/a/simple/placeholder_name/path"
echo #~placeholder_name~#
And now we can use sed:
sed 's%#~placeholder_name~#%foobar%g' my_better_script.skel > output.bash
The output now is better:
#!/bin/bash
AN_INNOCENT_PATH="/a/simple/placeholder_name/path"
echo foobar
Now everything works as intended.
You can use type to check if a command will evaluate.
$ placeholder1="testing"
$ placeholder2="test"
$ type $placeholder1
-bash: type: testing: not found
$ type $placeholder2
test is a shell builtin
This seems like it should be simple, but I'm pulling out my remaining hair trying to get it to work. In a shell script I want to run some Applescript code that defines a string, then pass that string (containing a single quote) to a shell command that calls PHP's addslashes function, to return a string with that single quote escaped properly.
Here's the code I have so far - it's returning a syntax error.
STRING=$(osascript -- - <<'EOF'
set s to "It's me"
return "['test'=>'" & (do shell script "php -r 'echo addslashes(\"" & s & "\");") & "']"
EOF)
echo -e $STRING
It's supposed to return this:
['test'=>'It\'s me']
First, when asking a question like this, please include what's happening, not just what you're trying to do. When I try this, I get:
42:99: execution error: sh: -c: line 0: unexpected EOF while looking for matchin
sh: -c: line 1: syntax error: unexpected end of file (2)
(which is actually two error messages, with one partly overwriting the other.) Is that what you're getting?
If it is, the problem is that the inner shell command you're creating has quoting issues. Take a look at the AppleScript snippet that tries to run a shell command:
do shell script "php -r 'echo addslashes(\"" & s & "\");"
Since s is set to It's me, this runs the shell command:
php -r 'echo addslashes("It's me");
Which has the problem that the apostrophe in It's me is acting as a close-quote for the string that starts 'echo .... After that, the double-quote in me"); is seen as opening a new quoted string, which doesn't get closed before the end of the "file", causing the unexpected EOF problem.
The underlying problem is that you're trying to pass a string from AppleScript to shell to php... but each of those has its own rules for parsing strings (with different ideas about how quoting and escaping work). Worse, it looks like you're doing this so you can get an escaped string (following which set of escaping rules?) to pass to something else... This way lies madness.
I'm not sure what the real goal is here, but there has to be a better way; something that doesn't involve a game of telephone with players that all speak different languages. If not, you're pretty much doomed.
BTW, there are a few other dubious shell-scripting practices in the script:
Don't use all-caps variable named in shell scripts. There are a bunch of all-caps variables that have special meanings, and if you accidentally use one of those for something else, weird results can happen.
Put double-quotes around all variable references in scripts, to avoid them getting split into multiple "words" and/or expanded as shell wildcards. For example, if the variable string was set to "['test'=>'It\'s-me']", and you happened to have files named "t" and "m" in the current directory, echo -e $string will print "m t" because those are the files that match the [] pattern.
Don't use echo with options and/or to print strings that might contain escapes, since different versions treat these things differently. Some versions, for example, will print the "-e" as part of the output string. Use printf instead. The first argument to printf is a format string that tells it how to format all of the rest of the arguments. To emulate echo -e "$string" in a more reliable form, use printf '%b\n' "$string".
To complement Gordon Davisson's helpful answer with a pragmatic solution:
Shell strings cannot contain \0 (NUL) characters, but the following sed command emulates all other escaping that PHP's (oddly named) addslashes PHP function performs (\-escaping ', " and \ instances):
string=$(osascript <<'EOF'
set s to "It's me\\you and we got 3\" of rain."
return "['test'=>'" & (do shell script "sed 's/[\"\\\\'\\'']/\\\\&/g' <<<" & quoted form of s) & "']"
EOF
)
printf '%s\n' "$string"
yields
['test'=>'It\'s me\\you and we got 3\" of rain.']
Note the use of quoted form of, which is crucial for passing a string from AppleScript to a do shell script shell command with proper quoting.
Also note how the closing here-doc delimiter, EOF, is on its own line to ensure that it is properly recognized (in Bash 3.2.57, as used on macOS 10.12, (also when called as /bin/sh, which is what do shell script does), this isn't strictly necessary, but Bash 4.x would rightfully complain about EOF) with warning: here-document at line <n> delimited by end-of-file (wanted 'EOF')
I tried following code.
command1="echo"
"${command1}" 'case1'
command2="echo -e"
"${command2}" 'case2'
echo -e 'case3'
The outputs are following,
case1
echo -e: command not found
case3
The case2 results in an error but similar cases, case1 and case3 runs well. It seems command with option cannot be recognized as valid command.
I would like to know why it does not work. Please teach me. Thank you very much.
Case 1 (Unmodified)
command1="echo"
"${command1}" 'case1'
This is bad practice as an idiom, but there's nothing actively incorrect about it.
Case 2 (Unmodified)
command2="echo -e"
"${command2}" 'case2'
This is looking for a program named something like /usr/bin/echo -e, with the space as part of its name.
Case 2 (Reduced Quotes)
# works in this very specific case, but bad practice
command2="echo -e"
$command2 'case2' # WITHOUT THE QUOTES
...this one works, but only because your command isn't interesting enough (doesn't have quotes, doesn't have backslashes, doesn't have other shell syntax). See BashFAQ #50 for a description of why it isn't an acceptable practice in general.
Case X (eval -- Bad Practice, Oft Advised)
You'll often see this:
eval "$command1 'case1'"
...in this very specific case, where command1 and all arguments are hardcoded, this isn't exceptionally harmful. However, it's extremely harmful with only a small change:
# SECURITY BUGS HERE
eval "$command1 ${thing_to_echo}"
...if thing_to_echo='$(rm -rf $HOME)', you'll have a very bad day.
Best Practices
In general, commands shouldn't be stored in strings. Use a function:
e() { echo -e "$#"; }
e "this works"
...or, if you need to build up your argument list incrementally, an array:
e=( echo -e )
"${e[#]}" "this works"
Aside: On echo -e
Any implementation of echo where -e does anything other than emit the characters -e on output is failing to comply with the relevant POSIX standard, which recommends using printf instead (see the APPLICATION USAGE section).
Consider instead:
# a POSIX-compliant alternative to bash's default echo -e
e() { printf '%b\n' "$*"; }
...this not only gives you compatibility with non-bash shells, but also fixes support for bash in POSIX mode if compiled with --enable-xpg-echo-default or --enable-usg-echo-default, or if shopt -s xpg_echo was set, or if BASHOPTS=xpg_echo was present in the shell's environment at startup time.
If the variable command contains the value echo -e.
And the command line given to the shell is:
"$command" 'case2'
The shell will search for a command called echo -e with spaces included.
That command doesn't exist and the shell reports the error.
The reason of why this happen is depicted in the image linked below, from O'Reilly's Learning the Bash Shell, 3rd Edition:
Learning the bash Shell, 3rd Edition
By Cameron Newham
...............................................
Publisher: O'Reilly
Pub Date: March 2005
ISBN: 0-596-00965-8
Pages: 352
If the variable is quoted (follow the right arrows) it goes almost (passing steps 6,7, and 8) directly to execution in step 12.
Therefore, the command searched has not been split on spaces.
Original image (removed because of #CharlesDuffy complaint, I don't agree, but ok, let's move to the impossible to be in fault side) is here:
Link to original image in the web site where I found it.
If the command line given to the shell is un-quoted:
$command 'case2'
The string command gets expanded in step 6 (Parameter expansion) and then the value of the variable $command: echo -e gets divided in step 9: "Word splitting".
Then the shell search for command echo with argument -e.
The command echo "see" an argument of -e and echo process it as an option.
Trying to store commands inside an string is a very bad idea.
Try this, think very carefully of what you would expect the out put to be, and then be surprised on execution:
$ command='echo -e case2; echo "next line"'; $command
To take a look at what happens, execute the command as this:
$ set -vx; $command; set +vx
It works on my machine if I give the command this way:
cmd2="echo -e"
if you are still facing a problem I would suggest storing the options in another variable so that if you are doing shell scripting then multiple commands that use similar option values you can leverage the variable so also try something like this.
cmd1="echo"
opt1="-e"
$cmd1 $opt1 Hello
I see in shell scripts a variable assignment followed (without a semicolon) by a command. What is the meaning of that? It does not seem to affect this command, and it does not seem to affect the next command the way normal assignment would:
>echo $FOOBAR
>FOOBAR=1 echo $FOOBAR
>echo $FOOBAR
>
So, what does it do?
It sets that environment variable to that value only for that process.
Here, step by step, is what happens:
Original command: FOOBAR=1 echo $FOOBAR
Shell performs substitution: command is now FOOBAR=1 echo
Shell fork(2)s a new process
Environment variable is created in the new process: command is now echo (with $FOOBAR equal to 1)
New process is exec(3)ed: \n is output
New process exits and is reaped by the shell
At no point does the parent process see the assigned value of $FOOBAR.
On first run of echo $FOOBAR you will get empty result on your screen as the FOOBAR variable is not set yet.
Then when you set it to 1,and run it on the second time it will display 1.
Note: This value will be lost when you close your terminal where you wrote these. $ notation in shell and shell scripting in Unix like environments simply denotes a variable.
What happens in your case in second command i.e FOOBAR=1 echo $FOOBAR is that the line is not syntactically incorrect but it breaches the contract of equalto(=) which actually does not accept a space thus no error is thrown and FOOBAR=1 is just treated as a parameter and it fails while the command for echo executes successfully.
I would suggest try this :
FOOBAR=1
echo $FOOBAR
in different lines or
FOOBAR=1 ; echo $FOOBAR in one line.