This question already has answers here:
Stop shell wildcard character expansion?
(4 answers)
Closed 3 years ago.
This will be better explained with an example. I'm building a little bash script that reruns the passed commandline every 1 second. this is my first attempt:
-- file wtbash
watch -n1 "$#"
If I issue:
wt "df -h | grep sda"
It works ok. I want to be able to do:
wt df -h | grep sda
And to work just the same
That is, I want to treat "df -h | grep sda" as a single string param
There's an app a variable for that! From man bash:
* Expands to the positional parameters, starting from one. When the expan‐
sion is not within double quotes, each positional parameter expands to a
separate word. In contexts where it is performed, those words are sub‐
ject to further word splitting and pathname expansion. When the expan‐
sion occurs within double quotes, it expands to a single word with the
value of each parameter separated by the first character of the IFS spe‐
cial variable. That is, "$*" is equivalent to "$1c$2c...", where c is
the first character of the value of the IFS variable. If IFS is unset,
the parameters are separated by spaces. If IFS is null, the parameters
are joined without intervening separators.
So change your script to:
#!/bin/bash
watch -n1 "$*"
But that won't work for pipes. You will still need to escape those:
wt df -h \| grep sda
Related
I have read here Difference between single and double quotes in Bash that single quotes will not allow any kind of interpolation. However I came across a situation where single quotes were actually the only way to interpolate my variable' value.
For the sake of an example I created a file named 'test_file' containing
1
2
3
4
5
6
7
8
9
10
Than in the same directory I created a script whose purpose was to extract parts of this file, I called it test.sh and this is its content:
#!/bin/bash
BEGIN=3
END=9
cat test_file | sed -n ''$BEGIN', '$END'p'
Now I tried different setups of the last line like:
cat test_file | sed -n '${BEGIN}, ${END}p'
cat test_file | sed -n '"${BEGIN}", "${END}"p'
cat test_file | sed -n '"$BEGIN", "$END"p'
But none of them worked.
My question is: why does it work that way?
Quotes of both flavors are a shortcut: they cause the enclosed characters to be treated as if they were each escaped by \. The difference between single and double quotes, then, is that double quotes do not escape $ (or following characters which form part of an expansion), allowing parameter and command substitutions to occur (with the results being treated as double-quoted strings as well).
Your first consists of an empty string '', followed by an unquoted expansion, then the literal string ,, the unquoted expansion, and a literal p.
Your second is a single literal sting.
Your third adds literal double quotes around the literal strings ${BEGIN} and ${END}.
The fourth is similar to the third, but drops the literal { and }.
The correct string is
sed -n "$BEGIN, ${END}p" # sed -n "3, 9p"
This question already has answers here:
How can I preserve quotes in printing a bash script's arguments
(7 answers)
Closed 4 years ago.
I have a script that logs the user argument list. This list is later processed by getopt.
If the script is started like:
./script.sh -a 'this is a sentence' -b 1
... and then I save "$#", I get:
-a this is a sentence -b 1
... without the single quotes. I think (because of the way Bash treats quotes) these are removed and are not available to the script.
For logging accuracy, I'd like to include the quotes too.
Can the original argument list be obtained without needing to quote-the-quotes?
No, there is no way to obtain the command line from before the shell performed whitespace tokenization, wildcard expansion, and quote removal on it.
If you want to pass in literal quotes, try
./script.sh '"-a"' '"this is a sentence"' '"-b"' '"1"'
Notice also how your original command line could have been written
'./script.sh' '-a' 'this is a sentence' '-b' '1'
This question already has answers here:
Expansion of variables inside single quotes in a command in Bash
(8 answers)
Closed 4 years ago.
I have a very simple bash script with three commands.
The first command strips the first word off of the last git commit, the second command attempts to make a POST call to an api endpoint, with that same variable as part of the call, and the third command just prints that variable, to ensure it was working properly. See the code below
SOMETHING=$(git log -1 --pretty=%B | head -n1 | sed -e 's/\s.*$//' | cut -d ' ' -f1)
curl -X POST \
http://www.someurl.com/ \
-H 'Cache-Control: no-cache' \
-d '{"item":"$SOMETHING"}'
echo "variable was $SOMETHING"
When I run that bash script, I get a response from the service saying that "item was not set properly" in XML, however it does correctly echo the correct variable. So I know that first line is working. If I copy that curl command and paste it into bash, replacing $SOMETHING with the actual value, it works fine.
Single quotes do not expand the $variables inside them.
Try
'{"item":"'"$SOMETHING"'"}'
instead. Brief explanation:
'{"item":"' is a string delimited by single quotes that contains double quotes
"$SOMETHING" is a string delimited by double quotes, that expands the variable $SOMETHING
'"}' is again a ''-delimited string that contains double quotes
Simply writing those strings in a row without gaps is string concatenation
In this way, you get your variable expansion, but don't have to insert any backslashes to escape the double quotes.
This question already has answers here:
Why does shell ignore quoting characters in arguments passed to it through variables? [duplicate]
(3 answers)
Closed 6 years ago.
If i use this command in pipeline, it's working very well;
pipeline ... | grep -P '^[^\s]*\s3\s'
But if I want to set grep into variable like:
var="grep -P '^[^\s]*\s3\s'"
And if I put variable in pipeline;
pipeline ... | $var
nothing happens, like there isn't any matches.
Any help what am I doing wrong?
The robust way to store a simple command in a variable in Bash is to use an array:
# Store the command names and arguments individually
# in the elements of an *array*.
cmd=( grep -P '^[^\s]*\s3\s' )
# Use the entire array as the command to execute - be sure to
# double-quote ${cmd[#]}.
echo 'before 3 after' | "${cmd[#]}"
If, by contrast, your command is more than a simple command and, for instance, involves pipes, multiple commands, loops, ..., defining a function is the right approach:
# Define a function encapsulating the command...
myGrep() { grep -P '^[^\s]*\s3\s'; }
# ... and use it:
echo 'before 3 after' | myGrep
Why what you tried didn't work:
var="grep -P '^[^\s]*\s3\s'"
causes the single quotes around the regex to become a literal, embedded part of $var's value.
When you then use $var - unquoted - as a command, the following happens:
Bash performs word-splitting, which means that it breaks the value of $var into words (separate tokens) by whitespace (the chars. defined in special variable $IFS, which contains a space, a tab, and a newline character by default).
Bash also performs globbing (pathname expansion) on the resulting works, which is not a problem here, but can have unintended consequences in general.
Also, if any of your original arguments had embedded whitespace, word splitting would split them into multiple words, and your original argument partitioning is lost.
(As an aside: "$var" - i.e., double-quoting the variable reference - is not a solution, because then the entire string is treated as the command name.)
Specifically, the resulting words are:
grep
-P
'^[^\s]*\s3\s' - including the surrounding single quotes
The words are then interpreted as the name of the command and its arguments, and invoked as such.
Given that the pattern argument passed to grep starts with a literal single quote, matching won't work as intended.
Short of using eval "$var" - which is NOT recommended for security reasons - you cannot persuade Bash to see the embedded single quotes as syntactical elements that should be removed (a process appropriate called quote removal).
Using an array bypasses all these problems by storing arguments in individual elements and letting Bash robustly assemble them into a command with "${cmd[#]}".
What you are doing wrong is trying to store a command in a variable. For simplicity, robustness, etc. commands are stored in aliases (if no arguments) or functions (if arguments), not variables. In this case:
$ alias foo='grep X'
$ echo "aXb" | foo
aXb
I recommend you read the book Shell Scripting Recipes by Chris Johnson ASAP to get the basics of shell programming and then Effective Awk Programming, 4th Edition, by Arnold Robbins when you're ready to start writing scripts to manipulate text.
I've got a variable named formatted_deploy_history that has multi-lined content. I'm trying to pipe that content into a series of commands and save the end result to a variable. Here's what I've done:
nwhin_version="$(echo "${formatted_deploy_history}" | grep "productName" | grep "NHINC" | head -n1 | grep -o "[0-9]*")"
This works the way I want, but it seems like it should be a syntax error. Why is it that the second quote doesn't interfere with the rest of the command? I'd expect a syntax error because this would be interpreted like this:
nwhin_version="$(echo "
with trailing characters.
Your string contains command substitution ($(...)), which is its own world - delimited by $( and ).
Inside that world, you're free to use double quotes without escaping.
See section 2.3, "Token Recognition", in the POSIX Shell Command Language specification:
If the current character is an unquoted $ or `, the shell shall identify the start of any candidates for parameter expansion ( Parameter Expansion), command substitution ( Command Substitution), or arithmetic expansion ( Arithmetic Expansion) from their introductory unquoted character sequences: $ or ${, $( or ```, and "$((", respectively. The shell shall read sufficient input to determine the end of the unit to be expanded (as explained in the cited sections). While processing the characters, if instances of expansions or quoting are found nested within the substitution, the shell shall recursively process them in the manner specified for the construct that is found. The characters found from the beginning of the substitution to its end, allowing for any recursion necessary to recognize embedded constructs, shall be included unmodified in the result token, including any embedded or enclosing substitution operators or quotes. The token shall not be delimited by the end of the substitution.
I believe you don't need quotes around the command substitution.
Try this:
nwhin_version=$(echo "${formatted_deploy_history}" | grep "productName" | grep "NHINC" | head -n1 | grep -o "[0-9]*")