How to match a single quote in sed if the expression is enclosed in single quotes:
sed -e '...'
For example need to match this text:
'foo'
You can either use:
"texta'textb" (APOSTROPHE inside QUOTATION MARKs)
or
'texta'\''textb' (APOSTROPHE text APOSTROPHE, then REVERSE SOLIDUS, APOSTROPHE, then APOSTROPHE more text APOSTROPHE)
I used unicode character names. REVERSE SOLIDUS is more commonly known as backslash.
In the latter case, you close your apostrophe, then shell-quote your apostrophe with a backslash, then open another apostrophe for the rest of the text.
As noted in the comments to the question, it's not really about sed, but how to include a quote in a quoted string in a shell (e.g. bash).
To clarify a previous answer, you need to escape the quote with a backslash, but you can't do that within a single-quoted expression. From the bash man page:
Enclosing characters in single quotes
preserves the literal value of each
character within the quotes. A single
quote may not occur between single
quotes, even when preceded by a
backslash.
Therefore, you need to terminate the quoted expression, insert the escaped quote, and start a new quoted expression. The shell's quote removal does not add any extra spaces, so in effect you get string concatenation.
So, to answer the original question of how to single quote the expression 'foo', you would do something like this:
sed -e '...'\''foo'\''...'
(where '...' is the rest of the sed expression).
Overall, for the sake of readability, you'd be much better off changing the surrounding quotes to double quotes if at all possible:
sed -e "...'foo'..."
[As an example of the potential maintenance nightmare of the first (single quote) approach, note how StackOverflow's syntax highlighting colours the quotes, backslashes and other text -- it's definitely not correct.]
For sed, a very simple solution is to change the single quotation format to a double quote.
For a given variable that contains single quotes
var="I'm a string with a single quote"
If double quotes are used for sed, this will match the single quote.
echo $var | sed "s/'//g"
Im a string with a single quote
Rather than single quotes, which will hang
echo $var | sed 's/'//g'
You can also use ['] to match a literal single quote without needing to do any shell quoting tricks.
myvar="stupid computers can't reason about life"
echo "$myvar" | sed -e "s/[']t//"
Outputs:
stupid computers can reason about life
Related
I am on Mac OS X and using sed for an in-place replacement.
Essentially I have this:
#!/bin/sh -e
PREFIX="$1"
sed -i bak -e 's|OCAMLDIR|"${PREFIX}"|g' ocamloptrev
Where PREFIX is a path, hence I'm using the |.
Unfortunately, the variable in the file path is not getting evaluated as I expected, I end up with:
OCAMLC="${PREFIX}"/bin/ocamlopt
How can I get the right evaluation of ${PREFIX} into the sed command?
Try this:
#!/bin/sh -e
PREFIX="$1"
sed -i bak -e 's|OCAMLDIR|'"${PREFIX}"'|g' ocamloptrev
What you're basically doing, is "exiting"/getting outside the single-quoted string, entering into a double-quoted string, interpreting the variable inside double-quotes, and then entering the single quotes again.
With this simple example, we could also just use double-quotes, which allow variables to be interpreted:
#!/bin/sh -e
PREFIX="$1"
sed -i bak -e "s|OCAMLDIR|${PREFIX}|g" ocamloptrev
If you try to use double-quotes ("") inside single-quotes, they don't get interpreted either. This part of the Bash manual explains this in more detail.
3.1.2.2 Single Quotes
Enclosing characters in single quotes (‘'’) preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.
3.1.2.3 Double Quotes
Enclosing characters in double quotes (‘"’) preserves the literal value of all characters within the quotes, with the exception of $, `, \, and, when history expansion is enabled, !. The characters $ and ` retain their special meaning within double quotes (see Shell Expansions). ...
Shell variables are not expanded inside single quotes (there are no metacharacters within single quotes, not even backslashes), so you need to use something like this, with the double quotes around ${PREFIX} ensuring that spaces etc in the value are handled correctly:
sed -i bak -e 's|OCAMLDIR|'"${PREFIX}"'|g' ocamloptrev
Or you could even use:
sed -i bak -e "s|OCAMLDIR|${PREFIX}|g" ocamloptrev
The latter is safe because the material inside the double quotes does not contain shell metacharacters (dollar signs, backslashes and back-quotes are the main danger signs). If there were dodgy characters in the rest of the string, the first version is safer to use.
Personally, I'd use .bak rather than just bak as the suffix.
I'm trying to write a script in bash which extracts a database name from a PHP file. For example I want to copy CRM_123456789 from the below line:
$sugar_config['dbconfig']['db_name'] = 'CRM_123456789';
I have tried using sed, so essentially I want to copy the text between
['db_name'] = '
and
';
sed -n '/['db_name'] = /,/';/p' myfile.php
However this does not return anything. Does anyone know what I'm doing wrong?
Thanks
You cannot nest single quotes. Your expression evaluates to single-quoted /[ next to unquoted db_name where clearly you want to match on a literal single quote.
One workaround is to use double quotes for the outermost quoting, but make sure you make any necessary changes, because double quotes are weaker than single quotes in the shell. In your case, there's nothing to change in that respect, though.
However, you also appear to misunderstand how sed address expressions work. They identify lines, not substrings on a line. So your script would print between a line matching ['db_name'] and a line matching ';. To extract something from within a line, the common idiom is to substitute out the parts you don't want, then print what's left.
Also, because opening square bracket is a metacharacter in sed, you need to backslash-escape it to match it literally.
sed -n "s/.*\['db_name'] = '\([^']*\)'.*/\1/p" myfile.php
This matches up through ['db_name'] = ', then captures whatever is inside the single-quoted string into \1, then matches anything from the next single quote through the end of line, and substitutes it with just the captured string; and prints that line after performing the substitution.
If the config file supports variable whitespace, a useful improvement would be to allow for optional whitespace around the equals sign, and possibly also within the square brackets. [ ]* will match zero or more spaces (the square brackets aren't really necessary around a single space, but I include them here for legibility reasons).
You could try the below sed command.
$ sed -n "s/.*\['db_name'\] = '\([^']*\)';.*/\1/p" file
CRM_123456789
I've got this in a for loop in a bash script:
xpath $f '//bad/objdesc/desc[$i]' > $f.$i.xml
$i is the counter.
It doesn't work.
How do I refer to $i properly in the brackets of the desc element?
Thanks.
Use double quotes instead, so that the variable gets expanded instead of being treated literally as $i:
xpath "$f" "//bad/objdesc/desc[$i]" > "$f.$i.xml"
From the bash manual:
3.1.2.2 Single Quotes
Enclosing characters in single quotes (‘'’) preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.
3.1.2.3 Double Quotes
Enclosing characters in double quotes (‘"’) preserves the literal value of all characters within the quotes, with the exception of ‘$’, ‘’, ‘\’, and, when history expansion is enabled, ‘!’. The characters ‘$’ and ‘’ retain their special meaning within double quotes (see Shell Expansions). The backslash retains its special meaning only when followed by one of the following characters: ‘$’, ‘`’, ‘"’, ‘\’, or newline. Within double quotes, backslashes that are followed by one of these characters are removed. Backslashes preceding characters without a special meaning are left unmodified. A double quote may be quoted within double quotes by preceding it with a backslash. If enabled, history expansion will be performed unless an ‘!’ appearing in double quotes is escaped using a backslash. The backslash preceding the ‘!’ is not removed.
Here's a much more efficient alternate approach, for the specific indexing-into-a-list case.
counter=1
while IFS='' read -r -d $'\x03' line; do
printf '%s' "$line" >"%f.$(( counter++ )).xml"
done < <(xmlstarlet sel -t -m //bad/objdesc/desc -v . -o $'\x03')
This runs xmlstarlet (a more capable alternative to xpath) only once, and retrieves the entire set of values (even should they be multiline) into a bash array, where they can be accessed as ${array[0]}, ${array[1]}, etc.
It has the caveat that values containing the literal EOT character within their contents will be split into multiple values on that character.
A word on safety:
Expanding query parameters -- whether XPath, SQL, or otherwise -- within double quotes in bash opens you up to query injection attacks. Let's say you had a variable with a customer IDs, and you were looking up account balances; if you used xpath "//record[customer='$id' and record='$rec']", a customer who could provide literal values for $rec could also look up content belonging to customers with a different ID by escaping the quotes.
The safe alternative is to use a tool that lets you pass your query in single-quotes, and pass your parameters out-of-band, as with a --var name="$value" parameter. Until it has this capability, I cannot advise the Perl xpath tool for general use.
Should someone other than the original poster of the question be considering using the xpath tool with user-provided or untrusted data, please keep the above in mind.
Should I double quote or escape with \ special characters like ',
$ echo "'"
'
$ echo \'
'
Here is apparently doesn't matter, but are there situations where there is a difference, except for $, `` or`, when I know there is a difference.
Thanks,
Eric J.
You can use either backslashes, single quotes, or (on occasion) double quotes.
Single quotes suppress the replacement of environment variables, and all special character expansions. However, a single quote character cannot be inside single quotes -- even when preceded by a backslash. You can include double quotes:
$ echo -e 'The variable is called "$FOO".'
The variable is called "$FOO".
Double quotes hide the glob expansion characters from the shell (* and ?), but it will interpolate shell variables. If you use echo -e or set shopt -s xpg_echo, the double quotes will allow the interpolation of backslash-escaped character sequences such as \a, and \t. To escape those, you have to backslash-escape the backslash:
$ echo -e "The \\t character sequence represents a tab character."
The \t character sequence represents a tab character."
The backslash character will prevent the expansion of special characters including double quotes and the $ sign:
$ echo -e "The variable is called \"\$FOO\"."
The variable is called "$FOO".
So, which one to choose? Which everyone looks the best. For example, in the preceding echo command, I would have been better off using single quotes and that way I wouldn't have the confusing array of backslashes one right after another.
On the other hand:
$ echo -e "The value of \$FOO is '$FOO'."
The value of FOO is 'bar'.
is probably better than trying something like this:
$ echo -e 'The value of $FOO is '"'$FOO'."
Readability should be the key.
I need use sed into bash script, for add lines after any line numer of script with some pair of values (below work)
sed -i.bak '14i\some_text=some_text' file
But I need on script bash (sh) for expand variables (below not work)
sed -i.bak '$number_linei\$var1=$var2' $var3
Just use double quotes instead of single quotes. You'll also need to use {} to delimit the number_line variable correctly and escape the \, too.
sed -i.bak "${number_line}i\\$var1=$var2" $var3
I'd personally prefer to see all of the variables use the {}, ending up with something like:
sed -i.bak "${number_line}i\\${var1}=${var2}" ${var3}
Change single quotes to double quotes:
man bash:
Enclosing characters in single quotes preserves the literal value of
each character within the quotes.
Enclosing characters in double quotes preserves the literal value of
all characters within the quotes, with the exception of $, `, \, and,
when history expansion is enabled, !. The characters $ and ` retain
their special meaning within double quotes.