Related
Below is the snippet of a shell script from a larger script. It removes the quotes from the string that is held by a variable. I am doing it using sed, but is it efficient? If not, then what is the efficient way?
#!/bin/sh
opt="\"html\\test\\\""
temp=`echo $opt | sed 's/.\(.*\)/\1/' | sed 's/\(.*\)./\1/'`
echo $temp
Use tr to delete ":
echo "$opt" | tr -d '"'
NOTE: This does not fully answer the question, removes all double quotes, not just leading and trailing. See other answers below.
There's a simpler and more efficient way, using the native shell prefix/suffix removal feature:
temp="${opt%\"}"
temp="${temp#\"}"
echo "$temp"
${opt%\"} will remove the suffix " (escaped with a backslash to prevent shell interpretation).
${temp#\"} will remove the prefix " (escaped with a backslash to prevent shell interpretation).
Another advantage is that it will remove surrounding quotes only if there are surrounding quotes.
BTW, your solution always removes the first and last character, whatever they may be (of course, I'm sure you know your data, but it's always better to be sure of what you're removing).
Using sed:
echo "$opt" | sed -e 's/^"//' -e 's/"$//'
(Improved version, as indicated by jfgagne, getting rid of echo)
sed -e 's/^"//' -e 's/"$//' <<<"$opt"
So it replaces a leading " with nothing, and a trailing " with nothing too. In the same invocation (there isn't any need to pipe and start another sed. Using -e you can have multiple text processing).
If you're using jq and trying to remove the quotes from the result, the other answers will work, but there's a better way. By using the -r option, you can output the result with no quotes.
$ echo '{"foo": "bar"}' | jq '.foo'
"bar"
$ echo '{"foo": "bar"}' | jq -r '.foo'
bar
There is a straightforward way using xargs:
> echo '"quoted"' | xargs
quoted
xargs uses echo as the default command if no command is provided and strips quotes from the input, see e.g. here. Note, however, that this will work only if the string does not contain additional quotes. In that case it will either fail (uneven number of quotes) or remove all of them.
If you came here for aws cli --query, try this. --output text
You can do it with only one call to sed:
$ echo "\"html\\test\\\"" | sed 's/^"\(.*\)"$/\1/'
html\test\
The shortest way around - try:
echo $opt | sed "s/\"//g"
It actually removes all "s (double quotes) from opt (are there really going to be any more double quotes other than in the beginning and the end though? So it's actually the same thing, and much more brief ;-))
The easiest solution in Bash:
$ s='"abc"'
$ echo $s
"abc"
$ echo "${s:1:-1}"
abc
This is called substring expansion (see Gnu Bash Manual and search for ${parameter:offset:length}). In this example it takes the substring from s starting at position 1 and ending at the second last position. This is due to the fact that if length is a negative value it is interpreted as a backwards running offset from the end of parameter.
Update
A simple and elegant answer from Stripping single and double quotes in a string using bash / standard Linux commands only:
BAR=$(eval echo $BAR) strips quotes from BAR.
=============================================================
Based on hueybois's answer, I came up with this function after much trial and error:
function stripStartAndEndQuotes {
cmd="temp=\${$1%\\\"}"
eval echo $cmd
temp="${temp#\"}"
eval echo "$1=$temp"
}
If you don't want anything printed out, you can pipe the evals to /dev/null 2>&1.
Usage:
$ BAR="FOO BAR"
$ echo BAR
"FOO BAR"
$ stripStartAndEndQuotes "BAR"
$ echo BAR
FOO BAR
This is the most discrete way without using sed:
x='"fish"'
printf " quotes: %s\nno quotes: %s\n" "$x" "${x//\"/}"
Or
echo $x
echo ${x//\"/}
Output:
quotes: "fish"
no quotes: fish
I got this from a source.
Linux=`cat /etc/os-release | grep "ID" | head -1 | awk -F= '{ print $2 }'`
echo $Linux
Output:
"amzn"
Simplest ways to remove double quotes from variables are
Linux=`echo "$Linux" | tr -d '"'`
Linux=$(eval echo $Linux)
Linux=`echo ${Linux//\"/}`
Linux=`echo $Linux | xargs`
All provides the Output without double quotes:
echo $Linux
amzn
I know this is a very old question, but here is another sed variation, which may be useful to someone. Unlike some of the others, it only replaces double quotes at the start or end...
echo "$opt" | sed -r 's/^"|"$//g'
If you need to match single or double quotes, and only strings that are properly quoted. You can use this slightly more complex regex...
echo $opt | sed -E "s|^(['\"])(.*)\1$|\2|g"
This uses backrefences to ensure the quote at the end is the same as at the start.
In Bash, you could use the following one-liner:
[[ "${var}" == \"*\" || "${var}" == \'*\' ]] && var="${var:1:-1}"
This will remove surrounding quotes (both single and double) from the string stored in var while keeping quote characters inside the string intact. Also, this won't do anything if there's only a single leading quote or only a single trailing quote or if there are mixed quote characters at start/end.
Wrapped in a function:
#!/usr/bin/env bash
# Strip surrounding quotes from string [$1: variable name]
function strip_quotes() {
local -n var="$1"
[[ "${var}" == \"*\" || "${var}" == \'*\' ]] && var="${var:1:-1}"
}
str="'hello world'"
echo "Before: ${str}"
strip_quotes str
echo "After: ${str}"
My version
strip_quotes() {
while [[ $# -gt 0 ]]; do
local value=${!1}
local len=${#value}
[[ ${value:0:1} == \" && ${value:$len-1:1} == \" ]] && declare -g $1="${value:1:$len-2}"
shift
done
}
The function accepts variable name(s) and strips quotes in place. It only strips a matching pair of leading and trailing quotes. It doesn't check if the trailing quote is escaped (preceded by \ which is not itself escaped).
In my experience, general-purpose string utility functions like this (I have a library of them) are most efficient when manipulating the strings directly, not using any pattern matching and especially not creating any sub-shells, or calling any external tools such as sed, awk or grep.
var1="\"test \\ \" end \""
var2=test
var3=\"test
var4=test\"
echo before:
for i in var{1,2,3,4}; do
echo $i="${!i}"
done
strip_quotes var{1,2,3,4}
echo
echo after:
for i in var{1,2,3,4}; do
echo $i="${!i}"
done
I use this regular expression, which avoids removing quotes from strings that are not properly quoted, here the different outputs are shown depending on the inputs, only one with begin-end quote was affected:
echo '"only first' | sed 's/^"\(.*\)"$/\1/'
Output: >"only first<
echo 'only last"' | sed 's/^"\(.*\)"$/\1/'
Output: >"only last"<
echo '"both"' | sed 's/^"\(.*\)"$/\1/'
Output: >both<
echo '"space after" ' | sed 's/^"\(.*\)"$/\1/'
Output: >"space after" <
echo ' "space before"' | sed 's/^"\(.*\)"$/\1/'
Output: > "space before"<
STR='"0.0.0"' ## OR STR="\"0.0.0\""
echo "${STR//\"/}"
## Output: 0.0.0
There is another way to do it. Like:
echo ${opt:1:-1}
If you try to remove quotes because the Makefile keeps them, try this:
$(subst $\",,$(YOUR_VARIABLE))
Based on another answer: https://stackoverflow.com/a/10430975/10452175
Below is the snippet of a shell script from a larger script. It removes the quotes from the string that is held by a variable. I am doing it using sed, but is it efficient? If not, then what is the efficient way?
#!/bin/sh
opt="\"html\\test\\\""
temp=`echo $opt | sed 's/.\(.*\)/\1/' | sed 's/\(.*\)./\1/'`
echo $temp
Use tr to delete ":
echo "$opt" | tr -d '"'
NOTE: This does not fully answer the question, removes all double quotes, not just leading and trailing. See other answers below.
There's a simpler and more efficient way, using the native shell prefix/suffix removal feature:
temp="${opt%\"}"
temp="${temp#\"}"
echo "$temp"
${opt%\"} will remove the suffix " (escaped with a backslash to prevent shell interpretation).
${temp#\"} will remove the prefix " (escaped with a backslash to prevent shell interpretation).
Another advantage is that it will remove surrounding quotes only if there are surrounding quotes.
BTW, your solution always removes the first and last character, whatever they may be (of course, I'm sure you know your data, but it's always better to be sure of what you're removing).
Using sed:
echo "$opt" | sed -e 's/^"//' -e 's/"$//'
(Improved version, as indicated by jfgagne, getting rid of echo)
sed -e 's/^"//' -e 's/"$//' <<<"$opt"
So it replaces a leading " with nothing, and a trailing " with nothing too. In the same invocation (there isn't any need to pipe and start another sed. Using -e you can have multiple text processing).
If you're using jq and trying to remove the quotes from the result, the other answers will work, but there's a better way. By using the -r option, you can output the result with no quotes.
$ echo '{"foo": "bar"}' | jq '.foo'
"bar"
$ echo '{"foo": "bar"}' | jq -r '.foo'
bar
There is a straightforward way using xargs:
> echo '"quoted"' | xargs
quoted
xargs uses echo as the default command if no command is provided and strips quotes from the input, see e.g. here. Note, however, that this will work only if the string does not contain additional quotes. In that case it will either fail (uneven number of quotes) or remove all of them.
If you came here for aws cli --query, try this. --output text
You can do it with only one call to sed:
$ echo "\"html\\test\\\"" | sed 's/^"\(.*\)"$/\1/'
html\test\
The shortest way around - try:
echo $opt | sed "s/\"//g"
It actually removes all "s (double quotes) from opt (are there really going to be any more double quotes other than in the beginning and the end though? So it's actually the same thing, and much more brief ;-))
The easiest solution in Bash:
$ s='"abc"'
$ echo $s
"abc"
$ echo "${s:1:-1}"
abc
This is called substring expansion (see Gnu Bash Manual and search for ${parameter:offset:length}). In this example it takes the substring from s starting at position 1 and ending at the second last position. This is due to the fact that if length is a negative value it is interpreted as a backwards running offset from the end of parameter.
Update
A simple and elegant answer from Stripping single and double quotes in a string using bash / standard Linux commands only:
BAR=$(eval echo $BAR) strips quotes from BAR.
=============================================================
Based on hueybois's answer, I came up with this function after much trial and error:
function stripStartAndEndQuotes {
cmd="temp=\${$1%\\\"}"
eval echo $cmd
temp="${temp#\"}"
eval echo "$1=$temp"
}
If you don't want anything printed out, you can pipe the evals to /dev/null 2>&1.
Usage:
$ BAR="FOO BAR"
$ echo BAR
"FOO BAR"
$ stripStartAndEndQuotes "BAR"
$ echo BAR
FOO BAR
This is the most discrete way without using sed:
x='"fish"'
printf " quotes: %s\nno quotes: %s\n" "$x" "${x//\"/}"
Or
echo $x
echo ${x//\"/}
Output:
quotes: "fish"
no quotes: fish
I got this from a source.
Linux=`cat /etc/os-release | grep "ID" | head -1 | awk -F= '{ print $2 }'`
echo $Linux
Output:
"amzn"
Simplest ways to remove double quotes from variables are
Linux=`echo "$Linux" | tr -d '"'`
Linux=$(eval echo $Linux)
Linux=`echo ${Linux//\"/}`
Linux=`echo $Linux | xargs`
All provides the Output without double quotes:
echo $Linux
amzn
I know this is a very old question, but here is another sed variation, which may be useful to someone. Unlike some of the others, it only replaces double quotes at the start or end...
echo "$opt" | sed -r 's/^"|"$//g'
If you need to match single or double quotes, and only strings that are properly quoted. You can use this slightly more complex regex...
echo $opt | sed -E "s|^(['\"])(.*)\1$|\2|g"
This uses backrefences to ensure the quote at the end is the same as at the start.
In Bash, you could use the following one-liner:
[[ "${var}" == \"*\" || "${var}" == \'*\' ]] && var="${var:1:-1}"
This will remove surrounding quotes (both single and double) from the string stored in var while keeping quote characters inside the string intact. Also, this won't do anything if there's only a single leading quote or only a single trailing quote or if there are mixed quote characters at start/end.
Wrapped in a function:
#!/usr/bin/env bash
# Strip surrounding quotes from string [$1: variable name]
function strip_quotes() {
local -n var="$1"
[[ "${var}" == \"*\" || "${var}" == \'*\' ]] && var="${var:1:-1}"
}
str="'hello world'"
echo "Before: ${str}"
strip_quotes str
echo "After: ${str}"
My version
strip_quotes() {
while [[ $# -gt 0 ]]; do
local value=${!1}
local len=${#value}
[[ ${value:0:1} == \" && ${value:$len-1:1} == \" ]] && declare -g $1="${value:1:$len-2}"
shift
done
}
The function accepts variable name(s) and strips quotes in place. It only strips a matching pair of leading and trailing quotes. It doesn't check if the trailing quote is escaped (preceded by \ which is not itself escaped).
In my experience, general-purpose string utility functions like this (I have a library of them) are most efficient when manipulating the strings directly, not using any pattern matching and especially not creating any sub-shells, or calling any external tools such as sed, awk or grep.
var1="\"test \\ \" end \""
var2=test
var3=\"test
var4=test\"
echo before:
for i in var{1,2,3,4}; do
echo $i="${!i}"
done
strip_quotes var{1,2,3,4}
echo
echo after:
for i in var{1,2,3,4}; do
echo $i="${!i}"
done
I use this regular expression, which avoids removing quotes from strings that are not properly quoted, here the different outputs are shown depending on the inputs, only one with begin-end quote was affected:
echo '"only first' | sed 's/^"\(.*\)"$/\1/'
Output: >"only first<
echo 'only last"' | sed 's/^"\(.*\)"$/\1/'
Output: >"only last"<
echo '"both"' | sed 's/^"\(.*\)"$/\1/'
Output: >both<
echo '"space after" ' | sed 's/^"\(.*\)"$/\1/'
Output: >"space after" <
echo ' "space before"' | sed 's/^"\(.*\)"$/\1/'
Output: > "space before"<
STR='"0.0.0"' ## OR STR="\"0.0.0\""
echo "${STR//\"/}"
## Output: 0.0.0
There is another way to do it. Like:
echo ${opt:1:-1}
If you try to remove quotes because the Makefile keeps them, try this:
$(subst $\",,$(YOUR_VARIABLE))
Based on another answer: https://stackoverflow.com/a/10430975/10452175
my question seems to be general, but i can't find any answers.
In sed command, how can you replace the substitution pattern by a value returned by a simple bash function.
For instance, I created the following function :
function parseDates(){
#Some process here with $1 (the pattern found)
return "dateParsed;
}
and the folowing sed command :
myCatFile=`sed -e "s/[0-3][0-9]\/[0-1][0-9]\/[0-9][0-9]/& parseDates &\}/p" myfile`
I found that the caracter '&' represents the current pattern found, i'd like it to be passed to my bash function and the whole pattern to be substituted by the pattern found +dateParsed.
Does anybody have an idea ?
Thanks
you can use the "e" option in sed command like this:
cat t.sh
myecho() {
echo ">>hello,$1<<"
}
export -f myecho
sed -e "s/.*/myecho &/e" <<END
ni
END
you can see the result without "e":
cat t.sh
myecho() {
echo ">>hello,$1<<"
}
export -f myecho
sed -e "s/.*/myecho &/" <<END
ni
END
Agree with Glenn Jackman.
If you want to use bash function in sed, something like this :
sed -rn 's/^([[:digit:].]+)/`date -d #&`/p' file |
while read -r line; do
eval echo "$line"
done
My file here begins with a unix timestamp (e.g. 1362407133.936).
Bash function inside sed (maybe for other purposes):
multi_stdin(){ #Makes function accepet variable or stdin (via pipe)
[[ -n "$1" ]] && echo "$*" || cat -
}
sans_accent(){
multi_stdin "$#" | sed '
y/àáâãäåèéêëìíîïòóôõöùúûü/aaaaaaeeeeiiiiooooouuuu/
y/ÀÁÂÃÄÅÈÉÊËÌÍÎÏÒÓÔÕÖÙÚÛÜ/AAAAAAEEEEIIIIOOOOOUUUU/
y/çÇñÑߢÐð£Øø§µÝý¥¹²³ªº/cCnNBcDdLOoSuYyY123ao/
'
}
eval $(echo "Rogério Madureira" | sed -n 's#.*#echo & | sans_accent#p')
or
eval $(echo "Rogério Madureira" | sed -n 's#.*#sans_accent &#p')
Rogerio
And if you need to keep the output into a variable:
VAR=$( eval $(echo "Rogério Madureira" | sed -n 's#.*#echo & | desacentua#p') )
echo "$VAR"
do it step by step. (also you could use an alternate delimiter , such as "|" instead of "/"
function parseDates(){
#Some process here with $1 (the pattern found)
return "dateParsed;
}
value=$(parseDates)
sed -n "s|[0-3][0-9]/[0-1][0-9]/[0-9][0-9]|& $value &|p" myfile
Note the use of double quotes instead of single quotes, so that $value can be interpolated
I'd like to know if there's a way to do this too. However, for this particular problem you don't need it. If you surround the different components of the date with ()s, you can back reference them with \1 \2 etc and reformat however you want.
For instance, let's reverse 03/04/1973:
echo 03/04/1973 | sed -e 's/\([0-9][0-9]\)\/\([0-9][0-9]\)\/\([0-9][0-9][0-9][0-9]\)/\3\/\2\/\1/g'
sed -e 's#[0-3][0-9]/[0-1][0-9]/[0-9][0-9]#& $(parseDates &)#' myfile |
while read -r line; do
eval echo "$line"
done
You can glue together a sed-command by ending a single-quoted section, and reopening it again.
sed -n 's|[0-3][0-9]/[0-1][0-9]/[0-9][0-9]|& '$(parseDates)' &|p' datefile
However, in contrast to other examples, a function in bash can't return strings, only put them out:
function parseDates(){
# Some process here with $1 (the pattern found)
echo dateParsed
}
I am trying to pass in a string containing a newline to a PHP script via BASH.
#!/bin/bash
REPOS="$1"
REV="$2"
message=$(svnlook log $REPOS -r $REV)
changed=$(svnlook changed $REPOS -r $REV)
/usr/bin/php -q /home/chad/www/mantis.localhost/scripts/checkin.php <<< "${message}\n${changed}"
When I do this, I see the literal "\n" rather than the escaped newline:
blah blah issue 0000002.\nU app/controllers/application_controller.rb
Any ideas how to translate '\n' to a literal newline?
By the way: what does <<< do in bash? I know < passes in a file...
try
echo -e "${message}\n${changed}" | /usr/bin/php -q /home/chad/www/mantis.localhost/scripts/checkin.php
where -e enables interpretation of backslash escapes (according to man echo)
Note that this will also interpret backslash escapes which you potentially have in ${message} and in ${changed}.
From the bash manual:
Here Strings
A variant of here documents, the format is:
<<<word
The word is expanded and supplied to the command on its standard input.
So I'd say
the_cmd <<< word
is equivalent to
echo word | the_cmd
newline=$'\n'
... <<< "${message}${newline}${changed}"
The <<< is called a "here string". It's a one line version of the "here doc" that doesn't require a delimiter such as "EOF". This is a here document version:
... <<EOF
${message}${newline}${changed}
EOF
in order to avoid interpretation of potential escape sequences in ${message} and ${changed}, try concatenating the strings in a subshell (a newline is appended after each echo unless you specify the -n option):
( echo "${message}" ; echo "${changed}" ) | /usr/bin/php -q /home/chad/www/mantis.localhost/scripts/checkin.php
The parentheses execute the commands in a subshell (if no parentheses were given, only the output of the second echo would be piped into your php program).
It is better to use here-document syntax:
cat <<EOF
copy $VAR1 $VAR2
del $VAR1
EOF
You can use magical Bash $'\n' with here-word:
cat <<< "copy $VAR1 $VAR2"$'\n'"del $VAR1"
or pipe with echo:
{ echo copy $VAR1 $VAR2; echo del $VAR1; } | cat
or with printf:
printf "copy %s %s\ndel %s" "$VAR1" "$VAR2" "$VAR1" | cat
Test it:
env VAR1=1 VAR2=2 printf "copy %s %s\ndel %s" "$VAR1" "$VAR2" "$VAR1" | cat
I have a fairly simple bash shell scripting problem.
I want to sed a piece of text and then assign the result of the sed to a variable.
#!/bin/bash
MOD_DATE=echo $(date) | sed 's/\ /_/g'
echo $MOD_DATE // should show date with spaces replaced with underscores.
I have tried the above and it doesn't work. Can anyone point out what I'm doing wrong?
To collect the output in stdout into a variable, use a command substitution:
MOD_DATE=`echo $(date) | sed 's/\ /_/g'`
# ^ ^
or
MOD_DATE=$(echo $(date) | sed 's/\ /_/g')
# ^^ ^
Maybe this can help:
mod_date = "$(date +"%d_%m_%Y")"
echo "$mod_date"