While writing to a file from terminal using 'echo', I could write everything except both quotes together. When trying to write/append lines of python code, to a file, that contains both single and double quotation marks like
echo "url = "http://mydummyurl.xyz"
names = ['name1', 'name2']" > somefile.txt
or
echo 'url = "http://mydummyurl.xyz"
names = ['name1', 'name2']' > somefile.txt
What is written in somefile.txt is either
url = http://mydummyurl.xyz
names = ['name1', 'name2']
or
url = "http://mydummyurl.xyz"
names = [name1, name2]
depending on what I am using to enclose the string after echo. Observe that in strings written in files are missing either single quote or double code used as python syntax. How can I write to file keeping all the necessary syntax intact including both single and double quotations?
you need escape the quotes:
kent$ echo "single:''';double:\"\"\"" > foo
kent$ cat foo
single:''';double:"""
Related
How can I escape double quotes inside a double string in Bash?
For example, in my shell script
#!/bin/bash
dbload="load data local infile \"'gfpoint.csv'\" into table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY \"'\n'\" IGNORE 1 LINES"
I can't get the ENCLOSED BY '\"' with double quote to escape correctly. I can't use single quotes for my variable, because I want to use variable $dbtable.
Use a backslash:
echo "\"" # Prints one " character.
A simple example of escaping quotes in the shell:
$ echo 'abc'\''abc'
abc'abc
$ echo "abc"\""abc"
abc"abc
It's done by finishing an already-opened one ('), placing the escaped one (\'), and then opening another one (').
Alternatively:
$ echo 'abc'"'"'abc'
abc'abc
$ echo "abc"'"'"abc"
abc"abc
It's done by finishing already opened one ('), placing a quote in another quote ("'"), and then opening another one (').
More examples: Escaping single-quotes within single-quoted strings
Keep in mind that you can avoid escaping by using ASCII codes of the characters you need to echo.
Example:
echo -e "This is \x22\x27\x22\x27\x22text\x22\x27\x22\x27\x22"
This is "'"'"text"'"'"
\x22 is the ASCII code (in hex) for double quotes and \x27 for single quotes. Similarly you can echo any character.
I suppose if we try to echo the above string with backslashes, we will need a messy two rows backslashed echo... :)
For variable assignment this is the equivalent:
a=$'This is \x22text\x22'
echo "$a"
# Output:
This is "text"
If the variable is already set by another program, you can still apply double/single quotes with sed or similar tools.
Example:
b="Just another text here"
echo "$b"
Just another text here
sed 's/text/"'\0'"/' <<<"$b" #\0 is a special sed operator
Just another "0" here #this is not what i wanted to be
sed 's/text/\x22\x27\0\x27\x22/' <<<"$b"
Just another "'text'" here #now we are talking. You would normally need a dozen of backslashes to achieve the same result in the normal way.
Bash allows you to place strings adjacently, and they'll just end up being glued together.
So this:
echo "Hello"', world!'
produces
Hello, world!
The trick is to alternate between single and double-quoted strings as required. Unfortunately, it quickly gets very messy. For example:
echo "I like to use" '"double quotes"' "sometimes"
produces
I like to use "double quotes" sometimes
In your example, I would do it something like this:
dbtable=example
dbload='load data local infile "'"'gfpoint.csv'"'" into '"table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '"'"'"' LINES "'TERMINATED BY "'"'\n'"'" IGNORE 1 LINES'
echo $dbload
which produces the following output:
load data local infile "'gfpoint.csv'" into table example FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY "'\n'" IGNORE 1 LINES
It's difficult to see what's going on here, but I can annotate it using Unicode quotes. The following won't work in Bash – it's just for illustration:
dbload=‘load data local infile "’“'gfpoint.csv'”‘" into ’“table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '”‘"’“' LINES ”‘TERMINATED BY "’“'\n'”‘" IGNORE 1 LINES’
The quotes like “ ‘ ’ ” in the above will be interpreted by bash. The quotes like " ' will end up in the resulting variable.
If I give the same treatment to the earlier example, it looks like this:
echo “I like to use” ‘"double quotes"’ “sometimes”
Store the double quote character in a variable:
dqt='"'
echo "Double quotes ${dqt}X${dqt} inside a double quoted string"
Output:
Double quotes "X" inside a double quoted string
Check out printf...
#!/bin/bash
mystr="say \"hi\""
Without using printf
echo -e $mystr
Output: say "hi"
Using printf
echo -e $(printf '%q' $mystr)
Output: say \"hi\"
Make use of $"string".
In this example, it would be,
dbload=$"load data local infile \"'gfpoint.csv'\" into table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY \"'\n'\" IGNORE 1 LINES"
Note (from the man page):
A double-quoted string preceded by a dollar sign ($"string") will cause the string to be translated according to the current locale. If the current locale is C or POSIX, the dollar sign is ignored. If the string is translated and replaced, the replacement is double-quoted.
For use with variables that might contain spaces in you Bash script, use triple quotes inside the main quote, e.g.:
[ "$(date -r """$touchfile""" +%Y%m%d)" -eq "$(date +%Y%m%d)" ]
Add "\" before double quote to escape it, instead of \
#! /bin/csh -f
set dbtable = balabala
set dbload = "load data local infile "\""'gfpoint.csv'"\"" into table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '"\""' LINES TERMINATED BY "\""'\n'"\"" IGNORE 1 LINES"
echo $dbload
# load data local infile "'gfpoint.csv'" into table balabala FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY "''" IGNORE 1 LINES
I'm writing a script for looping over a set of files in a directory searching for a string (stringA) in one file (srcFile), copying the line that follows it (stringToCopy), and pasting it on the line after another search string (stringB) in another file (outputFile). The copy/paste script that I have so far is as follows
stringA="This is string A"
stringB="This is string B"
srcFile=srcFile.txt
outpuFile=outputFile.txt
replacement="/$stringA/{getline; print}"
stringToCopy="$(awk "$replacement" $srcFile)"
sed -i "/$stringB/!b;n;c${stringToCopy}" $outputFile
The script works great, except when stringToCopy ends up containing curly braces. Example is
srcFile.txt:
This is string A
text to copy: {0}
outputFile.txt:
This is string B
line to be replaced
Once the script is done, I would expect outputFile.txt to be
This is string B
text to copy: {0}
But sed chokes with
sed: -e expression #1, char 106: unknown command: `m'
I've tried hardcoding the problematic string and trying different variations of escaping the curlies and quoting the string, but haven't found a winning combination and I'm at a loss for how to make it work.
EDIT
I had a derp moment and forgot that my stringA also has curly braces, that happened to cause my awk command to math multiple lines. This caused my stringToCopy to have newlines in it which is my real issue, not the curly braces. So the real question is, how to make awk treat curly braces as literal characters so that
srcFile.txt
This is string A: {0}
text to copy: {0}
This is string A:
Other junk
And stringA="This is string A: {0}"
Doesn't set stringToCopy to
text to copy: {0}
Other junk
A bit of a kludge in that we're going to add some extra coding specifically for braces ...
Current situation:
$ awk '/This is string A: {0}/{getline; print}' srcFile.txt
text to copy: {0} # this is the line we want
Other junk # we do not want this line
We can eliminate the second line by escaping the braces in the search pattern, eg:
$ awk '/This is string A: \{0\}/{getline; print}' srcFile.txt
text to copy: {0}
So, how to escape the braces? We can use some explicit parameter expansions to replace the braces with escaped braces in the $stringA variable, keeping in mind that we also need to escape the braces in the parameter expansion phase, too:
$ stringA="This is string A: {0}"
$ stringA="${stringA//\{/\\{}" # replace '{' with '\{'
$ stringA="${stringA//\}/\\}}" # replace '}' with '\}'
$ echo "${stringA}"
This is string A: \{0\}
We can then proceed with the rest of the code as is:
$ replacement="/$stringA/{getline; print}"
$ echo "${replacement}"
/This is string A: \{0\}/{getline; print}
$ stringToCopy="$(awk "$replacement" $srcFile)"
$ echo "${stringToCopy}"
text to copy: {0}
As for the final sed step I had to remove the ! to get it to work correctly:
$ sed -i "/$stringB/b;n;c${stringToCopy}" $outputFile
$ cat "${outputFile}"
This is string B
text to copy: {0}
NOTES:
if you preface your coding with set -xv you can see how variables are being interpreted at each step; use set +xv to turn off
obviously you'll probably run into issues if you do in fact have more than 1 matching row in $srcFile
if you find other characters that need to be escaped then you'll need to add additional parameter expansions for said characters
I want to use backticks in ruby for a programm call.
The parameter is a String variable containing one or more backticks, i.e.
"&E?##A`?". The following command yields a new label as its return value:
echo "&E?##A\`?" | nauty-labelg 2>/dev/null
From a ruby program I can call it as follows and get the correct result:
new_label = `echo "&E?##A\\\`?" | nauty-labelg 2>/dev/null`
I want to achieve the same using a variable for the label.
So I have to insert three slashes into my variable label = "&E?##A`?" in order to escape the backtick. The following seems to work, though it is not very elegant:
escaped_label = label.gsub(/`/, '\\\`').gsub(/`/, '\\\`').gsub(/`/, '\\\`')
But the new variable cannot be used in the program call:
new_label = `echo "#{escaped_label}" | nauty-labelg 2>/dev/null`
In this case I do not get an answer from nauty-labelg.
So I have to insert three slashes into my variable label = "&E?##A`?" in order to escape the backtick.
No, you only need to add one backslash for the output. To escape the ` special bash character. The other other two are only for representation proposes, otherwise it isn't valid Ruby code.
new_label = `echo "&E?##A\\\`?" | nauty-labelg 2>/dev/null`
The first backslash will escape the second one (outputting one single backslash). The third backslash escapes the ` character (outputting one single `).
You should only add backslashes before characters that have a special meaning within double quoted bash context. These special characters are: $, `, \ and \n. Those can be escaped with the following code:
def escape_bash_string(string)
string.gsub(/([$`"\\\n])/, '\\\\\1')
end
For label = "&E?##A`?" only the ` should be escaped.
escaped_string = escape_bash_string("&E?##A\`?")
puts escaped_string
# &E?##A\`?
I have this file:
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/I_Cuestionario_general_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/IV_Ingresos_y_recursos_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VI_ambito_escolar_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VII_ambito_laboral_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VIII_ambito_comunitario_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/IX_Atencion_Obstetrica_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/X_ambito_familiar_estimaciones_endireh2016.xlsx
And this bash script:
while read p; do
echo "\"$p\""
done < file.txt
I would expect the same file but with double quotes around each line, but this is what bash is outputting:
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/I_Cuestionario_general_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/IV_Ingresos_y_recursos_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VI_ambito_escolar_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VII_ambito_laboral_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VIII_ambito_comunitario_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/IX_Atencion_Obstetrica_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/X_ambito_familiar_estimaciones_endireh2016.xlsx
Anyone know why bash is behaving this way? And how to output both " double quotes? (beginning and end)
I'm near certain that the line endings on your input file are CR/LF rather than just LF. This would output:
";
the web address;
a CR returning the cursor to the beginning of the line;
"; and, finally,
moving to a new line.
Capture the output to a file and pass it through a dump utility like od -xcb, that should show you the raw bytes being output.
As a test, creating a file consisting of the two lines 123<CR> and 456, I see:
pax> while read p; do echo "\"$p\""; done <testfile
"123
"456"
which seems to indicate the problem is as described.
If you're having trouble escaping the leading and trailing double quotes, you can just use single quotes around your echo statement. Any double quotes inside of single quotes have no significance in terms of defining a string literal, and vice versa:
while read p; do
echo '"$p"'
done < file.txt
How can I escape double quotes inside a double string in Bash?
For example, in my shell script
#!/bin/bash
dbload="load data local infile \"'gfpoint.csv'\" into table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY \"'\n'\" IGNORE 1 LINES"
I can't get the ENCLOSED BY '\"' with double quote to escape correctly. I can't use single quotes for my variable, because I want to use variable $dbtable.
Use a backslash:
echo "\"" # Prints one " character.
A simple example of escaping quotes in the shell:
$ echo 'abc'\''abc'
abc'abc
$ echo "abc"\""abc"
abc"abc
It's done by finishing an already-opened one ('), placing the escaped one (\'), and then opening another one (').
Alternatively:
$ echo 'abc'"'"'abc'
abc'abc
$ echo "abc"'"'"abc"
abc"abc
It's done by finishing already opened one ('), placing a quote in another quote ("'"), and then opening another one (').
More examples: Escaping single-quotes within single-quoted strings
Keep in mind that you can avoid escaping by using ASCII codes of the characters you need to echo.
Example:
echo -e "This is \x22\x27\x22\x27\x22text\x22\x27\x22\x27\x22"
This is "'"'"text"'"'"
\x22 is the ASCII code (in hex) for double quotes and \x27 for single quotes. Similarly you can echo any character.
I suppose if we try to echo the above string with backslashes, we will need a messy two rows backslashed echo... :)
For variable assignment this is the equivalent:
a=$'This is \x22text\x22'
echo "$a"
# Output:
This is "text"
If the variable is already set by another program, you can still apply double/single quotes with sed or similar tools.
Example:
b="Just another text here"
echo "$b"
Just another text here
sed 's/text/"'\0'"/' <<<"$b" #\0 is a special sed operator
Just another "0" here #this is not what i wanted to be
sed 's/text/\x22\x27\0\x27\x22/' <<<"$b"
Just another "'text'" here #now we are talking. You would normally need a dozen of backslashes to achieve the same result in the normal way.
Bash allows you to place strings adjacently, and they'll just end up being glued together.
So this:
echo "Hello"', world!'
produces
Hello, world!
The trick is to alternate between single and double-quoted strings as required. Unfortunately, it quickly gets very messy. For example:
echo "I like to use" '"double quotes"' "sometimes"
produces
I like to use "double quotes" sometimes
In your example, I would do it something like this:
dbtable=example
dbload='load data local infile "'"'gfpoint.csv'"'" into '"table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '"'"'"' LINES "'TERMINATED BY "'"'\n'"'" IGNORE 1 LINES'
echo $dbload
which produces the following output:
load data local infile "'gfpoint.csv'" into table example FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY "'\n'" IGNORE 1 LINES
It's difficult to see what's going on here, but I can annotate it using Unicode quotes. The following won't work in Bash – it's just for illustration:
dbload=‘load data local infile "’“'gfpoint.csv'”‘" into ’“table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '”‘"’“' LINES ”‘TERMINATED BY "’“'\n'”‘" IGNORE 1 LINES’
The quotes like “ ‘ ’ ” in the above will be interpreted by bash. The quotes like " ' will end up in the resulting variable.
If I give the same treatment to the earlier example, it looks like this:
echo “I like to use” ‘"double quotes"’ “sometimes”
Store the double quote character in a variable:
dqt='"'
echo "Double quotes ${dqt}X${dqt} inside a double quoted string"
Output:
Double quotes "X" inside a double quoted string
Check out printf...
#!/bin/bash
mystr="say \"hi\""
Without using printf
echo -e $mystr
Output: say "hi"
Using printf
echo -e $(printf '%q' $mystr)
Output: say \"hi\"
Make use of $"string".
In this example, it would be,
dbload=$"load data local infile \"'gfpoint.csv'\" into table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY \"'\n'\" IGNORE 1 LINES"
Note (from the man page):
A double-quoted string preceded by a dollar sign ($"string") will cause the string to be translated according to the current locale. If the current locale is C or POSIX, the dollar sign is ignored. If the string is translated and replaced, the replacement is double-quoted.
For use with variables that might contain spaces in you Bash script, use triple quotes inside the main quote, e.g.:
[ "$(date -r """$touchfile""" +%Y%m%d)" -eq "$(date +%Y%m%d)" ]
Add "\" before double quote to escape it, instead of \
#! /bin/csh -f
set dbtable = balabala
set dbload = "load data local infile "\""'gfpoint.csv'"\"" into table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '"\""' LINES TERMINATED BY "\""'\n'"\"" IGNORE 1 LINES"
echo $dbload
# load data local infile "'gfpoint.csv'" into table balabala FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY "''" IGNORE 1 LINES