I want to use backticks in ruby for a programm call.
The parameter is a String variable containing one or more backticks, i.e.
"&E?##A`?". The following command yields a new label as its return value:
echo "&E?##A\`?" | nauty-labelg 2>/dev/null
From a ruby program I can call it as follows and get the correct result:
new_label = `echo "&E?##A\\\`?" | nauty-labelg 2>/dev/null`
I want to achieve the same using a variable for the label.
So I have to insert three slashes into my variable label = "&E?##A`?" in order to escape the backtick. The following seems to work, though it is not very elegant:
escaped_label = label.gsub(/`/, '\\\`').gsub(/`/, '\\\`').gsub(/`/, '\\\`')
But the new variable cannot be used in the program call:
new_label = `echo "#{escaped_label}" | nauty-labelg 2>/dev/null`
In this case I do not get an answer from nauty-labelg.
So I have to insert three slashes into my variable label = "&E?##A`?" in order to escape the backtick.
No, you only need to add one backslash for the output. To escape the ` special bash character. The other other two are only for representation proposes, otherwise it isn't valid Ruby code.
new_label = `echo "&E?##A\\\`?" | nauty-labelg 2>/dev/null`
The first backslash will escape the second one (outputting one single backslash). The third backslash escapes the ` character (outputting one single `).
You should only add backslashes before characters that have a special meaning within double quoted bash context. These special characters are: $, `, \ and \n. Those can be escaped with the following code:
def escape_bash_string(string)
string.gsub(/([$`"\\\n])/, '\\\\\1')
end
For label = "&E?##A`?" only the ` should be escaped.
escaped_string = escape_bash_string("&E?##A\`?")
puts escaped_string
# &E?##A\`?
Let's say I have a file test.txt with contents:
+-foo.bar:2.4
| bar.foo:1.1:test
\| hello.goobye:3.3.3
\|+- baz.yeah:4
I want to use the tr command to delete all instances of the following set of characters:
{' ', '+', '-', '|', '\'}
Done some pretty extensive research on this but found no clear/concise answers.
This is the command that works:
input:
cat test.txt | tr -d "[:blank:]|\\\+-"
output:
foo.bar:2.4
bar.foo:1.1:test
hello.goobye:3.3.3
baz.yeah:4
I experimented with many combinations of that set and I found out that the '-' was being treated as a range indicator (like... [a-z]) and therefore must be put at the end. But I have two main questions:
1) Why must the backslash be double escaped in order to be included in the set?
2) Why does putting the '|' at the end of the set string cause the tr program to delete everything in the file except for trailing new line characters?
Like this:
tr -d '\-|\\+[:blank:] ' < file
You have to escape the - because it is used for denoting ranges of characters like:
tr -d '1-5'
and must therefore being escaped if you mean a literal hyphen. You can also put it at the end. (learned that, thanks! :) )
Furthermore the \ must be escaped when you mean a literal \ because it has a special meaning needed for escape sequences.
The remaining characters must not being escaped.
Why must the \ being doubly escaped in your example?
It's because you are using a "" (double quoted) string to quote the char set. A double quoted string will be interpreted by the shell, a \\ in a double quoted string means a literal \. Try:
echo "\+"
echo "\\+"
echo "\\\+"
To avoid to doubly escape the \ you can just use single quotes as in my example above.
Why does putting the '|' at the end of the set string cause the tr program to delete everything in the file except for trailing new line characters?
Following CharlesDuffy's comment having the | at the end means also that you had the unescaped - not at the end, which means it was describing a range of characters where the actual range depends on the position you had it in the set.
another approach is to define the allowed chars
$ tr -cd '[:alnum:]:.\n' <file
foo.bar:2.4
bar.foo:1.1:test
hello.goobye:3.3.3
baz.yeah:4
or, perhaps delete all the prefix non-word chars
$ sed -E 's/\W+//' file
I have this file:
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/I_Cuestionario_general_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/IV_Ingresos_y_recursos_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VI_ambito_escolar_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VII_ambito_laboral_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VIII_ambito_comunitario_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/IX_Atencion_Obstetrica_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/X_ambito_familiar_estimaciones_endireh2016.xlsx
And this bash script:
while read p; do
echo "\"$p\""
done < file.txt
I would expect the same file but with double quotes around each line, but this is what bash is outputting:
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/I_Cuestionario_general_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/IV_Ingresos_y_recursos_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VI_ambito_escolar_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VII_ambito_laboral_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VIII_ambito_comunitario_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/IX_Atencion_Obstetrica_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/X_ambito_familiar_estimaciones_endireh2016.xlsx
Anyone know why bash is behaving this way? And how to output both " double quotes? (beginning and end)
I'm near certain that the line endings on your input file are CR/LF rather than just LF. This would output:
";
the web address;
a CR returning the cursor to the beginning of the line;
"; and, finally,
moving to a new line.
Capture the output to a file and pass it through a dump utility like od -xcb, that should show you the raw bytes being output.
As a test, creating a file consisting of the two lines 123<CR> and 456, I see:
pax> while read p; do echo "\"$p\""; done <testfile
"123
"456"
which seems to indicate the problem is as described.
If you're having trouble escaping the leading and trailing double quotes, you can just use single quotes around your echo statement. Any double quotes inside of single quotes have no significance in terms of defining a string literal, and vice versa:
while read p; do
echo '"$p"'
done < file.txt
Consider test file csf.conf:
CC_DENY = ""
Running the command:
sed -i -E 's/(CC_DENY *= *")[^"]+/\1AR,BE,CL,CN,CO,CS,ES,FR,GR,HK,IT,KO,PA,PE,PH,PL,RS,RU,SG,SK,TH,UA,VN,AE,AF,AL,AS,AZ,BA,BD,BF,BH,BJ,BN,CI,DJ,EG,EH,ER,ET,GM,GN,GW,IQ,IR,IS,JO,KG,KM,KW,KZ,LB,LY,MC,MK,ML,MR,MV,MY,NE,NG,OM,PK,PS,QA,SA,SD,SL,SN,SO,SY,TD,TJ,TM,TN,TR,UZ,XK,YE,YT/g' csf.conf
Does not replace the match inside the file. Output should look like this:
CC_DENY="AR,BE,CL,CN,CO,CS,ES,FR,GR,HK,IT,KO,PA,PE,PH,PL,RS,RU,SG,SK,TH,UA,VN,AE,AF,AL..."
Sed v4.2.2, same result on Debian 8, and Centos 7
This has nothing to do with long text, your regexp just doesn't match the content of your file. Change [^"]+ to [^"]* so it'll match even when there's nothing between the double quotes "". Look:
$ cat csf.conf
CC_DENY = ""
$ sed -E 's/(CC_DENY *= *")[^"]+/\1foo/' csf.conf
CC_DENY = ""
$ sed -E 's/(CC_DENY *= *")[^"]*/\1foo/' csf.conf
CC_DENY = "foo"
wrt the comment below from the OP that this sed command works:
$ cat file
LF_SPI = ""
$ sed -E 's/(LF_SPI *= *\")[^\"]+/\1blah/g' file
LF_SPI = ""
Clearly and predictably, no it does not. It simply can't because the regexp metacharacter + means 1 or more so [^\"]+ states there must be at least one non-" after the " and that just does not exist in the input file. There is no reason to escape the double quotes btw.
Suppose the current variable value in the file is empty. Then your regular expression doesn't match because [^"]+ means "any character, except double quote repeated one or more times".
You might fix it by replacing + quantifier with * (zero or more times). But suppose the value contains a double quote:
CC_DENY = "\""
Then the [^"]* will match everything until it gets to the double quote within the value.
Thus, I suggest the following command:
# Put the variable value here
value='AR,BE\\" ... YE,YT';
sed -i -r 's/^( *CC_DENY *= *").*"/\1'"$value"'"/' csf.conf
Also note, that the expression above uses an anchor for the beginning of the line. Otherwise, it will fail to match as expected, if such a CC_DENY = "... exists in the variable value in the configuration file: CC_DENY = "SOMETHING_CC_DENY = \"value\"".
Sed is certainly the wrong tool for this:
#!/usr/bin/awk -f
BEGIN {
FS = OFS = "\42"
}
$2 = "AR,BE,CL,CN,CO,CS,ES,FR,GR,HK,IT,KO,PA,PE,PH,PL,RS,RU,SG,SK,TH,UA,VN," \
"AE,AF,AL,AS,AZ,BA,BD,BF,BH,BJ,BN,CI,DJ,EG,EH,ER,ET,GM,GN,GW,IQ,IR,IS,JO,KG," \
"KM,KW,KZ,LB,LY,MC,MK,ML,MR,MV,MY,NE,NG,OM,PK,PS,QA,SA,SD,SL,SN,SO,SY,TD,TJ," \
"TM,TN,TR,UZ,XK,YE,YT"