How do I add an escape character "\" before each ", " in a file? (that's a comma followed by a space).
P.S. Those quotes are for formatting purposes and are not presented (nor should appear in the file).
see this example, if it is helpful for you:
kent$ echo "foo,bar,blah"|sed 's/,/\\,/g'
foo\,bar\,blah
if you want to replace the comma, only if there is a space followed, do this:
kent$ echo "foo, bar, blah,skipme"|sed 's/, /\\&/g'
foo\, bar\, blah,skipme
For each character
put wanted character to escape in a class [ ,] (space and ,in this case`)
echo 'Your, string or data' | sed 's/[, ]/\\&/g'
there is special rules for ], - and \ (see RegEx class documentation for this)
For specifing pattern
echo 'Your, string or data' | sed 's/, /\\&/g'
Related
How can I escape double quotes inside a double string in Bash?
For example, in my shell script
#!/bin/bash
dbload="load data local infile \"'gfpoint.csv'\" into table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY \"'\n'\" IGNORE 1 LINES"
I can't get the ENCLOSED BY '\"' with double quote to escape correctly. I can't use single quotes for my variable, because I want to use variable $dbtable.
Use a backslash:
echo "\"" # Prints one " character.
A simple example of escaping quotes in the shell:
$ echo 'abc'\''abc'
abc'abc
$ echo "abc"\""abc"
abc"abc
It's done by finishing an already-opened one ('), placing the escaped one (\'), and then opening another one (').
Alternatively:
$ echo 'abc'"'"'abc'
abc'abc
$ echo "abc"'"'"abc"
abc"abc
It's done by finishing already opened one ('), placing a quote in another quote ("'"), and then opening another one (').
More examples: Escaping single-quotes within single-quoted strings
Keep in mind that you can avoid escaping by using ASCII codes of the characters you need to echo.
Example:
echo -e "This is \x22\x27\x22\x27\x22text\x22\x27\x22\x27\x22"
This is "'"'"text"'"'"
\x22 is the ASCII code (in hex) for double quotes and \x27 for single quotes. Similarly you can echo any character.
I suppose if we try to echo the above string with backslashes, we will need a messy two rows backslashed echo... :)
For variable assignment this is the equivalent:
a=$'This is \x22text\x22'
echo "$a"
# Output:
This is "text"
If the variable is already set by another program, you can still apply double/single quotes with sed or similar tools.
Example:
b="Just another text here"
echo "$b"
Just another text here
sed 's/text/"'\0'"/' <<<"$b" #\0 is a special sed operator
Just another "0" here #this is not what i wanted to be
sed 's/text/\x22\x27\0\x27\x22/' <<<"$b"
Just another "'text'" here #now we are talking. You would normally need a dozen of backslashes to achieve the same result in the normal way.
Bash allows you to place strings adjacently, and they'll just end up being glued together.
So this:
echo "Hello"', world!'
produces
Hello, world!
The trick is to alternate between single and double-quoted strings as required. Unfortunately, it quickly gets very messy. For example:
echo "I like to use" '"double quotes"' "sometimes"
produces
I like to use "double quotes" sometimes
In your example, I would do it something like this:
dbtable=example
dbload='load data local infile "'"'gfpoint.csv'"'" into '"table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '"'"'"' LINES "'TERMINATED BY "'"'\n'"'" IGNORE 1 LINES'
echo $dbload
which produces the following output:
load data local infile "'gfpoint.csv'" into table example FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY "'\n'" IGNORE 1 LINES
It's difficult to see what's going on here, but I can annotate it using Unicode quotes. The following won't work in Bash – it's just for illustration:
dbload=‘load data local infile "’“'gfpoint.csv'”‘" into ’“table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '”‘"’“' LINES ”‘TERMINATED BY "’“'\n'”‘" IGNORE 1 LINES’
The quotes like “ ‘ ’ ” in the above will be interpreted by bash. The quotes like " ' will end up in the resulting variable.
If I give the same treatment to the earlier example, it looks like this:
echo “I like to use” ‘"double quotes"’ “sometimes”
Store the double quote character in a variable:
dqt='"'
echo "Double quotes ${dqt}X${dqt} inside a double quoted string"
Output:
Double quotes "X" inside a double quoted string
Check out printf...
#!/bin/bash
mystr="say \"hi\""
Without using printf
echo -e $mystr
Output: say "hi"
Using printf
echo -e $(printf '%q' $mystr)
Output: say \"hi\"
Make use of $"string".
In this example, it would be,
dbload=$"load data local infile \"'gfpoint.csv'\" into table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY \"'\n'\" IGNORE 1 LINES"
Note (from the man page):
A double-quoted string preceded by a dollar sign ($"string") will cause the string to be translated according to the current locale. If the current locale is C or POSIX, the dollar sign is ignored. If the string is translated and replaced, the replacement is double-quoted.
For use with variables that might contain spaces in you Bash script, use triple quotes inside the main quote, e.g.:
[ "$(date -r """$touchfile""" +%Y%m%d)" -eq "$(date +%Y%m%d)" ]
Add "\" before double quote to escape it, instead of \
#! /bin/csh -f
set dbtable = balabala
set dbload = "load data local infile "\""'gfpoint.csv'"\"" into table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '"\""' LINES TERMINATED BY "\""'\n'"\"" IGNORE 1 LINES"
echo $dbload
# load data local infile "'gfpoint.csv'" into table balabala FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY "''" IGNORE 1 LINES
Let's say I have a file test.txt with contents:
+-foo.bar:2.4
| bar.foo:1.1:test
\| hello.goobye:3.3.3
\|+- baz.yeah:4
I want to use the tr command to delete all instances of the following set of characters:
{' ', '+', '-', '|', '\'}
Done some pretty extensive research on this but found no clear/concise answers.
This is the command that works:
input:
cat test.txt | tr -d "[:blank:]|\\\+-"
output:
foo.bar:2.4
bar.foo:1.1:test
hello.goobye:3.3.3
baz.yeah:4
I experimented with many combinations of that set and I found out that the '-' was being treated as a range indicator (like... [a-z]) and therefore must be put at the end. But I have two main questions:
1) Why must the backslash be double escaped in order to be included in the set?
2) Why does putting the '|' at the end of the set string cause the tr program to delete everything in the file except for trailing new line characters?
Like this:
tr -d '\-|\\+[:blank:] ' < file
You have to escape the - because it is used for denoting ranges of characters like:
tr -d '1-5'
and must therefore being escaped if you mean a literal hyphen. You can also put it at the end. (learned that, thanks! :) )
Furthermore the \ must be escaped when you mean a literal \ because it has a special meaning needed for escape sequences.
The remaining characters must not being escaped.
Why must the \ being doubly escaped in your example?
It's because you are using a "" (double quoted) string to quote the char set. A double quoted string will be interpreted by the shell, a \\ in a double quoted string means a literal \. Try:
echo "\+"
echo "\\+"
echo "\\\+"
To avoid to doubly escape the \ you can just use single quotes as in my example above.
Why does putting the '|' at the end of the set string cause the tr program to delete everything in the file except for trailing new line characters?
Following CharlesDuffy's comment having the | at the end means also that you had the unescaped - not at the end, which means it was describing a range of characters where the actual range depends on the position you had it in the set.
another approach is to define the allowed chars
$ tr -cd '[:alnum:]:.\n' <file
foo.bar:2.4
bar.foo:1.1:test
hello.goobye:3.3.3
baz.yeah:4
or, perhaps delete all the prefix non-word chars
$ sed -E 's/\W+//' file
I have SNMP outputs like:
IP-MIB::ipNetToMediaPhysAddress.5122.192.19.3.25 = STRING: 34:8:4:56:f4:70
As you can see mac-address output is incorrect, and i fix it with sed:
echo IP-MIB::ipNetToMediaPhysAddress.5122.192.19.3.25 = STRING: 34:8:4:56:f4:70 |
sed -e 's/\b\(\w\)\b/0\1/g'
Output:
IP-MIB::ipNetToMediaPhysAddress.5122.192.19.03.25 = STRING: 34:08:04:56:f4:70
It fixes address but changes IP as well from 192.19.3.25 to 192.19.03.25. How can I avoid it and force to perform sed only after STRING: or only after last space in the string ?
The MAC address is colon-separated. You can use that to limit the substitutions. This will perform the substitutions that you are interested in but only if the word character is next to a colon:
sed -e 's/\b\w:/0&/g; s/:\(\w\)\b/:0\1/g'
For example:
$ echo IP-MIB::ipNetToMediaPhysAddress.5122.192.19.3.25 = STRING: 34:8:4:56:f4:70 | sed -e 's/\b\w:/0&/g; s/:\(\w\)\b/:0\1/g'
IP-MIB::ipNetToMediaPhysAddress.5122.192.19.3.25 = STRING: 34:08:04:56:f4:70
How it works
s/\b\w:/0&/g
This performs the substitution if the word character is preceded by a word break, \b, and followed by a colon. Since we just need to put a zero in front of the entire matched text, not just some section of it, we can omit the parens and just use & to copy the matched text.
s/:\(\w\)\b/:0\1/g
If there are any remaining substitutions that need to be done where the word character is preceded by a colon and followed by a word break, this does them.
Note: We are using GNU extensions that may not be portable.
Another way with sed if the MAC address is at end of line
echo IP-MIB::ipNetToMediaPhysAddress.5122.192.19.3.25 = STRING: 4:8:d:56:f4:7 |
sed -E '
s/$/:/
:A
s/([^[:xdigit:]])([[:xdigit:]]:)/\10\2/
tA
s/:$//'
I have strings which contain double quotes like this one:
"[{"clientid":"*", "identityzone":"*"}]"
I would like to use set or grep to delete the double quotes at the beginning and at the end of it, the output should look like :
[{"clientid":"*", "identityzone":"*"}]
I have used : sed -e 's/\"//g' but this deletes all the " in a string
You need to use line anchors
$ echo '"[{"clientid":"*", "identityzone":"*"}]"' | sed 's/^"//; s/"$//'
[{"clientid":"*", "identityzone":"*"}]
^" match " only at start of line
"$ match " only at end of line
You can also combine them using | as sed 's/^"\|"$//g'
See Overview of basic regular expression syntax
easy:
sed 's/^\"\(.*\)\"$/\1/g' <<<'"[{"clientid":"*", "identityzone":"*"}]"'
I want to replace all single quotes in a string with two single quotes using sed. But when the string contains the & character, the sed command is not replacing single quotes that come after that. How can I escape the & character so that the single quotes after it are still replaced?
You don't need to escape anything in the input:
$ echo "123 ' foo & b'ar" | sed "s/'/''/g"
123 '' foo & b''ar
However, in the 'replacement' part of the s command & has a special meaning: it means 'match'. That's why the above command can be re-written as:
$ echo "123 ' foo & b'ar" | sed "s/'/&&/g"
123 '' foo & b''ar
Escape it with a \ like everything else that needs to be escaped, if needed:
$ echo "123 ' foo & b'ar" | sed "s/'/'\&'/g"
123 '&' foo & b'&'ar
It's easier to answer if you post your code, but I'm guessing you're not escaping the ampersand. Change & to \& if you want a literal ampersand.
See section 3.1.2 of The sed FAQ for a more detailed explantion, if you're curious.
It's working for me
bash>echo "'This is a string with 'single quote' & '&'" | sed "s/'/''/g"
''This is a string with ''single quote'' & ''&''