I want to use sed to replace each occurence of a particular text in a full source file tree. I've attempted the following:
$ grep -rlI name2port\(\"Wan1\"\) . --exclude-dir=.svn --exclude=*.vxs | xargs sed -i 's/name2port\(\"Wan1\"\)/T_PORT_ID_WAN1/g' but it doesn't seem to work, I think my sed cmd isn't correct. How do I do this?
The problem is, that the replacements just do not happen.
I tried this: $ sed -i 's/name2port\(\"Wan1\"\)/T_PORT_ID_WAN1/g' ./rtos_core/jpax_switch/api/src/nms/switch_l3_route.c but turns out, the occurences of name2port("Wan1") would not be replaced.
sed uses BREs (basic regular expressions) by default, which, for historical reasons - and surprisingly for someone used to modern regular expressions - require escaping of certain metacharacters in order to be recognized as such.
In BREs, ( and ) are ordinary (literal) characters, and only become special when \-escaped.
Therefore, to match literal name2port("Wan1"), use that literal as-is in a BRE (given that you also don't need to \-escape " instances):
sed -i 's/name2port("Wan1")/T_PORT_ID_WAN1/g' ./rtos_core/jpax_switch/api/src/nms/switch_l3_route.c
If you're not concerned about portability, you can use -r (or -E for limited portability to macOS, though with -i that won't work), which then enables EREs (extended regular expressions), whose syntax and features are more likely to work as you expect:
sed -r -i 's/name2port\("Wan1"\)/T_PORT_ID_WAN1/g' ./rtos_core/jpax_switch/api/src/nms/switch_l3_route.c
Note how literals ( and ) now do need to be \-escaped, lest they be interpreted as enclosing a capture group.
In this particular case, it is the BRE that requires less escaping than the ERE; generally, though, it is the opposite.
Related
I have a file r. I want to replace the words File and MINvac.pdb in it with nothing. The commands I used are
sed -i 's/File//g' /home/kanika/standard_minimizer_prosee/r
and
sed -i 's/MINvac.pdb//g' /home/kanika/standard_minimizer_prosee/r
I want to combine both sed commands into one, but I don't know the way. Can anyone help?
The file looks like this:
-6174.27 File10MINvac.pdb
-514.451 File11MINvac.pdb
4065.68 File12MINvac.pdb
-4708.64 File13MINvac.pdb
6674.54 File14MINvac.pdb
8563.58 File15MINvac.pdb
sed is a scripting language. You separate commands with semicolon or newline. Many sed dialects also allow you to pass each command as a separate -e option argument.
sed -i 's/File//g;s/MINvac\.pdb//g' /home/kanika/standard_minimizer_prosee/r
I also added a backslash to properly quote the literal dot before pdb, but in this limited context that is probably unimportant.
For completeness, here is the newline variant. Many newcomers are baffled that the shell allows literal newlines in quoted strings, but it can be convenient.
sed -i 's/File//g
s/MINvac\.pdb//g' /home/kanika/standard_minimizer_prosee/r
Of course, in this limited case, you could also combine everything into one regex:
sed -i 's/\(File\|MINvac\.pdb\)//g' /home/kanika/standard_minimizer_prosee/r
(Some sed dialects will want this without backslashes, and/or offer an option to use extended regular expressions, where they should be omitted. BSD sed, and thus also MacOS sed, demands a mandatory argument to sed -i which can however be empty, like sed -i ''.)
Use the -e flag:
sed -i -e 's/File//g' -e 's/MINvac.pdb//g' /home/kanika/standard_minimizer_prosee/r
Once you get more commands than are convenient to define with -es, it is better to store the commands in a separate file and include it with the -f flag.
In this case, you'd make a file containing:
s/File//g
s/MINvac.pdb//g
Let's call that file 'sedcommands'. You'd then use it with sed like this:
sed -i -f sedcommands /home/kanika/standard_minimizer_prosee/r
With only two commands, it's probably not worthwhile using a separate file of commands, but it is quite convenient if you have a lot of transformations to make.
I've been using grep and sed on some logcat output to make it more readable and I noticed my output was noticeably slower than just grep-ing the output.
I understand sed is obviously going to add more runtime, but I wanted to check for any optimization techniques.
My commands look something like this for reference:
adb logcat | grep arg | sed $'s/{/\\\n{/g
The useless grep is well-documented and easy to get rid of.
adb logcat | sed $'/\\*arg/s/{/\\\n{/g'
To briefly reiterate the linked web page, anything that looks like grep 'x' | sed 'y' can be refactored to sed '/x/y' (and similarly for grep 'x' | awk 'y', which reduces to awk '/x/ y'). sed and Awk are both generalized regex tools which can do everything grep can do (though in fairness some complex grep options are tedious to reimplement in a sed or Awk script; but this is obviously not one of these cases).
However, *arg* is not a well-defined regex; so I have to guess what you actually mean.
* at the beginning of a regex isn't well-defined; but many grep implementations will understand it to mean a literal asterisk. If that's not what you meant, probably take away the first \\*.
arg* is exactly equivalent to ar; if you don't care whether there are g characters after the match, just don't specify them. But perhaps you actually meant arg followed by anything?
But then I guess you probably meant just arg (implicitly preceded by and followed by anything).
In case it's not already obvious, * is not a wildcard character in regex. Instead, it says to repeat the preceding expression as many times as possible, zero or more (and thus the way to say "any string at all" in regex is .*, i.e. the "any character (except newline)" wildcard character . repeated zero or more times).
Also, grep (and sed, and Awk) look for the regex anywhere in a line (unless you put in explicit regex anchors or use grep -x or equivalent options in sed or Awk) so you don't need to specify "preceded by anything" or "followed by anything".
The Bash "C-style string" $'...' offers some conveniences, but also requires any literal backslash to be doubled. So $'/\\*/' is equivalent to '/\*/' in regular single quotes.
The reason the sed slows you down is probably buffering, but getting rid of the useless grep also coincidentally gets rid of that buffering.
I am trying to remove ./ between two characters using sed but not getting the desired output.
Sample:
e2b66a3d84ee448c33d7f2a2f7e51c58 ./2017_06_10_0400.txt
I tried the below but it is not working as expected, even the . in the ".txt" is getting removed.
sed -i 's/[./,]//g'
Beware: don't even think of using the -i option until you know the code is working. You can screw things up big time!
Use:
sed -e 's%[.]/%%g'
You can choose the delimiter in a s/// command, and when the regular expressions involve /, it is sensible to choose something else — I often use % when it doesn't figure in the text. The -e is optional. Using [.] to detect an actual dot is one way; you can write \. if you prefer, but I'm allergic to avoidable backslashes (if you've never had to write 16 backslashes in a row to get troff to do what you want, you haven't suffered enough).
Be aware that the -i option behaves differently in GNU sed and BSD (macOS) sed. Using -i.bak works in both (for an arbitrary, non-empty string such as .bak). Otherwise, your code isn't portable (which may or may not matter to you now, but might well do later on).
You have:
sed -i 's/[./,]//g'
The trouble with this is that it looks for any of the characters ., / or , in isolation — so it removes the . in .txt as well as the . and / in ./. You need to look for consecutive characters — as in my suggested solution.
try this:
echo "e2b66a3d84ee448c33d7f2a2f7e51c58 ./2017_06_10_0400.txt" | sed -e 's|\./||'
You need to use escape character \
's#\.\/##g'
:=>echo "e2b66a3d84ee448c33d7f2a2f7e51c58 ./2017_06_10_0400.txt" | sed 's#\.\/##g'
e2b66a3d84ee448c33d7f2a2f7e51c58 2017_06_10_0400.txt
:=>
I'm trying to find something in a file with a pattern using the '*' but is not working, any idea how to do this?
this is what I'm trying to do:
grep "files*.txt" $myTestFile
is not returning anything, it's suppose that "*" should be all.
By default, grep doesn't support extended regular express, but grep -E or egrep do.
egrep "files.*\.txt" $myTestFile
or
grep -E "files.*\.txt" $myTestFile
In addition, three variant programs egrep, fgrep and rgrep are available. egrep is the same as grep -E. fgrep is the same as grep -F. rgrep is the same as grep -r. Direct
invocation as either egrep or fgrep is deprecated, but is provided to allow historical applications that rely on them to run unmodified.
Matcher Selection
-E, --extended-regexp
Interpret PATTERN as an extended regular expression (ERE, see below). (-E is specified by POSIX.)
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings, separated by newlines, any of which is to be matched. (-F is specified by POSIX.)
-G, --basic-regexp
Interpret PATTERN as a basic regular expression (BRE, see below). This is the default.
-P, --perl-regexp
Interpret PATTERN as a Perl regular expression (PCRE, see below). This is highly experimental and `grep -P` may warn of unimplemented features.
If you only want to match the exact string files*.txt, that would be:
# match exactly "files*.txt"
grep -e "files[*][.]txt" "$myTestFile"
...or, more simply put using fgrep to match only the exact string given:
# match exactly "files*.txt"
fgrep -e 'files*.txt' "$myTestFile"
[*] defines a character class with only a single character -- the * -- contained, and thus matches only that one character. Backslash-based escaping is also possible, but can have different meanings in different contexts and thus is less reliable.
If you want to match any line that contains files, and later .txt, then:
# match any line containing "files" and later ".txt"
grep -e "files.*[.]txt" "$myTestFile"
.* matches zero-or-more characters, and is thus the regex equivalent to the glob-pattern *. Likewise, whereas in a glob pattern . matches only itself, in a regex . can match any character, so the . in .txt needs to be escaped, as in [.]txt, to prevent it from matching anything else.
This question already has answers here:
Using different delimiters in sed commands and range addresses
(3 answers)
Closed 6 years ago.
I am having trouble figuring out how to use sed to search and replace strings containing the / character in a text file /etc/myconfig.
For instance, in my existing text file, I have:
myparam /path/to/a argB=/path/to/B xo
and I want this replaced by:
myparam /path/to/c argB=/path/to/D xo
I attempted doing this in bash:
line='myparam /path/to/a argB=/path/to/B xo'
line_new='myparam /path/to/c argB=/path/to/D xo'
sed -i 's/$line/$line_new/g' /etc/myconfig
But nothing happens.
Attempting
grep -rn "$line" /etc/myconfig
does return me 'myparam /path/to/a argB=/path/to/B xo' though.
What's the correct way to express my sed command to execute this search and replace and correctly deal with the / command? (I reckon that the / character in my strings are the ones giving me the problem because I used a similar sed command to search and replace another line in the text file with no problems and that line does not have a / character.
Don't escape the backslashes; you'll confuse yourself. Use a different symbol after the s command that doesn't appear in the text (I'm using % in the example below):
line_old='myparam /path/to/a argB=/path/to/B xo'
line_new='myparam /path/to/c argB=/path/to/D xo'
sed -i "s%$line_old%$line_new%g" /etc/myconfig
Also, enclose the whole string in double quotes; using single quotes means that sed sees $line (in the original) instead of the expanded value. Inside single quotes, there is no expansion and there are no metacharacters. If your text can contain almost any plain text character, use a control character (e.g. control-A or control-G) as the delimiter.
Note that the use of -i here mirrors what is in the question, but that assumes the use of GNU sed. BSD sed (found on Mac OS X too) requires a suffix. You can use sed -i '' … to replace in situ; that does not work with GNU sed. To be portable between the two, use -i.bak; that will work with both — but gives you a backup file that you'll probably want to delete. Other Unix platforms (e.g. AIX, HP-UX, Solaris) may have variants of sed that do not support -i at all. It is not required by the POSIX specification for sed.
This might work for you:
sed -i "s|$line|$line_new|g" /etc/myconfig
You must use "'s so that the $line and $new_line are interpolated. Also use | or any character not found in the match or replacement strings as a delimiter.