This question already has answers here:
Search and replace with sed when dots and underscores are present
(5 answers)
Closed 6 years ago.
I am trying to replace (for instance) 6.0 by 6.1 in a file, without 640 being replaced by 6.1
I have currently:
sed -i "s/$previousName/$newName/" 'myFile'
I think that the solution could be in here, but I don't find the right solution.
EDIT both string are inside a variable and the question this is supposed to be a duplicate of doesn't treat this case
Using an inner sed:
sed -i "s#$(echo $previousName | sed 's/\./\\./g')#$newName#g" myFile
Try this:
sed -i "s/6\.0/6.1/" 'myFile'
The key is to escape the . character in the pattern which has special meaning. By default it matches any character (including 0 in 640), whereas with a \ in front of it, it only matches a literal ..
Since you have the pattern in a variable, you could escape the . in it first like this:
previousNameE="$(sed -e 's/\./\\./' <<< "$previousName")"
sed -i "s/$previousNameE/$newName/" 'myFile'
if perl is acceptable:
perl -i -pe "s/\Q$previousName/$newName/" 'myFile'
From perldoc for \Q
Returns the value of EXPR with all the ASCII non-"word" characters
backslashed. (That is, all ASCII characters not matching
/[A-Za-z_0-9]/ will be preceded by a backslash in the returned string,
regardless of any locale settings.) This is the internal function
implementing the \Q escape in double-quoted strings
Another example:
$ echo '*.^[}' | perl -pe 's/\Q*.^[}/q($abc$)/e'
$abc$
Further reading: Perl flags -pe, -pi, -p, -w, -d, -i, -t?
Related
This question already has answers here:
Using different delimiters in sed commands and range addresses
(3 answers)
Closed 1 year ago.
I have a Visual Studio project, which is developed locally. Code files have to be deployed to a remote server. The only problem is the URLs they contain, which are hard-coded.
The project contains URLs such as ?page=one. For the link to be valid on the server, it must be /page/one .
I've decided to replace all URLs in my code files with sed before deployment, but I'm stuck on slashes.
I know this is not a pretty solution, but it's simple and would save me a lot of time. The total number of strings I have to replace is fewer than 10. A total number of files which have to be checked is ~30.
An example describing my situation is below:
The command I'm using:
sed -f replace.txt < a.txt > b.txt
replace.txt which contains all the strings:
s/?page=one&/pageone/g
s/?page=two&/pagetwo/g
s/?page=three&/pagethree/g
a.txt:
?page=one&
?page=two&
?page=three&
Content of b.txt after I run my sed command:
pageone
pagetwo
pagethree
What I want b.txt to contain:
/page/one
/page/two
/page/three
The easiest way would be to use a different delimiter in your search/replace lines, e.g.:
s:?page=one&:pageone:g
You can use any character as a delimiter that's not part of either string. Or, you could escape it with a backslash:
s/\//foo/
Which would replace / with foo. You'd want to use the escaped backslash in cases where you don't know what characters might occur in the replacement strings (if they are shell variables, for example).
The s command can use any character as a delimiter; whatever character comes after the s is used. I was brought up to use a #. Like so:
s#?page=one&#/page/one#g
A very useful but lesser-known fact about sed is that the familiar s/foo/bar/ command can use any punctuation, not only slashes. A common alternative is s#foo#bar#, from which it becomes obvious how to solve your problem.
add \ before special characters:
s/\?page=one&/page\/one\//g
etc.
In a system I am developing, the string to be replaced by sed is input text from a user which is stored in a variable and passed to sed.
As noted earlier on this post, if the string contained within the sed command block contains the actual delimiter used by sed - then sed terminates on syntax error. Consider the following example:
This works:
$ VALUE=12345
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
MyVar=12345
This breaks:
$ VALUE=12345/6
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
sed: -e expression #1, char 21: unknown option to `s'
Replacing the default delimiter is not a robust solution in my case as I did not want to limit the user from entering specific characters used by sed as the delimiter (e.g. "/").
However, escaping any occurrences of the delimiter in the input string would solve the problem.
Consider the below solution of systematically escaping the delimiter character in the input string before having it parsed by sed.
Such escaping can be implemented as a replacement using sed itself, this replacement is safe even if the input string contains the delimiter - this is since the input string is not part of the sed command block:
$ VALUE=$(echo ${VALUE} | sed -e "s#/#\\\/#g")
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
MyVar=12345/6
I have converted this to a function to be used by various scripts:
escapeForwardSlashes() {
# Validate parameters
if [ -z "$1" ]
then
echo -e "Error - no parameter specified!"
return 1
fi
# Perform replacement
echo ${1} | sed -e "s#/#\\\/#g"
return 0
}
this line should work for your 3 examples:
sed -r 's#\?(page)=([^&]*)&#/\1/\2#g' a.txt
I used -r to save some escaping .
the line should be generic for your one, two three case. you don't have to do the sub 3 times
test with your example (a.txt):
kent$ echo "?page=one&
?page=two&
?page=three&"|sed -r 's#\?(page)=([^&]*)&#/\1/\2#g'
/page/one
/page/two
/page/three
replace.txt should be
s/?page=/\/page\//g
s/&//g
please see this article
http://netjunky.net/sed-replace-path-with-slash-separators/
Just using | instead of /
Great answer from Anonymous. \ solved my problem when I tried to escape quotes in HTML strings.
So if you use sed to return some HTML templates (on a server), use double backslash instead of single:
var htmlTemplate = "<div style=\\"color:green;\\"></div>";
A simplier alternative is using AWK as on this answer:
awk '$0="prefix"$0' file > new_file
You may use an alternative regex delimiter as a search pattern by backs lashing it:
sed '\,{some_path},d'
For the s command:
sed 's,{some_path},{other_path},'
This question already has answers here:
How can I add quotation marks to fields in a CSV file?
(4 answers)
Closed 2 years ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Original close reason(s) were not resolved
Let's say I have a file with this structure:
1|2|3|4|
5|6|7|8|
9|10|11|12|
However, I want my file to look like this (expected output):
"1"|"2"|"3"|"4"|
"5"|"6"|"7"|"8"|
"9"|"10"|"11"|"12"|
I am trying to used sed command in the following way:
sed 's/^/"/g'
Unfortunately, it only adds quotation marks at the beginning of each line:
"1|2|3|4|
"5|6|7|8|
"9|10|11|12|
^ means "the beginning of a line". Use [^|] instead which means "anything but |". If your implementation of sed supports +, you can use
sed -E 's/[^|]+/"&"/g'
otherwise, you need to be more verbose
sed 's/[^|][^|]*/"&"/g'
& represents the matched part.
You can use
sed -E 's/[^|]+/"&"/g' file > newfile
The -E option enables the POSIX ERE syntax and [^|]+ thus matches one or more chars other than |, and "&" replaces each with its copy enclosed with " on both sides.
See the online sed demo:
s='1|2|3|4|
5|6|7|8|
9|10|11|12|'
sed -E 's/[^|]+/"&"/g' <<< "$s"
Output:
"1"|"2"|"3"|"4"|
"5"|"6"|"7"|"8"|
"9"|"10"|"11"|"12"|
Here is a gnu awk way of doing the same:
awk -v RS="[|\n]+" '{ORS=RT; print "\"" $0 "\""}' file
"1"|"2"|"3"|"4"|
"5"|"6"|"7"|"8"|
"9"|"10"|"11"|"12"|
This question already has answers here:
Is it possible to escape regex metacharacters reliably with sed
(4 answers)
Closed 4 years ago.
In a file x, a line containing 4*4 4*1 4*4 4*0 is to be replaced by 4*4 4*1 3*4 1*4 4*0 which is in file y. I'm using the following code:
#!/bin/bash
old=`grep "4*4" x`;
new=`grep "4*4" y`;
sed -i "s/${old}/${new}/g" x
but it yields no change at all in x. I'm a novice and this might be a silly question for this site but I'm unable to replace this expression with multiple special characters with another expression.
Welcome to stackOverflow - An awesome platform for developers 😃
If you stick around, you will also find it very rewarding. 🙂
Now to your question.
* is a special character for sed when it is included in pattern to match. More info here.
Thus, we need to escape it with a \.
You can use something like old=$(echo "${old}" | sed -r 's/[*]/\\*/g') to do so for each * inside variable old.
echo "${old}" | feeds the value of variable old to sed.
Let me write the sed command in an expanded form: sed -r 's/ [*] / \\* /g'
-r because we are using regex in pattern to match.
[*] is the regex and also the pattern to match. Info
\\* is the replacement string - The first \ is escaping the second \ and * is being treated as a normal character.
$( ) has been used to assign the final output to variable old.
Here is the modified script:
#!/bin/bash
# Read the strings
old=`grep "4*4" x`;
new=`grep "4*4" y`;
# Escape * i.e. add \ before it - As * is a special character for sed
old=$(echo "${old}" | sed -r 's/[*]/\\*/g')
# Replace
sed -i "s/${old}/${new}/g" x
This question already has answers here:
Substitute all characters between two strings by char 'X' using sed
(3 answers)
Closed 6 years ago.
file=mylog.log
search_str="&Name="
end_str="&"
sed -i -E ':a; s/('"$search_str"'X*)[^X'"$end_str"']/\1X/; ta' "$file"
Ex 1:
something&Name=JASON&else
to
something&Name=XXXXX&else
And actually, my current sed command works fine when instead of a '"$end_str"' if I use '&' character... Like this :
sed -i -E ':a; s/('"$search_str"'X*)[^X&]/\1X/; ta' "$file"
So, to summariz, it, after ^X if a single character comes than my given sed command works fine... But the same command does not work, if instead of character, i use a string...
For example, my sed command won't work in this case :
end_str="\%26"
sed -i -E ':a; s/('"$search_str"'X*)[^X'"$end_str"']/\1X/; ta' "$file"
Eg:
something&Name=JASON_MATTHEW_DONALD%26else
TO
something&Name=XXXXXXXXXXXXXXXXXXXX%26else
Eg:2
something&Name=JASON%26else
TO
something&Name=XXXXX%26else
Please let me know
Place your string variable outside the character class and capture it to check for further substitutions:
sed -i -E ':a; s/('"$search_str"'X*)[^X](.*'"$end_str"')/\1X\2/; ta' "$file"
If the number of X doesn't matter you can simplify it to:
search="Name"
sed "s/$search=[^&]*/$search=XXX/" input.file
This assumes that $search won't contain special characters which have a meaning in sed's regex syntax. If special characters can be a problem you need to prepare the $search variable, as explained here: Is it possible to escape regex metacharacters reliably with sed
I insist on the point that keeping the length of passwords in logs is a very bad idea (security wise).
Having said that:
First, a character list [...] is not the right tool to match an string. For that we need to use an alternating value (...|...).
end_str="(&|%26)"
But it is quite difficult to express a "not a string" in regex.
not_end_str="([^&]|[^%]|%[^2]|%2[^6])"
Using all that we may build a pure bash solution (maybe not fast, but works).
It prints to stdout to show how it works.
Redirect to a file to store the result.
file=mylog.log
search_str="&Name="
end_str="(&|%26)" # write end_str as an alternate value.
not_end_str="([^&]|[^%]|%[^2]|%2[^6])" # regex negate end_string.
# Build a regex that split each part of the text.
myreg="(.*${search_str}X*)(${not_end_str}*)([&%].*)"
while IFS=$'\n' read line; do
[[ $line =~ $myreg ]]
len=$((${#BASH_REMATCH[#]}-2)) # do not count [0] and last.
arr=("${BASH_REMATCH[#]:1}") # remove [0] and last.
arr[1]=${arr[1]//?/X} # Replace name with "X"'s.
arr[2]='' # Clear not_end_str match
printf '%s' "${arr[#]}"; echo # Print modified line.
done <"$file"
Further reading:
Regular expression to match line that doesn't contain a word?
Regular expression that doesn't contain certain string
This question already has answers here:
Using different delimiters in sed commands and range addresses
(3 answers)
Closed 6 years ago.
I am having trouble figuring out how to use sed to search and replace strings containing the / character in a text file /etc/myconfig.
For instance, in my existing text file, I have:
myparam /path/to/a argB=/path/to/B xo
and I want this replaced by:
myparam /path/to/c argB=/path/to/D xo
I attempted doing this in bash:
line='myparam /path/to/a argB=/path/to/B xo'
line_new='myparam /path/to/c argB=/path/to/D xo'
sed -i 's/$line/$line_new/g' /etc/myconfig
But nothing happens.
Attempting
grep -rn "$line" /etc/myconfig
does return me 'myparam /path/to/a argB=/path/to/B xo' though.
What's the correct way to express my sed command to execute this search and replace and correctly deal with the / command? (I reckon that the / character in my strings are the ones giving me the problem because I used a similar sed command to search and replace another line in the text file with no problems and that line does not have a / character.
Don't escape the backslashes; you'll confuse yourself. Use a different symbol after the s command that doesn't appear in the text (I'm using % in the example below):
line_old='myparam /path/to/a argB=/path/to/B xo'
line_new='myparam /path/to/c argB=/path/to/D xo'
sed -i "s%$line_old%$line_new%g" /etc/myconfig
Also, enclose the whole string in double quotes; using single quotes means that sed sees $line (in the original) instead of the expanded value. Inside single quotes, there is no expansion and there are no metacharacters. If your text can contain almost any plain text character, use a control character (e.g. control-A or control-G) as the delimiter.
Note that the use of -i here mirrors what is in the question, but that assumes the use of GNU sed. BSD sed (found on Mac OS X too) requires a suffix. You can use sed -i '' … to replace in situ; that does not work with GNU sed. To be portable between the two, use -i.bak; that will work with both — but gives you a backup file that you'll probably want to delete. Other Unix platforms (e.g. AIX, HP-UX, Solaris) may have variants of sed that do not support -i at all. It is not required by the POSIX specification for sed.
This might work for you:
sed -i "s|$line|$line_new|g" /etc/myconfig
You must use "'s so that the $line and $new_line are interpolated. Also use | or any character not found in the match or replacement strings as a delimiter.