I want to replace some strings with other strings in a folder containing .ly files (.ly files are files used to write music, which can then be converted to svg files). One problem is that the files contain backslashes (\).
Anyway, what I want to do is to replace the following strings:
\f with \fffff
\p with \ppppp
mordent with reverseturn
r4 with \mordent
Each string I want to replace (and each string it is replaced for) has a space in the end. This is because I don't want strings like
\pt to be replaced with \pppppt
Similarly, because backslashes matter, strings like
\ff should stay the same (only f with a slash before and a space after should be replaced).
The code that I am using to achieve this is:
for f in *.ly
do
filename="${f}"
sed -i -- "s#\p #\ppppp #g; s#\f #\fffff #g; s#mordent #reverseturn #g; s#r4 #\maxima #g" $filename;
done
I am using delimited # instead of backslash exactly to achieve this.
However, the program is giving some totally weird behavior. In particular:
\f stays the same (wrong behavior, the right behavior is to get transformed to \fffff)
\p is converted to \ppppp (correct behavior)
\stemUp is converted to \stemUppppp (wrong behavior, the right behavior is to remain as it was)
This is perplexing me and giving me a headache. In particular, why it works correctly for \p but not for \f considering that it is the exact same code.
For reference, I attached a file before and after applying the sed commands.
Thanks in advance for any help!
NB: I am using Ubuntu 16.04 and sed (GNU 4.2.2).
$ echo '12p3 p \p foo' | sed 's/\p /\ppppp /g'
12p3 ppppp \ppppp foo
$ echo '12p3 p \p foo' | sed 's/\\p /\\ppppp /g'
12p3 p \ppppp foo
\ is a meta-character, to match it literally, it needs to be escaped, i.e \\
\ is special within double quotes too(because of bash, nothing to do with sed) - so don't use double quotes unless needed(for ex: sed substitution with bash variables). See also Difference between single and double quotes in Bash
Related
I have the following string variable defined in bash as version
Uploaded registry/version1.3.1
I need to escape the / twice so that the string looks like:
registry\\/version1.3.1
Is there a way to use sed to find and replace all / to \\/?
One backslash is used to change an argument to text or vice versa.
echo 'Uploaded registry/version1.3.1' | sed 's/\//\\\\\//g'
Uploaded registry\\/version1.3.1
In this case you need \\ for one backslash and \/for a slash.
Using sed
You need two backslashes, \\, in the RHS (right-hand side) of the sed command to replaces with a single backslash.
Also, it is a good idea to use a character other than / as a regex delimiter to avoid overescaping.
You can thus use
sed 's,/,\\\\/,g'
See this online demo:
echo 'Uploaded registry/version1.3.1' | sed 's,/,\\\\/,g'
# => Uploaded registry\\/version1.3.1
Using string manipulation
You may also just use string manipulation in Bash:
s='Uploaded registry/version1.3.1'
echo "${s//\//\\\\\/}"
# => Uploaded registry\\/version1.3.1
Here, the backslash needs doubling again, and a single \ is added to escape /s. Note that "${s//a/b} means: replace all occurrences of a with b in s variable.
I'm trying to use sed as root to alter the default PS1.
The first 2 lines were just to make sure I had the syntax structured right and to see if the quotation marks made a difference,
running them consecutively allows me to change a comment near the top of the file and then change it back.
Opening the file in nano confirms the changes are effective, which should rule out 'write permissions'.
sed -i 's/If not running interactively,/stringtoreplaceitwith/' /etc/skel/.bashrc
sed -i "s/stringtoreplaceitwith/If not running interactively,/" /etc/skel/.bashrc
sed -i "s/\[\033[01;32m\]\u#\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ /Replace PS1/" /etc/skel/.bashrc
I'm not sure if it's something else about the string's structure,
but for some reason, it's not finding what I'd like to substitute.
\[\033[01;32m\]\u#\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$
(with a trailing blank).
Your problem is due to quoting and escaping. Let's reduce the string you're trying to substitute to just this:
\w\[\033[00m\]\$
(with a trailing blank) – this contains all the difficulties, but is less verbose.
To match this, we have to
escape all the backslashes: \w becomes \\w etc.
escape [ and ], because they're special to sed: [ becomes \[, and \[ becomes \\\[ (escaped backslash, escaped [)
escape $: \$ becomes \\\$
Then, we have to use single quotes around our sed command so the shell doesn't modify the string:
$ sed 's/\\w\\\[\\033\[00m\\\]\\\$ /Replace PS1/' <<< '\w\[\033[00m\]\$ '
Replace PS1
It can also work with double quotes, but then we have to add another round of escaping for the shell: \[ becomes \\\[ (escaped for sed) becomes \\\\\\[ (escaped for the shell), and so on.
$ sed "s/\\\\w\\\\\\[\\\\033\\[00m\\\\\\]\\\\\\$ /Replace PS1/" <<< '\w\[\033[00m\]\$ '
Replace PS1
Double quoting only makes sense if you're using variables, though, for obvious reasons.
Finally, for your actual string:
$ echo '\[\033[01;32m\]\u#\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ ' \
| sed 's/\\\[\\033\[01;32m\\\]\\u#\\h\\\[\\033\[00m\\\]:\\\[\\033\[01;34m\\\]\\w\\\[\\033\[00m\\\]\\\$ /Replace PS1/'
Replace PS1
This question already has answers here:
Is it possible to escape regex metacharacters reliably with sed
(4 answers)
Closed 5 years ago.
I have replaced urls in the past using sed with no problem before. However, this url imparticular is giving me trouble. It has quite a few ampersands and I need to replace them. How would I go about doing that?
sed -i.bak "s#<string>https://www.url1toreplace.com?blah=1234585474738743874386328764287364238746283764287346872364fN&blah=Y&blah=%2Fwebapp%2Fwcs%2Fblahblah%2Fblah%2Fen%2Fblahahah%3Fblah%3e212e123152%26cm_mmc%3DBLAH-_-BLAH-_-Null-_-Null</string>#<string>https://www.urltoreplace.com/blah/blah/blah/blah/en/blah?blah=129i312093132&cm_mmc=BLAH-_-BLAH-_-Null-_-Null</string>#g" path/to/xml/file
My problem is that it's not fully replacing the url. How do I escape the ampersands so I can successfully replace www.url1toreplace.com with www.urltoreplace.com and everything that follows?
In the replacement text, you need to escape &.
For example, without the escape, the whole of the original match is substituted in for each &:
$ echo '&' | sed 's#&#a & b & c#'
a & b & c
With the escape, \&, & is treated as an ordinary character:
$ echo '&' | sed 's#&#a \& b \& c#'
a & b & c
Your example
Let's take this test file:
$ cat file
<string>https://www.url1toreplace.com?blah=1234585474738743874386328764287364238746283764287346872364fN&blah=Y&blah=%2Fwebapp%2Fwcs%2Fblahblah%2Fblah%2Fen%2Fblahahah%3Fblah%3e212e123152%26cm_mmc%3DBLAH-_-BLAH-_-Null-_-Null</string>
And run the original command:
$ sed "s#<string>https://www.url1toreplace.com?blah=1234585474738743874386328764287364238746283764287346872364fN&blah=Y&blah=%2Fwebapp%2Fwcs%2Fblahblah%2Fblah%2Fen%2Fblahahah%3Fblah%3e212e123152%26cm_mmc%3DBLAH-_-BLAH-_-Null-_-Null</string>#<string>https://www.urltoreplace.com/blah/blah/blah/blah/en/blah?blah=129i312093132\&cm_mmc=BLAH-_-BLAH-_-Null-_-Null</string>#g" file
<string>https://www.urltoreplace.com/blah/blah/blah/blah/en/blah?blah=129i312093132&cm_mmc=BLAH-_-BLAH-_-Null-_-Null</string>
The above command fails. If we escape the &, however, we get:
$ sed 's#<string>https://www.url1toreplace.com?blah=1234585474738743874386328764287364238746283764287346872364fN&blah=Y&blah=%2Fwebapp%2Fwcs%2Fblahblah%2Fblah%2Fen%2Fblahahah%3Fblah%3e212e123152%26cm_mmc%3DBLAH-_-BLAH-_-Null-_-Null</string>#<string>https://www.urltoreplace.com/blah/blah/blah/blah/en/blah?blah=129i312093132\&cm_mmc=BLAH-_-BLAH-_-Null-_-Null</string>#g' file
<string>https://www.urltoreplace.com/blah/blah/blah/blah/en/blah?blah=129i312093132&cm_mmc=BLAH-_-BLAH-_-Null-_-Null</string>
This succeeds: the & in the replacement string successfully appears in the output.
Sample data file:
$ cat xfile
<string>https://www.old.home.com?x=123&y=abc&z=ABC_mmc%3D</string>
Desired output:
<string>https://www.new.home.biz?A=XYZ&B=123&C=987_jjj%2XD</string>
As John1024's already pointed out, if a sed replacement string contains &'s, the &'s have to be escaped (\&) (because & has a special meaning to sed).
Hmmmm, but that could be a major pain in the keister if ya gotta go through and (manually?) change all sed replacement patterns from & to \&. But this replacement can be automated with a few minor assumptions ...
Assumptions:
search and replace patterns can be stored in variables before and after, respectively (actually, only the after variable is needed for this idea to work, but for this example I'll use before and after variables)
before and after contain normal strings w/out any special escapes
your version of bash supports character replacement via the ${var// /} construct
Apply escapes to the after variable on the fly:
$ before='old.home.com?x=123&y=abc&z=ABC_mmc%3D'
$ after='new.home.biz?A=XYZ&B=123&C=987_jjj%2XD'
$ sed "s#${before}#${after//\&/\\\&}#g" xfile
<string>https://www.new.home.biz?A=XYZ&B=123&C=987_jjj%2XD</string>
${after//\&/\\\&} : in the after variable, replace all occurrences of & with \&
This eliminates the need to go through and manually escape all occurrences of & in the replacement string.
I am having a sql file (samplesqlfile) and I want to replace a string which contains backticks with another string. Below is the code.
actualtext="FROM sampledatabase.\`Datatype\`"
replacetext="FROM sampledatabase.\`Datatype_details\`"
sed -i "s/\<${actualtext}\>/${replacetext}/g" samplesqlfile
This is not working. The actual word to be replaced is
FROM sampledatabase.`Datatype`
I added back slashes to escape the backticks. But still it is not working. Please help.
Observe that this does not work:
$ sed "s/\<${actualtext}\>/${replacetext}/g" samplesqlfile
FROM sampledatabase.`Datatype`
But this does:
$ sed "s/\<${actualtext}/${replacetext}/g" samplesqlfile
FROM sampledatabase.`Datatype_details`
The problem was the \>. The string variable $actualtext does not end with a word-character. It ends with a quote. Consequently, \> will never match there. The solution is to remove \>.
To clarify, \> matches at the boundary between a word character and a non-word character where the word character appears first. Word characters can be alphanumerics or underlines.
\> is a GNU extension. The behavior under BSD/OSX sed will be different.
For purposes of illustration here, I removed the -i option. For your intended use, of course, add it back.
I have file with comments like this:
\$max_servers = 2;
\#\## BLOCKED ANYWHERE
I'm trying to
Replace all instances of \$ with $.
Replace all instances of \#\## with ###.
I wonder how I can go about doing that via sed or awk
What I have tried so far without much success using vi or vim
%s/^\//gc
%s/^#/\\/###/gc
Thank you
Another option to replace all [#$] in one pass is to use the following regular expression. The following is VI syntax:
:%s/\\\([$#]\)/\1/g
Replace the characters in the brackets [] with whatever you need if its more than just # and $.
The first \\ is a backslash - escaped since its inside a regular expression
The expression between the \( and \) is saved and later used in the replacement as \1.
Escaping backslash will work
#echo "\#\##"| sed "s/\\\\#\\\\##/###/g"
###
# echo "\\$"| sed "s/\\\\\\$/$/g"
$
In order to replace a backslash, you have to double it up, so it can quote itself much the way other special characters must be quoted. You can use sed instead of vim to help automate the process a bit:
$ sed -e 's/^\\\$/$/' -e 's/^\\#\\##/###/' $file > $new_file
Note that you have to put a backslash in front of dollar signs since they are used to mark an end of line in regular expressions. That's why I have \\\$ in my first expression. One backslash to quote the backslash and another backslash to quote the dollar sign.
By the way, these same sed expressions will also work inside Vim depending upon your Vim settings.
You escape special characters with the backslash. So for example, to replace everything with \$, you would do
%s/\\\$/$/g
sed 's|^\\\([$#]\)\\\{0,1\}|\1|' YourFile
work for your sample bu will also remove the 2 \ in \$\ ...,