I'm using macOS Sierra, and I'm and trying to manipulate a value of a key from a config file.
In order to achieve that I'm using (which works fine for simple values):
sed -i .bak "/^$KEY/s/\(.[^=]*\)\([ \t]*=[ \t]*\)\(.[^=]*\)/\1\2$VALUE/" $CONFIG_FILE
Unfortunately, my string $VALUE is quite complex with many special characters, giving me the error:
bad flag in substitute command: '/'
My $VALUE is being declared as:
VALUE='<Request xmlns="urn:oasis:names:tc:xacml:3.0:core:schema:wd-17" ReturnPolicyIdList="false" CombinedDecision="false"> <Attributes Category="urn:oasis:names:tc:xacml:3.0:attribute-category:resource"> <Attribute IncludeInResult="false" AttributeId="urn:oasis:names:tc:xacml:1.0:resource:resource-id"> <AttributeValue DataType="http://www.w3.org/2001/XMLSchema#string">test </AttributeValue> </Attribute> </Attributes> <Attributes Category="urn:oasis:names:tc:xacml:3.0:attribute-category:action"> <Attribute IncludeInResult="false" AttributeId="urn:oasis:names:tc:xacml:1.0:action:action-id"> <AttributeValue DataType="http://www.w3.org/2001/XMLSchema#string">testing something</AttributeValue> </Attribute> </Attributes> </Request>'
Since I have double quotes being part of the value of $VALUE, I can't use double quotes instead of single quote when declaring it... Any ideas to workaround this?
The problem is that $VALUE contains slashes which should be escaped because it conflicts with the separator for the substiture command
That's not convenient because if it changes, you have to escape them again. That's a solution, nevertheless.
Another simpler solution is to use an alternate separating character for s command which isn't in the $VALUE string, like % for instance (% has less chance to be in such a string, otherwise | can also be used).
sed -i .bak "/^$KEY/s%\(.[^=]*\)\([ \t]*=[ \t]*\)\(.[^=]*\)%\1\2$VALUE%" $CONFIG_FILE
or with pipe:
sed -i .bak "/^$KEY/s|\(.[^=]*\)\([ \t]*=[ \t]*\)\(.[^=]*\)|\1\2$VALUE|" $CONFIG_FILE
Related
This question already has answers here:
Using different delimiters in sed commands and range addresses
(3 answers)
Closed 1 year ago.
I have a Visual Studio project, which is developed locally. Code files have to be deployed to a remote server. The only problem is the URLs they contain, which are hard-coded.
The project contains URLs such as ?page=one. For the link to be valid on the server, it must be /page/one .
I've decided to replace all URLs in my code files with sed before deployment, but I'm stuck on slashes.
I know this is not a pretty solution, but it's simple and would save me a lot of time. The total number of strings I have to replace is fewer than 10. A total number of files which have to be checked is ~30.
An example describing my situation is below:
The command I'm using:
sed -f replace.txt < a.txt > b.txt
replace.txt which contains all the strings:
s/?page=one&/pageone/g
s/?page=two&/pagetwo/g
s/?page=three&/pagethree/g
a.txt:
?page=one&
?page=two&
?page=three&
Content of b.txt after I run my sed command:
pageone
pagetwo
pagethree
What I want b.txt to contain:
/page/one
/page/two
/page/three
The easiest way would be to use a different delimiter in your search/replace lines, e.g.:
s:?page=one&:pageone:g
You can use any character as a delimiter that's not part of either string. Or, you could escape it with a backslash:
s/\//foo/
Which would replace / with foo. You'd want to use the escaped backslash in cases where you don't know what characters might occur in the replacement strings (if they are shell variables, for example).
The s command can use any character as a delimiter; whatever character comes after the s is used. I was brought up to use a #. Like so:
s#?page=one&#/page/one#g
A very useful but lesser-known fact about sed is that the familiar s/foo/bar/ command can use any punctuation, not only slashes. A common alternative is s#foo#bar#, from which it becomes obvious how to solve your problem.
add \ before special characters:
s/\?page=one&/page\/one\//g
etc.
In a system I am developing, the string to be replaced by sed is input text from a user which is stored in a variable and passed to sed.
As noted earlier on this post, if the string contained within the sed command block contains the actual delimiter used by sed - then sed terminates on syntax error. Consider the following example:
This works:
$ VALUE=12345
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
MyVar=12345
This breaks:
$ VALUE=12345/6
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
sed: -e expression #1, char 21: unknown option to `s'
Replacing the default delimiter is not a robust solution in my case as I did not want to limit the user from entering specific characters used by sed as the delimiter (e.g. "/").
However, escaping any occurrences of the delimiter in the input string would solve the problem.
Consider the below solution of systematically escaping the delimiter character in the input string before having it parsed by sed.
Such escaping can be implemented as a replacement using sed itself, this replacement is safe even if the input string contains the delimiter - this is since the input string is not part of the sed command block:
$ VALUE=$(echo ${VALUE} | sed -e "s#/#\\\/#g")
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
MyVar=12345/6
I have converted this to a function to be used by various scripts:
escapeForwardSlashes() {
# Validate parameters
if [ -z "$1" ]
then
echo -e "Error - no parameter specified!"
return 1
fi
# Perform replacement
echo ${1} | sed -e "s#/#\\\/#g"
return 0
}
this line should work for your 3 examples:
sed -r 's#\?(page)=([^&]*)&#/\1/\2#g' a.txt
I used -r to save some escaping .
the line should be generic for your one, two three case. you don't have to do the sub 3 times
test with your example (a.txt):
kent$ echo "?page=one&
?page=two&
?page=three&"|sed -r 's#\?(page)=([^&]*)&#/\1/\2#g'
/page/one
/page/two
/page/three
replace.txt should be
s/?page=/\/page\//g
s/&//g
please see this article
http://netjunky.net/sed-replace-path-with-slash-separators/
Just using | instead of /
Great answer from Anonymous. \ solved my problem when I tried to escape quotes in HTML strings.
So if you use sed to return some HTML templates (on a server), use double backslash instead of single:
var htmlTemplate = "<div style=\\"color:green;\\"></div>";
A simplier alternative is using AWK as on this answer:
awk '$0="prefix"$0' file > new_file
You may use an alternative regex delimiter as a search pattern by backs lashing it:
sed '\,{some_path},d'
For the s command:
sed 's,{some_path},{other_path},'
This is the output of cat command and I don't know what this special character is called that is at the end of the file to even search for. How to remove this special character in bash?
EDIT:
Here is the actual xml file(I am just copy pasting):
<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
<types>
<name>ApexClass</name>
<members>CreditNotesManager</members>
<members>CreditNotesManagerTest</members>
</types>
<version>47.0</version>
</Package>%
It's unclear how the % (percent sign) is ending up in your file; it's easy to remove with sed:
sed -i '' 's/\(</.*>\)%.*/\1/g' file.xml
This will remove the percent and re-save your file. If you want to do a dry-run omit the -i '' portion as this is tells sed to save the file in-line.
As mentioned in the comments, there are many ways to do it. Just be sure you aren't removing something that you want to keep.
If it is just at the last line, this should work. Using ed(1)
printf '%s\n' '$s/%//' w | ed -s file.xml
If you don't need to save changes, you could use grep:
grep -v "%" <file.xml
This uses grep along with it's inverse matching flag -v. This method will remove all instances of the character % and print the result to STOUT. The < character is a method to tell grep which file you're talking about.
EDIT: actually you don't even need the redirection, so:
grep -v "%" file.xml
This is actually a feature of zsh, not bash.
To disable it, unsetopt prompt_cr prompt_sp
The reverse prompt character showing up means that line had an end-of-file before a final ascii linefeed (newline) character.
How to remove this special character at the end of the file
I have a source code file in which comments are prefixed with // (ie. double slashes and an empty space), I want to convert the source code into a document so I tried to cat file.c and pipe it to sed, the thinking is to replace "double slash and a space" if a line starts with it, with empty string, but it looks like the slash has some special meaning in sed, so what's the best way of constructing the sed arguments?
Thanks!
If you want to remove the special meaning of / from sed then following may help you in same.
sed 's/^\/\/ //g' Input_file
So I am escaping / here by using \ before it, so it will be taken as a literal character rather than it's special meaning in code. Also if you are happy with above command's result then use -i to save the changes in Input_file itself. Hope this helps.
The slash only has meaning if you allow it.
sed 's#^// +##' < file.c
Part of a shell script that I am creating takes a plain text list of files...
11111.jpg
22222.jpg
33333.jpg
...and appends a user-defined prefix that is stored in a variable to create a list of paths that looks like this:
user/defined/prefix/11111.jpg
user/defined/prefix/22222.jpg
user/defined/prefix/33333.jpg
I am attempting to use sed to add the prefix in this manner:
sed -e 's/^/prefix/' oldFile > newFile.new
The variable is getting assigned correctly:
echo $selectedPrefix
user/defined/prefix
Put no combinations of single quotes, double quotes of whatever seem to get sed to use the ACTUAL value of the variable instead of just the variable name.
sed -e 's/^/$selectedPrefix/' oldFile > newFile.new
Yields:
$selectedPrefix11111.jpg
$selectedPrefix22222.jpg
$selectedPrefix33333.jpg
Help! I'm sure the solution is simple but I feel like I've tried everything....
As mentionned by Cyrus, you need to used " (double quote) instead ' (single quote) if you want the variable replacement because single quoted string are interpreted literally so it doesn't see $selectedPrefix as a variable but as the string value of $selectedPrefic hence what you saw.
Since you are working with paths in you sed, you are correct in assuming that you should use a different separator for your sed comment. I usually prefer using | but ~ would also work.
so basically you could have:
sed -e "s~^~$selectedPrefix~" oldFile > newFile.new
This code would solve your problem:
selectedPrefixEscaped="$(echo "$selectedPrefix" | sed 's/\//\\\//g')" && sed -e "s/^/$selectedPrefixEscaped/" oldFile > newFile.new
Just using a different delimiter on sed would leave you open to problems when (if) the path contains the new delimiter (ex.: /folder/folder#5/file.txt would be problematic if using # as sed delimiter).
This question already has answers here:
Using different delimiters in sed commands and range addresses
(3 answers)
Closed 6 years ago.
I am having trouble figuring out how to use sed to search and replace strings containing the / character in a text file /etc/myconfig.
For instance, in my existing text file, I have:
myparam /path/to/a argB=/path/to/B xo
and I want this replaced by:
myparam /path/to/c argB=/path/to/D xo
I attempted doing this in bash:
line='myparam /path/to/a argB=/path/to/B xo'
line_new='myparam /path/to/c argB=/path/to/D xo'
sed -i 's/$line/$line_new/g' /etc/myconfig
But nothing happens.
Attempting
grep -rn "$line" /etc/myconfig
does return me 'myparam /path/to/a argB=/path/to/B xo' though.
What's the correct way to express my sed command to execute this search and replace and correctly deal with the / command? (I reckon that the / character in my strings are the ones giving me the problem because I used a similar sed command to search and replace another line in the text file with no problems and that line does not have a / character.
Don't escape the backslashes; you'll confuse yourself. Use a different symbol after the s command that doesn't appear in the text (I'm using % in the example below):
line_old='myparam /path/to/a argB=/path/to/B xo'
line_new='myparam /path/to/c argB=/path/to/D xo'
sed -i "s%$line_old%$line_new%g" /etc/myconfig
Also, enclose the whole string in double quotes; using single quotes means that sed sees $line (in the original) instead of the expanded value. Inside single quotes, there is no expansion and there are no metacharacters. If your text can contain almost any plain text character, use a control character (e.g. control-A or control-G) as the delimiter.
Note that the use of -i here mirrors what is in the question, but that assumes the use of GNU sed. BSD sed (found on Mac OS X too) requires a suffix. You can use sed -i '' … to replace in situ; that does not work with GNU sed. To be portable between the two, use -i.bak; that will work with both — but gives you a backup file that you'll probably want to delete. Other Unix platforms (e.g. AIX, HP-UX, Solaris) may have variants of sed that do not support -i at all. It is not required by the POSIX specification for sed.
This might work for you:
sed -i "s|$line|$line_new|g" /etc/myconfig
You must use "'s so that the $line and $new_line are interpolated. Also use | or any character not found in the match or replacement strings as a delimiter.