I wanted to change #232-12-3424 into these symbols #%%%-%%-%%% but still having same numbers whenever I run it.
Here's my code I tried but didn't worked out.
s/#[0-9]{3}-[0-9]{2}-[0-9]{4}*/#%%%-%%-%%%%/;
Two things :
putting \ before '{' to escape
Putting a quantifiable token before * (ex: .) or removing it .
Please note if you are using '.*' it will replace whole string (Please check following
example.
cat boo
#232-12-3424
#456-23-7896 some text
cat boo | sed -e 's/#[0-9]\{3\}-[0-9]\{2\}-[0-9]\{4\}/#%%%-%%-%%%%/'
#%%%-%%-%%%%
#%%%-%%-%%%% some text
cat boo | sed -e 's/#[0-9]\{3\}-[0-9]\{2\}-[0-9]\{4\}.*/#%%%-%%-%%%%/'
#%%%-%%-%%%%
#%%%-%%-%%%%
Related
How can I write a regex in a shell script that would target only the targeted substring between two given values? Give the example
https://www.stackoverflow.com
How can I match only the ":" in between "https" and "//".
If possible please also explain the approach.
The context is that I need to prepare a file that would fetch a config from the server and append it to the .env file. The response comes as JSON
{
"GRAPHQL_URL": "https://someurl/xyz",
"PUBLIC_TOKEN": "skml2JdJyOcrVdfEJ3Bj1bs472wY8aSyprO2DsZbHIiBRqEIPBNg9S7yXBbYkndX2Lk8UuHoZ9JPdJEWaiqlIyGdwU6O5",
"SUPER_SECRET": "MY_SUPER_SECRET"
}
so I need to adjust it to the .env syntax. What I managed to do this far is
#!/bin/bash
CURL_RESPONSE="$(curl -s url)"
cat <<< ${CURL_RESPONSE} | jq -r '.property.source' | sed -r 's/:/=/g;s/[^a-zA-Z0-9=:_/-]//g' > .env.test
so basically I fetch the data, then extract the key I am after with jq, and then I use sed to first replace all ":" to "=" and after that I remove all the quotations and semicolons and white spaces that comes from JSON and leave some characters that are necessary.
I am almost there but the problem is that now my graphql url (and only other) would look like so
https=//someurl/xyz
so I need to replace this = that is in between https and // back with the colon.
Thank you very much #Nic3500 for the response, not sure why but I get error saying that
sed: 1: "s/:/=/g;s#https\(.*\)// ...": \1 not defined in the RE
I searched SO and it seems that it should work since the brackets are escaped and I use -r flag (tried -E but no difference) and I don't know how to apply it. To be honest I assume that the replacement block is this part
#\1#
so how can I let this know to what character should it be replaced?
This is how I tried to use it
#!/bin/bash
CURL_RESPONSE="$(curl -s url)"
cat <<< ${CURL_RESPONSE} | jq -r '.property.source' | sed -r 's/:/=/g;s#https\(.*\)//.*#\1#;s/[^a-zA-Z0-9=:_/-]//g' > .env.test
Hope with this context you would be able to help me.
echo "https://www.stackoverflow.com" | sed 's#https\(.*\)//.*#\1#'
:
sed operator s/regexp/replacement/
regexp: https\(.*)//.*. So "https" followed by something (.*), followed by "//", followed by anything else .*
the parenthesis are back slashed since they are not part of the pattern. They are used to group a part of the regex for the replacement part of the s### operator.
replacement: \1, means the first group found in the regex \(.*\)
I used s###, but the usual form is s///. Any character can take the place of the / with the s operator. I used # as using / would have been confusing since you use / in the url.
The problem is that your sed substitutions are terribly imprecise. Anyway, you want to do it in jq instead, where you have more control over which parts you are substituting, and avoid spawning a separate process for something jq quite easily does natively in the first place.
curl -s url |
jq -r '.property.source | to_entries[] |
"\(.key)=\"\(.value\)\""' > .env.test
Tangentially, capturing the output of curl into a variable just so you can immediately cat it once to standard output is just a waste of memory.
I am having a problem with sed. Basically, I want to insert some text after a match a small piece of Java code after a match. It seems to be somehow interpreted as something else and ends up failing.
Consider the following basic example that does what I want:
match="string_I_want_to_match"
str_i_want_above="str I want above"
echo $match | sed -e "/$match/i \\
$str_i_want_above"
outputs what I want:
xyzt
string_I_want_to_match
However, if I use read to create a multiline string, it fails with an obscure error. Consider the following example:
match="string_I_want_to_match"
read -r -d '' str_i_want_above <<- 'EOF'
> I
> Want
> This
> Above the match
EOF
echo $match | sed -e "/$match/i \\
$str_i_want_above"
I expect
I
Want
This
Above the match
string_I_want_to_match
but instead I get
sed: 3: "/string_I_want_to_match ...": invalid command code W
It's somehow failing when trying to interpret the second line. I have isolated this to be a problem with read but if I'm wrong, I'd like to understand why and to fix this.
I'm using BSD sed on mac 12.6. I would like to achieve this with sed and read because I need to do in-place replacing and this is going to be a larger script where I need to do several in-place insertions of multiline strings
You need to insert a \ before the line breaks in replacement, so you may use:
match="string_I_want_to_match"
echo "$match" | sed "/$match/i \\
> ${str_i_want_above//$'\n'/\\$'\n'}
> "
I
Want
This
Above the match
string_I_want_to_match
Here:
${str_i_want_above//$'\n'/\\$'\n'}: Inserts \\ before each line break
Code Demo
This question already has answers here:
Using different delimiters in sed commands and range addresses
(3 answers)
Closed 1 year ago.
I have a Visual Studio project, which is developed locally. Code files have to be deployed to a remote server. The only problem is the URLs they contain, which are hard-coded.
The project contains URLs such as ?page=one. For the link to be valid on the server, it must be /page/one .
I've decided to replace all URLs in my code files with sed before deployment, but I'm stuck on slashes.
I know this is not a pretty solution, but it's simple and would save me a lot of time. The total number of strings I have to replace is fewer than 10. A total number of files which have to be checked is ~30.
An example describing my situation is below:
The command I'm using:
sed -f replace.txt < a.txt > b.txt
replace.txt which contains all the strings:
s/?page=one&/pageone/g
s/?page=two&/pagetwo/g
s/?page=three&/pagethree/g
a.txt:
?page=one&
?page=two&
?page=three&
Content of b.txt after I run my sed command:
pageone
pagetwo
pagethree
What I want b.txt to contain:
/page/one
/page/two
/page/three
The easiest way would be to use a different delimiter in your search/replace lines, e.g.:
s:?page=one&:pageone:g
You can use any character as a delimiter that's not part of either string. Or, you could escape it with a backslash:
s/\//foo/
Which would replace / with foo. You'd want to use the escaped backslash in cases where you don't know what characters might occur in the replacement strings (if they are shell variables, for example).
The s command can use any character as a delimiter; whatever character comes after the s is used. I was brought up to use a #. Like so:
s#?page=one&#/page/one#g
A very useful but lesser-known fact about sed is that the familiar s/foo/bar/ command can use any punctuation, not only slashes. A common alternative is s#foo#bar#, from which it becomes obvious how to solve your problem.
add \ before special characters:
s/\?page=one&/page\/one\//g
etc.
In a system I am developing, the string to be replaced by sed is input text from a user which is stored in a variable and passed to sed.
As noted earlier on this post, if the string contained within the sed command block contains the actual delimiter used by sed - then sed terminates on syntax error. Consider the following example:
This works:
$ VALUE=12345
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
MyVar=12345
This breaks:
$ VALUE=12345/6
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
sed: -e expression #1, char 21: unknown option to `s'
Replacing the default delimiter is not a robust solution in my case as I did not want to limit the user from entering specific characters used by sed as the delimiter (e.g. "/").
However, escaping any occurrences of the delimiter in the input string would solve the problem.
Consider the below solution of systematically escaping the delimiter character in the input string before having it parsed by sed.
Such escaping can be implemented as a replacement using sed itself, this replacement is safe even if the input string contains the delimiter - this is since the input string is not part of the sed command block:
$ VALUE=$(echo ${VALUE} | sed -e "s#/#\\\/#g")
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
MyVar=12345/6
I have converted this to a function to be used by various scripts:
escapeForwardSlashes() {
# Validate parameters
if [ -z "$1" ]
then
echo -e "Error - no parameter specified!"
return 1
fi
# Perform replacement
echo ${1} | sed -e "s#/#\\\/#g"
return 0
}
this line should work for your 3 examples:
sed -r 's#\?(page)=([^&]*)&#/\1/\2#g' a.txt
I used -r to save some escaping .
the line should be generic for your one, two three case. you don't have to do the sub 3 times
test with your example (a.txt):
kent$ echo "?page=one&
?page=two&
?page=three&"|sed -r 's#\?(page)=([^&]*)&#/\1/\2#g'
/page/one
/page/two
/page/three
replace.txt should be
s/?page=/\/page\//g
s/&//g
please see this article
http://netjunky.net/sed-replace-path-with-slash-separators/
Just using | instead of /
Great answer from Anonymous. \ solved my problem when I tried to escape quotes in HTML strings.
So if you use sed to return some HTML templates (on a server), use double backslash instead of single:
var htmlTemplate = "<div style=\\"color:green;\\"></div>";
A simplier alternative is using AWK as on this answer:
awk '$0="prefix"$0' file > new_file
You may use an alternative regex delimiter as a search pattern by backs lashing it:
sed '\,{some_path},d'
For the s command:
sed 's,{some_path},{other_path},'
Hi I am trying to remove part of string using sed command but looks like all options that i came across stack overflow doesn't seem to work.
sub-285345_task-WM_dir-28_epi.nii
sub-285345_task-LANGUAGE_dir-11_epi.nii.gz
I want to remove _task-*** part of it. I want to remove task-**, key value pair.
sed s/_task-.*//g
This removes even dir-** after task. sub-285345_epi.nii.gz
How can i remove only task key value pair?
Do:
sed 's/_task-[^_]*//'
[^_]* will match upto the next _.
Example:
$ sed 's/_task-[^_]*//' <<<'sub-285345_task-WM_dir-28_epi.nii'
sub-285345_dir-28_epi.nii
$ sed 's/_task-[^_]*//' <<<'sub-285345_task-LANGUAGE_dir-11_epi.nii.gz'
sub-285345_dir-11_epi.nii.gz
.* is greedy.
Try something like
$ echo sub-285345_task-WM_dir-28_epi.nii | sed -r 's/_task-.*?_dir/_dir/'
sub-285345_dir-28_epi.nii
The underscore seems to be a delimiter. In that case you can use
echo "sub-285345_task-WM_dir-28_epi.nii" | cut -d"_" -f1,3-
While using Sed to search/ insert a config file, I'm greeted by errors. What's causing them, and how can I fix them?
The Heredoc I'm looking to insert can be defined as follows:
read -d '' APPLICATION_ENV_STATE <<'EOF'
defined('APPLICATION_ENV') || define('APPLICATION_ENV',(getenv('APPLICATION_ENV')
? getenv('APPLICATION_ENV') : 'production'));
EOF
While my Sed command uses the variable like this:
sed -i "/\/\/ \*\* MySQL settings \*\* \/\//i$APPLICATION_ENV_STATE" wp-config.php
Which results in:
sed: -e expression #1, char 1: unknown command: `?'
In addition to an extra characters after command error.
However, the following Heredoc works, but results in some less than pretty formatting in my text file:
read -d '' APPLICATION_ENV_STATE <<'EOF'
defined('APPLICATION_ENV') || define('APPLICATION_ENV', (getenv('APPLICATION_ENV') ? getenv('APPLICATION_ENV') : 'production'));
EOF
How do I get the first example to work?
AIUI, it's not the heredoc that's the problem, it's understanding which process is doing what at various times.
In your script that runs the sed command, Bash is substituting the variable before sed even sees it. Being a multi-line string, it would need escaping for sed). From the man page for sed, under the i command:
i \ Insert text, which has each embedded newline preceded by a back-slash.
Personally, I'd recommend using cat or echo if you can (or a scriping language like Python / Ruby / PHP), having broken the template up into atomic elements, so you can simply concatenate the relevant pieces together.
If you do want to continue with the current method though, you'll at least need to replace the newlines with backslashed newlines - try something like:
echo $APPLICATION_ENV_STATE | sed 's/$/\\/'
You're using the wrong tool. The only constructs you should be using in sed are s, g, and p (with -n). Just use awk and avoid all the quoting/escaping nonsense:
$ cat file
foo
// ** MySQL settings ** //
bar
$ awk -v app="defined('APPLICATION_ENV') || define('APPLICATION_ENV',(getenv('APPLICATION_ENV')
? getenv('APPLICATION_ENV') : 'production'));" '
{print} index($0,"// ** MySQL settings ** //"){print app}' file
foo
// ** MySQL settings ** //
defined('APPLICATION_ENV') || define('APPLICATION_ENV',(getenv('APPLICATION_ENV')
? getenv('APPLICATION_ENV') : 'production'));
bar
Notice that you don't need to escape the RE metachars in the string you want to search for because awk can treat a string as a string and you don't need to escape newlines in the string you're adding and you don't need a here doc with a shell variable, etc.
Your read/sed command as written would fail for various character combinations in your search and/or replacement strings - see Is it possible to escape regex metacharacters reliably with sed for how to robustly search/replace "strings" in sed but then just use awk so you don't have to worry about any of it.