I have some app config file tmp.cfg. And need to change some given values inside.
Here are the string examples:
app-stat!error!25871a5f-9f50-40ac-923d-c80a660fe21d!1!2
app-stat!queued!25871a5f-9f50-40ac-923d-c80a660fe21d!5!10
app-stat!error!fbbf0e80-8a21-4ebf-9a78-b1017c58a19d!1!2
app-stat!error!5670b363-6a5d-4fcd-819e-85786c5957f1!120!200
For all strings that contains
!error! then following some GUID and then values !1!2 change to
!error! then preserve some GUID and then NEW values !7!10
I do not need to touch other string that contains !error! then GUID but different values in the end
Here what I've tried:
sed -i "s/error\!.*\!1\!2/error\!.*\!4\!8/g" tmp.cfg
It finds all string that I need but replaces a GUID actually with symbols .* instead of GUID number itself.
How to build sed expression in that way to preserve the wildcard part?
The expected result is:
app-stat!error!fbbf0e80-8a21-4ebf-9a78-b1017c58a19d!4!8
The actual result is:
app-stat!error!.*!4!8
sed 's/\(!error!.*\)!1!2/\1!4!8/g' file
Guess you need something like this.
Pattern enclosed within
\( ... \)
are saved in registers for later use and can be accessed as \1, \2 … upto \9.
In the above sed expression, pattern from !error!<GUID> is captured in \1 and used while replacing as \1!4!8.
You can omit g from the sed expression if you are sure that the same pattern won't occur twice on a line.
This is easy to do with awk
awk '$2=="error" && $4==1 && $5==2 {$4=7;$5=10}1' FS="!" OFS="!" file
app-stat!error!25871a5f-9f50-40ac-923d-c80a660fe21d!7!10
app-stat!queued!25871a5f-9f50-40ac-923d-c80a660fe21d!5!10
app-stat!error!fbbf0e80-8a21-4ebf-9a78-b1017c58a19d!7!10
app-stat!error!5670b363-6a5d-4fcd-819e-85786c5957f1!120!200
Separate fields by !
Then if field 2=error, filed 4=1 and field 5=1
Set field 4 and 5 to 7 and 10
1 do print the lines
This sed command should work:
sed -r 's/(.*)!error!(.*)!1!2$/\1!error!\2!4!8/g' file_name
Related
I have the following string:
|**barak**.version|2001.0132012031539|
in file text.txt.
I would like to replace it with the following:
|**barak**.version|2001.01.2012031541|
So I run:
sed -i "s/\|\*\*$module\*\*.version\|2001.0132012031539/|**$module**.version|$version/" text.txt
but the result is a duplicate instead of replacing:
|**barak**.version|2001.01.2012031541|**barak**.version|2001.0132012031539|
What am I doing wrong?
Here is the value for module and version:
$ echo $module
barak
$ echo $version
2001.01.2012031541
Assumptions:
lines of interest start and end with a pipe (|) and have one more pipe somewhere in the middle of the data
search is based solely on the value of ${module} existing between the 1st/2nd pipes in the data
we don't know what else may be between the 1st/2nd pipes
the version number is the only thing between the 2nd/3rd pipes
we don't know the version number that we'll be replacing
Sample data:
$ module='barak'
$ version='2001.01.2012031541'
$ cat text.txt
**barak**.version|2001.0132012031539| <<<=== leave this one alone
|**apple**.version|2001.0132012031539|
|**barak**.version|2001.0132012031539| <<<=== replace this one
|**chuck**.version|2001.0132012031539|
|**barak**.peanuts|2001.0132012031539| <<<=== replace this one
One sed solution with -Extended regex support enabled and making use of a capture group:
$ sed -E "s/^(\|[^|]*${module}[^|]*).*/\1|${version}|/" text.txt
Where:
\| - first occurrence (escaped pipe) tells sed we're dealing with a literal pipe; follow-on pipes will be treated as literal strings
^(\|[^|]*${module}[^|]*) - first capture group that starts at the beginning of the line, starts with a pipe, then some number of non-pipe characters, then the search pattern (${module}), then more non-pipe characters (continues up to next pipe character)
.* - matches rest of the line (which we're going to discard)
\1|${version}| - replace line with our first capture group, then a pipe, then the new replacement value (${version}), then the final pipe
The above generates:
**barak**.version|2001.0132012031539|
|**apple**.version|2001.0132012031539|
|**barak**.version|2001.01.2012031541| <<<=== replaced
|**chuck**.version|2001.0132012031539|
|**barak**.peanuts|2001.01.2012031541| <<<=== replaced
An awk alternative using GNU awk:
awk -v mod="$module" -v vers="$version" -F \| '{ OFS=FS;split($2,map,".");inmod=substr(map[1],3,length(map[1])-4);if (inmod==mod) { $3=vers } }1' file
Pass two variables mod and vers to awk using $module and $version. Set the field delimiter to |. Split the second field into array map using the split function and using . as the delimiter. Then strip the leading and ending "**" from the first index of the array to expose the module name as inmod using the substr function. Compare this to the mod variable and if there is a match, change the 3rd delimited field to the variable vers. Print the lines with short hand 1
Pipe is only special when you're using extended regular expressions: sed -E
There's no reason why you need extended here, stick with basic regex:
sed "
# for lines matching module.version
/|\*\*$module\*\*.version|/ {
# replace the version
s/|2001.0132012031539|/|$version|/
}
" text.txt
or as an unreadable one-liner
sed "/|\*\*$module\*\*.version|/ s/|2001.0132012031539|/|$version|/" text.txt
I want to search for a string and then from that string i want to replace 10 characters with another 10 characters.
For Example,
/prd/edm/hadoop/ifrs/eglex/hdata/ifrs_sri_open/eglex_*_txnacbal/ods=2020_02_23/
i want to search for string "ods=" and replace "2020_02_23" with "2020_02_30"
Since that date "2020_02_23" is not consistent i wanted to search with "ods=" which is static and one time for a line.
Like this more lines are there in the file.
I tried:
cat dta_1.sh | sed 's/.*ods=//' | cut -c1-10
I want to search for string "ods=" and replace "2020_02_23" with
"2020_02_30"
sed 's|\(.*ods=\).*|\12020_02_30/|g' inputfile
The regex inside the parenthesis \( \) is reproduced by \1, and the string after ods= is replaced by 2020_02_30/.
If there could be something else besides / after the date, then go for this:
sed 's|\(.*ods=\)...._.._..|\12020_02_30|g' inputfile
I've completed this using the below,
sed -i '/ods=/ s|ods="[^"]*"|ods='"${DATE_FORMAT2}"'|g' dta_2.sh
Thanks all for the response
I have a string like this:
/var/cpanel/users/joebloggs:DNS9=domain.example
I need to extract the username (joebloggs) from this string and store it in a variable.
The format of the string will always be the same with exception of joebloggs and domain.example so I am thinking the string can be split twice using cut?
The first split would split by : and we would store the first part in a variable to pass to the second split function.
The second split would split by / and store the last word (joebloggs) into a variable
I know how to do this in PHP using arrays and splits but I am a bit lost in bash.
To extract joebloggs from this string in bash using parameter expansion without any extra processes...
MYVAR="/var/cpanel/users/joebloggs:DNS9=domain.example"
NAME=${MYVAR%:*} # retain the part before the colon
NAME=${NAME##*/} # retain the part after the last slash
echo $NAME
Doesn't depend on joebloggs being at a particular depth in the path.
Summary
An overview of a few parameter expansion modes, for reference...
${MYVAR#pattern} # delete shortest match of pattern from the beginning
${MYVAR##pattern} # delete longest match of pattern from the beginning
${MYVAR%pattern} # delete shortest match of pattern from the end
${MYVAR%%pattern} # delete longest match of pattern from the end
So # means match from the beginning (think of a comment line) and % means from the end. One instance means shortest and two instances means longest.
You can get substrings based on position using numbers:
${MYVAR:3} # Remove the first three chars (leaving 4..end)
${MYVAR::3} # Return the first three characters
${MYVAR:3:5} # The next five characters after removing the first 3 (chars 4-9)
You can also replace particular strings or patterns using:
${MYVAR/search/replace}
The pattern is in the same format as file-name matching, so * (any characters) is common, often followed by a particular symbol like / or .
Examples:
Given a variable like
MYVAR="users/joebloggs/domain.example"
Remove the path leaving file name (all characters up to a slash):
echo ${MYVAR##*/}
domain.example
Remove the file name, leaving the path (delete shortest match after last /):
echo ${MYVAR%/*}
users/joebloggs
Get just the file extension (remove all before last period):
echo ${MYVAR##*.}
example
NOTE: To do two operations, you can't combine them, but have to assign to an intermediate variable. So to get the file name without path or extension:
NAME=${MYVAR##*/} # remove part before last slash
echo ${NAME%.*} # from the new var remove the part after the last period
domain
Define a function like this:
getUserName() {
echo $1 | cut -d : -f 1 | xargs basename
}
And pass the string as a parameter:
userName=$(getUserName "/var/cpanel/users/joebloggs:DNS9=domain.example")
echo $userName
What about sed? That will work in a single command:
sed 's#.*/\([^:]*\).*#\1#' <<<$string
The # are being used for regex dividers instead of / since the string has / in it.
.*/ grabs the string up to the last backslash.
\( .. \) marks a capture group. This is \([^:]*\).
The [^:] says any character _except a colon, and the * means zero or more.
.* means the rest of the line.
\1 means substitute what was found in the first (and only) capture group. This is the name.
Here's the breakdown matching the string with the regular expression:
/var/cpanel/users/ joebloggs :DNS9=domain.example joebloggs
sed 's#.*/ \([^:]*\) .* #\1 #'
Using a single Awk:
... | awk -F '[/:]' '{print $5}'
That is, using as field separator either / or :, the username is always in field 5.
To store it in a variable:
username=$(... | awk -F '[/:]' '{print $5}')
A more flexible implementation with sed that doesn't require username to be field 5:
... | sed -e s/:.*// -e s?.*/??
That is, delete everything from : and beyond, and then delete everything up until the last /. sed is probably faster too than awk, so this alternative is definitely better.
Using a single sed
echo "/var/cpanel/users/joebloggs:DNS9=domain.example" | sed 's/.*\/\(.*\):.*/\1/'
I like to chain together awk using different delimitators set with the -F argument. First, split the string on /users/ and then on :
txt="/var/cpanel/users/joebloggs:DNS9=domain.com"
echo $txt | awk -F"/users/" '{print$2}' | awk -F: '{print $1}'
$2 gives the text after the delim, $1 the text before it.
I know I'm a little late to the party and there's already good answers, but here's my method of doing something like this.
DIR="/var/cpanel/users/joebloggs:DNS9=domain.example"
echo ${DIR} | rev | cut -d'/' -f 1 | rev | cut -d':' -f1
I have a string which i need to insert at a specific position in a file :
The file contains multiple semicolons(;) i need to insert the string just before the last ";"
Is this possible with SED ?
Please do post the explanation with the command as I am new to shell scripting
before :
adad;sfs;sdfsf;fsdfs
string = jjjjj
after
adad;sfs;sdfsf jjjjj;fsdfs
Thanks in advance
This might work for you:
echo 'adad;sfs;sdfsf;fsdfs'| sed 's/\(.*\);/\1 jjjjj;/'
adad;sfs;sdfsf jjjjj;fsdfs
The \(.*\) is greedy and swallows the whole line, the ; makes the regexp backtrack to the last ;. The \(.*\) make s a back reference \1. Put all together in the RHS of the s command means insert jjjjj before the last ;.
sed 's/\([^;]*\)\(;[^;]*;$\)/\1jjjjj\2/' filename
(substitute jjjjj with what you need to insert).
Example:
$ echo 'adad;sfs;sdfsf;fsdfs;' | sed 's/\([^;]*\)\(;[^;]*;$\)/\1jjjjj\2/'
adad;sfs;sdfsfjjjjj;fsdfs;
Explanation:
sed finds the following pattern: \([^;]*\)\(;[^;]*;$\). Escaped round brackets (\(, \)) form numbered groups so we can refer to them later as \1 and \2.
[^;]* is "everything but ;, repeated any number of times.
$ means end of the line.
Then it changes it to \1jjjjj\2.
\1 and \2 are groups matched in first and second round brackets.
For now, the shorter solution using sed : =)
sed -r 's#;([^;]+);$#; jjjjj;\1#' <<< 'adad;sfs;sdfsf;fsdfs;'
-r option stands for extented Regexp
# is the delimiter, the known / separator can be substituted to any other character
we match what's finishing by anything that's not a ; with the ; final one, $ mean end of the line
the last part from my explanation is captured with ()
finally, we substitute the matching part by adding "; jjjj" ans concatenate it with the captured part
Edit: POSIX version (more portable) :
echo 'adad;sfs;sdfsf;fsdfs;' | sed 's#;\([^;]\+\);$#; jjjjj;\1#'
echo 'adad;sfs;sdfsf;fsdfs;' | sed -r 's/(.*);(.*);/\1 jjjj;\2;/'
You don't need the negation of ; because sed is by default greedy, and will pick as much characters as it can.
sed -e 's/\(;[^;]*\)$/ jjjj\1/'
Inserts jjjj before the part where a semicolon is followed by any number of non-semicolons ([^;]*) at the end of the line $. \1 is called a backreference and contains the characters matched between \( and \).
UPDATE: Since the sample input has no longer a ";" at the end.
Something like this may work for you:
echo "adad;sfs;sdfsf;fsdfs"| awk 'BEGIN{FS=OFS=";"} {$(NF-1)=$(NF-1) " jjjjj"; print}'
OUTPUT:
adad;sfs;sdfsf jjjjj;fsdfs
Explanation: awk starts with setting FS (field separator) and OFS (output field separator) as semi colon ;. NF in awk stands for number of fields. $(NF-1) thus means last-1 field. In this awk command {$(NF-1)=$(NF-1) " jjjjj" I am just appending jjjjj to last-1 field.
I have an input stream like this:
afs=1;bgd=1;cgd=1;djh=1;fgjhh=1;
Now the rule I have to edit the stream is:
(1)if we have
"djh=number;"
replace it with
"djh=number,"
(2)else replace "string=number;"it with
"string,"
I can handle case 2 as:
sed 's/afs=1/afs,/g;s/dbg=1/dbg,/g;..... so on for rest
How to take care for condition 1?
The "djh" number can be any number(1,12,100), the other numbers are always 1.
all the double quotes I have used are for reference only; no double quotes are present in the input stream. "afs" can be "Afs" also.
Thanks in advance.
sed -e 's/;/,/g; s/,djh=/,#=/; s/\([a-z][a-z]*\)=[0-9]*,/\1,/g; s/#/djh/g'
This does the following
replace all ; by ,
replace djh with #
remove =number from all lower cased strings
replace # with djh
This results in afs,bgd,cgd,djh=1,fgjhh, for your input. Of course you could substitute djh with any other character that makes it easy to match the other strings. This is just illustrating the idea.
echo 'afs=1;bgd=1;cgd=1;djh=1;fgjhh=1;' |
sed -e 's/\(djh=[0-9]\+\);/\1,/g' -e 's/\([a-zA-Z0-9]\+\)=1;/\1,/g'
This might work for you:
echo "afs=1;bgd=1;cgd=1;djh=1;fgjhh=1;" |
sed 's/^/\n/;:a;/\n\(djh=[0-9]*\);/s//\1,\n/;ta;s/\n\([^=]*\)=1;/\1,\n/;ta;s/.$//'
afs,bgd,cgd,djh=1,fgjhh,
Explanation:
This method uses a unique marker (\n is a good choice because it cannot appear in the pattern space as it is used by sed as the line delimiter) as anchor for comparing throughout the input string. It is slow but can scale if more than one exception is needed.
Place the marker in front of the string s/^/\n/
Name a loop label :a
Match the exception(s) /\n\(djh=[0-9]*\)/
If the exception occurs substitute as necessary. Also bump the marker along /s//\1,\n/
If the above is true break to loop label ta
Match the normal and substitute. Also bump the marker along s/\n\([^=]*\)=1;/\1,\n/
If the above is true break to loop label ta
All done remove the marker s/.$//
or:
echo "afs=1;bgd=1;cgd=1;djh=1;fgjhh=1;" |
sed 's/\<djh=/\n/g;s/=[^;]*;/,/g;s/\n\([^;]*\);/djh=\1,/g'
afs,bgd,cgd,djh=1,fgjhh,
Explanation:
This is fast but does not scale for multiple exceptions:
Globaly replace the exception string with a new line s/\<djh=/\n/g
Globaly replace the normal condition s/=[^;]*;/,/g
Globaly replace the \n by the exception string s/\n\([^;]*\);/djh=\1,/g
N.B. When replacing the exception string make sure that it begins on a word boundary \<