In bash, how can I find and replace some text containing a new line?
I want to match exactly 2 lines as specified (I can't match them separately as both lines appear at different places separately & I only want to replace where both lines appear consecutively). Using sed I was able to find and replace the individual lines and new line separately, but not together!
In case if needed, below are the lines I want to find and replace (from multiple files at once!):
} elseif ($this->aauth->is_member('Default')) {
$form_data['userstat'] = $this->aauth->get_user()->id;
In general you can used sed -z which tells sed to use the null-character to split lines. Assume you have the file text containing
Hello World
This is a line
line1
line2
Hello World, again
line1
line2
end
Executing sed -z -e 's/line1\nline2/xxx/g' text yields
Hello World
This is a line
xxx
Hello World, again
xxx
end
You can add * (that is <space><star>) to handle inconsistent white spaces.
In your specific case if you want to delete the second line you can use a block statement to advance to the next line and delete it if it matches the second line
sed -e '/line1/{n;/line2/d}' text
This might work for you (GNU sed):
sed -i 'N;s/first line\nsecond line/replacement/;P;D' file ...
Keep a moving window of two lines in the pattern space and replace when necessary.
N.B. -i option updates file(s) in place.
Also using a range and the change command:
sed -i '/first line/,/second line/c\replacement1\nreplacement2\netc' file ...
Related
How can you refresh the line numbers of a sed output inside the same sed command?
I have a sed script as follows -
#!/usr/bin/sed -f
/pattern/i #inserting a line
1~10i ####
What this does is that it inserts lines wherever the pattern is matched and then inserts #### every ten lines. The problem is that it inserts the hashes every 10 lines according to the line numbers of the original file before inserting the lines for the matching pattern. I want to refresh the line numbers after inserting the lines and use them for inserting the 4 hashes every 10 lines.
Anyway this can be done without piping the output into a new sed?
Interesting challenge. If your file is not too large, the following may work for you (tested with GNU sed):
#!/usr/bin/sed -nEf
:a; N; $!ba
{
s/([^\n]*pattern[^\n]*\n)/#inserting a line\n\1/g
s/\n/ \n/g
s/\`/####\n/
:b
s/(.*####\n([^\n]* \n){9}[^\n]*) \n/\1\n####\n/
tb
s/ \n/\n/g
p
}
Explanations, line by line:
No print, extended RE mode (-nE).
Loop around label a to concatenate the whole file in the pattern space (reason why its size matters).
Add #inserting a line\n before each line containing pattern.
Add a space before all endline characters.
Insert ####\n before the first line.
Label b.
Append ####\n' to anything followed by ####\n` and 10 space-terminated lines, removing the final space (to prevent subsequent matches).
Goto b if there was a substitution.
Remove all spaces at the end of a line.
print.
Note: if your file does not contain NUL characters the -z option of GNU sed saves a few commands:
#!/usr/bin/sed -Ezf
s/([^\n]*pattern[^\n]*\n)/#inserting a line\n\1/g
s/\n/ \n/g
s/\`/####\n/
:a
s/(.*####\n([^\n]* \n){9}[^\n]*) \n/\1\n####\n/
ta
s/ \n/\n/g
Note: with the hold space we could probably do the same on the fly, instead of storing the whole file in the pattern space.
This might work for you (GNU sed):
sed -zE 's/.*pattern/# insert line\n&/mg
s/([^\n]*\n){10}/&####\n/g
s/^/####\n/' file
Slurp the file into memory.
Insert desired text before lines containing pattern.
Insert #### every 10 lines and before the first line.
Is there anyway to insert new lines in-between 2 specific patterns of characters? I want to insert a new line every time "butterfly" occurs in a text file, however I want this new line to be inserted between the "butter" and "fly". For example butter\nfly
I also want to find the length of each line after splitting.
Eg:
if textfile contains:
fgsccgewvdhbejbecbecboubutterflybvdcvhkebcjl
vdjchvhecbihbutterflyglehblejkbedkbutterflyr
Then, I want a result like the following:
29 fgsccgewvdhbejbecbecboubutter
33 flybvdcvhkebcjlvdjchvhecbihbutter
22 flyglehblejkbedkbutter
4 flyr
I believe one way to tackle it would be to insert a new line using "sed" everywhere "butter" occurs and is followed by "fly". Strip out all blank line using grep with a -v flag. Then get the length of each line. However, even after trying a lot, I am unable to get the correct answer.
The Sed 's' sub-command + awk can work together:
sed -e "s/butterfly/butter\\nfly/g" < input.txt | awk '{ print length, $0 }'
This might work for you (GNU sed & bash):
sed -Ez 's/\n//g;s/(butter)(fly)/\1\n\2/g;s/^.*$/l=&;printf "%d %s\n" ${#l} &/meg' file
Slurp the file into memory using the -z sed option. Remove all existing newlines and then insert new ones between butter and fly. Using the m, g and e flags of the sed substitute command, split into separate lines and using bash make a variable l and via printf print the required format.
I have to deal with an output from a command. Just plain text. What I want is to remove unnecessary spaces by using
sed 's/ */ /g'
But it's doing it wrong - it seems to remove new line characters as well...
The other problem is about the $ character representing the 'new line'.
When I'm writing sth like this:
sed 's/$/FOO/g'
It is really replacing all the new line chars with the word FOO.
So its basically like:
text
text
text
Is converted to
textFOOtextFOOtext
The problem starts when 2 new lines occur. The text:
text
text
Is converted to:
textFOOFOOtext
BUT -- absolutely no SED line is able to convert the 2 new lines into one. I tried everything I found on the web.
How do I remove that additional new line?
Check if this helps:
tr -s ' ' < File
This will change multiple spaces to single space, if thats what u want
I need to insert the text at each line only if the given pattern matches with that line.
For example,
sed -n '/pattern/p' /etc/inittab/
so, if the pattern matches with any of the lines in inittab file, then i need to insert '#' at the beginning of those lines in the same file itself.
Kindly suggest me, how to make this.
Using sed:
sed '/pattern/s/^/#/' file
This will look for lines matching the pattern and once it finds it, it will place # in front of it. This will not modify the file. In order to do so, you need to use -i option to make in-place changes. You can put an extension like -i.bak to make an optional back if you'd like.
Using awk:
awk '/pattern/{$0="#"$0}1' file
awk is made up by pattern action statements. For the matching pattern, the action we do is modify the line by placing # in front of it. The 1 at the end will print the lines for us. GNU awk v4.1 or later has in-place editing just like sed. If you are using an older version you can redirect the output to another file and mv it back to original by saying:
awk '/pattern/{$0="#"$0}1' file > tmp && mv tmp file
The in-place changes is nothing special. It does the same job as redirecting to a temp file and then moving it back. It just does all the dirty work for you behind the scenes.
This is achieved with the following sed invocation
% sed -i.orig -e '/pattern/s/^/#/' inittab
The -i.orig option tells sed to operate in place on the file, previously saving the original as inittab.orig. The editing pattern
/pattern/ selects lines matching pattern
s/^/#/ and substitute the empty word at the beginning of line with #
Using a bash script, I am trying to insert a line in a file (eventually there will be 4 extra lines, one after the other).
I am trying to implement the answer by iiSeymour to the thread:
Insert lines in a file starting from a specific line
which I think is the same comment that dgibbs made in his own thread:
Bash: Inserting a line in a file at a specific location
The line after which I want to insert the new text is very long, so I save it in a variable first:
field1=$(head -2 file847script0.xml | tail -1)
The text I want to insert is:
insert='newtext123'
When running:
sed -i".bak" "s/$field1/$field1\n$insert/" file847script0.xml
I get the error:
sed: 1: "s/<ImageAnnotation xmln ...": bad flag in substitute command: 'c'
I also tried following the thread
sed throws 'bad flag in substitute command'
but the command
sed -i".bak" "s/\/$field1/$field1\n$insert/" file847script0.xml
still gives me the same error:
sed: 1: "s/\/<ImageAnnotation xm ...": bad flag in substitute command: 'c'
I am using a Mac OS X 10.5.
Any idea of what am I doing wrong? Thank you!
Good grief, just use awk. No need to worry about special characters in your replacement text or random single-character commands and punctuation.
In this case it looks like all you need is to print some new text after the 2nd line so that's just:
$ cat file
a
b
c
$ insert='absolutely any text you want, including newlines
slashes (/), backslashes (\\), whatever...'
$ awk -v insert="$insert" '{print} NR==2{print insert}' file
a
b
absolutely any text you want, including newlines
slashes (/), backslashes (\), whatever...
c
Isn't it easier to do it by line number? If you know it's the second line or the nth line (and grep will tell you line numbers if you are pattern matching) then you can simply use sed to find the correct line and then append a new line (or 4 new lines).
cat <<EOF > testfile
one two three
four five six
seven eight nine
EOF
sed -re '2a\hello there' testfile
will output
one two three
four five six
hello there
seven eight nine