I have files like:
text2insert
filewithpattern
and also known:
pattern
How can i insert lines from text2insert into filewithpattern but after the pattern line?
Using bash 2.05b
UPDATE:
Before filewithpattern should look like:
garbage
pattern1
pattern2
garbage
and after:
garbage
pattern1
text2insert lines
text2insert lines
text2insert lines
pattern2
garbage
sed -e '/pattern/r text2insert' filewithpattern
awk 'FNR==NR{ a[c++]=$0;next }
/pattern1/{g=1;next}
/pattern2/{g=0;next}
g{
for(i=1;i<=c;i++){
print a[i]
}
}' text2insert filewithpattern
And why do you need "garbage" in your output? Your previous question seems not to include "garbage"
Related
I am fresher in writing perl scripts,so i am asking this as question or support on this, below is the code
start pattern1
line1
Matching pattern can be here
line2
Matching Pattern can be here
line3
line4
...
end pattern1
.
start pattern1
line1
line2
start pattern1
start pattern1
line1
Matching pattern can be here
line2
start pattern1
so from perl i need to grep the lines between start pattern1 ... end pattern1,
for this i am using awk command to grep
$cmd = q(awk '/start pattern1/,end pattern1 /' x.file );
$n1 = system($cmd);
For this output works fine,Below is the output,
start pattern1
line1
**Matching pattern can be here**
line2
**Matching Pattern can be here**
...
end pattern1
But in the files i have 1000 of lines like this, so i need to grep those lines which have Matching pattern. i.e i need to grep only those starting pattern lines to ending pattern lines has matching pattern
For this i tried
$cmd = q(awk '/start pattern1/,end pattern1 /' x.file | grep '$n2\|line4');
$n1 = system($cmd);
But when i use the above command i don't see any output
Here $n2 contains some pattern which is grepped from another file.
if i use direct matched patternin place of $n2 it works fine, why cant i use $n2 here?
Note:i am using this in perl script
From the Awk command i get all the lines between start pattern1...end pattern1,But i have 1000 of such prints, so i need the bunch of the lines of start pattern1 to end pattern1 of thos which get matched with the matching pattern
The expected output when i do is,
start pattern1
line1
Matching pattern can be here
line2
Matching Pattern can be here
line3
line4
...
end pattern1
start pattern1
line1
Matching pattern can be here
line2
start pattern1
No need to summon awk from within perl since perl is much more powerful than awk.
It's not clear to me whether you want every line between start pattern1 and end pattern1 if there is at least one match inside, or just the matching lines.
If every line between start and end if match:
my #blocks = join("",<>)=~/start pattern1\s*(.*?)end pattern1/gsi;
print grep /matching pattern/i, #blocks;
If every line INCLUDING start|end pattern1:
my #blocks = join("",<>)=~/(start pattern1.*?end pattern1\s*)/gsi;
If just lines with /matching pattern/ between start and end:
print grep { /start pattern1/i../end pattern1/i and /matching pattern/i } <>;
Put that inside a file program.pl and run:
perl program.pl inputfile > outputfile
Some explanation might be needed: join("",<>) returns the whole inputfile as one multi-line string. The /gsi modifiers means: g matches globally so that the #block array will contain what is matched by the parentheses, one array element for each match (without g the #block array would just get the first block of lines), s means that . also matches newline characters which it otherwise wouldn't and i matches by ignoring case (sees no difference between a-z and A-Z letters). The question mark in .*? means no-greedy matching of every character, that is, match until the next end pattern1 and not the last. The <> returns the lines of inputfile (args after perl program.pl) as an array of strings. The .. is the flip-flop operator which is true after the left side becomes true and false after the right side becomes true and stays false until the left side is true again and so on.
I am trying to match nested text, including the line immediately prior to the nested text with sed or grep.
An example of what I'm working with:
pattern3
abcde
fghij
pattern3
pattern1
abcde
fghij
pattern1
pattern1
klmno
pattern1
pattern3
abcde
pattern1
pqrst
patterh3
fghij
Note that there are always four (4) spaces prefixing the nested text. Also, there may or may not be nested text after a matching pattern.
I'm interested in all pattern1 lines, plus the lines following pattern1 that are proceeded by spaces.
The output I'm looking for is:
pattern1
abcde
fghij
pattern1
pattern1
klmno
pattern1
pattern1
pqrst
I got close with:
sed -n '/^pattern1/,/^pattern1/p' data.txt
But it seems to skip nested text after the right hand side pattern1 match, and move onto the next iteration.
I also tried sed -n '/^\"pattern1\"$/,/^\"pattern1\"$/p' data.txt | sed '1d;$d' with no luck either.
With GNU sed:
sed -n '/pattern1/{p;:x;n;s/^ .*/&/;p;tx}' file
or simplified:
sed -n '/pattern1/{p;:x;n;p;/^ /bx}' file
Output:
pattern1
abcde
fghij
pattern1
pattern1
klmno
pattern1
pattern1
pqrst
Could you please try following.
awk '/pattern[23]/{flag=""} /pattern1/{flag=1} flag' Input_file
OR
awk '/pattern[^1]/{flag=""} /pattern1/{flag=1} flag' Input_file
Explanation: Adding explanation too here.
awk '
/pattern[^1]/{ ##Checking condition if a line is having string pattern with apart from digit 1 in it then do following.
flag="" ##Nullifying variable flag value here.
}
/pattern1/{ ##Checking condition here if a line is having string pattern1 then do following.
flag=1 ##Setting value of variable flag as 1 here.
}
flag ##Checking condition if value of flag is NOT NULL then print the line value.
' Input_file ##Mentioning Input_file name here.
$ awk '/^[^ ]/{f=/^pattern1$/} f' file
pattern1
abcde
fghij
pattern1
pattern1
klmno
pattern1
pattern1
pqrst
This might work for you (GNU sed):
sed '/^\S/h;G;/pattern1/P;d' file
Store the current pattern in the hold space and append it to each line. If the current pattern is pattern1, print the current line and/or delete the current line.
I have a big file with content separated by headers.
How would I go about substituting the text between the line after a particular header and the line before a specific string (non-inclusive) in the file via sed or awk?
Example: I want to replace the text between the line after header1 and the line beforestring_near_end_of_file (non-inclusive) with EXAMPLE in the file below...
...
header1
#######
content1
header2
#######
content2
...
string_near_end_of_file
...
Intended outcome:
...
header1
#######
EXAMPLE
string_near_end_of_file
...
I know that sed '/pattern1/,/.. pattern2/c\substition_string' file substitutes substition_string between pattern1 and pattern2 and that sed -n '/pattern/{n;p}' file prints the line after the pattern match but I'm not sure now to put this together to achieve my stated goal.
Any help would be greatly appreciated!
You can use this awk command:
awk '/string_near_end_of_file/{p=0} !p; $1=="header1"{p=1; print "######\nEXAMPLE\n"}' file
Output:
...
header1
######
EXAMPLE
string_near_end_of_file
...
GNU sed:
sed -e '/header1/,/string_near/{/header1/{n;s/$/\nEXAMPLE\n/;n};/string_near/!d}' file
In the range:
/header1/{n; - skip the header1 line
s/$/\nEXAMPLE\n/; - replace the ### end of line with \nEXAMPLE\n
n} leave the ### and EXAMPLE lines alone
/string_near/!d; - delete other lines except string_near
sed sucks for requirements to repeat the /START/,/END/ patterns within the block. You can sometimes use // to mean the last regex, but any nested regexes limit its use in /START/,/END/ ranges.
Note that some sed implementations require semicolons before closing curly braces.
Note also that I use replacement of the line end s/$/\nEXAMPLE\n/ because it is much easier to inline at the command line then it is to use sed's c a or i commands, which require a newline in the command string.
Awk solution (assuming that header line is always followed by ####### line):
awk -v repl="EXAMPLE" '!f && /header/{ f=1; n=NR; getline nl; r=$0 ORS nl }
/string_near_end_of_file/{ f=0; print r ORS repl ORS ORS $0; next }
f && NR>n{ next }1' file
The output:
...
header1
#######
EXAMPLE
string_near_end_of_file
...
Using awk or sed how can I select lines which are occurring between two different marker patterns? There may be multiple sections marked with these patterns.
For example:
Suppose the file contains:
abc
def1
ghi1
jkl1
mno
abc
def2
ghi2
jkl2
mno
pqr
stu
And the starting pattern is abc and ending pattern is mno
So, I need the output as:
def1
ghi1
jkl1
def2
ghi2
jkl2
I am using sed to match the pattern once:
sed -e '1,/abc/d' -e '/mno/,$d' <FILE>
Is there any way in sed or awk to do it repeatedly until the end of file?
Use awk with a flag to trigger the print when necessary:
$ awk '/abc/{flag=1;next}/mno/{flag=0}flag' file
def1
ghi1
jkl1
def2
ghi2
jkl2
How does this work?
/abc/ matches lines having this text, as well as /mno/ does.
/abc/{flag=1;next} sets the flag when the text abc is found. Then, it skips the line.
/mno/{flag=0} unsets the flag when the text mno is found.
The final flag is a pattern with the default action, which is to print $0: if flag is equal 1 the line is printed.
For a more detailed description and examples, together with cases when the patterns are either shown or not, see How to select lines between two patterns?.
Using sed:
sed -n -e '/^abc$/,/^mno$/{ /^abc$/d; /^mno$/d; p; }'
The -n option means do not print by default.
The pattern looks for lines containing just abc to just mno, and then executes the actions in the { ... }. The first action deletes the abc line; the second the mno line; and the p prints the remaining lines. You can relax the regexes as required. Any lines outside the range of abc..mno are simply not printed.
This might work for you (GNU sed):
sed '/^abc$/,/^mno$/{//!b};d' file
Delete all lines except for those between lines starting abc and mno
sed '/^abc$/,/^mno$/!d;//d' file
golfs two characters better than ppotong's {//!b};d
The empty forward slashes // mean: "reuse the last regular expression used". and the command does the same as the more understandable:
sed '/^abc$/,/^mno$/!d;/^abc$/d;/^mno$/d' file
This seems to be POSIX:
If an RE is empty (that is, no pattern is specified) sed shall behave as if the last RE used in the last command applied (either as an address or as part of a substitute command) was specified.
From the previous response's links, the one that did it for me, running ksh on Solaris, was this:
sed '1,/firstmatch/d;/secondmatch/,$d'
1,/firstmatch/d: from line 1 until the first time you find firstmatch, delete.
/secondmatch/,$d: from the first occurrance of secondmatch until the end of file, delete.
Semicolon separates the two commands, which are executed in sequence.
something like this works for me:
file.awk:
BEGIN {
record=0
}
/^abc$/ {
record=1
}
/^mno$/ {
record=0;
print "s="s;
s=""
}
!/^abc|mno$/ {
if (record==1) {
s = s"\n"$0
}
}
using: awk -f file.awk data...
edit: O_o fedorqui solution is way better/prettier than mine.
Don_crissti's answer from Show only text between 2 matching pattern?
firstmatch="abc"
secondmatch="cdf"
sed "/$firstmatch/,/$secondmatch/!d;//d" infile
which is much more efficient than AWK's application, see here.
perl -lne 'print if((/abc/../mno/) && !(/abc/||/mno/))' your_file
I tried to use awk to print lines between two patterns while pattern2 also match pattern1. And the pattern1 line should also be printed.
e.g.
source
package AAA
aaa
bbb
ccc
package BBB
ddd
eee
package CCC
fff
ggg
hhh
iii
package DDD
jjj
should has an ouput of
package BBB
ddd
eee
Where pattern1 is package BBB, pattern2 is package \w*. Note that CCC isn't a known value so can't be literally matched.
In this case, neither #scai 's awk '/abc/{a=1}/mno/{print;a=0}a' file nor #fedorqui 's awk '/abc/{a=1} a; /mno/{a=0}' file works for me.
Finally, I managed to solve it by awk '/package BBB/{flag=1;print;next}/package \w*/{flag=0}flag' file, haha
A little more effort result in awk '/package BBB/{flag=1;print;next}flag;/package \w*/{flag=0}' file, to print pattern2 line also, that is,
package BBB
ddd
eee
package CCC
This can also be done with logical operations and increment/decrement operations on a flag:
awk '/mno/&&--f||f||/abc/&&f++' file
Using awk or sed how can I select lines which are occurring between two different marker patterns? There may be multiple sections marked with these patterns.
For example:
Suppose the file contains:
abc
def1
ghi1
jkl1
mno
abc
def2
ghi2
jkl2
mno
pqr
stu
And the starting pattern is abc and ending pattern is mno
So, I need the output as:
def1
ghi1
jkl1
def2
ghi2
jkl2
I am using sed to match the pattern once:
sed -e '1,/abc/d' -e '/mno/,$d' <FILE>
Is there any way in sed or awk to do it repeatedly until the end of file?
Use awk with a flag to trigger the print when necessary:
$ awk '/abc/{flag=1;next}/mno/{flag=0}flag' file
def1
ghi1
jkl1
def2
ghi2
jkl2
How does this work?
/abc/ matches lines having this text, as well as /mno/ does.
/abc/{flag=1;next} sets the flag when the text abc is found. Then, it skips the line.
/mno/{flag=0} unsets the flag when the text mno is found.
The final flag is a pattern with the default action, which is to print $0: if flag is equal 1 the line is printed.
For a more detailed description and examples, together with cases when the patterns are either shown or not, see How to select lines between two patterns?.
Using sed:
sed -n -e '/^abc$/,/^mno$/{ /^abc$/d; /^mno$/d; p; }'
The -n option means do not print by default.
The pattern looks for lines containing just abc to just mno, and then executes the actions in the { ... }. The first action deletes the abc line; the second the mno line; and the p prints the remaining lines. You can relax the regexes as required. Any lines outside the range of abc..mno are simply not printed.
This might work for you (GNU sed):
sed '/^abc$/,/^mno$/{//!b};d' file
Delete all lines except for those between lines starting abc and mno
sed '/^abc$/,/^mno$/!d;//d' file
golfs two characters better than ppotong's {//!b};d
The empty forward slashes // mean: "reuse the last regular expression used". and the command does the same as the more understandable:
sed '/^abc$/,/^mno$/!d;/^abc$/d;/^mno$/d' file
This seems to be POSIX:
If an RE is empty (that is, no pattern is specified) sed shall behave as if the last RE used in the last command applied (either as an address or as part of a substitute command) was specified.
From the previous response's links, the one that did it for me, running ksh on Solaris, was this:
sed '1,/firstmatch/d;/secondmatch/,$d'
1,/firstmatch/d: from line 1 until the first time you find firstmatch, delete.
/secondmatch/,$d: from the first occurrance of secondmatch until the end of file, delete.
Semicolon separates the two commands, which are executed in sequence.
something like this works for me:
file.awk:
BEGIN {
record=0
}
/^abc$/ {
record=1
}
/^mno$/ {
record=0;
print "s="s;
s=""
}
!/^abc|mno$/ {
if (record==1) {
s = s"\n"$0
}
}
using: awk -f file.awk data...
edit: O_o fedorqui solution is way better/prettier than mine.
Don_crissti's answer from Show only text between 2 matching pattern?
firstmatch="abc"
secondmatch="cdf"
sed "/$firstmatch/,/$secondmatch/!d;//d" infile
which is much more efficient than AWK's application, see here.
perl -lne 'print if((/abc/../mno/) && !(/abc/||/mno/))' your_file
I tried to use awk to print lines between two patterns while pattern2 also match pattern1. And the pattern1 line should also be printed.
e.g.
source
package AAA
aaa
bbb
ccc
package BBB
ddd
eee
package CCC
fff
ggg
hhh
iii
package DDD
jjj
should has an ouput of
package BBB
ddd
eee
Where pattern1 is package BBB, pattern2 is package \w*. Note that CCC isn't a known value so can't be literally matched.
In this case, neither #scai 's awk '/abc/{a=1}/mno/{print;a=0}a' file nor #fedorqui 's awk '/abc/{a=1} a; /mno/{a=0}' file works for me.
Finally, I managed to solve it by awk '/package BBB/{flag=1;print;next}/package \w*/{flag=0}flag' file, haha
A little more effort result in awk '/package BBB/{flag=1;print;next}flag;/package \w*/{flag=0}' file, to print pattern2 line also, that is,
package BBB
ddd
eee
package CCC
This can also be done with logical operations and increment/decrement operations on a flag:
awk '/mno/&&--f||f||/abc/&&f++' file