I've a unix text file and I want to search from a specific pattern. When I find the first occurrence of the pattern, from that position onward, I want to read complete file till end.
How I can achieve this via bash commands.
Regards,
DKamran
Something like this perhaps?
seq 15 | sed '0,/4/d'
Will delete up to the first 4 and print the rest.
perl -ne 'if (/your_pattern/..EOF){print}' your_file.txt
This uses the flip flop operator so that it ignores every line until your pattern is matched. Until it reaches the end of the file, it prints each line.
awk variant:
$ seq 1 15 | awk '/6/{t=1}t'
6
7
8
9
10
11
12
13
14
15
grep for the pattern Using the -n option to get the line number. Cut out the line number using awk or cut and then use tail with the -n option and a + suffixed with the line number you've obtained to give you the file from that point onwards.
Related
I have a log file which contains several repeats of the pattern Fre --. I need to remove only first occurrence of this pattern and the next 20 lines after that and keep other matches intact. I need to do it in a bash terminal, using sed preferably or awk or perl. I would highly appreciate your help.
I tried
sed -e '/Fre --/,+20d' log.log
but it deletes all the patterns and next 20 lines after that. I want only first pattern to be removed
There is a more or less similar question and some answers here: How to remove only the first occurrence of a line in a file using sed but I don't know how to change it to remove 20 lines after the first match
Pretty sure that someone will find a nice sed command but I know awk better.
You can try :
awk '/Fre --/ && !found++{counter=21}--counter<0' log.log
Explanations :
/Fre --/ -> if it finds pattern Fre --
&& !found++ -> and if it didn't find it before
{counter=21} -> it sets counter value at 21 (because you want to remove the line + the next 20s)
--counter<0 -> decreases the counter and prints the line only if counter < 0
As mentioned by #Sundeep, #EdMorton solution is safer on very big files.
awk '/Fre --/ && !found++{counter=21}!(counter&&counter--)' log.log
NOTE
If you want the deletions to be saved into the original file, you will have to copy the contents of the awk command into a temp file, and then move the temp file into the original file. Always be careful before editing the original file since you may lose precious informations.
Run the first command first :
awk '/Fre --/ && !found++{counter=21}!(counter&&counter--)' log.log > log.log.tmp
Then check the .tmp file and you can run the second command to apply the changes if .tmp file looks ok :
mv log.log.tmp log.log
$ seq 20 | awk '!f && /3/{c=4; f=1} !(c&&c--)'
1
2
7
8
9
10
11
12
13
14
15
16
17
18
19
20
See Printing with sed or awk a line following a matching pattern
I used following script to search every line of one file in another file and if it is found printing 2nd column of that line :
#!/bin/csh
set goldFile=$1
set regFile=$2
set noglob
foreach line ("`cat $goldFile`")
set searchString=`echo $line | awk '{print $1}'`
set id=`grep -w -F "$searchString" $regFile | awk '{print $2}'`
echo "$searchString" "and" "$id"
end
unset noglob
Gold file is as follows :
\$#%$%escaped.Integer%^^&[10]
\$#%$%escaped.Integer%^^&[10][0][0][31]
\$#%$%escaped.Integer%^^&[10][0][0][30]
\$#%$%escaped.Integer%^^&[10][0][0][29]
\$#%$%escaped.Integer%^^&[10][0][0][28]
\$#%$%escaped.Integer%^^&[10][0][0][27]
\$#%$%escaped.Integer%^^&[10][0][0][26]
and RegFile is as follows :
\$#%$%escaped.Integer%^^&[10] 1
\$#%$%escaped.Integer%^^&[10][0][0][31] 10
\$#%$%escaped.Integer%^^&[10][0][0][30] 11
\$#%$%escaped.Integer%^^&[10][0][0][29] 12
\$#%$%escaped.Integer%^^&[10][0][0][28] 13
\$#%$%escaped.Integer%^^&[10][0][0][27] 14
\$#%$%escaped.Integer%^^&[10][0][0][26] 15
Output is coming :
\$#%$%escaped.Integer%^^&[10] and 1 10 11 12 13 14 15
\$#%$%escaped.Integer%^^&[10][0][0][31] and 10
\$#%$%escaped.Integer%^^&[10][0][0][30] and 11
\$#%$%escaped.Integer%^^&[10][0][0][29] and 12
\$#%$%escaped.Integer%^^&[10][0][0][28] and 13
\$#%$%escaped.Integer%^^&[10][0][0][27] and 14
\$#%$%escaped.Integer%^^&[10][0][0][26] and 15
But expected Output is :
\$#%$%escaped.Integer%^^&[10] and 1
\$#%$%escaped.Integer%^^&[10][0][0][31] and 10
\$#%$%escaped.Integer%^^&[10][0][0][30] and 11
\$#%$%escaped.Integer%^^&[10][0][0][29] and 12
\$#%$%escaped.Integer%^^&[10][0][0][28] and 13
\$#%$%escaped.Integer%^^&[10][0][0][27] and 14
\$#%$%escaped.Integer%^^&[10][0][0][26] and 15
Please help me to figure out how to search exact word having some special character using grep.
csh and bash are completely different variants of shell. They're not even supposed to be compatible. Your problem is more associated with usage of grep
Because of the -F flag in grep which lets your string to be fixed pattern, prone to contain all sorts of regex special characters like ,,[],(),.,*,^,$,-,\
The error result is because the -F flag, the line \$#%$%escaped.Integer%^^&[10] in Gold file matches all the input lines on the RegFile.
So normally the exact words of search can be filtered by the word boundary constructs ^ and $ as part of the pattern, but it won't work in your case because of the -F, --fixed-strings flag they will be treated as being part of the search string.
So assuming from the input file, there could be only one match for each line in the Gold file to RegFile you could stop the grep search after the first hit
Using the -m1 flag, which according to the man grep page says,
-m NUM, --max-count=NUM
Stop reading a file after NUM matching lines. If the input is standard input
from a regular file, and NUM matching lines are output, grep ensures that the
standard input is positioned to just after the last matching line before
exiting, regardless of the presence of trailing context lines.
So adding it like,
grep -w -F -m1 "$searchString" $regFile
should solve your problem.
I want to replace the first value (in first column and line so here 1) and add one to this value, so I have a file like this
1
1 1
2 5
1 6
I use this sentence
read -r a < file
echo $aa
sed "s/$aa/$(($aa + 1))/" file
# or
sed 's/$aa/$(($aa + 1))/' file
But when I make that, he change all first column one into two. I have try to change the quote but it make nothing.
restrict the script to first line only, i.e.
sed '1s/old/new/'
awk might be a better tool for this.
awk 'NR==1{$1=$1+1}1'
for the first line add 1 to the first field and print. Can be rewritten as
awk 'NR==1{$1+=1}1'
or
awk 'NR==1{$1++}1'
perl -p0e 's/(\d+)/$1+1/e' file
I have a set of text files and I'd like to display lines 12-14 by running a bash script on each file.
For one of the files, this works:
tail -14 | head -11
But since other files have different lengths, I cannot run the same script on them.
What is the command I'm looking for to output lines 12-24 of the text file?
Use sed with -n argument
sed -n 12,24p <FILENAME>
For a funny pure Bash (≥4) possibility:
mapfile -t -s 11 -n 13 lines < file
printf '%s\n' "${lines[#]}"
This will skip the first 11 lines (with -s 11) and read 13 lines (with -n 13) and store each line in a field of the array lines.
Using awk:
awk '12<= NR && NR <= 24' file
In awk, NR is the line number. The above condition insists that NR be both greater than or equal to 12 and less than or equal to 24. If it is, then the line is printed. Otherwise, it isn't.
A more efficient solution
It would be more efficient to stop reading the file after the upper line limit has been reached. This solution does that:
awk 'NR>24 {exit;} NR>=12' file
I've a text file with 2 million lines. Each line has some transaction information.
e.g.
23848923748, sample text, feild2 , 12/12/2008
etc
What I want to do is create a new file from a certain unique transaction number onwards. So I want to split the file at the line where this number exists.
How can I do this form the command line?
I can find the line by doing this:
cat myfile.txt | grep 23423423423
use sed like this
sed '/23423423423/,$!d' myfile.txt
Just confirm that the unique transaction number cannot appear as a pattern in some other part of the line (especially, before the correctly matching line) in your file.
There is already a 'perl' answer here, so, i'll give one more AWK way :-)
awk '{BEGIN{skip=1} /number/ {skip=0} // {if (skip!=1) print $0}' myfile.txt
On a random file in my tmp directory, this is how I output everything from the line matching popd onwards in a file named tmp.sh:
tail -n+`grep -n popd tmp.sh | cut -f 1 -d:` tmp.sh
tail -n+X matches from that line number onwards; grep -n outputs lineno:filename, and cut extracts just lineno from grep.
So for your case it would be:
tail -n+`grep -n 23423423423 myfile.txt | cut -f 1 -d:` myfile.txt
And it should indeed match from the first occurrence onwards.
It's not a pretty solution, but how about using -A parameter of grep?
Like this:
mc#zolty:/tmp$ cat a
1
2
3
4
5
6
7
mc#zolty:/tmp$ cat a | grep 3 -A1000000
3
4
5
6
7
The only problem I see in this solution is the 1000000 magic number. Probably someone will know the answer without using such a trick.
You can probably get the line number using Grep and then use Tail to print the file from that point into your output file.
Sorry I don't have actual code to show, but hopefully the idea is clear.
I would write a quick Perl script, frankly. It's invaluable for anything like this (relatively simple issues) and as soon as something more complex rears its head (as it will do!) then you'll need the extra power.
Something like:
#!/bin/perl
my $out = 0;
while (<STDIN>) {
if /23423423423/ then $out = 1;
print $_ if $out;
}
and run it using:
$ perl mysplit.pl < input > output
Not tested, I'm afraid.