I just want to delete the line which contain the number of selected rows in a query. I mean the one in the last line. please help.
[root#machine-test scripts]# ./hit_ratio.sh
193830 432
185260 125
2 rows selected.
If you know you want to delete the last line, but not other lines which contain similar text, or you don't know what text it will contain, sed is uniquely suitable.
./hit_ratio.sh | sed '$d'
You don't need the power of sed or the super-powers of awk if all you want is to delete a line based on a pattern. You can use:
./hit_ratio.sh | grep -v ' rows selected.'
You can do it with awk and sed but it's a bit like trying to kill a fly with a thermo-nuclear warhead:
pax> ./hit_ratio.sh | sed '/ rows selected./d'
193830 432
185260 125
pax> ./hit_ratio.sh | awk '$2!="rows"{print}'
193830 432
185260 125
Alternatively, do something with your SQL script. Sometimes, turning on the set nocount on statement eliminates the "rows affected" line.
My first recommendation is not to have that line outputted, list hit_ratio.sh here maybe it can be modified not to output that line
Anyway if you have to remove only the last line the easiest would be to use head:
./hit_ratio.sh | head -n -1
Using -n and a negative number, makes head print all but the last N lines of the input
Use head to get the first N - 1 lines of your file, where N is the length of the file (calculated with wc -l)
head -n $(($(cat lipsum.log | wc -l) - 1)) lipsum.log works
Pipe through
sed -e '/\w*[0-9]\+ rows\? selected/d'
Related
I have seen How can I extract a predetermined range of lines from a text file on Unix? but I have a slightly different use case: I want to specify a starting line number, and a count/amount/number of lines to extract, from a text file.
So, I tried to generate a text file, and then compose an awk command to extract a count of 10 lines starting from line number 100 - but it does not work:
$ seq 1 500 > test_file.txt
$ awk 'BEGIN{s=100;e=$s+10;} NR>=$s&&NR<=$e' test_file.txt
$
So, what would be an easy approach to extract lines from a text file using a starting line number, and count of lines, in bash? (I'm ok with awk, sed, or any such tool, for instance in coreutils)
This gives you text that is inclusive of both end points
(eleven output lines, here).
$ START=100
$
$ sed -n "${START},$((START + 10))p" < test_file.txt
The -n says "no print by default".
And then the p says "print this line",
for lines within the example range of 100,110
When you want to use awk, use something like
seq 1 500 | awk 'NR>=100 && NR<=110'
Advantage of awk is the flexibility for changing the requirements.
When you want to use a variable start and skip the endpoints, it will be
start=100
seq 1 500 | awk -v start="${start}" 'NR > start && NR < start + 10'
Another alternative with tail and head:
tail -n +$START test_file.txt | head -n $NUMBER
If test_file.txt is very large and $START and $NUMBER are small, the following variant should be the fastest:
head -n $((START+NUMBER)) test_file.txt | tail -n +$START
Anyway, I prefer the sed solution noticed above for short input files:
sed -n "$START,$((START+NUMBER)) p" test_file.txt
sed -n "$Start,$End p" file
is likely a better way to get those lines.
$ seq 1 500 > test_file.txt
$ awk 'BEGIN{s=100;e=$s+10;} NR>=$s&&NR<=$e' test_file.txt
$
$s in GNU AWK means value of s-th field, $e in GNU AWK means value of e-th field. There are not fields yet in BEGIN clause so $s for any s is not set, as you use in arithemtic context it will be assumed to be 0 and therefore e will be set to value 10. Output of seq is single number per line, so there is not 10th field, so GNU AWK assumes it to be zero when asked to compare it with number, as NR is always strictly bigger than 0 your condition never holds so output is empty.
Use Range if you are able to prepare condition which holds solely for starting line and condition which holds solely for ending line, in this case
awk 'BEGIN{s=100}NR==s,NR==s+10' test_file.txt
gives output
100
101
102
103
104
105
106
107
108
109
110
Keep in mind that this will process whole file, if you have huge file and area of interest is relatively near begin, then you might decrease time consumption by ending processing at end of area of interest following way
awk 'BEGIN{s=100}NR>=s{print}NR==s+10{exit}' test_file.txt
(tested in GNU Awk 5.0.1)
This command extracts 30 lines starting from line 100
sed -n '100,$p' test_file.txt | head -30
I want to delete one or more specific line numbers from a file. How would I do this using sed?
If you want to delete lines from 5 through 10 and line 12th:
sed -e '5,10d;12d' file
This will print the results to the screen. If you want to save the results to the same file:
sed -i.bak -e '5,10d;12d' file
This will store the unmodified file as file.bak, and delete the given lines.
Note: Line numbers start at 1. The first line of the file is 1, not 0.
You can delete a particular single line with its line number by
sed -i '33d' file
This will delete the line on 33 line number and save the updated file.
and awk as well
awk 'NR!~/^(5|10|25)$/' file
$ cat foo
1
2
3
4
5
$ sed -e '2d;4d' foo
1
3
5
$
This is very often a symptom of an antipattern. The tool which produced the line numbers may well be replaced with one which deletes the lines right away. For example;
grep -nh error logfile | cut -d: -f1 | deletelines logfile
(where deletelines is the utility you are imagining you need) is the same as
grep -v error logfile
Having said that, if you are in a situation where you genuinely need to perform this task, you can generate a simple sed script from the file of line numbers. Humorously (but perhaps slightly confusingly) you can do this with sed.
sed 's%$%d%' linenumbers
This accepts a file of line numbers, one per line, and produces, on standard output, the same line numbers with d appended after each. This is a valid sed script, which we can save to a file, or (on some platforms) pipe to another sed instance:
sed 's%$%d%' linenumbers | sed -f - logfile
On some platforms, sed -f does not understand the option argument - to mean standard input, so you have to redirect the script to a temporary file, and clean it up when you are done, or maybe replace the lone dash with /dev/stdin or /proc/$pid/fd/1 if your OS (or shell) has that.
As always, you can add -i before the -f option to have sed edit the target file in place, instead of producing the result on standard output. On *BSDish platforms (including OSX) you need to supply an explicit argument to -i as well; a common idiom is to supply an empty argument; -i ''.
The shortest, deleting the first line in sed
sed -i '1d' file
As Brian states here, <address><command> is used, <address> is <1> and <command> <d>.
I would like to propose a generalization with awk.
When the file is made by blocks of a fixed size
and the lines to delete are repeated for each block,
awk can work fine in such a way
awk '{nl=((NR-1)%2000)+1; if ( (nl<714) || ((nl>1025)&&(nl<1029)) ) print $0}'
OriginFile.dat > MyOutputCuttedFile.dat
In this example the size for the block is 2000 and I want to print the lines [1..713] and [1026..1029].
NR is the variable used by awk to store the current line number.
% gives the remainder (or modulus) of the division of two integers;
nl=((NR-1)%BLOCKSIZE)+1 Here we write in the variable nl the line number inside the current block. (see below)
|| and && are the logical operator OR and AND.
print $0 writes the full line
Why ((NR-1)%BLOCKSIZE)+1:
(NR-1) We need a shift of one because 1%3=1, 2%3=2, but 3%3=0.
+1 We add again 1 because we want to restore the desired order.
+-----+------+----------+------------+
| NR | NR%3 | (NR-1)%3 | (NR-1)%3+1 |
+-----+------+----------+------------+
| 1 | 1 | 0 | 1 |
| 2 | 2 | 1 | 2 |
| 3 | 0 | 2 | 3 |
| 4 | 1 | 0 | 1 |
+-----+------+----------+------------+
cat -b /etc/passwd | sed -E 's/^( )+(<line_number>)(\t)(.*)/--removed---/g;s/^( )+([0-9]+)(\t)//g'
cat -b -> print lines with numbers
s/^( )+(<line_number>)(\t)(.*)//g -> replace line number to null (remove line)
s/^( )+([0-9]+)(\t)//g #remove numbers the cat printed
Is it possible, in UNIX, to print a particular line of a file? For example I would like to print line 10 of file example.c. I tried with cat, ls, awk but apparently either these don't have the feature or I'm not able to properly read the man :-).
Using awk:
awk 'NR==10' file
Using sed:
sed '10!d' file
sed -n '10{p;q;}' example.c
will print the tenth line of example.c for you.
Try head and tail, you can specify the amount of lines and where to start.
To get the third line:
head -n 3 yourfile.c | tail -n 1
head -n 10 /tmp/asdf | tail -n 1
Unfortunately, all other solutions which use head/tail will NOT work incorrectly if line number provided is larger than total number of lines in our file.
This will print line number N or nothing if N is beyond total number of lines:
grep "" file | grep "^20:"
If you want to cut line number from output, pipe it through sed:
grep "" file | grep "^20:" | sed 's/^20://'
Try this:
cat -n <yourfile> | grep ^[[:space:]]*<NUMBER>[[:space:]].*$
cat -n numbers the file
the regex of grep searches the line numbered ;-)
The original mismatched as mentioned in the comments.
Te current one looks for the exact match.
- i.e. in the particular cas we need a line starting with an arbitrary amount () of spaces the followed by a space followed by whatever (.)
In case anyone thumbles over this regex and doesn't get it at all - here is a good tutorial to get you started: http://regex.learncodethehardway.org/book/ (it uses python regex as an example tough).
This might work for you:
sed '10q;d' file
I have very long log files, is it possible to ask grep to only search the first 10 lines?
The magic of pipes;
head -10 log.txt | grep <whatever>
For folks who find this on Google, I needed to search the first n lines of multiple files, but to only print the matching filenames. I used
gawk 'FNR>10 {nextfile} /pattern/ { print FILENAME ; nextfile }' filenames
The FNR..nextfile stops processing a file once 10 lines have been seen. The //..{} prints the filename and moves on whenever the first match in a given file shows up. To quote the filenames for the benefit of other programs, use
gawk 'FNR>10 {nextfile} /pattern/ { print "\"" FILENAME "\"" ; nextfile }' filenames
Or use awk for a single process without |:
awk '/your_regexp/ && NR < 11' INPUTFILE
On each line, if your_regexp matches, and the number of records (lines) is less than 11, it executes the default action (which is printing the input line).
Or use sed:
sed -n '/your_regexp/p;10q' INPUTFILE
Checks your regexp and prints the line (-n means don't print the input, which is otherwise the default), and quits right after the 10th line.
You have a few options using programs along with grep. The simplest in my opinion is to use head:
head -n10 filename | grep ...
head will output the first 10 lines (using the -n option), and then you can pipe that output to grep.
grep "pattern" <(head -n 10 filename)
head -10 log.txt | grep -A 2 -B 2 pattern_to_search
-A 2: print two lines before the pattern.
-B 2: print two lines after the pattern.
head -10 log.txt # read the first 10 lines of the file.
You can use the following line:
head -n 10 /path/to/file | grep [...]
The output of head -10 file can be piped to grep in order to accomplish this:
head -10 file | grep …
Using Perl:
perl -ne 'last if $. > 10; print if /pattern/' file
An extension to Joachim Isaksson's answer: Quite often I need something from the middle of a long file, e.g. lines 5001 to 5020, in which case you can combine head with tail:
head -5020 file.txt | tail -20 | grep x
This gets the first 5020 lines, then shows only the last 20 of those, then pipes everything to grep.
(Edited: fencepost error in my example numbers, added pipe to grep)
grep -A 10 <Pattern>
This is to grab the pattern and the next 10 lines after the pattern. This would work well only for a known pattern, if you don't have a known pattern use the "head" suggestions.
grep -m6 "string" cov.txt
This searches only the first 6 lines for string
I want to get the second last line from the ls -l output.
I know that
ls -l|tail -n 2| head -n 1
can do this, just wondering if sed can do this in just one command?
ls -l|sed -n 'x;$p'
It can't do third to last though, because sed only has 1 hold space, so can only remember one older line. And since it processes the lines one at a time, it does not know the line will be next to last when processing it. awk could return thrid to last, because you can have arbitrary number of variables there, but the script would be much longer than the tail -n X|head -n 1.
In a awk one-liner :
echo -e "aaa\nbbb\nccc\nddd" | awk '{v[c++]=$0}END{print v[c-2]}'
ccc
Try this to delete second-last line in file
sed -e '$!{h;d;}' -e x filename
tac filename | sed -n 2p
-- but involves a pipe, too