trying to format output from grep to make it look better, code is
grep "$1" "$2" | grep -E -o "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)" | sort | uniq -c
$ bash myScript.sh "Failed password for root" /home/user/auth.log
5 108.166.98.9
1426 108.53.208.61
1 113.108.211.131
1 117.79.91.195
370 122.224.49.124
3480 144.0.0.32
11 162.144.94.250
6 162.253.66.74
3 186.67.83.58
1 222.190.114.98
205 59.90.242.69
705 60.172.228.226
3 64.251.21.104
and want it to look more like
ip: xxx.xxx.xxx.xxx attempts: X
Add the following command to the end of your pipe in your script, after uniq:
... | awk '{print "ip: " $2 " attempts: " $1}'
The output will be
ip: 108.166.98.9 attempts: 5
ip: 108.53.208.61 attempts: 1426
...
Related
file
ID First_Name Last_Name(s) Average_Winter_Grade
323 Popa Arianna 10
317 Tabarcea Andreea 5.24
326 Balan Ionut 9.935
327 Balan Tudor-Emanuel 8.4
329 Lungu Iulian-Gabriel 7.78
365 Brailean Mircea 7.615
365 Popescu Anca-Maria 7.38
398 Acatrinei Andrei 8
How do I sort it by last column, except for the header ?
This is what file should look like after the changes:
ID First_Name Last_Name(s) Average_Winter_Grade
323 Popa Arianna 10
326 Balan Ionut 9.935
327 Balan Tudor-Emanuel 8.4
398 Acatrinei Andrei 8
329 Lungu Iulian-Gabriel 7.78
365 Brailean Mircea 7.615
365 Popescu Anca-Maria 7.38
317 Tabarcea Andreea 5.24
If it's always 4th column:
head -n 1 file; tail -n +2 file | sort -n -r -k 4,4
If all you know is that it's the last column:
head -n 1 file; tail -n +2 file | awk '{print $NF,$0}' | sort -n -r | cut -f2- -d' '
You'd like to just sort by the last column, but sort doesn't allow you to do that easily. So rewrite the data with the column to be sorted at the beginning of each line:
Ignoring the header for the moment (although this will often work by itself):
awk '{print $NF, $0 | "sort -nr" }' input | cut -d ' ' -f 2-
If you do need to trim the order (eg, it's getting mixed in the sort), you can do things like:
< input awk 'NR==1; NR>1 {print $NF, $0 | "sh -c \"sort -nr | cut -d \\\ -f 2-\"" }'
or
awk 'NR==1{ print " ", $0} NR>1 {print $NF, $0 | "sort -nr" }' OFS=\; input | cut -d \; -f 2-
I would like to get specific file contains from single file and put into separate files via bash. I have tried getting test1 file contain using below code and able to get it but i'm failed when getting everything in respected files.
Tried code:
reportFile=/report.txt
test1File=/test1.txt
test2File=/test2.txt
test3File=/test3.txt
totalLineNo=`cat ${reportFile} | wc -l`
test1LineNo=`grep -n "Test1 file content :" ${reportFile} | grep -Eo '^[^:]+'`
test2LineNo=`grep -n "Test2 file content :" ${reportFile} | grep -Eo '^[^:]+'`
test3LineNo=`grep -n "Test3 file content :" ${reportFile} | grep -Eo '^[^:]+'`
exactTest1LineNo=`echo $(( ${test1LineNo} - 1 ))`
exactTest2LineNo=`echo $(( ${test2LineNo} -1 ))`
exactTest3LineNo=`echo $(( ${test3LineNo} -1 ))`
test1Content=`cat ${reportFile} | head -n ${exactTest1LineNo}`
test3Content=`cat ${reportFile} | tail -n ${exactTest3LineNo}`
echo -e "${test1Content}\r" >> ${test1File}
echo -e "${test3Content}\r" >> ${test3File}
report.txt:
-------------------------------------
My Report:
Test1 file content:
1
2
3
4
5
6
Test2 file content:
7
8
9
10
Test3 file content:
11
12
13
14
15
Note: Find my report above.
-------------------------------------
test1.txt (expected):
1
2
3
4
5
6
test2.txt (expected):
7
8
9
10
test3.txt (expected):
11
12
13
14
15
With single awk command:
awk '/^Test[0-9] file content:/{ f=1; fn=tolower($1)".txt"; next }
f && NF{ print > fn }!NF{ f=0 }' report.txt
Viewing results:
$ head test[0-9].txt
==> test1.txt <==
1
2
3
4
5
6
==> test2.txt <==
7
8
9
10
==> test3.txt <==
11
12
13
14
15
If I understand you correctly: you have a long file report.txt and you want to extract short files from it. The name of each file is followed by the string " file content:" in the file report.txt.
This is my solution:
#!/bin/bash
reportFile=report.txt
Files=`grep 'file content' $reportFile | sed 's/ .*$//'`
for F in $Files ; do
f=${F,}.txt # first letter lowercase and append .txt
awk "/$F file content/,/^\$/ {print}" $reportFile |
tail -n +2 | # remove first line with "Test* file content:"
head -n -1 > $f # remove last blank line
done
I am using following code snippet where I export the shell variables in awk as follows:
half_buffer1=$((start_buffer/2))
half_buffer2=$((end_buffer/2))
echo $line | awk -v left="$half_buffer1" -v right="$half_buffer2" 'BEGIN {print $1"\t"$2-left"\t"$3+right"\t"$4"\t"$5"\t"$6"\t"$7"\t"$8}'
However for the variable 'right' in awk at times the $3 variable is being subtracted from instead of adding the 'right' variable to $3.
Observe that the following provides the "wrong" answers:
$ echo 1 2 3 4 5 | awk -v left=10 -v right=20 'BEGIN {print $1"\t"$2-left"\t"$3+right"\t"$4"\t"$5"\t"$6"\t"$7"\t"$8}'
-10 20
To get the right answers, remove BEGIN:
$ echo 1 2 3 4 5 | awk -v left=10 -v right=20 '{print $1"\t"$2-left"\t"$3+right"\t"$4"\t"$5"\t"$6"\t"$7"\t"$8}'
1 -8 23 4 5
The problem is that the BEGIN block is executed before any input is read. Consequently, the variables $1, $2, etc., do not yet have useful values.
If BEGIN is removed, the code is executed on each line read. This gives you the answers that you want.
Examples
Using real input lines from the comments:
$ echo ID1 14389398 14389507 109 + ABC 608 831 | awk -v left=10 -v right=20 '{print $1"\t"$2-left"\t"$3+right"\t"$4"\t"$5"\t"$6"\t"$7"\t"$8}'
ID1 14389388 14389527 109 + ABC 608 831
$ echo ID1 14390340 14390409 69 + ABC 831 32 – | awk -v left=10 -v right=20 '{print $1"\t"$2-left"\t"$3+right"\t"$4"\t"$5"\t"$6"\t"$7"\t"$8}'
ID1 14390330 14390429 69 + ABC 831 32
Also, this shell script:
start_buffer=10
end_buffer=100
half_buffer1=$((start_buffer/2))
half_buffer2=$((end_buffer/2))
echo ID1 14390340 14390409 69 + ABC 831 32 – | awk -v left="$half_buffer1" -v right="$half_buffer2" '{print $1"\t"$2-left"\t"$3+right"\t"$4"\t"$5"\t"$6"\t"$7"\t"$8}'
produces this output:
ID1 14390335 14390459 69 + ABC 831 32
Normally when I do cat number.txt | sort -n | uniq -c , I get numbers like this:
3 43
4 66
2 96
1 97
But what I need is the number shows of occurrences at the back, like this:
43 3
66 4
96 2
97 1
Please give advice on how to change this. Thanks.
Use awk to change the order of columns:
cat number.txt | sort -n | uniq -c | awk '{ print $2, $1 }'
Perl version:
perl -lne '$occ{0+$_}++; END {print "$_ $occ{$_}" for sort {$a <=> $b} keys %occ}' < numbers.txt
Through GNU sed,
cat number.txt | sort -n | uniq -c | sed -r 's/^([0-9]+) ([0-9]+)$/\2 \1/g'
Let me present an example and than try to explain my problem:
noob#noob:~/Downloads$ ls | grep srt$
Elementary - 01x01 - Pilot.LOL.English.HI.C.orig.Addic7ed.com.srt
Haven - 01x01 - Welcome to Haven.DVDRip.SAiNTS.English.updated.Addic7ed.com.srt
Haven - 01x01 - Welcome to Haven.FQM.English.HI.C.updated.Addic7ed.com.srt
Supernatural - 08x01 - We Need to Talk About Kevin.LOL.English.HI.C.updated.Addic7ed.com.srt
The Big Bang Theory - 06x02 - The Decoupling Fluctuation.LOL.English.HI.C.orig.Addic7ed.com.srt
Torchwood - 1x01 - Everything changes.0TV.English.orig.Addic7ed.com.srt
Torchwood - 1x01 - Everything changes.divx.English.updated.Addic7ed.com.srt
Now I only want to delete the first four results of the above command. Normally if I have to delete all the files I would do ls | grep srt$ | xargs -I {} rm {} but in this case I only want to delete the top four.
So, how can limit the output of ls and grep or suggest me an alternate way to achieve this.
You can pipe your commands to head -n to limit to n lines:
ls | grep srt | head -4
$ for i in `seq 1 345`; do echo $i ;done | sed -n '1,4p'
1
2
3
4
geee: ~
$ for i in `seq 1 345`; do echo $i ;done | sed -n '335,360p'
335
336
337
338
339
340
341
342
343
344
345
If you don't have too many files, you can use a bash array:
matching_files=( *.srt )
rm "${matching_files[#]:0:4}"