Count the most common occurrences of a unknown strings in a file - windows

Question

I have a large file full of lines like this...
19:54:05 10.10.8.5 [SERVER] Response sent: www.example.com. type A by 192.168.4.5
19:55:10 10.10.8.5 [SERVER] Response sent: ns1.example.com. type A by 192.168.4.5
19:55:23 10.10.8.5 [SERVER] Response sent: ns1.example.com. type A by 192.168.4.5
I don't care about any of the other data, only what's after the "response sent:"
I'd like a sorted list of the most common occurrences of the domain-names.
Problem is I won't know all the domain-names in advance, so I can't just do a search for the string.
Using the example above I'd like the output to be along the lines of
ns1.example.com (2)
www.example.com (1)
...where the number in ( ) is the counts of that occurrence.
How/what could I use to do this on Windows? The input file is .txt - the output file can be anything. Ideally a command-line process, but I'm really lost so I'd be happy with anything.

Answer 1

Cat is kinda out of the bag so lets try and help a little. This is a PowerShell solution. If you are having issues with how this works I encourage you to research the individual parts.
If you text file was "D:\temp\test.txt" then you could do something like this.
$results = Select-String -Path D:\temp\test.txt -Pattern "(?<=sent: ).+(?= type)" | Select -Expand Matches | Select -Expand Value
$results | Group-Object | Select-Object Name,Count | Sort-Object Count -Descending
Using your input you would get this for output
Name Count
---- -----
ns1.example.com. 2
www.example.com. 1
Since there is regex I have saved a link that explains how it works.
Please keep in mind that SO is, of course, a site that helps programmers and programming enthusiasts. We are devoting our free time where as some people get paid to do this.

Answer 2

Can you do it in PHP?
<?php
$lines = file($filename, FILE_IGNORE_NEW_LINES);
foreach($lines as $value) {
$arr = explode(' ', $value);
$domainarr[] = $arr[5];
}
$occurence = array_count_values($domainarr);
print_r($occurence);
?>

Answer 3

This is in batch:
#echo off
setlocal enabledelayedexpansion
if exist temp.txt del temp.txt
for /f "tokens=6" %%a in (input.txt) do (Echo %%a >> temp.txt)
for /f %%a in (temp.txt) do (
set /a count=0
set v=%%a
if "!%%a!" EQU "" (
for /f %%b in ('findstr /L "%%a" "temp.txt"') do set /a count+=1
set %%a=count
Echo !v:~0,-1! ^(!count!^)
)
)
del temp.txt
Currently it prints it out to the screen. If you would like to redirect it to a text file replace:
Echo !v:~0,-1! ^(!count!^)
with:
Echo !v:~0,-1! ^(!count!^) >> output.txt
This outputted:
www.example.com (1)
ns1.example.com (2)
With the sample data

Answer 4

This Batch file solution should run faster:
#echo off
setlocal
rem Accumulate each occurance in its corresponding array element
for /F "tokens=6" %%a in (input.txt) do set /A "count[%%a]+=1"
rem Show the result
for /F "tokens=2,3 delims=[]=" %%a in ('set count[') do echo %%a (%%b)
Output:
ns1.example.com. (2)
www.example.com. (1)
To store the result in a file, change the last line by this one:
(for /F "tokens=2,3 delims=[]=" %%a in ('set count[') do echo %%a (%%b^)) > output.txt