Manually calculate requests per minute from a HTTP access log - ruby

Question

I have a logfile
123.123.123.123 - - [26/Apr/2000:00:23:48 -0400] "GET /pics/wpaper.gif HTTP/1.0" 200 6248 "http://www.jafsoft.com/asctortf/" "Mozilla/4.05 (Macintosh; I; PPC)"
123.123.123.123 - - [26/Apr/2000:00:23:47 -0400] "GET /asctortf/ HTTP/1.0" 200 8130 "http://search.netscape.com/Computers/Data_Formats/Document/Text/RTF" "Mozilla/4.05 (Macintosh; I; PPC)"
123.123.123.123 - - [26/Apr/2000:00:23:48 -0400] "GET /pics/5star2000.gif HTTP/1.0" 200 4005 "http://www.jafsoft.com/asctortf/" "Mozilla/4.05 (Macintosh; I; PPC)"
123.123.123.123 - - [26/Apr/2000:00:23:50 -0400] "GET /pics/5star.gif HTTP/1.0" 200 1031 "http://www.jafsoft.com/asctortf/" "Mozilla/4.05 (Macintosh; I; PPC)"
123.123.123.123 - - [26/Apr/2000:00:23:51 -0400] "GET /pics/a2hlogo.jpg HTTP/1.0" 200 4282 "http://www.jafsoft.com/asctortf/" "Mozilla/4.05 (Macintosh; I; PPC)"
123.123.123.123 - - [26/Apr/2000:00:23:51 -0400] "GET /cgi-bin/newcount?jafsof3&width=4&font=digital&noshow HTTP/1.0" 200 36 "http://www.jafsoft.com/asctortf/" "Mozilla/4.05 (Macintosh; I; PPC)"
I want to calculate the requests per minute and output something like this.
2012/01/01 00:00 2
2012/01/01 00:01 33
I was thinking of looping over the whole file and extracting out the timestamps to an array, using regex like this.
File.open("log.txt") do |f|
f.each_line do |line|
timestamps << line[/\[(\d{2})\/([a-zA-Z]{3})\/(\d{4}):(\d{2}):(\d{2}) (\d{2})\s(-\d{4})]/]
end
Then using that array to somehow calculate the requests per second, is there a better way I can do this using ruby? Without using CLI tools.

Answer 1

It's not the prettiest, but this is what you're going to want to do.
require 'time'
TIMESTAMP_REGEX = %r{\[(.*?)\]} # extract everything between the []
datetimes = log.scan(TIMESTAMP_REGEX).flatten.map { |log_time| DateTime.parse(log_time.sub(":", ' ')) } # get the results from the regex and make an array of DateTime objects
results = Hash.new(0)
datetimes.each do |datetime|
time = datetime.strftime('%Y/%m/%d %H:%M')
results[time] += 1
end
results.each do |k,v|
puts "#{k}: #{v} requests"
end
There are more optimal ways to do this — including a moderately-lengthy one-liner — but if you're looking for straightforwardness this is the way to go.