Ruby check uniqueness - ruby

Question

I build some code for going through a textfile (webserver logfile). My code works fine, but i have two questions. The code works fine, only the first username visible in the logfile is not printed and not counted. Does anyone know why?
My second question is about my count_unique. What do I need to do to count only the unique usernames?
My Code:
count_tot = 0
count_unique = 0
file = File.new("text.txt", "r")
line = file.gets
while (line = file.gets)
substrings = line.split("&")
substrings.each do |sub|
if sub.include? 'username'
puts sub
count_tot += 1
else
end
end
end
file.close
puts ""
puts "Total found input values:"
puts count_tot
puts count_unique
Example input (2 lines)
[11/Mar/2014:00:15:02 +0100] "GET /web/show/id=568296 HTTP/1.1" 200 8499 "https://www.site.com/csc/default.aspx?sid=ertett4353452445.orker2&username=username1&timestamp=20140311001443&hashkey=847823786547385243678&" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.74.9 (KHTML, like Gecko) Version/7.0.2 Safari/537.74.9" 52345 1FD323C0D681D2F10AE789F8A6C0900D.wm9worker5
[11/Mar/2014:00:35:50 +0100] "GET /web/show/id=568296 HTTP/1.1" 200 8499 "https://www.site.com/csc/default.aspx?sid=gfdgdfdgfgdfdfg._worker1&username=username2&timestamp=20140311003517&hashkey=fdsfsdffsffds&" "Mozilla/5.0 (iPad; CPU OS 7_0_6 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) CriOS/33.0.1750.14 Mobile/11B651 Safari/9537.53" 62415 5852920B165D2E39559241BA8B5FB36A.wm9worker6

Answer 1

only the first username visible in the logfile is not printed and not counted. Does anyone know why?
For that you need to do
line = file.gets # remove this.
while (line = file.gets) # keep only this.
line = file.gets ( which is before while loop), is not being processed. Before entering into the while loop that line data got lost.
update
string = <<_
[11/Mar/2014:00:15:02 +0100] "GET /web/show/id=568296 HTTP/1.1" 200 8499 "https://www.site.com/csc/default.aspx?sid=ertett4353452445.orker2&username=username1&timestamp=20140311001443&hashkey=847823786547385243678&" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.74.9 (KHTML, like Gecko) Version/7.0.2 Safari/537.74.9" 52345 1FD323C0D681D2F10AE789F8A6C0900D.wm9worker5
[11/Mar/2014:00:35:50 +0100] "GET /web/show/id=568296 HTTP/1.1" 200 8499 "https://www.site.com/csc/default.aspx?sid=gfdgdfdgfgdfdfg._worker1&username=username2&timestamp=20140311003517&hashkey=fdsfsdffsffds&" "Mozilla/5.0 (iPad; CPU OS 7_0_6 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) CriOS/33.0.1750.14 Mobile/11B651 Safari/9537.53" 62415 5852920B165D2E39559241BA8B5FB36A.wm9worker6
[11/Mar/2014:00:35:50 +0100] "GET /web/show/id=568296 HTTP/1.1" 200 8499 "https://www.site.com/csc/default.aspx?sid=gfdgdfdgfgdfdfg._worker1&username=username2&timestamp=20140311003517&hashkey=fdsfsdffsffds&" "Mozilla/5.0 (iPad; CPU OS 7_0_6 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) CriOS/33.0.1750.14 Mobile/11B651 Safari/9537.53" 62415 5852920B165D2E39559241BA8B5FB36A.wm9worker6
_
File.write('f1',string)
#usernames = []
File.foreach('f1') do |line|
#collect all the usernames
#usernames << line[/username=(\w+)/,1]
# do other tasks with *line*
end
#usernames # => ["username1", "username2", "username2"]
# to get the uniq usernames
#usernames.uniq # => ["username1", "username2"]
# if you want to see, which username present how many times, think something
# like below
Hash[#usernames.group_by { |s| s }.map { |k,v| [k,v.size]}]
# => {"username1"=>1, "username2"=>2}
Look at the method IO::foreach to understand why I used it. Checkout the Array#uniq and group_by methods also. Documentation of these are much clear.

Answer 2

First of all, the IO class, and by extension File, has an each method which yields lines to the block. There is also a foreach class method that makes it even more concise.
File.foreach 'text.txt' do |line|
# Count stuff ...
end
Regarding your first question, that happens because you read the first line into a variable and then proceed to overwrite said variable immediately after in the while loop's clause. This effectively skips the first line. The example above gets rid of that problem.
It is hard to answer the second question without looking at the input we're dealing with.
A simple String#scan-based solution might suffice:
line.scan /[?&]username=([^&]*)/ do |user_name|
puts user_name
end
Everything can thus be simplified to:
user_names = File.foreach('text.txt').map do |line|
line.scan /[?&]username=([^&]*)/
end.flatten
user_name_counts = user_names.uniq.inject Hash.new do |hash, user_name|
hash.tap do |hash|
hash[user_name] = user_names.count user_name
end
end
p user_name_counts
# => {"username1"=>1, "username2"=>2}