How to get words frequency in efficient way with ruby? - ruby

Question

Sample input:
"I was 09809 home -- Yes! yes! You was"
and output:
{ 'yes' => 2, 'was' => 2, 'i' => 1, 'home' => 1, 'you' => 1 }
My code that does not work:
def get_words_f(myStr)
myStr=myStr.downcase.scan(/\w/).to_s;
h = Hash.new(0)
myStr.split.each do |w|
h[w] += 1
end
return h.to_a;
end
print get_words_f('I was 09809 home -- Yes! yes! You was');

Answer 1

This works but I am kinda new to Ruby too. There might be a better solution.
def count_words(string)
words = string.split(' ')
frequency = Hash.new(0)
words.each { |word| frequency[word.downcase] += 1 }
return frequency
end
Instead of .split(' '), you could also do .scan(/\w+/); however, .scan(/\w+/) would separate aren and t in "aren't", while .split(' ') won't.
Output of your example code:
print count_words('I was 09809 home -- Yes! yes! You was');
#{"i"=>1, "was"=>2, "09809"=>1, "home"=>1, "yes"=>2, "you"=>1}

Answer 2

def count_words(string)
string.scan(/\w+/).reduce(Hash.new(0)){|res,w| res[w.downcase]+=1;res}
end
Second variant:
def count_words(string)
string.scan(/\w+/).each_with_object(Hash.new(0)){|w,h| h[w.downcase]+=1}
end

Answer 3

def count_words(string)
Hash[
string.scan(/[a-zA-Z]+/)
.group_by{|word| word.downcase}
.map{|word, words|[word, words.size]}
]
end
puts count_words 'I was 09809 home -- Yes! yes! You was'

Answer 4

This code will ask you for input and then find the word frequency for you:
puts "enter some text man"
text = gets.chomp
words = text.split(" ")
frequencies = Hash.new(0)
words.each { |word| frequencies[word.downcase] += 1 }
frequencies = frequencies.sort_by {|a, b| b}
frequencies.reverse!
frequencies.each do |word, frequency|
puts word + " " + frequency.to_s
end

Answer 5

This works, and ignores the numbers:
def get_words(my_str)
my_str = my_str.scan(/\w+/)
h = Hash.new(0)
my_str.each do |s|
s = s.downcase
if s !~ /^[0-9]*\.?[0-9]+$/
h[s] += 1
end
end
return h
end
print get_words('I was there 1000 !')
puts '\n'

Answer 6

You can look at my code that splits the text into words. The basic code would look as follows:
sentence = "Ala ma kota za 5zł i 10$."
splitter = SRX::Polish::WordSplitter.new(sentence)
histogram = Hash.new(0)
splitter.each do |word,type|
histogram[word.downcase] += 1 if type == :word
end
p histogram
You should be careful if you wish to work with languages other than English, since in Ruby 1.9 the downcase won't work as you expected for letters such as 'Ł'.

Answer 7

class String
def frequency
self.scan(/[a-zA-Z]+/).each.with_object(Hash.new(0)) do |word, hash|
hash[word.downcase] += 1
end
end
end
puts "I was 09809 home -- Yes! yes! You was".frequency