Anagrams Code Kata, Ruby Solution very slow - ruby

Question

I've been having a play with Ruby recently and I've just completed the Anagrams Code Kata from http://codekata.pragprog.com.
The solution was test driven and utilises the unique prime factorisation theorem, however it seems to run incredibly slow. Just on the 45k file it's been running for about 10 minutes so far. Can anyone give me any pointers on improving the performance of my code?
class AnagramFinder
def initialize
#words = self.LoadWordsFromFile("dict45k.txt")
end
def OutputAnagrams
hash = self.CalculatePrimeValueHash
#words.each_index{|i|
word = #words[i]
wordvalue = hash[i]
matches = hash.select{|key,value| value == wordvalue}
if(matches.length > 1)
puts("--------------")
matches.each{|key,value|
puts(#words[key])
}
end
}
end
def CalculatePrimeValueHash
hash = Hash.new
#words.each_index{|i|
word = #words[i]
value = self.CalculatePrimeWordValue(word)
hash[i] = value
}
hash
end
def CalculatePrimeWordValue(word)
total = 1
hash = self.GetPrimeAlphabetHash
word.downcase.each_char {|c|
value = hash[c]
total = total * value
}
total
end
def LoadWordsFromFile(filename)
contentsArray = []
f = File.open(filename)
f.each_line {|line|
line = line.gsub(/[^a-z]/i, '')
contentsArray.push line
}
contentsArray
end
def GetPrimeAlphabetHash
hash = { "a" => 2, "b" => 3, "c" => 5, "d" => 7, "e" => 11, "f" => 13, "g" =>17, "h" =>19, "i" => 23, "j" => 29, "k" => 31, "l" => 37, "m" => 41, "n" =>43, "o" =>47, "p" => 53, "q" =>59, "r" => 61, "s" => 67, "t" => 71, "u" => 73, "v" => 79, "w" => 83, "x" => 89, "y" => 97, "z" => 101 }
end
end

Answer 1

Frederick Cheung has a few good points, but I thought I might provide you with a few descriptive examples.
I think your main problem is that you create your index in a way that forces you to do linear searches in it.
Your word list (#words) seems to look something like this:
[
"ink",
"foo",
"kin"
]
That is, it is just an array of words.
Then you create your hash index with CalculatePrimeValueHash, with hash keys being equal to the word's index in #words.
{
0 => 30659, # 23 * 43 * 31, matching "ink"
1 => 28717, # 13 * 47 * 47, matching "foo"
2 => 30659 # 31 * 23 * 43, matching "kin"
}
I would consider this a good start, but the thing is if you keep it like this, you will have to iterate through the hash to find what hash keys (i.e. indexes in #words) that belong together, and then iterate through those to join them. That is, the basic problem here is that you do things too granularly.
If you instead were to build this hash with the prime values as hash keys, and have them point to an array of the words with that key, you would get a hash index like this instead:
{
30659 => ["ink", "kin"],
28717 => ["foo"]
}
With this kind of structure, the only thing you have to do to write your output, is to just iterate over the hash values and print them, since they are already grouped.
Another thing with your code, is that it seems to generate a whole bunch of throwaway objects , which will make sure to keep your garbarge collector busy, and that is generally quite a big choke point in ruby.
It might also be a good thing to go find either a benchmark tool and/or a profiler to analyze your code and see where it could be approved upon.

Answer 2

Fundamentally your code is slow because for each word (45k) of them you iterate over the entire hash (45k of them) looking for words with the same signature, so you're doing 45k * 45k of these comparisons. Another way of phrasing that is to say that your complexity is n^2 in the number of words.
The code below implements your basic idea but runs in a few seconds on the 236k word file I happen to have lying around. It could definitely be faster - the second pass over the data to find the things with > 1 items could be eliminated but would be less readable
It's also a lot shorter than your code, around a third, while staying readable, largely because I used more standard library functions and idiomatic ruby.
For example, the load_words method uses collect to turn one array into another, rather than iterating over one array and adding things to a second one. Similarly the signature function uses inject rather than iterating over the characters. Lastly I've used group_by to do the actual grouping. All of these methods happen to be in Enumerable - it's well worth becoming very familiar with these.
signature_for_word could become even pithier with
word.each_char.map {|c| CHAR_MAP[c.downcase]}.reduce(:*)
This takes the word, splits it into characters and then maps each one of those to the right number. reduce(:*) (reduce is an alias for inject) then multiplies them all together.
class AnagramFinder
CHAR_MAP ={ "a" => 2, "b" => 3, "c" => 5, "d" => 7, "e" => 11, "f" => 13, "g" =>17, "h" =>19, "i" => 23, "j" => 29, "k" => 31, "l" => 37, "m" => 41, "n" =>43, "o" =>47, "p" => 53, "q" =>59, "r" => 61, "s" => 67, "t" => 71, "u" => 73, "v" => 79, "w" => 83, "x" => 89, "y" => 97, "z" => 101 }
def initialize
#words = load_words("/usr/share/dict/words")
end
def find_anagrams
words_by_signature = #words.group_by {|word| signature_for_word word}
words_by_signature.each do |signaure, words|
if words.length > 1
puts '----'
puts words.join('; ')
end
end
end
def signature_for_word(word)
word.downcase.each_char.inject(1) {| total, c| total * CHAR_MAP[c]}
end
def load_words(filename)
File.readlines(filename).collect {|line| line.gsub(/[^a-z]/i, '')}
end
end

Answer 3

You can start limiting the slowness by using the Benchmark tool. Some examples here:
http://www.skorks.com/2010/03/timing-ruby-code-it-is-easy-with-benchmark/
First of all it would be interesting to see how long it takes to run self.calculate_prime_value_hash and after that the calculate_prime_word_value.
Quite often the slowness boils down to the number of times the inners loops are run so you can also log how many times they are run.
One very quick improvement you can do is to set the prime alhabet hash as a constant because it's not changed at all:
PRIME_ALPHABET_HASH = { "a" => 2, "b" => 3, "c" => 5, "d" => 7, "e" => 11, "f" => 13, "g" =>17, "h" =>19, "i" => 23, "j" => 29, "k" => 31, "l" => 37, "m" => 41, "n" =>43, "o" =>47, "p" => 53, "q" =>59, "r" => 61, "s" => 67, "t" => 71, "u" => 73, "v" => 79, "w" => 83, "x" => 89, "y" => 97, "z" => 101 }