Ruby: String reconstruction algo, working only partially - ruby

Question

I'm working on a string reconstruction algorithm (a classic in dynamic programming examples, turning space less text into normal spaced text) in Ruby. The code below is pure ruby, you can copy paste and start testing immediately, it's working 80% of the time and tends to break, the larger the dictionary becomes. I've tested it with more then 80k words dictionaries and it works less good, about 70% of the time.
If there's a way to make it work 100% if the word is present in the dictionary, please show me.
Here's the code: (it's well spaced and should be very readable)
# Partially working string reconstruction algo in pure Ruby
# the dictionary
def dict(someWord)
myArray = [" ", "best", "domain", "my", "successes", "image", "resizer", "high", "tech", "crime", "unit", "name", "edge", "times", "find", "a", "bargain", "free", "spirited", "style", "i", "command", "go", "direct", "to", "harness", "the", "force"]
return !!(myArray.index(someWord))
end
# inspired by http://cseweb.ucsd.edu/classes/wi12/cse202-a/lecture6-final.pdf
## Please uncomment the one you wanna use
#
# (all the words used are present in the dictionary above)
#
# working sentences
x = ' ' + "harnesstheforce"
# x = ' ' + "hightechcrimeunit"
#
# non working sentences
# x = ' ' + "findabargain"
# x = ' ' + "icommand"
puts "Trying to reconstruct #{x}"
# useful variables we're going to use in our algo
n = x.length
k = Array.new(n)
s = Array.new(n)
breakpoints = Hash.new
validBreakpoints = Hash.new
begin
# let's fill k
for i in 0..n-1
k[i] = i
end
# the core algo starts here
s[0] = true
for k in 1..n-1
s[k] = false
for j in 1..k
if s[j-1] && dict(x[j..k])
s[k] = true
# using a hash is just a trick to not have duplicates
breakpoints.store(k, true)
end
end
end
# debug
puts "breakpoints: #{breakpoints.inspect} for #{x}"
# let's create a valid break point vector
i=1
while i <= n-1 do
# we choose the longest valid word
breakpoints.keys.sort.each do |k|
if i >= k
next
end
# debug: when the algo breaks, it does so here and goes into an infinite loop
#puts "x[#{i}..#{k}]: #{x[i..k]}"
if dict(x[i..k])
validBreakpoints[i] = k
end
end
if validBreakpoints[i]
i = validBreakpoints[i] + 1
end
end
# debug
puts "validBreakpoints: #{validBreakpoints.inspect} for #{x}"
# we insert the spaces at the places defined by the valid breakpoints
x = x.strip
i = 0
validBreakpoints.each_key do |key|
validBreakpoints[key] = validBreakpoints[key] + i
i += 1
end
validBreakpoints.each_value do |value|
x.insert(value, ' ')
end
puts "Debug: x: #{x}"
# we capture ctrl-c
rescue SignalException
abort
# end of rescue
end

Answer 1

Note that your algorithm fails for strings containing single-character words. This is an off-by-one error. You are ignoring the breakpoints after such words, thus you end up with a word ("abargain") not contained in your dictionary.
Change
if i >= k
next
end
to
if i > k
next
end
or more Ruby-like
next if i > k
Note also that you are running into an endless loop whenever your string contains something which is not a word:
if validBreakpoints[i] # will be false
i = validBreakpoints[i] + 1 # i not incremented, so start over at the same position
end
You better treat this as an error
return '<no parse>' unless validBreakpoints[i] # or throw if you are not in a function
i = validBreakpoints[i] + 1
The problem with "inotifier" is a deficiency of your algorithm. Always choosing the longest word is not good. In this case, the first "valid" breakpoint detected is after the "in" which leaves you the non-word "otifier".