Ruby - Microsoft Translator unexpected token error - ruby

Question

I'm using the following method to translate a simple word from English to Russian by calling:
translate("hello")
This is my method:
def translate(text)
begin
uri = "http://api.microsofttranslator.com/V2/Ajax.svc/GetTranslations?appId=#{#appid}&text=#{text.strip}&from=en&to=ru&maxTranslations=1"
page = HTTParty.get(uri).body
show_info = JSON.parse(page) # this line throws the error
rescue
puts $!
end
end
The JSON output:
{"From":"en","Translations":[{"Count":0,"MatchDegree":100,"MatchedOriginalText":"","Rating":5,"TranslatedText":"Привет"}]}
The error:
unexpected token at '{"From":"en","Translations":[{"Count":0,"MatchDegree":100,"MatchedOriginalText":"","Rating":5,"TranslatedText":"Привет"}]}'
Not sure what it means by unexpected token. It's the only error I'm receiving. Unfortunately I can't modify the JSON output as it's returned by the API itself.
UPDATE:
Looks like the API is returning some illegal characters (bad Microsoft):
'´╗┐{"From":"en","Translations":[{"Count":0,"MatchDegree":0,"Matched OriginalText":"","Rating":5,"TranslatedText":"Hello"}]}'
Full error:
C:/Ruby193/lib/ruby/1.9.1/json/common.rb:148:in `parse': 743: unexpected token at '´╗┐{"From":"en","Translations":[{"Count":0,"MatchDegree":0,"Matched
OriginalText":"","Rating":5,"TranslatedText":"Hello"}]}' (JSON::ParserError)
from C:/Ruby193/lib/ruby/1.9.1/json/common.rb:148:in `parse'
from trans.rb:13:in `translate'
from trans.rb:17:in `<main>'

Answer 1

Try ensuring UTF-8 encoding and stripping any leading BOM indicators in the string:
# encoding: UTF-8
# ^-- Make sure this is on the first line!
def translate(text)
begin
uri = "http://api.microsofttranslator.com/V2/Ajax.svc/GetTranslations?appId=#{#appid}&text=#{text.strip}&from=en&to=ru&maxTranslations=1"
page = HTTParty.get(uri).body
page.force_encoding("UTF-8").gsub!("\xEF\xBB\xBF", '')
show_info = JSON.parse(page) # this line throws the error
rescue
puts $!
end
end
Sources:
Ruby 1.9's String
Wikipedia: Byte order mark
Using awk to remove the Byte-order mark