We have been using JSON.parse for a while now and we just updated to the latest version.
We are getting the following error.
Has the functionality been updated to return nil instead of throwing a ParserError?
2.3.0 :001 > gem 'json', '2.0.2'
=> true
2.3.0 :002 > require 'json'
=> true
2.3.0 :003 > JSON.parse("null")
=> nil
2.3.0 :001 > gem 'json', '1.8.3'
=> true
2.3.0 :003 > require 'json'
=> true
2.3.0 :004 > JSON.parse("null")
JSON::ParserError: 784: unexpected token at 'null'
from /Users/ryannealmes/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/json/common.rb:156:in `parse'
from /Users/ryannealmes/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/json/common.rb:156:in `parse'
from (irb):4
from /Users/ryannealmes/.rvm/rubies/ruby-2.3.0/bin/irb:11:in `<main>'
2.3.0 :005 >
The functionality has been updated to comply with the JSON specification according to RFC 7159, which says:
A JSON value MUST be an object, array, number, or string, or one of
the following three literal names:
false null true
The literal names MUST be lowercase. No other literal names are allowed.
So, JSON.parse("null") returning nil is expected functionality, which you can confirm by checking out the tests for parsing single values.
Related
I’m using Rails 4.2.7. I have a request I set up like so:
2.3.0 :001 > url = 'http://www.mydomeina.com/results/browse.cfm?ID=4187141102&Gen=B&Begin=1&End=31&Max=31'
=> "http://www.mydomeina.com/results/browse.cfm?ID=4187141102&Gen=B&Begin=1&End=31&Max=31"
2.3.0 :002 > uri = URI(url)
=> #<URI::HTTP http://www.mydomeina.com/results/browse.cfm?ID=4187141102&Gen=B&Begin=1&End=31&Max=31>
2.3.0 :003 > req = Net::HTTP::Get.new uri
=> #<Net::HTTP::Get GET>
2.3.0 :004 > req["Referer"] = 'http://www.mydomeina.com/results/browse.cfm?ID=4187141102&Gen=B&Begin=1&End=31&Max=31'
=> "http://www.mydomeina.com/results/browse.cfm?ID=4187141102&Gen=B&Begin=1&End=31&Max=31"
I set req["Referer"] as an example of setting a header. There may be other headers in my request object. How do I figure out what they all are? This fails:
2.3.0 :009 > req.keys
NoMethodError: undefined method `keys' for #<Net::HTTP::Get GET>
Did you mean? key?
as does req.headers. How can I iterate over my request object to figure out what all the headers are? It is not an option to construct the request object in a different way then what I am doing now.
try
req.each_key{|k| puts k}
or
req.each{|k,v| puts "#{k} => #{v}"}
to see methods on an object from inside console you can always do
req.methods - Ojbect.methods
I'm currently working on a code where I'm requesting a URL with space on request path:
http://example.com/path1/path2/some details.detail?keyword=key word
I'm using rest-client gem.
url = http://example.com/path1/path2/some details.detail?keyword=key word
RestClient::Request.execute(method: :get, url: url)
But it throws an error:
URI::InvalidURIError: bad URI(is not URI?): http://example.com/path1/path2/some details.detail?keyword=key word
The documentation suggests to parse and normalize the URL first:
parsed_url = Addressable::URI.parse(url).normalize.to_str
# parsed_url is now: http://example.com/path1/path2/some%20details.detail?keyword=key%20word
RestClient::Request.execute(method: :get, url: parsed_url)
But the error still exists.
URI::InvalidURIError: bad URI(is not URI?): http://example.com/path1/path2/some details.detail?keyword=key%20word
It looks like it cannot parse the /some details.detail part of the url.
Does anyone encountered the same issue?
Try URI.encode:
$ irb
2.0.0-p247 :001 > require "uri"
=> true
2.0.0-p247 :002 > URI.encode("http://example.com/path1/path2/some details.detail?keyword=key word")
=> "http://example.com/path1/path2/some%20details.detail?keyword=key%20word"
2.0.0-p247 :003 >
Using rest-client 1.6.7 and addressable 2.3.5, on Ruby 2.0.0, this is an irb session trying to replicate your fault:
:001 > require 'rest-client'
=> true
:002 > require "addressable/uri"
=> true
:003 > url = 'http://example.com/path1/path2/some details.detail?keyword=key word'
=> "http://example.com/path1/path2/some details.detail?keyword=key word"
:004 > parsed_url = Addressable::URI.parse(url).normalize.to_str
=> "http://example.com/path1/path2/some%20details.detail?keyword=key%20word"
:005 > RestClient::Request.execute(method: :get, url: parsed_url)
RestClient::ResourceNotFound: 404 Resource Not Found: <!doctype html>
<html>
<head>
<title>Example Domain</title>
It seems to work ok according to your own solution. As far as I can tell you either missed out repeating a step whilst investigating the problem (try again, from the start, just in case), or perhaps it is a bug in an older version of one of the two gems you are using - in which case you could fix it by installing latest versions.
I got an error JSON::GeneratorError: source sequence is illegal/malformed utf-8 when trying to convert a hash into json string. I am wondering if this has anything to do with encoding, and how can I make to_json just treat \xAE as it is?
$ irb
2.0.0-p247 :001 > require 'json'
=> true
2.0.0-p247 :002 > a = {"description"=> "iPhone\xAE"}
=> {"description"=>"iPhone\xAE"}
2.0.0-p247 :003 > a.to_json
JSON::GeneratorError: source sequence is illegal/malformed utf-8
from (irb):3:in `to_json'
from (irb):3
from /Users/cchen21/.rvm/rubies/ruby-2.0.0-p247/bin/irb:16:in `<main>'
\xAE is not a valid character in UTF-8, you have to use \u00AE instead:
"iPhone\u00AE"
#=> "iPhone®"
Or convert it accordingly:
"iPhone\xAE".force_encoding("ISO-8859-1").encode("UTF-8")
#=> "iPhone®"
Every string in Ruby has a underlaying encoding. Depending on your LANG and LC_ALL environment variables, the interactive shell might be executing and interpreting your strings in a given encoding.
$ irb
1.9.3p392 :008 > __ENCODING__
=> #<Encoding:UTF-8>
(ignore that I’m using Ruby 1.9 instead of 2.0, the ideas are still the same).
__ENCODING__ returns the current source encoding. Yours will probably also say UTF-8.
When you create literal strings and use byte escapes (the \xAE) in your code, Ruby is trying to interpret that according to the string encoding:
1.9.3p392 :003 > a = {"description" => "iPhone\xAE"}
=> {"description"=>"iPhone\xAE"}
1.9.3p392 :004 > a["description"].encoding
=> #<Encoding:UTF-8>
So, the byte \xAE at the end of your literal string will be tried to be treated as a UTF-8 stream byte, but it is invalid. See what happens when I try to print it:
1.9.3-p392 :001 > puts "iPhone\xAE"
iPhone�
=> nil
You either need to provide the registered mark character in a valid UTF-8 encoding (either using the real character, or providing the two UTF-8 bytes):
1.9.3-p392 :002 > a = {"description1" => "iPhone®", "description2" => "iPhone\xc2\xae"}
=> {"description1"=>"iPhone®", "description2"=>"iPhone®"}
1.9.3-p392 :005 > a.to_json
=> "{\"description1\":\"iPhone®\",\"description2\":\"iPhone®\"}"
Or, if your input is ISO-8859-1 (Latin 1) and you know it for sure, you can tell Ruby to interpret your string as another encoding:
1.9.3-p392 :006 > a = {"description1" => "iPhone\xAE".force_encoding('ISO-8859-1') }
=> {"description1"=>"iPhone\xAE"}
1.9.3-p392 :007 > a.to_json
=> "{\"description1\":\"iPhone®\"}"
Hope it helps.
I'm trying to parse some JSON containing escaped unicode characters using JSON.parse. But on one machine, using json/ext, it gives back incorrect values. For example, \u2030 should return E2 80 B0 in UTF-8, but instead I'm getting 01 00 00. It fails with either the escaped "\\u2030" or the unescaped "\u2030".
1.9.2p180 :001 > require 'json/ext'
=> true
1.9.2p180 :002 > s = JSON.parse '{"f":"\\u2030"}'
=> {"f"=>"\u0001\u0000\u0000"}
1.9.2p180 :003 > s["f"].encoding
=> #<Encoding:UTF-8>
1.9.2p180 :004 > s["f"].valid_encoding?
=> true
1.9.2p180 :005 > s["f"].bytes.map do |x| x; end
=> [1, 0, 0]
It works on my other machine with the same version of ruby and similar environment variables. The Gemfile.lock on both machines is identical, including json (= 1.6.3). It does work with json/pure on both machines.
1.9.2p180 :001 > require 'json/pure'
=> true
1.9.2p180 :002 > s = JSON.parse '{"f":"\\u2030"}'
=> {"f"=>"‰"}
1.9.2p180 :003 > s["f"].encoding
=> #<Encoding:UTF-8>
1.9.2p180 :004 > s["f"].valid_encoding?
=> true
1.9.2p180 :005 > s["f"].bytes.map do |x| x; end
=> [226, 128, 176]
So is there something else in my environment or setup that could be causing it to parse incorrectly?
Recently ran into this same problem, and I tracked it down to this Ruby bug caused by the declaration of this buffer in Ruby 1.9.2 and how it gets optimized by GCC. It's fixed in this commit.
You can recompile Ruby with -O0 or use a newer version of Ruby (1.9.3 or better) to fix it.
Try upgrade your JSON Gem (at least to 1.6.6) or newest 1.7.1.
I am using JSON implementation for Ruby in my rails project to parse the JSON string sent by ajax, but I found that although the json string is in UTF-8, the result coming out is in ASCII-8BIT by default, see below
jruby-1.6.7 :068 > json_text = '["に到着を待っている"]'
=> "[\"に到着を待っている\"]"
jruby-1.6.7 :069 > json_text.encoding
=> #<Encoding:UTF-8>
jruby-1.6.7 :070 > json_parsed = JSON.parse(json_text)
=> ["\u00E3\u0081\u00AB\u00E5\u0088\u00B0\u00E7\u009D\u0080\u00E3\u0082\u0092\u00E5\u00BE\u0085\u00E3\u0081\u00A3\u00E3\u0081\u00A6\u00E3\u0081\u0084\u00E3\u0082\u008B"]
jruby-1.6.7 :071 > json_parsed.first.encoding
=> #<Encoding:ASCII-8BIT>
I don't want it being escaped, I would like to have a UTF-8 result. Is there a way to set that? I check the documentation of the JSON project, finding not encoding options for the method JSON.parse. Maybe I missed something, how could I do that?
UPDATE:
as notified by #fl00r, this example is working fine in MRI, but not in JRUBY
This looks like a bug, as this actually works when using the pure version:
jruby-1.6-head :001 > require 'json/pure'
=> true
jruby-1.6-head :002 > json_text = '["に到着を待っている"]'
=> "[\"に到着を待っている\"]"
jruby-1.6-head :003 > json_parsed = JSON.parse(json_text)
=> ["に到着を待っている"]
jruby-1.6-head :004 > json_parsed.first.encoding
=> #<Encoding:UTF-8>
jruby-1.6-head :005 >
Edit: Just saw you opened a ticket for this...
Edit 2: This actually seems to have already been fixed by this commit. To install latest code from json:
$ git clone https://github.com/flori/json.git
$ cd json
$ rake jruby_gem
$ jruby -S gem install pkg/json-1.6.6-java.gem