I'm still new to Ruby and trying to use the HTTParty gem to help me write an API Wrapper. I feed HTTParty::get a URI and it parses JSON data. From a quick glance and the way the returned result behaves, it looks like a Hash, but is it? I can't seem to find information online. Another post on StackOverflow shows to use HTTParty::get(...).parsed_response to get the Hash, but this seems outdated.
Do this in the console:
>require 'httparty'
=> true
> response = HTTParty.get( "..." )
....
> response.class
=> HTTParty::Response
So the response from HTTParty.get is an HTTParty::Response object.
See this blogpost titled "It's Time To HTTParty!" to learn more about how to work this response.
Related
This question is probably poorly structured so please bear with me, I'm new at this.
I'm trying to build a simple web scraper but every time i run my code i get this warning in terminal. I have tried to follow the link to the github issues hoping i would get a clearer explanation but i didn't understand it there either. Tried googling, but nothing there either.
[DEPRECATION] HTTParty will no longer override `response#nil?`.
This functionality will be removed in future versions.
Please, add explicit check `response.body.nil? || response.body.empty?`.
For more info refer to: https://github.com/jnunemaker/httparty/issues/568
I guess what i want to know is, in basic terms, what does this warning mean? and is there something i should do different from now on when using the HTTParty gem?
This was annoying me as well. You are probably passing the response from HTTParty.get to some other code that is testing for nil. I was passing the body to Nokogiri::HTML and that code was eventually calling HTTParty.response.nil?. To fix, I started sending the "body" to Nokogiri instead of the response object. Quite a few people will probably run into this as the Nokogiri example code with this pattern is all over the place.
I now use this:
doc = HTTParty.get('https://some/site')
parsed ||= Nokogiri::HTML(doc.body)
instead of before when getting the warning it was:
doc = HTTParty.get('https://some/site')
parsed ||= Nokogiri::HTML(doc)
url = "https://www.startech.com.bd/laptop-notebook/laptop"
unparsed_page = HTTParty.get(url)
parsed_page = Nokogiri::HTML(unparsed_page)
I was facing same error as yours
[DEPRECATION] HTTParty will no longer override `response#nil?`. This functionality will be removed in future versions. Please, add explicit check `response.body.nil? || response.body.empty?`. For more info refer to: https://github.com/jnunemaker/httparty/issues/568
/home/abir/.rvm/gems/ruby-2.7.2/gems/nokogiri-1.11.1-x86_64-linux/lib/nokogiri/html/document.rb:209:in `parse'
So changed the code as Jeremy Mullin said and it is working fine now.
url = "https://www.startech.com.bd/laptop-notebook/laptop"
unparsed_page = HTTParty.get(url)
parsed_page = Nokogiri::HTML(unparsed_page.body)
I want to use the API of a website in a Ruby script, and the only return from the API is a number through the HTTPS protocol. Nothing more, not even tags or something, so I was wondering if there is a way to get that number in a string or integer in my script without using any XML parsing livrary or gem like REXML or hpricot or libXML, because the webpages that I want to parse are, as I said, extremely basic...
If I understand. A request to https://www.website.com/api/getid return 2.
Then, I guess this would do:
require 'net/https'
require 'uri'
def open(url)
Net::HTTP.get(URI.parse(url))
end
response = open("https://www.website.com/api/getid")
EDIT
You'll find much usefull examples here.
As it is mentioned in the link above, HTTParty is quite popular. An example:
require 'httparty'
response = HTTParty.get('http://twitter.com/statuses/public_timeline.json')
puts response.body, response.code, response.message, response.headers.inspect
I'm trying to update a style for one of my Fusion Tables by using the Ruby gem RestClient.
Here's my code:
require 'rest_client'
tableId = '<STRING CONTAINING TABLEID>'
styleId = '<STRING CONTAINING STYLEID>'
key = '<STRING CONTAINING MY FUSION TABLES API KEY>'
table_url = "https://www.googleapis.com/fusiontables/v1/tables/#{tableId}/styles/#{styleId}?key=#{key}"
update = '{"polygonOptions": {"strokeColor":"#ffffff"}}'
token = 'STRING CONTAINING AUTHORIZATION TOKEN'
RestClient.put table_url,update,{"Authorization" => "Bearer #{token}"}
When I run that code, I get this error:
C:/Ruby193/lib/ruby/gems/1.9.1/gems/rest-client-1.6.7/lib/restclient/abstract_response.rb:48:in `return!': 400 Bad Request (RestClient::BadRequest)
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/rest-client-1.6.7/lib/restclient/request.rb:230:in `process_result'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/rest-client-1.6.7/lib/restclient/request.rb:178:in `block in transmit'
from C:/Ruby193/lib/ruby/1.9.1/net/http.rb:745:in `start'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/rest-client-1.6.7/lib/restclient/request.rb:172:in `transmit'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in `execute'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in `execute'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/rest-client-1.6.7/lib/restclient.rb:80:in `put'
When I input the update code into Google's official Style request PUT maker thingie, the update works. But it does not work when I run my Ruby code.
Does anyone know what I am doing wrong?
EDIT: Extra output I get from adding in RestClient.log = logger
RestClient.put "https://www.googleapis.com/fusiontables/v1/tables/<MY TABLE ID HERE>/styles/4?key=<AND HERE'S WHERE MY FUSION TABLE API KEY GOES>", "{\"polygonOptions\":{\"strokeColor\":\"#ffffff\"}}", "Accept"=>"*/*; q=0.5, application/xml", "Accept-Encoding"=>"gzip, deflate", "Authorization"=>"Bearer <THIS CONTAINS THE BEARER STRING>", "Content-Length"=>"44"
# => 400 BadRequest | application/json 255 bytes
You really should be using the google-api-ruby-client library instead of building your own REST calls. The library abstracts a lot of the OAuth stuff and formatting of the parameters for you.
Having said that, can you enable debugging for your RestClient and post the output of the RestClient call along with the output from Google official PUT maker thingie (I like your technical jargon there)? Comparing the two should show how they differ and what Google doesn't like with yours.
I think the problem is that you are not setting the content-type to application/json. Try to do something like this:
RestClient.put(table_url, update, {"Authorization" => "Bearer #{token}", "Content-type" => "application/json"})
The payload in this case needs to be json so you can either use your json-string from your example or run to_json on you data structure.
In answer to your question about setting up a logger without Rails, in the comments to #Jay Lee's answer…
Here is a logger set up to output to standard out:
logger = Logger.new STDOUT
logger.level = Logger::WARN # INFO/DEBUG… whatever level you find is needed
logger.datetime_format = '%a %d-%m-%Y %H%M '
Then put the rest of your code into a console (e.g. IRB) and before the last line add:
RestClient.log = logger
and you should get some helpful information output to the terminal. See the documents for the Logger class for more information on the levels available.
It could be because your hash here
update = '{"polygonOptions": {"strokeColor":"#ffffff"}}'
should probably be
update = {"polygonOptions" => {"strokeColor" => "#ffffff"}}
Good luck!
FYI,
Alternatives to try:
1) Remove HTTPS request in project settings and token regeneration.
2) Try to use SSL in that case.
3) Sometimes this error occurs when the values exceeds more than 255 characters that may be the possibility here with you. The same problem occurred sometime back with someone and was resolved after debugging. Check the link for more information.
I was having the same 400 Bad Request problem, specifically when posting styles. I was able to solve the problem by making sure any values for "kind" in the style were namespaced, which the examples in the docs don't always get right -- for example:
{
...
"kind": "fusiontables#buckets"
...
}
Instead of just "buckets".
When I do a find query from the mongodb JavaScript console, I get a beautiful JSON response. However, when I do a find with the ruby driver, it stores the JSON as some Ruby object. How do I convert this back to JSON? Or, better yet, how do I just get the JSON without doing the extra conversions?
I got it working. I was using ruby-1.9.1-p378 with rvm. I removed that. Now, I'm using the system ruby-1.8.7-p174 that came with the SnowLeopard install DVD. But, I was still getting an error with the to_json method except this time it was saying, stack level too deep. I did:
require 'json/pure'
instead of
require 'json'
Then, I changed the code to look something like this:
http://github.com/banker/mongulator/blob/master/mongulator.rb#L53
Here's the relevant part of the code:
cursor = persons.find(
{"loc" => {"$near" => [params[:lat].to_f, params[:lng].to_f]}},
{:limit => 20})
content_type "application/json"
JSON.pretty_generate(cursor.to_a)
The complete file is here:
http://github.com/acani/acani-sinatra/blob/master/acani.rb
And, it worked, even with pretty json like the facebook graph api gives you. To return the JSON all on one line, just do something like:
cursor.to_a.to_json
The JSON Code you get on the javascript console is also converted. It's not the native output of MongoDB, the Native format is BSON. To Get JSON on the javascript console it must be converted. In Ruby you should be able to do the same thing with the .to_json instance method of your Object.
now you can do cursor.find.to_a.inspect
I'm working on an application that reaches out to a web service. I'd like to develop a proxy class that returns a fake response from the service, so I don't have to constantly be hitting it with requests while I'm developing/testing other parts of the app.
My application is expecting a response generated via Net::HTTP.
response = Net::HTTP.get(URI.parse('http://foo.com'))
case response
when Net::HTTPOK
# do something fun
when Net::HTTPUnauthorized
# you get the idea
How can I manufacture a response object, give it all the right headers, return a body string, etc?
response = ProxyClass.response_object
case response
when Net::HTTPOk
# my app doesn't know it's being lied to
Thanks.
It's actually not that hard to roll your own fake responses directly with Net::HTTP. Here's a simple 200 OK with a cookie header:
def fake_response
net_http_resp = Net::HTTPResponse.new(1.0, 200, "OK")
net_http_resp.add_field 'Set-Cookie', 'Monster'
RestClient::Response.create("Body goes here", net_http_resp, nil)
end
Since few of us are using raw Net::HTTP anymore, the (optional) last line wraps it up as a RestClient::Response, which can then be stubbed into RestClient:
stub(RestClient).post(anything) { fake_response }
I would start with FakeWeb and see if that meets your needs. If it doesn't you can probably gut whatever you need out of the internals and create your own solution.
I know this post is old, but instead of FakeWeb which seems to be largely dead, try webmock. It seems to be more full-featured and very active.
I would look into a mocking library like mocha.
Then you should be able to setup a mock object to help test:
Then following example is from Tim Stephenson's RaddOnline blog, which also includes a more complete tutorial:
def setup
#http_mock = mock('Net::HTTPResponse')
#http_mock .stubs(:code => '200', :message => "OK", :content_type => > "text/html", :body => '<title>Test</title><body>Body of the page</body>')
end
For testing a web service client, we use Sinatra, a lovely little lightweight web framework that lets you get something up and running very quickly and easily. Check out the home page; it has an entire Hello World app in 5 lines of code, and two commands to install and run the whole thing.
I ended up using a Struct.
FakeHttpResponse = Struct.new(:status, :body)
http = FakeHttpResponse.new('success', 'body goes here')
http['status'] # = 'success'
http.body # = 'body goes here'
The drawback is that .status and ['body'] are also valid, but I don't think that matters much.
I would either use FakeWeb as mentioned above, or have my rake test task start a Webrick instance to a little sinatra app which mocks the various test responses you're hoping to see.
You could look into using Rack for this which should allow you to do everything you need.