I have always used open-uri, and open("").read to get content through http. I am using this to access an API, and now I need to specify that what content type is accepted. They provide this example from curl
curl -LH "Accept: text/bibliography; style=bibtex" http://dx.doi.org/10.1038/nrd842
How do I do this with open-uri or another Ruby function?
I'd suggest using the optional options hash of open, e.g.
open("http://dx.doi.org/10.1038/nrd842","Accept" => "text/bibliography; style=bibtex"){|f| f.each {|line| print line}}
yields a result.
See also http://www.ruby-doc.org/stdlib-1.9.3/libdoc/open-uri/rdoc/OpenURI.html
Related
Firstly I want to make clear that I am not familiar with Ruby, at all.
I'm building a Discord Bot in Go as an exercise, the bot fetches UrbanDictionary definitions and sends them to whoever asked in Discord.
However, UD doesn't have an official API, and so I'm using this. It's an Heroku App written in Ruby. From what I understood, it scrapes the UD page for the given search.
I want to add random to my Bot, however the API doesn't support it and I want to add it.
As I see it, it's not hard since http://www.urbandictionary.com/random.php only redirects you to a normal link of the site. This way if I can follow the link to the "normal" one, get the link and pass it on the built scraper it can return just as any other link.
I have no idea how to follow it and I was hoping I could get some pointers, samples or whatsoever.
Here's the "ruby" way using net/http and uri
require 'net/http'
require 'uri'
uri = URI('http://www.urbandictionary.com/random.php')
response = Net::HTTP.get_response(uri)
response['Location']
# => "http://www.urbandictionary.com/define.php?term=water+bong"
Urban Dictionary is using an HTTP redirect (302 status code, in this case), so the "new" URL is being passed back as an http header (Location). To get a better idea of what the above is doing, here's a way just using curl and a system call
`curl -I 'http://www.urbandictionary.com/random.php'`. # Get the headers using curl -I
split("\r\n"). # Split on line breaks
find{|header| header =~ /^Location/}. # Get the 'Location' header
split(' '). # Split on spaces
last # Get the last element in the split array
I have this Sinatra::Base code:
class Crush < Sinatra::Base
post '/upload' do
erb params.inspect
end
end
I am using Postman and its interface for uploading a file. So I send a POST request with form-data, where in the body of the request the name is hello and the value is a file test.txt which contains just a simple string hey there.
When I do params.inspect I get this long string
{"------WebKitFormBoundaryocOEEr26iZGSe75n\r\nContent-Disposition: form-data; name"=>"\"hello\"; filename=\"test.txt\"\r\nContent-Type: text/plain\r\n\r\nhey there\r\n------WebKitFormBoundaryocOEEr26iZGSe75n--\r\n"}
So basically a long has with a single key and a single value. Reading most Sinatra tutorials (where the file is accepted from a form), there's a nice way Sinatra handles this using params[:file], but this doesn't seem to be the case when the file is coming straight from the body of an HTTP request.
I tried a non-modular approach too withou Sinatra::Base, thinking it's some parsing middle-ware missing, but got the same result.
Is there something I'm missing here? Must I go and make my own custom parser to get the content of this long hash? Or is there an easier way?
I figured it's Postman issue. When I switch from 'x-www-form-urlencoded' to 'form-data' in Postman, in the Header section, the field: Content-Type => application/x-www-form-urlencoded is NOT removed. So for those who encounter this problem, make sure you remove it manually.
When connecting to a website using Net::HTTP you can parse the URL and output each of the URL headers by using #.each_header. I understand what the encoding and the user agent and such means, but not what the "accept"=>["*/*"] part is. Is this the accepted payload? Or is it something else?
require 'net/http'
uri = URI('http://www.bible-history.com/subcat.php?id=2')
http://www.bible-history.com/subcat.php?id=2>
http_request = Net::HTTP::Get.new(uri)
http_request.each_header { |header| puts header }
# => {"accept-encoding"=>["gzip;q=1.0,deflate;q=0.6,identity;q=0.3"], "accept"=>["*/*"], "user-agent"=>["Ruby"], "host"=>["www.bible-history.com"]}
From https://www.w3.org/Protocols/HTTP/HTRQ_Headers.html#z3
This field contains a semicolon-separated list of representation schemes ( Content-Type metainformation values) which will be accepted in the response to this request.
Basically, it specifies what kinds of content you can read back. If you write an api client, you may only be interested in application/json, for example (and you couldn't care less about text/html).
In this case, your header would look like this:
Accept: application/json
And the app will know not to send any html your way.
Using the Accept header, the client can specify MIME types they are willing to accept for the requested URL. If the requested resource is e.g. available in multiple representations (e.g an image as PNG, JPG or SVG), the user agent can specify that they want the PNG version only. It is up to the server to honor this request.
In your example, the request header specifies that you are willing to accept any content type.
The header is defined in RFC 2616.
When I run the curl command
curl -v -H "Content-type: application/json" -X POST -d '{"name":"abc", "id":"12", "subject":"my subject"}' http://localhost:9292
to send a POST request with data to my Rack application, my code prints out {}. That is coming from puts req.POST() in the code below.
Why does it print out {} instead of the POST data? And how do I correctly access the POST data in my Rack application?
require 'json'
class Greeter
def call(env)
req = Rack::Request.new(env)
if req.post?
puts req.POST()
end
[200, {"Content-Type" => "application/json"}, [{x:"Hello World!"}.to_json]]
end
end
run Greeter.new
From reading the docs for POST, looks like it is giving you parsed data based on other content types. If you want to process "application/json", you probably need to
JSON.parse( req.body.read )
instead. To check this, try
puts req.body.read
where you currently have puts req.POST.
req.body is an I/O object, not a string. See the body documentation and view the source. You can see that this is in fact the same as mudasobwa's answer.
Note that other code in a Rack application may expect to read the same I/O, such as the param parsers in Sinatra or Rails. To ensure that they see the same data and not get an error, you may want to call req.body.rewind, possibly both before and after reading the request body in your code. However, if you are in such a situation, you might instead consider whether your framework has options to process JSON directly via some option on the controller or request content-type handler declaration etc - most likely there will be an option to handle this kind of request within the framework.
Try:
env['rack.input'].read
I found it in "How to receive a JSON object with Rack" and, though it still sounds weird to me, it likely works.
You can try:
req.params
Hope this can help you.
I have a very simple Sinatra app which only does log out the params in the action, and then I use curl to send post data in xml format, but sinatra didn't get the xml parsed:
echo '<something>tyrael tong</something>' | curl -X POST -H 'Content-Type: text/xml' -d #- http://localhost:9528/status/update
I searched through google with no solution to this. Am I suppose to parse the xml post data by myself?
try this code
require 'plist'
post 'some/route'
content_type :xml
value=Plist::parse_xml(request.body)
end
Find a rack parser: https://github.com/achiu/rack-parser which could do the job I want: parse the post body into parameter.
P.S. And need to set the content type to "application/xml"
Yes, content type is just a hint for a server how to handle it. If your server can receive say XML or JSON, content type can tell you how to parse it.
As the body of your post message is an xml it will not be parsed by either rails/sinatra. If you want parsing to be done you will need to change the format of data that you are sending and set the content type to "application/x-www-form-urlencoded". Then Sinatra will parse that data and put it into the params hash.