I am trying to upload data as multipart using RestClient like so:
response = RestClient.post(url, io, {
:cookies => {
'JSESSIONID' => #sessionid
},
:multipart => true,
:content_type => 'multipart/form-data'
})
The io argument is a StringIO that contains my file, so it's from memory instead of from the disk.
The server (Tomcat servlet) is unable to read the multipart data, giving an error:
org.apache.commons.fileupload.FileUploadException: the request was rejected because no multipart boundary was found
So I believe that RestClient is not sending it in multipart format? Anyone see the problem? I am assuming the problem is on the Ruby (client) side, but I can post my servlet (Spring) code if anyone thinks it might be a server-side problem.
I also wonder what RestClient would use for the uploaded filename, since there isn't an actual file... Can you have a multipart request without a filename?
You can do this, it simply requires subclassing StringIO and adding a non-nil path method to it:
class MailIO < StringIO
def path
'message'
end
end
I've just checked this, and the Mailgun api is pretty down with this.
After consulting with the author of the rest-client library (Archiloque), it seems that if this is possible, the API is not set up to handle it easily. Using the :multipart => true parameter will cause the IO to be treated like a file, and it looks for a non-nil #path on the IO, which for a StringIO is always nil.
If anyone needs this in the future, you'll need to consult with the library's mailing list (code#archiloque.net), as the author seems to think it is possible but perhaps not straightforward.
It CAN easily do streaming uploads from an IO as long as it's not multipart format, which is what I ended up settling for.
Related
I'm currently stuck with a response time problem with the Faraday HTTP library. I'm using the following code to initialize a POST request to a server running on my local machine at 127.0.0.1. The server returns a valid JSON string. The code to initialize the Faraday object on the client side is as follows:
url = 'http://127.0.0.1'
conn = Faraday.new(:url => url) do |faraday|
faraday.request :url_encoded
faraday.response :json, :content_type => 'application/json'
faraday.adapter Faraday.default_adapter
end
Then I send a JSON string via a POST request to the server. The code for sending the request looks like this (text size up to 5000 char):
payload = {:language => 'en', :text => 'some text'}.to_json
response = conn.post do |req|
req.url '/api'
req.headers['Content-Type'] = 'application/json'
req.body = payload
end
The expected result is a JSON string of the following structure:
{
"level1.1" : [
{
"level2.1" : [
{
"value1" : "Text",
"value2" : "Text",
"value(...)" : "Text",
"value(n)" : "Text"
},
{...
}
],
<and so on... - of course in a valid JSON structure ending>
When I run the code without doing anything with the result it performs pretty well and finishes in a reasonable time (< 0.5s). But as soon as I try to access the response object the script becomes terrible slow.
Just by adding the line:
p response.body
The processing time goes up to > 8s.
I have checked the server response with Postman and it works absolutely fine without any visible problems. Response time in Postman is >0.5 s as well. The slow down appears only on the client side when I try to access the response object. Just inspecting the response object doesn't effect the processing time either. But as soon as I start "to do" something with the response it becomes terribly slow.
I'm running Ruby 2.5.3 and Faraday 0.15.4 / Middleware 0.12.2
Any idea what might cause this slowdown is greatly appreciated.
Krid
EDIT
Despite my comment that HTTParty solved this issue this was just due to a smaller payload. HTTParty and Faraday both perform badly on my POST request while a POST request from Postman runs very fast even with large payloads. I have no idea what causes this different runtime behavior when querying exactly the same server app with exactly the same payload.
As said any ideas that might point me in the right direction greatly appreciated.
Krid
...after searching for a solution for days I finally found the issue. When you have Python and Ruby code combined within a folder that is connected with a virtualenv Python (and activated Python env of course) Ruby apparently has problems resolving the local address space. It works but it takes some strange detours that make the whole thing terribly slow.
My solution: I moved the Ruby code that has to play together with the Python code via HTTP connections in a folder outside of the Python code folder.
I am trying to perform a file upload to google drive's API using Rest calls. The API says that if you want to name the file metadata must be passed and it should be uploaded as multipart.
I am getting a '400 Bad Request' error. I think that this might be due to having multiple content types using Rest client.
RestClient::Request.execute(
:method => "post",
:url => "https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart",
:headers => {:Authorization => "Bearer #{access_token}", :content_type => 'multipart/related'},
:payload => {:metadata => "{'title' : 'testing'}", :file => File.new(file, 'rb'), :multipart => true}
)
Any help would be great! Thanks!
I believe, using the given library (rest-client), this is not possible as the request, according to Google's requirements, the multi-parts need to be ordered and must have the correct mime type for each of the multipart.
So, I'd say, the syntax you've used is closest to correct, but unfortunately the gem doesn't seem to support this [1] as of the time of this comment.
If you still are looking for a solution, the closest thing I found was digging directly into Google's API client library, where they hand-craft a multipart request with the help of hurley [2].
You can check the source code for some ideas.[3]
Hope this helps.
[1] https://github.com/rest-client/rest-client/pull/222
[2] https://github.com/lostisland/hurley
[3] https://github.com/google/google-api-ruby-client/blob/d2e51b4e7d4cb5a18cb08b2aed9c0d8ffff14b22/lib/google/apis/core/multipart.rb
I am working on a data integration app which need to fetch images from one API (with XML's urls) and post the images to a rails built REST API.
I tried paperclip to download all the images however don't know how to handle the Paperclip::Attachment type when trying to post the images with HTTMultiParty.
I am thinking about use open-uri instead of paperclip which will store file into binary. Can anyone give me an example on that? And is there any good option for posting image to API apart from httmultiparty.
It's better to answer this question myself because the solution can be varied.
So image fetch and feed through api can be done by httparty(download&upload text)+paperclip(download image by url)+httmultiparty(upload image), here are some code example I use in my application.
To me, httparty is easiest way to deal with api, codes can be easily done like this:
response = HTTParty.get('url')
response = HTTParty.post('url',
:headers => 'head content',
:body => {'data':'data content'})
Code example on paperclip is here: answer on stack over flow
The important part is parsing the paperclip image to binary file, code goes:
Paperclip.io_adapters.for(productData[0].image).read
The last example is HTTmultiparty, When you pass a query with an instance of a File as a value for a PUT or POST request, the wrapper will use a bit of magic and multipart-post to execute a multipart upload,apart from that it is pretty much the same as httparty:
class ImgClient
include HTTMultiParty
base_uri 'http://localhost:3000'
end
respond = ImgClient.post('url',
:headers => head,
:query => {
:image => Paperclip.io_adapters.for(product.image)
})
Hope this will be helpful for other api newbies.
I'm developing a RESTful web application in Ruby with Sinatra. It should support CRUD operations, and to respond to Read requests I have the following function that formats the data according to what the request specified:
def handleResponse(data, haml_path, haml_locals)
case true
when request.accept.include?("application/json") #JSON requested
return data.to_json
when request.accept.include?("text/html") #HTML requested
return haml(haml_path.to_sym, :locals => haml_locals, :layout => !request.xhr?)
else # Unknown/unsupported type requested
return 406 # Not acceptable
end
end
Only I don't know what is best to do in the else statement. The main problem is that browsers and jQuery AJAX will accept */*, so technically a 406 error is not really the best idea. But: what do I send? I could do data.to_s which is meaningless. I could send what HAML returns, but they didn't ask for text/html and I would rather notify them of that somehow.
Secondly, supposing the 406 code is the right way to go, how do I format the response to be valid according to the W3 spec?
Unless it was a HEAD request, the response SHOULD include an entity containing a list of available entity characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content-Type header field. Depending upon the format and the capabilities of the user agent, selection of the most appropriate choice MAY be performed automatically. However, this specification does not define any standard for such automatic selection.
It looks like you're trying to do a clearing-house method for all the data types you could return, but that can be confusing for the user of the API. Instead, they should know that a particular URL will always return the same data type.
For my in-house REST APIs, I create certain URLs that return HTML for documentation, and others that return JSON for data. If the user crosses the streams, they'll do it during their development phase and they'll get some data they didn't expect and will fix it.
If I had to use something like you're writing, and they can't handle 'application/json' and can't handle 'text/html', I'd return 'text/plain' and send data.to_s and let them sort out the mess. JSON and HTML are pretty well established standards now.
Here's the doc for Setting Sinatra response headers.
I'm using Net::HTTP in Ruby 1.9.2p290 to handle some, obviously, networking calls.
I now have a need to see the complete request that is sent to the server (as one long big String conforming to HTTP 1.0/1.1.
In other words, I want Net::HTTP to handle the heavy lifting of generating the HTTP standard-compliant request+body, but I want to send the string with a custom delivery mechanism.
Net::HTTPRequest doesn't seem to have any helpful methods here -- do I need to go lower down the stack and hijack something?
Does anyone know of a good library, maybe other than Net::HTTP, that could help?
EDIT: I'd also like to do the same going the other way (turning a string response into Net::HTTP::* -- although it seems I may be able to instantiate Net::HTTPResponse by myself?
Request:
post = Net::HTTP::Post.new('http://google.com')
post.set_form_data :query => 'ruby http'
sio = StringIO.new
post.exec si, Net::HTTP::HTTPVersion, post.path
puts sio.string
Response:
si = StringIO.new("HTTP/1.1 200 OK\n")
bio = Net::BufferedIO.new(si)
Net::HTTPResponse.read_new(bio)