How to send a multipart streaming request using httpclient in ruby - ruby

Question

I'm trying to get a multipart streaming post working with HTTPClient for Ruby. Upon which I encounter two problems.
Problem 1:
First, if I try to do a normal post via the way described in the docs; and I return it via httpbin.org
I see this happening:
Code
File.open(path_to_my_file) do |file|
body = [{ 'Content-Type' => 'application/atom+xml; charset=UTF-8',
:content => '<entry>...</entry>' },
{ 'Content-Type' => 'video/mp4',
'Content-Transfer-Encoding' => 'binary',
:content => file }]
res = #http_client.post('http://httpbin.org/post', body: body)
response = res
puts res.body
Result
{
"args": {},
"data": "",
"files": {},
"form": {
"{\"Content-Type\"=>\"application/atom+xml; charset=UTF-8\", :content=>\"<entry>...</entry>\"}": "",
"{\"Content-Type\"=>\"video/mp4\", \"Content-Transfer-Encoding\"=>\"binary\", :content=>#<File:{{path_to_file}}>}": ""
},
"headers": {
"Accept": "*/*",
"Content-Length": "322",
"Content-Type": "application/x-www-form-urlencoded",
"Date": "Mon, 20 May 2019 06:43:17 GMT",
"Host": "httpbin.org",
"User-Agent": "HTTPClient/1.0 (2.8.3, ruby 2.6.0 (2018-12-25))"
},
"json": null,
"origin": "IP, IP",
"url": "https://httpbin.org/post"
}
As you can see, I cannot see the content of the file, but only the identifier containing the URI to the File; and as such, I don't know how to fix it so that I can see and return the content. As far as I know it looks like it tries to see the File object as a String, which is of course not what I'd like it doing.
Problem2:
Whenever I create the body dynamically, that is; creating the hash-array dynamically with objects set in my code, and I try to send it asynchronously (which I believe is the correct way of getting it streaming in HTTPClient for Ruby, although I'm not entirely sure)
it sends the entire body as data, and no longer as either a form or headers.
code
request_body = []
body.files.each do |k, v|
request_body.push( { 'Content-Type' => v.content_type, :content => v.content })
end
body.values.each { |k, v| request_body << { k.to_sym => v }}
#This creates the array correctly, although I just wanted to show how it was generated
connection = #http_client.send(method + '_async', uri, body: request_body, header: headers)
response = connection.pop
# Starts reading result here
Response
"args": {},
"data": "%7B%22Content-Type%22%3D%3E%22video%2Fmp4%22%2C+%3Acontent%3D%3E%23%3CFile%3A%2Fhome%2Fuser%2Ffiles%2file.mp4%3E%7D=&%7B%3Avalue%3D%3E%22hello+world%22%7D=",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Content-Length": "247",
"Content-Type": "application/json",
"Date": "Mon, 20 May 2019 06:44:11 GMT",
"Host": "httpbin.org",
"User-Agent": "HTTPClient/1.0 (2.8.3, ruby 2.6.0 (2018-12-25))",
"ApplicationIdentifier": "identifier"
},
"json": null,
"origin": "IP, IP",
"url": "https://httpbin.org/post"
}
As you can see, it puts everything in data, and I honestly don't know how to get it to send the post with the body as a form instead of as data. I send the file here as a File object.
I have tried sending it as a normal post instead of post_async, but it does not seem to be working.
Has anyone encountered these problems before and know how I should go about fixing these? (or can you see where I went wrong so that I can at least try getting a bit further)
Thanks in advance

Answer 1

Regarding your first issue, it appears that the documentation is not correct. I've performed some tests, and the content type appears to be application/x-www-form-urlencoded. You need to set the content type explicitly:
body: body, header: {'Content-Type': 'multipart/form-data; boundary=ABC'}
That's not sufficient though. You also need to set the content disposition manually. For instance:
body = [{ 'Content-Type' => 'application/atom+xml; charset=UTF-8',
'Content-Disposition' => 'form-data; name="name1"',
:content => '<entry>...</entry>' },
{ 'Content-Type' => 'video/mp4',
'Content-Disposition' => 'form-data; name="file"; filename="myfile.pdf"',
'Content-Transfer-Encoding' => 'binary',
:content => file }]
With this, httpbin reports one file and one form parameter.
Your second issue is related to the same. Both your files and values need to have the content disposition set. For instance (quick and dirty, style can probably be better):
files.each do |k, v|
request_body << { 'Content-Type' => v.content_type, :content => v.content, 'Content-Disposition' => 'form-data; name="' + k + '"; filename="' + v.filename + '"' }
end
values.each { |k, v| request_body << { :content => v, 'Content-Disposition' => 'form-data; name="' + k + '"' }}
Note that you should escape any " in a name or filename.