Adapt multipart/form-data request to curl with image and json in the same request - bash

I need to rewrite this curl request extracted from the browser to execute it from bash, it is a multipart/form-data that combines the upload of an image together with a json in the same request, I have carried out several tests substituting the raw data for two -F but the server responds with error.
What would be the correct way?
I also fail to understand the key that is next to WebKitFormBoundary is a value that seems random, is it just a separator? can i use anything?
I have removed the headers that I considered irrelevant
curl 'https://localhost' \
-H 'Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryXXXXXXXXXXXXXXXX'
-H 'Accept: */*' \
--data-raw $'------WebKitFormBoundaryXXXXXXXXXXXXXXXX\r\nContent-Disposition: form-data; name="image"; filename="image.jpg"\r\nContent-Type: image/jpeg\r\n\r\n\r\n------WebKitFormBoundaryXXXXXXXXXXXXXXXX\r\nContent-Disposition: form-data; name="item"; filename="blob"\r\nContent-Type: application/json\r\n\r\n{"item1":"xxxx","item2":"yyyy"}\r\n------WebKitFormBoundaryXXXXXXXXXXXXXXXX--\r\n' \
Thanks

I also fail to understand the key that is next to WebKitFormBoundary is a value that seems random, is it just a separator? can i use anything?
yes and yes and practically speaking yes. it's supposed to be a random-guaranteed-unique string used as separator to signal start and end of forms.. if you know beforehand that none of your uploads contains the string stackoverflow, you could use Content-Type: multipart/form-data; boundary=stackoverflow for example (but this may break if you upload a file actually containing the string stackoverflow !), but you should let curl generate that string automatically, rather than specifying it yourself, curl will auto-generate a string that is practically guaranteed to be unique.
I have removed the headers that I considered irrelevant
... if you guessed wrong, the server may give you an error for missing a required header. assuimg that you didn't make any mistakes in removing irrelevant headers though, i think it's
curl 'https://localhost'\
--form "image=#image.jpg;type=image/jpeg"\
--form "item=#blob;type=application/json"
the request generated by that command is roghly:
POST / HTTP/1.1
Host: localhost
User-Agent: curl/7.81.0
Accept: */*
Content-Length: 359
Content-Type: multipart/form-data; boundary=------------------------7d0262831ce8f248
--------------------------7d0262831ce8f248
Content-Disposition: form-data; name="image"; filename="image.jpg"
Content-Type: image/jpeg
image.jpg content
--------------------------7d0262831ce8f248
Content-Disposition: form-data; name="item"; filename="blob"
Content-Type: application/json
["json lol"]
--------------------------7d0262831ce8f248--
as you can see above, Accept: */* is curl's default accept-header anyway, so there's no need to specify it, and curl auto-generated the boundary ------------------------7d0262831ce8f248 so there's no need to manually specify a boundary either.

Related

Is it possible to send RingCentral SMS / MMS using multipart/form-data?

The OpenAPI spec for the Create SMS Message endpoint includes the following request content types:
consumes:
- application/json
- multipart/mixed
- multipart/form-data
https://netstorage.ringcentral.com/dpw/api-reference/specs/rc-platform.yml?v=2019082420190816-0828
I found the SMS / MMS instructions to include a multipart/mixed example in the API Reference, but don't see any information on using multipart/form-data. I'm specifically interested in sending files.
https://developers.ringcentral.com/api-reference/SMS/createSMSMessage
The same API Reference shows support for both multipart/form-data and multipart/mixed for sending faxes.
https://developers.ringcentral.com/api-reference/Fax/createFaxMessage
Since both APIs send files and metadata so I'm wondering if the SMS API also supports multipart/form-data and, if so, how to send it?
No, it does not appear so.
The example you'd linked for the SMS message uses multipart/mixed to separate the API call itself (which is in turn sent as application/json) from the payload being sent as an MMS (image/png).
The use of multipart/form-data in the fax API is specific to the way that particular metadata is included, but there isn't an equivalent system for SMS/MMS as they both need that particular meta information encoded either as a single JSON document or as the JSON element of a multipart/mixed message.
To send a file, though, multipart/mixed is fine. Your request would then be something like:
POST /restapi/v1.0/account/403391985008/extension/403391985008/sms
Content-Type: multipart/mixed; boundary=Boundary_1_14413901_1361871080888
--Boundary_1_14413901_1361871080888
Content-Type: application/json; charset=UTF-8
Content-Transfer-Encoding: 8bit
{"to" :[{"phoneNumber": "+18772004569"},{"phoneNumber": "+18772094569"}],
"text" :"hello",
"from" :{"phoneNumber": "+18882004237"}}
--Boundary_1_14413901_1361871080888
Content-Type: application/octet-stream
Content-Disposition: attachment; filename="filename.zip"
[Some encoded binary stream here ...]
--Boundary_1_14413901_1361871080888--
It'd be up to you to set the file's mime type properly and ensure things are encoded. The key points here are that the message information is encoded in the first JSON component in your multipart message, while the file attached to the MMS is encoded in the second.
multipart/form-data can be sent as shown in the following example:
POST / HTTP/1.1
HOST: platform.ringcentral.com/restapi/v1.0/account/~/extension/~/sms
Authorization: Bearer <MyToken>
Content-Type: multipart/form-data; boundary=12345
--12345
Content-Disposition: form-data; name="to"
+16505550101
--12345
Content-Disposition: form-data; name="to"
+16505550102
--12345
Content-Disposition: form-data; name="from"
+16505550100
--12345
Content-Disposition: form-data; name="text"
Hello World
--12345
Content-Disposition: form-data; name="attachment" filename="picture.jpg"
content of picture.jpg ...
--12345--
This can be done using curl as follows:
curl -XPOST https://platform.ringcentral.com/restapi/v1.0/account/~/extension/~/sms \
-H 'Authorization: Bearer <MyToken>' \
-F 'to=+16505550101' \
-F 'to=+16505550102' \
-F 'from=+16505550100' \
-F 'text=Hello World' \
-F 'attachment=#picture.jpg'

Special characters in curl post

I am trying to use the following curl
curl -k 'https://myhost.com:9091/nifi-api/access/token' -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' --data "username=$USERNAME&password=$PASSWORD" --compressed —insecure
But get an error with a % in the password saying
URLDecoder: Incomplete trailing escape (%) pattern
I have seen suggestions online to use --data-urlencode but then I get the following error.
The username and password must be specified.
How can I resolve this issue?
--data-urlencode is the way to go adding it multiple times, one per parameter
userh='vf&re'
passwordh='asdaf%'
curl 'http://127.0.0.1:8080/' --data-urlencode "username=$userh" --data-urlencode "password=$passwordh"
Let's start a netcat listener to check what was sent
netcat -l 8080
Received data:
POST / HTTP/1.1
Host: 127.0.0.1:8080
User-Agent: curl/7.60.0
Accept: */*
Content-Length: 34
Content-Type: application/x-www-form-urlencoded
username=vf%26re&password=asdaf%25
Sending all parameters in one option will not work since password parameter gets lost triggering the server error "The username and password must be specified."
curl 'http://127.0.0.1:8080/' --data-urlencode "username=$userh&password=$passwordh"
Response:
POST / HTTP/1.1
Host: 127.0.0.1:8080
User-Agent: curl/7.60.0
Accept: */*
Content-Length: 38
Content-Type: application/x-www-form-urlencoded
username=vf%26re%26password%3Dasdaf%25
First command from OP with --data will not work either as it sends the % sign as is triggering a server error that "sees" and incomplete percent encoded request.
POST / HTTP/1.1
Host: 127.0.0.1:8080
User-Agent: curl/7.60.0
Accept: */*
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Content-Length: 33
username=asdasdas&password=asdaf%
Based on this answer.

Microsoft Bot Framework (WebChat) - 500 Internal Server Error

From "Test in Web Chat" window in Azure Dashboard, I entered "hello" which is correctly received in my backend, but I'm still figuring out how to reply.
Note: WORDS IN CAPS means variable.
curl -d "grant_type=client_credentials&client_id=CLI-ENT-ID&client_secret=SECRET&scope=https%3A%2F%2Fapi.botframework.com%2F.default" -X POST https://login.microsoftonline.com/botframework.com/oauth2/v2.0/token
Response: {"token_type":"Bearer","expires_in":3600,"ext_expires_in":0,"access_token":"VERY_LONG_STRING"}
curl -d '{"type":"message","from":{"id":"mybot#SOME_ID","name":"mybot"},"conversation":{"id":"CONVERSATION_ID"},"recipient":{"id":"RECIPIENT_ID","name":"You"},"locale":"en","text":"hello too","replyToId":"CONVERSATION_ID|0000008"}' -H "Content-Type: application/json" -H "Authorization: Bearer VERY_LONG_STRING" -X POST https://api.botframework.com/v3/conversations/CONVERSATION_ID/activities/CONVERSATION_ID%7C0000008
Response: The page cannot be displayed because an internal server error has occurred.
Request from my app (also same error):
POST /v3/conversations/CONVERSATION_ID/activities/CONVERSATION_ID%7C0000008 HTTP/1.1
Accept: application/json
Content-Type: application/json
Authorization: Bearer VERY_LONG_STRING
User-Agent: Java-SDK
Content-Length: 273
Host: api.botframework.com
Connection: Keep-Alive
Accept-Encoding: gzip,deflate
{"type":"message","from":{"id":"mybot#SOME_ID","name":"mybot"},"conversation":{"id":"CONVERSATION_ID"},"recipient":{"id":"RECIPIENT_ID","name":"You"},"locale":"en","text":"hello too","replyToId":"CONVERSATION_ID|0000008"}
What do I miss?
Thanks to comment from Eric Dahlvang, I figure it out.
By default, ApiClient generated from swagger will have its basePath hardcoded to https://api.botframework.com.
Before replying, take the serviceUrl from incoming Activity and use it to change the basePath.

Cannot accept post in Sinatra using Curl

Just been tinkering with Sinatra and trying to get a bit of a restful web service going.
The error I'm getting at the moment is very specific though.
Take this example post method
post '/postMan/:someParam' do
#Edited here. This code can be anything. 411 is still the response
puts params[:someParam]
end
Seems simple enough. Take a param, make an object out of it, then go store it in whatever way the objects save method defines.
Heres what I use to post the data using Curl
$curl -I -X POST http://127.0.0.1/postman/123456
The only problem is, I'm getting 411 back and have no idea why.
To the best of my knowledge, 411 is length required. Here is the trace
HTTP/1.1 411 Length Required
Content-Type: text/html; charset=ISO-8859-1
Server: WEBrick/1.3.1 (Ruby/1.9.2/2011-07-09)
Date: Fri, 02 Mar 2012 22:27:09 GMT
Content-Length: 303
Connection: close
I 'Cannot' change the curl message in any way. So might anyone have a way to set the content length to be ignored in sinatra? Or some fix which doesn't involve changing the curl request?
For the record, it doesn't matter whether I use the parameters in the Post method or not. I could have some crazy code inside it, it will still throw the same error
As others said above, WEBrick wrongly requires POST requests to have a Content-Length header. I just pass an empty body, because it's less typing than passing in the header:
curl -X POST -d '' http://webrickwhyyounotakeemptyposts.com/
Are you sure you're on port 80 for your app?
When I run:
ruby -r sinatra -e "post('/postMan/:someParam'){puts params[:someParam]}"
and curl it:
curl -I -X POST http://127.0.0.1:4567/postMan/123456
HTTP/1.1 200 OK
X-Frame-Options: sameorigin
X-XSS-Protection: 1; mode=block
Content-Type: text/html;charset=utf-8
Content-Length: 0
Connection: keep-alive
Server: thin 1.3.1 codename Triple Espresso
it's ok. Had to change the URL to postManthough, your example threw a 404because you had postman.
The output was also as expected:
== Sinatra/1.3.2 has taken the stage on 4567 for development with backup from Thin
>> Thin web server (v1.3.1 codename Triple Espresso)
>> Maximum connections set to 1024
>> Listening on 0.0.0.0:4567, CTRL+C to stop
123456
Ah. Try it without -I. It's probably sending a HEAD request and as such, not sending what you expect. Use -v if you want to show the headers.
curl -v -X POST http://127.0.0.1/postman/123456
curl -v -X POST -d "key=val" http://127.0.0.1/postman/123456
WEBrick erroneously requires POST requests to include the Content-Length header.
curl -H 'Content-Length: 0' -X POST http://example.com
Standardly, however, POST requests don't require a body and therefore don't require the Content-Length header.

support multipart POST for apachebench?

I want to use apachebench (ab) to test file upload performance.
I have read the manual and can't find a way to achieve my goal.
My goal is try to upload a file by a HTTP Request with POST method and multipart/form-data format.
The ab support "-p POST-FILE", but I only can find the format key=value&key2=value2
What the post data I want to send is
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryuUlX4554LPBjInc5
------WebKitFormBoundaryuUlX4554LPBjInc5
Content-Disposition: form-data; name="file"; filename="411c40d9.jpg"
Content-Type: image/jpeg
XXXXXXXXXXXXXXX (IMAGE DATA)
YYYYYYYYYYYYYYYYYYYYYYYYYYYYY
I googled for a while and can't find any related article or way to achieve that.
I use cURL and it works fine, but I want to do stress testing.
So I need use ab to achieve that goal.
Any suggestions?
This may be very late but might help others who were searching for the same thing...
$ ab -c 5 -n 5 -v 4 -p example_post.txt -T "multipart/form-data; boundary=miuxupsktcqtriloonfbdudrgtawascl" http://yourapplication.local/upload/
Credit to this guy.

Resources