How to set the body of a POST request using Ruby Mechanize? - ruby

How can you set the body of a POST request using the Ruby Mechanize gem. I know you can do
mechanize.post(url, query, headers)
but I want to set the body of the POST request with a JSON string. Is that possible? So, similar to something like this with jQuery:
$.ajax({
type: 'POST',
url: 'myurl',
data: "{'key1':'value1','key2':'value2'}",
...
});

I don't really like the answer you linked to in your comment because it employs to_json() which is a rails method, and the tags for your question do not indicate that your question pertains to rails. In any case, I think the answer needs some discussion.
Here is the mechanize method:
Mechanize#post(url, query, headers)
...and your stated goal is:
I want to set the body of the POST request
Mechanize#post() allows you to set the body of the request to anything you want, but you also have to consider the question:
What is the server side expecting?
You gave an example of a jquery ajax() request for what you want to do. jquery uses the following default Content-Type header when sending an ajax() request:
application/x-www-form-urlencoded; charset=UTF-8
That tells the server that the body of the post request is going to be written in a specific secret code. Well, it's not much of a secret; it looks like this:
name1=val1&name2=val2
That secret code's name is x-www-form-urlencoded. Because the server is given the name of the secret code in the Content-Type header, the server knows how to read the body of the post request.
In the Mechanize#post() method, the second parameter is 'query', and the mechanize docs say this about the query argument:
The query is specified by either a string, or
a list of key-value pairs represented by a hash, or
an array of arrays.
http://rubydoc.info/gems/mechanize/Mechanize#post-instance_method
If you want to use the secret code named x-www-form-urlencoded in the body of your Mechanize#post() request, then you can provide a Hash with name/value pairs, e.g.
my_hash = {
'data' => '{"key1":"value1","key2":"value2"}'
}
Then you call Mechanize#post() like this:
my_agent.post(
'http://target_site.com',
my_hash,
{'Content-Type' => 'application/x-www-form-urlencoded; charset=UTF-8'},
)
Then Mechanize will convert the 'query' Hash into a String using the secret code named x-www-form-urlencoded and insert the string into the body of the post request. On the server side, the application that receives the post request can retrieve the json string doing something like this:
json_str = post_variables['data']
You should be aware that there are other secret codes that can be used for the body of a post request. One of them is called json, which is a string formatted using javascript syntax, for example:
'{
"id": 1,
"name": "A green door",
"price": 12.50,
"tags": ["home", "green"]
}'
Note how there are no '=' signs or '&' symbols in the json format--as there are with the x-www-form-urlencoded format, so the json secret code is much different from the x-www-form-urlencoded secret code.
If you want to use the json secret code in the body of your post request, you need to change two things when you call Mechanize#post(url, query, headers):
Provide a String for the 'query' argument.
Tell the server that the body of the post request uses the json secret code.
Like this:
json_str = '{"key1":"value1","key2":"value2"}'
my_agent.post(
'http://target_site.com',
json_str,
{'Content-Type' => 'application/json'},
)
When you pass a String argument for the query parameter, Mechanize doesn't do any processing of the String before inserting the String into the body of the post request. On the server side, the application that receives the post request can retrieve the json string by doing something like this:
json_str = request.body.read
#Then probably:
hash = JSON.parse(json_str)
The one hitch is that the server can ignore the Content-Type header and try to read the body of the post request using a secret code that it has already decided upon. If the body of your post request is not written in the secret code that the server expects, then you will get an error.
Note that the 'data' string you posted isn't valid json because it uses single quotes around the properties and values.

Related

Is it possible to make a x-www-form-urlencoded POST request through InvokeHttp

I just started using Nifi and i would like to know if it's possible to perform a POST with a content-type set to x-www-form-urlencoded. I need to pass these key/values to my request :
grant_type: refresh_token
client_id: myClientId
client_secret: myClientSecret
refresh_token: myRefreshToken
I try to made somethink with the FlowFile Form Data Name property
But the request sended looks like this :
I would like the key/value like this (sended as Content-type : application/x-www-urlencoded) :
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.15.3/org.apache.nifi.processors.standard.InvokeHTTP/index.html
When the HTTP Method is POST, dynamic properties with the property name in the form of post:form:<NAME>, where the will be the form data name, will be used to fill out the multipart form parts.
so, post:form:<NAME> could be used for multipart/form-data
and you want application/x-www-form-urlencoded
format of http body should be parm1=Value1&parm2=Value2
before calling invokeHttp use replaceText with approximately this expression in replacement value:
client_secret=${client_secret:urlEncode()}&client_secret=${client_secret:urlEncode()}&...
then use invokehttp with
Send Message Body = true
Content-Type = application/x-www-form-urlencoded

Getting Myob AccountRight API Access token error

I am trying to get access token from MYOB. The POST call i make returns a "400 Bad Request error"
i'm using "axios" to make the POST call
i already got the Access code which i use in the data i'm sending in the POST call
here is my code
const config= { headers:{'Content-Type':"application/x-www-form-urlencoded"}}
const data={
client_id:"xxxxxxxxxxxxxxxxxxxxxxx",
client_secret:"xxxxxxxxxxxxxxxxxxxxx",
scope:"CompanyFile",
code: code,
redirect_uri:"http%3A%2F%2Flocalhost%3A30002Fcallback",
grant_type : "authorization_code"
}
axios.post("https://secure.myob.com/oauth2/v1/authorize", data, config)
.then((res) =>{
console.log ("response ...............", res
}
)
.catch((error) => {
console.error("Error here is ........",error)
}
)
Axios will, by default, attempt to POST your data fields as JSON which is not correct.
Instead, you want to url encode them and post the url-encoded string in the HTTP body. See the 'example call' in the docs.
There's a good example of how to url encode w/ axios here.
I also note that your redirect_uri field is already url encoded, so attempting to simply encode it a second time means you'll end up with something like http%253A%252F%252Flocalhost which is not correct. Double check your URL encoding against the example call to make sure you're not accidentally encoding certain fields twice. From memory the access code is already encoded appropriately so you might need to fiddle with decoding it before re-encoding it to get it working.

Power Query call to google.webmaster.api , Post, request problem

I call the google.webmasters.api via Power-Query(M) and managed to configure the oath2 and made my first successfull call to get & list.
Now i try to call the /searchAnalytics/query? which is working only with Post.
This always responds in a 400 error. Formating of the Query or the Url is not working correctly.
Here some additional Infomations:
Power Query - Reference
Google Webmaster Api - Reference
PowerBi Community
format Date different:
body = "{ ""startDate"": ""2019-01-01"", ""endDate"": ""2019-02-02"" }",
to
body = "{ ""startDate"": ""2019/01/01"", ""endDate"": ""2019/02/02"" }",
let
body = "{ ""startDate"": ""2019-01-01"", ""endDate"": ""2019-02-02"" }",
AccessTokenList = List.Buffer(api_token),
access_token = AccessTokenList{0},
AuthKey = "Bearer " & access_token,
url = "https://www.googleapis.com/webmasters/v3/sites/https%3A%2F%2Fxxxxxxxxx.xxx/searchAnalytics/query?",
Response = Web.Contents(url, [Headers=[Authorization=AuthKey, ContentType="application/json", Accept="application/json"], Content=Text.ToBinary(body) ]),
JsonResponse = Json.Document(Response)
in
Response
getting a 400 and is shows as 400 call in Gooogle-Api Overview
Any Ideas whats wrong?
Thx
Ensure request headers are valid. Server expects Content-Type header, not ContentType.
The documentation (https://developers.google.com/webmaster-tools/search-console-api-original/v3/searchanalytics/query#try-it) suggest requests should be something like:
POST https://www.googleapis.com/webmasters/v3/sites/[SITEURL]/searchAnalytics/query HTTP/1.1
Authorization: Bearer [YOUR_ACCESS_TOKEN]
Accept: application/json
Content-Type: application/json
{}
So seems like main takeaways are:
HTTP POST method must be used
Web.Contents documentation (https://learn.microsoft.com/en-us/powerquery-m/web-contents) suggests including the Content field in the options record to change request from GET to POST.
URL must be valid
You haven't provided your actual URL, so you'll have to validate it for yourself. I would get rid of the trailing ? in your url (as you aren't including a query string -- and even if you were, you should pass them to the Query field of the options record instead of building the query string yourself).
Headers (Authorization, Accept, Content-Type) should be valid/present.
Build your headers in a separation expression. Then pass that expression to the Headers field of the options record. This gives you the chance to review/inspect your headers (to ensure they are as intended).
Body should contain valid JSON to pass to the API method.
Creating valid JSON via manual string concatenation is liable to error. Using Json.FromValue (https://learn.microsoft.com/en-us/powerquery-m/json-fromvalue) seems a better approach.
All in all, your M code might look something like:
let
// Some other code is needed here, in which you define the expression api_token
AccessTokenList = List.Buffer(api_token),
access_token = AccessTokenList{0},
AuthKey = "Bearer " & access_token,
requestHeaders = [Authorization = AuthKey, #"Content-Type" = "application/json", Accept = "application/json"],
parametersToPost = [startDate = "2019-01-01", endDate = "2019-02-02"], // Can include other parameters here e.g. dimensions, as mentioned in Search Console API documentaton.
jsonToPost = Json.FromValue(parametersToPost, TextEncoding.Utf8), // Second argument not required (as is default), but just be explicit until you've got everything working.
url = "https://www.googleapis.com/webmasters/v3/sites/https%3A%2F%2Fxxxxxxxxx.xxx/searchAnalytics/query", // Uri.EscapeDataString function can be use for URL encoding
response = Web.Contents(url, [Headers=requestHeaders, Content=jsonToPost])
in
response
Untested (as I don't have an account or API credentials).

MultiJson::LoadError: 795: unexpected token when trying to parse incoming body request

I'm losing my sanity trying to parse an incoming request on a Sinatra app.
This is my spec
payload = File.read("./spec/support/fixtures/payload.json")
post "/api/v1/verify_payload", { :payload => payload }
last_response.body.must_equal payload
where is simply spec/support/fixtures/payload.json
{"ref":"refs/heads/master"}
My route looks like
post '/verify_payload' do
params = MultiJson.load(request.body.read, symbolize_keys: true)
params[:payload]
end
And running the spec I get the following error:
MultiJson::LoadError: 795: unexpected token at 'payload=%7B%22ref%22%3A%22refs%2Fheads%2Fmaster%22%7D'
I have tried to parse the body request in different ways without luck.
How can I make the request valid JSON?
THANKS
If you want to send a JSON-encoded POST body, you have to set the Content-Type header to application/json. With Rack::Test, you should be able to do this:
post "/api/v1/verify_payload", payload, 'CONTENT_TYPE' => 'application/json'
Alternatively:
header 'Content-Type' => 'application/json'
post '/api/v1/verify_payload'
More info here: http://www.sinatrarb.com/testing.html
The problem it is that you are passing a ruby hash, that is not well formated, you should pass a json object.
Something like this, should work:
post "/api/v1/verify_payload", { :payload => payload.to_json }

Mechanize send a POST to populate data on a page

I have a site that sends a POST to populate some data on a page. I usually look at the POST in Charles Proxy and pass the parameters like so:
bot.post('https://www.google.com?', {
"parameter" => "value",
"SESSION_parameter_ID" => "value2})
However, when I look at the request in Charles it is just sending text like this:
callCount=1
page=/eplus/mao.portal?_nfpb=true&_pageLabel=pBillPayHistory&_nfls=false
httpSessionId=2GQQQj3McPh2vQzvxnFb5KM9qgfn80Sqv2L8sC16p66nvxc5yJv5!1006025334
scriptSessionId=22A83635CAD97A33C8255AC8D559FD27672
c0-scriptName=BillingService
How do I send a POST to a URL and send the request parameters as text?
That Content-Type header should be: application/x-www-form-urlencoded
So try:
bot.post url, vars, ({'Content-Type' => 'application/x-www-form-urlencoded'})

Resources