I am trying to send data within the body of my http post. The following code makes the POST but I can't figure out how to get data into the body or any other field besides the http version and URL fields. I've looked and tried different things from other sources on the net to no avail. Ultimately this will be within a bash script - I can\t use CURL or WGET.
printf "POST /postresearch HTTP/5.0\r\n\r\nTHIS IS THE BODY" |nc -n -i 1 10.0.1.11 3000
You need to make a valid HTTP request, which includes using the correct protocol version number and for POST requests, providing an entity length, and possibly a content type.
POST /postresearch HTTP/1.0
Content-Length: 25
Content-Type: application/x-www-form-urlencoded
field=value&field2=value2
I issue a HTTP 1.0 request, specify that the body will be 25 bytes, and specify that I'm sending URL-encoded form parameters (this is how browsers send forms when you submit them).
BTW, don't do this. There is absolutely no situation where you should be implementing your own HTTP client. Use a proper HTTP tool or library.
Related
I am trying to write a simple function which would easily extract the contact information from a classified listing.
Background
The URL I'm looking at is
https://www.idealista.pt/imovel/27542922/
Looking through the developer tools in Chrome, I see that it makes a GET request to this URL. https://www.idealista.pt/pt/ajax/listingController/adContactInfoForListing.ajax?adId=27542922
If I make a GET request in Postman or just copy the second URL into Chrome I get a JSON containing various details.
My code
(Ruby)
uri = URI('https://www.idealista.pt/pt/ajax/listingController/adContactInfoForListing.ajax?adId=27542922')
foo = Net::HTTP.get(uri)
JSON.parse(foo)
The problem
The response is a 403 with a body saying that the system has detected that many requests have been made in a short period of time.
I can replicate this in Postman by doing seven or eight consecutive requests, but then if I wait a minute or two before trying again I get back to seeing the JSON.
Through Ruby it happens straight away.
What I've tried
I've tried copying some or all of the temporary headers created by Postman into my request in Ruby but I still get the same error or 404
User-Agent - PostmanRuntime/7.22.0
Accept - */*
Cache-Control - no-cache
Postman-Token - 6c68a9eb-83d5-4724-9f41-3fc51971db9f
Host - www.idealista.pt
Accept-Encoding - gzip, deflate, br
Cookie - userUUID=c017919a-6115-4905-95b3-5d949c6fb447; _pxhd=34ed938caca242bf6050147e1514cda07b704cc7681245a4beec5a64e0a5cf66:d4f21381-522a-11ea-a954-6f59910ff05b; SESSION=887b6dbc-78a4-4abd-9600-7ce401507331; WID=15a353ca7aab3446|XlEN6|XlEN4
Connection - keep-alive
you have to use a proxy, and chanfe the ip
I'm trying to pull data from a post API that expects authentication and request parameters as part of the request body. I guess, what they do is access the data from the post variables.
What I have seen so far in documentation is how to send POST via JSON or headers. In my case, no headers just the post body parameters.
call to this api via curl --data option works just fine.
curl --data "username=xyz&password=xyz&function=xyz" http://example.com/api.php
How can I replicate above call in nifi?
I have tried multiple methods without success. Latest has been Generate flow file, update attributes (where i fill in the parameters), invoke http then putfile.
But I'm getting errors - the api is not abe to authenticate my request.
Thanks
If you need to send the following data in body, then put it into content of your flowfile.
username=xyz&password=xyz&function=xyz
The easiest way to put it into the Custom Text property of the GenerateFlowFile processor.
Usually for this kind of body you have to provide content type header:
content-type: application/x-www-form-urlencoded
If you don't need any additional headers then you don't have to define any additional attributes of the flow file.
This is my code.
<iron-ajax auto
id="requestRepos"
url="myurl"
params='{"mycommaseperatedparams"}'
handle-as="json"
on-response="handleResponse"></iron-ajax>
If i manually hit the url in the broswer, it works. But this one does't.
It's a GET request.
HTTP status code 406 means that the server cannot return a representation which conforms to the Accept- headers. From the specs:
The resource identified by the request is only capable of generating response entities which have content characteristics not acceptable according to the accept headers sent in the request.
For more answers see here: What is "406-Not Acceptable Response" in HTTP?
This is most likely the the Accept header set to application/json by the iron-ajax element. The browser (Chrome) on the other hand by default sends requests with
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Where the */* bit matches any content type.
To fix you would have to work on the server side to allow JSON responses. You could also try setting the header explicitly although I expect iron-ajax to override Accept header anyway
<iron-ajax headers='{"Accept": "*/*"}' handle-as="json"></iron-ajax>
Again, the */* is just an example. You probably need a more specific media type.
As I am not able to find a way to set the content-type to JSON for the data I sent via XDomainRequest,
I naturally ask:
Is it IMPOSSIBLE to POST JSON with XDomainRequest?
"Only text/plain is supported for the request's Content-Type header"
Looking at some other questions, I'd say this is possible.
For example: How to send JSON data in XDR using the POST method
You can still POST the JSON as plain text and have the server side read the POST body directly and handle it, even Content-Type is missing for the sick of IE.
According to http://blogs.msdn.com/b/ieinternals/archive/2010/05/13/xdomainrequest-restrictions-limitations-and-workarounds.aspx
You cannot send content-type : application/json in xdr. You can not even send any other custom header.
xdr is only limited to text/plain
The best you can do is use JSON.stringify() and parse it on server side.
xhr.send(JSON.stringify(jsonObject))
I am developing an API using CodeIgniter and the RestServer for CI (see below). I am also using the Firefox RestClient plugin to test the API.
What I am wondering is how to do the test post (what format).
Tried {"desc":"value"} but it did not work. The API is not "seeing" the incoming post fields.
http://net.tutsplus.com/tutorials/php/working-with-restful-services-in-codeigniter-2/
the post body doesn't need to have a specific format, but the most convenient is to encode the body in the same way web browsers encode form data, specifically Content-Type: application/x-www-form-urlencoded. In particular, the Host and Content-Length headers are not optional, and the Content-Type header is usually needed to tell the server how to interpret the body. A well formed POST request will look like:
POST /path/to/resource HTTP/1.0
Host: example.com:80
Content-Length: 21
Content-Type: application/x-www-form-urlencoded
key=value&key2=value2
It's still up to the server to recognize the content-type header and parse the body that way.
Note that the data is after all the headers, not as part of the request path (in the first line).
Optionally, you can use Proxy Library which i wrote for CI. With that, you can simulate any of possible call to your API(its works for popular REST API too), with more simple syntax instead using cURL...
// An example call to your API end point using POST, will be simply
$this->load->library('proxy');
$this->proxy->http('POST', 'http://somesite.com/api/users', array('username' => 'foo', 'password' => 'bar'));
You can define whatsoever HTTP header too (like API Key or whatever else).