In jQuery/AJAX you can make POST requests and get their response with something like
$.post(url, data, function(res){
//do something
});
Where res contains the server's response. I am trying to replicate this with cURL in bash.
curl -d "data=data" --cookie cookies.txt --header "Content-Type:text/html" https://example.com/path > result.html
Returns gibberish (some sort of js object maybe?), but I am expecting html. Is there a way to retrieve the data that would be in res using cURL?
Thanks in advance.
Sometimes servers send back compressed content. It looks like random garbage. Use curl --compressed to get the decompressed result.
It seems that you are trying to get a data from a POST request. Server returns you a javascript object. And you are getting it right way, javascript converts returned data to HTML, not cURL. So basically you can't do that.
Related
I have an API (via hug) that sits between a UI and Elasticsearch.
The API uses elasticsearch-py to run searches, for example:
es = Elasticsearch([URL], http_compress=True)
#hug.post('/search')
def search(body):
return es.search(index='index', body=body)
This works fine; however, I cannot figure out how to obtain a compressed JSON result.
Elasticsearch is capable of this because a curl test checks out — the following returns a mess of characters to the console instead of JSON and this is what I want to emulate:
curl -X GET -H 'Accept-Encoding: gzip' <URL>/<INDEX>/_search
I've tried the approach here to modify HTTP headers, but interestingly enough the "Accept-Encoding": "gzip" header is already there: it just doesn't appear to be passed to Elastic because the result is always uncompressed.
Lastly, I'm passing http_compress=True when creating the Elastic instance; however, this only compresses the payload — not the result.
Has anyone had a similar struggle and figured it out?
I am trying to post multipart/form-data in laravel using ixudra/curl that specify the data is application json. The problem Im facing right know is to assign the type for data and still make the header content type is multipart/form-data.
$contents = storage_path('app/curl.txt');
$dataJson = '{"bId":"79", "docId":"23"}';
$response = Curl::to($url)
->withHeaders( array(
'Authorization: Bearer 123432',
'grant_type: jwt-bearer' ) )
->withData( array ('data' => $dataJson ))
->withFile('file', $contents, 'text/*', 'curl.txt')
->containsFile()
->withResponseHeaders()
->returnResponseObject()
->post();
For the curl, it is like this. Somehow, the error is "Failure to Authenticate OAuth Token" and the header content type is not multipart/form-data
curl -v -H "Authorization:Bearer 123432"
-H "grant_type:jwt-bearer"
-F "file=#\"/jet/app/www/default/test/storage/app/curl.txt\""
-F "data={\"bId\":\"79\", \"docId\":\"23\"};type=application/json"
"https://url/private"
Any idea? thanks for your time.
is this still an issue? Looking at your code, I don't see any errors in there. You don't need to include the containsFile() method in there but that's really a detail that won't effect your result in any way.
Based on your description, I doubt that the file has anything to do with the error. I'd recommend trying to make sure if the authentication is correct by using it on a simple GET request, just to make sure. If that works, you should dig deeper into the form.
I'm trying to make a POST request from the command line to my Flask app, and I want it to include an image. But I don't know how to include it with the command. I've only used strings as data successfully.
So, if my POST request looks like this:
curl -i -H "Content-Type: application/json" -X POST -d '{"username":"user1", "password":"password", "image":##What do I put here?##}' http://localhost:5000/my_app/api/users
I don't know what to put in that image part of the JSON. I'm tagging flask in this question because it might be a specific answer with regards to flask.
I would like to include an actual image here, and then on the Flask side of things, put the image in a folder of the app where all the uploads go, then save the path to the image in the database for later access. But, to do that, I need to know how to send an image in the first place. Any thoughts?
Seems you're mixing things up here.
From your example seems you want to upload an image in a JSON object. This is generally bad for 2 reasons:
Overhead: the image data should be encoded in printable characters, e.g. using base64. This creates a huge overhead on the data itself causing the JSON decoder to slow down.
Testing: You can't try this using curl on the commandline. You should make some command line utility to test the request.
HTTP knows about data uploads. So to mantain the JSON structure without the slowdown of above, you should upload your image as traditional data upload using a field, and another field for the JSON data structure.
Using curl this is achieved with the -F option.
curl -i -Ffiledata=#file.jpg -Fdata='{"username":"user1", "password":"password"}' http://localhost:5000/my_app/api/users
With the above command you are sending an HTTP POST with a file upload named filedata and a 'data` field containing your 'JSON' payload to parse in the receiving view handler.
You must use 2 HTTP fields because in HTTP upload you're using multipart encoding.
Behind the scenes
You're sending the image contents encoded in base64 but since the data is decoded by the application framework knowing exactly what to do (JSON parse must figure it out while parsing) it's a lot faster.
I am playing with the open data project at spogo.co.uk (sport england).
See here for a search example: https://spogo.co.uk/search#all/Football Pitch/near-london/range-5.
I have been using cygwin and curl to POST JSON data to the MVC controller. An example is below:
curl -i -X POST -k -H "Accept: application/json" -H "Content-Type: application/json; charset=utf-8" https://spogo.co.uk/search/all.json --data '{"searchString":"","address": {"latitude":55,"longitude":-3},"page":0}'
Question:
How can I find out what other variables can be included in the post data?
How can I return all results, rather than just 20 at a time? Cycling through page numbers doesn't deliver all at once.
AJAX is simply a technique of posting data over a connection asynchronously, JSON is just a string format that can contain data. Neither of which have built in mechanisms for querying information such as what fields are accepted or the amount of data returned.
You will want to check the web service documentation for on spogo.co.uk for these answers, IF their web service exposes such functionality they will be the final authority on what the commands and formats are.
I have a very simple Sinatra app which only does log out the params in the action, and then I use curl to send post data in xml format, but sinatra didn't get the xml parsed:
echo '<something>tyrael tong</something>' | curl -X POST -H 'Content-Type: text/xml' -d #- http://localhost:9528/status/update
I searched through google with no solution to this. Am I suppose to parse the xml post data by myself?
try this code
require 'plist'
post 'some/route'
content_type :xml
value=Plist::parse_xml(request.body)
end
Find a rack parser: https://github.com/achiu/rack-parser which could do the job I want: parse the post body into parameter.
P.S. And need to set the content type to "application/xml"
Yes, content type is just a hint for a server how to handle it. If your server can receive say XML or JSON, content type can tell you how to parse it.
As the body of your post message is an xml it will not be parsed by either rails/sinatra. If you want parsing to be done you will need to change the format of data that you are sending and set the content type to "application/x-www-form-urlencoded". Then Sinatra will parse that data and put it into the params hash.