I have a standard embedded tomcat REST API application. I would like clients to send zipped JSON bodies using the header Content-Encoding: gzip
Below is an idea of how this might work:
echo '{"field1":"hello","field2":"there"}' > request.json
gzip request.json
curl -u user:password -H "Content-Encoding: gzip" -H 'Content-Type:application/json;charset=utf8' --data-binary #request.json.gz HTTP://localhost:8100/some/path
I noticed (in the docs) support for compressing the response. I enabled this using server.compression.enabled: true.
I get this error (which indicates to me that the binary data is interpreted as a JSON:
JSON parse error: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\\r, \\n, \\t) is allowed between tokens; nested exception is com.fasterxml.jackson.core.JsonParseException: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\\r, \\n, \\t) is allowed between tokens\n at [Source: (org.springframework.util.StreamUtils$NonClosingInputStream); line: 1, column: 2]
How can I tell Spring Boot to uncompress requests with this header?
I guessed server:http2.enabled: true would work, but this does not seem to have an effect. Is http2 somehow tied to this feature?
Related
I'm writing a bash script to interface with my ecobee (query info and change settings). I have the authorization all worked out (access token and refresh token) and am now trying to request info from the ecobee. This json parameter list is dynamically created. None of the curl examples in the Developers API Doc seem to work.
I've tried assigning the json to a variable (?json="$selection") and to a file (?json=#"$location"). My latest attempt (hard coding the json and escaping the braces) of the curl is as follows:
response=$(curl -s "https://api.ecobee.com/1/thermostat?json=\{"selection":\{"selectionType":"registered","selectionMatch":null,"includeRuntime":true,"includeSettings":true/}/}" -H "Content-Type: application/json;charset=UTF-8" -H "Authorization: Bearer $access_token")
and I received a null response declare -- response=""
If I have curl read from a file:
{"selection":{"selectionType":"registered","selectionMatch":null,"includeRuntime":true,"includeSettings":true}}
then I receive:
response='{
"status": {
"code": 4,
"message": "Serialization error. Malformed json. Check your request and parameters are valid."
}
}'
Which I assuming it' an escape issue?
Can anyone offer any insight? Again, I thought this would be the easy part. I'm using jq to build my request json. Other alternatives to curl that can better deal with json? I'm limited to bash (which is what I know)
Thanks
To integrate (arbitrary) data into a URL (or URI in general) you need to prepare it using percent-encoding first.
As you have mentioned to use jq to compose that data, you could add #uri to your jq filter to perform the encoding, and use the --raw-output (or -r) option to then output it properly.
For example, jq -r '#uri' applied to your JSON
{
"selection": {
"selectionType": "registered",
"selectionMatch": null,
"includeRuntime": true,
"includeSettings": true
}
}
would output
%7B%22selection%22%3A%7B%22selectionType%22%3A%22registered%22%2C%22selectionMatch%22%3Anull%2C%22includeRuntime%22%3Atrue%2C%22includeSettings%22%3Atrue%7D%7D
which you can use within the query string ?json=….
It is unclear to me how Tornado expects to parse body arguments for all content types. It makes sense for JSON, and maybe form data, but for other body content types it is unclear to me how to format the request with CURL to get it to work.
Question: How can one use CURL to send a shell script as the POST body to Tornado? What is the formatting for the body arguments?
What I've tried:
curl -X POST -H "Content-Type: application/x-sh" http://localhost:8888/ -d script='#!/bin/bash \necho "scale=500\; 4*a(1)" | bc -l'
What I would hope or expect this to do is for Tornado to process the HTTP body so that (1) the body argument is script, and (2) the value corresponding to it is the shell script to compute pi:
#!/bin/bash
echo "scale=500\; 4*a(1)" | bc -l
Instead I just get a 400: Bad Request error, and my debugging logs show that the value of self.request.body_arguments is an empty dictionary {}.
You're sending POST data with this header - Content-Type: application/x-sh. But Tornado only supports application/x-www-form-urlencoded to parse submitted form/POST data.
Just submit the data with Content-Type: application/x-www-form-urlencoded and Tornado will make the submitted data available in body_arguments.
I'm writing a bash script using cURL to call an API. I'm generating some part of the contents transforming strings coming out from other sources.
From the db the spaces are replaced with "_SPACE_" string for application reasons.
I'm replacing it using
ITEM=${M//_SPACE_/ }
and then I call cURL as follow:
curl ... --data '{"field": "'$ITEM'"}' ...
Running the script it returns an exception on curl execution
curl: (3) [globbing] unmatched close brace/bracket in column 21
org.codehaus.jackson.JsonParseException: Unexpected end-of-input in VALUE_STRING
at [Source: org.apache.catalina.connector.CoyoteInputStream#185ff06c; line: 4, column: 139]
The exception
org.codehaus.jackson.JsonParseException: Unexpected end-of-input in VALUE_STRING ...
says that your lib from codehouse is not able to parse the data object which is send via curl. i could imagin that your --data '{"field": "'$ITEM'"}' object fails because of the single quotes '$ITEM'. could u send the complete data object or script ?
I'm following Spotify's client credentials authorization flow, but all of my curl requests are returning {"error":"invalid_client"} each time. Here are the instructions from Spotify:
The request will include parameters in the request body:
grant_type - Set to “client_credentials”.
The header of this POST request must contain the following parameter:
Authorization - A Base 64 encoded string that contains the client ID and client secret key. The field must have the format: Authorization: Basic <base64 encoded client_id:client_secret>
They also include an example of a curl request:
$ curl -H "Authorization: Basic ZjM4ZjAw...WY0MzE=" -d grant_type=client_credentials https://accounts.spotify.com/api/token
Following their example, so far I've tried curl requests with:
client_id and client_secret plain
both base64 encoded seperately
only one or the other encoded
both encoded as one string with the colon
both encoded as one string without the colon
each of the above with a regenerated client secret
I'm using Ruby's Base64#encode64 method to encode. Still no luck. Any helpful hints?
Okay, I got it working - passing my client_id and client_secret, separated by a colon, to Base64.strict_encode64 (NOT Base64.encode64) and then passing that to the above curl request gets a 200 response with an access token. Apparently encode64 was not enough.
I ran into this error when I ran the curl command in my terminal.
-bash: unexpected EOF while looking for matching `"'
Solved by using single quotes instead of double quotes.
I am building a script that uses a cURL command against an API. I send the cURL command formatted as an application/json request and get the result, which I parse into a Ruby hash.
This works great when I use cURL POST commands, getting the correctly formatted JSON responses. However, when using cURL GET commands I am returned a JSON document that has headers:
puts r:
HTTP/1.1 200 OK
X-Compute-Request-Id: req-7e625990-068b-47d1-8c42-9d3dd3b27050
Content-Type: application/json
Content-Length: 1209
Date: Wed, 16 Jan 2013 20:47:41 GMT
{ <JSON DATA> }
When I try and do a JSON.parse(r) I get an unexpected token error at 'HTTP/1.1.'.
My method for this:
def list_flavors
r = %x(curl -s -k -D - -H \"X-Auth-Token: #{$token}\" -X 'GET' http://10.30.1.49:8774/v2/27e60c130c7748f48b0e3e9175702c30/flavors -H 'Content-type: application/json')
response = JSON.parse(r)
response
end
Is there a way to use regular expressions to pull the body out of the JSON doc and then parse?
Or am I going about this the wrong way when getting the response from cURL?
You'll need to find a way to cut out that header before passing the string into JSON.parse. JSON.parse expects valid json only.
Rather than curling and using the thing wholesale as a string, I'd suggest you use the very versatile ruby Net::HTTP and/or OpenURI libraries, which will allow you to easily access just your response's body without the header.