I am trying to call an API using Curl. The request was timing out after 60 secs due to TCP Timeout. I added "-H "Connection: keep-alive" -H "Keep-Alive: timeout=90, max=100" to the existing code.
curl -s -X POST -u $regressionTestAuthToken -H Content-Type: application/json -H Accept: application/json -H "Connection: keep-alive" -H "Keep-Alive: timeout=90, max=100" $URL -d#"${WORKSPACE}"/tmp.json > "${WORKSPACE}"/out.json
After adding the Connection Keep-Alive, I am getting a parse error
parse error: Invalid numeric literal at line 1, column 7
Can someone help me to triage this issue?
Related
I have tried to create a method in a bash script which should be able to perform a curl operation with a variable number of headers, however I seem to get stuck in that the curl command consider the header arguments to be multiple arguments as they contain spaces.
When I run the following line in bash I get a 201 response:
response=$($executable -X POST localhost:9200/index-template/globalmetadata --write-out '%{http_code}' --silent --output /dev/null --verbose --data "#${full_path}" -H "Content-Type: application/json" )
If I run the following:
#!/bin/bash
submit_request () {
full_path=/home/mat/globalmetadata.json
header_option=""
header_options=""
for header in "${#:1}"; do # looping over the elements of the $# array ($1, $2...)
header_option=$(printf " -H %s" "$header")
header_options=$(printf '%s%s' "$header_options" "$header_option")
done
echo Headers: $header_options
executable=curl
#response=$($executable -X POST localhost:9200/index-template/globalmetadata --write-out '%{http_code}' --silent --output /dev/null --verbose --data "#${full_path}" -H "Content-Type: application/json" )
response=$($executable -X POST localhost:9200/index-template/globalmetadata --write-out '%{http_code}' --silent --output /dev/null --verbose --data "#${full_path}" $header_option )
echo $response
}
submit_request "\"Content-Type: application/json\""
I get this output:
Headers: -H "Content-Type: application/json"
======= 3
* Trying 127.0.0.1:9200...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 9200 (#0)
> POST /index-template/globalmetadata HTTP/1.1
> Host: localhost:9200
> User-Agent: curl/7.68.0
> Accept: */*
> Content-Length: 3232
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 100 Continue
} [3232 bytes data]
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 406 Not Acceptable
< X-elastic-product: Elasticsearch
< content-type: application/json; charset=UTF-8
< content-length: 97
<
{ [97 bytes data]
* Connection #0 to host localhost left intact
* Could not resolve host: application
* Closing connection 1
406000
What I have noticed is that even though the headers are -H "Content-Type: application/json, curl says Could not resolve host: application. I suspect that it splits the arguments into two due to space between Content-Type: and application/json.
I tried to mix and match quotes and double quotes in all kinds of combination but nothing really worked.
#GordonDavisson is right, you'll have to put your header_options in an array. Just be aware that using the array "${header_options[#]}" when it is empty will result in an empty argument in your curl command, but You can get rid of this problem by storing the whole command in an other array.
submit_request () {
local -a header_options
for arg; do header_options+=(-H "$arg"); done
local -a curl_command=( \
curl \
-X POST \
localhost:9200/index-template/globalmetadata \
--write-out '%{http_code}' \
--silent \
--output /dev/null \
--verbose \
--data '#/home/mat/globalmetadata.json' \
"${header_options[#]}" \
)
local response=$( "${curl_command[#]}" )
printf '%s\n' "$response"
}
I'm trying to pass Headers and x-www-form-urlencoded body in Autocannon POST request, but my response is always:
0 2xx responses, 5 non 2xx responses
Example from command line:
autocannon -c 1 -a 5 -H Authorization="Bearer xxxx",Content-Type="application/x-www-form-urlencoded" -b "Key"="Value" http://host.com:8080/path
Use -H flag for each header like this :
autocannon -c 1 -a 5 -H "Authorization":"Bearer xxxx" -H "Content-Type":"application/x-www-form-urlencoded" -b '{"Key"="Value"}'
http://host.com:8080/path
Let's say you have the following data and you want to make a POST request using x-www-form-urlencoded:
URL
https://api.something.com
Headers
Authorization: Bearer some-extra-long-uuid=
Content-Type: application/x-www-form-urlencoded
Body
field_1: value_1
field_2: value_2
For the above data, the autocannon CLI command would look something like below:
autocannon -m POST -H "Authorization":"Bearer some-extra-long-uuid=" -H "Content-Type":"application/x-www-form-urlencoded" -b "field_1=value_1&field_2=value_2" https://api.something.com
Using keycloak 6.0.1
I have created the realm role and the user. I have the user uuid and the role uuid.
If I run
curl -v -X POST -w "\n" http://localhost:8080/auth/admin/realms/SpringBootKeycloak/users/$USER_ID/role-mappings/realm -H "Content-Type: application/json" -H "Authorization: bearer $TOKEN" --data '[{"id":$ROLE_ID,"name":"user"}]'
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /auth/admin/realms/SpringBootKeycloak/users/fe38bcb5-258b-44e4-a056-cf8c1a29b99f/role-mappings/realm HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Type: application/json
> Authorization: bearer very long string
> Content-Length: 31
>
* upload completely sent off: 31 out of 31 bytes
< HTTP/1.1 500 Internal Server Error
< Connection: keep-alive
< Content-Length: 0
< Date: Fri, 13 Sep 2019 08:53:14 GMT
<
* Connection #0 to host localhost left intact
I get a server error and a log entry of
10:00:55,903 ERROR [org.keycloak.services.error.KeycloakErrorHandler] (default task-10) Uncaught server error: com.fasterxml.jackson.databind.JsonMappingException: Unrecognized token '$ROLE_ID': was expecting ('true', 'false' or 'null')
at [Source: (io.undertow.servlet.spec.ServletInputStreamImpl); line: 1, column: 3] (through reference chain: java.util.ArrayList[0])
Alternatively if I run curl with data of
--data '[{"id":"$ROLE_ID","name":"user"}]'
then get return of 404 not found.
If I set data to be hardcoded then it works
--data '[{"id":"9b6371f2-646d-4927-b3b6-6e208935517e","name":"user"}]'
but get return of 204 No Content which is success. And the role is added.
Further investigation -
--data '[{"id":"'$ROLE_ID'","name":"user"}]'
gives a 204 status and the role is added to the user.
So get there eventually.
That is a problem with variable substitution:
$ export ROLE_ID=9b6371f2-646d-4927-b3b6-6e208935517e
$ echo '[{"id":"'$ROLE_ID'","name":"user"}]'
[{"id":"9b6371f2-646d-4927-b3b6-6e208935517e","name":"user"}]
$ echo '[{"id":$ROLE_ID,"name":"user"}]'
[{"id":$ROLE_ID,"name":"user"}]
$ echo '[{"id":"$ROLE_ID","name":"user"}]'
[{"id":"$ROLE_ID","name":"user"}]
I am calling the same curl command once from terminal and once from Shellscript - but the outcome is different:
#!/usr/bin/env bash
url="dev.test.ch"
http="http://"
test="192.168.178.107"
curl -H "'Host: $url'" "$http$test"
echo "'Host: $url'" "$http$test"
and
curl -H 'Host: dev.test.ch' http://192.168.178.107
Once I get the NGINX Startpage (from Shellscript) and once the right response (HTML Page from my App)
Any ideas?
Stupid me -
curl -H 'Host: '$newline "$http$test"
the braces were the problem.
You can use curl with -v option to see what exactly HTTP request was sent to server. Problem that in your script you curl adds 'Host header.
With this line curl -H "'Host: $url'" "$http$test" it sends
> GET http://192.168.178.107/ HTTP/1.1
> Host: 192.168.178.107
> User-Agent: curl/7.49.1
> Accept: */*
> 'Host: dev.test.ch'
If you remove single qoutes curl -v -H "Host: $url" "$http$test" it will send proper request:
> GET http://192.168.178.107/ HTTP/1.1
> Host: dev.test.ch
> User-Agent: curl/7.49.1
> Accept: */*
I have following sh script
#!/usr/bin/env bash
headers='-H "custom1: ololo1" -H "custom2: ololo2"'
value_for_header="value"
curl -X "PUT" -H "Content-Type: application/json" -H "custom_ololo: $value_for_header" $headers http://localhost:8000/ -d '{"a": true}' -vv
Log when execute it:
* Rebuilt URL to: ololo1"/
* Hostname was NOT found in DNS cache
* Could not resolve host: ololo1"
* Closing connection 0
curl: (6) Could not resolve host: ololo1"
* Rebuilt URL to: ololo2"/
* Hostname was NOT found in DNS cache
* Could not resolve host: ololo2"
* Closing connection 1
curl: (6) Could not resolve host: ololo2"
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8000 (#2)
> PUT / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:8000
> Accept: */*
> Content-Type: application/json
> custom_ololo: value
> Content-Length: 40
>
* upload completely sent off: 40 out of 40 bytes
* HTTP 1.0, assume close after body
< HTTP/1.0 400
< Date: Thu, 21 Jul 2016 12:32:13 GMT
< Server: WSGIServer/0.1 Python/2.7.6
< X-Frame-Options: SAMEORIGIN
< Content-Type: application/json
<
* Closing connection 2
As we can see -H "custom_ololo: $value_for_header" works well > custom_ololo: value
But string $headers is not inserted correctly. I've tried put "$headers" and ${headers} but no result
So, my question is: How is correctly insert strings with several headers into sh script with curl.
You need to use an array, at which point you can put all the headers in the array and simplify your call.
#!/usr/bin/env bash
value_for_header="value"
headers=(
-H "custom1: ololo1"
-H "custom2: ololo2"
-H "Content-Type: application/json"
-H "custom_ololo: $value_for_header"
)
curl -X "PUT" "${headers[#]}" http://localhost:8000/ -d '{"a": true}' -vv
You need to put $headers into ""
curl -X "PUT" -H "Content-Type: application/json" -H "custom_ololo: $value_for_header" "$headers" http://localhost:8000/ -d '{"a": true}' -vv