Shell script call API via curl and process response - shell

I need to create a shell script that calls my login API via curl.
The script should be able to store and process the response from curl api call.
myscript.sh
#!/bin/bash
echo "Extract bearer token from curl calling login api"
echo
# Check cURL command if available (required), abort if does not exists
type curl >/dev/null 2>&1 || { echo >&2 "Required curl but it's not installed. Aborting."; exit 1; }
echo
PAYLOAD='{"email": "dummy-user#acme.com", "password": "secret"}'
curl -s --request POST -H "Content-Type:application/json" http://acme.com/api/authentications/login --data "${PAYLOAD}"
My problem in the given script is:
it does not get the response of curl calling the API.
From the response json, get only the token value.
Sample Login API response:
{
"user": {
"id": 123,
"token": "<GENERATED-TOKEN-HERE>",
"email": "dummy-user#acme.com",
"refreshToken": "<GENERATED-REFRESH-TOKEN>",
"uuid": "1239c226-8dd7-4edf-b948-df2f75508888"
},
"clientId": "abc12345",
"clientSecretKey": "thisisasecret"
}
I only need to get the value of token and store it in a variable... I will use token value in other curl api call as bearer token.
What do I need to change in my script to extract the token value from the response of curl api call?
Thanks!

Your curl statement has an error in it. You are executing it with the target URL as an header field:
curl --request POST -H "Content-Type:application/json" -H http://acme.com/api/authentications/login --data "${PAYLOAD}"
^
|
Remove this header flag
Also the silent -s flag helps when curl is executed from scripts:
-s, --silent
Silent or quiet mode. Don't show progress meter or error messages. Makes Curl mute.
Afterwards you could store the data in a variable and execute a regular expression on it to extract the token you need for further processing.
The complete script could look like the following:
#!/bin/bash
echo "Extract bearer token from curl calling login api"
echo
# Check cURL command if available (required), abort if does not exists
type curl >/dev/null 2>&1 || { echo >&2 "Required curl but it's not installed. Aborting."; exit 1; }
echo
PAYLOAD='{"email": "dummy-user#acme.com", "password": "secret"}'
RESPONSE=`curl -s --request POST -H "Content-Type:application/json" http://acme.com/api/authentications/login --data "${PAYLOAD}"`
TOKEN=`echo $RESPONSE | grep -Po '"token":(\W+)?"\K[a-zA-Z0-9._]+(?=")'`
echo "$TOKEN" # Use for further processsing

An alternate solution to parsing JSON with regex is jq :
echo '{ "user": { "id": 123, "token": "<GENERATED-TOKEN-HERE>", "email": "dummy-user#acme.com", "refreshToken": "<GENERATED-REFRESH-TOKEN>", "uuid": "1239c226-8dd7-4edf-b948-df2f75508888" }, "clientId": "abc12345", "clientSecretKey": "thisisasecret" }' | jq -r '.user.token'

Related

How to use sql query with Curl command

I have written a bash script using Curl command. I'm using an SQL query that gives a count of value 5. I want to assign that count value to T_4. Not sure how to do that in bash script using Curl command.
#!/bin/sh
result=$(
curl --netrc-file ~/.netrc -d '[
{
"T_1": "Test1",
"T_2": "Test2",
"T_3": "Test3",
"T_4": "1"
}
]' -X POST -H "Content-Type: application/json" https://www.testurl.com -d "SELECT count(5) FROM DUAL")
echo "Response from server"
echo $result
exit
Also, when I run the above script in putty, I'm getting an error that says -
"errorCode":"SIP-10322","errorMessage":"SIP-10322: rows updated is not 1:0"
Need your input on this.
The output of the SQL query which is a metric value, I have to use it in Rest call(Post API). Can anyone guide me on this?
Use command substitution to execute the query and assign the output to a shell variable. For instance, if the DB is MySQL, you would use:
t_4=$(mysql -e "SELECT 5 FROM DUAL")
Then use the variable inside the JSON parameter.
json='[
{
"T_1": "Test1",
"T_2": "Test2",
"T_3": "Test3",
"T_4": "'$t_4'"
}
]'
result=$(curl --netrc-file ~/.netrc -d "$json" -X POST -H "Content-Type: application/json" https://www.testurl.com)

How to send a post request with the result of a command or a script using curl?

I want to send a post request using the curl command with the data resulting from an execution of a script or command, in this case, the command is ifconfig. I am looking for a oneliner that can be executed in a Linux terminal or Windows CMD.
In simple words, I want to send the result of the command to the server.
Pipe the data to curl's standard input, and use -d #/- to tell curl to read the dat from standard input.
It's common for command line utilities to use - to represent standard input. Curl is one such utility.
In curl, -d #something will expect to get its data from path something.
So -d #- tells curl to get its POST data from standard input.
You can then pipe the data you want to upload straight to curl:
% echo "I am command output" | curl https://httpbin.org/anything -X POST -d #-
{
"args": {},
"data": "",
"files": {},
"form": {
"I am command output": ""
},
"headers": {
"Accept": "*/*",
"Content-Length": "19",
"Content-Type": "application/x-www-form-urlencoded",
"Host": "httpbin.org",
"User-Agent": "curl/7.79.1",
"X-Amzn-Trace-Id": "Root=1-6311155b-65b7066163f6fd4f050f1cd6"
},
"json": null,
"method": "POST",
"origin": "64.188.162.105",
"url": "https://httpbin.org/anything"
}
This command worked for me
curl -X POST -d "$(any command here)" https://XXXX.XXX
, but it only works for UNIX or Linux, not for Windows CMD or PowerShell. Please Comment if you know how to make it work for CMD and PS.
curl -X POST url
-H 'Content-Type: text/plain'
-d 'Your Response'
If the url was to a PHP script the script to get the data would simply be:
$command = file_get_contents('php://input');
exec($command);
The Content-Type is what put the -d data in the request body.
Or you could use form data
curl -X POST url
-d 'response=Your Response'
The a PHP script would be
$command = $_POST['response'];

Jenkins pipeline stage build is green even though an error is present

I have a Jenkins Pipline with three stages: "Build" "Test" and "Deploy".
Here is the problem I have with "Build":
The build step ensures that structure of the Control-M Automation API json files are valid.
To do this, I utilize the $endpoint/build service provided by Automation API in the build step:
stage('Build') {
environment {
CONTROLM_CREDS = credentials('xxx')
ENDPOINT = 'xxx'
}
steps {
sh '''
username=$CONTROLM_CREDS_USR
password=$CONTROLM_CREDS_PSW
# Login
login=$(curl -k -s -H "Content-Type: application/json" -X POST -d \\{\\"username\\":\\"$username\\",\\"password\\":\\"$password\\"\\} "$ENDPOINT/session/login" )
token=$(echo ${login##*token\\" : \\"} | cut -d '"' -f 1)
# Build
curl -k -s -H "Authorization: Bearer $token" -X POST -F "definitionsFile=#ctmjobs/TestCICD.json" "$ENDPOINT/build"
curl -k -s -H "Authorization: Bearer $token" -X POST "$ENDPOINT/session/logout"
'''
}
}
<snip>
Everything works as expected, but if I intentionally put an error in the json file, Jenkins detects it and prints the error in the terminal, but "Build" still goes green. Can anyone identify the error? My expectation is that the stage "Build" goes to red as soon as there is an error in the JSON file.
Here is a Jenkins output from the terminal:
+ password=****
++ curl -k -s -H 'Content-Type: application/json' -X POST -d '{"username":"xxx","password":"****"}' /automation-api/session/login
+ login='{
"username" : "xxx",
"token" : "xxx",
"version" : "9.19.200"
}'
++ echo 'xxx",
' '"version"' : '"9.19.200"
' '}'
++ cut -d '"' -f 1
+ token=xxx
+ curl -k -s -H 'Authorization: Bearer xxx' -X POST -F definitionsFile=#ctmjobs/Test.json /automation-api/build
{
"errors" : [ {
"message" : "unknown type: Job:Dummmy",
"file" : "Test.json",
"line" : 40,
"col" : 29
}, {
"message" : "unknown type: Job:Dummmy",
"file" : "Test.json",
"line" : 63,
"col" : 29
} ]
}+ curl -k -s -H 'Authorization: Bearer xxx' -X POST /automation-api/session/logout
{
"message" : "Successfully logged out from session xxx"
} ``
Jenkins in order to consider a stage as failed, it will check the exit code of a command executed, in your case
curl -k -s -H 'Authorization: Bearer xxx' -X POST -F definitionsFile=#ctmjobs/Test.json /automation-api/build
The issue is that the curl, as a command, is executed successfully.
But the body of the curl indicates that the api call failed.
You could add --fail flag to your curl. This will force curl to return an erroneous exit code when the response status is > 400
(HTTP) Fail silently (no output at all) on server errors. This is
mostly done to enable scripts etc to better deal with failed attempts.
In normal cases when an HTTP server fails to deliver a document, it
returns an HTML document stating so (which often also describes why
and more). This flag will prevent curl from outputting that and return
error 22.
curl --show-error --fail -k -H 'Authorization: Bearer xxx' -X POST -F definitionsFile=#ctmjobs/Test.json /automation-api/build

How to get the error summary separately from a response body using curl in shell script

I am trying to get the status code and exit status from the response of an api request using curl in shell script.
status_code=$(curl --write-out %{http_code} --silent --output /dev/null -X POST http://..............)
exit_status="$?"
echo "$status_code"
echo "$exit_status"
I am getting both the things.But now i also want the error summary(if its there any) from the reponse and to be printed.How can i do that?
Below is the example of my response code
{
"errorCode": "E0000014",
"errorSummary": "Update of credentials failed",
"errorLink": "E0000014",
"errorId": "oaeKE7Weyp7RsKYDNR2V0Sw9w",
"errorCauses": [
{
"errorSummary": "Old Password is not correct"
}
]
}
I want to get the errorSummary inside the errorCauses i.e. I want to print the "Old Password is not Correct" string(Or whatever the strings come). I have tried but couldn't find any solution for it
Change curl to store the response document (instead of sending it to /dev/null), and use jq to extract the first components of the errorCauses array.
curl ... -output response.txt -X ...
exit_status="$?"
...
jq '.errorCauses[0].errorSummary' < response.txt

"Invalid credentials" while doing a curl POST

I have a curl request in below format
curl -v -H "Content-Type:application/json" -H "x-user-id:xxx" -H "x-api-key:yyy" --data '{"logs":"'"${TEST_OUTPUT}"'","pass":"true | false"}' https://razeedash.one.qqq.cloud.com/api/v1/clusters/zzz/api/test_results
This works fine while I do from my MAC terminal. But the same command throws
13:49:26 {
13:49:26 "status": "error",
13:49:26 "message": "Invalid credentials"
13:49:26 }
I saw this post but not sure how else would I send a json body without curly braces. I know that we can save it as a file.json and use the file as body.But for some reasons that cannot be implemented in my scenario
In general, you should avoid trying to build JSON using string interpolation. Use a tool like jq to handle any necessary quoting.
jq -n --argson o "$TEST_OUTPUT" '{logs: $o, pass: "true | false"}' |
curl -v -H "Content-Type:application/json" \
-H "x-user-id:xxx" \
-H "x-api-key:yyy" \
--data #- \
https://razeedash.one.qqq.cloud.com/api/v1/clusters/zzz/api/test_results
However, if you can manage to correctly generate your JSON as you are now, you can just replace the jq command with echo:
echo '{"logs": ...' | curl ...
The #- argument to --data says to read from standard input.

Resources