GET CURL doesn't escape some of the characters [duplicate] - bash

This question already has answers here:
Escaping characters in bash (for JSON)
(13 answers)
Parsing JSON with Unix tools
(45 answers)
Closed 10 days ago.
I have the following GET CURL from which I get an xml.
curl -X 'GET' \
'http://local/something/something2' \
-H 'accept: application/json' \
-H 'authorization: auth'
Now I want to use the previous xml received above within this POST CURL:
curl -X 'POST' \
'http://something/something2' \
-H 'accept: application/json' \
-H 'authorization: auth' \
-H 'Content-Type: application/json' \
-d '{
"components": [
{
"locator": "sample",
"config": xml file from above
}
]
}'
How can I make the second CURL with POST?

See this post to see how to capture the output of the first command into a variable. Use it like this:
output=$(curl -X 'GET' \
'http://local/something/something2' \
-H 'accept: application/json' \
-H 'authorization: auth')
# Assuming the $output variable is a JSON object, with a property
# called 'result', use 'jq' to extract the value of that property
result=$(jq -r '.result' <<< "$output")
# As noted above, escape the double quotes with backslashes
curl -X 'POST' \
'http://something/something2' \
-H 'accept: application/json' \
-H 'authorization: auth' \
-H 'Content-Type: application/json' \
-d "{
\"components\": [
{
\"locator\": \"sample\",
\"config\": \"$result\"
}
]
}"
Note the double quotes - double quotes must be there so $output variable can be used. As a result, the double quotes in the JSON need to be escaped.

Related

Use CURL POST using a CURL GET output in bash [duplicate]

This question already has answers here:
Escaping characters in bash (for JSON)
(13 answers)
Parsing JSON with Unix tools
(45 answers)
Closed 10 days ago.
I have the following GET CURL from which I get an xml.
curl -X 'GET' \
'http://local/something/something2' \
-H 'accept: application/json' \
-H 'authorization: auth'
Now I want to use the previous xml received above within this POST CURL:
curl -X 'POST' \
'http://something/something2' \
-H 'accept: application/json' \
-H 'authorization: auth' \
-H 'Content-Type: application/json' \
-d '{
"components": [
{
"locator": "sample",
"config": xml file from above
}
]
}'
How can I make the second CURL with POST?
See this post to see how to capture the output of the first command into a variable. Use it like this:
output=$(curl -X 'GET' \
'http://local/something/something2' \
-H 'accept: application/json' \
-H 'authorization: auth')
# Assuming the $output variable is a JSON object, with a property
# called 'result', use 'jq' to extract the value of that property
result=$(jq -r '.result' <<< "$output")
# As noted above, escape the double quotes with backslashes
curl -X 'POST' \
'http://something/something2' \
-H 'accept: application/json' \
-H 'authorization: auth' \
-H 'Content-Type: application/json' \
-d "{
\"components\": [
{
\"locator\": \"sample\",
\"config\": \"$result\"
}
]
}"
Note the double quotes - double quotes must be there so $output variable can be used. As a result, the double quotes in the JSON need to be escaped.

Passing Bearer token as variable in bash

I am trying to pass a Bearer token (variable $token)to invoke a job via curl command. However the single quote after the -H is not letting the value of variable $token being passed to curl command.
curl -X POST 'https://server.domain.com/v2/jobs/28723316-9373-44ba-9229-7c796f21b099/runs?project_id=aff59748-260a-476e-9578-b4f4a93e7a92' -H 'Content-Type: application/json' -H 'Authorization: Bearer $token -d { "job_run": {} }'
I get this error:
{"code":400,"error":"Bad Request","reason":"Bearer token format is invalid. Expected: 'Bearer '. Received: 'Bearer $token -d { "job_run": {} }'.","message":"Bearer token is invalid."}
I tried adding like the escape character with the variable $token:
curl -X POST 'https://server.domain.com/v2/jobs/28723316-9373-44ba-9229-7c796f21b099/runs?project_id=aff59748-260a-476e-9578-b4f4a93e7a92' -H 'Content-Type: application/json' -H 'Authorization: Bearer "\$token\" -d { "job_run": {} }'
I get the same error:
{"code":400,"error":"Bad Request","reason":"Bearer token format is invalid. Expected: 'Bearer '. Received: 'Bearer "\$token\" -d { "job_run": {} }'.","message":"Bearer token is invalid."}
I tried double quotes as well, it has been a few hours and I am unable to extract he variable value $token within single quotes.
Could some please assist and give me the correct syntax?
Thanks in advance
The problem here are the quotes. It should be like this:
curl -X POST 'https://server.domain.com/v2/jobs/28723316-9373-44ba-9229-7c796f21b099/runs?project_id=aff59748-260a-476e-9578-b4f4a93e7a92' -H 'Content-Type: application/json' -H "Authorization: Bearer $token" -d '{ "job_run": {} }'
In multiline:
curl -X POST 'https://server.domain.com/v2/jobs/28723316-9373-44ba-9229-7c796f21b099/runs?project_id=aff59748-260a-476e-9578-b4f4a93e7a92' \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $token" \
-d '{ "job_run": {} }'
Specifically, variables in bash aren't interpolated when in single quotes ('). Thus, we set the dynamic string inside double quotes (")
-H "Authorization: Bearer $token"
Also the -H and -d arguments are distinct, they should be quoted separately, in your code you have them combined in a single argument.
this works for me
token=$(curl 'https://example.com/v1.2/auths/login' \
-H 'Accept: application/json, text/plain, */*' \
-H 'Content-Type: application/json' \
--data-raw '{"username":"username","password":"password"}' | jq '.token')
echo $token
#to remove the double quotes from the token string
token=`sed -e 's/^"//' -e 's/"$//' <<<"$token"`
curl 'https://example.com/otherapi' \
-H 'Accept: application/json, text/plain, */*' \
-H "Authorization: Bearer $token"

Bash unable to read variable [duplicate]

This question already has answers here:
Difference between single and double quotes in Bash
(7 answers)
Closed 4 years ago.
I am curling Azure Log Analytics for some info, but first I need to grab an OAuth token from 1 command and pass it into the next. I have the following Curl commands which I have tested fine on their own (copying pasting the output for the next input), however I want to pass the OAuth token output as a variable for an automation task, but for some reason it is not able to read the variable into the next command.
token=$(curl -X POST \
https://login.microsoftonline.com/{{subscriptionID}}/oauth2/token \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-d 'grant_type=client_credentials&client_id={{clientID}}&client_secret={{clientSECRET}}&resource=https%3A%2F%2Fapi.loganalytics.io' \
| jq .access_token)
curl -X POST \
https://api.loganalytics.io/v1/workspaces/{{workspaceID}}/query \
-H 'Authorization: Bearer $token' \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{ "query": "AzureActivity | summarize count() by Category" }'
Unfortunately when I run this command it responds back that a token is needed.
{"error":{"message":"Valid authentication was not provided","code":"AuthorizationRequiredError"}}
However, if I were to echo the $token variable it shows that it was saved
beefcake#ubuntu:~$ echo $token
"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1...."
As I said, the commands work fine if I remove the token=$(..) and just copy/paste the output into the next input. Any ideas why this won't work for automation?
#Aserre had the right mindset. Turns out that jq copies the inverted commas " " from the string, whereas the bearer token requires none. Thus my first command should have looked like this:
token=$(curl -X POST \
https://login.microsoftonline.com/{{subscriptionID}}/oauth2/token \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-d 'grant_type=client_credentials&client_id={{clientID}}&client_secret={{clientSECRET}}&resource=https%3A%2F%2Fapi.loganalytics.io' \
| jq -r .access_token)
Note the last line that has the -r command for removing the double quotes. Which showed an echo of:
beefcake#ubuntu:~$ echo $token
eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIs....
Note the " " removed from the echo. In addition to that, I had to alter the next command where I replaced 'Authorization: Bearer $token' with "Authorization: Bearer $token":
curl -X POST \
https://api.loganalytics.io/v1/workspaces/{{workspaceID}}/query \
-H "Authorization: Bearer $token" \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{ "query": "AzureActivity | summarize count() by Category" }'

Bash: Command output as variable to curl error

I trying to get this bash script to run a speedtest (speedtest-cli) then pass the output as a variable to pushbullet via curl.
#!/bin/bash
speed=$(speedtest --simple)
curl --header 'Access-Token: <-ACCESS-TOKEN->' \
--header 'Content-Type: application/json' \
--data-binary {"body":"'"$speed"'","title":"SpeedTest","type":"note"}' \
--request POST \
https://api.pushbullet.com/v2/pushes
Other commands have worked well using this method (eg. whoami) but speedtest and ifconfig just get an error like this:
{"error":{"code":"invalid_request","type":"invalid_request","message":"Failed to decode JSON body.","cat":"(=^‥^=)"},"error_code":"invalid_request"}
Your quoting is wrong:
speed=$(speedtest --simple)
curl --header 'Access-Token: o.4q87SC5INy6nMQZqVHJeymwRsvMXW74j' \
--header 'Content-Type: application/json' \
--data-binary "{\"body\":\"$speed\",\"title\":\"SpeedTest\",\"type\":\"note\"}" \
--request POST \
https://api.pushbullet.com/v2/pushes
Reading from a here document simplifies the quoting:
speed=$(speedtest --simple)
curl --header 'Access-Token: o.4q87SC5INy6nMQZqVHJeymwRsvMXW74j' \
--header 'Content-Type: application/json' \
--data-binary #- \
--request POST \
https://api.pushbullet.com/v2/pushes <<EOF
{ "body": "$speed",
"title": "SpeedTest",
"type": "note"
}
EOF
However, in general you should not assume that the contents of the variable are a properly encoded JSON string, so use a tool like jq to generate the JSON for you.
jq -n --arg data "$(speedtest --simple)" \
'{body: $data, title: "SpeedTest", type: "note"}' |
curl --header 'Access-Token: o.4q87SC5INy6nMQZqVHJeymwRsvMXW74j' \
--header 'Content-Type: application/json' \
--data-binary #- \
--request POST \
https://api.pushbullet.com/v2/pushes
This can be refactored easily:
post_data () {
url=$1
token=$2
data=$3
jq -n --arg d "$data" \
'{body: $d, title: "SpeedTest", type: "note"}' |
curl --header "Access-Token: $token" \
--header 'Content-Type: application/json' \
--data-binary #- \
--request POST \
"$url"
}
post_data "https://api.pushbullet.com/v2/pushes" \
"o.4q87SC5INy6nMQZqVHJeymwRsvMXW74j" \
"$(speedtest ---simple)"

How to use a variable in a CURL request with bash?

Goal:
I'm using a bash CURL script to connect to the Cloudflare APIv4. The goal is to update an A-record. My script:
# Get current public IP
current_ip=curl --silent ipecho.net/plain; echo
# Update A record
curl -X PUT "https://api.cloudflare.com/client/v4/zones/ZONEIDHERE/dns_records/DNSRECORDHERE" \
-H "X-Auth-Email: EMAILHERE" \
-H "X-Auth-Key: AUTHKEYHERE" \
-H "Content-Type: application/json" \
--data '{"id":"ZONEIDHERE","type":"A","name":"example.com","content":"'"${current_ip}"'","zone_name":"example.com"}'
Problem:
The current_ip variable is not printed when I call it in my script. The output will be "content" : "" and not "content" : "1.2.3.4".
I used other stackoverflow posts and I'm trying to follow their examples but I think I'm still doing something wrong, just can't figure out what. :(
Using jq for this, as Charles Duffy's answer suggests, is a very good idea. However, if you can't or do not want to install jq here is what you can do with plain POSIX shell.
#!/bin/sh
set -e
current_ip="$(curl --silent --show-error --fail ipecho.net/plain)"
echo "IP: $current_ip"
# Update A record
curl -X PUT "https://api.cloudflare.com/client/v4/zones/ZONEIDHERE/dns_records/DNSRECORDHERE" \
-H "X-Auth-Email: EMAILHERE" \
-H "X-Auth-Key: AUTHKEYHERE" \
-H "Content-Type: application/json" \
--data #- <<END;
{
"id": "ZONEIDHERE",
"type": "A",
"name": "example.com",
"content": "$current_ip",
"zone_name": "example.com"
}
END
The reliable way to edit JSON from shell scripts is to use jq:
# set shell variables with your contents
email="yourEmail"
authKey="yourAuthKey"
zoneid="yourZoneId"
dnsrecord="yourDnsRecord"
# make sure we show errors; --silent without --show-error can mask problems.
current_ip=$(curl --fail -sS ipecho.net/plain) || exit
# optional: template w/ JSON content that won't change
json_template='{"type": "A", "name": "example.com"}'
# build JSON with content that *can* change with jq
json_data=$(jq --arg zoneid "$zoneid" \
--arg current_ip "$current_ip" \
'.id=$zoneid | .content=$current_ip' \
<<<"$json_template")
# ...and submit
curl -X PUT "https://api.cloudflare.com/client/v4/zones/$zoneid/dns_records/$dnsrecord" \
-H "X-Auth-Email: $email" \
-H "X-Auth-Key: $authKey" \
-H "Content-Type: application/json" \
--data "$json_data"

Resources