Hi There,
I'm working on bash script to take snapshots of Digital Ocean volumes and droplets through cURL api. I can get all snapshot list and post new snapshot through the script. However I can delete the snapshot through API by manual selecting the ID, but I want to automatation. For that I have to pick the ID of oldest snapshot from get response of snapshot list. can someone please help me in this regard?
My script:
#! /bin/bash
token= xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
get_list=$(curl -X GET \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $token" \
"https://api.digitalocean.com/v2/snapshots?page=1&per_page=100&resource_type=volume")
echo $get_list > get_list
d=$(date +'%Y%m%d%H%M')
echo -e "Get List of Snapshots before delete\n" $get_list "\n\n\n\n"
y=1
for ((x=21;x<=2602;x+=265))
do
id=${get_list:$x:36}
echo $id > id$y
((y += 1))
done
rm -rf id
#curl -X DELETE \
#-H 'Content-Type: application/json' \
#-H "Authorization: Bearer $token" \
#"https://api.digitalocean.com/v2/snapshots/${get_list:274:36}"
#get_list= $(curl -X GET -H "Content-Type: application/json" \
#-H "Authorization: Bearer $token" \
#"https://api.digitalocean.com/v2/droplets?page=1&per_page=100")
#echo -e "Get List of Snapshots after delete\n" $get_list "\n\n\n\n"
#post_snap=$(curl -X POST \
# -H 'Content-Type: application/json' \
# -H "Authorization: Bearer $token" \
# -d '{"name":"'"$d"'"}' \
# "https://api.digitalocean.com/v2/volumes/6d44e0e2-4fc4-11ed-b137-0a58ac14c316/snapshots")
#get_list=$(curl -X GET \
# -H 'Content-Type: application/json' \
# -H "Authorization: Bearer $token" \
# "https://api.digitalocean.com/v2/snapshots?page=1&per_page=100&resource_type=volume")
#echo -e "Get List of Snapshots after new snap\n" $get_list "\n\n\n\n"
The list of snapshots from api response is pasted below:
{"snapshots":[{"id":"0e24bf13-515e-11ed-b137-0a58ac14c316","name":"202210211633","regions":["lon1"],"created_at":"2022-10-21T16:33:12Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"2ef69541-5162-11ed-9450-0a58ac14c2c7","name":"202210211702","regions":["lon1"],"created_at":"2022-10-21T17:02:45Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"689047e8-517a-11ed-a0ab-0a58ac14c028","name":"202210211955","regions":["lon1"],"created_at":"2022-10-21T19:56:10Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0,"tags":[]},{"id":"6cc0b0f2-516d-11ed-9fc6-0a58ac14c1c9","name":"2022-10-21_18:23","regions":["lon1"],"created_at":"2022-10-21T18:23:13Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"7e20fcfa-5171-11ed-a0ab-0a58ac14c028","name":"2022-10-21_18:52","regions":["lon1"],"created_at":"2022-10-21T18:52:20Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"96a3a010-515f-11ed-9fc6-0a58ac14c1c9","name":"2022-10-21-16-44","regions":["lon1"],"created_at":"2022-10-21T16:44:10Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"a13c0105-5178-11ed-9f93-0a58ac14c247","name":"2022-10-21-19-43","regions":["lon1"],"created_at":"2022-10-21T19:43:26Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"c4dfd95b-5174-11ed-b137-0a58ac14c316","name":"2022-10-21_19:15","regions":["lon1"],"created_at":"2022-10-21T19:15:47Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"d517a7f2-5169-11ed-a0ab-0a58ac14c028","name":"2022-10-21_17:57","regions":["lon1"],"created_at":"2022-10-21T17:57:30Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"f7180242-515a-11ed-b137-0a58ac14c316","name":"2022-10-21-16-11","regions":["lon1"],"created_at":"2022-10-21T16:11:05Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]}],"links":{},"meta":{"total":10}}
I have command I am running with works for one enviornment in a bash script e.g.
so for example
ZONE_ID=prod
#!/bin/bash
curl -X DELETE "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/purge_cache" \
-H "X-Auth-Email: EMAIL" \
-H "X-Auth-Key: AUTH_KEY" \
-H "Content-Type: application/json" \
--data '{"purge_everything":true}'
What i would like to do is change the data item if possible for a different environment depending on the zone_id
ZONE_ID=prod
ZONE_ID=UAT
for example UAT would be
curl -X DELETE "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/purge_cache" \
-H "X-Auth-Email: EMAIL" \
-H "X-Auth-Key: AUTH_KEY" \
-H "Content-Type: application/json" \
--data '{"files":["http://www.example.com/css/styles.css",{"url":"http://www.example.com/cat_picture.jpg","headers":{"Origin":"https://www.cloudflare.com","CF-IPCountry":"US","CF-Device-Type":"desktop"}}]}'
So what I kind of want to do is the following, but this doesnt work
#!/bin/bash
curl -X DELETE "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/purge_cache" \
-H "X-Auth-Email: EMAIL" \
-H "X-Auth-Key: AUTH_KEY" \
-H "Content-Type: application/json" \
if [$ZONE_ID -q prod]; then
--data '{"purge_everything":true}'
else
--data '{"files":["http://www.example.com/css/styles.css",{"url":"http://www.example.com/cat_picture.jpg","headers":{"Origin":"https://www.cloudflare.com","CF-IPCountry":"US","CF-Device-Type":"desktop"}}]}'
One possible way to do it is:
#! /bin/bash
curl -X DELETE "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/purge_cache" \
-H "X-Auth-Email: EMAIL" \
-H "X-Auth-Key: AUTH_KEY" \
-H "Content-Type: application/json" \
--data "$(if [[ $ZONE_ID == prod ]]; then
echo '{"purge_everything":true}'
else
echo '{"files":["http://www.example.com/css/styles.css",{"url":"http://www.example.com/cat_picture.jpg","headers":{"Origin":"https://www.cloudflare.com","CF-IPCountry":"US","CF-Device-Type":"desktop"}}]}'
fi)"
However, a much better option (for clarity and maintainability) is to put the --data argument in a variable.
I would recommend set variable before use it in curl command
example as follow
DATA=""
if [ "$ZONE_ID" = "prod" ]; then
DATA='{"purge_everything":true}'
else
DATA='{"files":["http://www.example.com/css/styles.css",{"url":"http://www.examp
le.com/cat_picture.jpg","headers":{"Origin":"https://www.cloudflare.com","CF-IPC
ountry":"US","CF-Device-Type":"desktop"}}]}'
fi
curl -X DELETE "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/purge_cache"
\
-H "X-Auth-Email: EMAIL" \
-H "X-Auth-Key: AUTH_KEY" \
-H "Content-Type: application/json" \
--data $DATA
This question already has answers here:
Difference between single and double quotes in Bash
(7 answers)
Closed 4 years ago.
I am curling Azure Log Analytics for some info, but first I need to grab an OAuth token from 1 command and pass it into the next. I have the following Curl commands which I have tested fine on their own (copying pasting the output for the next input), however I want to pass the OAuth token output as a variable for an automation task, but for some reason it is not able to read the variable into the next command.
token=$(curl -X POST \
https://login.microsoftonline.com/{{subscriptionID}}/oauth2/token \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-d 'grant_type=client_credentials&client_id={{clientID}}&client_secret={{clientSECRET}}&resource=https%3A%2F%2Fapi.loganalytics.io' \
| jq .access_token)
curl -X POST \
https://api.loganalytics.io/v1/workspaces/{{workspaceID}}/query \
-H 'Authorization: Bearer $token' \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{ "query": "AzureActivity | summarize count() by Category" }'
Unfortunately when I run this command it responds back that a token is needed.
{"error":{"message":"Valid authentication was not provided","code":"AuthorizationRequiredError"}}
However, if I were to echo the $token variable it shows that it was saved
beefcake#ubuntu:~$ echo $token
"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1...."
As I said, the commands work fine if I remove the token=$(..) and just copy/paste the output into the next input. Any ideas why this won't work for automation?
#Aserre had the right mindset. Turns out that jq copies the inverted commas " " from the string, whereas the bearer token requires none. Thus my first command should have looked like this:
token=$(curl -X POST \
https://login.microsoftonline.com/{{subscriptionID}}/oauth2/token \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-d 'grant_type=client_credentials&client_id={{clientID}}&client_secret={{clientSECRET}}&resource=https%3A%2F%2Fapi.loganalytics.io' \
| jq -r .access_token)
Note the last line that has the -r command for removing the double quotes. Which showed an echo of:
beefcake#ubuntu:~$ echo $token
eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIs....
Note the " " removed from the echo. In addition to that, I had to alter the next command where I replaced 'Authorization: Bearer $token' with "Authorization: Bearer $token":
curl -X POST \
https://api.loganalytics.io/v1/workspaces/{{workspaceID}}/query \
-H "Authorization: Bearer $token" \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{ "query": "AzureActivity | summarize count() by Category" }'
I want to upload a big file with curl.
For that, I want to split it, without saving it to disk (like with split). I tried to use --continue-at with Content-Length.
curl -s \
--request PATCH \
--header "Content-Type: application/offset+octet-stream" \
--header "Content-Length: ${length}" \
--header "Upload-Offset: ${offset}" \
--continue-at "${offset}" \
--upload-file "${file}" \
"${dest}"
But curl "overshoots" and ignores Content-Length. Is there something like --stop-at? Alternatively, I have to use dd, if necessary.
EDIT
dd solution:
curl -s \
--request PATCH \
--header "Content-Type: application/offset+octet-stream" \
--header "Content-Length: ${length}" \
--header "Upload-Offset: ${offset}" \
--data-binary "#-" \
"${dest}" < <(dd if=${file} skip=${offset} count=${length} iflag=skip_bytes,count_bytes 2>/dev/null)
but if possible I would like to use only cURL..
I trying to get this bash script to run a speedtest (speedtest-cli) then pass the output as a variable to pushbullet via curl.
#!/bin/bash
speed=$(speedtest --simple)
curl --header 'Access-Token: <-ACCESS-TOKEN->' \
--header 'Content-Type: application/json' \
--data-binary {"body":"'"$speed"'","title":"SpeedTest","type":"note"}' \
--request POST \
https://api.pushbullet.com/v2/pushes
Other commands have worked well using this method (eg. whoami) but speedtest and ifconfig just get an error like this:
{"error":{"code":"invalid_request","type":"invalid_request","message":"Failed to decode JSON body.","cat":"(=^‥^=)"},"error_code":"invalid_request"}
Your quoting is wrong:
speed=$(speedtest --simple)
curl --header 'Access-Token: o.4q87SC5INy6nMQZqVHJeymwRsvMXW74j' \
--header 'Content-Type: application/json' \
--data-binary "{\"body\":\"$speed\",\"title\":\"SpeedTest\",\"type\":\"note\"}" \
--request POST \
https://api.pushbullet.com/v2/pushes
Reading from a here document simplifies the quoting:
speed=$(speedtest --simple)
curl --header 'Access-Token: o.4q87SC5INy6nMQZqVHJeymwRsvMXW74j' \
--header 'Content-Type: application/json' \
--data-binary #- \
--request POST \
https://api.pushbullet.com/v2/pushes <<EOF
{ "body": "$speed",
"title": "SpeedTest",
"type": "note"
}
EOF
However, in general you should not assume that the contents of the variable are a properly encoded JSON string, so use a tool like jq to generate the JSON for you.
jq -n --arg data "$(speedtest --simple)" \
'{body: $data, title: "SpeedTest", type: "note"}' |
curl --header 'Access-Token: o.4q87SC5INy6nMQZqVHJeymwRsvMXW74j' \
--header 'Content-Type: application/json' \
--data-binary #- \
--request POST \
https://api.pushbullet.com/v2/pushes
This can be refactored easily:
post_data () {
url=$1
token=$2
data=$3
jq -n --arg d "$data" \
'{body: $d, title: "SpeedTest", type: "note"}' |
curl --header "Access-Token: $token" \
--header 'Content-Type: application/json' \
--data-binary #- \
--request POST \
"$url"
}
post_data "https://api.pushbullet.com/v2/pushes" \
"o.4q87SC5INy6nMQZqVHJeymwRsvMXW74j" \
"$(speedtest ---simple)"