Hi There,
I'm working on bash script to take snapshots of Digital Ocean volumes and droplets through cURL api. I can get all snapshot list and post new snapshot through the script. However I can delete the snapshot through API by manual selecting the ID, but I want to automatation. For that I have to pick the ID of oldest snapshot from get response of snapshot list. can someone please help me in this regard?
My script:
#! /bin/bash
token= xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
get_list=$(curl -X GET \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $token" \
"https://api.digitalocean.com/v2/snapshots?page=1&per_page=100&resource_type=volume")
echo $get_list > get_list
d=$(date +'%Y%m%d%H%M')
echo -e "Get List of Snapshots before delete\n" $get_list "\n\n\n\n"
y=1
for ((x=21;x<=2602;x+=265))
do
id=${get_list:$x:36}
echo $id > id$y
((y += 1))
done
rm -rf id
#curl -X DELETE \
#-H 'Content-Type: application/json' \
#-H "Authorization: Bearer $token" \
#"https://api.digitalocean.com/v2/snapshots/${get_list:274:36}"
#get_list= $(curl -X GET -H "Content-Type: application/json" \
#-H "Authorization: Bearer $token" \
#"https://api.digitalocean.com/v2/droplets?page=1&per_page=100")
#echo -e "Get List of Snapshots after delete\n" $get_list "\n\n\n\n"
#post_snap=$(curl -X POST \
# -H 'Content-Type: application/json' \
# -H "Authorization: Bearer $token" \
# -d '{"name":"'"$d"'"}' \
# "https://api.digitalocean.com/v2/volumes/6d44e0e2-4fc4-11ed-b137-0a58ac14c316/snapshots")
#get_list=$(curl -X GET \
# -H 'Content-Type: application/json' \
# -H "Authorization: Bearer $token" \
# "https://api.digitalocean.com/v2/snapshots?page=1&per_page=100&resource_type=volume")
#echo -e "Get List of Snapshots after new snap\n" $get_list "\n\n\n\n"
The list of snapshots from api response is pasted below:
{"snapshots":[{"id":"0e24bf13-515e-11ed-b137-0a58ac14c316","name":"202210211633","regions":["lon1"],"created_at":"2022-10-21T16:33:12Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"2ef69541-5162-11ed-9450-0a58ac14c2c7","name":"202210211702","regions":["lon1"],"created_at":"2022-10-21T17:02:45Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"689047e8-517a-11ed-a0ab-0a58ac14c028","name":"202210211955","regions":["lon1"],"created_at":"2022-10-21T19:56:10Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0,"tags":[]},{"id":"6cc0b0f2-516d-11ed-9fc6-0a58ac14c1c9","name":"2022-10-21_18:23","regions":["lon1"],"created_at":"2022-10-21T18:23:13Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"7e20fcfa-5171-11ed-a0ab-0a58ac14c028","name":"2022-10-21_18:52","regions":["lon1"],"created_at":"2022-10-21T18:52:20Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"96a3a010-515f-11ed-9fc6-0a58ac14c1c9","name":"2022-10-21-16-44","regions":["lon1"],"created_at":"2022-10-21T16:44:10Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"a13c0105-5178-11ed-9f93-0a58ac14c247","name":"2022-10-21-19-43","regions":["lon1"],"created_at":"2022-10-21T19:43:26Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"c4dfd95b-5174-11ed-b137-0a58ac14c316","name":"2022-10-21_19:15","regions":["lon1"],"created_at":"2022-10-21T19:15:47Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"d517a7f2-5169-11ed-a0ab-0a58ac14c028","name":"2022-10-21_17:57","regions":["lon1"],"created_at":"2022-10-21T17:57:30Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]},{"id":"f7180242-515a-11ed-b137-0a58ac14c316","name":"2022-10-21-16-11","regions":["lon1"],"created_at":"2022-10-21T16:11:05Z","resource_id":"6d44e0e2-4fc4-11ed-b137-0a58ac14c316","resource_type":"volume","min_disk_size":1,"size_gigabytes":0.0248,"tags":[]}],"links":{},"meta":{"total":10}}
I have command I am running with works for one enviornment in a bash script e.g.
so for example
ZONE_ID=prod
#!/bin/bash
curl -X DELETE "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/purge_cache" \
-H "X-Auth-Email: EMAIL" \
-H "X-Auth-Key: AUTH_KEY" \
-H "Content-Type: application/json" \
--data '{"purge_everything":true}'
What i would like to do is change the data item if possible for a different environment depending on the zone_id
ZONE_ID=prod
ZONE_ID=UAT
for example UAT would be
curl -X DELETE "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/purge_cache" \
-H "X-Auth-Email: EMAIL" \
-H "X-Auth-Key: AUTH_KEY" \
-H "Content-Type: application/json" \
--data '{"files":["http://www.example.com/css/styles.css",{"url":"http://www.example.com/cat_picture.jpg","headers":{"Origin":"https://www.cloudflare.com","CF-IPCountry":"US","CF-Device-Type":"desktop"}}]}'
So what I kind of want to do is the following, but this doesnt work
#!/bin/bash
curl -X DELETE "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/purge_cache" \
-H "X-Auth-Email: EMAIL" \
-H "X-Auth-Key: AUTH_KEY" \
-H "Content-Type: application/json" \
if [$ZONE_ID -q prod]; then
--data '{"purge_everything":true}'
else
--data '{"files":["http://www.example.com/css/styles.css",{"url":"http://www.example.com/cat_picture.jpg","headers":{"Origin":"https://www.cloudflare.com","CF-IPCountry":"US","CF-Device-Type":"desktop"}}]}'
One possible way to do it is:
#! /bin/bash
curl -X DELETE "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/purge_cache" \
-H "X-Auth-Email: EMAIL" \
-H "X-Auth-Key: AUTH_KEY" \
-H "Content-Type: application/json" \
--data "$(if [[ $ZONE_ID == prod ]]; then
echo '{"purge_everything":true}'
else
echo '{"files":["http://www.example.com/css/styles.css",{"url":"http://www.example.com/cat_picture.jpg","headers":{"Origin":"https://www.cloudflare.com","CF-IPCountry":"US","CF-Device-Type":"desktop"}}]}'
fi)"
However, a much better option (for clarity and maintainability) is to put the --data argument in a variable.
I would recommend set variable before use it in curl command
example as follow
DATA=""
if [ "$ZONE_ID" = "prod" ]; then
DATA='{"purge_everything":true}'
else
DATA='{"files":["http://www.example.com/css/styles.css",{"url":"http://www.examp
le.com/cat_picture.jpg","headers":{"Origin":"https://www.cloudflare.com","CF-IPC
ountry":"US","CF-Device-Type":"desktop"}}]}'
fi
curl -X DELETE "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/purge_cache"
\
-H "X-Auth-Email: EMAIL" \
-H "X-Auth-Key: AUTH_KEY" \
-H "Content-Type: application/json" \
--data $DATA
I am writing a bash script that does this :
Get a list of customers
For each customer, query an API (get files) This returns id and name
For each file, do a PUT.
My question is, I want to both "id" and "name" in my PUT body, so how can I get this in a single loop on "files" API. I have written this:
URL="https://some-url.com"
API_TOKEN="some-token"
get_customers() {
curl --insecure \
--fail \
--header "Accept: application/json" \
--header "Authorization: Bearer ${API_TOKEN}" \
-k ${URL}/api/org/customers
}
get_files() {
curl --insecure \
--fail \
--header "Accept: application/json" \
--header "Authorization: Bearer ${API_TOKEN}" \
-k ${URL}/api/files/files
}
put_capability() {
echo ""
curl --insecure \
--fail \
--silent \
--header "Content-Type: application/json" \
--header "Authorization: Bearer ${API_TOKEN}" \
--data #- \
${URL}/api/files/files/{id}
echo ""
}
customers=($(get_customers | jq --raw-output '.values[].id'))
for customer_id in "${customers[#]}"; do
fileshares=($(get_files ${customer_id} | jq --raw-output '.values[].id'))
fileshares-names=($(get_files ${customer_id} | jq --raw-output '.values[].name'))
for fileshare_id in "${fileshares[#]}"; do
#call function put_capability and do subsequent PUT.
#PUT needs two parameters, fileshares and fileshares-names
done
done
Basically, I don't want to call "get_files" twice as I am doing now. I need both the "fileshares" and "fileshares-names" in my subsequent PUT API. Can this be done in one for loop on file shares?
You can parse get_files output once by storing it in a temp file like that :
customers=($(get_customers | jq --raw-output '.values[].id'))
for customer_id in "${customers[#]}"; do
get_files ${customer_id} > /tmp/cust.txt
fileshares=($(cat /tmp/cust.txt | jq --raw-output '.values[].id'))
fileshares-names=($(cat /tmp/cust.txt | jq --raw-output '.values[].name'))
for fileshare_id in "${fileshares[#]}"; do
#call function put_capability and do subsequent PUT.
#PUT needs two parameters, fileshares and fileshares-names
done
done
If I run the following command, everything works great:
curl -s -o /dev/null -w %{http_code} \
http://www.example.com/mywebservice/generic.example.com \
--header "Content-type: application/json" --request PUT \
--data '{"hostname":"generic.example.com", "project":"none", "usage":"generic", "environment":"production"}'
When I put this command into a variable in a /bin/sh script, and execute it, I get "HTTP 415 Unsupported Media Type" showing up in the server logs:
#!/bin/sh
read -d '' CMD << EOF
curl -s -o /dev/null -w %{http_code} http://www.example.com/mywebservice/generic.example.com \
--header "Content-type: application/json" --request PUT \
--data '{"hostname":"generic.example.com", "project":"none", "usage":"generic", "environment":"production"}'
EOF
`$CMD`
I verified the command by printing it out and it looks example like the manual one I run:
#!/bin/sh
read -d '' CMD << EOF
curl -s -o /dev/null -w %{http_code} http://www.example.com/mywebservice/generic.example.com \
--header "Content-type: application/json" --request PUT \
--data '{"hostname":"generic.example.com", "project":"none", "usage":"generic", "environment":"production"}'
EOF
echo $CMD
Output:
curl -s -o /dev/null -w %{http_code} http://www.example.com/mywebservice/generic.example.com --header "Content-type: application/json" --request PUT --data '{"hostname":"generic.example.com", "project":"none", "usage":"generic", "environment":"production"}'
For me it worked this way:
#!/bin/sh
read -d '' CMD << EOF
`/usr/bin/curl -s http://localhost:8080/someurl \
--header "Content-type: application/json"`
EOF
echo $CMD
What if evaluate it inside of EOF?
I trying to get this bash script to run a speedtest (speedtest-cli) then pass the output as a variable to pushbullet via curl.
#!/bin/bash
speed=$(speedtest --simple)
curl --header 'Access-Token: <-ACCESS-TOKEN->' \
--header 'Content-Type: application/json' \
--data-binary {"body":"'"$speed"'","title":"SpeedTest","type":"note"}' \
--request POST \
https://api.pushbullet.com/v2/pushes
Other commands have worked well using this method (eg. whoami) but speedtest and ifconfig just get an error like this:
{"error":{"code":"invalid_request","type":"invalid_request","message":"Failed to decode JSON body.","cat":"(=^‥^=)"},"error_code":"invalid_request"}
Your quoting is wrong:
speed=$(speedtest --simple)
curl --header 'Access-Token: o.4q87SC5INy6nMQZqVHJeymwRsvMXW74j' \
--header 'Content-Type: application/json' \
--data-binary "{\"body\":\"$speed\",\"title\":\"SpeedTest\",\"type\":\"note\"}" \
--request POST \
https://api.pushbullet.com/v2/pushes
Reading from a here document simplifies the quoting:
speed=$(speedtest --simple)
curl --header 'Access-Token: o.4q87SC5INy6nMQZqVHJeymwRsvMXW74j' \
--header 'Content-Type: application/json' \
--data-binary #- \
--request POST \
https://api.pushbullet.com/v2/pushes <<EOF
{ "body": "$speed",
"title": "SpeedTest",
"type": "note"
}
EOF
However, in general you should not assume that the contents of the variable are a properly encoded JSON string, so use a tool like jq to generate the JSON for you.
jq -n --arg data "$(speedtest --simple)" \
'{body: $data, title: "SpeedTest", type: "note"}' |
curl --header 'Access-Token: o.4q87SC5INy6nMQZqVHJeymwRsvMXW74j' \
--header 'Content-Type: application/json' \
--data-binary #- \
--request POST \
https://api.pushbullet.com/v2/pushes
This can be refactored easily:
post_data () {
url=$1
token=$2
data=$3
jq -n --arg d "$data" \
'{body: $d, title: "SpeedTest", type: "note"}' |
curl --header "Access-Token: $token" \
--header 'Content-Type: application/json' \
--data-binary #- \
--request POST \
"$url"
}
post_data "https://api.pushbullet.com/v2/pushes" \
"o.4q87SC5INy6nMQZqVHJeymwRsvMXW74j" \
"$(speedtest ---simple)"