Loop through list for curl requests in bash - bash

I have a bash script that sends a curl request and displays the response.
#!/bin/bash
token=$(curl -k -X GET \
'https://v.mytesting.io/oauth/token?grant_type=password&username=user1&password=123' \
-H 'Authorization: Basic 12345678' \
-H 'Host: v.mytesting.io.io')
v=$( jq -r ".access_token" <<<"$token" )
ts=$(curl -k -X POST \
https://timeseries.mytimeseries.io/v5/time_series/query \
-H 'Authorization: Bearer '"$v" \
-H 'Content-Type: application/json' \
-H 'Host: timeseries.mytimeseries.io' \
-H 'tenant: 123-123-123' \
-d '{"operation" : "raw","responseFormat" : "kairosDB","startTime": "1d-ago","stopTime": "now","tagList" : [ {"tagId" : "V.S.23164117.AVG.10M"}]}')
p=$(jq '.queries[].sample_size, .queries[].results[].name' <<<"$ts")
echo "$p"
My current output is just a value and the name of the tagId.
My query only allows for 1 tagId ( you can see above )
I want to be able to set a list of tagId's.
Then when I run this script it should loop through the list of tagId's and execute the curl request replacing the V.S.23164117.AVG.10M with each value
in the list.
Then output the entire list of results into a file.
list would be like so - (I would love to be able to enter this list into a seperate file and the bash script calls that file. Sometimes this list can be a few hundred lines.
V.S.23164117.AVG.10M
V.S.23164118.AVG.10M
V.S.23164119.AVG.10M
V.S.23164115.AVG.10M
V.S.23164114.AVG.10M
output would like look so.
value tagId
value tagId
value tagId
100 V.S.23164117.AVG.10M
etc..
thank you for any help

You can loop over list of tags using a small script. I'm not 100% clean of the output format. You can change the 'echo' to match the required format.
Note minor change to quotes to allow variable expansion in the body.
The tags will be stored in a file, for examples, tags.txt
V.S.23164117.AVG.10M
V.S.23164118.AVG.10M
V.S.23164119.AVG.10M
And the script will be use the file
#! /bin/bash
# Use user defined list of tags
tags=tags.txt
token=$(curl -k -X GET \
'https://v.mytesting.io/oauth/token?grant_type=password&username=user1&password=123' \
-H 'Authorization: Basic 12345678' \
-H 'Host: v.mytesting.io.io')
v=$( jq -r ".access_token" <<<"$token" )
for tag in $(<$tags) ; do
ts=$(curl -k -X POST \
https://timeseries.mytimeseries.io/v5/time_series/query \
-H 'Authorization: Bearer '"$v" \
-H 'Content-Type: application/json' \
-H 'Host: timeseries.mytimeseries.io' \
-H 'tenant: 123-123-123' \
-d '{"operation" : "raw","responseFormat" : "kairosDB","startTime": "1d-ago","stopTime": "now","tagList" : [ {"tagId" : "'"$tag"'"}]}')
p=$(jq '.queries[].sample_size, .queries[].results[].name' <<<"$ts")
echo "$tag $p"
done

Related

Automate cURL to run through a list of ID's

I am attempting to run through a list of 10 contact ID's every ten minutes with cURL
This is the code I am trying to run
curl --request POST \
--url https://youraccountname.api-us1.com/api/3/contactAutomations \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--header 'api-token: test' \
--data '
{
"contactAutomation": {
"contact": THE VARIABLE,
"automation": 23
}
}'
The list of contact variables could just be line items in a text file.
Is this something a bash script with cron could handle - and it just deleted the 10 IDs it runs?
Or is this something I would need to use python and a database to run?
This is really a one off thing - so a cron with a script would be easiest since it doesn't need to be used more than one time through.
If you have a constant list of IDs:
for id in 123 456 555 777
do
curl --request POST \
--url https://youraccountname.api-us1.com/api/3/contactAutomations \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--header 'api-token: test' \
--data "
{
\"contactAutomation\": {
\"contact\": $id,
\"automation\": 23
}
}"
done

Not able to replace the value of the variable inside expression in bash script

I am trying to run a bash script, where I would like to make POST calls in a for loop as follows:
for depId in "${depIds[#]}"
do
echo "$depId" <--------------------------------- THIS IS PRINTING PROPER VALUE
curl 'https://student.service.com/api/student' \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-H 'Cookie: UISESSION=abcd' \
--data-raw '{"name":"Student Name","description":"Dummy","depId":$depId}' \ <---- HERE I CANNOT GET THE VALUE OF THE VARIABLE
--compressed
echo "$content"
done
As mentioned above, I cannot get the value of the department id in the URL, with the above form, I am getting a Request Malformed exception. I have even tried with ${depId}, but no luck.
Could anyone please help here ?
Try flipping your quotes around the variable.
--data-raw '{"name":"Student Name","description":"Dummy","depId":'"$depId"'}' \

Replacing IP in curl command with bash variable

I'm currently trying to make a DDNS script that interacts with the Cloudflare API to catch changes in my ip address and automatically fix the ip address change for my web server. Everything is working correctly so far except I can't get $IP to be put properly in the curl statement. I first run a python script from within the bash script to get the ip address, then run the curl statement in the bash script. Here's what the python script looks like (it returns an ip address like "1.1.1.1" with quotations included because the curl command requires the quotations)
#!/usr/bin/python3
import subprocess as sp
def main():
command = "dig +short myip.opendns.com #resolver1.opendns.com";
ip = sp.check_output(command, shell=True).decode('utf-8').strip('\n');
ip_tmp = ip;
ip_tmp = '"' + ip + '"';
ip = ip_tmp;
print(ip);
if __name__ == "__main__":
main();
And the bash script looks like this:
#!/bin/bash
IP=$("./getIP.py")
curl -X PUT "https://api.cloudflare.com/client/v4/zones/zone_id/dns_records/dns_id" \
-H "X-Auth-Email: example.com" \
-H "X-Auth-Key: authkey" \
-H "Content-Type: application/json" \
--data '{"type":"A","name":"example.com","content":$IP,"ttl":120,"proxied":true}'
I've tried to have the python script only return numbers and then added the quotations in the bash script and now vice versa and I can't seem to get it to work. The last line should end up looking like this once the variable replaces with quotations around the ip address:
'{"type":"A","name":"example.com","content":"127.0.0.1","ttl":120,"proxied":true}'
The single quotes around your json structure prevent the variable from expanding.
You have a few options that are readily available.
Ugly quote escaping inside/around your json.
"{\"type\":\"A\",\"name\":\"example.com\",\"content\":$IP,\"ttl\":120,\"proxied\":true}"
Having the python write this data to a file and telling curl to use that file for the source of the post data.
curl -X PUT "https://api.cloudflare.com/client/v4/zones/zone_id/dns_records/dns_id" \
-H "X-Auth-Email: example.com" \
-H "X-Auth-Key: authkey" \
-H "Content-Type: application/json" \
--data #file_you_wrote_your_json_to.json
Using the python requests or urllib modules to issue the request to cloud flare.
Update your main() function to return the IP instead of print it.
my_ip = main()
url = "https://api.cloudflare.com/client/v4/zones/zone_id/dns_records/dns_id"
myheaders = {
"X-Auth-Email": "example.com",
"X-Auth-Key": "authkey",
"Content-Type": "application/json"
}
myjson = {
"type":"A",
"name":"example.com",
"content":my_ip,
"ttl":120,
"proxied":true
}
requests.put(url, headers=myheaders, data=myjson)
Better yet, just do it in bash. Cloudflare DDNS on github.
One shot to fetch the dynamic A-record ID:
curl -X GET "https://api.cloudflare.com/client/v4/zones/**Zone ID** \
/dns_records?type=A&name=dynamic" \
-H "Host: api.cloudflare.com" \
-H "User-Agent: ddclient/3.9.0" \
-H "Connection: close" \
-H "X-Auth-Email: example#example.com" \
-H "X-Auth-Key: "**Authorization key**" \
-H "Content-Type: application/json"
Cron job (* * * * *) to set the dynamic A-record:
#/usr/bin/env sh
AUTH_EMAIL=example#example.com
AUTH_KEY=** CF Authorization key **
ZONE_ID=** CF Zone ID **
A_RECORD_NAME="dynamic"
A_RECORD_ID=** CF A-record ID from cloudflare-dns-id.sh **
IP_RECORD="/tmp/ip-record"
RECORDED_IP=`cat $IP_RECORD`
PUBLIC_IP=$(curl --silent https://api.ipify.org) || exit 1
if [ "$PUBLIC_IP" = "$RECORDED_IP" ]; then
exit 0
fi
echo $PUBLIC_IP > $IP_RECORD
RECORD=$(cat <<EOF
{ "type": "A",
"name": "$A_RECORD_NAME",
"content": "$PUBLIC_IP",
"ttl": 180,
"proxied": false }
EOF
)
curl "https://api.cloudflare.com/client/v4/zones/$ZONE_ID \
/dns_records/$A_RECORD_ID" \
-X PUT \
-H "Content-Type: application/json" \
-H "X-Auth-Email: $AUTH_EMAIL" \
-H "X-Auth-Key: $AUTH_KEY" \
-d "$RECORD"

Data cannot GET by REST API

I use curl to query data from Parse, like this:
curl -X GET \
-H "X-Parse-Application-Id: xxxxxxxxxxxx" \
-H "X-Parse-REST-API-Key: xxxxxxx" \
https://api.parse.com/1/classes/TestObject
and it always return {"results":[]}. But I could GET 'https://api.parse.com/1/users' successfully.

rest api openam add user parametrised

hello I have this script in bash:
first part : authentication against server - works fine.
second part: adds user - and this works fine when I define user directly in curl code - but when I want to add user from parameters.. then I got a message error:
"{"code":400,"reason":"Bad Request","message":"The request could not
be processed because the provided content is not valid
JSON","detail":"Unexpected character ('$' (code 36)): expected a valid
value (number, String, array, object, 'true', 'false' or 'null')\n at
[Source: org.apache.catalina.connector.CoyoteInputStream#3a248e6a;
line: 2, column: 18]"}"
Any ideas how can I pass values to this script ? :)
Regards
#!/bin/bash
adm_user="admin"
adm_pass="secret"
user_name=""
user_pass=""
auth_url="http://url/OpenAM-11.0.0/json/authenticate"
add_user_url="http://url/OpenAM-11.0.0/json/users/?_action=create"
session_id=$(curl \
--request POST --header "X-OpenAM-Username: $adm_user " \
--header "X-OpenAM-Password: $adm_pass" \
--header "Content-Type: application/json" \
--data "{}" $auth_url | cut -d"\"" -f4 )
sleep 1
curl \
--request POST \
--header "iplanetDirectoryPro: $session_id" \
--header "Content-Type: application/json" \
--data \
'{
"username": "$1",
"userpassword": "secret12",
"mail": "bjensen#example.com"
}' \
$add_user_url
OK, mates I got the answer:
'{
"username": "'"$1"'",
"userpassword": "'"$2"'" ,
"mail": "bjensen#example.com"
}' \

Resources