Curl for loop issue - shell

for a in {P02183606,P02183608}
do
for b in {PID,PID2}
do
curl -i -H "Authorization:Token sample" "Content-type: application/json" -X GET "http://mastindia//project/push?applicationRefNo=$a&applicationFormId=$b"
done;
done;
Expected Output
Service should call 2 times not 4 times.(Issue is only $a and $ b values should be below)
http://applicationRefNo=P02183606&applicationFormId=PID
http://applicationRefNo=P02183608&applicationFormId=PID2

A nested for loop will always run n*n2 times.
In your case, there is no need to nest the loop, you can just do it like this:
#bin/bash
whatever=("P02183606" "PID" "P02183608" "PID2");
for((i = 0; i<${#whatever[#]}; i=i+2));
do
curl -i -H "Authorization:Token dont share it ffs" "Content-type: application/json" -X GET "http://indiafirstlife.com//onlineInsurance-rest/uploadDocument/uploadDocumentsOmniMannualPush?applicationRefNo=${whatever[i]}&applicationFormId=${whatever[i+1]}"
echo "Accessing: http://indiafirstlife.com/onlineInsurance-rest/uploadDocument/uploadDocumentsOmniMannualPush?applicationRefNo=${whatever[i]}&applicationFormId=${whatever[i+1]}"
done
Output:
Accessing: http://indiafirstlife.com/onlineInsurance-rest/uploadDocument/uploadDocumentsOmniMannualPush?applicationRefNo=P02183606&applicationFormId=PID
Accessing: http://indiafirstlife.com/onlineInsurance-rest/uploadDocument/uploadDocumentsOmniMannualPush?applicationRefNo=P02183608&applicationFormId=PID2

Related

How to detect end of _find in CouchDB?

Using CouchDB (3.1.1 on windows) I want to iterate on the results of a /db/_find using the bookmark mechanism.
It works pretty well, but I don't know how to end the looping. The code is similar to this:
while true
do
curl -s -X POST $dburl/db/_find -H "Content-Type: application/json" \
-d '{"selector":{bla bla},"bookmark":'$b'}' > result.dat
Extract bookmark from result.dat and assign it to b variable
done
How to end such loop? The result contains a valid bookmark even when there are no more documents to be returned and curl always succeeds.
Thanks for your help!
mario
Thanks for the suggestion RamblinRose!
Here is the solution that works beautifully:
n=200
b=null
while [ $n = 200 ]
do
x=`curl -s -X POST $db/kb/_find -H "Content-Type: application/json" \
-d '{"selector": {bla bla},"limit":200,"bookmark":'$b'}' | tee -a $tmp | jq '.bookmark,(.docs | length)'`
a=($x)
b=${a[0]}
n=${a[1]}
done

Not able to replace the value of the variable inside expression in bash script

I am trying to run a bash script, where I would like to make POST calls in a for loop as follows:
for depId in "${depIds[#]}"
do
echo "$depId" <--------------------------------- THIS IS PRINTING PROPER VALUE
curl 'https://student.service.com/api/student' \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-H 'Cookie: UISESSION=abcd' \
--data-raw '{"name":"Student Name","description":"Dummy","depId":$depId}' \ <---- HERE I CANNOT GET THE VALUE OF THE VARIABLE
--compressed
echo "$content"
done
As mentioned above, I cannot get the value of the department id in the URL, with the above form, I am getting a Request Malformed exception. I have even tried with ${depId}, but no luck.
Could anyone please help here ?
Try flipping your quotes around the variable.
--data-raw '{"name":"Student Name","description":"Dummy","depId":'"$depId"'}' \

Loop through list for curl requests in bash

I have a bash script that sends a curl request and displays the response.
#!/bin/bash
token=$(curl -k -X GET \
'https://v.mytesting.io/oauth/token?grant_type=password&username=user1&password=123' \
-H 'Authorization: Basic 12345678' \
-H 'Host: v.mytesting.io.io')
v=$( jq -r ".access_token" <<<"$token" )
ts=$(curl -k -X POST \
https://timeseries.mytimeseries.io/v5/time_series/query \
-H 'Authorization: Bearer '"$v" \
-H 'Content-Type: application/json' \
-H 'Host: timeseries.mytimeseries.io' \
-H 'tenant: 123-123-123' \
-d '{"operation" : "raw","responseFormat" : "kairosDB","startTime": "1d-ago","stopTime": "now","tagList" : [ {"tagId" : "V.S.23164117.AVG.10M"}]}')
p=$(jq '.queries[].sample_size, .queries[].results[].name' <<<"$ts")
echo "$p"
My current output is just a value and the name of the tagId.
My query only allows for 1 tagId ( you can see above )
I want to be able to set a list of tagId's.
Then when I run this script it should loop through the list of tagId's and execute the curl request replacing the V.S.23164117.AVG.10M with each value
in the list.
Then output the entire list of results into a file.
list would be like so - (I would love to be able to enter this list into a seperate file and the bash script calls that file. Sometimes this list can be a few hundred lines.
V.S.23164117.AVG.10M
V.S.23164118.AVG.10M
V.S.23164119.AVG.10M
V.S.23164115.AVG.10M
V.S.23164114.AVG.10M
output would like look so.
value tagId
value tagId
value tagId
100 V.S.23164117.AVG.10M
etc..
thank you for any help
You can loop over list of tags using a small script. I'm not 100% clean of the output format. You can change the 'echo' to match the required format.
Note minor change to quotes to allow variable expansion in the body.
The tags will be stored in a file, for examples, tags.txt
V.S.23164117.AVG.10M
V.S.23164118.AVG.10M
V.S.23164119.AVG.10M
And the script will be use the file
#! /bin/bash
# Use user defined list of tags
tags=tags.txt
token=$(curl -k -X GET \
'https://v.mytesting.io/oauth/token?grant_type=password&username=user1&password=123' \
-H 'Authorization: Basic 12345678' \
-H 'Host: v.mytesting.io.io')
v=$( jq -r ".access_token" <<<"$token" )
for tag in $(<$tags) ; do
ts=$(curl -k -X POST \
https://timeseries.mytimeseries.io/v5/time_series/query \
-H 'Authorization: Bearer '"$v" \
-H 'Content-Type: application/json' \
-H 'Host: timeseries.mytimeseries.io' \
-H 'tenant: 123-123-123' \
-d '{"operation" : "raw","responseFormat" : "kairosDB","startTime": "1d-ago","stopTime": "now","tagList" : [ {"tagId" : "'"$tag"'"}]}')
p=$(jq '.queries[].sample_size, .queries[].results[].name' <<<"$ts")
echo "$tag $p"
done

Replacing IP in curl command with bash variable

I'm currently trying to make a DDNS script that interacts with the Cloudflare API to catch changes in my ip address and automatically fix the ip address change for my web server. Everything is working correctly so far except I can't get $IP to be put properly in the curl statement. I first run a python script from within the bash script to get the ip address, then run the curl statement in the bash script. Here's what the python script looks like (it returns an ip address like "1.1.1.1" with quotations included because the curl command requires the quotations)
#!/usr/bin/python3
import subprocess as sp
def main():
command = "dig +short myip.opendns.com #resolver1.opendns.com";
ip = sp.check_output(command, shell=True).decode('utf-8').strip('\n');
ip_tmp = ip;
ip_tmp = '"' + ip + '"';
ip = ip_tmp;
print(ip);
if __name__ == "__main__":
main();
And the bash script looks like this:
#!/bin/bash
IP=$("./getIP.py")
curl -X PUT "https://api.cloudflare.com/client/v4/zones/zone_id/dns_records/dns_id" \
-H "X-Auth-Email: example.com" \
-H "X-Auth-Key: authkey" \
-H "Content-Type: application/json" \
--data '{"type":"A","name":"example.com","content":$IP,"ttl":120,"proxied":true}'
I've tried to have the python script only return numbers and then added the quotations in the bash script and now vice versa and I can't seem to get it to work. The last line should end up looking like this once the variable replaces with quotations around the ip address:
'{"type":"A","name":"example.com","content":"127.0.0.1","ttl":120,"proxied":true}'
The single quotes around your json structure prevent the variable from expanding.
You have a few options that are readily available.
Ugly quote escaping inside/around your json.
"{\"type\":\"A\",\"name\":\"example.com\",\"content\":$IP,\"ttl\":120,\"proxied\":true}"
Having the python write this data to a file and telling curl to use that file for the source of the post data.
curl -X PUT "https://api.cloudflare.com/client/v4/zones/zone_id/dns_records/dns_id" \
-H "X-Auth-Email: example.com" \
-H "X-Auth-Key: authkey" \
-H "Content-Type: application/json" \
--data #file_you_wrote_your_json_to.json
Using the python requests or urllib modules to issue the request to cloud flare.
Update your main() function to return the IP instead of print it.
my_ip = main()
url = "https://api.cloudflare.com/client/v4/zones/zone_id/dns_records/dns_id"
myheaders = {
"X-Auth-Email": "example.com",
"X-Auth-Key": "authkey",
"Content-Type": "application/json"
}
myjson = {
"type":"A",
"name":"example.com",
"content":my_ip,
"ttl":120,
"proxied":true
}
requests.put(url, headers=myheaders, data=myjson)
Better yet, just do it in bash. Cloudflare DDNS on github.
One shot to fetch the dynamic A-record ID:
curl -X GET "https://api.cloudflare.com/client/v4/zones/**Zone ID** \
/dns_records?type=A&name=dynamic" \
-H "Host: api.cloudflare.com" \
-H "User-Agent: ddclient/3.9.0" \
-H "Connection: close" \
-H "X-Auth-Email: example#example.com" \
-H "X-Auth-Key: "**Authorization key**" \
-H "Content-Type: application/json"
Cron job (* * * * *) to set the dynamic A-record:
#/usr/bin/env sh
AUTH_EMAIL=example#example.com
AUTH_KEY=** CF Authorization key **
ZONE_ID=** CF Zone ID **
A_RECORD_NAME="dynamic"
A_RECORD_ID=** CF A-record ID from cloudflare-dns-id.sh **
IP_RECORD="/tmp/ip-record"
RECORDED_IP=`cat $IP_RECORD`
PUBLIC_IP=$(curl --silent https://api.ipify.org) || exit 1
if [ "$PUBLIC_IP" = "$RECORDED_IP" ]; then
exit 0
fi
echo $PUBLIC_IP > $IP_RECORD
RECORD=$(cat <<EOF
{ "type": "A",
"name": "$A_RECORD_NAME",
"content": "$PUBLIC_IP",
"ttl": 180,
"proxied": false }
EOF
)
curl "https://api.cloudflare.com/client/v4/zones/$ZONE_ID \
/dns_records/$A_RECORD_ID" \
-X PUT \
-H "Content-Type: application/json" \
-H "X-Auth-Email: $AUTH_EMAIL" \
-H "X-Auth-Key: $AUTH_KEY" \
-d "$RECORD"

How to pass a variable in a curl command in shell scripting

I have a curl command:
curl -u ${USER_ID}:${PASSWORD} -X GET 'http://blah.gso.woo.com:8080/rest/job-execution/job-details/${job_id}'
The variable job_id has a value in it, say, 1160. When I execute the curl command in shell it gives me the following error:
{"message":"Sorry. An unexpected error occured.", "stacktrace":"Bad Request. The request could not be understood by the server due to malformed syntax."}
If I pass the number '1160' directly in the command, as shown below, the curl command works.
curl -u ${USER_ID}:${PASSWORD} -X GET 'http://blah.gso.woo.com:8080/rest/job-execution/job-details/1160'
I want to be able to pass the value of the variable in the curl command.
When using variables in shell, you can only use doubles quotes, not single quotes : the variables inside single quotes are not expanded.
Learn the difference between ' and " and `. See http://mywiki.wooledge.org/Quotes and http://wiki.bash-hackers.org/syntax/words
I ran into this problem with passing as well, it was solved by using ' " $1 " '
See connection.uri below
curl -X POST -H "Content-Type: application/json" --data '
{"name": "mysql-atlas-sink",
"config": {
"connector.class":"com.mongodb.kafka.connect.MongoSinkConnector",
"tasks.max":"1",
"topics":"mysqlstock.Stocks.StockData",
"connection.uri":"'"$1"'",
"database":"Stocks",
"collection":"StockData",
"key.converter":"io.confluent.connect.avro.AvroConverter",
"key.converter.schema.registry.url":"http://schema-registry:8081",
"value.converter":"io.confluent.connect.avro.AvroConverter",
"value.converter.schema.registry.url":"http://schema-registry:8081",
"transforms": "ExtractField",
"transforms.ExtractField.type":"org.apache.kafka.connect.transforms.ExtractField$Value",
"transforms.ExtractField.field":"after"
}}' http://localhost:8083/connectors -w "\n"
How to pass json to curl with shell variable(s):
myvar=foobar
curl -H "Content-Type: application/json" --data #/dev/stdin<<EOF
{ "xkey": "$myvar" }
EOF
With the switch -d or --data, the POST request is implicit
use variable in a double-quote single-quote "' $variable '"
#!/usr/bin/bash
token=xxxxxx
curl --location --request POST 'http://127.0.0.1:8009/submit/expense/' \
--form 'token="'$token'"' \
--form 'text="'$1'"' \
--form 'amount="'$2'"'
userdetails="$username:$apppassword"
base_url_part='https://api.XXX.org/2.0/repositories'
path="/$teamName/$repoName/downloads/$filename"
base_url="$base_url_part$path"**strong text**
curl -L -u "$userdetails" "$base_url" -o "$downloadfilename"

Resources