Pass Associative Array as Data Param in POST request using cURL - bash

I have an associative array that I want to pass to cURL as POST data. However i have tried multiple things, still it doesn't work.
The array:
declare -A details
details[name]="Honey"
details[class]="10"
details[section]="A"
details[subject]="maths"
The cURL commands have tried so far (all of these failed):
resp = $(cURL --request POST --data details "https://somedomain.net/getMarks")
resp = $(cURL --request POST --data variables=details "https://somedomain.net/getMarks")
resp = $(cURL --request POST --data "variables=$details" "https://somedomain.net/getMarks")
resp = $(cURL --request POST --data "variables=${details}" "https://somedomain.net/getMarks")
resp = $(cURL --request POST --data $details "https://somedomain.net/getMarks")
resp = $(cURL --request POST --data ${details} "https://somedomain.net/getMarks")
resp = $(cURL --request POST --data variables=details "https://somedomain.net/getMarks")
Something like shown below, I want the above request to be (indirectly), however I want to pass the array directly instead of writing its contents.
resp = $(cURL --request POST --data '{"variables":[{"name": "Honey"},{"class": "10"},{"section": "A"},{"subject": "maths"}]}' "https://somedomain.net/getMarks")
Please note that to begin with I will always have the associative array ONLY (not any json array or string).
This question rose when I was trying calling cURL command with the associative array as on this link (GITLAB API)(the example does not contain variables array example). Here they have mentioned a variables array (array of hashes).

Since I had to use an older version of bash, which does not involve
the name referencing as stated on the answer, I had to try to
code string creation of the associative array without passing it to a function
Since I always had an associative array to begin with, the process of passing the array as accepted by the gitlab API normally was:
resp=$(cURL --request POST --data '{"variables":[{"name": "Honey"},{"class": "10"},{"section": "A"},{"subject": "maths"}]}' "https://somedomain.net/getMarks")
OR
resp=$(cURL --request POST --data "variables[name]=Honey" --data "variables[class]=10" --data "variables[section]=A" --data "variables[subject]=maths" "https://somedomain.net/getMarks")
So tried some tweaks on the second way and what worked for me was:
_sep=""
_string=""
for index in "${!details[#]}"
do
_string="${_string}${_sep}variables[${index}]="${details[$index]}"
_sep="&"
done
resp=$(cURL --request POST --data "$_string" "https://somedomain.net/getMarks")
#which indirectly was:
resp=$(cURL --request POST --data "variables[name]=Honey&variables[class]=10&variables[section]=A&variables[subject]=maths" "https://somedomain.net/getMarks")
And it was a success. Thanks to #markp-fuso for giving me an intuition of creating a string with his logic above.

Assumptions/understandings:
no need to list the array entries in any particular order
neither the array indices or values contain newlines
One bash idea:
# use a function to build the --data component
build_data() {
local -n _arr="$1" # use nameref so we can pass in the name of any associative array
local _sep=""
local _string='{"variables":['
local _i
for i in "${!_arr[#]}"
do
_string="${_string}${_sep}{\"${i}\": \"${_arr[$i]}\"}"
_sep=","
done
printf "%s]}" "${_string}"
}
Adding this to the curl call:
resp=$(cURL --request POST --data "$(build_data details)" "https://somedomain.net/getMarks")
NOTES:
no spaces allowed on either side of the =, ie, resp = $(curl ...) needs to be resp=$(curl ...)
without an actual/valid URL I'm guessing a bit on if/where the escaped quotes belong so may need to tweak the escaped quotes to get working correcly

Related

Escape quotes and brackets in Jenkins Groovy script -> shell -> curl

I need to make API call with payload (everything needs to be exactly like that)
{"file": "//'HLQ.DATASET(MEMBER)'"}
in Jenkins pipeline. I can't figure correct escaping of the payload. Problem is with the round brackets, single escape - Groovy complains, double escape - one of the slashes bubbles all the way into curl call.
def String job = """{\\"file\\": \\"\\'HLQ.DATASET\\(MEMBER\\)\\'\\"}"""
...
script {
def String response = sh(script: " curl -X PUT -w %{http_code} -v --header 'Content-Type: application/json' --cookie cookies.txt --header 'X-CSRF-ZOSMF-HEADER: dummy' --header 'X-IBM-Notification-URL: ${hook.getURL()}' https://.../zosmf/restjobs/jobs --data '$job'", returnStdout: true).trim()
}
If you are checking the Jenkins console output to determine whether the message is correctly sent it would mislead you. What you see in the console output is not always the interpreted string.
Can you try something like the below? Also inorder to check what Curl is sending out you can use a flag like --trace
def job = "{\"file\": \"//'HLQ.DATASET(MEMBER)'\"}"
writeFile(file: 'payload.txt', text: job)
sh 'cat payload.txt'
def String response = sh(script: "curl -X PUT -w %{http_code} -v --header 'Content-Type: application/json' --cookie cookies.txt --header 'X-CSRF-ZOSMF-HEADER: dummy' --header 'X-IBM-Notification-URL: ${hook.getURL()}' https://.../zosmf/restjobs/jobs --data #payload.txt", returnStdout: true).trim()

Not able to replace the value of the variable inside expression in bash script

I am trying to run a bash script, where I would like to make POST calls in a for loop as follows:
for depId in "${depIds[#]}"
do
echo "$depId" <--------------------------------- THIS IS PRINTING PROPER VALUE
curl 'https://student.service.com/api/student' \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-H 'Cookie: UISESSION=abcd' \
--data-raw '{"name":"Student Name","description":"Dummy","depId":$depId}' \ <---- HERE I CANNOT GET THE VALUE OF THE VARIABLE
--compressed
echo "$content"
done
As mentioned above, I cannot get the value of the department id in the URL, with the above form, I am getting a Request Malformed exception. I have even tried with ${depId}, but no luck.
Could anyone please help here ?
Try flipping your quotes around the variable.
--data-raw '{"name":"Student Name","description":"Dummy","depId":'"$depId"'}' \

Running Curl POST w/ JSON Payload Sequentially Against File

I am hitting a wall trying to build a script to save myself quite a good bit of time. I am working in a system in which I need to run a curl POST against a list of values. The list is about 400 lines long, so I am hoping to find a way of scripting this in Bash instead of running that call manually for each entry. Below are some details to help understand what I'm trying to accomplish:
If I were to be doing this task manually, each call would be formatted like the below:
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d #json_payload.json
This points to my JSON in the listed file which shows as the below:
{
"groupId": "12345678987654321",
"type": "serial",
"memberInfo": "apple"
}
If I run the above, the call works, and the expected operation occurs. The issue is that I need to run this against roughly 400 values for that "memberInfo" field in the JSON payload. I'm trying to identify a way to run a single bash script, which will run this curl command over and over, and update the JSON payload to use each row in a file as the below:
memberList.txt
apple
banana
peach
pear
orange
.
.
And then maybe insert a pointer in my JSON for the "memberInfo" field over to this file.
Any and all help/suggestions are greatly appreciated!
.
This will do as you intend. Its a little convoluted but you might polish it a bit.
#!/bin/bash
function getString(){
echo $1 | python3 -c '
import json
import sys
payload="""
{
"groupId": "12345678987654321",
"type": "serial",
"memberInfo": ""
}
"""
obj = json.loads(payload)
obj["memberInfo"] = sys.stdin.read().strip()
print(json.dumps(obj, indent = " "))
'
}
while read member
do
getString "$member" > json_payload.json
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d #json_payload.json
done <<< "$( cat fruits.txt )"
Hope it helps!
while read member; do
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d '{"groupId": "12345678987654321","type": "serial","memberInfo": "$member"}'
done <members.txt
This will work if you only care about the memberInfo field, another method could be writing your json line by line to payloads.txt file.
payloads.txt
{"groupId": "12345678987455432","type": "stereo","memberInfo": "apple"}
{"groupId": "34532453453453465","type": "serial","memberInfo": "banana"}
...
then use this as the script
while read payload; do
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d '$payload'
done <payloads.txt
here is a collection of bash scripting common uses I've had to use
https://github.com/felts94/advanced-bash/blob/master/bash_learn.sh

Can I define an object as a variable in a shell script?

I know how to store a string as a variable, for example: API="http://localhost:4741"
However, for the sake of a CURL request I would like to be able to store on object as a variable that I can access values on, something like OBJ="{name : Joe}". Is this possible?
Right now my CURL request looks like this:
curl --include --request POST localhost:3000/scrape \
--header "Content-Type: application/json" \
--data '{
"url": "http://www.oddsshark.com/stats/gamelog/basketball/nba/20736",
"team": "LA Clippers"
}'
I would like to be able to do something like this, using a dictionary or an object:
TEAM=( ["Clippers"]="http://www.oddsshark.com/stats/gamelog/basketball/nba/20736" )
curl --include --request POST localhost:3000/scrape \
--header "Content-Type: application/json" \
--data '{
"url": "http://www.oddsshark.com/stats/gamelog/basketball/nba/20736",
"team": "${TEAM[Clippers]}"
}'

Calculate OAuth signature for NetSuite restlet using bash

I'm currently trying to generate an OAuth signature for my curl request header. These point to a NetSuite restlet. Resources online are either inconclusive or too high level for my understanding/lacking examples. How do I go about calculating the oauth_signature value for my request?
The following is my request with credentials ommitted:
curl --request GET \
--url 'https://rest.na1.netsuite.com/app/site/hosting/restlet.nl?script=foo&deploy=bar' \
--header 'Authorization: OAuth realm="'"$realm"'",oauth_consumer_key="'"$oauth_consumer_key"'",oauth_token="'"$oauth_token"'",oauth_signature_method="HMAC-SHA1",oauth_timestamp="'"$(OAuth_timestamp)"'",oauth_nonce="'"$(OAuth_nonce)"'",oauth_version="1.0",oauth_signature="'"$(OAuth_signature)"'"' \
--header 'cache-control: no-cache' \
--header 'content-type: application/json' \
| jq
Below is a list of the parameters I'm passing for the sake of readability:
params=(
oauth_consumer_key='foo'
oauth_signature_method='HMAC-SHA1'
oauth_version='1.0'
oauth_nonce=$(OAuth_nonce)
oauth_timestamp=$(OAuth_timestamp)
oauth_token='tokenfoo'
realm='4478811'
)
I am generating the timestamp and nonce like so:
OAuth_nonce () {
md5 <<< "$RANDOM-$(date +%s.%N)" | cut -d' ' -f 1
}
OAuth_timestamp () {
echo "$(date +%s)"
}
I got most of my resources from https://github.com/livibetter-backup/bash-oauth but no docs exist, the examples are poor, and the library itself doesn't seem to work when I've tested the functions.
All the values I use in the script (confirmed passing with bash +x) work when ran in Postman, but I can't calculate a oauth_signature value outside of it.
How do I create a OAuth_signature function that I can return a valid signature with? What parameters am I going to have to pass that function to calculate correctly? Is it possible or easier to generate perhaps using perl or python?

Resources