bash encapsulate command options in variables - bash

I have a bunch of these to test my RESTful API
$CURL \
-v \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-X POST \
-d '{"user":{"email":"user#example.com","password":"secret"}}' \
$URL/$PATH/sessions
I kinda want to shorten it to something like
CURLOPTS="-v -H 'Content-Type: application/json' -H 'Accept: application/json'"
$CURL \
$CURLOPTS \
-X POST \
-d '{"user":{"email":"user#example.com","password":"secret"}}' \
$URL/$PATH/sessions
but the options don't seem to be passed in. Any clues?

Short answer: see BashFAQ #50:'m trying to put a command in a variable, but the complex cases always fail!.
Long answer: Putting commands (or parts of commands) into variables and then getting them back out intact is complicated. The reason your script doesn't work is because of the order in which the shell parses the command line: it parses (and removes) quotes and escapes, then replaces variable values. By the time $CURLOPTS gets replaced, it's too late for the quotes to have their intended effect; instead, they're passed to curl as part of the arguments, which confuses curl greatly.
The solution: store the options in an array rather than a plain string:
CURLOPTS=(-v -H 'Content-Type: application/json' -H 'Accept: application/json')
$CURL \
"${CURLOPTS[#]}" \
-X POST \
-d '{"user":{"email":"user#example.com","password":"secret"}}' \
"$URL/$PATH/sessions"

You can use an array and trigger word splitting
$ set -x
$ CURLOPTS=(-v -H 'Content-Type: application/json' -H 'Accept: application/json')
$ : curl "${CURLOPTS[#]}"
+ : curl -v -H 'Content-Type: application/json' -H 'Accept: application/json'

Related

Trying to allow dynamic bash var for the prompt but escape char not working any ideas?

Trying to allow dynamic bash var for the prompt but escape char not working any ideas?
output=$(curl https://api.openai.com/v1/images/generations \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
"prompt": "$prompt",
"n": 1,
"size": "1024x1024"
}'
Tried escaping char but get error please help me allow bash vars in curl request.
The single quotes around the '-d' (--data) values are preventing the expansion of the variable.
Note that we are escaping the other double quotes in the json so they show up when the command is run. I assume you wanted them there, hence the single quotes.
You can do something along these lines (I like using the long version of options to show what the arguments are doing in a script).
$ prompt="A cute baby sea otter"
$ OPENAI_API_KEY=S0MeCo0lk3y
$ output=$(curl https://api.openai.com/v1/images/generations \
--header "Content-Type:application/json" \
--header "Authorization: Bearer ${OPENAI_API_KEY}" \
--data "{ \"prompt\":\"${prompt}\",\"n\": 1,\"size\": \"1024x1024\"}")
That sends the following:
curl https://api.openai.com/v1/images/generations --header Content-Type:application/json --header 'Authorization: Bearer S0MeCo0lk3y' --data '{ "prompt":"A cute baby sea otter","n": 1,"size": "1024x1024"}'
Checkout https://mywiki.wooledge.org/BashFAQ/050 as it might help with what you are trying to do.

How to use Bash command line to curl an API with token and payload as parameters

I am novice and first timer to bash. Trying to run bash under command line to invoke an API, by passing token and payload received from two different APIs and are set as parameters. Below is my command. I am trying to add this bash script to a task in AzureBatch service job.
It has 3 curl requests,
First one (Line#1 in the code snippet below)- gets payload by
calling an API. ---- This is working fine, I am able to verify the
payload using the echo statement following the first curl command.
Second one(Line#3 in the code snippet below) - gets token by
calling the token provider ----- This is working fine as well,
verified using the echo statement.
Third one (Line#5 in the code snippet below),This is the problematic command. I am trying to pass the token and payload received from the above two commands and the curl is not able to resolve them.
both token and payload or not resolving to their values..
My Bash COmmand
/bin/bash -c
"payload=$(curl --location --request GET 'http://url/OutreachData')
&& echo -e \"The value of payload is: "'$payload'"\"
&& token=$(curl --location --request POST 'https://login.microsoftonline.com/<<tenantId>>/oauth2/v2.0/token' --header 'Content-Type: application/x-www-form-urlencoded' --data-urlencode 'client_id=<<clientId>>' --data-urlencode 'scope=api://<<applicationId>>/.default' --data-urlencode 'client_secret=<<clientSecret>>' --data-urlencode 'grant_type=client_credentials' --data-urlencode 'Audience=api://<<applicationId>>'|jq -j '.access_token')
&& echo -e \"value of token is "'$token'"\n\"
&& result=$(curl --location --request POST 'https://url/api/<<Resource>>' --header 'accept: */*' --header 'Content-Type: application/json' --header 'Authorization: Bearer '"'$token'" --data-raw "'$payload'")
&& echo -e \"Result is "'$result'"\""
This is how the third Curl is resolving to, payload and token are not getting replaced as we can see in the authorization header and data-raw elements
++ curl --location --request POST https://url/api/ --header 'accept: /' --header 'Content-Type: application/json' --header 'Authorization: Bearer ' --data-raw ''''''''
There should be no need to explicitly run this with bash -c unless you are in a very constrained environment where you simply cannot run Bash by any other means.
The immediate problem is that code like
bash -c "echo "'$token'" && true"
ends up with $token being single-quoted in the shell which you run bash -c from. But the blazingly obvious fix is to not have this complex quoting in the first place.
payload=$(curl --location --request GET 'http://url/OutreachData')
echo "The value of payload is: '$payload'"
token=$(curl --location --request POST \
'https://login.microsoftonline.com/<<tenantId>>/oauth2/v2.0/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'client_id=<<clientId>>' \
--data-urlencode 'scope=api://<<applicationId>>/.default' \
--data-urlencode 'client_secret=<<clientSecret>>' \
--data-urlencode 'grant_type=client_credentials' \
--data-urlencode 'Audience=api://<<applicationId>>' |
jq -j '.access_token')
echo "value of token is "'$token'"
result=$(curl --location --request POST \
'https://url/api/<<Resource>>' --header 'accept: */*' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer '"$token" \
--data-raw "$payload")
echo "Result is "'$result'"
If your current shell is not Bash and you need these commands to be run in Bash, a simpler workaround is to put the script in a here document, which drastically simplifies the quoting needs (or if this is in an interactive session, just run bash and run these commands at the interactive Bash prompt, then exit when you no longer want to be in Bash).

Executing curl command with variables in bash

I am using AWX curl for update template in bash script. Somehow this curl command is not able to run
curl --insecure -v -X PATCH https://somedomain.corp.com/api/v2/job_templates/\"$test_2_template\"/ -H 'Authorization: Basic keys==' -H 'Content-Type: application/json' -d '{"extra_vars":"{\"module_name\": \"$module_name\", \"env\": \"$appenv\", \"host_group_name\": \"$host_group_name\", \"rolename\": \"$rolename\", \"app_artifact_url\": \"$app_artifact_url\"}"}'
Looks like there is some issue in passing variables to curl command via bash
However if i run this from my terminal , it's working fine.
curl --insecure -v -X PATCH https://somedomain.com/api/v2/job_templates/279/ -H 'Authorization: Basic keys==' -H 'Content-Type: application/json' -d '{"extra_vars":"{\"module_name\": \"myservice\", \"env\": \"test\", \"host_group_name\": \"env2\", \"rolename\": \"myservice\", \"artifact_version\": \"2.1.0-SNAPSHOT\"}"}'
Please suggest some solution.
You can't substitute variables inside a single quoted string.
For terrible long commands like this, breaking them up into pieces is essential for readability and maintainability:
data=$(printf \
'{"extra_vars": "{\"module_name\": \"%s\", \"env\": \"%s\", \"host_group_name\": \"%s\", \"rolename\": \"%s\", \"app_artifact_url\": \"%s\"}"}' \
"$module_name" "$appenv" "$host_group_name" "$rolename" "$app_artifact_url"
)
curl_args=(
--insecure
-v
-X PATCH
-H 'Authorization: Basic keys=='
-H 'Content-Type: application/json'
-d "$data"
)
url="https://somedomain.example.com/api/v2/job_templates/$test_2_template/"
curl "${curl_args[#]}" "$url"

curl command has ! character results auth failure

I am getting authentication failure error while using curl command in bash shell which password has ! char.
Below is curl command i am trying.
curl -D- -u 'some_user:somename#2019!' -X POST --data #data.txt -H 'Content-Type: application/json' https://help.myjira.com/rest/api/2/issue
I have also tried escaping ! char using \, as below, but no luck.
curl -D- -u 'some_user:somename#2019''\!' -X POST --data #data.txt -H 'Content-Type: application/json' https://help.myjira.com/rest/api/2/issue
can any one suggest.
You can try with entity code for ! = %21 (and # = %40). So:
curl -D- -u 'some_user:somename%402019%21' -X POST --data #data.txt -H 'Content-Type: application/json' https://help.myjira.com/rest/api/2/issue

Bash: Curl multiple lines, write output to file

I have a simple bash script that call cURL with several params.
I need to write the output in a file (also overwriting). But I cannot do.
The call itself works, but I have only an empty file (and the answer is a json, I can read on the shell with that echo)
Thank you in advance for your help
1st try
curl -X POST "https://www.example.com"\
-H "X-Auth-Email: $email"\
-H "X-Auth-Key: $auth_key"\
-H "Content-Type: application/json"\
--data '{"name":"'$name'","surname":"'$surname'"}'
>> id.txt
echo
2nd try
curl -X POST "https://www.example.com"\
-H "X-Auth-Email: $email"\
-H "X-Auth-Key: $auth_key"\
-H "Content-Type: application/json"\
--data '{"name":"'$name'","surname":"'$surname'"}'
o id.txt
echo
curl -X POST "https://www.example.com"\
-H "X-Auth-Email: $email"\
-H "X-Auth-Key: $auth_key"\
-H "Content-Type: application/json"\
--data '{"name":"'$name'","surname":"'$surname'"}'\
-o id.txt

Resources