This question already has answers here:
Expansion of variables inside single quotes in a command in Bash
(8 answers)
Closed 3 years ago.
I'm trying to use two variables in a curl command on my ksh program, but it doesn't work.
Example:
Original URL
curl -s --header "Content-Type:application/json" --header "Accept:application/json" -X POST --data-binary '{"username":"foo","password":"foo_pwd"}' URL site
On my program
user=foo
pwd=foo_pwd
curl -s --header "Content-Type:application/json" --header "Accept:application/json" -X POST --data-binary '{"username":"'"$user"'","password":"'"$pwd"'"}' URL site
I've tried also to escape double quote with backslash but it also doesn't work.
First examine your script
user=foo
pwd=foo_pwd
echo '{"username":"'"$user"'","password":"'"$pwd"'"}'
Then run
user=foo
pwd=foo_pwd
curl -s --header "Content-Type:application/json" --header "Accept:application/json" -X POST https://example.com --data-binary '{"username":"'"$user"'","password":"'"$pwd"'"}'
Related
I am novice and first timer to bash. Trying to run bash under command line to invoke an API, by passing token and payload received from two different APIs and are set as parameters. Below is my command. I am trying to add this bash script to a task in AzureBatch service job.
It has 3 curl requests,
First one (Line#1 in the code snippet below)- gets payload by
calling an API. ---- This is working fine, I am able to verify the
payload using the echo statement following the first curl command.
Second one(Line#3 in the code snippet below) - gets token by
calling the token provider ----- This is working fine as well,
verified using the echo statement.
Third one (Line#5 in the code snippet below),This is the problematic command. I am trying to pass the token and payload received from the above two commands and the curl is not able to resolve them.
both token and payload or not resolving to their values..
My Bash COmmand
/bin/bash -c
"payload=$(curl --location --request GET 'http://url/OutreachData')
&& echo -e \"The value of payload is: "'$payload'"\"
&& token=$(curl --location --request POST 'https://login.microsoftonline.com/<<tenantId>>/oauth2/v2.0/token' --header 'Content-Type: application/x-www-form-urlencoded' --data-urlencode 'client_id=<<clientId>>' --data-urlencode 'scope=api://<<applicationId>>/.default' --data-urlencode 'client_secret=<<clientSecret>>' --data-urlencode 'grant_type=client_credentials' --data-urlencode 'Audience=api://<<applicationId>>'|jq -j '.access_token')
&& echo -e \"value of token is "'$token'"\n\"
&& result=$(curl --location --request POST 'https://url/api/<<Resource>>' --header 'accept: */*' --header 'Content-Type: application/json' --header 'Authorization: Bearer '"'$token'" --data-raw "'$payload'")
&& echo -e \"Result is "'$result'"\""
This is how the third Curl is resolving to, payload and token are not getting replaced as we can see in the authorization header and data-raw elements
++ curl --location --request POST https://url/api/ --header 'accept: /' --header 'Content-Type: application/json' --header 'Authorization: Bearer ' --data-raw ''''''''
There should be no need to explicitly run this with bash -c unless you are in a very constrained environment where you simply cannot run Bash by any other means.
The immediate problem is that code like
bash -c "echo "'$token'" && true"
ends up with $token being single-quoted in the shell which you run bash -c from. But the blazingly obvious fix is to not have this complex quoting in the first place.
payload=$(curl --location --request GET 'http://url/OutreachData')
echo "The value of payload is: '$payload'"
token=$(curl --location --request POST \
'https://login.microsoftonline.com/<<tenantId>>/oauth2/v2.0/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'client_id=<<clientId>>' \
--data-urlencode 'scope=api://<<applicationId>>/.default' \
--data-urlencode 'client_secret=<<clientSecret>>' \
--data-urlencode 'grant_type=client_credentials' \
--data-urlencode 'Audience=api://<<applicationId>>' |
jq -j '.access_token')
echo "value of token is "'$token'"
result=$(curl --location --request POST \
'https://url/api/<<Resource>>' --header 'accept: */*' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer '"$token" \
--data-raw "$payload")
echo "Result is "'$result'"
If your current shell is not Bash and you need these commands to be run in Bash, a simpler workaround is to put the script in a here document, which drastically simplifies the quoting needs (or if this is in an interactive session, just run bash and run these commands at the interactive Bash prompt, then exit when you no longer want to be in Bash).
This question already has answers here:
When to wrap quotes around a shell variable?
(5 answers)
Closed 2 years ago.
I am having issues with sending the proper json data in a curl -X POST command. I have successfully run the POST command locally on my mac by copy and pasting the hardcoded values in but I am now trying to create a .sh script to automate this process. Upon running the code below I get this error:
{"message":"Unexpected end of JSON input"}
Here is the output json from JSON_STRING with names made generic and nothing else changed:
{ "url": "api_url", "tileset": "username.filename" }
Once I can figure out how to properly format the json in the POST command I know it will work, but I can't seem to get the syntax right. Hoping a set of fresh/experience bash eyes will be able to catch my mistake:). Also, all variables that I have are correct and already been confirmed by running variable values in mac terminal. Thanks in advance for the help!
curl_url="http://${bucket}.s3.amazonaws.com/${key}"
echo $curl_url
tileset_id="username.filename"
JSON_STRING=$(jq -n \
--arg bn "$curl_url" \
--arg on "$tileset_id" \
'{url: $bn, tileset: $on}')
echo $JSON_STRING
curl -X POST -H "Content-Type: application/json" -H "Cache-Control: no-cache" -d $JSON_STRING 'apiurl'
Absolutely required to quote the shell variable:
curl ... -d "$JSON_STRING" http://example.com/end/point
Otherwise, the shell will do word splitting, and the argument to -d becomes just {"url":
as a side note, bash arrays can help with the readability:
curl_url="http://${bucket}.s3.amazonaws.com/${key}"
tileset_id="username.filename"
JSON_STRING=$(
jq -n \
--arg bn "$curl_url" \
--arg on "$tileset_id" \
'{url: $bn, tileset: $on}'
)
curl_opts=(
-X POST
-H "Content-Type: application/json"
-H "Cache-Control: no-cache"
-d "$JSON_STRING"
)
curl "${curl_opts[#]}" 'apiurl'
I am using AWX curl for update template in bash script. Somehow this curl command is not able to run
curl --insecure -v -X PATCH https://somedomain.corp.com/api/v2/job_templates/\"$test_2_template\"/ -H 'Authorization: Basic keys==' -H 'Content-Type: application/json' -d '{"extra_vars":"{\"module_name\": \"$module_name\", \"env\": \"$appenv\", \"host_group_name\": \"$host_group_name\", \"rolename\": \"$rolename\", \"app_artifact_url\": \"$app_artifact_url\"}"}'
Looks like there is some issue in passing variables to curl command via bash
However if i run this from my terminal , it's working fine.
curl --insecure -v -X PATCH https://somedomain.com/api/v2/job_templates/279/ -H 'Authorization: Basic keys==' -H 'Content-Type: application/json' -d '{"extra_vars":"{\"module_name\": \"myservice\", \"env\": \"test\", \"host_group_name\": \"env2\", \"rolename\": \"myservice\", \"artifact_version\": \"2.1.0-SNAPSHOT\"}"}'
Please suggest some solution.
You can't substitute variables inside a single quoted string.
For terrible long commands like this, breaking them up into pieces is essential for readability and maintainability:
data=$(printf \
'{"extra_vars": "{\"module_name\": \"%s\", \"env\": \"%s\", \"host_group_name\": \"%s\", \"rolename\": \"%s\", \"app_artifact_url\": \"%s\"}"}' \
"$module_name" "$appenv" "$host_group_name" "$rolename" "$app_artifact_url"
)
curl_args=(
--insecure
-v
-X PATCH
-H 'Authorization: Basic keys=='
-H 'Content-Type: application/json'
-d "$data"
)
url="https://somedomain.example.com/api/v2/job_templates/$test_2_template/"
curl "${curl_args[#]}" "$url"
I am using below code to Post a curl command. But it is not taking any space or line-break as message input.
I tried with %20 and other answers that is already in SO regarding this problem. None is working. It is giving error as
"$error":"Unexpected end-of-input: was expecting closing quote for a string value\n
line="abc def"
curl --user "USER":"Password" -H "Content-Type: application/json" -X POST -d '{"message":"'${line}'"}}' --url http://${host}:${port}${REST_URL}
There is a an unnecessary extra close brace } for your data segment. Also, the variables in the middle of the data argument should be quoted. Also double-quote your --url string to prevent word-splitting by the shell.
curl --user "USER":"Password" \
-H "Content-Type: application/json" \
-X POST -d '{"message":"'"${line}"'"}' \
--url "http://${host}:${port}${REST_URL}"
I'm trying to get files from specific folder, load them with curl and get result to variable:
#!/bin/bash
shopt -s nullglob
FILES=*
for f in $FILES
do
core=$f
#echo "curl -H \"Content-Type: application/json\" --data-binary #$f http://v-cdh-master:8983/solr/$core/update/json?commit=true"
result=$(curl -H "Content-Type: application/json" -s --data-binary #$f http://v-cdh-master:8983/solr/$core/update/json?commit=true)
done
The result of running this script is
curl: no URL specified!
curl: try 'curl --help' or 'curl --manual' for more information
If I remove comment from echo string I could see the valid url like this
curl -H "Content-Type: application/json" --data-binary #affiliates_multival http://v-cdh-master:8983/solr/affiliates_multival/update/json?commit=true
Which works perfect when I copy/paste it to terminal.
Where is my mistake?
The ? in the URL causes the entire argument to be treated as a glob, which is not matching a file. Since you have the nullglob option enabled, the word is removed from the call to curl. Quoting the argument, as you do with the echo command, will resolved the problem by preventing pathname generation.
result=$(curl -H "Content-Type: application/json" -s --data-binary #"$f" "http://v-cdh-master:8983/solr/$core/update/json?commit=true")