This question already has answers here:
When to wrap quotes around a shell variable?
(5 answers)
Closed 2 years ago.
I am having issues with sending the proper json data in a curl -X POST command. I have successfully run the POST command locally on my mac by copy and pasting the hardcoded values in but I am now trying to create a .sh script to automate this process. Upon running the code below I get this error:
{"message":"Unexpected end of JSON input"}
Here is the output json from JSON_STRING with names made generic and nothing else changed:
{ "url": "api_url", "tileset": "username.filename" }
Once I can figure out how to properly format the json in the POST command I know it will work, but I can't seem to get the syntax right. Hoping a set of fresh/experience bash eyes will be able to catch my mistake:). Also, all variables that I have are correct and already been confirmed by running variable values in mac terminal. Thanks in advance for the help!
curl_url="http://${bucket}.s3.amazonaws.com/${key}"
echo $curl_url
tileset_id="username.filename"
JSON_STRING=$(jq -n \
--arg bn "$curl_url" \
--arg on "$tileset_id" \
'{url: $bn, tileset: $on}')
echo $JSON_STRING
curl -X POST -H "Content-Type: application/json" -H "Cache-Control: no-cache" -d $JSON_STRING 'apiurl'
Absolutely required to quote the shell variable:
curl ... -d "$JSON_STRING" http://example.com/end/point
Otherwise, the shell will do word splitting, and the argument to -d becomes just {"url":
as a side note, bash arrays can help with the readability:
curl_url="http://${bucket}.s3.amazonaws.com/${key}"
tileset_id="username.filename"
JSON_STRING=$(
jq -n \
--arg bn "$curl_url" \
--arg on "$tileset_id" \
'{url: $bn, tileset: $on}'
)
curl_opts=(
-X POST
-H "Content-Type: application/json"
-H "Cache-Control: no-cache"
-d "$JSON_STRING"
)
curl "${curl_opts[#]}" 'apiurl'
Related
I am very new to bash scripting, please can someone point me in the right direction on how to accomplish my task?
I have a curl call which returns a string, which I want to convert to json.
My curl statement -
curl --insecure -X POST 'https://url/api/IPAM/GetIP' --header 'Content-Type: application/json' --header -d '{"key1": "value1"}'
This curl statement returns a string ,for example: 10.100.100.100
I want to fetch this string and return the output in json format:
{"IP":"10.100.100.100"}
I don't want to use jquery or python to do this because this entire script will be run by a wrapper that only understands bash.
You can use jq to process your IP string into a JSON string and package it into a JSON object of your choice.
ip="10.100.100.100"
jq --arg ip "$ip" -cn '{"IP":$ip}'
Result:
{"IP":"10.100.100.100"}
Now if working with the result of your example curl POST request:
rawip_string=$(curl --insecure -X POST 'https://url/api/IPAM/GetIP' --header 'Content-Type: application/json' --header -d '{"key1": "value1"}')
jq --arg ip "$rawip_string" -cn '{"IP":$ip}'
One way to not rely on external tools like jq, you can get the output ip attribute that to a variable and concatenate it.
$ return=$(echo 10.100.100.100)
$ echo "{\"IP\":\"${return}\"}"
{"IP":"10.100.100.100"}
Like this
printf '{"IP":"%s"}' "$(curl --insecure -X POST 'https://url/api/IPAM/GetIP' --header 'Content-Type: application/json' --header -d '{"key1": "value1"}')"
I have the following bash script that uses its arguments to hit a RESTful web service (via curl) and prints out both the curl request made as well as the response:
#! /bin/bash
# arguments:
# $1 - username
# $2 - password
#
# outputs:
# if the script exits with a non-zero status then something went wrong
# verify that we have all 6 required arguments and fail otherwise
if [ "$#" -ne 2 ]; then
echo "Required arguments not provided"
exit 1
fi
# set the script arguments to meaningful variable names
username=$1
password=$2
# login and fetch a valid auth token
req='curl -k -i -H "Content-Type: application/json" -X POST -d ''{"username":"$username","password":"$password"}'' https://somerepo.example.com/flimflam'
resp=$(curl -k -i -H "Content-Type: application/json" -X POST -d ''{"username":"$username","password":"$password"}'' https://somerepo.example.com/flimflam)
# echo the request for troubleshooting
echo "req = $req"
if [ -z "$resp" ]; then
echo "Login failed; unable to parse response"
exit 1
fi
echo "resp = $resp"
When I run this I get:
$ sh myscript.sh myUser 12345#45678
curl: (3) Port number ended with '"'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: 12345#45678"
100 1107 100 1093 100 14 2849 36 --:--:-- --:--:-- --:--:-- 2849
req = curl -k -i -H "Content-Type: application/json" -X POST -d {"username":"$username","password":"$password"} https://somerepo.example.com/flimflam
resp = HTTP/1.1 400 Bad Request...(rest omitted for brevity)
Obviously, I'm not escaping the various layers of single- and double-quotes inside the curl statement correctly, as is indicated by outputs like:
curl: (6) Could not resolve host: 12345#45678"
and:
req = curl -k -i -H "Content-Type: application/json" -X POST -d {"username":"$username","password":"$password"} https://somerepo.example.com/flimflam
where the username/password variables are not parsing.
In reality my script takes a lot more than 2 arguments, which is why I'm changing them to have meaningful variable names (such as $username instead of $1) so its more understandable and readable.
Can anyone spot where I'm going awry? Thanks in advance!
Update
I tried the suggestion which turns the req into:
curl -k -i -H 'Content-Type: application/json' -X POST -d "{'username':'myUser','password':'12345#45678'}" https://somerepo.example.com/flimflam
However this is still an illegal curl command and instead needs to be:
curl -k -i -H 'Content-Type: application/json' -X POST -d '{"username":"myUser","password":"12345#45678"}' https://somerepo.example.com/flimflam
First, as I said in a comment, storing commands in variables just doesn't work right. Variables are for data, not executable code. Second, you have two levels of quoting here: quotes that're part of the shell syntax (which are parsed, applied, and removed by the shell before the arguments are passed to `curl), and quotes that're part of the JSON syntax.
But the second problem is actually worse than that, because simply embedding an arbitrary string into some JSON may result in JSON syntax errors if the string contains characters that're part of JSON syntax. Which passwords are likely to do. To get the password (and username for that matter) embedded correctly in your JSON, use a tool that understands JSON syntax, like jq:
userinfo=$(jq -n -c --arg u "$username" --arg p "$password" '{"username":$u,"password":$p}')
Explanation: this uses --arg to set the jq variables u and p to the shell variables $username and $password respectively (and the double-quotes around the shell variables will keep the shell from doing anything silly to the values), and creates a JSON snippet with them embedded. jq will automatically add appropriate quoting/escaping/whatever is needed.
Then, to use it with curl, use something like this:
resp=$(curl -k -i -H "Content-Type: application/json" -X POST -d "$userinfo" https://somerepo.example.com/flimflam)
Again, the double-quotes around $userinfo keep the shell from doing anything silly. You should almost always put double-quotes around variables references in the shell.
Note that I never used the req variable to store the command. If you need to print the command (or its equivalent), use something like this:
printf '%q ' curl -k -i -H "Content-Type: application/json" -X POST -d "$userinfo" https://somerepo.example.com/flimflam
echo
The %q format specifier tells the shell to add appropriate quoting/escaping so that you could run the result as a shell command, and it'd work properly. (And the echo is there because printf doesn't automatically add a newline at the end of its output.)
try changing this:
req='curl -k -i -H "Content-Type: application/json" -X POST -d ''{"username":"$username","password":"$password"}'' https://somerepo.example.com/flimflam'
to this
req="curl -k -i -H 'Content-Type: application/json' -X POST -d \"{'username':'$username','password':'$password'}\" https://somerepo.example.com/flimflam"
and similarly for the resp
ah those pesky "curly" thingies...
how 'bout...
req="curl -k -i -H 'Content-Type: application/json' -X POST -d '{\"username\":\"$username\",\"password\":\"$password\"}' https://somerepo.example.com/flimflam"
This needs even more escaping:
With:
resp=$(curl -k -i -H "Content-Type: application/json" -X POST -d "{\"username\":\"$username\",\"password\":\"$password\"}" https://somerepo.example.com/flimflam)
In bash, the variables are still expanded when they're inside single quotes that are inside double quotes.
And you'll need the \" double quotes in the payload as per the JSON definition.
EDIT: I rerun the curl through a HTTP proxy and corrected the script line (see above, removed the single quotes). Results (in raw HTTP) are now:
POST /flimflam HTTP/1.1
Host: somerepo.example.com
User-Agent: curl/7.68.0
Accept: */*
Content-Type: application/json
Content-Length: 44
Connection: close
{"username":"user","password":"12345#abcde"}
(which should be fine)
This question already has answers here:
Expansion of variables inside single quotes in a command in Bash
(8 answers)
Closed 3 years ago.
I'm trying to use two variables in a curl command on my ksh program, but it doesn't work.
Example:
Original URL
curl -s --header "Content-Type:application/json" --header "Accept:application/json" -X POST --data-binary '{"username":"foo","password":"foo_pwd"}' URL site
On my program
user=foo
pwd=foo_pwd
curl -s --header "Content-Type:application/json" --header "Accept:application/json" -X POST --data-binary '{"username":"'"$user"'","password":"'"$pwd"'"}' URL site
I've tried also to escape double quote with backslash but it also doesn't work.
First examine your script
user=foo
pwd=foo_pwd
echo '{"username":"'"$user"'","password":"'"$pwd"'"}'
Then run
user=foo
pwd=foo_pwd
curl -s --header "Content-Type:application/json" --header "Accept:application/json" -X POST https://example.com --data-binary '{"username":"'"$user"'","password":"'"$pwd"'"}'
I am using below code to Post a curl command. But it is not taking any space or line-break as message input.
I tried with %20 and other answers that is already in SO regarding this problem. None is working. It is giving error as
"$error":"Unexpected end-of-input: was expecting closing quote for a string value\n
line="abc def"
curl --user "USER":"Password" -H "Content-Type: application/json" -X POST -d '{"message":"'${line}'"}}' --url http://${host}:${port}${REST_URL}
There is a an unnecessary extra close brace } for your data segment. Also, the variables in the middle of the data argument should be quoted. Also double-quote your --url string to prevent word-splitting by the shell.
curl --user "USER":"Password" \
-H "Content-Type: application/json" \
-X POST -d '{"message":"'"${line}"'"}' \
--url "http://${host}:${port}${REST_URL}"
I'm trying to send a curl command in Automator via the 'Run Shell Script', with arguments, but having no luck. I'm using /bin/bash and passing info as arguments. Here is my script but keep getting Bad Request from IFTTT. I get it's to do with not using the args correctly (if I just put "value1":"test" it works fine), how should I format the $1?
for f
do
curl -X POST -H "Content-Type: application/json" -d '{"value1":$1}' https://maker.ifttt.com/trigger/Automator/with/key/heremykey
done
Thanks!
You should pass a valid JSON. There is no built-in JSON support in Bash, so you need to use external tools, such as PHP, or Node:
#!/bin/bash -
function json_encode {
printf "$1" | php -r 'echo json_encode(stream_get_contents(STDIN));'
}
for f
do
value=`json_encode "$f"`
curl -X POST -H "Content-Type: application/json" -d "{\"value1\":$value}" \
https://maker.ifttt.com/trigger/Automator/with/key/heremykey
done
The script is supposed to send {"value1": ...} string for each item in $#(because the short version of the for loop operates on $#).