This question already has answers here:
How does the leading dollar sign affect single quotes in Bash?
(3 answers)
Closed 2 years ago.
Chrome has given me a curl command that I don't fully recognize. What does the $'{...}' do in the following curl command? I know that the --data-binary is the switch to put that string in the body of the POST, but I'm trying to understand if $'... gets "expanded" or something.
curl 'http://localhost.example.com:8080/graphql' \
-H 'cookie: auth.token_key=zztopTuNQ61YYbHGJpfA00' \
-H 'content-type: application/json' \
--data-binary $'{"query":"query workPlanSearch($searchParameters: WorkPlanSearchParameters\u0021) { workPlanSearch( searchParameters: $searchParameters ) { workPlanId } }","variables":{"searchParameters":{"partNumbers":"000-662-7980-001"}},"operationName":"workPlanSearch"}'
Is it a curl thing or a bash thing?
I believe this is duplicated of How does the leading dollar sign affect single quotes in Bash?
It causes escape sequences to be interpreted. < This is the accepted answer in the link
And yes, I believe this is a bashism, take a look:
cat sh_test
#!/bin/sh
echo $'{"name":"joao\npedro"}'
./sh_test
${"name":"joao
pedro"}
cat bash_test
#!/bin/bash
echo $'{"name":"joao\npedro"}'
$ ./bash_test
{"name":"joao
pedro"}
It is used for disambiguation, in your case it means to pass
'{"query":"query workPlanSearch($searchParameters: WorkPlanSearchParameters\u0021) { workPlanSearch( searchParameters: $searchParameters ) { workPlanId } }","variables":{"searchParameters":{"partNumbers":"000-662-7980-001"}},"operationName":"workPlanSearch"}'
as argument
It can used be used for separating a car from text, for example: ${var}text means the content of var and the string text while $vartext means the content of the variable vartext
Related
This question already has answers here:
When to wrap quotes around a shell variable?
(5 answers)
Closed last year.
I want to send a message to a slack app in a script:
...
declare -a models=("TR10"
"TR100"
"TR1000")
for i in "${models[#]}"
do
#Send Message
if [ $use_slack_app ]
then
message="Starting model $i ($c of $lenght)!"
data=''\''{"text":"'"$message"'"}'''\'
echo $data
curl -X POST -H 'Content-type: application/json' --data $data $slack_app_url
echo "curl -X POST -H 'Content-type: application/json' --data $data $slack_app_url"
fi
#Counter
((c=c+1))
done
...
return of echo $data:
'{"text":"Starting model TR10 (1 of 3)!"}'
curl message:
curl: (3) unmatched close brace/bracket in URL position 5:
3)!"}'
^
curl: (6) Could not resolve host: model
curl: (6) Could not resolve host: TR10
curl: (6) Could not resolve host: (1
curl: (6) Could not resolve host: of
invalid_payload
It looks like it sees the string as several parameters, but I don't understand why. If I echo the line out, and run it in the terminal it works. I don't have much experience with bash scripting, so I don't understand what the problem here is.
Quote the variable expansions to ensure values with spaces are sent as one word: "$var" vs. $var.
Get rid of the single quotes (''\') surrounding $data.
data='{"text":"'"$message"'"}'
curl -X POST -H 'Content-Type: application/json' --data "$data" "$slack_app_url"
I am trying to update a password using curl, see example code below:
# Define the required variables
keystorePassword="Password123"
userPassword="Password321"
user="username"
# Update the password using curl
response=$(curl -s -u "elastic:${keystorePassword}" \
-XPOST "http://localhost:9200/_xpack/security/user/${user}/_password" \
-d'{"password":"'"${userPassword}"'"}' -H "Content-Type: application/json")
echo "Output: $response"
This works OK
But using the below password:
userPassword="Password!/\5"
Returns the error output: "json_parse_exception", "reason":"Unrecognized character escape '5'", "status":400
Is there a way to escape/handle special characters in the userPassword variable?
\ is the escape character. It escapes the character that follows. Since the character that follows is 5, it attempts to escape 5. So, you need to escape \ in order to use it as a character, so use \\5.
I am hitting a wall trying to build a script to save myself quite a good bit of time. I am working in a system in which I need to run a curl POST against a list of values. The list is about 400 lines long, so I am hoping to find a way of scripting this in Bash instead of running that call manually for each entry. Below are some details to help understand what I'm trying to accomplish:
If I were to be doing this task manually, each call would be formatted like the below:
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d #json_payload.json
This points to my JSON in the listed file which shows as the below:
{
"groupId": "12345678987654321",
"type": "serial",
"memberInfo": "apple"
}
If I run the above, the call works, and the expected operation occurs. The issue is that I need to run this against roughly 400 values for that "memberInfo" field in the JSON payload. I'm trying to identify a way to run a single bash script, which will run this curl command over and over, and update the JSON payload to use each row in a file as the below:
memberList.txt
apple
banana
peach
pear
orange
.
.
And then maybe insert a pointer in my JSON for the "memberInfo" field over to this file.
Any and all help/suggestions are greatly appreciated!
.
This will do as you intend. Its a little convoluted but you might polish it a bit.
#!/bin/bash
function getString(){
echo $1 | python3 -c '
import json
import sys
payload="""
{
"groupId": "12345678987654321",
"type": "serial",
"memberInfo": ""
}
"""
obj = json.loads(payload)
obj["memberInfo"] = sys.stdin.read().strip()
print(json.dumps(obj, indent = " "))
'
}
while read member
do
getString "$member" > json_payload.json
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d #json_payload.json
done <<< "$( cat fruits.txt )"
Hope it helps!
while read member; do
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d '{"groupId": "12345678987654321","type": "serial","memberInfo": "$member"}'
done <members.txt
This will work if you only care about the memberInfo field, another method could be writing your json line by line to payloads.txt file.
payloads.txt
{"groupId": "12345678987455432","type": "stereo","memberInfo": "apple"}
{"groupId": "34532453453453465","type": "serial","memberInfo": "banana"}
...
then use this as the script
while read payload; do
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d '$payload'
done <payloads.txt
here is a collection of bash scripting common uses I've had to use
https://github.com/felts94/advanced-bash/blob/master/bash_learn.sh
I'm a bit new to batch scripting, so I apologize if this is glaringly obvious, but I couldn't find any similar information.
I'm trying to perform the following cURL call:
curl -H "Content-Type:application/json" -d '{"lt":"f","sort":"n","max":"1500","offset":"1500"}' [API_KEY]#api.nal.usda.gov/ndb/list
When I run that line in a command line (or Cygwin) it does exactly what I want it to.
However, when I try to call it from a bat file, it seems my parameters are getting messed up somehow.
FOR /L %%A IN (0, 1500, 77500) DO (
curl -H "Content-Type:application/json" -d '{"lt":"f","sort":"n","max":"1500","offset":"%%A"}' [API_KEY]#api.nal.usda.gov/ndb/list > %%A.txt
)
I'm getting output into the correct .txt file, but it doesn't seem like the %%A in offset is getting replaced. I'm getting a "bad parameter" exception from the API. From the output on the command line, it looks accurate.
I'm open to using bash scripting instead if it would make more sense, but I was having the same issue using bash.
(Note: I replaced my API key with a placeholder in the example.. that's not the problem)
In bash at least, the problem is that variable expansion does not occur inside single quotes; you need to use double quotes and escape the nested double quotes:
for a in 0 1500 77500; do
curl -H "Content-Type:application/json" -d "{\"lt\":\"f\",\"sort\":\"n\",\"max\":\"1500\",\"offset\":\"$a\"}" [API_KEY]#api.nal.usda.gov/ndb/list > "$a".txt
)
I suspect you need to do the equivalent in a batch file.
You can concatenate adjacent single- and double-quoted strings to minimized the number of escaped quotes:
... -d '{"lt": "f", "sort": "n", "max": "1500", "offset": "'"$a"'"}' ...
but you may want to consider one of two other options. First, read the data from a here document instead of using a hard-coded string:
curl -H "..." -d#- [API_KEY]#api.nal.usda.gov/ndb/list > "$a".txt <<EOF
{"lt": "f", "sort": "n", "max": "1500", "offset": "$a"}
EOF
or use something like jq to generate the JSON for your:
curl -H "..." \
-d "$(jq --arg a "$a" '{lt: "f", sort: "n", max: "1500", offset: $a}') \
[API_KEY]#api.nal.usda.gov/ndb/list > "$a".txt
The jq solution would be preferable in general, since you don't have to worry about pre-escaping any variable values.
I have a curl command:
curl -u ${USER_ID}:${PASSWORD} -X GET 'http://blah.gso.woo.com:8080/rest/job-execution/job-details/${job_id}'
The variable job_id has a value in it, say, 1160. When I execute the curl command in shell it gives me the following error:
{"message":"Sorry. An unexpected error occured.", "stacktrace":"Bad Request. The request could not be understood by the server due to malformed syntax."}
If I pass the number '1160' directly in the command, as shown below, the curl command works.
curl -u ${USER_ID}:${PASSWORD} -X GET 'http://blah.gso.woo.com:8080/rest/job-execution/job-details/1160'
I want to be able to pass the value of the variable in the curl command.
When using variables in shell, you can only use doubles quotes, not single quotes : the variables inside single quotes are not expanded.
Learn the difference between ' and " and `. See http://mywiki.wooledge.org/Quotes and http://wiki.bash-hackers.org/syntax/words
I ran into this problem with passing as well, it was solved by using ' " $1 " '
See connection.uri below
curl -X POST -H "Content-Type: application/json" --data '
{"name": "mysql-atlas-sink",
"config": {
"connector.class":"com.mongodb.kafka.connect.MongoSinkConnector",
"tasks.max":"1",
"topics":"mysqlstock.Stocks.StockData",
"connection.uri":"'"$1"'",
"database":"Stocks",
"collection":"StockData",
"key.converter":"io.confluent.connect.avro.AvroConverter",
"key.converter.schema.registry.url":"http://schema-registry:8081",
"value.converter":"io.confluent.connect.avro.AvroConverter",
"value.converter.schema.registry.url":"http://schema-registry:8081",
"transforms": "ExtractField",
"transforms.ExtractField.type":"org.apache.kafka.connect.transforms.ExtractField$Value",
"transforms.ExtractField.field":"after"
}}' http://localhost:8083/connectors -w "\n"
How to pass json to curl with shell variable(s):
myvar=foobar
curl -H "Content-Type: application/json" --data #/dev/stdin<<EOF
{ "xkey": "$myvar" }
EOF
With the switch -d or --data, the POST request is implicit
use variable in a double-quote single-quote "' $variable '"
#!/usr/bin/bash
token=xxxxxx
curl --location --request POST 'http://127.0.0.1:8009/submit/expense/' \
--form 'token="'$token'"' \
--form 'text="'$1'"' \
--form 'amount="'$2'"'
userdetails="$username:$apppassword"
base_url_part='https://api.XXX.org/2.0/repositories'
path="/$teamName/$repoName/downloads/$filename"
base_url="$base_url_part$path"**strong text**
curl -L -u "$userdetails" "$base_url" -o "$downloadfilename"