Why does curl see several parameters? [duplicate] - bash

This question already has answers here:
When to wrap quotes around a shell variable?
(5 answers)
Closed last year.
I want to send a message to a slack app in a script:
...
declare -a models=("TR10"
"TR100"
"TR1000")
for i in "${models[#]}"
do
#Send Message
if [ $use_slack_app ]
then
message="Starting model $i ($c of $lenght)!"
data=''\''{"text":"'"$message"'"}'''\'
echo $data
curl -X POST -H 'Content-type: application/json' --data $data $slack_app_url
echo "curl -X POST -H 'Content-type: application/json' --data $data $slack_app_url"
fi
#Counter
((c=c+1))
done
...
return of echo $data:
'{"text":"Starting model TR10 (1 of 3)!"}'
curl message:
curl: (3) unmatched close brace/bracket in URL position 5:
3)!"}'
^
curl: (6) Could not resolve host: model
curl: (6) Could not resolve host: TR10
curl: (6) Could not resolve host: (1
curl: (6) Could not resolve host: of
invalid_payload
It looks like it sees the string as several parameters, but I don't understand why. If I echo the line out, and run it in the terminal it works. I don't have much experience with bash scripting, so I don't understand what the problem here is.

Quote the variable expansions to ensure values with spaces are sent as one word: "$var" vs. $var.
Get rid of the single quotes (''\') surrounding $data.
data='{"text":"'"$message"'"}'
curl -X POST -H 'Content-Type: application/json' --data "$data" "$slack_app_url"

Related

Send data to API containing special characters

I am trying to send JSON data containing a mac address to an api using this command:
$value={ "pcModel": "KAT12", "displayType": "DELL U2311H", "graphicsType": "Microsoft Remote Display Adapter", "displayServiceTag": "HV8XP08Q079L", "ipAddress": "172.16.4.194", "recDate": "2022-10-06 16:57:55", "serviceTag": "18LQ9X1;Diskwear:(4.91TBW ; 15393 Hours)", "wolMode": "lanwithpxeboot;CC:101010-0118ZH;os:Ubuntu", "sleepState": "disable", "macAddress": "90:B1:1C:8E:D5:11", "hostName": "CI-KR95-05", "diskMode": "raid", "diskType": "Samsung SSD 850 PRO 512GB;TBW+Hrs:(4.91TB;15393 HrH) ;Clock:3.4GHz;Max Clock:3.67GHz(108%);RAM:32GB" }
curl -X "PUT" -H "Accept: application/json" -H "Authorization: Token 62d85330rf562c4cd6c1fb1a64d1" -d "$value" --url "https://my_api.org/api/inventory/84:2b:2b:a0:0s2:18
but I get the following answer:
user#ubuntu:~$ curl -X "PUT" -H "Accept: application/json" -H "Authorization: Token 62d85df90210cd1a827bc1518c4cd6c1fb1a64d1" -d "$value" --url "https:/my_api/api/inventory/84:2b:2b:a0:0s2:18"
curl: (3) URL using bad/illegal format or missing URL
I tried to escape the ":" colon characters with \ like this
curl -X "PUT" -H "Accept: application/json" -H "Authorization: Token 62d85330rf562c4cd6c1fb1a64d1" -d "$value" --url "https://my_api.org/api/inventory/84\:2b\:2b\:a0\:0s\:18"
but I get no output and it sends nothing.
any Idea how to send this data without having the bad format error?
Thanks a lot
I removed the "?" from $value and added single quotes.
Try this:
#!/bin/bash
value='{ "pcModel": "KAT12", "displayType": "DELL U2311H", "graphicsType": "Microsoft Remote Display Adapter", "displayServiceTag": "HV8XP08Q079L", "ipAddress": "172.16.4.194", "recDate": "2022-10-06 16:57:55", "serviceTag": "18LQ9X1;Diskwear:(4.91TBW ; 15393 Hours)", "wolMode": "lanwithpxeboot;CC:101010-0118ZH;os:Ubuntu", "sleepState": "disable", "macAddress": "90:B1:1C:8E:D5:11", "hostName": "CI-KR95-05", "diskMode": "raid", "diskType": "Samsung SSD 850 PRO 512GB;TBW+Hrs:(4.91TB;15393 HrH) ;Clock:3.4GHz;Max Clock:3.67GHz(108%);RAM:32GB" }'
curl -X "PUT" -H "Accept: application/json" -H "Authorization: Token 62d85330rf562c4cd6c1fb1a64d1" -d "$value" --url "https://my_api.org/api/inventory/84:2b:2b:a0:0s2:18"
Didn't your mother teach you how to format?
value='{
"pcModel": "KAT12",
"displayType": "DELL U2311H",
"graphicsType": "Microsoft Remote Display Adapter",
"displayServiceTag": "HV8XP08Q079L",
"ipAddress": "172.16.4.194",
"recDate": "2022-10-06 16:57:55",
"serviceTag": "18LQ9X1;Diskwear:(4.91TBW ; 15393 Hours)",
"wolMode": "lanwithpxeboot;CC:101010-0118ZH;os:Ubuntu",
"sleepState": "disable",
"macAddress": "90:B1:1C:8E:D5:11",
"hostName": "CI-KR95-05",
"diskMode": "raid",
"diskType": "Samsung SSD 850 PRO 512GB;TBW+Hrs:(4.91TB;15393 HrH) ;Clock:3.4GHz;Max Clock:3.67GHz(108%);RAM:32GB"
}'
$value is not a legal variable name in bash for declaration, only when you want to look it up. use value=
Also, you are missing a " after the URL.
$ curl "http:/cake.com/nom-nom
> "
curl: (3) URL using bad/illegal format or missing URL
The protocol schema is followed by colon-slash-slash ://. so you need one more:
https://
Maybe you want to read the URL rfc:
https://www.ietf.org/rfc/rfc2718.txt
2.1.2 Improper use of "//" following ":"
Contrary to some examples set in past years, the use of double
slashes as the first component of the of a URL
is not simply an artistic indicator that what follows is a URL:
Double slashes are used ONLY when the syntax of the URL's contains a hierarchical structure as described in RFC
2396. In URLs from such schemes, the use of double slashes indicates that what follows is the top hierarchical element for a
naming authority. (See section 3 of RFC 2396 for more details.)
URL schemes which do not contain a conformant hierarchical
structure in their should not use double
slashes following the ":" string.
Thanks for your answers.
I found the reasons why my code was not updating the API
1 - I needed to specify the "Content-Type: "application/json" parameter to tell the API that the data sent hat JSON format otherwise I saw in verbose mode that the data has a "x-www-form-urlencoded" format that the API can't understand.
2 - The data sent with curl must accept a value with the format like this
value='{
"pcModel": "KAT12",
"displayType": "DELL U2311H",
"diskType": "Samsung SSD .."
}'
3 - The curl request must be sent with double quotes around the url and the data ""
So the end request looks like this:
curl -k -X "PUT" -H "Content-Type: application/json"\
-H "Accept: application/json" -H "Authorization: Token 62d85df902101828g84kc4cd6c1fb1a64d1" \
--url "https://my_api.org/api/inventory/84:2b:2b:a0:0s2:18" \
-d "$value"

Not able to replace the value of the variable inside expression in bash script

I am trying to run a bash script, where I would like to make POST calls in a for loop as follows:
for depId in "${depIds[#]}"
do
echo "$depId" <--------------------------------- THIS IS PRINTING PROPER VALUE
curl 'https://student.service.com/api/student' \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-H 'Cookie: UISESSION=abcd' \
--data-raw '{"name":"Student Name","description":"Dummy","depId":$depId}' \ <---- HERE I CANNOT GET THE VALUE OF THE VARIABLE
--compressed
echo "$content"
done
As mentioned above, I cannot get the value of the department id in the URL, with the above form, I am getting a Request Malformed exception. I have even tried with ${depId}, but no luck.
Could anyone please help here ?
Try flipping your quotes around the variable.
--data-raw '{"name":"Student Name","description":"Dummy","depId":'"$depId"'}' \

adding snapshot to elasticsearch Windows

I have two snapshots that I want to insert into elasticsearch in path:
C:\Users\name\Downloads\book_backup\agg_example
C:\Users\name\Downloads\book_backup\search_example
which I properly listed in elasticsearch.yml
path.repo: ["C:\\Users\\olulo\\Downloads\\book_backup\\agg_example", "C:\\Users\\olulo\\Downloads\\book_backup\\search_example"]
my elasticsearch starts fine and creating a new index works too.
Now when I try to insert snapshot into my elasticsearch so I can work on it:
curl -XPUT "http://localhost:9200/movies/" -d '{"type":"fs", "settings":{"location":"C:\\Users\\name\\Downloads\\book_backup\\search_example", "compress":true}}'
It gives me:
curl: (1) Protocol "'http" not supported or disabled in libcurl
curl: (3) [globbing] unmatched brace in column 10
curl: (3) [globbing] unmatched close brace/bracket in column 14
In https://www.elastic.co/guide/en/elasticsearch/reference/5.2/modules-snapshots.html it seems like they've added a name at the end of path in location so I did
curl -XPUT "http://localhost:9200/movies/" -d '{"type":"fs", "settings":{"location":"C:\\Users\\name\\Downloads\\book_backup\\search_example\\test", "compress":true}}'
but still gives me same error.
following https://superuser.com/questions/1322567/http-not-supported-or-disabled-in-libcurl I've change everything to double quotes:
curl -XPUT "http://localhost:9200/_snapshot/javacafe" -d "{"type":"fs", "settings":{"location":"C:\\Users\\olulo\\Downloads\\book_backup\\search_example", "compress":true}}"
giving me :
{"error":"Content-Type header [application/x-www-form-urlencoded] is not supported","status":406}
Tried adding curl option as suggested Content-Type header [application/x-www-form-urlencoded] is not supported on Elasticsearch by
curl -XPUT "localhost:9200/_snapshot/javacafe" -H 'Content-Type: application/json' -d "{
"type":"fs",
"settings"{"location":"C:\\Users\\olulo\\Downloads\\book_backup\\search_example\\test", "compress":true}
}"
which outputs similar error with added statement at the end:
{"error":"Content-Type header [application/x-www-form-urlencoded] is not supported","status":406}curl: (6) Could not resolve host: application
SOLVED : for windows you need to use backslash infront of double quotation mark inside {}:
curl -XPUT "localhost:9200/_snapshot/javacafe" -H "Content-Type: application/json" -d "{
\"type\":\"fs\",
\"settings\"{\"location\":"C:\\Users\\olulo\\Downloads\\book_backup\\search_example\\test", \"compress\":true}
}"
From my understanding windows use \ to consider " as it is. If so why not add backslash to all curl commands as well?
Try this:
curl -XPUT "localhost:9200/_snapshot/javacafe" -H 'Content-Type: application/json' -d '{
"type":"fs",
"settings" : {
"location":"C:\\Users\\olulo\\Downloads\\book_backup\\search_example\\test",
"compress":true
}
}'

how to pass arguments in curl command

i am getting the below error while executing the curl command in unix. Not sure if this is the right way to put the arguments in curl. Can you please advise on this?
curl -X POST -H Authorization:'Bearer AAEAAJ6ZNfGbzJkSuJ1o5rXLBec5Q' -H 'Content-Type: application/json' -d {"Filter": {"ClientName": "ABC","WorkflowName": "sk_lask"},"SortingName": "StartDate","SortingOrder": "Desc"} http://someaddress.com/api/status/search HTTP/1.1
error
curl: (3) [globbing] unmatched brace at pos 13
curl: (6) Could not resolve host: ABC,WorkflowName; Unknown error
curl: (3) [globbing] unmatched close brace/bracket at pos 9
curl: (6) Could not resolve host: StartDate,SortingOrder; Unknown error
curl: (3) [globbing] unmatched close brace/bracket at pos 5
You need to quote your header, payload and also remove HTTP/1.1 from end as this command:
curl -X POST -H 'Authorization: Bearer AAEAAJ6ZNfGbzJkSuJ1o5rXLBec5Q' \
-H 'Content-Type: application/json' \
-d '{"Filter": {"ClientName": "ABC","WorkflowName": "sk_lask"},"SortingName": "StartDate","SortingOrder": "Desc"}' \
'http://someaddress.com/api/status/search'

How to pass a variable in a curl command in shell scripting

I have a curl command:
curl -u ${USER_ID}:${PASSWORD} -X GET 'http://blah.gso.woo.com:8080/rest/job-execution/job-details/${job_id}'
The variable job_id has a value in it, say, 1160. When I execute the curl command in shell it gives me the following error:
{"message":"Sorry. An unexpected error occured.", "stacktrace":"Bad Request. The request could not be understood by the server due to malformed syntax."}
If I pass the number '1160' directly in the command, as shown below, the curl command works.
curl -u ${USER_ID}:${PASSWORD} -X GET 'http://blah.gso.woo.com:8080/rest/job-execution/job-details/1160'
I want to be able to pass the value of the variable in the curl command.
When using variables in shell, you can only use doubles quotes, not single quotes : the variables inside single quotes are not expanded.
Learn the difference between ' and " and `. See http://mywiki.wooledge.org/Quotes and http://wiki.bash-hackers.org/syntax/words
I ran into this problem with passing as well, it was solved by using ' " $1 " '
See connection.uri below
curl -X POST -H "Content-Type: application/json" --data '
{"name": "mysql-atlas-sink",
"config": {
"connector.class":"com.mongodb.kafka.connect.MongoSinkConnector",
"tasks.max":"1",
"topics":"mysqlstock.Stocks.StockData",
"connection.uri":"'"$1"'",
"database":"Stocks",
"collection":"StockData",
"key.converter":"io.confluent.connect.avro.AvroConverter",
"key.converter.schema.registry.url":"http://schema-registry:8081",
"value.converter":"io.confluent.connect.avro.AvroConverter",
"value.converter.schema.registry.url":"http://schema-registry:8081",
"transforms": "ExtractField",
"transforms.ExtractField.type":"org.apache.kafka.connect.transforms.ExtractField$Value",
"transforms.ExtractField.field":"after"
}}' http://localhost:8083/connectors -w "\n"
How to pass json to curl with shell variable(s):
myvar=foobar
curl -H "Content-Type: application/json" --data #/dev/stdin<<EOF
{ "xkey": "$myvar" }
EOF
With the switch -d or --data, the POST request is implicit
use variable in a double-quote single-quote "' $variable '"
#!/usr/bin/bash
token=xxxxxx
curl --location --request POST 'http://127.0.0.1:8009/submit/expense/' \
--form 'token="'$token'"' \
--form 'text="'$1'"' \
--form 'amount="'$2'"'
userdetails="$username:$apppassword"
base_url_part='https://api.XXX.org/2.0/repositories'
path="/$teamName/$repoName/downloads/$filename"
base_url="$base_url_part$path"**strong text**
curl -L -u "$userdetails" "$base_url" -o "$downloadfilename"

Resources