Variable inside double quotes doesn't work as argument [duplicate] - bash

This question already has answers here:
Why does shell ignore quoting characters in arguments passed to it through variables? [duplicate]
(3 answers)
Closed 3 years ago.
I'm trying to pass arguments in a variable to curl and it fails. I turned on set -x to get more information, and I'm really confused about all the single quotes I'm seeing here.
curlopts="-ksS -H 'Content-Type:application/x-www-form-urlencoded'"
$ curl "$curlopts" https://localhost
+ curl '-ksS -H '\''Content-Type:application/x-www-form-urlencoded'\''' https://localhost
curl: option -ksS -H 'Content-Type:application/x-www-form-urlencoded': is unknown
curl: try 'curl --help' or 'curl --manual' for more information
It works without quotes like curl $curlopts https://localhost but I thought it's generally bad practice to use a variable without quotes.

This:
curlopts="-ksS -H 'Content-Type:application/x-www-form-urlencoded'"
sets curlopts to -ksS -H 'Content-Type:application/x-www-form-urlencoded'.
After the above, this:
curl "$curlopts" https://localhost
is equivalent to this:
curl "-ksS -H 'Content-Type:application/x-www-form-urlencoded'" https://localhost
meaning that it calls curl with two arguments: -ksS -H 'Content-Type:application/x-www-form-urlencoded', and https://localhost.
But of course, you want to call curl with four arguments: -ksS, -H, Content-Type:application/x-www-form-urlencoded, and https://localhost.
To achieve that, I recommend using an array:
curlopts=(-ksS -H Content-Type:application/x-www-form-urlencoded)
curl "${curlopts[#]}" https://localhost
where the "${arrayname[#]}" syntax expands, magically, into a separate double-quoted word for each element of the array.
(Note: I've removed the single-quotes because they aren't needed here — none of the characters in Content-Type:application/x-www-form-urlencoded require quoting — but if they make you feel happier, you can safely re-add them.)

Related

Curl request to batch or windows command line [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 10 months ago.
Improve this question
I would like to create a github repository using a command line/batch script or whatever on windows. This request works on https://reqbin.com/curl, but I can't get it to work on windows.
Thanks
curl -i -H "Authorization: token MY_TOKEN"
-d '{
"name": "test",
"auto_init": true,
"private": false
}'
https://api.github.com/user/repos
curl on Windows is weird and doesn't recognize ' as a valid character, so you need double quotes for everything. Unfortunately, the only way to have double quotes inside of double quotes is to escape the inner quotes:
curl -ki -X POST -H "Accept: application/vnd.github.v3+json" -u "username:TOKEN" -d "{\"name\": \"test\", \"auto_init\": true, \"private\": false}" https://api.github.com/user/repos
Replace username with your actual username and TOKEN with your auth token. You also need -X POST in order to create repos, and the header string that I've added is optional, but strongly recommended.
(Note that while the escape character in batch is ^, the escape character that curl looks for is \. Like I said, curl on Windows is weird.) I've also put everything on one line because multi-line commands have varying degrees of success, while a one-liner always works (as long as it's valid, of course).
See https://docs.github.com/en/rest/repos/repos#create-a-repository-for-the-authenticated-user for more options.

Curl using Environment variables in GitLab-ci.yml

I'm trying to send a slack notification using the .gitlab-ci.yml and I need to pass the Commit's message in the message like this:
"The version ${CI_COMMIT_TAG} Version is available!"
But i'm still not able to get the environment variable desired when receiving the notification on my channel and passing it like this in the file:
script:
- "curl -X POST -H 'Content-type: application/json' --data '{\"text\":\"The version ${CI_COMMIT_TAG} version is available!\"} ' https://hooks.slack.com/services/....../......"
Do you have any clues ? I'm not used to Curl and Yaml
Thanks and Have a good day!
--data '...'
Variable expansion in bash does not work within single quotes. Use double quotes instead.
Alternatively, use a data file to avoid formatting JSON inline:
curl -d "#data.json" ...

Escaping curl command in Windows

I'm trying to run a curl command from the command line in Windows, but for the life of me I can't figure out how I'm supposed to escape it.
I'm executing this:
C:\WINDOWS\system32>curl --anyauth --user user:password -X POST -d "{\"rest-api\":{\"name\":\"BizSimDebug3\"}}" -H "Content-type: application/xml" http://localhost:8002/v1/rest-apis
And I'm getting this:
<rapi:error xmlns:rapi="http://marklogic.com/rest-api">
<rapi:status-code>400</rapi:status-code>
<rapi:status>Bad Request</rapi:status>
<rapi:message-code>RESTAPI-INVALIDCONTENT</rapi:message-code>
<rapi:message>Your bootstrap payload caused the server to throw an error. Underlying error message: XDMP-DOCROOTTEXT: xdmp:get-request-body() -- Invalid root text "{&quot;rest-api&quot;:{&quot;name&quot;:&quot;BizSimDebug3&quot;}}" at line 1</rapi:message>
</rapi:error>
Is there something else I need to do to escape the inner quotes in the -d flag? Or am I overlooking the real issue entirely?
This works in Windows:
curl -i -X POST -H "Content-Type: application/json" -d "{\"Field1\": 123, \"Field2\": 456 }" "http://localhost:8080"
The XDMP-DOCROOTTEXT error indicates the server is trying to parse the payload as XML and failing.
The Content-Type header is telling the server that you're sending XML, but the payload is JSON.
Try changing the Content-Type header to application/json
Quoting is hell. By "Windows Command Line and your prompt I presume you mean cmd.com ?. That doest quote the same as linux shells.
For this simplistic experiment I recommend going for 2 kinds of quotes to avoid escaping But even then its unlikely to work
curl --anyauth --user user:password -X POST -d "{'rest-api':{'name':'BizSimDebug3'}}" -H "Content-type: application/xml" http://localhost:8002/v1/rest-apis
Better luck might be had by going with a unix-like shell such as running cygwin (http://www.cygwin.com/) or maybe xmlsh (www.xmlsh.org) which escape like linux does.
You really are going to have a nightmare running anything complex through the windows command line natively.
-David

Sanitizing read entry in Bash

I have a script which prompts for user credentials in order to phrase a curl command. The issue I am having is if the password has a special character it can mess up the curl command.
example code:
curl -k -o ~/Desktop/output.txt
https://$name:$password#'server.example.com/serverpage.asp?action=MakeList&User='$enduser
example inputs
name:test
password:P#55w0rd!
output:
curl: (6) Could not resolve host: 55w0rd!#server.example.com; nodename nor servname provided, or not known
I understand that the curl command is hitting the "#" in the password and trying to connect to 55w0rd!#server.example.com in stead of server.example.com.
How do I "sanitize" the input to escape special characters?
Thank you!
Try to use the "-u" parameter for curl. On the other hand try to use " for start and end the parameters and finally use ${varname} format to access to variables to prevent bash escaping problems.
curl -k -u "${name}:${password}" -o "~/Desktop/output.txt" "https://server.example.com/serverpage.asp?action=MakeList&User=${enduser}"
You want to urlencode your password (Pp%340ssword!). AFAIK there is no simple way to do it in bash, this previous thread has some suggestions involving Perl etc. Testing or looking at the curl source might reveal that just replacing # with %40 is sufficient, I haven't tried for the general case.
curl -k -o ~/Desktop/output.txt "https://${name}:${password}#server.example.com/serverpage.asp?action=MakeList&User=${enduser}"

How to post PUT request under the Windows using curl?

I need to post XML data via curl.exe under windows using PUT request.
In the curl help I found:
-d/--data <data> HTTP POST data (H)
What should I supply for <data>?
curl sample calls
# with inlining plain data
curl -X PUT -d "payload" http://localhost
# referrring file
curl -X PUT -d #myXmlFile.xml http://localhost
If your windows curl-port does not support it go for cygwin. It is a linux-like environment for windows and also offers "a proper" curl.
In windows, if a double-quoted argument itself contains a double quote character, the double quote must be doubled.
For example, enter 'This is "quoted" payload' as "This is ""quoted"" payload" which is very different than in Unix.
Example:
curl -X PUT -d "This is ""quoted"" payload" http://localhost
in windows you'll need to put the # inside the quotes for the file you're sending:
curl -XPUT --data-binary "#uploadme.txt"
otherwise you'll get weird errors as it tries to use the content of the file as the url:
curl: (6) Couldn't resolve host 'upload'
curl: (6) Couldn't resolve host 'me!'
(uploadme.txt contains "upload me!")
on Windows CMD, curl refers to C:\Windows\System32\curl.exe so you can use
curl -X PUT -d "payload" http://localhost
instead on Windows PowerShell curl refers to Invoke-WebRequest so it is not working with curl syntax. you can use curl.exe on PowerShell to call C:\Windows\System32\curl.exe so it will solve the issue.
curl.exe -X PUT -d "payload" http://localhost

Resources