Curl command in Windows - windows

I'm learning feathersjs as per below link
http://feathersjs.com/quick-start/
I need to run below command and monitor the output at http://localhost:3000/todo
curl 'http://localhost:3000/todos/' -H 'Content-Type: application/json' --data-binary '{ "text": "Do something" }'
When I tried to run below on cmd, it shows 'curl' is not recognized in cmd prompt.
If I tried to run it using git-bash.exe, bash.exe or sh.exe (in Git/bin or shell.w32-ix86), Cygwin.bat (in cygwin64), it will run fine and showing result in browser.
[{"text":"Do something","id":0}]
But if tried to run it by including into my PATH "C:\Program Files\Git\mingw64\bin, which has curl.exe", I will be getting below error, but "C:\Program Files\Git\usr\bin" will do just fine...
curl: (1) Protocol "'http" not supported or disabled in libcurl
curl: (6) Couldn't resolve host 'application'
curl: (6) Couldn't resolve host text'
curl: (6) Couldn't resolve host 'Do something'
curl: (3) [globbing] unmatched close brace/bracket in column 1
I managed to remove the error with below command
curl "http://localhost:3000/todos/" -H "Content-Type: 'application/json'" --data-binary "{ 'text': 'Do something' }"
But the resulting output will have the Json object "text" missing...
[{"text":"Do something","id":0},{"id":1}]
Question:
1) After modifying the command, the Json object is not parsed successfully. Is there a syntax problem?
2) If there is no syntax problem, does this means that curl need to be run in Unix environment as per original attempt, that it cant be run in cmd directly, but will function ok in bash, cygwin, etc2?
3) What is the difference between curl.exe in C:\Program Files\Git\usr\bin and C:\Program Files\Git\mingw64\bin, which has curl.exe?
Update:
Not OK Raw Cap output
http.content_type == "'application/json'"
OK Raw Cap output
http.content_type == "application/json"
Update2:
Removing single quote in application/json on the 2nd command... shows error
C:\Users\testUser>"C:\Program Files\Git\mingw64\bin\curl.exe" "http://localhost:3000/todos/" -H "Content-Type: 'application/json'" --data-binary "{ 'text': 'Do something' }"
{"id":1}
C:\Users\testUser>"C:\Program Files\Git\mingw64\bin\curl.exe" "http://localhost:3000/todos/" -H "Content-Type: application/json" --data-binary "{ 'text': 'Do something' }"
SyntaxError: Unexpected token '<br> at Object.parse (native)<br> at parse (C:\Users\testUser\Documents\Framework\Javascript\featherstestNewService\node_modules\body-parser\lib\types\json.js:88:17)<br> at C:\Users\testUser\Documents\Framework\Javascript\featherstestNewService\node_modules\body-parser\lib\read.js:116:18<br> at invokeCallback (C:\Users\testUser\Documents\Framework\Javascript\featherstestNewService\node_modules\body-parser\node_modules\raw-body\index.js:262:16)<br> at done (C:\Users\testUser\Documents\Framework\Javascript\featherstestNewService\node_modules\body-parser\node_modules\raw-body\index.js:251:7)<br> at IncomingMessage.onEnd (C:\Users\testUser\Documents\Framework\Javascript\featherstestNewService\node_modules\body-parser\node_modules\raw-body\index.js:308:7)<br> at emitNone (events.js:67:13)<br> at IncomingMessage.emit (events.js:166:7)<br> at endReadableNT (_stream_readable.js:905:12)<br> at doNTCallback2 (node.js:441:9)
Update 3:
Tried to replace libcurl-4.dll used by curl.exe. Downloaded libcurl from "http://curl.haxx.se/download.html", MingW32 from "http://sourceforge.net/projects/mingw/files/MSYS/" and added "C:\MinGW\bin" to PATH. Then grep -rl "libcurl.dll" . | xargs sed -i 's/libcurl.dll/libcurl-4.dll/g' to create libcurl-4.dll as per suggested "http://curl.haxx.se/mail/lib-2010-11/0174.html". Then execute ./buildconfig, make, make install. Then copied the libcurl-4.dll to C:\Program Files\Git\mingw64\bin folder, but the result is the same...
Update 4
Changing origin of curl.exe but still using same command, shows error on the mingw64 version. I'm suspecting that mingw64 curl need special escaping to make it work?
C:\Users\testUser\Documents\Framework\Javascript\featherstestNewService>"C:\Program Files\Git\mingw64\bin\curl.exe" 'http://localhost:3000/todos/' -H 'Content-Type: application/json' --data-binary '{ "text": "Do something" }'
curl: (1) Protocol "'http" not supported or disabled in libcurl
curl: (6) Couldn't resolve host 'application'
curl: (6) Couldn't resolve host 'text'
curl: (6) Couldn't resolve host 'Do something'
curl: (3) [globbing] unmatched close brace/bracket in column 1
C:\Users\testUser\Documents\Framework\Javascript\featherstestNewService>"C:\Program Files\Git\usr\bin\curl.exe" 'http://localhost:3000/todos/' -H 'Content-Type: application/json' --data-binary '{ "text": "Do something" }'
{"text":"Do something","id":38}
Update 5
From manual... curl --manual
-d, --data <data>
(HTTP) Sends the specified data in a POST request to the HTTP
server, in the same way that a browser does when a user has
filled in an HTML form and presses the submit button. This will
cause curl to pass the data to the server using the content-type
application/x-www-form-urlencoded. Compare to -F, --form.
-d, --data is the same as --data-ascii. --data-raw is almost the
same but does not have a special interpretation of the # charac-
ter. To post data purely binary, you should instead use the
--data-binary option. To URL-encode the value of a form field
you may use --data-urlencode.
If any of these options is used more than once on the same com-
mand line, the data pieces specified will be merged together
with a separating &-symbol. Thus, using '-d name=daniel -d
skill=lousy' would generate a post chunk that looks like
'name=daniel&skill=lousy'.
If you start the data with the letter #, the rest should be a
file name to read the data from, or - if you want curl to read
the data from stdin. Multiple files can also be specified. Post-
ing data from a file named 'foobar' would thus be done with
--data #foobar. When --data is told to read from a file like
that, carriage returns and newlines will be stripped out. If you
don't want the # character to have a special interpretation use
--data-raw instead.
So I tried...
C:\Users\testUser>"C:\Program Files\Git\mingw64\bin\curl.exe" "http://localhost:3000/todos/" -H "'Content-Type:' 'application/json'" --data-binary text=doing --data complete=false
{"text":"doing","complete":"false","id":145}
C:\Users\testUser>"C:\Program Files\Git\mingw64\bin\curl.exe" "http://localhost:3000/todos/" -H "'Content-Type:' 'application/json'" --data-binary text=ding
{"text":"ding","id":146}
But I cant figure out how to make more than 1 word for the JSON object for example instead of "doing", I need "doing something". Seems that MingW64 git curl is accepting different format...

After modifying the curl command it works for you, because you need to use double quote for windows system.
After modifying the command, you have mistakenly added single quote around the application/json. That's why despite of having working curl command server was unsure what you have exactly sent to them!
"Content-Type: 'application/json'"
^ ^ notice the unwanted singles
So it should be
"Content-Type: application/json"
If you do not provide path for any binary (i.e. curl.exe, mysql.exe, php.exe, etc) then system looks for them inside the available paths provided in PATH variable and if they found multiple path there then it will only select one, and I don't know which one!

Related

Get Funding Rate Data From CoinGlass in Windows 10

I am new to using API and not quite sure how to use the CoinGlass API to retrieve data.
According to the documentation, it supports using the curl command
curl --location --request GET 'https://open-api.coinglass.com/api/pro/v1/futures/funding_rates_chart?symbol=BTC&type=C' \
--header 'coinglassSecret: fabe11b61f5a4963a4227f4ac2cbcf2f'
https://coinglass.github.io/API-Reference/#liquidation-chart
However, when I tried to run it in the command prompt in Windows 10, it gave me the below error.
C:\Users\User1>curl --request GET 'https://open-api.coinglass.com/api/pro/v1/futures/funding_rates_chart?symbol=BTC&type=C' --header 'coinglassSecret: fabe11b61f5a4963a4227f4ac2cbcf2f'
curl: (3) URL using bad/illegal format or missing URL
Please kindly point me to right direction, million thanks!
I removed the single quotes around the URL. Executed in Win 10 CMD prompt, and got this
curl --request GET https://open-api.coinglass.com/api/pro/v1/futures/funding_rates_chart?symbol=BTC&type=C --header 'coinglassSecret: fabe11b61f5a4963a4227f4ac2cbcf2f'
{"code":"30001","msg":"secret invalid","success":false}
The system cannot find the file specified.
Error occurred while processing: C.
The system cannot find the file specified.
Error occurred while processing: --header.
The filename, directory name, or volume label syntax is incorrect.

curl error 18 transfer closed with outstanding read data remaining

Setup
I'm Using curl in the following bash script to push a JSON file to a REST API running in tomcat sitting behind nginx.
while IFS= read -d '' -r file; do
base=$(basename "$file")
datetime=$(find $file -maxdepth 0 -printf "%TY/%Tm/%Td %TH:%TM:%.2TS")
curl -vX POST -H "Content-Type: application/json" -H "Cache-Control: no-cache" \
-d #"$file" -u vangeeij:eian12 \
"http://192.168.105.10/homeaccess/services/aCStats/uploadData?username=vangeeij&filename=$base&datetime=$datetime"
#sudo mv "$file" /home/vangeeij/acserver/resultsOld
done < <(sudo find . -type f -print0)
Problem
When running this script I get a http 400 response with curl error:
curl: (18) transfer closed with outstanding read data remaining
What I have tried
I have found 2 things. First running the same URL and body through Postman yields a successful POST.
I found that this error goes away when the last parameter is removed from the URL &datetime=$datetime
I have also found a few connections between this error and setting a curl option something like
curl_setopt($curl, CURLOPT_HTTPHEADER, array('Expect:'));
But I'm not sure where/how to set this exactly when using curl in a simple bash script
Question
What do I need to change in my curl command to get rid of the error and still be able to use all parameters?
UPDATE
Starting a new question, as further investigation has lead me to a better understanding of the problem.
New Question Link
The error has to do with the fact that the parameter datetime= ends up with text in it that needs to be URL encoded.
This was confirmed by replacing the variable with 2017%2F03%2F01%2008%3A50%3A56
and it worked.
So now the problem is, that I can't get --data-urlencode datetime=$datetime to work. It seems this just gets appended to the JSON data or something.
This error is being generated by the fact that the datetime= paramater is being passed in with non encoded non URL friendly characters... (eg. space).
The fix to this would be to find a way to convert the $datetime to a URLEncoded String.
eg. convert:
2017/03/01 08:50:56
TO
2017%2F03%2F01%2008%3A50%3A56
See the following discussion for one method to accomplish this.
Post JSON data to Rest with URLEncoded query paramaters

what does `curl -e` or `curl --referer` mean?

I've seen some example :-
curl --referer http://example.com/bot.html http://www.cyberciti.biz/
what is the different if I use curl http://www.cyberciti.biz/
shell script type:-
curl -e http://example.com/bot.html \
'http://www.cyberciti.biz/'
what is \ for?
From the man page of cURL
-e, --referer <URL>
(HTTP) Sends the "Referrer Page" information to the HTTP server. This
can also be set with the -H, --header flag of course. When used with
-L, --location you can append ";auto" to the --referer URL to make curl automatically set the previous URL when it follows a Location:
header. The ";auto" string can be used alone, even if you don't set an
initial --referer.
Essentially this tells the server which page sent you there.

How to update a TeamCity build parameter using REST+cURL

I have a configuration parameter called "testing" in one of my build configurations in TeamCity. After taking a look at the TeamCity REST API doc here I could get information about this parameter using the following cURL command line commands on Windows:
(1) curl -X GET -H "Authorization: Basic (...)" http://teamcity:8080/httpAuth/app/rest/buildTypes/id:bt7/parameters
(2) curl -X GET -H "Authorization: Basic (...)" http://teamcity:8080/httpAuth/app/rest/buildTypes/id:bt7/parameters/testing
Response:
(1) <?xml version="1.0" encoding="UTF-8" standalone="yes"?><property name="testing" value="11"/></properties>
(2) 11
But then, when I try to update this "testing" build parameter using the following command, I get an error message:
curl -X PUT -d "1" -H "Authorization: Basic (...)" http://teamcity:8080/httpAuth/app/rest/buildTypes/id:bt7/parameters/testing
Response:
Error has occurred during request processing (Unsupported Media Type).
Error: javax.ws.rs.WebApplicationException
Not supported request. Please check URL, HTTP method and transfered data are correct.
I already successfully use a similar command to update the buildNumberCounter setting of the same build configuration:
curl -X PUT -d "1" -H "Authorization: Basic (...)" http://teamcity:8080/httpAuth/app/rest/buildTypes/id:bt7/settings/buildNumberCounter
That's why I thought I can do the same with a build parameter in a similar way. What am I missing here?
UPDATE:
I managed to update the "testing" build parameter with value "1" using Fiddler. The request I composed had the following content:
Request: PUT
URL: http://teamcity:8080/httpAuth/app/rest/buildTypes/id:bt7/parameters/testing
Request headers: Authorization: Basic (...)
Request body: 1
So the problem with my cURL command above is probably somewhere around the -d "1" option. But where?
UPDATE 2:
I'm not sure if that makes any difference, but I use this cURL build on Windows 7.
Instead of fixing the failing cURL command, as a workaround, we use now Node.js to compose and send the REST request to TeamCity.
The script that needs to be fed to node.exe is as follows:
// Equivalent cURL command:
// curl -X PUT -d "1" -H "Authorization: Basic (...)" http://teamcity:8080/httpAuth/app/rest/buildTypes/id:bt7/parameters/testing
var http = require('http'),
options = {
hostname: 'teamcity',
port: 8080,
path: '/httpAuth/app/rest/buildTypes/id:bt7/parameters/testing',
method: 'PUT',
headers: { 'Authorization': 'Basic (...)' }
},
req;
req = http.request(options, function(res) { });
// write data to request body
req.write('1');
req.end();
Although the workaround works perfectly, I would still like to know what's wrong with the above cURL command?
For parameters that are non-XML like the one you are asking about, just add:
--Header "Content-Type: text/plain"
For parameters that are XML then you'll want to switch that to:
--Header "Content-Type: application/xml"
I was having a hard time figuring this out too but I found the answer. Instead of using -d and -H at the front. Use --data and --header at the end as shown below . I found this in the TeamCity docs buried in a "click to expand" example.
Set build number counter:
curl -v --basic --user <username>:<password> --request PUT http://<teamcity.url>/app/rest/buildTypes/<buildTypeLocator>/settings/buildNumberCounter --data <new number> --header "Content-Type: text/plain"
Set build number format:
curl -v --basic --user <username>:<password> --request PUT http://<teamcity.url>/app/rest/buildTypes/<buildTypeLocator>/settings/buildNumberPattern --data <new format> --header "Content-Type: text/plain"
I guess the REST API expect XML as input, add
-H "Content-type:text/xml"
and put XML as input. If you have a XML file file.xml :
curl -d "#/path/to/file.xml" -H "Content-type:text/xml" (...)

Bash curl POST a binary variable

How do you POST a binary variable in curl bash?
#!/usr/bin/env bash
IMAGE=$(curl "http://www.google.com/images/srpr/logo3w.png")
curl --data-binary "$IMAGE" --request "POST" "http://www.somesite.com"
Curl seems to do corrupt the image when uploading.
Curl has the option to write response to disk and then read from it, but it'd be more efficient to do it solely in memory.
Try to eliminate the variable ... as follows:
curl "http://www.google.com/images/srpr/logo3w.png" | curl --data-binary - --request "POST" "http://www.somesite.com"
From the curl man page:
If you start the data with the letter #, the rest should be a file name to read the data from, or - if you want curl to read the data from stdin.
EDIT: From the man page, too:
--raw When used, it disables all internal HTTP decoding of content or transfer encodings and instead makes them passed on unaltered, raw. (Added in 7.16.2)
What happens, if applied on either or both sides?
I had a related problem, where I wanted to dynamically curl a file from a given folder.
curl --data-binary directory/$file --request "POST" "http://www.somesite.com"
did not work - uploaded the string "directory/myFile.jar" instead of the actual file.
Adding the # symbol
curl --data-binary #directory/$file --request "POST" "http://www.somesite.com" fixed it.

Resources