I am trying to use cURL in the Command Prompt, but I dont understand where I have problems. I have been told that I need to configure a proxy tothe Command Prompt so that it can access the sites I am calling on.
This is what I want to run: curl -glob "api.fda.gov/drug/event.json?&search=receivedate:[20040101+TO+20150101]&limit=1"
I have cURL installed, but always face errors because it is not connecting. Is there a simple way to set up a proxy for/through the Command Prompt in Windows 7?
I also do not have admin rights, so I cannot change the system settings.
You can set your proxy using a set command in windows:
set http_proxy=http://<yourproxyaddress>:<port>
Then you can connect your curl requests to external sites.
Some proxies require specific authentication headers to be set, so be aware of those as well. In my case, it's --proxy-ntlm in the example below:
curl -x webproxy.net:8080 -U usernaname:password http://google.com --proxy-ntlm
But there're other options:
--proxy-digest and --proxy-negotiate
Lastly, cURL has a super friendly doc page, so be sure to check it out.
Related
Is it possible to bash script the setup process of jenkins, for example i have jenkins container set-up on my local machine, i would like to complete jenkins set-up entirely on bash for the purpose of scripting this entirely from a dockerfile.
I need to be able to pass initialAdminPassword without the use of a browser and just from the terminal.
Is it possible to complete the set-up from the terminal?
Yes it is possible to skip the manual setup.
I don't know your particular setup, but let's assume you retrieve the password from jenkins instance :
cat /var/jenkins_home/secrets/initialAdminPassword
or
docker exec "myjenkinscontainer" bash -c 'cat $JENKINS_HOME/secrets/initialAdminPassword'
You could then connect to Jenkins as admin, using that password.
curl --silent -u "admin:$mypassword" http://localhost:8080/manage
If you've configured Jenkins Security to not allow API calls, then you might require to generate first a crumb token that you would use in every request, instead of the password. To issue a crumb token, you might do something similar :
curl -s 'http://admin:$mypassword#localhost:8080/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,\":\",//crumb)'
Then you might need pass this crumb value instead of password in further requests.
Depending on your situation / Jenkins configuration, I might provide more details.
I have a script written in bash and tested working in Linux (CentOS 7) and on MacOS. The script uses cURL to interact with a REST API compliant data platform (XNAT).
I was hoping that Windows users could use the same script within git-bash that comes packaged with Git for Windows. Unfortunately there seems to be an issue when using cURL in git-bash.
The first use I make of cURL is to retrieve a JSESSION cookie:
COOKIE=`curl -k -u $USERNAME https://theaddress/JSESSION`
On Linux, this asks the user for password and stores the cookie in COOKIE.
In git-bash, issuing the command hangs, until using a "ctrl + C" to interrupt it. Strangely at that point the query message for the password is displayed, but too late, the script has terminated.
I have a suspicion that this may have to do with CR or LF issues, but cannot find some info I understand regarding this.
Any pointers would be welcome !
Thank you
EDIT:
It appears the above command works fine if I pass the password in the command like this:
COOKIE=`curl -k -u $USERNAME:$PASSWORD https://theaddress/JSESSION`
However, as pointed here:
Using cURL with a username and password?
I would rather avoid having the user typing their password as a command argument.
So the question is now "why is cURL not prompting for a password when I use the first command?" when in git-bash on Windows, while that command behaves as expected in Linux or MacOS:
COOKIE=`curl -k -u $USERNAME https://theaddress/JSESSION`
Ending up replying to my own question, hope this may be useful to someone else.
It appears this issue is a known problem when running cURL from within git-bash, according to this thread:
https://github.com/curl/curl/issues/573
In particular, see the answer of dscho on 30 Dec 2015:
The problem is the terminal emulator we use with Git Bash since Git for Windows 2.5, MinTTY.
This terminal emulator is not associated with a Win32 Console, therefore the user does not see anything when cURL wants to interact with the user via said Console.
This issue has a workaround, which is documented here:
https://github.com/git-for-windows/build-extra/blob/master/ReleaseNotes.md#known-issues
The workaround is to run curl via winpty as follows:
winpty curl [arguments]
Not an issue with CR or LF after all.
Soooo, git-bash may not be the magic-bullet (tm) to run my bash scripts in Windows with zero effort. Sigh...
I want to use the tool cntlm to authenticate via basic authentication against our corporate proxy server, which uses the ntlm protocol.
CNTLM uses a cntlm.conf file to get your proxy authentication credentials.
If you don't want to use your password in plain text, you can generate hashes with the command. (That's what I want!)
cntlm -u <user> -d <domain> -f -H.
The problem here is, that we use a rolling password policy here and I want to write a bash script for automating the hash generation, inserting in the config file and restarting the cntlm service.
Unfortunately the above shown command generates the password hashes interactively and I couldn't find a way to wrap this in a bash script.
I am a bash scripting newbie though and could use some advice here ;)
For my cntlm version (0.92.3) a simple echo pipe works like charm:
echo "PASSWORD" | cntlm -u USER -d DOMAIN -f -H
I'm new to cURL, just got it installed but it seems to only do what it feels like. I'm using the 64 bit version I got from here: http://curl.haxx.se/latest.cgi?curl=win64-ssl-sspi with installation instructions I found here: http://guides.instructure.com/m/4214/l/83393-how-do-i-install-and-use-curl-on-a-windows-machine. Opening a new Powershell window I'm able to use a simple GET request like so:
curl http://localhost:3000
but if I run a POST
curl -d "hello world" http://localhost:3000
it tells me "Invoke-WebRequest : Parameter cannot be processed because the parameter name 'd' is ambiguous. Possible matches include: -DisableKeepAlive -Debug."
Trying to get help I type
curl -h or curl --help
gives me "Invoke-WebRequest : Missing an argument for parameter 'Headers'. Specify a parameter of type 'System.Collections.IDictionary' and try again."
As I mentioned, I'm a cURL newbie but it seems strange that it can do get requests but nothing else. Any ideas what I'm doing wrong?
Windows 7 64 bit
Powershell version 4
Your problem is that your are not using the Curl you installed but a CmdLet called Invoke-WebRequest.
Just execute :
Remove-item alias:curl
And test your curl again, then store it in your profile.
The explanation is that it exists a native alias to the Invoke-WebRequest which is a CmdLet that is supposed to deliver a kind of curl service.
From Windows 10 build 17063 and later (April 2018), Curl is included into Windows, so that you can execute it directly from Cmd.exe or PowerShell.exe. To use it in PowerShell be careful to unalias this CmdLet or explicitly call curl.exe.
Built with Schannel (Microsoft's native TLS engine), libcurl still perform peer certificate verification, but instead of using a CA cert bundle, it uses the certificates that are built into the OS.
You can execute curl commands with Command Prompt instead of Windows Powershell. Command prompt doesn't alias curl commands like Windows Powershell does.
To open command prompt, hit Win + R, type cmd in the input box, <Enter>.
Is there a way to perform http commands GET/PUT/SET whatever via a command line in ubuntu or windows xp? Preferably without installing 3rd party products. Being that http is text based I thought it would be alot easier to run in the cmd line.
I've been able to get what I want out of GET in ubuntu in bash via
$wget google.com
$cat index.html
This is kinda clunky. It would be nice to pipe the output or something, but even that isn't straight forward. C programs are fine too. I'm trying to do something like what we get with Fiddler, but more basic.
telnet google.com 80
GET / HTTP/1.0
Host: google.com
You have to hit return twice after the Host line. It doesn't get any more basic.
If you are familiar with HTTP use telnet.
If you are looking for a browser take a look for Links.
Although it requires a 3rd party tool, these days I use curl. The -X option allows me to specify the HTTP verb. Windows has a few bash clients that allow you to run curl including Cygwin.
Sample Execution
$ curl -H "Content-Type: application/json" -X POST -d '{value: "600"}' http://localhost:8888/my/endpoint