I have a script written in bash and tested working in Linux (CentOS 7) and on MacOS. The script uses cURL to interact with a REST API compliant data platform (XNAT).
I was hoping that Windows users could use the same script within git-bash that comes packaged with Git for Windows. Unfortunately there seems to be an issue when using cURL in git-bash.
The first use I make of cURL is to retrieve a JSESSION cookie:
COOKIE=`curl -k -u $USERNAME https://theaddress/JSESSION`
On Linux, this asks the user for password and stores the cookie in COOKIE.
In git-bash, issuing the command hangs, until using a "ctrl + C" to interrupt it. Strangely at that point the query message for the password is displayed, but too late, the script has terminated.
I have a suspicion that this may have to do with CR or LF issues, but cannot find some info I understand regarding this.
Any pointers would be welcome !
Thank you
EDIT:
It appears the above command works fine if I pass the password in the command like this:
COOKIE=`curl -k -u $USERNAME:$PASSWORD https://theaddress/JSESSION`
However, as pointed here:
Using cURL with a username and password?
I would rather avoid having the user typing their password as a command argument.
So the question is now "why is cURL not prompting for a password when I use the first command?" when in git-bash on Windows, while that command behaves as expected in Linux or MacOS:
COOKIE=`curl -k -u $USERNAME https://theaddress/JSESSION`
Ending up replying to my own question, hope this may be useful to someone else.
It appears this issue is a known problem when running cURL from within git-bash, according to this thread:
https://github.com/curl/curl/issues/573
In particular, see the answer of dscho on 30 Dec 2015:
The problem is the terminal emulator we use with Git Bash since Git for Windows 2.5, MinTTY.
This terminal emulator is not associated with a Win32 Console, therefore the user does not see anything when cURL wants to interact with the user via said Console.
This issue has a workaround, which is documented here:
https://github.com/git-for-windows/build-extra/blob/master/ReleaseNotes.md#known-issues
The workaround is to run curl via winpty as follows:
winpty curl [arguments]
Not an issue with CR or LF after all.
Soooo, git-bash may not be the magic-bullet (tm) to run my bash scripts in Windows with zero effort. Sigh...
Related
Edit: As of 01/31/2023 the scripts that I am using below ARE working. Any patterns of inconsistencies that I find I will report here. Would like to leave this open in case others have findings/advice they are interested in sharing in relation to bash scripting/LastPass CLI/WSL
I am looking to use the LastPass CLI to make some changes to Shared Sites within our LastPass enterprise. I was able to write the scripts (fortunately with some help from others on here), however I am unable to get the commands to work properly within a script.
One of the commands that I WAS having troubles with was lpass share create. This command worked directly from the command line, but I was unable to run this command within a script successfully. I have a very simple script, similar to the one below:
#!/bin/bash
folderpath=$1
lpassCreateStoreFolder(){
lpass share create "$folderpath"
}
lpassLogin(){
echo 'testPWD' | LPASS_DISABLE_PINENTRY=1 lpass login --trust --force tester#test.com
}
lpassLogin
lpassCreateStoreFolder
I've been invoking my script through the PowerShell command line like so:
wsl "path/to/script" "Shared-00 Test LastPass CLI"
Sometimes this command works within the script and other times it does not. When I tried running this script around mid December, I had no success at all. The script would run through all the way, the CLI would even give me a response
Folder Shared-00 Test LastPass CLI created.
and the LastPass Admin Console logs show me a report of "Create Shared Folder". The problem is when I go to my LastPass Vault, the Shared Folder was rarely/if ever created. Running the command without a script, directly from the command line worked almost 100% of the time. I initially chalked this up to inconsistencies on their end, but now I am experiencing these same problems with a different command.
Similarly I have been using the lpass edit command to make edits to sites within our LastPass vault. Once again, I have a relatively simple script to make the edit to the site:
#!/bin/bash
lpassId=$1
lpassSetNotes(){
printf "Notes:\n What are your notes?\nThese are my notes" | lpass edit --non-interactive --sync=now "$lpassId"
}
lpassLogin(){
echo 'testPWD' | LPASS_DISABLE_PINENTRY=1 lpass login --trust --force test#test.com
}
lpassLogin
lpassSetNotes
and have been invoking this script through Powershell like so:
wsl "path/to/script" "000LastPassID000"
like the lpass share create command, running the script does not produce the desired output. The script runs all the way through and my changes are reflected in the logs, but when I go to the vault the site itself is never changed. The command DOES however work when I run it from the command line directly within WSL.
I am relatively new to writing Bash scripts/the Linux operating system, so I'm not entirely sure if this something wrong on my end or just the vendor's tool that I am utilizing producing inconsistencies. Any help would be appreciated, I know this issue might be hard to replicate without a LastPass account
Example LastPass CLI calls that work directly from command line in WSL
lpass share create "Shared-00 Testing LastPass CLI"
printf "Notes:\n What are your notes?\nThese are my notes" | lpass edit --non-interactive --sync=now "$lpassId"
References
LastPass CLI
CLI Manual
CLI GitHub
I'm trying to connect to my server via SSH and issue some commands to it. For some reason it seems like the commands are getting cut "off".
Here is the code that does the putty connection as well as issuing the SSH commands:
./plink.exe ${USER}#${HOSTNAME} -pw ${PASS}<<SSH
cd /some/foo/bar
deploy_artifact.sh --instance development1 some_artifact.ear
APP_development1.sh restart
exit
SSH
For me it works, but on the machine of my colleague the issued SSH commands are getting cut off and thus are not interpreted correctly. For example deploy_artifact.sh is getting turned into ploy_artifact.sh (See the following the screenshot).
How can i prevent this? And what is causing this?
Thanks in advance for any help!
It appears the problem is with the plink and how it is used. The given example sends commands as a standard input, however I did not find in the plink manual any mention that it reads commands from the STDIN. It is better to avoid undocumented features, since they may not work correctly or the author may remove them without any notice. Instead, if you want to pass commands inline you should provide them as an argument, ie you either have to use a quoted text, or you can wrap heredoc in the "$(cat *heredoc* )" code, eg:
./plink.exe ${USER}#${HOSTNAME} -pw ${PASS} "$(cat <<SSH
cd /some/foo/bar
deploy_artifact.sh --instance development1 some_artifact.ear
APP_development1.sh restart
exit
SSH
)"
Or, you can keep the commands in a file and run the plink with the -m commands_file option.
Having read this question and my answer there, I would like to do a similar thing on Windows.
My Linux solution is this:
#!/bin/bash
[[ $1 =~ password: ]] && cat || SSH_ASKPASS="$0" DISPLAY=nothing:0 exec setsid "$#"
How can I do a similar thing on Windows, something I can use like this from a Windows Command Prompt or batch file:
C:> echo password | pass ssh user#host ...
Points to note:
ssh here was installed using the free edition of crwsync. It uses Cygwin DLLs but does not require a Cygwin install.
the solution should not require further dependencies: it work from a typical Windows Command Prompt or batch file.
I'm looking for an answer to the above, even if the answer is "it can't be done". I know I can use keys (and their relative merits), or other tools such as Python/Paramiko, PuTTY plink, and so-on. I know I can do it in a Cygwin environment. I don't want to do those things... I need to do it from a plain old Windows command prompt or batch file without incurring additional dependencies because, if this is possible, it will reduce existing dependencies.
Here is what I have so far:
#echo off
echo.%1 | findstr /C:"password">nul
if errorlevel 1 (
set SSH_ASKPASS="%0"
set DISPLAY="nothing:0"
%*
) else (
findstr "^"
)
The idea is to save that as, say pass.bat and use it like this:
C:> echo password | pass.bat ssh user#host ...
What happens is that the SSH session is launched but ssh still interactively prompts for the password. I think that, in theory, the script is ok becuse the below works:
C:> echo mypassword | pass.bat pass.bat "password"
mypassword
As far as I understand, the underlying Cygwin DLLs should see the Windows environment so the setting of SSH_ASKPASS should propagate into ssh.
I think the problem is that ssh is connected to the terminal. According to man ssh, If ssh needs a passphrase, it will read the passphrase from the current terminal if it was run from a terminal. This is why I use setsid in the Linux example. I think a way to detach the process from the terminal in Windows is required but I am not sure there is one (I did try start /B).
So I'm stuck - I don't know enough about scripting windows to know what should work. Any solution that uses native windows techniques (i.e. batch or perhaps powershell) and does not require anything not available on a vanilla Windows would be welcome.
The solution will be used by a cross platform application that I am working on that needs to use SSH to interact with an external service. The current prototype version is Python and is aready wired up to launch ssh as a subprocess. The Linux version already uses the above method so I would like a Windows solution that does not require reworking of the application.
SSH will never read password from stdin. I would give a shot sshpass utility, which is quite standard for this task. The other common solution is using expect script (which should work the same way on the Cygwin as on Linux).
I'm new to cURL, just got it installed but it seems to only do what it feels like. I'm using the 64 bit version I got from here: http://curl.haxx.se/latest.cgi?curl=win64-ssl-sspi with installation instructions I found here: http://guides.instructure.com/m/4214/l/83393-how-do-i-install-and-use-curl-on-a-windows-machine. Opening a new Powershell window I'm able to use a simple GET request like so:
curl http://localhost:3000
but if I run a POST
curl -d "hello world" http://localhost:3000
it tells me "Invoke-WebRequest : Parameter cannot be processed because the parameter name 'd' is ambiguous. Possible matches include: -DisableKeepAlive -Debug."
Trying to get help I type
curl -h or curl --help
gives me "Invoke-WebRequest : Missing an argument for parameter 'Headers'. Specify a parameter of type 'System.Collections.IDictionary' and try again."
As I mentioned, I'm a cURL newbie but it seems strange that it can do get requests but nothing else. Any ideas what I'm doing wrong?
Windows 7 64 bit
Powershell version 4
Your problem is that your are not using the Curl you installed but a CmdLet called Invoke-WebRequest.
Just execute :
Remove-item alias:curl
And test your curl again, then store it in your profile.
The explanation is that it exists a native alias to the Invoke-WebRequest which is a CmdLet that is supposed to deliver a kind of curl service.
From Windows 10 build 17063 and later (April 2018), Curl is included into Windows, so that you can execute it directly from Cmd.exe or PowerShell.exe. To use it in PowerShell be careful to unalias this CmdLet or explicitly call curl.exe.
Built with Schannel (Microsoft's native TLS engine), libcurl still perform peer certificate verification, but instead of using a CA cert bundle, it uses the certificates that are built into the OS.
You can execute curl commands with Command Prompt instead of Windows Powershell. Command prompt doesn't alias curl commands like Windows Powershell does.
To open command prompt, hit Win + R, type cmd in the input box, <Enter>.
I have installed msysgit: Git-1.7.8-preview20111206.exe from http://code.google.com/p/msysgit/downloads/list (Edit: I have also installed msysGit-fullinstall-1.7.8-preview20111206.exe, which conveniently installs to a different directory, and I am seeing the same problems with that instance).
And I am having a problem with ssh-agent, where ssh-add always reports: "Could not open a connection to your authentication agent."
Unfortunately, ssh-add does not give me any further details on the error it encountered. It feels like it is timing out when it tries to connect to a socket, but it does not actually admit that this is the issue, and ssh-agent acts like it is running normally. Thinking about this combination makes me suspect a permissions issue, but I have no idea where I would look to investigate that kind of problem (and I do no see anything that I recognize as significant in windows event log).
This error message baffles me: I have reason to believe that I could get this error message if my environmental variables were not set up properly, but I have reason to believe that I have my environmental variables set properly. I have reason to believe that I could get this error message if my environmental variables refer to an instance of ssh-agent which is not running, but I have exactly one instance of ssh-agent running and it matches what I see in the environmental variables. And this is repeatable. I have started (and eventually shut down) dozens of instances of ssh-agent, using techniques like ssh-agent > ~/.ssh/environment; . ~/.ssh/environment, and eval `ssh-agent` and ssh-agent bash and so on... and I cannot get msys ssh-add to connect to msys ssh-agent.
bash-3.1$ ps | grep ssh-agent
10304 1 10304 10304 ? 500 09:01:24 /bin/ssh-agent
bash-3.1$ bash -c 'set | grep SSH_A'
BASH_EXECUTION_STRING='set | grep SSH_A'
SSH_AGENT_PID=10304
SSH_AUTH_SOCK=/tmp/ssh-oAFwa11048/agent.11048
bash-3.1$ time ssh-add -L
Could not open a connection to your authentication agent.
real 0m10.730s
user 0m0.000s
sys 0m0.015s
bash-3.1$ wc /tmp/ssh-oAFwa11048/agent.11048
0 3 52 /tmp/ssh-oAFwa11048/agent.11048
bash-3.1$ cat /tmp/ssh-oAFwa11048/agent.11048; echo
!<socket >59261 060A4541-9831B739-519220DE-57936738
bash-3.1$
I am not concerned with scripting anything, just yet -- I believe that if I can get ssh-add working from the command line that I can script it...
And note also that the 10 seconds of real time that ssh-add uses happens before it issues the "Could not open..." message.
And, I am using a current version of windows 7, here. And, I also have cygwin installed, but it's not in my path when I am using msys. Task manager shows me that I only have one instance of ssh-agent running. The ssh-agent which is running is the msys ssh-agent, and was started from msysgit's bash prompt (without anything from cygwin in path). The ssh key was generated using cygwin's ssh-keygen, but superficial testing shows me that msys ssh utilities have no problem with the file. ssh-agent does not give me any error messages or warnings.
I can use cygwin's ssh-agent, but mixing cygwin and msys introduces other complexities (and you can see some of that just from my description of this issue), and I am trying to get msys working without anything from cygwin.
What do I need to do to find out why the msys ssh-add does not work? (Or: what do I need to do to get that ssh-add to work properly?)
I can't tell you much about the why's and how's, but I can tell you what I tried and what finally solved it.
In git bash:
exec ssh-agent bash
Then (in the changed shell):
ssh-add
That worked to a degree, I only had to type the passphrase once, but I lost the normal look. After some more searching I found a .bashrc script on this Github help page.