I am trying to implement a terminal emulator in Java. It is supposed to be able to host both cmd.exe on Windows and bash on Unix-like systems (I would like to support at least Linux and Mac OS X). The problem I have is that both cmd.exe and bash repeat on their standard output whatever I send to their standard input.
For example, in bash, I type "ls", hit enter, at which point the terminal emulator sends the input line to bash's stdin and flushes the stream. The process then outputs the input line again "ls\n" and then the output of the ls command.
This is a problem, because other programs apart from bash and cmd.exe don't do that. If I run, inside either bash, or cmd.exe, the command "python -i", the python interactive shell does not repeat the input in the way bash and cmd.exe does. This means a workaround would have to know what process the actual output came from. I doubt that's what actual terminal emulators do.
Running "bash -i" doesn't change this behaviour. As far as I know, cmd.exe doesn't have distinct "interactive" and "noninteractive" modes.
EDIT: I am creating the host process using the ProcessBuilder class. I am reading the stdout and stderr and writing to the stdin of the process using a technique similar to the stream gobbler. I don't set any environment variables before I start the host process. The exact commands I use to start the processes are bash -i for bash and cmd for cmd.exe. I'll try to post minimal code example as soon as I manage to create one.
On Unix, run stty -echo to disable "local echo" (i.e. the shell repeating everything that you type). This is usually enabled so a user can edit what she types.
In your case, BASH must somehow allocate a pseudo TTY; otherwise, it would not echo every command. set +x would have a similar effect but then, you'd see + ls instead of ls in the output.
With cmd.exe the command #ECHO OFF should achieve the same effect.
Just execute those after the process has been created and it should work.
Related
I know that one can close stdin when executing a command from a Linux CLI using logic such as:
0<&- <command>
This simulates/emulates running an application unattended, or within a CI/CD system such as Jenkins/Gitlab that has stdin closed.
Why do I care? Because we have an application that has a "Press any key to continue..." prompt.
When run under Jenkins/Gitlab or anything that doesn't have stdin open, it just moves on..... when run on Linux.
How would I do this, and is this possible to do this from the Windows CMD Window CLI?
I've tried 0<&- and that results in the message
0<& was unexpected at this time.
Google search gives many hits for stdin but all the documentation is on redirection. I haven't found anything for windows for closing.
You can try
command <nul
which basically means "take any input from the NUL device (as an infinite source of "CRLF"'s). This works as long as command takes its input from STDIN (like the pause command) and doesn't flush the input buffer (like the choice command)
If you need textual input (like the set /p command), you need another approach:
echo inputtext|command
In command line it is possible to use the output of a command as the stdin of an executable. For example, pbpaste returns the value of the clipboard on OSX. I could run a program using this, e.g. pbpaste | ./program
Is this also possible in LLDB?
lldb only has access to a program's stdio if it launched the program and is sharing the terminal with it. So you can't always do this.
There isn't an lldb command to send text to the debuggee's stdin, but you can write to the process stdin (when that's possible) from Python using SBProcess.PutSTDIN:
https://lldb.llvm.org/python_api/lldb.SBProcess.html#lldb.SBProcess.PutSTDIN
So you could pretty easily cons up Python command that runs the shell command you want, gets the output, and uses this API to write it to the target.
I'm trying to run bash.exe (Bash on Ubuntu for Windows) as a build command for Sublime Text. However, bash.exe has a bug and does not support outputting its stdout to any pipe.
Question is this: how can I run a cmd line (i.e. "bash.exe -c ls") and capture the output without ever making bash.exe output into pipes on windows?
I'm open to using any languages or environment on Windows to make this tool.
Edit
I ran
bashTest = subprocess.Popen(["bash.exe", "-c", "ls"]), stdout=subproccess.PIPE)
Which yielded:
bashTest.communicate()[0] b'E\x00r\x00r\x00o\x00r\x00:\x00\x000\x00x\x008\x000\x000\x007\x000\x000\x005\x007\x00\r\x00\r\x00\n\x00'
This is currently not possible. There's a github issue about it which was closed as a known limitation. If you want to increase awareness of it, I see 2 related User Voice ideas: Allow Windows programs to spawn Bash and Allow native Win32 applications to launch Linux tools/commands.
There are ways you could hack around it, however. One way would be to write a script which loops forever in a bash.exe console. When the script gets a signal, it runs Linux commands with the output piped to a file then signals that it is complete. Here's some pseudo code:
Linux:
while true
while not exists /mnt/c/dobuild
sleep 1
end
gcc foo.c > /mnt/c/build.log
rm /mnt/c/dobuild
end
Windows:
touch C:\dobuild
while exists C:\dobuild
sleep 1
end
cat C:\build.log
This does require keeping a bash.exe console always open with the script running, which is not ideal.
Another potential workaround, which was already mentioned, is to use ReadConsoleOutput.
You need to use the option shell=True in Popen() to have pipes work.
like this example dont need to split this command.
>>> import subprocess as sp
>>> cmd = 'echo "test" | cat'
>>> process = sp.Popen(cmd,stdout=sp.PIPE,shell=True)
>>> output = process.communicate()[0]
>>> print output
test
Your only realistic option, if you can't wait for a fix, would be to use ReadConsoleOutput and/or the related functions.
I want to send commands in the ADB shell itself as if i had done the following in cmd.
>adb shell
shell#:/ <command>
I am using python 3.4 on a windows 7 OS 64bit machine. I can send one-line shell commands simply using subprocess.getoutput such as:
subprocess.getoutput ('adb pull /storage/sdcard0/file.txt')
as long as the adb commands themselves are recognized by ADB specifically, such as pull and push, however there are other commands such as grep that need to be run IN the shell, like above, since they are not recognized by adb. for example, the following line will not work:
subprocess.getoutput ('adb shell ls -l | grep ...')
To enter the commands in the shell I thought I needed some kind of expect library as that is what 'everyone' suggests, however pexpect, wexpect, and winexpect all failed to work. they were written for python 2 and after being ported to python 3 and my going through the .py files by hand, even those tweaked for windows, nothing was working - each of them for different reasons.
how can i send the input i want to the adb shell directly?
If none of the already recommended shortcuts work for you you can still go the 'regular' way using 'subprocess.Popen' for entering commands in the adb shell with Popen:
cmd1 = 'adb shell'
cmd2 = 'ls -l | grep ...'
p = subprocess.Popen(cmd1.split(), stdin=PIPE)
time.sleep(1)
p.stdin.write(cmd2.encode('utf-8'))
p.stdin.write('\n'.encode('utf-8'))
p.stdin.flush()
time.sleep(3)
p.kill()
Some things to remember:
even though you import subprocess you still need to invoke subprocess.Popen
sending cmd1 as a string or as items in a list should work too but '.split()' does the trick and is easier on the eyes
since you only specidfied you want to enter input to the shell you only need stdin=PIPE. stdout would only be necessary if you wanted to receive output from the shell
time.sleep(1) isn't really necessary, however since many complained about input issues being faster or slower in python 2 vs 3 consider maybe using it. 'they' might have been using versions of 'expect' that need the shell's reply first. this code also worked when i tested it with simply swapping out and in the process with time.sleep(0)
stdin.write will return an error if the input is not encoded properly. python's default is unicode. entering by binary did not work for me in my tests like this "b\ls ..." but .encode() worked. dont forget the endline!
if you use .encode() there is a worry that the line might not get sent properly, so to be sure it might be good to include a flush().
time.sleep(3) is completely uneccesary, but if your command takes a long time to execute (eg a regressive search through the entire device piped out to a txt file on the memory card) maybe give it some extra time before killing anyhting.
remember to kill. if you didnt kill it, the pipe may remain open, and even after exiting the test app on the console the next commend still went to the shell even though the prompt appearsed to be my regular cmd prompt.
Amichai, I have to start with pointing out that your own "solution" is pretty awful. And your explanation makes it even worse. Doing all those unnecessary things just because you do not understand how shell (here I mean your PC's OS shell, not adb) command parsing works.
When all you needed was just this one command:
subprocess.check_output(['adb', 'shell', 'ls /storage/sdcard0 | grep ...']).decode('utf-8')
In Windows you can use the following command in Matlab to start a new instance of MATLAB which will run in the background (i.e. you can keep executing commands in your first version of MATLAB).
system('matlab &')
An analogous call in OSX,
system([matlabroot '/bin/matlab &'])
however results in the display of the splash image, then nothing. If I take out the ampersand, the new instance opens as expected. Unfortunately, this won't work for me, I really need to be able to control the first instance of MATLAB while the second is running.
Does anyone know why this discrepancy between the operating systems exists? By the way, I'm using OSX 10.7, Windows 7 64 bit, and MATLAB R2012a on Mac and R2012b on PC.
As some background, I'm trying to write a generic tester for an interactive command line interface that uses the input() function extensively.
Edit: I should have mentioned that the command
/Applications/MATLAB_R2012a.app/bin/matlab &
works as expected from the OSX terminal. In other words, a new instance of MATLAB opens and new commands can be entered into the terminal. So this problem seems to be specific to the system() function in OSX matlab.
Also, I tried adding that command to a bash script and calling the script from matlab, but had the same problem that I did with putting the command into the system() function.
Thanks
This is a long shot, but it might be happening because when you invoke the new instance of Matlab from Matlab with the system() command on Unix or OS X, the matlab_helper process forks and runs a shell process to run the new application. If you omit the ampersand, the shell blocks and waits for the program to finish, and system() waits for it, so the first Matlab locks up. And (here's the speculation part) if you add the ampersand, Matlab launches in the background, and then the forked shell exits, which then causes the new Matlab process to exit because its parent process (the shell) has exited. (Windows doesn't have the same parent/child process relationships, process launch mechanism, or shells, which would explain the different behavior.)
You could try prefixing the command with nohup, which protects processes from getting killed by SIGHUP, which might be what's happening here to your second Matlab process.
system(['nohup ' matlabroot '/bin/matlab &'])
You could also try using the OS X open command to launch a new independent instance. Something like this. You may need to fiddle with the options and path, but -n should be what gives you a new instance. It should be pointing at /Applications/MATLAB_R2012a.app; I'm assuming that's what matlabroot is returning on OS X.
system(['open -na ' matlabroot])
You could also try running it from the Java process-launching features from within Matlab instead of with system(). Runtime.exec() doesn't block like system() does, and there may be other quirks to system(), like the matlab_helper architecture. Try launching it with java.lang.Runtime from Matlab.
jrt = java.lang.Runtime.getRuntime();
newMatlabProcess = jrt.exec([matlabroot '/bin/matlab']);
You can try the other command line variants above using this mechanism too, and you may need to redirect stdout to /dev/null, since the new processes input and output are buffered in to that newMatlabProcess object.
You can use applescript to do this. I do something like this:
! osascript -e "tell application \"Terminal\" to do script \"cd `pwd`;matlab -nojvm -nosplash -r 'why'\""
This example opens a new Matlab instance, in the current directory, and runs the command "why". You can remove the "-nojvm" if you need java in your background Matlab process