When a console application is started from another console application, how does console ownership work?
I see four possibilities:
The second application inherits the console from the first application for its lifetime, with the console returning to the original owner on exit.
Each application has its own console. Windows then somehow merges the content of the two into what the "console" visible to the user
The second application get a handle to the console that belongs to the first application.
The console is placed into shared memory and both applications have equal "ownership"
It's quite possible that I missed something and none of these four options adequately describe what Windows does with its consoles.
If the answer is close to option 4. My follow-up question is which of the two processes is responsible for managing the window? (Handling graphical updates when the screen needs to be refreshed / redrawn, etc)
A concrete example: Run CMD. Then, using CMD, run [console application]. The [console application] will write to what appears to be the same console window that CMD was using.
None of your four possibilities is actually the case, and the answer to your follow-on question, "Which of the two processes is responsible for managing the window?", is that neither process is responsible. TUI programs don't have to know anything about windows at all, and, under the covers, aren't necessarily even plumbed in to the GUI.
Consoles are objects, accessed via handles just like files, directories, pipes, processes, and threads. A single process doesn't "own" a console via its handle to it any more than a process "owns" any file that it has an open handle to. Handles to consoles are inherited by child processes from their parents in the same way that all other (inheritable) handles are. Your TUI application, spawned by CMD, simply inherits the standard handles that CMD said that it should inherit, when it called CreateProcess() — which are usually going to be CMD's standard input, output, and error (unless the command-line told CMD to use some other handles as the child's standard input, output, and error).
Consoles aren't dependent upon CMD. They exist as long as there are (a) any open handles to the console's input or output buffers or (b) any processes otherwise "attached" to the console. So in your example you could kill CMD, but only when you terminated the child process too would the console actually be destroyed.
The process that is in charge of displaying the GUI windows in which consoles are presented is, in Windows NT prior to version 6.1, CSRSS, the Client-Server Runtime SubSystem. The window handling code is in WINSRV.DLL, which contains the "console server" that — under the covers — Win32 programs performing console I/O make LPC calls to. In Windows NT 6.1, this functionality, for reasons covered by Raymond Chen, moved out of CSRSS into a less-privileged process that CSRSS spawns.
My guess is somewhere between 3 and 4. The console is a self-standing object, which has standard input, output and error streams. These streams are attached to the first process that uses the console. Subsequent processes can also inherit these streams if not redirected (e.g. running a command with redirect to a file.)
Normally there is no contention, since parent processes usually wait for their child process to complete, and asynchronous processes typically start their own console (e.g. try "start cmd" in a command prompt) or redirect standard output.
However, there is nothing to stop both processes writing to the output stream at the same time - the streams are shared. This can be a problem when using some runtime libraries since writes to standard output/error may not be immediately flushed, leading to mixed garbled output. In general, having to processes actively writing to the same output stream is usually not a good idea, unless you take measures to co-ordinate their output through concurrency primitives like Mutexes, Events and the like.
The way the SDK talks about it strongly resembles 1. It is an option with CreateProcess, described as follows:
CREATE_NEW_CONSOLE
The new process has a new console, instead of inheriting its parent's console (the default). For more information, see Creation of a Console.
Output however happens through handles, you'd get one with GetStdHandle(). Passing STD_OUTPUT_HANDLE returns the console handle, assuming output isn't redirected. Actual output is done through WriteFile() or WriteConsole/Output(). If both processes keep writing output to the handle then their output will be randomly intermingled. This is otherwise indistinguishable from what would happen when two programs write to the same file handle.
Logically, there's a screen buffer associated with a console. You can tinker with it with SetConsoleScreenBufferXxx(). From that point of view you could call it shared memory. The actual implementation is undiscoverable, handles abstract them away, like any Win32 API. It is sure to have changed considerably in Vista with the new conhost.exe process.
CMD 'owns' the console. When it creates a process for an app, that app inherits handles to the console. It can read and write those. When the process goes away, CMD continues ownership.
Note: I'm not entirely sure that 'ownership' is the right word here. Windows will close the Console when CMD exits, but that may be a simple setting.
Each application will run in it's own AppDomain. Each AppDomain should be running it's own console.
Ah, you're right. I was thinking about running executables within a process and forgot they start their own process - I didn't drill down far enough.
I think it's spelled out fairly well in the documentation.
Related
I've been trying to understand the relationship between applications, instances, exes and processes.
When i open two different word files on my windows laptop, 2 different 'Processes' show up in Task manager. I can independently kill each one
When i open Process Explorer - I can only see 1 instance of WINWORD.exe. I can't seem to find the processes related to each of the files i opened. Can someone explain whether each file has an associated process or a single process is used to host multiple files?
There's a mixup of several concepts here; executables, processes, what "opening a file" means and "whatever Task Manager shows in its processes tab", which is mostly a lie.
Double-clicking on a document generally1 boils down to a look up on the file extension in the registry and launching the associated executable, passing the path to the clicked file as command line argument.
Whenever an executable is launched, the kernel spawns a process (which roughly means, allocates some virtual memory for it and creates its main thread of execution) and loads the executable in it ("copies"2 it in memory, loads its dependencies and in general performs a lot of tasks necessary to be able to execute the machine code that is stored in it).
What happens next depends entirely on the code of the executable. Most programs will keep executing in the new process; some instead will look for an instance of the program already executing (which may or may not be the same executable), and communicate to it that a new instance has been launched (possibly forwarding to the existing instance the arguments obtained on the command line). This is generally the case for applications where there's a lot of data to share between different instances (so, to save e.g. memory) or where there's interaction between e.g. the various opened documents that is less cumbersome to implement in a single process (IPC is often a pain).
When instead you open a file e.g. from the File → Open menu of an application, that's generally done in the same process; again, when you actually click open it's up to the application to decide whether to open it inside the same process, or to spawn a different one for the other document.
The important concept here, though, is that if you see different top-level windows (≈ windows that you see in the taskbar) it does not mean they come from different processes, a single process may spawn multiple top-level windows (and indeed applications that deal with multiple files in multiple windows often do that). Also, remember that the fact that the 1 open file = 1 window (or 1 application, for what that matters) is just a GUI concept; you may have files that are handled concurrently by multiple processes (databases are a common instance of this).
As for Task Manager: the processes tab doesn't really list processes as they are seen by the system; some are grouped according to some unspecified rule/heuristics, some other processes are expanded to show sub-components (e.g. the various svchost.exe processes); processes that have multiple top-level windows, in particular, are shown "as if" having sub-processes. Clicking on "End task" on these fake sub-processes will actually send a "polite" quit request to the associated window, which will be handled in an application-specific way.
Ultimately, you shouldn't really think too hard about what is shown here. What are actually the "real" processes as understood by the operating system kernel is what is shown in the "Details" tab, which should match what you see in Process Exporer. If you do kill the process here, you'll see that all windows that are managed by it disappear.
I say "generally" because it's actually a lot more complicated than that; file association can do some really complicated stuff that avoids process creation, generally to directly communicate to an already running program that it has to load another file, essentially skipping the middleman described in the section that follows.
In quotes because it's smarter than that; what is generally done is a copy-on-write file mapping, which instructs the OS virtual memory manager to load the parts of the executable that are needed from disk when they are accessed in memory, and to copy on the fly the sections that are modified.
I want to create a generic logging application (Windows), which shall be able to execute an arbitrary executable as a child process and log the complete standard streams of a specific to a file (stdin, stdout, stderr).
Further, this logging application shall be acting like a kind of a "Man-in-the-Middle" application completely transparent, so that for user who is calling either the original executable or the wrapper logging application, there is not difference.
This functionality is somehow similar to the UNIX command line tool "tee", but as far as I found out, this tool is unfortunately not be able to log also the stdin (only stdout and stderr are supported). In addition, I want to log some more information e.g. who was the calling parent process, timestamps...
I want to use this especially for the command prompt: "cmd.exe" and rename the original and replace it with my own "cmd.exe" to automatically get a history file of all entered (stdin) commands, with its outputs (stdout, stderr).
Does someone have a good idea how to get this easily realized, perhaps with C# and pipes?
Best Regards,
Andreas
A simple version would indeed use pipes. In a native win32 app you would create some inheritable pipes and set them and the STARTF_USESTDHANDLES flag in the STARTUPINFO struct passed to CreateProcess. There is example code for this on MSDN.
You cannot however use this to replace cmd.exe because cmd.exe also uses the special Windows console API in interactive mode to implement features like the F7 history "dialog". You could take a look at some of the free terminal replacements like Console2 to see how they do it.
I want to make a windows application whose GUI will be streamed to another device (allowing remote control). The point is that I'm not willing to rely on creating Windows Sessions to isolate the GUI I/O's (1)
To achieve this, I started observing some existing solutions that are able to enable remote access using this premise to see if I could get a clue about where to start.
One of these solution is Winflector (BTW: it is free up to 2 connections).
I got interested in this solution because it seems (I'm guessing) it detects only the repainted regions. What I took from my observations are that:
While the streamed application is "invisibly" running locally in the same session I'm logged in (it shows the application process in my task manager), the application window seems not to be created anyhow - at least Inspect can't get any window information/handle of the application process - It looks like sort of a "GUI StdOut Redirection".
Apparently, no additional Desktop is created;
Also apparently, no Mirror Driver is installed;
Using Process Explorer, I found out Winflector adds some thread's to the original application process. I suspect it is about the GUI redirection (by the thread's names);
The application is started by the Winflector server - that is, it has control about the CreateProcess arguments.
What is the most likely technique to be used in this case?
Windows Hook?
Windows Messages interception?
Special Display Driver?
Sort of Memory Device Context?
Where should I start researching to get a similar approach? Any open source project regarding this subject would also be very welcomed.
PS: By my programming experience, this is sort of a whole "new world" - sorry if my questions are redundant/obvious/non-sense.
(1) That is, this application could be spawned, streamed and interact
with the remote client using the same session which a local user is
already logged in, without conflicting the IO (like a regular VNC
would do, for example). PS: At this moment, I'm focusing only at the
output.
I've got a Windows application (written in C, compiled with MSVC Express edition, 32-bit mode), which has two main modes of operation:
Windowed mode -- create a window, and draw stuff in it (namely, a fractal).
Benchmark mode -- when run with --benchmark as an argument, don't make a window but just print some benchmark statistics to stdout.
During development I've compiled as a Console app, and used SDL to create the window and perform other GUI functions. So benchmark mode runs fine (no window is created), and graphical mode just has a lingering console window.
However for my release compilation I've enabled the Windows subsystem instead of Console. (As explained in this question). This works great except I've suddenly discovered I can't run benchmarks any more. :o
I'm just wondering, is there a way for an application to choose at run time (e.g. based on the command line it's given) which kind of subsystem behaviour to use?
I've done some experimentation with EXE files in Windows (explorer, notepad, winword) and none of them seem to print anything to the console when run with an argument like "/?" (which most Windows console apps support). So it doesn't look like it, but I thought it's worth asking here in case there's a special trick.
Update. It looks like, no, you can't. Thanks for the answers guys.
Additional academic question. Does this mean that the subsystem choice is marked in the EXE header, and it's the operating system that examines this and sets up the Window or connects it to the console it's run from? I don't know much about EXE loading, but I would be curious to learn a few details here.
Conclusion. I think there are four good solutions (plus two semi-solutions, making five total :p) to choose from:
Use the console subsystem, but use FreeConsole when running in GUI mode.
Use the windows system, and use AllocConsole when running in benchmark mode. Not perfect if fractal.exe is run from an existing console, so I'll count this as half a solution ;-).
Just have one executable for each subsystem, fractal.exe and fractalgui.exe.
Have two (or more) executables, one of which does the work and passes it to the other to be displayed on the console or in a Window as appropriate. Needs some thought on how to divide the programs and how to communicate between them.
Another half-solution: have fractalgui.exe print the benchmark to standard out, and pipe that to a utility that will simply print it.
I haven't yet chosen, but I'm leaning towards #3.
Thanks to Matteo and smerlin for the ideas!
There is no way a application can choose her subsystem at runtime (well there are some really ugly workarounds, but those are full of quirks).
Then general solution for this problem is to have a console application, which starts your gui application if necessary
For your benchmark case, it would just print your benchmark statistics.
example setup:
- fractalgui.exe (subsystem: windows)
- fractal.exe (subsystem: console)
* the shortcut on the user desktop links to your fractalgui.exe
* if the user starts fractal.exe from the console, fractal exe starts fractalgui.exe
* if the user starts fractal.exe --benchmark, it either does the benchmark itself (if its possible to add this benchmark logic to another executable) and prints the information directly to console, or - if thats not possible - it will need to start fractalgui.exe --nogui --benchmark. The tricky case here is to get your output from fractalgui.exe to fractal.exe, so you can print it on the appropriate console. There are several ways to do this, e.g. named pipes (there are ways to start fractalgui.exe in a way, that you can just use stdout / cout there, and the data will be piped to the stdout of fractal.exe, but i dont recall how excactly this works anymore (edit: maybe this works)). The easiest way would be to start fractalgui.exe --nogui --benchmark > mylogfile and then print mylogfile after fractalgui.exe finished (since stdout/cout of fractalgui.exe will work if the output is redirected to a file), however you wont get "live" output, since all the output will be printed on the console when fractalgui.exe is already finished.
To add to #smerlin's answer, the other oft-seen method (cited into the articles I linked inside the comment) is to mark your application as a console application, but free the console (using FreeConsole) when you determine that you don't need it.
This is how ildasm does it, but it has the disadvantage of flashing the console for a brief moment between the start of the application and the call to FreeConsole.
Additional academic question. Does this mean that the subsystem choice is marked in the EXE header, and it's the operating system that examines this and sets up the Window or connects it to the console it's run from? I don't know much about EXE loading, but I would be curious to learn a few details here.
Yes, the loader checks the PE header and sets up everything according to the subsystem specified here.
Contrast with the *NIX approach: no executable is "special", and everyone has a working stdin/stdout/stderr; applications that want to display something will call the appropriate functions of Xlib. The drawback is that GUI applications have no clue if the application you are starting normally uses the console, so the system has to ask if you want to spawn also a terminal emulator or to discard the standard streams and just wait for it to spawn a window (obviously shortcuts store this information).
I described a technique for achieving this in my question here.
Matteo already mentioned the .com trick, but that's only part of a viable solution.
this is not yet another "I need a console in my GUI app" that has been discussed quite frequently. My situation differs from that.
I have a Windows GUI application, that is run from a command line. Now if you pass wrong parameters to this application, I do not want a popup to appear stating the possible switches, but I want that printed into the console that spawned my process.
I got that far that I can print into the console (call to AttachConsole(...) for parent process) but the problem is my application is not "blocking". As soon as I start it, the command prompt returns, and all output is written into this window (see attached image for illustration).
I played around a bit, created a console app, ran it, and see there, it "blocks", the prompt only re-appears after the app has terminated. Switching my GUI app to /SUBSYSTEM:Console causes strange errors (MSVCRTD.lib(crtexe.obj) : error LNK2019: nonresolved external Symbol "_main" in function "___tmainCRTStartup".)
I have seen the pipe approach with an ".exe" and a ".com" file approach from MSDEV but I find it horrible. Is there a way to solve this prettier?
This is not behaviour that you can change by modifying your application (aside from re-labelling it as already discussed). The command interpreter looks at the subsystem that an executable is labelled with, and decides whether to wait for the application to terminate accordingly. If the executable is labelled as having a GUI, then the command interpreter doesn't wait for it to terminate.
In some command interpreters this is configurable. In JP Software's TCC/LE, for example, one can configure the command interpreter to always wait for applications to terminate, even GUI ones. In Microsoft's CMD, this is not configurable behaviour, however. The Microsoft answer is to use the START command with the /WAIT option.
Once again: This is not the behaviour of your application. There is, apart from relabelling as a TUI program, no programmatic way involving your code to change this.
Maybe write a console-based wrapper app that checks the parameters, prints the error message on bad parameters, and calls/starts up the actual program when the parameters are correct?