I've been trying to understand the relationship between applications, instances, exes and processes.
When i open two different word files on my windows laptop, 2 different 'Processes' show up in Task manager. I can independently kill each one
When i open Process Explorer - I can only see 1 instance of WINWORD.exe. I can't seem to find the processes related to each of the files i opened. Can someone explain whether each file has an associated process or a single process is used to host multiple files?
There's a mixup of several concepts here; executables, processes, what "opening a file" means and "whatever Task Manager shows in its processes tab", which is mostly a lie.
Double-clicking on a document generally1 boils down to a look up on the file extension in the registry and launching the associated executable, passing the path to the clicked file as command line argument.
Whenever an executable is launched, the kernel spawns a process (which roughly means, allocates some virtual memory for it and creates its main thread of execution) and loads the executable in it ("copies"2 it in memory, loads its dependencies and in general performs a lot of tasks necessary to be able to execute the machine code that is stored in it).
What happens next depends entirely on the code of the executable. Most programs will keep executing in the new process; some instead will look for an instance of the program already executing (which may or may not be the same executable), and communicate to it that a new instance has been launched (possibly forwarding to the existing instance the arguments obtained on the command line). This is generally the case for applications where there's a lot of data to share between different instances (so, to save e.g. memory) or where there's interaction between e.g. the various opened documents that is less cumbersome to implement in a single process (IPC is often a pain).
When instead you open a file e.g. from the File → Open menu of an application, that's generally done in the same process; again, when you actually click open it's up to the application to decide whether to open it inside the same process, or to spawn a different one for the other document.
The important concept here, though, is that if you see different top-level windows (≈ windows that you see in the taskbar) it does not mean they come from different processes, a single process may spawn multiple top-level windows (and indeed applications that deal with multiple files in multiple windows often do that). Also, remember that the fact that the 1 open file = 1 window (or 1 application, for what that matters) is just a GUI concept; you may have files that are handled concurrently by multiple processes (databases are a common instance of this).
As for Task Manager: the processes tab doesn't really list processes as they are seen by the system; some are grouped according to some unspecified rule/heuristics, some other processes are expanded to show sub-components (e.g. the various svchost.exe processes); processes that have multiple top-level windows, in particular, are shown "as if" having sub-processes. Clicking on "End task" on these fake sub-processes will actually send a "polite" quit request to the associated window, which will be handled in an application-specific way.
Ultimately, you shouldn't really think too hard about what is shown here. What are actually the "real" processes as understood by the operating system kernel is what is shown in the "Details" tab, which should match what you see in Process Exporer. If you do kill the process here, you'll see that all windows that are managed by it disappear.
I say "generally" because it's actually a lot more complicated than that; file association can do some really complicated stuff that avoids process creation, generally to directly communicate to an already running program that it has to load another file, essentially skipping the middleman described in the section that follows.
In quotes because it's smarter than that; what is generally done is a copy-on-write file mapping, which instructs the OS virtual memory manager to load the parts of the executable that are needed from disk when they are accessed in memory, and to copy on the fly the sections that are modified.
Related
I have a scenario in Windoze where I'd like to monitor for program A to be running before launching program B. I've google'd for related topics and all I've come across are posts for using batch files to kick off multiple apps in sequence (with/out delays). The use case I'm facing is that the user can download a file which causes program A to launch, so I don't always know when A will be started (i.e. can't just make a batch file with a shortcut on the desktop for the user).
I'd like to find a somewhat native solution that would routinely scan for program A running (could simply set up a Task that repeats) and then kicks off program B.
If a more complex method needs to be employed, so be it, but I'd prefer a minimally-invasive solution... simple language, simple development environment, etc.
I've been reading about the functions of Threads and Process. One major difference between a process and a thread is that a thread shares the resources with the process from which it has been created whereas a process will have its own set of resources. I wanted to test this, so I opened up my Task Manager and opened MS Word and in the task manager, there was a new process called WINWORD.EXE loaded and I again opened another MS Word(Keeping the previous one opened) yet the Task Manager shows only one WINWORD.EXE process running. I just don't get it, since two MS Words are needed to be considered as two separate process(as they don't share resources).
There only is one process in that case. It's just that Word is implemented in such a way that it prevents you creating multiple processes. When you try to start another one, the new process forwards the request to the existing process, and the window is shown by the existing process.
But yes, separate processes do indeed have distinct and isolated address spaces. Try your experiment with a simple program, for instance Notepad.
I would like to write a C++ function, on Microsoft Windows, that spawns a process and returns, in addition to the process's termination status, a list of all the files the process read or wrote. It should not require any cooperation from the spawned application.
For example, if the program spawned is the Visual Studio C++ compiler, the function would produce a list containing the source file the compiler opened, all header files it read, and the .OBJ file it created. If it also contained things like .DLL files the program contained, that would be fine. But again, it should work regardless of the program spawned; the compiler is just an example.
A twist: if the process creates subprocesses, I need to monitor their file accesses as well.
A second twist: if the process tries to open a file, I would like to be able to make it wait until I can create that file—and only then let it resume and open the file. (I think this rules out ETW.)
I know this probably sounds like an ingredient for some horrible kludge. But if I can get this working, the end result will be really cool.
A second twist: if the process tries to open a file, I would like to be able to make it wait until I can create that file—and only then let it resume and open the file
You just put yourself into Hack City with that requirement - you're right that ETW would've been a far easier solution, but it also has no way to block the file call.
Basically, here's what you're going to have to do:
Create the process suspended
Create two named pipes in opposite directions whose names are well known (perhaps containing the PID of the process)
Hook LoadModule, and the hook will watch for Kernel32 to get loaded
When Kernel32 gets loaded, hook CreateFileW and CreateFileA - also hook CreateProcessEx and ShellExecute
When your CreateFile hook hits, you write the name to one of the named pipes, then execute a ReadFile on the other one until the parent process signals you to continue.
When your CreateProcessEx hook hits, you get to do the same process all over again from inside the current process (remember that you can't have the parent process do the CreateProcess'ing because it'll mess up inherited handles).
Start the child process.
Keep in mind that you'll be injecting code and making fixups to an in-memory image that may be a different bitness than yours (i.e. your app is 64-bit, but it's starting a 32-bit process), so you'll have to have both x86 and amd64 versions of your shim code to inject. I hope by writing this lengthy diatribe you have convinced yourself that this is actually an awful idea that is very difficult to get right and that people who hook Win32 functions make Windows OS developers sad.
When a console application is started from another console application, how does console ownership work?
I see four possibilities:
The second application inherits the console from the first application for its lifetime, with the console returning to the original owner on exit.
Each application has its own console. Windows then somehow merges the content of the two into what the "console" visible to the user
The second application get a handle to the console that belongs to the first application.
The console is placed into shared memory and both applications have equal "ownership"
It's quite possible that I missed something and none of these four options adequately describe what Windows does with its consoles.
If the answer is close to option 4. My follow-up question is which of the two processes is responsible for managing the window? (Handling graphical updates when the screen needs to be refreshed / redrawn, etc)
A concrete example: Run CMD. Then, using CMD, run [console application]. The [console application] will write to what appears to be the same console window that CMD was using.
None of your four possibilities is actually the case, and the answer to your follow-on question, "Which of the two processes is responsible for managing the window?", is that neither process is responsible. TUI programs don't have to know anything about windows at all, and, under the covers, aren't necessarily even plumbed in to the GUI.
Consoles are objects, accessed via handles just like files, directories, pipes, processes, and threads. A single process doesn't "own" a console via its handle to it any more than a process "owns" any file that it has an open handle to. Handles to consoles are inherited by child processes from their parents in the same way that all other (inheritable) handles are. Your TUI application, spawned by CMD, simply inherits the standard handles that CMD said that it should inherit, when it called CreateProcess() — which are usually going to be CMD's standard input, output, and error (unless the command-line told CMD to use some other handles as the child's standard input, output, and error).
Consoles aren't dependent upon CMD. They exist as long as there are (a) any open handles to the console's input or output buffers or (b) any processes otherwise "attached" to the console. So in your example you could kill CMD, but only when you terminated the child process too would the console actually be destroyed.
The process that is in charge of displaying the GUI windows in which consoles are presented is, in Windows NT prior to version 6.1, CSRSS, the Client-Server Runtime SubSystem. The window handling code is in WINSRV.DLL, which contains the "console server" that — under the covers — Win32 programs performing console I/O make LPC calls to. In Windows NT 6.1, this functionality, for reasons covered by Raymond Chen, moved out of CSRSS into a less-privileged process that CSRSS spawns.
My guess is somewhere between 3 and 4. The console is a self-standing object, which has standard input, output and error streams. These streams are attached to the first process that uses the console. Subsequent processes can also inherit these streams if not redirected (e.g. running a command with redirect to a file.)
Normally there is no contention, since parent processes usually wait for their child process to complete, and asynchronous processes typically start their own console (e.g. try "start cmd" in a command prompt) or redirect standard output.
However, there is nothing to stop both processes writing to the output stream at the same time - the streams are shared. This can be a problem when using some runtime libraries since writes to standard output/error may not be immediately flushed, leading to mixed garbled output. In general, having to processes actively writing to the same output stream is usually not a good idea, unless you take measures to co-ordinate their output through concurrency primitives like Mutexes, Events and the like.
The way the SDK talks about it strongly resembles 1. It is an option with CreateProcess, described as follows:
CREATE_NEW_CONSOLE
The new process has a new console, instead of inheriting its parent's console (the default). For more information, see Creation of a Console.
Output however happens through handles, you'd get one with GetStdHandle(). Passing STD_OUTPUT_HANDLE returns the console handle, assuming output isn't redirected. Actual output is done through WriteFile() or WriteConsole/Output(). If both processes keep writing output to the handle then their output will be randomly intermingled. This is otherwise indistinguishable from what would happen when two programs write to the same file handle.
Logically, there's a screen buffer associated with a console. You can tinker with it with SetConsoleScreenBufferXxx(). From that point of view you could call it shared memory. The actual implementation is undiscoverable, handles abstract them away, like any Win32 API. It is sure to have changed considerably in Vista with the new conhost.exe process.
CMD 'owns' the console. When it creates a process for an app, that app inherits handles to the console. It can read and write those. When the process goes away, CMD continues ownership.
Note: I'm not entirely sure that 'ownership' is the right word here. Windows will close the Console when CMD exits, but that may be a simple setting.
Each application will run in it's own AppDomain. Each AppDomain should be running it's own console.
Ah, you're right. I was thinking about running executables within a process and forgot they start their own process - I didn't drill down far enough.
I think it's spelled out fairly well in the documentation.
UNIX file-locking is dead-easy: The operating system assumes that you know what you are doing and lets you do what you want:
For example, if you try to delete a file which another process has opened the operating system will usually let you do it. The original process still keeps it's file-handles until it terminates - at which point the the file-system will quietly re-cycle the disk-resources. No fuss, that's the way I like it.
How different things are on Windows: If I try to delete a file which another process is using I get an Operating-System error. The file is untouchable until the original process releases it's lock on the file. That was great back in the single-user days of MS-DOS when any locking process was likely to be on the same computer that contained the files, however on a network it's a nightmare:
Consider what happens when a process hangs while writing to a shared file on a Windows file-server. Before the file can be deleted we have to locate the computer and ID the process on that computer which originally opened the file. Only then can we kill the process and delete our unwanted file.
What a nuisance!
Is there a way to make this better? What I want is for file-locking on Windows to behave a like file-locking in UNIX. I want the operating system to just let me do what I want because I'm in charge and I know what I'm doing...
...so can it be done?
No. Windows is designed for the "average user", that is people who don't understand anything about a computer. Therefore, the OS tries to be smart to avoid PEBKACs. To quote Bill Gates: "There are no issues with Windows that any number of people want to be fixed." Of course, he knows that 99.9999% of all Windows users can't tell whether the program just did something odd because of them or the guy who wrote it.
Unix was designed when the world was more simple and anyone close enough to a computer to touch it, probably knew how to assemble it from dirty sand. Therefore, the OS usually lets you do what you want because it assumes that you know better (and if you didn't, you will next time).
Technical answer: Unix allocates an "i-nodes" if you create a file. I-nodes can be shared between processes. If two processes create the same file (that is, two processes call create() with the same path), then you end up with two i-nodes. This is by design. It allows for a fancy security feature: You can create files which no one can open but yourself:
Open a file
Delete it (but keep the file handle)
Use the file any way you like
Close the file
After step #2, the only process in the universe who can access the file is the one who created it (unless you want to read the hard disk block by block). The OS will keep the data alive until you either close the file or your process dies (at which time Unix will clean up after you).
This design is the foundation of all Unix filesystems. The Windows file system NTFS works much the same way but the high level API is different. Many applications open files in exclusive mode (which prevents anyone, even backup programs) to read the file. This is even true for applications which just display information like PDF viewers.
That means you'll have to fix all the Windows applications to achieve the desired effect. If you have access to the source, you can create a file in a shared mode. That would allow other processes to access it at the same time but then, you will have to check before every read/write if the file still exists, whether someone has made changes, etc.
According to MSDN you can specify to CreateFile() 3rd parameter (dwSharedMode) shared mode flag FILE_SHARE_DELETE which:
Enables subsequent open operations on a file or device to request delete access.
Otherwise, other processes cannot open the file or device if they request delete access.
If this flag is not specified, but the file or device has been opened for delete access, the function fails.
Note Delete access allows both delete and rename operations.
http://msdn.microsoft.com/en-us/library/aa363858(VS.85).aspx
So if you're can control your applications you can use this flag.
Note that Process Explorer allow for force closing of file handles (for processes local to the box on which you are running it) via Handle -> Close Handle.
Unlocker purports to do a lot more, and provides a helpful list of other tools.
Also deleting on reboot is an option (though this sounds like not what you want)
That doesn't really help if the hung process still has the handle open. It won't release the resources until that hung process releases the handle. But anyway, in Windows it is possible to force close a file out from under a process that's using it. Process Explorer from sysinternals.com will let you look at and close handles that a process has open.