forking() and CreateProcess() - fork

Are forking() and CreateProcess(with all required arguments), the same thing for Linux and WinXP, respectively?
If they are different, then could someone explain the difference in terms of what happens in each of the two cases?
Thanks

They do different things, and on different systems. CreateProcess is a Windows-only function, while fork is only on POSIX (e.g. Linux and Mac OSX) systems.
The fork system call creates a new process and continue execution in both the parent and the child from the point where the fork function was called. CreateProcess creates a new process and load a program from disk. The only similarity is that the end result is a new process is created.
For more information, read the respective manual page on CreateProcess and fork.
To summarize: CreateProcess is similar to fork() followed by one of the exec() functions.

CreateProcess takes the following steps:
Create and initialize the process control block (PCB) in the kernel.
Create and initialize a new address space.
Load the program prog into the address space.
Copy arguments args into memory in the address space.
Initialize the hardware context to start execution at “start”.
Inform the scheduler that the new process is ready to run.
Unix's fork takes the following steps:
Create and initialize the process control block (PCB) in the kernel
Create a new address space
Initialize the address space with a copy of the entire contents of
the address space of the parent
Inherit the execution context of the parent (e.g., any open files)
Inform the scheduler that the new process is ready to run
It creates a complete copy of the parent process, and the parent doesn't set up the runtime environment for the child, because the parent process trusts its own set up. The child is a complete copy of the parent except for its process id (what fork returns). A forked process continues to run the same program as its parent until it performs an explicit exec. When the child calls exec which, the new executable image into memory and runs.
How is it efficient to make a complete copy? copy-on-write. It really only copies the virtual memory map. All of the segments in the segment table are read only. If the parent or child edits the data in a segment, an exception is thrown and the kernel creates a full memory copy of that. This is explained nicely in this answer
There are several benefits to shared resources between parent and child:
- intuitively, resource management: less memory is needed to maintain the states of the processes
- Cache resources are shared means greater temporal locality of data when data is not over written, which improves performance because retrieving data from larger caches/disk is time consuming.
Disadvantages to shared resources:
- when writes are common, it puts the data in an invalid state for the other process, and this leads to coherency misses which is costly if the child process is running on a separate core, because the changes will have to propagate up to the L3 cache.
In general though, programs read a heck of a lot more than writes, typically the child/parent will only need to make writes to its stack, and that's a small portion of the their program block.
Additionally Unix fork is different because it returns twice, once in the parent (the process id of its child), once in the child (0, congrats you're a new baby process), which is how we distinguish in our code if we are the child or parent.
Unix Exec does the following:
Load the program prog into the current address space.
Copy arguments args into memory in the address space.
Initialize the hardware context to start execution at “start.”
the parent has the option to wait for the child to finish. When the child finishes, when exit is called is when the parent's wait is notified.

I will give two examples to show the difference:
fork():
#include "stdio.h"
#include "stdlib.h"
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
int fac(int);
int main(void)
{
int child_ret,input_num=-1;
pid_t pid1;
while(input_num<0){
printf("\nPlease input a non-negative number: ");
scanf("%d",&input_num);
}
if((pid1=fork())<0){
printf("fork error");
}
else if(pid1==0){
printf("\nI am the child process,my PID is %d.\n\nThe first %d numbers of fibonacci sequence is:\n", getpid(),input_num);
for (int i=0;i<input_num;i++)
{printf("%d\n", fac(i+1));}
}
else{
wait(&child_ret);
printf("\nI am the parent process,my PID is %d.\n\n", getpid());
}
return 0;
}
int fac(int n)
{
if (n<=2) return n-1;
else
{
return fac(n-1)+fac(n-2);
}
}
In this program, fork will do a copy and return two values. We called the copied process parent process and the other one child process.If we call the exec() function, the whole process will be replaced by a new program except the PID.
CreateProcess():
#include <windows.h>
#include <stdio.h>
#include <tchar.h>
void _tmain( VOID )
{
STARTUPINFO si;
PROCESS_INFORMATION pi;
LPTSTR szCmdline=_tcsdup(TEXT("MyChildProcess"));
ZeroMemory( &si, sizeof(si) );
si.cb = sizeof(si);
ZeroMemory( &pi, sizeof(pi) );
// Start the child process.
if( !CreateProcess( NULL, // No module name (use command line)
szCmdline, // Command line
NULL, // Process handle not inheritable
NULL, // Thread handle not inheritable
FALSE, // Set handle inheritance to FALSE
0, // No creation flags
NULL, // Use parent's environment block
NULL, // Use parent's starting directory
&si, // Pointer to STARTUPINFO structure
&pi ) // Pointer to PROCESS_INFORMATION structure
)
{
printf( "CreateProcess failed (%d)./n", GetLastError() );
return;
}
// Wait until child process exits.
WaitForSingleObject( pi.hProcess, INFINITE );
// Close process and thread handles.
CloseHandle( pi.hProcess );
CloseHandle( pi.hThread );
}
This is an example from MSDN. What we call to create a new process must be a separate *.exe program in Windows system. The new process is a whole new one which just has the only connection of return value with the older one.
In conclusion, we often see fork()+exec() as CreateProcess(). In fact, fork() is more similar with CreateThread() in Windows.

Related

Is there a race between starting and seeing yourself in WinApi's EnumProcesses()?

I just found this code in the wild:
def _scan_for_self(self):
win32api.Sleep(2000) # sleep to give time for process to be seen in system table.
basename = self.cmdline.split()[0]
pids = win32process.EnumProcesses()
if not pids:
UserLog.warn("WindowsProcess", "no pids", pids)
for pid in pids:
try:
handle = win32api.OpenProcess(
win32con.PROCESS_QUERY_INFORMATION | win32con.PROCESS_VM_READ,
pywintypes.FALSE, pid)
except pywintypes.error, err:
UserLog.warn("WindowsProcess", str(err))
continue
try:
modlist = win32process.EnumProcessModules(handle)
except pywintypes.error,err:
UserLog.warn("WindowsProcess",str(err))
continue
This line caught my eye:
win32api.Sleep(2000) # sleep to give time for process to be seen in system table.
It suggests that if you call EnumProcesses() too fast after starting, you won't see yourself. Is there any truth to this?
There is a race, but it's not the race the code tried to protect against.
A successful call to CreateProcess returns only after the kernel object representing the process has been created and enqueued into the kernel's process list. A subsequent call to EnumProcesses accesses the same list, and will immediately observe the newly created process object.
That is, unless the process object has since been destroyed. This isn't entirely unusual since processes in Windows are initialized in-process. The documentation even makes note of that:
Note that the function returns before the process has finished initialization. If a required DLL cannot be located or fails to initialize, the process is terminated.
What this means is that if a call to EnumProcesses immediately following a successful call to CreateProcess doesn't observe the newly created process, it does so because it was late rather than early. If you are late already then adding a delay will only make you more late.
Which swiftly leads to the actual race here: Process IDs uniquely identify processes only for a finite time interval. Once a process object is gone, its ID is up for grabs, and the system will reuse it at some point. The only reliable way to identify a process is by holding a handle to it.
Now it's anyone's guess what the author of _scan_for_self was trying to accomplish. As written, the code takes more time to do something that's probably altogether wrong1 anyway.
1 Turns out my gut feeling was correct. This is just your average POSIX developer, that, in the process of learning that POSIX is insufficient would rather call out Microsoft instead of actually using an all-around superior API.
The documentation for EnumProcesses (WIn32 API - EnumProcesses function), does not mention anything about a delay needed to see the current process in the list it returns.
The example from Microsoft how to use EnumProcess to enumerate all running processes (Enumerating All Processes), also does not contain any delay before calling EnumProcesses.
A small test application I created in C++ (see below) always reports that the current process is in the list (tested on Windows 10):
#include <Windows.h>
#include <Psapi.h>
#include <iostream>
#include <vector>
const DWORD MAX_NUM_PROCESSES = 4096;
DWORD aProcesses[MAX_NUM_PROCESSES];
int main(void)
{
// Get the list of running process Ids:
DWORD cbNeeded;
if (!EnumProcesses(aProcesses, MAX_NUM_PROCESSES * sizeof(DWORD), &cbNeeded))
{
return 1;
}
// Check if current process is in the list:
DWORD curProcId = GetCurrentProcessId();
bool bFoundCurProcId{ false };
DWORD numProcesses = cbNeeded / sizeof(DWORD);
for (DWORD i=0; i<numProcesses; ++i)
{
if (aProcesses[i] == curProcId)
{
bFoundCurProcId = true;
}
}
std::cout << "bFoundCurProcId: " << bFoundCurProcId << std::endl;
return 0;
}
Note: I am aware that the fact that the program reported the expected result does not mean that there is no race. Maybe I just couldn't catch it manifest. But trying to run code like that can give you a hint sometimes (especially if the result would have been that there is a race).
The fact that I never had a problem running this test (did it many times), together with the lack of any mention of the need for a delay in Microsoft's documentation make me believe that it is not required.
My conclusion is that either:
There is a unique issue when using it from python (doubt it).
or:
The code you found is doing something unnecessary.
There is no race.
EnumProcesses calls a NT API function that switches to kernel mode to walk the linked list of processes. Your own process has been added to the list before it starts running.

Windows DuplicateHandle named pipe handle strange error 183 "file already exists"

I have a problem with DuplicateHandle (Win32). I try to duplicate a named pipe handle and I always get the Error 183 "file already exists". I do not understand this error message, because I try to create a copy of a file handle and the new file handle does not exist before. (Is there a start value required to overwrite?) This is my call:
return DuplicateHandle (MeshellProcessHandle, sourcehandle, HelperProcess, targethandle, 0, TRUE, DUPLICATE_SAME_ACCESS) != 0;
To understand what I am doing, I have to explain more extensively: I am working on a convenient editor frontend for the command line program cmd.exe. This project already works fine on the OS/2 Operating System which is very familiar with Win32, because from a historical point of view, the two OS were developed together until one year before finishing, where Microsoft and IBM went different ways.
The implementation of this program was quite tricky: There's a windowed front end editor program. This program creates named pipes for stderr, stdout and stderr, but from the reverse point of view (output from cmd.exe is input for the editor). Because of limited communication between different sessions, I had to program a "cmd helper program" which is a tiny commandline program holding several API calls and running in the same session as the cmd.exe program. The helper gets the editor process ID via commandline parameter and opens the existing pipes created by the windowed editor program and then redirects stdin/stdout/stderr to the pipes. The helper gets the process handle of the editor from the editor process ID via "OpenProcess" API call. Then the helper executes cmd.exe which automatically inherits the stdin/stdout/stderr handles and now cmd.exe writes to and reads from the pipe.
Another option would be to parse the full pipe names to cmd.exe without using DuplicateHandle, but I would prefer to be as close as to my solution which already works fine on the OS/2 Operating System.
I am still not sure why I have no access rights to duplicate the pipe handles. But I have found another solution: My helper console program starts the child process (cmd.exe) and within this child process, I want to use the named pipes instead of stdin/stdout/stderr - this is the reason why I want to use DuplicateHandle. Windows offers a convenient solution when starting the child process by using
CreateProcess (...)
With CreateProcess, you have always to hold parameters in the STARTUPINFO structure. And there are three handle variables you can set to redirect stdin/stdout/stderr to the three named pipes cmd_std*:
STARTUPINFO StartupInfo;
HFILE cmd_stdout, cmd_stdin, cmd_stderr;
//Open the existing pipes with CreateFile
//Start child process program
StartupInfo.hStdOutput = &cmd_stdout;
StartupInfo.hStdInput = &cmd_stdin;
StartupInfo.hStdError = &cmd_stderr;
CreateProcess (..., &StartupInfo, ...);
This solution has much less code than the variant with DuplicateHandle, because I also have to save and to restore the original file handles, so this way replaces 9 DuplicateHandle calls.
I did program the changes now but my new code described above does not work. If I set the "handle inheritance flag" in CreateProcess to TRUE, cmd.exe does not get executed, but CreateProcess returns TRUE (=OK). Here's my detailed code:
STARTUPINFO StartupInfo;
PROCESS_INFORMATION ProcessInformation;
HFILE cmd_stdout, cmd_stdin, cmd_stderr;
char *pprogstr, *pargstr;
//Open the existing pipes with CreateFile
//Start child process program cmd.exe
StartupInfo.hStdOutput = &cmd_stdout;
StartupInfo.hStdInput = &cmd_stdin;
StartupInfo.hStdError = &cmd_stderr;
StartupInfo.dwFlags = STARTF_USESTDHANDLES;
pprogstr = "C:\\WINDOWS\\system32\\cmd.exe";
pargstr = "/K";
CreateProcess (
pprogstr, // pointer to name of executable module
pargstr, // pointer to command line string
NULL, // pointer to process security attributes
NULL, // pointer to thread security attributes
TRUE, // handle inheritance flag
0, // creation flags
NULL, // pointer to new environment block
NULL, // pointer to current directory name
&StartupInfo, // pointer to STARTUPINFO
&ProcessInformation) // pointer to PROCESS_INFORMATION
Any idea?

Windows DLL global variables

A DLL loaded into a Windows app is composed of an entry point (DLLMain()) and other functions that can be called from there. Now, how do I make a variable global amongst all those functions for every process and thread that attaches to this DLL?
Currently, I need to open/close a file every time one of the functions is called. Since these functions are called very often it is a real performance problem opening/closing the file each time.
wofstream wofstr;
int WINAPI func(LPCWSTR lpString)
{
...
//wofstr.open(...);
wofstr << lpString << endl; // *** NOT working without opening in this very function
...
//wofstr.close(...);
...
}
BOOL APIENTRY DllMain(HMODULE hModule, DWORD ul_reason_for_call, LPVOID lpReserved)
{
...
case DLL_PROCESS_ATTACH:
wofstr.open(...); // *** NOT seen in function above
...
case DLL_THREAD_ATTACH:
...
case DLL_THREAD_DETACH:
...
case DLL_PROCESS_DETACH:
wofstr.close(...)
...
}
What am I missing here? What about threads?
Each global variable in a Windows DLL has a single instance for each process that loads the DLL. There is a one-to-one mapping between processes and global variable instances. Processes are completely isolated from each other and one process cannot directly access memory in a different process.
If you want to share a variable between processes then you will need some form of inter-process communication (IPC). For example you could set up a logging app to which your DLL could send messages.

How to know what amount of memory I'm using in a process? win32 C++

I'm using
Win32 C++ in
CodeGear Builder 2009
Target is Windows XP Embedded.
I found the PROCESS_MEMORY_COUNTERS_EX struct
and I have created a siple function to return the
Memory consumption of my process
SIZE_T TForm1::ProcessPrivatBytes( DWORD processID )
{
SIZE_T lRetval = 0;
HANDLE hProcess;
PROCESS_MEMORY_COUNTERS_EX pmc;
hProcess = OpenProcess( PROCESS_QUERY_INFORMATION |
PROCESS_VM_READ,
FALSE, processID );
if (NULL == hProcess)
{
lRetval = 1;
}
else
{
if ( GetProcessMemoryInfo( hProcess, (PROCESS_MEMORY_COUNTERS*)&pmc, sizeof(pmc)) )
{
lRetval = pmc.WorkingSetSize;
lRetval = pmc.PrivateUsage;
}
CloseHandle( hProcess );
}
return lRetval;
}
//---------------------------------------------------------------------------
Do i have to use lRetval = pmc.WorkingSetSize; or lRetval = pmc.PrivateUsage;
the privateUsage are what I see in perfmon.
but what is that WorkingSetSize exactly.
I what to see every byte I allocate in the counter when I allocate it. Is this Posible?
regards
jvdn
This is a much tougher question than you probably realized. The reason is that Windows shares most executable code between processes (especially the ones that make up most of Windows itself) between processes. For example, there's normally ONE copy of kernel32.dll loaded into memory, but it'll normally be mapped into every process. Do you consider that part of the memory your process is "using" or not?
Private memory is what's unique to that particular process. This can be somewhat misleading too. Since the executable for your process could potentially be shared with another process (i.e. two instances of your program could be run), that's not counted as part of the private memory, even if (as is often the case) there's only one instance of it running.
The working set size is about 99.999% meaningless. What it returns is whatever has been set as the preferred working set size for the process. You can adjust that with SetProcessWorkingSetSize(). Windows has a working set trimmer that attempts to trim down working sets. If memory serves, it uses the working set size to guess at whether it's worth trying to trim the working set of this process -- i.e. if its current working set is larger than the working set size was set to, it tries to trim it down. Otherwise, it (mostly) leaves it alone.
Chances are that nothing you do will show you ever byte you allocate as you allocate it though. Calling Windows to allocate memory is fairly slow, so what's normally done is that the run-time library allocates a fairly big chunk of memory from Windows. When you allocate memory, the run-time library gives you a piece of that big chunk. Only when that chunk is gone does it go back to Windows and ask for more.

GetExitCodeProcess() returns 128

I have a DLL that's loaded into a 3rd party parent process as an extension. From this DLL I instantiate external processes (my own) by using CreateProcess API. This works great in 99.999% of the cases but sometimes this suddenly fails and stops working permanently (maybe a restart of the parent process would solve this but this is undesirable and I don't want to recommend that until I solve the problem.) The failure is symptomized by external process not being invoked any more even though CreteProcess() doesn't report an error and by GetExitCodeProcess() returning 128. Here's the simplified version of what I'm doing:
STARTUPINFO si;
ZeroMemory(&si, sizeof(si));
si.cb = sizeof(si);
si.dwFlags = STARTF_USESHOWWINDOW;
si.wShowWindow = SW_HIDE;
PROCESS_INFORMATION pi;
ZeroMemory(&pi, sizeof(pi));
if(!CreateProcess(
NULL, // No module name (use command line).
"<my command line>",
NULL, // Process handle not inheritable.
NULL, // Thread handle not inheritable.
FALSE, // Set handle inheritance to FALSE.
CREATE_SUSPENDED, // Create suspended.
NULL, // Use parent's environment block.
NULL, // Use parent's starting directory.
&si, // Pointer to STARTUPINFO structure.
&pi)) // Pointer to PROCESS_INFORMATION structure.
{
// Handle error.
}
else
{
// Do something.
// Resume the external process thread.
DWORD resumeThreadResult = ResumeThread(pi.hThread);
// ResumeThread() returns 1 which is OK
// (it means that the thread was suspended but then restarted)
// Wait for the external process to finish.
DWORD waitForSingelObjectResult = WaitForSingleObject(pi.hProcess, INFINITE);
// WaitForSingleObject() returns 0 which is OK.
// Get the exit code of the external process.
DWORD exitCode;
if(!GetExitCodeProcess(pi.hProcess, &exitCode))
{
// Handle error.
}
else
{
// There is no error but exitCode is 128, a value that
// doesn't exist in the external process (and even if it
// existed it doesn't matter as it isn't being invoked any more)
// Error code 128 is ERROR_WAIT_NO_CHILDREN which would make some
// sense *if* GetExitCodeProcess() returned FALSE and then I were to
// get ERROR_WAIT_NO_CHILDREN with GetLastError()
}
// PROCESS_INFORMATION handles for process and thread are closed.
}
External process can be manually invoked from Windows Explorer or command line and it starts just fine on its own. Invoked like that it, before doing any real work, creates a log file and logs some information about it. But invoked like described above this logging information doesn't appear at all so I'm assuming that the main thread of the external process never enters main() (I'm testing that assumption now.)
There is at least one thing I could do to try to circumvent the problem (not start the thread suspended) but I would first like to understand the root of the failure first. Does anyone has any idea what could cause this and how to fix it?
Quoting from the MSDN article on GetExitCodeProcess:
The following termination statuses can be returned if the process has terminated:
The exit value specified in the
ExitProcess or TerminateProcess
function
The return value from the
main or WinMain function of the
process
The exception value for an
unhandled exception that caused the
process to terminate
Given the scenario you described, I think the most likely cause ist the third: An unhandled exception. Have a look at the source of the processes you create.
Have a look at Desktop Heap memory.
Essentially the desktop heap issue comes down to exhausted resources (eg starting too many processes). When your app runs out of these resources, one of the symptoms is that you won't be able to start a new process, and the call to CreateProcess will fail with code 128.
Note that the context you run in also has some effect. For example, running as a service, you will run out of desktop heap much faster than if you're testing your code in a console app.
This post has a lot of good information about desktop heap
Microsoft Support also has some useful information.
There are 2 issues that i could think of from your code sample
1.Get yourusage of the first 2 paramaters to the creatprocess command working first. Hard code the paths and invoke notepad.exe and see if that comes up. keep tweaking this until you have notepad running.
2.Contrary to your comment, If you have passed the currentdirectory parameter for the new process as NULL, it will use the current working directory of the process to start the new process from and not the parent' starting directory.
I assume that your external process exe cannot start properly due to dll dependencies that cannot be resolved in the new path.
ps : In the debugger watch for #err,hr which will tell you the explanation for the last error code,

Resources