Some libraries might register some handlers with pthread_atfork(). I don't need them as I only use fork() together with exec(). Also, they can cause trouble in some cases. So, is there a way to reset the registered handler list?
Related: calling fork() without the atfork handlers, fork() async signal safety.
POSIX does not document any mechanism for fork handlers installed by pthread_atfork() to be removed, short of termination of the process or replacing the process image. If you don't want them, then don't install them. If they are installed by a third-party library, as you describe, then your options are to find a way to avoid that behavior of the library (possibly by avoiding the library altogether) or to live with it.
Related
Im writing an application in Rust that checks for certain processes. I know it's possible to get a list of running processes, but i rather not create an infinite loop to poll them.
Is there an event that gets triggered when a process is started?
Rust can't do anything that the OS doesn't already provide, and Rust doesn't have its own runtime, so you can just use whatever the OS offers.
When there isn't already a crate for some thing, the problem boils down to: How would you do that in C? Find answer to that, and then use Rust's FFI (or some lower-level sys crate like winapi to call that.
I've been looking at some hooking code which selectively loads a library into certain processes and then hooks certain native API functions (using Detours). The chain of events looks like this:
Kernel driver loads A.dll into every process.
A.dll::DllMain() decides whether to load B.dll (LoadLibraryEx) which contains actual Detours hooks.
B.dll runs for the duration of the process hooking said functions.
The second bullet here appears to break the DllMain rules specified here, but I'm trying to work out if the way the driver loads A.dll works around the limitations. Specifically, the kernel driver uses PsSetLoadImageNotifyRoutine to get notifications when each process starts and then queues an APC to call LoadLibraryEx on A.dll which means it's pretty much the first DLL loaded when the process starts. Does this circumvent the problems with calling LoadLibrary within DllMain?
Doesn't matter how the LoadLibraryEx was triggered. Once triggered, the DLL loading process is the same, and the same rules apply.
The documentation very specifically says not to call LoadLibrary in DllMain. Even in the unlikely event that you figured out a safe way to make it work, it may not work in the next version (or even the next service pack) of Windows.
I know about Cygwin, and I know of its shortcomings. I also know about the slowness of fork, but not why on Earth it's not possible to work around that. I also know Cygwin requires a DLL. I also understand POSIX defines a whole environment (shell, etc...), that's not really what I care about here.
My question is asking if there is another way to tackle the problem. I see more and more of POSIX functionality being implemented by the MinGW projects, but there's no complete solution providing a full-blown (comparable to Linux/Mac/BSD implementation status) POSIX functionality.
The question really boils down to:
Can the Win32 API (as of MSVC20??) be efficiently used to provide a complete POSIX layer over the Windows API?
Perhaps this will turn out to be a full libc that only taps into the OS library for low-level things like filesystem access, threads, and process control. But I don't know exactly what else POSIX consists of. I doubt a library can turn Win32 into a POSIX compliant entiity.
POSIX <> Win32.
If you're trying to write apps that target POSIX, why are you not using some variant of *N*X? If you prefer to run Windows, you can run Linux/BSD/whatever inside Hyper-V/VMWare/Parallels/VirtualBox on your PC/laptop/etc.
Windows used to have a POSIX compliant environment that ran alongside the Win32 subsystem, but was discontinued after NT4 due to lack of demand. Microsoft bought Interix and released Services For Unix (SFU). While it's still available for download, SFU 3.5 is now deprecated and no longer developed or supported.
As to why fork is so slow, you need to understand that fork isn't just "Create a new process", it's "create a new process (itself an expensive operation) which is a duplicate of the calling process along with all memory".
In *N*X, the forked process is mapped to the same memory pages as the parent (i.e. is pretty quick) and is only given new pages as and when the forked process tried to modify any shared pages. This is known as copy on write. This is largely achievable because in UNIX, there is no hard barrier between the parent and forked processes.
In NT, on the other hand, all processes are separated by a barrier enforced by CPU hardware. In NT, the easiest way to spawn a parallel activity which has access to your process' memory and resources, is to create a thread. Threads run within the memory space of the creating process and have access to all of the process' memory and resources.
You can also share data between processes via various forms of IPC, RPC, Named Pipes, mailslots, memory-mapped files but each technique has its own complexities, performance characteristics, etc. Read this for more details.
Because it tries to mimic UNIX, CygWin's 'fork' operation creates a new child process (in its own isolated memory space) and has to duplicate every page of memory in the parent process within the newly forked child. This can be a very costly operation.
Again, if you want to write POSIX code, do so in *N*X, not NT.
How about this
Most of the Unix API is implemented by the POSIX.DLL dynamically loaded (shared) library. Programs linked with POSIX.DLL run under the Win32 subsystem instead of the POSIX subsystem, so programs can freely intermix Unix and Win32 library calls.
From http://en.wikipedia.org/wiki/UWIN
The UWIN environment may be what you're looking for, but note that it is hosted at research.att.com, while UWIN is distributed under a liberal license it is not the GNU license. Also, as it is research for att, and only 2ndarily something that they are distributing for use, there are a lot of issues with documentation.
See more info see my write-up as the last answer for Regarding 'for' loop in KornShell
Hmm main UWIN link is bad link in that post, try
http://www2.research.att.com/sw/download/
Also, You can look at
https://mailman.research.att.com/pipermail/uwin-users/
OR
https://mailman.research.att.com/pipermail/uwin-developers/
To get a sense of the features vs issues.
I hope this helps.
The question really boils down to: Can the Win32 API (as of MSVC20??)
be efficiently used to provide a complete POSIX layer over the Windows
API?
Short answer: No.
"Complete POSIX" means fork(), mmap(), signal() and such, and these are [almost] impossible to implement on NT.
To drive the point home: GNU Hurd has problems with fork() as well, because Hurd kernel is not POSIX.
NT is not POSIX too.
Another difference is persisence:
In POSIX-compliant systems it is possible to create system objects and leave them there. Examples of such objects are named pipes and shared memory objects (shms). You can create a named pipe or a shm, and leave it in the filesystem (or in a special filesystem-like place) where other processes will be able to access it. The downside is that a process might die and fail to clean up after itself, leaving unused objects behind (you know about zombie processes? same thing).
In NT every object is reference-counted, and is destroyed as soon as its last handle is closed. Files are among the few objects that persist.
Symlinks are a filesystem feature, and don't exactly depend on NT kernel, but current implementation (in Vista and later) is incapable of creating object-type-agnostic symlinks. That is, a symlink is either a file or a directory, and must link to either a file or a directory. If the target has wrong type, the symlink won't work. You can give it the right type if the target exists when you create the symlink, but POSIX requires that symlinks may be created without their target existing. I can't imagine a use-case for a symlink that points first to a file, then to a directory, but POSIX says that this should work, and if it doesn't, you're not completely POSIX-compliant. Or if your symlinking API/utility can be given an option that specifies the right type, when target doesn't exist, that also breaks POSIX compatibility.
It is possible to replicate some POSIX features to some degree (such as "integer descriptors from in a single namespace, referencing any I/O object, and being select()able" without sacrificing [much] performance, but that is still a major undertaking, and POSIX interface is really restrictive (that is, if you could just add one more argument to that function, it would have been possible to Do The Right Thing...but you couldn't, unless you want to throw POSIX compliance away).
Your best bet is to not to rely on POSIX features that are difficult to port to non-POSIX systems, or abstract in such a way that lower levels may have separate implementations for different OSes, and upper levels do not care about the details.
I am using pthread win32 library to implement mqueue.
But when it runs into following code, it throw #40 error should be ENOSYS, means system not supported.
pthread_mutexattr_setpshared(&mattr, PTHREAD_PROCESS_SHARED);
i = pthread_mutex_init(&mqhdr->mqh_lock, &mattr);
pthread_mutexattr_destroy(&mattr); /* be sure to destroy */
i is 40 after it goes wrong. Any body has idea about this? or do you have some other alternative solution, like use what kind of WIN32 thread function to replace it.
Note: If anyone successfully implement a mqueue in win32?
Thanks
You will want to read up on Windows interprocess synchronization functions.
For an inter-process mutex in Windows, your choices are to implement your own using shared memory and InterlockedCompareExchange (spin then sleep or watch for Event).
Or easier to program but not as performant is to use the OS provided named Mutex object. These perform about 10 times worse than using CriticalSection within threads of a process.
In my own production code I was porting from Linux pthreads, I played with the first solution, but ended up releasing the code using the Mutex solution. It was more reliable and I was sure it would work in all cases.
I recognize the code you are using ...just comment the 2 lines in the code
pthread_mutexattr_setpshared(&mattr, PTHREAD_PROCESS_SHARED);
pthread_condattr_setpshared(&cattr, PTHREAD_PROCESS_SHARED);
...it works fine as a intra-process message queue ...unless you need it across processes.
I don't know if you feel comfortable hacking inside the Win32 PThread library, but, while the full PTHREAD_PROCESS_SHARED behavior cannot be attained, it IS possible to duplicate handles to kernel objects into other processes using the DuplicateHandle API - so it should be possible to add some windows specific extensions (that would compile out in unix builds) that allow a mutex to be shared between processes.
•A child process created by the CreateProcess function can inherit a handle to a mutex object if the lpMutexAttributes parameter of CreateMutex enabled inheritance. This mechanism works for both named and unnamed mutexes.
•A process can specify the handle to a mutex object in a call to the DuplicateHandle function to create a duplicate handle that can be used by another process. This mechanism works for both named and unnamed mutexes.
•A process can specify a named mutex in a call to the OpenMutex or CreateMutex function to retrieve a handle to the mutex object.
I believe that is Aurelio Medina's code from 2000.
Unfortunately, his test code was a single process, so it didn't care if the PTHREAD_PROCESS_SHARED flag was set or not, since pthreads-32 has never supported it. When he built it in 2000, I bet that pthreads did't even throw an error, so his test code run fine.
Unfortunately for all of us, it seems he died in 2013, so he's not going to finish his opus.
I've taken up the torch and rewrote the mutex/signal handling to use native windows mutex and events. Please look here for the code:
https://github.com/marklakata/mqueue-w32
Global Windows hooks must be in a DLL because the hook is going to be called in the context of a different process, so the hook procedure's code must be injected into that process. However, there are limitations:
SetWindowsHookEx can be used to inject
a DLL into another process. A 32-bit
DLL cannot be injected into a 64-bit
process, and a 64-bit DLL cannot be
injected into a 32-bit process. If an
application requires the use of hooks
in other processes, it is required
that a 32-bit application call
SetWindowsHookEx to inject a 32-bit
DLL into 32-bit processes, and a
64-bit application call
SetWindowsHookEx to inject a 64-bit
DLL into 64-bit processes. The 32-bit
and 64-bit DLLs must have different
names.
For this reason, I'd rather use the low-level hooks WH_MOUSE_LL and WH_KEYBOARD_LL, instead of WH_MOUSE and WH_KEYBOARD. As seen from their documentation:
This hook is called in the context of
the thread that installed it. The call
is made by sending a message to the
thread that installed the hook.
Therefore, the thread that installed
the hook must have a message loop.
This leads me to think that these particular hook procedures do not need to be in a separate DLL, and can just live inside the EXE that hooked them up. The documentation for SetWindowsHookEx, however, says:
lpfn
[in] Pointer to the hook procedure. If the dwThreadId parameter
is zero or specifies the identifier of
a thread created by a different
process, the lpfn parameter must point
to a hook procedure in a DLL.
No explicit exception for the two low-level hooks is mentioned.
I have seen several .NET applications that use the low-level hooks without having their hook procedures in a separate DLL. That is another hint that this is acceptable. However, I'm a bit scared to do this myself since the documentation forbids it.
Does anyone foresee any trouble if I don't use a DLL and just put these low-level hook procedures straight into my EXE?
Edit: For the bounty, I would like a definitive "yes, this is ok, because..." or "no, this can go wrong, because...".
Turns out that this is actually in the documentation. Although not in the documentation of SetWindowsHookEx and friends, but in a .NET knowledge base article.
Low-level hook procedures are called on the thread that installed the hook. Low-level hooks do not require that the hook procedure be implemented in a DLL.
There is one exception to the global hooking function in dll rule. Low level mouse and keyboard hooks are executed in the context of the calling process, not the process being hooked (internally, Windows notifies your hook via a windows message). Therefore the hook code is not executed in an arbitrary process and can be written in .Net. See http://www.codeproject.com/KB/cs/CSLLKeyboardHook.aspx for an example.
For other hooks you do need to call the 32 bit version of SetWindowsHookEx and pass a hook function in a 32bit process and call the 64bit version of SetWindowsHookEx and pass a hook function in a 64bit process, though.
Global hooks, whether low or high level, have to be in a separate DLL that can be loaded into each process. The documentation you quoted makes that pretty clear, and if there was an exception that applied to the low-level hooks, that documentation would say so as well.
Rule of thumb: When the docs say not to do something, there's usually a pretty good reason for it. While it may work in some cases, that fact that it works may be an implementation detail, and subject to change. If that happens, then your code will be broken if the implementation is ever modified.
Edit: I take back my previous answer. It turns out that WH_MOUSE_LL and WH_KEYBOARD_LL are exceptions to the usual rule about global hooks:
What is the HINSTANCE passed to SetWindowsHookEx used for?