I'm writing a package in go that creates a connection to Kafka.
I create the connection in the init() function in the main file in the package. After the program that uses the package stops, I want to call a function called Close()
Is there a way to enforce it in the package level, instead of giving this responsibility to the user?
Assume that the connection should be available throughout the run of the user's program, I don't want to initialize it every time.
As for just about any resource (and a connection is no different), a user should call the functions to return resources to the system after use. Most users understand this across languages and architectures, so I don't see why your package users should have any concern. Go addresses this problem very elegantly by including the defer statement which makes it very easy to provide for releasing a resource.
Related
I want to implement (in C++) a feature, using MPI, in an existing (non-MPI) application. I am thinking of using mpich-3.4.1 for this.
I am planning to create a .so file for that feature, which the original application can link to. I initially thought to have a function in the .so file that starts with an MPI_Init() and ends with MPI_Finalize() and, in between, calls all required MPI apis to do the parallel job. As part of the MPI job, the new feature makes the current application an MPI server by calling APIs like 'MPI_Open_port' and 'MPI_Comm_accept'. Other worker processes (possibly running on different machines) connect to this server, send/receive messages, and complete a heavy computation in parallel. The application then resumes its other non-mpi work.
It seems to me that Singleton MPI_INIT mechanism will be useful for this. I found the following page on Singleton Init:
https://www.mpi-forum.org/docs/mpi-3.1/mpi31-report/node254.htm
This page says, "A high-quality implementation will allow any process (including those not started with a ``parallel application'' mechanism) to become an MPI process by calling MPI_INIT. Such a process can then connect to other MPI processes...".
However, the comments in mpich-3.4.1/src/mpi/init/init.c says, "The MPI standard does not say what a program can do before an 'MPI_INIT' or after an 'MPI_FINALIZE'. In the MPICH implementation, you should do as little as possible. In particular, avoid anything that changes the external state of the program, such as opening files, reading standard input or writing to standard output."
Based on the above comments, it seems we should not have MPI_Init(NULL, NULL) and MPI_Finalize() as part of any implementation in a library. In that case, I am thinking to have the init and finalize APIs in the original application's main function, and have rest of the API calls made from the .so file. My original application is a working large software, and may not need to execute my mpi feature at all, in some situations.
My questions are:
(1) Does it make sense to have MPI_Init(NULL, NULL) and MPI_Finalize() called in the main function of this application, and rest of the MPI functionalities in a .so file?
(2) Once MPI_Init(NULL, NULL) is called in the main, would it interfere with the normal execution of the software in any way? Would there be any performance impact on the existing application?
(3) Is there an MPI implementation that handles this better?
(4) Is MPI a good approach to handle this requirement, or other mechanisms like ZeroMQ better? In the comments made by Wesley Bland in the following link, he says that "MPI may not be right for you if you're looking for a client/server model. Yes, it's possible, but it's not really optimized for that use case and you might have better luck using a different communication mechanism". Is that true in 2022?
client relationship within MPI server
I read here that the inverse is possible, but how does one achieve such a thing? I'd hope there's a simple way, similar to calling from a loaded DLL, but my google research comes up with nothing. My only other thought is that one could pass some predefined constants through WriteFile or DeviceIoControl that the driver parses like a switch statement to execute the relevant function; does a more direct method exist?
The question is why would you want to do it? Generally if you have to rely on some mechanism like this, you need to revisit the design of the application/driver that you are writing.
The correct way to do something in context of your user mode application is exactly what you described. You can do a DeviceIoControl call to your driver and the driver validates all the parameters that you have passed, then carries out the operation on behalf of the user mode call.
If for some reason, you need to call into kernel directly, you will have to resort to undocumented methods. There are ways to hook into kernel dispatch table and overwrite one of the dispatch handler to redirect the call to your function. But I hope you never ever ship anything like this to your customer. This is good for learning how the dispatch table works, etc but introduces several security nightmares. Ultimately your software should not be responsible for someone's machine getting hacked.
I am trying to override the singe instance limit of an application for which I don't have the source. I know that the app is using the good ol' trick of using CreateMutex to determine whether there is another instance running. (If the mutex is created successfully it proceeds, if getlasterror says that the mutex has been created it quits immediately). I found that through sniffing the Win32 api calls.
I thought using Detours would do the trick, but it doesn't quite work out. I am intercepting CreateMutexW, but for some reason, it doesn't catch the first four calls to it. (Again I know what these calls are by sniffing win32 calls and looking at the name of the mutexes). I do get the fifth one intercepted, but the one I actually want to intercept is the first one.
I am using detours through the sample application withdll. I wonder if the problem is that detours is kicking in too late or because of some kind of protection these calls may have. Is detours the best approach? Perhaps using something else may be a better idea?
There might be several reasons for the situation you describe. Here are the most probable of them:
The CreateMutexW call you need to catch occurs within the DllMain
method of one of the DLLs that are imported by the process, and you
are using the DetoursCreateProcessWithDll() function to inject your
code. Detours injects your DLL by placing it at the end of the
process executable import list, and hence all the DLLs that are
imported by the process would be loaded and initialized within the
process prior to yours. In order to overcome this, try using
CreateProcess(CREATE_SUSPENDED) and CreateRemoteThread()-based
injection, although this method raises its own challenges.
The API that is used in the first call is different. Have you tried
overriding CreateMutexExW? Are you sure ANSI methods call Unicode
ones?
Hope this helps.
I am assuming that once a semaphore is created by a process, it will be accessible by any process/user.
Is it possible to put access restrictions on a particular semaphore so that it can be accessible by only certain processes/users or only certain processes can release the semaphore.
I see some problems if we make a semaphore accessible by all processes.Eg: a dummy process can read the semaphore and release the lock at wish making a false signal to the actual process that is really waiting for the semaphore lock.
All these questions are arising as I am getting very weird output with the following code snippet:
use Win32::Semaphore;
$sem = Win32::Semaphore->new(0, 1,"reliance2692")
or print "Can't create semaphore\n";
$sem = Win32::Semaphore->open("reliance2692")
or print "Can't open semaphore\n";
print "Semaphore:" . $sem . "\n";
By running the above program, I am getting the following output
Can't create semaphore
Can't open semaphore
The output shows that its failed to create a semaphore and even failed to open semaphore.
creating a semaphore might have failed if a semaphore already exists with the given name.
I don't understand why opening a semaphore failed.
Can some clarify the scenario where both creating semaphore & opening semaphore fails.
Win32::Semaphore->new calls the Windows API function CreateSemaphore and gets the process's default security descriptor, which usually means that processes running as the same user as your script can have full access whereas processes running as other accounts get no access. So, for starters, your assumption is false.
The name you choose in your Perl code is passed directly to the API function, so it's subject to the same namespace rules as all other Win32 kernel objects.
Win32::Semaphore provides no interface for specifying access restrictions. Even if it did, Windows does not provide per-process permissions. Permissions are attached to the user, not the process.
If you're getting "access denied" from new, then that suggests there's another program running that chose to use that same name for something else — maybe another semaphore, or maybe something else, like an event or a mutex — and that process is running as a different user.
If you're getting "access denied" from open, then, in addition to the possibilities for new, it could be that another process has already opened a semaphore with the same name but has not granted full permissions to other users. Win32::Semaphore->open requests SEMAPHORE_ALL_ACCESS permission.
If the semaphore has already been opened by a process running as the same user, then you should not get "access denied." Neither new nor open should fail in that case, although $^E might hold 183 (ERROR_ALREADY_EXISTS) anyway.
For the record, I'm the author of Win32::Semaphore. As mobrule and Rob have explained, Windows security is user/group based. It's not possible to have a semaphore that only certain processes can access. If any process belonging to a user can access a semaphore, then any process of that user can access that semaphore.
Normally, the default access allows only the current user to access the semaphore. Nobody's ever requested the ability to have Win32::Semaphore specify a non-default security descriptor, and the associated API is non-trivial. If somebody created a module to manage a SECURITY_ATTRIBUTES structure, I'd be happy to add support for that to Win32::Semaphore and the related IPC modules. Win32-Security does not appear to be that module, although it might be a start.
If you need a semaphore to work across multiple users, your only solution right now is to create the semaphore outside of Win32::Semaphore, passing an appropriate SECURITY_ATTRIBUTES pointer. You could do that with a small helper program written in C, or using Inline::C. (Remember that once created, a semaphore exists as long as any process has an open handle to it, so your helper program needs to keep the semaphore handle open until you've called Win32::Semaphore->open on it.)
From Win32::Semaphore pod
$semaphore = Win32::Semaphore->new($initial, $maximum, [$name])
Constructor for a new semaphore object. $initial is the initial count, and $maximum is
the maximum count for the semaphore. If $name is omitted or undef, creates an unnamed
semaphore object.
If $name signifies an existing semaphore object, then $initial and $maximum are ignored
and the object is opened. If this happens, $^E will be set to 183
(ERROR_ALREADY_EXISTS).
If I'm reading this correctly, if your call to Win32::Semaphore->new refers to an existing semaphore, then the new call will open the semaphore as well, and the subsequent open call will be redundant (it's not clear to me from the pod what should happen if you open a sempahore that is already open).
Perhaps you could step through the code, checking the value of $sem as well as $! and $^E at each step.
Additional reply: the Windows API does have methods for setting access control of semaphores, but
they don't appear to be exposed in the Perl Win32::Semaphore module
access control can't be set unless it was already allowed by the other process that created the semaphore
I don't know if you have any good options for this problem. Can you modify the process that creates the semaphore to relax access restrictions? Ask the Win32::Semaphore author to update his module? Try to fix Win32::Semaphore yourself?
MSDN article on CoRevokeGetClassObject() says that when the COM server calls it the class object referenced by clients is not released. Then the following comes:
If other clients still have pointers to the class object and have caused the reference >count to be incremented by calls to IUnknown::AddRef, the reference count will not be >zero. When this occurs, applications may benefit if subsequent calls (with the obvious >exceptions of IUnknown::AddRef and IUnknown::Release) to the class object fail.
What is meant by "applications may benefit"? The class object is not released, but creation requests fail. Sounds reasonable but where's the benefit?
Yeah, it's a pretty strange turn of words...
I think what they're trying to say is that clients may end up in a tricky situation if they create objects from a server that just called CoRevokeClassObjects, because it's likely it'll disappear very soon (CoRevokeClassObjects is routinely called when a server is shut down.)
So, if the activation calls (IClassFactory::CreateInstance) don't fail, the client will get an interface pointer back, and as soon as they call a method on it, they'll get an error from the RPC layer that the server is gone.
I suppose that's 'beneficial' in some way :-)
That said, I'm not sure how to detect the case where IUnknown::Release is called via CoRevokeClassObjects vs some other client, but I suppose the code revoking the factories could set some global state or per-factory state that they can check before letting creation requests come through.