In my C++ program I have a class CEvent with trigger and wait member functions based on pthreads (running on Linux). Implementation is quite obvious (i.e. many examples online) if there is one waiting process. However now I need to satisfy the requirement that multiple threads are waiting on the event and should ALL wake up reliably when trigger() is called. As a second condition, only threads that were waiting when trigger() was called should wake up.
My current code:
void CEvent::trigger() {
pthread_mutex_lock(&mutex);
wakeUp = true;
pthread_cond_broadcast(&condition)
pthread_mutex_unlock(&mutex);
wakeUp = false;
}
void CEvent::wait() {
pthread_mutex_lock(&mutex);
while (!wakeUp)
pthread_cond_wait(&condition, &mutex)
pthread_mutex_unlock(&mutex);
}
This seems to almost work, insofar as that all threads waiting wake up before I set wakeUp back to false. However between the broadcast and the reset of wakeUp, other (or the same) threads calling wait() will also wake up right away which is not acceptable. Putting wakeUp = false before the mutext unlocking prevents the threads from waking up.
My questions:
* When does pthread_cond_broadcast return? I.e. is there a guarantee it will only return after all threads have woken up or could it return before?
* Are there any recommended solutions to this problem?
Please disregard my previous bogus answer. There is a race between the time that the trigger thread unlocks the mutex (and thus frees the waiting threads), and then sets the wakeUp value. This means that another (not waiting) thread can come in, grab the mutex, and see a true value in wakeUp and exit without waiting. Another bug is that a thread which was waiting will wake up after wakeUp is reset and immediately resume waiting.
One way to resolve this is to use count - each thread that is waiting will increment the count, then the trigger will wait until that many threads have woken before resuming. You would then have to ensure that non-waiting threads are not allowed to start waiting until this had happened.
// wake up "waiters" count of waiting threads
void CEvent::trigger()
{
pthread_mutex_lock(&mutex);
// wakey wakey
wakeUp = true;
pthread_cond_broadcast(&condition);
// wait for them to awake
while (waiters>0)
pthread_cond_wait(&condition, &mutex);
// stop waking threads up
wakeUp = false;
// let any "other" threads which were ready to start waiting, do so
pthread_cond_broadcast(&condition);
pthread_mutex_unlock(&mutex);
}
// wait for the condition to be notified for us
void CEvent::wait()
{
pthread_mutex_lock(&mutex);
// wait for us to be allowed to start waiting
// we have to wait until any currrently being woken threads have gone
while (wakeUp)
pthread_cond_wait(&condition, &mutex);
// our turn to start waiting
waiters ++;
// waiting
while (!wakeUp)
pthread_cond_wait(&condition, &mutex);
// finished waiting, we were triggered
waiters --;
// let the trigger thread know we're done
pthread_cond_broadcast(&condition);
pthread_mutex_unlock(&mutex);
}
Related
I am trying to write a simple script that spawns a thread that performs a task that may timeout. (For the sake of writing a simple example for StackOverflow I replaced the actual process with a sleep command).
This program spawns a thread and then uses a cond_timedwait to monitor the thread and check if it has timed out. If a timeout occurs it calls the kill method on the thread with a "STOP" signal to notify the thread that it should exit.
use strict;
use threads;
use threads::shared;
use warnings;
my $var :shared;
my $thread = threads->create(sub {
# Tell the thread how to handle the STOP signal
local $SIG{'STOP'} = sub {
print "Stop signal received\n";
threads->exit();
};
# Perform a process that takes some time
sleep 10;
# Signal that the thread is complete
lock($var); cond_signal($var);
});
# Current time + 1 second
my $wait_time = time() + 1;
my $timeout;
{
# Wait for the thread to complete or until a timeout has occurred
lock($var); $timeout = !cond_timedwait($var, $wait_time);
}
# Check if a timeout occurred
if ($timeout) {
print "A timeout has occurred\n";
# Signal the thread to stop
$thread->kill('STOP')->join();
}
else {
$thread->join();
}
This code runs successfully and prints the following output:
1 second passes...
A timeout has occurred
9 seconds pass...
Stop signal received
The problem is, even though a timeout is detected and the "STOP" signal is sent to the thread, the program still seems to be waiting the full 10 seconds before printing "Stop signal received" and exiting.
I tried changing it so it calls detach instead of join after killing the thread, but then the "Stop signal received" message is never printed which means the program is exiting before the thread cleanly exits. I want to make sure the thread is actually interrupted and exits, because in the real program I will want to kill and retry the process after the timeout has occurred and the process won't work if there is another instance already running on a detached thread.
How can I make it so the thread instantly prints the message and exits when it receives the "STOP" signal?
These "signals" aren't the actual OS signals, and there are operations they won't interrupt
CAVEAT: The thread signalling capability provided by this module does not actually send signals via the OS. It emulates signals at the Perl-level such that signal handlers are called in the appropriate thread. For example, sending $thr->kill('STOP') does not actually suspend a thread (or the whole process), but does cause a $SIG{'STOP'} handler to be called in that thread (as illustrated above).
...
Correspondingly, sending a signal to a thread does not disrupt the operation the thread is currently working on: The signal will be acted upon after the current operation has completed. For instance, if the thread is stuck on an I/O call, sending it a signal will not cause the I/O call to be interrupted such that the signal is acted up immediately.
The granularity of what the "operation" is isn't stated but sleep is clearly uninterruptable so the signal handler runs only after it completes. With a different job to interrupt
use warnings;
use strict;
use feature 'say';
use threads;
say "Start at ", scalar localtime, " (", time, ")";
my $thread = threads->create(sub {
# Tell the thread how to handle the STOP signal
$SIG{'STOP'} = sub {
say "\tStop signal received. Exiting at ", time;
threads->exit();
};
say "\tIn the thread ", threads->tid;
# Perform a process that takes some time
#sleep 10;
do { sleep 1; say "\tnappin'... ($_ sec)" } for 1..10;
});
sleep 3;
$thread->kill('STOP')->join(); # works differently with detach()
say "Main thread done, exiting at ", time;
Output
Start at Thu Jul 7 11:11:27 2022 (1657217487)
In the thread 1
nappin'... (1 sec)
nappin'... (2 sec)
Stop signal received. Exiting at 1657217490
Main thread done, exiting at 1657217490
With detach instead of join it still stops that loop at the right time but I see no indication that a signal handler ran. (In my tests I have the signal handler also write a file and with detach it doesn't.) It all works the same for me with a shared variable etc, like in the question.
This sleep doesn't matter of course -- but it is all a warning to carefully test with actual jobs that the signal is aimed to stop.
Signals can only be sent to processes. As such, $thread->kill('STOP') can't possibly be sending an actual signal. As such, nothing interrupts sleep.
Between each statement, Perl checks if a "signal" came in. If it has, it handles it. So the "signal" is only handled once sleep completes.
If you had ten one second sleeps instead of one ten second sleep, the wait would be at most one second.
I have window service that polls a web service for new items every 30 seconds. If it finds any new items, it checks to see if they need to be "processed" and then puts them in a list to process. I spawn off different threads to process 5 at a time, and when one finishes, another one will fill the empty slot. Once everything has finished, the program sleeps for 30 seconds and then polls again.
My issue is, while the items are being processed(which could take up to 15 minutes), new items are being created which also may need to be processed. My problem is the main thread gets held up waiting for every last thread to finish before it sleeps and starts the process all over.
What I'm looking to do is have the main thread continue to poll the web service every 30 seconds, however instead of getting held up, add any new items it finds to a list, which would be processed in a separate worker thread. In that worker thread, it would still have say only 5 slots available, but they would essentially always all be filled, assuming the main thread continues to find new items to process.
I hope that makes sense. Thanks!
EDIT: updated code sample
I put together this as a worker thread that operates on a ConcurrentQueue. Any way to improve this?
private void ThreadWorker() {
DateTime dtStart = DateTime.Now;
int iNumOfConcurrentSlots = 6
Thread[] threads = new Thread[iNumOfConcurrentSlots];
while (true) {
for (int i = 0; i < m_iNumOfConcurrentSlots; i++) {
if (m_tAssetQueue.TryDequeue(out Asset aa)) {
threads[i] = new Thread(() => ProcessAsset(aa));
threads[i].Start();
Thread.Sleep(500);
}
}
}
}
EDIT: Ahh yeah that won't work above. I need a way of being able to not hard code the number of ConcurrentSlots, but have each thread basically waiting and looking for something in the Queue and if it finds it, process it. But then I also need a way of signalling that the ProcessAsset() function has completed to release the thread and allow another thread to be created....
One simple way to do it is to have 5 threads reading from a concurrent queue. The main thread queues items and the worker threads do blocking reads from the queue.
Note: The workers are in an infinite loop. They call TryDequeue, process the item if they got one or sleep one second if they fail to get something. They can also check for an exit flag.
To have your service property behaved, you might have an independent polling thread that queues the items. The main thread is kept to respond to start, stop, pause requests.
Pseudo code for worker thread:
While true
If TryDequeue then
process data
If exit flag is true, break
While pause flag, sleep
Sleep
Pseudo code for polling thread:
While true
Poll web service
Queue items in concurrent queue
If exit flag true, break
While pause flag, sleep
Sleep
Pseudo code for main thread:
Start polling thread
Start n worker threads with above code
Handle stop:
set exit flag to true
Handle pause
set pause flag to true
I am curious about the event parameter that gets passed to IOLockWakeup and IOLockSleep{Deadline}.
i understand that the event is an address that gets passed to both functions. i am assuming this address is used to essentially notify the thread.
so my question is: assuming i is an int, and we are using its address, how do these functions know when to sleep and wakeup?
is the assumption that:
when IOLockWakeup is called, that the contents of event are 0 (which it then changes to a non-zero value), and
when IOLockSleepDeadline is called, that the contents of the event were 0 at the time it was called, and it will stop sleeping because the contents are nonzero
and when we keep calling these functions (in a workloop context) are the contents of the event parameter automatically set to zero when iolocksleep* is called (and when it wakes up), since iolockwakeup presumably changs this to a nonzero value?
You'll notice that the event parameter is of type void*, not int*:
int IOLockSleep( IOLock * lock, void *event, UInt32 interType);
The event parameter is an arbitrary pointer, it’s never dereferenced, and it doesn’t matter what’s stored there, it's used purely for identification purposes: so for example don't pass NULL, because that's not a unique value.
IOLockSleep always suspends the running thread, and IOLockWakeup wakes up any thread that’s sleeping on that address. If no such thread is waiting, nothing at all happens. This is why you’ll usually want to pair the sleep/wakeup with some condition that’s protected by the lock, and send the wakeup while holding the lock - the thing to avoid is going to sleep after the wakeup was sent, in which case your sleeping thread might sleep forever.
So, you'll have some condition for deciding whether or not to sleep, and you'll update that condition before calling wakeup, while holding the lock:
IOLock* myLock;
bool shouldSleep;
…
// sleep code:
IOLockLock(myLock);
while (shouldSleep)
{
IOLockSleep(myLock, &shouldSleep, THREAD_UNINT);
}
IOLockUnlock(myLock);
…
// wakeup code:
IOLockLock(myLock);
shouldSleep = false;
IOLockWakeup(myLock, &shouldSleep, true /* or false, if we want to wake up multiple sleeping threads */);
IOLockUnlock(myLock);
Here, I've used the address of shouldSleep for the event parameter, but this could be anything, it's just convenient to use this because I know no other kext will be using that pointer, as no other kext has access to that variable.
I have the classic IOCP callback that dequeues i/o pending requests, process them, and deallocate them, in this way:
struct MyIoRequest { OVERLAPPED o; /* ... other params ... */ };
bool is_iocp_active = true;
DWORD WINAPI WorkerProc(LPVOID lpParam)
{
ULONG_PTR dwKey;
DWORD dwTrans;
LPOVERLAPPED io_req;
while(is_iocp_active)
{
GetQueuedCompletionStatus((HANDLE)lpParam, &dwTrans, &dwKey, (LPOVERLAPPED*)&io_req, WSA_INFINITE);
// NOTE, i could use GetQueuedCompletionStatusEx() here ^ and set it in the
// alertable state TRUE, so i can wake up the thread with an ACP request from another thread!
printf("dequeued an i/o request\n");
// [ process i/o request ]
...
// [ destroy request ]
destroy_request(io_req);
}
// [ clean up some stuff ]
return 0;
}
Then, in the code I will have somewhere:
MyIoRequest * io_req = allocate_request(...params...);
ReadFile(..., (OVERLAPPED*)io_req);
and this just works perfectly.
Now my question is: What about I want to immediately close the IOCP queue without causing leaks? (e.g. application must exit)
I mean: if i set is_iocp_active to 'false', the next time GetQueuedCompletionStatus() will dequeue a new i/o request, that will be the last i/o request: it will return, causing thread to exit and when a thread exits all of its pending i/o requests are simply canceled by the system, according to MSDN.
But the structures of type 'MyIoRequest' that I have instanced when calling ReadFile() won't be destroyed at all: the system has canceled pending i/o request, but I have to manually destroy those structures I have
created, or I will leak all pending i/o requests when I stop the loop!
So, how I could do this? Am I wrong to stop the IOCP loop with just setting that variable to false? Note that is would happen even if i use APC requests to stop an alertable thread.
The solution that come to my mind is to add every 'MyIoRequest' structures to a queue/list, and then dequeue them when GetQueuedCompletionStatusEx returns, but shouldn't that make some bottleneck, since the enqueue/dequeue process of such MyIoRequest structures must be interlocked? Maybe I've misunderstood how to use the IOCP loop. Can someone bring some light on this topic?
The way I normally shut down an IOCP thread is to post my own 'shut down now please' completion. That way you can cleanly shut down and process all of the pending completions and then shut the threads down.
The way to do this is to call PostQueuedCompletionStatus() with 0 for num bytes, completion key and pOverlapped. This will mean that the completion key is a unique value (you wont have a valid file or socket with a zero handle/completion key).
Step one is to close the sources of completions, so close or abort your socket connections, close files, etc. Once all of those are closed you can't be generating any more completion packets so you then post your special '0' completion; post one for each thread you have servicing your IOCP. Once the thread gets a '0' completion key it exits.
If you are terminating the app, and there's no overriding reason to not do so, (eg. close DB connections, interprocess shared memory issues), call ExitProcess(0).
Failing that, call CancelIO() for all socket handles and process all the cancelled completions as they come in.
Try ExitProcess() first!
According to MSDN:
The WaitForSingleObject function can wait for the following objects:
Change notification
Console input
Event
Memory resource notification
Mutex
Process
Semaphore
Thread
Waitable timer
Then we can use WaitForSingleObject to make the parent-thread wait for child ones.
int main()
{
HANDLE h_child_thread = CreateThread(0,0, child, 0,0,0); //create a thread in VC
WaitForSingleObject(h_child_thread, INFINITE); //So, parent-thread will wait
return 0;
}
Question
Is there any other way to make parent-thread wait for child ones in VC or Windows?
I don't quite understand the usage of WaitForSingleObject here, does it mean that the thread's handle will be available when the thread terminates?
You can establish communication between threads in multiple ways and the terminating thread may somehow signal its waiting thread. It could be as simple as writing some special value to a shared memory location that the waiting thread can check. But this won't guarantee that the terminating thread has terminated when the waiting thread sees the special value (ordering/race conditions) or that the terminating thread terminates shortly after that (it can just hang or block on something) and it won't guarantee that the special value gets ever set before the terminating thread actually terminates (the thread can crash). WaitForSingleObject (and its companion WaitForMultipleObjects) is a sure way to know of a thread termination when it occurs. Just use it.
The handle will still be available in the sense that its value won't be gone. But it is practically useless after the thread has terminated, except you need this handle to get the thread exit code. And you still need to close the handle in the end. That is unless you're OK with handle/memory leaks.
for the first queation - yes. The method commonly used here is "Join". the usage is language dependant.
In .NET C++ you can use the Thread's Join method. this is from the msdn:
Thread* newThread = new Thread(new ThreadStart(0, Test::Work));
newThread->Start();
if(newThread->Join(waitTime + waitTime))
{
Console::WriteLine(S"New thread terminated.");
}
else
{
Console::WriteLine(S"Join timed out.");
}
Secondly, the thread is terminated when when you are signaled with "WaitForSingleObject" but the handle is still valid (for a terminated thread). So you still need to explicitly close the handle with CloseHandle.