I tried to write a simple producer/consumer by using condition_variable,
include <iostream>
#include <thread>
#include <condition_variable>
#include <mutex>
#include <chrono>
#include <queue>
#include <chrono>
using namespace std;
condition_variable cond_var;
mutex m;
int main()
{
int c = 0;
bool done = false;
cout << boolalpha;
queue<int> goods;
thread producer([&](){
for (int i = 0; i < 10; ++i) {
m.lock();
goods.push(i);
c++;
cout << "produce " << i << endl;
m.unlock();
cond_var.notify_one();
this_thread::sleep_for(chrono::milliseconds(100));
}
done = true;
cout << "producer done." << endl;
cond_var.notify_one();
});
thread consumer([&](){
unique_lock<mutex> lock(m);
while(!done || !goods.empty()){
/*
cond_var.wait(lock, [&goods, &done](){
cout << "spurious wake check" << done <<endl;
return (!goods.empty() || done);
});
*/
while(goods.empty())
{
cout<< "consumer wait" <<endl;
cout<< "consumer owns lock " << lock.owns_lock() <<endl;
cond_var.wait(lock);
}
if (!goods.empty()){
cout << "consume " << goods.front()<<endl;
goods.pop();
c--;
}
}
});
producer.join();
consumer.join();
cout << "Net: " << c << endl;
}
The problem I have now is when the consumer consumes the last item before the producer set done to true, the consumer thread will stuck in
while(goods.empty())
{
cout<< "consumer wait" <<endl;
cout<< "consumer owns lock " << lock.owns_lock() <<endl;
cond_var.wait(lock);
}
My understanding is cond_var.wait(lock) will wake up spuriously and thus exit the while(good.empty()) loop, but it seems not the case?
Spurious wakeups are not a regular occurance which you can rely on to break a loop in the manner you're suggesting. The risk of having a spurious wakeup is an unfortunate side-effect of the current implementations of condition variables which you must account for, but there is no guarantee about when (if ever) you will experience a spurious wakeup.
If you want to ensure that the consumer thread doesn't get stuck waiting for a notify that never comes, you might try using std::condition_variable::wait_for() instead. It takes a duration and will timeout and reaquire the lock if the duration expires. It might be viewed as closer to a busy wait but if the timeout is long enough the implications on performance should be negligible.
As #Karlinde says, and as the name implies, spurious wakeups are not guaranteed to happen. Rather, they will normally not happen at all.
But, even if spurious wakeups would happen, that would not fix your issue: you simply have an infinite loop in your program. Once the producer has stopped, goods.empty() is true and it will never change again. So change the while loop to:
while(!done && goods.empty())
{
...
}
Now it should exit... most of the time. You still have a possible race condition, because in the producer, you set done = true without holding the lock.
If the Producer notifies {cond_var.notify_one();} without any consumer waiting {cond_var.wait(lock);} then the 1'st notification that is sent to the consumer has gone unnoticed.
#tesla1060
"The problem I have now is when the consumer consumes the last item before the producer set done to true, the consumer thread will stuck in" , this is not ture. The fact is that the Consumer has not received any notification from the Producer (as it has missed one notification (the 1'st one)).
Related
Problem criteria:
my service is Windows-only, so portability is not a constraint for me
my service uses threadpools with overlapped I/O
my service needs to open a connection to a remote service, ask a question and receive a reply
the remote service may refuse to answer (root cause is not important)
The solution is trivial to describe: set a timeout on the read.
The implementation of said solution has been elusive.
I think I may have finally tracked down something that is viable, but I am so weary from false starts that I seek someone's approval who has done this sort of thing before before moving ahead with it.
By calling GetOverlappedResultsEx with a non-zero timeout:
https://learn.microsoft.com/en-us/windows/win32/api/ioapiset/nf-ioapiset-getoverlappedresultex
If dwMilliseconds is nonzero, and an I/O completion routine or APC is queued, GetLastError returns WAIT_IO_COMPLETION.
If dwMilliseconds is nonzero and the specified timeout interval elapses, GetLastError returns WAIT_TIMEOUT.
Thus, I can sit and wait until IO has been alerted or the timeout exceeded and react accordingly:
WAIT_TIMEOUT: CancelIoEx on the overlapped structure from the WSARecv, which will trigger my IO complete callback and allow me to do something meaningful (e.g. force the socket closed).
WAIT_IO_COMPLETION: Do nothing. Timeout need not be enforced.
Is it really that simple, though? Because I have yet to find any questions or example code, etc. that closely resembles what I got going on here (which is largely based on a codebase I inherited) and as a consequence, have failed to find any examples/suggestions to support that this is appropriate.
Demo program: https://github.com/rguilbault-mt/rguilbault-mt/blob/main/WinSock.cpp
to run:
-p -d -t -gor
Make the read delay > timeout to force the timeout condition.
Relevant bits for this question:
StartThreadpoolIo(gIoTp[s]);
if (WSARecv(s, bufs, 1, &readBytes, &dwFlags, &ioData->ol, NULL) == SOCKET_ERROR)
{
std::lock_guard<std::mutex> log(gIoMtx);
switch (WSAGetLastError())
{
case WSA_IO_PENDING:
std::cout << preamble(__func__) << "asynchronous" << std::endl;
break;
default:
std::cerr << preamble(__func__) << "WSARecv() failed: " << WSAGetLastError() << std::endl;
CancelThreadpoolIo(gIoTp[s]);
return false;
}
}
else
{
std::lock_guard<std::mutex> log(gIoMtx);
std::cout << preamble(__func__) << "synchronous - " << readBytes << " read" << std::endl;
}
if (gGetOverlappedResult)
{
{
std::lock_guard<std::mutex> log(gIoMtx);
std::cout << preamble(__func__) << "wait until I/O occurs or we timeout..." << std::endl;
}
DWORD bytesTransferred = 0;
if (!GetOverlappedResultEx((HANDLE)s, &ioData->ol, &bytesTransferred, gTimeout, true))
{
DWORD e = GetLastError();
std::lock_guard<std::mutex> log(gIoMtx);
switch (e)
{
case WAIT_IO_COMPLETION:
std::cout << preamble(__func__) << "read activity is forthcoming" << std::endl;
break;
case WAIT_TIMEOUT:
// we hit our timeout, cancel the I/O
CancelIoEx((HANDLE)s, &ioData->ol);
break;
default:
std::cerr << preamble(__func__) << "GetOverlappedResult error is unhandled: " << e << std::endl;
}
}
else
{
std::lock_guard<std::mutex> log(gIoMtx);
std::cerr << preamble(__func__) << "GetOverlappedResult success: " << bytesTransferred << std::endl;
}
}
Confirmation/other suggestions welcomed/appreciated.
I was debating what the proper protocol was and decided I'm just going to answer my own question for the benefit of the world (if anyone bumps into my similar criteria/issue) even though I would have preferred that #HansPassant get credit for the answer.
Anyway, with his suggestion, using the wait mechanism provided by Microsoft allows me to pull of what I need without orchestrating any thread-based monitoring of my own. Here are the relevant bits:
after calling WSARecv, register a wait callback:
else if (gRegisterWait)
{
if (!RegisterWaitForSingleObject(&ioData->waiter, (HANDLE)s, waitOrTimerCallback, ioData, gTimeout, WT_EXECUTEONLYONCE))
{
std::lock_guard<std::mutex> log(gIoMtx);
std::cerr << preamble(__func__) << "RegisterWaitForSingleObject failed: " << GetLastError() << std::endl;
}
else
{
std::lock_guard<std::mutex> log(gIoMtx);
std::cout << preamble(__func__) << "RegisterWaitForSingleObject success: " << ioData->waiter << std::endl;
}
}
when the wait callback is invoked, use the second parameter to decide if the callback was called because of a timeout (true) or other signal (false):
VOID CALLBACK waitOrTimerCallback(
PVOID lpParameter,
BOOLEAN TimedOut
)
{
IoData* ioData = (IoData*)lpParameter;
{
std::lock_guard<std::mutex> log(gIoMtx);
std::cout << preamble(__func__) << (TimedOut ? "true" : "false") << std::endl;
std::cout << "\tSocket: " << ioData->socket << std::endl;
}
if (!TimedOut)
{
std::lock_guard<std::mutex> log(gIoMtx);
std::cout << preamble(__func__) << "read activity is forthcoming" << std::endl;
}
else
{
// we hit our timeout, cancel the I/O
CancelIoEx((HANDLE)ioData->socket, &ioData->ol);
std::lock_guard<std::mutex> log(gIoMtx);
std::cout << preamble(__func__) << "timeout reached, cancelling I/O" << std::endl;
}
// need to unregister the waiter but not supposed to do it in the callback
if (!TrySubmitThreadpoolCallback(unregisterWaiter, &ioData->waiter, NULL))
{
std::lock_guard<std::mutex> log(gIoMtx);
std::cerr << preamble(__func__) << "failed to unregister waiter...does this mean I have a memory leak?" << std::endl;
}
}
per the recommendations of the API:
https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-registerwaitforsingleobject
When the wait is completed, you must call the UnregisterWait or UnregisterWaitEx function to cancel the wait operation. (Even wait operations that use WT_EXECUTEONLYONCE must be canceled.) Do not make a blocking call to either of these functions from within the callback function.
submit the unregistering of the waiter to the threadpool to be dealt with outside of the callback:
VOID CALLBACK unregisterWaiter(
PTP_CALLBACK_INSTANCE Instance,
PVOID Context
)
{
PHANDLE pWaitHandle = (PHANDLE)Context;
{
std::lock_guard<std::mutex> log(gIoMtx);
std::cout << preamble(__func__) << std::endl;
std::cout << "\Handle: " << (HANDLE)*pWaitHandle << std::endl;
}
if (!UnregisterWait(*pWaitHandle))
{
std::lock_guard<std::mutex> log(gIoMtx);
std::cerr << preamble(__func__) << "UnregisterWait failed: " << GetLastError() << std::endl;
}
}
Managing the pointer to the handle created needs to be accounted for, but I think you can tuck it into the structure wrapping the overlapped IO and then pass the pointer to your wrapper around. Seems to work fine. The documentation makes no indication of whether I'm on the hook for freeing anything, so I assume that is why we're required to call the UnregisterWait function regardless of whether we're only executing once, etc. That detail can be considered outside the scope of the question.
Note, for others' benefit, I've updated the github link from my question with the latest version of the code.
I'm thinking of coding something up that will change a laptop's refresh rate based on whether or not the device is plugged in.
From my research, these are two links I came across. One is 20 years old and the other is from Microsoft, but I don't see any mentions of refresh rate specifically.
https://www.codeproject.com/Articles/558/Changing-your-monitor-s-refresh-rate
https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-changedisplaysettingsa?redirectedfrom=MSDN
Does anyone have any insight into how to do this? I'm not too particular about what language would have to be used for it, so let me know whatever would be most viable. Of course I'd also have to be able to check a change in state for plugged in/unplugged, but I haven't gotten to that point yet.
I'm mostly targeting Windows 10 since that's what my device is on.
You can use EnumDisplaySettings to enumerate the information of the current display device, and then set the display by ChangeDisplaySettingsA.
If you want to modify the refresh rate, you only need to modify the dmDisplayFrequency parameter of DEVMODEA.
Here is the sample:
#include <Windows.h>
#include <iostream>
using namespace std;
int main(int argc, const char* argv[])
{
DEVMODE dm;
ZeroMemory(&dm, sizeof(dm));
dm.dmSize = sizeof(dm);
if (0 != EnumDisplaySettings(NULL, ENUM_CURRENT_SETTINGS, &dm))
{
cout << "DisplayFrequency before setting = " << dm.dmDisplayFrequency << endl;
dm.dmDisplayFrequency = 60; //set the DisplayFrequency
LONG ret = ChangeDisplaySettingsEx(NULL, &dm, NULL, 0, NULL);
std::cout << "ChangeDisplaySettingsEx returned " << ret << '\n';
if (0 != EnumDisplaySettings(NULL, ENUM_CURRENT_SETTINGS, &dm))
{
cout << "DisplayFrequency after setting = " << dm.dmDisplayFrequency << endl;
}
switch (ret)
{
case DISP_CHANGE_SUCCESSFUL:
std::cout << "display successfully changed\n";
break;
case DISP_CHANGE_BADDUALVIEW:
std::cout << "The settings change was unsuccessful because the system is DualView capable\n";
break;
case DISP_CHANGE_BADFLAGS:
std::cout << "An invalid set of flags was passed in.\n";
break;
case DISP_CHANGE_BADMODE:
std::cout << "The graphics mode is not supported.\n";
break;
case DISP_CHANGE_BADPARAM:
std::cout << "An invalid parameter was passed in. This can include an invalid flag or combination of flags.\n";
break;
case DISP_CHANGE_FAILED:
std::cout << "The display driver failed the specified graphics mode.\n";
break;
case DISP_CHANGE_NOTUPDATED:
std::cout << "Unable to write settings to the registry.\n";
break;
case DISP_CHANGE_RESTART:
std::cout << "The computer must be restarted for the graphics mode to work.\n";
break;
}
}
system("pause");
}
This example is not always successful. Whether you can modify the refresh rate depends on whether your monitor supports it. This is the output of successful setup:
Say I have a PUB server that zmq_send()'s realtime messages to SUB client. If client is busy and can not zmq_recv() messages quick enough, then messages will be buffered in client (and/or server).
If the buffer grows too large (high water mark) then NEW messages will be dropped. For realtime messages this is the opposite of what one wants. OLD messages should be dropped to make place for NEW ones.
Is there some way to do this?
Ideally I would like the SUB client's receive queue to be either empty or contain the most recent message only. When a new message is received it would replace the old one. ( I guess the problem here would be that the client would block on zmq_recv() when the queue is empty, wasting time doing so. )
So how are realtime feeds usually implemented in ZeroMQ?
I'll answer my own question here. The setting ZMQ_CONFLATE "Keep only last message" seemed promising but it doesn't work with subscription filters. It only ever keeps one message in the queue. If you have more than one filter, both old and new messages of the other filters type gets thrown away.
Likewise the recommendation of the zeromq guide to simply to kill slow subscribers, but that doesn't seem like realistic solution. Having subscribers with different read speeds, subscribed to the same fast publisher, should be a normal use case. Some of these subscribers might live on slow computers others on fast ones, etc. ZeroMQ should be able to handle that somehow.
http://zguide.zeromq.org/page:all#Slow-Subscriber-Detection-Suicidal-Snail-Pattern
I ended up doing manual dropping of old queued up messages on the client side. It seems to work fine. I get subscribed messages to the client that are less than 3ms old (through tcp localhost) that way. This works even in cases where I have five thousand, 10 second old messages, in the queue in front of those few real-time message at the back. This is good enough for me.
I cant help but think this is something that should be provided by the library. It could probably do a better job of it.
Anyways here is the client side, old message dropping, code:
bool Empty(zmq::socket_t& socket) {
bool ret = true;
zmq::pollitem_t poll_item = { socket, 0, ZMQ_POLLIN, 0 };
zmq::poll(&poll_item, 1, 0); //0 = no wait
if (poll_item.revents & ZMQ_POLLIN) {
ret = false;
}
return ret;
}
std::vector<std::string> GetRealtimeSubscribedMessageVec(zmq::socket_t& socket_sub, int timeout_ms)
{
std::vector<std::string> ret;
struct MessageTmp {
int id_ = 0;
std::string data_;
boost::posix_time::ptime timestamp_;
};
std::map<int, MessageTmp> msg_map;
int read_msg_count = 0;
int time_in_loop = 0;
auto start_of_loop = boost::posix_time::microsec_clock::universal_time();
do {
read_msg_count++;
//msg format sent by publisher is: filter, timestamp, data
MessageTmp msg;
msg.id_ = boost::lexical_cast<int>(s_recv(socket_sub));
msg.timestamp_ = boost::posix_time::time_from_string(s_recv(socket_sub));
msg.data_ = s_recv(socket_sub);
msg_map[msg.id_] = msg;
auto now = boost::posix_time::microsec_clock::universal_time();
time_in_loop = (now - start_of_loop).total_milliseconds();
if (time_in_loop > timeout_ms) {
std::cerr << "Timeout reached. Publisher is probably sending messages quicker than we can drop them." << std::endl;
break;
}
} while ((Empty(socket_sub) == false));
if (read_msg_count > 1) {
std::cout << "num of old queued up messages dropped: " << (read_msg_count - 1) << std::endl;
}
for (const auto &pair: msg_map) {
const auto& msg_tmp = pair.second;
auto now = boost::posix_time::microsec_clock::universal_time();
auto message_age_ms = (now - msg_tmp.timestamp_).total_milliseconds();
if (message_age_ms > timeout_ms) {
std::cerr << "[SUB] Newest message too old. f:" << msg_tmp.id_ << ", age: " << message_age_ms << "ms, s:" << msg_tmp.data_.size() << std::endl;
}
else {
std::cout << "[SUB] f:" << msg_tmp.id_ << ", age: " << message_age_ms << "ms, s:" << msg_tmp.data_.size() << std::endl;
ret.push_back(msg_tmp.data_);
}
}
return ret;
}
I don't see this program having any practical usage, but while experimenting with c++ 11 concurrency and conditional_variables I stumbled across something I don't fully understand.
At first I assumed that using notify_one() would allow the program below to work. However, in actuality the program just froze after printing one. When I switched over to using notify_all() the program did what I wanted it to do (print all natural numbers in order). I am sure this question has been asked in various forms already. But my specific question is where in the doc did I read wrong.
I assume notify_one() should work because of the following statement.
If any threads are waiting on *this, calling notify_one unblocks one of the waiting threads.
Looking below only one of the threads will be blocked at a given time, correct?
class natural_number_printer
{
public:
void run()
{
m_odd_thread = std::thread(
std::bind(&natural_number_printer::print_odd_natural_numbers, this));
m_even_thread = std::thread(
std::bind(&natural_number_printer::print_even_natural_numbers, this));
m_odd_thread.join();
m_even_thread.join();
}
private:
std::mutex m_mutex;
std::condition_variable m_condition;
std::thread m_even_thread;
std::thread m_odd_thread;
private:
void print_odd_natural_numbers()
{
for (unsigned int i = 1; i < 100; ++i) {
if (i % 2 == 1) {
std::cout << i << " ";
m_condition.notify_all();
} else {
std::unique_lock<std::mutex> lock(m_mutex);
m_condition.wait(lock);
}
}
}
void print_even_natural_numbers()
{
for (unsigned int i = 1; i < 100; ++i) {
if (i % 2 == 0) {
std::cout << i << " ";
m_condition.notify_all();
} else {
std::unique_lock<std::mutex> lock(m_mutex);
m_condition.wait(lock);
}
}
}
};
The provided code "works" correctly and gets stuck by design. The cause is described in the documentation
The effects of notify_one()/notify_all() and
wait()/wait_for()/wait_until() take place in a single total order, so
it's impossible for notify_one() to, for example, be delayed and
unblock a thread that started waiting just after the call to
notify_one() was made.
The step-by-step logic is
The print_odd_natural_numbers thread is started
The print_even_natural_numbers thread is started also.
The m_condition.notify_all(); line of print_even_natural_numbers is executed before than the print_odd_natural_numbers thread reaches the m_condition.wait(lock); line.
The m_condition.wait(lock); line of print_odd_natural_numbers is executed and the thread gets stuck.
The m_condition.wait(lock); line of print_even_natural_numbers is executed and the thread gets stuck also.
I have been working on implementing a half duplex serial driver by learning from a basic serial terminal example using boost::asio::basic_serial_port:
http://lists.boost.org/boost-users/att-41140/minicom.cpp
I need to read asynchronously but still detect when the handler is finished in the main thread so I pass async_read_some a callback with several additional reference parameters in a lambda function using boost:bind. The handler never gets invoked but if I replace the async_read_some function with the read_some function it returns data without an issue.
I believe I'm satisfying all of the necessary requirements for this function to invoke the handler because they are the same for the asio::read some function which returns:
The buffer stays in scope
One or more bytes is received by the serial device
The io service is running
The port is open and running at the correct baud rate
Does anyone know if I'm missing another assumption unique to the asynchronous read or if I'm not setting up the io_service correctly?
Here is an example of how I'm using the code with async_read_some (http://www.boost.org/doc/libs/1_56_0/doc/html/boost_asio/reference/basic_serial_port/async_read_some.html):
void readCallback(const boost::system::error_code& error, size_t bytes_transfered, bool & finished_reading, boost::system::error_code& error_report, size_t & bytes_read)
{
std::cout << "READ CALLBACK\n";
std::cout.flush();
error_report = error;
bytes_read = bytes_transfered;
finished_reading = true;
return;
}
int main()
{
int baud_rate = 115200;
std::string port_name = "/dev/ttyUSB0";
boost::asio::io_service io_service_;
boost::asio::serial_port serial_port_(io_service_,port_name);
serial_port_.set_option(boost::asio::serial_port_base::baud_rate(baud_rate));
boost::thread service_thread_;
service_thread = boost::thread(boost::bind(&boost::asio::io_service::run, &io_service_));
std::cout << "Starting byte read\n";
boost::system::error_code ec;
bool finished_reading = false;
size_t bytes_read;
int max_response_size = 8;
uint8_t read_buffer[max_response_size];
serial_port_.async_read_some(boost::asio::buffer(read_buffer, max_response_size),
boost::bind(readCallback,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred,
finished_reading, ec, bytes_read));
std::cout << "Waiting for read to finish\n";
while (!finished_reading)
{
boost::this_thread::sleep(boost::posix_time::milliseconds(1));
}
std::cout << "Finished byte read: " << bytes_read << "\n";
for (int i = 0; i < bytes_read; ++i)
{
printf("0x%x ",read_buffer[i]);
}
}
The result is that the callback does not print out anything and the while !finished loop never finishes.
Here is how I use the blocking read_some function (boost.org/doc/libs/1_56_0/doc/html/boost_asio/reference/basic_serial_port/read_some.html):
int main()
{
int baud_rate = 115200;
std::string port_name = "/dev/ttyUSB0";
boost::asio::io_service io_service_;
boost::asio::serial_port serial_port_(io_service_,port_name);
serial_port_.set_option(boost::asio::serial_port_base::baud_rate(baud_rate));
boost::thread service_thread_;
service_thread = boost::thread(boost::bind(&boost::asio::io_service::run, &io_service_));
std::cout << "Starting byte read\n";
boost::system::error_code ec;
int max_response_size = 8;
uint8_t read_buffer[max_response_size];
int bytes_read = serial_port_.read_some(boost::asio::buffer(read_buffer, max_response_size),ec);
std::cout << "Finished byte read: " << bytes_read << "\n";
for (int i = 0; i < bytes_read; ++i)
{
printf("0x%x ",read_buffer[i]);
}
}
This version prints from 1 up to 8 characters that I send, blocking until at least one is sent.
The code does not guarantee that the io_service is running. io_service::run() will return when either:
All work has finished and there are no more handlers to be dispatched
The io_service has been stopped.
In this case, it is possible for the service_thread_ to be created and invoke io_service::run() before the serial_port::async_read_some() operation is initiated, adding work to the io_service. Thus, the service_thread_ could immediately return from io_service::run(). To resolve this, either:
Invoke io_service::run() after the asynchronous operation has been initiated.
Create a io_service::work object before starting the service_thread_. A work object prevents the io_service from running out of work.
This answer may provide some more insight into the behavior of io_service::run().
A few other things to note and to expand upon Igor's answer:
If a thread is not progressing in a meaningful way while waiting for an asynchronous operation to complete (i.e. spinning in a loop sleeping), then it may be worth examining if mixing synchronous behavior with asynchronous operations is the correct solution.
boost::bind() copies its arguments by value. To pass an argument by reference, wrap it with boost::ref() or boost::cref():
boost::bind(..., boost::ref(finished_reading), boost::ref(ec),
boost::ref(bytes_read));
Synchronization needs to be added to guarantee memory visibility of finished_reading in the main thread. For asynchronous operations, Boost.Asio will guarantee the appropriate memory barriers to ensure correct memory visibility (see this answer for more details). In this case, a memory barrier is required within the main thread to guarantee the main thread observes changes to finished_reading by other threads. Consider using either a Boost.Thread synchronization mechanism like boost::mutex, or Boost.Atomic's atomic objects or thread and signal fences.
Note that boost::bind copies its arguments. If you want to pass an argument by reference, wrap it with boost::ref (or std::ref):
boost::bind(readCallback, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred, boost::ref(finished_reading), ec, bytes_read));
(However, strictly speaking, there's a race condition on the bool variable you pass to another thread. A better solution would be to use std::atomic_bool.)