Getting wrong output from boost lock free spsc queue - c++11

I am trying to implement lock free queue of user defined data type using boost library, but I am getting wrong result.
Please help me out where I am doing wrong.
#include <boost/lockfree/spsc_queue.hpp>
#include <thread>
#include <iostream>
#include <string.h>
#include <time.h>
class Queue
{
private:
unsigned char *m_data;
int m_len;
public:
Queue(unsigned char *data,int len);
Queue(const Queue &obj);
~Queue();
Queue & operator =(const Queue &obj);
unsigned char *getdata()
{
return m_data;
}
int getint()
{
return m_len;
}
};
Queue::Queue(unsigned char* data, int len)
{
m_len=len;
m_data=new unsigned char[m_len];
memcpy(m_data,data,m_len);
}
Queue::Queue(const Queue& obj)
{
m_len= obj.m_len;
m_data=new unsigned char[m_len];
memcpy(m_data,(unsigned char *)obj.m_data,m_len);
}
Queue::~Queue()
{
delete[] m_data;
m_len=0;
}
Queue & Queue::operator =(const Queue &obj)
{
if(this != &obj)
{
m_len=obj.m_len;
m_data=new unsigned char[m_len];
memcpy(m_data,(unsigned char *)obj.m_data,m_len);
}
return *this;
}
boost::lockfree::spsc_queue<Queue*> q(10);
void produce()
{
int i=0;
unsigned char* data=(unsigned char *)malloc(10);
memset(data,1,9);
Queue obj(data,10);
Queue *pqueue=&obj;
printf("%d\n",pqueue->getint());
q.push(pqueue);
}
void consume()
{
Queue *obj;
q.pop(&obj);
printf("%d\n",obj->getint());
}
int main(int argc, char** argv) {
// std::thread t1{produce};
// std::thread t2{consume};
//
// t1.join();
// t2.join();
produce();
consume();
return 0;
}
As per boost::lockfree::queue requirements I created following in class.
Copy Constructor
Assignment Operator
Destructor
Please let me know if anything other requires.
Thanks.

You're using malloc in C++.
You die.
You have 2 lives left.
Seriously, don't do that. Especially since using it with delete[] is clear cut Undefined Behaviour.
Sadly you lose another life here:
Queue obj(data,10);
Queue *pqueue=&obj;
q.push(pqueue);
You store a pointer to a local. More Undefined Behaviour
You have 1 life left.
Last life at
q.pop(&obj);
You pop using an iterator. It will be treated as an output iterator.
You get a return that indicates the number of elements popped, and items
will be written to &obj[0], &obj[1], &obj[2], etc.
Guess what? Undefined Behaviour.
See also: Boost spsc queue segfault
You died.
You're already dead. But you forsake your afterlife with
printf("%d\n",obj->getint());
Since pop might not have popped anything (the queue may have been empty), this in itself is Undefined Behaviour.
The funny part is, you talk about all these constructor requirements but you store pointers in the lockfree queue...?! Just write it:
typedef std::vector<unsigned char> Data;
class Queue {
private:
Data m_data;
public:
Queue(Data data) : m_data(std::move(data)) {}
Queue() : m_data() {}
unsigned char const *getdata() const { return m_data.data(); }
size_t getint() const { return m_data.size(); }
};
boost::lockfree::spsc_queue<Queue> q(10);
Live On Coliru
Notes:
you need to make the consumer check the return code of pop. The push might not have happened, and lock free queues don't block.
you don't need that contraption. Just pass vectors all the way:
C++ Code
Live On Coliru
#include <boost/lockfree/spsc_queue.hpp>
#include <thread>
#include <iostream>
#include <vector>
typedef std::vector<unsigned char> Queue;
boost::lockfree::spsc_queue<Queue> q(10);
void produce() {
Queue obj(10, 1);
std::cout << __FUNCTION__ << " - " << obj.size() << "\n";
q.push(std::move(obj));
}
void consume() {
Queue obj;
while (!q.pop(obj)) { }
std::cout << __FUNCTION__ << " - " << obj.size() << "\n";
}
int main() {
std::thread t1 {produce};
std::thread t2 {consume};
t1.join();
t2.join();
}

Related

Boost asio: including <arpa/inet.h> causes socket to always output 0 bytes

I'm trying to include <arpa/inet.h> in a low-level library so that I have access to hton* and ntoh* functions in the library. The low-level library gets called into by higher-level code running a Boost asio socket. I'm aware Boost asio contains the hton* and ntoh* functions, but i'd like to avoid linking all of Boost asio to the library since hton*/ntoh* are all I need.
However, if I simply include <arpa/inet.h> in the low-level library, 0 bytes always will be sent from the Boost asio socket. Confirmed by Wireshark.
Here's the class where i'd like to include <arpa/inet.h> but not Boost. If <arpa/inet.h> is included, 0 bytes will be sent.
#pragma pack(push, 1)
#include "PduHeader.h"
#include <arpa/inet.h>
class ClientInfoPdu
{
public:
ClientInfoPdu(const uint16_t _client_receiver_port)
{
set_client_receiver_port(_client_receiver_port);
}
PduHeader pdu_header{CLIENT_INFO_PDU, sizeof(client_receiver_port)};
inline void set_client_receiver_port(const uint16_t _client_receiver_port)
{
//client_receiver_port = htons(_client_receiver_port);
client_receiver_port = _client_receiver_port;
}
inline uint16_t get_client_receiver_port()
{
return client_receiver_port;
}
inline size_t get_total_size()
{
return sizeof(PduHeader) + pdu_header.get_pdu_payload_size();
}
private:
uint16_t client_receiver_port;
};
#pragma pack(pop)
Here's the higher level code that includes Boost and attempts to send the data via a socket. The printout indicates 5 bytes were sent, however 0 bytes were actually sent.
#include "ServerConnectionThread.h"
#include "config/ClientConfig.h"
#include "protocol_common/ClientInfoPdu.h"
#include <boost/asio.hpp>
#include <unistd.h>
using boost::asio::ip::udp;
void ServerConnectionThread::execute()
{
boost::asio::io_service io_service;
udp::endpoint remote_endpoint =
udp::endpoint(boost::asio::ip::address::from_string(SERVER_IP), SERVER_PORT);
udp::socket socket(io_service);
socket.open(udp::v4());
ClientInfoPdu client_info_pdu = ClientInfoPdu(RECEIVE_PORT);
while (true)
{
uint16_t total_size = client_info_pdu.get_total_size();
socket.send_to(boost::asio::buffer(&client_info_pdu, total_size), remote_endpoint);
printf("sent %u bytes\n", total_size);
usleep(1000000);
}
}
Again, simply removing "#include <arpa/inet.h>" will cause this code to function as expected and send 5 bytes per packet.
How is ClientInfoPdu defined? This looks like it is likely UB:
boost::asio::buffer(&client_info_pdu, total_size)
The thing is total size is sizeof(PduHeader) + pdu_header.get_pdu_payload_size() (so sizeof(PduHeader) + 2);
First problem is that you're mixing access modifiers, killing the POD/standard_layout properties of your types.
#include <type_traits>
static_assert(std::is_standard_layout_v<PduHeader> && std::is_trivial_v<PduHeader>);
static_assert(std::is_standard_layout_v<ClientInfoPdu> && std::is_trivial_v<ClientInfoPdu>);
This will fail to compile. Treating the types as POD (as you do) invokes
Undefined Behaviour.
This is likely the explanation for the fact that "it stops working" with some changes. It never worked: it might just accidentally have appeared to work, but it was undefined behaviour.
It's not easy to achieve POD-ness while still getting the convenience of the
constructors. In fact, I don't think that's possible. In short, if you want to
treat your structs as C-style POD types, make them... C-style POD types.
Another thing: a possible implementation of `PduHeader I
can see working for you looks a bit like so:
enum MsgId{CLIENT_INFO_PDU=0x123};
struct PduHeader {
MsgId id;
size_t payload_size;
size_t get_pdu_payload_size() const { return payload_size; }
};
Here, again you might have/need endianness conversions.
Suggestion
In short, if you want this to work, I'd say keep it simple.
Instead of creating non-POD types all over the place that are responsible for endianness conversion by adding getters/setters for each value, why not create a simple user-defined-type that does this always, and use them instead?
struct PduHeader {
Short id; // or e.g. uint8_t
Long payload_size;
};
struct ClientInfoPdu {
PduHeader pdu_header; // or inheritance, same effect
Short client_receiver_port;
};
Then just use it as a POD struct:
while (true) {
ClientInfoPdu client_info_pdu;
init_pdu(client_info_pdu);
auto n = socket.send_to(boost::asio::buffer(&client_info_pdu, sizeof(client_info_pdu)), remote_endpoint);
printf("sent %lu bytes\n", n);
std::this_thread::sleep_for(1s);
}
The function init_pdu can be implemented with overloads per submessage:
void init_pdu(ClientInfoPdu& msg) {
msg.pdu_header.id = CLIENT_INFO_PDU;
msg.pdu_header.payload_size = sizeof(msg);
}
There are variations on this where it can become a template or take a
PduHeder& (if your message inherits instead of aggregates). But the basic
principle is the same.
Endianness Conversion
Now you'll noticed I avoided using uint32_t/uint16_t directly (though uint8_t is fine because it doesn't need byte ordering). Instead, you could define Long and Short as simple POD wrappers around them:
struct Short {
operator uint16_t() const { return ntohs(value); }
Short& operator=(uint16_t v) { value = htons(v); return *this; }
private:
uint16_t value;
};
struct Long {
operator uint32_t() const { return ntohl(value); }
Long& operator=(uint32_t v) { value = htonl(v); return *this; }
private:
uint32_t value;
};
The assignment and conversions mean that you can use it as just another
int32_t/int16_t except that the necessary conversions are always done.
If you want to satnd on the shoulder of giants instead, you can use the better types from Boost Endian, which also has lots more advanced facilities
DEMO
Live On Coliru
#include <type_traits>
#include <cstdint>
#include <thread>
#include <arpa/inet.h>
using namespace std::chrono_literals;
#pragma pack(push, 1)
enum MsgId{CLIENT_INFO_PDU=0x123};
struct Short {
operator uint16_t() const { return ntohs(value); }
Short& operator=(uint16_t v) { value = htons(v); return *this; }
private:
uint16_t value;
};
struct Long {
operator uint32_t() const { return ntohl(value); }
Long& operator=(uint32_t v) { value = htonl(v); return *this; }
private:
uint32_t value;
};
static_assert(std::is_standard_layout_v<Short>);
static_assert(std::is_trivial_v<Short>);
static_assert(std::is_standard_layout_v<Long>);
static_assert(std::is_trivial_v<Long>);
struct PduHeader {
Short id; // or e.g. uint8_t
Long payload_size;
};
struct ClientInfoPdu {
PduHeader pdu_header; // or inheritance, same effect
Short client_receiver_port;
};
void init_pdu(ClientInfoPdu& msg) {
msg.pdu_header.id = CLIENT_INFO_PDU;
msg.pdu_header.payload_size = sizeof(msg);
}
static_assert(std::is_standard_layout_v<PduHeader> && std::is_trivial_v<PduHeader>);
static_assert(std::is_standard_layout_v<ClientInfoPdu> && std::is_trivial_v<ClientInfoPdu>);
#pragma pack(pop)
#include <boost/asio.hpp>
//#include <unistd.h>
using boost::asio::ip::udp;
#define SERVER_IP "127.0.0.1"
#define SERVER_PORT 6767
#define RECEIVE_PORT 6868
struct ServerConnectionThread {
void execute() {
boost::asio::io_service io_service;
udp::endpoint const remote_endpoint =
udp::endpoint(boost::asio::ip::address::from_string(SERVER_IP), SERVER_PORT);
udp::socket socket(io_service);
socket.open(udp::v4());
while (true) {
ClientInfoPdu client_info_pdu;
init_pdu(client_info_pdu);
auto n = socket.send_to(boost::asio::buffer(&client_info_pdu, sizeof(client_info_pdu)), remote_endpoint);
printf("sent %lu bytes\n", n);
std::this_thread::sleep_for(1s);
}
}
};
int main(){ }

How to use boost::shared_ptr/std::shared_ptr with boost::object_pool?

Should I ever prevent memory leaks using shared pointers with boost::object_pool (in case of an exception inside malloc-destroy block)?
If yes, what is the correct way to initialize shared_ptr?
How to clean up memory afterwards?
#include <boost/pool/object_pool.hpp>
#include <boost/shared_ptr.hpp>
int main() {
boost::object_pool<int> pool;
// Which is the correct way of initializing the shared pointer:
// 1)
int *i = pool.malloc();
boost::shared_ptr<int> sh(i);
int *j = pool.construct(2);
boost::shared_ptr<int> sh2(j);
// or 2)
boost::shared_ptr<int> sh(pool.malloc());
// Now, how should i clean up the memory?
// without shared_ptr I'd call here pool.destroy(i) and pool.destroy(j)
}
Does it need to be a shared pointer?
unique_ptr has the benefit to encode the deleter into the type:
#include <boost/pool/object_pool.hpp>
template <typename T, typename Pool = boost::object_pool<T> >
struct pool_deleter {
pool_deleter(Pool& pool) : _pool(pool) {}
void operator()(T*p) const { _pool.destroy(p); }
private:
Pool& _pool;
};
#include <memory>
template <typename T> using pool_ptr
= std::unique_ptr<T, pool_deleter<T, boost::object_pool<T> > >;
int main() {
boost::object_pool<int> pool;
pool_ptr<int> i(pool.construct(42), pool);
std::cout << *i << '\n';
}

Reading data with boost asio on client

I am learning boost asio and have mistake. I have written simple client ( I can send data from it but when I read data I cant even compile it) I used protocol buffer to serialize data . So file #include "test.pb.h" is probuffer class
My code client :
#include <boost/array.hpp>
#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <boost/foreach.hpp>
#include <boost/lexical_cast.hpp>
#include <boost/thread/thread.hpp>
#include <cstdlib>
#include <iostream>
#include <string>
#include <thread>
#include "test.pb.h"
using boost::asio::ip::tcp;
int main(int argc, char** argv)
{
try
{
// connect to the server:
boost::asio::io_service io_service;
tcp::resolver resolver(io_service);
std::string const server_address = "localhost";
std::string const server_port = "10000";
tcp::resolver::query query(server_address, server_port);
tcp::resolver::iterator endpoint_iterator = resolver.resolve(query);
tcp::socket socket(io_service);
boost::asio::connect(socket, endpoint_iterator);
//while ( true)
{
Person p = Person();
p.set_id(22);
p.set_name("Jack vorobey");
// std::cout << p.id();
// std::cout << p.name();
std::string data; // this will hold serialized data
bool ok = p.SerializeToString(&data);
assert(ok);
// std::cout << data.size() << std::endl;
boost::asio::write(socket, boost::asio::buffer(data))
boost::asio::read(socket, boost::asio::buffer(data));;
// break; // Connection closed cleanly by peer.
// std::cout << data.size() << std::endl; // shows a reduction in amount of
remaining data
// boost::asio::read(socket, boost::asio::buffer(data) /*,
}
boost::asio::transfer_exactly(65536) */);
}
catch (std::exception& e)
{
//std::cerr << e.what(luuu) << std::endl;
}
std::cout << "\nClosing";
std::string dummy;
}
The code of my mistake I dont understand :
error C2679: binary '=' : no operator found which takes a right-hand operand of type 'const boost::asio::const_buffer' (or there is no acceptable conversion)
1> c:\local\boost_1_55_0\boost\asio\buffer.hpp(136): could be 'boost::asio::mutable_buffer &boost::asio::mutable_buffer::operator =(const boost::asio::mutable_buffer &)'
1> while trying to match the argument list '(boost::asio::mutable_buffer, const boost::asio::const_buffer)'
This is because template<typename Elem, typename Traits, typename Allocator> const_buffers_1 buffer(const std::basic_string<Elem, Traits, Allocator> &) returns an instance of const_buffers_1 (which is a model of ConstBufferSequence concept). Certainly, you cannot read data into a constant buffer.
Do not read data into a std::string, because it's not intended for that (note that its c_str() and data() member functions return const char*). Instead, allocate another buffer or use asio::streambuf.
You can use a streambuf, or specify the (preallocated!) size:
#include <boost/asio.hpp>
#include <string>
using boost::asio::ip::tcp;
int main()
{
boost::asio::io_service io_service;
tcp::resolver resolver(io_service);
tcp::socket socket(io_service);
boost::asio::connect(socket, resolver.resolve(tcp::resolver::query("localhost", "10000")));
std::string request("request");
boost::asio::write(socket, boost::asio::buffer(request));
#if 0
std::string response;
response.resize(32);
boost::asio::read(socket, boost::asio::buffer(&response[0], response.size()));
#else
boost::asio::streambuf response;
boost::asio::read(socket, response);
#endif
}

Boost.Asio: What's the Behavior of a deadline_timer with 0 Millisecond Expiration

What the behavior of a deadline_timer whose expiration is 0 milliseconds?
In my code, I have:
boost::asio::io_service ios;
...
boost::asio::deadline_timer ptimer(ios);
ptimer.expires_from_now(boost::posix_time::milliseconds(duration)); // Duration might be 0 sometimes
boost::system::error_code ec;
ptimer.async_wait(boost::bind(&SomeTimeOutHandler, this, ec));
I found that if duration == 0, the handler SomeTimeOutHandler never gets called.
I want it gets called.
However, if I change to duration == 1, the handler does get called.
So what the exact behavior it should be when the deadline_timer's expiration is 0?
Edit:
But the following HelloWorld test program is working (suggested by #Roger):
#include <iostream>
#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <boost/thread.hpp>
void SomeTimeOutHandler(const boost::system::error_code& ec)
{
if (ec)
{
std::cout << "SomeTimeOutHandler error_code" << std::endl;
}
else
{
std::cout << "I'm in good shape" << std::endl;
}
}
void Test(int duration)
{
boost::asio::io_service ios;
boost::asio::deadline_timer ptimer(ios);
ptimer.expires_from_now(boost::posix_time::milliseconds(duration));
boost::system::error_code ec;
ptimer.async_wait(boost::bind(&SomeTimeOutHandler, ec));
ios.run();
// boost::this_thread::sleep(boost::posix_time::milliseconds(duration * 2 + 1000));
}
int main(int argc, char* argv[])
{
Test(10); // Test(0);
return 0;
}

boost::asio::async_write, writing data larger than 65536 bytes

I'm attempting to write jpeg frames via a socket to a client using async_write(). I used the boost asynchronous TCP daytime server example as a starting point.
#include <ctime>
#include <iostream>
#include <string>
#include <boost/bind.hpp>
#include <boost/shared_ptr.hpp>
#include <boost/enable_shared_from_this.hpp>
#include <boost/asio.hpp>
using boost::asio::ip::tcp;
std::string make_daytime_string()
{
using namespace std; // For time_t, time and ctime;
time_t now = time(0);
return ctime(&now);
}
class tcp_connection
: public boost::enable_shared_from_this<tcp_connection>
{
public:
typedef boost::shared_ptr<tcp_connection> pointer;
static pointer create(boost::asio::io_service& io_service)
{
return pointer(new tcp_connection(io_service));
}
tcp::socket& socket()
{
return socket_;
}
void start()
{
message_ = make_daytime_string();
boost::asio::async_write(socket_, boost::asio::buffer(message_),
boost::bind(&tcp_connection::handle_write, shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
private:
tcp_connection(boost::asio::io_service& io_service)
: socket_(io_service)
{
}
void handle_write(const boost::system::error_code& /*error*/,
size_t /*bytes_transferred*/)
{
}
tcp::socket socket_;
std::string message_;
};
class tcp_server
{
public:
tcp_server(boost::asio::io_service& io_service)
: acceptor_(io_service, tcp::endpoint(tcp::v4(), 13))
{
start_accept();
}
private:
void start_accept()
{
tcp_connection::pointer new_connection =
tcp_connection::create(acceptor_.io_service());
acceptor_.async_accept(new_connection->socket(),
boost::bind(&tcp_server::handle_accept, this, new_connection,
boost::asio::placeholders::error));
}
void handle_accept(tcp_connection::pointer new_connection,
const boost::system::error_code& error)
{
if (!error)
{
new_connection->start();
start_accept();
}
}
tcp::acceptor acceptor_;
};
int main()
{
try
{
boost::asio::io_service io_service;
tcp_server server(io_service);
io_service.run();
}
catch (std::exception& e)
{
std::cerr << e.what() << std::endl;
}
return 0;
}
I have modified the method that performs the async_write() as follows:
void start()
{
// fileToVector method reads contents of file to vector;
std::vector<unsigned char> message_ = fileToVector("/tmp/test");
boost::asio::async_write(socket_, boost::asio::buffer(message_),
boost::bind(&tcp_connection::handle_write, shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
When reading a large file from the server using a client, the server will only write a maximum of 65536 bytes. If I replace the boost::asio::async_write() call with a synchronous call boost::asio::write() the correct amount of bytes are transferred to the client.
So I suppose my question is, how can I send more than 65536 bytes using boost::asio::async_write()? Any help would be greatly appreciated.
An issue is that using the async_write function data will be send not immediately by the function but in some time after the start method is finished and the local message_ variable will be destroyed and the boost::asio::buffer does not copy the content of message_. It stores only a reference to it. The result is unpredictable. May be transmission of 65536 bytes is the result of this behavior.

Resources