Serializing multiple std::shared_ptr via boost::serialization and sending over boost::asio - boost

Question

I'd like to transmit a shared_ptr object via boost asio from a client to a server. Here is my code:
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/asio.hpp>
#include <boost/serialization/export.hpp>
#include <boost/serialization/shared_ptr.hpp>
#include <chrono>
#include <iostream>
#include <sstream>
#include <string>
#include <thread>
using namespace std;
class Message {
public:
Message() {
}
virtual ~Message() {
}
string text;
private:
friend class boost::serialization::access;
template <class Archive>
void serialize(Archive &ar, const unsigned int version) {
ar &text;
}
};
BOOST_CLASS_EXPORT(Message)
void runClient() {
// Give server time to startup
this_thread::sleep_for(chrono::milliseconds(3000));
boost::asio::ip::tcp::iostream stream("localhost", "3000");
boost::archive::text_oarchive archive(stream);
for (int i = 0; i < 10; i++) {
std::shared_ptr<Message> dl = std::make_shared<Message>();
stringstream ss;
ss << "Hello " << i;
dl->text = ss.str();
archive << dl;
}
stream.close();
cout << "Client shutdown" << endl;
}
void handleIncommingClientConnection(boost::asio::ip::tcp::acceptor &acceptor) {
boost::asio::ip::tcp::iostream stream;
acceptor.accept(*stream.rdbuf());
boost::archive::text_iarchive archive(stream);
while (true) {
std::shared_ptr<Message> m;
try {
archive >> m;
cout << m->text << endl;
} catch (std::exception &ex) {
cout << ex.what() << endl;
if (stream.eof()) {
cout << "eof" << endl;
stream.close();
cout << "Server: shutdown client handling..." << endl;
break;
} else
throw ex;
}
}
}
void runServer() {
boost::asio::io_service ios;
boost::asio::ip::tcp::endpoint endpoint = boost::asio::ip::tcp::endpoint(boost::asio::ip::tcp::v4(), 3000);
boost::asio::ip::tcp::acceptor acceptor(ios, endpoint);
handleIncommingClientConnection(acceptor);
}
int main(int argc, char **argv) {
thread clientThread(runClient);
thread serverThread(runServer);
clientThread.join();
serverThread.join();
return 0;
}
Here is the program output:
Hello 0
Hello 1
Hello 2
Hello 3
Hello 3
Hello 3
Hello 3
Hello 3
Client shutdown
Hello 3
Hello 3
input stream error
eof
Server: shutdown client handling...
I am expecting the following output:
Hello 0
Hello 1
Hello 2
Hello 3
Hello 4
Hello 5
Hello 6
Hello 7
Client shutdown
Hello 8
Hello 9
input stream error
eof
Server: shutdown client handling...
When changing the shared_ptr to a simple object (std::shared_ptr<Message> m; to Message m) everything works as expected. I want to stick to the shared_ptr. What do I need to change?
Serialization alone seems to work:
stringstream stream;
{
boost::archive::text_oarchive archive(stream);
std::shared_ptr<Message> dl = std::make_shared<Message>();
stringstream ss;
ss << "Hello World!";
dl->text = ss.str();
archive << dl;
}
{
boost::archive::text_iarchive archive(stream);
std::shared_ptr<Message> m;
archive >> m;
cout << m->text << endl;
}
Output: Hello World!

Answer 1

The issues you're encountering are due to object tracking done by Boost.Serialization.
Depending on how the class is used and other factors, serialized
objects may be tracked by memory address. This prevents the same
object from being written to or read from an archive multiple times.
These stored addresses can also be used to delete objects created
during a loading process that has been interrupted by throwing of an
exception.
The documentation actually foreshadows this specific issue happening:
This could cause problems in progams[sic] where the copies of different
objects are saved from the same address.
Furthermore, the Class Serialization Traits documentation on object tracking tells us that in this particular situation, object tracking is enabled:
Default tracking traits are:
For primitive, track_never.
For pointers, track_never. That is, addresses of addresses are not tracked by default.
All current serialization wrappers such as boost::serialization::nvp, track_never.
For all other types, track_selectively. That is addresses of serialized objects are tracked if and only if one or more of the
following is true:
an object of this type is anywhere in the program serialized through a pointer.
the class is explicitly "exported" - see below.
the class is explicitly "registered" in the archive
Going back to your situation -- in the client, due to how your loop body is written, the 5th (and following) Message instance were allocated at the same address as the 4th Message instance. You can verify this by inspecting the values of dl.get() in each iteration. (In my tests on coliru, all of the instances were allocated at the same address, so YMMV).
Due to how object tracking works, all those shared_ptr instances were considered to point to the same Message instance (even though you changed the value meanwhile -- the library does not expect this happening), so the additional occurrences were just serialized as additional references. Upon deserialization... to be honest this smells of memory leaks and/or dangling reference issues (opinion, haven't investigated this in detail).
Summed up, the main issue with the code as shown is that it breaks a prerequisite of the serialization library, which is that you're serializing some constant state, and on deserialization you recreate that same state.
One way to address this would be to have an initialized std::vector of shared_ptr<Message> containing all the messages to transmit in this particular transaction. Similarly, you'd deserialize the whole vector on the other side. If you expect to have some persistent connection, then add framing to the protocol, with each frame containing an archive that contains one sequence of messages.
Minimal code modifications to make this work -- add include
#include <boost/serialization/vector.hpp>
Change runClient() as such:
void runClient() {
// Give server time to startup
this_thread::sleep_for(chrono::milliseconds(3000));
boost::asio::ip::tcp::iostream stream("127.0.0.1", "3000");
std::vector<std::shared_ptr<Message>> messages;
for (int i = 0; i < 10; i++) {
std::shared_ptr<Message> dl = std::make_shared<Message>();
stringstream ss;
ss << "Hello " << i;
dl->text = ss.str();
messages.emplace_back(dl);
}
boost::archive::text_oarchive archive(stream);
archive << messages;
stream.close();
cout << "Client shutdown" << endl;
}
And change handleIncommingClientConnection(...) as such:
void handleIncommingClientConnection(boost::asio::ip::tcp::acceptor &acceptor) {
boost::asio::ip::tcp::iostream stream;
acceptor.accept(*stream.rdbuf());
boost::archive::text_iarchive archive(stream);
while (true) {
try {
std::vector<std::shared_ptr<Message>> messages;
archive >> messages;
for (auto const& m : messages) {
cout << m->text << endl;
}
} catch (std::exception &ex) {
cout << ex.what() << endl;
if (stream.eof()) {
cout << "eof" << endl;
stream.close();
cout << "Server: shutdown client handling..." << endl;
break;
} else
throw ex;
}
}
}
NB: This doesn't add any support for multiple frames -- the client is expected to close the connection after it sent one vector of messages, otherwise the behaviour is undefined.
Sample on Coliru
Further resources:
boost serialization multiple objects