I am a novice trying to use google protobuf for work project. I want to find out difference between protobuf messages and hence trying to use the MessageDifferencer APIs. I get the SEGV while running the code below. Commenting the line "reporter->ReportModified(*Obj1, *Obj2, field_path);" results in no segv
Any help in usage of differencer appreciated!
google::protobuf::util::MessageDifferencer diff;
diff.set_report_matches(false);
diff.set_report_moves(false);
std::string reportDiff;
google::protobuf::io::StringOutputStream* opstream = new google::protobuf::io::StringOutputStream(&reportDiff);
google::protobuf::util::MessageDifferencer::StreamReporter* reporter = new google::protobuf::util::MessageDifferencer::StreamReporter(opstream);
diff.ReportDifferencesTo(reporter);
std::vector<google::protobuf::util::MessageDifferencer::SpecificField> field_path;
try
{
reporter->ReportModified(*Obj1, *Obj2, field_path);
}
catch (const std::exception& e)
{
std::cout << e.what() <<"\n";
}
cout << __func__ << " Report added " << field_path.size();
//Cleanup objects
delete Obj1;
delete Obj2;
delete reporter;
Thanks,
Maddy
You shouldn't be calling the ReportModified method directly, the MessageDifferencer class calls it when it finds a difference.
MessageDifferencer::Compare is the correct method to call, according to the docs. Assuming all else is correct, I believe changing your code inside the try-loop to call that should work.
Moving your code to a function, you could have something like
std::string CompareMessages(
const google::protobuf::Message& m1,
const google::protobuf::Message& m2) {
using google::protobuf::util::MessageDifferencer;
MessageDifferencer diff;
diff.set_report_matches(false);
diff.set_report_moves(false);
std::string reportDiff;
{
google::protobuf::io::StringOutputStream opstream(&reportDiff);
MessageDifferencer::StreamReporter reporter(&opstream);
diff.ReportDifferencesTo(&reporter);
diff.Compare(m1, m2);
}
return std::move(reportDiff);
}
Related
I have a function which needs to parse some arguments and several if clauses inside it need to perform similar actions. In order to reduce typing and help keep the code readable, I thought I'd use a lambda to encapsulate the recurring actions, but I'm having trouble finding sufficient info to determine whether I'm mistakenly invoking undefined behavior or what I need to do to actualize my approach.
Below is a simplified code snippet of what I have currently:
int foo(int argc, char* argv[])
{
Using ss = std::istringstream;
auto sf = [&](ss&& stream) -> ss& {
stream.exceptions(ss::failbit);
return stream;
};
int retVal = 0;
bool valA = false;
bool valB = false;
try
{
for(int i=1; i < argc; i++)
{
std::string arg( argv[i] );
if( !valA )
{
valA = true;
sf( ss(arg) ) >> myInt;
}
else
if( !valB )
{
valB = true;
sf( ss(arg) ) >> std::hex >> myOtherInt;
}
}
}
catch( std::exception& err )
{
retVal = -1;
std::cerr << err.what() << std::endl;
}
return retVal;
}
First, based on what I've read, I don't think that specifying the lambda argument as an rvalue reference (ss&&) is doing quite what I want it to do, however, trying to compile with it declared as a normal reference (ss&) failed with the error cannot bind non-const lvalue reference of type 'ss&'. Changing ss& to ss&& got rid of the error and did not produce any warnings, but I'm not convinced that I'm using that construct correctly.
I've tried reading up on the various definitions for each, but the wording is a bit confusing.
I guess ultimately my questions are:
Can I expect the lifetime of my temporary ss(arg) object to extend through the entire extraction expression?
What is the correct way to define a lambda such that I can use the lambda in the way I demonstrate above, assuming that such a thing is actually possible?
I am using protobuf 3 to serialize a simple message.
I get a bad alloc when i set a string value for one of the memebers of my protobuf message like so.
std::string a("eeee");
hello_in.set_name(a);
The bad alloc exception happens in the libprotobuf.dll in this function...
void CreateInstance(Arena* arena, const ::std::string* initial_value) {
GOOGLE_DCHECK(initial_value != NULL);
// uses "new ::std::string" when arena is nullptr
ptr_ = Arena::Create< ::std::string>(arena, *initial_value);
}
But i think the real problem is that initial_value has been corrupted somehow and has a size of [size] = 3435973836.
Not sure how this is being corrupted. CreateInstance does get called a few times prior to this but its the first time it is called from main.cpp. Which leads me to believe that it has something to do with dll's and ownership of memeory.
Using any of the other set_name functions also cause a bad alloc exception.
Setting the bool or int in the message works fine.
Here is the message and the main.cpp. I didnt include the hello.pb.h/pb.cc as they are quite big but can if it helps.
// See README.txt for information and build instructions.
//
// Note: START and END tags are used in comments to define sections used in
// tutorials. They are not part of the syntax for Protocol Buffers.
//
// To get an in-depth walkthrough of this file and the related examples, see:
// https://developers.google.com/protocol-buffers/docs/tutorials
// [START declaration]
syntax = "proto3";
package commands;
import "google/protobuf/timestamp.proto";
// [END declaration]
// [START messages]
message Hello {
string name = 1;
int32 id = 2; // Unique ID number for this person.
bool on = 3;
google.protobuf.Timestamp last_updated = 4;
}
// [END messages]
#include "hello.pb.h"
// stl
#include <fstream>
#include <iostream>
int main()
{
GOOGLE_PROTOBUF_VERIFY_VERSION;
commands::Hello hello_in;
hello_in.set_id(2);
std::string a("eeee");
hello_in.set_name(a);
hello_in.set_on(false);
{
// Write the new address book back to disk.
std::fstream output("hello.txt", std::ios::out | std::ios::trunc | std::ios::binary);
if (!hello_in.SerializeToOstream(&output)) {
std::cerr << "Failed to write address book." << std::endl;
return -1;
}
}
commands::Hello hello_out;
{
// Read the existing address book.
std::fstream input("hello.txt", std::ios::in | std::ios::binary);
if (!input) {
std::cout << "hello.txt" << ": File not found. Creating a new file." << std::endl;
}
else if (!hello_out.ParseFromIstream(&input)) {
std::cerr << "Failed to parse address book." << std::endl;
return -1;
}
}
// Optional: Delete all global objects allocated by libprotobuf.
google::protobuf::ShutdownProtobufLibrary();
return 0;
}
I have observed same behavior (Visual Studio 2019 C++ project). The solution which helped me: libprotobuf.lib and libprotobuf.dll were replaced in debug/x86 mode by its debug version, libprotobufd.lib and libprotobufd.dll.
Used Versions: OMNeT++ 5.0 with iNET 3.4.0
I created some code, which gives me reliable results in ‘step-by-step’- or ‘animated’ simulation mode. The moment I change to ‘fast’ or ‘express’ mode, it gets buggy. The following simplified example will explain my problems:
void MyMacSlave::handleSelfMessage(cMessage *msg)
{
if (msg == CheckAck) {
std::cout << “CheckAck: “ << msg << std::endl;
}
if (msg == transmissionAnnouncement) {
std::cout << “transmissionAnncouncement: “ << msg << std::endl;
}
if (msg == transmissionEvent) {
std::cout << “transmissionEvent: “ << msg << std::endl;
}
delete msg;
}
There is a function, which is called for handling self-messages. Depending on what self-message I got, I need to run different if queries.
I get this correct output in step-by-step or animated mode:
CheckAck: (omnetpp::cMessage)CheckAck
transmissionAnncouncement: (omnetpp::cMessage)transmissionAnncouncement
transmissionEvent: (omnetpp::cMessage)transmissionEvent
And this is the strange output I get using fast or express mode:
CheckAck: (omnetpp::cMessage)CheckAck
transmissionAnncouncement: (omnetpp::cMessage)transmissionAnncouncement
transmissionAnncouncement: (omnetpp::cMessage)transmissionEvent
transmissionEvent: (omnetpp::cMessage)transmissionEvent
The third output line shows that the self-message is ‘transmissionEvent’, but the ‘if (msg == transmissionAnnouncement)’ is mistakenly considered as true as well.
As shown above I get different simulation results, depending on the simulation mode I am using. What is the reason for the different output? Why is there even a difference?
As Christoph and Rudi mentioned there was something wrong with the memory allocation. When a pointer is de-allocated and a new one is allocated on the same memory, there will be something wrong. The difference regarding the usage of different running modes is just a sign that there are errors to that effect.
In my case it was useful to check for message-kinds like:
if (msg->getKind() == checkAckAckType) {
instead of the method used in the originally question. I defined the message-kinds using simple enums.
I have the following system architecture (cannot be changed - legacy code): One main application invokes one or more other applications and these applications interact over a IP protocol.
All applications write to one console window. Unfortunately the console output can get messed up (one character from app 1, next char from app 2, next character from app 4 etc.).
All applications write to console via one Logger.dll (provides static logging functions) using cout/cerr.
Is there a way how I can prevent mixed logging messages in this setup?
Thanks in advance.
EDIT code added:
void Logger::Log(const std::string & componentName, const std::string & Text, LogLevel logLevel, bool logToConsole, bool beep)
{
std::ostringstream stream;
switch (logLevel)
{
case LOG_INFO:
if (logToConsole)
{
stream << componentName << ": INFO " << Text;
mx_console.lock(); // this is a static boost::mutex
std::cout << stream.str() << std::endl;
std::cout.flush();
mx_console.unlock();
}
break;
case LOG_STATUS:
stream << componentName << ": STATUS " << Text;
mx_console.lock();
std::cout << stream.str() << std::endl;
std::cout.flush();
mx_console.unlock();
break;
case LOG_WARNING:
stream << componentName << ": WARNING " << Text;
mx_console.lock();
std::cout << stream.str() << std::endl;
std::cout.flush();
mx_console.unlock();
break;
default:;
}
if (beep)
Beep( 500, 50 );
}
Since you have separate logging functionality you can at minimum use some kind of locking (global mutex, etc.) to avoid interspersing messages from different applications too much. To make it more readable and grepable, add some identifying information, like process name or PID. Wrapping your Logger.dll around existing logging library sounds like an option as well.
Alternatively, you could have logging functions just forward messages to your main application and let that to sort out the synchronization and interspersing.
Syslog might be a solution for you as it is intended to handle logs from various places. Syslog is developed for unix, but this answer shows versions for windows.
You can change your logger to log to syslog instead of the console.
I replaced now all the
std::cout << stream.str();
statements with
std::string str = stream.str();
printf(str.c_str());
and now the output isn't messed up character-wise anymore.
But I don't have a good explanation for this behavior, does anybody know why?
I serialize multiple objects into a binary archive with Boost.
When reading back those objects from a binary_iarchive, is there a way to know how many objects are in the archive or simply a way to detect the end of the archive ?
The only way I found is to use a try-catch to detect the stream exception.
Thanks in advance.
I can think of a number of approaches:
Serialize STL containers to/from your archive (see documentation). The archive will automatically keep track of how many objects there are in the containers.
Serialize a count variable before serializing your objects. When reading back your objects, you'll know beforehand how many objects you expect to read back.
You could have the last object have a special value that acts as a kind of sentinel that indicates the end of the list of objects. Perhaps you could add an isLast member function to the object.
This is not very pretty, but you could have a separate "index file" alongside your archive that stores the number of objects in the archive.
Use the tellp position of the underlying stream object to detect if you're at the end of file:
Example (just a sketch, not tested):
std::streampos archiveOffset = stream.tellg();
std::streampos streamEnd = stream.seekg(0, std::ios_base::end).tellg();
stream.seekg(archiveOffset);
while (stream.tellp() < streamEnd)
{
// Deserialize objects
}
This might not work with XML archives.
Do you have all your objects when you begin serializing? If not, you are "abusing" boost serialization - it is not meant to be used that way. However, I am using it that way, using try catch to find the end of the file, and it works for me. Just hide it away somewhere in the implementation. Beware though, if using it this way, you need to either not serialize pointers, or disable pointer tracking.
If you do have all the objects already, see Emile's answer. They are all valid approaches.
std::istream* stream_;
boost::iostreams::filtering_streambuf<boost::iostreams::input>* filtering_streambuf_;
...
stream_ = new std::istream(memoryBuffer_);
if (stream_) {
filtering_streambuf_ = new boost::iostreams::filtering_streambuf<boost::iostreams::input>();
if (filtering_streambuf_) {
filtering_streambuf_->push(boost::iostreams::gzip_decompressor());
filtering_streambuf_->push(*stream_);
archive_ = new eos::portable_iarchive(*filtering_streambuf_);
}
}
using zip when reading data from the archives, and filtering_streambuf have such method as
std::streamsize std::streambuf::in_avail()
Get number of characters available to read
so i check the end of archive as
bool IArchiveContainer::eof() const {
if (filtering_streambuf_) {
return filtering_streambuf_->in_avail() == 0;
}
return false;
}
It is not helping to know how many objects are last in the archive, but helping to detect the end of them
(i'm using eof test only in the unit test for serialization/unserialization my classes/structures - to make sure that i'm reading all what i'm writing)
Sample code which I used to debug the similar issue
(based on Emile's answer) :
#include <fstream>
#include <iostream>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
struct A{
int a,b;
template <typename T>
void serialize(T &ar, int ){
ar & a;
ar & b;
}
};
int main(){
{
std::ofstream ofs( "ff.ar" );
boost::archive::binary_oarchive ar( ofs );
for(int i=0;i<3;++i){
A a {2,3};
ar << a;
}
ofs.close();
}
{
std::ifstream ifs( "ff.ar" );
ifs.seekg (0, ifs.end);
int length = ifs.tellg();
ifs.seekg (0, ifs.beg);
boost::archive::binary_iarchive ar( ifs );
while(ifs.tellg() < length){
A a;
ar >> a;
std::cout << "a.a-> "<< a.a << " and a.b->"<< a.b << "\n";
}
}
return 0;
}
you just read a byte from the file.
If you do not reach the end,
backword a byte then.