I am using protobuf 3 to serialize a simple message.
I get a bad alloc when i set a string value for one of the memebers of my protobuf message like so.
std::string a("eeee");
hello_in.set_name(a);
The bad alloc exception happens in the libprotobuf.dll in this function...
void CreateInstance(Arena* arena, const ::std::string* initial_value) {
GOOGLE_DCHECK(initial_value != NULL);
// uses "new ::std::string" when arena is nullptr
ptr_ = Arena::Create< ::std::string>(arena, *initial_value);
}
But i think the real problem is that initial_value has been corrupted somehow and has a size of [size] = 3435973836.
Not sure how this is being corrupted. CreateInstance does get called a few times prior to this but its the first time it is called from main.cpp. Which leads me to believe that it has something to do with dll's and ownership of memeory.
Using any of the other set_name functions also cause a bad alloc exception.
Setting the bool or int in the message works fine.
Here is the message and the main.cpp. I didnt include the hello.pb.h/pb.cc as they are quite big but can if it helps.
// See README.txt for information and build instructions.
//
// Note: START and END tags are used in comments to define sections used in
// tutorials. They are not part of the syntax for Protocol Buffers.
//
// To get an in-depth walkthrough of this file and the related examples, see:
// https://developers.google.com/protocol-buffers/docs/tutorials
// [START declaration]
syntax = "proto3";
package commands;
import "google/protobuf/timestamp.proto";
// [END declaration]
// [START messages]
message Hello {
string name = 1;
int32 id = 2; // Unique ID number for this person.
bool on = 3;
google.protobuf.Timestamp last_updated = 4;
}
// [END messages]
#include "hello.pb.h"
// stl
#include <fstream>
#include <iostream>
int main()
{
GOOGLE_PROTOBUF_VERIFY_VERSION;
commands::Hello hello_in;
hello_in.set_id(2);
std::string a("eeee");
hello_in.set_name(a);
hello_in.set_on(false);
{
// Write the new address book back to disk.
std::fstream output("hello.txt", std::ios::out | std::ios::trunc | std::ios::binary);
if (!hello_in.SerializeToOstream(&output)) {
std::cerr << "Failed to write address book." << std::endl;
return -1;
}
}
commands::Hello hello_out;
{
// Read the existing address book.
std::fstream input("hello.txt", std::ios::in | std::ios::binary);
if (!input) {
std::cout << "hello.txt" << ": File not found. Creating a new file." << std::endl;
}
else if (!hello_out.ParseFromIstream(&input)) {
std::cerr << "Failed to parse address book." << std::endl;
return -1;
}
}
// Optional: Delete all global objects allocated by libprotobuf.
google::protobuf::ShutdownProtobufLibrary();
return 0;
}
I have observed same behavior (Visual Studio 2019 C++ project). The solution which helped me: libprotobuf.lib and libprotobuf.dll were replaced in debug/x86 mode by its debug version, libprotobufd.lib and libprotobufd.dll.
I'm writing a little Domain Specific Language for my program, using JUCE::JavascriptEngine as the scripting engine. This takes a string as input and then parses it, but I need to do some pre-processing on the string to adapt it from my DSL to JavaScript. The pre-processing mainly consists of wrapping some terms inside functions, and placing object names in front of functions. So, for instance, I want to do something like this:
take some special string input "~/1/2"...
wrap it inside a function: "find("~/1/2")"...
and then attach an object to it: "someObject.find("~/1/2")" (the object name has to be a variable).
I've been using regex for this (now I have two problems...). The regexes are getting complicated and unreadable, and it's missing a lot of special cases. Since what I'm doing is grammatical, I thought I'd upgrade from regex to a proper parser (now I have three problems...). After quite a lot of research, I chose Boost.Spirit. I've been going through the documentation, but it's not taking me in the right direction. Can someone suggest how I might use this library to manipulate strings in the way I am looking for? Given that I am only trying to manipulate a string and am not interested in storing the parsed data, do I need to use karma for the output, or can I output the string with qi or x3, during the parsing process?
If I'm headed down the wrong path here, please feel free to re-direct me.
This seems too broad to answer.
What you're doing is parsing input, and transforming it to something else. What you're not doing is find/replace (otherwise you'd be fine using regular expressions).
Of course you can do what regular expressions do, but I'm not sure it buys you anything:
template <typename It, typename Out>
Out preprocess(It f, It l, Out out) {
namespace qi = boost::spirit::qi;
using boost::spirit::repository::qi::seek;
auto passthrough = [&out](boost::iterator_range<It> ignored, auto&&...) {
for (auto ch : ignored) *out++ = ch;
};
auto transform = [&out](std::string const& literal, auto&&...) {
for (auto ch : "someObject.find(\"~"s) *out++ = ch;
for (auto ch : literal) *out++ = ch;
for (auto ch : "\")"s) *out++ = ch;
};
auto pattern = qi::copy("\"~" >> (*~qi::char_('"')) >> '"');
qi::rule<It> ignore = qi::raw[+(!pattern >> qi::char_)] [passthrough];
qi::parse(f, l, -qi::as_string[pattern][transform] % ignore);
return out;
}
The nice thing about this way of writing it, is that it will work with any source iterator:
for (std::string const input : {
R"(function foo(a, b) { var path = "~/1/2"; })",
})
{
std::cout << "Input: " << input << "\n";
std::string result;
preprocess(begin(input), end(input), back_inserter(result));
std::cout << "Result: " << result << "\n";
}
std::cout << "\n -- Or directly transformed stdin to stdout:\n";
preprocess(
boost::spirit::istream_iterator(std::cin >> std::noskipws), {},
std::ostreambuf_iterator<char>(std::cout));
See it Live On Coliru, printing the output:
Input: function foo(a, b) { var path = "~/1/2"; }
Result: function foo(a, b) { var path = someObject.find("~/1/2"); }
-- Or directly transformed stdin to stdout:
function bar(c, d) { var path = someObject.find("~/1/42"); }
But this is very limited since it will not even do the right thing if such things are parts of comments or multiline strings etc.
So instead you probably want a dedicated library that knows how to parse javascript and use it to do your transformation, such as (one of the first hits when googling tooling library preprocess javascript transform): https://clojurescript.org/reference/javascript-library-preprocessing
UPDATE 2 / TL;DR
Is there some way to prevent dirty pages of a windows FILE_FLAG_DELETE_ON_CLOSE temporary file from being flushed as a result of closing memory maps opened on these files?
Yes. If you do not need to do anything with the files themselves after their initial creation and you implement some naming conventions, this is possible through the strategy explained in this answer.
Note: I am still quite interested in finding out the reasons for why there is so much difference in the behavior depending on how maps are created and the order of disposal/unmapping.
I have been looking into some strategies for an inter-process shared memory data structure that allows growing and shrinking its committed capacity on windows by using a chain of "memory chunks."
One possible way is to use pagefile backed named memory maps as the chunk memory. An advantage of this strategy is the possibility to use SEC_RESERVE to reserve a big chunk of memory address space and incrementally allocate it using VirtualAlloc with MEM_COMMIT. Disadvantages appear to be (a) the requirement to have SeCreateGlobalPrivilege permissions to allow using a shareable name in the Global\ namespace and (b) the fact that all committed memory contributes to the system commit charge.
To circumvent these disadvantages, I started investigating the use of temporary file backed memory maps. I.e. memory maps over files created using the FILE_FLAG_DELETE_ON_CLOSE | FILE_ATTRIBUTE_TEMPORARY flags combination. This appears to be a recommended strategy that according to e.g. this blog post should prevent flushing the mapped memory to disk (unless memory pressure causes dirty mapped pages to be paged out).
I am however observing that closing the map/file handle before the owning process exits, causes dirty pages to be flushed to disk. This occurs even if the view/file handle is not the one through which the dirty pages were created and when these views/file handles were opened after the pages were 'dirtied' in a different view.
It appears that changing the order of disposal (i.e. unmapping the view first or closing the file handle first) has some impact on when the disk flush is initiated, but not on the fact that flushing takes place.
So my questions are:
Is there some way to use temporary file backed memory maps and prevent them from flushing dirty pages when the map/file is closed, taking into account that multiple threads within a process/multiple processes may have open handles/views to such a file?
If not, what is/could be the reason for the observed behavior?
Is there an alternative strategy that I may have overlooked?
UPDATE
Some additional info: When running the "arena1" and "arena2" parts of the sample code below in two separate (independent) processes, with "arena1" being the process that creates the shared memory regions and "arena2" the one that opens them, the following behavior is observed for maps/chunks that have dirty pages:
If closing the view before the file handle in the "arena1" process, it flushes each of these chunks to disk in what seems a (partially) synchronous process (i.e. it blocks the disposing thread for several seconds), independent of whether or not the "arena2" process was started.
If closing the file handle before the view, disk flushes only occur for those maps/chunks that are closed in the "arena1" process while the "arena2" process still has an open handle to those chunks, and they appear to be 'asynchronous', i.e. not blocking the application thread.
Refer to the (c++) sample code below that allows reproducing the problem on my system (x64, Win7):
static uint64_t start_ts;
static uint64_t elapsed() {
return ::GetTickCount64() - start_ts;
}
class PageArena {
public:
typedef uint8_t* pointer;
PageArena(int id, const char* base_name, size_t page_sz, size_t chunk_sz, size_t n_chunks, bool dispose_handle_first) :
id_(id), base_name_(base_name), pg_sz_(page_sz), dispose_handle_first_(dispose_handle_first) {
for (size_t i = 0; i < n_chunks; i++)
chunks_.push_back(new Chunk(i, base_name_, chunk_sz, dispose_handle_first_));
}
~PageArena() {
for (auto i = 0; i < chunks_.size(); ++i) {
if (chunks_[i])
release_chunk(i);
}
std::cout << "[" << ::elapsed() << "] arena " << id_ << " destructed" << std::endl;
}
pointer alloc() {
auto ptr = chunks_.back()->alloc(pg_sz_);
if (!ptr) {
chunks_.push_back(new Chunk(chunks_.size(), base_name_, chunks_.back()->capacity(), dispose_handle_first_));
ptr = chunks_.back()->alloc(pg_sz_);
}
return ptr;
}
size_t num_chunks() {
return chunks_.size();
}
void release_chunk(size_t ndx) {
delete chunks_[ndx];
chunks_[ndx] = nullptr;
std::cout << "[" << ::elapsed() << "] chunk " << ndx << " released from arena " << id_ << std::endl;
}
private:
struct Chunk {
public:
Chunk(size_t ndx, const std::string& base_name, size_t size, bool dispose_handle_first) :
map_ptr_(nullptr), tail_(nullptr),
handle_(INVALID_HANDLE_VALUE), size_(0),
dispose_handle_first_(dispose_handle_first) {
name_ = name_for(base_name, ndx);
if ((handle_ = create_temp_file(name_, size)) == INVALID_HANDLE_VALUE)
handle_ = open_temp_file(name_, size);
if (handle_ != INVALID_HANDLE_VALUE) {
size_ = size;
auto map_handle = ::CreateFileMappingA(handle_, nullptr, PAGE_READWRITE, 0, 0, nullptr);
tail_ = map_ptr_ = (pointer)::MapViewOfFile(map_handle, FILE_MAP_ALL_ACCESS, 0, 0, size);
::CloseHandle(map_handle); // no longer needed.
}
}
~Chunk() {
if (dispose_handle_first_) {
close_file();
unmap_view();
} else {
unmap_view();
close_file();
}
}
size_t capacity() const {
return size_;
}
pointer alloc(size_t sz) {
pointer result = nullptr;
if (tail_ + sz <= map_ptr_ + size_) {
result = tail_;
tail_ += sz;
}
return result;
}
private:
static const DWORD kReadWrite = GENERIC_READ | GENERIC_WRITE;
static const DWORD kFileSharing = FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE;
static const DWORD kTempFlags = FILE_ATTRIBUTE_NOT_CONTENT_INDEXED | FILE_FLAG_DELETE_ON_CLOSE | FILE_ATTRIBUTE_TEMPORARY;
static std::string name_for(const std::string& base_file_path, size_t ndx) {
std::stringstream ss;
ss << base_file_path << "." << ndx << ".chunk";
return ss.str();
}
static HANDLE create_temp_file(const std::string& name, size_t& size) {
auto h = CreateFileA(name.c_str(), kReadWrite, kFileSharing, nullptr, CREATE_NEW, kTempFlags, 0);
if (h != INVALID_HANDLE_VALUE) {
LARGE_INTEGER newpos;
newpos.QuadPart = size;
::SetFilePointerEx(h, newpos, 0, FILE_BEGIN);
::SetEndOfFile(h);
}
return h;
}
static HANDLE open_temp_file(const std::string& name, size_t& size) {
auto h = CreateFileA(name.c_str(), kReadWrite, kFileSharing, nullptr, OPEN_EXISTING, kTempFlags, 0);
if (h != INVALID_HANDLE_VALUE) {
LARGE_INTEGER sz;
::GetFileSizeEx(h, &sz);
size = sz.QuadPart;
}
return h;
}
void close_file() {
if (handle_ != INVALID_HANDLE_VALUE) {
std::cout << "[" << ::elapsed() << "] " << name_ << " file handle closing" << std::endl;
::CloseHandle(handle_);
std::cout << "[" << ::elapsed() << "] " << name_ << " file handle closed" << std::endl;
}
}
void unmap_view() {
if (map_ptr_) {
std::cout << "[" << ::elapsed() << "] " << name_ << " view closing" << std::endl;
::UnmapViewOfFile(map_ptr_);
std::cout << "[" << ::elapsed() << "] " << name_ << " view closed" << std::endl;
}
}
HANDLE handle_;
std::string name_;
pointer map_ptr_;
size_t size_;
pointer tail_;
bool dispose_handle_first_;
};
int id_;
size_t pg_sz_;
std::string base_name_;
std::vector<Chunk*> chunks_;
bool dispose_handle_first_;
};
static void TempFileMapping(bool dispose_handle_first) {
const size_t chunk_size = 256 * 1024 * 1024;
const size_t pg_size = 8192;
const size_t n_pages = 100 * 1000;
const char* base_path = "data/page_pool";
start_ts = ::GetTickCount64();
if (dispose_handle_first)
std::cout << "Mapping with 2 arenas and closing file handles before unmapping views." << std::endl;
else
std::cout << "Mapping with 2 arenas and unmapping views before closing file handles." << std::endl;
{
std::cout << "[" << ::elapsed() << "] " << "allocating " << n_pages << " pages through arena 1." << std::endl;
PageArena arena1(1, base_path, pg_size, chunk_size, 1, dispose_handle_first);
for (size_t i = 0; i < n_pages; i++) {
auto ptr = arena1.alloc();
memset(ptr, (i + 1) % 256, pg_size); // ensure pages are dirty.
}
std::cout << "[" << elapsed() << "] " << arena1.num_chunks() << " chunks created." << std::endl;
{
PageArena arena2(2, base_path, pg_size, chunk_size, arena1.num_chunks(), dispose_handle_first);
std::cout << "[" << ::elapsed() << "] arena 2 loaded, going to release chunks 1 and 2 from arena 1" << std::endl;
arena1.release_chunk(1);
arena1.release_chunk(2);
}
}
}
Please refer to this gist that contains the output of running the above code and links to screen captures of system free memory and disk activity when running TempFileMapping(false) and TempFileMapping(true) respectively.
After the bounty period expired without any answers that provided more insight or solved the mentioned problem, I decided to dig a little deeper and experiment some more with several combinations and sequences of operations.
As a result, I believe I have found a way to achieve memory maps shared between processes over temporary, delete-on-close files, that are not being flushed to disk when they are closed.
The basic idea involves creating the memory map when a temp file is newly created with a map name that can be used in a call to OpenFileMapping:
// build a unique map name from the file name.
auto map_name = make_map_name(file_name);
// Open or create the mapped file.
auto mh = ::OpenFileMappingA(FILE_MAP_ALL_ACCESS, false, map_name.c_str());
if (mh == 0 || mh == INVALID_HANDLE_VALUE) {
// existing map could not be opened, create the file.
auto fh = ::CreateFileA(name.c_str(), kReadWrite, kFileSharing, nullptr, CREATE_NEW, kTempFlags, 0);
if (fh != INVALID_HANDLE_VALUE) {
// set its size.
LARGE_INTEGER newpos;
newpos.QuadPart = desired_size;
::SetFilePointerEx(fh, newpos, 0, FILE_BEGIN);
::SetEndOfFile(fh);
// create the map
mh = ::CreateFileMappingA(mh, nullptr, PAGE_READWRITE, 0, 0, map_name.c_str());
// close the file handle
// from now on there will be no accesses using file handles.
::CloseHandle(fh);
}
}
Thus, the file handle is only used when the file is newly created, and closed immediately after the map is created, while the map handle itself remains open, to allow opening the mapping without requiring access to a file handle. Note that a race condition exists here, that we would need to deal with in any "real code" (as well as adding decent error checking and handling).
So if we got a valid map handle, we can create the view:
auto map_ptr = MapViewOfFile(mh, FILE_MAP_ALL_ACCESS, 0, 0, 0);
if (map_ptr) {
// determine its size.
MEMORY_BASIC_INFORMATION mbi;
if (::VirtualQuery(map_ptr, &mbi, sizeof(MEMORY_BASIC_INFORMATION)) > 0)
map_size = mbi.RegionSize;
}
When, some time later closing a mapped file: close the map handle before unmapping the view:
if (mh == 0 || mh == INVALID_HANDLE_VALUE) {
::CloseHandle(mh);
mh = INVALID_HANDLE_VALUE;
}
if (map_ptr) {
::UnmapViewOfFile(map_ptr);
map_ptr = 0;
map_size = 0;
}
And, according to the test I have performed so far, this does not cause flushing dirty pages to disk on close, problem solved. Well partially anyway, there may still be a cross-session map name sharing issue.
If I take it correctly, commenting out Arena2 part of code shall reproduce the issue without the need for second process. I have tried this:
I edited base_path as follows for convenience:
char base_path[MAX_PATH];
GetTempPathA(MAX_PATH, base_path);
strcat_s(base_path, MAX_PATH, "page_pool");
I edited n_pages = 1536 * 128 to bring the used memory to 1.5GB, compared to your ~800mb.
I have tested TempFileMapping(false) and TempFileMapping(true), one at a time, for the same results.
I have tested with Arena2 commented out and intact, for the same results.
I have tested on Win8.1 x64 and Win7 x64, for ±10% same results.
In my tests, code runs in 2400ms ±10%, only 500ms ±10% spent on deallocating. That's clearly not enough for a flush of 1.5GB on a low-spinning silent HDDs I have there.
So, the question is, what are you observing? I'd suggest that you:
Provide your times for comparison
Use a different computer for tests, paying attention to excluding software issues such as "same antivirus"
Verify that you're not experiencing a RAM shortage.
Use xperf to see what's happening during the freeze.
Update
I have tested on yet another Win7 x64, and times are 890ms full, 430ms spent on dealloc. I have looked into your results, and what is VERY suspicious is that almost exactly 4000ms is spent in freeze each time on your machine. That can't be a mere coincidence, I believe. Also, it's rather obvious now the the problem is somehow bound to a specific machine you're using. So my suggestions are:
As stated above, test on another computer yourself
As stated above, Use XPerf, it will allow you to see what exactly happens in user mode and kernel mode during the freeze (I really suspect some non-standard driver in the middle)
Play with number of pages and see how it affects the freeze length.
Try to store files on a different disk drive on the same computer where you have tested initially.
I populate a very large array using a stxxl::VECTOR_GENERATOR<MyData>::result::bufwriter_type (something like 100M entries) which I need to sort in parallel.
I use the stxxl::sort(vector->begin(), vector->end(), cmp(), memoryAmount) method, which in theory should do what I need: sort the elements very efficiently.
However, during the execution of this method I noticed that only one processor is fully utilised, and all the other cores are quite idle (I suspect there is little activity to fetch the input, but in practice they don't do anything).
This is my question: is it possible to exploit more cores during the sorting phase, or is the parallelism used only to fetch the input asynchronously? If so, are there documents that explain how to enable it? (I looked extensively the documentation on the website, but I couldn't find anything).
Thanks very much!
EDIT
Thanks for the suggestion. I provide below some more information.
First of all I use MacOs for my experiments. What I do is that I launch the following program and I study its behaviour.
typedef struct Triple {
long t1, t2, t3;
Triple(long s, long p, long o) {
this->t1 = s;
this->t2 = p;
this->t3 = o;
}
Triple() {
t1 = t2 = t3 = 0;
}
} Triple;
const Triple minv(std::numeric_limits<long>::min(),
std::numeric_limits<long>::min(), std::numeric_limits<long>::min());
const Triple maxv(std::numeric_limits<long>::max(),
std::numeric_limits<long>::max(), std::numeric_limits<long>::max());
struct cmp: std::less<Triple> {
bool operator ()(const Triple& a, const Triple& b) const {
if (a.t1 < b.t1) {
return true;
} else if (a.t1 == b.t1) {
if (a.t2 < b.t2) {
return true;
} else if (a.t2 == b.t2) {
return a.t3 < b.t3;
}
}
return false;
}
Triple min_value() const {
return minv;
}
Triple max_value() const {
return maxv;
}
};
typedef stxxl::VECTOR_GENERATOR<Triple>::result vector_type;
int main(int argc, const char** argv) {
vector_type vector;
vector_type::bufwriter_type writer(vector);
for (int i = 0; i < 1000000000; ++i) {
if (i % 10000000 == 0)
std::cout << "Inserting element " << i << std::endl;
Triple t;
t.t1 = rand();
t.t2 = rand();
t.t3 = rand();
writer << t;
}
writer.finish();
//Sort the vector
stxxl::sort(vector.begin(), vector.end(), cmp(), 1024*1024*1024);
std::cout << vector.size() << std::endl;
}
Indeed there seems to be only one or maximum two threads working during the execution of this program. Notice that the machine has only a single disk.
Can you please confirm me whether the parallelism work on macos? If not, then I will try to use linux to see what happens. Or is perhaps because there is only one disk?
In principle what you are doing should work out-of-the-box. With everything working you should see all cores doing processing.
Since it doesnt work, we'll have to find the error, and debugging why we see no parallel speedups is still tricky business these days.
The main idea is to go from small to large examples:
what platform is this? There is no parallelism on MSVC, only on Linux/gcc.
By default STXXL builds on Linux/gcc with USE_GNU_PARALLEL. you can turn it off to see if it has an effect.
Try reproducing the example values shown in http://stxxl.sourceforge.net/tags/master/stxxl_tool.html - with and without USE_GNU_PARALLEL
See if just in memory parallel sorting scales on your processor/system.
I serialize multiple objects into a binary archive with Boost.
When reading back those objects from a binary_iarchive, is there a way to know how many objects are in the archive or simply a way to detect the end of the archive ?
The only way I found is to use a try-catch to detect the stream exception.
Thanks in advance.
I can think of a number of approaches:
Serialize STL containers to/from your archive (see documentation). The archive will automatically keep track of how many objects there are in the containers.
Serialize a count variable before serializing your objects. When reading back your objects, you'll know beforehand how many objects you expect to read back.
You could have the last object have a special value that acts as a kind of sentinel that indicates the end of the list of objects. Perhaps you could add an isLast member function to the object.
This is not very pretty, but you could have a separate "index file" alongside your archive that stores the number of objects in the archive.
Use the tellp position of the underlying stream object to detect if you're at the end of file:
Example (just a sketch, not tested):
std::streampos archiveOffset = stream.tellg();
std::streampos streamEnd = stream.seekg(0, std::ios_base::end).tellg();
stream.seekg(archiveOffset);
while (stream.tellp() < streamEnd)
{
// Deserialize objects
}
This might not work with XML archives.
Do you have all your objects when you begin serializing? If not, you are "abusing" boost serialization - it is not meant to be used that way. However, I am using it that way, using try catch to find the end of the file, and it works for me. Just hide it away somewhere in the implementation. Beware though, if using it this way, you need to either not serialize pointers, or disable pointer tracking.
If you do have all the objects already, see Emile's answer. They are all valid approaches.
std::istream* stream_;
boost::iostreams::filtering_streambuf<boost::iostreams::input>* filtering_streambuf_;
...
stream_ = new std::istream(memoryBuffer_);
if (stream_) {
filtering_streambuf_ = new boost::iostreams::filtering_streambuf<boost::iostreams::input>();
if (filtering_streambuf_) {
filtering_streambuf_->push(boost::iostreams::gzip_decompressor());
filtering_streambuf_->push(*stream_);
archive_ = new eos::portable_iarchive(*filtering_streambuf_);
}
}
using zip when reading data from the archives, and filtering_streambuf have such method as
std::streamsize std::streambuf::in_avail()
Get number of characters available to read
so i check the end of archive as
bool IArchiveContainer::eof() const {
if (filtering_streambuf_) {
return filtering_streambuf_->in_avail() == 0;
}
return false;
}
It is not helping to know how many objects are last in the archive, but helping to detect the end of them
(i'm using eof test only in the unit test for serialization/unserialization my classes/structures - to make sure that i'm reading all what i'm writing)
Sample code which I used to debug the similar issue
(based on Emile's answer) :
#include <fstream>
#include <iostream>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
struct A{
int a,b;
template <typename T>
void serialize(T &ar, int ){
ar & a;
ar & b;
}
};
int main(){
{
std::ofstream ofs( "ff.ar" );
boost::archive::binary_oarchive ar( ofs );
for(int i=0;i<3;++i){
A a {2,3};
ar << a;
}
ofs.close();
}
{
std::ifstream ifs( "ff.ar" );
ifs.seekg (0, ifs.end);
int length = ifs.tellg();
ifs.seekg (0, ifs.beg);
boost::archive::binary_iarchive ar( ifs );
while(ifs.tellg() < length){
A a;
ar >> a;
std::cout << "a.a-> "<< a.a << " and a.b->"<< a.b << "\n";
}
}
return 0;
}
you just read a byte from the file.
If you do not reach the end,
backword a byte then.