I am trying to build a "failsafe" layer on top of libzip but libzip is giving me some trouble here.
First I add a file to my (empty) archive with zip_file_add(...). This has 3 possible user-defined encodings available. Then I try to locate the name with zip_name_locate(...) which also has 3 possible user-defined encodings available.
This mcve checks all possible encoding combinations and all of them fail for the specific filename x%²»Ã-ØÑ–6¨wx.txt. When using a more conventional file.txt filename, zip_name_locate() succeeds every time.
#include <zip.h>
#include <include/libzip.h>//<.pragmas to include the .lib's...
#include <iostream>
#include <vector>
#include <utility>
/*
'zip_file_add' possible encodings:
ZIP_FL_ENC_GUESS
ZIP_FL_ENC_UTF_8
ZIP_FL_ENC_CP437
'zip_name_locate' possible encodings:
ZIP_FL_ENC_RAW
ZIP_FL_ENC_GUESS
ZIP_FL_ENC_STRICT
*/
/*
build encoding pairs (trying all possibilities)
*/
std::vector<std::pair<unsigned, unsigned>>
encoding_pairs{
{ ZIP_FL_ENC_GUESS, ZIP_FL_ENC_RAW },
{ ZIP_FL_ENC_UTF_8, ZIP_FL_ENC_RAW },
{ ZIP_FL_ENC_CP437, ZIP_FL_ENC_RAW },
{ ZIP_FL_ENC_GUESS, ZIP_FL_ENC_GUESS },
{ ZIP_FL_ENC_UTF_8, ZIP_FL_ENC_GUESS },
{ ZIP_FL_ENC_CP437, ZIP_FL_ENC_GUESS },
{ ZIP_FL_ENC_GUESS, ZIP_FL_ENC_STRICT },
{ ZIP_FL_ENC_UTF_8, ZIP_FL_ENC_STRICT },
{ ZIP_FL_ENC_CP437, ZIP_FL_ENC_STRICT },
};
int main(int argc, char** argv) {
const char* file_buf = "hello world";
#if 0
const char* file_name = "file.txt";
#else
const char* file_name = "x%²»Ã-ØÑ–6¨wx.txt";
#endif
zip_error_t ze;
zip_error_init(&ze);
{
zip_source_t* zs = zip_source_buffer_create(nullptr, 0, 1, &ze);
if (zs == NULL)
return -1;
zip_t* z = zip_open_from_source(zs, ZIP_CHECKCONS, &ze);
if (z == NULL)
return -1;
{
zip_source_t* s = zip_source_buffer(z, file_buf, strlen(file_buf), 0);//0 = don't let libzip auto-free the const char* buffer on the stack
if (s == NULL)
return -1;
for (size_t ep = 0; ep < encoding_pairs.size(); ep++) {
std::cout << "ep = " << ep << std::endl;
zip_uint64_t index;
if ((index = zip_file_add(z, file_name, s, encoding_pairs[ep].first)) == -1) {
std::cout << "could not zip_file_add() with encoding " << encoding_pairs[ep].first << std::endl;
continue;
}
if (zip_name_locate(z, file_name, encoding_pairs[ep].second) == -1) {
std::cout << "the name '" << file_name << "' could not be located." << std::endl;
std::cout << " encoding pair: " << encoding_pairs[ep].first << " <-> " << encoding_pairs[ep].second << std::endl;
}
else {
std::cout << "the name was located." << std::endl;
}
if (zip_delete(z, index) == -1)
return -1;
}
}
zip_close(z);
}
zip_error_fini(&ze);
return 0;
}
I don't understand what I might be doing wrong here or if libzip just can't even resolve such a name.
If it can't then what would be the criteria on names to avoid ?
It turns out the problem was the encoding of my source file itself. It was ANSI - So I converted it to UTF8 and it solved the issue.
What I still don't understand is why libzip can't zip_name_locate() a name from an input c-string that is exactly the same as the input c-string used in zip_file_add() (whatever the source file encoding might be). "Lost in translation" perhaps ?
(Special thanks to Thomas Klausner for helping me find the issue).
My first version of this question was rich with misunderstandings. My answer below suits my needs. But I kept at it to understand what could be done with with<>. What I get is that it intended to inject context into a parser. Then the parser is called from with_directive::parse (in x3's with.hpp) In the following code, that is just what happens.
#include <boost/spirit/home/x3.hpp>
using namespace boost::spirit::x3;
struct eol_parser_cnt : parser<eol_parser_cnt>
{
struct context {
int line = 0;
std::string::iterator iter_pos;
};
template <typename Iterator, typename Context, typename Attribute>
bool parse(Iterator& first, Iterator const& last
, Context const& context, unused_type, Attribute& attr) const
{
//std::cout << context.line;
auto& ctx = context;
return boost::spirit::x3::parse(first, last, lit(' ') | (lit("//") >> *(char_ - eol) >> eol));
}
};
const auto& our_skipper = eol_parser_cnt{};
eol_parser_cnt::context lines;
auto with_skipper = with<eol_parser_cnt::context>(lines)[our_skipper];
int main()
{
std::string str("12 word");
auto first = str.begin();
phrase_parse(first, str.end(), int_ >> *char_("a-z"), with_skipper);
}
Putting a break point in eol_parser_cnt::parse and I see it working. The debugger consistently shows the context is there and that it is the structure of eol_parser_cnt::context. I can change the value of line in the debugger and the next hit shows that value, it is a real object. But, try to uncomment the line std::cout << context.line; and the compiler complains that is it an unused_type. So, I just don't get it.
test.cpp(15,30): error C2039: 'line': is not a member of 'boost::spirit::x3::context<ID,T,Context>'
with
[
ID=eol_parser_cnt::context,
T=eol_parser_cnt::context,
Context=boost::spirit::x3::unused_type
]
F:\cpp\boost_1_76_0\boost\spirit\home\x3\support\context.hpp(18): message : see declaration of 'boost::spirit::x3::context<ID,T,Context>'
with
[
ID=eol_parser_cnt::context,
T=eol_parser_cnt::context,
Context=boost::spirit::x3::unused_type
]
F:\cpp\boost_1_76_0\boost\spirit\home\x3\directive\with.hpp(62): message : see reference to function template instantiation 'bool eol_parser_cnt::parse<Iterator,boost::spirit::x3::context<ID,T,Context>,Attribute>(Iterator &,const Iterator &,const boost::spirit::x3::context<ID,T,Context> &,boost::spirit::x3::unused_type,Attribute &) const' being compiled
with
[
Iterator=std::_String_iterator<std::_String_val<std::_Simple_types<char>>>,
ID=eol_parser_cnt::context,
T=eol_parser_cnt::context,
Context=boost::spirit::x3::unused_type,
Attribute=const boost::spirit::x3::unused_type
]
Well, it took a while but I was going to understand this. I re-read C++ Template Metaprogramming seriously this time. Then looking at the x3 code I finally understood that with<> is just another parser wrapper. It is interesting to debug an optimized build and see just how much code disappears. VS shows all the disappeared stuff on the stack as inlined and what was 9 layers of parser calls becomes 2 into the likes of with_error_handling::on_error. Everything from my call to parse_rhs_main (in rule.hpp, line 232), is gone.
So because with<> is just another parser wrapper and if the skipper is wrapped, the context is only avaliable to the skipper. But no big deal as the skipper is in the context of the main parser. The difference is that in the skipper object a get<skipper_eol_cnt>(context) returns a reference to the skipper_eol_cnt::context{}. Whereas in the main parser we have to us get<skipper_tag>(context) and this returns the with<> parser object. val is a member of that parser, the reference to the skipper_eol_cnt::context{}. So get<skipper_tag>(context).val retrieves the context we are looking for.
So here it is, using with<> applied to a skipper.
#include<iostream>
#include <iomanip>
#include <vector>
//#define BOOST_SPIRIT_X3_DEBUG
#include <boost/spirit/home/x3.hpp>
#include <boost/fusion/adapted.hpp>
using namespace boost::spirit::x3;
struct skipper_eol_cnt : parser<skipper_eol_cnt>
{
struct context {
int line = 1;
std::string::iterator iter_pos;
};
template <typename Iterator, typename Context, typename Attribute>
bool parse(Iterator& first, Iterator const& last
, Context const& context, unused_type, Attribute& attr) const
{
const char* start_cmt = "/*";
const char* end_cmt = "*/";
if (first == last)
return false;
bool matched = false;
//here we are getting from the 'with<>' wrapper
auto& ctx = get<skipper_eol_cnt>(context);
//skip: space | '//comment'
boost::spirit::x3::parse(first, last, lit(' ') | (lit("//") >> *(char_ - eol)));//eol counted below
//skip: '/*comment*/'
if (detail::string_parse(start_cmt, first, last, unused, case_compare<Iterator>())) {
for (; first != last; ++first) {
if (detail::string_parse(end_cmt, first, last, unused, case_compare<Iterator>()))
break;
if (*first == '\n')
++ctx.line, ctx.iter_pos = first;
}
}
Iterator iter = first;
for (; iter != last && (*iter == '\r' || *iter == '\n'); ++iter) {
matched = true;
if (*iter == '\n') // LF
++ctx.line, ctx.iter_pos = iter;
}
//{static int pos = 0; if (pos < ctx.line) { pos = ctx.line; std::cout << pos << std::endl; }}
if (matched) first = iter;
return matched;
}
};
auto const& skip_eol_cnt = skipper_eol_cnt{};
struct with_error_handling {
template<typename It, typename Ctx>
error_handler_result on_error(It f, It l, expectation_failure<It> const& ef, Ctx const& ctx) const {
It erit = f + std::distance(f, ef.where());
//here we are getting the wrapped skipper so need the 'with<>.val'
const auto& sctx = get<skipper_tag>(ctx).val;
It bit = erit;
for (; *bit != '\n'/* && bit != f*/; --bit)
;
It eit = erit;
for (; *eit != '\n' && eit != l; ++eit)
;
int str_pos = erit - bit - 1;
std::ostringstream oss;
oss << "Expecting " << ef.which() << "\n at line: " << sctx.line
<< "\n\t" << std::string(bit + 1, eit)
<< "\n\t" << std::setw(str_pos) << std::setfill('-') << "" << "^";
get<with_error_handling>(ctx).push_back(oss.str());;
return error_handler_result::fail;
}
};
//attr sections
struct section_type {
std::string name;
int line;
std::string::iterator iter_pos;
};
BOOST_FUSION_ADAPT_STRUCT(section_type, name)
using sections_type = std::vector<section_type>;
struct parser_find_sections : parser<parser_find_sections> {
template<typename Iterator, typename Context, typename RContext, typename Attribute>
bool parse(Iterator& first, Iterator const& last, Context const& context, RContext const& rcontext, Attribute& section) const {
const auto& sssctx = get<skipper_eol_cnt>(context); //now here this doesn't work, unused_type, but
const auto& sctx = get<skipper_tag>(context).val;
auto to_line = [&sctx, first](auto& ctx) {
_attr(ctx).line = sctx.line;
//_attr(ctx).iter_pos = first; // this one will get at 'section color(x,x)'
_attr(ctx).iter_pos = _where(ctx).begin(); // this one is '(x,x)'
};
static_assert(BOOST_VERSION / 100 % 1000 >= 77);
////NOTE!!! if you have a boost version of less than 1.77, x3::seek will fail here
////quick fix, copy from: https://github.com/boostorg/spirit/blob/boost-1.78.0/include/boost/spirit/home/x3/directive/seek.hpp
////and paste to your boost file...
return phrase_parse(first, last, *(seek["section"] >> (*alpha)[to_line]), get<skipper_tag>(context), section);
}
};
auto const parse_section = rule<with_error_handling, std::pair<int, int>>("the_sec_parser") = [] {
return '(' > int_ > ',' > int_ > ')';
}();
template<typename T>
std::ostream& operator << (std::ostream& os, std::pair<T, T>& t) {
return os << t.first << ',' << t.second;
}
//errors
std::vector<std::string> errors;
auto with_errors = with<with_error_handling>(errors)[parse_section];
auto test_section = [](auto& content, auto& section) {
//attr
std::pair<int, int> attr;
skipper_eol_cnt::context ctx{ section.line, section.iter_pos };
auto with_skip_cnt = with< skipper_eol_cnt>(ctx)[skip_eol_cnt];
auto first(section.iter_pos);
return std::tuple(phrase_parse(first, content.end(), with_errors, with_skip_cnt, attr), attr);
};
int main() {
std::string str(R"(//line 1
section red(5, 6)
section green( 7, 8) //line 3
section blue(9, 10) //no error
/*comment
bunch of lines of stuff....
*/ section white(11, a 12) //error on line 7
section black( 13,14)
)");
//get the list of sections
auto with_skip_cnt = with<skipper_eol_cnt>(skipper_eol_cnt::context{})[skip_eol_cnt];
sections_type secs;
auto first(str.begin());
phrase_parse(first, str.end(), parser_find_sections(), with_skip_cnt, secs);
for (auto& item : secs)
std::cout << item.name << "\t at line: " << item.line << std::endl;
//section 'blue', at 2, has no error
auto [r, attr] = test_section(str, secs.at(2));
if (r)
std::cout << "\nthe " << secs.at(2).name << " hase vals: " << attr << "\n\n";
//section 'white', at 3, has an error
test_section(str, secs.at(3));
if (errors.size())
std::cout << errors.front() << std::endl;
return 0;
}
I have trying to solve a bug on the following code, where I get an hour difference from boost last_write_time.
To explain it better: I create a file, and then I try to extract the time it was created with boost::filesystem::path.
void PrintTime(boost::filesystem::path _file) {
time_t sys_time{ last_write_time(_file) };
ptime p_time{ boost::posix_time::from_time_t(sys_time) };
boost::posix_time::time_duration time_dur{ p_time.time_of_day() };
long h{ time_dur.hours() }; //1a
long m{ time_dur.minutes() };
long s{ time_dur.seconds() };
//...print h, m, s.
}
//1a: Here when for example the time I expect is 12:34:56,
//I always get 11:34:56
Any idea why is that?
Is there timezone somewhere in boost last_write_time?
My os displays the right time when I check the file through the system.
You have to translate to the "presentation" time-zone, like "when [you] check the file through the system". The timestamp from the filesystem is UTC time.
E.g. if you do
std::cout << boost::posix_time::second_clock::local_time() << "\n";
std::cout << boost::posix_time::second_clock::universal_time() << "\n";
you'll probably get
2018-Feb-27 16:03:12
2018-Feb-27 15:03:12
Fix:
#include <boost/date_time/c_local_time_adjustor.hpp>
void PrintTime(boost::filesystem::path _file) {
using boost::posix_time::ptime;
using adj = boost::date_time::c_local_adjustor<ptime>;
time_t const sys_time = last_write_time(_file);
ptime const utc = boost::posix_time::from_time_t(sys_time);
ptime const local = adj::utc_to_local(utc);
DEMO
See it Live On Coliru
#include <boost/filesystem.hpp>
#include <boost/date_time/posix_time/posix_time_io.hpp>
#include <boost/date_time/posix_time/conversion.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>
#include <boost/date_time/c_local_time_adjustor.hpp>
void PrintTime(boost::filesystem::path _file) {
using boost::posix_time::ptime;
using adj = boost::date_time::c_local_adjustor<ptime>;
time_t const sys_time = last_write_time(_file);
ptime const utc = boost::posix_time::from_time_t(sys_time);
ptime const local = adj::utc_to_local(utc);
std::cout << "utc: " << utc << "\n";
std::cout << "local: " << local << "\n";
{
long h{ local.time_of_day().hours() };
long m{ local.time_of_day().minutes() };
long s{ local.time_of_day().seconds() };
//...print h, m, s.
std::cout << h << ":" << m << ":" << s << '\n';
}
}
int main() {
PrintTime("main.cpp");
}
Prints (on my system):
utc: 2018-Feb-27 15:19:45
local: 2018-Feb-27 16:19:45
16:19:45
I am using boost message queue to communicate among different processes. I am transmitting an object of type Packet. To do this, I am using serialization and deserialization in send and receive functions.
However, when I try to send the data, I am getting this error:
boost::interprocess_exception::library_error
No other information is given.
This is how I create message queues.
for(i = 0; i< PROC_MAX_E ; i++){
std::string mqName = std::string("mq") + std::to_string(i);
std::cout << " Size of Packet is " << sizeof(Packet) << std::endl;
message_queue mq(open_or_create, mqName.c_str(), MAX_QUEUE_SIZE_E, 100*sizeof(Packet)); // size of packet later
}
This is my Packet :
class Packet{
public :
Packet();
Packet(uint32_t aType, uint32_t aProcId);
~Packet();
uint32_t getType();
union{
uint32_t mFuncId;
//uint8_t mResult8;
uint32_t mResult32;
//uint64_t mResult64;
//bool mResult;
//uint8_t* mAddr8;
//uint32_t* mAddr32;
//uint64_t* mAddr64;
//char mData[MAX_PACKET_SIZE]; // This will be used to store serialized data
};
friend class boost::serialization::access;
template <class Archive>
void serialize(Archive & ar, const unsigned int version){
ar & _mType;
ar & _mProcId;
//ar & mData;
ar & mFuncId;
//ar & mResult32;
}
private :
uint32_t _mType;
uint32_t _mProcId;
}; // end class
} // end namespace
This is my serialize and deserialize functions:
std::string IPC::_serialize(Packet aPacket){
std::stringstream oss;
boost::archive::text_oarchive oa(oss);
oa << aPacket;
std::string serialized_string (oss.str());
return serialized_string;
}
Packet IPC::_deserialize(std::string aData){
Packet p;
std::stringstream iss;
iss << aData;
boost::archive::text_iarchive ia(iss);
ia >> p;
return p;
}
And this is my send and receive functions:
bool IPC::send(uint32_t aProcId, Packet aPacket){
try{
_mLogFile << "<-- Sending Data to Process : " << aProcId << std::endl;
//uint32_t data = aPacket;
std::string mqName = std::string("mq") + std::to_string(aProcId);
message_queue mq(open_only, mqName.c_str());
//serialize Packet
std::cout << "Serializing \n";
std::string data = _serialize(aPacket);
std::cout << " Serialized data =" << data.data() << "Size = " << data.size()<< std::endl;
mq.send(data.data(), data.size(), 0);
//mq.send(&data, sizeof(uint32_t), 0);
}catch(interprocess_exception &ex){
_mLogFile << "***ERROR*** in IPC Send to process : " << aProcId << " " << ex.what() << std::endl;
std::cout << "***ERROR*** in IPC Send to process : " << aProcId << " " << ex.what() << std::endl;
_ipc_exit();
}
}
I am getting exception during mq.send
When I transmit only integers it works fine. Only with serialization and deserialization, I get this error
Any help is greatly appreciated.I am a little stuck as the exception message is also not clear.
I am using boost 1_57_0
Rgds
Sapan
Try closing or flushing the string steam before using the string.
std::string IPC::_serialize(Packet aPacket){
std::stringstream oss;
{
boost::archive::text_oarchive oa(oss);
oa << aPacket;
}
return oss.str();
}
I have a code that finds files that are less than 20 seconds old. It's finding them, but I can't get them deleted. I am using remove(). I've tried using remove with wildcards but no luck. It has to be something with the syntax. Any help is appreciated. Thank you.
using namespace std;
typedef vector<WIN32_FIND_DATA> tFoundFilesVector;
std::wstring LastWriteTime;
int getFileList(const char * filespec, tFoundFilesVector &foundFiles)
{
WIN32_FIND_DATA findData;
HANDLE h;
int validResult=true;
int numFoundFiles = 0;
h = FindFirstFile((LPCSTR)filespec, &findData);
if (h == INVALID_HANDLE_VALUE)
return 0;
while (validResult)
{
numFoundFiles++;
foundFiles.push_back(findData);
validResult = FindNextFile(h, &findData);
}
return numFoundFiles;
}
void showFileAge(tFoundFilesVector &fileList)
{
unsigned _int64 fileTime, curTime, age;
tFoundFilesVector::iterator iter;
FILETIME ftNow;
CoFileTimeNow(&ftNow);
curTime = ((_int64) ftNow.dwHighDateTime << 32) + ftNow.dwLowDateTime;
for (iter=fileList.begin(); iter<fileList.end(); iter++)
{
fileTime = ((_int64)iter->ftLastWriteTime.dwHighDateTime << 32) + iter- >ftLastWriteTime.dwLowDateTime;
age = curTime - fileTime;
if (age <= (_int64)200000000UL)
{
wcout << " Delete: '" <<endl;
wcout << "FILE: '" << iter->cFileName << "', AGE: " << (_int64)age/10000000UL << " seconds" << endl;
remove("C:\\mapper\\iter->cFileName");
}
else
{
//wcout << " Quit: '" <<endl;
//return;
}
}
}
int main()
{
string fileSpec = "*.*";
tFoundFilesVector foundFiles;
tFoundFilesVector::iterator iter;
int foundCount = 0;
getFileList("c:\\Mapper\\*.txt", foundFiles);
getFileList("c:\\Mapper\\*.jpg", foundFiles);
foundCount = foundFiles.size();
if (foundCount)
{
wcout << "Found "<<foundCount<<" matching files.\n";
showFileAge(foundFiles);
}
system("pause");
return 0;
}
You need to concatenate the strings "C:\mapper\" with whatever is in iter->cFileName. The most simple way of doing that would be to use the std::string class.
The result would look something like this:
remove(string("c:\\mapper\\").append(string(iter->cFileName)).c_str());
This assumes you do not use unicode encoding of your strings, otherwise you need to use std::wstring. To use these classes you will also need to include
#include <string>
in the top of your file.