I have a mesh from which I need to read the vertex positions of but I can just get a buffer with that data, which I seemingly can get as an utf-8 char array.
Currently I'm getting the data from the buffer into the array I metioned and wirte it into a char* but i can't get the decoding correctly or so it seems.
The following code reads the dara from the buffer:
char* GetDataFromIBuffer(Windows::Storage::Streams::IBuffer^ container)
{
unsigned int bufferLength = container->Length;
auto dataReader = Windows::Storage::Streams::DataReader::FromBuffer(container);
Platform::Array<unsigned char>^ managedBytes =
ref new Platform::Array<unsigned char>(bufferLength);
dataReader->ReadBytes(managedBytes);
char * bytes = new char[bufferLength];
for (unsigned int i = 0; i < bufferLength; i++)
{
if (managedBytes[i] == '\0')
{
bytes[i] = '0';
}
else
{
bytes[i] = managedBytes[i];
}
}
}
I can see the data in debug mode but i need a method to make it readable and write it into a file, where i can copy the mesh data and draw the mesh in a seperate program.
The following image shows the array data which can be seen in the array:
debug mode
Be careful not to mix up text encoding and data types.
char is a type often used for buffers because it has the size of a byte, but that doesn't mean that the data contained in the buffer is text.
Your debug view seem to confirm that the data inside your buffer is not text, because when interpreted as text, it gives weird characters such as 'ΓΏ', '^', etc...
UTF-8 is a way to encode unicode text, so it has nothing to do with binary data.
You need to find a way to cast your buffer data info the internal type of the data, it should be documented where you got that data (maybe it's just an array of floats ?)
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
Recently started exploring nanopb - apology if my questions sound sily. I faced some issues while assigning and retrieval of strings and integers when I modified simple example of nanopb. Let me give my steps before my questions -
I defined simple.proto file
message Device{
optional string devid =1;
optional string mac = 2;
optional string cpu=3 [default = "x86"] ;
optional bool isSecured=4;
optional int32 uptime = 5 [default = 1234];
}
Also defined simple.option
Device.devid max_size:64
Device.cpu max_size:64
Then I compiled as usual : protoc -osimple.pb simple.proto
Here is my code :
3a) Using same string encode decode utility function as in How to encode a string when it is a pb_callback_t type
//string encode decode to pb
bool encode_string(pb_ostream_t* stream, const pb_field_t* field, void* const* arg)
{
const char* str = (const char*)(*arg);
if (!pb_encode_tag_for_field(stream, field))
return false;
return pb_encode_string(stream, (uint8_t*)str, strlen(str));
}
bool print_string(pb_istream_t *stream, const pb_field_t *field, void **arg)
{
uint8_t buffer[1024] = {0};
/* We could read block-by-block to avoid the large buffer... */
if (stream->bytes_left > sizeof(buffer) - 1)
return false;
if (!pb_read(stream, buffer, stream->bytes_left))
return false;
/* Print the string, in format comparable with protoc --decode.
* Format comes from the arg defined in main(). */
printf((char*)*arg, buffer);
return true;
}
3b) here is my main code snippet based on simple.c of example -
/* This is the buffer where we will store our message. */
uint8_t buffer[128];
size_t message_length;
bool status;
/* Encode our message */
{
/* Allocate space on the stack to store the message data, check out the contents of simple.pb.h
* good to always initialize your structures so that no garbage data from RAM in there. */
//Device message = Device_init_zero; //init zero for empty
Device message = Device_init_default; //init default for default
/* Create a stream that will write to our buffer. */
pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
/* Fill in the data */
message.devid.arg = "device1";
message.devid.funcs.encode = &encode_string;
//strcpy(message.devid,"device1"); // no easier way like this ?
message.isSecured = true; // should be 1 if printed with %d
message.uptime=9876; // should change, why it is not working ?
/* Now we are ready to encode the message! */
// encode stream to buffer , also get the buffer length
status = pb_encode(&stream, Device_fields, &message);
message_length = stream.bytes_written;
/* Then just check for any errors.. */
if (!status)
{
printf("Encoding failed: %s\n", PB_GET_ERROR(&stream));
return 1;
}
}
/* Now we could transmit the message over network, store it in a file etc. */
/* just decode it immediately. */
{
/* Allocate space for the decoded message. */
Device message = Device_init_zero;
/* Create a stream that reads from the buffer. */
pb_istream_t stream = pb_istream_from_buffer(buffer, message_length);
message.devid.funcs.decode = &print_string;
message.devid.arg = "before decode Device - devid: %s \n"; //works here
message.cpu.funcs.decode = &print_string;
message.cpu.arg = "before decode Device -cpu: %s \n"; //where in op?
printf("before decode isSecured %d\n",message.isSecured); // doesn't work
printf("before decode uptime %d\n",message.uptime); //doesn't work
/* Now ready to decode the message. */
// decode stream buffer into message
status = pb_decode(&stream, Device_fields, &message);
/* Check for errors... */
if (!status)
{
printf("Decoding failed: %s\n", PB_GET_ERROR(&stream));
return 1;
}
/* Print the data contained in the message. */
message.devid.funcs.decode = &print_string;
message.devid.arg = "after decode Devic - devid: %s \n"; // doesn't work here
printf(" after decode isSecured %d\n",message.isSecured); // prints default 0
printf(" after decode uptime %d\n",(int)message.uptime); //prints default assigned in proto
}
The output after build and run :
$ ./simple
before decode isSecured 0
before decode uptime 0
before decode Device - devid: device1
after decode isSecured 0
after decode uptime 1234
My queries ( also added my inline comments in code) :
In original simple.c message.lucky_number=13 assignment works but here message.uptime assignment is not working , it is taking default value. Similarly assigning boolean value to message.isSecured is not working. Please tell where is my fault.
I used Device_init_default before pb_encode as some have default values and Device_init_zero before pb_decode call as it will populate after decode. Is my approach correct ?
Is there any simpler way to assign string value using strcpy and printing it in C way by printf("%s",strvar) apart from the encode_string and decode_string util ?
The string is printed only before pb_decode call but uptime default value is printed after pb_decode call. Also boolean value assignment is not working. Why ? What is my mistake ?
I saw encode string and int functions in https://github.com/nanopb/nanopb/blob/master/tests/callbacks/encode_callbacks.c
How to encode and decode float and boolean ?
Thanks in anticipation
In original simple.c message.lucky_number=13 assignment works but here message.uptime assignment is not working , it is taking default value. Similarly assigning boolean value to message.isSecured is not working. Please tell where is my fault.
If you look into the generated .pb.h file, you'll find that every optional field has a boolean has_field. You'll have to set that to true also, to signify that the field is present.
I used Device_init_default before pb_encode as some have default values and Device_init_zero before pb_decode call as it will populate after decode. Is my approach correct ?
That's fine.
Is there any simpler way to assign string value using strcpy and printing it in C way by printf("%s",strvar) apart from the encode_string and decode_string util ?
Because you have already set a max_size for your string fields, they should be generated as a char array instead of callbacks. You can try passing -v switch like: ../generator/nanopb_generator.py -v simple.pb to see more verbose messages that can point out why the option is not applying. Perhaps the file name is incorrect or the message name is incorrect.
The string is printed only before pb_decode call but uptime default value is printed after pb_decode call. Also boolean value assignment is not working. Why ? What is my mistake ?
I saw encode string and int functions in https://github.com/nanopb/nanopb/blob/master/tests/callbacks/encode_callbacks.c How to encode and decode float and boolean ?
Well, usually you wouldn't have to resort to callbacks. But if you decide you need them, you can encode booleans with pb_encode_varint() and floats with pb_encode_fixed32(). For studying callbacks, the protobuf encoding documentation and this test case can be helpful.
You may find the network_server example useful to study.
Also, the Stack Overflow format works best when you have only a single question per post. That way the questions and answers remain focused and easy to follow.
When performing shm-related development on MacOS, the searched processes are shown in the following code (verification is indeed correct).
However, there is a new problem that cannot be solved. It is found that when ftruncat adjusts the memory size for shm_fd, it is allocated according to the multiple of the page size.
But in this case, when the shared memory file is opened by other processes, the actual data size cannot be obtained correctly. The obtained file size is an integer multiple of the page, which will cause an error when appending data.
// write data_size = 12
char *data = "....";
long data_size = 12;
shmFD = shm_open(...);
ftruncate(shmFD, data_size); // Actually the size actually allocated is not 12, but 4096
shmAddr = (char *)mmap(NULL, data_size, ... , shmFD, 0);
memcpy(shmAddr, data, data_size);
// read
...
fstat(shmFD, &sb)
long context_len_in_shm = sb.st_size;
// get wrong shm size -> context_len_in_shm = 4096
Temporarily use the following structure to record data into shm. The first operation before writing or reading is to get the value of the data_len field, and then determine the length of the data to be read and written from the back. Hope for a more concise way, just like the use of lseek() under Linux.
shm mem map :
----shm mem----
struct {
long data_len;
data[1];
data[2];
...
data[data_len];
}
---------------
long *shm_mem = (long *)shmAddr;
long data_size = shm_mem[0]; // Before reading, you need to determine whether the shm file is empty and whether the pointer is valid. It is omitted here.
char *shm_data = (char *)&(shm_mem[1]);
char *buffer = (char *)malloc(data_size);
memcpy(buffer, shm_data, data_size);
Is there a way to attach some user private data to AVPacket before decoding to be able to match input AVPacket and decoded output AVFrame? Some sort of AVFrame::opaque.
Specifically, decoding process of h264 codestream can do reorder in a case B-frames are present and I would like to identify which AVPacket was decoded to which AVFrame.
Thanks to #Gyan I was able to solve the issue with the following code in the main decoding loop.
static uint64_t privateId = 0;
// Allocate dictionary and add appropriate key/value record
AVDictionary * frameDict = NULL;
av_dict_set(&frameDict, "private_id", std::to_string(privateId++).c_str(), 0);
// Pack dictionary to be able to use it as a side data in AVPacket
int frameDictSize = 0;
uint8_t *frameDictData = av_packet_pack_dictionary(frameDict, &frameDictSize);
// Free dictionary not used any more
av_dict_free(&frameDict);
// Add side_data to AVPacket which will be decoded
av_packet_add_side_data(&avPacket, AVPacketSideDataType::AV_PKT_DATA_STRINGS_METADATA, frameDictData, frameDictSize);
// Do the actual decoding
...
// Free side data from packet
av_packet_free_side_data(&avPacket);
// Obtain privateId from decoded frame
uint64_t privateId = std::stoul(av_dict_get(avFrame->metadata, "private_id", NULL, 0)->value);
// Free dictionary from decoded frame
av_dict_free(&avFrame->metadata);
I have a problem in regards of extracting signed int from string in c++.
Assuming that i have a string of images1234, how can i extract the 1234 from the string without knowing the position of the last non numeric character in C++.
FYI, i have try stringstream as well as lexical_cast as suggested by others through the post but stringstream returns 0 while lexical_cast stopped working.
int main()
{
string virtuallive("Images1234");
//stringstream output(virtuallive.c_str());
//int i = stoi(virtuallive);
//stringstream output(virtuallive);
int i;
i = boost::lexical_cast<int>(virtuallive.c_str());
//output >> i;
cout << i << endl;
return 0;
}
How can i extract the 1234 from the string without knowing the position of the last non numeric character in C++?
You can't. But the position is not hard to find:
auto last_non_numeric = input.find_last_not_of("1234567890");
char* endp = &input[0];
if (last_non_numeric != std::string::npos)
endp += last_non_numeric + 1;
if (*endp) { /* FAILURE, no number on the end */ }
auto i = strtol(endp, &endp, 10);
if (*endp) {/* weird FAILURE, maybe the number was really HUGE and couldn't convert */}
Another possibility would be to put the string into a stringstream, then read the number from the stream (after imbuing the stream with a locale that classifies everything except digits as white space).
// First the desired facet:
struct digits_only: std::ctype<char> {
digits_only(): std::ctype<char>(get_table()) {}
static std::ctype_base::mask const* get_table() {
// everything is white-space:
static std::vector<std::ctype_base::mask>
rc(std::ctype<char>::table_size,std::ctype_base::space);
// except digits, which are digits
std::fill(&rc['0'], &rc['9'], std::ctype_base::digit);
// and '.', which we'll call punctuation:
rc['.'] = std::ctype_base::punct;
return &rc[0];
}
};
Then the code to read the data:
std::istringstream virtuallive("Images1234");
virtuallive.imbue(locale(locale(), new digits_only);
int number;
// Since we classify the letters as white space, the stream will ignore them.
// We can just read the number as if nothing else were there:
virtuallive >> number;
This technique is useful primarily when the stream contains a substantial amount of data, and you want all the data in that stream to be interpreted in the same way (e.g., only read numbers, regardless of what else it might contain).
I want to convert a UINT16 monochrome image to a 8 bits image, in C++.
I have that image in a
char *buffer;
I'd like to give the new converted buffer to a QImage (Qt).
I'm trying with freeImagePlus
fipImage fimage;
if (fimage.loadfromMemory(...) == false)
//error
loadfromMemory needs a fipMemoryIO adress:
loadfromMemory(fipMemoryIO &memIO, int flag = 0)
So I do
fipImage fimage;
BYTE *buf = (BYTE*)malloc(gimage.GetBufferLength() * sizeof(BYTE));
// 'buf' is empty, I have to fill it with 'buffer' content
// how can I do it?
fipMemoryIO memIO(buf, gimage.GetBufferLength());
fimage.loadFromMemory(memIO);
if (fimage.convertTo8Bits() == true)
cout << "Good";
Then I would do something like
fimage.saveToMemory(...
or
fimage.saveToHandle(...
I don't understand what is a FREE_IMAGE_FORMAT, which is the first argument to any of those two functions. I can't find information of those types in the freeImage documentation.
Then I'd finish with
imageQt = new QImage(destiny, dimX, dimY, QImage::Format_Indexed8);
How can I fill 'buf' with the content of the initial buffer?
And get the data from the fipImage to a uchar* data for a QImage?
Thanks.
The conversion is simple to do in plain old C++, no need for external libraries unless they are significantly faster and you care about such a speedup. Below is how I'd do the conversion, at least as a first cut. The data is converted inside of the input buffer, since the output is smaller than the input.
QImage from16Bit(void * buffer, int width, int height) {
int size = width*height*2; // length of data in buffer, in bytes
quint8 * output = reinterpret_cast<quint8*>(buffer);
const quint16 * input = reinterpret_cast<const quint16*>(buffer);
if (!size) return QImage;
do {
*output++ = *input++ >> 8;
} while (size -= 2);
return QImage(output, width, height, QImage::Format_Indexed8);
}