I am working on an ECG module, which is giving out data in bytes. There's a protocol document about it explaining like how to structure the packets, that are coming out of the module.
I want to decode that data.
I am confused whether protocol buffers will help in this or not. Any other methods that would be helpful in this decoding and writing that protocol in Python ?
Protocol buffers only works with its own encoding format.
For decoding a manufacturer-specific custom binary format, I would suggest Python's built-in struct module. Here is a basic tutorial which is easy get started with.
If the manufacturer-specific format is text based instead, you can either use basic string manipulation to split it into tokens, or one of the parsing libraries for Python.
Related
I have a nfc card which supports multiple technologies. (For example NfcA,MifareClassic, ISOdep). I want to understand what technology has been used to write to the tag.
Are these technologies independent? I mean can I use mifareclassic to write some data and use NfcA (or ISOdep) to read that data?
Or these technologies each have their own memory?
I have been reading a lot about this subject last few days but have not found a good reference.
I also did some tests myself. I wrote an android app to write a NdefMessage to a tag. and I found the corresponding data bytes when I used MifareClassic APIs to dump the memory.
I took a look at the code inside MifareClassic library, and found out that all of the relative functions(e.g readblock,writeblock) create a byte array and pass it to transceive(). In android documention it is mentioned that calling MifareClassic.transceive is the same as calling NfcA.transceive.
which is a bit ironic since it is mentioned in NfcA documents that NfcA and MifareClassic are not the same and they have different transmission protocols.
another thing that I realized was that Ndef is not a protocol by itself. it a standard format to store data. apparently Ndef class has different implementations for different tags. on a MifareClassic tag you can only use Ndef class to write data to it if the tag is using the default keys. otherwise you will not be able to write to the tag.
I need to design a binary format to save data from a scientific application. This data has to be encoded in a binary format that can't be easily read by any other application (it is a requirement by some of our clients). As a consequence, we decided to build our own binary format, its encoder and its decoder.
We got some inspiration from many binary format, including protobuf. One thing that puzzles me is the way protobuf encodes the length of embedded messages. According to https://developers.google.com/protocol-buffers/docs/encoding, the size of an embedded message is encoded at its very beginning as a varint.
But before we encode an embedded message, we don't know yet its size (think for instance of an embedded message that contains many integers encoded as varint). As a consequence, we need to encode the message entirely, before we write it to the disk so we know its size.
Imagine that this message is huge. As a consequence, it is very difficult to encode it in an efficient way. We could encode this size as a full int and seek back to this part of the file once the embedded message is written, but we loose the nice property of varints: you don't need to specify if you have a 32-bit or a 64-bit integer. So going back to Google's implementation using a varint:
Is there an implementation trick I am missing, or is this scheme likely to be inefficient for large messages?
Yes, the correct way to do this is to write the message first, at the back of the buffer, and then prepend the size. With proper buffer management, you can write the message in reverse.
That said, why write your own message format when you can just use protobuf? It would be better to just use Protobuf directly and encrypt the file format. That would be easy for you to use, and still be hard for other applications to read.
I am implementing a WebSocket server in C and was wondering what's the purpose of the text/binary frame indicators (opcode 1 and 2). Why they are there? In the end in both cases the payload contains bits. And when there is a protocol using websocket or so then I know what expect in the data. Is it because when it's a text message I can be sure that payload only contains UTF-8 valid data?
I will start my answer by pointing out that WebSockets are often implemented with a Javascript client in mind (i.e., a browser).
When you're using C, the different opcode might be used in different ways, but when using Javascript, this difference controls the type of the data in the event (Blob vs. String).
As you point out in the question, a string is always a valid UTF-8 stream of bytes, whereas a blob isn't.
This affects some data transport schemes (such as JSON parsing, which requires a UTF-8 valid stream).
Obviously, in C, this opcode could be used in different ways, but it would be better to use the opcode in the same manner as a potential javascript client.
P.S.
There are a number of Websocket C libraries and frameworks out there (I'm the author of facil.io).
Unless this is a study project, I would consider using one of the established frameworks / libraries.
I am working on a project where we are using protocol buffers to create and parse some of our messages (protobuf-net). This is so elegant, that I would like to use this same deserialization method to parse other messages emanating from external non-protobuf generated sources. Is this possible?
I would imagine that it could be possible to specify all of the .proto fields to be fixed size (i.e. not like variable ints). The question is then whether you could replace the protobuf headers with other magic numbers or whichever header the 3rd party protocol uses.
If this is a bit confusing, an example may shed some light:
Let's say you buy a fancy toaster that exposes an ethernet port. It supports a proprietary but well documented protocol. Can you burn heart shaped patterns on your toast using protobuf?
At the moment, no: the library is tied to the protobuf wire specification; it does not have support for non-protobuf data.
In a way, it is a bit like asking: "can XmlSerializer read/write json?". It isn't something that is on my list of things to look at, to be honest.
What are the pros and cons of protocol buffer (protobuf) over GSON?
In what situation protobuf is more appropriate than GSON?
I am sorry for a very generic question.
Both json (via the gson library) and protobuf are portable between platorms; but
protobuf is smaller (bandwidth) and cheaper (CPU) to read/write
json is human readable / editable (protobuf is binary; hard to parse without library support)
protobuf is trivial to merge fragments - just concatenate
json is easily passed to web page clients
the main java version of protobuf needs contract-definition (.proto) and code-generation; gson seems to allow arbitrary pojo usage (there are protobuf implementations that work on such objects, but not for java afaik)
If performance is key : protubuf
For use with a web page (JavaScript), or human readable: json (perhaps via gson)
If you want efficiency and cross-platform you should send raw messages between applications containing the information that is necessary and nothing more or less.
Serialising classes via Java's own mechanisms, gson, protobufs or whatever, creates data that contains not only the information you wish to send, but also information about the logical structures/hierarchies of the data structures that has been used to represent the data inside your application.
This makes those classes and data mapping dual purpose, one to represent the data internally in the application, and two to be transmitted to another application. Those two roles can be conflicting there is an onus on the developer to remember that the classes, collections and data layout he is working with at any time will also be serialised.