We use protocol buffers for storing data in a database (as blobs). At some point, we read them again, need to modify them and store them again.
The problem is, protocol buffer message objects are immutable. What we had in mind was just creating a new builder object using the protocol buffer message as prototype:
Foo.Builder.newBuilder(prototype)
This basically works. But as we have a nested structure, so the prototype object actually contains attributes which are messages themselves, this does not work. Excerpt from the documentation:
Since embedded message and string objects are immutable, they are shared between the original and the copy.
Is there a way of cloning a whole structure to new builder objects which are mutable?
Well, sort of. I've done this using DynamicMessage.Builder, Descriptor and FieldDescriptor. I recursively walk the object graph using a fully qualified name to the property I want to update. Once found, I update it and call build on the DyynamicMessage.Builder(s) back up the stack. It is not straight forward particularly when dealing with repeated fields.
Related
Am working on finding the differences between two Json objects and storing the respective Diff in Json format(using Javers), so that can be retrieve later and presented to the user in a meaningful manner. Now, the challenge is, the Affected Object(where the change has happened) is not being serialized during serialization.
I understand that it is not possible to retrieve the domain objects from Diff.
Could see that the "affectedCdo" has been marked as transient not allowing to serialize it.
Any other alternative for doing this?
Thanks,
Ravi.
Restoring orginal domain objects from persisted snapshots will be possible when we implemen the shadows feature in JaVers. We are working on it. https://github.com/javers/javers/issues/133
I'm struggling to find something in the documentation that seems like it should be there...
In Phoenix I see validation at the point of trying to create an Ecto change set, but I'm not seeing much prior to that, upon validating the actual user input.
I'm not really a fan of exposing my data models across API boundaries, and I would rather just have structs representing the requests and responses, as they are likely very different shapes to my actual data models.
I'd like a way of converting user input to a struct and using some kind of validation framework to determine if the input is valid before I even think about hitting a database.
I've found https://github.com/CargoSense/vex and have gone down the route of converting the input to a struct, and using their validation, but there are a few things that worry me about this approach, namely:
I hear that there are issues with atoms in Elixir, and as structs are basically atom keyed maps, am I going to run into this atom exhaustion issue converting user input to these?
I also have some structs that would have nested structs. I'm currently checking the default value provided. If it's a struct doing some magic, based on the answers here In Elixir how do you initialize a struct with a map variable, to automatically convert a nested map to my nested struct. But again, I'm not sure if this is sensible.
The validations I'm defining in one DSL will be very similar in my Ecto models, and I would rather be using this for both.
Basically, how would you go about validating user input correctly in a Phoenix app. Am I on the right lines, or way off?
According to Apple's documentation, Archives convert data into architecture-independent byte streams. What is the difference between serializing data (like saving values to a property list) and archiving it?
Can archives be created with or without NSCoder? Is the NSCoder protocol only used to archive custom objects?
I understand that archives are a way to save object graph relationships and maintain object mutability - am I correct in thinking that for saving an object graph with custom objects, I would need to encode those objects into NSData, archive them to a byte stream to maintain the graph relationship, and then have the option of saving to the defaults database with NSUserDefaults or saving to disk?
Additionally, what's the difference between the defaults database and saving to disk?
I just want to get a better understanding of how the terms all relate to one another.
I've laid out my general idea of how these things interrelate like so:
I THINK -
NSKeyedArchiver is used to encode and store an object graph as a byte stream. It traverses the object graph, maintaining relationships, and calls the encoding protocol methods on each object. NSKeyedArchiver just keeps track of the object graph and saves it - we can either save it to disk or we can save to NSUserDefaults (which is a property list).
*it is advisable to archive data to a file directory on disk instead of NSUserDefaults.
Objects can be serialized to a property list (or to NSUserDefaults) without encoding them as long as they are arrays, strings, integers, etc.
Let’s say we have an array (which is automatically NSKeyedArchiver and PropertyListSerializer compliant) and it’s full of custom objects. Those custom objects must implement the NSCoder protocol methods. Then, somewhere else, we can create an instance of NSData that is equal to an NSKeyedArchive using that array, and save that instance of NSData to our desired location.
What happens is, the archiver traverses over the contents of that array, implementing the NSCoder protocol methods that each object adheres to. It keeps track of relationships in the object (the array), persisting the object graph.
When we want that data back, we can go into the file we saved to, check its contents, and create an instance of NSData. We make that instance equal to the dataWithContentsOfFile of the path specified. Then, we create an array and unpackaged the NSData into that array by called the NSKeyedUnarchiver unarchiveObjectWithData method. The archiver traverses over each object in the NSData instance and calls initWithCoder, essentially deserializing the byte stream.
Places we can possible store objects -
NSUserDefaults
Disk (file path)
Ways to store custom objects -
NSKeyedArchiver, which implements NSCoder protocol methods to convert custom objects into a byte stream that represents an object graph.
What must we use to store custom objects? NSCoder protocol methods.
I can easily see how to add hierarchical data to a plist file via the CFPreferences api.
However, whats far less obvious how to read from a CFPreferences a value stored inside a CFDictionary (that might be stored in turn, in a CFDictionary), and change it.
You can’t, you have to replace the root element. If this is too cumbersome, that’s a sign that you should be using model objects rather than collections and possibly move away from CFPreferences/NSUserDefaults to some other storage mechanism, perhaps Core Data.
I'm trying to figure out how to decide when to use NSDictionary or NSCoder/NSCoding?
It seems that for general property lists and such that NSDictionary is the easy way to go that generates XML files that are easily editable outside of the application.
When dealing with custom classes that holds data or possibly other custom classes nested inside, it seems like NSCoder/NSCoding would be the better route since it will step through all the contained object classes and encode them as well when an archive command is used.
NSDictionary seems like it would take more work to get all the properties or data characteristics to a single level to be able to save it, where as NSCoder/NSCoding would automatically encode nested custom classes that implement the NSCoding interface.
Outside of it being binary data and not editable outside of your application is there a real reason to use one over the other? And along those lines is there an indicator of which way you should lean between the two? Am I missing something obvious?
Apple's documentation on object graphs has this to say:
Mac OS X serializations store a simple hierarchy of value objects, such as dictionaries, arrays, strings, and binary data. The serialization only preserves the values of the objects and their position in the hierarchy. Multiple references to the same value object might result in multiple objects when deserialized. The mutability of the objects is not maintained.
…
Mac OS X archives store an arbitrarily complex object graph. The archive preserves the identity of every object in the graph and all the relationships it has with all the other objects in the graph. When unarchived, the rebuilt object graph should, with few exceptions, be an exact copy of the original object graph.
The way I interpret this is that, if you want to store simple values, serialization (using an NSDictionary, for example) is a fine way to go. If you want to store an object graph of arbitrary types, with uniqueness and mutability preserved, using archives (with NSCoder, for example) is your best bet.
You may also want to read Apple's Archives and Serializations Programming Guide for Cocoa, of which the aforelinked page on object graphs is a part, as it covers this topic well.
I am NOT a big fan of using NSCoding/NSCoder/NSArchiver (we need to pick a name!) to serialise an object graph to a file.
Archives created in this way are incredibly fragile. If you save an object of class Foo then by golly you need to make sure when you load the data back in you have a class Foo in your application.
This makes NSCoder based serialisation difficult from the perspective of sharing files with other applications or even forwards compatibility with your future application.
I forgot to list what I would recommend.
NSCoding can be ok in certain situations: if you're just doing something quick and simple (although you do have to write a lot of code - two methods per class to be serialised). It can also be ok if you're not worried about compatibility with other applications.
Export/import via property lists (perhaps using the NSPropertyListSerializaion class) is a fine solution. XML based plists are easy to create and edit. Main advantage to plists is that you're not tying the file format to just your application.
You can also create your own XML based file format and read/write to it using NSXMLDocument API and friends. This really isn't much more work than using property lists.
I think you're a bit confused, NSDictionary is a data structure, it also happens to implement the NSCoding protocol. So in essence, you could either put all your data into a NSDictionary and have that encode itself later on, or you can implement the NSCoding protocol and encode your object tree using the NSCoder API. Based on the type of NSCoder object passed in to the encodeWithCoder: method, is the output of your encoding.