MongoDB - override default Serializer for a C# primitive type - mongodb-.net-driver

I'd like to change the representation of C# Doubles to rounded Int64 with a four decimal place shift in the serialization C# Driver's stack for MongoDB. In other words, store (Double)29.99 as (Int64)299900
I'd like this to be transparent to my app. I've had a look at custom serializers but I don't want to override everything and then switch on the Type with fallback to the default, as that's a bit messy.
I can see that RegisterSerializer() won't let me add one for an existing type, and that BsonDefaultSerializationProvider has a static list of primitive serializers and it's marked as internal with private members so I can't easily subclass.
I can also see that it's possible to RepresentAs Int64 for Doubles, but this is a cast not a conversion. I need essentially a cast AND a conversion in both serialization directions.
I wish I could just give the default serializer a custom serializer to override one of it's own, but that would mean a dirty hack.
Am I missing a really easy way?

You can definitely do this, you just have to get the timing right. When the driver starts up there are no serializers registered. When it needs a serializer, it looks it up in the dictionary where it keeps track of the serializers it knows about (i.e. the ones that have been registered). Only it it can't find one in the dictionary does it start figuring out where to get one (including calling the serialization providers) and if it finds one it registers it.
The limitation in RegisterSerializer is there so that you can't replace an existing serializer that has already been used. But that doesn't mean you can't register your own if you do it early enough.
However, keep in mind that registering a serializer is a global operation, so if you register a custom serializer for double it will be used for all doubles, which could lead to unexpected results!
Anyway, you could write the custom serializer something like this:
public class CustomDoubleSerializer : BsonBaseSerializer
{
public override object Deserialize(BsonReader bsonReader, Type nominalType, Type actualType, IBsonSerializationOptions options)
{
var rep = bsonReader.ReadInt64();
return rep / 100.0;
}
public override void Serialize(BsonWriter bsonWriter, Type nominalType, object value, IBsonSerializationOptions options)
{
var rep = (long)((double)value * 100);
bsonWriter.WriteInt64(rep);
}
}
And register it like this:
BsonSerializer.RegisterSerializer(typeof(double), new CustomDoubleSerializer());
You could test it using the following class:
public class C
{
public int Id;
public double X;
}
and this code:
BsonSerializer.RegisterSerializer(typeof(double), new CustomDoubleSerializer());
var c = new C { Id = 1, X = 29.99 };
var json = c.ToJson();
Console.WriteLine(json);
var r = BsonSerializer.Deserialize<C>(json);
Console.WriteLine(r.X);

You can also use your own serialization provider to tell Mongo which serializer to use for certain types, which I ended up doing to mitigate some of the timing issues mentioned when trying to override existing serializers. Here's an example of a serialisation provider that overrides how to serialize decimals:
public class CustomSerializationProvider : IBsonSerializationProvider
{
public IBsonSerializer GetSerializer(Type type)
{
if (type == typeof(decimal)) return new DecimalSerializer(BsonType.Decimal128);
return null; // falls back to Mongo defaults
}
}
If you return null from your custom serialization provider, it will fall back to using Mongo's default serialization provider.
Once you've written your provider, you just need to register it:
BsonSerializer.RegisterSerializationProvider(new CustomSerializationProvider());

I looked through the latest iteration of the driver's code and checked if there's some sort of backdoor to set custom serializers. I am afraid there's none; you should open an issue in the project's bug tracker if you think this needs to be looked at for future iterations of the driver (https://jira.mongodb.org/).
Personally, I'd open a ticket -- and if a quick workaround is necessary or required, I'd subclass DoubleSerializer, implement the new behavior, and then use Reflection to inject it into either MongoDB.Bson.Serialization.Serializers.DoubleSerializer.__instance or MongoDB.Bson.Serialization.BsonDefaultSerializationProvider.__serializers.

Related

protobuf-net concurrent performance issue in TakeLock

We're using protobuf-net for sending log messages between services. When profiling stress testing, under high concurrency, we see very high CPU usage and that TakeLock in RuntimeTypeModel is the culprit. The hot call stack looks something like:
*Our code...*
ProtoBuf.Serializer.SerializeWithLengthPrefix(class System.IO.Stream,!!0,valuetype ProtoBuf.PrefixStyle)
ProtoBuf.Serializer.SerializeWithLengthPrefix(class System.IO.Stream,!!0,valuetype ProtoBuf.PrefixStyle,int32)
ProtoBuf.Meta.TypeModel.SerializeWithLengthPrefix(class System.IO.Stream,object,class System.Type,valuetype ProtoBuf.PrefixStyle,int32)
ProtoBuf.Meta.TypeModel.SerializeWithLengthPrefix(class System.IO.Stream,object,class System.Type,valuetype ProtoBuf.PrefixStyle,int32,class ProtoBuf.SerializationContext)
ProtoBuf.ProtoWriter.WriteObject(object,int32,class ProtoBuf.ProtoWriter,valuetype ProtoBuf.PrefixStyle,int32)
ProtoBuf.BclHelpers.WriteNetObject(object,class ProtoBuf.ProtoWriter,int32,valuetype
ProtoBuf.BclHelpers/NetObjectOptions)
ProtoBuf.Meta.TypeModel.GetKey(class System.Type&)
ProtoBuf.Meta.RuntimeTypeModel.GetKey(class System.Type,bool,bool)
ProtoBuf.Meta.RuntimeTypeModel.FindOrAddAuto(class System.Type,bool,bool,bool)
ProtoBuf.Meta.RuntimeTypeModel.TakeLock(int32&)
[clr.dll]
I see that we can use the new precompiler to get a speed boost, but I'm wondering if that will get rid of the issue (sounds like it doesn't use reflection); it would be a bit of work for me to integrate this, so I haven't tested it yet. I also see the option to call Serializer.PrepareSerializer. My initial (small scale) testing didn't make the prepare seem promising.
A little more info about the type we're serializing:
[ProtoContract]
public class SomeMessage
{
[ProtoMember(1)]
public SomeEnumType SomeEnum { get; set; }
[ProtoMember(2)]
public long SomeId{ get; set; }
[ProtoMember(3)]
public string SomeString{ get; set; }
[ProtoMember(4)]
public DateTime SomeDate { get; set; }
[ProtoMember(5, DynamicType = true, OverwriteList = true)]
public Collection<object> SomeArguments
}
Thanks for your help!
UPDATE 9/17
Thanks for your response! We're going to try the workaround you suggest and see if that fixes things.
This code lives in our logging system so, in the SomeMessage example, SomeString is really a format string (e.g. "Hello {0}") and the SomeArguments collection is a list of objects used to fill in the format string, just like String.Format. Before we serialize, we look at each argument and call DynamicSerializer.IsKnownType(argument.GetType()), if it isn't known, we convert it to a string first. I haven't looked at the ratios of data, but I'm pretty sure we have a lot of different strings coming in as arguments.
Let me know if this helps. If you need, I'll try to get more details.
TakeLock is only used when it is changing the model, for example because it is seeing a type for the first time. You shouldn't normally see TakeLock after the first time a particular type has been used. In most cases, using Serializaer.PrepareSerializer<SomeMessage>() should perform all the necessary initialization (and similar for any other contracts you are using).
However! I wonder if perhaps this is also related to your use of DynamicType; what are the actual objects being used here? It might be that I need to tweak the logic here, so that it doesn't spend any time on that step. If you let me know the actual objects (so I can repro), I will try to run some tests.
As for whether the precompiler would change this; yes it would. A fully compiled static model has a completely different implementation of the ProtoBuf.Meta.TypeModel.GetKey method, so it would never call TakeLock (you don't need to protect a model that can never change!). But you can actuallydo something very similar without needing to use precompile. Consider the following, run as part of your app's initialization:
static readonly TypeModel serializer;
...
var model = TypeModel.Create();
model.Add(typeof(SomeMessage), true);
// TODO add other contracts you use here
serializer = model.Compile();
This will create a fully static-compiled serializer assembly in memory (instead of a mutable model with individual operations compiled). If you now use serializer.Serialize(...) instead of Serializer.Serialize (i.e. the instance method on your stored TypeModel rather than the static method on Serializer) then it will essentially be doing something very similar to "precompiler", but without the need to actualy precompile it (obviously this will only be available on "full" .NET). This will then never call TakeLock, as it is running a fixed model, rather than a flexible model. It does, however, require you to know what contract-types you use. You could use reflection to find these, by looking for all those types with a given attribute:
static readonly TypeModel serializer;
...
var model = TypeModel.Create();
Type attributeType = typeof(ProtoContractAttribute);
foreach (var type in typeof(SomeMessage).Assembly.GetTypes()) {
if (Attribute.IsDefined(type, attributeType)) {
model.Add(type, true);
}
}
serializer = model.Compile();
But emphasis: the above is a workaround; it sounds like there's a glitch, which I'll happily investigate if I can see an example where it actually happens; most importantly: what are the objects in SomeArguments?

Separation of Concerns: Returning Projected Data between layers From a Linq Query

I'm using Linq and having trouble doing something that I believe should be trivial. I want to return data from one layer so it can be used independently of linq in another layer.
Suppose I have a Data Access Layer. It knows about the entity framework and how to interact with it. But, it doesn't care who accesses it. The one interesting requirement I have is that the queries in the entity framework return projected data that is not part of the Entity Model itself. Please don't ask me to change this part of the requirement and make POCOs for each return type, as it is not the best design given the problem I am trying to solve. Below is an example.
public class ChartData
{
public function <<returnType??>> GetData()
{
MyEntities context = new MyEntities();
var results = from context.vManyColumnsOfData as v
where v.CompanyName = "acme"
select new {Year = v.SalesYear, Income = v.Income};
return ??;
}
}
Then, I would like to have an ASP.Net UI layer be able to call into the Data Access Layer to get the data in order to bind it to a control. The UI layer should have no notion of where the data came from. It should only know that it has the data it needs to bind. Below is an example.
protected void chart_Load(object sender, EventArgs e)
{
// set some chart properties
chart.Skin = "Default";
...
// Set the data source
ChartData dataMgr = new ChartData();
<<returnType?>> data = dataMgr.GetData();
chart.DataSource = data;
chart.DataBind();
}
What is the best way to send linq projected data back to another layer?
If you don't need to use the projected type statically, just return IEnumerable<object>.
Please don't ask me to change this part of the requirement and make
POCOs for each return type, as it is not the best design given the
problem I am trying to solve.
I feel like I should rightly ignore this, as the best thing to do is to return a defined type. Anonymous types are useful when they are wholly contained within the method that creates them. Once you start passing them around, it is time to go ahead and give them the proper class treatment.
However, to live within your imposed limitations, you can return IEnumerable<object> from the method and use that or var at the callsite and rely upon the dynamic binding of the control to get at the data. It's not going to help you if you need to deal with the object programmatically, but it will serve fine for databinding.
You can not return an anonymous type, so basically for this you will need POCO's even though you don't want them.
"not the best design given the problem I am trying to solve"
Could you explain what you are trying to achieve a little more? It might be possible to return some type of list containing a dictionary of items (ie rows and columns). Think something like an untyped dataset (yuck)
Your GetData method can use IEnumerable (the "old" non-generic interface) as its return type.
Any dynamic resolution (e.g. ASP.NET or XAML bindings) should work as expected, which seems to be what you want to do.
However, if you want to use the results in your code, you will probably have to resort to .NET 4's dynamic keyword.
The following example can be run in LINQPad (in "C# Program" mode) and illustrates this:
void Main()
{
var v = GetData();
foreach (dynamic element in v)
{
((string)element.Name).Dump();
}
}
public IEnumerable GetData()
{
return from i in Enumerable.Range(1, 10)
select new
{
Name = "Item " + i,
Value = i
};
}
Keep in mind that, design-wise, coding like this will make most people frown and can affect performance.

Custom serialization in Caliburn.Micro 1.1

I'm upgrading a 1.0 WP7 application to CM 1.1. Among other stuff, I'm removing the old attribute-based tombstoning and implementing storage classes.
This typically involves creating a class for each VM for storage purposes, deriving it from StorageHandler<T> (where T is the type of the VM) and overriding its Configure method like e.g.:
public override void Configure()
{
Property(x => x.SomeSerializableProperty).InPhoneState().RestoreAfterViewLoad();
// ...
}
In this context, how can I implement a custom serialization mechanism using my own serialize/deserialize code for objects which could not be automatically serialized? For instance, one of my VM's has a StrokeCollection property and I'd like to serialize the strokes in it, but to this end I need to replace the default mechanism which would raise security exceptions.
Could anyone show a fake CM WP7 sample to illustrate how to customize the serialization of some property, so that I can place my own code for serializing/deserializing it?
Thanks!
I don't know if this is the right path, but it works; here is a code sample:
Property(x => x.Strokes).InPhoneState().RestoreAfterViewReady().Configure(x =>
{
x.Save = SaveStrokes;
x.Restore = RestoreStrokes;
});
with their implementations like:
void SaveStrokes(BoardViewModel vm, Func<string> serialize, StorageMode nMode)
{
IsolatedStorageSettings.ApplicationSettings[vm.DisplayName + "ThePropertyKey"] =
// ...get data from vm and serialize
}
and conversely:
void RestoreStrokes(BoardViewModel vm, Func<string> serialize, StorageMode nMode)
{
// use IsolatedStorageSettings.ApplicationSettings[vm.DisplayName + "ThePropertyKey"]
// to check if the key exists, and if it is there get the serialized data and deserialize
}
As for strokes, I'm using my own serialization class as my usual tool for this purpose (SharpSerializer) seems having issues in restoring (it throws an ambiguous match reflection exception).

Is there any way to clean up the following generic method using any of the new C# 4 features?

I've just modified a method for handling my DDD commands (previously it had no return type):
public static CommandResult<TReturn> Execute<TCommand, TReturn>(TCommand command)
where TCommand : IDomainCommand
{
var handler = IoCFactory.GetInstance<ICommandHandler<TCommand, TReturn>>();
return handler.Handle(command);
}
The method is fine, and does what I want it to do, however using it creates some fugly code:
CommandResult<Customer> result =
DomainCommands.Execute<CustomerCreateCommand, Customer>
(
new CustomerCreateCommand(message)
);
Before I added the Customer return type TReturn, it was nice and tidy and the method could infer the types from its usage. However that's no longer possible.
Is there any way using any new C# features that I could rewrite the above to make it tidier, i.e. using Func, Action, Expression, etc? I'm probably expecting the impossible, but I'm getting fed up of writing so much code to just call a single method that used to be very simple.
One option to reduce it slightly is to have a static generic type for the type parameter that can't be inferred, allowing you to have a generic method with just one type parameter that can be inferred:
public static class DomainCommands<TReturn>
{
public static CommandResult<TReturn> Execute<TCommand>(TCommand command)
where TCommand : IDomainCommand
{
var handler = IoCFactory.GetInstance<ICommandHandler<TCommand, TReturn>>();
return handler.Handle(command);
}
}
Then:
var result = DomainCommands<Customer>.Execute(new CustomerCreateCommand(msg));
It's not much nicer, but it's slightly better. Of course, if the domain command type itself could be generic, that might help - so CustomerCreateCommand would implement IDomainCommand<Customer> for example. If you still needed a nongeneric IDomainCommand, you could make IDomainCommand<T> derive from IDomainCommand.

Lazy generic delegate initialisation using Ninject

I'm using Ninject 1.0 and would like to be able to inject lazy initialisation delegates into constructors. So, given the generic delegate definition:
public delegate T LazyGet<T>();
I'd simply like to bind this to IKernel.Get() so that I can pass a lazy getter into constructors, e.g.
public class Foo
{
readonly LazyGet<Bar> getBar;
public Foo( LazyGet<Bar> getBar )
{
this.getBar = getBar;
}
}
However, I can't simply call Bind<LazyGet<T>>() because it's an open generic type. I need this to be an open generic so that I don't have to Bind all the different lazy gets to explicit types. In the above example, it should be possible to create a generic delegate dynamically that invokes IKernel.Get<T>().
How can this be achieved with Ninject 1.0?
Don't exactly understand the question, but could you use reflection? Something like:
// the type of T you want to use
Type bindType;
// the kernel you want to use
IKernel k;
// note - not compile tested
MethodInfo openGet = typeof(IKernel).GetMethod("Get`1");
MethodInfo constGet = openGet.MakeGenericMethod(bindType);
Type delegateType = typeof(LazyGet<>).MakeGenericType(bindType);
Delegate lazyGet = Delegate.CreateDelegate(delegateType, k, constGet);
Would using lazyGet allow you to do what you want? Note that you may have to call the Foo class by reflection as well, if bindType isn't known in the compile context.
I am fairly certain that the only way to do this (without some dirty reflection code) is to bind your delegate with type params. This will mean it needs to be done for each individual type you use. You could possibly use a BindingGenerator to do this in bulk, but it could get a bit ugly.
If there is a better solution (a clean one) I would love to hear it as I run into this problem from time to time.
From another similar question I answered:
public class Module : NinjectModule
{
public override void Load()
{
Bind(typeof(Lazy<>)).ToMethod(ctx =>
GetType()
.GetMethod("GetLazyProvider", BindingFlags.Instance | BindingFlags.NonPublic)
.MakeGenericMethod(ctx.GenericArguments[0])
.Invoke(this, new object[] { ctx.Kernel }));
}
protected Lazy<T> GetLazyProvider<T>(IKernel kernel)
{
return new Lazy<T>(() => kernel.Get<T>());
}
}

Resources