I'm using the Mongo c# driver 2.0 and am running into BsonSerializer registration issues when registering AbstractClassSerializers for my Id value objects.
MongoDB.Bson.BsonSerializationException: There is already a serializer registered for type HistoricalEventId.
When I peek into the BsonSerializer I'm seeing that a BsonClassMapSerializer is already registered for my type.
I'm assuming that a BsonClassMapSerializer is being created for my entity types and it's also creating a BsonClassMapSerializer for the Id field as well. Has anyone run into this before? The Bson serializer code is shown below if that helps.
Sorry if the formatting is wrong, c# doesn't seem to be showing up well.
HistoricalEventIdBsonSerializer
public class HistoricalEventIdBsonSerializer : ToObjectIdBsonSerializer<HistoricalEventId>
{
public override HistoricalEventId CreateObjectFromObjectId(ObjectId serializedObj)
{
HistoricalEventId parsedObj;
HistoricalEventId.TryParse(serializedObj, out parsedObj);
return parsedObj;
}
}
ToObjectIdBsonSerializer
public abstract class ToObjectIdBsonSerializer<T> : AbstractClassSerializer<T> where T : class
{
private static readonly Type _convertibleType = typeof(IConvertible<ObjectId>);
public abstract T CreateObjectFromObjectId(ObjectId serializedObj);
public override T Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
{
var bsonType = context.Reader.GetCurrentBsonType();
ObjectId value;
switch (bsonType)
{
case BsonType.Undefined:
value = ObjectId.Empty;
context.Reader.ReadUndefined();
break;
case BsonType.Null:
value = ObjectId.Empty;
context.Reader.ReadNull();
break;
case BsonType.ObjectId:
value = context.Reader.ReadObjectId();
break;
case BsonType.String:
value = new ObjectId(context.Reader.ReadString());
break;
default:
throw new NotSupportedException("Unable to create the type " +
args.NominalType.Name + " from the bson type " + bsonType + ".");
}
return this.CreateObjectFromObjectId(value);
}
public override void Serialize(BsonSerializationContext context, BsonSerializationArgs args, T value)
{
if (value == null)
{
context.Writer.WriteObjectId(ObjectId.Empty);
}
else
{
if (!_convertibleType.IsAssignableFrom(args.NominalType))
{
throw new NotSupportedException("The type " + args.NominalType.Name +
" must implement the " + _convertibleType.Name + " interface.");
}
var typedObj = (IConvertible<ObjectId>)value;
context.Writer.WriteObjectId(typedObj.ToValueType());
}
}
}
IConvertible
public interface IConvertible<out T>
{
T ToValueType();
}
My assumption must have been correct because I just fixed this by doing the BsonSerializer registration before creating the MongoClient and getting the database. Hopefully this will help someone else.
Related
As per the documentation the difference between
orElse and orElseGet is in the fact that in the case of orElseGet(x) the x part
is called incase Optional.isPresent is false
But in below program in case the supplier x is a another method call in orElseGet(x)
the x part is called even if Optional.isPresent is true similiar to the behavior of
orElse(). Why is the behavior of orElseGet() in this scenario same as orElse()?
import java.util.Optional;
import java.util.function.Supplier;
public class TestOrElseGet {
public static void main(String[] args) {
checkOrElseGet();
}
private static void checkOrElseGet() {
System.out.println("------Start Optional.orElseGet-------");
String first= getFirst();
// else part is not initialized
// else doesn't get initialized if its inside OrElseGet()
String myOptional = Optional.of(first).orElseGet(() -> {
System.out.println("OrElseGet - Create & Return Second");
return "Second";
});
System.out.println(
"Result of OrElseGet " + myOptional);
// if supplier is a method then its gets called
myOptional = Optional.of(getThird()).orElseGet(getNumberFromSupplier());
System.out.println(" Result of OrElseGet is "
+ myOptional);
Optional<String> empty = Optional.empty();
// OrElseGet - else is initialized as optional was empty
myOptional = empty.orElseGet(() -> {
System.out.println("OrElseGet - Create & Return Fourth");
return "Four" ;
});
System.out.println("Result of OrElseGEt is " + myOptional);
System.out.println("----------Completed-----------");
}
private static Supplier<String> getNumberFromSupplier() {
System.out.println("Inside Supplier call..");
return ()->{
return "Supplying TEN";
};
}
private static String getThird() {
System.out.println("Inside Third");
return "Third";
}
private static String getFirst() {
System.out.println("Inside GetFirst");
return "First";
}
}
The contract of orElseGet(Supplier) says the →Supplier← will only be invoked if the value is not present. More specifically, it's the Supplier#get() method that's lazily invoked (on the Supplier instance passed to orElseGet).
And your code shows that. However, this bit:
private static Supplier<String> getNumberFromSupplier() {
System.out.println("Inside Supplier call..");
return ()->{
return "Supplying TEN";
};
}
Indicates a misunderstanding. Your println log says "Inside Supplier call...", but at that point you are not inside the supplier. The Supplier is the lambda expression and is returned from the method. That said, you're correct that the getNumberFromSupplier method is invoked eagerly, but that's because its return value is passed as an argument to the orElseGet call. Even for orElseGet(() -> ... ) the Supplier is created eagerly for the same reason.
If you had the following:
private static Supplier<String> getNumberFromSupplier() {
// Remember, the lambda is the implementation of Supplier#get()
return () -> {
System.out.println("Inside Supplier call...");
return "Supplying TEN";
};
}
Then you'd see the Supplier is lazily invoked. Notice the new location for the println call. Additionally, the method would be better named getNumberSupplier as that's a more apt description of what the method actually does.
I am trying to implement the Hive UDF with Parameter and so I am extending GenericUDF class.
The problem is my UDF works find on String Datatype however it throws error if I run on other data types. I want UDF to run regardless of data type.
Would someone please let me know what's wrong with following code.
#Description(name = "Encrypt", value = "Encrypt the Given Column", extended = "SELECT Encrypt('Hello World!', 'Key');")
public class Encrypt extends GenericUDF {
StringObjectInspector key;
StringObjectInspector col;
#Override
public ObjectInspector initialize(ObjectInspector[] arguments) throws UDFArgumentException {
if (arguments.length != 2) {
throw new UDFArgumentLengthException("Encrypt only takes 2 arguments: T, String");
}
ObjectInspector keyObject = arguments[1];
ObjectInspector colObject = arguments[0];
if (!(keyObject instanceof StringObjectInspector)) {
throw new UDFArgumentException("Error: Key Type is Not String");
}
this.key = (StringObjectInspector) keyObject;
this.col = (StringObjectInspector) colObject;
return PrimitiveObjectInspectorFactory.javaStringObjectInspector;
}
#Override
public Object evaluate(DeferredObject[] deferredObjects) throws HiveException {
String keyString = key.getPrimitiveJavaObject(deferredObjects[1].get());
String colString = col.getPrimitiveJavaObject(deferredObjects[0].get());
return AES.encrypt(colString, keyString);
}
#Override
public String getDisplayString(String[] strings) {
return null;
}
}
Error
java.lang.ClassCastException: org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaIntObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.primitive.StringObjectInspector
I would suggest you to replace StringObjectInspector col with PrimitiveObjectInspector col and the corresponding cast this.col = (PrimitiveObjectInspector) colObject. Then there are two ways:
First is to process every possible Primitive type, like this
switch (((PrimitiveTypeInfo) colObject.getTypeInfo()).getPrimitiveCategory()) {
case BYTE:
case SHORT:
case INT:
case LONG:
case TIMESTAMP:
cast_long_type;
case FLOAT:
case DOUBLE:
cast_double_type;
case STRING:
everyting_is_fine;
case DECIMAL:
case BOOLEAN:
throw new UDFArgumentTypeException(0, "Unsupported yet");
default:
throw new UDFArgumentTypeException(0,
"Unsupported type");
}
}
Another way, is to use PrimitiveObjectInspectorUtils.getString method:
Object colObject = col.getPrimitiveJavaObject(deferredObjects[0].get());
String colString = PrimitiveObjectInspectorUtils.getString(colObject, key);
It just pseudocode like examples. Hope it helps.
I'm trying to work with the reddit JSON API. There are post data objects that contain a field called edited which may contain a boolean false if the post hasn't been edited, or a timestamp int if the post was edited.
Sometimes a boolean:
{
"edited": false,
"title": "Title 1"
}
Sometimes an int:
{
"edited": 1234567890,
"title": "Title 2"
}
When trying to parse the JSON where the POJO has the field set to int, I get an error: JsonDataException: Expected an int but was BOOLEAN...
How can I deal with this using Moshi?
I also ran into a similar problem where I had fields that were sometimes booleans, and sometimes ints. I wanted them to always be ints. Here's how I solved it with Moshi and kotlin:
Make a new annotation that you will use on fields to should convert from boolean to int
#JsonQualifier
#Retention(AnnotationRetention.RUNTIME)
#Target(AnnotationTarget.FIELD, AnnotationTarget.VALUE_PARAMETER, AnnotationTarget.FUNCTION)
annotation class ForceToInt
internal class ForceToIntJsonAdapter {
#ToJson
fun toJson(#ForceToInt i: Int): Int {
return i
}
#FromJson
#ForceToInt
fun fromJson(reader: JsonReader): Int {
return when (reader.peek()) {
JsonReader.Token.NUMBER -> reader.nextInt()
JsonReader.Token.BOOLEAN -> if (reader.nextBoolean()) 1 else 0
else -> {
reader.skipValue() // or throw
0
}
}
}
}
Use this annotation on the fields that you want to force to int:
#JsonClass(generateAdapter = true)
data class Discovery(
#Json(name = "id") val id: String = -1,
#ForceToInt #Json(name = "thanked") val thanked: Int = 0
)
The easy way might be to make your Java edited field an Object type.
The better way for performance, error catching, and appliaction usage is to use a custom JsonAdapter.
Example (edit as needed):
public final class Foo {
public final boolean edited;
public final int editedNumber;
public final String title;
public static final Object JSON_ADAPTER = new Object() {
final JsonReader.Options options = JsonReader.Options.of("edited", "title");
#FromJson Foo fromJson(JsonReader reader) throws IOException {
reader.beginObject();
boolean edited = true;
int editedNumber = -1;
String title = "";
while (reader.hasNext()) {
switch (reader.selectName(options)) {
case 0:
if (reader.peek() == JsonReader.Token.BOOLEAN) {
edited = reader.nextBoolean();
} else {
editedNumber = reader.nextInt();
}
break;
case 1:
title = reader.nextString();
break;
case -1:
reader.nextName();
reader.skipValue();
default:
throw new AssertionError();
}
}
reader.endObject();
return new Foo(edited, editedNumber, title);
}
#ToJson void toJson(JsonWriter writer, Foo value) throws IOException {
writer.beginObject();
writer.name("edited");
if (value.edited) {
writer.value(value.editedNumber);
} else {
writer.value(false);
}
writer.name("title");
writer.value(value.title);
writer.endObject();
}
};
Foo(boolean edited, int editedNumber, String title) {
this.edited = edited;
this.editedNumber = editedNumber;
this.title = title;
}
}
Don't forget to register the adapter on your Moshi instance.
Moshi moshi = new Moshi.Builder().add(Foo.JSON_ADAPTER).build();
JsonAdapter<Foo> fooAdapter = moshi.adapter(Foo.class);
Trying to save selections from a CheckBoxList as a comma-separated list (string) in DB (one or more choices selected). I am using a proxy in order to save as a string because otherwise I'd have to create separate tables in the DB for a relation - the work is not worth it for this simple scenario and I was hoping that I could just convert it to a string and avoid that.
The CheckBoxList uses an enum for it's choices:
public enum Selection
{
Selection1,
Selection2,
Selection3
}
Not to be convoluted, but I use [Display(Name="Choice 1")] and an extension class to display something friendly on the UI. Not sure if I can save that string instead of just the enum, although I think if I save as enum it's not a big deal for me to "display" the friendly string on UI on some confirmation page.
This is the "Record" class that saves a string in the DB:
public virtual string MyCheckBox { get; set; }
This is the "Proxy", which is some sample I found but not directly dealing with enum, and which uses IEnumerable<string> (or should it be IEnumerable<Selection>?):
public IEnumerable<string> MyCheckBox
{
get
{
if (String.IsNullOrWhiteSpace(Record.MyCheckBox)) return new string[] { };
return Record
.MyCheckBox
.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries)
.Select(r => r.Trim())
.Where(r => !String.IsNullOrEmpty(r));
}
set
{
Record.MyCheckBox = value == null ? null : String.Join(",", value);
}
}
To save in the DB, I am trying to do this in a create class:
proxy.MyCheckBox = record.MyCheckBox; //getting error here
but am getting the error:
Cannot implicitly convert 'string' to System.Collections.Generic.IEnumerable'
I don't know, if it's possible or better, to use Parse or ToString from the API for enum values.
I know that doing something like this will store whatever I put in the ("") into the DB, so it's just a matter of figuring out how to overcome the error (or, if there is an alternative):
proxy.MyCheckBox = new[] {"foo", "bar"};
I am not good with this stuff and have just been digging and digging to come up with a solution. Any help is much appreciated.
You can accomplish this using a custom user type. The example below uses an ISet<string> on the class and stores the values as a delimited string.
[Serializable]
public class CommaDelimitedSet : IUserType
{
const string delimiter = ",";
#region IUserType Members
public new bool Equals(object x, object y)
{
if (ReferenceEquals(x, y))
{
return true;
}
var xSet = x as ISet<string>;
var ySet = y as ISet<string>;
if (xSet == null || ySet == null)
{
return false;
}
// compare set contents
return xSet.Except(ySet).Count() == 0 && ySet.Except(xSet).Count() == 0;
}
public int GetHashCode(object x)
{
return x.GetHashCode();
}
public object NullSafeGet(IDataReader rs, string[] names, object owner)
{
var outValue = NHibernateUtil.String.NullSafeGet(rs, names[0]) as string;
if (string.IsNullOrEmpty(outValue))
{
return new HashSet<string>();
}
else
{
var splitArray = outValue.Split(new[] {Delimiter}, StringSplitOptions.RemoveEmptyEntries);
return new HashSet<string>(splitArray);
}
}
public void NullSafeSet(IDbCommand cmd, object value, int index)
{
var inValue = value as ISet<string>;
object setValue = inValue == null ? null : string.Join(Delimiter, inValue);
NHibernateUtil.String.NullSafeSet(cmd, setValue, index);
}
public object DeepCopy(object value)
{
// return new ISet so that Equals can work
// see http://www.mail-archive.com/nhusers#googlegroups.com/msg11054.html
var set = value as ISet<string>;
if (set == null)
{
return null;
}
return new HashSet<string>(set);
}
public object Replace(object original, object target, object owner)
{
return original;
}
public object Assemble(object cached, object owner)
{
return DeepCopy(cached);
}
public object Disassemble(object value)
{
return DeepCopy(value);
}
public SqlType[] SqlTypes
{
get { return new[] {new SqlType(DbType.String)}; }
}
public Type ReturnedType
{
get { return typeof(ISet<string>); }
}
public bool IsMutable
{
get { return false; }
}
#endregion
}
Usage in mapping file:
Map(x => x.CheckboxValues.CustomType<CommaDelimitedSet>();
A task at university was to implement a simple proxy generator / interceptor mechanism using Reflection.Emit.
I came up with the following program.
It seems to work just fine inside Visual Studio in debug mode [F5] (Debug -> Start Debugging) but crashes most of the time when started without debugging [Ctrl + F5] (Debug -> Start Without Debugging).
What is the difference between these two modes? (I do not refer to Debug <> Release mode).
The issue occurs on multiple machines/setups (Win XP SP3 32bit and 64bit, Windows 7 32bit).
Click for pastebin.
// The proxy generator; I assume that the error is buried along the lines emitting the IL code
public static class ProxyGenerator
{
public static T Create<T>(object obj, IInterception interception)
{
Type type = obj.GetType();
TypeBuilder proxy = DefineProxy(type);
FieldBuilder wrappedField = DefinePrivateField(proxy, "wrappedObject", type);
FieldBuilder interceptionField = DefinePrivateField(proxy, "interception", interception.GetType());
DefineConstructor(proxy, wrappedField, interceptionField);
DefineInterfaceMethods(type, proxy, wrappedField, interceptionField);
return (T) Activator.CreateInstance(proxy.CreateType(), obj, interception);
}
private static TypeBuilder DefineProxy(Type type)
{
var assemblyName = new AssemblyName {Name = "GeneratedProxyAssembly"};
AssemblyBuilder assemblyBuilder = AppDomain.CurrentDomain.DefineDynamicAssembly(
assemblyName, AssemblyBuilderAccess.Run);
ModuleBuilder moduleBuilder = assemblyBuilder.DefineDynamicModule("GeneratedProxyModule");
return moduleBuilder.DefineType(
type.Name + "Proxy",
type.Attributes,
typeof (object),
type.GetInterfaces());
}
private static FieldBuilder DefinePrivateField(TypeBuilder typeBuilder, string fieldName, Type fieldType)
{
return typeBuilder.DefineField(fieldName, fieldType, FieldAttributes.Private);
}
private static void DefineConstructor(TypeBuilder typeBuilder, params FieldBuilder[] parameters)
{
ConstructorBuilder ctor = typeBuilder.DefineConstructor(
MethodAttributes.Public, CallingConventions.Standard, parameters.Select(f => f.FieldType).ToArray());
// Emit constructor
ILGenerator g = ctor.GetILGenerator();
// Load "this" pointer and call base constructor
g.Emit(OpCodes.Ldarg_0);
g.Emit(OpCodes.Call, typeof(object).GetConstructor(new Type[0]));
// Store parameters in private fields
for (int i = 0; i < parameters.Length; i++)
{
// Load "this" pointer and parameter and store paramater in private field
g.Emit(OpCodes.Ldarg_0);
g.Emit(OpCodes.Ldarg, i + 1);
g.Emit(OpCodes.Stfld, parameters[i]);
}
// Return
g.Emit(OpCodes.Ret);
}
private static void DefineInterfaceMethods(Type type, TypeBuilder proxy, FieldInfo wrappedField, FieldInfo interceptionField)
{
// Loop through all interface methods
foreach (MethodInfo interfaceMethod in type.GetInterfaces().SelectMany(i => i.GetMethods()))
{
MethodInfo method = type.GetMethod(interfaceMethod.Name);
MethodBuilder methodBuilder = proxy.DefineMethod(
method.Name,
method.Attributes,
method.ReturnType,
method.GetParameters().Select(p => p.ParameterType).ToArray());
// Emit method
ILGenerator g = methodBuilder.GetILGenerator();
// Intercept before
EmitMethodCallOnMember(g, interceptionField, "Before", false);
// Delegate method call
EmitMethodCallOnMember(g, wrappedField, method.Name, true);
// Intercept after
EmitMethodCallOnMember(g, interceptionField, "After", false);
// Return
g.Emit(OpCodes.Ret);
}
}
private static void EmitMethodCallOnMember(ILGenerator g, FieldInfo field, string methodName, bool delegateParameters)
{
// Load "this" pointer to get address of field
g.Emit(OpCodes.Ldarg_0);
g.Emit(OpCodes.Ldflda, field);
MethodInfo method = field.FieldType.GetMethod(methodName);
if (delegateParameters)
{
// Load method parameters
for (int i = 0; i < method.GetParameters().Length; i++)
{
g.Emit(OpCodes.Ldarg, i + 1);
}
}
// Emit call
g.Emit(OpCodes.Call, method);
}
}
// Some infrastructure
public interface IInterception
{
void Before();
void After();
}
public class LogInterception : IInterception
{
public void Before()
{
Console.WriteLine("Before ... ");
}
public void After()
{
Console.WriteLine("... After");
}
}
public interface ITest
{
string DoSomething(string s1, string s2);
}
public class Test : ITest
{
public string DoSomething(string s1, string s2)
{
Console.WriteLine("... doing something ...");
return s1 + s2;
}
}
// The test program, expected output is down below
internal class Program
{
internal static void Main(string[] args)
{
var test = new Test();
var proxy = ProxyGenerator.Create<ITest>(test, new LogInterception());
Console.WriteLine(test.DoSomething("Hello", " World"));
Console.WriteLine("----------------------------------------");
Console.WriteLine(proxy.DoSomething("Hello", " World"));
Console.ReadKey();
}
}
Another question: What's the best way to narrow down such issues?
I tried to save the generated assembly to disk and open the resulting dll in Reflector but it appeared to be empty.
As mentioned above, when started in debug mode the program seems to work and prints the following output.
... doing something ...
Hello World
----------------------------------------
Before ...
... doing something ...
... After
Hello World
Thanks for your time.
Try to explicitly set x86 mode on project settings tab.
I got the fatal exception only when run program in x64 or AnyCpu mode.
Ah, I've got it. Replace Ldflda with Ldfld. It works fine even without debugger (I just ran .exe).
Ldflda is for fields you pass into method as parameters with ref or out keyword.