I would like to track changes to a registry key, for instance addition/removal of a subkey, addition/removal/edition of a value. How could I create an IObservable sequence that exposes these changes?
One way is to p/invoke the RegNotifyChangeKeyValue, a Win32 function which notifies the caller about changes to the attributes or contents of a specified registry key. This function sets an event whenever it detects a change. Note that it it must be called on a persistent thread, otherwise it will signal whenever the thread exits (even though no change happened). See below for a possible implementation of this with Rx.Net.
using System;
using System.ComponentModel;
using System.Reactive;
using System.Reactive.Concurrency;
using System.Reactive.Disposables;
using System.Reactive.Linq;
using System.Runtime.InteropServices;
using System.Threading;
using Microsoft.Win32;
public class RegistryMonitoringOperations
{
[Flags]
public enum RegChangeNotifyFilter
{
/// <summary>Notify the caller if a subkey is added or deleted.</summary>
Key = 1,
/// <summary>Notify the caller of changes to the attributes of the key,
/// such as the security descriptor information.</summary>
Attribute = 2,
/// <summary>Notify the caller of changes to a value of the key. This can
/// include adding or deleting a value, or changing an existing value.</summary>
Value = 4,
/// <summary>Notify the caller of changes to the security descriptor
/// of the key.</summary>
Security = 8
}
private const int KeyQueryValue = 0x0001;
private const int KeyNotify = 0x0010;
private const int StandardRightsRead = 0x00020000;
public static IObservable<Unit> CreateKeyValuesChangedObservable(
RegistryHive hive,
string subKey,
RegChangeNotifyFilter filter,
IScheduler registrationScheduler)
{
return Observable.Create<Unit>(
obs =>
{
try
{
var key = OpenKey(hive, subKey);
return new CompositeDisposable(
CreateKeyValuesChangedObservable(key, filter).SubscribeOn(registrationScheduler).Subscribe(obs),
Disposable.Create(() => RegCloseKey(key)));
}
catch (Win32Exception e)
{
obs.OnError(e);
return Disposable.Empty;
}
});
}
private static IDisposable SetCallbackWhenSignalled(WaitHandle waitObject, Action action)
{
var registeredWait = ThreadPool.RegisterWaitForSingleObject(waitObject, (s, t) => action(), null, -1, true);
return Disposable.Create(() => registeredWait.Unregister(null));
}
private static IObservable<Unit> CreateKeyValuesChangedObservable(IntPtr key, RegChangeNotifyFilter filter)
{
return Observable.Create<Unit>(
obs =>
{
var eventNotify = new AutoResetEvent(false);
var result = RegNotifyChangeKeyValue(key, true, filter, eventNotify.SafeWaitHandle.DangerousGetHandle(), true);
if (result != 0)
{
obs.OnError(new Win32Exception(Marshal.GetLastWin32Error()));
}
return new CompositeDisposable(
eventNotify,
SetCallbackWhenSignalled(
eventNotify,
() =>
{
obs.OnNext(Unit.Default);
obs.OnCompleted();
}));
}).Repeat();
}
private static IntPtr OpenKey(RegistryHive hive, string subKey)
{
IntPtr registryKey;
var result = RegOpenKeyEx((int)hive, subKey, 0, StandardRightsRead | KeyQueryValue | KeyNotify, out registryKey);
if (result != 0)
{
throw new Win32Exception(Marshal.GetLastWin32Error());
}
return registryKey;
}
Here's a typical usage of this function:
RegistryMonitoringOperations.CreateKeyValuesChangedObservable(
RegistryHive.LocalMachine,
"somepath",
RegistryMonitoringOperations.RegChangeNotifyFilter.Value,
DispatcherScheduler.Instance)
As you can see above, one way to avoid dedicating a thread for calling this function is to use the UI thread which is persistent (so in rx terms, using the dispatcher scheduler). RegNotifyChangeKeyValue returns immediatly when in asynchronous mode so it won't block the UI.
Related
In Protobuf3 zero is the default value for numeric types, and so they are filtered out when serialized.
I have an application where I need to send a value only when it has changed. For example, x was 1, now x is 0, send this value.
It isn't possible to send only the delta, eg -1, because some of these values are floats or doubles, and we do not want to accrue errors.
There are over 200 different variables in some classes I need to serialize, so solutions like "add a boolean to flag which fields have changed" are possible but not fun. Other suggestions that have a large amount of per-field work or processing are undesirable too.
Is there a simple mechanism to tell protobuf3 to explicitly keep a value even though it is the default value?
Possible solutions:
Send the entire class each time. The main downside here is some fields may have a lot of data.
Use a boolean "has changed" in the schema to indicate if a variable has changed, even if it is 0
Use a magic value. Terrible idea, but possible. Not going to do this.
If you need to distinguish 0 and null then you can use proto3 wrapper types: https://developers.google.com/protocol-buffers/docs/reference/csharp-generated#wrapper_types
There are special wrapper types for such case: StringWrapper, Int32Wrapper and etc. All of the wrapper types that correspond to C# value types (Int32Wrapper, DoubleWrapper, BoolWrapper etc) are mapped to Nullable<T> where T is the corresponding non-nullable type.
Since you tagged protobuf-net, you can do this at the field level:
[ProtoMember(..., IsRequired = true)]
// your field
or globally (here I'm assuming you are using the default model, which is a pretty safe assumption usually):
RuntimeTypeModel.Default.ImplicitZeroDefault = false;
and you're done;
Note: if you're interested in deltas, you can also do this conditionally - for a property Foo, you can add:
private bool ShouldSerializeFoo() { /* your rules here */ }
(this is a name-based pattern used by many serializers and other tools; in some scenarios, it needs to be public, but protobuf-net is usually happy with it non-public)
As a non-trivial example of an object that tracks delta state internally:
using ProtoBuf;
using System;
using System.Collections.Generic;
using System.IO;
static class P
{
static void Main()
{
var obj = new MyType
{
Foo = 42,
Bar = "abc",
Blap = DateTime.Now
};
ShowPayloadSize("original", obj);
obj.MarkClean();
ShowPayloadSize("clean", obj);
obj.Foo = 42;
obj.Bar = "abc";
ShowPayloadSize("set property to same", obj);
obj.Foo = 45;
obj.Bar = "new value";
ShowPayloadSize("set property to different", obj);
obj.MarkClean();
ShowPayloadSize("clean again", obj);
}
static void ShowPayloadSize<T>(string caption, T obj)
{
using var ms = new MemoryStream();
Serializer.Serialize(ms, obj);
Console.WriteLine($"{caption}: {ms.Length} bytes");
}
}
[ProtoContract]
public class MyType
{
private int _dirty = -1; // treat everything as dirty by default
public void MarkClean() => _dirty = 0;
public bool IsDirty => _dirty != 0;
private bool ShouldSerialize(int flag) => (_dirty & flag) != 0;
private void Set<T>(ref T field, T value, int flag)
{
if (!EqualityComparer<T>.Default.Equals(field, value))
{
field = value;
_dirty |= flag;
}
}
[ProtoMember(1)]
public int Foo
{
get => _foo;
set => Set(ref _foo, value, 1 << 0);
}
public bool ShouldSerializeFoo() => ShouldSerialize(1 << 0);
private int _foo;
[ProtoMember(2)]
public string Bar
{
get => _bar;
set => Set(ref _bar, value, 1 << 1);
}
public bool ShouldSerializeBar() => ShouldSerialize(1 << 1);
private string _bar;
[ProtoMember(3)]
public DateTime Blap
{
get => _blap;
set => Set(ref _blap, value, 1 << 2);
}
public bool ShouldSerializeBlap() => ShouldSerialize(1 << 2);
private DateTime _blap;
}
I encapsulated the odata container because of some extensions:
public class Connector
{
public string apiKey{get;set;}
public Connector(string apiKey)
{
this.apiKey = apiKey;
}
public Default.Container Get()
{
Default.Container container = new Default.Container(new Uri("http://documents.omnisoftonline.be/odata"));
container.BuildingRequest += container_BuildingRequest;
return container;
}
/// <summary>
/// When fetching a single Document, you can enable this method to including fetch the binary data
/// </summary>
/// <param name="container"></param>
/// <returns></returns>
internal bool doIO = false;
internal void container_BuildingRequest(object sender, BuildingRequestEventArgs e)
{
e.Headers.Add("X-ApiKey", apiKey);
if (doIO && e.RequestUri.ToString().Contains("/Documents"))
{
e.RequestUri = new Uri(e.RequestUri.ToString() + (e.RequestUri.Query.Contains("?") ? "&" : "?") + "doIO=true");
}
this.container_BuildingRequest(sender, e);
}
}
When i use my .dll ( using the Connector class), i have an empty result ( statuscode = -1, no headers, ...)
This is how i call the DLL
documentsConnector.Get().AddToDocuments(docToCreate);
var serviceResponse = documentsConnector.Get().SaveChanges(Microsoft.OData.Client.SaveChangesOptions.None); //tried several options
foreach(var operationResponse in serviceResponse)
{
Console.WriteLine("Response: {0}", operationResponse.StatusCode); //no operationResponses, so this isn't executed
}
It could be because my object isn't valid. But it's weird that i don't see any validation happening...
Any thoughts on how to propagate the SaveChanges() or pre-validate ( before submit) the Entity? The post isn't happening ( checked with Fiddler)
My wrapper class created a new container every time, so the entities got deleted from the Default.Container
My table is like this (pseudo code):
CREATE TABLE DOCUMENT_LOCATOR (DOCUMENT_ID VARCHAR2(6) PRIMARY KEY)
When using the latest ODP.NET Managed Driver v121.1.2 downloaded via NuGet, the following Entity Framework commands do NOT use the index:
context.DOCUMENT_LOCATOR.Find("ABC123");
context.DOCUMENT_LOCATOR.Find(EntityFunctions.AsNonUnicode("ABC123"));
context.DOCUMENT_LOCATOR.FirstOrDefault(i => i.DOCUMENT_ID == "ABC123");
The following command does use the index:
context.DOCUMENT_LOCATOR.FirstOrDefault(i => i.DOCUMENT_ID == EntityFunctions.AsNonUnicode("ABC123"));
Unfortunately, UPDATEs and DELETEs do not use the index, so they basically render EF/Oracle useless for large tables. Has anyone figured out how to work around this?
After a fair amount of searching, I found a very nice solution. It sets all the string properties of all the entities to IsUnicode(false). To use it, you simply add one line to the OnModelCreating method of your DbContext class:
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Configurations.Add(new ASSUMED_NAME_FILESMap());
modelBuilder.Configurations.Add(new ASSUMED_NAME_FILENAME_ONLYMap());
modelBuilder.Configurations.Add(new DOCUMENT_LOCATORMap());
modelBuilder.Configurations.Add(new IMS_IMAGE_TABLEMap());
modelBuilder.DisableUnicodeForAllEntityStrings(this);
}
As you can see, I changed the code to be an extension. Here's my adapted code:
public static class DbModelBuilderExtensions
{
/// <summary>
/// Disables unicode for all string properties on entities in the given context.
/// MUST BE CALLED AFTER modelBuilder.Configurations.Add(map) CALLS.
/// </summary>
/// <remarks>
/// Adapted from http://www.johncoder.com/Post/EFCodeFirstDisableUnicodeforallStringProperties
/// </remarks>
public static void DisableUnicodeForAllEntityStrings(this DbModelBuilder builder, DbContext context)
{
// Get all IDbSet<> properties from the context
var entityTypes = from property in context.GetType().GetProperties()
where property.PropertyType.ImplementsInterface(typeof(IDbSet<>))
let entityType = property.PropertyType.GetGenericArguments()[0]
select entityType;
// Disable Unicode support for each table
foreach (var entityType in entityTypes)
DisableUnicodeForEntityStrings(builder, entityType);
}
private static void DisableUnicodeForEntityStrings(DbModelBuilder modelBuilder, Type entityType)
{
// Get all string properties with setters
var stringProperties = from property in entityType.GetProperties()
where property.PropertyType == typeof(string)
&& property.CanWrite
select property;
// Each table field must be varchar for now,
// so take the string property & call IsUnicode(false).
foreach (var property in stringProperties)
{
// Don't remove this line.
// Lambda might not work without it.
PropertyInfo prop = property;
// Create the correct expression type,
// should be Expression<Func<TModel, string>>
var exprType = typeof(Expression<>)
.MakeGenericType(typeof(Func<,>)
.MakeGenericType(prop.ReflectedType, typeof(string)));
// Find and execute the Entity() method,
// using TModel generic parameter
var obj = modelBuilder.GetType()
.GetMethod("Entity")
.MakeGenericMethod(prop.ReflectedType)
.Invoke(modelBuilder, null);
// Runtime Lambda expression to represent
// something like Property(p => p.Suffix)
ParameterExpression pe = Expression.Parameter(prop.ReflectedType, "p");
var expression = Expression.Lambda(Expression.Property(pe, prop.Name), pe);
// Find the Property method that takes an expression as a parameter
// and then invoke it using the expression that was just built
var p = obj.GetType()
.GetMethod("Property", new[] { exprType })
.Invoke(obj, new[] { expression });
// If all goes well, we'll have a StringPropertyConfiguration.
var propertyConfig = p as StringPropertyConfiguration;
if (propertyConfig != null)
propertyConfig.IsUnicode(false);
}
}
public static bool ImplementsInterface(this Type value, Type interfaceType)
{
return value.GetInterface(interfaceType.FullName) != null;
}
}
A task at university was to implement a simple proxy generator / interceptor mechanism using Reflection.Emit.
I came up with the following program.
It seems to work just fine inside Visual Studio in debug mode [F5] (Debug -> Start Debugging) but crashes most of the time when started without debugging [Ctrl + F5] (Debug -> Start Without Debugging).
What is the difference between these two modes? (I do not refer to Debug <> Release mode).
The issue occurs on multiple machines/setups (Win XP SP3 32bit and 64bit, Windows 7 32bit).
Click for pastebin.
// The proxy generator; I assume that the error is buried along the lines emitting the IL code
public static class ProxyGenerator
{
public static T Create<T>(object obj, IInterception interception)
{
Type type = obj.GetType();
TypeBuilder proxy = DefineProxy(type);
FieldBuilder wrappedField = DefinePrivateField(proxy, "wrappedObject", type);
FieldBuilder interceptionField = DefinePrivateField(proxy, "interception", interception.GetType());
DefineConstructor(proxy, wrappedField, interceptionField);
DefineInterfaceMethods(type, proxy, wrappedField, interceptionField);
return (T) Activator.CreateInstance(proxy.CreateType(), obj, interception);
}
private static TypeBuilder DefineProxy(Type type)
{
var assemblyName = new AssemblyName {Name = "GeneratedProxyAssembly"};
AssemblyBuilder assemblyBuilder = AppDomain.CurrentDomain.DefineDynamicAssembly(
assemblyName, AssemblyBuilderAccess.Run);
ModuleBuilder moduleBuilder = assemblyBuilder.DefineDynamicModule("GeneratedProxyModule");
return moduleBuilder.DefineType(
type.Name + "Proxy",
type.Attributes,
typeof (object),
type.GetInterfaces());
}
private static FieldBuilder DefinePrivateField(TypeBuilder typeBuilder, string fieldName, Type fieldType)
{
return typeBuilder.DefineField(fieldName, fieldType, FieldAttributes.Private);
}
private static void DefineConstructor(TypeBuilder typeBuilder, params FieldBuilder[] parameters)
{
ConstructorBuilder ctor = typeBuilder.DefineConstructor(
MethodAttributes.Public, CallingConventions.Standard, parameters.Select(f => f.FieldType).ToArray());
// Emit constructor
ILGenerator g = ctor.GetILGenerator();
// Load "this" pointer and call base constructor
g.Emit(OpCodes.Ldarg_0);
g.Emit(OpCodes.Call, typeof(object).GetConstructor(new Type[0]));
// Store parameters in private fields
for (int i = 0; i < parameters.Length; i++)
{
// Load "this" pointer and parameter and store paramater in private field
g.Emit(OpCodes.Ldarg_0);
g.Emit(OpCodes.Ldarg, i + 1);
g.Emit(OpCodes.Stfld, parameters[i]);
}
// Return
g.Emit(OpCodes.Ret);
}
private static void DefineInterfaceMethods(Type type, TypeBuilder proxy, FieldInfo wrappedField, FieldInfo interceptionField)
{
// Loop through all interface methods
foreach (MethodInfo interfaceMethod in type.GetInterfaces().SelectMany(i => i.GetMethods()))
{
MethodInfo method = type.GetMethod(interfaceMethod.Name);
MethodBuilder methodBuilder = proxy.DefineMethod(
method.Name,
method.Attributes,
method.ReturnType,
method.GetParameters().Select(p => p.ParameterType).ToArray());
// Emit method
ILGenerator g = methodBuilder.GetILGenerator();
// Intercept before
EmitMethodCallOnMember(g, interceptionField, "Before", false);
// Delegate method call
EmitMethodCallOnMember(g, wrappedField, method.Name, true);
// Intercept after
EmitMethodCallOnMember(g, interceptionField, "After", false);
// Return
g.Emit(OpCodes.Ret);
}
}
private static void EmitMethodCallOnMember(ILGenerator g, FieldInfo field, string methodName, bool delegateParameters)
{
// Load "this" pointer to get address of field
g.Emit(OpCodes.Ldarg_0);
g.Emit(OpCodes.Ldflda, field);
MethodInfo method = field.FieldType.GetMethod(methodName);
if (delegateParameters)
{
// Load method parameters
for (int i = 0; i < method.GetParameters().Length; i++)
{
g.Emit(OpCodes.Ldarg, i + 1);
}
}
// Emit call
g.Emit(OpCodes.Call, method);
}
}
// Some infrastructure
public interface IInterception
{
void Before();
void After();
}
public class LogInterception : IInterception
{
public void Before()
{
Console.WriteLine("Before ... ");
}
public void After()
{
Console.WriteLine("... After");
}
}
public interface ITest
{
string DoSomething(string s1, string s2);
}
public class Test : ITest
{
public string DoSomething(string s1, string s2)
{
Console.WriteLine("... doing something ...");
return s1 + s2;
}
}
// The test program, expected output is down below
internal class Program
{
internal static void Main(string[] args)
{
var test = new Test();
var proxy = ProxyGenerator.Create<ITest>(test, new LogInterception());
Console.WriteLine(test.DoSomething("Hello", " World"));
Console.WriteLine("----------------------------------------");
Console.WriteLine(proxy.DoSomething("Hello", " World"));
Console.ReadKey();
}
}
Another question: What's the best way to narrow down such issues?
I tried to save the generated assembly to disk and open the resulting dll in Reflector but it appeared to be empty.
As mentioned above, when started in debug mode the program seems to work and prints the following output.
... doing something ...
Hello World
----------------------------------------
Before ...
... doing something ...
... After
Hello World
Thanks for your time.
Try to explicitly set x86 mode on project settings tab.
I got the fatal exception only when run program in x64 or AnyCpu mode.
Ah, I've got it. Replace Ldflda with Ldfld. It works fine even without debugger (I just ran .exe).
Ldflda is for fields you pass into method as parameters with ref or out keyword.
I have an extension method for the Microsoft.ApplicationServer.Caching.DataCache object found in the Windows Server AppFabric SDK that looks like this:
using System;
using System.Collections.Generic;
using Microsoft.ApplicationServer.Caching;
namespace Caching
{
public static class CacheExtensions
{
private static Dictionary<string, object> locks = new Dictionary<string, object>();
public static T Fetch<T>(this DataCache #this, string key, Func<T> func)
{
return #this.Fetch(key, func, TimeSpan.FromSeconds(30));
}
public static T Fetch<T>(this DataCache #this, string key, Func<T> func, TimeSpan timeout)
{
var result = #this.Get(key);
if (result == null)
{
lock (GetLock(key))
{
result = #this.Get(key);
if (result == null)
{
result = func();
if (result != null)
{
#this.Put(key, result, timeout);
}
}
}
}
return (T)result;
}
private static object GetLock(string key)
{
object #lock = null;
if (!locks.TryGetValue(key, out #lock))
{
lock (locks)
{
if (!locks.TryGetValue(key, out #lock))
{
#lock = new object();
locks.Add(key, #lock);
}
}
}
return #lock;
}
}
}
The intent is to let the developer write code that says, "fetch me some data by trying the cache first. if it's not available in cache execute the specified function, put the results in cache for the next caller, then return the results". Like this:
var data = dataCache.Fetch("key", () => SomeLongRunningOperation());
The locking limits executing the potentially long running function call to a single thread but only within a single process on the same machine. How would you expand on this pattern to make the locking distributed to prevent multiple processes/machines from executing the function at once?
AppFabric has it's own distributed locking mechanism which you can access through the GetAndLock/PutAndUnlock family of methods. If your item is locked, a normal Get call will still succeed and return the last value, but further GetAndLock calls will throw an Exception. In the case where your client is requesting a cached object for the first time, you can still lock the key even though it doesn't really exist yet (it's kind of more like a reservation than a solid lock).
public static T Fetch<T>(this DataCache #this, string key, Func<T> func, TimeSpan timeout)
{
var result = #this.Get(key);
if (result == null)
(
DataCacheLockHandle handle;
// We need a timespan to allow func time to run
TimeSpan funcTimespan = New TimeSpan(0,1,0);
try
{
// Lock the key
// If something goes wrong here it will unlock at the end of funcTimespan
var result = #this.GetAndLock(key, funcTimespan, handle);
if (result == null)
{
// Still no value so go and run func
result = func();
#this.PutAndUnlock(key, result, handle, timeout);
}
else
{
// There's a value now so we'll unlock the key and reset it's timeout
#this.Unlock(key, handle, timeout);
}
}
catch (DataCacheException ex)
{
if (ex.ErrorCode == DataCacheErrorCode.ObjectLocked)
{
// Another process has locked the key so func must be running right now
// We'll return null to the client
result = null;
}
}
if (result == null)
{
return null;
}
else
{
return (T)result;
}
)
}
I was looking for a good implementation of this and came up with my own:
Distributed Lock with AppFabric Caching
Essentially it's an AcquireLock() extension method to the DataCache class which you can use like this:
DataCache cache = factory.GetCache("MyCache");
using (cache.AcquireLock("MyLock"))
{
// Lock acquired, perform synchronized work here
}