Distributed Lock Service with Windows Server AppFabric Caching - caching

I have an extension method for the Microsoft.ApplicationServer.Caching.DataCache object found in the Windows Server AppFabric SDK that looks like this:
using System;
using System.Collections.Generic;
using Microsoft.ApplicationServer.Caching;
namespace Caching
{
public static class CacheExtensions
{
private static Dictionary<string, object> locks = new Dictionary<string, object>();
public static T Fetch<T>(this DataCache #this, string key, Func<T> func)
{
return #this.Fetch(key, func, TimeSpan.FromSeconds(30));
}
public static T Fetch<T>(this DataCache #this, string key, Func<T> func, TimeSpan timeout)
{
var result = #this.Get(key);
if (result == null)
{
lock (GetLock(key))
{
result = #this.Get(key);
if (result == null)
{
result = func();
if (result != null)
{
#this.Put(key, result, timeout);
}
}
}
}
return (T)result;
}
private static object GetLock(string key)
{
object #lock = null;
if (!locks.TryGetValue(key, out #lock))
{
lock (locks)
{
if (!locks.TryGetValue(key, out #lock))
{
#lock = new object();
locks.Add(key, #lock);
}
}
}
return #lock;
}
}
}
The intent is to let the developer write code that says, "fetch me some data by trying the cache first. if it's not available in cache execute the specified function, put the results in cache for the next caller, then return the results". Like this:
var data = dataCache.Fetch("key", () => SomeLongRunningOperation());
The locking limits executing the potentially long running function call to a single thread but only within a single process on the same machine. How would you expand on this pattern to make the locking distributed to prevent multiple processes/machines from executing the function at once?

AppFabric has it's own distributed locking mechanism which you can access through the GetAndLock/PutAndUnlock family of methods. If your item is locked, a normal Get call will still succeed and return the last value, but further GetAndLock calls will throw an Exception. In the case where your client is requesting a cached object for the first time, you can still lock the key even though it doesn't really exist yet (it's kind of more like a reservation than a solid lock).
public static T Fetch<T>(this DataCache #this, string key, Func<T> func, TimeSpan timeout)
{
var result = #this.Get(key);
if (result == null)
(
DataCacheLockHandle handle;
// We need a timespan to allow func time to run
TimeSpan funcTimespan = New TimeSpan(0,1,0);
try
{
// Lock the key
// If something goes wrong here it will unlock at the end of funcTimespan
var result = #this.GetAndLock(key, funcTimespan, handle);
if (result == null)
{
// Still no value so go and run func
result = func();
#this.PutAndUnlock(key, result, handle, timeout);
}
else
{
// There's a value now so we'll unlock the key and reset it's timeout
#this.Unlock(key, handle, timeout);
}
}
catch (DataCacheException ex)
{
if (ex.ErrorCode == DataCacheErrorCode.ObjectLocked)
{
// Another process has locked the key so func must be running right now
// We'll return null to the client
result = null;
}
}
if (result == null)
{
return null;
}
else
{
return (T)result;
}
)
}

I was looking for a good implementation of this and came up with my own:
Distributed Lock with AppFabric Caching
Essentially it's an AcquireLock() extension method to the DataCache class which you can use like this:
DataCache cache = factory.GetCache("MyCache");
using (cache.AcquireLock("MyLock"))
{
// Lock acquired, perform synchronized work here
}

Related

Mocking a class with internal SDK calls using NSubstitute

First time trying to use NSubstitute.
I have the following method in my Web API.
For those who don't know Couchbase, lets say that a collection/bucket is like a DB table and a key is like a DB row.
Couchbase_internal.Collection_GET returns Task<ICouchbaseCollection>
I would like to write 2 unit tests.
One that tests the returned class when the key exist and one when it doesn't (couchbaseServiceResultClass).
I don't really understand where is the part where I control whether or not the key exist in the mocked data.
public class CouchbaseAPI : ControllerBase, ICouchbaseAPI
{
// GET /document_GET?bucketName=<bucketName>&key=<key>
[HttpGet]
[Consumes("application/x-www-form-urlencoded")]
[Produces(MediaTypeNames.Application.Json)]
public async Task<couchbaseServiceResultClass> document_GET([FromQuery, BindRequired] string bucketName, [FromQuery, BindRequired] string key)
{
var collection = await Couchbase_internal.Collection_GET(bucketName);
if (collection != null)
{
IGetResult result;
try
{
// get document
result = await collection.GetAsync(key);
}
catch (CouchbaseException ex)
{
return new ErrorHandling().handleCouchbaseException(ex);
}
couchbaseServiceResultClass decryptResult = new();
try
{
// decrypt document
decryptResult = Encryption.decryptContent(result);
}
catch (Exception ex)
{
return new ErrorHandling().handleException(ex, null);
}
// remove document if decryption failed
if (!decryptResult.DecryptSuccess)
{
try
{
await collection.RemoveAsync(key);
}
catch (CouchbaseException ex)
{
return new ErrorHandling().handleCouchbaseException(ex);
}
}
decryptResult.Message = "key retrieved successfully";
// return result
return decryptResult;
}
else
{
return new ErrorHandling().handleError("Collection / bucket was not found.");
}
}
This is what I have so far for the first test:
public class CouchbaseAPITests
{
private readonly CouchbaseAPI.Controllers.ICouchbaseAPI myClass = Substitute.For<CouchbaseAPI.Controllers.ICouchbaseAPI>();
[Fact]
public async Task document_GET_aKeyIsRetrievedSuccessfully()
{
// Arrange
string bucketName = "myBucket";
string keyName = "myKey";
couchbaseServiceResultClass resultClass = new();
resultClass.Success = true;
resultClass.Message = "key retrieved successfully";
myClass.document_GET(bucketName, keyName).Returns(resultClass);
// Act
var document = await myClass.document_GET(bucketName, keyName);
// Assert
Assert.True(document.Success);
Assert.Equal("key retrieved successfully", document.Message);
}
}
If we want to test that we are retrieving documents from the Couchbase API properly, then generally we want to use a real instance (local test setup) of that API where possible. If we are mocking this then our tests are not really telling us about whether our code is working correctly (just that our mock is working the way we want it to).
When certain APIs are difficult to use real instances for (e.g. non-deterministic code, difficult to reproduce conditions such as network errors, slow dependencies, etc), that's when it can be useful to introduce an interface for that dependency and to mock that for our test.
Here's a very rough example that doesn't quite match the code snippets posted, but hopefully will give you some ideas on how to proceed.
public interface IDataAdapter {
IEnumerable<IGetResult> Get(string key);
}
public class CouchbaseAdapter : IDataAdapter {
/* Implement interface for Couchbase */
}
public class AppApi {
private IDataAdapter data;
public AppApi(IDataAdapter data) {
this.data = data;
}
public SomeResult Lookup(string key) {
try {
var result = data.Get(key);
return Transform(Decrypt(result));
} catch (Exception ex) { /* error handling */ }
}
}
[Fact]
public void TestWhenKeyExists() {
var testAdapter = Substitute.For<IDataAdapter>();
var api = new AppApi(testAdapter);
testAdapter.Get("abc").Returns(/* some valid data */);
var result = api.Lookup("abc");
/* assert that result is decrypted/transformed as expected */
Assert.Equal(expectedResult, result);
}
[Fact]
public void TestWhenKeyDoesNotExist() {
var testAdapter = Substitute.For<IDataAdapter>();
var api = new AppApi(testAdapter);
var emptyData = new List<IGetResult>();
testAdapter.Get("abc").Returns(emptyData);
var result = api.Lookup("abc");
/* assert that result has handled error as expected */
Assert.Equal(expectedError, result);
}
Here we've introduced a IDataAdapter type that our class uses to abstract the details of which implementation we are using to get data. Our real code can use the CouchbaseAdapter implementation, but our tests can use a mocked version instead. For our tests, we can simulate what happens when the data adapter throws errors or returns specific information.
Note that we're only testing AppApi here -- we are not testing the CouchbaseAdapter implementation, only that AppApi will respond in a certain way if its IDataAdapter has certain behaviour. To test our CouchbaseAdapter we will want to use a real instance, but we don't have to worry about those details for testing our AppApi transformation and decryption code.

Implement `Process.waitFor(long timeout, TimeUnit unit)` in Java 6

I am working on a legacy (Java 6/7) project that uses ProcessBuilder to request a UUID from the machine in an OS-agnostic way. I would like to use the Process.waitFor(long timeout, TimeUnit unit) method from Java 8, but this isn't implemented in Java 6. Instead, I can use waitFor(), which blocks until completion or an error.
I would like to avoid upgrading the version of Java used to 8 if possible as this necessitates a lot of other changes (migrating code away from removed internal APIs and upgrading a production Tomcat server, for example).
How can I best implement the code for executing the process, with a timeout? I was thinking of somehow implementing a schedule that checks if the process is still running and cancelling/destroying it if it is and the timeout has been reached.
My current (Java 8) code looks like this:
/** USE WMIC on Windows */
private static String getSystemProductUUID() {
String uuid = null;
String line;
List<String> cmd = new ArrayList<String>() {{
add("WMIC.exe"); add("csproduct"); add("get"); add("UUID");
}};
BufferedReader br = null;
Process p = null;
SimpleLogger.debug("Attempting to retrieve Windows System UUID through WMIC ...");
try {
ProcessBuilder pb = new ProcessBuilder().directory(getExecDir());
p = pb.command(cmd).start();
if (!p.waitFor(TIMEOUT, SECONDS)) { // No timeout in Java 6
throw new IOException("Timeout reached while waiting for UUID from WMIC!");
}
br = new BufferedReader(new InputStreamReader(p.getInputStream()));
while ((line = br.readLine()) != null) {
if (null != line) {
line = line.replace("\t", "").replace(" ", "");
if (!line.isEmpty() && !line.equalsIgnoreCase("UUID")) {
uuid = line.replace("-", "");
}
}
}
} catch (IOException | InterruptedException ex) {
uuid = null;
SimpleLogger.error(
"Failed to retrieve machine UUID from WMIC!" + SimpleLogger.getPrependedStackTrace(ex)
);
// ex.printStackTrace(System.err);
} finally {
if (null != br) {
try {
br.close();
} catch (IOException ex) {
SimpleLogger.warn(
"Failed to close buffered reader while retrieving machine UUID!"
);
}
if (null != p) {
p.destroy();
}
}
}
return uuid;
}
You can use the following code which only uses features available under Java 6:
public static boolean waitFor(Process p, long t, TimeUnit u) {
ScheduledExecutorService ses = Executors.newSingleThreadScheduledExecutor();
final AtomicReference<Thread> me = new AtomicReference<Thread>(Thread.currentThread());
ScheduledFuture<?> f = ses.schedule(new Runnable() {
#Override public void run() {
Thread t = me.getAndSet(null);
if(t != null) {
t.interrupt();
me.set(t);
}
}
}, t, u);
try {
p.waitFor();
return true;
}
catch(InterruptedException ex) {
return false;
}
finally {
f.cancel(true);
ses.shutdown();
// ensure that the caller doesn't get a spurious interrupt in case of bad timing
while(!me.compareAndSet(Thread.currentThread(), null)) Thread.yield();
Thread.interrupted();
}
}
Note that unlike other solutions you can find somewhere, this will perform the Process.waitFor() call within the caller’s thread, which is what you would expect when looking at the application with a monitoring tool. It also helps the performance for short running sub-processes, as the caller thread will not do much more than the Process.waitFor(), i.e. does not need to wait for the completion of background threads. Instead, what will happen in the background thead, is the interruption of the initiating thread if the timeout elapsed.

Xamarin Realm - when to close Realm

I have a Shared Project where I have changed the database to Realm instead of SQLite.
My problem is, if I close the Realm in my DatabaseManager, the result is removed. Therefore i have created a static singelton instance of the Realm, which all my DatabaseManager use. Now my app crash after short time on memory, and if i remove all my database-functions, it works.
I create my Realm-instance here:
public class RealmDatabase
{
private Realm mRealmDB;
public Realm RealmDB
{
get
{
if (mRealmDB == null || mRealmDB.IsClosed)
{
SetRealm ();
}
return mRealmDB;
}
}
static RealmDatabase cCurrentInstance;
public static RealmDatabase Current
{
get
{
if (cCurrentInstance == null)
cCurrentInstance = new RealmDatabase ();
return cCurrentInstance;
}
}
public RealmDatabase ()
{
}
private void SetRealm ()
{
var config = new RealmConfiguration ("DBName.realm", true);
mRealmDB = Realm.GetInstance (config);
}
public Transaction BeginTransaction ()
{
return RealmDB.BeginWrite ();
}
}
The I have my DatabaseManagler looking like this:
public class NewFreeUserManager
{
internal Realm RealmDB = RealmDatabase.Current.RealmDB;
static NewFreeUserManager cCurrentInstance;
public static NewFreeUserManager Current
{
get
{
if (cCurrentInstance == null)
cCurrentInstance = new NewFreeUserManager ();
return cCurrentInstance;
}
}
private NewFreeUserManager ()
{
}
internal bool Save (FreeUser freeuser)
{
try
{
using (var trans = RealmDB.BeginWrite ())
{
RealmDB.RemoveAll<FreeUser> ();
var fu = RealmDB.CreateObject<FreeUser> ();
fu = freeuser;
trans.Commit ();
}
return true;
}
catch (Exception e)
{
Console.WriteLine ("FreeUser save: " + e.ToString ());
return false;
}
}
internal FreeUser Get ()
{
return RealmDB.All<FreeUser> ().FirstOrDefault ();
}
}
Can anyone help me?
there are a few issues with your current setup that prevent you from persisting objects properly.
The first and very important one is that Realm instances are not thread-safe. That is, using them as singletons is strongly discouraged, unless you are certain that you'll never access them from another thread.
The second is more subtle, but in your save method you are calling:
var fu = RealmDB.CreateObject<FreeUser>();
fu = freeuser;
What it does is, effectively, you are creating an object in the Realm, and then assigning the variable to another object. This will not assign freeuser's properties to fu, it just replaces one reference with another. What you're looking for is Realm.Manage so your code should look like this:
using (var trans = RealmDB.BeginWrite())
{
RealmDB.Manage(freeuser);
trans.Commit();
}
Once you fix the second bug, you should be able to go back and close Realm instances when you don't need them anymore.

Android getContentResolver insert not returning full URI

I have an activity that is being swapped out when I raise an intent for another activity. onPause calls saveState() to save work so far:
private void saveState() {
...
...
if (myUri == null) {
// Inserting a new record
*** myUri = getContentResolver().insert(ContentProvider.CONTENT_URI, values);
} else {
// Update an existing record
getContentResolver().update(myUri, values, null, null);
}
}
Before calling getContentResolver(), ContentProvider.CONTENT_URI = 'content://nz.co.bkd.extraTime.contentprovider/times'.
After the call, myUri = 'times/#' where #=row ID. My question is; where is the 'content:...' prefix to the returned uri?
During the call, ContentResolver.java is called and returns CreatedRow uri
ContentResolver.java
....
....
public final Uri insert(Uri url, ContentValues values)
{
IContentProvider provider = acquireProvider(url);
if (provider == null) {
throw new IllegalArgumentException("Unknown URL " + url);
}
try {
long startTime = SystemClock.uptimeMillis();
*** Uri createdRow = provider.insert(url, values);
long durationMillis = SystemClock.uptimeMillis() - startTime;
maybeLogUpdateToEventLog(durationMillis, url, "insert", null /* where */);
return createdRow;
} catch (RemoteException e) {
// Arbitrary and not worth documenting, as Activity
// Manager will kill this process shortly anyway.
return null;
} finally {
releaseProvider(provider);
}
}
At this point, createdRow = 'times/#'.
The record does actually get saved in the Sqlite database.
Do I have to add the uri prefix in my code or should the full uri be returned?

Force tags to update from external event

this is a somewhat exotic question but I will try anyway.
I am writing a Visual Studio 2010 Extension using MEF. At some point in my code, I am required to provide some error glyphs ( like brakepoints ) whenever a document is saved. The problem is that I can't seem to find how to force GetTags of my ITagger to be called without having the user writing anything.
I can catch the document save event but I am missing the "link" between this and somehow getting GetTags method to be called. Any ideas?
Here is some code :
internal class RTextErrorTag : IGlyphTag
{}
class RTextErrorGlyphTagger : ITagger<RTextErrorTag>
{
private IClassifier mClassifier;
private SimpleTagger<ErrorTag> mSquiggleTagger;
private EnvDTE._DTE mMSVSInstance = null;
private List<object> mDTEEvents = new List<object>();
internal RTextErrorGlyphTagger(IClassifier classifier, IErrorProviderFactory squiggleProviderFactory, ITextBuffer buffer, IServiceProvider serviceProvider)
{
this.mClassifier = classifier;
this.mSquiggleTagger = squiggleProviderFactory.GetErrorTagger(buffer);
mMSVSInstance = serviceProvider.GetService(typeof(EnvDTE._DTE)) as EnvDTE._DTE;
var eventHolder = mMSVSInstance.Events.DocumentEvents;
mDTEEvents.Add(eventHolder);
eventHolder.DocumentSaved += new EnvDTE._dispDocumentEvents_DocumentSavedEventHandler(OnDocumentSaved);
}
void OnDocumentSaved(EnvDTE.Document Document)
{
//fire up event to renew glyphs - how to force GetTags method?
var temp = this.TagsChanged;
if (temp != null)
temp(this, new SnapshotSpanEventArgs(new SnapshotSpan() )); //how to foce get
}
IEnumerable<ITagSpan<RTextErrorTag>> ITagger<RTextErrorTag>.GetTags(NormalizedSnapshotSpanCollection spans)
{
foreach (SnapshotSpan span in spans)
{
foreach (var error in this.mSquiggleTagger.GetTaggedSpans(span))
{
yield return new TagSpan<RTextErrorTag>(new SnapshotSpan(span.Start, 1), new RTextErrorTag());
}
}
}
public event EventHandler<SnapshotSpanEventArgs> TagsChanged;
}

Resources