Service Bus - Retrieve message from session by sequence number - session

I am currently trying to retrieve a specific message from a session.
To do so I use want to use the .Receive(Int64) on the MessageSession where I pass in the sequence number of the message.
Here is my code -
long msgSequenceNr = 1337;
QueueClient queueClient = QueueClient.CreateFromConnectionString(Constants.ServiceBusConnectionString, Constants.TestQueueEntityName, ReceiveMode.PeekLock);
MessageSession msgSession = queueClient.AcceptMessageSession(Constants.TestSessionId);
var peekedMsg = msgSession.Peek(msgSequenceNr); // <-- Works fine!
var receivedMsg = msgSession.Receive(msgSequenceNr); // <-- MessageNotFoundException
Unfortunately the Receive will result in a MessageNotFoundException while the Peek works fine.
Is this a limitation that I missed or is there another way to achieve this.
Note that it is possible that there are multiple messages in the session

Receive with the SequenceNumber can only be used in combination with the Defer method. This is how you would implement it:
Message received, but it can't be processed right now (maybe it's waiting for a different process to complete).
Persist the SequenceNumber in some persistent storage (Table Storage, SQL Database, ...)
When you know that processing can continue (eg: the dependent process is complete), load all SequenceNumbers from your persistent storage.
Use Receive(int sequenceNumber) or ReceiveBatch(int[] sequenceNumbers) to received and process your deferred messages.
Sample application: https://code.msdn.microsoft.com/windowsazure/Brokered-Messaging-ccc4f879#content
Update:
Form your comment I noticed that "undeferring" a deferred message could be a solution. Here's some sample code to undefer the message which copies the deferred message to a new message, Completes the deferred message and sends the new message back in the queue. This uses a TransactionScope to transactionally Complete and Resend the message to avoid the risk of losing the message:
var messageId = "12434539828282";
// Send.
var msg = new BrokeredMessage {SessionId = "user1", MessageId = messageId };
msg.Properties.Add("Language", "Dutch");
queue.Send(msg);
// Receive.
var session = queue.AcceptMessageSession();
msg = session.Receive();
// Store the sequence number.
var sequenceNumber = msg.SequenceNumber;
// Defer.
msg.Defer();
// Change to true to test if the transaction worked.
var shouldThrow = false;
// Later processing of deferred message.
msg = session.Receive(sequenceNumber);
try
{
using (var ts = new TransactionScope())
{
// Create a new message.
var undeferredMessage = new BrokeredMessage {SessionId = msg.SessionId, MessageId = msg.MessageId};
foreach (var prop in msg.Properties)
undeferredMessage.Properties.Add(prop);
// Complete and send within the same transaction.
msg.Complete();
if (shouldThrow)
throw new InvalidOperationException("Some error");
queue.Send(undeferredMessage);
// Complete the transaction.
ts.Complete();
}
}
catch (Exception ex)
{
msg.Abandon();
}
if (shouldThrow)
{
msg = session.Receive(sequenceNumber);
Console.WriteLine(msg.MessageId + " should match: " + messageId);
}
else
{
try
{
msg = session.Receive(sequenceNumber);
}
catch (Exception ex)
{
Console.WriteLine("Message not found, transaction worked OK.");
}
}
Note: here I'm simply taking a copy of the Properties. Keep into account that you might want to copy the Body and any other additional information.

Related

Ibm Wmq - Xms .Net - Read and delete message from Queue

Greetings of the day.
Please help on the below requirement:
Requirement:
We want to delete message from MQ only after it is processed successfully.
Use event based message detection technique and avoid loop
So, to achieve above:
I have created message listener and consumer class below:
{
sessionIn = connectionIn.CreateSession(false, AcknowledgeMode.ClientAcknowledge);
// Create message listener and assign it to consumer
messageListener = new MessageListener(OnMessageCallback);
consumerAsync.MessageListener = messageListener;
Console.WriteLine("Message Listener set. Starting the connection now.");
// Start the connection to receive messages.
connectionWMQ.Start();
}
Reading the message from the call back event and push the message into other system:
OnMessageCallback(Message) {
if (xmsMessage is IBytesMessage)
{
IBytesMessage bytesMessage = (IBytesMessage)xmsMessage;
byte[] arrayMessage = new byte[bytesMessage.BodyLength];
bytesMessage.ReadBytes(arrayMessage);
string message = System.Text.Encoding.Default.GetString(arrayMessage);
}
}
Once the message processed, external system will fire the below over ride method:
Response method override:
protected override Task OnResponse(ReponseMessage message)
{
 //Read the message and get the message id and correlation id.
//Delete the message from the queue.
//I am trying to do like this, but Its not working:
messageConsumerDelete = sessionDelete.CreateConsumer(destinationDelete, query);
if (messageConsumerDelete != null)
{
IMessage m = messageConsumerDelete.Receive(1000);
LogWrite("Receive Message=" + m);
m.Acknowledge();
}
}
Please suggest a best solution for this requirement.
I am trying to find a solution for this since weeks, but no breakthrough. 
Thanks,
Balaji

IBM MQ transactions and .net

I have used .net C# (IBM MQ version 9.1.5) to pull messages from the queue. So I have no issues connecting to the queue and getting messages.
I have read that there is the concept of transactions Distributed Transactions.
I tried the following:
var getMessageOptions = new MQGetMessageOptions();
getMessageOptions = new MQGetMessageOptions();
getMessageOptions.Options += MQC.MQGMO_WAIT + MQC.MQGMO_SYNCPOINT;
getMessageOptions.WaitInterval = 20000; // 20 seconds wait
Transaction oldAmbient = Transaction.Current;
using (var tx = new CommittableTransaction())
{
try
{
int i = queue.CurrentDepth;
Log.Information($"Current queue depth is {i} message(s)");
var message = new MQMessage();
queue.Get(message, getMessageOptions);
string messageStr = message.ReadString(message.DataLength);
Log.Information(messageStr);
tx.Commit();
}
catch (MQException e) when (e.Reason == 2033)
{
// Report exceptions other than "no messages in the queue"
Log.Information("No messages in the queue");
tx.Rollback();
}
catch (Exception ex)
{
Log.Error($"Exception when trying to capture a message from the queue: {ex.Message}");
tx.Rollback();
}
I am getting an error code of 2035.
Looking at the documents on Recovering Transactions, where does the "SYSTEM.DOTNET.XARECOVERY.QUEUE" live, is it on the queuemanger?
Do I need to get permissions enabled on this?
Also I see that Microsoft Distributed Transaction Manager is mentioned, is this something that we need to have running on the local host in order for distributed transactions to work?
If MQ Distributed transactions feature is being used then the user running the application should have the authority to "SYSTEM.DOTNET.XARECOVERY.QUEUE".If a transaction is incomplete "SYSTEM.DOTNET.XARECOVERY.QUEUE" queue holds the information of incomplete transaction as message in that queue,which later can be used to resolve the transaction.
Based on your scenario which you had put in comments i.e "we want to just save the message to a file. My thinking is if there is a problem with that, I could roll back the transaction." .If MQ is the only resource manager then you don't have to use Distributed transactions. Getting a message under syncpoint can also be used instead of Distributed Transactions. Distributed Transactions will be useful if more than one resource manager is being used.
To get a message under syncpoint following sample code can be used by updating hostname,channel,port,queue and queue manager name:
var getMessageOptions = new MQGetMessageOptions();
getMessageOptions = new MQGetMessageOptions();
getMessageOptions.Options += MQC.MQGMO_WAIT + MQC.MQGMO_SYNCPOINT;
getMessageOptions.WaitInterval = 20000; // 20 seconds wait
Hashtable props = new Hashtable();
props.Add(MQC.HOST_NAME_PROPERTY, "localhost");
props.Add(MQC.CHANNEL_PROPERTY, "DOTNET.SVRCONN");
props.Add(MQC.PORT_PROPERTY, 3636);
MQQueueManager qm = new MQQueueManager("QM", props);
MQQueue queue = qm.AccessQueue("Q1", MQC.MQOO_INPUT_AS_Q_DEF);
try
{
var message = new MQMessage();
queue.Get(message, getMessageOptions);
//to commit the message
qm.Commit();
string messageStr = message.ReadString(message.DataLength);
}
catch (MQException e) when (e.Reason == 2033)
{
// Report exceptions other than "no messages in the queue"
Log.Information("No messages in the queue");
}
catch (Exception ex)
{
Log.Error($"Exception when trying to capture a message from the queue:
}

Replay a particular type of event from eventstore

I am currently using the Event Store to handle my events. I currently need to replay a particular type of event as I have made changes in the way they are subscribed and written to DB.
Is this possible? If so, how can it be done? Thanks.
You cannot tell EventStore to replay a specific event onto a persistent subscription because the point of the persistent subscription is to keep state for the subscribers.
To achieve this kind of fix you would really need a catch up application to do the work.
And really if you think about, if you replayed ALL the events to a new database then you would have the correct data in there?
So I have a console application that reuses the same logic as the persistent connection but the only difference is:
I change the target database connection string - So this would be a new Database or Collection (not the broken one)
It connects to EventStore and replays all the events from the start
It rebuilds the entire database to the correct state
Switch the business over to the new database
This is the point of EventStore - You just replay all the events to build any database at any time and it will be correct
Your persistent connections deal with new, incoming events and apply updates.
If you enable $by_event_type projection than you can access that projection stream under
/streams/$et-{event-type}
https://eventstore.org/docs/projections/system-projections/index.html
Then you can read it using .net api if you wish.
Here is some code to get you started
private static T GetInstanceOfEvent<T>(ResolvedEvent resolvedEvent) where T : BaseEvent
{
var metadataString = Encoding.UTF8.GetString(resolvedEvent.Event.Metadata);
var eventClrTypeName = JObject.Parse(metadataString).Property(EventClrTypeHeader).Value;
var #event = JsonConvert.DeserializeObject(Encoding.UTF8.GetString(resolvedEvent.Event.Data), Type.GetType((string) eventClrTypeName));
if (!(#event is BaseEvent))
{
throw new MessageDeserializationException((string) eventClrTypeName, metadataString);
}
return #event as T;
}
private static IEventStoreConnection GetEventStoreConnection()
{
var connectionString = System.Configuration.ConfigurationManager.ConnectionStrings["EventStore"].ConnectionString;
var connection = EventStoreConnection.Create(connectionString);
connection.ConnectAsync().Wait();
return connection;
}
private static string GetStreamName<T>() where T : BaseEvent
{
return "$et-" + typeof(T).Name;
}
And to read events you can use this code snippet
StreamEventsSlice currentSlice;
long nextSliceStart = StreamPosition.Start;
const int sliceCount = 200;
do
{
currentSlice = await esConnection.ReadStreamEventsForwardAsync(streamName, nextSliceStart, sliceCount, true);
foreach (var #event in currentSlice.Events)
{
var myEvent = GetInstanceOfEvent<OrderMerchantFeesCalculatedEvent>(#event);
TransformEvent(myEvent);
}
nextSliceStart = currentSlice.NextEventNumber;
} while (currentSlice.IsEndOfStream == false);

Multiple Async tasks with timeout and results

I'm a new in the async world. Regarding your static method (from https://stackoverflow.com/a/25733275/1596974):
static async Task<TResult[]> WhenAll<TResult>(IEnumerable<Task<TResult>> tasks, TimeSpan timeout)
{
var timeoutTask = Task.Delay(timeout).ContinueWith(_ => default(TResult));
var completedTasks =
(await Task.WhenAll(tasks.Select(task => Task.WhenAny(task, timeoutTask)))).
Where(task => task != timeoutTask);
return await Task.WhenAll(completedTasks);
}
How should I use it in order to retrieve the results of those tasks?
Just to be clear, what I need to achieve here is basically this:
For each task I'm calling to several shipping providers in order to get the different shipping rates from them.
Aggregate the response from all the shipping providers into a big list of shipping rates.
Sometimes one (or more) of the shipping providers can be down. So, I need to retrieve the shipping rates from the tasks that completed successfully and just skip the ones that failed.
I hope I was clear enough.
Ok, I ended up with this code that worked pretty well:
var providers = GetShippingProviders().ToList();
var tasks = new Task<Task>[providers.Count];
var timeout = TimeSpan.FromMilliseconds(10000);
try
{
var shippingRates = new List<IShippingRate>();
for (var i = 0; i < tasks.Length; i++)
{
var provider = providers[i];
tasks[i] = Task.WhenAny(Task.Run(() => provider.GetShippingRates(origin, destination, weight)), Task.Delay(timeout));
}
Task.WaitAll(tasks);
foreach (var tasksResult in tasks.Select(x => x.Result).Where(x => x.Status == TaskStatus.RanToCompletion))
{
var shippingRatesResult = tasksResult as Task<List<IShippingRate>>;
if (shippingRatesResult != null)
shippingRates.AddRange(shippingRatesResult.Result.ToList());
}
}
catch (AggregateException ae)
{
Log.Error("An exception occurred when retrieving the shipping Rates.", ae.Flatten());
}
All the tasks that completed successfully are processed. The ones that failed can be just skipped. There is a way to add some code using the "Task.ContinueWith(...)" so the faulted tasks can be catched for logging the exceptions as well.
For the ones that fail, do they throw an exception? Or do they just hang for a long time?
I might do something like this:
var shippingProviderRateTasks = ...;
var results = ...;
foreach (var task in shippingProviderRateTasks) {
try {
results.Add(await task);
} catch (Exception e) {
// log the error here, if you want, and skip this provider
}
}
One of the primary reasons to use Task.WhenAll is to catch an exception when it occurs, and not later on. If you want to swallow all the exceptions and essentially ignore those errors, you might as well just await each of them one at a time - it shouldn't be any slower.

email read from EWS (exchange web service) server cannot process this request error c#

I have a task where i need to check the emails delivered to my mailbox and read them ,based on subject i have to do some task. But for demo purpose I have put only the basic functionality of just updating the email read status
The basic connection and creating service object everything is fine:
///////////
NetworkCredential credentials = new NetworkCredential(securelyStoredEmail, securelyStoredPassword);
ExchangeService _service = new ExchangeService(ExchangeVersion.Exchange2010_SP2);
_service.Credentials = credentials;
_service.AutodiscoverUrl("User1#contoso.com");
/////////////////////////
Here Everything works fine. However I will invoke the below method for every 60s using observable event of reactive linq. THis is to go and poll the my emailbox and read 100 emails for every 60 seconds.
Everything works fine till sometime. Sometimes when the control reaches the line of code inside parallel.foreach loop, it shows error message like 'server cannot process this request now. Please try later' something like this. THis error comes exactly at the line
var email = EmailMessage.Bind(_service, findItemsResult.Id, emailProps);
so for every 60 seconds, i will get this error sometimes.sometimes it works fine.
Below is the method which is executed for every 60seconds. Its like i try to read the emails from "myaccount.com" for every 60s and i vil get the error 'server cannot process'.
internal void GetEmailsFrommymailbox()
{
try
{
var view = new ItemView(100);
var userMailbox = new Mailbox(userMailbox);
var folderId = new FolderId(WellKnownFolderName.Inbox, userMailbox);
SearchFilter sf = new SearchFilter.SearchFilterCollection(LogicalOperator.And,
new SearchFilter.IsEqualTo(EmailMessageSchema.IsRead, false));
var findResults = _service.FindItems(folderId, sf, view);
var emailProps = new PropertySet(ItemSchema.MimeContent, ItemSchema.Body,
ItemSchema.InternetMessageHeaders);
Parallel.ForEach(findResults, findItemsResult =>
{
///////////// this is the line where i get error////////
var email = EmailMessage.Bind(_service, findItemsResult.Id, emailProps);
//// above is the place where i get error
var emailMatching = email;
try
{
email.IsRead = true;
email.Update(ConflictResolutionMode.AutoResolve);
}
catch (Exception emailreadFromBccException)
{
Logger.Warn(emailreadFromBccException + " Unable to update email read status");
}
});
}
}
Your getting that error because you being throttled https://msdn.microsoft.com/en-us/library/office/jj945066%28v=exchg.150%29.aspx and your being throttled because you code isn't very efficient.
Instead of doing
Parallel.ForEach(findResults, findItemsResult =>
{
///////////// this is the line where i get error////////
var email = EmailMessage.Bind(_service, findItemsResult.Id, emailProps);
You should use LoadPropertiesFromItems http://blogs.msdn.com/b/exchangedev/archive/2010/03/16/loading-properties-for-multiple-items-with-one-call-to-exchange-web-services.aspx . Which will reduce the number of call you need to make to the server.
I would also suggest you use Streaming notification https://msdn.microsoft.com/en-us/library/office/hh312849%28v=exchg.140%29.aspx?f=255&MSPPError=-2147217396 which will mean you won't need to poll the server every 60 seconds and just take an action when a new item arrives.
Cheers
Glen

Resources