I need to implement multithreading background job for import file.
I have implemented it with background job(Hangfire). But if i use one thread it goes very slow.
The function look like this.
I using non-transaction unit to save changes to db immediately.
var contactFound = await _contactRepository.FirstOrDefaultAsync(x => x.Email.ToLower() == contact.Email.ToLower());
if (contactFound != null)
{
await _bjInfoManager.AddLog(args.JobId, "Found duplicated email: " + contact.Email);
}
else
{
contact.ContactListId = args.ContactListId;
contact.Email = contact.Email.ToLower();
await _contactRepository.InsertAsync(contact);
//Save changes in db
await CurrentUnitOfWork.SaveChangesAsync();
}
The problem occur when I tries to use this with Producer-Consumer Dataflow Pattern. I throws the exception "A second operation started on this context before a previous asynchronous operation completed."
The question is how to create isolated DbContext inside this method.
Please help me.
Transactions should not be multi-threaded. If you create a new task/thread in a UOW, you can create a seperated UOW using IUnitOfWork.Begin(TransactionScopeOption.RequiresNew) in a using block.
See the links
https://github.com/aspnetboilerplate/aspnetboilerplate/issues/619
Does Entity Framework support Multi-Threading?
Entity Framework and Multi threading
If you are using Microsoft SQL Server, then I recommend you to use bulk insert. It's super fast than entity framework.
https://learn.microsoft.com/en-us/sql/t-sql/statements/bulk-insert-transact-sql
Related
After reading pretty much every question on Stack Overflow and Microsoft's documentation about NetworkStream, I dont understand what is wrong with my code.
The problem I see is that my method GetDataAsync() hangs very often. I call this method from Init Method like so:
public MyView(string id)
{
InitializeComponent();
MyViewModel myViewModel = session.Resolve<MyViewModel>(); //Autofac
myiewModel.Init(id);
BindingContext = myViewModel;
}
Above, my View does its initialization, then resolves MyViewModel from Autofac DiC and then calls MyViewModel Init() method to do some additional setup on the VM.
The Init method then calls my Async method GetDataAsync which return a IList like so:
public void Init()
{
// call this Async method to populate a ListView
foreach (var model in GetDataAsync("111").Result)
{
// The List<MyModel> returned by the GetDataAsync is then
// used to load ListView's ObservableCollection<MyModel>
// This ObservableCollection is data-bound to a ListView in
// this View. So, the ListView shows its data once the View
// displays.
}
}
, and here is my GetDataAsync() method including my comments:
public override async Task<IList<MyModel>> GetDataAsync(string id)
{
var timeout = TimeSpan.FromSeconds(20);
try
{
byte[] messageBytes = GetMessageBytes(Id);
using (var cts = new CancellationTokenSource(timeout))
using (TcpClient client = new TcpClient(Ip, Port))
using (NetworkStream stream = client.GetStream())
{
await stream.WriteAsync(messageBytes, 0, messageBytes.Length, cts.Token);
await stream.FlushAsync(cts.Token);
byte[] buffer = new byte[1024];
StringBuilder builder = new StringBuilder();
int bytesRead = 0;
await Task.Delay(500);
while (stream.DataAvailable) // need to Delay to wait for data to be available
{
bytesRead = await stream.ReadAsync(buffer, 0, buffer.Length, cts.Token);
builder.AppendFormat("{0}", Encoding.ASCII.GetString(buffer, 0, bytesRead));
}
string msg = buffer.ToString();
}
return ParseMessageIntoList(msg); // parses message into IList<MyModel>
}
catch (OperationCanceledException oce)
{
return await Task.FromResult<IList<RoomGuestModel>>(new List<RoomGuestModel>());
}
catch (Exception ex)
{
return await Task.FromResult<IList<RoomGuestModel>>(new List<RoomGuestModel>());
}
}
I would expect that a ReadAsync or WriteAsync either complete successfully, throw some exception, or get cancelled after 10 seconds in which case I would catch OperationCanceledException.
However, it just hangs endlessly when I call method above. If I am debugging and have some breakpoints in the code above, I will be able to go through the method entirely but if I call it 2nd time, app just hangs forever.
I am new to Tasks and Async programming, so I am also not sure I do my cancellations and exception handling properly here?
UPDATE AND FIX
I figured out how to fix the deadlock issue. In hope this will help others sho might run into the same issue, I'll first explain it. The articles that helped me a lot are:
https://devblogs.microsoft.com/pfxteam/await-and-ui-and-deadlocks-oh-my/ by Stephen Taub
https://montemagno.com/c-sharp-developers-stop-calling-dot-result/ by James Montemagno
https://msdn.microsoft.com/en-us/magazine/jj991977.aspx by StephenCleary
https://blog.xamarin.com/getting-started-with-async-await/ by Jon Goldberger
#StephenCleary was great help understanding the issue. Calling Result or Wait (above, I call Result when calling GetDataAsync) will lead to dead-lock.
The context thread (UI in this case) is now waiting for GetDataAsync to complete, but GetDataAsync captures the current context-thread (UI thread), so it can resume on it once it gets data from TCP. But since this context-thread is now blocked by call to Result, it cannot resume.
The end result is that it looks like call to GetDataAsync has deadlocked but in reality, it is call to Result that deadlocked.
After reading tons of articles from #StephenTaub, #StephenCleary, #JamesMontemagno, #JoeGoldenberger (thank you all), I started getting understanding of the issue (I am new to TAP/async/await).
Then I discovered continuations in Tasks and how to use them to resolve the issue (thanks to Stephen Taub's article above).
So, instead of calling it like:
IList<MyModel> models = GetDataAsync("111").Result;
foeach(var model in models)
{
MyModelsObservableCollection.Add(model);
}
, I call it with continuation like this:
GetDataAsync(id)
.ContinueWith((antecedant) =>
{
foreach(var model in antecedant.Result)
{
MyModelsObservableCollection.Add(model);
}
}, TaskContinuationOptions.OnlyOnRanToCompletion)
.ContinueWith((antecedant) =>
{
var error = antecedant.Exception.Flatten();
}, TaskContinuationOptions.OnlyOnFaulted);
This seam to have fixed my deadlocking issue and now my list will load fine even though it is loaded from the constructor.
So, this seam to work just fine. But #JoeGoldenberger also suggests another solution in his article https://blog.xamarin.com/getting-started-with-async-await/ which is to use Task.Run(async()=>{...}); and inside that await GetDataAsync and load ObservableCollection. So, I gave that a try as well and that is not blocking either, so working great:
Task.Run(async() =>
{
IList<MyModel> models = await GetDataAsync(id);
foreach (var model in models)
{
MyModelsObservableCollection.Add(model);
}
});
So, it looks like either of these 2 will remove deadlock just fine. And since above my Init method is called from a c-tor; therefore, I cannot make it Async and await on this, using one of the 2 methods described above resolves my problem. I dont know which one is better but in my tests, they do work.
Your problem is most likely due to GetDataAsync("111").Result. You shouldn't block on async code.
This can cause deadocks. E.g., if you're on a UI thread, the UI thread will start GetDataAsync and run it until it hits an await. At this point, GetDataAsync returns an incomplete task, and the .Result call blocks the UI thread until that task is completed.
Eventually, the inner async call completes and GetDataAsync is ready to resume executing after its await. By default, await captures its context and resumes on that context. Which in this example is the UI thread. Which is blocked since it called Result. So, the UI thread is waiting for GetDataAsync to complete, and GetDataAsync is waiting for the UI thread so it can complete: deadlock.
The proper solution is to go async all the way; replace .Result with await, and make the necessary changes to other code for that to happen.
As stated in my update, going async all the way by providing an async lambda like below resolved the issue for me
Task.Run(async() =>
{
IList<MyModel> models = await GetDataAsync(id);
foreach (var model in models)
{
MyModelsObservableCollection.Add(model);
}
});
Loading asynchronously an observable collection in a ctor this way (in my case, ctor calls Init which then uses this Task.Run) solves problem
Imagine there is a method in appservice which upgrade data and finally a report is made in Pdf form through Ssrs. All methods are unit of work, so data is not committed in server until calling method uow.CompleteAsync(). therefore GenerateSampleReport() can not be called. To solve the problem we have used [unit of work (Isdisabled = true) so the method is written so:
[UnitOfWork(IsDisabled =true)]
public async Task Method()
{
using (var uow = UnitOfWorkManager.Begin())
{
//some data manipulation
await uow.CompleteAsync();
}
var pdfFile GenerateSampleReport();
}
But, while providing the report, if there is any exception, report can not be generated and data manipulation must be roll backed(problem about data integrity). How to solve the problem?
you shouldn't begin your own Unit Of Work. And don't disable UnitOfWork. Normally if Method() is an application service method then this should work as you want.
public async Task Method()
{
//some data manipulation...
var pdfFile GenerateSampleReport();//if any exception occurs in this line, the db transaction will be rolled back.
}
have you checked this before?
I am some need help understanding the latest recommended approach to wire up and use reactiveui for a WPF project.
In doing research on the internet on reactiveui I came across various (few) posts spanning a long time period during which the library evolved with the unfortunate result that some of these how-to articles now refer to older ways of doing things which are no longer applicable
I am trying to understand the recommended way to wire up commands (usually to invoke web service which returns a DTO) and I’ve found multiple ways mentioned to do it.
My current understanding is that
// this is the first thing to do
MyCommand = ReactiveCommand.Create()
// variations to wire up the delegates / tasks to be invoked
MyCommand.CreateAsyncTask()
MyCommand.CreateAsyncFunc()
MyCommand.CreateAsyncAction()
// this seems to be only way to wire handler for receiving result
MyCommand.Subscribe
// not sure if these below are obsolete?
MyCommand.ExecuteAsync
MyCommand.RegisterAsyncTask()
Could someone try to explain which of these variations is the latest API and which are obsolete, with perhaps a few words about when to use each of them
The changes on the ReactiveCommand API are documented in this blog post:
http://log.paulbetts.org/whats-new-in-reactiveui-6-reactivecommandt/
The first option - ReactiveCommand.Create() - just creates a reactive command.
To define a command which asynchronously returns data from a service you would use :
MyCommand = ReactiveCommand.CreateAsyncTask(
canExec, // optional
async _ => await api.LoadSomeData(...));
You may use the Subscribe method to handle data when it is received:
this.Data = new ReactiveList<SomeDTO>();
MyCommand.Subscribe(items =>
{
this.Data.Clear();
foreach (var item in items)
this.Data.Add(item);
}
Though, the simplest thing is to use instead the ToProperty method like this:
this._data = MyCommand
.Select(items => new ReactiveList<SomeDTO>(items))
.ToProperty(this, x => x.Data);
where you have defined an output property for Data:
private readonly ObservableAsPropertyHelper<ReactiveList<SomeDTO>> _data;
public ReactiveList<SomeDTO> Data
{
get { return _data.Value; }
}
Say I have a DelegatingHandler that I am using to log api requests. I want to access the request and perhaps response content in order to save to a db.
I can access directly using:
var requestBody = request.Content.ReadAsStringAsync().Result;
which I see in a lot of examples. It is also suggested to use it here by somebody that appears to know what they are talking about. Incidentally, the suggestion was made because the poster was originally using ContinueWith but was getting intermittent issues.
In other places, here, the author explicitly says not to do this as it can cause deadlocks and recommends using ContinueWith instead. This information comes directly from the ASP.net team apparently.
So I am a little confused. The 2 scenarios looks very similar in my eyes so appear to be conflicting.
Which should I be using?
You should use await.
One of the problems with Result is that it can cause deadlocks, as I describe on my blog.
The problem with ConfigureAwait is that by default it will execute the continuation on the thread pool, outside of the HTTP request context.
You can get working solutions with either of these approaches (though as Youssef points out, Result will still have sub-optimal performance), but why bother? await does it all for you: no deadlocks, optimal threading, and resuming within the HTTP request context.
var requestBody = await request.Content.ReadAsStringAsync();
Edit for .NET 4.0: First, I strongly recommend upgrading to .NET 4.5. The ASP.NET runtime was enhanced in .NET 4.5 to properly handle Task-based async operations. So the code below may or may not work if you install WebAPI into a .NET 4.0 project.
That said, if you want to try properly using the old-school ContinueWith, something like this should work:
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
var context = TaskScheduler.FromCurrentSynchronizationContext();
var tcs = new TaskCompletionSource<HttpResponseMessage>();
HttpResponseMessage ret;
try
{
... // logic before you need the context
}
catch (Exception ex)
{
tcs.TrySetException(ex);
return tcs.Task;
}
request.Content.ReadAsStringAsync().ContinueWith(t =>
{
if (t.Exception != null)
{
tcs.TrySetException(t.Exception.InnerException);
return;
}
var content = t.Result;
try
{
... // logic after you have the context
}
catch (Exception ex)
{
tcs.TrySetException(ex);
}
tcs.TrySetResult(ret);
}, context);
return tcs.Task;
}
And now it becomes clear why await is so much better...
Calling .Result synchronously blocks the thread until the task completes. This is not a good thing because the thread is just spinning waiting for the async operation to complete.
You should prefer using ContinueWith or even better, async and await if you're using .NET 4.5. Here's a good resource for you to learn more about that:
http://msdn.microsoft.com/en-us/library/vstudio/hh191443.aspx
Search several blogs about it but always are same examples.
I dunno if misunderstood or'm not knowing use but see no parallel process when work with AsyncController.
The following tests performed
Create a new project of type Asp.net MVC
HomeController.cs
public void IndexAsync()
{
AsyncManager.OutstandingOperations.Increment();
var bg = new BackgroundWorker();
bg.DoWork += (o, e) => GetEntriesBlog();
bg.RunWorkerCompleted += (o, e) =>
{
AsyncManager.Parameters["items"] = e.Result;
AsyncManager.OutstandingOperations.Decrement();
};
bg.RunWorkerAsync();
ViewBag.Message = "Modify this template to kick-start your ASP.NET MVC application.";
}
public ActionResult IndexCompleted(IEnumerable<SyndicationItem> items)
{
return View(items);
}
[NonAction]
public IEnumerable<SyndicationItem> GetEntriesBlog(int page = 0)
{
using (var reader = XmlReader.Create("http://blog.bindsolution.com/rss"))
{
Thread.Sleep(20000);
var rssData = SyndicationFeed.Load(reader);
if (rssData != null)
{
return (from item in rssData.Items
orderby item.PublishDate descending
select item).Take(3).Skip(3 * page).ToList();
}
return null;
}
}
Always delay 20 seconds browsing the site!
I was thinking of using PartialView AsyncController in to perform this task. Work?
I think you are misunderstanding what the Asynchronous Background worker would do.
If the operation takes 20 seconds using a background worker will not reduce that time or make the view render any faster. Using an asynchronous operations will free up the worker process on the server to process other requests while this long running request keep chugging along.
In your case I think you should create a very simple view that returns quickly to the user and kick of the long running operation as an asynch request from the client. For example, render the fast portions of your page (e.g. header, menus, et cetera) and make an AJAX request for the blog entries.
Depending on the nature of the code in GetEntriesBlog you might not need to make the controller operation asynchronous. In theory, since most of the time in this method will be spend waiting for the HTTP GET request to http://blog.bindsolution.com/rss to complete, it might be a good idea but in practice those things need to be bench marked (and perhaps under heavy load) to make sure you are getting the benefit that you expect. Keep in mind that your server code side will be more complex (and harder to maintain) if you make it asynch. I would suggest you go this route only if you do get a significant benefit.