I am worried about a race condition situation in my code.
I have this function that fetches information from the database:
public async Task ExecuteBroadcasts()
{
List<Broadcast> broadcasts =_iSMSFetcherUnitOfWork.GetAllBroadCastsInPastTimeThatDidntRun();
foreach (Broadcast broadcast in broadcasts)
{
foreach (SenderPhone sender in broadcast.SenderPhoneList.SenderPhones)
{
foreach (ReceiverPhone receiverPhone in broadcast.RecipeintPhoneList.ReceiverPhones)
{
var tasks= broadcast.RecipeintPhoneList.ReceiverPhones.Select(receiverPhone => _iSMSProcessorAndSender.ProcessSMSes(new PreProcessedSMS(sender.Phone, receiverPhone, broadcast.SMSTemplate))).ToArray();
await Task.WhenAll(tasks);
}
}
}
}
This type 'PreProcessedSMS' is a record. The ProcessSMSes is a function that sends the information fetched from the database to be sent through an SMS provider. Here is the content of this function.
public async Task ProcessSMSes(PreProcessedSMS preProcessedSMS)
{
_iSMSExpressionInterpreter.ReceiverPhone = preProcessedSMS.receiverPhoneObj;
object sync = new object();
string smsBody = string.Empty;
lock (sync) smsBody = preProcessedSMS.sMSTemplate;
Task longRunning = Task.Factory.StartNew(() =>
{
smsBody = _iSMSExpressionInterpreter.TranslateSpinner(preProcessedSMS.sMSTemplate);
});
await longRunning.ContinueWith(async (sms) =>
{
smsBody = _iSMSExpressionInterpreter.TranslateFirstName(smsBody);
smsBody = _iSMSExpressionInterpreter.TranslateLastName(smsBody);
smsBody = _iSMSExpressionInterpreter.TranslateCustom(smsBody);
smsBody = _iSMSExpressionInterpreter.TranslateUnsubscribe(smsBody);
await _iSMSDispatcher.SendSMS(new SMSRecord(preProcessedSMS.senderPhone, preProcessedSMS.receiverPhoneObj.Phone, smsBody));
}, TaskContinuationOptions.NotOnFaulted);
longRunning.Start();
}
I locked the 'smsBody' parameter from fear that there will be a next function execute that will set a different value for it. I am wary about the implementation. Whats the best way to execute the code efficiently while not encountering a possible race condition.
update:
public async Task ProcessSMSes(PreProcessedSMS preProcessedSMS)
{
await Task.Run(async () =>
{
_iSMSExpressionInterpreter.ReceiverPhone = preProcessedSMS.receiverPhoneObj;
string smsBody = _iSMSExpressionInterpreter.TranslateSpinner(preProcessedSMS.sMSTemplate);
smsBody = _iSMSExpressionInterpreter.TranslateFirstName(smsBody);
smsBody = _iSMSExpressionInterpreter.TranslateLastName(smsBody);
smsBody = _iSMSExpressionInterpreter.TranslateCustom(smsBody);
smsBody = _iSMSExpressionInterpreter.TranslateUnsubscribe(smsBody);
await _iSMSDispatcher.SendSMS(new SMSRecord(preProcessedSMS.senderPhone, preProcessedSMS.receiverPhoneObj.Phone, smsBody));
});
}
locks only work if everything else also uses the same lock. In this case, you're locking on a local variable, which will do nothing. You can't use a lock to protect against any other code changing anything. In this case, you could use lock in the implementation of sMSTemplate so that every time it's read/written, it uses a lock, but that seems very odd. I recommend using locks judiciously, where multithreaded access is expected, and not "from fear".
Side notes:
Don't use Task.Factory.StartNew; use Task.Run if you want to run code on a thread pool thread.
Don't use ContinueWith; use await instead.
Related
I have two entities bound as one-to-one via foreignkey: CreateTenantDto and SaasTenantCreateDto.
I need to use TWO repositories (_abpTenantRepository is an instance of 3rd party repository from ABP Framework) to insert those entities into DB. I am trying to use ABP UnitOfWork implementation for this. After SaasTenantCreateDto entity is inserted, I am trying to insert CreateTenantDto entry which depends on it. If I use OnCompleted event to insert a CreateTenantDto record - the method does not enter OnCompleted before returning newTenantDto and the latter is returned as a null (the records are inserted finally, but I want to return the inserted entity if it's inserted successfully). If I don't use OnCompleted at all - the method hangs (looks like DB lock). If I use two nested UnitOfWork objects - the method hangs as well. If I use the scope for working with two repositories -
using (var scope = ServiceProvider.CreateScope())
{
var unitOfWorkManager = scope.ServiceProvider.GetRequiredService<IUnitOfWorkManager>();
using (var tenantUow = unitOfWorkManager.Begin(new AbpUnitOfWorkOptions { IsTransactional = true }))
{ ... }
}
it hangs also... It is definitely the lock and it has to do with accessing the id from the newly created newAbpTenant: I can see that in SQL Developer Sessions
enq: TX - row lock contention
and guilty session is another my HttpApi host session. Probably, the reason is as Oracle doc says: "INSERT, UPDATE, and DELETE statements on the child table do not acquire any locks on the parent table, although INSERT and UPDATE statements wait for a row-lock on the index of the parent table to clear." - SaveChangesAsync causes new record row lock?
How to resolve this issue?
//OnModelCreatingBinding
builder.Entity<Tenant>()
.HasOne(x => x.AbpTenant)
.WithOne()
.HasPrincipalKey<Volo.Saas.Tenant>(x => x.Id)
.HasForeignKey<Tenant>(x => x.AbpId);
...
b.Property(x => x.AbpId).HasColumnName("C_ABP_TENANT").IsRequired();
//Mapping ignoration to avoid problems with 'bound' entities, since using separate repositories for Insert / Update
CreateMap<CreateTenantDto, Tenant>().ForMember(x => x.AbpTenant, opt => opt.Ignore());
CreateMap<UpdateTenantDto, Tenant>().ForMember(x => x.AbpTenant, opt => opt.Ignore());
public class CreateTenantDto
{
[Required]
public int Id { get; set; }
...
public Guid? AbpId { get; set; }
public SaasTenantCreateDto AbpTenant { get; set; }
}
public async Task<TenantDto> CreateAsync(CreateTenantDto input)
{
try
{
TenantDto newTenantDto = null;
using (var uow = _unitOfWorkManager.Begin(new AbpUnitOfWorkOptions { IsTransactional = true, IsolationLevel = System.Data.IsolationLevel.Serializable }))
{
var abpTenant = await _abpTenantManager.CreateAsync(input.AbpTenant.Name, input.AbpTenant.EditionId);
input.AbpTenant.MapExtraPropertiesTo(abpTenant);
var newAbpTenant = await _abpTenantRepository.InsertAsync(abpTenant);
await uow.SaveChangesAsync();
var tenant = ObjectMapper.Map<CreateTenantDto, Tenant>(input);
tenant.AbpId = newAbpTenant.Id;
var newTenant = await _tenantRepository.InsertAsync(tenant);
newTenantDto = ObjectMapper.Map<Tenant, TenantDto>(newTenant);
await uow.CompleteAsync();
}
return newTenantDto;
}
//Implementation by ABP Framework
public virtual async Task CompleteAsync(CancellationToken cancellationToken = default)
{
if (_isRolledback)
{
return;
}
PreventMultipleComplete();
try
{
_isCompleting = true;
await SaveChangesAsync(cancellationToken);
await CommitTransactionsAsync();
IsCompleted = true;
await OnCompletedAsync();
}
catch (Exception ex)
{
_exception = ex;
throw;
}
}
I have finally resolved the problem using the following approach (but it is not using TWO repositories which seems to be impossible to implement, since we need to manipulate DbContext directly):
Application service layer:
//requiresNew: true - to be able to use TransactionScope
//isTransactional: false, otherwise it won't be possible to use TransactionScope, since we would have active ambient transaction
using var uow = _unitOfWorkManager.Begin(requiresNew: true);
var abpTenant = await _abpTenantManager.CreateAsync(input.AbpTenant.Name, input.AbpTenant.EditionId);
input.AbpTenant.MapExtraPropertiesTo(abpTenant);
var tenant = ObjectMapper.Map<CreateTenantDto, Tenant>(input);
var newTenant = await _tenantRepository.InsertAsync(tenant, abpTenant);
await uow.CompleteAsync();
return ObjectMapper.Map<Tenant, TenantDto>(newTenant);
Handmade InsertAsync method on Repository (EntityFrameworkCore) layer:
using (new TransactionScope(asyncFlowOption: TransactionScopeAsyncFlowOption.Enabled))
{
var newAbpTenant = DbContext.AbpTenants.Add(abpTenant).Entity;
tenant.AbpId = newAbpTenant.Id;
var newTenant = DbContext.Tenants.Add(tenant).Entity;
if (autoSave)
{
await DbContext.SaveChangesAsync(GetCancellationToken(cancellationToken));
}
return newTenant;
}
I use a derived store in the code below. It feels like a strange construct because I only use the derived construct for the dynamic $session dependency and to get the normData. But not with $norm. I use $norm only once to kick off the derived store.
Nevertheless it seem to work fine. But I have to renew the subscription if the $session changes. Is it possible to update the RxFire / RxJs subscription without unsubscribing first?
let normDocRef = null;
let normData = null;
let normSubscription = null;
const norm = derived(
session,
$session => {
normDocRef = db.doc(`uploads/${$session.a_id}_${$session.year}`);
// renew the subscription if $session changes
if (normSubscription)
normSubscription.unsubscribe();
normSubscription = doc(normDocRef).subscribe(snapshot => {
if (snapshot.exists) {
normData = snapshot.data();
} else {
normData = null;
};
});
},
);
$norm; // kick off the derived store to monitor $session
// show the data and updates
$: console.log(normData);
onDestroy(() => {
if (normSubscription) normSubscription.unsubscribe();
});
Update: I can use the set and return options of the derived store to change $norm in a real $norm Svelte store. Code below in my own answer.
But the real question is: Can I update a subscription. Change the subscription without the unsubscribe?
I already had the answer, but did not realize it.
Below the derived store code with the set() and return() options.
When the session changes the return() will unsubscribe automatically.
So still an unsubscribe and not an update ... but this feels good. Nice!
let normDocRef = null;
let normSubscription = null
const norm = derived(
session,
($session, set) => {
normDocRef = db.doc(`uploads/${$session.a_id}_${$session.year}`);
normSubscription = doc(normDocRef).subscribe(snapshot => {
if (snapshot.exists) {
set(snapshot.data());
} else {
set({}); // clear
};
});
return () => {
normSubscription.unsubscribe();
};
}, {} // initial value
);
$: console.log('$norm', $norm); // Now it is a real store
onDestroy(() => {
if (!normSubscription.closed) {
normSubscription.unsubscribe();
}
});
API docs derived store:
Derives a store from one or more other stores. Whenever those dependencies change (like the $session), the callback runs.
If you "return a function" from the callback, it will be called (before the callback) when a) the callback runs again (because the dependency changed), or b) ...
Ok, roughly get what you trying to describe over here.
You can actually use the reactive declaration to execute code when a variable / store changed.
In this case is to execute the resubscribe method:
let normDocRef = null;
let normData = null;
let normSubscription = null;
$: {
normDocRef = db.doc(`uploads/${$session.a_id}_${$session.year}`);
// renew the subscription if $session changes
if (normSubscription) {
normSubscription.unsubscribe();
normSubscription = doc(normDocRef).subscribe(snapshot => {
if (snapshot.exists) {
normData = snapshot.data();
} else {
normData = null;
};
});
}
}
onDestroy(() => {
if (normSubscription) normSubscription.unsubscribe();
});
The key here, is that when compiling this, Svelte knows that the block is depending on $session, so it will re-execute the code block whenever $session changed.
Should you want to refactor it out into another function, you need to make sure that Svelte knows that function depends on $session, ie:
$: resubscribe_norm($session);
Here, Svelte can tell that, if $session changed, need to call resubscribe_norm again.
Hi i have a global service for several applications and i wish to make a method with several subscribes in order to stock and initialize all my datas and i wish make a subscribe to this method in my appComponent but i don't know how to make that
In my service
private initData(isLogged: boolean) {
this.http.get('/api/conf').subscribe(
conf => {
this.http.get('api/param').subscribe(
tokResp => {
this.appParams.token = tkResp.queoval;
this.appParams.culture = tkResp.culture;
this.appParams.GMT = tkResp.gmt;
this.http.get('/api/trad').subscribe(
trad => {
this.label = trad
// return an Observable
}
)
}
)
}
)
}
In my AppComponent
this.service.initData().subscribe(
result => {
this.test = result
}
How can i make that? I can't find the information in the documentation. Thank you for your help. It's important for my work, i used so much time to research for nothing :(
So since you want to make multiple async requests one after the other, you should use the observable function ".flatMap" (this is very similar to a Promises ".then"). The ".flatMap" function allows you to wait until the first request is completed before you continue.
So in your case, you would want your service to look something like this:
private initData(isLogged: boolean) {
return this.http.get('/api/conf').flatMap(
conf => {
return this.http.get('api/param');
}
).flatMap(
tokResp => {
this.appParams.token = tkResp.queoval;
this.appParams.culture = tkResp.culture;
this.appParams.GMT = tkResp.gmt;
return this.http.get('/api/trad');
}
).flatMap(
trad => {
this.label = trad;
return trad;
}
);
}
This function has all of the async requests chained together through ".flatMap" so that they are only called after the previous request completes.
The component file looks fine and should work with this new service.
As a general note, you should never subscribe to the observable inside
of the service. You should instead use functions like map, flatMap,
forkJoin ...
I have a data stream, with rapidly incoming data. I want to insert them into a database by keeping order. I have a database, which returns a promise, which is resolved when an insert is successful.
I would like to make an Rx stream, which buffers the new data, until the buffered data is inserted.
How can I do that?
I believe to get exactly what you desire you would need to create your own operator. Breaking from RxJS slightly you can get something like (warning, have not tested)...
export class BusyBuffer<T> {
private itemQueue = new Subject<T>();
private bufferTrigger = new Subject<{}>();
private busy = false;
constructor(consumerCallback: (items: T[]) => Promise<void>) {
this.itemQueue.buffer(this.bufferTrigger).subscribe(items => {
this.busy = true;
consumerCallback(items).then(() => {
this.busy = false;
this.bufferTrigger.next(null);
});
});
}
submitItem(item: T) {
this.itemQueue.next(item);
if(!busy) {
this.bufferTrigger.next(null);
}
}
}
Which can then be used as
let busyBuffer = new BusyBuffer<T>(items => {
return database.insertRecords(items);
});
items.subscribe(item => busyBuffer.submitItem(item));
It isn't exactly purely reactive though and someone may be able to come up with something better.
Theory: If a web crawler crawls my entire site, my default caching mechanism (Redis for example) will be swamped, and may age out the wrong data. (depending on the cache policy).
Assuming that web crawlers don't need the performance gains I'm offering to end users, I could edit my app to "protect the cache"
Question
Is this a good idea?
Do web crawlers measure the time difference between delivered content?
Aside from user agent, should I "tag" a session that references robots.txt and assume they are a crawler?
How should I administratively, or programmatically handle this delivery?
In an extreme example, can I throttle a web crawler?
If I implement this programmatically, I need to tell GetFromCacheAsync to not update the cache based on some client information.
Is it a violation of any domain driven design theory to add a method overload to determine if the cache should be updated?
Where should I place the logic of "to update Redis" or "don't update Redis" ... this aspect I think is most relevant to DDD
HomeController.cs
public async Task<ActionResult> Events()
{
ViewBag.Events = await eventSvc.GetLiveEvents(DateTime.Now);
return View();
}
Services.EventManagementService.cs
public async Task<List<Event>> GetLiveEvents(DateTime currentDate)
{
//return ctx.Events.Where(e => e.StatusId == (int)EventStatus.Live && e.EventDate >= DateTime.Now).ToList();
return await cloudCtx.GetLiveEvents(DateTime.Now);
}
Data.CloudContext.cs
public async Task<List<Event>> GetLiveEvents(DateTime currentDate)
{
string year = currentDate.Year.ToString();
var key = GenerateLiveEventsKey(year);
var yearEvents = await cache.GetFromCacheAsync<List<Event>>(key, async () =>
{
List<Event> events = new List<Event>();
string partitionKey = year;
TableQuery<EventRead> query = new TableQuery<EventRead>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, partitionKey));
TableQuerySegment<EventRead> currentSegment = null;
var result = tableEvents.ExecuteQuery(query);
while (currentSegment == null || currentSegment.ContinuationToken != null)
{
currentSegment = await tableMyEvents.ExecuteQuerySegmentedAsync(query, currentSegment != null ? currentSegment.ContinuationToken : null);
foreach (EventRead nosqlEvent in currentSegment.Results)
{
var eventObj = nosqlEvent.ToEvent(true);
events.Add(eventObj);
}
}
return events;
});
return yearEvents.Where(e => e.EventDate >= currentDate).ToList();
}
Data.Cache.cs
public async Task<T> GetFromCacheAsync<T>(string key, Func<Task<T>> missedCacheCall, TimeSpan timeToLive)
{
if (!IsCacheAvailable)
{
var ret = await missedCacheCall();
return ret;
}
IDatabase cache = Connection.GetDatabase();
var obj = await cache.GetAsync<T>(key);
if (obj == null)
{
obj = await missedCacheCall();
if (obj != null)
{
cache.Set(key, obj);
}
}
return obj;
}
Is this a good idea?
Don't fix it if it is not broken (or if there is no real issue in the first place).
Do web crawlers measure the time difference between delivered content?
Google does. Its bots will slow down the crawling if necessary.
Aside from user agent, should I "tag" a session that references
robots.txt and assume they are a crawler?
User agent is good enough.
How should I administratively, or programmatically handle this delivery?
Return a 503 to tell bots they are coming too often.
In an extreme example, can I throttle a web crawler?
Yes, see above.