Entity Framework Core CompileAsyncQuery syntax to do a query returning a list? - async-await

Documentation and examples online about compiled async queries are kinda sparse, so I might as well ask for guidance here.
Let's say I have a repository pattern method like this to query all entries in a table:
public async Task<List<ProgramSchedule>> GetAllProgramsScheduledList()
{
using (var context = new MyDataContext(_dbOptions))
{
context.ChangeTracker.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking;
return await context.ProgramsScheduledLists.ToListAsync();
}
}
This works fine.
Now I want to do the same, but with an async compiled query.
One way I managed to get it to compile is with this syntax:
static readonly Func<MyDataContext, Task<List<ProgramSchedule>>> GetAllProgramsScheduledListQuery;
static ProgramsScheduledListRepository()
{
GetAllProgramsScheduledListQuery = EF.CompileAsyncQuery<MyDataContext, List<ProgramSchedule>>(t => t.ProgramsScheduledLists.ToList());
}
public async Task<List<ProgramSchedule>> GetAllProgramsScheduledList()
{
using (var context = new MyDataContext(_dbOptions))
{
context.ChangeTracker.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking;
return await GetAllProgramsScheduledListQuery(context);
}
}
But then on runtime this exception get thrown:
System.ArgumentException: Expression of type 'System.Collections.Generic.List`1[Model.Scheduling.ProgramSchedule]' cannot be used for return type 'System.Threading.Tasks.Task`1[System.Collections.Generic.List`1[Model.Scheduling.ProgramSchedule]]'
The weird part is that if I use any other operator (for example SingleOrDefault), it works fine. It only have problem returning List.
Why?

EF.CompileAsync for set of records, returns IAsyncEnumrable<T>. To get List from such query you have to enumerate IAsyncEnumrable and fill List,
private static Func<MyDataContext, IAsyncEnumerable<ProgramSchedule>> compiledQuery =
EF.CompileAsyncQuery((MyDataContext ctx) =>
ctx.ProgramsScheduledLists);
public static async Task<List<ProgramSchedule>> GetAllProgramsScheduledList(CancellationToken ct = default)
{
using (var context = new MyDataContext(_dbOptions))
{
context.ChangeTracker.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking;
var result = new List<ProgramSchedule>();
await foreach (var s in compiledQuery(context).WithCancellation(ct))
{
result.Add(s);
}
return result;
}
}

Related

Cannot insert two one-to-one entities using two EF repositories in one transaction (method hangs)

I have two entities bound as one-to-one via foreignkey: CreateTenantDto and SaasTenantCreateDto.
I need to use TWO repositories (_abpTenantRepository is an instance of 3rd party repository from ABP Framework) to insert those entities into DB. I am trying to use ABP UnitOfWork implementation for this. After SaasTenantCreateDto entity is inserted, I am trying to insert CreateTenantDto entry which depends on it. If I use OnCompleted event to insert a CreateTenantDto record - the method does not enter OnCompleted before returning newTenantDto and the latter is returned as a null (the records are inserted finally, but I want to return the inserted entity if it's inserted successfully). If I don't use OnCompleted at all - the method hangs (looks like DB lock). If I use two nested UnitOfWork objects - the method hangs as well. If I use the scope for working with two repositories -
using (var scope = ServiceProvider.CreateScope())
{
var unitOfWorkManager = scope.ServiceProvider.GetRequiredService<IUnitOfWorkManager>();
using (var tenantUow = unitOfWorkManager.Begin(new AbpUnitOfWorkOptions { IsTransactional = true }))
{ ... }
}
it hangs also... It is definitely the lock and it has to do with accessing the id from the newly created newAbpTenant: I can see that in SQL Developer Sessions
enq: TX - row lock contention
and guilty session is another my HttpApi host session. Probably, the reason is as Oracle doc says: "INSERT, UPDATE, and DELETE statements on the child table do not acquire any locks on the parent table, although INSERT and UPDATE statements wait for a row-lock on the index of the parent table to clear." - SaveChangesAsync causes new record row lock?
How to resolve this issue?
//OnModelCreatingBinding
builder.Entity<Tenant>()
.HasOne(x => x.AbpTenant)
.WithOne()
.HasPrincipalKey<Volo.Saas.Tenant>(x => x.Id)
.HasForeignKey<Tenant>(x => x.AbpId);
...
b.Property(x => x.AbpId).HasColumnName("C_ABP_TENANT").IsRequired();
//Mapping ignoration to avoid problems with 'bound' entities, since using separate repositories for Insert / Update
CreateMap<CreateTenantDto, Tenant>().ForMember(x => x.AbpTenant, opt => opt.Ignore());
CreateMap<UpdateTenantDto, Tenant>().ForMember(x => x.AbpTenant, opt => opt.Ignore());
public class CreateTenantDto
{
[Required]
public int Id { get; set; }
...
public Guid? AbpId { get; set; }
public SaasTenantCreateDto AbpTenant { get; set; }
}
public async Task<TenantDto> CreateAsync(CreateTenantDto input)
{
try
{
TenantDto newTenantDto = null;
using (var uow = _unitOfWorkManager.Begin(new AbpUnitOfWorkOptions { IsTransactional = true, IsolationLevel = System.Data.IsolationLevel.Serializable }))
{
var abpTenant = await _abpTenantManager.CreateAsync(input.AbpTenant.Name, input.AbpTenant.EditionId);
input.AbpTenant.MapExtraPropertiesTo(abpTenant);
var newAbpTenant = await _abpTenantRepository.InsertAsync(abpTenant);
await uow.SaveChangesAsync();
var tenant = ObjectMapper.Map<CreateTenantDto, Tenant>(input);
tenant.AbpId = newAbpTenant.Id;
var newTenant = await _tenantRepository.InsertAsync(tenant);
newTenantDto = ObjectMapper.Map<Tenant, TenantDto>(newTenant);
await uow.CompleteAsync();
}
return newTenantDto;
}
//Implementation by ABP Framework
public virtual async Task CompleteAsync(CancellationToken cancellationToken = default)
{
if (_isRolledback)
{
return;
}
PreventMultipleComplete();
try
{
_isCompleting = true;
await SaveChangesAsync(cancellationToken);
await CommitTransactionsAsync();
IsCompleted = true;
await OnCompletedAsync();
}
catch (Exception ex)
{
_exception = ex;
throw;
}
}
I have finally resolved the problem using the following approach (but it is not using TWO repositories which seems to be impossible to implement, since we need to manipulate DbContext directly):
Application service layer:
//requiresNew: true - to be able to use TransactionScope
//isTransactional: false, otherwise it won't be possible to use TransactionScope, since we would have active ambient transaction
using var uow = _unitOfWorkManager.Begin(requiresNew: true);
var abpTenant = await _abpTenantManager.CreateAsync(input.AbpTenant.Name, input.AbpTenant.EditionId);
input.AbpTenant.MapExtraPropertiesTo(abpTenant);
var tenant = ObjectMapper.Map<CreateTenantDto, Tenant>(input);
var newTenant = await _tenantRepository.InsertAsync(tenant, abpTenant);
await uow.CompleteAsync();
return ObjectMapper.Map<Tenant, TenantDto>(newTenant);
Handmade InsertAsync method on Repository (EntityFrameworkCore) layer:
using (new TransactionScope(asyncFlowOption: TransactionScopeAsyncFlowOption.Enabled))
{
var newAbpTenant = DbContext.AbpTenants.Add(abpTenant).Entity;
tenant.AbpId = newAbpTenant.Id;
var newTenant = DbContext.Tenants.Add(tenant).Entity;
if (autoSave)
{
await DbContext.SaveChangesAsync(GetCancellationToken(cancellationToken));
}
return newTenant;
}

ObjectDisposedException during tests

READ EDIT
I have a similar implementation to AsyncCrudAppService related to filtering queries. When I run tests on top of ABPs implementation of Application Services derived of AsyncCrudAppServiceBase, everything runs fine. When I do the same running on top of my custom "filters", I get the following error:
System.ObjectDisposedException : Cannot access a disposed object [...]
Object name: 'DataManagerDbContext'.
I know the solution is using IUnitOfWorkManager and calling Begin() method to define a UnitOfWork, but since I am working with AppServices, I thought there was already a UnitOfWork defined. These are my methods:
public PagedResultDto<StateDetails> GetEditorList(EditorRequestDto input)
{
var query = _stateRepository.GetAllIncluding(p => p.Country).AsQueryable();
query = ApplySupervisorFilter(query);
query = query.ApplyFiltering(input, "Name");
var totalCount = query.Count();
query = query.ApplySorting<State, int, PagedAndSortedResultRequestDto>(input);
query = query.ApplyPaging<State, int, PagedAndSortedResultRequestDto>(input);
var entities = query.ToList();
return new PagedResultDto<StateDetails>(totalCount, ObjectMapper.Map<List<StateDetails>>(entities));
}
private IQueryable<State> ApplySupervisorFilter(IQueryable<State> query)
{
if (!SettingManager.GetSettingValue<bool>(AppSettingNames.SupervisorFlag))
{
query = ApplyUncategorizedFilter(query);
}
return query;
}
private IQueryable<State> ApplyUncategorizedFilter(IQueryable<State> query)
{
return query.Where(
p => !p.CountryId.HasValue);
}
My passing test (with manual UnitOfWork):
[Fact]
public async Task GetEditorListWithouSupervisorFlag_Test()
{
using (UnitOfWorkManager.Begin())
{
await ChangeSupervisorFlag(false);
var result = _stateAppService.GetEditorList(
new EditorRequestDto
{
MaxResultCount = 10,
});
result.Items.Any(p => p.Country == null).ShouldBe(true);
}
}
Does anybody know an solution to this "issue"? It would be annoying to define a UnitOfWork for every test I perform. It also seems like I am doing something wrong
EDIT
I have solved the issue. I must use an interface for my Application Service when running tests so it is able to mock it properly
I have solved the issue. I must use an interface for my Application Service when running tests so it is able to mock it properly

The best way to modify a WebAPI OData QueryOptions.Filter

I am using the OData sample project at http://www.asp.net/web-api/overview/odata-support-in-aspnet-web-api/working-with-entity-relations. In the Get I want to be able to change the Filter in the QueryOptions of the EntitySetController:
public class ProductsController : EntitySetController<Product, int>
{
ProductsContext _context = new ProductsContext();
[Queryable(AllowedQueryOptions=AllowedQueryOptions.All)]
public override IQueryable<Product> Get()
{
var products = QueryOptions.ApplyTo(_context.Products).Cast<Product>();
return products.AsQueryable();
}
I would like to be able to find properties that are specifically referred to. I can do this by parsing this.QueryOptions.Filter.RawValue for the property names but I cannot update the RawValue as it is read only. I can however create another instance of FilterQueryOption from the modified RawValue but I cannot assign it to this.QueryOptions.Filter as this is read only too.
I guess I could call the new filter's ApplyTo passing it _context.Products, but then I will need to separately call the ApplyTo of the other properties of QueryOptions like Skip and OrderBy. Is there a better solution than this?
Update
I tried the following:
public override IQueryable<Product> Get()
{
IQueryable<Product> encryptedProducts = _context.Products;
var filter = QueryOptions.Filter;
if (filter != null && filter.RawValue.Contains("Name"))
{
var settings = new ODataQuerySettings();
var originalFilter = filter.RawValue;
var newFilter = ParseAndEncyptValue(originalFilter);
filter = new FilterQueryOption(newFilter, QueryOptions.Context);
encryptedProducts = filter.ApplyTo(encryptedProducts, settings).Cast<Product>();
if (QueryOptions.OrderBy != null)
{
QueryOptions.OrderBy.ApplyTo<Product>(encryptedProducts);
}
}
else
{
encryptedProducts = QueryOptions.ApplyTo(encryptedProducts).Cast<Product>();
}
var unencryptedProducts = encryptedProducts.Decrypt().ToList();
return unencryptedProducts.AsQueryable();
}
and it seems to be working up to a point. If I set a breakpoint I can see my products in the unencryptedProducts list, but when the method returns I don't get any items. I tried putting the [Queryable(AllowedQueryOptions=AllowedQueryOptions.All)] back on again but it had no effect. Any ideas why I am not getting an items?
Update 2
I discovered that my query was being applied twice even though I am not using the Queryable attribute. This meant that even though I had items to return the List was being queried with the unencrypted value and therefore no values were being returned.
I tried using an ODataController instead:
public class ODriversController : ODataController
{
//[Authorize()]
//[Queryable(AllowedQueryOptions = AllowedQueryOptions.All)]
public IQueryable<Products> Get(ODataQueryOptions options)
{
and this worked! Does this indicate that there is a bug in EntitySetController?
You would probably need to regenerate ODataQueryOptions to solve your issue. Let's say if you want to modify to add $orderby, you can do this like:
string url = HttpContext.Current.Request.Url.AbsoluteUri;
url += "&$orderby=name";
var request = new HttpRequestMessage(HttpMethod.Get, url);
ODataModelBuilder modelBuilder = new ODataConventionModelBuilder();
modelBuilder.EntitySet<Product>("Product");
var options = new ODataQueryOptions<Product>(new ODataQueryContext(modelBuilder.GetEdmModel(), typeof(Product)), request);

PrepareResponse().AsActionResult() throws unsupported exception DotNetOpenAuth CTP

Currently I'm developing an OAuth2 authorization server using DotNetOpenAuth CTP version. My authorization server is in asp.net MVC3, and it's based on the sample provided by the library. Everything works fine until the app reaches the point where the user authorizes the consumer client.
There's an action inside my OAuth controller which takes care of the authorization process, and is very similar to the equivalent action in the sample:
[Authorize, HttpPost, ValidateAntiForgeryToken]
public ActionResult AuthorizeResponse(bool isApproved)
{
var pendingRequest = this.authorizationServer.ReadAuthorizationRequest();
if (pendingRequest == null)
{
throw new HttpException((int)HttpStatusCode.BadRequest, "Missing authorization request.");
}
IDirectedProtocolMessage response;
if (isApproved)
{
var client = MvcApplication.DataContext.Clients.First(c => c.ClientIdentifier == pendingRequest.ClientIdentifier);
client.ClientAuthorizations.Add(
new ClientAuthorization
{
Scope = OAuthUtilities.JoinScopes(pendingRequest.Scope),
User = MvcApplication.LoggedInUser,
CreatedOn = DateTime.UtcNow,
});
MvcApplication.DataContext.SaveChanges();
response = this.authorizationServer.PrepareApproveAuthorizationRequest(pendingRequest, User.Identity.Name);
}
else
{
response = this.authorizationServer.PrepareRejectAuthorizationRequest(pendingRequest);
}
return this.authorizationServer.Channel.PrepareResponse(response).AsActionResult();
}
Everytime the program reaches this line:
this.authorizationServer.Channel.PrepareResponse(response).AsActionResult();
The system throws an exception which I have researched with no success. The exception is the following:
Only parameterless constructors and initializers are supported in LINQ to Entities.
The stack trace: http://pastebin.com/TibCax2t
The only thing I've done differently from the sample is that I used entity framework's code first approach, an I think the sample was done using a designer which autogenerated the entities.
Thank you in advance.
If you started from the example, the problem Andrew is talking about stays in DatabaseKeyNonceStore.cs. The exception is raised by one on these two methods:
public CryptoKey GetKey(string bucket, string handle) {
// It is critical that this lookup be case-sensitive, which can only be configured at the database.
var matches = from key in MvcApplication.DataContext.SymmetricCryptoKeys
where key.Bucket == bucket && key.Handle == handle
select new CryptoKey(key.Secret, key.ExpiresUtc.AsUtc());
return matches.FirstOrDefault();
}
public IEnumerable<KeyValuePair<string, CryptoKey>> GetKeys(string bucket) {
return from key in MvcApplication.DataContext.SymmetricCryptoKeys
where key.Bucket == bucket
orderby key.ExpiresUtc descending
select new KeyValuePair<string, CryptoKey>(key.Handle, new CryptoKey(key.Secret, key.ExpiresUtc.AsUtc()));
}
I've resolved moving initializations outside of the query:
public CryptoKey GetKey(string bucket, string handle) {
// It is critical that this lookup be case-sensitive, which can only be configured at the database.
var matches = from key in db.SymmetricCryptoKeys
where key.Bucket == bucket && key.Handle == handle
select key;
var match = matches.FirstOrDefault();
CryptoKey ck = new CryptoKey(match.Secret, match.ExpiresUtc.AsUtc());
return ck;
}
public IEnumerable<KeyValuePair<string, CryptoKey>> GetKeys(string bucket) {
var matches = from key in db.SymmetricCryptoKeys
where key.Bucket == bucket
orderby key.ExpiresUtc descending
select key;
List<KeyValuePair<string, CryptoKey>> en = new List<KeyValuePair<string, CryptoKey>>();
foreach (var key in matches)
en.Add(new KeyValuePair<string, CryptoKey>(key.Handle, new CryptoKey(key.Secret, key.ExpiresUtc.AsUtc())));
return en.AsEnumerable<KeyValuePair<string,CryptoKey>>();
}
I'm not sure that this is the best way, but it works!
It looks like your ICryptoKeyStore implementation may be attempting to store CryptoKey directly, but it's not a class that is compatible with the Entity framework (due to not have a public default constructor). Instead, define your own entity class for storing the data in CryptoKey and your ICryptoKeyStore is responsible to transition between the two data types for persistence and retrieval.

compiling a linq to sql query

I have queries that are built like this:
public static List<MyObjectModel> GetData (int MyParam)
{
using (DataModel MyModelDC = new DataModel())
{ var MyQuery = from....
select MyObjectModel { ...}
}
return new List<MyObjectModel> (MyQuery)
}
}
It seems that using compiled linq-to-sql queries are about as fast as stored procedures and so the goal is to convert these queries into compiled queries. What's the syntax for this?
Thanks.
Put something like this inside of your DataContext (or in your case your "DataModel"):
private static Func<DataModel, int, MyObjectModel> _getObjectModelById =
CompiledQuery.Compile<DataModel, int, MyObjectModel>(
(dataModel, myParam) =>
dataModel.PersonDtos.where(c => c.ObjectModelId == myParam).FirstOrDefault()
);
Then add amethod in there to call it like this:
internal List<MyObjectModel> GetObjectModel(int myParam)
{
var results = _getObjectModelById(this, myParam);
return results.SingleOrDefault();
}
Inside of your repository where your original method was call the internal function to get the result you are looking for.
Hope this helps -> I can post more code if necessary :)

Resources