Where clause using Expression tree builder - linq

I have to fetch data from database and filter data using the linq where clause.
My filter is an integer column and it contains value more than 1000.
What i am doing in the code is, breaking this huge array into chunk of 1000's of each and putting it in the where clause of a base query
int j = 0;
int batchsize = 1000;
while ((j * batchsize) < items.Count())
{
List<long> batch = items.Skip(j * batchsize)
.Take(batchsize).ToList();
prequery = prequery.Where(x => batch.Contains(x.Id));
j++;
}
the query which is getting generated in sql is below,
SELECT
x.name,
x.email
FROM
table x
WHERE
x.Id IN (1,2,3,...,1000) AND
x.Id IN (1001,1002,1003....,2000)
i want the query to be generated as below,
SELECT
x.name,
x.email
FROM
table x
WHERE
x.Id IN (1,2,3,...,1000) OR
x.Id IN (1001,1002,1003....,2000)
can i achieve this using expression tree builder and generate the query dynamically, if so please help in doing

you could use the "Concat" API:
int j = 0;
int batchsize = 1000;
IQueryable<YourType> finalQuery = null;
while ((j * batchsize) < items.Count())
{
List<long> batch = items.Skip(j * batchsize)
.Take(batchsize).ToList();
if (finalQuery == null) {
finalQuery = prequery.Where(x => batch.Contains(x.Id));
}
else
finalQuery = finalQuery.Concat (prequery.Where(x => batch.Contains(x.Id)));
j++;
}
This will logically get you what you want: basically you want an "OR" operation between batches. Concat is translated into an "UNION ALL" database call.
BUT ... I don't understand why are you doing this, after all you are grabbing all your data, chunks are not helping you because in the end there will be only one statement executed.

Related

Spring JPA - Page<Company> - append Page contents

Inside a for loop I have my 'Page companies' object which stores the result of method called 'findByNameOrGroupIdOrTerritoryIdAndCompanyIdsInobject' 'n' number of times. If it was a List we could go with companies.addAll() method so that it would append to the same object.
Is there any way to append to companies object instead of assigning every time which only stores last iteration result?
Here is the code snippet:
Page<Company> companies = null;
int parametersLimit = 500;
int companyBatches = companyIds.size() / parametersLimit;
for (int companyBatchIndex = 0; companyBatchIndex <= companyBatches; companyBatchIndex++) {
int lowerIndex = companyBatchIndex * parametersLimit;
int upperIndex = Math.min((companyBatchIndex + 1) * parametersLimit, companyIds.size());
companies = companyRepository.findByNameOrGroupIdOrTerritoryIdAndCompanyIdsIn(nameOrGroupId, territoryId, companyIds.subList(lowerIndex, upperIndex), pageable);
}
Thank you!
Simply use a list you should not use page like this
Simply put, there is no such way to append to Page contents. So I modified the logic to return List of Companies and then using PageImpl I converted the List to Page.
Page<Company> companies = null;
List<Company> companiesList = new ArrayList<Company>();;
int parametersLimit = 500;
int companyBatches = companyIds.size() / parametersLimit;
for (int companyBatchIndex = 0; companyBatchIndex <= companyBatches; companyBatchIndex++) {
int lowerIndex = companyBatchIndex * parametersLimit;
int upperIndex = Math.min((companyBatchIndex + 1) * parametersLimit, companyIds.size());
companiesList.addAll(companyRepository.findByNameOrGroupIdOrTerritoryIdAndCompanyIdsIn(nameOrGroupId, territoryId, companyIds.subList(lowerIndex, upperIndex), pageable));
}
int fromIndex = pageable.getPageNumber() * pageable.getPageSize();
int toIndex = Math.min( (int) (pageable.getOffset() + pageable.getPageSize()), companiesList.size());
companies = new PageImpl<>(companiesList.subList(fromIndex, toIndex), pageable, companiesList.size());

Linq to entity with dynamically growing where clause

I am trying to fetch data from an entity based on some criteria. The WHERE clause will have any number of 'OR' conditions. How can I acheive this?
var skill = skillset.split(";"); //array can have any number of items
var EmployeeList1 = EmployeeList
.Where(x => x.Primary1.ToUpper().Contains(skill[0])
|| x.Primary2.ToUpper().Contains(skill[0])
|| x.Primary1.ToUpper().Contains(skill[1])
|| x.Primary2.ToUpper().Contains(skill[1]))
.ToList();
return EmpMapper.Mapper.Map<List<EmployeeModel>>(EmployeeList1);
I need to build an expression for where clause with 'n' number of 'OR' conditions. I tried using predicate, but got exception(Index was outside the bounds of the array.).
var predicate = PredicateBuilder.New<ResourceEntity>(true);
for (int i = 0; i < skill.Count(); i++)
{
predicate= predicate.Or(p => p.Primary1.Contains(skill[i]));
}
You have to copy for variable into local variable:
var predicate = PredicateBuilder.New<ResourceEntity>(true);
for (int i = 0; i < skill.Length; i++)
{
var s = skill[i];
predicate = predicate.Or(p => p.Primary1.ToUpper().Contains(s));
predicate = predicate.Or(p => p.Primary2.ToUpper().Contains(s));
}

EF Core performance issue when using dynamic query using Expressions in .Net

Requirement : Select list of records from table for members whose ID and First name matches with records.
The number of members in the request may vary as per the request:
I have written following code to generate dynamic query using expressions.
private Expression<Func<TmpOncMemberAuth, bool>> CreateQueryExpression(List<MemberRequest> memberRequests)
{
var type = typeof(TmpOncMemberAuth);
// member =>
var memberParam = Expression.Parameter(type, "member");
var requests = memberRequests;
Expression selectLeft = null;
Expression filterExpression = null;
foreach (var member in requests)
{
var comparison = BuildSubQuery(member, memberParam, type);
if (selectLeft == null)
{
selectLeft = comparison;
filterExpression = selectLeft;
continue;
}
filterExpression =
Expression.Or(filterExpression, comparison);
}
return Expression.Lambda<Func<TmpOncMemberAuth, bool>>
(filterExpression, memberParam);
}
private Expression BuildSubQuery(MemberRequest member, ParameterExpression memberParam, Type type)
{
// LHS
// member.MemberID
Expression leftMemberID = Expression.Property(memberParam, type.GetProperty("MemberId"));
Expression leftMemberFirstName = Expression.Property(memberParam, type.GetProperty("MemberFirstName"));
// RHS
Expression requestMemberID = Expression.Constant(member.MemberID);
Expression requestFirstName = Expression.Constant(member.FirstName);
// member.MemberID == requestMemberID
Expression compareID =
Expression.Equal(leftMemberID, requestMemberID);
Expression compareFName =
Expression.Equal(leftMemberFirstName, requestFirstName);
// condition for a member
Expression comparison = Expression.AndAlso(compareID, compareFName);
return comparison;
}
// query call in the main
var whereQuery = CreateQueryExpression(memberData);
var memberAuthData = await FindAllAsyncFromTemp(whereQuery);
This generate a SQL query in the below format, which take a lot time
SELECT [#].[CaseID] AS [CaseId], [#].[MemberID] AS [MemberId], [#].[MemberFirstName], [#].[MemberLastName], [#].[MemberDOB]
FROM [#ONCMemberAuth] AS [#]
WHERE ((CASE
WHEN (([#].[MemberID] = '12345') AND ([#].[MemberFirstName] = 'James') ) THEN CAST(1 AS bit)
ELSE CAST(0 AS bit)
END | CASE
WHEN (([#].[MemberID] = '6789') AND ([#].[MemberFirstName] = 'WILLERS') ) THEN CAST(1 AS bit)
ELSE CAST(0 AS bit)
END)
) = CAST(1 AS bit)
I need to create SQL query like below which executes faster than the above one
SELECT [#].[CaseID] AS [CaseId], [#].[MemberID] AS [MemberId], [#].[MemberFirstName], [#].[MemberLastName], [#].[MemberDOB]
FROM [#ONCMemberAuth] AS [#]
WHERE ((([#].[MemberID] = '1234') AND ([#].[MemberFirstName] = 'James')) OR (([#].[MemberID] = '56789') AND ([#].[MemberFirstName] = 'WILLERS') ) )

EF6 LINQ cast column

My entity has a field called Value. This field is a string, but it hold int and DateTime values also.
My regular query is something like
SELECT * FROM tableA
WHERE entityType=#eID and CAST(value as int/datetime/varchar)==#value
How can I make EF generate a query with a cast?
Edit: create suggestion on uservoice
Here is example:
int eId = 1;
int valueInt = 1;
DateTime valueDateTime = DateTime.Now;
string valueString = "Test";
TableA result;
switch (eId)
{
case 1: result = context.tableA.Where(x => x.entityType == eId && Convert.ToInt32(x.value) == valueInt); break;
case 2: result = context.tableA.Where(x => x.entityType == eId && Convert.ToDateTime(x.value) == valueDateTime); break;
case 3: result = context.tableA.Where(x => x.entityType == eId && x.value == valueString); break;
}
It was implemented and completed by the EF team. Check the user invoice link listed on the question

Combining Sub Queries Into 1 Query Linq

Is there a way I can rewrite the following query to make it just one query?
try
{
var fileIds = (from f in context.SignalTo
where f.SignalFileID == 2
select new { f.GFileID }).ToList();
foreach (var id in fileIds)
{
var pp = (from p in context.ProjectFiles
where p.FileID == id.GFileID && p.ProjectID == ProjectID
select p);
if (pp != null)
{
ProjectFiles projectFile =(ProjectFiles) pp;
projectFile.MStatus = Status;
projectFile.DateLastUpdated = DateTime.Now;
context.SaveChanges();
}
}
}
You can combine the two query parts of your code into one.
You would then need to loop over the result set, making your updates. You would then call context.SaveChanges to submit all changes in one batch.
I can't tell if your existing code actually runs or compiles, but you need something like this:
Get the list of file ids you're interested in:
var fileIds = from f in context.SignalTo
where f.SignalFileID == 2
select f.GFileID;
fileIds is at this point an IQueryable where I assume T is an Int. No query has been excuted yet.
Now you can do
var pps = from p in context.ProjectFiles
where fileIds.Contains(p.FileID) && p.ProjectID == ProjectID
select p;
Still no query executed.
Then iterate over the result set
foreach( var pp in pps ) // query executed now
{
pp.MStatus = Status;
pp.DateLastUpdated = DateTime.Now;
}
context.SaveChanges(); // batch of updates executed now

Resources