I wanted to formulate 1) query dynamically as shown in 2)
1)
Select col1, Sum(col2)
from t1
where col3 = 1
and co11 between '01 jan 2009' and '31 dec 2013'
group by co11
2)
And I would like to formulate above as following in 2012 (linq to entity)
" select "+ #selectclause + " from t1 where " + whereclause
somebody suggested me, that the above query can be formulated using http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx
Can any body formulate it using the System.linq.Dynamic or System.Linq.Expression? I am not able to do it due to lack of time.
I was trying some thing like following using System.Linq.dynamic
var var1 = context.table1.Where(strWhereClause)
.Where(i => i.co11 >= dtDateFrom && i.co11 <= dtDateTo).GroupBy("new(" + varGroupBy + ")").Select("new(" + varAggregateCol + ")");
Please also tell me if my bunch of queries can be forumlated by System.Linq.Expression
then should I use System.Linq.Dynamic or shouldI give priority to System.Linq.Expression.
yours sincerley
Here is an example of what this would look like:
var query = t1.Where("col3 = 1 AND col1 >= #0 AND col1 <= #1", dtDateFrom, dtDateTo)
.GroupBy("col1")
.Select(varAggregateCol)
While the standard Dynamic Linq library simply returns a IQueryable that is hard to use, I've been maintaining an updated library (https://www.nuget.org/packages/System.Linq.Dynamic.Library) that returns IQueryable{dynamic} (assuming you are using .NET 4.0+), allowing you to easily access the resulting members dynamically.
Related
Hi I am using a Jqwidgets Grid to display my data. It has a build in possibility to use filters but if you filter your records on the server side you have to build your own query. As I am working with Linq I thought to use the Dynamic Linq Library for Asp net core. Problem is there are not many examples or explanations how to do this. But I am busy for days now and not getting very far.The way I am setup; I have a normal Linq query:
var Mut = from M in _DB.Mutations
join S in _DB.Shifts on M.ShiftId equals S.ShiftId
join U in _DB.RoosterUsers on M.UserId equals U.RoosterUserId
join D in deps on M.UserId equals D.UserId
join DD in _DB.Departements on D.DepartementID equals DD.DepartementId
select new MutationModel
{
MutId=M.MutationId,
Naam=U.FirstName + " " + U.LastName,
UserId=M.UserId,
Departement= DD.DepartementName,
MutationType = S.publicName,
MutationGroup = S.ShiftType.ToString(),
DateTot =M.DateTill,
TijdVan=M.DateStartOn,
TijdTot=M.DateTill,
Status=CreateStatus(M.Tentative, M.ApprovedOn, M.Processed, M.CancelRefId, M.Deleted)
};
This query is running OK and gives me all the data I need for the Grid.
Then for the filter I would like to add a dynamic Linq Query using the System.Linq.Dynamic.Core library
But this is as far as I get things working until now:
var outQuery = Mut.Where("Status = #0 and UserId = #1", "Nieuw", "KLM22940").Select("Status");
My questions now :
1. In the where clause If I make the fieldname variable I get an error. how to do this??
2. In the Select Clause, how to add multiple Columns? (actually I just like to output all columns.)
Best would be to see an example. has somebody used Dynamic Linq to build a dynamic linq query for the JQWidgets Grid?
Thank you very much.
In what way you are trying to use fieldname variable in where clause ?
If you want to output all columns you can use ToList()
like
var outQuery = Mut.Where("Status = #0 and UserId = #1", "Nieuw", "KLM22940").ToList();
If you want to get some specific columns you can use Select clause like this
var outQuery = Mut.Where("Status = #0 and UserId = #1", "Nieuw", "KLM22940").Select("new(Status,UserId )");
This Select clause creates data class which contains Status and UserId properties and returns a sequence of instances of that data class.
I have a table which contains ~600k records and 33 columns. In my project I am using EF Core (2.0.1) to retrieve data from database. I am having issues with below code:
var theCounter = (from f in _context.tblData.Take(100000)
group f by f.TypeId into data
select new DataDto { ID = data.Key, Count = data.Count() }).ToList();
This code is a part of REST API and when I am testing it from SOAP UI, I am gettin timeout error. When I tested the code for
Take(1000)
There are around 300 unique TypeIds.
it works fine. Any ideas how I can make it work?
-- EDIT 1:
Here is what I see when debugging the code:
Microsoft.EntityFrameworkCore.Query:Warning: Query: '(from TblData <generated>_1 in DbSet<TblData> select [<generated>_1]).Take(__p_0)' uses a row limiting operation (Skip/Take) without OrderBy which may lead to unpredictable results.
Microsoft.EntityFrameworkCore.Query:Warning: Query: '(from TblData <generated>_1 in DbSet<TblData> select [<generated>_1]).Take(__p_0)' uses a row limiting operation (Skip/Take) without OrderBy which may lead to unpredictable results.
Microsoft.EntityFrameworkCore.Query:Warning: The LINQ expression 'GroupBy([f].TypeId, [f])' could not be translated and will be evaluated locally.
Microsoft.EntityFrameworkCore.Query:Warning: The LINQ expression 'GroupBy([f].TypeId, [f])' could not be translated and will be evaluated locally.
Microsoft.EntityFrameworkCore.Query:Warning: The LINQ expression 'Count()' could not be translated and will be evaluated locally.
Microsoft.EntityFrameworkCore.Database.Command:Information: Executed DbCommand (131ms) [Parameters=[#__p_0='?'], CommandType='Text', CommandTimeout='30']
SELECT [t2].[Id], [t2].[at], [t2].[add], [t2].[AddDate], [t2].[aftc], [t2].[aftcd], [t2].[aid], [t2].[afl], [t2].[prdid], [t2].[cid], [t2].[TypeId], [t2].[env], [t2].[ext], [t2].[extddcode], [t2].[fn], [t2].[fn], [t2].[fic], [t2].[gid], [t2].[grp], [t2].[hnm], [t2].[IP], [t2].[icid], [t2].[ln], [t2].[lg], [t2].[pcid], [t2].[ret], [t2].[rts], [t2].[rnam], [t2].[sled], [t2].[seq], [t2].[sid], [t2].[styp]
FROM (
SELECT TOP(#__p_0) [t1].[Id], [t1].[at], [t1].[add], [t1].[AddDate], [t1].[aftc], [t1].[aftcd], [t1].[aid], [t1].[afl], [t1].[prdid], [t1].[cid], [t1].[TypeId], [t1].[env], [t1].[ext], [t1].[extddcode], [t1].[fn], [t1].[fn], [t1].[fic], [t1].[gid], [t1].[grp], [t1].[hnm], [t1].[IP], [t1].[icid], [t1].[ln], [t1].[lg], [t1].[pcid], [t1].[ret], [t1].[rts], [t1].[rnam], [t1].[sled], [t1].[seq], [t1].[sid], [t1].[styp]
FROM [TblData] AS [t1]
) AS [t2]
WHERE [t2].[TypeId] IS NOT NULL
ORDER BY [t2].[TypeId]
I think it is not translated properly. Any ideas why?
-- EDIT 2:
I have changed my queries to:
var query = _context.TblData
.Select(a => new {ID = a.Id, TypeId= a.TypeId})
.Distinct();
var q1 = query.GroupBy(p => p.TypeId)
.Select(g => new DataDto {TypeId= g.Key, Count = g.Count()});
return await q1.ToListAsync();
But it was translated to:
SELECT DISTINCT [a0].[Id], [a0].[TypeId] AS [TypeId]
FROM [tblData] AS [a0]
ORDER BY [a0].[TypeId]
When I checked directly in the database this query takes 14 seconds to execute. Any idea why it was not translated to something like:
SELECT DISTINCT [a0].[Id], COUNT([TypeId]) AS [TypeId]
FROM [tblData] AS [a0]
GROUP BY COUNT([a0].[Id])
ORDER BY [a0].[TypeId]
I had to upgrade EF Core version to 2.1 and LINQ is now translated properly into SQL.
I am trying to compare two tables (i.e values, count, etc..) in linq to sql but I am not getting the way to achieve it. I tried the following,
Table1.Any(i => i.itemNo == Table2.itemNo)
It gives error. Could you please help me?
Thanks in Advance.
how about
var isDifferent =
Table1.Zip(Table2, (j, k) => j.itemNo == k.itemMo).Any(m => !m);
EDIT
if Linq-To-Sql does not support Zip.
var one = Table1.ToList();
var two = Table2.ToList();
var isDifferent =
one.Zip(two, (j, k) => j.itemNo == k.itemMo).Any(m => !m);
if the tables are vary large this could cause performance problems. In that case you will need a much more sophisticated solution, if so, please ask.
EDIT2
If the tables are very large you don't want to get all the data from the server and hold it memory. Additionaly, Linq and SQL server do not garauntee the order of the rows unless you specify an order in the query. This becomes espcially relavent for large result sets returned by a multi processor server where the effects of parallelism are likely to come into play.
I suggest that Linq-to-Sql doesen't really cater well for your scenario so you will have to help it out using ExecuteQuery somthing like this.
string zipQuery =
#"SELECT TOP 1
1
FROM
[Table1] [one]
WHERE
NOT EXISTS (
SELECT * FROM [Table2] [two] WHERE [two].[itemNo] = [one].[itemNo]
)
UNION ALL
SELECT
1
FROM
[Table2] [two]
WHERE
NOT EXISTS (
SELECT * FROM [Table1] [one] WHERE [one].[itemNo] = [two].[itemNo]
)
UNION ALL
SELECT 0";
var isDifferent = context.ExecuteQuery<int>(zipQuery).Single() == 1;
This will do the select on the server without returning lots of data to the client but, I think you will agree is much more complicated.
EDIT3
Okay, the zip approach should be fine for 1000 rows. I've read your comment and I suggest changing the code accordingly.
var one = Table1.ToList();
var two = Table2.ToList();
var isDifferent =
one.Count != two.Count ||
one.Zip(two, (o, t) => o.itemNo == k.itemNo).Any(m => !m);
You should probably consider putting an order by on the list retrievers, like this.
var one = Table1.OrderBy(o => o.itemNo).ToList();
Strictly, the results of a Linq-to-Sql come back in any order unless an order is specified.
I am having trouble with the following piece of code. Before I paste it, Let me give a bit of history on what should happen.
I have a model containing 2 fields of interest at the moment, which is the name of the order the customer placed, and the date at which he/she placed it. A pre-calculated date will be used to query the dateplaced field (and should only query the dates , and not the time). The query counts the amount of duplicates that occur in the MondayOrder field, and groups them together. Now , when I exclude the where clause which should query the dates, the query runs great. However, The goal of this query is to count the amount of orders for the following week based on the date the order has been placed.
List<string> returnlist = new List<string>();
DateTime dt = getNextWeekMondaysDate().Date;
switch (day)
{
case DayOfWeek.Monday:
{
var CountOrders =
from x in Data.EntityDB.Orders
group x by x.MondayOrder into m
let count = m.Count()
select new
{
MondayOrderItem = m.Key, Amount = count
};
foreach (var item in CountOrders)
{
returnlist.Add(item.MondayOrderItem + " : " +
item.Amount);
}
}
break;
The getNextWeekMondaysDate() method has an overload which I can use, where if I supply it a date, it will get the following Monday's date from the parameter given. The problem is though, LINQ does not accept queries such as the following:
var CountOrders =
from x in Data.EntityDB.Orders
where getNextWeekMondaysDate(x.DatePlaced.Value).Date == dt
group x by x.MondayOrder into m
let count = m.Count()
select new { MondayOrderItem = m.Key, Amount = count };
This is exactly what I must achieve. Is there any workaround for this situation?
UPDATE
Here is the exception I get when I try the 2nd query.
LINQ to Entities does not recognize the method 'System.DateTime getNextWeekMondaysDate(System.DateTime)' method, and this method cannot be translated into a store expression.
You cannot do this directly, as user-defined method calls cannot be translated to SQL by the EF query provider. The provider recognizes a limited set of .NET methods that can be translated to SQL and also a number of canonical functions as well. Anything that cannot be expressed using these methods only is off-limits unless you write your own query provider (which is only theoretically an option).
As a practical workaround, you can calculate an appropriate range for x.DatePlaced.Value in code before the query and then use specific DateTime values on the where clause.
As an intellectual exercise, note that this method is recognized by the query provider and can be used as part of the expression. So this abomination should work too:
var CountOrders =
from x in Data.EntityDB.Orders
where EntityFunctions.AddDays(
x.DatePlaced.Date.Value,
(9 - DateAndTime.DatePart(DateInterval.WeekDay, x.DatePlaced.Value)) % 7)
.Date == dt
group x by x.MondayOrder into m
let count = m.Count()
select new { MondayOrderItem = m.Key, Amount = count };
Linq to Entities doesn't know how to convert arbitrary C# methods into SQL - it's not possible in general.
So, you have to work with the methods it does understand.
In this case, you could do something like this:
DateTime weekBegin = CalculateWeekBegin( dt );
DateTime weekEnd = CalculateWeekEnd( dt );
var CountOrders =
from x in Data.EntityDB.Orders
where x.DatePlaced.Value >= weekBegin && x.DatePlaced.Value < weekEnd
group x by x.MondayOrder into m
let count = m.Count()
select new { MondayOrderItem = m.Key, Amount = count });
In another posting: Does Linq-To-Sql support composable queries there was discussion on how to compose/concat where clauses dynamically. This appears to be done with an "AND" (i.e. the first where clause and the second where clause are joined by an AND). What I am wondering is if there is a way to compose Linq queries with an OR.
Example:
var people = from p in Person
where p.age < 18
select p
var otherPeople = from p in people
where p.firstName equals "Daniel"
select p
This gives people with a first name of "Daniel" and that are under 18. I'm looking for the syntax to join these to find people who have a first name of "Daniel" or are under 18.
Note: I am using ADO.net Data Services so I do not have .Contains() available to me.
EDIT: The Union Suggestion (by Garry Shutler) is exactly what I am looking for functionality-wise. I did run into two possible issues with it:
It looks like it would make multiple database hits if I was to do a third condition (union seems to take an IEnumerable as its parameter) - I was hoping to build up multiple AND and OR statements in code and then execute one request.
Union is not supported by ADO.Net Data Services (very disappointing)
Is what you want as simple as:
var people = from p in Person
where p.age < 18 || p.firstName == "Daniel"
select p;
or have you just given a simple example?
In which case you can use:
var under18 = from p in Person
where p.age < 18
select p;
var daniels = from p in Person
where p.firstName == "Daniel"
select p;
var combined = under18.Union(daniels);
LinqToSql may be intelligent enough to convert that to an OR but I'm not so sure.
What about using PredicateBuilder by Joe Albahari?
var predicate = PredicateBuilder.False<Person>();
predicate = predicate.Or(p => p.age < 18);
predicate = predicate.Or(p => p.firstName == "Daniel");
var query = Person.Where(predicate);
The predicate option is the way to go. The Union option DOES NOT build good sql. Reference http://social.msdn.microsoft.com/forums/en-US/linqprojectgeneral/thread/925b245d-5529-4a64-8cd4-4bc83ee6fe7a/
I wrote about how to achieve queries which search for a key value within a set on my blog .
Here are the relevant links.
Contains Operations in ADO.NET Data Services Part I
Contains Operations in ADO.NET Data Services Part II
Using this , you can write queries which look like this
//The set in which we have to search for a match
List<string> citiesIWillVisit = new List<string>() {"London","Berlin","Prague"};
var customersAround = nwContext.Customers
.IsIn<Customers>(citiesIWillVisit, c=> c.City);
foreach (Customers localCustomer in customersAround) {
System.Console.WriteLine(localCustomer.ContactName);
}