I have a T-SQL 2005 query which returns:
pid propertyid displayname value
----------- ----------- --------------- ---------------
14270790 74 Low Price 1.3614
14270790 75 High Price 0
14270791 74 Low Price 1.3525
14270791 75 High Price 0
14270792 74 Low Price 1.353
14270792 75 High Price 0
14270793 74 Low Price 1.3625
14270793 75 High Price 0
14270794 74 Low Price 1.3524
14270794 75 High Price 0
What I would like to do is essentially pivot on the displayname field, hopefully producing:
pid Low Price High Price
14270790 1.3614 0
14270791 1.3525 0
14270792 1.353 0
14270793 1.3625 0
14270794 1.3524 0
(Not sure how the propertyid field would be output, so I left it out (was hoping it would simply sit alongside the Low Price and High Price fields, to indicate their IDs, but I don't think that will work.)
The problem is that the content of the original displayname field is dynamic - it is produced from a join with a PropertyName' table, so the number of pivoted columns is variable. It could therefore containHigh Price,Low Price,OpenandClose`, depending on what the join with that table returns.
It is, of course, relatively easy (regardless of the trouble I'm having writing the initial query!) to produce this pivot in a fixed query or stored proc. However, is it possible to get LINQ to generate a SQL query which would name each column to be produced rather than having to write a dynamic (probably in a stored proc) query which lists out the column names?
Thanks,
Matt.
I'll give you a sample with a different data (that I needed). You can adapt that to your need. Note only two linq queries are used, most of the other fluff is to convert a list into a datatable.
var data = new[] {
new{Student=1, Subject="English", Marks=40},
new{Student=1, Subject="Maths", Marks=50},
new{Student=1, Subject="Science", Marks=60},
new{Student=1, Subject="Physics", Marks=70},
new{Student=1, Subject="Chemistry", Marks=80},
new{Student=1, Subject="Biology", Marks=90},
new{Student=2, Subject="English", Marks=4},
new{Student=2, Subject="Maths", Marks=5},
new{Student=2, Subject="Science", Marks=6},
new{Student=2, Subject="Physics", Marks=7},
new{Student=2, Subject="Chemistry", Marks=8},
new{Student=2, Subject="Biology", Marks=9}
};
/*Here the pivot column is the subject and the static column is student
group the data against the static column(s)*/
var groups = from d in data
group d by d.Student into grp
select new
{
StudentId = grp.Key,
Marks = grp.Select(d2 => new { d2.Subject, d2.Marks }).ToArray()
};
/*get all possible subjects into a separate group*/
var subjects = (from d in data
select d.Subject).Distinct();
DataTable dt = new DataTable();
/*for static cols*/
dt.Columns.Add("STUDENT_ID");
/*for dynamic cols*/
foreach (var subject in subjects)
{
dt.Columns.Add(subject.ToString());
}
/*pivot the data into a new datatable*/
foreach (var g in groups)
{
DataRow dr = dt.NewRow();
dr["STUDENT_ID"] = g.StudentId;
foreach (var mark in g.Marks)
{
dr[mark.Subject] = mark.Marks;
}
dt.Rows.Add(dr);
}
This is the closest I could get, but it's not LINQ...
create table #t
(
pointid [int],
doublevalue [float],
title [nvarchar](50)
)
insert into #t
select
distinct top 100
v.pointid, v.doublevalue, p.title
from [property] p
inner join pointvalue v on p.propertyid = v.propertyid
inner join point pt on v.pointid = pt.pointid
where v.pointid in (select top 5 p.pointid from point p where p.instanceid = 36132)
declare #fields nvarchar(250)
set #fields = (select STUFF((SELECT N',[' + title + ']' FROM [property] FOR XML PATH('')), 1, 1, N''))
--select #fields
declare #sql nvarchar(500)
set #sql = 'select * from #t
pivot
(
sum(doublevalue)
for [title] in ('+#fields+')
) as alias'
--select #sql
exec (#sql)
drop table #t
The kicker is that I'm simply asking for every entry in the Property table, meaning there's a lot of columns, in the resulting pivot, which have NULL values.
the code I think is like this:
var list = from table in Property
group table by table.pid into g
select new
{
pid = g.key,
LowPrice = g.Where(w => w.pid== g.key && w.priceType == "low").Select(s => s.value).FirstorDefault(),
HighPrice = g.Where(w => w.pid== g.key && w.priceType == "high").Select(s => s.value).FirstorDefault(),
};
Hope it can help you and have a nice day.
Related
I am trying to query a table with nested linq query. My query working but is too slow. I have almost 400k row. And this query work 10 seconds for 1000 rows. For 400k I think its about to 2 hours.
I have rows like this
StudentNumber - DepartmentID
n100 - 1
n100 - 1
n105 - 1
n105 - 2
n107 - 1
I want the students which have different department ID. My results looks like this.
StudentID - List
n105 - 1 2
And my query provides it. But slowly.
var sorgu = (from yok in YOKAktarim
group yok by yok.StudentID into g
select new {
g.Key,
liste=(from birim in YOKAktarim where birim.StudentID == g.Key select new { birim.DepartmentID }).ToList().GroupBy (x => x.DepartmentID).Count()>1 ? (from birim in YOKAktarim where birim.StudentID == g.Key select new { birim.DepartmentID }).GroupBy(x => x.DepartmentID).Select(x => x.Key).ToList() : null,
}).Take(1000).ToList();
Console.WriteLine(sorgu.Where (s => s.liste != null).OrderBy (s => s.Key));
I wrote this query with linqpad C# statement.
For 400K records you should be able to return the student ids and department ids into an in-memory list.
var list1 = (from r in YOKAktarim
group r by new { r.StudentID, r.DepartmentID} into g
select g.Key
).ToList();
Once you have this list, you should be able to group by StudentID and select those students who have more than one record.
var list2 = (from r in list1 group r by r.StudentID into g
where g.Count() > 1
select new
{
StudentID = g.Key,
Departments = g.Select(a => a.DepartmentID).ToList()
}
).ToList();
This should be faster as it only hits the sql database once, rather than hundreds of thousands of times.
You're iterating your source collection (YOKAktarim) three times, which makes your query *O(n^3)` query. It's going to be slow.
Instead of going back to source collection to get content of the group you can simply iterate over g.
var sorgu = (from yok in YOKAktarim
group yok by yok.StudentID into g
select new {
g.Key,
liste = from birim in g select new { birim.DepartmentID }).ToList().GroupBy (x => x.DepartmentID).Count()>1 ? (from birim in g select new { birim.DepartmentID }).GroupBy(x => x.DepartmentID).Select(x => x.Key).ToList() : null,
}).Take(1000).ToList();
However, that's still not optimal, because you're doing a lot of redundant subgrouping. Your query is pretty much equivalent to:
from yok in YOKAktarim
group yok by yok.StudentID into g
let departments = g.Select(g => g.DepartmentID).Distinct().ToList()
where departments.Count() > 1
select new {
g.Key,
liste = departments
}).Take(1000).ToList();
I can't speak for the correctness of that monster, but simply removing all ToList() calls except the outermost one will fix your issue.
I have the following LINQ query using EF5 and generic repository, unit of work patterns to a SQL Server 2008 db
var countriesArr = GetIdsFromDelimStr(countries);
var competitionsArr = GetIdsFromDelimStr(competitions);
var filterTeamName = string.Empty;
if (teamName != null)
{
filterTeamName = teamName.ToUpper();
}
using (var unitOfWork = new FootballUnitOfWork(ConnFooty))
{
// give us our selection of teams
var teams =
(from team in
unitOfWork.TeamRepository.Find()
where ((string.IsNullOrEmpty(filterTeamName) || team.Name.ToUpper().Contains(filterTeamName)) &&
(countriesArr.Contains(team.Venue.Country.Id) || countriesArr.Count() == 0))
select new
{
tId = team.Id
}).Distinct();
// give us our selection of contests
var conts = (
from cont in
unitOfWork.ContestRepository.Find(
c =>
((c.ContestType == ContestType.League && competitionsArr.Count() == 0) ||
(competitionsArr.Contains(c.Competition.Id) && competitionsArr.Count() == 0)))
select new
{
contId = cont.Id
}
).Distinct();
// get selection of home teams based on contest
var homecomps = (from fixt in unitOfWork.FixtureDetailsRepository.Find()
where
teams.Any(t => t.tId == fixt.HomeTeam.Id) &&
conts.Any(c => c.contId == fixt.Contest.Id)
select new
{
teamId = fixt.HomeTeam.Id,
teamName = fixt.HomeTeam.Name,
countryId = fixt.HomeTeam.Venue.Country.Id != null ? fixt.HomeTeam.Venue.Country.Id : 0,
countryName = fixt.HomeTeam.Venue.Country.Id != null ? fixt.HomeTeam.Venue.Country.Name : string.Empty,
compId = fixt.Contest.Competition.Id,
compDesc = fixt.Contest.Competition.Description
}).Distinct();
// get selection of away teams based on contest
var awaycomps = (from fixt in unitOfWork.FixtureDetailsRepository.Find()
where
teams.Any(t => t.tId == fixt.AwayTeam.Id) &&
conts.Any(c => c.contId == fixt.Contest.Id)
select new
{
teamId = fixt.AwayTeam.Id,
teamName = fixt.AwayTeam.Name,
countryId = fixt.AwayTeam.Venue.Country.Id != null ? fixt.AwayTeam.Venue.Country.Id : 0,
countryName = fixt.AwayTeam.Venue.Country.Id != null ? fixt.AwayTeam.Venue.Country.Name : string.Empty,
compId = fixt.Contest.Competition.Id,
compDesc = fixt.Contest.Competition.Description
}).Distinct();
// ensure that we return the max competition based on id for home teams
var homemax = (from t in homecomps
group t by t.teamId
into grp
let maxcomp = grp.Max(g => g.compId)
from g in grp
where g.compId == maxcomp
select g).Distinct();
// ensure that we return the max competition based on id for away teams
var awaymax = (from t in awaycomps
group t by t.teamId
into grp
let maxcomp = grp.Max(g => g.compId)
from g in grp
where g.compId == maxcomp
select g).Distinct();
var filteredteams = homemax.Union(awaymax).OrderBy(t => t.teamName).AsQueryable();
As you can see we want to return the following format which is passed across to a WebAPI so we cast the results to types we can relate to in the UI.
Essentially what we are trying to do is get the home and away teams from a fixture, these fixtures have a contest which relates to a competition. We then get the highest competition id from the grouping and then this is returned with that team. The country is related to the team based on the venue id, when I was originally doing this i had problems figuring out how to do OR joins in linq which is why i split it down to getting home teams and away team and then grouping them based on competition then unioning them together.
An idea of current table size is fixtures has 7840 rows, teams has 8581 rows, contests has 337 rows and competitions has 96 rows. The table that is likely to increase rapidly is the fixture table as this is related to football.
The output we want to end up with is
Team Id, Team Name, Country Id, Country Name, Competition Id, Competition Name
Using no filtering this query takes on average around 5 secs, just wondering if anybody has any ideas/pointers on how to make it quicker.
thanks in advance Mark
I can't judge whether it will speed up things, but your homemax and awaymax queries could be
var homemax = from t in homecomps
group t by t.teamId into grp
select grp.OrderByDescending(x => x.compId).FirstOrDefault();
var awaymax = from t in awaycomps
group t by t.teamId into grp
select grp.OrderByDescending(x => x.compId).FirstOrDefault();
Further, as you are composing one very large query it may perform better when you cut it up in a few smaller queries that fetch intermediary results. Sometimes a few more roundtrips to the database perform better than one very large query for which the database engine can't find a good execution plan.
Another thing is all these Distinct()s. Do you always need them? I think you can do without because you are always fetching data from one table without joining a child collection. Removing them may save a bunch.
Yet another optimization could be to remove the ToUpper. The comparison is done by the database engine in SQL and chances are that the database has a case-insensitive collation. If so, the comparison is never case sensitive even if you'd want it to be! Constructs like Name.ToUpper cancel the use of any index on Name (it is not sargable).
How we can write a LINQ query for following select sql query:
string brandid="1,2,3"
string bodystyleid="1,2,3"
-------------------
-----------------
select * from car
where brandid in (brandid)
and bodystyleid in (brandid)
----------------------
-------------------
My specific requirement is that if brandid or bodystyleid is blank(if user does not select
any checkbox of a particular search option) query should return all record for that particular where condition.
Please guide me.
Thanks,
Paul
In order to fulfil your requirement about returning all items if none are specified, you need to check for the lists being empty.
var brands = brandid.Split(',').Select(x => Int32.Parse(x));
var styles = bodystyleid.Split(',').Select(x => Int32.Parse(x));
var result = from c in car
where (!brands.Any() || brands.Contains(c.brandid))
&& (!styles.Any() || styles.Contains(c.bodystyleid))
select c;
(similar to sgmoore's solution, but includes the check for no brand/style specified)
I've not actually checked how this gets converted back to SQL - it may be more efficient to use a flag to indicate whether there are any values:
var brands = ....; // As above
bool anyBrands = brands.Any()
var result = from c in car
where (!anyBrands || brands.Contains(c.brandid))
.....
Is bodystyleid meant to check brandid or bodystyleid? (I am assuming bodystyleid, however have wrote the query to match the query in the question (brandid))
As a start you could do:
var results = (from c in car
where c.brandid.Contains(brandid)
&& c.bodystyleid.Contains(brandid)
select c).ToList();
var brandids = brandid .Split(',').Select(n => int.Parse(n)).ToList();
var bodyStyleids = bodystyleid.Split(',').Select(n => int.Parse(n)).ToList();
var results =
(from c in car where
brandids.Contains(c.brandid) &&
bodyStyleids.Contains(c.bodystyleid)
select c
).ToList();
the Ids you have are as strings with comma delimiter, you need them to be collections like List of the same type as your Ids of the Car table, so if brandid column is int then brandids has to be List<long>, then you can do
var results = (
from c in cars
where brandids.Contains(c.brandid) && bodystyleid.Contains(c.bodystyleid)
select c).ToList();
I am using BLToolKit in a project of mine and I was trying to get this to work. What I don't like is that I am trying to average a bunch of temps down to the minute, but the select statement that is being generated groups by the minute but then selects the original time. I think I am doing the linq expression correctly (but then again, i am not getting the results i expect). (this is C#, if you care) Anyone know what is going wrong?
var test = (from r in db.SensorReadingRaws
where r.TimeLogged < DateTime.Now.AddMinutes(-2)
group r by new
{
Sensor = r.SensorNumber,
//group time down to the minute
Time = r.TimeLogged.AddSeconds(-1 * r.TimeLogged.Second).AddMilliseconds(-1 * r.TimeLogged.Millisecond)
} into grouped
select new SensorReading
{
SensorNumber = grouped.Key.Sensor,
TimeLogged = grouped.Key.Time,
Reading = (int)grouped.Average(x => x.Reading)
}).ToList();
textBox1.Text = db.LastQuery;
and the resulting query is this
SELECT
[r].[SensorNumber],
[r].[TimeLogged],
Avg([r].[Reading]) as [c1]
FROM
[SensorReadingRaw] [r]
WHERE
[r].[TimeLogged] < #p1
GROUP BY
[r].[SensorNumber],
DateAdd(Millisecond, Convert(Float, -DatePart(Millisecond, [r].[TimeLogged])), DateAdd(Second, Convert(Float, -DatePart(Second, [r].[TimeLogged])), [r].[TimeLogged])),
[r].[TimeLogged]
I discovered that
BLToolkit.Data.Linq.Sql.AsSql<T>(T obj)
can be used as a workaround for this case.
When applying this function to the required grouped key properties in select statement you get rid of grouping/selecting an original field.
It may look something like:
_queryStore.Leads().
GroupBy(x => new {
x.LeadDate.Hour,
x.LeadDate.Minute
}).
Select(x => new {
Hour = Sql.AsSql(x.Key.Hour),
Minute = Sql.AsSql(x.Key.Minute),
Count = x.Count()
});
and in your particular case:
var test = (from r in db.SensorReadingRaws
where r.TimeLogged < DateTime.Now.AddMinutes(-2)
group r by new
{
Sensor = r.SensorNumber,
//group time down to the minute
Time = r.TimeLogged.AddSeconds(-1 * r.TimeLogged.Second).AddMilliseconds(-1 * r.TimeLogged.Millisecond)
} into grouped
select new SensorReading
{
SensorNumber = grouped.Key.Sensor,
TimeLogged = Sql.AsSql(grouped.Key.Time),
Reading = (int)grouped.Average(x => x.Reading)
}).ToList();
I got same issue yesterday.
Today I found a workaround. The idea is to write 2 linq queries. First transforming the data and the second grouping the result:
var bandAndDate =
(from r in repo.Entities
select new {Band = r.Score / 33, r.StartTime.Date});
var examsByBandAndDay =
(from r in bandAndDate
group r by new {r.Band, r.Date } into g
select new { g.Key.Date, g.Key.Band, Count = g.Count() }).ToList();
Both this queries run one SQL that do the job:
SELECT
[t1].[c1] as [c11],
[t1].[c2] as [c21],
Count(*) as [c3]
FROM
(
SELECT
[r].[Score] / 33 as [c2],
Cast(Floor(Cast([r].[StartTime] as Float)) as DateTime) as [c1]
FROM
[Results] [r]
) [t1]
GROUP BY
[t1].[c2],
[t1].[c1]
I have divAssignments that has potential multiple rows by rNI, an official id, according to a compound key of Indictment and booking numbers.
rNI Booking Indictment
12345 954445 10 12345
12345 954445 10 12346
12345 954445 10 12347
So ID has a count of 3 for a single booking number for this rni.
I get lost attempting to generate a count and a group by booking Number:
var moreThen = from dA in divAssignments
select new { dA.rNI, IndictmentCount = dA.indictmentNumber.Count() };
Most of the examples are dealing with static int[] and don't seem to work in my case.
How do I get a group and then a count? If I could put in a having that would be fantastic.
from a t-sql POV I'd use this:
Select rni, bookingNumber, count(*) IndictmentCount
from divAssignments
group by rni, bookingNumber
having count(*) > 0
TIA
How about something like this:
var query = from item in divAssignments
group item by item.rNI into grouping
select new
{
Id = grouping.Key,
Count = grouping.Count()
}
If you're interested in grouping by both the rNI and the booking number, I would change it to this:
var query = from item in divAssignements
group item by new { item.rNI, a.Booking } into grouping
select new
{
Id = grouping.Key,
Count = grouping.Count
};
OR
var query = from item in divAssignments
group item by item into grouping
select new
{
Id = grouping.Key,
Count = grouping.Count()
}
and implement IEquatable on the divAssignment object to support equality comparison. The other option if you'd like is to write an IEqualityComparer instance to do the composite key comparison. Your query could then look like:
var query =
divAssignments
.GroupBy(i => i, new MyCustomEqualityComparer())
.Select(i => new { Key = i.Key, Count = i.Count());
var query =
from dA in divAssignments
group dA by new { dA.rNI, dA.bookingNumber };
foreach(var grp in query)
{
Console.WriteLine("rNI={0}, bookingNumber={1} => Count={2}", grp.Key.rNI, grp.Key.bookingNumber, grp.Count());
}
If you use a Grouping operator in Linq you will get what you need. The code:
var count = from a in divAssignments
group a by new { a.rNI, a.Booking } into b
select b;
will return a collection of IGrouping objects. This will give you the Key (in my example this will be an anonymous type with an rNI and a Booking property) and a collection of the divAssignments that match the key.
Using Method syntax (much easier to read in my opinion):
First group the records, then select a new result for each group that contains the count.
var groups = divAssignments.GroupBy(d => new { d.rNI, d.Booking });
groups.Select(g=> new { g.Key.rNI, g.Key.Booking, IndictmentCount = g.Count() });