LINQ partial crosstab query when the potential columns are not fixed - linq

I have a requirement to produce an "flattened" version of data for export, in which the customer wants the related table rows to appear as columns. We're using Entity Framework 4 & .NET 4.0
The customer has the ability to create custom questions for their events. There may be zero to infinite questions. Users attending the event may answer none, some or all of those questions.
What I need to produce is like this:
User 1 User 1 info field A User 1 info field B User 1 Answer to question A User 1 Answer to Question B
User 2 User 2 info field A User 2 info field B User 2 Answer to question A User 2 Answer to Question B
Creating the user info portion of the result set is not problem, but I'm stumped on how to take my related questions and answers, i.e.:
from urf in tblUserRegistrationFields.Include("tblRegistrationFields")
where urf.UserID == eachUserID && urf.tblRegistrationField.EventID == thisEventID
select new
{
FieldLabel = urf.tblRegistrationField.FieldLabel, //this is the question name
value = urf.value //this is the user's answer
}
And add those to my user select statement as individual columns, i.e.:
var users = from u in context.tblUsers
<snip>
select new
{
UserID = u.UserID,
<snip>,
CustomQuestionA = AnswerA,
CustomQuestionB = AnswerB,
... etc.
};
Ultimately I need to create a grid which I can then export to their desired format and I'm open to non-LINQ solutions (although the purchase of 3rd party controls is not an option.) I suspect there is a way to leverage LINQ's grouping features, but I haven't been able to apply it to this scenario.

I found a way to do dynamic pivots in SQL, but couldn't get it to work with two related tables, only with one. Dynamic SQL pivots here:
http://www.simple-talk.com/blogs/2007/09/14/pivots-with-dynamic-columns-in-sql-server-2005/
Ultimately, I fell back and created a dataset manually. It's certainly not an optimized solution, but it gives the customer what they want. I wouldn't recommend this approach for a high volume of data.
I create the dataset and datatable fixed columns, then loop through the list of questions:
//get the list of questions
var context = new EventRegistrationEntities();
var customQuestions = from rf in context.tblRegistrationFields
where rf.EventID == vEventID
select new
{
RegistrationFieldID = rf.RegistrationFieldID,
FieldLabel = rf.FieldLabel
};
//add a question column for each question
List<string> extracolumns = new List<string>();
foreach (var q in customQuestions)
{
dt.Columns.Add(q.FieldLabel, typeof(string));
//store the question names for later user
extracolumns.Add(q.RegistrationFieldID.ToString() + "-" + q.FieldLabel.ToString());
}
Next I then loop through the list of users and insert the fixed data, then within that loop (ugh) I add the users' answers:
foreach (var c in extracolumns) //these are the custom *questions*
{
int regID = Convert.ToInt32(c.Substring(0, c.IndexOf("-")));
string question = c.Substring(c.IndexOf("-") + 1); //we need to pass the question text (aka column header) in because if the user hasn't answered the question we have no access to it from here
//get the question answer
var userAnswer = (from urf in context.tblUserRegistrationFields.Include("tblRegistrationFields")
where urf.UserID == u.UserID && urf.RegistrationFieldID == regID
select new
{
Question = urf.tblRegistrationField.FieldLabel,
Answer = urf.value
}).FirstOrDefault();
//add the answer to the data row
if (userAnswer != null)
{
dr[question] = userAnswer.Answer.ToString();
}
else
{
dr[question] = ""; //if the user has not answered the question, insert a blank
}
}

Related

Can I join a table to a list using linq? [duplicate]

This question already has answers here:
EntityFramework - contains query of composite key
(12 answers)
Closed 2 years ago.
I have a table as follows:
PersonalDetails
Columns are:
Name
BankName
BranchName
AccountNo
Address
I have another list that contains 'Name' and 'AccountNo'.
I have to find all the records from table that whose respective 'Name' and 'AccountNo' are present in given list.
Any suggestion will be helpful.
I have done following but not of much use:
var duplicationhecklist = dataAccessdup.MST_FarmerProfile
.Join(lstFarmerProfiles,
t => new { t.Name,t.AccountNo},
t1 => new { t1.Name, t1.AccountNo},
(t, t1) => new { t, t1 })
.Select(x => new {
x.t1.Name,
x.t1.BankName,
x.t1.BranchName,
x.t1.AccountNo
}).ToList();
where lstFarmerProfiles is a list.
You probably found out that you can't join an Entity Framework LINQ query with a local list of entity objects, because it can't be translated into SQL. I would preselect the database data on the account numbers only and then join in memory.
var accountNumbers = lstFarmerProfiles.Select(x => x.AccountNo).ToArray();
var duplicationChecklist =
from profile in dataAccessdup.MST_FarmerProfile
.Where(p => accountNumbers
.Contains(p.AccountNo))
.AsEnumerable() // Continue in memory
join param in lstFarmerProfiles on
new { profile.Name, profile.AccountNo} equals
new { param.Name, param.AccountNo}
select profile
So you will never pull the bulk data into memory but the smallest selection you can probably get to proceed with.
If accountNumbers contains thousands of items, you may consider using a better scalable chunky Contains method.
Since you have the lists in .net of values you want to find, try to use the Contains method, for sample:
List<string> names = /* list of names */;
List<string> accounts = /* list of account */;
var result = db.PersonalDetails.Where(x => names.Contains(x.Name) && accounts.Contains(x.AccountNo))
.ToList();
If MST_FarmerProfile is not super large I think you best option is to bring it into memory using AsEnumerable() and do the joining there.
var duplicationhecklist =
(from x in dataAccessdup.MST_FarmerProfile
.Select(z => new {
z.Name,
z.BankName,
z.BranchName,
z.AccountNo
}).AsEnumerable()
join y in lstFarmerProfiles
on new { x.Name, x.AccountNo} equals new { y.Name, y.AccountNo}
select x).ToList();
Since data is usually located on different machines or in separate processes at least: DB - is one and your in-memory list is your app, there is just 2 ways to do it.
Download as small data part from DB to local as possible and join locally (usually using AsEnumerable() or basically ToList()). You got many good thoughts on this in other answers.
Another one is different - upload your local data to server somehow and perform query on DB side. Uploading can be done differently: using some temp tables OR using VALUES. Fortunately there is a small extension for EF now (for both EF6 and EF Core) which you could try. It is EntityFrameworkCore.MemoryJoin (name might be confusing, but it supports both EF6 and EF Core). As stated in author's article it modifies SQL query passed to server and injects VALUES construction with data from your local list. And query is executed on DB server.
If accountNo identifies the record then you could use:
var duplicationCheck = from farmerProfile in dataAccessdup.MST_FarmerProfile
join farmerFromList in lstFarmerProfiles
on farmerProfile.AccountNo equals farmerFromList.AccountNo
select new {
farmerProfile.Name,
farmerProfile.BankName,
farmerProfile.BranchName,
farmerProfile.AccountNo
};
If you need to join on name and account then this should work:
var duplicationCheck = from farmerProfile in dataAccessdup.MST_FarmerProfile
join farmerFromList in lstFarmerProfiles
on new
{
accountNo = farmerProfile.AccountNo,
name = farmerProfile.Name
}
equals new
{
accountNo = farmerFromList.AccountNo,
name = farmerFromList.Name
}
select new
{
farmerProfile.Name,
farmerProfile.BankName,
farmerProfile.BranchName,
farmerProfile.AccountNo
};
If you are only going to go through duplicateChecklist once then leaving .ToList() out will be better for performance.

Multiple rows update without select

An old question for Linq 2 Entities. I'm just asking it again, in case someone has came up with the solution.
I want to perform query that does this:
UPDATE dbo.Products WHERE Category = 1 SET Category = 5
And I want to do it with Entity Framework 4.3.1.
This is just an example, I have a tons of records I just want 1 column to change value, nothing else. Loading to DbContext with Where(...).Select(...), changing all elements, and then saving with SaveChanges() does not work well for me.
Should I stick with ExecuteCommand and send direct query as it is written above (of course make it reusable) or is there another nice way to do it from Linq 2 Entities / Fluent.
Thanks!
What you are describing isnt actually possible with Entity Framework. You have a few options,
You can write it as a string and execute it via EF with .ExecuteSqlCommand (on the context)
You can use something like Entity Framework Extended (however from what ive seen this doesnt have great performance)
You can update an entity without first fetching it from db like below
using (var context = new DBContext())
{
context.YourEntitySet.Attach(yourExistingEntity);
// Update fields
context.SaveChanges();
}
If you have set-based operations, then SQL is better suited than EF.
So, yes - in this case you should stick with ExecuteCommand.
I don't know if this suits you but you can try creating a stored procedure that will perform the update and then add that procedure to your model as a function import. Then you can perform the update in a single database call:
using(var dc = new YourDataContext())
{
dc.UpdateProductsCategory(1, 5);
}
where UpdateProductsCategory would be the name of the imported stored procedure.
Yes, ExecuteCommand() is definitely the way to do it without fetching all the rows' data and letting ChangeTracker sort it out. Just to provide an example:
Will result in all rows being fetched and an update performed for each row changed:
using (YourDBContext yourDB = new YourDBContext()) {
yourDB.Products.Where(p => p.Category = 1).ToList().ForEach(p => p.Category = 5);
yourDB.SaveChanges();
}
Just a single update:
using (YourDBContext yourDB = new YourDBContext()) {
var sql = "UPDATE dbo.Products WHERE Category = #oldcategory SET Category = #newcategory";
var oldcp = new SqlParameter { ParameterName = "oldcategory", DbType = DbType.Int32, Value = 1 };
var newcp = new SqlParameter { ParameterName = "newcategory", DbType = DbType.Int32, Value = 5 };
yourDB.Database.ExecuteSqlCommand(sql, oldcp, newcp);
}

Can I refer to items within a LINQ result set by index?

I'm trying to work with a LINQ result set of 4 tables retrieved with html agility pack. I'd like to process each one slightly differently by setting a variable for each (switch statement below), and then processing the rows within the table. The variable would ideally be the index for each of the tables in the set, 0 to 3, and would be used in the switch statement and to select the rows. I haven't been able to locate the index property, but I see it used in situations such as SelectChildNode.
My question is can I refer to items within a LINQ result set by index? My "ideal scenario" is the last commented out line. Thanks in advance.
var ratingsChgs = from table in htmlDoc.DocumentNode
.SelectNodes("//table[#class='calendar-table']")
.Cast<HtmlNode>()
select table;
String rtgChgType;
for (int ratingsChgTbl = 0; ratingsChgTbl < 4; ratingsChgTbl++)
{
switch (ratingsChgTbl)
{
case 0:
rtgChgType = "Upgrades";
break;
case 1:
rtgChgType = "Downgrades";
break;
case 2:
rtgChgType = "Coverage Initiated";
break;
case 3:
rtgChgType = "Coverage Reit/ Price Tgt Changed";
break;
//This is what I'd like to do.
var tblRowsByChgType = from row in ratingsChgs[ratingsChgTbl]
.SelectNodes("tr")
select row;
//Processing of returned rows.
}
}
ElementAt does what you're asking for. I don't recommend using it in your example, though, because each time you call it, your initial LINQ query will be executed. The easy fix is to have ratingsChgs be a List or Array.
You can also refactor out the switch statement. It is overkill when you only need to iterate through a list of items. Here is a possible solution:
var ratingsChgs = from table in htmlDoc.DocumentNode
.SelectNodes("//table[#class='calendar-table']")
.Cast<HtmlNode>()
select table;
var rtgChgTypeNames = new List
{
"Upgrades",
"Downgrades",
"Coverage Initiated",
"Coverage Reit/ Price Tgt Changed"
};
var changeTypes = ratingsChgs.Zip(rtgChgTypeNames, (changeType, name) => new
{
Name = name,
Rows = changeType.SelectNodes("tr")
});
foreach( var changeType in changeTypes)
{
var name = changeType.Name;
var rows = changeType.Rows;
//Processing of returned rows.
}
Also, why not store your rating change types in the HTML doc? It seems odd to have table information defined in the business logic.

LINQ to sharepoint . join list help

I am currently using SharePoint 2010,and in my business layer I am using LINQ to SharePoint. I generated all my Entity classes using SPMetal.
We are creating a Library system, my system has 2 Lists . The first one is Contribution and the second one is Contributor. Each Contributor contains a reference to the Contribution List (a PrimaryISBN reference). The Contribution List contains list of books and PrimaryISBN is not unique in this list.
Contribution
ID PrimaryISBN TITLE
1 PRIM1 HardcoverLOTR
2 PRIM1 AudioBookLOTR
3 PRIM2 HardcoverHP
Contributor
ID Name PrimaryISBNLookup
1 ABC PRIM1
2 DEF PRIM2
I am currently trying to fetch All the Books Contributed by a Particular user based on the Name.
My query is something like this
var result = from _contributor in data.contributor
where _contributor.Name= "ABC"
select new Book
{
Title = contributor.PrimaryISBNLookup.Title
}
The problem that I am currently facing is in retrieving records that have same ISBN but different title (Each format will have a title i.e. a Audio book will have a title and a Hardcover of the same book will have a different one).
This query returns me only 1 records even thought I have 2 records in my system i.e. the record with ID (in Contribution) that I am forced to insert during the insertion of record into Contributor List.
Your help is highly appreciated.
From what I understand, you try to implement a simple join, like this:
var results = from _contributor in data.contributor
join _contribution in data.contribution
on _contributor.PrimaryISBNLookup equals _contribution.PrimaryISBN
where _contributor.Name == "ABC"
select new Book
{
Title = _contribution.Title
}
If you want to use DataTable in SPList, try this:
SPList cList = spWeb.Lists.TryGetList("Customer");
SPList oList = spWeb.Lists.TryGetList("Order");
DataTable cTable= cList.Items.GetDataTable();
DataTable oTable= oList.Items.GetDataTable();
var coList = from tbl1 in cTable.AsEnumerable()
join tbl2 in oTable.AsEnumerable() on tbl1["Title"] equals tbl2["CustomerName"]
select new
{
ItemName = tbl2["Title"],
CustomerName = tbl1["Title"],
Mobile = tbl1["MobileNo"]
};

Linq query, how to build nested objects from single table

I have a single table and I need to build a bunch of nested objects based on the single table.
Data:
PointA PointB Month Time Price
1 2 11 11:00 10.99
1 2 12 11:00 9.99
Objects are
POINTS {PointA, PointB, Details}
Details {Month, ExtraDetails}
ExtraDetails {Time, Price}
I want to avoid having loads of loops and if statements, so should be able to use linq to do this. but its beyond my linq experience.
edit: These need grouping aswell
any help would be great.
Thanks
Just tried out a solution:
var nestedObjects = from row in data
select new {row.PointA, row.PointB, Details = new {
row.Month, ExtraDetails = new {
row.Time, row.Price
}
}};
This is assuming that you have already got your data into data.
Group by
If you want to group the Points together, you need 'Group By':
var nestedObjects = from row in data
group row by new { row.PointA, row.PointB } into Points
select new {
Points = Points.Key,
Details = from details in Points
select new { row.Month, ExtraDetails = new {
row.Time, row.Price
}}
};
A little more complicated - of course you might want to group by month as well, in which case, you need to follow the same pattern as for the Points bit. Note, this will not create tables, because the group by doesn't quite do that, but it at least creates the structure for you.
Assuming you got your classes defined for the objects you mentioned, and you have a constructor or properties so you can propery create the object in one line you could have a LINQ query returning a list of a POINTS.
If would go something lik this :
var res =
from item in table.AsEnumerable()
select new Points(){PointA = item["PointA"];
PointB = item["PointB"];
Details = from item2 in table.AsEnumberable()
where item["PointA"] = item2["PointA"] and item["PointB"] = item2["PointB"]
select new Details(){
month=item2["month"],
extraDetails = from item3 in table.AsEnumerable()...
}
};
At the end res will be a IEnumerable of Points
I am sorry for the code, I am not at a computer with .NET 3.5 so I cannot write a proper testable query

Resources