I'm using LINQ to SQL to obtain data from a set of database tables. The database design is such that given a unique ID from one table (Table A) one and only one instance should be returned from an associated table (Table B).
Is there a more concise way to compose this query and ensure that only one item was returned without using the .Count() extension method like below:
var set = from itemFromA in this.dataContext.TableA
where itemFromA.ID == inputID
select itemFromA.ItemFromB;
if (set.Count() != 1)
{
// Exception!
}
// Have to get individual instance using FirstOrDefault or Take(1)
FirstOrDefault helps somewhat but I want to ensure that the returned set contains only one instance and not more.
It sounds like you want Single:
var set = from itemFromA in this.dataContext.TableA
where itemFromA.ID == inputID
select itemFromA.ItemFromB;
var onlyValue = set.Single();
Documentation states:
Returns the only element of a sequence, and throws an exception if there is not exactly one element in the sequence.
Of course that means you don't get to customize the message of the exception... if you need to do that, I'd use something like:
// Make sure that even if something is hideously wrong, we only transfer data
// for two elements...
var list = set.Take(2).ToList();
if (list.Count != 1)
{
// Throw an exception
}
var item = list[0];
The benefit of this over your current code is that it will avoid evaluating the query more than once.
Related
I'm using EF Core but I'm not really an expert with it, especially when it comes to details like querying tables in a performant manner...
So what I try to do is simply get the max-value of one column from a table with filtered data.
What I have so far is this:
protected override void ReadExistingDBEntry()
{
using Model.ResultContext db = new();
// Filter Tabledata to the Rows relevant to us. the whole Table may contain 0 rows or millions of them
IQueryable<Measurement> dbMeasuringsExisting = db.Measurements
.Where(meas => meas.MeasuringInstanceGuid == Globals.MeasProgInstance.Guid
&& meas.MachineId == DBMatchingItem.Id);
if (dbMeasuringsExisting.Any())
{
// the max value we're interested in. Still dbMeasuringsExisting could contain millions of rows
iMaxMessID = dbMeasuringsExisting.Max(meas => meas.MessID);
}
}
The equivalent SQL to what I want would be something like this.
select max(MessID)
from Measurement
where MeasuringInstanceGuid = Globals.MeasProgInstance.Guid
and MachineId = DBMatchingItem.Id;
While the above code works (it returns the correct value), I think it has a performance issue when the database table is getting larger, because the max filtering is done at the client-side after all rows are transferred, or am I wrong here?
How to do it better? I want the database server to filter my data. Of course I don't want any SQL script ;-)
This can be addressed by typing the return as nullable so that you do not get a returned error and then applying a default value for the int. Alternatively, you can just assign it to a nullable int. Note, the assumption here of an integer return type of the ID. The same principal would apply to a Guid as well.
int MaxMessID = dbMeasuringsExisting.Max(p => (int?)p.MessID) ?? 0;
There is no need for the Any() statement as that causes an additional trip to the database which is not desirable in this case.
I am trying to group by my custom method. For example, if the group id is something, then I want to return 1 or 0 from the method of GetClientGroup, then I want to group by the value. But I am getting error such as this.
Error
could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync(). See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
await (from o in _cdsContext.Order
where o.ClienteleId == clienteleId && o.DeliveryDate >= new DateTime(2020, 06, 29).Date
&& o.DeliveryDate != null
group o by new
{
o.ClienteleId,
o.DeliveryDate,
ClientGroup= o.OrderTypeId == 22 ? 259 : GetClientGroup(clienteleId, (int)o.GroupId),
}
into g
select new { ClienteleId = g.Key.ClienteleId}).ToListAsync()
I think you get this error at run time, not at compile time. Am I right?
IEnumerable and IQueryable
You should be aware of the difference between IEnumerable<...> and IQueryable<...>.
Object that implement IEnumerable<...> or IQueryable<...> represents the potentional to give you an enumerable sequence. Once you've got the sequence, you can ask for the first element, and once you've got this, you can ask for the next element as long as there is an element.
This iterating over the elements is usually done using a foreach (var element in sequence) {...}. This translates into the following:
IEnumerable<MyType> sequence = ... // the potential to get iterator
IEnumerator<MyType> enumerator = sequence.GetEnumerator(); // get the iterator
while (enumerator.MoveNext()) // iterate
{ // as long as there are items
MyType item = enumerator.Current; // fetch the item
ProcessItem(item); // and process it.
}
The LINQ methods that don't return IEnumerable<...> or IQueryable<...>, like ToList, ToDictionary, Count, Any, FirstOrDefault, etc internally all use foreach or GetEnumerator
An object that implements IEnumerable<...> is meant to be processed by your local process. The object holds everything to be able to iterate, inclusive calls to local methods.
On the other hand, an object that implements IQueryable<...>, like your _cdsContext.Order is meant to be processed by another process, usually a database management system.
This object holds an Expression and a Provider. The Expression is a generic form of the data that you want to query. The Provider knows who has to execute the query, and what language is used (usually SQL)
Concatenating LINQ statements won't execute the query, they will only change the Expression. When (deep inside) GetEnumerator() is called, the Expression is sent to the Provider, who will translate it into SQL and execute the query at the DBMS. The fetched data is represented as an iterator to your process, who will repeatedly call MoveNext() and Current.
Back to your question
Your GroupBy contains a call to a local method. The GroupBy won't execute the query, it will only change the Expression. In the end you do a ToList. The Tolist will do a GetEnumerator(). The Expression is sent to the Provider who will try to translate it into SQL.
Alas, your provider doesn't know your local method GetClientGroup, and thus can't convert it into SQL. In fact, apart from all your local methods, there are also several LINQ methods that can't be translated into SQL. See Supported and Unsupported LINQ methods (LINQ to entities)
Your compiler doesn't know which methods the provider can translate, so the compiler won't complain. Only at run time, when you do a ToList, the problem is detected.
How to solve the problem
The problem is in parameter KeySelector of Queryable.GroupBy
Expression<Func<TSource,TKey>> keySelector
Alas you forgot to write what GetClientGroup does. It seems that it takes the ClienteleId and the GroupId of an Order, and returns an integer that is similar to a ClientGroup.
The most easy would be to replace the call to GetClientGroup with the code that is in that method. Don't call any other methods
DateTime deliveryLimitDate = new DateTime(2020, 06, 29).Date;
var result = dbContext.Orders
.Where (order => order.ClienteleId == clienteleId
&& order.DeliveryDate != null
&& order.DeliveryDate >= deliveryLimitDate)
.GroupBy(order => new // Parameter KeySelector
{
ClienteleId = order.ClienteleId,
DeliveryDate = order.DeliveryDate,
ClientGroup= order.OrderTypeId == 22 ? 259 :
// formula in GetClientGroup(...)
// for example
(int)order.GroupId << 16 + order.ClienteleId
// parameter ResultSelector
group => new { ClienteleId = group.Key.ClienteleId});
Instead of a separate Select, I used the GroupBy overload with a parameter ResultSelector. Your result is a sequence of objects with only one property ClienteleId. Consider to return only a sequence of ClienteleId:
// parameter ResultSelector
group => group.Key.ClienteleId});
Alas, since I don't know your GetClientGroup, I can't give you parameter KeySelector
I'm using NHibernate 3.2 and I have a repository method that looks like:
public IEnumerable<MyModel> GetActiveMyModel()
{
return from m in Session.Query<MyModel>()
where m.Active == true
select m;
}
Which works as expected. However, sometimes when I use this method I want to filter it further:
var models = MyRepository.GetActiveMyModel();
var filtered = from m in models
where m.ID < 100
select new { m.Name };
Which produces the same SQL as the first one and the second filter and select must be done after the fact. I thought the whole point in LINQ is that it formed an expression tree that was unravelled when it's needed and therefore the correct SQL for the job could be created, saving my database requests.
If not, it means all of my repository methods have to return exactly what is needed and I can't make use of LINQ further down the chain without taking a penalty.
Have I got this wrong?
Updated
In response to the comment below: I omitted the line where I iterate over the results, which causes the initial SQL to be run (WHERE Active = 1) and the second filter (ID < 100) is obviously done in .NET.
Also, If I replace the second chunk of code with
var models = MyRepository.GetActiveMyModel();
var filtered = from m in models
where m.Items.Count > 0
select new { m.Name };
It generates the initial SQL to retrieve the active records and then runs a separate SQL statement for each record to find out how many Items it has, rather than writing something like I'd expect:
SELECT Name
FROM MyModel m
WHERE Active = 1
AND (SELECT COUNT(*) FROM Items WHERE MyModelID = m.ID) > 0
You are returning IEnumerable<MyModel> from the method, which will cause in-memory evaluation from that point on, even if the underlying sequence is IQueryable<MyModel>.
If you want to allow code after GetActiveMyModel to add to the SQL query, return IQueryable<MyModel> instead.
You're running IEnumerable's extension method "Where" instead of IQueryable's. It will still evaluate lazily and give the same output, however it evaluates the IQueryable on entry and you're filtering the collection in memory instead of against the database.
When you later add an extra condition on another table (the count), it has to lazily fetch each and every one of the Items collections from the database since it has already evaluated the IQueryable before it knew about the condition.
(Yes, I would also like to be the extensive extension methods on IEnumerable to instead be virtual members, but, alas, they're not)
I have created a linq query that returns my required data, I now have a new requirement and need to add an extra field into the returned results. My entity contains an ID field that I am trying to map against another table without to much luck.
This is what I have so far.
Dictionary<int, string> itemDescriptions = new Dictionary<int, string>();
foreach (var item in ItemDetails)
{
itemDescriptions.Add(item.ItemID, item.ItemDescription);
}
DB.TestDatabase db = new DB.TestDatabase(Common.GetOSConnectionString());
List<Transaction> transactionDetails = (from t db.Transactions
where t.CardID == CardID.ToString()
select new Transaction
{
ItemTypeID= t.ItemTypeID,
TransactionAmount = t.TransactionAmount,
ItemDescription = itemDescriptions.Select(r=>r.Key==itemTypeID).ToString()
}).ToList();
What I am trying to do is key the value from the dictonary where the key = itemTypeID
I am getting this error.
Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator.
What do I need to modify?
This is a duplicate of this question. The problem you're having is because you're trying to match an in-memory collection (itemDescriptions) with a DB table. Because of the way LINQ2SQL works it's trying to do this in the DB which is not possible.
There are essentially three options (unless I'm missing something)
1) refactor your query so you pass a simple primitive object to the query that can be passed accross to the DB (only good if itemDescriptions is a small set)
2) In your query use:
from t db.Transactions.ToList()
...
3) Get back the objects you need as you're doing, then populate ItemDescription in a second step.
Bear in mind that the second option will force LINQ to evaluate the query and return all transactions to your code that will then be operated on in memory. If the transaction table is large this will not be quick!
This is kinda theoretical question,
I was looking at someone else' code (below) and my simple solution was to instantiate the collection outside linq, but I can guess there will be cases where I'd want to instantiate the objects inside the query, and perhaps only on a selection of elements.
Here's a simplified example of how this was being done (badly).
var pods = (from n in ids
where new Node(Convert.ToInt32(n)).HasValue("propertyName")
select new
{
Id = Convert.ToInt32(n),
Url = new Node(Convert.ToInt32(n)).Url,
Name = new Node(Convert.ToInt32(n)).Title()
}).ToList();
Irrelevant Note: in this case the Node constructor is getting data from a memory cache.
How can I improve this example to only instantiate each object once using linq?
Cheers.
Murray.
Use a let clause like this:
var pods = (
from n in ids
let id = Convert.ToInt32(n)
let node = new Node(id)
where node.HasValue("propertyName")
select new
{
Id = id,
Url = node.Url,
Name = node.Title()
}
).ToList();
For more information please see let clause (C# Reference):
In a query expression, it is sometimes
useful to store the result of a
sub-expression in order to use it in
subsequent clauses. You can do this
with the let keyword, which creates a
new range variable and initializes it
with the result of the expression you
supply. Once initialized with a value,
the range variable cannot be used to
store another value. However, if the
range variable holds a queryable type,
it can be queried.