I'm designing an application using ASP.NET Web API and Entity Framework 5 and LINQ to Entities. The Web API doesn't serve up the entities directly, it converts them to a set of data transfer objects that are similar but not identical to my entities. The API will be used by a Silverlight application initially but I will have to support non-.NET clients (e.g. iOS apps) down the road. I'd also like to give the client the ability to run a robust set of queries against the API.
These requirements have lead me to consider the query object pattern. Essentially, I want to create a homegrown query object client-side, post it to the Web API, and convert the query object to a lambda expression that I can use in LINQ to Entities. This last part is what's tripping me up.
Starting with a simple comparison query, I want to be able to convert an object that looks like the following into a lambda expression at runtime.
public enum QueryOperator
{
None = 0,
GreaterThan,
GreaterThanOrEqualTo,
EqualTo,
NotEqualTo,
LessThanOrEqualTo,
LessThan
}
public class SimpleQuery<T>
{
public SimpleQuery()
{
this.Field = null;
this.Operator = QueryOperator.None;
this.Value = null;
}
public string Field { get; set; }
public QueryOperator Operator { get; set; }
public object Value { get; set; }
public IEnumerable<T> Execute(IQueryable<T> queryTarget)
{
// ????
}
}
How can I do this?
I've had to do things like this in the past. Here's what I came up with:
public IEnumerable<T> Execute(IQueryable<T> queryTarget)
{
return queryTarget.Where(this.GetWhereExpression<T>());
}
private Expression<Func<T, bool>> GetWhereExpression<T>()
{
var param = Expression.Parameter(typeof(T), "x");
var prop = Expression.Property(param, this.Field);
var value = Expression.Constant(this.Value, prop.Type);
Expression compare = null;
switch(this.Operator)
{
case QueryOperator.EqualTo:
compare = Expression.Equal(prop, value);
break;
...
}
return Expression.Lambda(compare, param);
}
Related
I'm moving some code from a traditional worker role to an Azure Function. I've found a line of code that returns a result when I call it from a console app, but null when I call it from a function.
Now, for some example code. I wrote a _resultProvider class that basically queries an underlying CosmosDB database -- at the base class, it creates an IOrderedQueryable query and filters it based on the predicate that you pass in as a parameter. The first line of code returns a result only when I call it from a console app, and null if I call it from an Azure Function. The second line returns a result from either platform.
Gets result when called from the worker role, but null when called from the function:
var res1 = _resultProvider.GetSpecialAsync(o => id == o.Id).Result.FirstOrDefault();
Gets result from either the worker role or the function:
var res2 = _resultProvider.GetSpecialAsync(o => 1 == 1).Result.Where(o=>id==o.Id).FirstOrDefault();
I'm guessing this is some kind of LINQ issue, because passing the predicate along doesn't seem to work from the function, but it works if I just get all the results and query that result set.
Here's the GetSpecialAsync code:
public async Task<IEnumerable<T>> GetItemsSpecialAsync(Expression<Func<T, bool>> predicate)
{
IDocumentQuery<T> query = client.CreateDocumentQuery<T>(
UriFactory.CreateDocumentCollectionUri(DatabaseId, CollectionId),
new FeedOptions { MaxItemCount = -1, EnableCrossPartitionQuery = true})
.Where(predicate)
.AsDocumentQuery();
List<T> results = new List<T>();
while (query.HasMoreResults)
{
results.AddRange(await query.ExecuteNextAsync<T>());
}
return results;
}
Here's the type I'm attempting to return, ResultDocVm:
public class ResultDocVm : DocViewModelBase
{
public string Name { get; set; }
public long AccountId { get; set; }
// ... insert more junk here with getters and setters
}
Here's DocViewModelBase:
public abstract class DocViewModelBase
{
[JsonProperty(PropertyName = "id")]
public string Id { get; set; }
public DateTime? CreatedAt { get; set; }
//... even more junk here
}
So after all the back and forth it looks like the Console App is taking the JsonProperty attribute into account while the Azure Function doesn't.
This generates a query which will return no results because the Id property will be uppercased and not lowercased ie id.
It sounds like a bug with the Azure Function at the Azure level and not with your code per se.
We have a lot of Dto classes in our project and on various occasions SELECT them using Expressions from the entity framework context. This has the benefit, that EF can parse our request, and build a nice SQL statement out of it.
Unfortunatly, this has led to very big Expressions, because we have no way of combining them.
So if you have a class DtoA with 3 properties, and one of them is of class DtoB with 5 properties, and again one of those is of class DtoC with 10 properties, you would have to write one big selector.
public static Expression<Func<ClassA, DtoA>> ToDto =
from => new DtoA
{
Id = from.Id,
Name = from.Name,
Size = from.Size,
MyB = new DtoB
{
Id = from.MyB.Id,
...
MyCList = from.MyCList.Select(myC => new DtoC
{
Id = myC.Id,
...
}
}
};
Also, they cannot be reused. When you have DtoD, which also has a propertiy of class DtoB, you would have to paste in the desired code of DtoB and DtoC again.
public static Expression<Func<ClassD, DtoD>> ToDto =
from => new DtoD
{
Id = from.Id,
Length = from.Length,
MyB = new DtoB
{
Id = from.MyB.Id,
...
MyCList = from.MyCList.Select(myC => new DtoC
{
Id = myC.Id,
...
}
}
};
So this will escalate pretty fast. Please note that the mentioned code is just an example, but you get the idea.
I would like to define an expression for each class and then combine them as required, as well as EF still be able to parse it and generate the SQL statement so to not lose the performance improvement.
How can i achieve this?
Have you thought about using Automapper ? You can define your Dtos and create a mapping between the original entity and the Dto and/or vice versa, and using the projection, you don't need any select statements as Automapper will do it for you automatically and it will project only the dto's properties into SQL query.
for example, if you have a Person table with the following structure:
public class Person
{
public int Id { get; set; }
public string Title { get; set; }
public string FamilyName { get; set; }
public string GivenName { get; set; }
public string Initial { get; set; }
public string PreferredName { get; set; }
public string FormerTitle { get; set; }
public string FormerFamilyName { get; set; }
public string FormerGivenName { get; set; }
}
and your dto was like this :
public class PersonDto
{
public int Id { get; set; }
public string Title { get; set; }
public string FamilyName { get; set; }
public string GivenName { get; set; }
}
You can create a mapping between Person and PersonDto like this
Mapper.CreateMap<Person, PersonDto>()
and when you query the database using Entity Framework (for example), you can use something like this to get PersonDto columns only:
ctx.People.Where(p=> p.FamilyName.Contains("John"))
.Project()
.To<PersonDto>()
.ToList();
which will return a list of PersonDtos that has a family name contains "John", and if you run a sql profiler for example you will see that only the PersonDto columns were selected.
Automapper also supports hierachy, if your Person for example has an Address linked to it that you want to return AddressDto for it.
I think it worth to have a look and check it, it cleans a lot of the mess that manual mapping requires.
I thought about it a little, and I didn't come up with any "awesome" solution.
Essentially you have two general choices here,
Use placeholder and rewrite expression tree entirely.
Something like this,
public static Expression<Func<ClassA, DtoA>> DtoExpression{
get{
Expression<Func<ClassA, DtoA>> dtoExpression = classA => new DtoA(){
BDto = Magic.Swap(ClassB.DtoExpression),
};
// todo; here you have access to dtoExpression,
// you need to use expression transformers
// in order to find & replace the Magic.Swap(..) call with the
// actual Expression code(NewExpression),
// Rewriting the expression tree is no easy task,
// but EF will be able to understand it this way.
// the code will be quite tricky, but can be solved
// within ~50-100 lines of code, I expect.
// For that, see ExpressionVisitor.
// As ExpressionVisitor detects the usage of Magic.Swap,
// it has to check the actual expression(ClassB.DtoExpression),
// and rebuild it as MemberInitExpression & NewExpression,
// and the bindings have to be mapped to correct places.
return Magic.Rebuild(dtoExpression);
}
The other way is to start using only Expression class(ditching the LINQ). This way you can write the queries from zero, and reusability will be nice, however, things get harder & you lose type safety. Microsoft has nice reference about dynamic expressions. If you structure everything that way, you can reuse a lot of the functionality. Eg, you define NewExpression and then you can later reuse it, if needed.
The third way is to basically use lambda syntax: .Where, .Select etc.. This gives you definitely better "reusability" rate. It doesn't solve your problem 100%, but it can help you to compose queries a bit better. For example: from.MyCList.Select(dtoCSelector)
I am building dynamic linq expressions which is working fine for a single entity.
For example:
I have a class called Employee and empeduinfo
public class Employee
{
public int Id { get; set; }
public string Name { get; set; }
}
public class EmpEduInfo
{
public int Id { get; set; }
public string Name { get; set; }
public int EmpId { get; set; }
}
I need to get all the the employees and empeduinfo class starts with "x"
I prepared expression for startswith("x")
var temp= entities.employees.Include("EmpEduInfo").Where(mydynamicexpression);
In this case it is filtering only parent table not on child.
I need to prepare generic expression so than i need to filter both parent and child objects dynamically.
Without using expression I know a solution:
var temp= (from ee in entities.Employee.Include("EmpEduInfo").Where(x => x.name.StartsWith("t"))
where ee.EmpEduInfo.Where(x => x.name.StartsWith("t")).Count()>0
select ee).ToList();
using expressions I am building generic expression to provide dynamic advance search rather than writing in each and every entity.
Here is my expression details
// Get the method information for the String.StartsWith() method
MethodInfo mi = typeof(string).GetMethod("StartsWith", new Type[] { typeof(string) });
// Build the parameter for the expression
ParameterExpression empparam= Expression.Parameter(typeof(employee), "ename");;
// Build the member that was specified for the expression
MemberExpression field = Expression.PropertyOrField(empparam, "name");
// Call the String.StartsWith() method on the member
MethodCallExpression startsWith = Expression.Call(field, mi, Expression.Constant("t"));
var namelamda = Expression.Lambda<Func<employee, bool>>(startsWith, new ParameterExpression[] { empparam });
var temp = entities.employees.Include("empedudetails").Where(namelamda).ToList();
You can look at the Expression the compiler generates using IQueryable:
IQueryable<Employee> query =
from ee in entities.Employee ...
var expression = query.Expression;
Look at expression in a debugger to see what you need to generate - LINQPad is good for this.
You might want to simplify your query a bit first:
IQueryable<Employee> query =
from ee in entities.Employee.Include("EmpEduInfo")
where
ee.name.StartsWith("t") &&
ee.EmpEduInfo.Any(x => x.name.StartsWith("t"))
select ee;
I was trying in EF 4.0 either we have write DB extentions for the same.
Option is provided in EF 4.1
http://www.asp.net/mvc/tutorials/getting-started-with-ef-using-mvc/reading-related-data-with-the-entity-framework-in-an-asp-net-mvc-application
Thanks.
I have the following EF class:
class Product
{
public int ProdId { get; set; }
public int ProdDesc { get; set; }
public int ProdKeywords { get; set; }
}
Now I have to implement a search function that looks at ProdDesc and ProdKeywords. The keywords are registered in a array and the collection of products in a IQueryable
string[] keywordsArray = new string[] {"kw1", "kw2", ..., "kwN"};
IQueryable<Product> products = repository.GetProducts();
To see if there are products matching the keywords I use the following LINQ:
var matchingProducts = products.Where(p => keywordsArray.Any(k => p.ProdDesc.Contains(k) ||
p.ProdKeywords.Contains(k));
which works like a charm in .NET 4.
The BIG problem is that I am forced to use this code in .NET 3.5 and I just discovered that Any and Contains (the LINQ method, not the one applied to strings) don't work in that framework. That's a real pain. The code is too big to rewrite everything and the deadline is too close.
I found this article really interesting but I can't make it work in my case. Anybody might help?
What's about:
static class Extension
{
public static bool Contains(this IEnumerable<object> source, object value)
{
foreach (object o in source)
if (o.Equals(value)) return true;
return false;
}
}
var mylist = keywordsArray.ToList();
matchingProducts = products.Where(p => mylist.Exists(k => p.ProdDesc.Contains(k) ||
p.ProdKeywords.Contains(k));
you could query first the any and store that in a enumerable and the check if the count is bigger then 0
I'm using Entity Framework 4.1 Code First. In my entity, I have three date/time properties:
public class MyEntity
{
[Key]
public Id { get; set; }
public DateTime FromDate { get; set; }
public DateTime ToDate { get; set; }
[NotMapped]
public DateTime? QueryDate { get; set; }
// and some other fields, of course
}
In the database, I always have the From/To dates populated. I query against them using a simple where clause. But in the result set, I want to include the date I queried for. I need to persist this for some other business logic to work.
I'm working on an extension method to do this, but I'm running into problems:
public static IQueryable<T> WhereDateInRange<T>(this IQueryable<T> queryable, DateTime queryDate) where T : MyEntity
{
// this part works fine
var newQueryable = queryable.Where(e => e.FromDate <= queryDate &&
e.ToDate >= queryDate);
// in theory, this is what I want to do
newQueryable = newQueryable.Select(e =>
{
e.QueryDate = queryDate;
return e;
});
return newQueryable;
}
This doesn't work. It works if I use an IEnumerable, but I want to keep it as IQueryable so everything runs on the database side, and this extention method can still be used in any part of another query. When it's IQueryable, I get a compile error of the following:
A lambda expression with a statement body cannot be converted to an expression tree
If this was SQL, I would just do something like this:
SELECT *, #QueryDate as QueryDate
FROM MyEntities
WHERE #QueryDate BETWEEN FromDate AND ToDate
So the question is, how can I transform the expression tree I already have to include this extra property assignment? I have looked into IQueryable.Expression and IQueryable.Provider.CreateQuery - there's a solution in there somewhere. Maybe an assignment expression can be appended to the existing expression tree? I'm not familiar enough with the expression tree methods to figure this out. Any ideas?
Example Usage
To clarify, the goal is to be able to perform something like this:
var entity = dataContext.Set<MyEntity>()
.WhereDateInRange(DateTime.Now)
.FirstOrDefault();
And have the DateTime.Now persisited into the QueryDate of the resulting row, WITHOUT having more than one row returned from the database query. (With the IEnumerable solution, multiple rows are returned before FirstOrDefault picks the row we want.)
Another Idea
I could go ahead and map QueryDate like a real field, and set its DatabaseGeneratedOption to Computed. But then I would need some way to inject the "#QueryDate as QueryDate" into the SQL created by EF's select statements. Since it's computed, EF won't try to provide values during update or insert. So how could I go about injecting custom SQL into the select statements?
Ladislav is absolutely right. But since you obviously want the second part of your question to be answered, here is how you can use Assign. This won't work with EF, though.
using System;
using System.Linq;
using System.Linq.Expressions;
namespace SO5639951
{
static class Program
{
static void Main()
{
AdventureWorks2008Entities c = new AdventureWorks2008Entities();
var data = c.Addresses.Select(p => p);
ParameterExpression value = Expression.Parameter(typeof(Address), "value");
ParameterExpression result = Expression.Parameter(typeof(Address), "result");
BlockExpression block = Expression.Block(
new[] { result },
Expression.Assign(Expression.Property(value, "AddressLine1"), Expression.Constant("X")),
Expression.Assign(result, value)
);
LambdaExpression lambdaExpression = Expression.Lambda<Func<Address, Address>>(block, value);
MethodCallExpression methodCallExpression =
Expression.Call(
typeof(Queryable),
"Select",
new[]{ typeof(Address),typeof(Address) } ,
new[] { data.Expression, Expression.Quote(lambdaExpression) });
var data2 = data.Provider.CreateQuery<Address>(methodCallExpression);
string result1 = data.ToList()[0].AddressLine1;
string result2 = data2.ToList()[0].AddressLine1;
}
}
}
Update 1
Here is the same code after some tweaking. I got rid of the "Block" expression, that EF choked on in the code above, to demonstrate with absolute clarity that it's "Assign" expression that EF does not support. Note that Assign works in principle with generic Expression trees, it is EF provider that does not support Assign.
using System;
using System.Linq;
using System.Linq.Expressions;
namespace SO5639951
{
static class Program
{
static void Main()
{
AdventureWorks2008Entities c = new AdventureWorks2008Entities();
IQueryable<Address> originalData = c.Addresses.AsQueryable();
Type anonType = new { a = new Address(), b = "" }.GetType();
ParameterExpression assignParameter = Expression.Parameter(typeof(Address), "value");
var assignExpression = Expression.New(
anonType.GetConstructor(new[] { typeof(Address), typeof(string) }),
assignParameter,
Expression.Assign(Expression.Property(assignParameter, "AddressLine1"), Expression.Constant("X")));
LambdaExpression lambdaAssignExpression = Expression.Lambda(assignExpression, assignParameter);
var assignData = originalData.Provider.CreateQuery(CreateSelectMethodCall(originalData, lambdaAssignExpression));
ParameterExpression selectParameter = Expression.Parameter(anonType, "value");
var selectExpression = Expression.Property(selectParameter, "a");
LambdaExpression lambdaSelectExpression = Expression.Lambda(selectExpression, selectParameter);
IQueryable<Address> finalData = assignData.Provider.CreateQuery<Address>(CreateSelectMethodCall(assignData, lambdaSelectExpression));
string result = finalData.ToList()[0].AddressLine1;
}
static MethodCallExpression CreateSelectMethodCall(IQueryable query, LambdaExpression expression)
{
Type[] typeArgs = new[] { query.ElementType, expression.Body.Type };
return Expression.Call(
typeof(Queryable),
"Select",
typeArgs,
new[] { query.Expression, Expression.Quote(expression) });
}
}
}
No, I don't think there is a solution. It is true that you can modify expression tree but you will get exactly the same exception as you got with your linq query because that query actually is what you will build in expression tree. The problem is not in expression tree but in the mapping. EF can't map QueryData to the result. Moreover you are trying to do projection. Projection can't be done to mapped entity and anonymous type can't be returned from the method.
You can off course do the select you mentioned but simply you can't map it to your entity. You must create a new type for that:
var query = from x in context.MyData
where x.FromDate <= queryDate && x.ToDate >= queryDate
select new MyDateWrapper
{
MyData = x,
QueryDate = queryDate
};
Automapper has Queryable Extensions, i think it can resolve your needs.
You can use ProjectTo to calculate property on runtime.
Ef Core 2 set value to ignored property on runtime
http://docs.automapper.org/en/stable/Queryable-Extensions.html
Example configuration:
configuration.CreateMap(typeof(MyEntity), typeof(MyEntity))
.ForMember(nameof(Entity.QueryDate), opt.MapFrom(src => DateTime.Now));
Usage:
queryable.ProjectTo<MyEntity>();
Thank you for all of the valuable feedback. It sounds like the answer is "no - you can't do it that way".
So - I figured out a workaround. This is very specific to my implementation, but it does the trick.
public class MyEntity
{
private DateTime? _queryDate;
[ThreadStatic]
internal static DateTime TempQueryDate;
[NotMapped]
public DateTime? QueryDate
{
get
{
if (_queryDate == null)
_queryDate = TempQueryDate;
return _queryDate;
}
}
...
}
public static IQueryable<T> WhereDateInRange<T>(this IQueryable<T> queryable, DateTime queryDate) where T : MyEntity
{
MyEntity.TempQueryDate = queryDate;
return queryable.Where(e => e.FromDate <= queryDate && e.ToDate >= queryDate);
}
The magic is that I'm using a thread static field to cache the query date so it's available later in the same thread. The fact that I get it back in the QueryDate's getter is specific to my needs.
Obviously this isn't an EF or LINQ solution to the original question, but it does accomplish the same effect by removing it from that world.