Using RavenDB, how do I efficiently retrieve a list of items related by a 'foreign key' - linq

I store User objects in RavenDB. Each User has a User.Id property.
I also have a Relationship class that links two User.Ids together to create a Mentor/Mentee relationship, like this:
public class User
{
public string Id { get; set; }
public string UserName { get; set; }
... more properties
}
public class Relationship
{
public string Id { get; set; }
public string MentorId { get; set; }
public string MenteeId { get; set; }
public RelationshipStatus Status { get; set; }
}
Now I want to retrieve a list of Mentees for a given Mentor. I have done this in the following way:
public static List<User> GetMentees(IDocumentSession db, string mentorId)
{
var mentees = new List<User>();
db.Query<Relationship>()
.Where(r => r.MentorId == mentorId)
.Select(r => r.MenteeId)
.ForEach(id => mentees.Add(db.Load<User>(id)));
return mentees;
}
This seems to work just fine but the coding-angel on my shoulder is wrinkling her nose at the smells emanating from the nested use of the IDocumentSession (db) and the need for multiple Load calls to fill the Mentees List.
How can I optimise this method using best practice RavenDB syntax?
Edit
Thanks to #Jonah Himango (see accepted answer below) who solved the problem of multiple calls to the database for me. In addition I have also created a new extension method called 'Memoize' to eliminate the need for the external 'mentees' result List (see code above).
Here is the optimised code. Please feel free to comment and refine further.
The Linq
public static List<User> GetMentees(IDocumentSession db, string mentorId)
{
return db.Query<Relationship>()
.Customize(x => x.Include<Relationship>(o => o.MenteeId))
.Where(r => r.MentorId == mentorId)
.Memoize()
.Select(r => db.Load<User>(r.MenteeId))
.ToList();
}
The extension method
public static List<T> Memoize<T>(this IQueryable<T> target)
{
return target.ToList();
}
Note : This extension method may seem completely superfluous (it is really) but it irritates my geek-gland that I have to call a function called ToList(), not to create a list, but to force the execution of the Linq statement. So my extension method just renames ToList() to the far more accurate Memoize().

You'll want to use the .Include query customization to tell Raven to include the related user object off of each Relationship:
db.Query<Relationship>()
.Customize(x => x.Include<Relationship>(o => o.MenteeId))
.Where(r => r.MentorId == mentorId)
.Select(r => r.MenteeId)
.ForEach(id => mentees.Add(db.Load<User>(id))); // .Load no longer makes a call to the DB; it's already loaded into the session!
Relevant documentation here:
The call to Load() is resolved completely client side (i.e. without
additional requests to the RavenDB server) because the [related]
object has already been retrieved via the .Include call.

Related

Is there a way I can paginate the included list in a linq query

Hope you're doing well,
I was trying to optimize my reads with entity framework, where I arrived at a position, where I get a record from database by id, and I want to include a one-to-many related list, but I don't want to get all data of the list, just a few, so I want to kind of paginate it.
I want to do this process as long as data is in IQueryable state, I don't want to load all data of list in memory and that paginate it as enumerable.
Let's say the query is like below:
var author = await _dbContext.Authors.Where(x => x.Id == id)
.Include(x => x.Books) // <-- paginate this !!??
.FirstOrDefaultAsync();
Entities represent Data state. Pagination and presentation concerns are View state. Entity Framework can help bridge that gap, but it does so by enabling projection so that you can build View state from Data state. Don't pass entities to views, instead build and pass ViewModels to represent the data in accordance to translations and limitations you want for the view.
For instance if you want to pass Author details with their 5 most recent books:
public class AuthorViewModel
{
public int Id { get; set; }
public string Name { get; set; }
// Any other relevant fields...
public ICollection<BookViewModel> RecentBooks = new List<BookViewModel>();
}
public class BookViewModel
{
public int Id { get; set; }
public string Name { get; set; }
public DateTime PublishedDate { get; set; }
// Any other relevant fields...
}
var author = await _dbContext.Authors
.Where(x => x.Id == id)
.Select( x => new AuthorViewModel
{
Id = x.Id,
Name = x.Name,
RecentBooks = x.Books
.OrderByDescending(b => b.PublishedDate)
.Select(b => new BookViewModel
{
Id = b.Id,
Name = b.Name,
PublishedDate = b.PublishedDate
}).Take(5)
.ToList()
}).SingleOrDefault();
This gives you the benefit of structuring the data how you want to present it, while generating efficient queries against the database. You can configure tools like Automapper to perform this kind of mapping to use it's ProjectTo<AuthorViewModel>() as a more succinct alternative.

net core API controller is returning incomplete json

I asked a question a few days ago regarding a net core game I'm making that is using Entity Framework.
I had one issue where a controller was returning duplicate JSON data.
Based on one of the answers, I changed that controller to this:
[HttpGet("GetDungeonAndRoomData/{dungeonId}")]
public async Task<ActionResult<GameDungeon>> GetDungeonAndRoomData(Guid dungeonID)
{
{
var dungeon = await _context.DungeonList
.Select(c => new GameDungeon
{
DungeonId = c.DungeonId,
DungeonName = c.DungeonName,
StartRoom = c.StartRoom,
Rooms = c.Rooms.Select(n => new GameDungeonRoom
{
RoomId = n.RoomId,
RoomText = n.RoomText,
TreasureId = n.TreasureId
})
}).SingleOrDefaultAsync(c => c.DungeonId == dungeonID);
Since I changed the controller, I had to modify this model class, so I added a new property called Rooms.
public partial class GameDungeon
{
[Key]
public string DungeonId { get; set; }
public string DungeonName { get; set; }
public string StartRoom { get; set; }
public IEnumerable<GameDungeonRoom> Rooms { get; set; }
}
Since I added that new "Rooms" property, I had to create a new model called GameDungeonRoom:
public partial class GameDungeonRoom
{
public Guid DungeonId { get; set; }
[Key]
public string RoomId { get; set; }
public string RoomText { get; set; }
public string TreasureId { get; set; }
}
Building and running the game, I now get one set of dungeon data, but it is not returning any rooms.
At first, and based off this Stack Overflow question, .net core : incomplete JSON response,I thought it was because I needed to add this to my Startup.cs file:
services.AddMvc()
.AddJsonOptions(
options => options.SerializerSettings.ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore
);
But that was not the case.
So I then spent the evening trying a bunch of different ways of writing the controller, but they either produced the same results or just threw an error.
Reviewing the code this morning, I realized something. In my controller, the first select statement that creates the new "GameDungeon" is getting data from _context.DungeonList.
DungeonList is a model generated by Entity Framework from a real table in my database.
But GameDungeonRoom is just a new class model I created. It's based off a table in my database called RoomList, but there is nothing in _context that specifically handles GameDungeonRoom.
So I am wondering, should I introduce another controller that kind of looks like this?
var rooms = await _context.RoomList
.Select(c => new GameDungeonRoom ...
And then somehow append that to the GameDungeon object.
I sort of tried that after lunch but ended up just creating a mess of code that created an even bigger mess of errors so I just deleted it all.
Anyway, after all that, this is where my JSON currently stands:
{
"dungeonId" : "293hf938",
"dungeonName" : "Dungeon of Dread",
"startRoom" : "bjgh39811ffr",
"roomId" : "fgf4h635j",
"roomText" : "A big empty room",
"treasureId" : "12a",
"rooms": [
You'll notice that "rooms" is empty. I'm not quite sure why it is, or what's going on.
One idea I had, is maybe I should just create an API controller that get's the dungeon data for a particular dungeon. Then create another API controller that gets the Room data for a particular dungeon.
Then let the client call both controllers(using the same dungeonId) and combine the data on the client side.
So I was wondering if anyone could think of an idea as to why the "rooms" object is empty.
Thanks!
Just guessing you might have hit a cyclic reference in your result set due to Data Context being cached. Hence Json serializer cannot serialize it properly and give incomplete json content. So can you try following to pin point that.
var dungeon = await _context.DungeonList
.Select(c => new GameDungeon
{
DungeonId = c.DungeonId,
DungeonName = c.DungeonName,
StartRoom = c.StartRoom,
Rooms = c.Rooms.Select(n => new GameDungeonRoom
{
RoomId = n.RoomId,
RoomText = n.RoomText,
TreasureId = n.TreasureId
})
})
.AsNoTracking() //This ignore the cached data
.SingleOrDefaultAsync(c => c.DungeonId == dungeonID);

How to pass a parameter to the Read method of signalr transport for Kendo UI grid data source

I was looking at the demo:
Telerik's Demo
and this
Telerik's Example
I still cannot figure out how, using this technique, to pass a parameter
to the Read method, so instead of reading ALL games or products, it reads ONLY a
subset of games or products, by category ID lest say. If I change the Read method in
the server-side code to take a parameter, it never gets hit anymore,
and I cannot figure out how to pass a parameter from the transport
client-side definition definition... Any help would be highly
appreciated!
I posted the question to the Telerik's forums and got this which should work!
Telerik's Answer
In case anyone else comes along in the future and the link in the other answer is gone, or for those who don't want to download a project file and sift through it themselves, here's more detail from the answer given on the Telerik forums:
They use Kendo.DynamicLinq to "bind the request parameters."
They created a custom class that mirrors the structure of the existing DataSourceRequest class typically used in AJAX grid actions.
using System.Collections.Generic;
using Kendo.DynamicLinq;
namespace SignalRLocalHub.Models
{
public class MyDataSourceRequest
{
public int Take { get; set; }
public int Skip { get; set; }
public int Page { get; set; }
public int PageSize { get; set; }
public Filter Filter { get; set; }
public IEnumerable<Sort> Sort { get; set; }
public IEnumerable<Aggregator> Aggregates { get; set; }
}
}
In the ProductHub (SignalR hub) class, this is the Read method:
public DataSourceResult Read(MyDataSourceRequest request)
{
return productService.Read()
.ToDataSourceResult(
request.Take,
request.Skip,
request.Sort,
request.Filter,
request.Aggregates);
}
The productService.Read method it calls, for reference, looks like this:
//int take, int skip, IEnumerable<Sort> sort, Filter filter, IEnumerable<Aggregator> aggregates
public IQueryable<ProductViewModel> Read()
{
return entities.Products
.OrderBy(p => p.ProductID)
.Select(product => new ProductViewModel
{
ProductID = product.ProductID,
ProductName = product.ProductName,
UnitPrice = product.UnitPrice.HasValue ? product.UnitPrice.Value : default(decimal),
UnitsInStock = product.UnitsInStock.HasValue ? product.UnitsInStock.Value : default(short),
Discontinued = product.Discontinued
});
}
And finally, here is the grid's DataSource configuration:
.DataSource(d => d
.SignalR()
.AutoSync(true)
.Transport(tr => tr
.Promise("hubStart")
.Hub("hub")
.Client(c => c.Read("read").Create("create").Update("update").Destroy("destroy"))
.Server(s => s.Read("read").Create("create").Update("update").Destroy("destroy"))
)
.ServerFiltering(true)
.Filter(f => f.Add(m => m.UnitPrice).IsEqualTo(10))
.ServerPaging(true)
.PageSize(10)
.Schema(s => s
.Data("Data")
.Total("Total")
.Aggregates("Aggregates")
.Model(m =>
{
m.Id(e => e.ProductID);
m.Field(e => e.ProductID).Editable(false);
})
)
)
This allows sorting, paging, and filtering by any field that is part of the grid view model. So as long as your "category ID" is a property of the grid view model and the grid is configured to be able to filter by that field, this method should work.

Queryable Attribute allow to get some properties not all of them

I have a web api 2 application, and in my controller , I have this code :
[Queryable]
public IQueryable<Title> GetTitles()
{
return db.Titles;
}
and here is the Title entity :
public partial class Title
{
public Title()
{
this.People = new HashSet<Person>();
}
public short Id { get; set; }
public string NameT { get; set; }
public virtual ICollection<Person> People { get; set; }
}
When people query the Titles, they must get only "NameT" property. but now they get all of the properties. and yes, I know about $select, but I want another way. means even they use $select, they should not able to get "Id" property for example. if I have to bring more information, please tell me. thanks.
There are two ways to solve your problem when you use ODataController. However, they won't affect ApiController non-query part.
In that condition, you can try what Zoe suggested.
1.Ignore those properties while building your model with model builder.
builder.EntityType<Title>().Ignore(title => title.Id);
2.Add ignore member attributes on those properties.
[IgnoreDataMember]
public short Id { get; set; }
More than these, we provide support for limiting the set of allowed queries in Web API 2.2 for OData v4.0.
http://blogs.msdn.com/b/webdev/archive/2014/03/13/getting-started-with-asp-net-web-api-2-2-for-odata-v4-0.aspx
We can use attributes like Unsortable, NonFilterable, NotExpandable or NotNavigable on the properties of the types in our model, or we can configure this explicitly in the model builder.
Maybe you can have filter in your action GetTitles(), like:
[Queryable]
public IQueryable<Title> GetTitles()
{
return db.Titles.Select(t=>t.Name);
}
Use the ODataModelBuilder class as opposed to the ODataConventionModelBuilder class.
var modelBuilder = new ODataModelBuilder();
var titles = modelBuilder.EntitySet<Title>("titles");
var title = titles.EntityType;
title.HasKey(x => x.Id);
title.Ignore(x => x.Id);
title.Property(x => x.TName);
titles.HasIdLink(x => { return x.GenerateSelfLink(true); }, true);
config.Routes.MapODataRoute("odata", "odata", modelBuilder.GetEdmModel());

NHibernate: Where clause containing child collection as subquery building improper T-SQL statements

I am using NHibernate 3.x, along with Fluent NHibernate, and have not had any issues constructing database queries until now.
To simplify my objects for the purposes of this post, I've included a subset of my object and mapping structures below:
IssueItem POCO entity class:
public class IssueItem : DomainEntity, IKeyed<Guid> {
public virtual Guid ID { get; set; }
public virtual string Subject { get; set; }
public virtual string Description { get; set; }
public virtual IList<IssueLocation> Locations { get; set; }
}
Location POCO entity class:
public class Location : DomainEntity, IKeyed<Guid> {
public virtual Guid ID { get; set; }
public virtual string City { get; set; }
public virtual string State { get; set; }
public virtual string Zip { get; set; }
public virtual string Organization { get; set; }
public virtual IssueItem Issue { get; set; }
}
IssueItem Fluent NHibernate map:
public class IssueItemMap : DomainEntityMapping<IssueItem> {
public IssueItemMap()
{
Table("IssueItem");
LazyLoad();
Map(x => x.ID).Column("ID");
Map(x => x.Subject).Column("Subject");
Map(x => x.Description).Column("Description");
HasMany(x => x.Locations).KeyColumn("IssueItemID").LazyLoad().ReadOnly().Inverse();
}
}
Location Fluent NHibernate map:
public class LocationMap : DomainEntityMapping<Location> {
public LocationMap()
{
Table("Location");
LazyLoad();
Map(x => x.ID).Column("ID");
Map(x => x.City).Column("City");
Map(x => x.State).Column("State");
Map(x => x.Zip).Column("Zip");
Map(x => x.Organization).Column("Organization");
References(x => x.IssueItem).ForeignKey("IssueItemID").LazyLoad().ReadOnly();
}
}
Now, I'm using a Unit of Work and Service/Repository pattern in my MVC app. Therefore, I have a domain layer of my project that contains my basic POCO entities, as well as validators and services. In my data layer, I've got my NHibernate-related stuff, such as my repositories that my domain layer access from my services. This is where my NHibernate maps live as well.
In order to ensure that no NHibernate-specific logic creeps into my domain layer (in case I want to use a different ORM in the future), I perform my LINQ statements in my services within my domain layer against IQueryable objects returned from the repositories in my data layer. Therefore, when I write my queries, I am using System.Linq and System.Linq.Expressions instead of the NHibernate.Linq class.
That said, here's my LINQ query I'm having issues with from within one of my service classes in my domain layer:
var issues = _issueRepo.All();
if (!string.IsNullOrWhiteSpace(searchWords)) {
issues = issues.Where(i => i.Subject.Contains(searchWords)
|| i.Description.Contains(searchWords)
|| i.Locations.Where(l => l.Organization.Contains(searchWords)
|| l.City.Contains(searchWords))
.Select(x => x.IssueItemID).Contains(i.ID)
);
}
Now, the IssueItems are queried just fine. However, the one-to-many table (Locations) is not properly queried. This is what I mean...
The generated T-SQL statement is perfect except for the very end of it. Example:
select TOP(100) issueitem0_.ID as ID2_, issueitem0_.Subject as Subject2_, issueitem0_.Description as Description2_
from IssueItem issueitem0_
where issueitem0_.Subject like ('%test%') or issueitem0_.Description like ('%test%')
or exists (select location1_.IssueItemID from Location location1_ where
issueitem0_.ID=location1_.IssueItemID and (location1_.Organization like ('%test%')
or location1_.City like ('%test%')) and location1_.ID=issueitem0_.ID)
See that last bit? It throws in that last "and" statement (and location1_.ID=issueitem0_.ID) that throws a wrench in the whole system. I have tweaked every configuration parameter I could think of with my mapping and have tried many different LINQ statements and I cannot get rid of that last part. I don't know why it adds it.
If I construct the same LINQ statement in LINQPad, it properly generates the T-SQL statement without the last part (and location1_.ID=issueitem0_.ID).
Any ideas?
Thanks!
Joel
Add Any() when you query locations. It will come true if any location property contains what you are looking for. You are trying to select in where clause, then trying to get IssueID from there. I think you will see this query is clearer.
var issues = _issueRepo.All();
if (!string.IsNullOrWhiteSpace(searchWords))
{
issues = issues.Where(i => i.Subject.Contains(searchWords)
|| i.Description.Contains(searchWords)
|| i.Locations.Any(l => l.Organization.Contains(searchWords))
|| i.Locations.Any(l => l.City.Contains(searchWords)) )
}

Resources