We have a collection contains documents in the server. Each document is like:
{ _id: "...", Prop1: "", Prop2: "", Prop3: "", LargeField: "", ... }
There're many other fields but they're not required by the client.
I want to load documents as MyDoc class whose definition is:
public class MyDoc {
public string Id { get; set; }
public string Prop1 { get; set; }
public string Prop2 { get; set; }
public string Prop3 { get; set; }
public string LargeField { get; set; }
}
I've tried:
var client = new MongoClient(uri);
var database = client.GetDatabase("MyDatabase");
var collection = database.GetCollection<MyDocs>("MyDocs");
var allDocs = collection.Find().ToList();
Then it will load all the fields for each document, so I have to put [BsonIgnoreExtraElements] on MyDoc. The issue here is that the document is large but I only needs a limit subset of fields. Is it possible to let the driver know I only need the fields defined in the class?
If not, is it possible to exclude some of the fields like the LargeField to make the result set smaller? I've tried:
var fieldsBuilder = Builders<MyDoc>.Projection;
var fields = fieldsBuilder.Exclude(d => d.LargeField);
var allDocs = collection.Find().Project(fields).ToList();
But now allDocs becomes BsonDocument list instead of the MyDoc list. How to query MyDoc with projection?
Can someone help? It's rather simple in legacy MongoDB driver but I don't know how to do it in the new driver. Thanks.
I had a similar issue, I believe you need to specify the generic type, for some reason Project automatically assumes BsonDocument. This should fix it from BsonDocument to your class.
Change:
var allDocs = collection.Find().Project(fields).ToList();
To:
var allDocs = collection.Find<MyDoc>().Project<MyDoc>(fields).ToList();
As for how to include only certain fields, this can be done just as you are doing it with builder (using Include) or with a string in json form like:
var allDocs = collection.Find<MyDoc>().Project<MyDoc>("{Prop1: 1, Prop2: 1}").ToList();
I would highly recommend checking out projection on this guy's post:
https://www.codementor.io/pmbanugo/working-with-mongodb-in-net-part-3-skip-sort-limit-and-projections-oqfwncyka
From this article:
This brings us to another difference: with a projection definition, it implicitly converts the document type from Student to BsonDocument, so what we get back is a fluent object that, in result, will be a BsonDocument (even though what we're working with is the Student type). If we want to work with Student, we have to indicate that we still want to keep the type to Student.
A newer way:
var fieldsBuilder = Builders<MyDoc>.Projection;
var fields = fieldsBuilder.Exclude(d => d.BigField1).Exclude(d => d.BigField2);
return Collection.Find(x => x.id.Equals(id)).Project<MyDoc>(fields).ToEnumerable();
Gina
We can use FindAsync as well. and to use projection in FindSync, we have to pass the FindOptions something like this: It will select countryName, ID and population.
FindAsync loads documents one by one from DB cursor in async manner, it's a good choice when you have a huge database.
Relevant Code:
cursor = await collection.FindAsync(y => y.CountryID > 205,
new FindOptions<MyDoc, MyDoc>()
{
BatchSize = 20,
NoCursorTimeout = true,
AllowPartialResults = true,
Projection = "{'_id':1,'Name':1,'Population':1}"
}).ConfigureAwait(false);
Related
I asked a question a few days ago regarding a net core game I'm making that is using Entity Framework.
I had one issue where a controller was returning duplicate JSON data.
Based on one of the answers, I changed that controller to this:
[HttpGet("GetDungeonAndRoomData/{dungeonId}")]
public async Task<ActionResult<GameDungeon>> GetDungeonAndRoomData(Guid dungeonID)
{
{
var dungeon = await _context.DungeonList
.Select(c => new GameDungeon
{
DungeonId = c.DungeonId,
DungeonName = c.DungeonName,
StartRoom = c.StartRoom,
Rooms = c.Rooms.Select(n => new GameDungeonRoom
{
RoomId = n.RoomId,
RoomText = n.RoomText,
TreasureId = n.TreasureId
})
}).SingleOrDefaultAsync(c => c.DungeonId == dungeonID);
Since I changed the controller, I had to modify this model class, so I added a new property called Rooms.
public partial class GameDungeon
{
[Key]
public string DungeonId { get; set; }
public string DungeonName { get; set; }
public string StartRoom { get; set; }
public IEnumerable<GameDungeonRoom> Rooms { get; set; }
}
Since I added that new "Rooms" property, I had to create a new model called GameDungeonRoom:
public partial class GameDungeonRoom
{
public Guid DungeonId { get; set; }
[Key]
public string RoomId { get; set; }
public string RoomText { get; set; }
public string TreasureId { get; set; }
}
Building and running the game, I now get one set of dungeon data, but it is not returning any rooms.
At first, and based off this Stack Overflow question, .net core : incomplete JSON response,I thought it was because I needed to add this to my Startup.cs file:
services.AddMvc()
.AddJsonOptions(
options => options.SerializerSettings.ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore
);
But that was not the case.
So I then spent the evening trying a bunch of different ways of writing the controller, but they either produced the same results or just threw an error.
Reviewing the code this morning, I realized something. In my controller, the first select statement that creates the new "GameDungeon" is getting data from _context.DungeonList.
DungeonList is a model generated by Entity Framework from a real table in my database.
But GameDungeonRoom is just a new class model I created. It's based off a table in my database called RoomList, but there is nothing in _context that specifically handles GameDungeonRoom.
So I am wondering, should I introduce another controller that kind of looks like this?
var rooms = await _context.RoomList
.Select(c => new GameDungeonRoom ...
And then somehow append that to the GameDungeon object.
I sort of tried that after lunch but ended up just creating a mess of code that created an even bigger mess of errors so I just deleted it all.
Anyway, after all that, this is where my JSON currently stands:
{
"dungeonId" : "293hf938",
"dungeonName" : "Dungeon of Dread",
"startRoom" : "bjgh39811ffr",
"roomId" : "fgf4h635j",
"roomText" : "A big empty room",
"treasureId" : "12a",
"rooms": [
You'll notice that "rooms" is empty. I'm not quite sure why it is, or what's going on.
One idea I had, is maybe I should just create an API controller that get's the dungeon data for a particular dungeon. Then create another API controller that gets the Room data for a particular dungeon.
Then let the client call both controllers(using the same dungeonId) and combine the data on the client side.
So I was wondering if anyone could think of an idea as to why the "rooms" object is empty.
Thanks!
Just guessing you might have hit a cyclic reference in your result set due to Data Context being cached. Hence Json serializer cannot serialize it properly and give incomplete json content. So can you try following to pin point that.
var dungeon = await _context.DungeonList
.Select(c => new GameDungeon
{
DungeonId = c.DungeonId,
DungeonName = c.DungeonName,
StartRoom = c.StartRoom,
Rooms = c.Rooms.Select(n => new GameDungeonRoom
{
RoomId = n.RoomId,
RoomText = n.RoomText,
TreasureId = n.TreasureId
})
})
.AsNoTracking() //This ignore the cached data
.SingleOrDefaultAsync(c => c.DungeonId == dungeonID);
Once again, I am facing an issue, this time with LINQ Expression builder and this time I am even struggling to find the reason why it's not working. I have a Database-First EF project with quite a few tables. For this specific case, I have to use 2 of them - DocHead and Contragent. MyService.metadata.cs looks like this:
[MetadataTypeAttribute(typeof(DocHead.DocHeadMetadata))]
public partial class DocHead
{
// This class allows you to attach custom attributes to properties
// of the DocHead class.
//
// For example, the following marks the Xyz property as a
// required property and specifies the format for valid values:
// [Required]
// [RegularExpression("[A-Z][A-Za-z0-9]*")]
// [StringLength(32)]
// public string Xyz { get; set; }
internal sealed class DocHeadMetadata
{
// Metadata classes are not meant to be instantiated.
private DocHeadMetadata()
{
}
public string doc_Code { get; set; }
public string doc_Name { get; set; }
public string doc_ContrCode { get; set; }
//...
[Include]
public Contragent Contragent { get; set; }
}
}
[MetadataTypeAttribute(typeof(Contragent.ContragentMetadata))]
public partial class Contragent
{
// This class allows you to attach custom attributes to properties
// of the Contragent class.
//
// For example, the following marks the Xyz property as a
// required property and specifies the format for valid values:
// [Required]
// [RegularExpression("[A-Z][A-Za-z0-9]*")]
// [StringLength(32)]
// public string Xyz { get; set; }
internal sealed class ContragentMetadata
{
// Metadata classes are not meant to be instantiated.
private ContragentMetadata()
{
}
public string Code { get; set; }
public string Name { get; set; }
//...
I take some docHeads like this:
IQueryable<DocHead> docHeads = new MyEntities().DocHead;
Then I try to sort them like this:
docHeads = docHeads.OrderByDescending(x => x.Contragent.Name);
It is all working like I want it. I get those docHeads sorted by the name of the joined Contragent. My problem is that I will have to sort them by a field, given as a string parameter. I need to be able to write something like this:
string field = "Contragent.Name";
string linq = "docHeads = docHeads.OrderByDescending(x => x." + field + ")";
IQueryable<DocHead> result = TheBestLinqLibraryInTheWorld.PrepareLinqQueryable(linq);
Unfortunately, TheBestLinqLibraryInTheWorld does not exist (for now). So, I have set up a method as a workaround.
public static IQueryable<T> OrderByField<T>(this IQueryable<T> q, string SortField, bool Ascending)
{
var param = Expression.Parameter(typeof(T), "x");
var prop = Expression.Property(param, SortField); // normally returns x.sortField
var exp = Expression.Lambda(prop, param); // normally returns x => x.sortField
string method = Ascending ? "OrderBy" : "OrderByDescending";
Type[] types = new Type[] { q.ElementType, exp.Body.Type };
var mce = Expression.Call(typeof(Queryable), method, types, q.Expression, exp); // normally returns sth similar to q.OrderBy(x => x.sortField)
return q.Provider.CreateQuery<T>(mce);
}
Normally... yes, when it comes to own properties of the class DocHead - those prefixed with doc_. The disaster strikes when I call this method like this:
docHeads = docHeads.OrderByField<DocHead>("Contragent.Name", true); // true - let it be Ascending order
To be more specific, the exception in the title is thrown on line 2 of the method OrderByField():
var prop = Expression.Property(param, SortField);
In My.edmx (the model), the tables DocHead and Contragent have got a relation already set up for me, which is the following: 0..1 to *.
Once again, I have no problem writing "static" queries at all. I have no problem creating "dynamic" ones via the method OrderByField(), but only when it comes to properties of the class DocHead. When I try to order by a prop of the joined Contragent class - the disaster strikes. Any help will be greatly appretiated, thank you!
The problem is that Expression.Property method does not support nested properties. It does exactly what it says - creates expression that represents a property denoted by propertyName parameter of the object denoted by the expression parameter.
Luckily it can easily be extended. You can use the following simple Split / Aggregate trick anytime you need to create a nested property access expression:
var prop = SortField.Split('.').Aggregate((Expression)param, Expression.Property);
I would like to have some flexibility and provide some interface to specify list of columns that should be included to the final select dynamically.
For example for this table
public class Person
{
[DatabaseGeneratedAttribute(DatabaseGeneratedOption.Identity), Key()]
public int Id { get; set; }
[Required]
public string FirstName { get; set; }
[Required]
public string LastName { get; set; }
public string Address { get; set; }
...
}
I want to have at one case
persons.Select(p => new { FirstName = p.FirstName, LastName = p.LastName }).ToList();
but at another I want to have
persons.Select(p => new { FirstName = p.FirstName, LastName = p.LastName, Address = p.Address }).ToList();
at third case I'll need something else... So, I was thinking it would be good to have some flexible mechanism that would allow to specify the list of columns to extract.
Any ideas how I can do this?
I was reading a bit about LINQ expressions and I have some feeling that this is the right way but do not understand how implement it yet...
If you include the NuGet package System.Linq.Dynamic it is pretty straightforward:
var columns = new [] { "FirstName", "LastName" };
var selectText = "new (" + string.Join(", ", columns) + ")";
var result = Users.AsQueryable().Select(selectText);
Or for type safety
var columns = new Expression<Func<User, object>> [] { x => x.FirstName, x => x.Surname };
var selectText = "new (" + string.Join(", ", columns.Select(x => ((MemberExpression)x.Body).Member.Name)) + ")";
var result = Users.AsQueryable().Select(selectText);
This is turning expression into string for a library that will turn them back so it isn't the most optimized approach, but the cost is likely to be insignificant compared to that database query.
You could do it by building expressions but the library does it all for you.
Basically it is the right way you're doing it. You could only make it a bit more comfortable and type safe by using classes as 'Models' for your data. Basically, create another Person class with either all fields of the table or only the ones you need in that 'person'. Doing this instead of just creating nameless objects and writing them to a list gives you more flexibility, since you can add other Methods that transform the data in this class.
You could also add different constructors to this class depending on which fields you want to initialize. Like this:
var modelPersons = persons.Select(p => new ModelPerson(p.FirstName, p.LastName)).ToList();
If you write a Constructor for every initialization you need, you'll have reusable and short code when querying. Of course, you still could use a object initializer like this:
var modelPersons = persons.Select(p => new ModelPerson {
FirstName = p.FirstName
LastName = p.LastName
}).ToList();
Which is as long as the one you posted, but it's better to use an object you can define yourself, add methods, format data etc. But all together you're using the right approach.
To expand on Mant101's answer. You can solve the duplicate namespace problem by using extern alias. See also this answer to an SO question.
I have a List<Product> contains properties of Bikes (Name, ProductName, Color, List Price). I'm struggling to figure out how to write a search function using LINQ. I'd like to find a name of Bike. Any suggest will be help me some ways.
Imagine that your name is taken from a variable called nameToSearch.
This is if you want to get the Product.
string nameToSearch = "BikeName";
List<Product> list = bikes.Where(x => x.Name == nameToSearch).ToList();
I assume you have the following Product class:
public class Product
{
public String Name { get; set; }
public String ProductName { get; set; }
public String Color { get; set; }
public String List { get; set; }
public String Price { get; set; }
}
You also mentioned you have your data in a List<Product>. I will give a demo name for it:
List<Product> myProductList = GetProductList();
// Where GetProductList() will create a new List<Product> and populate it.
String bikeNameFilter = GetNameFilter();
// You can chnage this by the string you want for filtering.
You can use the following to get your data:
List<Product> myFilteredProductList = (from p in myProductList
where p.Name = bikeNameFilter
select p;
).ToList()
Obviously you can change the filter you want to use to another property of your product. Finally to get the actual name, you can loop through the list you just got:
foreach (var p in myFilteredProductList)
{
Console.WriteLine(p.ProductName);
// Use this value wherever you want.
}
Take a look at a nuget package I have created
http://www.nuget.org/packages/NinjaNye.SearchExtensions
This will enable the following (and more) which will return results where the search term appears in any of the properties specified
var result = products.Search("searchTerm", p => p.Name, p => p.ProductName);
Performing a search against all string properties can be done as follows:
var result = products.Search("searchTerm");
Alternatively, you can perform an AND search where the search term exists in a set of properties as follows:
string searchTerm = "searchTerm";
var result = products.Search(searchTerm, p => p.Name)
.Search(searchTerm, p => p.ProductName);
For more information take a look at the projects GitHub page or my blog posts
UPDATE: don't forget the using directive...
using NinjaNye.SearchExtensions
I have the following class definition:
public class PriorityItem
{
public string uniqueID { get; set; }
public List<DelegationTask> DelegationTasks = new List<DelegationTask>();
public List<int> Priorities = new List<int>();
}
Now lets say I create a list from this class like this:
List<PriorityItem> pItems = new List<PriorityItem>();
And now I want to check for the presence of an item in this list based on the uniqueID property how is this done?
For example I will populate the list with a collection of PriorityItem, but only if the Priority item does not exist. To determine if it exists, I need to check using LINQ the value of uniqueID.
Hope this makes sense.
The Any linq extension method returns true if any items are found which match a certain criteria. You can use it as follows:
string idToSearchFor = "123"
bool exists = pItems.Any(item => item.uniqueId == idToSearchFor);
With the above code exists will be true, only if your list contains a PriorityItem with the uniqueId of "123".
Exists does the same thing as Any. Exists matches the SQL parlance a little more than Any does. Any and All naturally fit together as a kind of natural qualifier pair.
However in the sentence, "do we have this unique id?" Perhaps "exists" is a better fit in terms of readability and intention.
class Class1
{
[Test]
public void Test()
{
// initialise list
var pItems = new List<PriorityItem>(){new PriorityItem(){uniqueID = "1"}, new PriorityItem(){uniqueID = "2"}, new PriorityItem(){ uniqueID = "123"}};
var IDtoFind = "123";
var wasFound = pItems.Exists(o => o.uniqueID == IDtoFind);
}
}
public class PriorityItem
{
public string uniqueID { get; set; }
//public List<DelegationTask> DelegationTasks = new List<DelegationTask>();
public List<int> Priorities = new List<int>();
}