Say I have a collection of the following simple class:
public class MyEntity
{
public string SubId { get; set; }
public System.DateTime ApplicationTime { get; set; }
public double? ThicknessMicrons { get; set; }
}
I need to search through the entire collection looking for 5 consecutive (not 5 total, but 5 consecutive) entities that have a null ThicknessMicrons value. Consecutiveness will be based on the ApplicationTime property. The collection will be sorted on that property.
How can I do this in a Linq query?
You can write your own extension method pretty easily:
public static IEnumerable<IEnumerable<T>> FindSequences<T>(this IEnumerable<T> sequence, Predicate<T> selector, int size)
{
List<T> curSequence = new List<T>();
foreach (T item in sequence)
{
// Check if this item matches the condition
if (selector(item))
{
// It does, so store it
curSequence.Add(item);
// Check if the list size has met the desired size
if (curSequence.Count == size)
{
// It did, so yield that list, and reset
yield return curSequence;
curSequence = new List<T>();
}
}
else
{
// No match, so reset the list
curSequence = new List<T>();
}
}
}
Now you can just say:
var groupsOfFive = entities.OrderBy(x => x.ApplicationTime)
.FindSequences(x => x.ThicknessMicrons == null, 5);
Note that this will return all sub-sequences of length 5. You can test for the existence of one like so:
bool isFiveSubsequence = groupsOfFive.Any();
Another important note is that if you have 9 consecutive matches, only one sub-sequence will be located.
Related
If I have a class like this
`
class Person
{
public string First;
public string Last;
public bool IsMarried;
public int Age;
}`
Then how can I write a LINQ Expression where I could select properties of a Person. I want to do something like this (user can enter 1..n properties)
SelectData<Person>(x=>x.First, x.Last,x.Age);
What would be the input expression of my SelectData function ?
SelectData(Expression<Func<TEntity, List<string>>> selector); ?
EDIT
In my SelectData function I want to extract property names and then generate SELECT clause of my SQL Query dynamically.
SOLUTION
Ok, so what I have done is to have my SelectData as
public IEnumerable<TEntity> SelectData(Expression<Func<TEntity, object>> expression)
{
NewExpression body = (NewExpression)expression.Body;
List<string> columns = new List<string>();
foreach(var arg in body.Arguments)
{
var exp = (MemberExpression)arg;
columns.Add(exp.Member.Name);
}
//build query
And to use it I call it like this
ccc<Person>().SelectData(x => new { x.First, x.Last, x.Age });
Hopefully it would help someone who is looking :)
Thanks,
IY
I think it would be better to use delegates instead of Reflection. Apart from the fact that delegates will be faster, the compiler will complain if you try to fetch property values that do not exist. With reflection you won't find errors until run time.
Luckily there is already something like that. it is implemented as an extension function of IEnumerable, and it is called Select (irony intended)
I think you want something like this:
I have a sequence of Persons, and I want you to create a Linq
statement that returns per Person a new object that contains the
properties First and Last.
Or:
I have a sequence of Persns and I want you to create a Linq statement
that returns per Person a new object that contains Age, IsMarried,
whether it is an adult and to make it difficult: one Property called
Name which is a combination of First and Last
The function SelectData would be something like this:
IEnumerable<TResult> SelectData<TSource, TResult>(this IEnumerable<TSource> source,
Func<TSource, TResult> selector)
{
return source.Select(selector);
}
Usage:
problem 1: return per Person a new object that contains the
properties First and Last.
var result = Persons.SelectData(person => new
{
First = person.First,
Last = person.Last,
});
problem 2: return per Person a new object that contains Age, IsMarried, whether he is an adult and one Property called Name which is a combination
of First and Last
var result = Persons.SelectData(person => new
{
Age = person.Name,
IsMarried = person.IsMarried,
IsAdult = person.Age > 21,
Name = new
{
First = person.First,
Last = person.Last,
},
});
Well let's face it, your SelectData is nothing more than Enumerable.Select
You could of course create a function where you'd let the caller provide a list of properties he wants, but (1) that would limit his possibilities to design the end result and (2) it would be way more typing for him to call the function.
Instead of:
.Select(p => new
{
P1 = p.Property1,
P2 = p.Property2,
}
he would have to type something like
.SelectData(new List<Func<TSource, TResult>()
{
p => p.Property1, // first element of the property list
p -> p.Property2, // second element of the property list
}
You won't be able to name the returned properties, you won't be able to combine several properties into one:
.Select(p => p.First + p.Last)
And what would you gain by it?
Highly discouraged requirement!
You could achive similar result using Reflection and Extension Method
Model:
namespace ConsoleApplication2
{
class Person
{
public string First { get; set; }
public string Last { get; set; }
public bool IsMarried { get; set; }
public int Age { get; set; }
}
}
Service:
using System.Collections.Generic;
using System.Linq;
namespace Test
{
public static class Service
{
public static IQueryable<IQueryable<KeyValuePair<string, object>>> SelectData<T>(this IQueryable<T> queryable, string[] properties)
{
var queryResult = new List<IQueryable<KeyValuePair<string, object>>>();
foreach (T entity in queryable)
{
var entityProperties = new List<KeyValuePair<string, object>>();
foreach (string property in properties)
{
var value = typeof(T).GetProperty(property).GetValue(entity);
var entityProperty = new KeyValuePair<string, object>(property, value);
entityProperties.Add(entityProperty);
}
queryResult.Add(entityProperties.AsQueryable());
}
return queryResult.AsQueryable();
}
}
}
Usage:
using System;
using System.Collections.Generic;
using System.Linq;
namespace Test
{
class Program
{
static void Main(string[] args)
{
var list = new List<Person>()
{
new Person()
{
Age = 18,
First = "test1",
IsMarried = false,
Last = "test2"
},
new Person()
{
Age = 40,
First = "test3",
IsMarried = true,
Last = "test4"
}
};
var queryableList = list.AsQueryable();
string[] properties = { "Age", "Last" };
var result = queryableList.SelectData(properties);
foreach (var element in result)
{
foreach (var property in element)
{
Console.WriteLine($"{property.Key}: {property.Value}");
}
}
Console.ReadKey();
}
}
}
Result:
Age: 18
Last: test2
Age: 40
Last: test4
Hello I have a list of a custom object (its really just a tuple), I'm trying to remove from one list where the same object is in another list. But I'm having difficulty doing so.
Here is the code:
public class FileNameTuple
{
public FileNameTuple(string origFileName, string newFileName)
{
OrigFileName = origFileName;
NewFileName = newFileName;
}
public string OrigFileName
{
get;
set;
}
public string NewFileName
{
get;
set;
}
}
List<FileNameTuple> fileListing = new List<FileNameTuple>();
List<FileNameTuple> failedFileListing = new List<FileNameTuple>();
//doesn't work, has compliation error
fileListing.RemoveAll(i => failedFileListing.Contains(i.OrigFileName));
Contains compares the object itself, you want to compare a property. Use Any:
fileListing
.RemoveAll(i => failedFileListing.Any(fnt => i.OrigFileName == fnt.OrigFileName));
If you want to use Contains you need to override Equals (+ GetHashCode):
public class FileNameTuple
{
//...
public override bool Equals(object obj)
{
if(object.ReferenceEquals(obj, this) return true;
FileNameTuple t2 = obj as FileNameTuple;
if(t2 == null) return false;
return OrigFileName == t2.OrigFileName;
}
public override int GetHashCode()
{
return (OrigFileName ?? "").GetHashCode();
}
}
Now you can compare FileNameTuples:
fileListing.RemoveAll(i => failedFileListing.Contains(i));
If you just want a filtered list instead of changing an existing list, you can use the Except LINQ operator.
If you can't change your class like Tim Schmelter and don't want to build a custom equality comparer, you can always use my PredicateEqualityComparer.
fileListing.Except(
failedFileListing,
new PredicateEqualityComparer((f1, f2) => f1.OriginalName == f2.OriginalName));
I need to get differences between two IEnumerable. I wrote extension method for it. But as you can see, it has performance penalties. Anyone can write better version of it?
EDIT
After first response, I understand that I could not explain well. I'm visiting both arrays three times. This is performance penalty. It must be a single shot.
PS: Both is optional :)
public static class LinqExtensions
{
public static ComparisonResult<T> Compare<T>(this IEnumerable<T> source, IEnumerable<T> target)
{
// Looping three times is performance penalty!
var res = new ComparisonResult<T>
{
OnlySource = source.Except(target),
OnlyTarget = target.Except(source),
Both = source.Intersect(target)
};
return res;
}
}
public class ComparisonResult<T>
{
public IEnumerable<T> OnlySource { get; set; }
public IEnumerable<T> OnlyTarget { get; set; }
public IEnumerable<T> Both { get; set; }
}
Dependig on the use-case, this might be more efficient:
public static ComparisonResult<T> Compare<T>(this IEnumerable<T> source, IEnumerable<T> target)
{
var both = source.Intersect(target).ToArray();
if (both.Any())
{
return new ComparisonResult<T>
{
OnlySource = source.Except(both),
OnlyTarget = target.Except(both),
Both = both
};
}
else
{
return new ComparisonResult<T>
{
OnlySource = source,
OnlyTarget = target,
Both = both
};
}
}
You're looking for an efficient full outer join.
Insert all items into a Dictionary<TKey, Tuple<TLeft, TRight>>. If a given key is not present, add it to the dictionary. If it is present, update the value. If the "left member" is set, this means that the item is present in the left source collection (you call it source). The opposite is true for the right member. You can do that using a single pass over both collections.
After that, you iterate over all values of this dictionary and output the respective items into one of three collections, or you just return it as an IEnumerable<Tuple<TLeft, TRight>> which saves the need for result collections.
I've got an ILookup generated by some complicated expression. Let's say it's a lookup of people by last name. (In our simplistic world model, last names are unique by family)
ILookup<string, Person> families;
Now I've got two queries I'm interested in how to build.
First, how would I filter by last name?
var germanFamilies = families.Where(family => IsNameGerman(family.Key));
But here, germanFamilies is an IEnumerable<IGrouping<string, Person>>; if I call ToLookup() on it, I'd best bet would get an IGrouping<string, IGrouping<string, Person>>. If I try to be smart and call SelectMany first I'd end up with the computer doing a lot of unnecessary work. How would you convert this enumeration into a lookup easily?
Second, I'd like to get a lookups of adults only.
var adults = families.Select(family =>
new Grouping(family.Key, family.Select(person =>
person.IsAdult())));
Here I'm faced with two problems: the Grouping type doesn't exist (except as an internal inner class of Lookup), and even if it did we'd have the problem discussed above.
So, apart from implementing the ILookup and IGrouping interfaces completely, or make the computer do silly amounts of work (regrouping what has already been grouped), is there a way to alter existing ILookups to generate new ones that I missed?
(I'm going to assume you actually wanted to filter by last name, given your query.)
You can't modify any implementation of ILookup<T> that I'm aware of. It's certainly possible to implement ToLookup with an immutable lookup, as you're clearly aware :)
What you could do, however, is to change to use a Dictionary<string, List<Person>>:
var germanFamilies = families.Where(family => IsNameGerman(family.Key))
.ToDictionary(family => family.Key,
family.ToList());
That approach also works for your second query:
var adults = families.ToDictionary(family => family.Key,
family.Where(person => persion.IsAdult)
.ToList());
While that's still doing a bit more work than we might think necessary, it's not too bad.
EDIT: The discussion with Ani in the comments is worth reading. Basically, we're already going to be iterating over every person anyway - so if we assume O(1) dictionary lookup and insertion, we're actually no better in terms of time-complexity using the existing lookup than flattening:
var adults = families.SelectMany(x => x)
.Where(person => person.IsAdult)
.ToLookup(x => x.LastName);
In the first case, we could potentially use the existing grouping, like this:
// We'll have an IDictionary<string, IGrouping<string, Person>>
var germanFamilies = families.Where(family => IsNameGerman(family.Key))
.ToDictionary(family => family.Key);
That is then potentially much more efficient (if we have many people in each family) but means we're using groupings "out of context". I believe that's actually okay, but it leaves a slightly odd taste in my mouth, for some reason. As ToLookup materializes the query, it's hard to see how it could actually go wrong though...
For your first query, what about implementing your own FilteredLookup able to take advantage of coming from another ILookup ?
(thank to Jon Skeet for the hint)
public static ILookup<TKey, TElement> ToFilteredLookup<TKey, TElement>(this ILookup<TKey, TElement> lookup, Func<IGrouping<TKey, TElement>, bool> filter)
{
return new FilteredLookup<TKey, TElement>(lookup, filter);
}
With FilteredLookup class being:
internal sealed class FilteredLookup<TKey, TElement> : ILookup<TKey, TElement>
{
int count = -1;
Func<IGrouping<TKey, TElement>, bool> filter;
ILookup<TKey, TElement> lookup;
public FilteredLookup(ILookup<TKey, TElement> lookup, Func<IGrouping<TKey, TElement>, bool> filter)
{
this.filter = filter;
this.lookup = lookup;
}
public bool Contains(TKey key)
{
if (this.lookup.Contains(key))
return this.filter(this.GetGrouping(key));
return false;
}
public int Count
{
get
{
if (count >= 0)
return count;
count = this.lookup.Where(filter).Count();
return count;
}
}
public IEnumerable<TElement> this[TKey key]
{
get
{
var grp = this.GetGrouping(key);
if (!filter(grp))
throw new KeyNotFoundException();
return grp;
}
}
public IEnumerator<IGrouping<TKey, TElement>> GetEnumerator()
{
return this.lookup.Where(filter).GetEnumerator();
}
IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
private IGrouping<TKey, TElement> GetGrouping(TKey key)
{
return new Grouping<TKey, TElement>(key, this.lookup[key]);
}
}
and Grouping:
internal sealed class Grouping<TKey, TElement> : IGrouping<TKey, TElement>
{
private readonly TKey key;
private readonly IEnumerable<TElement> elements;
internal Grouping(TKey key, IEnumerable<TElement> elements)
{
this.key = key;
this.elements = elements;
}
public TKey Key { get { return key; } }
public IEnumerator<TElement> GetEnumerator()
{
return elements.GetEnumerator();
}
IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
}
So basically your first query will be:
var germanFamilies = families.ToFilteredLookup(family => IsNameGerman(family.Key));
This allows you to avoid re-flattening-filtering-ToLookup, or creating a new dictionary (and so hashing keys again).
For the second query the idea will be similar, you should just create a similar class not filtering for the whole IGrouping but for the elements of the IGrouping.
Just an idea, maybe it could not be faster than other methods :)
The Lookup creates an index with a Key type and a value type generic indexer. You can added to a lookup and remove from a lookup by using concat for add and iterate and removing the key items in a temp list then rebuilding the lookup. The look up then works like a dictionary by retrieving the value type by a key.
public async Task TestILookup()
{
// Lookup<TKey,TElement>
List<Product> products = new List<Product>
{
new Product { ProductID = 1, Name = "Kayak", Category = "Watersports", Price = 275m },
new Product { ProductID = 2, Name = "Lifejacket", Category = "Watersports", Price = 48.95m },
new Product { ProductID = 3, Name = "Soccer Ball", Category = "Soccer", Price = 19.50m },
new Product { ProductID = 4, Name = "Corner Flag", Category = "Soccer", Price = 34.95m }
};
// create an indexer
ILookup<int, Product> lookup = (Lookup<int,Product>) products.ToLookup(p => p.ProductID, p => p);
Product newProduct = new Product { ProductID = 5, Name = "Basketball", Category = "Basketball", Price = 120.15m };
lookup = lookup.SelectMany(l => l)
.Concat(new[] { newProduct })
.ToLookup(l => l.ProductID, l=>l);
foreach (IGrouping<int, Product> packageGroup in lookup)
{
// Print the key value of the IGrouping.
output.WriteLine("ProductID Key {0}",packageGroup.Key);
// Iterate over each value in the IGrouping and print its value.
foreach (Product product in packageGroup)
output.WriteLine("Name {0}", product.Name);
}
Assert.Equal(lookup.Count(), 5);
}
public class Product
{
public int ProductID { get; set; }
public string Name { get; set; }
public string Category { get; set; }
public decimal Price { get; set; }
}
Output:
ProductID Key 1
Name Kayak
ProductID Key 2
Name Lifejacket
ProductID Key 3
Name Soccer Ball
ProductID Key 4
Name Corner Flag
ProductID Key 5
Name Basketball
I'm building a rather large filter based on an SearchObject that has 50+ fields that can be searched.
Rather than building my where clause for each one of these individually I thought I'd use some slight of hand and try building custom attribute suppling the necessary information and then using reflection to build out each of my predicate statements (Using LinqKit btw). Trouble is, that the code finds the appropriate values in the reflection code and successfully builds a predicate for the property, but the "where" doesn't seem to actually generate and my query always returns 0 records.
The attribute is simple:
[AttributeUsage(AttributeTargets.Property, AllowMultiple=true)]
public class FilterAttribute: Attribute
{
public FilterType FilterType { get; set; } //enum{ Object, Database}
public string FilterPath { get; set; }
//var predicate = PredicateBuilder.False<Metadata>();
}
And this is my method that builds out the query:
public List<ETracker.Objects.Item> Search(Search SearchObject, int Page, int PageSize)
{
var predicate = PredicateBuilder.False<ETracker.Objects.Item>();
Type t = typeof(Search);
IEnumerable<PropertyInfo> pi = t.GetProperties();
string title = string.Empty;
foreach (var property in pi)
{
if (Attribute.IsDefined(property, typeof(FilterAttribute)))
{
var attrs = property.GetCustomAttributes(typeof(FilterAttribute),true);
var value = property.GetValue(SearchObject, null);
if (property.Name == "Title")
title = (string)value;
predicate.Or(a => GetPropertyVal(a, ((FilterAttribute)attrs[0]).FilterPath) == value);
}
}
var res = dataContext.GetAllItems().Take(1000)
.Where(a => SearchObject.Subcategories.Select(b => b.ID).ToArray().Contains(a.SubCategory.ID))
.Where(predicate);
return res.ToList();
}
The SearchObject is quite simple:
public class Search
{
public List<Item> Items { get; set; }
[Filter(FilterType = FilterType.Object, FilterPath = "Title")]
public string Title { get; set; }
...
}
Any suggestions will be greatly appreciated. I may well be going way the wrong direction and will take no offense if someone has a better alternative (or at least one that works)
You're not assigning your predicate anywhere. Change the line to this:
predicate = predicate.Or(a => GetPropertyVal(a, ((FilterAttribute)attrs[0]).FilterPath) == value);