I am trying to build a search predicate in code that compares two fields in Sitecore and I am getting a strange error message. Basically I have two date fields on each content item - FirstPublishDate (the date that the content item was first published) and LastPublishDate (the last date that the content item was published). I would like to find all content items where the LastPublishDate falls within a certain date range AND where the LastPublishDate does not equal the FirstPublishDate. Using Linq here is my method for generating the predicate...
protected Expression<Func<T, Boolean>> getDateFacetPredicate<T>() where T : MySearchResultItem
{
var predicate = PredicateBuilder.True<T>();
foreach (var facet in myFacetCategories)
{
var dateTo = System.DateTime.Now;
var dateFrom = dateTo.AddDays(facet.Value*-1);
predicate = predicate.And(i => i.LastPublishDate.Between(dateFrom, dateTo, Inclusion.Both)).And(j => j.LastPublishDate != j.FirstPublishDate);
}
return predicate;
}
Then I use this predicate in my general site search code to perform the search as follows: the above predicate gets passed in to this method as the "additionalWhere" parameter.
public static SearchResults<T> GeneralSearch<T>(string searchText, ISearchIndex index, int currentPage = 0, int pageSize = 20, string language = "", IEnumerable<string> additionalFields = null,
Expression<Func<T, Boolean>> additionalWhere = null, Expression<Func<T, Boolean>> additionalFilter = null, IEnumerable<string> facets = null,
Expression<Func<T, Boolean>> facetFilter = null, string sortField = null, SortDirection sortDirection = SortDirection.Ascending) where T : SearchResultItem {
using (var context = index.CreateSearchContext()) {
var query = context.GetQueryable<T>();
if (!string.IsNullOrWhiteSpace(searchText)) {
var keywordPred = PredicateBuilder.True<T>();
// take into account escaping of special characters and working around Sitecore limitation with Contains and Equals methods
var isSpecialMatch = Regex.IsMatch(searchText, "[" + specialSOLRChars + "]");
if (isSpecialMatch) {
var wildcardText = string.Format("\"*{0}*\"", Regex.Replace(searchText, "([" + specialSOLRChars + "])", #"\$1"));
wildcardText = wildcardText.Replace(" ", "*");
keywordPred = keywordPred.Or(i => i.Content.MatchWildcard(wildcardText)).Or(i => i.Name.MatchWildcard(wildcardText));
}
else {
keywordPred = keywordPred.Or(i => i.Content.Contains(searchText)).Or(i => i.Name.Contains(searchText));
}
if (additionalFields != null && additionalFields.Any()) {
keywordPred = additionalFields.Aggregate(keywordPred, (current, field) => current.Or(i => i[field].Equals(searchText)));
}
//query = query.Where(i => (i.Content.Contains(searchText) || i.Name.Contains(searchText))); // more explicit call to check the content or item name for our term
query = query.Where(keywordPred);
}
if (language == string.Empty) {
language = Sitecore.Context.Language.ToString();
}
if (language != null) {
query = query.Filter(i => i.Language.Equals(language));
}
query = query.Page(currentPage, pageSize);
if (additionalWhere != null) {
query = query.Where(additionalWhere);
}
if (additionalFilter != null) {
query = query.Filter(additionalFilter);
}
query = query.ApplySecurityFilter();
FacetResults resultFacets = null;
if (facets != null && facets.Any()) {
resultFacets = facets.Aggregate(query, (current, fname) => current.FacetOn(i => i[fname])).GetFacets();
}
// calling this before applying facetFilter should allow us to get a total facet set
// instead of just those related to the current result set
// var resultFacets = query.GetFacets();
// apply after getting facets for more complete facet list
if (facetFilter != null) {
query = query.Where(facetFilter);
}
if (sortField != null)
{
if (sortDirection == SortDirection.Ascending)
{
query = query.OrderBy(x => x[sortField]);
}
else
{
query = query.OrderByDescending(x => x[sortField]);
}
}
var results = query.GetResults(); // this enumerates the actual results
return new SearchResults<T>(results.Hits, results.TotalSearchResults, resultFacets);
}
}
When I try this I get the following error message:
Server Error in '/' Application.
No constant node in query node of type: 'Sitecore.ContentSearch.Linq.Nodes.EqualNode'. Left: 'Sitecore.ContentSearch.Linq.Nodes.FieldNode'. Right: 'Sitecore.ContentSearch.Linq.Nodes.FieldNode'.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.NotSupportedException: No constant node in query node of type: 'Sitecore.ContentSearch.Linq.Nodes.EqualNode'. Left: 'Sitecore.ContentSearch.Linq.Nodes.FieldNode'. Right: 'Sitecore.ContentSearch.Linq.Nodes.FieldNode'.
Source Error:
Line 548: FacetResults resultFacets = null;
Line 549: if (facets != null && facets.Any()) {
Line 550: resultFacets = facets.Aggregate(query, (current, fname) => current.FacetOn(i => i[fname])).GetFacets();
Line 551: }
Line 552: // calling this before applying facetFilter should allow us to get a total facet set
From what I can understand about the error message it seems to not like that I am trying to compare two different fields to each other instead of comparing a field to a constant. The other odd thing is that the error seems to be pointing to a line of code that has to do with aggregating facets. I did a Google search and came up with absolutely nothing relating to this error. Any ideas?
Thanks,
Corey
I think what you are trying is not possible, and if you look at this that might indeed be the case. A solution that is given there is to put your logic in the index: create a ComputedField that checks your dates and puts a value in the index that you can search on (can be a simple boolean).
You will need to split your logic though - the query on the date range can still be done in the predicate (as it is relative to the current date) but the comparison of first and last should be done on index time instead of on query time.
Related
***ctx.Load(listItemCollection,
eachItem => eachItem.Include(
item => item,
item => item["Column1"],
item => item["Column2"]
));***
i have list of fields in a array of string instead of column1 and column2, how can i pass it through in include linq, not able to create proper lambda on runtime. i tried following ways but couldn't get success. Static befor loops works but thw fields added in loop fails as it doesn't evaluate string value in loop
***Expression<Func<ListItem, object>>[] paramss = new
Expression<Func<ListItem, object>>[length];
paramss[0] = x => x.ContentType;
paramss[1] = x => x["Title"];
count = 2;
foreach (string item in solConnDefModel.Columns)
{ paramss[count] = x => x[item];
count++;
}***
Please take a reference of below code:
List dlist = context.Web.Lists.GetByTitle("listname");
context.Load(dlist);
context.ExecuteQuery();
string[] fieldNames = { "Id", "Title", "num", "mStartDate" };
// Create the expression used to define the fields to be included
List<Expression<Func<ListItemCollection, object>>> fieldsToBeIncluded = new List<Expression<Func<ListItemCollection, object>>>();
foreach (string s in fieldNames)
{
fieldsToBeIncluded.Add(items => items.Include(item => item[s]));
}
// Initialize the collection of list items
var listItems = dlist.GetItems(new CamlQuery());
context.Load(listItems, fieldsToBeIncluded.ToArray());
context.ExecuteQuery();
You can hover on load method to see what type parameter it requires, then generate a corresponding one and pass it.
i have to create lambda expression at runtime. following code i was able to get expected value
Expression<Func<ListItem, object>>[] paramss = new Expression<Func<ListItem, object>>[length];
foreach (string item in Columns)
{
if (item.ToLower() != "contenttype")
{
ParameterExpression parameter = Expression.Parameter(typeof(ListItem), "x");
var propertyInfo = typeof(ListItem).GetMethod("get_Item");
var arguments = new List<Expression> { Expression.Constant(item) };
var expression = Expression.Call(parameter, propertyInfo, arguments);
var lambda = Expression.Lambda<Func<ListItem, object>>(expression, parameter);
paramss[count] = lambda;
}
else
{
paramss[count] = x => x.ContentType;
}
count++;
}
my objective is to reindex an index with 10 million shards for the purposes of changing field mappings to facilitate significant terms analysis.
My problem is that I am having trouble using the NEST library to perform a re-index, and the documentation is (very) limited. If possible I need an example of the following in use:
http://nest.azurewebsites.net/nest/search/scroll.html
http://nest.azurewebsites.net/nest/core/bulk.html
NEST provides a nice Reindex method you can use, although the documentation is lacking. I've used it in a very rough-and-ready fashion with this ad-hoc WinForms code.
private ElasticClient client;
private double count;
private void reindex_Completed()
{
MessageBox.Show("Done!");
}
private void reindex_Next(IReindexResponse<object> obj)
{
count += obj.BulkResponse.Items.Count();
var progress = 100 * count / (double)obj.SearchResponse.Total;
progressBar1.Value = (int)progress;
}
private void reindex_Error(Exception ex)
{
MessageBox.Show(ex.ToString());
}
private void button1_Click(object sender, EventArgs e)
{
count = 0;
var reindex = client.Reindex<object>(r => r.FromIndex(fromIndex.Text).NewIndexName(toIndex.Text).Scroll("10s"));
var o = new ReindexObserver<object>(onError: reindex_Error, onNext: reindex_Next, completed: reindex_Completed);
reindex.Subscribe(o);
}
And I've just found the blog post that showed me how to do it: http://thomasardal.com/elasticsearch-migrations-with-c-and-nest/
Unfortunately the NEST implementation is not quite what I expected. In my opinion it's a bit over-engineered for possibly the most common use case.
Alot of people just want to update their mappings with zero downtime...
In my case - I had already taken care of creating the index with all its settings and mappings, but NEST insists that it must create a new index when reindexing. That among many other things. Too many other things.
I found it much less complicated to just implement directly - since NEST already has Search, Scroll, and Bulk methods. (this is adopted from NEST's implementation):
// Assuming you have already created and setup the index yourself
public void Reindex(ElasticClient client, string aliasName, string currentIndexName, string nextIndexName)
{
Console.WriteLine("Reindexing documents to new index...");
var searchResult = client.Search<object>(s => s.Index(currentIndexName).AllTypes().From(0).Size(100).Query(q => q.MatchAll()).SearchType(SearchType.Scan).Scroll("2m"));
if (searchResult.Total <= 0)
{
Console.WriteLine("Existing index has no documents, nothing to reindex.");
}
else
{
var page = 0;
IBulkResponse bulkResponse = null;
do
{
var result = searchResult;
searchResult = client.Scroll<object>(s => s.Scroll("2m").ScrollId(result.ScrollId));
if (searchResult.Documents != null && searchResult.Documents.Any())
{
searchResult.ThrowOnError("reindex scroll " + page);
bulkResponse = client.Bulk(b =>
{
foreach (var hit in searchResult.Hits)
{
b.Index<object>(bi => bi.Document(hit.Source).Type(hit.Type).Index(nextIndexName).Id(hit.Id));
}
return b;
}).ThrowOnError("reindex page " + page);
Console.WriteLine("Reindexing progress: " + (page + 1) * 100);
}
++page;
}
while (searchResult.IsValid && bulkResponse != null && bulkResponse.IsValid && searchResult.Documents != null && searchResult.Documents.Any());
Console.WriteLine("Reindexing complete!");
}
Console.WriteLine("Updating alias to point to new index...");
client.Alias(a => a
.Add(aa => aa.Alias(aliasName).Index(nextIndexName))
.Remove(aa => aa.Alias(aliasName).Index(currentIndexName)));
// TODO: Don't forget to delete the old index if you want
}
And the ThrowOnError extension method in case you want it:
public static T ThrowOnError<T>(this T response, string actionDescription = null) where T : IResponse
{
if (!response.IsValid)
{
throw new CustomExceptionOfYourChoice(actionDescription == null ? string.Empty : "Failed to " + actionDescription + ": " + response.ServerError.Error);
}
return response;
}
I second Ben Wilde's answer above. Better to have full control over index creation and the re-index process.
What's missing from Ben's code is support for parent/child relationship. Here is my code to fix that:
Replace the following lines:
foreach (var hit in searchResult.Hits)
{
b.Index<object>(bi => bi.Document(hit.Source).Type(hit.Type).Index(nextIndexName).Id(hit.Id));
}
With this:
foreach (var hit in searchResult.Hits)
{
var jo = hit.Source as JObject;
JToken jt;
if(jo != null && jo.TryGetValue("parentId", out jt))
{
// Document is child-document => add parent reference
string parentId = (string)jt;
b.Index<object>(bi => bi.Document(hit.Source).Type(hit.Type).Index(nextIndexName).Id(hit.Id).Parent(parentId));
}
else
{
b.Index<object>(bi => bi.Document(hit.Source).Type(hit.Type).Index(nextIndexName).Id(hit.Id));
}
}
gridview_new is a form class
private checkNulls[] CheckNulls()
{
checkNulls Cntrl;
checkNulls[] cntrlsToupdate = new checkNulls[15];
using (gridview_new IterateThroughCntrls = new gridview_new())
{
for (int i = 5; i < 18; i++)
{
var getCntrl =
IterateThroughCntrls.Controls.Cast<Control>().Where(x => x.TabIndex == i).SingleOrDefault();
if (!(getCntrl.Text == ""))
{
Cntrl = (checkNulls)(i);
cntrlsToupdate[i - 5] = Cntrl;
}
}
}
return cntrlsToupdate;
}
Get Control is getting a null value even though there is a control at tab-index 5.
First, ...OrDefault returns the default value for a given type, in case of reference types (like Control) you get null. So then you could simply check that:
if(getCntrl != null)
{
// safe....
}
If you want the text of the control and "" as default if the Where has found no matching controls, use Select + DefaultIfEmpty:
string getCntrlText = IterateThroughCntrls.Controls.Cast<Control>()
.Where(x => x.TabIndex == i)
.Select(ctrl => ctrl.Text)
.DefaultIfEmpty("")
.Single();
Note that i've used Single since i've provided a default value.
Note that Single... throws an exception (as opposed to First...) if multiple items match the predicate. Normally it's used with key properties where it should be impossible to find multiple elments. So First(or FirstOrDefault) seems to be more appropriate here.
I was trying to implement the jQgrid using MvcjQgrid and i got this exception.
System.NotSupportedException was unhandled by user code
Message=The method 'Skip' is only supported for sorted input in LINQ to Entities. The method 'OrderBy' must be called before the method 'Skip'.
Though OrdeyBy is used before Skip method why it is generating the exception? How can it be solved?
I encountered the exception in the controller:
public ActionResult GridDataBasic(GridSettings gridSettings)
{
var jobdescription = sm.GetJobDescription(gridSettings);
var totalJobDescription = sm.CountJobDescription(gridSettings);
var jsonData = new
{
total = totalJobDescription / gridSettings.PageSize + 1,
page = gridSettings.PageIndex,
records = totalJobDescription,
rows = (
from j in jobdescription
select new
{
id = j.JobDescriptionID,
cell = new[]
{
j.JobDescriptionID.ToString(),
j.JobTitle,
j.JobType.JobTypeName,
j.JobPriority.JobPriorityName,
j.JobType.Rate.ToString(),
j.CreationDate.ToShortDateString(),
j.JobDeadline.ToShortDateString(),
}
}).ToArray()
};
return Json(jsonData, JsonRequestBehavior.AllowGet);
}
GetJobDescription Method and CountJobDescription Method
public int CountJobDescription(GridSettings gridSettings)
{
var jobdescription = _dataContext.JobDescriptions.AsQueryable();
if (gridSettings.IsSearch)
{
jobdescription = gridSettings.Where.rules.Aggregate(jobdescription, FilterJobDescription);
}
return jobdescription.Count();
}
public IQueryable<JobDescription> GetJobDescription(GridSettings gridSettings)
{
var jobdescription = orderJobDescription(_dataContext.JobDescriptions.AsQueryable(), gridSettings.SortColumn, gridSettings.SortOrder);
if (gridSettings.IsSearch)
{
jobdescription = gridSettings.Where.rules.Aggregate(jobdescription, FilterJobDescription);
}
return jobdescription.Skip((gridSettings.PageIndex - 1) * gridSettings.PageSize).Take(gridSettings.PageSize);
}
And Finally FilterJobDescription and OrderJobDescription
private static IQueryable<JobDescription> FilterJobDescription(IQueryable<JobDescription> jobdescriptions, Rule rule)
{
if (rule.field == "JobDescriptionID")
{
int result;
if (!int.TryParse(rule.data, out result))
return jobdescriptions;
return jobdescriptions.Where(j => j.JobDescriptionID == Convert.ToInt32(rule.data));
}
// Similar Statements
return jobdescriptions;
}
private IQueryable<JobDescription> orderJobDescription(IQueryable<JobDescription> jobdescriptions, string sortColumn, string sortOrder)
{
if (sortColumn == "JobDescriptionID")
return (sortOrder == "desc") ? jobdescriptions.OrderByDescending(j => j.JobDescriptionID) : jobdescriptions.OrderBy(j => j.JobDescriptionID);
return jobdescriptions;
}
The exception means that you always need a sorted input if you apply Skip, also in the case that the user doesn't click on a column to sort by. I could imagine that no sort column is specified when you open the grid view for the first time before the user can even click on a column header. To catch this case I would suggest to define some default sorting that you want when no other sorting criterion is given, for example:
switch (sortColumn)
{
case "JobDescriptionID":
return (sortOrder == "desc")
? jobdescriptions.OrderByDescending(j => j.JobDescriptionID)
: jobdescriptions.OrderBy(j => j.JobDescriptionID);
case "JobDescriptionTitle":
return (sortOrder == "desc")
? jobdescriptions.OrderByDescending(j => j.JobDescriptionTitle)
: jobdescriptions.OrderBy(j => j.JobDescriptionTitle);
// etc.
default:
return jobdescriptions.OrderBy(j => j.JobDescriptionID);
}
Edit
About your follow-up problems according to your comment: You cannot use ToString() in a LINQ to Entities query. And the next problem would be that you cannot create a string array in a query. I would suggest to load the data from the DB with their native types and then convert afterwards to strings (and to the string array) in memory:
rows = (from j in jobdescription
select new
{
JobDescriptionID = j.JobDescriptionID,
JobTitle = j.JobTitle,
JobTypeName = j.JobType.JobTypeName,
JobPriorityName = j.JobPriority.JobPriorityName,
Rate = j.JobType.Rate,
CreationDate = j.CreationDate,
JobDeadline = j.JobDeadline
})
.AsEnumerable() // DB query runs here, the rest is in memory
.Select(a => new
{
id = a.JobDescriptionID,
cell = new[]
{
a.JobDescriptionID.ToString(),
a.JobTitle,
a.JobTypeName,
a.JobPriorityName,
a.Rate.ToString(),
a.CreationDate.ToShortDateString(),
a.JobDeadline.ToShortDateString()
}
})
.ToArray()
I had the same type of problem after sorting using some code from Adam Anderson that accepted a generic sort string in OrderBy.
After getting this excpetion, i did lots of research and found that very clever fix:
var query = SelectOrders(companyNo, sortExpression);
return Queryable.Skip(query, iStartRow).Take(iPageSize).ToList();
Hope that helps !
SP
I would like to optimize following lines of code for Sorting.
public ViewResult Index(string sortorder, int? pagesize, int? page)
{
int pageSize = pagesize ?? 10;
if (Request.HttpMethod != "GET")
{
page = 1;
pageSize = 10;
}
ViewBag.SelectedPageSize = pageSize;
ViewBag.CurrentSort = sortorder;
ViewBag.FirstNameSortParm = String.IsNullOrEmpty(sortorder) ? "FirstName desc" : "";
ViewBag.LastNameSortParm = sortorder == "LastName" ? "LastName desc" : "LastName";
ViewBag.DepNameSortParm = sortorder == "depName" ? "depName desc" : "depName";
var joined = from tm in db.TabMasters select tm;
switch (sortorder)
{
case "FirstName":
joined = joined.OrderBy(m => m.FirstName);
break;
case "FirstName desc":
joined = joined.OrderByDescending(m => m.FirstName);
break;
case "LastName":
joined = joined.OrderBy(m => m.LastName);
break;
case "LastName desc":
joined = joined.OrderByDescending(m => m.LastName);
break;
case "depName":
joined = joined.OrderBy(m => m.depName);
break;
case "depName desc":
joined = joined.OrderByDescending(m => m.depName);
break;
default:
joined = joined.OrderBy(m => m.FirstName);
break;
}
int pageIndex = (page ?? 1) - 1;
int start = (pageIndex * pageSize);
ViewBag.TotalRecord = joined.Count();
ViewBag.StartRecord = start + 1;
ViewBag.EndRecord = ((start + pageSize) >= ViewBag.TotalRecord) ? ViewBag.TotalRecord : (start + pageSize);
return View(joined.ToPagedList(pageIndex, pageSize));
}
Because this is very tedious way if i have more the 10 fields to perform sort.
Thanks,
Imdadhusen
It's a bit vague to me what your actual goal is but for the switch part you could use an extension method as the below.
public static class SortExtensions
{
public static IEnumerable<T> SortByField<T>(this IEnumerable<T> sequence, string sortOrder)
{
var tokens = sortOrder.Trim().Split(' ');
var field = tokens[0];
var direction = tokens.Skip(1).Single().ToLower();
var prop = typeof(T).GetProperty(field);
return direction == "desc"
? sequence.OrderByDescending(m => prop.GetValue(m, null))
: sequence.OrderBy(m => prop.GetValue(m, null));
}
}
It will make a very simplified parsing of the sort order. It puts the responsibility on the calling party which is generally not what you want to do, so you might want some error handling in case the sortorder string does not fulfill the requirements.
from the sortorder string it fetches a name used to identify a property which can be used to fetch the value used for sorting.
you can use it like this:
db.TabMasters.SortByField(sortOrder)
EDIT based on comment:
The line typeof(T).GetProperty(field) is fragile in the absence of any error handling. It relies on the first token to be a name of a public property of the type T. It will return null if the name doesn't match a property. Including if it matches a Field name. A similar function exist for getting a FieldInfo
prop.GetField(field) will return a fieldinfo object of there's a public field with the given name otherwise null. To get the value of a field simply omit the last parameter to the GetValue call.
You should take a look at Linq.DynamicQuery.
There's more info in this blogpost http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx
The library lets you write following code:
var query = northwind.Products
.Where("CategoryID = 3 AND UnitPrice > 3")
.OrderBy("SupplierID");
instead of
var query = from p in northwind.Products
where p.CategoryID == 3 && p.UnitPrice > 3
orderby p.SupplierID
select p;
If you want to add the sortdirection:
var query = northwind.Products.OrderBy("SupplierID Descending");