I am writing some queries in my ElasticSearch project that can be filtered by Date. I have written them like this:
var searchResponse = client.Search<mdl.Event>(s => s
.Query(q => q
.QueryString(qs => qs
.Query(search.Space))
&& q
.DateRange(r => r
.Field(f => f.CreatedTimeStamp)
.GreaterThanOrEquals(search.From)
.LessThanOrEquals(search.To))));
However, search.From and search.To are optional inputs, so they might turn out to be null. In the event that they are null, does this break the query? Or will it continue as if the DateRange part of the query is not included?
Nest queries are condition less. If input is determined to be null or empty string then the query will be omitted from the request.
So you don't need to check whether each filter property is null , NEST will perform this filtering by default.
In your case if search.From and search.To are null then range check will be removed from final query
Related
I have this C# code which is expected to match 2 fields using multi-match Elastic Search type. I am using NEST package.
var response = await _elasticClient.SearchAsync<FileDocument>(
s => s.Query(q => q.MultiMatch(c => c
.Fields(f => f.Field(p => p.FileName).Field(query))
.Fields(f => f.Field(p => p.Metadata).Field(query))
)));
Problem is no matter what text I passed in, it returns all the result. Anything I miss out?
In order to efficiently debug these types of issue, you need to inspect the HTTP request going to Elasticsearch, ultimately your query builder will be converted to search JSON and will be executed against Elasticsearch.
I am not aware of nest but have written the answer for Java code, which prints the Elasticsearch query in JSON format.
Although my guess is that, you are not sending the correct HTTP method which should be POST and you might be sending it with GET, which is causing ES to ignore your search query and return all documents.
Solved after adding .Query(query)
var response = await _elasticClient.SearchAsync<FileDocument>(
s => s.Query(q => q.MultiMatch(c => c
.Fields(f => f.Field(p => p.FileName).Field(p=>p.Metadata))
.Query(query)
))
);
Reference - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/multi-match-usage.html
Using Nest for ElasticSearch. enter image description here
part of the index looks like this "startdate" : "2018-11-01T00:00:00", "enddate" : "2018-11-01T00:00:00"
My fluentAPI query looks like below
var response = client.Search<ActivityReportsSearchViewModel>(x => x
.Size(500)
.Query(q => q.Bool(b => b.Must
(mu => mu.MultiMatch(m => m
.Fields(n => n.Fields(f => f.User, f => f.Activity
, f => f.County, f => f.Constituency, f => f.Country
, f => f.ActivityRequestDescription, f => f.ActivityDescription,
f => f.LessonsLearnt, f => f.Challenges, f => f.Recommendatinons, f => f.User
, f => f.Venue, f => f.Division))
.Query(**search**)))
.**Filter**(fi => fi
.DateRange(r => r
.Field(f => f.StartDate)
.GreaterThan("2018-08-20")
.LessThanOrEquals(DateMath.Now)
)))));
The query runs well and return results WITHOUT filter. After I add filter as shown above the query returns no results. Can anyone point the problem with the fluentAPI code
You can check response.IsValid to determine whether the request is valid for the API call to Elasticsearch.
The response.DebugInformation will provide more details about the API call in a friendly to read format, including if there is an error. If it is an error in Elasticsearch, response.ServerError will be populated with the details.
Without knowing further details about the response, I suspect the problem is an issue with parsing the string "2018-08-20" as a date on the Elasticsearch server side. Elasticsearch will attempt to parse the string into a date using the format specified in the date field mapping for StartDate, and if the date in the search request is in a different format, you can supply the format in the DateRange query with .Format(...).
Assuming that the document was originally indexed with the .NET client and the default DateTime serialization was used, you should just be able to use an instance of DateTime to supply the needed date
.DateRange(r => r
.Field(f => f.StartDate)
.GreaterThan(new DateTime(2018, 8, 20))
.LessThanOrEquals(DateMath.Now)
)
which will serialize as
"2018-08-20T00:00:00"
which is the same format as the field in the index. Alternatively, you can pass the string in this format.
Below is the NEST query and aggregations:
Func<QueryContainerDescriptor<ConferenceWrapper>, QueryContainer> query =
q =>
q.Term(p => p.type, "conference")
// && q.Term(p => p.conference.isWaitingAreaCall, true)
&& q.Range(d => d.Field("conference.lengthSeconds").GreaterThanOrEquals(minSeconds))
&& q.DateRange(qd => qd.Field("conference.firstCallerStart").GreaterThanOrEquals(from.ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:ss.fffZ")))
&& q.DateRange(qd => qd.Field("conference.firstCallerStart").LessThanOrEquals(to.ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:ss.fffZ")));
Func<AggregationContainerDescriptor<ConferenceWrapper>, IAggregationContainer> waitingArea =
a => a
.Terms("both", t => t
.Field(p => p.conference.orgNetworkId) // seems ignore this field
.Field(p => p.conference.isWaitingAreaCall)
// .Field(new Field( p => p.conference.orgNetworkId + "-ggg-" + p.conference.networkId)
.Size(300)
.Aggregations(a2 => a2.Sum("sum-length", d2 => d2.Field("conference.lengthSeconds"))));
I have called .Field(p => p.conference.orgNetworkId) followed by .Field(p => p.conference.isWaitingAreaCall) But it seems the NEST client tries to ignore the first field expression.
Is is possible to have multiple fields to be the terms group by?
Elasticsearch doesn't support a terms aggregation on multiple fields directly; the calls to .Field(...) within NEST are assignative rather than additive, so the last call will overwrite any previously set values.
In order to aggregate on multiple fields, you can either
Create a composite field at index time that incorporates the values that you wish to aggregate on
or
Use a Script to generate the terms on which to aggregate at query time, by combining the two field values.
The performance of the first option will be better than the second.
Using Query string OnFieldWithBoosts added different fields need to apply filter, string fields records are working fine.
For id field when I include .Add(id,2) it does not return ID based result.
When I use term then ID fields records working fine.
Now in the query section,
I have used OR condition, so when first condition satisfies, it does not check second one.
If I user AND condition, then it checks for both the condition matches.
But I need first query results and second query results concat into one result
CODE:
var result = client.Search<dynamic>(q => q
.Indices("test")
.Types("user")
.From(1)
.Size(10)
.MinScore(1.0)
.Fields("id", "createddate", "email", "modifieddate", "name", "companyname") // Result set Fields Fields
.Query(q1 =>
{
qq = (q1.ConstantScore(a => a.Filter(b => b.Term("id", searchKeyword))))
|| q1.QueryString(qs => qs.Query(searchKeyword).OnFieldsWithBoost(a => a.Add("notes",3).Add("email", 2).Add("name", 2)));
return qq;
})
);
I have got unique document ids (across all types) and I would like to check which document already exists in elasticsearch index. I try to search
var duplicateCheck = _elasticClient
.MultiGet(m => m.GetMany<object>(notices.Select(s => s.Id)).Fields("Id"));
but it returns wrong result - every document has set found property to false.
update
there is workaround here
var exisitngDocIds = _elasticClient.Search<ZPBase>(s => s
.AllTypes()
.Query(q => q.Ids(notices.Select(z=>z.Id)))
.Fields("Id")
.Take(notices.Count)
);
notices = notices.Where(q => !exisitngDocIds.Hits.Any(s => s.Id == q.Id)).ToList();
From the Multi Get API documentation I realized that you can use something similar to the following code to solve your problem:
var response = _elasticClient.MultiGet(m => m
.Index(MyIndex)
.Type("")
.GetMany<ZPBase>(noticeIds));
Note the empty string passed as the Type.