I have this C# code which is expected to match 2 fields using multi-match Elastic Search type. I am using NEST package.
var response = await _elasticClient.SearchAsync<FileDocument>(
s => s.Query(q => q.MultiMatch(c => c
.Fields(f => f.Field(p => p.FileName).Field(query))
.Fields(f => f.Field(p => p.Metadata).Field(query))
)));
Problem is no matter what text I passed in, it returns all the result. Anything I miss out?
In order to efficiently debug these types of issue, you need to inspect the HTTP request going to Elasticsearch, ultimately your query builder will be converted to search JSON and will be executed against Elasticsearch.
I am not aware of nest but have written the answer for Java code, which prints the Elasticsearch query in JSON format.
Although my guess is that, you are not sending the correct HTTP method which should be POST and you might be sending it with GET, which is causing ES to ignore your search query and return all documents.
Solved after adding .Query(query)
var response = await _elasticClient.SearchAsync<FileDocument>(
s => s.Query(q => q.MultiMatch(c => c
.Fields(f => f.Field(p => p.FileName).Field(p=>p.Metadata))
.Query(query)
))
);
Reference - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/multi-match-usage.html
Related
I have a combination of two queries with Elasticsearch and nest, the first one is a full-text search for a specific term and the second one is to filter or query another field which is file-path but it should be for many files paths and the path could be part or full path, I can query one file-path but I couldn't manage to do it for many file paths, any suggestion?
Search<SearchResults>(s => s
.Query(q => q
.Match(m => m.Field(f => f.Description).Query("Search_term"))
&& q
.Prefix(t => t.Field(f => f.FilePath).Value("file_Path"))
)
);
For searching for more than one path you can use bool Query in elasticsearch and then use Should Occur to search like logical OR, so you code should look like this:
Search<SearchResults>(s => s
.Query(q => q.
Bool(b => b
.Should(
bs => bs.Wildcard(p => p.FilePath, "*file_Pathfile_Path*"),
bs => bs.Wildcard(p => p.FilePath, "*file_Pathfile_Path*"),
....
))
&& q.Match(m => m.Field(f => f.description).Query("Search_term")
)));
Also you should use WildCard Query to get result for paths that could be part or full path. For more information check ES offical documentation about WildQuery and Bool Query below:
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html
https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/bool-queries.html
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-wildcard-query.html
Using Nest for ElasticSearch. enter image description here
part of the index looks like this "startdate" : "2018-11-01T00:00:00", "enddate" : "2018-11-01T00:00:00"
My fluentAPI query looks like below
var response = client.Search<ActivityReportsSearchViewModel>(x => x
.Size(500)
.Query(q => q.Bool(b => b.Must
(mu => mu.MultiMatch(m => m
.Fields(n => n.Fields(f => f.User, f => f.Activity
, f => f.County, f => f.Constituency, f => f.Country
, f => f.ActivityRequestDescription, f => f.ActivityDescription,
f => f.LessonsLearnt, f => f.Challenges, f => f.Recommendatinons, f => f.User
, f => f.Venue, f => f.Division))
.Query(**search**)))
.**Filter**(fi => fi
.DateRange(r => r
.Field(f => f.StartDate)
.GreaterThan("2018-08-20")
.LessThanOrEquals(DateMath.Now)
)))));
The query runs well and return results WITHOUT filter. After I add filter as shown above the query returns no results. Can anyone point the problem with the fluentAPI code
You can check response.IsValid to determine whether the request is valid for the API call to Elasticsearch.
The response.DebugInformation will provide more details about the API call in a friendly to read format, including if there is an error. If it is an error in Elasticsearch, response.ServerError will be populated with the details.
Without knowing further details about the response, I suspect the problem is an issue with parsing the string "2018-08-20" as a date on the Elasticsearch server side. Elasticsearch will attempt to parse the string into a date using the format specified in the date field mapping for StartDate, and if the date in the search request is in a different format, you can supply the format in the DateRange query with .Format(...).
Assuming that the document was originally indexed with the .NET client and the default DateTime serialization was used, you should just be able to use an instance of DateTime to supply the needed date
.DateRange(r => r
.Field(f => f.StartDate)
.GreaterThan(new DateTime(2018, 8, 20))
.LessThanOrEquals(DateMath.Now)
)
which will serialize as
"2018-08-20T00:00:00"
which is the same format as the field in the index. Alternatively, you can pass the string in this format.
I created an autocomplete filter with ElasticSearch using NEST API. I cant seem to get the word joiner to work.
So basically if I search for something like Transhex i also want to be able to return Trans Hex
My Index looks as follow...I think the WordDelimiter filter might be wrong.
Also, I followed the following article Link. They use the low-level API so it is possible that I am doing it completely wrong using the NEST API
var response = this.Client.CreateIndex(
"company-index",
index => index.Mappings(
ms => ms.Map<CompanyDocument>(m => m.Properties(p => p
.Text(t => t.Name(n => n.CompanyName).Analyzer("auto-complete")
.Fields(ff => ff.Keyword(k => k.Name("keyword")))))))
.Settings(f => f.Analysis(
analysis => analysis
.Analyzers(analyzers => analyzers
.Custom("auto-complete", a => a.Tokenizer("standard").Filters("lowercase", "word-joiner-filter", "auto-complete-filter")))
.TokenFilters(tokenFilter => tokenFilter
.WordDelimiter("word-joiner-filter", t => t.CatenateAll())
.EdgeNGram("auto-complete-filter", t => t.MinGram(3).MaxGram(30))))));
UPDATE
So I updated the Analyzer to look as follows, noticed that I updated the Analyzer from standard to keyword.
var response = this.Client.CreateIndex(
this.indexName,
index => index.Mappings(
ms => ms.Map<CompanyDocument>(m => m.Properties(p => p
.Text(t => t.Name(n => n.CompanyName).Analyzer("auto-complete")
.Fields(ff => ff.Keyword(k => k.Name("keyword")))))))
.Settings(f => f.Analysis(
analysis => analysis
.Analyzers(analyzers => analyzers
.Custom("auto-complete", a => a.Tokenizer("keyword").Filters("lowercase", "word-joiner-filter", "auto-complete-filter")))
.TokenFilters(tokenFilter => tokenFilter
.WordDelimiter("word-joiner-filter", t => t.CatenateAll())
.EdgeNGram("auto-complete-filter", t => t.MinGram(1).MaxGram(20))))));
The Results will look as follows
Search Keyword : perfect pools
Results
perfect pools -> This is the correct one at the top
EXCLUSIVE POOLS
Perfect Painters
Search Keyword : perfectpools Or PerfectPools
Results
Perfect Hideaways (Pty) Ltd -> this is the wrong one i would like to display perfect pools
PERFORMANTA APAC PTY LTD
Perfect Laser Technologies (PTY) LTD
Use Keyword tokenizer. The standard tokenizer will split the word in 2 tokens, then apply the filters on them.
UPDATE:
I used a search like this one and seems ok.
var searchResult = EsClient.Search<CompanyDocument>(q => q
.Index("test_index")
.Type("companydocument")
.TrackScores(true)
.Query(qq =>
{
QueryContainer queryContainer = null;
queryContainer = qq.QueryString(qs => qs.Fields(fs => fs.Field(f => f.CompanyName)).Query("perfectpools").DefaultOperator(Operator.And).Analyzer("auto-complete"));
return queryContainer;
})
.Sort(sort => sort.Descending(SortSpecialField.Score))
.Take(10)
);
According to this link, both scan and count are deprecated.
I am trying to change my queries to reflect this. So the count change is easy, just removing the search type and adding size=0 to the request, however, I am not 100% on the scan change.
Currently I have this query:
var result = ElasticClient.Search<Product>(s => s
.From(0)
.Size(10)
.SearchType(SearchType.Scan)
.Scroll("4s")
.Query
(qu =>
qu.Filtered
(fil =>
fil.Filter
(f =>
f.Bool(b => b.Must(m => m.Term("filedName", "abc")))))));
Am I correct in my understanding that all I need to change is remove the searchtype and add a sort? I.e:
var result = ElasticClient.Search<Product>(s => s
.From(0)
.Size(10)
.Scroll("4s")
.Sort(x => x.OnField("_doc"))
.Query
(qu =>
qu.Filtered
(fil =>
fil.Filter
(f => f.Bool(b => b.Must(m => m.Term("filedName", "abc")))))));
I have seen a enum SortSpecialField here, but I am not sure how to actually use this in the sort parameter.
You're correct in your understanding that the change (as you document in your question) to sort by _doc will replace the deprecated Scan searchtype. The SortSpecialField enum is just syntax sugar for sorting by _doc. If you prefer to use it, in NEST 2.0 [only], you can do this:
ElasticClient.Search<Product>(s => s
.From(0)
.Size(10)
.Scroll("4s")
.Sort(x => x.Ascending(SortSpecialField.DocumentIndexOrder))
...
I have got unique document ids (across all types) and I would like to check which document already exists in elasticsearch index. I try to search
var duplicateCheck = _elasticClient
.MultiGet(m => m.GetMany<object>(notices.Select(s => s.Id)).Fields("Id"));
but it returns wrong result - every document has set found property to false.
update
there is workaround here
var exisitngDocIds = _elasticClient.Search<ZPBase>(s => s
.AllTypes()
.Query(q => q.Ids(notices.Select(z=>z.Id)))
.Fields("Id")
.Take(notices.Count)
);
notices = notices.Where(q => !exisitngDocIds.Hits.Any(s => s.Id == q.Id)).ToList();
From the Multi Get API documentation I realized that you can use something similar to the following code to solve your problem:
var response = _elasticClient.MultiGet(m => m
.Index(MyIndex)
.Type("")
.GetMany<ZPBase>(noticeIds));
Note the empty string passed as the Type.