I want to do sorting on nested field quality score where the trade name matches the search term.
My code is as below which works fine when search term is one word. It fails to sort when search term is a phrase. How can I solve this? :
sortDesc = _sortTerm switch
{
"Quality" => new SortDescriptor<Provider>().Field(so => so
.Field(f => f.Metrics.First().Data.Trades.First().QualityScore)
.Order(SortOrder.Descending)
.Nested(n => n
.Path(p => p.Metrics)
.Filter(q => q.Match(m => m
.Field(f => f.Metrics.First().Data.Trades.First().Name.Suffix("keyword"))
.Query(_searchTerm?.ToLower()))))
)
}
Thank you so much in advance.
If you want to sort on the nested numeric field, based on the match phrase results, you need to use the query as shown below :
Adding a working example, to replicate your requirements
Index Mapping:
{
"mappings": {
"properties": {
"Metrics": {
"type": "nested"
}
}
}
}
Index API
{
"Metrics": {
"FIELDNAME": "hello worlds",
"age": 2
}
}
{
"Metrics": {
"FIELDNAME": "hello worlds",
"age": 3
}
}
{
"Metrics": {
"FIELDNAME": "hello world",
"age": 1
}
}
Search Query:
{
"sort": [
{
"Metrics.age": {
"order": "desc",
"nested": {
"path": "Metrics"
}
}
}
],
"query": {
"nested": {
"path": "Metrics",
"query": {
"match_phrase": {
"Metrics.FIELDNAME": "SEARCH PHRASE"
}
}
}
}
}
Related
I have an Elastic index that contains objects structured like this:
{
dogs: [
{
name: 'wiener dog',
id: 2,
cats: [
{
name: 'mean cat',
id: 5,
},
...
],
},
...
],
...
}
My question is: How do I search against this index for all documents that include a particular id in cats? A single match is fine.
What I have tried: I have tried many different queries, including nesting on dogs, and nesting on both dogs and cats. I have tried accessing the property directly via dogs.cats.id, and all combinations of the above. Here is an example in NEST:
query &= mst.Nested(n => n
.Path("dogs")
.Query(q => q
.Nested(n => n
.Path("dogs.cats")
.Query(q => q
.Terms(t => t
.Field("dogs.cats.id")
.Terms(catIds.ToList())
)
)
)
)
);
I have also tried with a single Nested with Field set to cats.id with no luck.
Any help here would be greatly appreciated. Changing the data structure at this point would be a much larger effort, and would be avoided if possible. Thanks!
From your information, I assume that the use of NestedQuery is ideal.
PUT bug_reports
{
"mappings": {
"properties": {
"dogs": {
"type": "nested",
"properties": {
"cats": {
"type": "nested"
}
}
}
}
}
}
POST bug_reports/_doc/1
{
"dogs": [
{
"name": "wiener dog",
"id": 1,
"cats": [
{
"name":"red cat",
"id": 4
},
{
"name":"mean cat",
"id": 5
}
]
}
]
}
POST bug_reports/_doc/2
{
"dogs": [
{
"name": "none dog",
"id": 2,
"cats": [
{
"name":"mean cat",
"id": 5
}
]
}
]
}
GET bug_reports/_search?filter_path=hits.hits
{
"query": {
"nested": {
"path": "dogs",
"query": {
"bool": {
"must": [
{
"nested": {
"path": "dogs.cats",
"query": {
"terms": {
"dogs.cats.id": [
4
]
}
}
}
},
{
"nested": {
"path": "dogs.cats",
"query": {
"terms": {
"dogs.cats.id": [
5
]
}
}
}
}
]
}
}
}
}
}
We need to get the data based on multiple values.
So I am trying to use terms query in elastic search for modelNumber field.
But it is not working as expected.can anyone let me know what is wrong with the query.
POST index_name/_Search
{
"query": {
"bool": {
"must": [
{
"terms": {
"modelNumber": [
"test 1234rthg-1234-1234512-2345",
"testMode11l-123-rtyu-xyz11"
]
}
},
{
"terms": {
"userId": [
"123",
"VALUE2"
]
}
}
]
}
}
}
Terms query returns documents that contain one or more exact terms in
a provided field.
If you have not explicitly defined any index mapping, then you need to add .keyword to the modelNumber field. This uses the keyword analyzer instead of the standard analyzer (notice the ".keyword" after modelNumber field).
{
"query": {
"bool": {
"must": [
{
"terms": {
"modelNumber.keyword": [ // note this
"test 1234rthg-1234-1234512-2345",
"testMode11l-123-rtyu-xyz11"
]
}
},
{
"terms": {
"userId": [
"123",
"VALUE2"
]
}
}
]
}
}
}
OR you need to modify the mapping of modelNUmber field as -
{
"mappings": {
"properties": {
"modelNumber": {
"type": "keyword"
}
}
}
}
I have the following issue in case of a full-text search in Elasticsearch. I would like to search for all indexed attributes. However, one of my Project attributes is a very complex array of hashes/objects:
[
{
"title": "Group 1 title",
"name": "Group 1 name",
"id": "group_1_id",
"items": [
{
"pos": "1",
"title": "Position 1 title"
},
{
"pos": "1.1",
"title": "Position 1.1 title",
"description": "<p>description</p>",
"extra_description": {
"rotation": "2 years",
"amount": "1.947m²"
},
"inputs": {
"unit_price": true,
"total_net": true
},
"additional_inputs": [
{
"name": "additonal_input_name",
"label": "Additional input label:",
"placeholder": "Additional input placeholder",
"description": "Additional input description",
"type": "text"
}
]
}
]
}
]
My mappings look like this:
{:title=>{:type=>"text", :analyzer=>"english"},
:description=>{:type=>"text", :analyzer=>"english"},
:location=>{:type=>"keyword"},
:company=>{:type=>"keyword"},
:created_at=>{:type=>"date"},
:due_date=>{:type=>"date"},
:specification=>
{:type=>:nested,
:properties=>
{:id=>{:type=>"keyword"},
:title=>{:type=>"text"},
:items=>
{:type=>:nested,
:properties=>
{:pos=>{:type=>"keyword"},
:title=>{:type=>"text"},
:description=>{:type=>"text", :analyzer=>"english"},
:extra_description=>{:type=>:nested, :properties=>{:rotation=>{:type=>"keyword"}, :amount=>{:type=>"keyword"}}},
:additional_inputs=>
{:type=>:nested,
:properties=>
{:label=>{:type=>"keyword"},
:placeholder=>{:type=>"text"},
:description=>{:type=>"text"},
:type=>{:type=>"keyword"},
:name=>{:type=>"keyword"}
}
}
}
}
}
}
}
The question is, how to properly seek through it? For no nested attributes, it works as a charm, but for instance, I would like to seek by title in the specification, no result is returned. I tried both:
query:
{ nested:
{
multi_match: {
query: keyword,
fields: ['title', 'description', 'company', 'location', 'specification']
}
}
}
Or
{
nested: {
path: 'specification',
query: {
multi_match: {
query: keyword
}
}
}
}
Without any result.
Edit:
It's with elasticsearch-ruby for Ruby.
I am trying to query by: MODEL_NAME.all.search(query: with_specification("Group 1 title")) where with_specification is:
def with_specification(keyword)
{
bool: {
should: [
{
nested: {
path: 'specification',
query: {
bool: {
should: [
{
match: {
'specification.title': keyword,
}
},
{
multi_match: {
query: keyword,
fields: [
'specification.title',
'specification.id'
]
}
},
{
nested: {
path: 'specification.items',
query: {
match: {
'specification.items.title': keyword,
}
}
}
}
]
}
}
}
}
]
}
}
end
Querying on multi-level nested documents must follow a certain schema.
You cannot multi-match on nested & non-nested fields at the same time and/or query on nested fields under different paths.
You can wrap your queries in a bool-should but keep the 2 rules above in mind:
GET your_index/_search
{
"query": {
"bool": {
"should": [
{
"nested": {
"path": "specification",
"query": {
"bool": {
"should": [
{
"match": {
"specification.title": "TEXT" <-- standalone match
}
},
{
"multi_match": { <-- multi-match but 1st level path
"query": "TEXT",
"fields": [
"specification.title",
"specification.id"
]
}
},
{
"nested": {
"path": "specification.items", <-- 2nd level path
"query": {
"match": {
"specification.items.title": "TEXT"
}
}
}
}
]
}
}
}
}
]
}
}
}
I am currently trying to migrate a solr-based application to elasticsearch.
I have this lucene query:
((
name:(+foo +bar)
OR info:(+foo +bar)
)) AND state:(1) AND (has_image:(0) OR has_image:(1)^100)
As far as I understand this is a combination of must clauses combined with boolean OR:
Get all documents containing (foo AND bar in name) OR (foo AND bar in info). After that filter results by condition state=1 and boost documents that have an image.
I have been trying to use a bool query with must but I am failing to get boolean OR into must clauses. Here is what I have:
GET /test/object/_search
{
"from": 0,
"size": 20,
"sort": {
"_score": "desc"
},
"query": {
"bool": {
"must": [
{
"match": {
"name": "foo"
}
},
{
"match": {
"name": "bar"
}
}
],
"must_not": [],
"should": [
{
"match": {
"has_image": {
"query": 1,
"boost": 100
}
}
}
]
}
}
}
As you can see, must conditions for info are missing.
** UPDATE **
I have updated my elasticsearch query and got rid of that function score. My base problem still exists.
OR is spelled should
AND is spelled must
NOR is spelled should_not
Example:
You want to see all the items that are (round AND (red OR blue)):
{
"query": {
"bool": {
"must": [
{
"term": {"shape": "round"}
},
{
"bool": {
"should": [
{"term": {"color": "red"}},
{"term": {"color": "blue"}}
]
}
}
]
}
}
}
You can also do more complex versions of OR, for example, if you want to match at least 3 out of 5, you can specify 5 options under "should" and set a "minimum_should" of 3.
Thanks to Glen Thompson and Sebastialonso for finding where my nesting wasn't quite right before.
Thanks also to Fatmajk for pointing out that "term" becomes a "match" in ElasticSearch Version 6.
I finally managed to create a query that does exactly what i wanted to have:
A filtered nested boolean query.
I am not sure why this is not documented. Maybe someone here can tell me?
Here is the query:
GET /test/object/_search
{
"from": 0,
"size": 20,
"sort": {
"_score": "desc"
},
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"state": 1
}
}
]
}
},
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"match": {
"name": "foo"
}
},
{
"match": {
"name": "bar"
}
}
],
"should": [
{
"match": {
"has_image": {
"query": 1,
"boost": 100
}
}
}
]
}
},
{
"bool": {
"must": [
{
"match": {
"info": "foo"
}
},
{
"match": {
"info": "bar"
}
}
],
"should": [
{
"match": {
"has_image": {
"query": 1,
"boost": 100
}
}
}
]
}
}
],
"minimum_should_match": 1
}
}
}
}
}
In pseudo-SQL:
SELECT * FROM /test/object
WHERE
((name=foo AND name=bar) OR (info=foo AND info=bar))
AND state=1
Please keep in mind that it depends on your document field analysis and mappings how name=foo is internally handled. This can vary from a fuzzy to strict behavior.
"minimum_should_match": 1 says, that at least one of the should statements must be true.
This statements means that whenever there is a document in the resultset that contains has_image:1 it is boosted by factor 100. This changes result ordering.
"should": [
{
"match": {
"has_image": {
"query": 1,
"boost": 100
}
}
}
]
Have fun guys :)
This is how you can nest multiple bool queries in one outer bool query
this using Kibana,
bool indicates we are using boolean
must is for AND
should is for OR
GET my_inedx/my_type/_search
{
"query" : {
"bool": { //bool indicates we are using boolean operator
"must" : [ //must is for **AND**
{
"match" : {
"description" : "some text"
}
},
{
"match" :{
"type" : "some Type"
}
},
{
"bool" : { //here its a nested boolean query
"should" : [ //should is for **OR**
{
"match" : {
//ur query
}
},
{
"match" : {}
}
]
}
}
]
}
}
}
This is how you can nest a query in ES
There are more types in "bool" like,
Filter
must_not
I recently had to solve this problem too, and after a LOT of trial and error I came up with this (in PHP, but maps directly to the DSL):
'query' => [
'bool' => [
'should' => [
['prefix' => ['name_first' => $query]],
['prefix' => ['name_last' => $query]],
['prefix' => ['phone' => $query]],
['prefix' => ['email' => $query]],
[
'multi_match' => [
'query' => $query,
'type' => 'cross_fields',
'operator' => 'and',
'fields' => ['name_first', 'name_last']
]
]
],
'minimum_should_match' => 1,
'filter' => [
['term' => ['state' => 'active']],
['term' => ['company_id' => $companyId]]
]
]
]
Which maps to something like this in SQL:
SELECT * from <index>
WHERE (
name_first LIKE '<query>%' OR
name_last LIKE '<query>%' OR
phone LIKE '<query>%' OR
email LIKE '<query>%'
)
AND state = 'active'
AND company_id = <query>
The key in all this is the minimum_should_match setting. Without this the filter totally overrides the should.
Hope this helps someone!
If you were using Solr's default or Lucene query parser, you can pretty much always put it into a query string query:
POST test/_search
{
"query": {
"query_string": {
"query": "(( name:(+foo +bar) OR info:(+foo +bar) )) AND state:(1) AND (has_image:(0) OR has_image:(1)^100)"
}
}
}
That said, you may want to use a boolean query, like the one you already posted, or even a combination of the two.
$filterQuery = $this->queryFactory->create(QueryInterface::TYPE_BOOL, ['must' => $queries,'should'=>$queriesGeo]);
In must you need to add the query condition array which you want to work with AND and in should you need to add the query condition which you want to work with OR.
You can check this: https://github.com/Smile-SA/elasticsuite/issues/972
Say my user provides a list of search terms which I've collected into an array/list, and now I want to combine those OR-wise into a NEST query using MatchPhrase. How would I do that? The code for a (single) search term would look something like this:
var search = client.Search<ElasticRequirement>(s => s
.Query(q =>
q.MatchPhrase(m => m.OnField(f => f.Title).Query(term.ToLower()).Slop(slop))
|| q.MatchPhrase(m => m.OnField(f => f.Description).Query(text).Slop(slop))
)
.LowercaseExpandedTerms()
.Explain()
.Query(q => q.Fuzzy(f => f.PrefixLength(1).OnField(c => c.Title).OnField(c => c.Description)))
);
This is fine, but I need to apply that same MatchPhrase filter once for each provided search term. Any help much appreciated.
You can use bool should expressions to build your query dynamically. I'll provide the complete solution below. Call BuildQuery() method with appropriate parameters.
ISearchResponse<ElasticRequirement> BuildQuery(IElasticClient client, IEnumerable<string> terms, int slop)
{
return client.Search<ElasticRequirement>(s => s
.Query(q => q
.Bool(b => b
.Should(terms.Select(t => BuildPhraseQueryContainer(q, t, slop)).ToArray())))
.LowercaseExpandedTerms()
.Explain()
.Query(q => q.Fuzzy(f => f.PrefixLength(1).OnField(c => c.Title).OnField(c => c.Description))));
}
QueryContainer BuildPhraseQueryContainer(QueryDescriptor<ElasticRequirement> qd, string term, int slop)
{
return qd.MatchPhrase(m => m.OnField(f => f.Title).Query(term.ToLower()).Slop(slop)) ||
qd.MatchPhrase(m => m.OnField(f => f.Description).Query(term.ToLower()).Slop(slop));
}
For terms = {"term1", "term2", "term3"} and slop = 0, the Elasticsearch search JSON command that will get built by my code is as under:
{
"explain": true,
"query": {
"bool": {
"should": [
{
"bool": {
"should": [
{
"match": {
"title": {
"type": "phrase",
"query": "term1",
"slop": 0
}
}
},
{
"match": {
"description": {
"type": "phrase",
"query": "term1",
"slop": 0
}
}
}
]
}
},
{
"bool": {
"should": [
{
"match": {
"title": {
"type": "phrase",
"query": "term2",
"slop": 0
}
}
},
{
"match": {
"description": {
"type": "phrase",
"query": "term2",
"slop": 0
}
}
}
]
}
},
{
"bool": {
"should": [
{
"match": {
"title": {
"type": "phrase",
"query": "term3",
"slop": 0
}
}
},
{
"match": {
"description": {
"type": "phrase",
"query": "term3",
"slop": 0
}
}
}
]
}
}
]
}
}
}
You can tweak this code such that all the match commands are under the same should node. I'll leave that up to you to figure out :)