Elasticsearch custom function score - elasticsearch

i am trying to do a search with custom functions to modify document score.
I have a mapping with specialities stored inside a hospital and every speciality has a priority with it:
Something like:
hospital:{
name: 'Fortis',
specialities: [
{
name: 'Cardiology',
priority: 10
},
{
name: 'Oncology',
priority: 15
}
]
}
Now i have a function score :
functions: [{
filter: {terms: {'specialities.name' => params[:search_text]}},
script_score: {script: "_score * doc['specialities.priority'].value"}
},
I have a filter query to match the search text to any speciality.
Like if i search Oncology, it will match and then I have specified a script_score to take priority of that speciality and add it to final score of document.
But, it is taking the priority of the first speciality it encounters that is 10 and a score of 1 for the filter matched and the end score is 11 not 21 (priority of oncology + 1 for filter match)

I solved it using nested mapping in elasticsearch.
Lucene internally has no concept of storing object mappings by default, so if I am looking to store priority for every speciality I should have a mapping like this:
hospital: {
properties: {
specialities: {
type: nested,
properties: {
name: {
type: 'string'
}priority: {
type: 'long'
}
}
}
}
}
Reference: https://www.elastic.co/guide/en/elasticsearch/reference/2.0/nested.html
After that, I was able to define function score with nested query and my query looks like this:
"query": {
"filtered": {
"query": {
"bool": {
"must": [
{
"nested": {
"path": "specialities",
"query": {
"function_score": {
"score_mode": "sum",
"boost_mode": "sum",
"filter": {
"terms": {
"specialities.name.raw": ["Oncology"]
}
},
"functions": [
{
"field_value_factor": {
"field": "specialities.priority"
}
}
]
}
}
}
}
]
}
}
}
}

Related

ElasticSearch: query nested objects, that match filter

I have such index:
{
"id":2,
"name":"test home",
"city_id":"31",
"county_id":"1",
"zip_code":"123",
"residencePlans":[
{
"id" : 1,
"unit_price_from":480240,
"bathrooms_count":3,
"interior_area_sqrft":23,
"floor_range_hight":5,
"bedrooms_count":5,
"elevator_type_id":4,
"price_psqft":3756,
},
{
"id" : 2,
"unit_price_from":123456,
"bathrooms_count":1,
"interior_area_sqrft":12,
"floor_range_hight":4,
"bedrooms_count":2,
"elevator_type_id":3,
"price_psqft":1234,
}
],
}
And then I use some filters. Some of them are applied to the top object, and some to nesting.
I need to query residencePlans, that match filter, applied for their. eg filter on residencePlans.bathrooms_count >= 3 should return only residence with id = 1 and not 2.
{
"id": [2],
"residencePlans.id": [1]
}
I marked residencePlans as nested mapping, but it doesn't help.
Checkout the documentation here: https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-query.html
And here: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-inner-hits.html
Something like this should do it
{
"query": {
"bool": {
"must": [
{ "match": { "id": 1 }},
{
"nested": {
"path": "residencePlans",
"query": {
"bool": {
"must": [
{ "gte": { "residencePlans.unit_price_from": 3 }}
]
}
}
}
}
]
},
inner_hits: {}
}
}
I've revised my answer to take into account the particulars of filtering your top level document and your nested documents. Please let me know if it works for you!

Elastic search returning wrong results

I am running a query against elastic search but the results returned are wrong. The idea is that I can check against a range of fields with individual queries. But when I pass the following query, items which don't have the included lineup are returned.
query: {
bool: {
must: [
{match:{"lineup.name":{query:"The 1975"}}}
]
}
}
The objects are events which looks like.
{
title: 'Glastonbury'
country: 'UK',
lineup: [
{
name: 'The 1975',
genre: 'Indie',
headliner: false
}
]
},
{
title: 'Reading'
country: 'UK',
lineup: [
{
name: 'The Strokes',
genre: 'Indie',
headliner: true
}
]
}
In my case both of these events are returned.
The mapping can be seen here:
https://jsonblob.com/567e8f10e4b01190df45bb29
You need to use match_phrase query, match query is looking for either The or 1975 and it find The in The strokes and it gives you that result.
Try
{
"query": {
"bool": {
"must": [
{
"match": {
"lineup.name": {
"query": "The 1975",
"type": "phrase"
}
}
}
]
}
}
}

Elasticsearch match nested field against array of values

I'm trying to apply a terms query on a nested field using mongoid-elasticsearch and ElasticSearch 2.0. This has come to be quite frustrating since the trial-error didn't pay off much and the docs on the subject are rather sparse.
Here is my query:
{
"query": {
"nested": {
"path": "awards",
"query": {
"bool": {
"must": [
{ "match": { "awards.year": "2010"}}
]
}
}
},
"nested":{
"path": "procuring_entity",
"query": {
"bool": {
"must": [
{ "terms": { "procuring_entity.country": ["ES", "PL"]}}
]
}
}
}
}
}
While "match" and "term", work just fine, when combined with the "terms" query it returns no results, even thought it should. My mappings looks like this:
elasticsearch!({
prefix_name: false,
index_name: 'documents',
index_options: {
mappings: {
document: {
properties: {
procuring_entity: {
type: "nested"
},
awards: {
type: "nested"
}
}
}
}
},
wrapper: :load
})
If "nested" doesn't count as analyzer (which as far as I know doesn't), than there's no problem with that. As for the second example,I don't think it's the case since the array of values that it's matched against comes from exterior.
Is terms query possible on nested fields? Am I doing something wrong?
Is there any other way to match a nested field against multiple values?
Any thoughts would be much appreciated.
I think you would need to change your mappings for your nested types for this - the terms query only works on not_analyzed fields. If you update your mapping to something like:
elasticsearch!({
prefix_name: false,
index_name: 'documents',
index_options: {
mappings: {
document: {
properties: {
procuring_entity: {
type: 'nested',
properties: {
country: {
'type': 'string',
'index': 'not_analyzed'
}
}
},
awards: {
type: 'nested'
}
}
}
}
},
wrapper: :load
})
I think the query should work if you do that.

Elasticsearch Nested Query matching and/or

For example, I have data containing the following:
{
author: "test",
books: [
{
name: "first book",
cost: 50
},
{
name: "second book",
cost: 100
}
]
}
I want to search the author which has ALL books with cost > 40. How would the query for that look like? The field books is mapped as nested property.
For author names with cost of one book greater than 40 (in hits), something as below in query would work
POST http://192.168.0.68:9200/library/Book/_search
{
"fields": ["author"],
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"nested": {
"path": "books",
"filter": {
"range": {
"books.cost": {
"gt": 40
}
}
}
}
}
}
}
}
For all books having cost greater than 40, I had to handle the collection of nested field manually in client side after getting the response.
Not sure if script applies here to apply filter to all nested objects.
Reference
document not in nested documents elasticsearch

How to filter terms aggregation

Currently I have something like this
aggs: {
categories: {
terms: {
field: 'category'
}
}
}
and this is giving me number of products in each category. But I have additional condition. I need to get number of products in each category which are not sold already, so I need to perform filter on terms somehow.
Is there some elegant way of doing this using aggregation framework, or I need to write filtered query?
Thank you
You can merge between Terms Aggregation and Filter Aggregation, and this is how it should look: (tested)
aggs: {
categories: {
filter: {term: {sold: true}},
aggs: {
names: {
terms: {field: 'category'}
}
}
}
}
You can add also more conditions to the filter, I hope this helps.
Just to add to the other answer, you can also use a nested query. This is similar to what I had to do. I'm using Elasticsearch 5.2.
From the docs, here is the basic syntax:
"aggregations" : {
"<aggregation_name>" : {
"<aggregation_type>" : {
<aggregation_body>
}
[,"aggregations" : { [<sub_aggregation>]+ } ]?
}
[,"<aggregation_name_2>" : { ... } ]*
}
This is how I implemented it:
GET <path> core_data/_search
{
"aggs": {
"NAME": {
"nested": {
"path": "ATTRIBUTES"
},
"aggs": {
"NAME": {
"filter": {
"term": {
"ATTRIBUTES.ATTR_TYPE": "EDUCATION_DEGREE"
}
},
"aggs": {
"NAME": {
"terms": {
"field": "ATTRIBUTES.DESCRIPTION",
"size": 100
}
}
}
}
}
}
}
}
This filtered the data down to one bucket, which is what I needed.

Resources