How can I append a script field to a result? - elasticsearch

I want to append a script field to a elasticsearch result. But I can't find a working solution.
I have a script field like this:
{
"script_fields": {
"distance": {
"script": "doc[my_field_name].arcDistance(my_lat, my_lon)",
"params": {
"my_field_name": "geopoint",
"my_lat": 52.5,
"my_lon": 13.4
}
}
}
}
As result I get something like that:
"hits": [
{
"fields": {
"distance": [
0
]
}
},
{
"fields": {
"distance": [
500
]
}
},
{
"fields": {
"distance": [
1000
]
}
}
]
But I need full documents together with the script fields. So I've tried this:
{
"script_fields": {
"distance": {
"script": "doc[my_field_name].arcDistance(my_lat, my_lon)",
"params": {
"my_field_name": "geopoint",
"my_lat": 52.5,
"my_lon": 13.4
}
},
"source": {
"script": "_source"
}
}
}
But as result I get something like this:
"hits": [
{
"fields": {
"distance": [
0
],
"source": [
{
"id": "101",
"geopoint": {
"lon": 52.5,
"lat": 13.4
},
}
]
}
},
{
"fields": {
"distance": [
500
],
"source": [
{
"id": "101",
"geopoint": {
"lon": 52.5,
"lat": 13.4
},
}
]
}
},
{
"fields": {
"distance": [
1000
],
"source": [
{
"id": "101",
"geopoint": {
"lon": 52.5,
"lat": 13.4
},
}
]
}
}
]
The source is in this case the same for all hits. I have thought the _source loads per document, but doesn't look so.
How can I achieve script field together with the document as result or isn't this possible?

I was on the wrong track. The solutions was to change the request to:
{
"fields": [
"_source"
],
"script_fields": {
"distance": {
"script": "doc[my_field_name].arcDistance(my_lat, my_lon)",
"params": {
"my_field_name": "geopoint",
"my_lat": 52.5,
"my_lon": 13.4
}
}
}
}
the result looks than something like this:
"hits": [
{
"fields": {
"distance": [
0
],
},
"_source": {
{
"id": "101",
"geopoint": {
"lat": 52.5,
"lon": 13.4
},
}
}
},
{
"fields": {
"distance": [
500
],
},
"_source": {
{
"id": "102",
"geopoint": {
"lat": 52.5,
"lon": 13.40739378
},
}
}
},
{
"fields": {
"distance": [
1000
],
},
"_source": {
{
"id": "103",
"geopoint": {
"lat": 52.5,
"lon": 13.4147875
},
}
}
}
]

Related

Is there a way to return the geo distance when NOT sorting with _geo_distance?

I need to return the computing distance in the result for the geo location.but not using the sort
currently I'm using sorting but it ignores the exist field
here is my query:
"query": {
"bool": {
"filter": [
{
"match": {
"field 1": "value"
}
},
{
"match": {
"field2": "A"
}
},
{
"geo_distance": {
"distance": "20km",
"location": "34,-2.99"
}
}
], "should": [
{
"exists": {
"field": "field3",
"boost": 10000
}
}
]
}
},
"size": 500,
"sort": [
{
"_geo_distance": {
"location": {
"lat": 34,
"lon": 2.99
},
"order": "asc",
"unit": "km",
"mode": "min",
"distance_type": "arc",
"ignore_unmapped": "true"
}
}
]
I need if the field3 exist it gets higher ranking
If I understand your problem correctly. This query should work:
"query": {
"bool": {
"filter": [
{
"match": {
"field 1": "value"
}
},
{
"match": {
"field2": "A"
}
},
{
"geo_distance": {
"distance": "20km",
"location": "34,-2.99"
}
}
]
}
},
"size": 500,
"sort": [
{
"_script": {
"type": "number",
"script": {
"source": "doc['field3'].size() > 0 ? 10000 : 0"
},
"order": "asc"
}
},
{
"_geo_distance": {
"location": {
"lat": 34,
"lon": 2.99
},
"order": "asc",
"unit": "km",
"mode": "min",
"distance_type": "arc",
"ignore_unmapped": "true"
}
}
]
I am checking if field exist and boosting it's score

Order documents by multiple geolocations

I am new to ElasticSearch and I try to create an index for companies that come with multiple branches in the city.
Each of the branches, it has its own geolocation point.
My companies document looks like this:
{
"company_name": "Company X",
"branch": [
{
"address": {
// ... other fields
"location": "0.0000,1.1111"
}
}
]
}
The index have the following mapping:
{
"companies": {
"mappings": {
"dynamic_templates": [
{
"ids": {
"match": "id",
"match_mapping_type": "long",
"mapping": {
"type": "long"
}
}
},
{
"company_locations": {
"match": "location",
"match_mapping_type": "string",
"mapping": {
"type": "geo_point"
}
}
}
],
"properties": {
"branch": {
"properties": {
"address": {
"properties": {
// ...
"location": {
"type": "geo_point"
},
// ...
}
},
}
}
}
}
}
}
Now, in the ElasticSearch I've indexed the following documents:
{
"company_name": "Company #1",
"branch": [
{
"address": {
"location": "39.615,19.8948"
}
}
]
}
and
{
"company_name": "Company #2",
"branch": [
{
"address": {
"location": "39.586,19.9028"
}
},
{
"address": {
"location": "39.612,19.9134"
}
},
{
"address": {
"location": "39.607,19.8946"
}
}
]
}
Now what is my problem. If I try to run the following search query, unfortunately the company displayed first is the Company #2 although the geodistance query has the location data of the Company #1:
GET companies/_search
{
"fields": [
"company_name",
"branch.address.location"
],
"_source": false,
"sort": [
{
"_geo_distance": {
"branch.address.location": {
"lon": 39.615,
"lat": 19.8948
},
"order": "asc",
"unit": "km"
}
}
]
}
Am I doing something wrong? Is there a way to sort the search results using this method?
Please keep in mind that if for example search with a geolocation that is more close to some geolocations of the "Comapny #2", in this case I need the Company #2 to be first.
Finally, if the setup I have isn't correct for what I require, if there's any other way to achieve that same result with different document structure, please let me know. I am still in the beginning of the project, and It's simple to adapt to what is more appropriate.
The documentation here says "Geopoint expressed as a string with the format: "lat,lon"."
Your location is "location": "39.615,19.8948", maybe the query must be below:
"branch.address.location": {
"lat": 39.615,
"lon": 19.8948
}
My Tests:
PUT idx_test
{
"mappings": {
"properties": {
"branch": {
"properties": {
"address": {
"properties": {
"location": {
"type": "geo_point"
}
}
}
}
}
}
}
}
POST idx_test/_doc/1
{
"company_name": "Company #1",
"branch": [
{
"address": {
"location": "39.615,19.8948"
}
}
]
}
POST idx_test/_doc/2
{
"company_name": "Company #2",
"branch": [
{
"address": {
"location": "39.586,19.9028"
}
},
{
"address": {
"location": "39.612,19.9134"
}
},
{
"address": {
"location": "39.607,19.8946"
}
}
]
}
Search by location "39.607,19.8946" company #2
GET idx_test/_search?
{
"fields": [
"company_name",
"branch.address.location"
],
"_source": false,
"sort": [
{
"_geo_distance": {
"branch.address.location": {
"lat": 39.607,
"lon": 19.8946
},
"order": "asc",
"unit": "km"
}
}
]
}
Response:
"hits": [
{
"_index": "idx_test",
"_id": "2",
"_score": null,
"fields": {
"branch.address.location": [
{
"coordinates": [
19.9028,
39.586
],
"type": "Point"
},
{
"coordinates": [
19.9134,
39.612
],
"type": "Point"
},
{
"coordinates": [
19.8946,
39.607
],
"type": "Point"
}
],
"company_name": [
"Company #2"
]
},
"sort": [
0
]
},
{
"_index": "idx_test",
"_id": "1",
"_score": null,
"fields": {
"branch.address.location": [
{
"coordinates": [
19.8948,
39.615
],
"type": "Point"
}
],
"company_name": [
"Company #1"
]
},
"sort": [
0.8897252783915647
]
}
]
Search by location "39.615,19.8948" company #1
GET idx_test/_search?
{
"fields": [
"company_name",
"branch.address.location"
],
"_source": false,
"sort": [
{
"_geo_distance": {
"branch.address.location": {
"lat": 39.615,
"lon": 19.8948
},
"order": "asc",
"unit": "km"
}
}
]
}
Response
"hits": [
{
"_index": "idx_test",
"_id": "1",
"_score": null,
"fields": {
"branch.address.location": [
{
"coordinates": [
19.8948,
39.615
],
"type": "Point"
}
],
"company_name": [
"Company #1"
]
},
"sort": [
0
]
},
{
"_index": "idx_test",
"_id": "2",
"_score": null,
"fields": {
"branch.address.location": [
{
"coordinates": [
19.9028,
39.586
],
"type": "Point"
},
{
"coordinates": [
19.9134,
39.612
],
"type": "Point"
},
{
"coordinates": [
19.8946,
39.607
],
"type": "Point"
}
],
"company_name": [
"Company #2"
]
},
"sort": [
0.8897285575578558
]
}
]

Elasticsearch - filter geo_distance query

I have a mapping type with two fields : location (geo_point) and type (short).
I want to list my places by geo proximity and use this kind of query
{
"query": {
"bool": {
"filter": {
"geo_distance": {
"distance": "20km",
"location": {
"lat": 48.856614,
"lon": 2.3522219
}
}
}
}
},
"aggs": {
"types": {
"terms": {
"field": "type"
}
}
},
"post_filter": [],
"page": 1,
"size": 50,
"sort": [
{
"_geo_distance": {
"location": {
"lat": 48.856614,
"lon": 2.3522219
},
"order": "asc",
"unit": "km",
"distance_type": "plane"
}
}
]
}
Is there any way to only include the first 2 places of a special type (e.g. type=2) ?
Add another clause to the filter like
{
"query": {
"bool": {
"filter": [{
"geo_distance": {
"distance": "20km",
"location": {
"lat": 48.856614,
"lon": 2.3522219
}
}
},
{
"term": {"type":"2"}
}]
}
},
"aggs": {
"types": {
"terms": {
"field": "type"
}
}
},
"post_filter": [],
"page": 1,
"size": 2,
"sort": [
{
"_geo_distance": {
"location": {
"lat": 48.856614,
"lon": 2.3522219
},
"order": "asc",
"unit": "km",
"distance_type": "plane"
}
}
]
}

Elasticsearch with range and exists filter

I have a ElasticSearch query to get every products within a set range. I would like to add a filter to select only documents which have the attribute "products". My tests with must exists had always error.
/zipcodes_at/zipcode/_search
{
"_source": [
"products"
],
"filter": {
"geo_distance": {
"distance": "100km",
"location": {
"lat": 48.232361,
"lon": 16.324659
}
}
},
"sort": [
{
"_geo_distance": {
"location": {
"lat": 48.232361,
"lon": 16.324695
},
"order": "asc",
"unit": "km",
"distance_type": "plane"
}
}
]
}
Try this:
POST /zipcodes_at/zipcode/_search
{
"_source": [
"products"
],
"query": {
"bool": {
"filter": [
{
"exists": {
"field": "products"
}
},
{
"geo_distance": {
"distance": "100km",
"location": {
"lat": 48.232361,
"lon": 16.324659
}
}
}
]
}
},
"sort": [
{
"_geo_distance": {
"location": {
"lat": 48.232361,
"lon": 16.324695
},
"order": "asc",
"unit": "km",
"distance_type": "plane"
}
}
]
}
You should must use bool filter , and combine geo distance filter along with exist filter.
{
"_source": ["products"],
"query": {
"filtered": {
"filter": {
"bool": {
"must": [{
"exists": {
"field": "products"
}
}, {
"geo_distance_range": {
"from": 0,
"to": 100,
"distance_unit": "km",
"location": {
"lat": 40.73,
"lon": -74.1
}
}
}]
}
}
}
},
"sort": [{
"_geo_distance": {
"location": {
"lat": 48.232361,
"lon": 16.324695
},
"order": "asc",
"unit": "km",
"distance_type": "plane"
}
}]
}

Elasticsearch Geo Distance query

I've got a list of places which have their latitude and longitude associated with them in the correct mapping of geo_point
I've also got a query successfully returning results based on geo distance which looks like this:
{
"filter": {
"geo_distance": {
"distance": "30mi",
"companies.locations": {
"lat": "51.8801595",
"lon": "0.577141"
}
}
},
"sort": {
"_geo_distance": {
"companies.locations": {
"lat": "51.8801595",
"lon": "0.577141"
},
"order": "asc",
"unit": "mi",
"mode": "min"
}
},
"from": 0,
"size": 500
}
So this currently returns results within 30miles of the latitude and longitude provided. And this works fine.
I'm struggling with the next step, which I'm hoping someone can point me in the right direction with.
Each place has a field called distance which is an integer. This is the maximum distance a place is willing to travel to a client. So if the distance is 20 (miles) but their latitude and longitude calculates as more than 20miles they should be excluded from the results.
The results come back like this:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": null,
"hits": [
{
"_index": "test",
"_type": "places",
"_id": "AUtvK2OILrMWSKLclj9Z",
"_score": null,
"_source": {
"id": "1",
"name": "Chubby Company",
"summary": "",
"content": "",
"locations": [
{
"lat": 51.8200763,
"lon": 0.5264076
}
],
"address": [
{
"addr1": "xxxx",
"addr2": "",
"town": "MyTown",
"county": "Essex",
"postcode": "XX1 2XX",
"tel1": "01111 111111",
"tel2": "",
"email": null
}
],
"website": "",
"media": {
"logo": "",
"image": "",
"video": ""
},
"product_ids": [
"1",
"3",
"2"
],
"distance": "20"
},
"sort": [
0.031774582056958885
]
}
]
}
}
The sort object is distance in miles, so the result above is 0.03 miles from the client.
I'm trying to utilize this to check against the record using result to exclude it from the results but this is where I'm falling down.
I've tried different combinations of this:
"script": {
"script": "doc['distance'].value < doc['sort'].value"
}
which combined with the query looks like this:
{
"filter": {
"geo_distance": {
"distance": "30mi",
"companies.locations": {
"lat": "51.8801595",
"lon": "0.577141"
}
}
},
"sort": {
"_geo_distance": {
"companies.locations": {
"lat": "51.8801595",
"lon": "0.577141"
},
"order": "asc",
"unit": "mi",
"mode": "min"
}
},
"filtered": {
"filter": {
"script": {
"script": "doc['distance'].value < doc['sort'].value"
}
}
},
"from": 0,
"size": 500
}
But i get an error of:
SearchPhaseExecutionException[Failed to execute phase [query], all
shards failed ... Parse Failure [No parser for element [filtered]
Any advice would be great.
UPDATE
Trying this also fails:
{
"filter": {
"geo_distance": {
"distance": "30mi",
"companies.locations": {
"lat": "51.8801595",
"lon": "0.577141"
}
},
"script": {
"script": "_source.distance < sort.value"
}
},
"sort": {
"_geo_distance": {
"companies.locations": {
"lat": "51.8801595",
"lon": "0.577141"
},
"order": "asc",
"unit": "mi",
"mode": "min"
}
},
"from": 0,
"size": 500
}
with
nested: ElasticsearchParseException[Expected field name but got START_OBJECT \"script\"]; }]","status":400}
I had a similar problem where I wanted to have a conditional distance for each record that had this value stored in the data(searchRadius for me). I ended up using the AndFilterBuilder and ScriptFilterBuilder classes in Java, so the AndFilterBuilder has an array of both a GeoDistanceFilterBuilder and a ScriptFilterBuilder.
{
"from": 0,
"size": 20,
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"and": {
"filters": [
{
"geo_distance": {
"location": [
-104.99230194091797,
39.74000930786133
],
"distance": "3000mi"
}
},
{
"script": {
"script": "doc['location'].arcDistanceInMiles(39.74000930786133, -104.99230194091797) < doc['searchRadius'].value"
}
}
]
}
}
}
},
"fields": "_source",
"script_fields": {
"distance": {
"script": "doc['location'].distance(39.74000930786133, -104.99230194091797)"
}
},
"sort": [
{
"_geo_distance": {
"location": [
-104.99230194091797,
39.74000930786133
],
"unit": "mi"
}
}
]
}

Resources