I can't achieve to only get documents with a maximum distance of 1km between two geopoints of the same document.
I have simple documents like this :
{
start_location: {
lat: 34.0583,
lon: -118.2476
},
end_location: {
lat: 33.989521,
lon: -117.531614
}
}
I want to get all the documents where start_location is located less than 1km from end_location.
Im stuck with this for a while, thank you in advance !
You can use the arcDistance function within a script:
PUT geo
{
"mappings": {
"properties": {
"start_location": {
"type": "geo_point"
},
"end_location": {
"type": "geo_point"
}
}
}
}
POST geo/_doc
{
"start_location": {
"lat": 34.0583,
"lon": -118.2476
},
"end_location": {
"lat": 33.989521,
"lon": -117.531614
}
}
GET geo/_search
{
"query": {
"script": {
"script": """
def distance_in_m = doc['start_location'].arcDistance(
doc['end_location'].getLat(),
doc['end_location'].getLon()
);
return distance_in_m < 1000"""
}
}
}
Related
From an Elasticsearch query I'd like to retrieve all the points within a variable distance.
Let say I have 2 shops, one is willing to deliver at maximum 3 km and the other one at maximum 5 km:
PUT /my_shops/_doc/1
{
"location": {
"lat": 40.12,
"lon": -71.34
},
"max_delivery_distance": 3000
}
PUT /my_shops/_doc/2
{
"location": {
"lat": 41.12,
"lon": -72.34
},
"max_delivery_distance": 5000
}
For a given location I'd like to know which shops are able to deliver. IE query should return shop1 if given location is within 3km and shop2 if given location is within 5km
GET /my_shops/_search
{
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": {
"geo_distance": {
"distance": max_delivery_distance,
"location": {
"lat": 40,
"lon": -70
}
}
}
}
}
}
There's another way to solve this without scripting (big performance hogger !!) and let ES sort it out using native Geo shapes.
I would model each document as a circle, with a center location and a (delivery) radius. First, your index mapping should look like this:
PUT /my_shops
{
"mappings": {
"properties": {
"delivery_area": {
"type": "geo_shape",
"strategy": "recursive"
}
}
}
}
Then, your documents then need to have the following form:
PUT /my_shops/_doc/1
{
"delivery_area" : {
"type" : "circle",
"coordinates" : [-71.34, 40.12],
"radius" : "3000m"
}
}
PUT /my_shops/_doc/2
{
"delivery_area" : {
"type" : "circle",
"coordinates" : [-72.34, 41.12],
"radius" : "5000m"
}
}
And finally the query simply becomes a geo_shape query looking at intersections between a delivery point and the delivery area of each shop.
GET /my_shops/_search
{
"query": {
"bool": {
"filter": {
"geo_shape": {
"delivery_area": {
"shape": {
"type": "point",
"coordinates": [ -70, 40 ]
},
"relation": "contains"
}
}
}
}
}
}
That's it! No scripting, just geo operations.
I think that you need to work with a script to use another field as parameter. After some research I come to this answer:
GET my_shops/_search
{
"query": {
"script": {
"script": {
"params": {
"location": {
"lat": 40,
"lon": -70
}
},
"source": """
return doc['location'].arcDistance(params.location.lat, params.location.lon)/1000 <= doc['max_delivery_distance'].value"""
}
}
}
}
Basically, we exploit the fact that the classes related to the GEO points are whitelisted in painless https://github.com/elastic/elasticsearch/pull/40180/ and that scripts accepts additional parameters (your fixed location).
According to the documentation of arcDistance we retrieve the size in meters, so you need to convert this value into km by dividing by 1000.
Additional Note
I assume that location and max_delivery_distance are always (for each document) defined. If it is not the case, you need to cover this case.
Reference
Another related question
https://github.com/elastic/elasticsearch/pull/40180/
I'm beginner in ElasticSearch. I'm trying to test if a list of geopoint (lat / long ) is existing in a list of geopoints.
For example I give this geopoint :
"lat": 49.01536940596998
"lon": 2.4967825412750244
and I want to test if this point exist in the list below. Thanks.
"positions": [
{
"millis": 12959023,
"lat": 49.01525113731623,
"lon": 2.4971945118159056,
"rawX": -3754,
"rawY": 605,
"rawVx": 0,
"rawVy": 0,
"speed": 9.801029291617944,
"accel": 0.09442740907572084,
"grounded": true
},
{
"millis": 12959914,
"lat": 49.01536940596998,
"lon": 2.4967825412750244,
"rawX": -3784,
"rawY": 619,
"rawVx": -15,
"rawVy": 7,
"speed": 10.841861737855924,
"accel": -0.09534648619563282,
"grounded": true
}
...
}
To be able to search in an array of objects, you need to use the nested data type. As the linked page explains, to keep the internal elements of the array as independent, you cannot use the default mapping. First, you will have to update the mapping.
Note: Mappings only take effect on new indexes. Reference.
PUT YOUR_INDEX
{
"mappings": {
"YOUR_TYPE": {
"properties": {
"positions": {
"type": "nested"
}
}
}
}
}
Now we can query the data. You're looking for a bool query, which combines other queries (in your case, term queries).
POST _search
{
"query": {
"nested": {
"path": "positions",
"query": {
"bool" : {
"must" : [
{ "term" : { "lat": 49.01536940596998 } },
{ "term" : { "lon": 2.4967825412750244 } }
]
}
}
}
}
}
I have an index that stores set of geo locations in nested object.
{
......,
"geo_points" : [
{
"lat" : ...
"lon" : ...
},
.......
]
}
and query is
{
sort: {
_geo_distance: {
geo_points: {
lat: "",
lon: ""
},
order: 'asc',
unit: 'km'
}
},
query: {
filtered: {
query: {
bool: {
must: [
{
range: {
endtime: {gte: ""}
}
},
{
range:{
starttime: {lte: ""}
}
}
],
should: [
{
nested: {
path: 'categories',
filter: {
bool: {
should: { terms: { 'categories.id' => [1,2,3,4]} }
}
}
}
}
],
minimum_number_should_match: 1
}
},
filter: {
geo_distance: {
distance: "25km",
geo_points: {lat: "",lon: ""}
}
}
}
},
from: 0, size: 100
}
and it is used for geo queries (sorts by distance), I would like to know if it is possible to return the geo_points with only matching location(s) or
can I say return only X location(s)?
Is there an existing way to do this inner_hits? and also, can you give me a sample query if it is possible?
Thanks.
I solved my issue with custom ES script since I couldn't find a proper inbuilt way so here what I did
import org.elasticsearch.common.geo.GeoDistance;
import org.elasticsearch.common.unit.DistanceUnit;
import org.elasticsearch.common.geo.GeoPoint;
/***
*
* usage
* "script_fields": {
* "closest_points": {
* "script": {
* "lang": "groovy",
* "file": "sortedGeoPoints",
* "params": {
* "field": "lokasyonlar",
* "lon": 26.954897,
* "lat": 38.7762021,
* "method": "PLANE",
* "unit": "km",
* "order": "asc",
* "limit": 5
* }
* }
* }
* }
*
*/
if (doc[field].values.size() < limit){
return doc[field]
}
else{
def distanceUnit
def geoCalculator
switch (method){
case "ARC" :
geoCalculator = GeoDistance.ARC
break
case "FACTOR" :
geoCalculator = GeoDistance.FACTOR
break
default:
geoCalculator = GeoDistance.PLANE
break
}
switch (unit) {
case "in" : //inch
distanceUnit = DistanceUnit.INCH
break
case "yd": //yards
distanceUnit = DistanceUnit.YARD
break
case "ft": //feet
distanceUnit = DistanceUnit.FEET
break
case "nmi": //NAUTICALMILES
distanceUnit = DistanceUnit.NAUTICALMILES
break
case "mm": // MILLIMETERS
distanceUnit = DistanceUnit.MILLIMETERS
break
case "cm": // CENTIMETERS
distanceUnit = DistanceUnit.CENTIMETERS
break
case "mi": // MILES
distanceUnit = DistanceUnit.MILES
break
case "m": // MILES
distanceUnit = DistanceUnit.METERS
break
default:
distanceUnit = DistanceUnit.KILOMETERS
break
}
def sortedGeoPoints = new TreeMap<Double, GeoPoint>()
for(i = 0; i < doc[field].values.size(); i++){
def loc = doc[field].values[i]
sortedGeoPoints.put(geoCalculator.calculate(loc.lon, loc.lat, lon, lat, distanceUnit), loc)
}
def list
if(order == "desc"){
list = new ArrayList<GeoPoint>(sortedGeoPoints.descendingMap().values()) //reversed
}
else{
list = new ArrayList<GeoPoint>(sortedGeoPoints.values())
}
return list.subList(0, limit)
}
and sample query
curl -XPOST /geo/test/_search -d '{
"fields": [
"_source",
"distances"
],
"query": {
"bool": {
"must": [
{
"geo_distance": {
"distance": "100km",
"locations": {
"lon": 28.442826999999966,
"lat": 37.101167
}
}
}
]
}
},
"script_fields": {
"closest_points": {
"script": {
"lang": "groovy",
"file": "sortedGeoPoints",
"params": {
"field" : "locations",
"lon": 28.442826999999966,
"lat": 37.101167,
"method" : "PLANE",
"unit" : "km",
"order" : "asc",
"limit" : 3
}
}
}
},
"sort": {
"_geo_distance": {
"locations": {
"lon": 28.442826999999966,
"lat": 37.101167
},
"order": "asc",
"unit": "km"
}
}
}'
Use case & full story can be found in gist
In fact, it can be extracted from _geo_distance sort. However, I don't believe enough to have such functionality at there.
I will keep this open for better inbuilt options if there is, please let us know
Im trying to search documents with wildcard and _all. But It does not seem like it's possible to get boosted result with wildcard on _all ?
MappingRequest:
"theboostingclass": {
"properties": {
"Important": {
"boost": 2.0,
"type": "string"
},
"LessImportant": {
"type": "string"
},
"Garbage": {
"type": "string"
}
}
}
}
Indexing:
{
"index" :
{
"_index":"boosting",
"_type":"theboostingclass"
}
}
{
"Important":"bomb",
"LessImportant":"kruka",
"Garbage":"kalkon"
}
{
"index" :
{
"_index":"boosting",
"_type":"theboostingclass"
}
}
{
"Important":"kalkon",
"LessImportant":"bomb",
"Garbage":"bomber"
}
{
"index" :
{
"_index":"boosting",
"_type":"theboostingclass"
}
}
{
"Important":"kruka",
"LessImportant":"bomber",
"Garbage":"bomb"
}
Query
"query": {
"wildcard": {
"_all": {
"value": "*bomb*"
}
}
}
The result returs all hits with a Score of 1 and a seemingly random order. Which is not really what Im after. I want the hit on "Important"field to yield a higher score.
If I do a wildcard search on all 3 fields the scoring seems correct. However I want to use it on _all. Any ideas?
Please see documentation here:
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-multi-term-rewrite.html. Note that the reason it works with a constant scoring by default is for performance.
I believe you need to modify your query as follows:
"query": {
"wildcard": {
"_all": {
"value": "*bomb*",
"rewrite": "scoring_boolean"
}
}
}
My index in elastic search has the following mapping:
"couchbaseDocument": {
"properties": {
"doc": {
"properties": {
"properties": {
"properties": {
"location": {
"type": "geo_point"
The source document is as follows:
{"properties" : {"location":"43.706596,-79.4030464"}}
I am trying to use the distance script to calculate the distance based on geo-points. I found this post Return distance in elasticsearch results? to help me out. I am trying to get all results,filter by radius 1km, get the distance, and sort on geo_point. The query is constructed as follows:
{
"query": {
"match_all": {}
},
"filter": {
"geo_distance": {
"distance": "1km",
"doc.properties.location": {
"lat": 43.710323,
"lon": -79.395284
}
}
},
"script_fields": {
"distancePLANE": {
"params": {
"lat": 43.710323,
"lon": -79.395284
},
"script": "doc[properties]['location'].distanceInKm(lat, lon)"
},
"distanceARC" :{
"params": {
"lat": 43.710323,
"lon": -79.395284
},
"script": "doc[properties]['location'].arcDistanceInKm(lat,lon)"
}
},
"sort": [
{
"_geo_distance":{
"doc.properties.location": [-79.395284,43.710323],
"order": "desc",
"unit": "km"
}
}
],
"track_scores": true
}
I get the following error with status 500:
"PropertyAccessException[[Error: could not access: properties; in class: org.elasticsearch.search.lookup.DocLookup]\n[Near : {... doc[properties]['location'].distan ....}]\n ^\n[Line: 1, Column: 5]]"
I tried rewriting the query in this way:
..."script": "doc['properties']['location'].arcDistanceInKm(lat,lon)"...
Then I get this error:
"CompileException[[Error: No field found for [properties] in mapping with types [couchbaseDocument]]\n[Near : {... doc['properties']['location']. ....}]\n ^\n[Line: 1, Column: 1]]; nested: ElasticSearchIllegalArgumentException[No field found for [properties] in mapping with types [couchbaseDocument]]; "
When I remove the script part from the query all together, the sorting and filtering works just fine. Is there a different way to access nested fields when using scripts? Any insights would be really appreciated!
Thank you!
Managed to get it done with
"script" : "doc.last_location.distance(41.12, -71.34)"
Don't know why but doc['last_location'] does not seem to work at all!
As mentioned in my comment when you sort by _geo_distance the "_sort" field that is returned, is the actual distance. So there is no need to do a separate computation. Details here: http://elasticsearch-users.115913.n3.nabble.com/search-by-distance-and-getting-the-actual-distance-td3317140.html#a3936224