elasticsearch: DISJOINT query relation not supported for Field - elasticsearch

I have created the simplest index in ES v7.10:
it maps mylocation field into geo_point (not geo_shape):
PUT /myindex
{
"mappings": {
"dynamic": "false",
"properties": {
"mylocation": {
"type": "geo_point"
}
}
}
}
then I pushed some data inside. I've omitted this to shorten the question...
When I query the below, all works fine:
{
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": {
"geo_shape": {
"mylocation": {
"shape": {
"type": "polygon",
"coordinates": [[ [ 13.0, 53.0 ], [ 0.0, 1.0 ], [ 0.0, 0.0 ], [ 13.0, 53.0 ] ]]
},
"relation": "intersects"
}
}
}
}
}
}
when I replace the intersects with disjoint, I get the error:
DISJOINT query relation not supported for Field [mylocation].
In elastic docs (for the relevant version, 7), it is mentioned that "Geoshape query filter documents indexed using the geo_shape or geo_point type" (and I am using the geo_point). Down the same page, it is written the "disjoint" is supported!
What am I missing? Why do I get the error?

No idea how I have missed it, but in v7.10 docs (link in the question) they write in the next paragraph:
When searching a field of type geo_point there is a single supported
spatial relation operator:
INTERSECTS - (default) Return all documents whose geo_point field
intersects the query geometry.
In v7.17 they do not have this limitation anymore...

Related

Elasticsearch : Check if GeoPoint exists in the radius of multiple circles

My requirement is to check if a partucular geoPoint falls in radius of a circle or not.
I am using geoShape : circle to store the location. My document is as below:
PUT location_test/doc/1
{
"location" : {
"type" : "circle",
"coordinates" : [73.7769,18.5642],
"radius": "10mi"
}
}
and querying is as below :
GET location_test/_search
{
"query": {
"bool": {
"must": [
{
"geo_shape": {
"location": {
"shape": {
"type": "point",
"coordinates": [
73.877097,
18.455303
],
"relation": "contains"
}
}
}
}
]
}
}
}
This query works perfectly for single circle geo shape.
However now I want to check if a particular geoPoint falls in radius of multiple circles.
Can we have our document something like :
{
"location": [
{
"type": "circle",
"coordinates": [
73.7769,
18.5642
],
"radius": "10mi"
},
{
"type": "circle",
"coordinates": [
-118.240853,
34.052997
],
"radius": "10mi"
}
]
}
and have a query to check if a geoPoint falls in which circle.
Or is there any another way to achieve this ?
EDIT
Is it a good practice to use array of geo-points to sort documents for a particular geo-point ?
Mapping :
{
"mappings": {
"doc": {
"properties": {
"locationPoint": {
"type": "geo_point"
}
}
}
}
}
PUT location_test2/doc/1
{
"locationPoint": ["34.075433, -118.307228","36.336356,-119.304597"]
}
PUT location_test2/doc/2
{
"locationPoint": ["34.075433, -118.307228"]
}
GET location_test2/_search
{
"sort": [
{
"_geo_distance": {
"locationPoint": "34.075433, -118.307228",
"order": "asc"
}
}
]
}
You can surely have multiple circles in one document and the search is still going to match if any of the circles contain your point. Collapsing the steps for brevity:
PUT location_test
{"mappings":{"properties":{"location":{"type":"geo_shape","strategy":"recursive"}}}}
Taking in your array of circles:
PUT location_test/_doc/2
{"location":[{"type":"circle","coordinates":[73.7769,18.5642],"radius":"10mi"},{"type":"circle","coordinates":[-118.240853,34.052997],"radius":"10mi"}]}
Same query as for a single circle.
GET location_test/_search
{"query":{"bool":{"must":[{"geo_shape":{"location":{"shape":{"type":"point","coordinates":[73.7769,18.5642],"relation":"contains"}}}}]}}}
which yields our doc of interest. The counterintuitive but nice thing about this is that it does not matter if you provide a single object or a list of objects. ElasticSearch handles both without a mapping change.
Just note that your circles are on opposite sides of the globe:
If you're aware of this and querying your locations makes sense like this, all is fine.
From a performance standpoint keep in mind that circles are represented as polygons
which, depending on your ES version are represented as a bunch of triangles.
So you may want to index circle-like polygons instead of circles to maybe speed your indexing up or even think about merging your circles in a set of polygons (a MultiPolygon) because from what it looks like, your list of circles represents
related geometries.

Using time filter in dashboard to change range of Vega in Kibana

I am using Kibana 7.1.
I have successfully created Vega line plot. I can allow it to show month of data however I want user to play around with time filter in dashboard and allow vega visualization to change with it.
From https://www.elastic.co/blog/getting-started-with-vega-visualizations-in-kibana and in Vega documentation I've read that inserting
"%context%": true,
"%timefield%": "#timestamp"
inside url will resolve this issue however when I do this it gives me
url.%context% and url.%timefield% must not be used when url.body.query is set
my full elasticsearch code look like this:
"data": {
"url": {
"%context%":"true",
"index": "access_log",
"body": {
"query": {
"bool": {
"must": [
{"term": {"request_1": "rent"}},
{"term": {"status": 200}}
]
}
},
"aggs": {
"histo": {
"date_histogram": {
"field": "date",
"interval": "day"
},
"aggs": {
"start_agg": {
"filter": {
"term": {"request_2": "start"}
}
},
"check_agg": {
"filter": {
"term": {"request_2": "check"}
}
},
"start_check": {
"bucket_script": {
"buckets_path": {
"start_count": "start_agg._count",
"check_count": "check_agg._count"
},
"script": "params.start_count / params.check_count"
}
}
}
}
}
}
},
"format": {
"property": "aggregations.histo.buckets"
}
},
"mark": {
"type":"line"
},
"encoding": {
"x": {
"field": "key",
"type": "temporal",
"axis": {"title": false}
},
"y": {
"field": "start_check.value",
"type": "quantitative",
"axis": {"title": "Document count"}
},
"tooltip":[
{"field":"start_check.value",
"type" : "quantitative"},
{"field":"key",
"type" :"temporal"}
]
}
}
Quoting Elastic's Vega reference for Kibana:
When using "%context%": true or defining a value for "%timefield%" the body cannot contain a query. To customize the query within the VEGA specification (e.g. add an additional filter, or shift the timefilter), define your query and use the placeholders as in the example above. The placeholders will be replaced by the actual context of the dashboard or visualization once parsed.
The "example above" they are talking about is the following:
{
body: {
query: {
bool: {
must: [
// This string will be replaced
// with the auto-generated "MUST" clause
"%dashboard_context-must_clause%"
{
range: {
// apply timefilter (upper right corner)
// to the #timestamp variable
#timestamp: {
// "%timefilter%" will be replaced with
// the current values of the time filter
// (from the upper right corner)
"%timefilter%": true
// Only work with %timefilter%
// Shift current timefilter by 10 units back
shift: 10
// week, day (default), hour, minute, second
unit: minute
}
}
}
]
must_not: [
// This string will be replaced with
// the auto-generated "MUST-NOT" clause
"%dashboard_context-must_not_clause%"
]
filter: [
// This string will be replaced
// with the auto-generated "FILTER" clause
"%dashboard_context-filter_clause%"
]
}
}
}
}
And, just as already defined in the docs:
"%dashboard_context-must_clause%": String replaced by object containing filters
"%dashboard_context-filter_clause%": String replaced by an object containing filters
"%dashboard_context-must_not_clause%": String replaced by an object containing filters
So, in case you want to use user-defined filters or the time filter having a customized query at the same time, you must use these three strings instead of "%context%": true. They will be parsed by Kibana and substituted by Elasticsearch query objects: one for "MUST", one for "FILTER" and one for "MUST_NOT", respectively.
A simple schema like this one might be useful:
{
body: {
query: {
bool: {
must: [
// {
// A "MUST" clause of yours
// },
"%dashboard_context-must_clause%"
]
must_not: [
// {
// A "MUST_NOT" clause of yours
// },
"%dashboard_context-must_not_clause%"
]
filter: [
// {
// A "FILTER" clause of yours
// },
"%dashboard_context-filter_clause%"
]
}
}
}
}
In case you don't have any clause in some of the categories, just leave the corresponding "%dashboard_context-XXXXX_clause%" string without further objects - just like in the first example for "FILTER" or "MUST_NOT".
inserting "%timefield%":"date" where date is one of my field worked.

Elasticsearch returning no geo query results in es6.7

I'm mapping congressional districts in elasticsearch but my efforts are returning no results when trying to query by geoshape even though the query is clearly within the boundary of my WKT values. Here's some relevant data:
The mapping:
"mappings":{
"house":{
"properties":{
"state_name":{
"type":"text"
},
"district":{
"type":"integer"
},
"shape":{
"type":"geo_shape",
"strategy":"recursive"
}
}
}
An example of a WKT on the shape property: POLYGON((-122.612285 37.815224,-122.501272 37.821212,-122.499764 37.819724,-122.41871 37.852491,-122.418673 37.852505,-122.430183 37.918425,-122.432283 37.929824,-122.373982 37.883884,-122.373782 37.883725,-122.28246 37.709309,-122.28178 37.70823,-122.419802 37.708231,-122.420082 37.708231,-122.440893 37.716405,-122.440999 37.716488,-122.428203 37.731851,-122.428038 37.732016,-122.451147 37.731567,-122.453411 37.731568,-122.463399 37.752981,-122.463433 37.753028,-122.469667 37.738505,-122.470597 37.737665,-122.512732 37.735087,-122.578294 37.733988,-122.584728 37.779156,-122.588174 37.789362,-122.612285 37.815224))
My search query in kibana that is clearly inside that WKT:
GET /house/_search
{
"query":{
"bool": {
"must": {
"match_all": {}
},
"filter": {
"geo_shape": {
"shape": {
"shape": {
"type": "circle",
"coordinates" : [-122.421151, 37.758442],
"radius": "100m"
}
}
}
}
}
}
I've also tried geoshape type point with intersects and contains relation (which is not currently supported).
What else can I do to debug why these documents are not being found?
I realized that my mapping was incorrect. I was manually setting a geo_shape mapping on shape but when my indexes were created they were creating Shape (note capitalization change) properties with type text. Thus none of my indexes had any geo_shape data to search.
The most useful thing I used to debug and find this issue was a readout of the GET /house/mapping data which showed the two distinct properties that should have been a single property.

Can not query geo_point using geo polygon filter

Hi I'm trying to query geo_point in ElasticSearch, using the query syntax geo polygon filter in the official document], but no reusult is returned.
Here are some details:
I use river plugin to index the data from MySQL into ES with the definition of mapping (a nested structure but coordinate.value is geo_point).
I can see the documents from head plugin:
The query json is:
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"geo_polygon": {
"coordinate.value": {
"points": [
[
-180,
90
],
[
-180,
-90
],
[
180,
-90
],
[
-180,
90
]
]
}
}
}
}
}
}
Can anyone tell me what's the correct query method to get geo_point? Thanks
polygon should be closed (ie first and last points should be the same).

Accessing nested property in Elasticsearch distance script.

My index in elastic search has the following mapping:
"couchbaseDocument": {
"properties": {
"doc": {
"properties": {
"properties": {
"properties": {
"location": {
"type": "geo_point"
The source document is as follows:
{"properties" : {"location":"43.706596,-79.4030464"}}
I am trying to use the distance script to calculate the distance based on geo-points. I found this post Return distance in elasticsearch results? to help me out. I am trying to get all results,filter by radius 1km, get the distance, and sort on geo_point. The query is constructed as follows:
{
"query": {
"match_all": {}
},
"filter": {
"geo_distance": {
"distance": "1km",
"doc.properties.location": {
"lat": 43.710323,
"lon": -79.395284
}
}
},
"script_fields": {
"distancePLANE": {
"params": {
"lat": 43.710323,
"lon": -79.395284
},
"script": "doc[properties]['location'].distanceInKm(lat, lon)"
},
"distanceARC" :{
"params": {
"lat": 43.710323,
"lon": -79.395284
},
"script": "doc[properties]['location'].arcDistanceInKm(lat,lon)"
}
},
"sort": [
{
"_geo_distance":{
"doc.properties.location": [-79.395284,43.710323],
"order": "desc",
"unit": "km"
}
}
],
"track_scores": true
}
I get the following error with status 500:
"PropertyAccessException[[Error: could not access: properties; in class: org.elasticsearch.search.lookup.DocLookup]\n[Near : {... doc[properties]['location'].distan ....}]\n ^\n[Line: 1, Column: 5]]"
I tried rewriting the query in this way:
..."script": "doc['properties']['location'].arcDistanceInKm(lat,lon)"...
Then I get this error:
"CompileException[[Error: No field found for [properties] in mapping with types [couchbaseDocument]]\n[Near : {... doc['properties']['location']. ....}]\n ^\n[Line: 1, Column: 1]]; nested: ElasticSearchIllegalArgumentException[No field found for [properties] in mapping with types [couchbaseDocument]]; "
When I remove the script part from the query all together, the sorting and filtering works just fine. Is there a different way to access nested fields when using scripts? Any insights would be really appreciated!
Thank you!
Managed to get it done with
"script" : "doc.last_location.distance(41.12, -71.34)"
Don't know why but doc['last_location'] does not seem to work at all!
As mentioned in my comment when you sort by _geo_distance the "_sort" field that is returned, is the actual distance. So there is no need to do a separate computation. Details here: http://elasticsearch-users.115913.n3.nabble.com/search-by-distance-and-getting-the-actual-distance-td3317140.html#a3936224

Resources