Why am I not able to query by geolocation in elastic search? - elasticsearch

So I am experimenting with ElasticSearch and this is what I did.
1) Added a geo_point mapping on my index called "test"
curl -XPOST localhost:9200/test -d '{
"mappings" : {
"type1" : {
"_source" : { "enabled" : false },
"properties" : {
"location" : { "type" : "geo_point", "index" : "not_analyzed" }
}
}
}
}'
2) Indexed a document under test:
curl -XPUT 'localhost:9200/test/type1/1?pretty' -d '{
"location" : {
"lat" : 74,
"lon" : 90
}
}'
3) Then wrote a query by geolocation filter like this:
curl -XPOST 'localhost:9200/test2/_search?pretty' -d '{
"filtered" : {
"query" : {
"match_all" : {}
},
"filter" : {
"geo_distance" : {
"distance" : "200km",
"location" : {
"lat" : 40,
"lon" : -70
}
}
}
}
}'
For this I get:
"error" : "SearchPhaseExecutionException[Failed to execute phase
[query], all shards failed; shardFailures
{[QpGEHtdcReeYmt8X2tG26g][test2][0]:
RemoteTransportException[[Jester][inet[/10.58.91.21:9301]][search/phase/query]];
nested: SearchParseException[[test2][0]: from[-1],size[-1]: Parse
Failure [Failed to parse source [na]]]; nested:
ElasticsearchParseException[Failed to derive xcontent from
org.elasticsearch.common.bytes.ChannelBufferBytesReference#60d8bc76];

First off, in your query (i.e. the 3rd piece of code), localhost:9200/test2/_search?pretty should be localhost:9200/test/_search?pretty, i.e. you're not querying the correct index.
Then your query is simply missing the query keyword (i.e. filtered should be enclosed in a query), it should look like this:
curl -XPOST 'localhost:9200/test/_search?pretty' -d '{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"geo_distance": {
"distance": "200km",
"location": {
"lat": 40,
"lon": -70
}
}
}
}
}
}

Related

How to order search results with related to slop in elasticsearch?

I have an index in ES:
curl -XGET 'http://127.0.0.1:9200/so/_settings?pretty=true'
{
"so" : {
"settings" : {
"index" : {
"number_of_shards" : "1",
"provided_name" : "so",
"creation_date" : "1594912442805",
"analysis" : {
"analyzer" : {
"my_simple_analyzer" : {
"type" : "simple",
"tokenizer" : "lowercase"
}
}
},
"number_of_replicas" : "1",
"uuid" : "8YVu4zU_Sdylr3KhOIwu9Q",
"version" : {
"created" : "7080099"
}
}
}
}
}
It has around 1.5M data.
curl -XGET 'http://127.0.0.1:9200/so/_count?pretty=true'
{
"count" : 15426942,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
}
}
I wanted to perform a full text search, so that the query string first does the phrase match and then followed by results which has slop of 1, then slop of 2 and so on.
So I came up with the below query for the same:
curl -XGET 'http://127.0.0.1:9200/so/_search?pretty=true' -H 'Content-Type: application/json' -d '{
"query": {
"bool": {
"should": [
{
"match_phrase": {
"posts": {
"query": "get the scanner on a specific family like this",
"_name": "exact_match"
}
}
},
{
"match": {
"posts": {
"query": "get the scanner on a specific family like this",
"_name": "partial_match"
}
}
}
]
}
}
}'
Is this the correct query? Because I do see the partial_match doesnt sort from slop of distance 1 and so on. How to achieve it?

Geo query using elasticsearch

I have created and index the same as the example tutorials, in here...
https://www.elastic.co/guide/en/elasticsearch/reference/2.0/geo-point.html
in specific writing the following:
curl -PUT 'localhost:9200/my_index?pretty' -d '
{
"mappings": {
"my_type": {
"properties": {
"location": {
"type": "geo_point"
}
}
}
}
}'
I have also added two points as data
curl -PUT 'localhost:9200/my_index/my_type/1?pretty' -d'
{
"text": "first geo-point",
"location": {
"lat": 41.12,
"lon": -71.34
}
}'
curl -PUT 'localhost:9200/my_index/my_type/1?pretty' -d'
{
"text": "second geo-point",
"location": {
"lat": 41.13,
"lon": -71.35
}
}'
The example geo bounding box query on the page works (i.e):
curl -XGET 'localhost:9200/my_index/_search?pretty' -d'
{
"query": {
"geo_bounding_box": {
"location": {
"top_left": {
"lat": 42,
"lon": -72
},
"bottom_right": {
"lat": 40,
"lon": -74
}
}
}
}
}'
But the example from this page (https://www.elastic.co/guide/en/elasticsearch/reference/2.0/query-dsl-geo-bounding-box-query.html) doesn't work:
What I have tried looks like the following:
curl -XGET 'localhost:9200/my_index/_search?pretty' -d'
{
"bool" : {
"must" : {
"match_all" : {}
},
"filter" : {
"geo_bounding_box" : {
"my_type.location" : {
"top_left" : {
"lat" : 42,
"lon" : -72
},
"bottom_right" : {
"lat" : 40,
"lon" : -74
}
}
}
}
}
}'
The error I get is as follows:
"error" : {
"root_cause" : [ {
"type" : "search_parse_exception",
"reason" : "failed to parse search source. unknown search element [bool]",
"line" : 3,
"col" : 5
} ],
"type" : "search_phase_execution_exception",
"reason" : "all shards failed",
"phase" : "query",
"grouped" : true,
"failed_shards" : [ {
"shard" : 0,
"index" : "my_index",
"node" : "0qfvkynhTRyjHFRurBLJeQ",
"reason" : {
"type" : "search_parse_exception",
"reason" : "failed to parse search source. unknown search element [bool]",
"line" : 3,
"col" : 5
}
} ]
},
"status" : 400
}
I hope its just a simple error, so would like to know what am i doing wrong?
You need to specify that the whole thing is a query:
curl -XGET 'localhost:9200/my_index/_search?pretty' -d'
{
"query": {
"bool" : {
"must" : {
"match_all" : {}
},
"filter" : {
"geo_bounding_box" : {
"my_type.location" : {
"top_left" : {
"lat" : 42,
"lon" : -72
},
"bottom_right" : {
"lat" : 40,
"lon" : -74
}
}
}
}
}
}
}'
However as far as I understand using bool with must and filter is the old way of doing things. In previous versions, geo queries were thought of as "filters", so you had to first run a match_all query to return all the results, and then filter using the geo bounding box. In Elasticssearch 2.0+, there is no separation between filters and queries - everything is a query. So you can run the geo query directly:
curl -XGET 'localhost:9200/my_index/_search?pretty' -d'
{
"query": {
"geo_bounding_box": {
"location": {
"top_left": {
"lat": 42,
"lon": -72
},
"bottom_right": {
"lat": 40,
"lon": -74
}
}
}
}
}'

how do you set a field to be not_analyized on a field that contains spaces?

I have a 'grade' field in an Elasticsearch index that contains text and numbers. I have set the field mapping to be 'not_analyized' but I can't search for grade ==== 'Year 1'.
I have read the finding exact values section of the docs but it doesn't seem to work for me.
Create the index.
curl -XPUT http://localhost:9200/my_test_index
Create the mapping template.
curl -XPUT http://localhost:9200/_template/my_test_index_mapping -d '
{
"template" : "my_test_index",
"mappings" : {
"my_type": {
"properties": {
"grade": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
'
Create some documents.
curl -XPUT 'http://localhost:9200/my_test_index/my_type/1' -d '{
"title" : "some title",
"grade" : "Year 1"
}'
curl -XPUT 'http://localhost:9200/my_test_index/my_type/3' -d '{
"title" : "some title",
"grade" : "preschool"
}'
Query for "Year 1" returns 0 results.
curl -XPOST http://localhost:9200/my_test_index/_search -d '{
"query": {
"filtered" : {
"filter" : {
"term": {
"grade": "Year 1"
}
}
}
}
}'
Query for 'preschool' returns 1 result.
curl -XPOST http://localhost:9200/my_test_index/_search -d '{
"query": {
"filtered" : {
"filter" : {
"term": {
"grade": "preschool"
}
}
}
}
}'
Checking the mapping and the 'grade' field does not show 'not_analyzed'.
curl -XGET http://localhost:9200/my_test_index/_mapping
{
"my_test_index" : {
"mappings" : {
"my_type" : {
"properties" : {
"grade" : {
"type" : "string"
},
"title" : {
"type" : "string"
}
}
}
}
}
}
The template will only impact newly created indices.
Re-Created the index after the template has been created.
Alternatively, specify the mappings while creating the index, instead of relying on templates to a single index.
If you don't want the field to be analysed you can specify "index" : "not_analyzed" in the mapping. You'll then be able to search for exact matches as desired.
See: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#string
In your case,Please try to Re-create your mapping.

How to index percolator queries containing filters on inner objects?

Using Elasticsearch 2.1.1
I have documents with inner objects:
{
"level1": {
"level2": 42
}
}
I want to register percolator queries applying filters on the inner property:
$ curl -XPUT http://localhost:9200/myindex/.percolator/myquery?pretty -d '{
"query": {
"filtered": {
"filter": {
"range": {
"level1.level2": {
"gt": 10
}
}
}
}
}
}'
It fails because I don't have a mapping:
{
"error" : {
"root_cause" : [ {
"type" : "query_parsing_exception",
"reason" : "Strict field resolution and no field mapping can be found for the field with name [level1.level2]",
"index" : "myindex",
"line" : 1,
"col" : 58
} ],
"type" : "percolator_exception",
"reason" : "failed to parse query [myquery]",
"index" : "myindex",
"caused_by" : {
"type" : "query_parsing_exception",
"reason" : "Strict field resolution and no field mapping can be found for the field with name [level1.level2]",
"index" : "myindex",
"line" : 1,
"col" : 58
}
},
"status" : 500
}
So I start again, but this time I add a mapping template before:
curl -XDELETE http://localhost:9200/_template/myindex
curl -XDELETE http://localhost:9200/myindex
curl -XPUT http://localhost:9200/_template/myindex?pretty -d 'x
{
"template": "myindex",
"mappings" : {
"mytype" : {
"properties" : {
"level1" : {
"properties" : {
"level2" : {
"type" : "long"
}
}
}
}
}
}
}
'
I try to register my percolator query again:
curl -XPUT http://localhost:9200/myindex/.percolator/myquery?pretty -d '{
"query": {
"filtered": {
"filter": {
"range": {
"level1.level2": {
"gt": 10
}
}
}
}
}
}'
And now it succeeds:
{
"_index" : "myindex",
"_type" : ".percolator",
"_id" : "myquery",
"_version" : 1,
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"created" : true
}
And I can see the mapping that has been created:
curl http://localhost:9200/myindex/_mapping?pretty
{
"myindex" : {
"mappings" : {
".percolator" : {
"properties" : {
"query" : {
"type" : "object",
"enabled" : false
}
}
},
"mytype" : {
"properties" : {
"level1" : {
"properties" : {
"level2" : {
"type" : "long"
}
}
}
}
}
}
}
}
Now my problem is that I also need to perform searches on my percolator queries and the default percolate mapping doesn’t index the query field.
So I start again, this time specifying in my mapping template that I want percolator queries to be indexed (note "enabled": true):
curl -XPUT http://localhost:9200/_template/myindex?pretty -d '
{
"template": "myindex",
"mappings" : {
".percolator" : {
"properties" : {
"query" : {
"type" : "object",
"enabled" : true
}
}
},
"mytype" : {
"properties" : {
"level1" : {
"properties" : {
"level2" : {
"type" : "long"
}
}
}
}
}
}
}
'
I try to register my percolator query again:
curl -XPUT http://localhost:9200/myindex/.percolator/myquery?pretty -d '{
"query": {
"filtered": {
"filter": {
"range": {
"level1.level2": {
"gt": 10
}
}
}
}
}
}'
But now I get an error:
{
"error" : {
"root_cause" : [ {
"type" : "mapper_parsing_exception",
"reason" : "Field name [level1.level2] cannot contain '.'"
} ],
"type" : "mapper_parsing_exception",
"reason" : "Field name [level1.level2] cannot contain '.'"
},
"status" : 400
}
How can I create and index a percolator query matching an inner property?

Elastic Search parent with same type

Sorry if this is a duplicate (I did try searching), or if this is a silly question. New to posting questions.
I am trying to do parent child relations and queries in ElasticSearch with the following:
#!/bin/bash
curl -XDELETE 'http://localhost:9200/test/'
echo
curl -XPUT 'http://localhost:9200/test/' -d '{
"settings" : {
"index" : {
"number_of_shards" : 1
}
}
}'
echo
curl -XPUT localhost:9200/test/_mapping/nelement -d '{
"nelement" : {
"_id" : { "path" : "nid", "store" : true, "index" : "not_analyzed"},
"_parent" : { "type" : "nelement"},
"properties" : {
"name" : { "type" : "string", "index" : "not_analyzed" },
"nid": { "type" : "string", "copy_to" : "_id" }
}
}
}'
echo
#curl -s -XPOST localhost:9200/_bulk --data-binary #test_data.json
test_data.json is as follows:
{"index":{"_index":"test","_type":"element", "_parent":"abc"}
{"nid":"1a","name":"parent1"}
{"index":{"_index":"test","_type":"element", "_parent":"1a"}
{"nid":"2b","name":"child1"}
{"index":{"_index":"test","_type":"element", "_parent":"2b"}
{"nid":"2c","name":"child2"}
curl -XGET 'localhost:9200/test/nelement/_search?pretty=true' -d '{
"query": {
"has_child": {
"child_type": "nelement",
"query": {
"match": {
"nid": "2c"
}
}
}
}
}'
echo
echo
curl -XGET 'localhost:9200/test/nelement/_search?pretty=true' -d '{
"query": {
"has_parent": {
"type": "nelement",
"query": {
"term": {
"nid": "2b"
}
}
}
}
}'
For some reason, my search queries get no results. I have confirmed that the objects are indexed....
Because you are using self referential(set parent and query in the same index type) to parent/child query.
For now Elasticsearch is not supporting it.
Explore parent/child self referential support

Resources