Elasticsearch context suggester, bool on contexts - elasticsearch

I'm using context suggester and am wondering if we can set the scope of the context to be used for suggestions rather that using all contexts.
Currently the query needs to match all contexts. Can we add an "OR" operation on the contexts and/or specify which context to use for a particular query?
Taking the example from here :
Mapping :
PUT /venues/poi/_mapping
{
"poi" : {
"properties" : {
"suggest_field": {
"type": "completion",
"context": {
"type": {
"type": "category"
},
"location": {
"type": "geo",
"precision" : "500m"
}
}
}
}
}
}
Then I index a document :
{
"suggest_field": {
"input": ["The Shed", "shed"],
"output" : "The Shed - fresh sea food",
"context": {
"location": {
"lat": 51.9481442,
"lon": -5.1817516
},
"type" : "restaurant"
}
}
}
Query:
{
"suggest" : {
"text" : "s",
"completion" : {
"field" : "suggest_field",
"context": {
"location": {
"value": {
"lat": 51.938119,
"lon": -5.174051
}
}
}
}
}
}
If I query using only one Context ("location" in the above example) it gives an error, I need to pass both the contexts, is it possible to specify which context to use? Or pass something like a "Context_Operation" parameter set to "OR".

You have 2 choices:
First you add all available type values as default in your mapping (not scalable)
{
"poi" : {
"properties" : {
"suggest_field": {
"type": "completion",
"context": {
"type": {
"type": "category",
"default": ["restaurant", "pool", "..."]
},
"location": {
"type": "geo",
"precision" : "500m"
}
}
}
}
}
}
Second option, you add a default value to every indexed document, and you add only this value as default
Mapping:
{
"poi" : {
"properties" : {
"suggest_field": {
"type": "completion",
"context": {
"type": {
"type": "category",
"default": "any"
},
"location": {
"type": "geo",
"precision" : "500m"
}
}
}
}
}
}
Document:
{
"suggest_field": {
"input": ["The Shed", "shed"],
"output" : "The Shed - fresh sea food",
"context": {
"location": {
"lat": 51.9481442,
"lon": -5.1817516
},
"type" : ["any", "restaurant"]
}
}
}

Related

Search by slug in Elasticsearch

I have an index named homes. Here is the simplified mapping of it:
{
"template": "homes",
"index_patterns": "homes",
"settings": {
"index.refresh_interval": "60s"
},
"mappings": {
"properties": {
"status": {
"type": "keyword"
},
"address": {
"type": "keyword",
"fields": {
"suggest": {
"type": "search_as_you_type"
},
"search": {
"type": "text"
}
}
}
}
}
}
As you can see, there is an address field which I query this way:
{
"query": {
"bool": {
"filter": [
{
"term": {
"status": "sale"
}
},
{
"term": {
"address": "406 - 533 Richmond St W"
}
}
]
}
}
}
Now my problem is that I need to be able to query with slugyfied version of the address field as well. For example, I need to query like this:
{
"query": {
"bool": {
"filter": [
{
"term": {
"status": "sale"
}
},
{
"term": {
"address": "406-533-richmond-st-w"
}
}
]
}
}
}
So, instead of 406 - 533 Richmond St W I need to query 406-533-richmond-st-w. How can I do that? I was thinking of adding a new field address_slug which is the slugyfied version of address but I need it to be auto populated so I don't need to manually fill this field every time that I insert or update a document in the index.
If you create a custom analyzer with the token filters below and another field for search that uses the custom analyzer, you can achieve this. Here is an example analyze result and output:
GET {index}/_analyze
{
"tokenizer": "keyword",
"filter": [
{
"type": "lowercase"
},
{
"type": "pattern_replace",
"pattern": """[^A-Za-z0-9]+""",
"replacement": "-"
}
],
"text": "406 - 533 Richmond St W"
}
Output:
{
"tokens" : [
{
"token" : "406-533-richmond-st-w",
"start_offset" : 0,
"end_offset" : 23,
"type" : "word",
"position" : 0
}
]
}

Range query on Nested Documents for Keyword Type

I have a document structure like this. For this below two documents, we have nested documents called interaction info. I just need to get only the documents that have title duration and their value is greater than 60
Here whats the catch the value field is keyword, not an integer. I know that only for the Integer Range query will get executed. Is there any possible way to find the documents that have a duration greater than 60 ( Painless Query or Script Query ). Like converting the value Field into Integer and then searching the document.
{
"key": "f07ff9ba-36e4-482a-9c1c-d888e89f926e",
"interactionInfo": [
{
"title": "duration",
"value": "11"
},
{
"title": "timetaken",
"value": "9"
},
{
"title": "talk_time",
"value": "145"
}
]
},
{
"key": "f07ff9ba-36e4-482a-9c1c-d888e89f926e",
"interactionInfo": [
{
"title": "duration",
"value": "120"
},
{
"title": "timetaken",
"value": "9"
},
{
"title": "talk_time",
"value": "60"
}
]
}
I have added script to get interactionInfo.value>"somevalue". Scripts are slow and it is better to resolve this at index time and use a range query.
Index:
{
"index15" : {
"mappings" : {
"properties" : {
"interactionInfo" : {
"type" : "nested",
"properties" : {
"title" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"value" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"key" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
}
Query:
{
"query": {
"nested": {
"path": "interactionInfo",
"query": {
"bool": {
"must": [
{
"term": {
"interactionInfo.title.keyword": {
"value": "duration"
}
}
},
{
"script": {
"script": {
"source":"def val=Integer.parseInt(doc['interactionInfo.value.keyword'].value); if(val>params.value) return true; else return false;",
"params": {
"value":10
}
}
}
}
]
}
},
"inner_hits": {}
}
}
}

Simple elasticsearch input - Rejecting mapping update final mapping would have more than 1 type: [_doc, doc]

I'm trying to send data to elasticsearch but running into an issue where my number field only comes up as a string. These are the steps I took.
Step 1. Add index & map
PUT http://123.com:5101/core_060619/
{
"mappings": {
"properties": {
"date": {
"type": "date",
"format": "HH:mm yyyy-MM-dd"
},
"data": {
"type": "integer"
}
}
}
}
Result:
{
"acknowledged": true,
"shards_acknowledged": true,
"index": "core_060619"
}
Step 2. Add data
PUT http://123.com:5101/core_060619/doc/1
{
"test" : [ {
"data" : "119050300",
"date" : "00:00 2019-06-03"
} ]
}
Result:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Rejecting mapping update to [zyxnewcoreyxbl_060619] as the final mapping would have more than 1 type: [_doc, doc]"
}
],
"type": "illegal_argument_exception",
"reason": "Rejecting mapping update to [zyxnewcoreyxbl_060619] as the final mapping would have more than 1 type: [_doc, doc]"
},
"status": 400
}
You can not have more than one type of document in Elasticsearch 6.0.0+. If you set your document type to doc, then you can add another document by simply PUT http://123.com:5101/core_060619/doc/1, PUT http://123.com:5101/core_060619/doc/2 etc.
Elasticsearch 6.+
PUT core_060619/
{
"mappings": {
"doc": { //type of documents in index is 'doc'
"properties": {
"date": {
"type": "date",
"format": "HH:mm yyyy-MM-dd"
},
"data": {
"type": "integer"
}
}
}
}
}
Since we created mapping to have doc type of documents, now we can add new documents by simply adding /doc/_id:
PUT core_060619/doc/1
{
"test" : [ {
"data" : "119050300",
"date" : "00:00 2019-06-03"
} ]
}
PUT core_060619/doc/2
{
"test" : [ {
"data" : "111120300",
"date" : "10:15 2019-06-02"
} ]
}
Elasticsearch 7.+
Types are removed, but you can use custom like field(s):
PUT twitter
{
"mappings": {
"_doc": {
"properties": {
"type": { "type": "keyword" },
"name": { "type": "text" },
"user_name": { "type": "keyword" },
"email": { "type": "keyword" },
"content": { "type": "text" },
"tweeted_at": { "type": "date" }
}
}
}
}
PUT twitter/_doc/user-kimchy
{
"type": "user",
"name": "Shay Banon",
"user_name": "kimchy",
"email": "shay#kimchy.com"
}
PUT twitter/_doc/tweet-1
{
"type": "tweet",
"user_name": "kimchy",
"tweeted_at": "2017-10-24T09:00:00Z",
"content": "Types are going away"
}
GET twitter/_search
{
"query": {
"bool": {
"must": {
"match": {
"user_name": "kimchy"
}
},
"filter": {
"match": {
"type": "tweet"
}
}
}
}
}
Removal of mapping types

Is it impossible to index a document where a property has multiple fields, one of them being a completion type with contexts?

Here is my mapping (some fields renamed/removed), I'm using ES 6.0
{
"mappings": {
"_doc" :{
"properties" : {
"username" : {
"type": "keyword",
"fields": {
"suggest" : {
"type" : "completion",
"contexts": [
{
"name": "user_id",
"type": "category"
}
]
}
}
},
"user_id": {
"type": "integer"
}
}
}
}
}
Now when I try to index a document with
PUT usernames/_doc/1
{
"username" : "JOHN",
"user_id": 1
}
OR
PUT usernames/_doc/1
{
"username" : {
"input": "JOHN",
"contexts: {
"user_id": 1
}
}
"user_id": 1
}
The first doesn't index with context and the second just fails. I've attempted to add a path like so,
{
"mappings": {
"_doc" :{
"properties" : {
"username" : {
"type": "keyword",
"fields": {
"suggest" : {
"type" : "completion",
"contexts": [
{
"name": "user_id",
"type": "category",
"path": "user_id",
}
]
}
}
},
"user_id": {
"type": "integer"
}
}
}
}
}
And attempting indexing again
PUT usernames/_doc/1
{
"username" : "JOHN",
"user_id": 1
}
But it just throws a context must be a keyword or text error. Do I have to give up and make a totally new property username-autocomplete instead? Or is there some magical way where I can have a context completion suggester and another field on the same property, and be able to index like I would other multifield properties?
The second approach is the right one (i.e. with the path inside the context), but you need to set the user_id field as a keyword and it will work:
{
"mappings": {
"_doc" :{
"properties" : {
"username" : {
"type": "keyword",
"fields": {
"suggest" : {
"type" : "completion",
"contexts": [
{
"name": "user_id",
"type": "category",
"path": "user_id",
}
]
}
}
},
"user_id": {
"type": "keyword" <--- change this
}
}
}
}
}
Then you can index your document without creating an additional field, like this:
PUT usernames/_doc/1
{
"username" : "JOHN",
"user_id": "1" <--- wrap in double quotes
}

Elasticsearch - Can't search using suggestion field (“is not a completion suggest field”)

I'm completely new to elasticsearch and I'm trying to use elasticsearch completion suggester on an existing field called "identity.full_name", index = "search" and type = "person".
I followed the below index to change the mappings of the field.
1)
POST /search/_close
2)
POST search/person/_mapping
{
"person": {
"properties": {
"identity.full_name": {
"type": "text",
"fields":{
"suggest":{
"type":"completion"
}
}
}
}
}
}
3)
POST /search/_open
When I check the mappings at this point, using
GET search/_mapping/person/field/identity.full_name
I get the result,
{
"search": {
"mappings": {
"person": {
"identity.full_name": {
"full_name": "identity.full_name",
"mapping": {
"full_name": {
"type": "text",
"fields": {
"completion": {
"type": "completion",
"analyzer": "simple",
"preserve_separators": true,
"preserve_position_increments": true,
"max_input_length": 50
},
"keyword": {
"type": "keyword",
"ignore_above": 256
},
"suggest": {
"type": "completion",
"analyzer": "simple",
"preserve_separators": true,
"preserve_position_increments": true,
"max_input_length": 50
}
}
}
}
}
}
}
}
}
Which is suggesting that it has been updated to be a completion field.
However, when I'm querying to check if this works using,
GET search/person/_search
{
"suggest": {
"person-suggest" : {
"prefix" : "EMANNUEL",
"completion" : {
"field" : "identity.full_name"
}
}
}
}
It is giving me the error "Field [identity.full_name] is not a completion suggest field"
I'm not sure why I'm getting this error. Is there anything else I can try?
sample data:
{
"_index": "search",
"_type": "person",
"_id": "3106105149",
"_score": 1,
"_source": {
"identity": {
"id": "3106105149",
"first_name": "FLORENT",
"last_name": "TEBOUL",
"full_name": "FLORENT TEBOUL"
}
}
}
{
"_index": "search",
"_type": "person",
"_id": "125296353",
"_score": 1,
"_source": {
"identity": {
"id": "125296353",
"first_name": "CHRISTINA",
"last_name": "BHAN",
"full_name": "CHRISTINA K BHAN"
}
}
}
so when I do a GET based on prefix "CHRISTINA"
GET search/person/_search
{
"suggest": {
"person-suggest" : {
"prefix" : "CHRISTINA",
"completion" : {
"field" : "identity.full_name.suggest"
}
}
}
}
I'm getting all the results like a match_all query.
You should use it like
GET search/person/_search
{
"suggest": {
"person-suggest" : {
"prefix" : "EMANNUEL",
"completion" : {
"field" : "identity.full_name.suggest"
}
}
}
}
Mapping for GET search/_mapping/person/field/identity.full_name
{
"search" : {
"mappings" : {
"person" : {
"identity.full_name" : {
"full_name" : "identity.full_name",
"mapping" : {
"full_name" : {
"type" : "text",
"fields" : {
"suggest" : {
"type" : "completion",
"analyzer" : "simple",
"preserve_separators" : true,
"preserve_position_increments" : true,
"max_input_length" : 50
}
}
}
}
}
}
}
}
}

Resources