I've an index on an Elasticsearch cluster, and I want to support phonetic matching.
This is my request:
curl -XPUT "http://localhost:9200/propertywebsites/_mapping/doc?pretty" -i -d '
{
"properties" : {
"phoneticbuilding" : {
"type" : "string",
"fields" : {
"phonetic" : {
"type" : "string",
"analyzer" : "dbl_metaphone"
}}}}
}
'
I received this error response:
HTTP/1.1 400 Bad Request
Content-Type: application/json; charset=UTF-8
Content-Length: 116
{
"error" : "MapperParsingException[Analyzer [dbl_metaphone] not found for field [phonetic]]",
"status" : 400
}
Anyone has any idea about why the dbl_metaphone analyzer couldn't be recognized for phonetic fields?
My elasticsearch's version is elasticsearch-1.7.2
Update 1
I already have the analyzer as the following
PUT myIndexName/
{
"settings": {
"analysis": {
"filter": {
"dbl_metaphone": {
"type": "phonetic",
"encoder": "double_metaphone"
}
},
"analyzer": {
"dbl_metaphone": {
"tokenizer": "standard",
"filter": "dbl_metaphone"
}
}
}
}
}
Update 2
Querying this request
curl -XGET "http://localhost:9200/propertywebsites/_settings?pretty"
I get the following response:
{
"propertywebsites" : {
"settings" : {
"index" : {
"creation_date" : "1451838136296",
"number_of_shards" : "5",
"number_of_replicas" : "1",
"version" : {
"created" : "1070299"
},
"uuid" : "KVOuKVgGRBudsSplownrgg",
"analsis" : {
"filter" : {
"dbl_metaphone" : {
"type" : "phonetic",
"encoder" : "double_metaphone"
}
},
"analyzer" : {
"dbl_metaphone" : {
"filter" : "dbl_metaphone",
"tokenizer" : "standard"
}
}
}
}
}
}
}
"dbl_metaphone" is a token filter and not an analyzer. You need to first install the Phonetic Analysis plug-in and then create a custom analyzer with it. Find more information at https://www.elastic.co/guide/en/elasticsearch/guide/current/phonetic-matching.html.
Related
I'm new in ELK. I have a problem with the followed search query:
curl --insecure -H "Authorization: ApiKey $ESAPIKEY" -X GET "https://localhost:9200/commsrch/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"should" : [
{"match" : {"cn" : "franc"}},
{"prefix" : {"srt" : "99889300200"}}
]
}
}
}
'
I need to find all documents that satisfies the condition: OR field "cn" contains "franc" OR field "srt" starts with "99889300200".
Index mapping:
{
"commsrch" : {
"mappings" : {
"properties" : {
"addr" : {
"type" : "text",
"index" : false
},
"cn" : {
"type" : "text",
"analyzer" : "compname"
},
"srn" : {
"type" : "text",
"analyzer" : "srnsrt"
},
"srt" : {
"type" : "text",
"analyzer" : "srnsrt"
}
}
}
}
}
Index settings:
{
"commsrch" : {
"settings" : {
"index" : {
"routing" : {
"allocation" : {
"include" : {
"_tier_preference" : "data_content"
}
}
},
"number_of_shards" : "1",
"provided_name" : "commsrch",
"creation_date" : "1675079141160",
"analysis" : {
"filter" : {
"ngram_filter" : {
"type" : "ngram",
"min_gram" : "3",
"max_gram" : "4"
}
},
"analyzer" : {
"compname" : {
"filter" : [
"lowercase",
"stop",
"ngram_filter"
],
"type" : "custom",
"tokenizer" : "whitespace"
},
"srnsrt" : {
"type" : "custom",
"tokenizer" : "standard"
}
}
},
"number_of_replicas" : "1",
"uuid" : "C15EXHnaTIq88JSYNt7GvA",
"version" : {
"created" : "8060099"
}
}
}
}
}
Query works properly with just only one condition. If query has only "match" condition, results has properly documents count. If query has only "prefix" condition, results has properly documents count.
In case of two conditions "match" and "prefix", i see in result documents that corresponds only "prefix" condition.
In ELK docs can't find any limitation about mixing "prefix" and "match", but as i see some problem exists. Please help to find where is the problem.
In continue of experince I have one more problem.
Example:
Source data:
1st document cn field: "put stone is done"
2nd document cn field:: "job one or two"
Mapping and index settings the same as described in my first post
Request:
{
"query": {
"bool": {
"should" : [
{"match" : {"cn" : "one"}},
{"prefix" : {"cn" : "one"}}
]
}
}
}
'
As I understand, the high scores got first document, because it has more repeats of "one". But I need high scores for documents, that has at least one word in field "cn" started from string "one". I have experiments with query:
{
"query": {
"bool": {
"should": [
{"match": {"cn": "one"}},
{
"constant_score": {
"filter": {
"prefix": {
"cn": "one"
}
},
"boost": 100
}
}
]
}
}
}
But it doesn't work properly. What's wrong with my query?
I am trying to query my products ElasticSearch index and create a script_score but I keep receiving the error Variable [relevancy] is not defined.
I tried replacing the script with just a number, then with Math.log(_score) to make sure the script_score was working properly and the math function is ok, and both queries executed as expected. I also tried doc['relevancy'].value and received the same error.
My query is:
curl -X GET "localhost:9200/products/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"function_score": {
"query": {
"multi_match" : {
"query": "KQ",
"fields": [ "item_id", "extended_desc", "mfg_part_no" ]
}
},
"script_score" : {
"script": "Math.log(_score) + Math.log(doc['relevancy'])"
},
"boost_mode": "replace"
}
}
}
'
And the mapping for this index is:
{
"products" : {
"mappings" : {
"properties" : {
"#timestamp" : {
"type" : "date"
},
"#version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"extended_desc" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"frecno" : {
"type" : "long"
},
"item_id" : {
"type" : "text",
"analyzer" : "my_analyzer"
},
"mfg_part_no" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"relevancy" : {
"type" : "long"
}
}
}
}
}
Replaced ' with \u0027 because this is curl.
curl -X GET "localhost:9200/products/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"function_score": {
"query": {
"multi_match" : {
"query": "KQ",
"fields": [ "item_id", "extended_desc", "mfg_part_no" ]
}
},
"script_score" : {
"script": "Math.log(_score) + Math.log(doc[\u0027relevancy\u0027].value)"
},
"boost_mode": "replace"
}
}
}
'
I created new index in elasticsearch (v6) using command:
curl -XPUT -H 'Content-Type: application/json' http://localhost:9200/sorttest -d '
{
"settings" : {
"index" : {
"sort.field" : ["username", "date"],
"sort.order" : ["asc", "desc"]
}
},
"mappings": {
"_doc": {
"properties": {
"username": {
"type": "keyword",
"doc_values": true
},
"date": {
"type": "date"
}
}
}
}
}
'
The response was
{"acknowledged":true,"shards_acknowledged":true,"index":"sorttest"}
Next I checked out generated mapping
curl -XGET localhost:9200/sorttest/_mapping?pretty
And the result was
{
"sorttest" : {
"mappings" : {
"_doc" : {
"properties" : {
"date" : {
"type" : "date"
},
"username" : {
"type" : "keyword"
}
}
}
}
}
}
The question is: how can I find out what kind of sorting is set for my index?
Just
curl -XGET localhost:9200/sorttest?pretty
and you will see:
"settings" : {
"index" : {
...
"sort" : {
"field" : [
"username",
"date"
],
"order" : [
"asc",
"desc"
]
},
...
}
}
I have created a synonym analyser on an index:
curl http://localhost:9200/test_index/_settings?pretty
{
"test_index" : {
"settings" : {
"index" : {
"creation_date" : "1429175067557",
"analyzer" : {
"search_synonyms" : {
"filter" : [ "lowercase", "search_synonym_filter" ],
"tokenizer" : "standard"
}
},
"uuid" : "Zq6Id8xsRWGofJrNCb7M8w",
"number_of_replicas" : "1",
"analysis" : {
"filter" : {
"search_synonym_filter" : {
"type" : "synonym",
"synonyms" : [ "sneakers,pumps" ]
}
}
},
"number_of_shards" : "5",
"version" : {
"created" : "1050099"
}
}
}
}
}
But when I try to use it with the mapping:
curl -XPUT 'http://localhost:9200/test_index/_mapping/product_catalog?pretty' -H "Content-Type: application/json" \
-d '{"product_catalog": {"properties" : {"name": {"type": "string", "include_in_all": true, "analyzer":"search_synonyms"} }}}'
I get the error:
{
"error" : "MapperParsingException[Analyzer [search_synonyms] not found for field [name]]",
"status" : 400
}
I have also tried to just check the analyser with:
curl 'http://localhost:9200/test_index/_analyze?analyzer=search_synonyms&pretty=1&text=pumps'
but still get an error:
ElasticsearchIllegalArgumentException[failed to find analyzer [search_synonyms]]
Any ideas, I may be missing something but I can't think what.
The analyzer element has to be inside your analysis component. Change your index creator as follows:
{
"settings": {
"index": {
"creation_date": "1429175067557",
"uuid": "Zq6Id8xsRWGofJrNCb7M8w",
"number_of_replicas": "0",
"analysis": {
"filter": {
"search_synonym_filter": {
"type": "synonym",
"synonyms": [
"sneakers,pumps"
]
}
},
"analyzer": {
"search_synonyms": {
"filter": [
"lowercase",
"search_synonym_filter"
],
"tokenizer": "standard"
}
}
},
"number_of_shards": "5",
"version": {
"created": "1050099"
}
}
}
}
I just installed and testing elastic search it looks great and i need to know some thing i have an configuration file
elasticsearch.json in config directory
{
"network" : {
"host" : "127.0.0.1"
},
"index" : {
"number_of_shards": 3,
"number_of_replicas": 1,
"refresh_interval" : "2s",
"analysis" : {
"analyzer" : {
"index_analyzer" : {
"tokenizer" : "nGram",
"filter" : ["lowercase"]
},
"search_analyzer" : {
"tokenizer" : "nGram",
"filter" : ["lowercase"]
}
},
"// you'll need lucene dep for this: filter" : {
"snowball": {
"type" : "snowball",
"language" : "English"
}
}
}
}
}
and i have inserted an doc that contains a word searching if i search for keyword
search it says nothing found...
wont it stem before indexing or i missed some thing in config ....
How looks your query?
your config does not look good. try:
...
"index_analyzer" : {
"tokenizer" : "nGram",
"filter" : ["lowercase", "snowball"]
},
"search_analyzer" : {
"tokenizer" : "nGram",
"filter" : ["lowercase", "snowball"]
}
},
"filter" : {
"snowball": {
"type" : "snowball",
"language" : "English"
}
}
I've had trouble overriding the "default_search" and "default_index" analyzer as well.
This works though.
You can add "index_analyzer" to default all string fields with unspecified analyzers within a type, if need be.
curl -XDELETE localhost:9200/twitter
curl -XPOST localhost:9200/twitter -d '
{"index":
{ "number_of_shards": 1,
"analysis": {
"filter": {
"snowball": {
"type" : "snowball",
"language" : "English"
}
},
"analyzer": { "a2" : {
"type":"custom",
"tokenizer": "standard",
"filter": ["lowercase", "snowball"]
}
}
}
}
}
}'
curl -XPUT localhost:9200/twitter/tweet/_mapping -d '{
"tweet" : {
"date_formats" : ["yyyy-MM-dd", "dd-MM-yyyy"],
"properties" : {
"user": {"type":"string"},
"message" : {"type" : "string", "analyzer":"a2"}
}
}}'
curl -XPUT http://localhost:9200/twitter/tweet/1 -d '{ "user": "kimchy", "post_date": "2009-11-15T13:12:00", "message": "Trying out searching teaching, so far so good?" }'
curl -XGET localhost:9200/twitter/tweet/_search?q=message:search
curl -XGET localhost:9200/twitter/tweet/_search?q=message:try