I have an index with the following mapping:
{
"hosts": {
"mappings": {
"host": {
"properties": {
"dn": {
"type": "keyword",
"fields": {
"fqdn": {
"type": "text"
}
}
},
"hostname": {
"type": "text"
},
.....
}
}
}
where my intention is to be able get exact matches on 'dn' and full text on 'fqdn'. What happens in practice is that searches on 'dn' work fine but searches on 'fqdn' always return no documents.
E.g.
{"query": {"term": {"dn": "ps346256.uoa.auckland.ac.nz" } } }
returns one document but
{"query": {"match": {"fqdn": "ps346256" } } }
returned none.
What am I missing?
fqdn should be referenced as dn.fqdn as shown below:
{"query": {"match": {"dn.fqdn": "ps346256" } } }
Find the reference for multi-fields here.
Hope this helps!
Related
{
"myindex": {
"mappings": {
"properties": {
"city": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
I tried to update by using below PUT request on the index, but still getting the above ouput of _mapping
{
"_doc" : {
"properties" : {
"city" : {"type" : "text"}
}
}
}
I am not able to query with inexact words because its type is "keyword", for the below the actual value in record is "Mumbai"
{
"query": {
"bool": {
"must": {
"match": {
"city": {
"query": "Mumbi",
"minimum_should_match": "10%"
}
}
}
}
}
}
Below mapping (What is shared in the question) will store 'city' as text and 'city.keyword' as a keyword.
{
"myindex": {
"mappings": {
"properties": {
"city": {
"type": "text", // ==========> Store city as text
"fields": {
"keyword": {
"type": "keyword", // =========> store city.keyword as a keyword
"ignore_above": 256
}
}
}
}
}
}
}
your's is the use case of Fuzzy search and not minimum_should_match.
ES Docs for Fuzzy Search: https://www.elastic.co/blog/found-fuzzy-search
Try below query
{
"query": {
"match": {
"city": {
"query": "mubai",
"fuzziness": "AUTO"
}
}
}
}
minimum_should_match
Minimum number of clauses that must match for a document to be returned
It signifies the percentage of clauses not the percentage of the string. Go through this documentation to frame the query to get the expected results. Invalid queries return invalid results.
Does Elasticsearch allow us to query documents case-insensitive? Or should I save them as case-insensitive before querying? Or is there some setting that I should set for the whole index to make it case-insensitive?
Can you clarify this moment please?
By Default, the fields are case-insensitive because of the mapping elastic applied.
Try below:
PUT myindex/doc/1
{
"name":"TEST"
}
GET myindex/_mapping
It should return :
{
"myindex": {
"mappings": {
"doc": {
"properties": {
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
Now if you query with below, it will return a match (notice the mapping[text and keyword]):
POST myindex/_search
{
"query": {
"match": {
"name2": "test"
}
}
}
Now, if you explicitly specify to index the field as keyword, then it will be case-sensitive search. Try below and see; it will not return any results.
PUT myindex/_mapping/doc
{
"properties": {
"name2": {
"type": "keyword"
}
}
}
PUT myindex/doc/1
{
"name2":"TEST"
}
POST myindex/_search
{
"query": {
"match": {
"name2": "test"
}
}
}
TLDR; Use default mapping or text type- if you specify the field to index only keyword type, it will be case-sensitive.
In ES 2.3.3, many queries in the system I'm working on use the _all field. Sometimes these are registered to a percolate index, and when running percolator on the doc, _all is generated automatically.
In converting to ES 5.X _all is being deprecated and so _all has been replaced with a copy_to field that contains the components that we actually care about, and it works great for those searches.
Registering the same query to a percolate index with the same document mapping including copy_to fields works fine. Sending a percolate query with the document never results in a hit for a copy_to field however.
Manually building the copy_to field via simple string concatenation seems to work, it's just that I'd expect to be able to Query -> DocIndex and get the same result as Doc -> PercolateQuery... So I'm just looking for a way to have ES generate the copy_to fields automatically on a document being percolated.
Ended up there was nothing wrong with ES of course, posting here in case it helps someone else. Figured it out while attempting to generate a simpler example to post here with details... Basically the issue came down to the fact that attempting to percolate a document of a type that doesn't exist in the percolate index doesn't give any errors back, but seems to apply all percolate queries without applying any mappings which was just confusing as it worked for simple test cases, but not complex ones. Here's an example:
From the copy_to docs, generate an index with a copy_to mapping. See that a query to the copy_to field works.
PUT my_index
{
"mappings": {
"my_type": {
"properties": {
"first_name": {
"type": "text",
"copy_to": "full_name"
},
"last_name": {
"type": "text",
"copy_to": "full_name"
},
"full_name": {
"type": "text"
}
}
}
}
}
PUT my_index/my_type/1
{
"first_name": "John",
"last_name": "Smith"
}
GET my_index/_search
{
"query": {
"match": {
"full_name": {
"query": "John Smith",
"operator": "and"
}
}
}
}
Create a percolate index with the same type
PUT /my_percolate_index
{
"mappings": {
"my_type": {
"properties": {
"first_name": {
"type": "text",
"copy_to": "full_name"
},
"last_name": {
"type": "text",
"copy_to": "full_name"
},
"full_name": {
"type": "text"
}
}
},
"queries": {
"properties": {
"query": {
"type": "percolator"
}
}
}
}
}
Create a percolate query that matches our other percolate query on the copy_to field, and a second query that just queries on a basic unmodified field
PUT /my_percolate_index/queries/1?refresh
{
"query": {
"match": {
"full_name": {
"query": "John Smith",
"operator": "and"
}
}
}
}
PUT /my_percolate_index/queries/2?refresh
{
"query": {
"match": {
"first_name": {
"query": "John"
}
}
}
}
Search, but with the wrong type... there will be a hit on the basic field (first_name: John) even though no document mappings match the request
GET /my_percolate_index/_search
{
"query" : {
"percolate" : {
"field" : "query",
"document_type" : "non_type",
"document" : {
"first_name": "John",
"last_name": "Smith"
}
}
}
}
{"took":7,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":1,"max_score":0.2876821,"hits":[{"_index":"my_percolate_index","_type":"queries","_id":"2","_score":0.2876821,"_source":{
"query": {
"match": {
"first_name": {
"query": "John"
}
}
}
}}]}}
Send in the correct document_type and see both matches as expected
GET /my_percolate_index/_search
{
"query" : {
"percolate" : {
"field" : "query",
"document_type" : "my_type",
"document" : {
"first_name": "John",
"last_name": "Smith"
}
}
}
}
{"took":7,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":2,"max_score":0.51623213,"hits":[{"_index":"my_percolate_index","_type":"queries","_id":"1","_score":0.51623213,"_source":{
"query": {
"match": {
"full_name": {
"query": "John Smith",
"operator": "and"
}
}
}
}},{"_index":"my_percolate_index","_type":"queries","_id":"2","_score":0.2876821,"_source":{
"query": {
"match": {
"first_name": {
"query": "John"
}
}
}
}}]}}
Here's the mapping:
PUT books-index
{
"mappings": {
"books": {
"properties": {
"tags": {
"type": "nested",
"fields": {
"name": {
"type": "string"
},
"weight": {
"type": "float"
}
}
}
}
}
}
}
Then doing a nested query using a field_value_factor fails with an error
GET books-index/books/_search
{
"query": {
"nested": {
"path": "tags",
"score_mode": "sum",
"query": {
"function_score": {
"query": {
"match": {
"tags.name": "world"
}
},
"field_value_factor": {
"field": "weight"
}
}
}
}
}
}
The error: "nested: ElasticsearchException[Unable to find a field mapper for field [weight]]"
Interestingly, if there's one book in the index with tags - there's no error and the query works well.
Why is this happening? how can I prevent the error when there are no books with tags in the index?
Any ideas?
Thank you!
P.S. There's also an issue on github for this: https://github.com/elastic/elasticsearch/issues/12871
it looks like your mapping is incorrect.
After PUTing the mapping you provided, try executing GET books-index/_mapping, It will show these results:
"books-index": {
"mappings": {
"books": {
"properties": {
"tags": {
"type": "nested"
}
}
}
}
}
It's missing name and weight! The problem with the mapping is that you used either you used fields instead of properties, or you forget to put in a second properties key.
I modified your mapping to reflect that you were looking for a nested name and type within tags, as it looks like that is what your query wants.
PUT books-index
{
"mappings": {
"books": {
"properties": {
"tags": {
"type": "nested",
"properties": { // <--- HERE!
"name": {
"type": "string"
},
"weight": {
"type": "float"
}
}
}
}
}
}
}
So my problem is basically the same as described here, however it still remains unanswered on the group.
My mapping:
{
"abstract": {
"properties": {
"summary": {
"type": "string"
}
}
},
"authors": {
"type": "nested",
"properties": {
"first_name": {
"type": "string"
},
"last_name": {
"type": "string"
}
}
}
}
And I would like to perform a full-text search on both of these fields, probably unequally weighted. The query that comes to my mind, but unfortunately doesn't work, would be this:
{
"query": {
"bool": {
"should": [{
"multi_match": {
"query": "higgs boson",
"fields": ["abstract.summary^5", "author.last_name^2"]
}
}]
}
}
}
I don't get any results from the authors field, because of its nested mapping. I also can't get rid of the nested property - I use it for aggregations. Any elegant idea how to solve it?
The only solution that I managed to work out, which is not handy nor elegant but somehow works is such query:
"query": {
"bool": {
"should": [
{
"nested": {
"path": "authors",
"query": {
"multi_match": {
"query": "higgs",
"fields": ["last_name^2"]
}
}
}
},
{
"multi_match": {
"query": "higgs",
"fields": ["abstract.summary^5"]
}
}
]
}
}
I'm also not sure if the boosting will work as expected, providing it's set in different queries. Any suggestions appreciated.
Changing your mapping to the following one which uses include_in_root: true will allow you to use the query you original wrote:
{
"abstract": {
"properties": {
"summary": {
"type": "string"
}
}
},
"authors": {
"type": "nested",
"include_in_root": true,
"properties": {
"first_name": {
"type": "string"
},
"last_name": {
"type": "string"
}
}
}
}
You may want to index inner objects both as nested fields and as flattened object fields. This can be achieved by setting include_in_parent to true. - Link
Note: include_in_root may be deprecated in future versions of elasticsearch in favor of copy_to.