ElasticSearch - Sort does not work - sorting

I'm trying make a search and sort the results. However, I'm getting a error dont know why.
EDIT - I'll provide my full mappings.
"myindex": {
"mappings": {
"mytype": {
"dynamic_templates": [
{
// Dynamic templates here!
}
],
"properties": {
"fieldid": {
"type": "keyword",
"store": true
},
"fields": {
"properties": {
"myfield": {
"type": "text",
"fields": {
"sort": {
"type": "keyword",
"ignore_above": 256
}
},
"analyzer": "myanalyzer"
}
}
},
"isDirty": {
"type": "boolean"
}
}
}
}
}
}
When I performed a search with sorting, like this:
POST /index/_search
{
"sort": [
{ "myfield.sort" : {"order" : "asc"}}
]
}
I get the following error:
{
"error": {
"root_cause": [
{
"type": "query_shard_exception",
"reason": "No mapping found for [myfield.sort] in order to sort on",
"index_uuid": "VxyKnppiRJCrrnXfaGAEfA",
"index": "index"
}
]
"status": 400
}
I'm following the documentation on elasticsearch.
DOCUMENTATION
I also check this link:
DOCUMENTATION
Can someone provided me help?

Hmm it might be your mapping isn't set properly. I followed along using the following:
PUT /your-index/
{
"settings": {
"number_of_replicas": "1",
"number_of_shards": "3",
"analysis": {
"customanalyzer": {
"ID": {
"type": "custom",
"tokenizer": "keyword",
"filter": ["lowercase"]
}
}
}
},
"mappings": {
"thingy": {
"properties": {
"myfield": {
"type": "text",
"analyzer": "ID",
"fields": {
"sort": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
To double check if the index actually has myfield.sort field look at
GET /your-index/thingy/_mapping
Then, upload some document (no need to specify the sub-field(s), elasticsearch will do it for you)
POST /your-index/thingy/
{
"myfield": "some-value"
}
Now I can search with the following:
POST /your-index/thingy/_search
{
"sort": [
{ "myfield.sort": { "order": "asc" } }
]
}
So be sure to check:
Naming/typo's (you never know)
Your mapping (does it have the "myfield.sort" field)
Are you searching in the correct index?
Hopefully this helps

Related

more like this search is not working on field in list object

this is the mapping of my index i am searching on payload nested field category
{
"mappings": {
"date_detection": false,
"properties": {
"#class": {
"type": "keyword"
},
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"payload": {
"type": "nested",
"properties": {
"#class": {
"type": "keyword"
},
"description": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"title": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"category": {
"type": "keyword"
},
"value": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
This is my index mapping, Whenever i try to search object with more like this query in elasticsearch , it does not return any object,
**I am searching on list of object
**
the queries are
{
"query": {
"more_like_this": {
"fields": [
"payload.category"
],
"like": [
"ASSEST"
],
"min_term_freq": 1,
"max_query_terms": 12
}
}
}
It does not return any object but the values are present in elastic search
I just want to search similar object values present in elastic search thorough more like this query
But the Payload is actually list of objects which has filed category, i need to find similar objects according to it
For nested fields use nested query
GET <index-name>/_search
{
"query": {
"nested": {
"path": "payload",
"query": {
"more_like_this": {
"fields": [
"payload.category"
],
"like": [
"assest doc"
],
"min_term_freq": 1,
"max_query_terms": 12
}
}
}
}
}
Also look for min_doc_freq
The minimum document frequency below which the terms will be ignored from the input document. Defaults to 5.
If you have less than 5 matching documents set "min_doc_freq" to 1

Elasticsearch query for multiple terms

I am trying to create a search query that allows to search by name and type.
I have indexed the values, and my record in Elasticsearch look like this:
{
_index: "assets",
_type: "asset",
_id: "eAOEN28BcFmQazI-nngR",
_score: 1,
_source: {
name: "test.png",
mediaType: "IMAGE",
meta: {
content-type: "image/png",
width: 3348,
height: 1890,
},
createdAt: "2019-12-24T10:47:15.727Z",
updatedAt: "2019-12-24T10:47:15.727Z",
}
}
so how would I create for example, a query that finds all assets that have the name "test' and are images?
I tried multi_mach query but that did not return the correct results:
{
"query": {
"multi_match" : {
"query": "*test* IMAGE",
"type": "cross_fields",
"fields": [ "name", "mediaType" ],
"operator": "and"
}
}
}
The query above returns 0 results, and if I change the operator to "or" it returns all this assets of type IMAGE.
Any suggestions would be greatly appreciated. TIA!
EDIT: Added Mapping
Below is the mapping:
{
"assets": {
"aliases": {},
"mappings": {
"properties": {
"__v": {
"type": "long"
},
"createdAt": {
"type": "date"
},
"deleted": {
"type": "date"
},
"mediaType": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"meta": {
"properties": {
"content-type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"width": {
"type": "long"
},
"height": {
"type": "long"
}
}
},
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"originalName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"updatedAt": {
"type": "date"
}
}
},
"settings": {
"index": {
"creation_date": "1575884312237",
"number_of_shards": "1",
"number_of_replicas": "1",
"uuid": "nSiAoIIwQJqXQRTyqw9CSA",
"version": {
"created": "7030099"
},
"provided_name": "assets"
}
}
}
}
You are unnecessary using the wildcard expression for this simple query.
First, change your analyzer on name field.
You need to create a custom analyzer which replaces . with space as default standard analyzer doesn't do that, so that you when searching for test you get test.png as there will be both test and png in the inverted index. The main benefit of doing this is to avoid the regex queries which are very costly.
Updated mapping with custom analyzer which would do the work for you. Just update your mapping and re-index again all the doc.
{
"aliases": {},
"mappings": {
"properties": {
"__v": {
"type": "long"
},
"createdAt": {
"type": "date"
},
"deleted": {
"type": "date"
},
"mediaType": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"meta": {
"properties": {
"content-type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"width": {
"type": "long"
},
"height": {
"type": "long"
}
}
},
"name": {
"type": "text",
"analyzer" : "my_analyzer"
},
"originalName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"updatedAt": {
"type": "date"
}
}
},
"settings": {
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "standard",
"char_filter": [
"replace_dots"
]
}
},
"char_filter": {
"replace_dots": {
"type": "mapping",
"mappings": [
". => \\u0020"
]
}
}
},
"index": {
"number_of_shards": "1",
"number_of_replicas": "1"
}
}
}
Second, you should change your query to bool query as below:
{
"query": {
"bool": {
"must": [
{
"match": {
"name": "test"
}
},
{
"match": {
"mediaType.keyword": "IMAGE"
}
}
]
}
}
}
Which is using must with 2 match queries means, that it would return docs only when there is a match in all the clauses of must query.
I already tested my solution by creating the index, inserting a few sample docs and query them, let me know if you need any help.
Did you tried with best_fields ?
{
"query": {
"multi_match" : {
"query": "Will Smith",
"type": "best_fields",
"fields": [ "name", "mediaType" ],
"operator": "and"
}
}
}

ElasticSearch xcontent for unknown value of type class java.math.BigInteger

I have created the index with the following mapping in elasticsearch:
PUT my_master
{
"mappings": {
"documents": {
"properties": {
"fields": {
"type": "nested",
"properties": {
"uid": {
"type": "keyword"
},
"value": {
"type": "text",
"copy_to": "fulltext",
"fields": {
"raw": {
"type": "keyword",
"ignore_above": 32766
}
}
}
}
},
"fulltext": {
"type": "text"
}
}
}
}
}
Added a document into it:
POST my_master/documents/1/_create
{
"fields": [
{
"uid": "number",
"value": 111111111111111000000000000001100000000000000
}
]
}
After adding I am using update API to update document :
POST my_master/documents/1/_update
{
"doc":{
"fields": [
{
"uid": "number",
"value": 1111111111111110000000000000011000000000000000
}
]
}}
But elasticsearch gives me following error while updating the document:
cannot write xcontent for unknown value of type class java.math.BigInteger
Please help me here? What is wrong here? How to solve this issue?
according to the mapping i would expect that you send the value as a string, therefore i think this might work:
"1111111111111110000000000000011000000000000000"

Add mapping on elastic search index field

I would like to add anaylyser uax_url_email for search email field in elasticsearch document. However, I get the error.
Here is the code I am using to create this index
{
"user": {
"aliases": {},
"mappings": {
"user": {
"properties": {
"created": {
"type": "date"
},
"email": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
},
"settings": {
"index": {
"number_of_shards": "5",
"provided_name": "user",
"creation_date": "1521016015646",
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "my_tokenizer"
}
},
"tokenizer": {
"my_tokenizer": {
"type": "uax_url_email",
"max_token_length": "255"
}
}
},
"number_of_replicas": "1",
"uuid": "RS96V9gFQbG5UmoQ2R_gLA",
"version": {
"created": "6010099"
}
}
}
}
}
PUT /user/_mapping/email
{
"mappings": {
"_doc": {
"properties": {
"email": {
"type": "text",
"fields": {
"my_analyzer": {
"email": "text",
"analyzer": "my_analyzer"
}
}
}
}
}
}
}
I got an error stating "root_cause":
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Root mapping definition has unsupported parameters: [mappings : {_doc={properties={email={type=text, fields={my_analyzer={email=text, analyzer=my_analyzer}}}}}}]"
}
],
"type": "mapper_parsing_exception",
"reason": "Root mapping definition has unsupported parameters: [mappings : {_doc={properties={email={type=text, fields={my_analyzer={email=text, analyzer=my_analyzer}}}}}}]"
},
"status": 400
}
Nothing will be found. I want my analyzer and tokenizer work on email field any help will be highly appreciated
This should work:
PUT /user/_mapping/user
{
"properties": {
"email": {
"type": "text",
"fields": {
"my_analyzer": {
"type": "text",
"analyzer": "my_analyzer"
}
}
}
}
}
Your mistake was that you thought that the index type was _doc, but looking at your mapping, the index type is user. Basically, you have a user index with a user type.
The format of the command is PUT /[INDEX_NAME]/_mapping/[TYPE_NAME] {"properties:" { "[FIELD_TO_BE_UPDATED]": {....} } }.

Search by size of object type field elastic search

I have a mapping like this:
{
"post": {
"properties": {
"author_gender": {
"type": "string",
"index": "not_analyzed",
"omit_norms": true,
"index_options": "docs"
},
"author_link": {
"type": "string",
"index": "no"
},
"content": {
"type": "string"
},
"mentions": {
"properties": {
"id": {
"type": "integer"
},
"name": {
"type": "string"
},
"profile_image_url": {
"type": "string"
},
"screen_name": {
"type": "string"
}
}
}
}
}
I need to search by the size of the mentions object. I have tried this:
{
"filter": {
"script": {
"script": "doc['mentions'].values.length == 2"
}
}
}
This is not working. Gives an error
nested: ElasticSearchIllegalArgumentException[No field found for
[mentions] in mapping with types [post]];
I have also tried replacing the script part with doc['mentions.id'].value.length == 2. It is also erroring
nested: ArrayIndexOutOfBoundsException[10];
How to query records with mentions object size 2 ?
The elasticsearch guide recommends using size() instead of length for objects. So try this:
{
"filter": {
"script": {
"script": "doc['mentions.id'].values.size() == 2"
}
}
}
All the best.
ElasticSearch will not store array of objects like you think. Instead, multiple arrays are stored for each object properties.
So that's why you must use "sub array" instead.
For me, it works like this:
ES version - 7.11.1
I have mapping like this:
{
"items": {
"properties": {
"description": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"quantity": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"value": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
My Query is like this:
{
"filter": {
"script": {
"script": "doc['items.description.keyword'].size() == 2"
}
}
}
I don't know why but it seems to me that Elasticsearch stores array not as array of particular objects. Such pattern worked for me: I used instead
"script": "doc['some_array'].values.size()"
this
"script": "doc['some_array.any_field'].size()"
Should work:
"script": "doc['mentions.id'].size() == 2"
or
"script": "doc['mentions.id'].length == 2"

Resources