I am using ElasticSearch 2.x with Nest 2 in my project.
I am facing a issue in which i need to update the nested records , but Elasticsearch doesn't do that, instead it deletes the records and re-index them.
So because of this scenario, i need to always send all the nested records along with the updated one to update the nested records.
So has anyone of you has a solution of this? Can i only update the record without re-indexing all records?
Thanks for your help in advance..!!
Try this,
It works for me
POST /yourindex/type/_id/_update
{
"script" : {
"inline" : "if (ctx._source.yourarray == null || ctx._source.yourarray.size() == 0){ ctx._source.yourarray = params.uuuser} else {ctx._source.yourarray.add(params.newarray[0]) } ",
"params" : {
"newarray" :[
{"c1":"dfgfgsdf",
"c2":"can2",
"ce":" can2#can.co",
"cp":475522778,
"d1":[
{
"e1":"fffff",
"ffff":[{"g1":"hhhhh"},{"g2":"iiiiii"}]
}
]
}
]
}
}
}
Related
I have field in index with mapping as :
"sequence_number" : {
"type" : "long",
"copy_to" : [
"_custom_all"
]
}
and using search query as
POST /my_index/_search
{
"query": {
"term": {
"sequence_number": {
"value": "we"
}
}
}
}
I am getting error message :
,"index_uuid":"FTAW8qoYTPeTj-cbC5iTRw","index":"my_index","caused_by":{"type":"number_format_exception","reason":"For input string: \"we\""}}}]},"status":400}
at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:260) ~[elasticsearch-rest-client-7.1.1.jar:7.1.1]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:238) ~[elasticsearch-rest-client-7.1.1.jar:7.1.1]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:212) ~[elasticsearch-rest-client-7.1.1.jar:7.1.1]
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1433) ~[elasticsearch-rest-high-level-client-7.1.1.jar:7.1.1]
at
How can i ignore number_format_exception errors, so the query just doesn't return anything or ignores this filter in particular - either is acceptable.
Thanks in advance.
What you are looking for is not possible, ideally, you should have coherce enabled on your numeric fields so that your index doesn't contain dirty data.
The best solution is that in your application which generated the Elasticsearch query(you should have a check for NumberFormatExcepton if you are searching for numeric fields as your index doesn't contain the dirty data in the first place and reject the query if you get an exception in your application).
Edit: Another interesting approach is to validate the data before inserting into ES, using the Validate API as suggested by #prakash, only thing is that it would add another network call but if your application is not latency-sensitive, it can be used as a workaround.
I want to delete the most current item in my Elasticsearch index sorted by myDateField which is a date type. Is that possible? I want something like this query but this would delete all matching items even though I have the size at 1.
{
"query" : {
"match_all" : {
}
},
"size" : "1",
"sort" : [
{
"myDateField" : {
"order" : "desc"
}
}
]
}
Delete by query is unlikely to support any sorting features.
If you try Delete by query - however you'll get the error: request does not support [sort]. I couldn't find any documentation saying that the "sort" parameter is not supported in delete by query.
I've one idea to do it but don't know it's the best way or not?
Step 1: Do a normal query based on your conditions+sorting and get those ids.
Step 2: Build a bulk query to delete all documents retrieved above by id those you got on Step 1.
When a time-based index is added to kibana, you have to pick the field that will act as a time field. If you want to switch from one field to another, normally I would delete the index and re-add it back. But you end up loosing scripted fields and filed formatting this way.
Is there any way to modify the existing index time field without loosing scripted fields/formatting?
It can probably be done by messing around directly with /.kibana/index-pattern/index_pattern_name but all my attempts with changing timeFieldName directly ended up dropping scripted fields.
This is what worked for me on Kibana 7.7.0 (Elastic Cloud):
find the {id} of the document whose title field corresponds to the index you want to make change
GET .kibana/_search
{
"query": {
"match": {
"index-pattern.title": "{NAME_OF_THE_INDEX}"
}
}
}
change the timefield with following code
POST .kibana/_update/{id}
{
"doc": {
"index-pattern": {
"timeFieldName" : "{NEW_TIME_FIELD_NAME}"
}
}
}
The easiest way seems to update the corresponding document:
POST /.kibana/index-pattern/YOUR_INDEX_NAME/_update
{
"doc": {
"timeFieldName": "NEW_TIME_FIELD_NAME"
}
}
It should preserve scripted fields.
This doesn't seem to work on Kibana 5.
Instead, following is the way worked on Kibana 5.
1. find the {id} of the document whose title field corresponds to the index you want to make change
GET .kibana/index-pattern/_search
{
"_source" : "_id",
"query" : {
"match" : {
"title": "{NAME_OF_THE_INDEX}"
}
}
}
2. change the timefield with following code
POST /.kibana/index-pattern/{id}/_update
{
"doc": {
"timeFieldName" : "{NEW_TIME_FIELD_NAME}"
}
}
This worked fine with me on Kibana 5.
In ElasticSearch I'm using upsert to update a document that may not exist:
POST /website/pageviews/1/_update
{
"script" : "ctx._source.online+=1",
"upsert": {
"online": 1
}
}
Since my data are going to change frequently I want to remove my document if online == 0.
It would be useless to use update if I need to get the document and check online value every time, and I don't want to accumulate a lot of trash documents.
Which is the best way to remove my document when online == 0? Something like:
POST /website/pageviews/1/_update
{
"script" : "ctx._source.online-=1",
"remove_doc": "ctx._source.online == 0"
}
You can use the delete operation like this:
POST /website/pageviews/1/_update
{
"script" : "if (online == 0) { ctx.op = 'delete' } else { ctx._source.online += 1 }",
"upsert": {
"online": 1
}
}
I'm new to ElasticSearch and was wondering, once you define a script field with mvel syntax, can you subsequently filter on, or refer to it in the query body as if it was any other field?
I can't find any examples of this while same time I don't see any mention of whether this is possible on the docs page
http://www.elasticsearch.org/guide/reference/modules/scripting/
http://www.elasticsearch.org/guide/reference/api/search/script-fields/
The book ElasticSearch Server doesn't mention if this is possible or not either
As for 2018 and Elastic 6.2 it is still not possible to filter by fields defined with script_fields, however, you can define custom script filter for the same purpose. For example, lets assume that you've defined the following script field:
{
"script_fields" : {
"some_date_fld_year":"doc["some_date_fld"].empty ? null : doc["some_date_fld"].date.year"
}
}
you can filter by it with
{
"query": {
"bool" : {
"must" : {
"script" : {
"script" : {
"source": " (doc["some_date_fld"].empty ? null : doc["some_date_fld"].date.year) >= 2017",
"lang": "painless"
}
}
}
}
}
}
It's not possible for one simple reason: the script_fields are calculated during final stage of search (fetch phase) and only for the records that you retrieve (top 10 by default). The script filter is applied to all records that were not filtered out by preceding filters and it happens during query phase, which precedes the fetch phase. In other words, when filters are applied the script_fields don't exist yet.