In SOLR I can use In-Place-Update to update the value of any fields. Here the value of popularity field will be incremented every time by 20 and add it to the the current value of popularity without considering any other things.
{
"id":"mydoc",
"price":{"set":99},
"popularity":{"inc":20}
}
For Elasticsearch, I can also use the _update api using script to In-Place-update.
POST /website/blog/1/_update
{
"script" : "ctx._source.popularity+=20"
}
But my problem is I want to use the _bulk api using python to in-place-update multiple documents at one with some incremental values. Here I've seen the documentation on how to use the _bulk endpoint to set different values with update action payload. I'm just having some difficulties how can I make the same POST JSON datasets for _bulk with python elasticsearch client for the script update way.
Related
I am fairly new to the elastic search, and I am using the Elastic App Search.
So I am trying to update data in elastic app-search through MongoDB Realm App which basically provide triggers on CRUD operations.
I am able to add documents or update existing fields.
But the problem is I am unable to add elements to the array field. I want to add or delete elements from array, after some research I found out that it can be done using some scripts:
"script": {
"source": "ctx._source.fieldToUpdate.add(elementToAdd);",
"lang": "painless"
}
But it's just not working. I am using REST APIs to add or update data in elastic app search. And I am using elastic cloud managed service.
UPDATE - 1
I was using ES App Search, and I created and named the engine as "articles", when I tried to run queries using kibana, I had to use some weird name ".ent-search-engine-documents-article".
So I tried using the same name in Elastic Search REST API
POST /.ent-search-engine-documents-article/_update/docid
And it worked perfectly fine, but I want to perform the same work using REST API of APP Search only.
To perform CRUD operations on your data stored through AppSearch, you should use the Documents API.
AppSearch does not handle nested objects and only provides 4 field types: text, number, date and geolocation. if you are posting objects, it will flatten and stringify them as you described in your comment.
It's also the case for arrays, so you can't just add elements to a field that holds an array as it's just a text field, you need to re-write the whole field (though it does detect them as arrays and handles each element separately if you use that field as a facet for instance).
as for how to patch with the AppSearch REST API, here's a small example inspired from the official documentation:
curl -X PATCH 'https://[instance id].ent-search.[region].[provider].cloud.es.io:443/api/as/v1/engines/articles/documents' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer private-xxxxxxxxxxxxxxxxxxxx' \
-d '[
{ "id": "your_article_id",
"source": "your article source",
"lang": "painless" }
]'
There are also clients for several programming languages that you may find helpful or more intuitive to use.
The weird names you see for your engines, like ".ent-search-engine-documents-article" are the underlying indices on ElasticSearch, and you normally would not like to manipulate them directly.
I am using Elasticsearch 7.4, and have a unique requirement to perform below operation in elasticsearch through painless script:-
Update the field through painless script
Get the updated value in response
#1 I can perform using painless script using below to update the counter value, but how can I get the updated value in response? I want the new value ctx._source.counter in response of the POST call I made, Is there any way?
curl -XPOST -H "Content-Type:application/json" localhost:9200/myindex/_update_by_query -d "{"query":{"bool":{"filter":[{"match":{"field1":"1"}}]}},"script":{"source":"ctx._source.counter = ctx._source.counter + params.incrementValue","lang":"painless","params":{"incrementValue":"2"}}}"
What you want is different from the specs of the _update_by_query API. In the specs you can see the response only contains stats & status of the update, and not the document body (neither initial or final).
One way to accomplish your goal is:
Query document to update ie. where field1==1
Get initial counter value
Update
With this, you can get compute the updated value at your end with the method of your choice. In any case it would be two API calls not just one.
I am taking some courses on udemy on elastic search and trying to set up an elastic search project. I have gotten the bulk api to work and can successfully send batch data into an index on elastic search. But I am having trouble sending data without the bulk api. Because I have read on the elastic search docs that there is a 'false' analogy that a type is like a table in a database and an index is like a database. I decided for this project to create an index for each of the entities that I want to persist which is users and statistics. Therefore I have indices which are called statisitics and users. When I make the following request from postman
headers: Content-Type application/json
POST http://localhost:9200/users
with body:
{"id": 1, "name":"myname"}
I get an error
{"error":"Incorrect HTTP method for uri [/users] and method [POST],
allowed: [PUT, DELETE, GET, HEAD]","status":405}
How can I allow this http method?
Hello I had the same issue using PHP and JAVA, this happen simply because you must add the _type after the index name (described here) :
https://www.elastic.co/guide/en/elasticsearch/guide/current/index-doc.html
Try curl -XPOST http://localhost:9200/users/_admin or something similar ;)
See https://www.elastic.co/guide/en/elasticsearch/guide/current/mapping.html
Since version 6.2, we need to use _doc for the type.
https://discuss.elastic.co/t/cant-use-doc-as-type-despite-it-being-declared-the-preferred-method/113837
https://www.elastic.co/guide/en/elasticsearch/reference/master/removal-of-types.html#_schedule_for_removal_of_mapping_types
In fact, when you say there is "no types" allowed in ElasticSearch, it is false (at least for versions up to 8.x). What is true is that there is only 1 type allowed for each index.
I am using Elasticsearch 6.2.
I have an index products with index_type productA having data with following structure:
{
"id": 1,
"parts": ["part1", "part2",...]
.....
.....
}
Now during the query time, I want to add or project a field parts_count to the response which simply represents the number of parts i.e the length of parts array. Also, if possible, I would also like to sort the documents of productA based on the generated field parts_count.
I have gone through most of the docs but haven't found a way to achieve this.
Note:
I don't want to update the mapping and add dynamic fields. I am not sure if Elasticsearch allows it. I just wanted to mention it.
Did you read about Script Fields and on Script Based Sorting?
I think you should be able to achieve both things and this not require any mapping updates.
Is it possible to specify a script be executed when inserting a document into ElasticSearch using its Index API? This functionality exists when updating an existing document with new information using its Update API, by passing in a script attribute in the HTTP request body. I think it would be useful too in the Index API because perhaps there are some fields the user wants to be auto-calculated and populated during insertion, without having to send an additional Update request after the insertion to have the script be executed.
Elasticsearch 1.3
If you just need to search/filter on the fields that you'd like to add, the mapping transform capabilities that were added into 1.3.0 could possibly work for you:
The document can be transformed before it is indexed by registering a
script in the transform element of the mapping. The result of the
transform is indexed but the original source is stored in the _source
field.
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-transform.html
You can also have the same transformation run when you get a document as well by adding the _source_transform url parameter to the request:
The get endpoint will retransform the source if the _source_transform
parameter is set.The transform is performed before any source
filtering but it is mostly designed to make it easy to see what was
passed to the index for debugging.
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/_get_transformed.html
However, I don't think the _search endpoint accepts the _source_transform url parameter so I don't think you can apply the transformation to search results. That would be a nice feature request.
Elasticsearch 1.4
Elasticsearch 1.4 added a couple features which makes all this much nicer. As you mentioned, the update API allows you to specify a script to be executed. The update API in 1.4 can also accept a default document to be used in the case of an upsert. From the 1.4 docs:
curl -XPOST 'localhost:9200/test/type1/1/_update' -d '{
"script" : "ctx._source.counter += count",
"params" : {
"count" : 4
},
"upsert" : {
"counter" : 1
}
}'
In the example above, if the document doesn't exist it uses the contents of the upsert key to initialize the document. So in the case above the counter key in the newly created document will have a value of 1.
Now, if we set scripted_upsert to true (scripted_upsert is another new option in 1.4), our script will run against the newly initialized document:
curl -XPOST 'localhost:9200/test/type1/2/_update' -d '{
"script": "ctx._source.counter += count",
"params": {
"count": 4
},
"upsert": {
"counter": 1
},
"scripted_upsert": true
}'
In this example, if the document didn't exist the counter key would have a value of 5.
Full documentation from Elasticsearch site.