Hi I'm trying to update the ttl of a document with the following way but it seems that it is not getting updated:
POST /my_index/my_type/AU4Gd1DVbqjanfsolMgP/_update
{
"doc": {
"_ttl": 60000
},
"doc_as_upsert": true
}
With the script way it is getting updated normally.. What is the problem? does anyone know?
I think you can only update it through script. From the documentation:
It also allows to update the ttl of a document using ctx._ttl and timestamp using ctx._timestamp. Note that if the timestamp is not updated and not extracted from the _source it will be set to the update date.
In addition to _source, the following variables are available through the ctx map: _index, _type, _id, _version, _routing, _parent, _timestamp, _ttl.
Related
I am implementing ElasticSearch 7.1.1 in my application using Python requests library. I have successfully created a document in the elastic index using
r = requests.put(url, auth=awsauth, json=document, headers=headers)
However, while updating an existing document, the JSON body(containing to be updated values) that I pass to the method, replaces the original document. How do I overcome this? Thank you.
You could do the following:
document = {
"doc": {
"field_1": "value_1",
"field_2": "value_2"
},
"doc_as_upsert": True
}
...
r = requests.post(url, auth=awsauth, json=document, headers=headers)
It should be POST instead of PUT
You can update existing fields and also add new fields.
Refer the doc in the comment posted by Nishant Saini.
I have a field that was originally set to "index": false because we did not think we would ever have to query on this particular field. Its now about 9 months later and we have a new feature request that is going to require us to query on this field.
I know ES offers some nice features like fields that allow you to add more functionality to a field, but it does not appear to allow you to go from index = false to index = true by simply adding a sub-field.
After some googling I was not able to find a solution to this issue that doesn't involve either 1) creating a new field altogether or 2) re-indexing all of the data.
Does anyone know of a clean/side effect-free way of adding this kind of functionality to an existing field? If not, what is the suggested process?
Here is what the current field looks like:
{
"mappings": {
"entity": {
"properties": {
"contentType": {
"type": "keyword",
"index": false
}
}
}
}
}
Like I said I am looking to find the cleanest way of changing "index" to true
Thanks!
You need to create a new index and reindex I’m afraid.
Reindex API can be a great help though.
I have documents which contains only "url"(analyzed) and "respsize"(not_analyzed) fields at first. I want to update documents that match the url and add new field "category"
I mean;
at first doc1:
{
"url":"http://stackoverflow.com/users/4005632/mehmet-yener-yilmaz",
"respsize":"500"
}
I have an external data and I know "stackoverflow.com" belongs to category 10,
And I need to update the doc, and make it like:
{
"url":"http://stackoverflow.com/users/4005632/mehmet-yener-yilmaz",
"respsize":"500",
"category":"10"
}
Of course I will do this all documents which url fields has "stackoverflow.com"
and I need the update each doc oly once.. Because category data of url is not changeable, no need to update again.
I need to use _update api with _version number to check it but cant compose the dsl query.
EDIT
I run this and looks works fine:
But documents not changed..
Although query result looks true, new field not added to docs, need refresh or etc?
You could use the update by query plugin in order to do just that. The idea is to select all document without a category and whose url matches a certain string and add the category you wish.
curl -XPOST 'localhost:9200/webproxylog/_update_by_query' -H "Content-Type: application/json" -d '
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"url": "stackoverflow.com"
}
},
{
"missing": {
"field": "category"
}
}
]
}
}
}
},
"script" : "ctx._source.category = \"10\";"
}'
After running this, all your documents with url: stackoverflow.com that don't have a category, will get category: 10. You can run the same query again later to fix new stackoverflow.com documents that have been indexed in the meantime.
Also make sure to enable scripting in elasticsearch.yml and restart ES:
script.inline: on
script.indexed: on
In the script, you're free to add as many fields as you want, e.g.
...
"script" : "ctx._source.category1 = \"10\"; ctx._source.category2 = \"20\";"
UPDATE
ES 2.3 now features the update by query functionality. You can still use the above query exactly as is and it will work (except that filtered and missing are deprecated, but still working ;).
That all sounds great but just to add to #Val answer, Update By Query is available form ElasticSearch 2.x but not for earlier versions. In our case we're using 1.4 for legacy reasons and there is no chance of upgrading in forseeable future so another solution is using the Update by query plugin provided here: https://github.com/yakaz/elasticsearch-action-updatebyquery
Question is pretty straight forward, is there a way to enable TTL on the index level. effectively means all types created under this index will inherit an enabled TTL.
on the documentation it is said that "You can provide a per index/type default _ttl value as follows", but I wasn't able to request TTL on an index level.
in case it isn't possible, what workaround can be suggested ? in our environment new types are created all the time, and the data has to be removed after it is not needed anymore.
You can accomplish this using default option under mapping. Under an index , if you put any configuration under_default_ it would be applied to all the mappings whose these configurations not defined under the same index.
curl -XPUT "http://localhost:9200/test_index" -d'{
"mappings": {
"_default_": {
"_ttl": {
"enabled": true
}
}
}
}'
If I change the mapping so certain properties have new/different boost values, does that work even if the documents have already been indexed? Or do the boost values get applied when the document is indexed?
You cannot change field level boost factors after indexing data. It's not even possible for new data to be indexed once the same fields have been indexed already for previous data.
The only way to change the boost factor is to reindex your data. The pattern to do this without changing the code of your application is to use aliases. An alias points to a specific index. In case you want to change the index, you create a new index, then reindex data from the old index to the new index and finally you change the alias to point to the new index. Reindexing data is either supported by the elasticsearch library or can be achieved with a scan/scroll.
First version of mapping
Index: items_v1
Alias: items -> items_v1
Change necessary, sencond version of the index with new field level boost values :
Create new index: items_v2
Reindex data: items_v1 => items_v2
Change alias: items -> items_v2
This might be useful in other situations where you want to change your mapping.
Field level boosts are, however, not recommended. The better approach is to use boosting at query time.
Alias commands are:
Adding an alias
POST /_aliases
{
"actions": [
{ "add": {
"alias": "tems",
"index": "items_v1"
}}
]
}
Removing an alias
POST /_aliases
{
"actions": [
{ "remove": {
"alias": "tems",
"index": "items_v1"
}}
]
}
They do not.
Index time boosting is generally not recommended. Instead, you should do your boosting when you search.