We have some Elastic rollover indices managed by ILM. Mappings are defined in templates.
When we try to add mappings fields concepts collide:
The obvious way is (1) update the template and (2) provoke a rollover so the new index has the new mapping. BUT rolling over an ILM managed index breaks things. _ilm/explain will tell you index [blah] is not the write index for alias [blah-000004].
So... what do people do?
Directly updating the write index mapping might work, but it feels like there should be a better option?
This is Elastic 6.8.
Related
I have an index which a few fields as keyword type. I need ot have these fields as text instead now.
Going through documentation, it seems to be not possible.
Documentation instead asks to create a new index and reindex it with documents from older index.
Can I keep new index name same as old one? Won't it cause issues during reindexing process?
no, you need to reindex to an index with a different name. One thing you could do, is to (1) reindex to e.g. original_index_name_v2, (2) create an index alias named original_index_name catching original_index_name_* indices, (3) delete the original index. This way, next time you'll need to change the mapping, you don't need to change the index name but just keep querying the same alias
I am working on a task of reindexing my Elastic search indexes in case any change happens. There are 2 ways that I can find to implement this but they look same to me unless I am missing something.
I am getting data to my Elastic search service from Postgres of service B, which has a paginated endpoint.
Approach 1:
Create alias which will point to our existing index.
When reindex is triggered, create a new index and once the reindexing is complete, point the alias, which was pointing to old index, to the newly created index.
Delete the old index.
Approach 2:
Create a new Index.
Use the reindex API to copy the data from old index to new index, which will apply the new changes to the old documents.
To me, both of these look same. Disadvantage of using approach 2 seems that it will create a new index name, hence we will have to change the index names while querying.
Also, considering my reindexing operation would not be a frequent task, I am reading the data from a paginated endpoint and then creating indexes again, Approach 1 seems to make more sense to me.
In approach1, you are using alias. In approach 2, you are not using alias.
Both would be same if you add alias to approach2 as step3 and step4 - delete the old index.
Refer As you need to do little often.
I'm trying to update settings for existing index with custom analyzer, but it is not allowing for open index, and aws is not allowing me to close the index.
Any thoughts?
AWS Elasticsearch only supports a subset of operations and indeed _close is not supported. You can get the list of supported operation here.
Since you are updating an analyzer you will probably have to reindex your documents, so you can create a new index with the correct mapping and use _reindex endpoint to copy the documents.
If you are not already doing it, I would advise you to use index alias to handle the index migration.
Is there a Recipe out there to Reindex all ElasticSearch Indices with Curator?
I'm seeing that it can Reindex a set of indices into one (Daily to Month use case), however I don't see anything that would suggest it could easily apply a new mapping file to every Elastic Index.
I'm taking a guess I'll need to write a wrapper script around Curator to grab index names and feed them into Curator.
I don't know if I got you right as you mentioned reindexing and mapping changes...
If you want to set/update a mapping in a collection of indices and if you know the indices to update by name (or pattern), you are able to apply the same mapping or a mapping change at once with https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html#_multi_index_2
For reindexing, there is no way to specify multiple source/target pairs at once but you can split one index into many. But as you sugessted, you can use subsequent calls to the reindex api.
BTW: The reindex api does not copy the settings nor mappings from the source into the destination index. You need to handle it by yourself, maybe using https://www.elastic.co/guide/en/elasticsearch/reference/6.4/indices-templates.html
I have a question about elasticsearch index template, there is a scene of my question.
Create a template for a series indices, named templateA, and there are some indices create from this template, named Index-yyyy.mm.dd2 and Index-yyyy.mm.dd2. After a period of time, I need create some new fields in indice, and I update the templateA.
SO, How to make the previously created indices use the new template? please give me some suggestion. Thanks a lot!
The template is only used at index creation. You'll have to modify your mapping or recreate your index and reindex your data.
You can use the PUT mapping API to modify your mapping.
At least in ElasticSearch 7.15 you could create or update an index template using the same endpoint, also:
Index templates are applied during data stream or index creation
It is ovious but "old" data needs to be updated in some way.
In case you are using Logstash, just restart it to do the reindex.