How to auto apply index policy to newly created indexes in AWS Elasticsearch - elasticsearch

Question

We push Nginx logs to AWS Elasticsearch using Filebeat and Logstash. We have created an index pattern with the name nginx-error-logs* & nginx-access-logs*. We can see in Kibana that daily new indices are being created based on the nginx log file date pattern. We created index policy and applied to existing indices but we would like to auto-apply the same ISM policy for all the newly created indices in Elasticsearch. Kindly help us to achieve this?
Is this the correct format to apply in Devtools console?
PUT _template/testindex_template
{
"index_patterns": ["*"],
"settings": {
"opendistro.index_state_management.policy_id": "index_lifecycle_management_policy"
}
}
Or should that be applied on the filebeat or Logstash config?

Answer 1

opendistro.index_state_management.policy_id is deprecated

Answer 2

opendistro.index_state_management.policy_id is deprecated
You have to add your index pattern in ism_template array of the policy. Below is the example.
PUT _opendistro/_ism/policies/policy_name
{
"policy": {
"description": "Policy to manage indices",
"default_state": "hot",
"states" : [
{
"name" : "hot",
"actions" : [
{
"rollover" : {
"min_size" : "20gb",
"min_index_age" : "2d"
}
}
]
}
],
"ism_template": {
"index_patterns": [
"nginx-error-logs*", // **sample index pattern**
"nginx-access-logs*"
],
"priority": 100
}
}
}
Whenever new index create, the index name pattern will match to the ism_template and the respective policy will be applied.
If same pattern available in multiple policy the it will attach the policy who has high prority.