Lets take the following scenario.
I have an alias A1 pointing to index I1. Now, I would like to use rollOver feature of ES and create index I2 and make alias point to I1 and I2.
Can I always keep rolling over and make my alias A1 point to last 2 indices or in general last 'n' indices ?
You can point one alias to multiple indices like this:
POST /_aliases
{
"actions" : [
{ "add" : { "indices" : ["l1", "l2"], "alias" : "A1" } }
]
}
or even point the alias to a wildcard index pattern like this:
POST /_aliases
{
"actions" : [
{ "add" : { "index" : "l*", "alias" : "A1" } }
]
}
EDIT: With rollover, you can only point the alias to one index - the latest index. If you want an alias that points to the last 2 indices, n indices, or all of the indices matching the pattern l*, you'll have to create an additional alias using the requests I showed above.
EDIT 2: If I wanted to maintain 30 days of logs in an index, this is how I would accomplish it. I stayed consistent with the naming of indices as 'l1' and alias of 'A1'. After the first 30 days, a new index will be created called l000002 (the naming convention is incrementing the number of the last index and zero padding with a length of 6) and the alias A1 will be pointing at the index l000002. I would create a second alias to refer to 'l*' like you originally desired.
PUT /l1
{ "aliases": { "A1": {} } }
POST /A1/_rollover
{ "conditions": { "max_age": "30d" } }
POST /_aliases
{
"actions" : [
{ "add" : { "index" : "l*", "alias" : "A2" } }
]
}
Related
I saw that following is the way to add aliases for one index.
https://www.elastic.co/guide/en/elasticsearch/reference/6.8/indices-aliases.html
Time series indices are usually formed everyday as per configuration, so how to give aliases those individual indices keeping the date part as it is?
If you scroll down the page you linked to a little bit, you'll find that you can do what you want using a glob pattern (i.e. time*)
POST /_aliases
{
"actions" : [
{ "add" : { "index" : "time*", "alias" : "all_time_indices" } }
]
}
Note, however, that if a new time series index is created it won't get the alias automatically. For that you'd need to set up an index template instead:
PUT _template/my-time-series
{
"index_patterns": ["time*"],
"aliases": {
"all_time_indices": {}
},
"settings": {
...
},
"mappings": {
...
}
}
I want to create an index and modify its setting with template and at the same time create an alias for it
"template_1" : {
"order" : 0,
"index_patterns" : [
"test*"
],
"settings" : {
"index" : {
"number_of_shards" : "2",
"number_of_replicas" : "2"
}
},
"mappings" : { },
"aliases" : {
"some-alias" : { }
}
}
}
when I am trying to put a document using alias, it tries to create an index with the alias name. However I am looking for something which will search for the index which has this alias and throws an error that there are no index exist with this alias
The problem is you are referencing multiple indexes with a single alias, so when you PUT a document ES does not know in which document to store it to.
Quoting the doc:
If no write index is specified and there are multiple indices referenced by an alias, then writes will not be allowed.
One solution, as per quote above, is to specify a write index (see docs) as the default destination for new documents (its also possible to specify rollover rules to update it).
The other solution, of course, is use the actual index name when putting docs.
I know it's possible to define two indices in an alias where one index has the is_write_index set to true while the other has it set to false -
POST /_aliases
{
"actions" : [
{
"add" : {
"index" : "test_index_1",
"alias" : "my_alias",
"is_write_index": true
}
}
]
}
POST /_aliases
{
"actions" : [
{
"add" : {
"index" : "test_index_2",
"alias" : "my_alias",
"is_write_index": false
}
}
]
}
As you can see, I've defined two indices test_index_1 and test_index_2 where the first one is a write index while the second one isn't.
Now, I want to be able to query the my_alias in such a way that searches happen only on the test_index_2 which has the is_write_index set to false while I write data to test_index_1, instead of reading from both the indices, which is the default behaviour. Meaning, I don't wish the search results come from the index where is_write_index is set to true.
Is this possible? I've tried setting index.blocks.read to true on the write index, but then search queries on the alias fail with an exception. Instead, I wish reads on the alias query only from that index which has the is_write_index set to false.
How can I achieve this?
This can be achieved by using filtered aliases.
The way you do this is you apply a custom filter while adding the write index to the alias. The filter property defines the bool condition based on which data is filtered on this index and presented as a new view of the dataset in this index. All search queries on this index happen on this new view that Elastic creates. So, if you want to avoid reading from the index you're currently writing to, apply a filter that is never satisfied across any documents in your dataset or an exists filter on some dummy field.
POST /_aliases
{
"actions": [
{
"add": {
"index": "test_index_2",
"alias": "my_alias",
"is_write_index": true,
"filter": {
"bool": {
"must_not": {
"exists": {
"field": "<field_that_always_exists_in_your_documents>"
}
}
}
}
}
}
]
}
Once you're done writing the data, update the alias by removing the filter property to allow reads from both the indices.
You are using this feature in an incorrect fashion. If you use alias for search, it will always attempt to read across all underlying indices. is_write_index is provided as feature to support rollover and index patterns, where writes are happening to 1 index, but reads happen across all indices with same alias or index pattern.
If your intent is to load data into one index, but allow application to continue to read from old index, when data loading is going on, you should use 2 separate alias - one for read and one for write and device a strategy to swap alias pointing to the indices, after your data loading is completed.
I've got an Elastic index transactions-internal and would like to point all the names like transactions-([a-z]+)-internal to this index using alias, so all the requests like
GET /transactions-a-internal/_search
GET /transactions-b-internal/_search
GET /transactions-c-internal/_search
...
etc
should give the same result as
GET /transactions-internal/_search
I've tried
POST /transactions-internal/_alias/transactions-*-internal
but it returned
Invalid alias name [...] must not contain the following characters [ , \", *, \\, <, |, ,, >, /, ?]
Is there any "smart" solution for that? I would strongly prefer co configure it on Elastic side, not anywhere else.
You're almost there. It's the other way around, i.e. POST /<index>/_alias/<alias>
POST /transactions-*-internal/_alias/transactions-internal
UPDATE:
If you want the other way around, then you can use the following (note that an alias name cannot contain wildcard characters):
POST /_aliases
{
"actions" : [
{ "add" : { "index" : "transactions-internal", "alias" : "transactions-a-internal" } },
{ "add" : { "index" : "transactions-internal", "alias" : "transactions-b-internal" } },
{ "add" : { "index" : "transactions-internal", "alias" : "transactions-c-internal" } }
]
}
Not quite sure if this is applicable to your situation but if you're starting from scratch a possible solution might be to use a index template.
PUT _index_template/transactions-internal
{
"priority": 1,
"template": {
"aliases": {
"transactions-internal": {}
}
},
"index_patterns": [
"transactions-*-internal"
],
"composed_of": []
}
As I'm quite new to elastic I don't know if this template can be applied to an existing index.But this approach will work for new indizes in v 7.12.1
POST /_aliases
{
"actions" : [
{ "add" : { "index" : "test*", "alias" : "all_test" } }
]
}
1)Based on the above configuration all the indices that start with test and also that are present in the cluster at that point of time will be kept in all_test alias.
2)If i create alias in Template by using below configuration
PUT localhost:9200/_template/test -d '
{
"template" : "test*",
"aliases" : {
"all_test" : {}
}
}
This will send the future indices which starts with test* to all_test alias.
Can i able to do 1+2 = I want to send the existing indices and the future indices to particular alias which i will define in ES Template?
If it is possible can you please share the sample template code for the above usecase?
Thanks