I have been exploring Elastic Search lately.
I have been going through aliases. I see ES provides an API to create multiple aliases to a single index like below:
{ "actions" : [{ "add" : { "indices" : ["test1", "test2"], "alias" : "alias1" } }] }
Refer: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html#indices-aliases
I'm wondering what is the use case of this.
Won't the queries on aliases get split if an alias point to multiple indices?
I have tried getting the info, but failed to do so as everywhere it's being explained how to acheive this but not the use case.
Directing me to a resource where I could get more info would also help.
A possible use case is when your application has to switch from an
old index to a newindex with zero downtime.
Let's say you want to reindex an index because of some reasons and you're not using aliases with your index then you need to update your application to use the new index name.
How this is helpful?
Assume that your application is using the alias instead of an index name.
Let's create an index:
PUT /my_index
Create its alias:
PUT /my_index/_alias/my_index_alias
Now you've decided to reindex your index (maybe you want to change the existing mapping).
Once documents have been reindexed correctly, you can switch your alias to point to the new index.
Note: You need to remove the alias from the old index at the same time as we add it to the new index. You can do it using _aliases endpoint atomically.
A good read : elastic
As per your question usage of maintaining two aliases for a single index:
Create “views” on a subset of the documents in an index.
Using multiple indices having same alias:
Group multiple indices under same name, which is helpful if you want to perform a single query on multiple index at the same time.
But you can't insert/index data using this strategy.
Lets say that you have to types of events, eventA & eventB. You want to "partition" them by time, so you use alias to map multiple indices (e.g. eventA-20220920) to one alias ('eventA' in this case). And you want make one alias for all the event types, so you need to give all the eventA-* and eventB-* indices another alias 'event'.
That way when you add a third type of event (eventC) you can just add them to the 'event' alias and don't change your queries
Related
I have created the index template in elastic search 8.5.3. Here, I need the alias creation with a dynamic name.
For example, if the index name is, es.contact100 then the alias should be as.contact100.model.
I used this one,
{ "as-{index}": {} }
But, the output is, as-es.contact100
Please help me to resolve this.
I've an alias setup for rolling indices in elastic search. Let's call the alias : "alias" for now. It points to a number of indexes and rolls over after every 100gb. Now, let's say the number of fields in previous indices associated with alias is 100 and I've added one more field while writing to latest index. so, the number of fields become 101.
I've setup an index pattern by the name of "alias" and I can see all the indices listed via that index pattern but I am unable to visualize the 101th field I just added in the recent indices. Is there a way to do it ?
Please let me know if more details are needed regarding the same.
Hope you added the new field in the write index that your alias is pointing to, an alias can have only one write index but can have many read index and if you added the new field to a read index of your alias, you will not be able to visualise it using your alias.
I would like to create a one to many relashanship for the purpose of aggregations.
The "join" will be according to a field called "common_id":
When I create the first document belonging to the same group I would like to use it's flakeId (it's _id) as the common_id.
When adding other document belonging to the same group I would like to explicitly set the common_id to have the same value as the first document I added. This can be done by my app since my application will know the common_id of the first element.
My problem is with the first document:
How can i tell elasticsearch to copy the _id into common_id in a single call to elastic (I know I can do it using update script, or using two calls one for index and one for update... but this requires two requests instead of one).
I would like a simple syntax for this.
thanks
I'm using logstash to create an elastic index. The steps are :
1. logstash start
2. datas are retrieve with a jdbc input plugin
3. datas are indexed with an elasticsearch output plugin (with a template includes an alias)
4. logstash stop
The time, I've got an index call myindex-1 which can be requested with the alias myindex.
The second time, I've got an index call myindex-2 which can be requested with the alias myindex. The first index is now deprecated and I need to delete it just before (or after the step 4).
Do you know how to do this ?
First things first, if you know the deprecated index name, then it's just a question of adding a step 5:
curl -XDELETE 'http://localhost:9200/myindex-1'
So you'd wrap your logstash run into a script with this additional step - as to my knowledge there is no option for logstash to delete an index, it's simply not its purpose.
But from the way you describe your situation, it seems you're trying to keep the data available during the new index creation, could you elaborate a bit on your use case?
Reason for the asking is that with the current procedure, you're likely to end up with duplicate data (old and new version) during the indexing period.
If there is indeed a need to refresh the data, and assuming that you have an id in the data retrieved from the DB,
you might consider another approach: configuring 2 elasticsearch outputs in your logstash,
first one with action set to "delete" targeting the old entry in previous index,
second being your standard create into new index.
Depending on the nature of your data, there might also be other possibilities.
Create and populate myindex-2, don't alias it yet
Simultaneously add alias to myindex-2 and remove it from myalias-1
REST request for step 2:
POST /_aliases
{
"actions" : [
{ "remove" : { "index" : "myindex-1", "alias" : "myindex" } },
{ "add" : { "index" : "myindex-2", "alias" : "myindex" } }
]
}
Documentation here
So I have following problem which I'm trying to solve last two days. I have python script which parses logs and inserts data in elastic search, dynamically creating indices via bulk function.
Problem is my mapping has one "type": "nested" property, something like "users" field. And particularly when I'm only adding "type": "nested" in this property I can't query objects from Kibana nor creating any vizualization (because nested objects are separate documents If I'm not making mistakes). First think I tried: adding aditional "include_in_parent": true parameter to users field, but as result I got "wrong" queries (i.e. running something like +users.name: 'test' +users.age: 30) would result in ANY document which has those two fields, not exactly referring to ONE user object. Also vizualization was obviously wrong too.
Second solution I found was adding parent-child relationship. But this could be potentially be waste of time as I don't know will it result in correct queries. So I'm asking, if it will be normal solution to my problem?
Found out that Kibana doesn't support nested objects.
But ppadovani made this fork which supports this feature.
https://github.com/homeaway/kibana/tree/nestedSupport-4.5.4