I want to create index alias in template file, I have specified index name as "test_2017_12_02" in logstash conf file, my template is as below
"aliases" : {"test_2017_12_02" : "test"}
but not working, the index getting created without alias
Try to use Dev Tools in kibana of elasticsearch.
POST /_aliases
{
"actions" : [
{ "add" : { "index" : "test_2017_12_02", "alias" : "test" } }
]
}
ref: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html
Related
I want to create an index and modify its setting with template and at the same time create an alias for it
"template_1" : {
"order" : 0,
"index_patterns" : [
"test*"
],
"settings" : {
"index" : {
"number_of_shards" : "2",
"number_of_replicas" : "2"
}
},
"mappings" : { },
"aliases" : {
"some-alias" : { }
}
}
}
when I am trying to put a document using alias, it tries to create an index with the alias name. However I am looking for something which will search for the index which has this alias and throws an error that there are no index exist with this alias
The problem is you are referencing multiple indexes with a single alias, so when you PUT a document ES does not know in which document to store it to.
Quoting the doc:
If no write index is specified and there are multiple indices referenced by an alias, then writes will not be allowed.
One solution, as per quote above, is to specify a write index (see docs) as the default destination for new documents (its also possible to specify rollover rules to update it).
The other solution, of course, is use the actual index name when putting docs.
I have a bulk upload for a new index that I'm sending to my ES cluster from logstash. As such I want replication and refreshing turned off until the load is done, and I'll re-enable those values after the upload is complete.
I have a config file that looks like the following
input {
stdin { type => stdin }
}
filter {
csv {
separator => " "
columns => [ ...]
}
}
output {
amazon_es {
hosts =>
["my-domain.us-east-1.es.amazonaws.com"]
index => "my-index"
template => "conf/my-index-template.json"
template_name => "my-index-template-name"
region => "us-east-1"
}
}
And the template file looks like
{
"template" : "my-index-template-name",
"mappings" : {
...
},
"settings" : {
"index" : {
"number_of_shards" : "48",
"number_of_replicas" : "0",
"refresh_interval": "-1"
}
}
}
And when I run logstash and go to look at the settings for that index, the mappings are all respected from this template which is good, but everything in the settings section is ignored and it takes on default values (i.e. number_of_shards=5, and number_of_replicas=1)
Some investigation notes:
If I get the template after it's installed from ES itself I see the proper values in the template (for both mappings and settings). They just don't seem to be applying to the index
Also if I take the contents of the template file and create the index manually w/ a PUT it shows up as I would expect
My logstash version is 7.3.0 and my elasticsearch version is 6.7
Not sure what I'm doing wrong here
Your index name is my-index, but the template setting in your mapping uses my-index-template-name, it needs to be a regular expression or the same name as your index.
Since you are using elasticsearch 6.7 you should use index_patterns instead of template in your mapping.
{
"index_patterns" : ["my-index"],
"mappings" : {
...
},
"settings" : {
"index" : {
"number_of_shards" : "48",
"number_of_replicas" : "0",
"refresh_interval": "-1"
}
}
}
POST /_aliases
{
"actions" : [
{ "add" : { "index" : "test*", "alias" : "all_test" } }
]
}
1)Based on the above configuration all the indices that start with test and also that are present in the cluster at that point of time will be kept in all_test alias.
2)If i create alias in Template by using below configuration
PUT localhost:9200/_template/test -d '
{
"template" : "test*",
"aliases" : {
"all_test" : {}
}
}
This will send the future indices which starts with test* to all_test alias.
Can i able to do 1+2 = I want to send the existing indices and the future indices to particular alias which i will define in ES Template?
If it is possible can you please share the sample template code for the above usecase?
Thanks
I would like to change the type of the field location to geo_point. I'm using ES with Logstash, as y'all know, indices are generated with the name logstash-yyyy-mm-dd
I first created a logstash index and named it logstash-2016-03-29, like so:
curl -XPUT 'http://localhost:9200/logstash-2016-03-29'
then, I changed the mapping for supposedly all the indices called Logstash-* using the following:
curl -XPOST "http://localhost:9200/logstash-*/_mapping/logs" -d '{
"properties" : {
"location" : { "type":"geo_point" }
}
}'
And when I ran the Logstash configuration file, all the location fields in the index logstash-2016-03-29 were indeed of type geo_point.
However, today, the auto-generated index logstash-2016-03-30 had field location of type String instead of geo_point. I thought the type should be applied on ANY index that starts with the name logstash-*. Apparently, I was wrong. How can I fix this so that any future index created by logstash that have the location field would have that field type set to geo_point instead of String?
Thanks.
You should define it using the index template
curl -XPUT localhost:9200/_template/template_2 -d '
{
"template" : "logstash-",
"mappings" : {
"logs" : {
"properties": {
"location" : { "type" : "geo_point" }
}
}
}
}
https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html
How do you add an alias to an index using a template?
If I look at my current templates there are empty sections named aliases in mappings and in the parent of mappings. I can't seem to find much documentation indicating their use. What I'm hoping to do is not have to execute an alias add after the index is created.
Thanks,
Pat
When creating an index template, the aliases section contains the aliases that you want to be created at the same time a new index is created. For instance, with the command below, every time a new index whose name matches index* is created, then the aliases named my_alias1 and my_alias2 are also created at the same time.
curl -XPUT localhost:9200/_template/my_template -d '
{
"template" : "index*",
"settings" : {
"number_of_shards" : 1
},
"aliases" : {
"my_alias1" : {},
"my_alias2" : {}
}
}'
UPDATE
Note that as of ES6, template has been renamed to index_patterns:
curl -XPUT localhost:9200/_template/my_template -d '
{
"index_patterns" : "index*",
"settings" : {
"number_of_shards" : 1
},
"aliases" : {
"my_alias1" : {},
"my_alias2" : {}
}
}'