I am new to elastic search. I have installed elastic search in my local machine. when I add any document, the result shards total is 2, when I see all indices through _cat API primary and replica are showing as 1 only. But by default shards should be 5, but its showing only 1 for me. for every index default shards are 1 only, I didn't changed any configuration.
So starting with Elasticsearch version 7.0 the default number of shards was reduced from 5 to 1.
You can see the difference by comparing the version 6.8 and version 7.0 of the documentation.
If you still want to have 5 shards created for your index, you have to create it like the following:
PUT my_index
{
"settings": {
"number_of_shards": 5,
"number_of_replicas": 1
}
}
The resulting shards after indexing a document is two because you have also one replica. Since you operate a single-node-cluster the replica shard can not be allocated onto another node and therefore only one shard succeeded.
Related
I have an issue with the shards in Elastic Search. Whenever I create my index it creates an unassigned node with unassigned shards as shown below.
Can anyone please help me out here? Any help is much appreciated.
The reason is simply because you have only one node and one shard replica is always created by default.
In order to prevent this, you can remove the replicas from your index by calling this:
PUT suppliercloudproductindex1/_settings
{
"index.number_of_replicas": 0
}
Alternately, you can also specify this setting when initially creating your index in order to make sure that no replica shard gets created:
PUT suppliercloudproductindex1
{
"settings": {
"index.number_of_replicas": 0
}
}
PUT */_settings { "index.number_of_replicas": 0 }
This will update all indexes to have zero replicas. This will only work if wild card operations are allowed. Otherwise the command has to be run on every index in your cluster.
I wanted to ask a question about Elasticsearch making 5 shards in each index by default. Well for some reason this is not the case for me. I was wondering whether it was an error on my side (even though I didn't make any changes to the custom template) or this is no longer a case (no longer 5 shards defaultly for each index)? I didn't find anything in documentation or in internet about it. I know I can change this by running:
PUT _template/default
{
"index_patterns": ["*"],
"order": -1,
"settings": {
"number_of_shards": "5",
"number_of_replicas": "1"
}
}
however this is not my point I just wanted to learn what is current way of working for Elasticsearch.
Thanks for all answers!
From the 7.x version, the default number of primary shard in each index is 1, as mentioned here in the documentation
Before the 7.x version, the default number of primary shared for each index were 5
You can refer to the breaking changes of the elasticsearch 7.0.0 version here
Index creation no longer defaults to five shards Previous versions of
Elasticsearch defaulted to creating five shards per index. Starting
with 7.0.0, the default is now one shard per index.
I have a newly setup Elasticsearch 7.5.2 cluster. When I create an index, only one shard is created default for it.
My cluster strategy is as below:
Total Nodes: 5
--------------
Node 1 & Node 2 - Master Only
Node 3 - Master & Data Node
Node 4 & Node 5 - Data Only
Could not find any cluster setting that is restricting the shards for index creation.
Is the issue with cluster strategy or am I missing any settings here?.
Please help me to find the the issue.
Earlier Elasticsearch had default number of primary shards to 5, which is changed from Elasticsearch 7.X, which you are using, hence you are seeing just 1 primary shard.
Elasticsearch link for this change and more info on this SO answer.
Apart from API which is applicable on a particular index, which #Kamal already mentioned, you can specify this setting in your elasticsearch.yml, which would be effective on every index created until you override using the API call.
Config to add in your elasticsearch.yml
index.number_of_shards: {your desired number of shards}
Note: This is for primary shards that can't be changed dynamically, so be cautious of setting this, Unlike the number of replicas which can be changed dynamically.
That is correct. Post version 7, Elasticsearch by default creates index with shard size 1 as mentioned here
You can always specify the index shard using the below settings, while creating the index.
PUT <your_index_name>
{
"settings" : {
"index" : {
"number_of_shards" : 5
}
}
}
Hope this helps!
I have script for uploading the data to ElasticSearch and it works fine with ES clusters containing 3 ES instances. But running the script against a 2-instance cluster throws that cluster into yellow status. Deleting the index restores them to green.
Found this: "A yellow cluster status means that the primary shards for all indices are allocated to nodes in a cluster, but the replica shards for at least one index are not."
How could I fix that? Should I improve my script somehow with a cluster size switch?
You certainly have in your index settings that you need 2 replicas. And as you cant have a replica and a primary shard on the same node, your cluster cant allocate all your shards in a 2 node cluster.
Could you try to decrease your number of replica to 1 ?
see here for the doc:
PUT /<your_index>/_settings
{
"index" : {
"number_of_replicas" : 1
}
}
Keep us posted !
i have a cluster with single node in it..
i had created an index A with default shard size(i.e. in elasticsearch.yml file the value of index.number_of_shards: 1). When i listed all my shards, i could see single shard for index A. After this i changed the value of index.number_of_shards: 4 in elasticsearch.yml and then created another index B. again when i listed all my shards in the cluster i could only see single shard created for index B instead of 4 shards.
Does Elasticsearch check for the total number of nodes present in a cluster before creating the index and assigning the shards(in my case i had not specified any no of shards while creating the index B, so i was expecting total of 4 shards to be created for my index). can you help me with this?
The index configuration on elasticsearch.yml is being deprecated in favor of passing this configuration on the index settings/mapping instead.
So what you have to do is to remove index configurations from the elasticsearch.yml file and pass them using index settings or template. From elastic docs:
curl -XPUT 'http://localhost:9200/twitter/' -d '{
"settings" : {
"index" : {
"number_of_shards" : 3,
"number_of_replicas" : 2
}
}
Using this approach you can create different configurations for each index.
Note: elasticsearch.yml is a "global" / static configuration file of elasticsearch that is read during startup and if you change it it will not affect current running instances.
Does Elasticsearch check for the total number of nodes present in a cluster before creating the index and assigning the shards
No. It will happily create many shards for the same index on the same node. Most likely you didn't restart ES after changing elasticsearch.yml. It's not read live, so any changes to it require a restart to take effect.