Unable to pause Elasticsearch instance in Elastic Cloud.
In Documentation, It is mention for pause but unable to get any option on Elastic Cloud Console.
Documentation Link :-
https://www.elastic.co/guide/en/cloud-enterprise/1.0/ece-maintenance-mode-clusters.html
The documentation you refer to is not the Elastic Cloud documentation but the Elastic Cloud Enterprise (ECE) documentation, which is the underlying product that Elastic uses to operate Elastic Cloud.
It is not possible to directly pause instances from the Elastic Cloud Console UI. You can only stop routing requests to a given node by clicking on the three dots at the top right of your node.
Do you know of any gotcha's or requirements that would not allow using a single ES/kibana as a target for fluentd in multiple k8 clusters?
We are engineering rolling out a new kubernetes model. I have requirements to run multiple kubernetes clusters, lets say 4-6. Even though the workload is split in multiple k8 clusters, I do not have a requirement to split the logging and believe it would be easier to find the logs for pods in all clusters in a centralized location. Also less maintenance for kibana/elasticsearch.
Using EFK for Kubernetes, can I point Fluentd from multiple k8 clusters at a single ElasticSearch/Kibana? I don't think I'm the first one with this thought however I haven't been able to find any discussion of doing this. Found lots of discussions of setting up efk but all that I have found only discuss a single k8 to its own elasticsearch/kibana.
Has anyone else gone down the path of using a single es/kibana to service logs from multiple kubernetes clusters? We'll plunge ahead with testing it out but seeing if anyone else has already gone down this road.
I dont think you should create an elastic instance for each kubernetes cluster, you can run a main elastic instance and index it all logs.
But even if you don`t have an elastic instance for each kubernetes client, i think you sohuld have a drp, so lets says instead moving your logs of all pods to elastic directly, maybe move it to kafka, and then split it to two elastic clusters.
Also it is very depend on the use case, if every kubernetes cluster is on different regions, and you need the pod`s logs in low latency (<1s), so maybe one elastic instance is not the right answer.
Based on [1] we can read:
Fluentd collects logs from pods running on cluster nodes, then routes
them to a centralized Elasticsearch.
Then Elasticsearch ingests these logs from Fluentd and stores them in a central location. It is also used to efficiently search text files.
Kibana is the UI; the user can visualize the collected logs and metrics and create custom dashboards based on queries.
There are several ways in which they can solve your dilemma:
a) Create a centralized dashboard and use each cluster’s Elasticsearch as backend. So you can see all your clusters logs in one place.
b) Create an Elasticsearch cluster and add each Elasticsearch into it. This is NOT the best option since you will duplicate your data several times, you will need to handle each index shards and you will need to fight with the split brain dilemma but it’s great for data resiliency.
c) Use another solution like an APM (New Relic, Instana, etc) to fully centralize your logs in one place.
[1] https://techbeacon.com/enterprise-it/9-top-open-source-tools-monitoring-kubernetes
I am using Elastic Cloud hosted service for elasticsearch and kibana instance. I have already asked help on https://www.elastic.co/blog/monitoring-the-search-queries article from Elastic Cloud team but it is relevant to on-premise cluster
With 6.2 Elastic version how to have single kibana instance to monitor multiple elastic clusters.
Cluster-1: Production Application cluster
Cluster-2: Log cluster
Need to monitor both the cluster's using single kibana instance with basic license. Is it possible ?
I recently completed a deployment of Elastic Search Cluster on GCP (Google Cloud Platform) using the link mentioned below.
The elastic search works perfectly fine and all operations associated with elastic search are functional, I have two questions associated with this deployment:
How many simultaneous search this elastic search can perform? (Considering the fact machine has 1cpu core and 3.75 GB memory)
And can we add more nodes with more compute power in later phases? Is there any way I can add more nodes to the cluster as my application scales?
Google Cloud Bitnami ElasticSearch