My GKE cluster is currently in default settings and is logging to stack driver. However, I would like to be able to log to elastic stack that I am deploying at elastic.co.
https://cloud.google.com/solutions/customizing-stackdriver-logs-fluentd
I see that I am able to customize filtering and parsing of default fluentd daemonset but how do I install elasticsearch output plugin so that I can stream logs to my elasticsearch endpoint instead of Stackdriver?
The tutorial you linked to answers your question. You need to create a GKE cluster without the built-in fluentd (by passing the --no-enable-cloud-logging flag when creating the cluster) and then install a custom daemon set with the fluentd configuration you want to use.
Related
I am trying to add ELK to my project which is running on kubernetes. I want to pass by filebeat -> logstach then elastic search. I prepared my filebeat.yml file and in my company the filebeat is configured as an agent in the cluster which i don't realy know what it means? I want to know how to configure the filebeat in this case ? just adding the file in the project and it will be taken into considiration once the pod started or how does it work ?
You can configure the Filebeat in some ways.
1 - You can configure it using the DeamonSet, meaning each node of your Kubernetes architecture will have one POD of Filebeat. Usually, in this architecture, you'll need to use only one filebeat.yaml configuration file and set the inputs, filters, outputs (output to Logstash, Elasticsearch, etc.), etc. In this case, your filebeat will need root access inside your cluster.
2 - Using Filebeat as a Sidecar with your application k8s resource. You can configure an emptyDir in the Deployment/StatefulSet, share it with the Filebeat Sidecar, and set the Filebeat to monitor this directory.
I have an EFK stack that outputs EKS cluster logs to both Elasticsearch and S3. I wonder if there's a way to add a switch to enable/disable outputs to S3 .. maybe using an ENV in the FluentD manifest file. Would appreciate help if anyone knows how to implement this feature.
P.S: can share files as needed
We have an EKS cluster running and we are looking for best practices to ship application logs from pods to Elastic.
In the EKS workshop there is an option to ship the logs to cloudwatch and then to Elastic.
Wondered if there is an option to ship the logs directly to Elastic, or to understand best practices.
Additional requirement:
We need the logs to determine from which namespace the logs is coming from and to deliver a dedicated index
You can deploy EFK stack in kubernetes cluster. Follow the reference --> https://github.com/acehko/kubernetes-examples/tree/master/efk/production
Fluentd would be deployed as DaemonSet so that one replica is run on each node collecting the logs from all pods and push them to elasticsearch
I'm running an elasticsearch cluster using StatefulSets on Google Container Engine (GKE) (my k8s configs are very similar to the ones here: https://github.com/pires/kubernetes-elasticsearch-cluster/tree/master/stateful)
I created the k8s cluster with --enable-stackdriver-kubernetes
Now I want to also install & use the Stackdriver elasticsearch plugin:
https://cloud.google.com/monitoring/agent/plugins/elasticsearch
Should I install the Stackdriver monitoring agent+plugin inside the ES pods? or on the nodes?
If you're using Stackdriver logging agent to generate/export logs for pods running elasticsearch on a Kubernetes cluster, you can have Stackdriver logging enabled for the cluster (this is enabled by default and can be enabled/disabled through the Console), the Stackdriver logging agent will be deployed on the cluster.
If the logging agent is running on the cluster, logs from each container will automatically be gathered, formatted and exported by the logging agent to Stackdriver Logging [1] for the deployed pods/containers, including elasticsearch.
Kubernetes does things differently to Compute Engine instances in terms of the monitoring agent. If Stackdriver Monitoring is enabled for the cluster, pods are deployed running the Kubernetes Engine version of the Stackdriver Agent and in the case of Kubernetes these come in the form of heapster pods as explained in more detail here [2].
[1] https://cloud.google.com/kubernetes-engine/docs/how-to/loggin
[2] https://cloud.google.com/monitoring/kubernetes-engine/customizing
I am trying to do the log monitoring of Kubernetes cluster using EFK. I got Kibana dashboard but it doesn't show any logs of Kubernetes cluster.
Here is the link which I followed in my task.By default my dashboard shows like
After that i changed the index-pattern in dashboard as
Then it showed as
My dought is, how Can i view the logs of each and every pod logs in kubernetes cluster?
Could anybody suggest me how to do the log monitoring of kubernetes cluster using EFK?
Note: in order for Fluentd to work, every Kubernetes node must be
labeled with beta.kubernetes.io/fluentd-ds-ready=true, as otherwise
the Fluentd DaemonSet will ignore them.
Have you made sure to address this?