I'm trying to forward kubernetes-event logs to elasticsearch using fluentd.I currently use fluent/fluentd-kubernetes-daemonset:v1.10.1-debian-elasticsearch7-1.0as container image to forward my application logs to elasticsearch cluster.I've searched enough & my problem is that this image doesn't have enough documentation as to accomplishing this task(i.e; forward kubernetes event related logs).
I've found this plugin from splunk which has desired output but this has overhead like :
add above plugin's gem to bundler.
install essential tools like make etc.
install the plugin .
Sure I can do above steps using init-container, but above operations are adding ~200MB to disk space .I'd like to know if it can be accomplished with smaller footprint or other way.
Any help is appreciated.
Thanks.
You can try this: https://github.com/opsgenie/kubernetes-event-exporter
It is able to export Kube events to Elasticsearch.
Related
I am currently working on the ELK setup for my Kubernetes clusters. I set up logging for all the pods and fortunately, it's working fine.
Now I want to push all terminated/crashed pod logs (which we get by describing but not as docker logs) as well to my Kibana instance.
I checked on my server for those logs, but they don't seem to be stored anywhere on my machine. (inside /var/log/)
maybe it's not enabled or I might not aware where to find them.
If these logs are available in a log file similar to the system log then I think it would be very easy to put them on Kibana.
It would be a great help if anyone can help me achieve this.
You need to use kube-state-metrics by which you can get all pod related metrics. You can configure to your kube-state-metrics to connect elastic search. It will create an index for a different kind of metrics. Then you can easily use that index to display your charts/graphs in Kibana UI.
https://github.com/kubernetes/kube-state-metrics
guys
I want to watch all kubernetes events and I find the source code here: https://github.com/kubernetes/client-go/blob/master/informers/events/v1beta1/event.go
However, I can not find any examples about how to use the functions.
Can anyone help me, thanks a lot!
I’d like to collect the event logs with kubectl or REST API[2] as JSON, then you
can send the logs to fluentd for centralized monitoring such as Elasticsearch.
Here is a good sample;[0], though it's OpenShift, but if oc cmd replace with kubectl cmd, it's same with Kubernetes. (Yeah, OpenShift is Enterprise Kubernetes).
[1] is how to implement the fluentd - Elasticsearch stack.
I hope this help you.
[0] [https://docs.openshift.com/container-platform/3.9/security/monitoring.html#security-monitoring-events]
[1] [https://docs.fluentd.org/v0.12/articles/recipe-json-to-elasticsearch]
[2] [https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.10/#list-all-namespaces-292]
One or several of these could help:
"watches" for (quote) "...efficient change notifications on resources" - see Kubernetes API Concepts as well as the API Reference for a particular version. Example: GET /api/v1/namespaces/test/pods?watch=1&resourceVersion=10245
Event Read Operations.
kubectl get allows you to specify the -w or --watch flag to start watching updates to a particular object.
I believe the events are for a particular resource or collection of resources, not for all resources.
I want to use ElasticSearch instead of MongoDB. How can I achieve this?
Is there a way to install everything from scratch and configure it? Configuration is the challenging part. I'm looking for tutorials explaining how to replace MongoDB with Elasticsearch.
There's not an easy way to substitute MongoDB with Elasticsearch on the stack.
However, you can easily install a Bitnami Elasticsearch stack (https://bitnami.com/stack/elasticsearch) on a different directory. For instance, if you have your MEAN stack on the default directory (/opt/bitnami/), you can install the Elasticsearch stack at /opt/elasticsearch/ and then edit the environment/control scripts of the original stack so you disable MongoDB and add the ability to control Elasticsearch.
If you want to have everything in the same VM, then I advise you to use our Elasticsearch installer: https://bitnami.com/stack/elasticsearch/installer
This way you would have your MEAN stack and, in addition, an Elasticsearch stack. Then you can disable MongoDB if you don't plan to use it at all.
sudo /opt/bitnami/ctlscript.sh stop mongodb
sudo mv /opt/bitnami/mongodb/scripts/ctl.sh
sudo /opt/bitnami/mongodb/scripts/ctl.sh.disabled
Credits -
jsalmeron - bitnami
I am trying to integrate Zabbix with Elastic Search through logstash and further generate dashboard on Kibana. Now there are many links which suggest it is possible to monitor elasticsearch through Zabbix but not the other way around.
http://logstash.net/docs/1.4.2/outputs/zabbix
Now I got one link which suggests zabbix servers can be monitored. I followed the same but not success
http://philippe.lewin.me/2014/10/06/send-zabbix-events-to-logstash/
I need some help to understand the possibilities and probable some workarounds.
OP, are you still having the same problem? I also am looking for a way to send my zabbix snmp event data to Elastic Search.
True, the plugins out there are helping the other way around though
I will try phillipe's way later, if all else fail, probably i will try to migrate specific table(s) with another tools directly like Logstash for example
I have been trying to write a aws sensu plugin which will get the instance id's of all the healthy instances which are under a load balancer and then get the stats for each of the instances like CPU Utilization Network In and Network Out etc and using graphite and graphane generate graphs.
I was searching the open source plugins in the sensu community, I could not find any. Is it possible write the script or plugin for this. Or anyone has done it before??
Kindly help me out
I don't believe a Sensu-specific plugin exists for this. However, since Sensu can run any Nagios plugin, you could use one of those: This one looks like it would get basic information on how many hosts are healthy. You could also write your own plugin using your language of choice (check out the available SDKs) to get more detailed metrics for each of the instances.
I wrote a plugin to do the same. It use to work fine then. I have testing on newer version of API. Let me know if you face any problem. I will help to fix the same.