The connection to InfluDb is correct, I can connect via CLI and I can send Jmeter data to InfluxDb correctly, I can even see it in influxDb, but Grafana Cloud does not plot graphs of InfluDb data.
I've researched everything and I can't find a way to debug to understand what's going wrong.
InfluxDB working correctly:
Grafana Data Source Settings:
Related
I am trying to send data from Elasticsearch to Influxdb. Is there any other way to do this except writing plugins and configuration files. I am new to both these databases, and trying to understand the overall picture.
Also, am I right in understanding that Kapacitor processes influx data and then sends it to Kafka for streaming? Or should I stream data using Kapacitor only?
I am trying to learn all these new technologies in as short time frame, and all the new terminologies have got me confused. Thanks for your time and help.
ElasticSearch is a search engine, not a database. InfluxDB is a time series data. I did't understand why you need to transfer data from search results to a time series database.
Kapacitor can process data in 2 different ways. Either in batch mode or in streaming mode. Assume some application streaming sensor data(or some time series data) to InfluxDb. You can set Kapacitor to process that data as soon it is available in InfluxDB by setting kapacitor in streaming mode. Or in case you need to process data from InfluxDB every 2 hour, you can configure that job as batch job. Once you process the data, Kapacitor can persist the data in InfluxDB. Or in case you need to stream the data to Kafka, Kpacitor has a Kafka plugin. Please note Kapacitor has more plugins than I mentioned in my answer.
I have a ton of services: Node(s), MySQL(s), Redis(s), Elastic(s)...
I want to monitor how they connect to each other: Connection rate, Number alive connection... (Node1 create 30 connection to Node2/MySQL/Redis per second...) like Haproxy stat image attached below.
Currently i have two options:
Haproxy (proxy): I want to use single service Haproxy to archive this but it's seem very hard to use ALC detect what connection need forward to what service.
ELK (log center): I need to create log files on each service (Node, MySQL, Redis...) and then show them on the log center. I see that a ton of works to do that without built-in feature like Haproxy stat page.
How to do this? Is log center good in this case?
The problem
I think your problem is not collecting and pipelining the statistics to Elasticsearch, but instead the ton of work extracting metrics from your services because most of them do not have metric files/logs.
You'd then need to export them with some custom script, log them and capture it with filebeat, stream to a logstash for text processing and metric extraction so they are indexed in a way you can do some sort of analytics, and then send it to elasticsearch.
My take on the answer
At least for the 3 services you've referenced, there are Prometheus exporters readily available and you can find them here. The exporters are simple processes that will query your services native statistics APIs and expose a prometheus metric API for Prometheus to Scrape (poll).
After you have Prometheus scraping the metrics, you can display them in dashboards via Grafana (which is the de facto visualization layer for Prometheus) or bulk export your metrics to wherever you want (Elasticsearch, etc..) for visualization and exploration.
Conclusion
The benefits of this approach:
Prometheus can auto-discover new nodes you add to your networks
Readily available exporters from haproxy, redis and mysql for
Prometheus
No code needed, each exporter requires minimal
configuration specific to each monitored technology, it can easily
be containerized and deployed if your environment is container
oriented, otherwise you just need to run each exporter in the
correct machines
Prometheus is very, very easy to deploy
Use ELK - elasticsearch logstash and kibana stack with filebeat. Filebeat -will share the log file content with logstash
Logstash-will scan, filter and share the needed content to elastic search
Elasticsearch- will work as a db, store the content from logstash in json format as documents.
Kibana- with kibana you can search the required info. Also you can plot graphs and other visuals with the relevant data.
I read the documentation , though I would like to gain more knowledge and usage of Influx DB how much worth before start using.
Can some one explain in detail on the below questions.
1.What is the use of Influx Db backed listener in jmeter.
2.What is the difference Influx Db backed listener Vs Graphs Generator?
3.What are steps involved installation and configuration of Influx Database on Windows?
4.Along with the Influx Db do we need to install and configure anything else?
5.How can we send the whole dashboard to the team generated from the Influx db.
6.I appreciate If you provide the detailed steps involved from #1 to # 5.
Thanks,
Raj
InfluxDB is a time-series db (a light-weight db used to store time dependant data such as a Performance Test).
Using InfluxDB along with Grafana you can monitor certain test metrics live during a JMeter test and can also configure other system metrics to be collected and monitored (cpu/network/memory).
To store data into InfluxDB, you need to configure the Graphite configuration within JMeter (see Real-Time results). Then you can add a Backend listener to throw this into the DB.
For InfluxDB installation on Windows read this answer.
As for the Dashboard, I guess you need to use Grafana to see the expected test live metrics in a graphical format.
I am trying to build the InfluxDB + Grafana + JMeter dashboard.
I am using InfluxDB 0.10, Grafana 2.6 and JMeter 2.13.
Backend Listener
My backend Listener looks good. I have listed a few names in the Sampler's List.
When I try to create graphs for the Sampler's I am unable to.
I also tried the Grafana Dashboard Generator, but still unable to create graphs for the individual Sampler's.
Any suggestions.
First check if Jmeter is shipping data in InfluxDB by running DB commands.
I’m trying to install grafana to work with OpenTSDB datasource. I’d like to know, what should I do to install it without elasticsearch?
I'm using grafana with Influxdb and I'm not using elasticsearch.
Grafana 2 is out in beta and I've been using that in production for a while. Grafana 2 now has its own data store, which either uses MySQL or SQLite. But you can always use Elasticsearch as well. You can read more about it here
Update: Stable version of Grafana 2 is now out, and it just works.
Grafana is a frontend, you will need some kind of database to store values and configuration in. I just grabbed the .tar.gz file from grafana's downloads page, created a config.js and pointed it at my influxdb server. No elasticsearch here, either.
You might want to take a look at gofana which will allow you to run Grafana without Elasticsearch. It's a self-contained binary that allows you to store dashboards on the filesystem and not in Elasticsearch or InfluxDB. It also supports HTTPS and basic authentication.
Note: I'm the author of gofana.