I have set up Flink UI for application running in Intellij IDEA. I would like to get some streaming metrics like - scheduling delay and processing time. However, I can not find the anywhere in UI. Should there be some specific setup for that or should I explicitly submit app jar?
Currently, Flink UI for the job looks like this:
All of the task metrics are exposed in the web UI, as Dominik mentioned, but for other metric scopes (e.g., job metrics) only some selected metrics are displayed. You can access all of the metrics via the REST API or by connecting a metrics reporter to send the metrics to an external metrics system.
I don't think any attempt has been to made to measure scheduling delay, but in the job metrics you will find things like restarting time and uptime.
In the UI you should have a tab Task Metrics when You select the currently running job. This tab allows You to choose a task and see all the metrics that are available. Although, I am not sure if the scheduling delay is one of currently available metrics.
Probably the better idea is to refer to expose the metrics for some collector of Your choice, You can find more info in the documentation: https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html.
Related
Requirements is to find out root cause analysis using Prometheus and Grafana and also need to know which method is taking more time, CPU and Memory Utilisation also?
Can anyone pls help me
Integrate Prometheus client library into your app
Configure Grafana dashboard to visualize the metrics you want to monitor from your app
Additionally you can integrate JMeter with Prometheus Listener for JMeter plugin (can be installed using JMeter Plugins Manager)
Come up with a test plan which simulates a real life usage of your application
Run a stress test (start with 1 user and gradually increase the load until response time starts exceeding acceptable threshold or errors start occurring)
Inspect Grafana dashboard to see what is the reason of the application being slow, i.e. trace down the "slow" request to the underlying function. Additionally you can use a profiler tool which can provide a better picture
I have a spring boot app that listens to a queue so it is non restful. Is there any easy way to get a time breakdown for the amount of time spent in each service?
The options are in:
Use a Profiler tool when running your application, this way you will be able to trace down the time even to a single function call
If you have an APM tool in place you can collect the same information plus metrics from the operating system, database, message broker, etc.
Just in case you need to generate messages and "feed" them to your application it could be done using i.e. Apache JMeter tool, see Building a JMS Testing Plan article for more information if needed
We are trying to revamp our batch job scheduling and monitoring process over the entire enterprise. Currently all our batch jobs are scheduled using Unix crontab and are monitored using log files generated by shell scripts.
This process has lot of disadvantages and as the number of applications grow this gets really complicated.
Two copies of applications need to be deployed one to App-Server and one as standalone(since business logic is shared between both). This is complicating our build process too.
There is no easy of use web-ui for us to see the status of jobs and manually run failed jobs remotely without getting onto the unix box.
There is no fail over or load balanced batch processing.
So I was thinking of using Quartz (with our existing Spring apps) in our applications and deploy them to App-Servers and no longer rely on the unix crontab.
Is there a way I can write a centralized web application from where I can schedule and monitor jobs running on different quartz schedulers on different app servers?
P.S: I know quartzdesk.com is one solution, but I don't want to enable RMI on my JVM.
You could use SpringBoot scheduler as an Orchestrator and call REST APIs for the remote (or local, if you are small) execution. This way, as your app grows you could easily leverage a load balancer.
If you have the possibility of using cloud services (like Amazon, Azure or Google Cloud), this could be done easily using their own load balancers. They also support docker and could take care of any peaks of utilization.
I have this pipeline: Kafka->Logstash->ElasticSearch->Kibana
I have found a producer performance tool that can be invoked with the script “ bin/kafka-producer-perf-test.sh”.
I am wondering if anyone had any suggestions for testing performance end-to-end testing?
Thanks.
Your pipeline Kafka->Logstash->ElasticSearch->Kibana involves 4 components. Kafka, Logstash, ElasticSearch, Kibana all of which serve a different purpose. Each component has different performance numbers and characteristics.
bin/kafka-producer-perf-test.sh You mentioned is kafka performance test tool which will only measure performance of Kafka and not others. If you have configured pipeline to read, process and display data generated using above test tool then you can get overall pipeline performance. This way you won't be able to find out limiting component in pipeline.
I suggest you to configure a replicate a similar data which is required by your pipeline. Kafka custom producer - PepperBox is good tool for that. Deploy monitoring on all components using influxDB/graphite (or any timeseries DB of your choice) and measure the end-to-end throughput as well as component wise throughput.
Sample benchmark is YSB. This will help you to get started.
I am trying to make Grafana display all my metrics (CPU, Memory, etc).
I have already configured Grafana on my server and have configured influxdb and of course I have configured Jmeter listener (Backend Listener) but still I cannot display all grpahas, any idea what should I do in order to make it work ?
It seems like that system metrics (CPU/Memory, etc.) are not in the scope of the JMeter Backend Listener implementation. Actually capturing those KPIs is a part of PerfMon plugin, which currently doesn't seem to support dumping the metrics to InfluxDB/Graphite (at least it doesn't seem to work for me). It might be a good idea to raise such a request at https://groups.google.com/forum/#!forum/jmeter-plugins. Until this gets done, I guess you also have the option of using some alternative metric-collection tools to feed data in InfluxDB/Graphite. Those would depend on the server OS you want to monitor (e.g. Graphite-PowerShell-Functions for Windows or collectd for everything else)
Are you sure that JMeter posts the data to InfluxDB? Did you see the default measurements created in influxDB?
I am able to send the data using backend listener to influxdb. I have given the steps in this site.
http://www.testautomationguru.com/jmeter-real-time-results-influxdb-grafana/