I am not able to understand the values in WAS PMI for ConnectionPoolModule.
In one application I am monitoring, I am getting perf metrics for "Allocate Count", and in other I am getting perf metrics for "Create Count"
In the case of the AllocateCount - i can see that this value keeps increasing over time, and not sure what the effects of this state is.
What are the differences between these count types?
What should I be looking for to review connection pools?
Why are these metrics not showing up at the same time?
Should I be bothered about the increase in AllocateCount, or should I match it with other metrics to review the application state?
Thanks!
With these metrics, an allocate is an application request for a connection, e.g. a DataSource.getConnection(). The WebSphere pool manager either satisfies the request with an already-pooled connection, or creates a new one, and in the latter case the create count gets incremented. So if your allocate and create counts were the same, you'd be doing no pooling, probably a bad thing!
But that's not necessarily the best thing to monitor. Things like the average wait time could be the best starting point.
Let me refer you to some other links to help you monitor WebSphere JDBC connection pool data:
JDBC chapter in WebSphere Application Server performance cookbook
WebSphere Application Server Performance Tuning Toolkit with video
Older but still relevant, some slides specifically detailing some connection pool monitoring techniques.
Related
I am currently using a IBM MQ Advanced for Developers server for testing our client and was able to achieve around 1000 messages per second using the sample consumer written in jms, which seems to be pretty slow. Is this a limit for dev server, and if yes that what throughput can be achieved using a licensed production IBM MQ server.
There is no artificial limit associated with IBM MQ Advanced for Developers. It is the same as the licensed production version of IBM MQ.
You don't say what type of machine you were using, what persistence your messages were, what size they were, or any other qualifying criteria.
You say client, but I don't know whether you mean "network attached application" or "driving application". Clearly if your program is running "client-attached" (MQ parlance for network attached), then the network performance will also come into this.
On my Windows laptop, I get 4500 non-persistent msgs/sec, or 2000 persistent msgs/sec using a simple C-language locally bound program. Over client connection (just using localhost, not actually going out over a real network connection) I get 2700 non-persistent msgs/sec, or 1500 persistent msgs/sec.
You should read the MQ Performance Reports for details of the expected rates you can get.
As an ex MQ performance person I would say - it depends.
At one level you can ask - what can one application in isolation process.
For persistent messages this will come down to the rate at which you can write to the log files.
If you have 10 applications in parallel each putting and getting from their own queue, then you will not get 10 times the throughput - you might get 8 or 9 times the throughput.
If they are all processing the same queue, then the throughput may drop a bit more as the queue usage is serialised.
If only one application is writing to the log, the application may see 1 millisecond response time. If you have 10 applications running concurrently, they may see a 3 milliseconds response time - so individual throughput goes down, but with more threads, the overall throughput goes up.
If you have requests coming in over the network, you need to add network time, but you can run more clients and so get improved throughput.
If your application has a delay built in - it may only process a low message rate. You can have lots (1000s) of these and get a high >overall< throughput.
If your application is putting and getting as fast as possible, you may find that you can run 10-100 instances before the throughput plateaus.
Let's say you want to run you box so it is using 75% of the CPU, and the logging is 50% busy.
If you have just MQ on the box, then this can run more messages than if you had DB2 on the box (with DB2 using 50% of the CPU)
If you have an application (DB2) hammering the disk, then the MQ throughput will go down.
If you have lots of applications putting to a server queue - and one server program, you will find the throughput is limited by the rate at which the server can process work. If it is doing DB2 work, it will be slower than no DB2 work. If you find the server queue depth is over 5 then you need more server instances.
As Morag said, see the performance reports, but they are not the clearest reports to understand.
The web application is running on Springboot and deployed on WebLogic.
We have assigned 400 as max threads and JDBC to be 100 connections.
When we perform load testing on the web application, the performance is optimal when the load is low (the response time is less than 200ms for most of the http request that we called).
When we increase the load, we can see that the thread count increases and jdbc count also increases gradually but no where near to max. However, the response time is getting much longer and it could take more than 5 seconds to response.
CPU usage, thread count, memory, JDBC connection seems to be normal during these period.
Another observation is that during testing and we saw that the performance is degrading, we used another machine to make a http call to the server that is only retrieving text without any DB or logic, and even this simple http call will take 10s to respond. (And the server resources is still not MAX!)
So, we are wondering what keep them waiting ?
Any other possible bottleneck?
If the server doesn't lack resources like CPU/RAM/etc. only a profiler can tell you where your application spends the most time which might be in:
Waiting in a queue for next thread/db connection from the pool to be available
Slow database query
Inefficient functions/algorithms which a subject to optimization
WebLogic configuration not suitable for high loads
JVM configuration not suitable for high loads (i.e. system is doing garbage collection to often/too long)
So I would recommend re-running your test with profiler tool telemetry enabled and at the same time monitoring essential JVM metrics using i.e. JMXMon Sample Collector which can be used for monitoring your application-specific metrics as well. It's a plugin which can be installed using JMeter Plugins Manager
For a detailed approach on how ago about identifying poor thread performance I suggest you take look at the TSA Method by Brendan Gregg.
Is there a practical use of JMeter's JDBC Pool configuration
I tried to use Max Number of Connections 10 and it caused issues with Oracle max connection reached.
It seems from documentation below that its usage is discourage, so I still wonder if there's scenarios that it could be useful.
Max Number of Connections Maximum number of connections allowed in the
pool. In most cases, set this to zero (0). This means that each thread
will get its own pool with a single connection in it, i.e. the
connections are not shared between threads. If you really want to use
shared pooling (why?), then set the max count to the same as the
number of threads to ensure threads don't wait on each other.
Note In code I see it uses for connection pool org.apache.commons.dbcp2.BasicDataSource.
The practical use is that you should start with JDBC Connection Configuration which will be a replica of your production JDBC pool configuration in order to have realistic load pattern(s).
If you detect a database performance problem you could play with pool settings (connections number, transaction isolation, etc) to determine the most performing configuration, once you have the evidence that these or that pool settings provide better performance you can report it to developers or devops and amend your application database connectivity settings according to your findings. Check out Using JDBC Sampler in JMeter for JMeter connection pool settings explained.
From Oracle perspective I believe Connection Pooling and Caching and High-Performance Oracle JDBC Programming will help a lot.
I have created JDBC Test Plan and i am using mysql database, with 'max number of connections:2' in 'connection pool configuration' but when i change it to 10 average response time increases.
My Question is 'max number of connections:2'in 'connection pool configuration' is referring to number of users?
I do not have ide about database performance testing, how database performance testing plan should be? i am assuming that increase number of users in thread group and report the response time. Can some one guid me with the sample database test plan as i do not know what component should be modified when testing for the performance.
Number of connections != number of users.
This "Max Number of Connections" is applicable for JDBC Connection Pool. JDBC Connections are very "expensive" to create hence normal practice is creating a pool so the connections can be established and used/reused by threads(users).
Ideally you should configure JDBC parameters to match your application settings, if you figure out that database responds slower due to the lack of available connections - you can test new configuration and suggest DBAs to change settings.
Number of users you should be setting on Thread Group level.
See The Real Secret to Building a Database Test Plan With JMeter article to learn more about concepts of database load testing using JMeter
I am using Pentaho-BI server installation in my web application as a third party installation.I am using its saiku analytics and reporting files by embedding their specific links in iframe of my application. Problem is I am not getting how it creates database connections, in terms of numbers?? Because many times it throws error regarding 'No connection is available in pool'. I know there are properties like max available connection, max idle connections , wait and sql validation. But How to release connections?? And if Pentaho handles it in its own way then how?? Because increasing number of max connections available will create load on database server, when many users are using my BI server.
One solution I found is just to restart my BI server, but It's not a valid solution for production environment. Other solution I think is scheduler, but I have no clues about it and not getting proper info on net.
The defaults for max connections are incredibly low. This is standard tomcat connection pooling stuff, I would definitely try increasing the default, see if that helps. you can monitor concurrent connections on the db side - just because you have 100 connections to the db it doesn't necessarily mean they'll be all used at once.
Also; Are you using mysql? You should try the c3po pooling driver it handles timeouts and things better than the standard driver so you shouldnt ever get dead connections sitting in the pool.