I have created JDBC Test Plan and i am using mysql database, with 'max number of connections:2' in 'connection pool configuration' but when i change it to 10 average response time increases.
My Question is 'max number of connections:2'in 'connection pool configuration' is referring to number of users?
I do not have ide about database performance testing, how database performance testing plan should be? i am assuming that increase number of users in thread group and report the response time. Can some one guid me with the sample database test plan as i do not know what component should be modified when testing for the performance.
Number of connections != number of users.
This "Max Number of Connections" is applicable for JDBC Connection Pool. JDBC Connections are very "expensive" to create hence normal practice is creating a pool so the connections can be established and used/reused by threads(users).
Ideally you should configure JDBC parameters to match your application settings, if you figure out that database responds slower due to the lack of available connections - you can test new configuration and suggest DBAs to change settings.
Number of users you should be setting on Thread Group level.
See The Real Secret to Building a Database Test Plan With JMeter article to learn more about concepts of database load testing using JMeter
Related
Is there a practical use of JMeter's JDBC Pool configuration
I tried to use Max Number of Connections 10 and it caused issues with Oracle max connection reached.
It seems from documentation below that its usage is discourage, so I still wonder if there's scenarios that it could be useful.
Max Number of Connections Maximum number of connections allowed in the
pool. In most cases, set this to zero (0). This means that each thread
will get its own pool with a single connection in it, i.e. the
connections are not shared between threads. If you really want to use
shared pooling (why?), then set the max count to the same as the
number of threads to ensure threads don't wait on each other.
Note In code I see it uses for connection pool org.apache.commons.dbcp2.BasicDataSource.
The practical use is that you should start with JDBC Connection Configuration which will be a replica of your production JDBC pool configuration in order to have realistic load pattern(s).
If you detect a database performance problem you could play with pool settings (connections number, transaction isolation, etc) to determine the most performing configuration, once you have the evidence that these or that pool settings provide better performance you can report it to developers or devops and amend your application database connectivity settings according to your findings. Check out Using JDBC Sampler in JMeter for JMeter connection pool settings explained.
From Oracle perspective I believe Connection Pooling and Caching and High-Performance Oracle JDBC Programming will help a lot.
I am not able to understand the values in WAS PMI for ConnectionPoolModule.
In one application I am monitoring, I am getting perf metrics for "Allocate Count", and in other I am getting perf metrics for "Create Count"
In the case of the AllocateCount - i can see that this value keeps increasing over time, and not sure what the effects of this state is.
What are the differences between these count types?
What should I be looking for to review connection pools?
Why are these metrics not showing up at the same time?
Should I be bothered about the increase in AllocateCount, or should I match it with other metrics to review the application state?
Thanks!
With these metrics, an allocate is an application request for a connection, e.g. a DataSource.getConnection(). The WebSphere pool manager either satisfies the request with an already-pooled connection, or creates a new one, and in the latter case the create count gets incremented. So if your allocate and create counts were the same, you'd be doing no pooling, probably a bad thing!
But that's not necessarily the best thing to monitor. Things like the average wait time could be the best starting point.
Let me refer you to some other links to help you monitor WebSphere JDBC connection pool data:
JDBC chapter in WebSphere Application Server performance cookbook
WebSphere Application Server Performance Tuning Toolkit with video
Older but still relevant, some slides specifically detailing some connection pool monitoring techniques.
We're in the process of rewriting a web application in Java, coming from PHP. I think, but I'm not really sure, that we might run into problems in regard to connection pooling. The application in itself is multitenant, and is a combination of "Separate database" and "Separate schema".
For every Postgres database server instance, there can be more than 1 database (named schemax_XXX) holding more than 1 schema (where the schema is a tenant). On signup, one of two things can happen:
A new tenant schema is created in the highest numbered schema_XXX database.
The signup process sees that a database has been fully allocated and creates a new schemas_XXX+1 database. In this new database, the tenant schema is created.
All tenants are known via a central registry (also a Postgres database). When a session is established the registry will resolve the host, database and schema of the tenant and a database session is established for that HTTP request.
Now, the problem I think I'm seeing here is twofold:
A JDBC connection pool is defined when the application starts. With that I mean that all databases (host+database) are known at startup. This conflicts with the signup process.
When I'm writing this we have ~20 database servers with ~1000 databases (for a total sum of ~100k (tenant) schemas. Given those numbers, I would need 20*1000 data sources for every instance of the application. I'm assuming that all pools are also, at one time or another, also started. I'm not sure how much resources a pool allocates, but it must be a non trivial amount for 20k pools.
So, is it feasable to even assume that a connection pool can be used for this?
For the first problem, I guess that a pool with support for JMX can be used, and that we create a new datasource when and if a new schemas_XXX database is created. The larger issue is that of the huge amount of pools. For this, I guess, some sort of pool manager should be used that can terminate a pool that have no open connections (and on demand also start a pool). I have not found anything that supports this.
What options do I have? Or should I just bite the bullet and fall back to an out of process connection pool such as PgBouncer and establish a plain JDBC connection per request, similar to how we're handling it now with PHP?
A few things:
A Connection pool need not be instantiated only at application start-up. You can create or destroy them whenever you want;
You obviously don't want to eagerly create one Connection pool per database or schema to be open at all times. You'd need to keep at least 20000 or 100000 Connections open if you did, a nonstarter even before you get to the non-Connection resources used by the DataSource;
If, as is likely, requests for Connections for a particular tenant tend to cluster, you might consider lazily, dynamically instantiating pools, and destroying them after some timeout if they've not handled a request for a while.
Good luck!
I am using Pentaho-BI server installation in my web application as a third party installation.I am using its saiku analytics and reporting files by embedding their specific links in iframe of my application. Problem is I am not getting how it creates database connections, in terms of numbers?? Because many times it throws error regarding 'No connection is available in pool'. I know there are properties like max available connection, max idle connections , wait and sql validation. But How to release connections?? And if Pentaho handles it in its own way then how?? Because increasing number of max connections available will create load on database server, when many users are using my BI server.
One solution I found is just to restart my BI server, but It's not a valid solution for production environment. Other solution I think is scheduler, but I have no clues about it and not getting proper info on net.
The defaults for max connections are incredibly low. This is standard tomcat connection pooling stuff, I would definitely try increasing the default, see if that helps. you can monitor concurrent connections on the db side - just because you have 100 connections to the db it doesn't necessarily mean they'll be all used at once.
Also; Are you using mysql? You should try the c3po pooling driver it handles timeouts and things better than the standard driver so you shouldnt ever get dead connections sitting in the pool.
What database connection pool could be used to load-balance connections from a Tomcat web container to one of several Oracle database servers without using RAC clustering?
I'm assuming these are read-only databases or you're not concerned connections will get different data. If you want the data to be the same, you can use streams replication which is doesn't require RAC.
The connection load balancing and failover happens in the listener. There's a lot of flexibility in how this works and this should get you started:
http://download.oracle.com/docs/cd/E11882_01/network.112/e10836/advcfg.htm#sthref858
The first part shows a simple client based load balance which is essentially picking a connection at random. Farther down it shows how to load balance based on actual server load.
Look into DRCP if using 11g