I have a servlet which connects to Oracle DB using JDBC (ojdbc6.jar) and BoneCP. I now need to port my BoneCP-using code to something which will work in WebLogic out-of-the-box, without having BoneCP in the package.
What would be the recommended approach? What WebLogic feature I can use, specifically to get an equivalent of BoneCP's:
Performance
Ability to log failed SQL statements
Auto-resume from lost DB connection
Thanks in advance.
The best approach would be to create a standard Oracle JDBC connection pool pointing to your database. Tune it according to your necessities (number of connections, etc.). Next you would need to refactor out of your code any explicit reference to your former connection pool implementation. If you have been working with java.sql.* interfaces in your code, there should be few to no references at all.
Once all that is refactorized, you will have only a bit of code (or config file) telling your app to recover something implementing javax.sql.DataSource from a given JNDI name and getting Connections out of it. The rest should be the same - just do whatever you need and close your ResultSets, Statements and Connections as you must have been doing until now.
About your questions, you will find extensive information on how to monitor your connection pool, and its fail recovery policies, here (depending on your app server version, I paste here the one I have used):
http://docs.oracle.com/cd/E15051_01/wls/docs103/jdbc_admin/jdbc_datasources.html
About performance, I have no accurate data nor benchmarks comparing both implementations; for your tranquility, I would say you that I have never found a database performance problem in the connection pool implementation - this does not mean that it cannot exist, but it is the last place I would look for it ;)
Related
Is there any problem on using many open connections at the same time from different threads?
From what I've read it's thread safe by default, but, can this be hurting performance rather than improving it?
Having multiple connection is not a problem, the only thing to keep in mind is that SQLite does not support concurrency of multiple write transactions. From the SQlite site:
SQLite supports an unlimited number of simultaneous readers, but it will only allow one writer at any instant in time. For many situations, this is not a problem. Writer queue up. Each application does its database work quickly and moves on, and no lock lasts for more than a few dozen milliseconds. But there are some applications that require more concurrency, and those applications may need to seek a different solution.
SQLite is an "untypical" database management system: in practice it is a library that offers SQL as language to access a simple "database-in-a-file", and a few other functionalities of DBMSs. For instance, it has no real concurrency control (it uses the Operating Systems functions to lock the db file).
So, if you need concurrent insertions into a database, you should use something else, for instance PostgreSQL.
The documentation say:
A connection can only be used from within the thread that created it.
Moving connections between threads or creating queries from a
different thread is not supported.
In addition, the third party libraries used by the QSqlDrivers can
impose further restrictions on using the SQL Module in a multithreaded
program. Consult the manual of your database client for more
information.
It is mean you have to create connection to database which will be linking with parent thread. At docs of QSqlDatabase class you can see description:
The QSqlDatabase class represents a connection to a database.
The QSqlDatabase class provides an interface for accessing a database
through a connection. An instance of QSqlDatabase represents the
connection. The connection provides access to the database via one of
the supported database drivers, which are derived from QSqlDriver.
Create a connection (i.e., an instance of QSqlDatabase) by calling one
of the static addDatabase() functions, where you specify the driver or
type of driver to use (i.e., what kind of database will you access?)
and a connection name.
Using static addDatabase() function is way to create connection.
But as Renzo said SQLite does not support multiple write transactions at the same time. So you need some mechanisms(wrapper) for synchronizing threads like task queue using low-level mutex or something like that. More information you can see at docs.
We're in the process of rewriting a web application in Java, coming from PHP. I think, but I'm not really sure, that we might run into problems in regard to connection pooling. The application in itself is multitenant, and is a combination of "Separate database" and "Separate schema".
For every Postgres database server instance, there can be more than 1 database (named schemax_XXX) holding more than 1 schema (where the schema is a tenant). On signup, one of two things can happen:
A new tenant schema is created in the highest numbered schema_XXX database.
The signup process sees that a database has been fully allocated and creates a new schemas_XXX+1 database. In this new database, the tenant schema is created.
All tenants are known via a central registry (also a Postgres database). When a session is established the registry will resolve the host, database and schema of the tenant and a database session is established for that HTTP request.
Now, the problem I think I'm seeing here is twofold:
A JDBC connection pool is defined when the application starts. With that I mean that all databases (host+database) are known at startup. This conflicts with the signup process.
When I'm writing this we have ~20 database servers with ~1000 databases (for a total sum of ~100k (tenant) schemas. Given those numbers, I would need 20*1000 data sources for every instance of the application. I'm assuming that all pools are also, at one time or another, also started. I'm not sure how much resources a pool allocates, but it must be a non trivial amount for 20k pools.
So, is it feasable to even assume that a connection pool can be used for this?
For the first problem, I guess that a pool with support for JMX can be used, and that we create a new datasource when and if a new schemas_XXX database is created. The larger issue is that of the huge amount of pools. For this, I guess, some sort of pool manager should be used that can terminate a pool that have no open connections (and on demand also start a pool). I have not found anything that supports this.
What options do I have? Or should I just bite the bullet and fall back to an out of process connection pool such as PgBouncer and establish a plain JDBC connection per request, similar to how we're handling it now with PHP?
A few things:
A Connection pool need not be instantiated only at application start-up. You can create or destroy them whenever you want;
You obviously don't want to eagerly create one Connection pool per database or schema to be open at all times. You'd need to keep at least 20000 or 100000 Connections open if you did, a nonstarter even before you get to the non-Connection resources used by the DataSource;
If, as is likely, requests for Connections for a particular tenant tend to cluster, you might consider lazily, dynamically instantiating pools, and destroying them after some timeout if they've not handled a request for a while.
Good luck!
I am using Pentaho-BI server installation in my web application as a third party installation.I am using its saiku analytics and reporting files by embedding their specific links in iframe of my application. Problem is I am not getting how it creates database connections, in terms of numbers?? Because many times it throws error regarding 'No connection is available in pool'. I know there are properties like max available connection, max idle connections , wait and sql validation. But How to release connections?? And if Pentaho handles it in its own way then how?? Because increasing number of max connections available will create load on database server, when many users are using my BI server.
One solution I found is just to restart my BI server, but It's not a valid solution for production environment. Other solution I think is scheduler, but I have no clues about it and not getting proper info on net.
The defaults for max connections are incredibly low. This is standard tomcat connection pooling stuff, I would definitely try increasing the default, see if that helps. you can monitor concurrent connections on the db side - just because you have 100 connections to the db it doesn't necessarily mean they'll be all used at once.
Also; Are you using mysql? You should try the c3po pooling driver it handles timeouts and things better than the standard driver so you shouldnt ever get dead connections sitting in the pool.
My application have performance issues, so i started to investigate this from the root: "The connection with the database".
The best practices says: "Open a connection, use it and close is as soon as possible", but i dont know the overhead that this causes, so the question is:
1 -"Open, Use, Close connections as soon as possible is the best aproach using ODP.NET?"
2 - Is there a way and how to use connection pooling with ODP.NET?
I thinking about to create a List to store some connections strings and create a logic to choose the "best" connection every time i need. Is this the best way to do it?
Here is a slide deck containing Oracle's recommended best practices:
http://www.oracle.com/technetwork/topics/dotnet/ow2011-bp-performance-deploy-dotnet-518050.pdf
You automatically get a connection pool when you create an OracleConnection. For most middle tier applications you will want to take advantage of that. You will also want to tune your pool for a realistic workload by turning on Performance Counters in the registry.
Please see the ODP.NET online help for details on connection pooling. Pool settings are added to the connection string.
Another issue people run into a lot with OracleConnections is that the garbage collector does not realize how truly resource intensive they are and does not clean them up promptly. This is compounded by the fact that ODP.NET is not fully managed and so some resources are hidden from the garbage collector. Hence the best practice is to Close() AND Dispose() all Oracle ODP.NET objects (including OracleConnection) to force them to be cleaned up.
This particular issue will be mitigated in Oracle's fully managed provider (a beta will be out shortly)
(EDIT: ODP.NET, Managed Driver is now available.)
Christian Shay
Oracle
The ODP.NET is a data provider for ADO.NET.
The best practice for ADO.Net is Open, Get Data (to memory), close, use in memory data.
For example using a OracleDataReader to load data in a DataTable in memory and close connection.
[]'s
For a single transaction this is best but for multiple transaction where you commit at the end this might not be the best solution. You need to keep the connection open until the transaction either committed or rolled back. How do you manage that and also how do you check the connection still exist in that case?(ie network failure) There is ConnectionState.Broken property which does not work at this point.
Does anyone know the best (or any) way to flush a JDBC connection pool? I can't find anything obvious in the documentation. It appears connection pools aren't meant to ever be deleted.
My current thought is to delete all DataSources from the hash we store them in, which will trigger our code to make new ones. However, my first attempt throws a ConcurrentModificationException.
You shouldn't be writing a connection pool. Even if you want to manage the pool yourself (as opposed to letting a container do it), you should use a library for that (such as Commons DBCP).
If you want to delete everything from a hash, you should use hash.clear().
If you want to avoid ConcurrentModificationException, you need to add synchronization.
If you delete references to Connections (are you sure you meant DataSources?), be sure to close() them first.
Why do you want to delete, rather don't create it at first place.
It should be based on your appserver, may be some JNDI programming could do the trick.
You shouldn't be writing a connection pool. That's handled by the Java EE app server.