We are using the websphere as application server and spring-data to access an oracle database.
If we call the start page on our application, different data and a lot of labels from three different frames are loaded parallely from the database.
Now, if we monitor the database while open the start page, different database sessions are opened and closed. However, three sessions (get labels) are kept open, but inactive.
I think three sessions, because I have three frames and they all load a lot of labels parallely.
Is there a functionality in websphere, spring-data or oracle that recognizes when a sql statement is often executed and keeps the session open?
Or do you know why the sessions is kept open?
WebSphere Application Server data sources support pooling of connections and caching of prepared statements. This is enabled by default. As long as your application is using the WebSphere Application Server data source (obtained via resource injection #Resource or JNDI lookup) and uses java.sql.PreparedStatement for your SQL commands, PreparedStatements will be kept in the cache and reused as needed. It isn't clear from your question if you want this behavior or if you want to avoid it. If you are looking to disable caching of all prepared statements, you can configure the statementCacheSize of the data source to 0. Alternately, if there are only specific prepared statements that you wish to avoid caching, you can programmatically set the JDBC poolability hint on them:
preparedStatement.setPoolable(false);
Related
Is there any problem on using many open connections at the same time from different threads?
From what I've read it's thread safe by default, but, can this be hurting performance rather than improving it?
Having multiple connection is not a problem, the only thing to keep in mind is that SQLite does not support concurrency of multiple write transactions. From the SQlite site:
SQLite supports an unlimited number of simultaneous readers, but it will only allow one writer at any instant in time. For many situations, this is not a problem. Writer queue up. Each application does its database work quickly and moves on, and no lock lasts for more than a few dozen milliseconds. But there are some applications that require more concurrency, and those applications may need to seek a different solution.
SQLite is an "untypical" database management system: in practice it is a library that offers SQL as language to access a simple "database-in-a-file", and a few other functionalities of DBMSs. For instance, it has no real concurrency control (it uses the Operating Systems functions to lock the db file).
So, if you need concurrent insertions into a database, you should use something else, for instance PostgreSQL.
The documentation say:
A connection can only be used from within the thread that created it.
Moving connections between threads or creating queries from a
different thread is not supported.
In addition, the third party libraries used by the QSqlDrivers can
impose further restrictions on using the SQL Module in a multithreaded
program. Consult the manual of your database client for more
information.
It is mean you have to create connection to database which will be linking with parent thread. At docs of QSqlDatabase class you can see description:
The QSqlDatabase class represents a connection to a database.
The QSqlDatabase class provides an interface for accessing a database
through a connection. An instance of QSqlDatabase represents the
connection. The connection provides access to the database via one of
the supported database drivers, which are derived from QSqlDriver.
Create a connection (i.e., an instance of QSqlDatabase) by calling one
of the static addDatabase() functions, where you specify the driver or
type of driver to use (i.e., what kind of database will you access?)
and a connection name.
Using static addDatabase() function is way to create connection.
But as Renzo said SQLite does not support multiple write transactions at the same time. So you need some mechanisms(wrapper) for synchronizing threads like task queue using low-level mutex or something like that. More information you can see at docs.
I'm looking to store sessions in an external data store instead of in memory. I want to do this in order to have session data available to all my Jetty server instances.
So I read the Jetty docs:
Session Clustering with a Database
Session Clustering with MongoDB
Both docs have the following statement:
The persistent session mechanism works in conjunction with a load balancer that supports stickiness.
My question is - why do I need stickiness in this mechanism? The whole point is to allow any server in the cluster serve any request because the session data is stored externally.
The problem is that the servlet specification does not provide any transactional semantics around sessions. Thus, unless your sessions are sticky its possible for concurrent requests involving the same session to go to multiple servers and produce inconsistent results, depending on the interleaving of the writes. If your sessions are read-mostly you may get away with non-sticky sessions, although you may find that sessions time out a little sooner or a little later than you'd expect, due to having multiple servers trying to manage the same session.
We're in the process of rewriting a web application in Java, coming from PHP. I think, but I'm not really sure, that we might run into problems in regard to connection pooling. The application in itself is multitenant, and is a combination of "Separate database" and "Separate schema".
For every Postgres database server instance, there can be more than 1 database (named schemax_XXX) holding more than 1 schema (where the schema is a tenant). On signup, one of two things can happen:
A new tenant schema is created in the highest numbered schema_XXX database.
The signup process sees that a database has been fully allocated and creates a new schemas_XXX+1 database. In this new database, the tenant schema is created.
All tenants are known via a central registry (also a Postgres database). When a session is established the registry will resolve the host, database and schema of the tenant and a database session is established for that HTTP request.
Now, the problem I think I'm seeing here is twofold:
A JDBC connection pool is defined when the application starts. With that I mean that all databases (host+database) are known at startup. This conflicts with the signup process.
When I'm writing this we have ~20 database servers with ~1000 databases (for a total sum of ~100k (tenant) schemas. Given those numbers, I would need 20*1000 data sources for every instance of the application. I'm assuming that all pools are also, at one time or another, also started. I'm not sure how much resources a pool allocates, but it must be a non trivial amount for 20k pools.
So, is it feasable to even assume that a connection pool can be used for this?
For the first problem, I guess that a pool with support for JMX can be used, and that we create a new datasource when and if a new schemas_XXX database is created. The larger issue is that of the huge amount of pools. For this, I guess, some sort of pool manager should be used that can terminate a pool that have no open connections (and on demand also start a pool). I have not found anything that supports this.
What options do I have? Or should I just bite the bullet and fall back to an out of process connection pool such as PgBouncer and establish a plain JDBC connection per request, similar to how we're handling it now with PHP?
A few things:
A Connection pool need not be instantiated only at application start-up. You can create or destroy them whenever you want;
You obviously don't want to eagerly create one Connection pool per database or schema to be open at all times. You'd need to keep at least 20000 or 100000 Connections open if you did, a nonstarter even before you get to the non-Connection resources used by the DataSource;
If, as is likely, requests for Connections for a particular tenant tend to cluster, you might consider lazily, dynamically instantiating pools, and destroying them after some timeout if they've not handled a request for a while.
Good luck!
I am experiencing very low performance in my web application in which trivial HTTP requests take dozens of seconds to be processed. Tracing down the application code I discovered the majority of time is spent executing the first DB query, even if it is as simple as a SELECT on a single row-single column table. This happens for every HTTP request, independently from the query performed. After this first pathological DB interaction the remaining queries go smoothly.
I am using Hibernate on top of an Oracle DB (using jdbc).
It is not a problem of connection pool since I am successfully using Hibernate-c3p0, neither it seems to be related to Oracle itself, because all query returns immediately if performed directly on DB.
Furthermore, Hibernate SessionFactory is correctly created only once, at application start up time and concurrency is not a problem at all since tests have been done with single user.
Finally, my DB IP address is correctly resolved in my application server /etc/hosts so that even DNS related issues can be discarded (I am using two distinct virtual machines, DB and APP server).
I do not know what to look for, any help?
This sounds like your session factory object is being spun up on the first query. Generally I try to initialize the session factory on application startup to avoid this when issuing the first query because generally the user can see this slowdown. When doing it up front in application startup you will avoid this.
My Java application is processing a lot of information parsed from XML. For some methods, I need to validate some info in an SQL Database on another machine. I am using JDBC, currently for each validation, I call a DB Handling method that opens a connection and returns the result set to validate. I am not sure if I am taking the best design option. This seems very dummy and expensive to me. I am wondering if there are better practices.
Is there is a proper way to have the connection open through the whole application run time so no time is wasted (opening connection) for every validation iteration (I might have hundreds of thousands).
Should I build another application that retrieves all needed tables and convert them to XMLs that are saved on my machine. Later, my application parses them normally and have better access and performance
I am open to any better suggestion
Of course you can use one db connection for whole life of application. Use Singleton pattern. If your program work long and do not use database for a longer time it may loose db connection (some kind of timeout on network equipment etc.). For such cases you can use db pool. Such pool should manage longer time of inactivity or give you separate db connections for separate threads.
I think your solution with converting data into XML is not good. Why do you want to convert it into XML? How often is this data changed? How big is the database you want to copy? How do you want to synchronize your local copy with database? I think that local copy in XML files adds too many problems. If data is small than maybe you can read it at start of your program, save it in some data structures and use to verify other data? It can even be SQLite or other small database. But I would go this way only if singleton or db pool performance is really weak.