I am implementing database schema based multitenancy strategy. I use PostgreSQL for db, HikariCP for pool, and it's a SpringBoot application. I will need to set the schema after getting the connection, like the code below. (this.ds is a hikari pool)
#Override public Connection getConnection() throws SQLException {
Connection connection = this.ds.getConnection();
// reset target schema to...
String schema = determineSchema();
connection.createStatement().execute("SET search_path to "+schema);
return connection;
}
So my question is, say two API calls with tenant1 and tenant2 comes nearly the same time visiting getConnection() function and set schema. I am wondering how will the code handle it? Ideally, I would like to see Hikari will give them 2 different connections and there would be no conflict in schema set. But is it the real case? Do we need a read/write lock here? Thanks ahead.
A connection pool keeps track of what connections it has checked out; once a connection is handed off to a thread it doesn’t give that same connection out again until the connection is handed back. If a connection isn’t handed back then you have a connection leak. The leak happens because the pool hasn’t gotten notified that the user of the connection is finished with it so it refuses to hand it out. It errs on the side of not handing out a connection if there is any doubt about if the connection is in use.
In the event the pool did mess up and hand out the same connection twice there is no locking you can do that would fix the problem, you would end up with one of the two threads writing to the wrong schema (unless both just happened to ask for the same schema, of course). If Hikari made this error it wouldn’t be useable. Fortunately connection pools don’t seem to have a problem getting this right.
Related
Because of too low set of max limit of processes and sessions in an Oracle DB, sometimes the following error occurs in mirth:
DBConnection - java.sql.SQLException: Listener refused the connection with the following error:
ORA-12516, TNS:listener could not find available handler with matching protocol stack
due to dbConn.executeCachedQuery(SQLStatement) with the DatabaseConnection Class in Mirth
So these are my questions:
Is there any way to throw this response/exception in the channel?
Is all data of the SQL query with the exception "lost", if this error occurs or is there an automatic retry?
And is there a best practice to handle this (e.g. to check first the the connection with a getConnection() method)?
I'll answer your questions in order:
1) If you are using the javascript connector, then you should have this in a try catch when you initiate the connection. In the catch, just put the error as Logger.Error(exceptionGoesHere).
If you are using the db connector this should get throw automatically in the logs. To make sure you have logging enabled at the channel level, access the Mirth Connect Server Manager, click on the Server tab and ensure that Channel Log Level is at least set to Error.
2) The way that Mirth Connect works, every time the message is initiated it will hit certain points of the Mirth DB to save the state of the message at that point in time. It's how Mirth guarantees message delivery. With that being said, you can always 'retry' the send manually. Otherwise, if you are using the DB connector there's an option that handles this for you under the Database Reader Settings section. The retry gives you the option to select the number of retries as well as the Retry interval in ms. When I was working there, by default it was set to 3 retries after 10 seconds.
3) Use the default database connector. Everything's already built in for you. Put the extra processing in the transformer to handle anything else. Don't try to re-invent the wheel if everything is already built is the best practice.
If you insist on using a code solution, then make sure all of your code is in a try catch, and make sure your catch is actually logging out the error exception.
I realize there is a method to set on MQConnectionFactory to attempt to reconnect if the connection of a consumer or producer is broken. However, I'm wondering if one can do something similar for an application that is starting up and setting up consumers and producers. The code I have right now will not recover if the server is down when my client application comes up.
Is there a common/recommended practice here?
My recommendation would simply be to use the tools that are provided in the Java language itself. For example, you could write a loop with exception handling to retry the initial connection or JNDI lookup a configurable number of times. It's hard to provide more specific recommendations when you haven't provided any client code of your own.
I used the Spring JMS cachingconnectionfactory to improve the performance of the my application based on Spring Integration and IBM MQ. I put sessioncachesize as 10 as we have the max of 10 concurrent threads working (ThreadPoolTaskExecutor) on consume/sending messages.
When I looked at the number of connections opened in MQ explorer (open output count for queue), it shows 10 and it stays on for days and never getting closed.
Is there a way to programatically to detect connections which are
potentially stale - say idle for half a day - I checked the
resetConnection() but not sure how to get the last used time for the
session.
Does Spring provides any connection time out parameter for
cacheconnection factory? or How to release these idle connections?
Also, the heartbeat/keepalive mechanism will not work for us as we want to physical close the cached connections based on last used time.
If the timeout is a property of the Session object returned by IBM, you could subclass the connection factory, override createSession(); call super.createSession(...) then set the property before returning it.
You might also have to override getSession(...) and keep calling it until you a get a session that is not closed. I don't see any logic to check the session state in the standard factory. (getSession() calls createSession() when the cache is empty).
I have an application that uses a Spring-EntityManager (JPA) and I wonder what happens if the database happens to be unavailable during the lifetime of my aforesaid application.
I expect in that situation it will throw an exception the first time to do anything on the database, right?
But, say I wait 10 minutes and try again then and the DB happens to be back. Will it recover? Can I arrange it so it does?
Thanks
Actually, neither Spring nor JPA have anything to do with it. Internally all persistence frameworks simply call DataSource.getConnection() and expect to receive (probably pooled) JDBC connection. Once they're done, they close() the connection effectively returning it to the pool.
Now when the DataSource is asked to give a connection but database is unaivalable it will throw an exception. That exception will propagate up and will be somehow handled by whatever framework you use.
Now to answer your question - typically DataSource implementation (like dbcp, c3p0, etc.) will discard connection known to be broken and replace it with a fresh one. It really depends on the provider, but you can safely assume that once the database is available again, the DataSource will gradually get rid of sick connections and replace them with healthy ones.
Also many DataSource implementors provide ways of testing the connection periodically and before it is returned to the client. This is important in pooled environemnts where the DataSource contains a pool of connections and when the database becomes unavailable it has no way to discover that. So some DataSources test connection (by calling SELECT 1 or similar) before giving it back to the client and do the same once in a while to get rid of broken connections, e.g. due to broken underlying TCP connection.
TL;DR
Yes, you will get an exception and yes the system will work normally once the database is back. BTW you can easily test this!
I'm creating a new Client / Server application in C# and expect to have a fairly high rate of connections. That made me think of database connection pools which help mitigate the expense of creating and disposing connections between the client and database.
I would like to create a similar capability for my application and haven't been able to find any good examples of how to apply this pattern. Do I really need to spin up an instance of a TcpClient every time I want to send a message to the server and receive a receipt message? Each connection is expected to transport between 1-5KB with each receiving a 1KB response message.
I realize this question is somewhat vague, but I am starting from scratch so I am open to suggestions. Even if that means my suppositions are all wrong.
Introducing connection pool is a sort of optimization. Premature optimization can be bad.
I would recommend you start develpment without introducing connection pool. When client and server code is stable enough you can create load tests and detect performance problems.
Connection pool is required if time to create a connection is considerable compared to the rate at which data is coming to/from server (load tests should indicate that).
If data from client is send not that often you may not even need connection pool.