I have a controller action that does the following things:
Gets a domain object from the database
Uses info on that object to find a data file (on disk) and writes contents of that file to the response output stream.
My problem is that the database connection is reserved for the duration of the action, including the (long) time required to stream the data. This results in a lot of unnecessary database connections when there are several users streaming data at the same time.
def stream() {
StreamDetails sd = StreamDetails.get(params.id)
// Extract info needed to read the stream
String filename = sd.filename
// The database connection is no longer needed, how to properly release it?
// Start writing the data stream to response output
// This may take a long time and does not use a db connection
streamService.writeToOutput(filename,response.getOutputStream())
}
I have tried:
Injecting the sessionFactory bean to the controller and calling sessionFactory.currentSession.close() before calling the service. However this causes a SessionException on the line calling the service, ie. before entering the writeToOutput() method (and nothing in that method needs a database connection). AND I don't think the session should be really closed, just released to the pool.
Copy-pasting the code from streamService.writeToOutput(...) to the controller to avoid the service call. In this case all the code gets executed but a SessionException is still thrown after the action is complete.
How to properly release the connection early?
Have you tried to inject the dataSource? You could use the DataSourceUtils to create a new connection that you can then use to get the filename. You can then manually close() this connection.
I don't know if you can use this connection in combination with gorm, so you might have to create a custom sql query as well.
Related
I have one method where some DB operations are happened with some API call. This is a scenario with Spring and postgres DB. We also have property set for idle transaction in postgres DB
idle_in_transaction_session_timeout = 10min
Now the issue is I do get Exception sometime
org.postgresql.util.PSQLException: This connection has been closed. Root Cause is FATAL: terminating connection due to idle-in-transaction timeout
For example, my code look like this:
#Transactional(value = "transactionManagerDC")
public void Execute()
{
// 1. select from DB - took 2 min
// 2. call another API - took 10 min. <- here is when the postgres close my connection
// 3. select from DB - throws exception.
};
What could be the correct design for it? We are using output for select from step 1 in API call and output of that API call used in step 3 for select. So these three steps are interdependent.
Ten minutes is a very long time to hold a transaction open. Your RDBMS server automatically disconnects your session, rolling back the transaction, because it cannot tell whether the transaction was started by an interactive (command-line) user who then went out to lunch without committing it, or by a long-running task like yours.
Open transactions can block other users of your RDBMS, so it's best to COMMIT them quickly.
Your best solution is to refactor your application code so it can begin, and then quickly commit, the transaction after the ten-minute response from that other API call.
It's hard to give you specific advice without knowing more. But you could set some sort of status = "API call in progress" column on a row before you call that slow API, and clear that status within your transaction after the API call completes.
Alternatively, you can set the transaction timeout for just that connection with something like this, to reduce the out-to-lunch risk on your system.
SET idle_in_transaction_session_timeout = '15m';
I have a publisher that executes a long-running and large query on MongoDB and returns the data in a Flux. Entities that are marked in the database as "processed" will be filtered out and the entities are then buffered and passed to a concatMap operator (so that all buffered ≤elements are processed before elements in the next buffer are processed). It looks something like this:
Flux<Entity> entitiesFromMongoDb = myMongoRepository.executeLargeQuery();
entitiesFromMongoDb.filter(entity -> !entity.isProcessed())
.buffer(10)
.concatMap(bufferedEntityList ->
Flux.fromIterable(bufferedEntityList)
.flatMap(makeExternalCall)
.then()));
Where makeExternalCall calls a third-party remote server and sets the entity to processed after the call has been made. In most cases this works fine, but when the remote server is really slow or has an error then makeExternalCall will retry (with exponential backoff) the operation to the remote server. In some cases it can take quite a while before all 10 external calls have been processed. In fact it can take so long that the myMongoRepository.executeLargeQuery() publisher is restarted and the query is executed again. Now we run into a problem that I'll try to describe here:
Entity A is read from the database (i.e. it's returned in the flux produced by myMongoRepository.executeLargeQuery()). It's not yet marked as "processed" which means that entity.isProcessed() will return false and it'll be retained in the stream.
The external server is really slow or down so that makeExternalCall is forced to retry the operation before entity A has been marked as "processed" in the DB.
myMongoRepository.executeLargeQuery() is restarted and the query is executed again.
Entity A is read from the database once more. But the problem is that there's already another instance of entity A in-flight since it has not yet been marked as "processed" by the previous call to myMongoRepository.executeLargeQuery().
This means that makeExternalCall will be called twice for entity A, which is not optimal!
I could make an additional request to the DB and check the status of processed for each entity in the makeExternalCall method, but this will cause additional load (since an extra request is necessary for each entity) to the DB which is not optimal.
So my question is:
Is there a way to somehow "restart" the entire stream, and thus clear intermediary buffers (i.e. remove entity A that is in-flight from the ongoing stream) when the MongoDB query triggered by myMongoRepository.executeLargeQuery() is restarted/re-executed? Or is there a better way to handle this?
I'm using Spring Boot 2.2.4.RELEASE, project reactor 3.3.2.RELEASE and spring-boot-starter-data-mongodb-reactive 2.2.4.RELEASE.
Not sure If I understood the problem completely. But trying to answer as it sounds interesting.
As you need to be aware of the requests which are already being processed by the makeExternalCall, can you maintain a set / local cache which contains the entities which are being processed?
Set<Entity> inProgress = new HashSet<>();
Flux<Entity> entitiesFromMongoDb = myMongoRepository.executeLargeQuery();
entitiesFromMongoDb.filter(entity -> !entity.isProcessed())
.buffer(10)
.map(bufferedEntityList -> { // remove the inprogress requests to avoid redundant processing
bufferedEntityList.removeIf(inProgress::contains);
return bufferedEntityList;
})
.concatMap(bufferedEntityList ->
inProgress.addAll(bufferedEntityList);
Flux.fromIterable(bufferedEntityList)
.flatMap(makeExternalCall) //assuming once processed, it emits the entity object
.map(entity -> { //clear growing set
inProgress.remove(entity);
return entity;
})
.then()));
This approach is not a good solution when you need to scale your application horizontally. In that case instead of maintaining a local cache, you could go for an external cache server like redis.
I have an injected JDBCTemplate instance, and the code basically executes
private JdbcTemplate template;
public OutputType getOutput(InputType input) {
CallType call = new CallType(input);
CallbackType callback = new CallbackType(input);
OutputType output = (OutputType) template.execute(call, callback);
...
}
I assume the execute method actually connects to the database and retrieves the result. However, I am having trouble finding out how the control flow works from the documentation.
Is the response from execute blocking (thread occupies a CPU core the entire time waiting for the database response)? Is it synchronous, but not blocking (i.e. thread sleeps/is not scheduled until the response is ready)? Is it asynchronous (execute returns immediately but output is incomplete/null, all database processing logic is in the callback)?
I have used several different databases so I am unsure of what actually happens in JdbcTemplate. If my terminology is incorrect please let me know. Thanks!
The JDBC protocol itself is synchronous and blocking - it will block on socket I/O awaiting the database response. While this doesn't mean you couldn't asynchronously call a JDBC provider (manually spawn a separate thread, use actors, etc.), the actual connection to the database will always be synchronous.
JDBCTemplate is also fully synchronous and blocking, there is no thread magic going on under the hood.
I'm using the TransactionScope class within a project based on Silverlight and RIA services. Each time I need to save some data, I create a TransactionScope object, save my data using Oracle ODP, then call the Complete method on my TransactionScope object and dispose the object itself:
public override bool Submit(ChangeSet changeSet)
{
TransactionOptions txopt = new TransactionOptions();
txopt.IsolationLevel = IsolationLevel.ReadCommitted;
using (TransactionScope tx = new TransactionScope(TransactionScopeOption.Required, txopt))
{
// Here I open an Oracle connection and fetch some data
GetSomeData();
// This is where I persist my data
result = base.Submit(changeSet);
tx.Complete();
}
return result;
}
My problem is, the first time I get the Submit method to be called, everything is fine, but if I call it a second time, the execution gets stuck for a couple of minutes after the call to Complete (so, when disposing tx), then I get the Oracle error "ORA-12154". Of course, I already checked that my persistence code completes without errors. Any ideas?
Edit: today I repeated the test and for some reason I'm getting a different error instead of the Oracle exception:
System.InvalidOperationException: Operation is not valid due to the current state of the object.
at System.Transactions.TransactionState.ChangeStatePromotedAborted(InternalTransaction tx)
at System.Transactions.InternalTransaction.DistributedTransactionOutcome(InternalTransaction tx, TransactionStatus status)
at System.Transactions.Oletx.RealOletxTransaction.FireOutcome(TransactionStatus statusArg)
at System.Transactions.Oletx.OutcomeEnlistment.InvokeOutcomeFunction(TransactionStatus status)
at System.Transactions.Oletx.OletxTransactionManager.ShimNotificationCallback(Object state, Boolean timeout)
at System.Threading._ThreadPoolWaitOrTimerCallback.PerformWaitOrTimerCallback(Object state, Boolean timedOut)
I somehow managed to solve this problem, although I still can't figure out the reason it showed up in the first place: I just moved the call to GetSomeData outside the scope of the distributed transaction. Since the call to Submit may open many connections and perform any kind of operations on the DB, I just can't tell why GetSomeData was causing this problem (it just opens a connection, calls a very simple stored function and returns a boolean). I can only guess that has something to do with the implementation of the Submit method and/or with the instantiation of multiple oracle connections within the same transaction scope.
I started using nhibernate the same time I started using castle activerecord so I have those mixed up questions sometimes
I was wondering when activerecord (or nhibernate) closes a connection when the following code runs:
Dim entity = TABLE_Y.TryFind(id)
if not entity is nothing then
'long running process here
response.redirect("...")
end if
I'm asking this because this long running process takes more than one hour to finish and whenever the code redirects to another page I get an ORA-03135 (connection lost contact) telling me the connection was lost, this other page has the following active record query:
Dim entity = TABLE_X.FindAll(Order.Desc(...), _
Expression.Eq(...) And _
Expression.Eq(...)).FirstOrDefault
which then returns the ora-03135
so I was thinking if couldn't be any connection from activerecord that didn't close before the long running process
this long running process is literally another process started by the application which wait for it to end before redirecting to another page, so even if the other process uses active records it doesn't use the same connection string or whatsoever
it's funny because I'm starting a new query on a completely different table, is activerecord trying to reuse an existing connection that timed out? I tried to add "Pooling=False" and "Validate Connection=true" with no luck
thanks in advance
The connection will be closed when the session is disposed of, where this happens depends on your application. If you're using the module that comes with ActiveRecord then it will happen when the Application.EndRequest event fires (ie. at the end of your request), if you aren't, then you need to see where a SessionScope or TransactionScope is created and disposed (the dispose is where the connection is closed).
If you want to start a long running task and redirect before it's complete you will need to start it in another thread (eg. using ThreadPool or Tasks). You will also need to configure ActiveRecord to use the HybridWebThreadScopeInfo so that it will store the session in a thread local when the HttpContext is unavailable (which is what will happen in your background thread).
<activerecord threadinfotype="Castle.ActiveRecord.Framework.Scopes.HybridWebThreadScopeInfo, Castle.ActiveRecord">
Then in your task, wrap it in a TransactionScope or SessionScope (I prefer the former):
using(var trans = new TransactionScope()) {
// do your stuff here...
trans.VoteCommit();
}