I have to write an operation to database only if a subsequent (non transactional) operation succeeds.
The condition is that the operation (let's say, send an e-mail) is slow, and I want to rollback de transaction if this operation fails. The structure is very simple:
#Transactional // A transaction starts and a connection is taken.
public void doStuff(String text) {
writeToDataBase(text);
sendEmail(text); //Rollback previous operation if this fails.
}
It works fine like this, the problem is that if the "sendEmail(text)" operation is slow, then the database connection keeps blocked until this operation finishes, the transaction is commited and the connection released.
I am performing load tests and the connection seems to be behaving in a way that I dont understand, it throws this error:
Connection oracle.jdbc.driver.T4CConnection#50e5e340 marked as broken because of SQLSTATE(72000), ErrorCode(1013)
How should this kind of logic be managed?
Thanks!
Related
I have one method where some DB operations are happened with some API call. This is a scenario with Spring and postgres DB. We also have property set for idle transaction in postgres DB
idle_in_transaction_session_timeout = 10min
Now the issue is I do get Exception sometime
org.postgresql.util.PSQLException: This connection has been closed. Root Cause is FATAL: terminating connection due to idle-in-transaction timeout
For example, my code look like this:
#Transactional(value = "transactionManagerDC")
public void Execute()
{
// 1. select from DB - took 2 min
// 2. call another API - took 10 min. <- here is when the postgres close my connection
// 3. select from DB - throws exception.
};
What could be the correct design for it? We are using output for select from step 1 in API call and output of that API call used in step 3 for select. So these three steps are interdependent.
Ten minutes is a very long time to hold a transaction open. Your RDBMS server automatically disconnects your session, rolling back the transaction, because it cannot tell whether the transaction was started by an interactive (command-line) user who then went out to lunch without committing it, or by a long-running task like yours.
Open transactions can block other users of your RDBMS, so it's best to COMMIT them quickly.
Your best solution is to refactor your application code so it can begin, and then quickly commit, the transaction after the ten-minute response from that other API call.
It's hard to give you specific advice without knowing more. But you could set some sort of status = "API call in progress" column on a row before you call that slow API, and clear that status within your transaction after the API call completes.
Alternatively, you can set the transaction timeout for just that connection with something like this, to reduce the out-to-lunch risk on your system.
SET idle_in_transaction_session_timeout = '15m';
In EAR application running on Websphere application server 8.5.5 we have to execute CallableStatement (call stored procedure in Oracle DB) which runs more than five minutes or more depending on input data. The operation is automatically rolled back because of transaction timeout (code WTRN0006W) which is set to 120 seconds by default in Websphere AS. We can't change this value due to customers requirements.
We can split input data to smaller chunks and execute CallableStatement several times to achieve shorter run time (30 seconds or so). Processing whole data chunks still takes more than 120 seconds (as expected). But the transaction timeout still occurs although for every statement execution with small chunk (in loop) we are getting connection from datasource configured in WAS, set autocommit to false, after statement execution doing commit and closing connection. Then again with next chunk in next loop cycle.
The whole process of statement executions is done in Stateless EJB which is called from Singleton EJB scheduled to run twice a day. We are not using JTA neither JPA, just JDBC.
Is it possible to avoid transaction timeout if we execute statement several times?
How we obtain datasource during application start:
javax.naming.Context ctx = new InitialContext();
javax.sql.Datasource ds = (javax.sql.Datasource) ctx.lookup("jndi/datasource1");
How we obtain Connection:
java.sql.Connection conn = m24sb.getConnection();
conn.setAutoCommit(false)
How we execute statement:
try (CallableStatement sta = conn.prepareCall("{ call SOME_STORED_PROC }"))) {
// ... setting statement params
sta.execute();
// ... resolving returned values
}
and then commit and closing connection.
Thanks in advance for answers!
Try marking your stateless session bean method as transaction NOT_SUPPORTED or NEVER, which will cause it to run outside of a global transaction. (Note that you would need to do this anyway for your connection.commit() call to be valid -- it likely just isn't getting that far due to the timeout).
#javax.ejb.TransactionAttribute(javax.ejb.TransactionAttributeType.NEVER)
public void yourStatelessEJBMethod() {
... code that invokes stored procedure and commits transaction
}
I am trying to implement hystrix with #Transactional on a method in spring boot.
#Transactional(transactionManager="primaryTrnsManager")
#HystrixCommand(commandKey ="createRecord", fallbackMethod="createRecordFallback", commandProperties={
#HystrixProperty(name="execution.siolation.thread.timeoutInMilliseconds",value="1000"),
#HystrixProperty(name="circuitBreaker.requestVoulumeThreshold",value="20"),
#HystrixProperty(name="circuitBreaker.sleepWindowInMilliseconds",value="5000"),
#HystrixProperty(name="circuitBreaker.errorThresholdPercentage",value="50")})
public String createRecord(String name){
...............
//Dbcall
}
#Transactional(transactionManager="backUptranManager",propagation=propagation.REQUIRES_NEW)
public String createRecordFallback(){
//dbcall
}
What what is happeneing is when the hystrix timeout happens the call i made to database is not getting rollback and the hystrix is falling back to secondary and again calling the database with same sql query. In case of timeout, i want to abort the previous transaction and start another one. Usually #Transactional does that but with hystrix,i am inserting dubplicate records.
Any Help?
This happen because the HistryxCommand is ran in a completely different thread, so, when the "Histryx Admin Thread" (the one which is watching your command thread) reaches the timeout, it's marking the command thread to be interrupted, but what happen inside of it can't be managed by Hyxtrix.
So, the best approach to reach out the behaviour that you want, is set the timeout for the transaction or even at the jdbc library level, so, the timeout will be managed from inside the command thread, if it's reached out, the Timeout Exception will be throwed from inside the command thread, and Hyxtrix will manage it properly.
I have an injected JDBCTemplate instance, and the code basically executes
private JdbcTemplate template;
public OutputType getOutput(InputType input) {
CallType call = new CallType(input);
CallbackType callback = new CallbackType(input);
OutputType output = (OutputType) template.execute(call, callback);
...
}
I assume the execute method actually connects to the database and retrieves the result. However, I am having trouble finding out how the control flow works from the documentation.
Is the response from execute blocking (thread occupies a CPU core the entire time waiting for the database response)? Is it synchronous, but not blocking (i.e. thread sleeps/is not scheduled until the response is ready)? Is it asynchronous (execute returns immediately but output is incomplete/null, all database processing logic is in the callback)?
I have used several different databases so I am unsure of what actually happens in JdbcTemplate. If my terminology is incorrect please let me know. Thanks!
The JDBC protocol itself is synchronous and blocking - it will block on socket I/O awaiting the database response. While this doesn't mean you couldn't asynchronously call a JDBC provider (manually spawn a separate thread, use actors, etc.), the actual connection to the database will always be synchronous.
JDBCTemplate is also fully synchronous and blocking, there is no thread magic going on under the hood.
I am using spring-boot(1.4.1) with hibernate(5.0.1.Final). I noticed that when I try to write to the db from within #TransactionalEventListener handler the call is simply ignored. A read call works just fine.
When I say ignore, I mean there is no write in the db and there are no logs. I even enabled log4jdbc and still no logs which mean no hibernate session was created. From this, I reckon, somewhere in spring-boot we identify that its a transaction event handler and ignore a write call.
Here is an example.
// This function is defined in a class marked with #Service
#TransactionalEventListener
open fun handleEnqueue(event: EnqueueEvent) {
// some code to obtain encodeJobId
this.uploadService.saveUploadEntity(uploadEntity, encodeJobId)
}
#Service
#Transactional
class UploadService {
//.....code
open fun saveUploadEntity(uploadEntity: UploadEntity, encodeJobId: String): UploadEntity {
// some code
return this.save(uploadEntity)
}
}
Now if I force a new Transaction by annotating
#Transactional(propagation = Propagation.REQUIRES_NEW)
saveUploadEntity
a new transaction with connection is made and everything works fine.
I dont like that there is complete silence in logs when this write is dropped (again reads succeed). Is there a known bug?
How to enable the handler to start a new transaction? If I do Propogation.Requires_new on my handleEnqueue event, it does not work.
Besides, enabling log4jdbc which successfully logs reads/writes I have following settings in spring.
Thanks
I ran into the same problem. This behavior is actually mentioned in the documentation of the TransactionSynchronization#afterCompletion(int) which is referred to by the TransactionPhase.AFTER_COMMIT (which is the default TransactionPhase attribute of the #TransactionalEventListener):
The transaction will have been committed or rolled back already, but the transactional resources might still be active and accessible. As a consequence, any data access code triggered at this point will still "participate" in the original transaction, allowing to perform some cleanup (with no commit following anymore!), unless it explicitly declares that it needs to run in a separate transaction. Hence: Use PROPAGATION_REQUIRES_NEW for any transactional operation that is called from here.
Unfortunately this seems to leave no other option than to enforce a new transaction via Propagation.REQUIRES_NEW. The problem is that the transactionalEventListeners are implemented as transaction synchronizations and hence bound to the transaction. When the transaction is closed and its resources cleaned up, so are the listeners. There might be a way to use a customized EntityManager which stores events and then publishes them after its close() was called.
Note that you can use TransactionPhase.BEFORE_COMMIT on your #TransactionalEventListener which will take place before the commit of the transaction. This will write your changes to the database but you won't know whether the transaction you're listening on was actually committed or is about to be rolled back.