Do we need to add any configuration on client side to avoid transaction retry errors in cockroach database - spring-boot

when there is concurrent updates happening on the same record, transaction 1 is able to update but as per the cockroach database documentation transaction t2 should be on queue, but transaction t2 is immediately failing. Do we need to add any other configuration to make transaction wait in queue instead of it throwing an immedaite retry transaction error. Thanks in advance

Yes, the client should handle transaction retry errors. Note that whenever possible, CockroachDB will auto-retry a transaction internally without notifying the client. CockroachDB will only send a serialization error to the client when it cannot resolve the error automatically without client-side intervention. In this case, cockroach provides client side tools to manually or automatically retry the txn. See the docs here.

As per CockroachDB Documentation
CockroachDB always uses SERIALIZABLE isolation, which is the strongest
of the four transaction isolation levels defined by the SQL standard
and is stronger than the SNAPSHOT isolation level developed later.
SERIALIZABLE isolation guarantees that even though transactions may
execute in parallel, the result is the same as if they had executed
one at a time, without any concurrency.
I am assuming you are using Spring Boot for you application and Spring Data for Database operations.
If there are concurrent updates in the database, it's always good to use SELECT FOR UPDATE when you are using sql query. Similarly, #Locks which does the same in JPA.
#Lock(LockModeType.PESSIMISTIC_READ)
#QueryHints({#QueryHint(name = "javax.persistence.lock.timeout", value = "3000")})
public Optional<Employee> findById(Long employeeId);
For more information you can refer documentation.

Related

Spring Boot DB connection release mode

Is there a way to control database connection release strategy in Spring Boot? Something like Hibernate's ConnectionReleaseMode?
My test code would be rougly following:
INSERT query
HTTP call
UPDATE query
INSERT and UPDATE queries are their own methods in a repository bean (either extending CrudRepository, or as a Mybatis #Mapper). HTTP call is in it's own bean.
Now my service bean is where I experiment with different #Transactional settings as my ultimate goal is to have these three steps executed within a single transaction.
The problem is, the HTTP call can take hundreds of millis and Spring is holding the database connection during that time. This quickly leads to an empty connection pool, while the connections themselves are idle.
I have done the same experiments with default configuration using spring-boot-starter-data-jpa and also using mybatis-spring-boot-starter.
The only thing which got me closer to my goal was setting spring.jpa.open-in-view=false for data-jpa, which would release db connections in case of not using #Transactional at all or with propagation set to NEVER. It doesn't work if it's all wrapped in a transactin though.
I'm feeling like I'm missing some vital part of the transaction concept in Spring. Though the Spring reference docs mentions release mode only in relation to JTA transaction manager + JEE container + Hibernate.
Transactions are bound to a connection/session so you can't release the connection in the middle of a transaction. You have to finish the transaction to close/release the connection. Keep in mind that a transaction could abort at any time, and since the HTTP call is not "part of" the transaction, but just runs while the transaction is in progress, it won't help you to keep the transaction open while the HTTP call is running. I'd suggest you go with one of the following solutions:
Use a circuit breaker to cancel the HTTP call if it hits a timeout to have an upper bound for how long a connection/transaction can be open/held
Move the HTTP call out of the transaction, either before or after the transaction
Use two database transactions
Note that you could use a transactional Job Scheduler to schedule a job in the first TX. The job could then, with at least once semantics, try to invoke the HTTP call and then continue with the second transaction or do some compensation if something fails.

Transacted Route and Transactional Endpoints, Transaction commit order

My route looks like below
from("jms:queue:IN_QUEUE) //(A) Transactional Endpoint
.transacted("required") //(B) TX Policy with PROPAGATION_REQUIRED and JPATxManager
.bean("someBean", "readFromDB()") //(C) Read from DB
.bean("someBean", "writeToDB()") //(D) Write to DB
.to("file:/home/src?fileName=demo_${id}.txt")
I know the JMS consumer at (A) will fork out JMS Transaction on each poll and attaches to the
thread.
Also the transacted node in (B) will fork out JPA transaction after an exchange reaches there
and attaches to the thread.
Please find my questions below:
Can two different transactions get attached to a single thread (like the one above) ?
If Yes, which one should get suspended ?
What should be the commit and rollback order of the above mentioned route ?
Note: I didn't find any obvious answer from Camel In Action 2nd Ed book, So please guide me
Good afternoon,
This is a variation on your other question.
The:
from("jms:queue:IN_QUEUE) //(A) Transactional Endpoint
endpoint is transacted, meaning that you have marked the JMS component as transacted and the JMS sessions will be managed by a JmsTransactionManager.
.transacted("required") //(B) TX Policy with PROPAGATION_REQUIRED and
JPATxManager
This should not be a JPA transaction manager, but a JTA transaction manager (like Arjuna). As in your other question, you now have a JMS local transaction for reading you message, and local JPA transacted sessions for your DB access. You want the PlatformTransactionManager (a JTA transaction manager) to synchronize the local transactions for you.
As to your questions:
Can two different transactions get attached to a single thread (like the one above) ?
That really doesn't make any sense.
If Yes, which one should get suspended ?
Nothing will get suspended.
What should be the commit and rollback order of the above mentioned route ?
The DB read is not transactional and does not need to be committed. The file write will actually happen as the JTA transactional context is closed. This leaves the DB write. If that fails, then the DB read does not matter, the message will get put back on the source destination and the file write will not be called.
Enabling the DEBUG logging for the various transaction managers is very helpful.
I could go on about this with this in painful detail. This is more for burki. I think that you will really appreciate this. Very subtle, and it happens a lot.
from("jms:queue:SRC_QUEUE")
.transacted("required")
.to("jms1:queue:DEST_QUEUE")
What happens if the two endpoints are marked as transacted ... but... you do not have the 'transacted' line? Well, a JMS local transaction was started on the message listener. This will be committed as the route ends. There are two independent local JMS transactions. These are not synchronized by a JTA transaction manager.
What actually happens is that the commit for the message 'get' is called. There is no actual commit for the message 'put'. The message 'put' is committed when the JMS session is closed. This is in the JMS spec, that closing the connection inherently commits any transaction. So, because there is no linkage between the two components, the 'get' is committed, and then the 'put' session is closed.
This means that you can lose messages if there is an outage between the commit for the message 'get' and the session close for the message 'put'.
Does that make sense? There is no linkage between the local transactions, so Camel closes them in order, starting with committing the 'get' before calling the 'put'.
JTA transaction synchronization is the key. You still have local transaction resources (not XA), but they can be very well managed in a pretty lightweight JTA transactional context.
from("jms:queue:SRC_QUEUE")
.transacted("required")
.to("DB:transactedwrite")
.to("jms1:queue:DEST_QUEUE")
I couldn't be bothered to look up the correct syntax for a database insert, but you get the idea. In this case, you can get duplicate DB inserts if the JMS 'put' fails. This is not 'all or nothing' XA transactions. The transactions are committed in order. If one in the middle succeeds, then the next transaction fails, well the 'get' will get rolled back and you will get duplicates up to the point of failure.
Sorry, I can't answer your specific questions, but I can give some specific infos about the transactions of your route.
You've got 3 different "systems" with different transaction "scopes"
A JMS broker from which you consume
A database you read and write from and you configured a JPA TxManager for
A file system (no transactions at all) as destination
First of all, if you want to have transaction safety across JMS and the database you have to use XA transactions.
Then, it is unclear if you expect the JMS consumer to be transacted (because of the transacted() in your route) or if you really configured a JMS connection with local JMS transactions. I assume that you really consume transacted.
Let's talk about what you got without line B of your route:
You consume transacted from the broker
Camel processes the message through your route
When any error occurs during route processing, the message is not committed on the broker and therefore redelivered to your route
The transaction that is opened by a transactional consumer is kept open by Camel until the route is successfully processed.
So the only obvious problem would be an error after the database write, that triggers a redelivery and the database write is done once again. Probably the write is not idempotent and therefore must not happen twice.
So to solve these problems, you either have to use XA transactions or you simply use local JMS transactions and implement compensation logic for the "gaps" like the one described above.
The database transaction on the other hand has no benefit unless the read and write operation must be done in a transaction (but I have doubts that this is the case with two individual bean calls and a JMS consumer).

Is it possible to wait transaction lock release with serializable isolation level?

Context:
We've built a JAVA application using spring framework (5.1.8) & spring boot (2.1.6) with the help of Jhipster.
It is a RESTFul application.
Data persistance is done in a PostGRES database managed by Hibernate (5.3.10).
Application constraints:
Because of our context, we have to process most of update transactions in the same order that they happen.
Problem:
All requests are processed by a business function declared as transactionnal (annoted with #Transactionnal(propagation=Propagation.REQUIRED)).
We set the isolation level to SERIALIZABLE to avoid execution of multiple transaction at the same time (concurrency).
However, with this isolation level, we have to manage the transactions repeat for case of concurrency exception.
Is there a way to force to wait transaction lock release before that isolation throws a concurrency exception (serialize access error), like with pessimistic locks ?
We tried to make transaction synchronized but it doesn't work because it seems commits are done outside the transactions.
Any help would be greatly appreciated.

TransactionSynchronizationRegistry vs TransactionSynchronizationManager

I have an application running no weblogic with ejb, with JTA provided by WLS. Now I am trying to adapt all application to work in tomcat without ejb, using spring instead. Also will use JPATransactionManager that spring provides, instead of JTA.
I used to use TransactionSynchronizationRegistry.getTransactionKey() in order to get tx object and use it for cache purposes. Now I am adapting spring and I have TransactionSynchronizationManager of spring, where I can't findv such an API.
My question is - do you know some analog of TransactionSynchronizationRegistry.getTransactionKey() in spring ?
Or where I could get some unique object with info about transaction?
Problem was solved by using some generated logical transaction ID. This transaction ID is stored by calling TransactionSynchronizationManager.bindResource(key, value) and retrieved by TransactionSynchronizationManager.getResource(key).
However it will not work properly in case if transaction interleaving.
Assume that you have flow with transaction interleaving when upstream transaction A is suspended whereas downstream transaction B is running.
In case if in two this transaction you try to store transaction ID with some key as mentioned above, you could have an issue that you mixing your resources.
In this case recommended use ResourceHolderSupport which is responsible to listen various states of transaction and bind or unbind resources.
In our case assume that tx1 is stores resource = "txID", "123456", and tx1 suspended because tx2 is starting to run, resource holder will unbind resource and when tx1 will be resumed - it will bind it again.

Sharing JMS and Hibernate transactions in a Spring MDB using Oracle Streams AQ?

I'm using Oracle 11g for my database and its Oracle Streams AQ feature as JMS implementation.
For all I know, it should be possible to implement a Spring based message-driven POJO (MDP) that uses the same data source for both transactional data access and JMS transactions -- all without XA-Transactions (IIRC, this was marketed as a feature of SpringSource Advanced Pack for Oracle).
Is this possible using Hibernate as well? Ideally, my MDP would start a JMS transaction and read a message from a queue, then re-use the transaction for data access through Hibernate. If anything goes wrong, the JMS and database transaction would both be rolled back, without using 2-phase commit (2PC).
I'm not much of a transaction guru, so before I start digging deeper, can anyone confirm that this is possible and makes sense as well?
Update:
What I want is an implementation of the Shared Transaction Resource pattern. The sample code demonstrates it for ActiveMQ and JDBC, but I need to use Oracle Streams AQ and Hibernate.
Update2:
The SpringSource Advanced Pack for Oracle has been open sourced as part of Spring Data JDBC and it "provides the option of using a single local transaction manager for both
database and message access without resorting to expensive distributed 2-phase commit
transaction management".
2PC shouldn't be necessary, as you say, since the appserver should take care of it. However, you'll pretty much have to use JTA (i.e. JavaEE container) transactions, rather than vanilla DataSource transactions, since JMS only works with JTA.
This isn't a big deal, it's just a bit more fiddly:
Your Spring config should use
<jee:jndi-lookup/> to get a
reference to your container's
DataSource, and you inject that
data source into your spring-managed
hibernate SessionFactory.
You then need to introduce a transaction manager into the context (<tx:jta-transaction-manager/> should work in most app-servers).
In your Spring JMS MessageListenerContainer, plug the above transaction manager reference into it.
Does that all make sense, or should I elaborate? This setup should ensure that the container-managed transactions are held across JMS and Hibernate interactions.

Resources