Hi I have played a lot with the following code and has read https://github.com/quarkusio/quarkus/issues/21111
I think I am facing a very similar issue, where it will work the first 4 times and then it stops working and things are stuck and eventually showing.
2022-09-15 23:21:21,029 ERROR [io.sma.rea.mes.provider] (vert.x-eventloop-thread-16) SRMSG00201: Error caught while processing a message: io.vertx.core.impl.NoStackTraceThrowable: Timeout
I have seen such exact behaviours in multiple bug reports and discussion threads.
I am using quarkus-hibernate-reactive-panache + quarkus-smallrye-reactive-messaging with kafka (v2.12)
#Incoming("words-in")
#ReactiveTransactional
public Uni<Void> storeToDB(Message<String> message) {
return storeMetamodels(message).onItemOrFailure().invoke((v, throwable) -> {
if (throwable == null) {
Log.info("Successfully stored");
message.ack();
} else {
Log.error(throwable, throwable);
message.nack(throwable);
}
});
}
private Uni<Void> storeMetamodels(Message<String> message) {
List<EntityMetamodel> metamodels = Lists.newArrayList();
for (String metamodelDsl : metamodelDsls.getMetamodelDsls()) {
try {
EntityMetamodel metamodel = new EntityMetamodel();
metamodel.setJsonSchema("{}")
metamodels.add(metamodel);
} catch (IOException e) {
Log.error(e, e);
}
}
return Panache.getSession().chain(session -> session.setBatchSize(10)
.persistAll(metamodels.toArray((Object[]) new EntityMetamodel[metamodels.size()])));
}
NOTE This same code works if it is running on RestEasy Reactive but I need to move the actual processing and storing to DB away from rest easy as it will be a large process and I do not want it to be stuck on the Rest API waiting for a few minutes.
Hope some Panache or smallrye reactive messaging experts can shed some lights.
Could you try this approach, please?
#Inject
Mutiny.SessionFactory sf;
#Incoming("words-in")
public Uni<Void> storeToDB(Message<String> message) {
return storeMetamodels(message).onItemOrFailure().invoke((v, throwable) -> {
if (throwable == null) {
Log.info("Successfully stored");
message.ack();
} else {
Log.error(throwable, throwable);
message.nack(throwable);
}
});
}
private Uni<Void> storeMetamodels(Message<String> message) {
List<EntityMetamodel> metamodels = Lists.newArrayList();
for (String metamodelDsl : metamodelDsls.getMetamodelDsls()) {
try {
EntityMetamodel metamodel = new EntityMetamodel();
metamodel.setJsonSchema("{}")
metamodels.add(metamodel);
} catch (IOException e) {
Log.error(e, e);
}
}
return sf
.withTransaction( session -> session
.setBatchSize(10)
.persistAll(metamodels.toArray((Object[]) new EntityMetamodel[metamodels.size()]))
);
}
I suspect you've hit a bug where the session doesn't get closed at the end of storeToDB. Because the session doesn't get closed when injected using Panache or dependency injection, the connection stays open and you hit the limit of connections that can stay open.
At the moment, using the session factory makes it easier to figure out when the session gets closed.
Related
I have a DefaultMessageListenerContainer which is processing a message from the queue.
While the message is being processed -- stop , shutdown methods are called on DefaultMessageListenerContainer. Does this close database connections?
Looks like it is closing the database connections and hence the message being processed is getting interrupted from completely processing.
I see these errors :
o.s.jdbc.support.SQLErrorCodesFactory : Error while extracting database name
Closed Connection; nested exception is java.sql.SQLRecoverableException: Closed Connection
could these be because the DefaultMessageListenerContainer was stopped and shutdown ?
My code is as follows . startStopContainer is where I am trying to stop and shutdown container. I want to shutdown container only if listener completed processing the current message. I added logic to figure out if listener completed processing .
Is the below logic the only way or is there a better way to figure out if listener completed processing. Please suggest. Thank you.
public class MyMessageConsumerFacade {
private ConnectionFactory connectionFactory() {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL(url);
connectionFactory.setUserName(userName);
connectionFactory.setPassword(password);
return connectionFactory;
}
#Bean
public MessageListenerContainer listenerContainer() {
DefaultMessageListenerContainer container = new DefaultMessageListenerContainer();
container.setConnectionFactory(connectionFactory());
container.setDestinationName(queueName);
container.setMessageListener(new MyJmsListener());
return container;
}
}
public class MyJmsListener implements MessageListener {
public boolean onMessageCompleted;
public void onMessage(Message message) {
onMessageCompleted = false;
processMessage(message);
onMessageCompleted = true;
}
}
private String startStopContainer(ExecutionContext etk) {
String response = "success";
AnnotationConfigApplicationContext context = null;
DefaultMessageListenerContainer myNewContainer = null;
if (context == null) {
context = new AnnotationConfigApplicationContext(MyMessageConsumerFacade.class);
}
if (myNewContainer == null) {
myNewContainer = context.getBean(DefaultMessageListenerContainer.class);
}
MyCaseMessageJmsListener messageJmsListener = (MyCaseMessageJmsListener) myNewContainer.getMessageListener();
if (!myNewContainer.isRunning()) {// container not running
myNewContainer.start();
}
//due to some business logic we need to stop listener every 5 minutes, so sleep for 5 minutes and then stop
try {
Thread.sleep(300000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
if (myNewContainer.isRunning()) {
myNewContainer.stop();
}
//Before shutting down container , make sure listener processed all msgs completely
if(!messageJmsListener.isOnMessageCompleted) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
if(messageJmsListener.isOnMessageCompleted) {
myNewContainer.shutdown();
}
if (context != null) {
context.destroy();
}
return response;
}
Is there a better way than this?
No; the container knows nothing about JDBC or any connections thereto.
Stopping the container only stops the JMS consumer(s) the consumers from receiving messages; shutDown() on the container closes the consumer(s).
Something else is closing your JDBC connection.
I can't seem to get an unbounded source to work with bufferTimeout without falling into unlimited demand. My source has a lot of data, but I can selectively pull from it, so there is no need to buffer a lot of data in memory if it isn't requested. However, I cannot figure out how to get reactor to both 1. Not request unlimited demand. 2. Not overflow when the source is a bit slow to respond.
Here is a JUnit test case:
#Test
void bufferAllowsRequested() throws InterruptedException {
ExecutorService workers = Executors.newFixedThreadPool(4);
AtomicBoolean down = new AtomicBoolean();
Flux.generate(sink -> {
produceRequestedTo(down, sink);
}).concatMap(Flux::fromIterable).bufferTimeout(400, Duration.ofMillis(200))
.doOnError(t -> {
t.printStackTrace();
down.set(true);
})
.publishOn(Schedulers.fromExecutor(workers), 4)
.subscribe(this::processBuffer);
Thread.sleep(3500);
workers.shutdownNow();
assertFalse(down.get());
}
private void processBuffer(List<Object> buf) {
System.out.println("Received " + buf.size());
try {
Thread.sleep(400);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
private void produceRequestedTo(AtomicBoolean down, SynchronousSink<Object> sink) {
Thread.sleep(new Random().nextInt(1000));
try {
sink.next(IntStream.range(0, 500).boxed().collect(Collectors.toList()));
} catch (Exception e) {
e.printStackTrace();
down.set(true);
}
}
I've tried both Flux.create and Flux.generate, but both seem to suffer from this problem. I don't understand how this isn't a common use case.
I filed an issue here: https://github.com/reactor/reactor-core/issues/1557
I have been trying to do load testing(500 request in 1 second) with gatling for following three request handlers created with spring boot. Inspired by http://callistaenterprise.se/blogg/teknik/2014/04/22/c10k-developing-non-blocking-rest-services-with-spring-mvc/
but i received reverse output. "getUser" -BLOCKING request handler was able to handle 500 request in 1 second(but launched more then 200 threads).
"getUser2" and "getUser3" which are NONBLOCKING resulted in 503 error after handling around 10 request out of 500 request.
screen shot of gatling output: https://ibb.co/ePEEBk
Can somebody please explain me the reason or what I am doing wrong.
#RequestMapping("/getUser2")
public CompletionStage<Object> getUser2() {
return CompletableFuture.supplyAsync(()-> {
try {
logger.info("");
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
return "result";
});
}
#RequestMapping("getUser3")
public DeferredResult<String> getUser3() {
DeferredResult<String> def = new DeferredResult<String>();
CompletableFuture.runAsync(()-> {
logger.info("");
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}).whenCompleteAsync((res,err)-> {
def.setResult("result");
});
return def;
}
#RequestMapping("getUser")
public String getUser() {
try {
logger.info("");
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
return "result";
}
I am not sure if you have got the answer.
But for getUser3(), it seems your REST api is timed out. By default it will use the default timeout from the Spring MVC.
One thing you can do is to create a DeferredResult with a timeout value.
DeferredResult(java.lang.Long timeout)
See this constructor for further information https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/web/context/request/async/DeferredResult.html#DeferredResult-java.lang.Long-
For getUser, it should be the same, you probably need a longer timeout to fix it
I want to bind a connection to a thread, and use that connection for any JdbcTemplate calls, to finally commit the changes or do a rollback.
I'm declaring all sentences from a Groovy script, so I can't control how many SQL query will be call, that's why I have to used this method and not a TransactionalTemplate or something like that. this script will call a helper class that will use that connection and JdbcTemplate, let's call that class SqlHelper.
Right now my non-working-as-needed solution is call from groovy script to that SqlHelper to initialize a transaction:
initTransaction(ecommerce)
which calls
public void initTransaction(DataSource dataSource) {
DefaultTransactionDefinition transactionDefinition = new DefaultTransactionDefinition();
transactionDefinition.setReadOnly(false);
transactionDefinition.setIsolationLevel(TransactionDefinition.ISOLATION_SERIALIZABLE);
// dataSourceTransactionManager.setDataSource(dataSource);
// dataSourceTransactionManager.setRollbackOnCommitFailure(true);
// dataSourceTransactionManager.setTransactionSynchronization(TransactionSynchronization.STATUS_COMMITTED);
// dataSourceTransactionManager.setDataSource(dataSource);
try {
Connection connection = DataSourceUtils.getConnection(dataSource);
connection.setAutoCommit(false);
DataSourceUtils.prepareConnectionForTransaction(connection, transactionDefinition);
} catch (CannotGetJdbcConnectionException e) {
throw new RuntimeException(e);
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
After that the script will call some SQL operations, like
sqlUpdate(ecommerce, insertSentence, insertParams)
which calls
public Integer update(DataSource dataSource, String sql, Map<String, Object> paramMap) {
return new NamedParameterJdbcTemplate(dataSource).update(sql, paramMap);
}
Finally I want to finish the transaction committing the changes using
commitTransaction(dataSource)
which calls
public void commitTransaction(DataSource dataSource) {
Connection connection = DataSourceUtils.getConnection(dataSource);
try {
connection.commit();
} catch (Exception e) {
rollbackTransaction(dataSource);
}
// DataSourceUtils.resetConnectionAfterTransaction(connection, TransactionDefinition.ISOLATION_DEFAULT);
// SimpleTransactionStatus transactionStatus = new SimpleTransactionStatus(false);
// try {
// dataSourceTransactionManager.commit(transactionStatus);
// jta.commit(transactionStatus);
// } catch (TransactionException e) {
// dataSourceTransactionManager.rollback(transactionStatus);
// throw new RuntimeException(e);
// }
}
private void rollbackTransaction(DataSource dataSource) {
Connection connection = DataSourceUtils.getConnection(dataSource);
try {
connection.rollback();
} catch (SQLException e) {
throw new RuntimeException(e);
}
DataSourceUtils.resetConnectionAfterTransaction(connection, TransactionDefinition.ISOLATION_DEFAULT);
}
I left commented blocks of some testing to show you what approaches I tried. I don't know very well how Spring transaction works, so I'm just trying different things and trying to learn how all this stuff works... I will provide you more information if you want ;)
Thank you in advance.
UPDATE
As M. Denium suggested, that's what I have for now:
I declared the variable, using the TransactionStatus as ThreadSafe and finally solved as:
private DataSourceTransactionManager dataSourceTransactionManager = null;
private DefaultTransactionDefinition transactionDefinition = null;
private static final ThreadLocal<TransactionStatus> transactionStatus = new ThreadLocal<TransactionStatus>() {
#Override
protected TransactionStatus initialValue() {
return null;
}
};
And then using the same call from Groovy script, using the helper methods:
public void initTransaction(DataSource dataSource) {
dataSourceTransactionManager = new DataSourceTransactionManager();
transactionDefinition = new DefaultTransactionDefinition();
transactionDefinition.setReadOnly(false);
transactionDefinition.setIsolationLevel(TransactionDefinition.ISOLATION_SERIALIZABLE);
dataSourceTransactionManager.setRollbackOnCommitFailure(true);
dataSourceTransactionManager.setTransactionSynchronization(TransactionSynchronization.STATUS_UNKNOWN);
dataSourceTransactionManager.setDataSource(dataSource);
transactionStatus.set(dataSourceTransactionManager.getTransaction(null));
}
public void commitTransaction() {
try {
LOG.info("Finishing transaction...");
dataSourceTransactionManager.commit(transactionStatus.get());
dataSourceTransactionManager.getDataSource().getConnection().close();
LOG.info("Done.");
} catch (Throwable e) {
dataSourceTransactionManager.rollback(transactionStatus.get());
throw new RuntimeException("An exception during transaction. Rolling back.");
}
}
You are trying to reimplement the things that are already implemented by the transaction abstraction of Spring. Simply use the proper PlatformTransactionManager (you can probably grab that from an ApplicationContext) keep a reference to the TransactionStatus instead of a DataSource and use that to commit/rollback.
public TransactionStatus initTransaction() {
return transactionManager.getTransaction(null);
}
public void commit(TransactionStatus status) {
transactionManager.commit(status);
}
Instead of passing the TransactionStatus around you could also store it in a ThreadLocal and retrieve it in the commit method. This would ease the pain.
Another tip you shouldn't be creating JdbcTemplates and NamedParameterJdbcTemplates those are heavy objects to create. Upon construction they consult a connection to determine which database and version this is needed for the exception conversion. So performance wise this isn't a smart thing to do. Create a single instance and reuse, the templates are thread safe so you would only be needing a single instance.
However I would strongly argue that you should actually be using Groovy and not to try to work around it. Groovy has the Sql class that can help you. You already have access to the DataSource so doing something like this would be all that is needed.
def sql = new Sql(dataSource);
sql.withTransaction {
sql.execute "INSERT INTO city (name, state, founded_year) VALUES ('Minneapolis', 'Minnesota', 1867)"
sql.execute "INSERT INTO city (name, state, founded_year) VALUES ('Orlando', 'Florida', 1875)"
sql.execute "INSERT INTO city (name, state, founded_year) VALUES ('Gulfport', 'Mississippi', 1887)"
}
This is plain Groovy, no need to develop additional classes or to write extensive documentation to get it working. Just Groovy...
I am processing three transaction inside a single method in stateless container managed bean .i want to persist three transaction while if one throws exception other two should complete their respective transaction ,error is that if first or any one is throwing exception other two are to executing please give some helpful suggestion
public void allocateSubjectToStudent(SubjectAllocatedToStudentDto dto)throws Exception {
logger.info("allocateSubjectToStudent method entry :");
List<Subject> coreList=dto.getCoreList();
Iterator<Subject> iterator=coreList.iterator();
while(iterator.hasNext()){
logger.info("inside while :");
SubjectAllocatedToStudentBo bo=new SubjectAllocatedToStudentBo();
bo.setBacthId(dto.getBacthId());
bo.setSemester(dto.getSemester());
bo.setStudentId(dto.getStudentId());
Subject subject=iterator.next();
bo.setSubjectName(subject.getSubjectName());
bo.setSubjectType(subject.getAbbreviation());
try{
manager.persist(bo);
}
catch(javax.persistence.PersistenceException e){
Throwable t = e.getCause();
while ((t != null) && !(t instanceof org.hibernate.exception.ConstraintViolationException)) {
t = t.getCause();
}//while
if (t instanceof org.hibernate.exception.ConstraintViolationException) {
throw new Exception("Core subject already allocated to student");
} //end of if
}//end of catch
}//end of while
List<Subject> departmentallist=dto.getDepartmentList();
Iterator<Subject> iterator1=departmentallist.iterator();
while(iterator1.hasNext()){
logger.info("inside while :");
SubjectAllocatedToStudentBo bo=new SubjectAllocatedToStudentBo();
bo.setBacthId(dto.getBacthId());
bo.setSemester(dto.getSemester());
bo.setStudentId(dto.getStudentId());
Subject subject=iterator1.next();
bo.setSubjectName(subject.getSubjectName());
bo.setSubjectType(subject.getAbbreviation());
try{
manager.persist(bo);
}
catch(javax.persistence.PersistenceException e){
Throwable t = e.getCause();
while ((t != null) && !(t instanceof org.hibernate.exception.ConstraintViolationException)) {
t = t.getCause();
}//while
if (t instanceof org.hibernate.exception.ConstraintViolationException) {
throw new Exception("InterDepartmental subject already allocated to student");
} //end of if
}//end of catch
}//end of while
List<Subject> electiveList=dto.getElectiveList();
Iterator<Subject> iterator2=electiveList.iterator();
while(iterator2.hasNext()){
logger.info("inside while :");
SubjectAllocatedToStudentBo bo=new SubjectAllocatedToStudentBo();
bo.setBacthId(dto.getBacthId());
bo.setSemester(dto.getSemester());
bo.setStudentId(dto.getStudentId());
Subject subject=iterator2.next();
bo.setSubjectName(subject.getSubjectName());
bo.setSubjectType(subject.getAbbreviation());
try{
manager.persist(bo);
}
catch(javax.persistence.PersistenceException e){
Throwable t = e.getCause();
while ((t != null) && !(t instanceof org.hibernate.exception.ConstraintViolationException)) {
t = t.getCause();
}//while
if (t instanceof org.hibernate.exception.ConstraintViolationException) {
throw new Exception("Elective subject already allocated to student");
} //end of if
}//end of catch
}//end of while
logger.info("allocateSubjectToStudent method exit :");
} //end of method
create three different method all with TranscationAttributeType REQUIRES_NEW
Please find below code snippet for EJB3 Bean
public void doYourWork()
{
a();
b();
c();
}
#TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public void a()
{
try
{
//Do the first transaction here
}catch(Exception e)
{
}
}
#TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public void b()
{
try
{
//Do the second transaction here
}catch(Exception e)
{
}
}
#TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public void c()
{
try
{
//Do the third transaction here
}catch(Exception e)
{
}
}
Within a single method invocation there is only a single transaction active. To achieve what you want, you must perform the three operations in different transactions. This would require one more level of abstracttion.
public class MyFreshTransaction {
#TransactionAttribute(REQUIRES_NEW)
public void updateO() {
//do something
}
#TransactionAttribute(REQUIRES_NEW)
public void update1() {
//do something
}
#TransactionAttribute(REQUIRES_NEW)
public void update2() {
//do something
}
}
#Stateless
public class MyTransactionProcessor {
#EJB
private MyFreshTransaction freshTransaction;
public void processTransaction() {
try {
//The current transaction context will be suspended, and a new one invoked
//if the new one fails and is rollback, the current one is not affected.
//you can then handle the exception, by rethrowing the exception,in which case
//the current transaction will also be rolled back, or continue based on your logic.
freshTransaction.update0();
} catch (Exception ex ) {//handle}
try {
//The current transaction context will be suspended, and a new one invoked
//if the new one fails and is rollback, the current one is not affected.
//you can then handle the exception, by rethrowing the exception,in which case
//the current transaction will also be rolled back, or continue based on your logic.
freshTransaction.update1();
} catch (Exception ex ) {//handle}
try {
//The current transaction context will be suspended, and a new one invoked
//if the new one fails and is rollback, the current one is not affected.
//you can then handle the exception, by rethrowing the exception,in which case
//the current transaction will also be rolled back, or continue based on your logic.
freshTransaction.update2();
} catch (Exception ex ) {//handle}
}
}
Note that if any of the update transaction was successful, and the the parent transaction is rolled back, it will not affect the status of the 'child' transactions, as they had already been committed and their effects (if DB effects) will be committed too.
Read on Java EE Transactions Java EE Transactions