Spring Cloud Stream Kafka Binder KafkaTransactionManager results in a cycle in application context - spring-boot

I am setting up a basic Spring Cloud Stream producer with Kafka. The intent is to accept a HTTP POST, save the result of the post to a database with Spring Data JPA, and write the results to a Kafka topic using Spring Cloud Stream Kafka Binder. I am following the latest binder documentation on how to setup a KafkaTransactionManager, but this code results in an error on Application startup.
***************************
APPLICATION FAILED TO START
***************************
Description:
The dependencies of some of the beans in the application context form a cycle:
┌─────┐
| kafkaTransactionManager defined in com.example.tx.Application
↑ ↓
| org.springframework.boot.autoconfigure.kafka.KafkaAnnotationDrivenConfiguration
└─────┘
I have the following Bean defined in my Application class, which is the same as documentation.
#Bean
public KafkaTransactionManager kafkaTransactionManager(BinderFactory binders) {
ProducerFactory<byte[], byte[]> pf = ((KafkaMessageChannelBinder) binders.getBinder(null, MessageChannel.class)).getTransactionalProducerFactory();
KafkaTransactionManager tm = new KafkaTransactionManager<>(pf);
tm.setTransactionIdPrefix("tx-test");
return tm;
}
It seems that calling getBinder causes Spring to create the context again. How can I resolve this circular dependency?
Dependencies: Spring Boot parent 2.4.6; Spring Cloud BOM 2020.0.3

Something must have changed in one of the layers; here is a work around:
#Bean
SmartInitializingSingleton ktmProvider(BinderFactory binders, GenericApplicationContext context) {
return () -> {
context.registerBean("kafkaTransactionManager", KafkaTransactionManager.class,
((KafkaMessageChannelBinder) binders.getBinder(null, MessageChannel.class))
.getTransactionalProducerFactory());
context.getBean(KafkaTransactionManager.class).setTransactionIdPrefix("tx-test");
};
}
i.e. wait for the other beans to be created before registering and configuring the tm.

Related

Transactions, Spring Boot Starter JDBC & R2DBC

I am trying to migrate a Spring Boot project, version 2.3.0.M3, that have used JDBC template to R2DBC. The project also uses Liquibase so I cannot get rid of JDBC altogether.
I have both the spring-boot-starter-data-r2dbc and the spring-boot-starter-jdbc dependencies in the project with which I get the following exception when trying to run one of my tests:
org.springframework.beans.factory.NoUniqueBeanDefinitionException: No qualifying bean of type 'org.springframework.transaction.TransactionManager' available: expected single matching bean but found 2: transactionManager,connectionFactoryTransactionManager
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveNamedBean(DefaultListableBeanFactory.java:1180)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveBean(DefaultListableBeanFactory.java:416)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBean(DefaultListableBeanFactory.java:349)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBean(DefaultListableBeanFactory.java:342)
at org.springframework.transaction.interceptor.TransactionAspectSupport.determineTransactionManager(TransactionAspectSupport.java:480)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:335)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
...
The bean connectionFactoryTransaction manager is defined like this in the Spring class R2dbcTransactionManagerAutoConfiguration:
#Bean
#ConditionalOnMissingBean(ReactiveTransactionManager.class)
public R2dbcTransactionManager connectionFactoryTransactionManager(ConnectionFactory connectionFactory) {
return new R2dbcTransactionManager(connectionFactory);
}
The bean transactionManager is defined like this in the Spring class DataSourceTransactionManagerAutoConfiguration:
#Bean
#ConditionalOnMissingBean(PlatformTransactionManager.class)
DataSourceTransactionManager transactionManager(DataSource dataSource,
ObjectProvider<TransactionManagerCustomizers> transactionManagerCustomizers) {
DataSourceTransactionManager transactionManager = new DataSourceTransactionManager(dataSource);
transactionManagerCustomizers.ifAvailable((customizers) -> customizers.customize(transactionManager));
return transactionManager;
}
As can be seen, the #ConditionalOnMissingBean annotation contains different types which will cause an instance of both beans to be created.
However, in the Spring class TransactionAspectSupport there is this line of code in the determineTransactionManager method:
defaultTransactionManager = this.beanFactory.getBean(TransactionManager.class);
Since both of the transaction manager types, DataSourceTransactionManager and R2dbcTransactionManager, implement the TransactionManager interface, both the transaction manager beans as above will be matched and the error will occur.
I am now reaching out to hear if there is anyone who has managed to solve or work around this issue?
Thanks in advance!
With inspiration from M. Deinums answer (thanks!), I applied the following steps to my project and the test that failed earlier now runs successfully:
Remove the spring-boot-starter-jdbc dependency.
Add a dependency to spring-jdbc.
Add a dependency to HikariCP (com.zaxxer).
Add spring.liquibase user and password properties (I already had the url and change-log properties).
Remove all spring.datasource properties (I had url and drive-class-name).
I had the spring.r2dbc properties username, password and url defined which I did not need to change.
Update:
In addition, I used Testcontainers in the tests and could not assign a static port. In order to be able to configure the database port on Liquibase, I overrode a bean name liquibase of the type SpringLiquibase and created a DataSource (not exposed as a bean) in the liquibase bean creation method and set it on the liquibase bean.
It's possible to have spring-boot-starter-jdbc and spring-boot-starter-data-r2dbc co-exist. There is a class org.springframework.transaction.annotation.TransactionManagementConfigurer that can be used to resolve the conflict.
Spring Boot 2.3.0 seems to disable automatic datasource config when r2dbc is present. It's possible to manually import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration class to make both co-exist.
#Bean
TransactionManagementConfigurer transactionManagementConfigurer(ReactiveTransactionManager reactiveTransactionManager) {
return new TransactionManagementConfigurer() {
#Override
public TransactionManager annotationDrivenTransactionManager() {
return reactiveTransactionManager;
}
};
}

How to disable SpringBoot autoconfiguration for TomcatServletWebServerFactory in order for a custom spring-starter to provide it?

so I was writing my own SpringBootStarter which was supposed to enable the JNDI lookup in the embedded tomcat of a SpringBoot application.
My sample SpringBoot application has a dependency of my custom SpringBootStarter, which in turn has a dependency on the spring-boot-starter-web. If I create a Configuration class like the following inside the sample SpringBoot application everything works perfectly:
#Configuration
public class SampleSpringBootAppConfig {
#Bean
public TomcatServletWebServerFactory tomcatFactory() {
return new TomcatServletWebServerFactory() {
#Override
protected TomcatWebServer getTomcatWebServer(org.apache.catalina.startup.Tomcat tomcat) {
System.out.println("CONFIGURING CUSTOM TOMCAT WEB SERVER FACTORY ");
tomcat.enableNaming();
return super.getTomcatWebServer(tomcat);
}
#Override
protected void postProcessContext(Context context) {
ContextResource resource = new ContextResource();
resource.setName("myDataSource");
resource.setType(DataSource.class.getName());
resource.setProperty("driverClassName", "org.postgresql.Driver");
resource.setProperty("url", "jdbc:postgresql://localhost:5432/postgres");
resource.setProperty("username", "postgres");
resource.setProperty("password", "postgres");
context.getNamingResources()
.addResource(resource);
}
};
}
Because SpringBoot finds a custom Bean, there won't be an autoconfigured default one / it is overridden and the JNDI is successfully enabled.
However, as soon as I extract this Bean configuration into my auto-configure module of my custom SpringBoot Starter, the following exception is thrown while trying to start the sample SpringBoot application:
org.springframework.context.ApplicationContextException: Unable to start web server; nested exception is org.springframework.context.ApplicationContextException: Unable to start ServletWebServerApplicationContext due to multiple ServletWebServerFactory beans : tomcatServletWebServerFactory,tomcatFactory
I reckon this is due to SpringBoot not finding a customized Bean and therefore creating an autoconfigured default one which also won't be overridden. So now there will be two ServletWebServerFactory beans, the default one and the one from my auto- configure module.
What i tried so far (to no avail) :
annotating my custom Bean with #Primary
setting spring.main.allow-bean-definition-overriding to true
Is there any way to make SpringBoot not initialize a autoconfigured default bean, or any other possible solution to this?
try this
#AutoConfigureBefore(ServletWebServerFactoryAutoConfiguration.class)
I was able to solve this myself by excluding the responsible AutoConfiguration class:
#SpringBootApplication ( exclude = ServletWebServerFactoryAutoConfiguration.class)

Spring Websocket TaskExecutor-beans displaces "applicationTaskExecutor"-bean

I have a Spring Boot 2.1.8 Application that uses #Async-Tasks. All #Async-Tasks used to be executed by an automatically configured ThreadPoolTaskExecutor-bean named applicationTaskExecutor.
What did I change?
With spring-boot-starter-websocket in the class path and a #EnableWebSocketMessageBroker configuration the applicationTaskExecutor-bean is gone and replaced by four beans with the names
clientInboundChannelExecutor,
clientOutboundChannelExecutor,
brokerChannelExecutor,
and messageBrokerTaskScheduler.
Spring logs to the console: AnnotationAsyncExecutionInterceptor : More than one TaskExecutor bean found within the context, and none is named 'taskExecutor'. Mark one of them as primary or name it 'taskExecutor' (possibly as an alias) in order to use it for async processing: [clientInboundChannelExecutor, clientOutboundChannelExecutor, brokerChannelExecutor, messageBrokerTaskScheduler]
#Async-tasks are now executed by SimpleAsyncTaskExecutor.
Question
Why can't all beans co-exist? Why won't Spring create a applicationTaskExecutor-bean when spring-websockets is configured?
As #M. Deinum mentioned in the comments a #ConditionalOnMissingBean in TaskExecutionAutoConfiguration.java [0] results in the described behavior
I solved it by creating the bean by myself.
#ConditionalOnClass(ThreadPoolTaskExecutor.class)
#Configuration
public class ApplicationTaskExecutorBeanConfig {
#Lazy
#Bean(name = {APPLICATION_TASK_EXECUTOR_BEAN_NAME, DEFAULT_TASK_EXECUTOR_BEAN_NAME})
public ThreadPoolTaskExecutor applicationTaskExecutor(TaskExecutorBuilder builder) {
return builder.build();
}
}
First I tried to prefer the bean from TaskExecutionAutoConfiguration.java by annotating my bean factory method with
#ConditionalOnMissingBean(name = {APPLICATION_TASK_EXECUTOR_BEAN_NAME, DEFAULT_TASK_EXECUTOR_BEAN_NAME})
but it didn't worked because my bean factory method gets invoked earlier than the one in TaskExecutionAutoConfiguration.java.
[0] https://github.com/spring-projects/spring-boot/blob/master/spring-boot-project/spring-boot-autoconfigure/src/main/java/org/springframework/boot/autoconfigure/task/TaskExecutionAutoConfiguration.java#L78
Thanks to #M. Deinum for your comment.

Determine Springs bean instantiation order in Grails 3.3 with GORM 6.1.x

I am using a Grails 3.3 application that uses GORM 6.1.6.RELEASE, Spring-Boot 1.5.4.RELEASE and Spring Core 4.3.9.RELEASE behind the scene. I am trying to declare a Spring bean that get initialized just before Hibernate starts to validate the underlying database schema.
Here is what I like to do. I want to register my Flyway as a Spring bean and inject the dataSource bean into it. In order to have Flyway run before Hibernate starts to validate the current database schema, I add my flyway bean as a dependency onto the sessionFactory bean. The order would be as follows:
dataSource bean
flyway bean
hiberateDatastore bean
GORM 6.1 uses org.grails.orm.hibernate.HibernateDatastore as a Spring bean to initialize the Hibernate ORM and the database. The sessionFactory bean declares the hibernateDatastore#getSessionFactory as factory class.
Therefore the hibernateDatastore always is created first.
What is the way in Grails 3.3 to create a custom Spring bean that has to run after the connection to the database is available but before the Hibernate stuff gets initialized?
In previous versions of Grails 3.x it was possible to declare it in resources.groovy like this.
beans = {
if (Environment.current == Environment.PRODUCTION) {
flyway(Flyway) { bean ->
bean.initMethod = 'migrate'
dataSource = ref('dataSource')
locations = 'classpath:db/h2'
baselineOnMigrate = true
}
BeanDefinition sessionFactoryBeanDef = getBeanDefinition('hibernateDatastore')
if (sessionFactoryBeanDef) {
def dependsOnList = ['flyway'] as Set
if (sessionFactoryBeanDef.dependsOn?.length > 0) {
dependsOnList.addAll(sessionFactoryBeanDef.dependsOn)
}
sessionFactoryBeanDef.dependsOn = dependsOnList as String[]
}
}
}
I don't think Spring provides a visualisation of a 'bean instantion tree' however you could set the log level for org.springframework.beans.factory.support.DefaultListableBeanFactory to DEBUG and you'll get output like this:
Creating shared instance of singleton bean '...fully qualified class name...'
Returning cached instance of singleton bean '...fully qualified class name...'
You could review this log output for beans from the Hibernate namespace.
I presume you'll use the results to declare a DependsOn relationship so just for completeness this would look like:
#Bean
public SomeHibernateClass createHibernate() {
...
}
#Bean
#DependsOn("createHibernate")
public MyClass createMine() {
...
}
Grails 3.3.0 changed the mechanism of the dataSource bean creation. The Grails project lead stated in the related issue:
Previous versions of Grails created the dataSource bean separately from the session factory. We would need to restore this behaviour I guess so the dataSource can be referenced without triggering the creation of the sessionFactory
After upgrading to Grails 3.3.1 the dataSource bean is again available before the session factory gets created. This solves the problem.

Wiring Activiti as a Spring Bean in Grails

I'm trying to create a Grails application that uses Activiti for its process engine. To that end, I'd like the main Activiti service classes (RuntimeService, TaskService, etc.) to be wired as Spring beans.
I believe that I have the wiring setup correctly, but when I run a simple integration test that calls the runtime service, I get an error that Spring couldn't open a hibernate session (see Full Stack Trace, below).
Update: I can start the application with run-app, then call a controller action which calls my service, and it all works. So, the Activiti wiring works, it just has some conflict with the Grails Integration Testing mixin.
I really want the Activiti services to use the same datasource connection as the Grails application. I'm assuming that the problem is that Activiti is trying to create its own ConnectionHolder instance, when Grails already has one setup for the integration tests.
My specific (and possibly misguided) question I have is, how do I configure my Activiti ProcessEngine so that it uses the same data source and hibernate connection that my Grails application does?
The more general question is, how can I best make the Activiti services available to my Grails application? I've looked at the Activiti plugin for grails, and the source of it has helped me get this far. However, I'd rather not use that plugin; it's not using the latest activiti, development on it is not terribly active, and it's not really what I need in any case.
Full Stack Trace
| Failure: start approver request(com.package.MyServiceSpec)
| org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is java.lang.IllegalStateException: Already value [org.springframework.jdbc.datasource.ConnectionHolder#7f2e1821] for key [org.springframework.jdbc.datasource.LazyConnectionDa
at grails.test.mixin.integration.IntegrationTestMixin.initIntegrationTest(IntegrationTestMixin.groovy:58)
at org.spockframework.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:138)
at org.spockframework.runtime.extension.builtin.JUnitFixtureMethodsExtension$FixtureType$FixtureMethodInterceptor.intercept(JUnitFixtureMethodsExtension.java:145)
at org.spockframework.runtime.extension.MethodInvocation.proceed(MethodInvocation.java:84)
at org.spockframework.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:138)
at org.spockframework.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:138)
at org.spockframework.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:138)
Caused by: org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is java.lang.IllegalStateException: Already value [org.springframework.jdbc.datasource.ConnectionHolder#7f2e1821] for key [org.springframework.jdbc.datasource.LazyConn
... 7 more
Caused by: java.lang.IllegalStateException: Already value [org.springframework.jdbc.datasource.ConnectionHolder#7f2e1821] for key [org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy#1b7aeb] bound to thread [main]
... 7 more
| Completed 1 integration test, 1 failed in 0m 1s
resources.groovy
import org.activiti.engine.ProcessEngine
import org.activiti.spring.ProcessEngineFactoryBean
import org.activiti.explorer.form.*
import org.activiti.spring.SpringProcessEngineConfiguration
import grails.util.Environment
//These imports are only needed in the test environment for building an h2 database for activiti during unit tests
import org.springframework.jdbc.datasource.DataSourceTransactionManager
import org.springframework.jdbc.datasource.SimpleDriverDataSource
beans = {
processEngineConfig(SpringProcessEngineConfiguration) {
dataSource = ref('dataSource')
transactionManager = ref('transactionManager')
databaseType = 'oracle'
databaseSchema = 'OURSCHEMA'
databaseSchemaUpdate = false
jobExecutorActivate = true
}
processEngine(ProcessEngineFactoryBean) {
processEngineConfiguration = ref("processEngineConfig")
}
runtimeService(processEngine: "getRuntimeService")
repositoryService(processEngine: "getRepositoryService")
taskService(processEngine: "getTaskService")
managementService(processEngine: "getManagementService")
historyService(processEngine: "getHistoryService")
formService(processEngine: "getFormService")
}
Service class
class MyService {
def foapAuthFoapService
def processEngine
def runtimeService
def repositoryService
def taskService
def managementService
def historyService
def formService
/**
* Start the activiti process.
*
*/
def startRequest(String requester, String subject, String designatedApprover) {
runtimeService.startProcessInstanceByKey('MyProcess', ["requester": requester, "subject": subject, "designatedApprover": designatedApprover])
}
}
Spock Test
def "start request"() {
setup:
def approverRequest = service.startRequest(requester, subject, designatedApprover)
def variables = runtimeService.getVariables(approverRequest.id) //approverRequest.getProcessVariables()
expect:
approverRequest instanceof ProcessInstance
variables.entrySet().contailsAll(["designatedApprover": designatedApprover, "requester": requester, "subject": subject].entrySet())
where:
requester | subject | designatedApprover
"abc123" | "def456"| "hij789"
}
we at Alephsa are maintaining the grails activiti plugin updated to the latest version of Activiti (5.17) and we are working to update to version 6. You can find the plugin at bintray also with the grails activiti spring security plugin. In our github account you can find the source code to guide you in your efforts.
Regards.

Resources