I am trying to create a spring boot application using spring cloud kafka stream which reads input from kafka cluster 1 and send it to kafka cluster 2 using single kafkastream application.
I am getting following exception during startup.
***************************
APPLICATION FAILED TO START
***************************
Description:
A component required a bean named 'sessionBrokers-KafkaStreamsBinderConfigurationProperties' that could not be found.
Action:
Consider defining a bean named 'sessionBrokers-KafkaStreamsBinderConfigurationProperties' in your configuration.
My configuration is as below
spring.cloud.function.definition=itemConsumedStream
#Input Configuration
spring.cloud.stream.bindings.itemConsumedStream-in-0.binder=sessionBrokers
spring.cloud.stream.bindings.itemConsumedStream-in-0.destination=${session.event.data.input.topic}
spring.cloud.stream.kafka.bindings.itemConsumedStream-in-0.consumer.configuration.key.deserializer=org.apache.kafka.common.serialization.StringSerializer
spring.cloud.stream.kafka.bindings.itemConsumedStream-in-0.consumer.configuration.value.deserializer=in.custom.JsonNodeDeserializer
#Output Configuration
spring.cloud.stream.bindings.itemConsumedStream-out-0.binder=searchBrokers
spring.cloud.stream.bindings.itemConsumedStream-out-0.destination=${itemConsumed.events.output.topic}
spring.cloud.stream.kafka.bindings.itemConsumedStream-out-0.producer.configuration.key.serializer=org.apache.kafka.common.serialization.StringSerializer
spring.cloud.stream.kafka.bindings.itemConsumedStream-out-0.producer.configuration.value.serializer=in.custom.JsonSerializer
#Input Binder Configurations
spring.cloud.stream.binders.sessionBrokers.type=kafka
spring.cloud.stream.binders.sessionBrokers.environment.spring.cloud.stream.kafka.streams.binder.brokers=${session.event.data.input.kafka.brokers}
#Output Binder Configurations
spring.cloud.stream.binders.searchBrokers.type=kafka
spring.cloud.stream.binders.searchBrokers.environment.spring.cloud.stream.kafka.binder.brokers=${search.kafka.brokers}
pom dependencies are
spring boot version
2.7.1
spring cloud version
2021.0.0
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-streams</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-kafka-streams</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-kafka</artifactId>
</dependency>
This is working if I use single binder ie
spring.cloud.stream.kafka.streams.binder.brokers=${session.event.data.input.kafka.brokers}
instead of multiple binders
Related
I trying to add Kubernetes to my project to use Service name rather than localhost. I added the below dependencies :
#AutoConfigureAfter
#EnableEurekaClient
#EnableFeignClients
#EnableDiscoveryClient
<!-- https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-kubernetes-client-all -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes-client-all</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-kubernetes-config</artifactId>
<version>1.1.10.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes</artifactId>
<version>1.1.10.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-kubernetes-discovery</artifactId>
<version>2.1.3</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-kubernetes-core</artifactId>
<version>1.1.10.RELEASE</version>
</dependency>
<!-- MicroServices -->
But I got the below error :
***************************
APPLICATION FAILED TO START
***************************
Description:
The bean 'kubernetesPodUtils', defined in class path resource [org/springframework/cloud/kubernetes/KubernetesAutoConfiguration.class], could not be registered. A bean with that name has already been defined in class path resource [org/springframework/cloud/kubernetes/client/KubernetesClientAutoConfiguration.class] and overriding is disabled.
Action:
Consider renaming one of the beans or enabling overriding by setting spring.main.allow-bean-definition-overriding=true
2022-06-14 12:28:08.696 ERROR 1 --- [ main] o.s.b.d.LoggingFailureAnalysisReporter :
***************************
APPLICATION FAILED TO START
***************************
Description:
The bean 'kubernetesPodUtils', defined in class path resource [org/springframework/cloud/kubernetes/KubernetesAutoConfiguration.class], could not be registered. A bean with that name has already been defined in class path resource [org/springframework/cloud/kubernetes/client/KubernetesClientAutoConfiguration.class] and overriding is disabled.
Action:
Consider renaming one of the beans or enabling overriding by setting spring.main.allow-bean-definition-overriding=true
Looks like KubernetesAutoConfiguration is going to be defined in multiple dependencies. Based on your need, try to identify the dependencies which are required. In your pom file you have spring-cloud-starter-kubernetes-client-all alongwith individual dependecies too.
https://docs.spring.io/spring-cloud-kubernetes/docs/current/reference/html/#starters
I am writing service based on Spring webflux, which reads data from PostgreSQL using r2dbc. I need to use latest release of r2dbc, however I am getting NoSuchMethodError exception in using TransactionSynchronizationManager spring-tx 5.2.0.RELEASE library.
I basically need to know what is correct spring-tx library version to be compatible with version of spring-data-r2dbc which works correctly with latest r2dbc-postgresql and r2dbc-spi libraries.
Here are my Maven dependencies.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-r2dbc</artifactId>
<version>1.0.0.M2</version>
</dependency>
<dependency>
<groupId>io.r2dbc</groupId>
<artifactId>r2dbc-postgresql</artifactId>
<version>0.8.0.RC2</version>
</dependency>
<dependency>
<groupId>io.r2dbc</groupId>
<artifactId>r2dbc-spi</artifactId>
<version>0.8.0.M8</version>
</dependency>
I am using interface extending ReactiveCrudRepository interface to retrieve table data per below.
#Query("...")
Flux<QuoteHistory> findAllBySecIdAndDateTimeBetweenAndUpdateTypeIn(LocalDate date, Long secId);
I was able to get this code to work with earlier versions of r2dbc-postgresql and r2dbc-spi but now I am getting following exception.
java.lang.NoSuchMethodError: org.springframework.transaction.reactive.TransactionSynchronizationManager.currentTransaction()Lreactor/core/publisher/Mono;
at org.springframework.data.r2dbc.connectionfactory.ConnectionFactoryUtils.doGetConnection(ConnectionFactoryUtils.java:88) ~[spring-data-r2dbc-1.0.0.M2.jar:1.0.0.M2]
at org.springframework.data.r2dbc.connectionfactory.ConnectionFactoryUtils.getConnection(ConnectionFactoryUtils.java:70) ~[spring-data-r2dbc-1.0.0.M2.jar:1.0.0.M2]
at org.springframework.data.r2dbc.core.DefaultDatabaseClient.getConnection(DefaultDatabaseClient.java:189) ~[spring-data-r2dbc-1.0.0.M2.jar:1.0.0.M2]
These are r2dbc dependencies which code works with.
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-r2dbc</artifactId>
<version>1.0.0.M1</version>
</dependency>
<dependency>
<groupId>io.r2dbc</groupId>
<artifactId>r2dbc-postgresql</artifactId>
<version>1.0.0.M7</version>
</dependency>
Please use the following dependency combination:
R2DBC Postgres: 0.8.0.RC2
R2DBC SPI: 0.8.0.RC2
Spring Data R2DBC: 1.0.0.RC1
Spring Framework: 5.2.0.RELEASE
Alternatively, go to https://start.spring.io to get a dependency-managed project with versions that work together.
I'm getting following error while running my Spring boot app, I'm new to Spring boot and elastic search, please help to solve this issue.And attached my pom dependencies below.
Thanks in advance,
*************************** APPLICATION FAILED TO START ***************************
Description:
An attempt was made to call the method org.elasticsearch.common.logging.Loggers.getLogger(Ljava/lang/String;)Lorg/apache/logging/log4j/Logger; but it does not exist. Its class, org.elasticsearch.common.logging.Loggers, is available from the following locations:
jar:file:/C:/Users/Sudhakar/.m2/repository/org/elasticsearch/elasticsearch/6.6.2/elasticsearch-6.6.2.jar!/org/elasticsearch/common/logging/Loggers.class
It was loaded from the following location:
file:/C:/Users/Sudhakar/.m2/repository/org/elasticsearch/elasticsearch/6.6.2/elasticsearch-6.6.2.jar
Action:
Correct the classpath of your application so that it contains a single, compatible version of org.elasticsearch.common.logging.Loggers
Process finished with exit code 1
Maven dependencies:
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-to-slf4j</artifactId>
<version>2.11.1</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.24</version>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>6.6.2</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>transport</artifactId>
<version>6.6.2</version>
</dependency>
Spring Boot uses Elasticsearch 6.4 by default. By using 6.6.2 as the versions for two Elasticsearch modules you will have ended up with a mixture of the two versions. You should remove the <version> configuration in your pom. If you are able to use Spring Boot’s default version there’s nothing more to do. If you need to use 6.6.2 you should add an entry in your pom’s <properties>:
<elasticsearch.version>6.6.2</elasticsearch.version>
I want to integration test a custom camel component and therefor need an embedded/in memory messanging I can easily use to test from/to endpoints.
I am hoping that I can achieve this via spring-boot-amqp-starter.
I used this example for a start, which has the dependencies:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-activemq</artifactId>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-broker</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-spring-boot-starter</artifactId>
</dependency>
and the config:
spring:
activemq:
broker-url: vm://embedded?broker.persistent=false,useShutdownHook=false
// ...
This is working, when I use regular Listener annotations in spring, I have a sender using the template and a consumer logging the messages.
Now I go one step further and use camel, but it does not recognize the vm:embedded broker but tries to connect to tcp://localhost, which is not running.
return new RouteBuilder() {
#Override
public void configure() throws Exception {
from("activemq:foo").to("log:sample");
from("timer:bar").setBody(constant("Hello from Camel")).to("activemq:foo");
}
};
How can I configure activemq-camel to use the embedded broker?
Update:
I use dependency management imports for spring-boot-dependencies (1.5.9)
and camel-spring-boot-dependencies (2.20.1).
This has been fixed in newer versions of activemq-camel when you use that with Spring Boot. Now the activemq-camel component will honor the spring boot configuration of the spring.activemq.* settings.
I am writing a spring boot app that accesses stuff from an s3 bucket, but I get a NoClassDefFoundError when I use the starter spring-cloud-starter-aws dependency from the spring initializer.
Am I missing some other dependency here?
Below are my dependencies.
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-aws</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-cassandra</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-integration</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
I also defined the dependencyManagement block for spring-cloud-dependencies and use Edgware.SR1 as my spring-cloud-version.
My app fails with the following error when starting up.
2018-01-24 12:20:25.642 INFO 1980 --- [ main] utoConfigurationReportLoggingInitializer :
Error starting ApplicationContext. To display the auto-configuration report re-run your application with 'debug' enabled.
2018-01-24 12:20:25.666 ERROR 1980 --- [ main] o.s.boot.SpringApplication : Application startup failed
org.springframework.beans.factory.BeanDefinitionStoreException: Failed to process import candidates for configuration class [com.art.productattribution.consumerintegration.ConsumerIntegrationApplication]; nested exception is java.lang.NoClassDefFoundError: com/amazonaws/AmazonClientException
at org.springframework.context.annotation.ConfigurationClassParser.processImports(ConfigurationClassParser.java:616) ~[spring-context-4.3.13.RELEASE.jar:4.3.13.RELEASE]
Not sure what am I missing here? Please let me know if you need any more details with this. The version of spring boot I am using is 1.5.9.RELEASE
The correct dependency is the spring-cloud-aws-context. Add the following in your pom file (version 1.2.2 as of Nov 22, 2017):
<!-- https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-aws-context -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-aws-context</artifactId>
<version>1.2.2.RELEASE</version>
</dependency>
Below are the Spring Cloud AWS modules:
Spring Cloud AWS Core is the core module of Spring Cloud AWS providing basic services for security and configuration setup. Developers will not use this module directly but rather through other modules. The core module provides support for cloud based environment configurations providing direct access to the instance based EC2 metadata and the overall application stack specific CloudFormation metadata.
Spring Cloud AWS Context delivers access to the Simple Storage Service via the Spring resource loader abstraction. Moreover developers can send e-mails using the Simple E-Mail Service and the Spring mail abstraction. Further the developers can introduce declarative caching using the Spring caching support and the ElastiCache caching service.
Spring Cloud AWS JDBC provides automatic datasource lookup and configuration for the Relational Database Service which can be used with JDBC or any other support data access technology by Spring.
Spring Cloud AWS Messaging enables developers to receive and send messages with the Simple Queueing Service for point-to-point communication. Publish-subscribe messaging is supported with the integration of the Simple Notification Service.
Ref: http://cloud.spring.io/spring-cloud-aws/spring-cloud-aws.html#_using_amazon_web_services
It seems com.amazonaws.AmazonClientException is not found in the classpath. I think you can add the following dependency in your POM.xml file to solve this issue.
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-aws-core</artifactId>
<version>1.2.2.RELEASE</version>
</dependency>
You need to include dependency mentioned by #alltej. Also you need to add below property in application.properties file if you are running on local.
cloud.aws.stack.auto=false