We have a spring boot microservice using spring-kafka. There are a couple of Kafka listeners and producers and the Kafka related metrics are successfully presented at the actuator endpoint. But when adding another maven dependency with custom metrics, the Kafka related metrics provided by spring boot are removed. I suspect it's related to introducing new MeterRegistry bean, but overriding it with the same bean definition as in NoOpMeterRegistryConfiguration in spring-boot-actuator-autoconfigure module of spring-boot didn't help. I've also tried debugging KafkaMetricsAutoConfiguration, and it looks like all the beans are successfully instantiated. Can somebody with more experience with spring-boot-actuator-autoconfigure help me to understand the default configurations for Kafka related metrics, please?
The spring configuration in the new maven dependency is pretty simple, just ProducerFactory and MeterRegistry beans are defined
#Bean
#Primary
public ProducerFactory<Object, Object> producerFactory(MessageCounter messageCounter) {
Map<String, Object> configs = this.kafkaProperties.buildProducerProperties();
configs.put("interceptor.classes", Collections.singletonList(RecordSuccessfullySentInterceptor.class));
configs.put("message.counter.bean", messageCounter);
DefaultKafkaProducerFactory<Object, Object> producerFactory = new DefaultKafkaProducerFactory(configs);
producerFactory.setTransactionIdPrefix(this.transactionId);
return producerFactory;
}
#Bean
public MeterRegistry meterRegistry() {
return new PrometheusMeterRegistry(PrometheusConfig.DEFAULT);
}
#Bean({"retryProducerFactory"})
public ProducerFactory<Object, Object> retryProducerFactory(MessageCounter messageCounter) {
Map<String, Object> configs = this.kafkaProperties.buildProducerProperties();
configs.put("interceptor.classes", Collections.singletonList(RecordSuccessfullySentInterceptor.class));
configs.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
configs.put("message.counter.bean", messageCounter);
DefaultKafkaProducerFactory<Object, Object> producerFactory = new DefaultKafkaProducerFactory(configs);
producerFactory.setTransactionIdPrefix(this.transactionId);
return producerFactory;
}
It may be possible some transient dependency is messing up spring boot autoconfiguration for Kafka related metrics though.
Related
How do I create and use the Redis connection that spring-boot-starter-data-redis creates? It doesn't seem like there is a Bean for RedisClient created by the default auto configuration so I'm not sure of the best way to do this.
The documentation does state that in this case you need to create the StorageProvider yourself which is fine, but can you reuse what Spring Boot has already created. I believe this would need to be a pooled connection which you would also need to enable through Spring Boot.
RedisTemplate offers a high-level abstraction for Redis interactions:
https://docs.spring.io/spring-data/data-redis/docs/current/reference/html/#redis:template
Redis autoconfiguration :
#AutoConfiguration
#ConditionalOnClass({RedisOperations.class})
#EnableConfigurationProperties({RedisProperties.class})
#Import({LettuceConnectionConfiguration.class, JedisConnectionConfiguration.class})
public class RedisAutoConfiguration {
public RedisAutoConfiguration() {
}
#Bean
#ConditionalOnMissingBean(
name = {"redisTemplate"}
)
#ConditionalOnSingleCandidate(RedisConnectionFactory.class)
public RedisTemplate<Object, Object> redisTemplate(RedisConnectionFactory redisConnectionFactory) {
RedisTemplate<Object, Object> template = new RedisTemplate();
template.setConnectionFactory(redisConnectionFactory);
return template;
}
#Bean
#ConditionalOnMissingBean
#ConditionalOnSingleCandidate(RedisConnectionFactory.class)
public StringRedisTemplate stringRedisTemplate(RedisConnectionFactory redisConnectionFactory) {
return new StringRedisTemplate(redisConnectionFactory);
}
}
Here you can find the corresponding configuration properties(including connection pool default configuration).
Simple implementation example :
https://www.baeldung.com/spring-data-redis-tutorial
I want to use Lettuce as a Redis Client, which is the default dependency for spring-boot-starter-data-redis-reactive. However I am inheriting Jedis as a dependency from another component written as pure Java code (no Spring). This is resulting in a conflict when initializing LettuceConnectionFactory due to presence of JedisConnectionFactory.
How can I keep Jedis in dependency for other component's use while ensuring that LettuceConnectionFactory is initialised for my own code? The main reason for using LetticeConnectionFactory is reactive programming in my service.
Both the connection factories are configured for initialization via RedisAutoConfiguration with no option of prioritisation.
https://github.com/spring-projects/spring-boot/blob/master/spring-boot-project/spring-boot-autoconfigure/src/main/java/org/springframework/boot/autoconfigure/data/redis/RedisAutoConfiguration.java
You could override RedisConnectionFactory by creating your own #Configuration class and adding new #Bean in it.
For example:
#Bean
RedisConnectionFactory myLettuceConnectionFactory() {
// your setup....
new LettuceConnectionFactory();
}
and then use myLettuceConnectionFactory bean to setup RedisTemplate #Bean
#Bean
public RedisTemplate<String, Object> redisTemplate() {
final RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(myLettuceConnectionFactory());
// other settings...
return template;
}
I have a activemq:queue inQueue in my JbossFuse. How do I consume those JMS messages which are enqueued so that my process instance is triggered in the Spring boot application integrated with Camunda ? Any link to references or samples would be helpful ?
Currently I am able to consume messages from activemq but I am not sure how to consume the messages from Jboss Fuse ActiveMQ ?
#Component
public class ActiveMQConsumer {
#Autowired
CamelContext camelContext;
#Autowired
ProducerTemplate producerTemplate;
#SuppressWarnings("unchecked")
#JmsListener(destination = "inQueue")
public void consumeMessage(JSONObject employeeRecord) throws Exception {
if (employeeRecord instanceof JSONObject) {
HashMap<String, Object> employeeRecordMap = (HashMap<String, Object>) employeeRecord.toMap();
Exchange exchange = ExchangeBuilder.anExchange(camelContext).withBody(employeeRecordMap).build();
HashMap<String, Object> employeeDetails = (HashMap<String, Object>) employeeRecordMap.get("employeeDetails");
exchange.setProperty("CamundaBpmBusinessKey", employeeDetails.get("employeeADId"));
producerTemplate.send("camunda-bpm:start?processDefinitionKey=camunda-camel-activeMQ", exchange);
}
}
}
application.properties
# activeMQ config
spring.activemq.broker-url=tcp://localhost:61616
spring.activemq.user=admin
spring.activemq.password=admin
Expected to consume messages from JbossFuse.
I would recommend using the maven archetype io.fabric8.archetypes spring-boot-camel-amq-archetype version 2.2.197. This can be found:
Spring Boot example running a Camel route connecting to ActiveMQ
http://repo1.maven.org/maven2/
This will get you a nice sample project that has all of the Camel and Spring dependencies and some nice samples.
I have successfully integrated kafka outbound channle adapter with fixed topic name. Now, i want to make the topic name configurable and hence, want to expose it via application properties.
application.properties contain one of the following entry:
kafkaTopic:testNewTopic
My configuration class looks like below:
#Configuration
#Component
public class KafkaConfig {
#Value("${kafkaTopic}")
private String kafkaTopicName;
#Bean
public String getTopic(){
return kafkaTopicName;
}
#Bean
public KafkaTemplate<String, String> kafkaTemplate() {
return new KafkaTemplate<>(producerFactory());
}
#Bean
public ProducerFactory<String, String> producerFactory() {
Map<String, Object> props = new HashMap<>();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");//this.brokerAddress);
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer");
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer");
// set more properties
return new DefaultKafkaProducerFactory<>(props);
}
}
and in my si-config.xml, i have used the following (ex: topic="getTopic") :
<int-kafka:outbound-channel-adapter
id="kafkaOutboundChannelAdapter" kafka-template="kafkaTemplate"
auto-startup="true" sync="true" channel="inputToKafka" topic="getTopic">
</int-kafka:outbound-channel-adapter>
However, the configuration is unable to pick up the topic name when exposed via bean. But it works fine when i hard code the value of the topic name.
Can someone please suggest what i am doing wrong here?
Does topic within kafka outbound channel accept the value referred as bean?
How do i externalize it as every application using my utility will supply different kafka topic names
The topic attribute is for string value.
However it supports property placeholder resolution:
topic="${kafkaTopic}"
and also SpEL evaluation for aforementioned bean:
topic="#{getTopic}"
Just because this is allowed by the XML parser configuration.
However you may pay attention that KafkaTemplate, which you inject into the <int-kafka:outbound-channel-adapter> has defaultTopic property. Therefore you won't need to worry about that XML.
And one more option available for you is Spring Integration Annotations configuration. Where you can define a #ServiceActivator for the KafkaProducerMessageHandler #Bean:
#ServiceActivator(inputChannel = "inputToKafka")
#Bean
KafkaProducerMessageHandler kafkaOutboundChannelAdapter() {
kafkaOutboundChannelAdapter adapter = new kafkaOutboundChannelAdapter( kafkaTemplate());
adapter.setSync(true);
adapter.setTopicExpression(new LiteralExpression(this.kafkaTopicName));
return adapter;
}
I am writing a Spring Boot application and configured Eclipselink as a JPA provider. I would like to disable Eclipselink external and internal connection pools and use tomcat to provide connection to Eclipselink.
Could you suggest is this approach is good to go? If I have to give pooling to Eclipselink control, I wouldn't be able to use spring orm and jpa features. Several things happens behind the scenes such as EntityManagerFactory and EntityManager I have to manage in application.
My questions are:
Which one is better/smooth approach with Spring Boot + Eclipselink for connection pooling.
How to disable/enable eclipselink internal or external connection pools. Looking at the documentation I couldn't get clear idea.
Thank you for your help!
Below is my eclipselink configuration:
public class EclipseLinkJpaConfig extends JpaBaseConfiguration {
#Override
protected AbstractJpaVendorAdapter createJpaVendorAdapter() {
final EclipseLinkJpaVendorAdapter adapter = new EclipseLinkJpaVendorAdapter();
return adapter;
}
#Override
protected Map<String, Object> getVendorProperties() {
final HashMap<String, Object> map = new HashMap<String, Object>();
map.put(PersistenceUnitProperties.WEAVING, "static");
map.put(PersistenceUnitProperties.DEPLOY_ON_STARTUP, "true");
map.put(PersistenceUnitProperties.TARGET_SERVER, "config.ConnectionPlatform");
return map;
}
}
Data source config in application.properties below:
spring.datasource.initial-size=5
spring.datasource.max-wait=30000
spring.datasource.max-active=10
spring.datasource.max-idle=10
spring.datasource.min-idle=5
spring.datasource.test-on-borrow=true