We develop a spring-boot service, which offers a rest api (spring-webflux) and sends data via RabbitMQ (spring-rabbit). The service is deployed on cloud foundry and we use spring-boot in version 2.1.4. We added spring-boot-starter-data-redis to use redis to cache some data and we got the following error:
[io.netty.util.ResourceLeakDetector] [] LEAK: HashedWheelTimer.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records:
Created at:
io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:284)
io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:217)
io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:196)
io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:178)
io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:162)
io.lettuce.core.resource.DefaultClientResources.<init>(DefaultClientResources.java:169)
io.lettuce.core.resource.DefaultClientResources$Builder.build(DefaultClientResources.java:532)
io.lettuce.core.resource.DefaultClientResources.create(DefaultClientResources.java:233)
io.lettuce.core.AbstractRedisClient.<init>(AbstractRedisClient.java:98)
io.lettuce.core.RedisClient.<init>(RedisClient.java:87)
io.lettuce.core.RedisClient.create(RedisClient.java:124)
org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.lambda$createClient$7(LettuceConnectionFactory.java:971)
java.base/java.util.Optional.orElseGet(Unknown Source)
org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.createClient(LettuceConnectionFactory.java:971)
org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.afterPropertiesSet(LettuceConnectionFactory.java:273)
org.springframework.cloud.service.keyval.RedisConnectionFactoryCreator.create(RedisConnectionFactoryCreator.java:88)
org.springframework.cloud.service.keyval.RedisConnectionFactoryCreator.create(RedisConnectionFactoryCreator.java:31)
org.springframework.cloud.Cloud.getServiceConnector(Cloud.java:288)
org.springframework.cloud.Cloud.getSingletonServiceConnector(Cloud.java:202)
org.springframework.cloud.config.java.CloudServiceConnectionFactory.redisConnectionFactory(CloudServiceConnectionFactory.java:260)
org.springframework.cloud.config.java.CloudServiceConnectionFactory.redisConnectionFactory(CloudServiceConnectionFactory.java:242)
...
This error only happens when we run the service on cloud foundry, if we run it locally, we don't get any error.
We don't do any configuration of the connection factory or the stringRedisTemplate on our side and only use stringRedisTemplate, which is configured by the spring-autoconfiguration.
We use following configuration for redis on cloud foundry:
#Configuration
#Profile( "cloud" )
public class CloudSpecificConfig extends AbstractCloudConfig {
#Bean
public RedisConnectionFactory redisConnectionFactory() {
return connectionFactory().redisConnectionFactory();
}
}
And this is how we use the template
#Component
#RequiredArgsConstructor
public final class RequestUtil {
private final StringRedisTemplate myRedisTemplate;
public String cacheId(String id, String value) {
myRedisTemplate.opsForValue().set( id, value );
}
}
These are our spring dependencies:
<properties>
<spring-boot-version>2.1.4.RELEASE</spring-boot-version>
</properties>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
<version>${spring-boot-version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<version>${spring-boot-version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
<version>${spring-boot-version}</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-sleuth</artifactId>
<version>${spring-boot-version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-cloud-connectors</artifactId>
<version>${spring-boot-version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
<version>${spring-boot-version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
<version>${spring-boot-version}</version>
</dependency>
We are quite confused on our side, since we didn't do any specific configuration on our side. It looks for us like there is something wrong with the spring configuration on the cloud. Are we doing something wrong? Do we need to configure something differently? Is this a bug?
This is what I had to do
I will see if I can find a more elegant way
#Bean(destroyMethod = "shutdown")
public DefaultClientResources lettuceClientResources() {
return DefaultClientResources.create();
}
#SuppressWarnings("unused")
#Bean
public RedisConnectionFactory redisConnectionFactory(DefaultClientResources dependency) {
return connectionFactory().redisConnectionFactory("redis-pcf-service");
}
In the end the issue disappeared on our side, because we changed the redis client from lettuce to jedis.
We had the problem with lettuce that we would lose the connection to our redis service on our cloud infrastructure. But since there was an update to the redis service at same time as we changed the client, we don't really know if it was related to lettuce.
Maybe there also just something wrong in the auto-configuration in conjunction with the redis service on our cloud instructure, which is based on cloudfoundry
Related
I'm trying to use Spring's state machine library inside of a JavaEE application running in a container to manage instances of a class named RunInfo. But I'm not sure how to integrate the two together.
I tried adding the following dependencies:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-autoconfigure</artifactId>
<version>2.2.11.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.statemachine</groupId>
<artifactId>spring-statemachine-core</artifactId>
<version>2.4.1</version>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-core</artifactId>
<version>5.7.2</version>
</dependency>
on spring-boot-autoconfigure and then adding the following configuration class:
#Configuration
#EnableAutoConfiguration
#ComponentScan(basePackages = {"com.ourdomain"})
public class StateConfiguration {}
I then wrote a simple state machine configuration:
#Configuration
#EnableStateMachine(name = "runInfoStateMachine")
public class RunInfoStateMachineConfiguration extends EnumStateMachineConfigurerAdapter<RunInfoState, RunInfoEvent> {
and state handler:
#Component
public class RunInfoStateHandler extends LifecycleObjectSupport {
// ...
#Autowired private StateMachine<RunInfoState, RunInfoEvent> stateMachine;
}
based on another project that I manage (which is a pure Spring Boot app).
The state handler is created, but the state machine reference is never injected; i.e., stateMachine in the above code is always null.
How does one integrate the Spring StateMachine framework into an EJB application?
I'm using spring boot 2 to create a project and use websocket using reactive web dependency. My application is worked correctly until I add datarest dependency. after I add datarest dependency application give
' failed: Error during WebSocket handshake: Unexpected response code: 404
is any way to resolve this conflict?.
pom.xml
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-integration</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<!-- https://mvnrepository.com/artifact/org.springframework.integration/spring-integration-file -->
<dependency>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-file</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-rest</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
WebSocketConfiguration
#Configuration
public class WebSocketConfiguration {
#Bean
public IntegrationFlow fileFlow(PublishSubscribeChannel channel, #Value("file://${HOME}/Desktop/in") File file) {
FileInboundChannelAdapterSpec in = Files.inboundAdapter(file).autoCreateDirectory(true);
return IntegrationFlows.from(
in,
p -> p.poller(pollerFactory -> {
return pollerFactory.fixedRate(1000);
})
).channel(channel).get();
}
#Bean
#Primary
public PublishSubscribeChannel incomingFilesChannel() {
return new PublishSubscribeChannel();
}
#Bean
public WebSocketHandlerAdapter webSocketHandlerAdapter() {
return new WebSocketHandlerAdapter();
}
#Bean
public WebSocketHandler webSocketHandler(PublishSubscribeChannel channel) {
return session -> {
Map<String, MessageHandler> connections = new ConcurrentHashMap<>();
Flux<WebSocketMessage> publisher = Flux.create((Consumer<FluxSink<WebSocketMessage>>) fluxSink -> {
connections.put(session.getId(), new ForwardingMessageHandler(session, fluxSink));
channel.subscribe(connections.get(session.getId()));
}).doFinally(signalType -> {
channel.unsubscribe(connections.get(session.getId()));
connections.remove(session.getId());
});
return session.send(publisher);
};
}
#Bean
public HandlerMapping handlerMapping(WebSocketHandler webSocketHandler) {
SimpleUrlHandlerMapping handlerMapping = new SimpleUrlHandlerMapping();
handlerMapping.setOrder(10);
handlerMapping.setUrlMap(Collections.singletonMap("/ws/files", webSocketHandler));
return handlerMapping;
}
}
spring-boot-starter-data-rest brings spring-boot-starter-web as a transitive dependency (so basically Spring MVC). This makes Spring Boot configure your application as a Spring MVC web application.
Spring Data REST does not currently support Spring WebFlux (see this issue for more information on that).
Your only choice is to remove the Spring Data REST dependency, as you can't have both Spring MVC and Spring WebFlux in the same Spring Boot application.
By default spring web flux uses netty which is single threaded event loop. How to configure spring boot so that a thread will be created for each core.
Thanks,
Lokesh
As described in the Spring Boot reference documentation, you can customize the Reactor Netty web server with a NettyServerCustomizer.
Here's an example with Spring Boot 2.1:
#Component
public class MyNettyWebServerCustomizer
implements WebServerFactoryCustomizer<NettyReactiveWebServerFactory> {
#Override
public void customize(NettyReactiveWebServerFactory factory) {
factory.addServerCustomizers(new EventLoopNettyCustomizer());
}
}
class EventLoopNettyCustomizer implements NettyServerCustomizer {
#Override
public HttpServer apply(HttpServer httpServer) {
LoopResources loopResources = LoopResources.create(...);
return httpServer.runOn(loopResources);
}
}
You can change your dependencies:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<!-- Exclude the Tomcat dependency -->
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- Use Jetty instead -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jetty</artifactId>
</dependency>
https://docs.spring.io/spring-boot/docs/current/reference/html/howto-embedded-web-servers.html
I'm using spring-boot with camel and ActiveMQ.
I'm using activemq autoconfiguration via #EnableJms annotation.
But creating my own ActiveMQComponent to enable "transacted(true)" on all queues.
#Bean(name = "activemq")
#ConditionalOnClass(ActiveMQComponent.class)
public ActiveMQComponent activeMQComponent(ConnectionFactory connectionFactory) {
ActiveMQComponent activeMQComponent = new ActiveMQComponent();
activeMQComponent.setConnectionFactory(connectionFactory);
activeMQComponent.setTransacted(true);
activeMQComponent.setTransactionManager(jmsTransactionManager(connectionFactory));
return activeMQComponent;
}
It works well but when I try to gracefully shutdown the application.
The PooledConnectionFactory get destroyed before the camel graceful shutdown happens.
Leading to a tons of error and the route unable to correctly stops.
Like 20 times this error :
2017-05-04 18:21:59.748 WARN 12188 --- [er[test.queue]] o.a.activemq.jms.pool.PooledSession : Caught exception trying rollback() when putting session back into the pool, will invalidate. javax.jms.IllegalStateException: The Session is closed
Followed by:
2017-05-04 18:21:59.748 INFO 12188 --- [ Thread-18] o.a.camel.spring.SpringCamelContext : Apache Camel 2.18.3 (CamelContext: route) is shutting down
Then later :
2017-05-04 18:21:59.766 INFO 12188 --- [ - ShutdownTask] o.a.camel.impl.DefaultShutdownStrategy : Waiting as there are still 1 inflight and pending exchanges to complete, timeout in 300 seconds. Inflights per route: [test2 = 1]
Anyone can help me configuring spring-boot camel activemq all together with graceful shutdown ?
Thanks
Update :
Here is a sample of my pom.xml:
<properties>
<!-- Spring -->
<spring-boot.version>1.4.3.RELEASE</spring-boot.version>
<!-- Camel -->
<camel-spring-boot.version>2.18.3</camel-spring-boot.version>
</properties>
....
<!-- Camel BOM -->
<dependency>
<!-- Import dependency management from Spring Boot -->
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>${spring-boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-spring-boot-dependencies</artifactId>
<version>${camel-spring-boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
...
<!-- Spring -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jetty</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-actuator</artifactId>
</dependency>
<!-- ActiveMQ -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-activemq</artifactId>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-camel</artifactId>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-pool</artifactId>
</dependency>
<!-- Camel -->
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-spring-boot-starter</artifactId>
</dependency>
Update 2:
After further investigation and the creation of a new project adding every modification one by one I have isolated the problem.
The shutdown works correcly until I add a specific endpoint :
#EndpointInject(uri = "direct:aaa")
private Endpoint errorHandling;
Using :
private String errorHandling = "direct:aaa";
Doesn't produce the bug.
It seems like using #EndpointInject is making the activemq close first
Update 3 :
Found that SpringCamelContext is not implementing ApplicationListener and thus its method "onApplicationEvent" its not called handling the "shutdownEager" of camel.
Important thing is to use Camel Spring Boot Starter.
http://camel.apache.org/spring-boot.html
How to enable Camel auto-configuration in my Spring Boot application?
Just drop camel-spring-boot jar into your classpath:
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-spring-boot</artifactId>
<version>${camel.version}</version> <!-- use the same version as your Camel core version -->
</dependency>
camel-spring-boot jar comes with the spring.factories file, so as soon as you add that dependency into your classpath, Spring Boot will automatically auto-configure the Camel for you. Yay! That was fast ;) .
Auto-configured Camel context
The most important piece of functionality provided by the Camel auto-configuration is the CamelContext instance.
Camel auto-configuration creates SpringCamelContext for your and take care of the proper initialization and shutdown of that context.
Created Camel context is also registered in the Spring application context (under camelContext bean name), so you can access it just as the any other Spring bean.
#Configuration
public class MyAppConfig {
#Autowired
CamelContext camelContext;
#Bean
MyService myService() {
return new DefaultMyService(camelContext);
}
}
Apparently since https://issues.apache.org/jira/browse/CAMEL-2607
the SpringCamelContext doesn't implement ApplicationListener interface anymore.
Since I'm using spring-boot autoconfiguration, I am not using CamelContextFactoryBean which is adding the listener.
Has a temporary fix, I created a component which listen to ApplicationEvent and dispatch them to the SpringCamelContext method :
public class SpringCamelContextFix implements ApplicationListener<ApplicationEvent> {
private SpringCamelContext camelContext;
public SpringCamelContextFix(SpringCamelContext camelContext) {
this.camelContext = camelContext;
}
#Override
public void onApplicationEvent(ApplicationEvent event) {
camelContext.onApplicationEvent(event);
}
}
I had this same problem running unit/integration tests with Spring Boot, ActiveMQ or A-MQ, and Camel (version 2.18.1.redhat-000012). Apparently, when Spring Boot shuts down, the JMS thread pool is closed before the Camel context is shutdown, which is the wrong order. #John D provided a code fix in a Camel users mailing list thread which is similar to what he provided in this thread. Here is the version of John D's code that worked for me:
#Component
public class SpringCamelContextFix implements
ApplicationListener<ApplicationEvent> {
#Inject
private SpringCamelContext camelContext;
public SpringCamelContextFix(SpringCamelContext camelContext) {
this.camelContext = camelContext;
}
#Override
public void onApplicationEvent(ApplicationEvent event) {
camelContext.onApplicationEvent(event);
}
}
I have a spring-boot-app with Actuator and Hystrix enabled.
Spring-Boot-Version: 1.3.1.RELEASE
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-eureka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-hystrix</artifactId>
</dependency>
When I add #HystrixCommandto some methods, the /metrics Endpoint shows all the metrics from hystrix:
gauge.hystrix.HystrixCommand.RestEndpoints.TestController.test.errorPercentage: 0,
gauge.hystrix.HystrixThreadPool.RestEndpoints.rollingMaxActiveThreads: 1,
...
Question:
How can I exclude the hystrix-metrics completely from the /metricsendpoint?
Update 1
I tried to exclude the ServoMetrics and SpectatorMetrics with these approaches:
1)
#EnableAutoConfiguration(exclude={ServoMetricsAutoConfiguration.class,
SpectatorMetricsAutoConfiguration.class} )
2)
#SpringBootApplication(exclude={ServoMetricServices.class, SpectatorMetricServices.class})
But both did not achieved the desired effect.
This issue was created and fixed. In snapshots you can now set the following:
hystrix.metrics.enabled=false.
If you don't want any other metrics, better solution would be to create your own MetricsRegistry. This way, any other future code changes (adding more jars with more in built metrics) would not affect.
#Configuration
public class MetricsConfiguration {
private MetricRegistry metricRegistry;
MetricsConfiguration() {
//avoid other metrics already registered
metricRegistry = new MetricRegistry();
}
}
Note: Creating a #Bean of type MetricRegistry would not help, as Spring autoconfigure will use this Bean object instead of MetricsRegistry object from MetricsDropwizardAutoConfiguration.java