How to configure netty in spring boot 2 - spring-boot

By default spring web flux uses netty which is single threaded event loop. How to configure spring boot so that a thread will be created for each core.
Thanks,
Lokesh

As described in the Spring Boot reference documentation, you can customize the Reactor Netty web server with a NettyServerCustomizer.
Here's an example with Spring Boot 2.1:
#Component
public class MyNettyWebServerCustomizer
implements WebServerFactoryCustomizer<NettyReactiveWebServerFactory> {
#Override
public void customize(NettyReactiveWebServerFactory factory) {
factory.addServerCustomizers(new EventLoopNettyCustomizer());
}
}
class EventLoopNettyCustomizer implements NettyServerCustomizer {
#Override
public HttpServer apply(HttpServer httpServer) {
LoopResources loopResources = LoopResources.create(...);
return httpServer.runOn(loopResources);
}
}

You can change your dependencies:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<!-- Exclude the Tomcat dependency -->
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- Use Jetty instead -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jetty</artifactId>
</dependency>
https://docs.spring.io/spring-boot/docs/current/reference/html/howto-embedded-web-servers.html

Related

Use tomcat instead of netty for rscket

I want to leverage on servlets and filters so I want to use tomcat and, in general, servlet 3.1 to handle the communication.
I tried to do the following:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-rsocket</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-reactor-netty</artifactId>
</exclusion>
</exclusions>
It works bringing up a tomcat but I lost the endpoint!
Here's what I do to register it:
spring.rsocket.server:
transport: websocket
mapping-path: /topics
and:
#Configuration
public class RSocketConfig {
#Bean
public Mono<RSocketRequester> rSocketRequester(
RSocketStrategies rSocketStrategies,
RSocketProperties rSocketProps) {
return RSocketRequester.builder()
.rsocketStrategies(rSocketStrategies)
.connectWebSocket(getURI(rSocketProps));
}
private URI getURI(RSocketProperties rSocketProps) {
return URI.create(String.format("ws://localhost:%d%s",
rSocketProps.getServer().getPort(), rSocketProps.getServer().getMappingPath()));
}

Resource leak detected when using spring-data-redis on cloud foundry

We develop a spring-boot service, which offers a rest api (spring-webflux) and sends data via RabbitMQ (spring-rabbit). The service is deployed on cloud foundry and we use spring-boot in version 2.1.4. We added spring-boot-starter-data-redis to use redis to cache some data and we got the following error:
[io.netty.util.ResourceLeakDetector] [] LEAK: HashedWheelTimer.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records:
Created at:
io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:284)
io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:217)
io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:196)
io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:178)
io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:162)
io.lettuce.core.resource.DefaultClientResources.<init>(DefaultClientResources.java:169)
io.lettuce.core.resource.DefaultClientResources$Builder.build(DefaultClientResources.java:532)
io.lettuce.core.resource.DefaultClientResources.create(DefaultClientResources.java:233)
io.lettuce.core.AbstractRedisClient.<init>(AbstractRedisClient.java:98)
io.lettuce.core.RedisClient.<init>(RedisClient.java:87)
io.lettuce.core.RedisClient.create(RedisClient.java:124)
org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.lambda$createClient$7(LettuceConnectionFactory.java:971)
java.base/java.util.Optional.orElseGet(Unknown Source)
org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.createClient(LettuceConnectionFactory.java:971)
org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.afterPropertiesSet(LettuceConnectionFactory.java:273)
org.springframework.cloud.service.keyval.RedisConnectionFactoryCreator.create(RedisConnectionFactoryCreator.java:88)
org.springframework.cloud.service.keyval.RedisConnectionFactoryCreator.create(RedisConnectionFactoryCreator.java:31)
org.springframework.cloud.Cloud.getServiceConnector(Cloud.java:288)
org.springframework.cloud.Cloud.getSingletonServiceConnector(Cloud.java:202)
org.springframework.cloud.config.java.CloudServiceConnectionFactory.redisConnectionFactory(CloudServiceConnectionFactory.java:260)
org.springframework.cloud.config.java.CloudServiceConnectionFactory.redisConnectionFactory(CloudServiceConnectionFactory.java:242)
...
This error only happens when we run the service on cloud foundry, if we run it locally, we don't get any error.
We don't do any configuration of the connection factory or the stringRedisTemplate on our side and only use stringRedisTemplate, which is configured by the spring-autoconfiguration.
We use following configuration for redis on cloud foundry:
#Configuration
#Profile( "cloud" )
public class CloudSpecificConfig extends AbstractCloudConfig {
#Bean
public RedisConnectionFactory redisConnectionFactory() {
return connectionFactory().redisConnectionFactory();
}
}
And this is how we use the template
#Component
#RequiredArgsConstructor
public final class RequestUtil {
private final StringRedisTemplate myRedisTemplate;
public String cacheId(String id, String value) {
myRedisTemplate.opsForValue().set( id, value );
}
}
These are our spring dependencies:
<properties>
<spring-boot-version>2.1.4.RELEASE</spring-boot-version>
</properties>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
<version>${spring-boot-version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<version>${spring-boot-version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
<version>${spring-boot-version}</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-sleuth</artifactId>
<version>${spring-boot-version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-cloud-connectors</artifactId>
<version>${spring-boot-version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
<version>${spring-boot-version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
<version>${spring-boot-version}</version>
</dependency>
We are quite confused on our side, since we didn't do any specific configuration on our side. It looks for us like there is something wrong with the spring configuration on the cloud. Are we doing something wrong? Do we need to configure something differently? Is this a bug?
This is what I had to do
I will see if I can find a more elegant way
#Bean(destroyMethod = "shutdown")
public DefaultClientResources lettuceClientResources() {
return DefaultClientResources.create();
}
#SuppressWarnings("unused")
#Bean
public RedisConnectionFactory redisConnectionFactory(DefaultClientResources dependency) {
return connectionFactory().redisConnectionFactory("redis-pcf-service");
}
In the end the issue disappeared on our side, because we changed the redis client from lettuce to jedis.
We had the problem with lettuce that we would lose the connection to our redis service on our cloud infrastructure. But since there was an update to the redis service at same time as we changed the client, we don't really know if it was related to lettuce.
Maybe there also just something wrong in the auto-configuration in conjunction with the redis service on our cloud instructure, which is based on cloudfoundry

Spring boot 2 reactive web websocket conflict with datarest

I'm using spring boot 2 to create a project and use websocket using reactive web dependency. My application is worked correctly until I add datarest dependency. after I add datarest dependency application give
' failed: Error during WebSocket handshake: Unexpected response code: 404
is any way to resolve this conflict?.
pom.xml
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-integration</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<!-- https://mvnrepository.com/artifact/org.springframework.integration/spring-integration-file -->
<dependency>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-file</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-rest</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
WebSocketConfiguration
#Configuration
public class WebSocketConfiguration {
#Bean
public IntegrationFlow fileFlow(PublishSubscribeChannel channel, #Value("file://${HOME}/Desktop/in") File file) {
FileInboundChannelAdapterSpec in = Files.inboundAdapter(file).autoCreateDirectory(true);
return IntegrationFlows.from(
in,
p -> p.poller(pollerFactory -> {
return pollerFactory.fixedRate(1000);
})
).channel(channel).get();
}
#Bean
#Primary
public PublishSubscribeChannel incomingFilesChannel() {
return new PublishSubscribeChannel();
}
#Bean
public WebSocketHandlerAdapter webSocketHandlerAdapter() {
return new WebSocketHandlerAdapter();
}
#Bean
public WebSocketHandler webSocketHandler(PublishSubscribeChannel channel) {
return session -> {
Map<String, MessageHandler> connections = new ConcurrentHashMap<>();
Flux<WebSocketMessage> publisher = Flux.create((Consumer<FluxSink<WebSocketMessage>>) fluxSink -> {
connections.put(session.getId(), new ForwardingMessageHandler(session, fluxSink));
channel.subscribe(connections.get(session.getId()));
}).doFinally(signalType -> {
channel.unsubscribe(connections.get(session.getId()));
connections.remove(session.getId());
});
return session.send(publisher);
};
}
#Bean
public HandlerMapping handlerMapping(WebSocketHandler webSocketHandler) {
SimpleUrlHandlerMapping handlerMapping = new SimpleUrlHandlerMapping();
handlerMapping.setOrder(10);
handlerMapping.setUrlMap(Collections.singletonMap("/ws/files", webSocketHandler));
return handlerMapping;
}
}
spring-boot-starter-data-rest brings spring-boot-starter-web as a transitive dependency (so basically Spring MVC). This makes Spring Boot configure your application as a Spring MVC web application.
Spring Data REST does not currently support Spring WebFlux (see this issue for more information on that).
Your only choice is to remove the Spring Data REST dependency, as you can't have both Spring MVC and Spring WebFlux in the same Spring Boot application.

ConnectionFactory get destroyed before camel

I'm using spring-boot with camel and ActiveMQ.
I'm using activemq autoconfiguration via #EnableJms annotation.
But creating my own ActiveMQComponent to enable "transacted(true)" on all queues.
#Bean(name = "activemq")
#ConditionalOnClass(ActiveMQComponent.class)
public ActiveMQComponent activeMQComponent(ConnectionFactory connectionFactory) {
ActiveMQComponent activeMQComponent = new ActiveMQComponent();
activeMQComponent.setConnectionFactory(connectionFactory);
activeMQComponent.setTransacted(true);
activeMQComponent.setTransactionManager(jmsTransactionManager(connectionFactory));
return activeMQComponent;
}
It works well but when I try to gracefully shutdown the application.
The PooledConnectionFactory get destroyed before the camel graceful shutdown happens.
Leading to a tons of error and the route unable to correctly stops.
Like 20 times this error :
2017-05-04 18:21:59.748 WARN 12188 --- [er[test.queue]] o.a.activemq.jms.pool.PooledSession : Caught exception trying rollback() when putting session back into the pool, will invalidate. javax.jms.IllegalStateException: The Session is closed
Followed by:
2017-05-04 18:21:59.748 INFO 12188 --- [ Thread-18] o.a.camel.spring.SpringCamelContext : Apache Camel 2.18.3 (CamelContext: route) is shutting down
Then later :
2017-05-04 18:21:59.766 INFO 12188 --- [ - ShutdownTask] o.a.camel.impl.DefaultShutdownStrategy : Waiting as there are still 1 inflight and pending exchanges to complete, timeout in 300 seconds. Inflights per route: [test2 = 1]
Anyone can help me configuring spring-boot camel activemq all together with graceful shutdown ?
Thanks
Update :
Here is a sample of my pom.xml:
<properties>
<!-- Spring -->
<spring-boot.version>1.4.3.RELEASE</spring-boot.version>
<!-- Camel -->
<camel-spring-boot.version>2.18.3</camel-spring-boot.version>
</properties>
....
<!-- Camel BOM -->
<dependency>
<!-- Import dependency management from Spring Boot -->
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>${spring-boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-spring-boot-dependencies</artifactId>
<version>${camel-spring-boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
...
<!-- Spring -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jetty</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-actuator</artifactId>
</dependency>
<!-- ActiveMQ -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-activemq</artifactId>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-camel</artifactId>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-pool</artifactId>
</dependency>
<!-- Camel -->
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-spring-boot-starter</artifactId>
</dependency>
Update 2:
After further investigation and the creation of a new project adding every modification one by one I have isolated the problem.
The shutdown works correcly until I add a specific endpoint :
#EndpointInject(uri = "direct:aaa")
private Endpoint errorHandling;
Using :
private String errorHandling = "direct:aaa";
Doesn't produce the bug.
It seems like using #EndpointInject is making the activemq close first
Update 3 :
Found that SpringCamelContext is not implementing ApplicationListener and thus its method "onApplicationEvent" its not called handling the "shutdownEager" of camel.
Important thing is to use Camel Spring Boot Starter.
http://camel.apache.org/spring-boot.html
How to enable Camel auto-configuration in my Spring Boot application?
Just drop camel-spring-boot jar into your classpath:
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-spring-boot</artifactId>
<version>${camel.version}</version> <!-- use the same version as your Camel core version -->
</dependency>
camel-spring-boot jar comes with the spring.factories file, so as soon as you add that dependency into your classpath, Spring Boot will automatically auto-configure the Camel for you. Yay! That was fast ;) .
Auto-configured Camel context
The most important piece of functionality provided by the Camel auto-configuration is the CamelContext instance.
Camel auto-configuration creates SpringCamelContext for your and take care of the proper initialization and shutdown of that context.
Created Camel context is also registered in the Spring application context (under camelContext bean name), so you can access it just as the any other Spring bean.
#Configuration
public class MyAppConfig {
#Autowired
CamelContext camelContext;
#Bean
MyService myService() {
return new DefaultMyService(camelContext);
}
}
Apparently since https://issues.apache.org/jira/browse/CAMEL-2607
the SpringCamelContext doesn't implement ApplicationListener interface anymore.
Since I'm using spring-boot autoconfiguration, I am not using CamelContextFactoryBean which is adding the listener.
Has a temporary fix, I created a component which listen to ApplicationEvent and dispatch them to the SpringCamelContext method :
public class SpringCamelContextFix implements ApplicationListener<ApplicationEvent> {
private SpringCamelContext camelContext;
public SpringCamelContextFix(SpringCamelContext camelContext) {
this.camelContext = camelContext;
}
#Override
public void onApplicationEvent(ApplicationEvent event) {
camelContext.onApplicationEvent(event);
}
}
I had this same problem running unit/integration tests with Spring Boot, ActiveMQ or A-MQ, and Camel (version 2.18.1.redhat-000012). Apparently, when Spring Boot shuts down, the JMS thread pool is closed before the Camel context is shutdown, which is the wrong order. #John D provided a code fix in a Camel users mailing list thread which is similar to what he provided in this thread. Here is the version of John D's code that worked for me:
#Component
public class SpringCamelContextFix implements
ApplicationListener<ApplicationEvent> {
#Inject
private SpringCamelContext camelContext;
public SpringCamelContextFix(SpringCamelContext camelContext) {
this.camelContext = camelContext;
}
#Override
public void onApplicationEvent(ApplicationEvent event) {
camelContext.onApplicationEvent(event);
}
}

Spring Boot/Spring Integration Servlet Context/DelegatingWebMvcConfiguration issue

I'm not sure the best way to word this issue, because I have no idea why this happens.
When I start up my Spring Boot application without the spring-integration-core jar, my app starts up fine. The second I add the spring-boot-starter-integration or any spring-integration-core library to my Maven pom file the servlet context is always null on my DelegatingWebMvcConfiguration class and I get the following error:
java.lang.IllegalArgumentException: A ServletContext is required to configure default servlet handling
After doing some debugging, I see the DelegatingWebMvcConfiguration class initialized before the EmbeddedWebApplicationContext sets the servlet context for my application, which makes sense why the servlet context is null for my DelegatingWebMvcConfiguration and set for all the other classes initialized with postProcessBeforeInitialization in ServletContextAwareProcessor. Simply removing the Spring Integration dependency fixes the issue, but I'd like to use Spring Integration in my Boot application.
My question is what would cause the DelegatingWebMvcConfiguration to be created before the EmbeddedWebApplicationContext can call prepareEmbeddedWebApplicationContext method.
Spring dependencies using Spring Boot 1.1.8.RELEASE:
org.springframework.boot
spring-boot-starter-web
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-solr</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-integration</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.hateoas</groupId>
<artifactId>spring-hateoas</artifactId>
<version>0.16.0.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-rest-core</artifactId>
<version>2.2.1.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-cassandra</artifactId>
<version>1.0.2.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.ldap</groupId>
<artifactId>spring-ldap-core</artifactId>
<version>2.0.2.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context-support</artifactId>
<version>4.1.1.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jms</artifactId>
<version>4.1.2.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-messaging</artifactId>
<version>4.1.2.RELEASE</version>
</dependency>
My root configuration file:
#Configuration
#EnableAutoConfiguration
#EnableAsync
#EnableEntityLinks
#EnableSpringDataWebSupport
#ComponentScan
#PropertySource(value = "file:../application.properties",
ignoreResourceNotFound = true)
public class ApplicationInitializer extends SpringBootServletInitializer
implements EmbeddedServletContainerCustomizer {
public static void main(String[] args) throws Exception {
SpringApplication.run(ApplicationInitializer.class, args);
}
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder application){
return application.sources(ApplicationInitializer.class);
}
#Override
public void customize(ConfigurableEmbeddedServletContainer container) {
container.setContextPath(APPLICATION_ROOT_PATH);
}
// Global beans defined here
}
Update: After trying many different things, I found out after disabling my SchedulingConfigurer my app now starts. Here is my SchedulingConfigurer
#Configuration
#EnableScheduling
#EnableConfigurationProperties(SchedulingProperties.class)
public class SchedulingConfig implements SchedulingConfigurer {
//Inject Runnables and properties
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
taskRegistrar.setScheduler(taskScheduler());
// call taskRegistrar.addFixedRateTask with runnable and property rate value
}
#Bean(destroyMethod = "shutdown")
public Executor taskScheduler() {
ThreadPoolTaskScheduler taskScheduler = new ThreadPoolTaskScheduler();
taskScheduler.setPoolSize(Runtime.getRuntime().availableProcessors());
return taskScheduler;
}
}

Resources