Getting connection timed out error in Java microservice - spring

I have built a microservice using Java 8 and SpringBoot 2. From this microservice, I'm trying to consume another REST API service. However, I'm getting the following error on Chrome
java.lang.IllegalStateException: The underlying HTTP client completed
without emitting a response.
2018-06-12 15:21:29.300 ERROR 17996 --- [ctor-http-nio-3]
.a.w.r.e.DefaultErrorWebExceptionHandler : Failed to handle request
[GET http://localhost:8080/category/search]
io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
timed out: no further information: test.usdemo.xyz.com/92.54.41.24:443
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.8.0_171]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
~[na:1.8.0_171]
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:325)
~[netty-transport-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340)
~[netty-transport-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
~[netty-transport-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
~[netty-transport-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
~[netty-transport-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
~[netty-transport-4.1.24.Final.jar:4.1.24.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
~[netty-common-4.1.24.Final.jar:4.1.24.Final]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_171] Caused by: java.net.ConnectException: Connection timed
out: no further information
... 10 common frames omitted
I'm able to consume the same service successfully using PostMan but not through my microservice.
Please assist to advise on this.

I do not know how to edit my own question above as I do not see the edit option hence I'm adding the additional details here to get the resolution of my query.
Controller class:-
#RestController
public class CategorySearchController {
private final CategorySearchService categorySearchService;
#Autowired
public CategorySearchController(CategorySearchService categorySearchService) {
this.categorySearchService = categorySearchService;
}
#GetMapping(path = "/search-category")
public Mono<CategoryResponse> searchCategories(SearchRequest categorySearchRequest){
return categorySearchService
.searchCategories()
.switchIfEmpty(Mono.error(
new EntityNotFoundException("No category matching " + categorySearchRequest.getSearchTerm() + " was found")));
}
}
Service class:-
#Service
public class CategorySearchServiceImpl implements CategorySearchService {
private String baseUrl = "https://demo0954903.mockable.io";
#Override
public Mono<CategoryResponse> searchCategories() {
WebClient webClient = WebClient.create(baseUrl);
return webClient.
get()
.uri("/category-search")
.retrieve()
.bodyToMono(CategoryResponse.class);
}
}

I found the solution for this issue. I need to add the proxy in the webclient as follows:-
private final WebClient webClient;
ReactorClientHttpConnector connector = new ReactorClientHttpConnector(
options -> options.httpProxy(addressSpec -> {
return addressSpec.host(PROXY_HOST).port(PROXY_PORT);
}));
this.webClient = webClientBuilder.baseUrl(BASE_URL).clientConnector(connector).build();

Check your proxy settings. In Spring applications, you can add the following to the VM arguments to set your proxy: -Dhttp.proxyHost=your.proxy.net -Dhttp.proxyPort=8080

I met a similar issue on a scala project. According to the error message, I found IntelliJ took Ant as the compiler which I was expected sbt as the compiler.
Solve the issue by turn on sbt shell use for builds option (Preferences > Build, Execution, Deployment > Build Tools > sbt > sbt projects > sbt shell > check builds )
Executing pre-compile tasks...
Loading Ant configuration...
Running Ant tasks...
Abnormal build process termination:
{remove some sensitive logs here}
io.netty.channel.AbstractChannel$AnnotatedConnectException: Operation timed out: /127.0.0.1:50154
Caused by: java.net.ConnectException: Operation timed out
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:707)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.lang.Thread.run(Thread.java:748)
Exiting.
Executing post-compile tasks...
Loading Ant configuration...
Running Ant tasks...
Synchronizing output directories...
2021/9/15, 8:52 AM - Build completed with 1 error and 0 warnings in 27 sec, 726 ms

Related

SpringBoot Camel (3.14.0) and ElasticSearch Integration

I'm trying to integrate Spring Boot (2.6.3) with Camel (3.14.0) to send a simple query to ElasticSearch.
The project is located at https://github.com/saavedrah/springboot-camel-elastic
I have a timer that sends a ElasticSearch query every 60 sec.
However, the following exception is generated:
2022-01-26 11:08:48.036 WARN 23668 --- [1 - timer://foo] o.a.camel.component.timer.TimerConsumer : Error processing exchange. Exchange[2578AAA06ED30CB-0000000000000000]. Caused by: [org.apache.camel.CamelExecutionException - Exception occurred during execution on the exchange: Exchange[]]
org.apache.camel.CamelExecutionException: Exception occurred during execution on the exchange: Exchange[]
at org.apache.camel.CamelExecutionException.wrapCamelExecutionException(CamelExecutionException.java:45) ~[camel-api-3.14.0.jar:3.14.0]
at org.apache.camel.support.AbstractExchange.setException(AbstractExchange.java:589) ~[camel-support-3.14.0.jar:3.14.0]
at org.apache.camel.support.DefaultExchange.setException(DefaultExchange.java:27) ~[camel-support-3.14.0.jar:3.14.0]
at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:69) ~[camel-support-3.14.0.jar:3.14.0]
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:172) ~[camel-core-processor-3.14.0.jar:3.14.0]
at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:469) ~[camel-core-processor-3.14.0.jar:3.14.0]
at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:187) ~[camel-base-engine-3.14.0.jar:3.14.0]
at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:64) ~[camel-base-engine-3.14.0.jar:3.14.0]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:184) ~[camel-core-processor-3.14.0.jar:3.14.0]
at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:398) ~[camel-base-engine-3.14.0.jar:3.14.0]
at org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:210) ~[camel-timer-3.14.0.jar:3.14.0]
at org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:76) ~[camel-timer-3.14.0.jar:3.14.0]
at java.base/java.util.TimerThread.mainLoop(Timer.java:556) ~[na:na]
at java.base/java.util.TimerThread.run(Timer.java:506) ~[na:na]
Caused by: java.lang.NoClassDefFoundError: org/elasticsearch/common/CheckedConsumer
at org.apache.camel.component.elasticsearch.ElasticsearchProducer$HighLevelClient.<init>(ElasticsearchProducer.java:347) ~[camel-elasticsearch-rest-3.14.0.jar:3.14.0]
at org.apache.camel.component.elasticsearch.ElasticsearchProducer$HighLevelClient.<init>(ElasticsearchProducer.java:345) ~[camel-elasticsearch-rest-3.14.0.jar:3.14.0]
at org.apache.camel.component.elasticsearch.ElasticsearchProducer.process(ElasticsearchProducer.java:124) ~[camel-elasticsearch-rest-3.14.0.jar:3.14.0]
at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66) ~[camel-support-3.14.0.jar:3.14.0]
... 10 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.elasticsearch.common.CheckedConsumer
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) ~[na:na]
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) ~[na:na]
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) ~[na:na]
... 14 common frames omitted
UPDATE 1
Accordingly with this site CAMEL-17183 the downloaded version of ElasticSearch (7.15.2) does not have the [org.elasticsearch.common.CheckedConsumer] class.
So, how should I update the gradle.properties to get the supported version of ElasticSearch(7.8.0) ?
UPDATE 2
Gradle properties was updated to force the version of ElasticSearch.
configurations.all {
resolutionStrategy {
dependencySubstitution {
substitute module('org.elasticsearch.client:elasticsearch-rest-high-level-client') with module('org.elasticsearch.client:elasticsearch-rest-high-level-client:7.8.0')
substitute module('org.elasticsearch.client:elasticsearch-rest-client') with module('org.elasticsearch.client:elasticsearch-rest-client:7.8.0')
substitute module('org.elasticsearch:elasticsearch') with module('org.elasticsearch:elasticsearch:7.8.0')
}
}
}
And now I'm getting the following exception, I guess it is related with the lack of parameters passed to the query...
java.lang.IllegalArgumentException: Wrong body type. Only Map, String or SearchRequest is allowed as a type
at org.apache.camel.component.elasticsearch.ElasticsearchProducer.process(ElasticsearchProducer.java:232)
at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:172)
at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:469)
at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:187)
at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:64)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:184)
at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:398)
at org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:210)
at org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:76)
at java.base/java.util.TimerThread.mainLoop(Timer.java:556)
at java.base/java.util.TimerThread.run(Timer.java:506)
13:13:25.219 [Camel (camel-1) thread #1 - timer://foo] WARN o.a.c.component.timer.TimerConsumer - Error processing exchange. Exchange[DB28E010007B18C-0000000000000006]. Caused by: [java.lang.IllegalArgumentException - Wrong body type. Only Map, String or SearchRequest is allowed as a type]
java.lang.IllegalArgumentException: Wrong body type. Only Map, String or SearchRequest is allowed as a type
at org.apache.camel.component.elasticsearch.ElasticsearchProducer.process(ElasticsearchProducer.java:232)
at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:172)
at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:469)
at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:187)
at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:64)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:184)
at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:398)
at org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:210)
at org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:76)
at java.base/java.util.TimerThread.mainLoop(Timer.java:556)
at java.base/java.util.TimerThread.run(Timer.java:506)
Any help is appreciated.
What Huang was telling me was that I should put some query in my timer component body to pass it to the Search operation, something like:
from("timer://foo?fixedRate=true&period=60000")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
exchange.getIn().setBody("{\"query\":{\"match_all\":{}}}");
}
})
.to("direct:search");
from("direct:search")
.to("elasticsearch-rest://elasticsearch?operation=Search&indexName=twitter");
Thank you.

Kotlin based Spring RabbitListener produces endlessloop trying to send back `kotlin.Unit`

We have a RabbitListener implemented in Kotlin. The method returns Unit, so I dont expect Spring to send a result but we are getting the following error message. It looks like Spring tries to iterprete Unit as the result and send a rabbit message back:
2021-04-16 16:39:21
msg="Execution of Rabbit message listener failed." thread="org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#0-2" level=WARN logger="org.springframework.amqp.rabbit.listener.ConditionalRejectingErrorHandler" exception="org.springframework.amqp.rabbit.support.ListenerExecutionFailedException: Listener threw exception
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.wrapToListenerExecutionFailedExceptionIfNeeded(AbstractMessageListenerContainer.java:1746)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1636)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.actualInvokeListener(AbstractMessageListenerContainer.java:1551)
at jdk.internal.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at brave.spring.rabbit.TracingRabbitListenerAdvice.invoke(TracingRabbitListenerAdvice.java:108)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at org.springframework.amqp.rabbit.listener.$Proxy146.invokeListener(Unknown Source)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:1539)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:1530)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:1474)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:967)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:913)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1600(SimpleMessageListenerContainer.java:83)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.mainLoop(SimpleMessageListenerContainer.java:1288)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1194)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: org.springframework.amqp.rabbit.listener.adapter.ReplyFailureException: Failed to send reply with payload 'InvocationResult [returnValue=kotlin.Unit, returnType=class java.lang.Object, bean=com.xxx.EventHandler#4aaf6902, method=public java.lang.Object com.xxx.EventHandler.receiveMessage(org.springframework.amqp.core.Message)]'
at org.springframework.amqp.rabbit.listener.adapter.AbstractAdaptableMessageListener.doHandleResult(AbstractAdaptableMessageListener.java:476)
at org.springframework.amqp.rabbit.listener.adapter.AbstractAdaptableMessageListener.handleResult(AbstractAdaptableMessageListener.java:400)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandlerAndProcessResult(MessagingMessageListenerAdapter.java:152)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:135)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1632)
... 20 common frames omitted
Caused by: java.lang.IllegalArgumentException: SimpleMessageConverter only supports String, byte[] and Serializable payloads, received: kotlin.Unit
at org.springframework.amqp.support.converter.SimpleMessageConverter.createMessage(SimpleMessageConverter.java:164)
at org.springframework.amqp.support.converter.AbstractMessageConverter.createMessage(AbstractMessageConverter.java:88)
at org.springframework.amqp.support.converter.AbstractMessageConverter.toMessage(AbstractMessageConverter.java:70)
at org.springframework.amqp.rabbit.listener.adapter.AbstractAdaptableMessageListener.convert(AbstractAdaptableMessageListener.java:519)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.buildMessage(MessagingMessageListenerAdapter.java:257)
at org.springframework.amqp.rabbit.listener.adapter.AbstractAdaptableMessageListener.doHandleResult(AbstractAdaptableMessageListener.java:464)
... 24 common frames omitted
" time="2021-04-16 14:39:21,726"
here a simplified version of the RabbitLister:
#Component
class EventHandler {
#RabbitListener(queues = ["someQueue"])
fun receiveMessage(message: Message): Unit {
log.debug("Received message")
}
}
We use Spring Boot 2.4.4, Spring Rabbit 2.3.6 and Spring Framework 5.3.6
debugging the issue, I got the following result:
What version are you using?
This works fine for me (with and without the Unit return type)...
#RabbitListener(queues = ["foo"])
open fun listen(data: String?): Unit {
println(data)
}
Even with : Unit, I see void as the return type and null:

grcp failure while posting in etcd via spring boot

I'm running a simple etcd connection with springbok example. I keep getting an error w.r.t grcp in both my spring boot logs and server logs. How can I fix this?
#Scheduled(fixedRate = 5000)
public void test() throws EtcdException, ExecutionException, InterruptedException {
// create client
Client client = Client.builder().endpoints("http://localhost:2379").build();
KV kvClient = client.getKVClient();
ByteSequence key = ByteSequence.fromBytes("test_key".getBytes());
ByteSequence val = ByteSequence.fromBytes("test_val".getBytes());
// put the key-value
kvClient.put(key, val);
// get the CompletableFuture
CompletableFuture<GetResponse> getFuture = kvClient.get(key);
// get the value from CompletableFuture
GetResponse response = getFuture.get();
System.out.println(response.toString());
}
Error
.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
Caused by: java.util.concurrent.ExecutionException: io.grpc.StatusRuntimeException: INTERNAL: Connection closed with unknown cause
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476) ~[guava-19.0.jar:na]
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455) ~[guava-19.0.jar:na]
at com.coreos.jetcd.internal.impl.Util.lambda$toCompletableFutureWithRetry$1(Util.java:125) ~[jetcd-core-0.0.1.jar:na]
... 3 common frames omitted
Caused by: io.grpc.StatusRuntimeException: INTERNAL: Connection closed with unknown cause
at io.grpc.Status.asRuntimeException(Status.java:526) ~[grpc-core-1.5.0.jar:1.5.0]
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:427) ~[grpc-stub-1.5.0.jar:1.5.0]
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41) ~[grpc-core-1.5.0.jar:1.5.0]
at com.coreos.jetcd.internal.impl.ClientConnectionManager$AuthTokenInterceptor$1$1.onClose(ClientConnectionManager.java:267) ~[jetcd-core-0.0.1.jar:na]
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:419) ~[grpc-core-1.5.0.jar:1.5.0]
at io.grpc.internal.ClientCallImpl.access$100(ClientCallImpl.java:60) ~[grpc-core-1.5.0.jar:1.5.0]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:493) ~[grpc-core-1.5.0.jar:1.5.0]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$500(ClientCallImpl.java:422) ~[grpc-core-1.5.0.jar:1.5.0]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:525) ~[grpc-core-1.5.0.jar:1.5.0]
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[grpc-core-1.5.0.jar:1.5.0]
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:102) ~[grpc-core-1.5.0.jar:1.5.0]
... 3 common frames omitted
At ETCD server
WARNING: 2020/04/16 00:23:15 grpc: Server.processUnaryRPC failed to write status: connection error: desc = "transport is closing"
Tried setting the keep alive property but didn't work
grpc.client.GLOBAL.keep-alive-time=5000
So it turns out that the grcp version was the reason, I was using verion 1.5.0 but when I moved to version 1.9.0, the issue was resolved for me.

Workaround for the slowness of the WebClient first request

I am using WebClient in a Spring Boot MVC 2.1 project and found that the first request made by the client takes up to 6 seconds. Subsequent requests are way faster (~30ms).
There's a closed issue in Spring's JIRA that advices using Jetty as the WebClient Http connector. I have tried that approach, improving the figures, with a ~800ms first request. This time is an improvement but it's still far from RestTemplate which usally takes <200ms.
Netty approach (5s first request):
Conf:
#Bean
public WebClient webClient() {
return WebClient.create();
}
Usage:
private final WebClient webClient;
#GetMapping(value="/wc", produces = APPLICATION_JSON_UTF8_VALUE)
public Mono<String> findWc() throws URISyntaxException {
URI uri = new URI("http://xxx");
final Mono<String> response = webClient.get().uri(uri).retrieve().bodyToMono(String.class);
return response;
}
Jetty approach (800ms first request):
Conf:
#Bean
public JettyResourceFactory resourceFactory() {
return new JettyResourceFactory();
}
#Bean
public WebClient webClient() {
ClientHttpConnector connector = new JettyClientHttpConnector(resourceFactory(), null);
return WebClient.builder().clientConnector(connector).build();
}
Usage: same as before.
There's another "problem" with the Jetty approach. On server shutdown it always produces the following exception:
27-Dec-2018 11:24:20.463 INFO [jetty-http#74305db9-65] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. Could not load [org.eclipse.jetty.io.ManagedSelector$StopSelector]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [org.eclipse.jetty.io.ManagedSelector$StopSelector]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1348)
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1336)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1195)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1157)
at java.lang.Class.getDeclaringClass0(Native Method)
at java.lang.Class.getDeclaringClass(Class.java:1235)
at java.lang.Class.getEnclosingClass(Class.java:1277)
at java.lang.Class.getSimpleBinaryName(Class.java:1443)
at java.lang.Class.getSimpleName(Class.java:1309)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.toString(ManagedSelector.java:534)
at java.lang.String.valueOf(String.java:2994)
at java.lang.StringBuilder.append(StringBuilder.java:131)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.getString(EatWhatYouKill.java:458)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.toStringLocked(EatWhatYouKill.java:447)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.toString(EatWhatYouKill.java:440)
at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:299)
at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:271)
at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:233)
at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:173)
at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:680)
at org.eclipse.jetty.util.log.JettyAwareLogger.debug(JettyAwareLogger.java:224)
at org.eclipse.jetty.util.log.Slf4jLog.debug(Slf4jLog.java:97)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:288)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
SLF4J: Failed toString() invocation on an object of type [org.eclipse.jetty.util.thread.strategy.EatWhatYouKill]
Reported exception:
java.lang.NoClassDefFoundError: org/eclipse/jetty/io/ManagedSelector$StopSelector
at java.lang.Class.getDeclaringClass0(Native Method)
at java.lang.Class.getDeclaringClass(Class.java:1235)
at java.lang.Class.getEnclosingClass(Class.java:1277)
at java.lang.Class.getSimpleBinaryName(Class.java:1443)
at java.lang.Class.getSimpleName(Class.java:1309)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.toString(ManagedSelector.java:534)
at java.lang.String.valueOf(String.java:2994)
at java.lang.StringBuilder.append(StringBuilder.java:131)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.getString(EatWhatYouKill.java:458)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.toStringLocked(EatWhatYouKill.java:447)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.toString(EatWhatYouKill.java:440)
at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:299)
at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:271)
at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:233)
at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:173)
at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:680)
at org.eclipse.jetty.util.log.JettyAwareLogger.debug(JettyAwareLogger.java:224)
at org.eclipse.jetty.util.log.Slf4jLog.debug(Slf4jLog.java:97)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:288)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: Illegal access: this web application instance has been stopped already. Could not load [org.eclipse.jetty.io.ManagedSelector$StopSelector]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1338)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1195)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1157)
... 25 more
Caused by: java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [org.eclipse.jetty.io.ManagedSelector$StopSelector]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1348)
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1336)
... 27 more
27-Dec-2018 11:24:20.467 INFO [jetty-http#74305db9-65] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.classic.spi.ThrowableProxy]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.classic.spi.ThrowableProxy]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1348)
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1336)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1195)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1157)
at ch.qos.logback.classic.spi.LoggingEvent.<init>(LoggingEvent.java:119)
at ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:419)
at ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:383)
at ch.qos.logback.classic.Logger.log(Logger.java:765)
at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:668)
at org.eclipse.jetty.util.log.JettyAwareLogger.warn(JettyAwareLogger.java:474)
at org.eclipse.jetty.util.log.Slf4jLog.warn(Slf4jLog.java:73)
at org.eclipse.jetty.util.log.Slf4jLog.warn(Slf4jLog.java:67)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.execute(EatWhatYouKill.java:375)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:305)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "jetty-http#74305db9-65" java.lang.NoClassDefFoundError: ch/qos/logback/classic/spi/ThrowableProxy
at ch.qos.logback.classic.spi.LoggingEvent.<init>(LoggingEvent.java:119)
at ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:419)
at ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:383)
at ch.qos.logback.classic.Logger.log(Logger.java:765)
at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:668)
at org.eclipse.jetty.util.log.JettyAwareLogger.warn(JettyAwareLogger.java:474)
at org.eclipse.jetty.util.log.Slf4jLog.warn(Slf4jLog.java:73)
at org.eclipse.jetty.util.log.Slf4jLog.warn(Slf4jLog.java:67)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:740)
at java.lang.Thread.run(Thread.java:748)
How can I avoid this exception?
Is there any other way we can use to improve the WebClient first request slowness?
I went through the same problem and managed to solve it by changing the connector used by the WebClient.
Below the configuration file with the appropriate imports
import org.eclipse.jetty.client.HttpClient;
import org.eclipse.jetty.util.ssl.SslContextFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.client.reactive.ClientHttpConnector;
import org.springframework.http.client.reactive.JettyClientHttpConnector;
import org.springframework.web.reactive.function.client.WebClient;
The class
SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
HttpClient httpClient = new HttpClient(sslContextFactory);
httpClient.setFollowRedirects(false);
httpClient.setConnectTimeout(TIMEOUT);
httpClient.start();
final ClientHttpConnector connector = new JettyClientHttpConnector(httpClient);
final WebClient webClient = WebClient.builder()
.clientConnector(connector)
.baseUrl(BASE_URL)
.build();
It is important to know how to add the right libs to your project, so below is how I managed to import using gradle:
implementation 'org.eclipse.jetty:jetty-client'
implementation 'org.eclipse.jetty:jetty-reactive-httpclient'
We upgraded Spring Boot to 2.4.2 with reactor-netty 1.0.3 but still encounter this issue.
Here is our upgraded configuration:
#Bean
public WebClient createWebClient(WebClient.Builder webClientBuilder) {
log.info("Initializing WebClient Bean");
final int timeoutInMillis = Long.valueOf(TimeUnit.SECONDS.toMillis(timeout)).intValue();
final HttpClient httpClient = HttpClient.create()
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, timeoutInMillis)
.responseTimeout(Duration.ofMillis(timeoutInMillis))
.doOnConnected(conn ->
conn.addHandlerLast(new ReadTimeoutHandler(timeoutInMillis, TimeUnit.MILLISECONDS))
.addHandlerLast(new WriteTimeoutHandler(timeoutInMillis, TimeUnit.MILLISECONDS)));
final ClientHttpConnector connector = new ReactorClientHttpConnector(httpClient);
final WebClient webClient = webClientBuilder
.clientConnector(connector)
.defaultHeader("x-clientname", clientname)
.build();
httpClient.warmup().block();
log.info("WebClient initialized");
return webClient;
}
Our call with WebClient:
ResoponseObject doCall() {
return this.webClient
.get()
.uri("http://***.de/api/rest/***")
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(ResponseObject.class)
.block();
}
With debugging enabled via application.yaml:
logging.level.reactor.netty: debug
We now see this during application startup:
2021-02-10 17:02:31,922 INFO d.t.e.b.c.c.WebClientAutoConfiguration - Initializing WebClient Bean
2021-02-10 17:02:31,959 DEBUG r.n.r.DefaultLoopIOUring - Default io_uring support : false
2021-02-10 17:02:31,967 DEBUG r.n.r.DefaultLoopEpoll - Default Epoll support : true
2021-02-10 17:02:31,997 INFO d.t.e.b.c.c.WebClientAutoConfiguration - WebClient initialized
This should be an indication that the warmup works as expected?
But on first request this happens:
2021-02-10 17:05:16,045 DEBUG o.s.w.r.f.c.ExchangeFunctions - [73d400c8] HTTP GET http://***.de/api/rest/***
2021-02-10 17:05:16,050 DEBUG r.n.r.PooledConnectionProvider - Creating a new [http] client pool [PoolFactory{evictionInterval=PT0S, leasingStrategy=fifo, maxConnections=500, maxIdleTime=-1, maxLifeTime=-1, metricsEnabled=false, pendingAcquireMaxCount=1000, pendingAcquireTimeout=45000}] for [***.de/<unresolved>:80]
2021-02-10 17:05:29,619 DEBUG r.n.r.DefaultPooledConnectionProvider - [id: 0x71b840f4] Created a new pooled channel, now 1 active connections and 0 inactive connections
2021-02-10 17:05:29,635 DEBUG r.n.t.TransportConfig - [id: 0x71b840f4] Initialized pipeline DefaultChannelPipeline{(reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
...
In our case, creating the client pool is the problem. On a decent machine, it takes about 13 seconds.
Can you give us any comment on that? This is very this is very frustrating for us.
Thanks a lot!

Getting connection timed out error in microservice

I have built a microservice using Java 8 and SpringBoot 2. From this microservice, I'm trying to consume another REST API service. However, I'm getting the following error on Chrome. I have already disabled the windows firewall and McAfee antivirus firewall but still getting the same error. I can call the REST API directly using the Postman tool but not through my microservice.
Error:-
java.lang.IllegalStateException: The underlying HTTP client completed
without emitting a response.
2018-06-12 15:21:29.300 ERROR 17996 --- [ctor-http-nio-3]
.a.w.r.e.DefaultErrorWebExceptionHandler : Failed to handle request
[GET http://localhost:8080/category/search]
io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
timed out: no further information: test.usdemo.xyz.com/92.54.41.24:443
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
~[na:1.8.0_171] at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
~[na:1.8.0_171] at
io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:325)
~[netty-transport-4.1.24.Final.jar:4.1.24.Final] at
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340)
~[netty-transport-4.1.24.Final.jar:4.1.24.Final] at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
~[netty-transport-4.1.24.Final.jar:4.1.24.Final] at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
~[netty-transport-4.1.24.Final.jar:4.1.24.Final] at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
~[netty-transport-4.1.24.Final.jar:4.1.24.Final] at
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
~[netty-transport-4.1.24.Final.jar:4.1.24.Final] at
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
~[netty-common-4.1.24.Final.jar:4.1.24.Final] at
java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_171] Caused by:
java.net.ConnectException: Connection timed out: no further
information ... 10 common frames omitted
Controller class:-
#RestController
public class CategorySearchController {
private final CategorySearchService categorySearchService;
#Autowired
public CategorySearchController(CategorySearchService categorySearchService) {
this.categorySearchService = categorySearchService;
}
#GetMapping(path = "/search-category")
public Mono<CategoryResponse> searchCategories(SearchRequest categorySearchRequest){
return categorySearchService
.searchCategories()
.switchIfEmpty(Mono.error(
new EntityNotFoundException("No category matching " + categorySearchRequest.getSearchTerm() + " was found")));
}
}
Service class:-
#Service
public class CategorySearchServiceImpl implements CategorySearchService {
private String baseUrl = "https://demo0954903.mockable.io";
#Override
public Mono<CategoryResponse> searchCategories() {
WebClient webClient = WebClient.create(baseUrl);
return webClient.
get()
.uri("/category-search")
.retrieve()
.bodyToMono(CategoryResponse.class);
}
}
I found the solution for this issue. I need to add the proxy in the webclient as follows:-
private final WebClient webClient;
ReactorClientHttpConnector connector = new ReactorClientHttpConnector(
options -> options.httpProxy(addressSpec -> {
return addressSpec.host(PROXY_HOST).port(PROXY_PORT);
}));
this.webClient = webClientBuilder.baseUrl(BASE_URL).clientConnector(connector).build();

Resources