Spring API Gateway java.net.ConnectException: Connection timed out: no further information - spring-boot

I have a spring boot app with simple GET method "sayHello" i have configured API gateway to route calls to my microservice module when uri pattern matches /ms1/**
Both gateway and microservice registered to the Registry service and dashboard displays both registered
API-GATEAY YAML configuration
server:
port: '8010'
spring:
application:
name: MY-GATEWAY
cloud:
compatibility-verifier:
enabled: false
gateway:
routes:
- id: MICRO-SERVICE1
uri: lb://MICRO-SERVICE1
predicates:
- Path=/ms1/**
eureka:
client:
register-with-eureka: true
fetch-registry: true
service-uri: http://localhost:8761/eureka/
intance:
hostname: localhost
MICRO-SERVICE1 YAML configuration
server:
port: '8030'
spring:
application:
name: MICRO-SERVICE1
eureka:
client:
register-with-eureka: true
fetch-registry: true
service-uri: http://localhost:8761/eureka/
intance:
hostname: localhost
Controller's GET method
#RestController
#RequestMapping("/ms1")
public class UATController {
#GetMapping(value = "/sayHello")
public #ResponseBody ResponseEntity<String> sayHello() {
String message = "Hello World";
return new ResponseEntity<String>(message, HttpStatus.OK);
}
}
I am getting following error when i access via gateway, however when i access my microservice directly it works any idea where i am wrong here?
http://localhost:8010/ms1/sayHello <- Not working via gateway
http://localhost:8030/ms1/sayHello <- Working direct access
Wed Dec 07 23:32:31 EST 2021
[83b324d4-2] There was an unexpected error (type=Internal Server Error, status=500).
Connection timed out: no further information: DESKTOP-1.verizon.net/192.168.1.160:8030
io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection timed out: no further information: DESKTOP-1.verizon.net/192.168.1.160:8030
Suppressed: The stacktrace has been enhanced by Reactor, refer to additional information below:
Error has been observed at the following site(s):
*__checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ HTTP GET "/ms1/sayHello" [ExceptionHandlingWebHandler]
Original Stack Trace:
Caused by: java.net.ConnectException: Connection timed out: no further information
at java.base/sun.nio.ch.Net.pollConnect(Native Method)
at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:707)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)

I have faced the same issue today as well when running the micro services, the gateway and the Eureka discovery all on localhost.
I was able to solve it by adding eureka.instance.preferIpAddress=true to my application.properties of the micro services. That property causes that the services will advertise their IP addresses instead of the host names. More info here under point 2.6: https://cloud.spring.io/spring-cloud-netflix/multi/multi_spring-cloud-eureka-server.html. My guess is that there was an issue with forwarding the requests to the microservices due to using localhost.
Edit: Here are my settings
There is nothing special in my setup. These are the values I am using in my application.properties
The microservice is using:
spring.cloud.discovery.enabled=true
spring.application.name=[The name of my app]
eureka.client.register.with-eureka=true
eureka.client.service-url.defaultZone=http://localhost:8999/eureka
eureka.instance.prefer-ip-address=true
The properties for the gateway are similar to the service one.
The load balancing part looks like this:
spring.cloud.gateway.routes[0].id=[Same name as my app]
spring.cloud.gateway.routes[0].uri=lb://[name of my app]
spring.cloud.gateway.routes[0].predicates=Path=/test/**
Where test/** is the part of the Rest URL of my application

Adding DESKTOP-1.verizon.net to hosts file worked, following is entry I have added to C:\Windows\System32\drivers\etc\hosts file. Unfortunately eureka.instance.prefer-ip-address=true didn't work for me
127.0.0.1 DESKTOP-1.verizon.net

have added this settings in yml file in all services and its work for me.
eureka:
instance:
prefer-ip-address: true

I just faced the same issue, and in my case, I had a problem with port number. I was using the code below to set up the port for me:
server:
port: ${random.int(8010,8013)}
I noticed when the service starts, even though the log shows "Tomcat started on port(s): 8010", on Eureka's page shows "host.docker.internal:my-service:8011"
This error happens randomly, because sometimes when you re-start the microservice, it is able to register on Eureka correctly.
You can do a simple test using a non-random port number.
I haven't figured out how to solve it for random port number.
UPDATING:
I did this and worked fine for me:
server:
port: 0
spring:
application:
name: user-service
eureka:
instance:
hostname: localhost
prefer-ip-address: true
client:
serviceUrl:
defaultZone: http://localhost:9000/eureka/

Related

spring cloud gateway + eureka + microservice on heroku is putting the port in the url result is error 503 on request to microservice via gateway

I have 3 services deployed to Heroku.
Spring cloud Gateway
Eureka
a Microservice
The Gateway and the Microservice are registered in eureka.
GATEWAY application.yml
server:
port: ${PORT:8080}
eureka:
instance:
hostname: gateway.herokuapp.com
homePageUrl: https://${eureka.instance.hostName}/
client:
serviceUrl:
defaultZone: http://eureka.herokuapp.com/eureka/
spring:
application:
name: gateway-service
cloud:
gateway:
routes:
- id: employeeModule
uri: lb://MICROSERVICE
predicates:
- Path=/employee/**
logging:
level:
org.springframework.cloud.gateway: TRACE
EUREKA application.yml
server:
port: ${PORT:8761}
eureka:
instance:
hostname: ${EUREKA_HOSTNAME:localhost}
client:
register-with-eureka: false
fetch-registry: false
service-url:
defaultZone: ${EUREKA_URI:http://${eureka.instance.hostname}:${server.port}/eureka/}
MICROSERVICE application.yml
spring:
application:
name: MICROSERVICE
server:
port: ${PORT:8081}
eureka:
instance:
hostname: microservice.herokuapp.com
homePageUrl: https://${eureka.instance.hostName}/
home-page-url-path: https://${eureka.instance.hostName}
client:
serviceUrl:
defaultZone: http://eureka.herokuapp.com/eureka/
I am getting the correct response, when i am calling directly https://MICROSERVICEURL/employee/message
when I am calling the gateway https://GATEWAYURL/employee/message it should ask eureka for the address of the microservice and then route the request to the microservice to get the response.
But instead of routing to https://MICROSERVICEURL/employee/message it routes to the eureka version with the random heroku port. e.g.: https://MICROSERVICEURL:RANDOMHEROKUPORT/employee/message. This leads to an error 503 because of a timeout.
Is there a way to tell the gateway to ignore the port number?
Or am I doing something else wrong?
GATEWAY Log
2020-08-18T19:59:56.683925+00:00 app[web.1]: 2020-08-18 19:59:56.683 TRACE 4 --- [or-http-epoll-1] o.s.c.g.filter.LoadBalancerClientFilter : LoadBalancerClientFilter url chosen: http://microservice.herokuapp.com:41632/employee/message
2020-08-18T20:00:26.673252+00:00 heroku[router]: at=error code=H12 desc="Request timeout" method=GET path="/employee/message" host=gateway.herokuapp.com request_id=236faa79-1d66-4e30-8c32-f879228d08db fwd="79.205.56.29" dyno=web.1 connect=1ms service=30001ms status=503 bytes=0 protocol=https
The solution was setting the secure and non secure port for the eureka instance in the appllication.yml
MICROSERVICE application.yml
spring:
application:
name: MICROSERVICE
server:
port: ${PORT:8081}
eureka:
instance:
hostname: microservice.herokuapp.com
homePageUrl: https://${eureka.instance.hostName}/
home-page-url-path: https://${eureka.instance.hostName}
client:
serviceUrl:
defaultZone: http://eureka.herokuapp.com/eureka/
non-secure-port: 80
secure-port: 443
You just need to add the non-secure port and make sure it is enabled in the application.properties or .yml
eureka.instance.non-secure-port-enabled=true
eureka.instance.secure-port-enabled=false
eureka.instance.non-secure-port=80
you could also run it on the secure port but then you can only run 1 service.
My application.properties

Could not connect to eureka server from docker

I'm trying connection to eureka server from docker container.
The eureka server is running via "java -jar eureka-server.jar" command.
The eureka client is running on docker container.
Eureka server configuration is:
eureka:
instance:
hostname: eurekaserver
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: http://eurekaserver:8080/eureka/
Eureka client configuration is:
spring:
application:
name: client-application
server:
port: 8081
eureka:
client:
serviceUrl:
defaultZone: http://eurekaserver:8080/eureka/
I also tried to setup directly ip-address for example
eureka:
client:
serviceUrl:
defaultZone: http://1.2.3.4:8080/eureka/
But, unfortunately, I'm facing with stack trace:
[2019-05-26 09:44:14.714] slf4j - 1 INFO [main] --- o.a.h.i.c.DefaultHttpClient: Retrying connect to {}-> http://eurekaserver:8080
[2019-05-26 09:44:15.717] slf4j - 1 ERROR [main] --- c.n.d.s.t.d.RedirectingEurekaHttpClient: Request execution error
com.sun.jersey.api.client.ClientHandlerException: java.net.NoRouteToHostException: No route to host (Host unreachable)
at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:187)
at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123)
at com.netflix.discovery.EurekaIdentityHeaderFilter.handle(EurekaIdentityHeaderFilter.java:27)
at com.sun.jersey.api.client.Client.handle(Client.java:652)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509)
...
Caused by: java.net.NoRouteToHostException: No route to host (Host unreachable)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:144)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:134)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:117)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:173)
... 72 common frames omitted
And it is cause of
com.netflix.discovery.shared.transport.TransportException: Cannot execute request on any known server
So my question is how to resolve connectivity?
Docker and eureka are placed on the same machine, because the command curl "http://1.2.3.4:8080/eureka/apps" returns app to me
How can I do some investigating, why a docker cannot resolve the host?
p.s. I've known about issue, when eureka-client looks at localhost hostname, but looks like my issue another.
Thank you.
Resolved.
The cause of the issue was, don't use
registerWithEureka: true
fetchRegistry: true
More about, find at first links after typing properties name in google
But if you use them, then you have to set serviceUrl as hostname and also add
instance.preferUseIpAddress: true
p.s. Not sure that is fully can solve this problem, but that, I have changed at application config.

Config server and eureka server in same application: tries to connect to localhost:8761

I have a spring-boot application which I use to setup a spring cloud config server and a eureka server in development and testing environments.
Strangely the application always tries to connect to localhost:8761, even though I have eureka.client.registerWithEureka set to false.
How can I deactivate this?
The error:
ERROR 3144 --- [et_localhost-12] c.n.e.cluster.ReplicationTaskProcessor : Network level connection to peer localhost; retrying after delay
com.sun.jersey.api.client.ClientHandlerException: org.apache.http.conn.ConnectTimeoutException: Connect to localhost:8761 timed out
at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:187) ~[jersey-apache-client4-1.19.1.jar:1.19.1]
at com.netflix.eureka.cluster.DynamicGZIPContentEncodingFilter.handle(DynamicGZIPContentEncodingFilter.java:48) ~[eureka-core-1.4.10.jar:1.4.10]
at com.netflix.discovery.EurekaIdentityHeaderFilter.handle(EurekaIdentityHeaderFilter.java:27) ~[eureka-client-1.4.10.jar:1.4.10]
at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.1.jar:1.19.1]
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) ~[jersey-client-1.19.1.jar:1.19.1]
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) ~[jersey-client-1.19.1.jar:1.19.1]
at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:570) ~[jersey-client-1.19.1.jar:1.19.1]
at com.netflix.eureka.transport.JerseyReplicationClient.submitBatchUpdates(JerseyReplicationClient.java:116) ~[eureka-core-1.4.10.jar:1.4.10]
at com.netflix.eureka.cluster.ReplicationTaskProcessor.process(ReplicationTaskProcessor.java:71) ~[eureka-core-1.4.10.jar:1.4.10]
at com.netflix.eureka.util.batcher.TaskExecutors$BatchWorkerRunnable.run(TaskExecutors.java:187) [eureka-core-1.4.10.jar:1.4.10]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_92]
Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to localhost:8761 timed out
My only class looks like this:
#EnableEurekaServer
#EnableConfigServer
#SpringBootApplication
public class MyApplication {
public static void main(String[] args) {
SpringApplication.run(MyApplication.class, args);
}
}
The application.yml:
server:
port: 8888
eureka:
client:
registerWithEureka: false
fetchRegistry: false
server:
waitTimeInMsWhenSyncEmpty: 0
renewal-percent-threshold: 0.49
security:
basic:
enabled: true
user:
password: mypassword
spring:
jmx:
default-domain: ${spring.application.name}
cloud:
config:
server:
git:
uri: https://example.com/myrepo.git
username: username
password: password
clone-on-start: true
search-paths: '{application},{application}/{profile}'
endpoints:
jmx:
domain: ${spring.application.name}
unique-names: true
In the bootstrap.yml I have only the application name set.
Versions:
spring-cloud-netflix-eureka: 1.1.6,
spring-cloud-config-server: 1.1.3
Could you try to change your configuration like below
eureka:
client:
registerWithEureka: false
fetchRegistry: false
service-url:
defaultZone: http://localhost:8888/eureka
You specify server.port: 8888. So your eureka is running on 8888 port. But you didn't specify any service-url for eureka. So I think that your eureka server is trying to replicate to localhost:8761 because it's default and you didn't specify service-url for eureka.
For me, below properties that worked for me.
eureka.client.registerWithEureka= false
eureka.client.fetchRegistry= false
eureka.server.maxThreadsForPeerReplication=0
Please Use 8761 port for your Eureka server as below.
server:
port: 8761
eureka:
client:
registerWithEureka: false
fetchRegistry: false
Here we’re configuring an application port – 8761 is the default one for Eureka servers. We are telling the built-in Eureka Client not to register with ‘itself’, because our application should be acting as a server.
It seems that eureka tries to connect to itself despite the below settings:
eureka.client.register-with-eureka=false
eureka.client.fetch-registry=false
you have two options to fix that 1-Keep the default port of eureka 2-Add below to your application.properties of your eureka server eureka.server.maxThreadsForPeerReplication=0

ZuulException: forward error with Zuul-Eureka

I'm facing a ZuulException: forward error when routing with Zuul and Eureka.
The error not occur during the first minutes but after 1 or 2 mn I get this weird error.
I'm using spring boot 1.4 and spring cloud Camden
If you want to reproduce the error or see my project: https://github.com/Seb69/Spring-demo-ZuulException/tree/master
Eureka config:
server:
port: 9999
spring:
application:
name: eureka-server
eureka:
instance:
hostname: localhost
client:
registerWithEureka: false
fetchRegistry: false
serviceUrl:
defaultZone: http://${eureka.instance.hostname}:${server.port}/eureka/
Service-Gateway config (Zuul):
server:
port: 1111
spring:
application:
name: service-gateway
# ZUUL (Load balancing)
zuul:
ignoredServices: '*'
routes:
service-server:
stripPrefix: true
path: /api/**
serviceId: SERVICE-SERVER
# EUREKA (Service registry)
eureka:
instance:
leaseRenewalIntervalInSeconds: 1
leaseExpirationDurationInSeconds: 2
client:
serviceUrl:
defaultZone: http://localhost:9999/eureka/
Service config:
server:
port: 8095
spring:
application:
name: service-server
eureka:
instance:
leaseRenewalIntervalInSeconds: 1
leaseExpirationDurationInSeconds: 2
client:
serviceUrl:
defaultZone: http://localhost:9999/eureka/
Here is a short version of my stack trace:
com.netflix.zuul.exception.ZuulException: Forwarding error
Caused by: com.netflix.client.ClientException: Number of retries on
next server exceeded max 1 retries, while making a call for:
mbp-de-andre:8095
Caused by: java.net.UnknownHostException: mbp-de-andre
FIX:
So, I finaly succeed to make it work !
My problem came from my Macbook Hostname,
Last hostname was: mbp-de-andre
I modify it and I set: `MacBook-Pro-de-ANDRE.local``
Just want to say another fix might be missing dependency:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-zuul</artifactId>
</dependency>

Zuul -> Eureka Server, Basic Authentication issue

I am able to hit the service, if the flow doesn't contain Basic Authorization.
If i use Basic Authorization, it throws "message": "Full authentication is required to access this resource"
Below are my observations:
In ZuulFilter, run() method, i get value for
request.getHeader("Authorization") --> Basic c29tOnNvbzz==
but once it reaches the Micro Service, i am getting value as 'null',
request.getHeader("Authorization") --> null
Using Spring Boot version : 1.4.0.RELEASE
This is my flow:
------------------
Zuul -> Service Discovery (Eureka Server) -> Service
Kindly help, not sure where the Authorization header is vanishing.
Eureka Server yml file:
-------------------------
server.port:4001
eureka.instance.hostname=localhost
eureka.client.fetch-registry:false
eureka.client.register-with-eureka:false
eureka.client.serviceUrl.defaultZone: http://${eureka.instance.hostname}:${server.port}/eureka/
eureka.client.healthcheck.enabled=true
Zuul yml file:
-----------------
server:
port: 8765
info:
component: Edge Server
eureka:
instance:
leaseRenewalIntervalInSeconds: 3
metadataMap:
instanceId: ${spring.application.name}:${random.value}
client:
# Default values comes from org.springframework.cloud.netflix.eurek.EurekaClientConfigBean
registryFetchIntervalSeconds: 5
instanceInfoReplicationIntervalSeconds: 5
initialInstanceInfoReplicationIntervalSeconds: 5
endpoints:
restart:
enabled: true
shutdown:
enabled: true
health:
sensitive: false
zuul.sensitive-headers: Cookie,Set-Cookie,Authorization
logging:
level:
ROOT: WARN
se.callista: INFO
# Get info regarding connection to the cofig server and retries if required
org.springframework.cloud.config.client.ConfigServicePropertySourceLocator: INFO
org.springframework.retry.support.RetryTemplate: DEBUG
# Set INFO to see the allocated port
org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainer: INFO
---
eureka:
instance:
preferIpAddress: true
client:
serviceUrl:
defaultZone: http://localhost:4001/eureka,http://localhost:4002/eureka
Authorization is by default a sensitive header, this means Zuul will not forward them. If you leave it out of the sensitive headers, Zuul will forward the header.
zuul.sensitiveHeaders: Cookie,Set-Cookie
It should also be camelCase instead of hyphenated.
Extra info: https://github.com/spring-cloud/spring-cloud-netflix/blob/master/docs/src/main/asciidoc/spring-cloud-netflix.adoc#cookies-and-sensitive-headers
This solved my problem, but is this the only solution we have ?
ctx.addZuulRequestHeader("Authorization",request.getHeader("‌​Authorization"))
Your property for zuul.sensitiveHeaders is wrong. It is camel case not hyphenated.
https://github.com/spring-cloud/spring-cloud-netflix/blob/master/docs/src/main/asciidoc/spring-cloud-netflix.adoc#cookies-and-sensitive-headers

Resources