I am working in a Spring boot project and deploy on 3 nodes docker swarm worker as a replicate, When goes under heavy load after 150 request it's get 504 gateway timeout and didn't response until reload. after reload container everything it we'll be OK.
Rest Template config :
#Bean("restTemplate")
public RestTemplate createRestTemplate(RestTemplateBuilder builder) {
log.info("RestTemplate..");
return builder
.setConnectTimeout(Duration.ofMillis(5000))
.setReadTimeout(Duration.ofMillis(40000))
.build();
}
tomcat webservice config in application.yml :
tomcat:
max-connections: 2000
connection-timeout: 600s
threads:
max: 100000 jetty:
connection-idle-timeout: 10s
how I can resole this problem ?
Related
I have two microservices deployed and a eureka server. The microservices are eureka clients, viz, eureka-client-1 and eureka-client-2 respectively.
Using, service discovery, I want to call an API from eureka-client-1 of eureka-client-2.
Following is the supported code and configurations:
discovery-server (eureka server) configuration:
server:
port: 8761
eureka:
client:
fetch-registry: false
register-with-eureka: false
spring:
application:
name: discovery-server
eureka-client-1 and eureka-client-2 have same configurations:
eureka:
client:
service-url:
defaultZone: http://localhost:8761/eureka
In eureka-client-1 I am using RestTemplate to communicate with the eureka-client-2
Notice that I have currently commented the LoadBalanced annotation.
#Bean
// #LoadBalanced
public RestTemplate restTemplate(){
return new RestTemplate();
}
I access the api of eureka-client-2 as follows:
restTemplate.getForObject(
"http://EUREKA-CLIENT-2/api/v1/check/{customerId}", Response.class, customer.getId());
When I deploy and test the application, I am getting UnknownHostException for EUREKA-CLIENT-2. But, as soon as I enable the annotation #LoadBalanced everything works fine and eureka-client-1 can successfully communicate with eureka-client-2.
My question is: Why #LoadBalanced annotation is absolutely compulsory? If I have deployed only single instances of each microservices, why can't the discovery-server return the information of the only service with given name that is deployed? And even if same service is deployed multiple times, why it can't internally apply the loadbalancing algorithm? Why the client has to specifically add the annotation
I have this scenario where I have an Eureka server, a Zuul proxy and the eureka's clients (some API).
The thing is that when I call the Zuul server it does discover the "service" on Eureka but it returns a 504 status time out.
{
"timestamp":"2020-07-21T18:38:46.408+00:00",
"status":504,
"error":"Gateway Timeout",
"message":""
}
In my application.properties I have:
spring.application.name=zuul-server
eureka.instance.preferIpAddress=true
eureka.client.registerWithEureka=true
eureka.client.fetchRegistry=true
eureka.client.healthcheck.enabled=true
zuul.ignored-services=*
zuul.host.time-to-live=-1
zuul.host.connect-timeout-millis=15000
zuul.host.max-per-route-connections=10000
zuul.host.max-total-connections=5000
zuul.host.socket-timeout-millis=60000
zuul.semaphore.max-semaphores=500
I can't figure it out what's wrong to cause this timeout.
Any helping tips?
For those who eventually face this same problem here is how I solved it.
This is how my final application.properties on the Zuul project is set and working:
spring.application.name=zuul-server
eureka.instance.preferIpAddress=true
eureka.client.registerWithEureka=true
eureka.client.fetchRegistry=true
eureka.client.healthcheck.enabled=true
#zuul.ignored-services=*
#zuul.host.time-to-live=-1
zuul.host.connect-timeout-millis=60000
zuul.host.max-per-route-connections=10000
zuul.host.max-total-connections=5000
zuul.host.socket-timeout-millis=60000
zuul.semaphore.max-semaphores=500
zuul.ribbon.eager-load.enabled= true
hystrix.command.default.execution.isolation.strategy=THREAD
hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds=40000
ribbon.ConnectTimeout=10000
ribbon.ReadTimeout: 10000
If your IDE complains about "unknown property" just ignore it.
I have a spring boot app and i am trying to use feign client to make a request to an endpoint but its not able to resolve feign client name from application yml.
My application yml has following code :
ribbon:
eureka:
enabled: false
beacon:
ribbon:
isSecure: true
listOfServers: https://beacon.org
My interface look like this :
#FeignClient("beacon")
public interface BeaconClient {
#GetMapping("/api/organizations")
List<Org> getOrgs();
}
I am getting an error 500 internal server error because i see that it making a request to http://beacon/api/organizations endpoint instead of https://beacon.org/api/organizations
Your code is correct. The default feign Logger is misleading. http://beacon/.. that you see in the log isn't indicative of actual url that is being called. The actual request is resolved with https://beacon.org/...
We are using Spring Boot 2.0.0.RELEASE with spring-cloud-starter-netflix-ribbon for our micro services. I set ribbon.readTimeout=1000 for slow requests and check it with our micro service setting breakpoint inside #GetMapping method without sending a response. In my test I have been waiting for 10 minutes and did not get any exception. It seems like there is no readTimeout at all.
Service configuration
ribbon:
ReadTimeout: 1000
my-service:
ribbon:
eureka:
enabled: false
listOfServers: localhost:8080
ReadTimeout: 1000
ConnectTimeout: 1000
The only way a can make it work is ribbon.restclient.enabled=true. But this client is deprecated and I don't wont to use it.
Not all the ribbon properties are supported by spring-cloud-netflix while being used with a Spring RestTemplate. There are some ribbon properties that work, as described in the docs, but ReadTimeout is not one of them. So it does not work, but it's by desing (as per the response to this issue). However, if you are using Spring's RestTemplate, you can set it there directly, like so:
#LoadBalanced
#Bean
RestTemplate restTemplate(RestTemplateBuilder restTemplateBuilder) {
return restTemplateBuilder
.setReadTimeout(2000)
.build();
}
I think you need to configure Hystrix timeouts. Take a look at this part of the documentation : http://cloud.spring.io/spring-cloud-static/Edgware.RELEASE/single/spring-cloud.html#_hystrix_timeouts_and_ribbon_clients
It could be something like that :
hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds: 1100
ribbon:
ConnectTimeout: 1000
ReadTimeout: 1000
We find this:
serviceA.ribbon.ReadTimeout=8000
work well with spring boot 2.1.0.RELEASE using spring cloud Finchley.SR2 and
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
However, we use ribbon via feign, in a client as so:
#FeignClient(value = "serviceA")
public interface ServiceAClient {
#GetMapping(value = "/test")
String getTest();
}
We then use a wiremock test to introduce a fixed delay above the read timeout to verify it is working fine.
According to this blog https://spring.io/blog/2015/07/14/microservices-with-spring
Was able to run the application without any issues. In this order:
java -jar microservice-demo-0.0.1-SNAPSHOT.jar registration 1111
java -jar microservice-demo-0.0.1-SNAPSHOT.jar accounts 2222
java -jar microservice-demo-0.0.1-SNAPSHOT.jar web 3333
But when trying to hit any service through the web application (http://localhost:3333/) which uses the http://ACCOUNTS-SERVICE url to access any accounts service endpoints like http://ACCOUNTS-SERVICE/accounts/123456789 I'm getting an error response:
Response Status: 500 (Internal Server Error)
Cause: org.springframework.web.client.ResourceAccessException I/O error on GET request for "http://ACCOUNTS-SERVICE/accounts/123456789": ACCOUNTS-SERVICE; nested exception is java.net.UnknownHostException: ACCOUNTS-SERVICE
When I provide the real address (http://localhost:2223/) of the accounts service to the web server instead of the http://ACCOUNTS-SERVICE everything works properly but there is no service discovery in this case.
The source code is stored at: https://github.com/paulc4/microservices-demo
This issue was due to the RestTemplate was no longer auto-created in the Brixton release-train (Spring Cloud 1.1.0.RELEASE), so the RestTemplate could not resolve properly the http://ACCOUNTS-SERVICE url using the service discovery server.
Was able to fix this issue after declaring a RestTemplate bean with #LoadBalanced as follows:
#Bean
#LoadBalanced
public RestTemplate restTemplate() {
return new RestTemplate();
}