I have a Spring boot based application with reactive netty server. The application is running on multiple nodes with VIP in front of the nodes. Client applications connect to VIP and VIP routes to one of the node with hash based routing. VIP is returning 502 occasionally and VIP says, it received 502 response from Application. Application seems to be healthy and not sure why app is returning 502 occasionally and no logs in application logs related to 502. Is there any way to enable netty logs for 502? Any clue on how to narrow down the issue?
Our application implements WebFilter and logs each request and response, but unable to see anything related to 502. Is there anything in spring boot netty to enable more logs to triage 502.
Related
Spring boot apps with Sleuth are deployed in pods with istio sidecar injected alongside.
istio virtual service ingresses traffic into the mesh and proxies the request to pods correctly.
Both logs are seen in kibana - one for istio-proxy (sidecar) and another for the spring app using logback + sleuth.
However I would like to see same traceId (x-request-id) header for both requests. Is this a good practice or is rather what is the right way to join/tie these request logs ?
I did one test and found out that istio-proxy is sending 'x-request-id' which corresponds to the value of traceId. However spring creates a new traceId and wondering if there is a way to make them same ?
This does not block me in any way, just curious. Today during debugging my application I run into a weird logs:
{"#timestamp":"2020-04-17T16:30:55.128+0000","#version":"1","message":"Mapped to Actuator web endpoint 'health'","logger_name":"org.springframework.boot.actuate.endpoint.web.servlet.WebMvcEndpointHandlerMapping","thread_name":"qtp250197932-43","level":"DEBUG","level_value":10000,"caller_class_name":"org.springframework.web.servlet.handler.AbstractHandlerMapping","caller_method_name":"getHandler","caller_file_name":"AbstractHandlerMapping.java","caller_line_number":414}
{"#timestamp":"2020-04-17T16:30:55.128+0000","#version":"1","message":"Mapped to Actuator web endpoint 'health'","logger_name":"org.springframework.boot.actuate.endpoint.web.servlet.WebMvcEndpointHandlerMapping","thread_name":"qtp250197932-45","level":"DEBUG","level_value":10000,"caller_class_name":"org.springframework.web.servlet.handler.AbstractHandlerMapping","caller_method_name":"getHandler","caller_file_name":"AbstractHandlerMapping.java","caller_line_number":414}
Does anyone have an idea why I got two identical records about accessing health endpoint but from different threads?
I have created micro service using spring boot and implment zuul proxy so when I call url via zuul proxy and somehow my service is down I get com.netflix.zuul.exception.ZuulException: Forwarding error I have also implement errorFilter and send custom message from errorFilter but it's not working for me. I get below response:
{
"timestamp": 1537261710692,
"status": 500,
"error": "Internal Server Error",
"exception": "com.netflix.zuul.exception.ZuulException",
"message": "GENERAL"}
This problem occurs when your back-end service is not available but your gateway make proxy calls to unavaiable back-end service.
If you read a log file, you can check it.
DynamicServerListLoadBalancer for client BACKEND-YOUR-SERVICE initialized:
DynamicServerListLoadBalancer:{NFLoadBalancer:name=BACKEND-YOUR-SERVICE,
current list of Servers=[]
It means that there are not available servers of your back-end service.
Please check in the following order.
Check whether your back-end service is running
(If you use Eureka Server) Check whether your back-end service is registered on Eureka server on Eureka webpage.
Check whether your zuul proxy is configured with correct back-end service name.
We are using Spring Boot v2.0.4 + Spring Cloud (Finchley release).
We have deployed Zuul, Auth Server, Eureka, and Config Server, each in their own separate applications / processes. We have half a dozen of our own services deployed. We are using OAuth2 authentication for all services. We are using the embedded undertow container for all apps.
When Zuul uses the embedded Undertow container, we can not log in- the login page is redisplayed even though the credentials are correct. When Zuul is switched to use the embedded Tomcat container, everything works correctly- logging in brings the user to the correct page. (All other apps use Undertow in both cases.)
When we debug the requests coming through Zuul for Tomcat vs Undertow, we see that the SPRING_SECURITY_CONTEXT session attribute is being set for Tomcat but is not being set for Undertow. I assume this means that the user will be redirected back to the login page, which is why we are seeing that behavior.
We know that it is an issue with Zuul + Tomcat vs. Undertow, since the configuration of all other apps remains exactly the same.
Any ideas? We are really scratching our heads over this one.
It turned out this was caused by JSESSIONID cookie name conflict between Zuul and Auth Server. Because Tomcat container has different handling than Wildfly for multiple Set-Cookies of the same name (which is in violation of RFC6265), we are only seeing it now.
I should have mentioned that we configured auth server to be behind the Zuul proxy, hence the multiple JSESSIONID cookies.
We fixed this for now by renaming the JSESSIONID cookie name on all back-end servers. We will investigate the best way to do this for production deployment going forward, perhaps by using Spring Session. Any suggestions welcome.
I'm attempting to establish a discovery server with spring cloud Eureka which needs to connect to a secured client. I understand how to secure the Eureka sever itself - that isn't the issue. The issue is in the other direction - how to get Eureka to successfully communicate with a client service that itself is secured.
In other words; I have a discovery client that registers itself with Eureka. That client implements http basic authentication. It can and does successfully register itself with the discovery service, however when I attempt to utilize that service with a lookup to the discovery service, I get authentication failures (on the client service itself) which of course makes sense because I haven't specified any credentials anywhere and have no idea how to do so. Any assistance would be greatly appreciated.
Guessing from your tags you are using spring-cloud.
When you use your service (with RestTemplate or Feign or manually looking up and interacting with it), your request has nothing to do with Eureka. Eureka only provides you information about your services whereabouts. Once you (or some undelying logic) obtained the address of the service, you are directly communicating with it.