I just created simple Zuul Proxy at the front end for our microservices environment but now I wanted to log all the entries into the log file which went through the proxy.
Do any properly which I need to enable to do this.
I assume an implementation of zuul as a regular spring boot driven microservice with a bunch of netflix's beans running under the hood.
In this case it can run on tomcat (probably for other services the idea is the same, but the technical implementation might be different).
So for tomcat:
As a first resort you can take advantage of tomcat feature of "access logs" that logs all the requests anyway. It also allows some level of customizations (what to log). The technical difficulty is that tomcat access log is not by default managed by logback, so you'll have to use some kind of adapter.
Here you can find ideas of how to resolve this technically and integrate access log with logback.
An another approach would be creating a Filter that will extract required pieces and log the request / response / whatever you want to log
Here is an example of creating a custom filter like this.
Of course I you also need to log something from response you should configure the filter type (see the java code example in the link)
One tip / caution: think about performance implications, so that this feature won't slow down the processing if the server is under high load of requests.
Related
I have recently changed my Quarkus application from RestEasy to Reactive Routes to implement my HTTP endpoints.
My Quarkus app had OpenTracing enabled and it was working fine. After changing the HTTP resource layer I can not see any trace in Jaeger.
After setting log level in DEBUG I can see my application is registered in Jaeger but I don't see any traceId or spanId in logs neither traces in Jaeger:
15:44:36 DEBUG traceId=, spanId=, sampled= [io.qu.ja.ru.JaegerDeploymentRecorder] (main) Registering tracer to GlobalTracer JaegerTracer(version=Java-0.34.3, serviceName=employee, reporter=RemoteReporter(sender=HttpSender(), closeEnqueueTimeout=1000), sampler=ConstSampler(decision=true, tags={sampler.type=const, sampler.param=true}), tags={hostname=employee-8569585469-tg8wg, jaeger.version=Java-0.34.3, ip=10.244.0.21}, zipkinSharedRpcSpan=false, expandExceptionLogs=false, useTraceId128Bit=false)
15:45:03 INFO traceId=, spanId=, sampled= [or.se.po.re.EmployeeResource] (vert.x-eventloop-thread-0) getEmployees
I'm using the latest version of Quarkus which is 1.9.2.Final.
Is it enabled OpenTracing when I'm using Reactive Routes?
Tracing is enabled by default for JAX-RS endpoints only, not for reactive routes at the moment. You can activate tracing by annotating your route with #org.eclipse.microprofile.opentracing.Traced.
Yes, adding #Traced enable to activate tracing on reactive routes.
Unfortunately, using both JAX-RS reactive and reactive routes bugs the tracing on event-loop threads used by JAX-RS reactive endpoint when they get executed.
I only started Quarkus 2 days ago so i don't really the reason of this behavior (and whether it's normal or it's a bug), but obviously switching between two completely mess up the tracing.
Here is an example to easily reproduce it:
Create a REST Easy reactive endpoint returning an empty Multi
Create a custom reactive route
set up the IO threads to 2 (easier to quickly reproduce it)
Run the application, and request the two endpoints alternatively
Here is a screenshot that show the issue
As you can see, as soon as the JAX-RS resource is it and executed on one of the two threads available, it "corrupts" it, messing the trace_id reported (i don't know if it's the generation or the reporting on logs that is broken) on logs for the next calls of the reactive route.
This does not happen on the JAX-RS resource, as you can notice on the screenshot as well. So it seems to be related to reactive routes only.
Another point here is the fact that JAX-RS Reactive resources are incorrectly reported on Jaeger. (with a mention to a missing root span) Not sure if it's related to the issue but that's also another annoying point.
I'm thinking to completely remove the JAX-RS Reactive endpoint and replace them by normal reactive route to eliminate this bug.
I would appreciate if someone with more experience than me could verify this or tell me what i did wrong :)
EDIT 1: I added a route filter with priority 500 to clear the MDC and the bug is still there, so definitely not coming from MDC.
EDIT 2: I opened a bug report on Quarkus
EDIT 3: It seems related to how both implementations works (thread locals versus context propagation in actor based context)
So, unless JAX-RS reactive resources are marked #Blocking (and get executed in a separated thread pool), JAX-RS reactive and Vertx reactive routes are incompatible when it comes to tracing (but also probably the same for MDC related informations since MDC is also thread related)
I am using Zuul as an api-gateway in a spring-cloud micro-service app, so that every access to api-gateway/some-service/a_route is redirected to /a_route in a generic way (the discovery is backed by consul).
I am trying to use Jaeger to instrument this system. And at this point I am using opentracing-spring-web-autoconfigure, because I cannot upgrade my spring boot/cloud version easily (I am using1.4.5.RELEASE Camden.SR7). So I just added the dependency, created the Jaeger tracer and redirect it to the docker all in one collector.
I have begin by instrumenting the gateway and It somewhat works => It generate span on the gateway, but all the route are marked :
apigateway-service: GET
and there is no information concerning the forwarded route at this level, the full route itself is store in a tag : http.url
"http://localhost:8080/collection-service/collections/projects/"
To be useful I would prefer to have span named :
apigateway-service: GET collection-service/collections/projects/
Can this be configured somewhere ?
No, it cannot, but it wouldn't hurt to open an issue there with this suggestion.
the technology stack in our company are:
Java, Spring MVC, Spring Boot, Jaxws etc..
and we provide webservices for the client to querying our services.
in terms of securing the SOAP service. some of the webservices uses spring OAuth security and some of them uses the Spring Basic Auth
recently one of the client flooded our server by sending huge amount of request in the short period of time.
we are going to implement something to provent this to happen. ideally a
per client based calling interval. which can recognize the high calling frequency. then ban the client or force the client to wait
before we code this from the scratch, I wonder if there are libraries we can reuse. Spring normally very good at providing solutions for most of the enterprise issues. but so far I have't found any thing. any hint, ideally a working sample. would be great!
EDIT1: ideally we want to implement this instead of fully rely on the HTTP server e.g tomcat or apache to handle this. because our own implementation would offer more fine grained rules, such as how long the interval should be,
what kind of customised message we can return, more important we can implement our own monitoring mechanism, and treating different client with different traffic allowance etc...
We are in the process of standing up a new microservices architecture with Zuul at the front-end and a bunch of tomcat enabled microservices at the backend. Each service as it starts up, will register itself with Eureka and any client that wants to call those service will do so through Zuul. We've got this all wired in and everything is working fine.
However, I have a couple questions as to how we can make this architecture much more dynamic.
One thing that we assumed was there out of the box with Ribbon/Eureka, but have yet to find a solution for is that as we add more services to the backend, that somehow (via Archiaus and update to Zuul's eureka-client.properties file) Zuul's Ribbon client would update itself with the new service details (e.g. vipaddress, load balancing algorithm, etc). So far, the only thing that works is to update the properties file and restart Zuul (ughhh).
For example, let's say today we have 2 microservices at the backend, therefore, Zuul's eureka/ribbon client configuration would include the below:
ribbon.client.niws.clientlist=service1|service2
zuul.ribbon.namespace=zuul.client
service1.zuul.client.DeploymentContextBasedVipAddresses=myService1
service1.zuul.client.NIWSServerListClassName=com.netflix.niws.loadbalancer.DiscoveryEnabledNIWSServerList
service2.zuul.client.DeploymentContextBasedVipAddresses=myService2
service2.zuul.client.NIWSServerListClassName=com.netflix.niws.loadbalancer.DiscoveryEnabledNIWSServerList
Now tomorrow, let's assume we need to add service3. What we have observed is that if we add those details to the same configuration (see below), they only become available to Zuul after a restart. Is there some other configuration parameter we are missing that would allow us to dynamically introduce the new service details or do we have to roll our own Eureka/Ribbon client to do this?
ribbon.client.niws.clientlist=service1|service2|service3
zuul.ribbon.namespace=zuul.client
service1.zuul.client.DeploymentContextBasedVipAddresses=myService1
service1.zuul.client.NIWSServerListClassName=com.netflix.niws.loadbalancer.DiscoveryEnabledNIWSServerList
service2.zuul.client.DeploymentContextBasedVipAddresses=myService2
service2.zuul.client.NIWSServerListClassName=com.netflix.niws.loadbalancer.DiscoveryEnabledNIWSServerList
service3.zuul.client.DeploymentContextBasedVipAddresses=myService3
service3.zuul.client.NIWSServerListClassName=com.netflix.niws.loadbalancer.DiscoveryEnabledNIWSServerList
My other question is related and that is do we really need to add a client configuration (in eureka-client.properties) for every service that Zuul could possibly route to? At some point, we may have 100's of services running and trying to maintain all the related client configurations in Zuul seems a bit clumsy. Is there a way to globally configure Zuul to load all services into its client list from Eureka (or based on some service metadata in Eureka) and dynamically update this list as new services register themselves with Eureka?
Thanks!
The issue is with namespaces.If we use the default namespace it should be able to pick up the new properties addedd by default.
My organization is using Websphere Application Server with RAD. My unit is developing Web Services that take data from a consumer, and often pass the data through other Web Services.
Currently, we have two ways of defining time-out's for data to be returned from called services:
Using the Spring framework
Websphere profile configuration
If we need to update the values using Spring we need to re-deploy our service. If we define them as JVM properties we need to do a JVM configuration change and 'restart/recycle/stop and start' the application.
My co-worker had it suggested to him that there may be some better solutions to this problem. I'm wondering if this is true, and if so what they are?
edit:
One option we might be considering is "Application Policy Sets". We'd like to know if this is a good alternative method:
can we define Spring to leverage policy sets
can we define multiple policy sets for an application when timeout values for different services called by our service are different?
Does this console change take affect at run-time, or do we need to recycle?
I suggest you use JAX-WS policy sets and bindings on WAS level, because it allows you to:
define policy set for each web service separately
in policy set define web service parameters, from HTTP transport parameters to WS-Security policies such as UsernameToken, digital signature and encryption
web services remain unchanged, the code remains the same
there is only one place where you change it: in WAS console, application is not changed
The only drawback here is that web services should be generated from RAD (you can use top-down or bottom-up approach).
Please see more details in this excellent 3-part DeveloperWorks tutorial