I have a distributed process which runs across two different servers (A and B) and I get two different log files A.log and B.log, I need this merged into a single
file.
I have referred to following links but I am unable to get a merged file from the same:
https://github.com/spring-cloud/spring-cloud-sleuth
https://howtodoinjava.com/spring-cloud/spring-cloud-zipkin-sleuth-tutorial/
Is there something that I am missing?
Edit: I need the logs in something along these lines
service1.log:2016-02-26 11:15:47.561 INFO [service1,2485ec27856c56f4,2485ec27856c56f4,true] 68058 --- [nio-8081-exec-1] i.s.c.sleuth.docs.service1.Application : Hello from service1. Calling service2
service2.log:2016-02-26 11:15:47.710 INFO [service2,2485ec27856c56f4,9aa10ee6fbde75fa,true] 68059 --- [nio-8082-exec-1] i.s.c.sleuth.docs.service2.Application : Hello from service2. Calling service3 and then service4
service3.log:2016-02-26 11:15:47.895 INFO [service3,2485ec27856c56f4,1210be13194bfe5,true] 68060 --- [nio-8083-exec-1] i.s.c.sleuth.docs.service3.Application : Hello from service3
service2.log:2016-02-26 11:15:47.924 INFO [service2,2485ec27856c56f4,9aa10ee6fbde75fa,true] 68059 --- [nio-8082-exec-1] i.s.c.sleuth.docs.service2.Application : Got response from service3 [Hello from service3]
service4.log:2016-02-26 11:15:48.134 INFO [service4,2485ec27856c56f4,1b1845262ffba49d,true] 68061 --- [nio-8084-exec-1] i.s.c.sleuth.docs.service4.Application : Hello from service4
service2.log:2016-02-26 11:15:48.156 INFO [service2,2485ec27856c56f4,9aa10ee6fbde75fa,true] 68059 --- [nio-8082-exec-1] i.s.c.sleuth.docs.service2.Application : Got response from service4 [Hello from service4]
service1.log:2016-02-26 11:15:48.182 INFO [service1,2485ec27856c56f4,2485ec27856c56f4,true] 68058 --- [nio-8081-exec-1] i.s.c.sleuth.docs.service1.Application : Got response from service2 [Hello from service2, response from service3 [Hello from service3] and from service4 [Hello from service4]]
Related
I have a spring-boot application where using webflux and r2dbc-postgres. I have discovered a strange issue when trying to do some db operations in a flatMap().
Code example:
#Transactional
public Mono<Void> insertDummyFooBars() {
return Flux.fromIterable(IntStream.rangeClosed(1, 260).boxed().collect(Collectors.toList()))
.log()
.flatMap(i -> this.repository.save(FooBar.builder().foo("test-" + i).build()))
.log()
.concatMap(i -> this.repository.findAll())
.then();
}
It seems like flatMap can process max 256 elements in batches. (Queues.SMALL_BUFFER_SIZE default value is 256). So when I tried to run the code above (with 260 elements) I've got an exception - TransientDataAccessResourceException and the following message:
Cannot exchange messages because the request queue limit is exceeded; nested exception is io.r2dbc.postgresql.client.ReactorNettyClient$RequestQueueException
There is no Releasing R2DBC Connection after this exception. The pgdb connection/session remains in idle in transaction state and the app is not able to run properly when pool max size is reached and all of the connections are in idle in transaction state. I think the connection should be released even if an exception happened or not.
If I use concatMap instead of flatMap it works as expected - no exception, connection released! It's also ok with flatMap when the elements are less than or equal to 256.
Is it possible to force pgdb connection closure? What should I do If I have lot of db operations in flatMap like this? Should I replace all of them with concatMap? Is there a global solution for this?
Versions:
Postgres: 12.6, Spring-boot: 2.7.6
Demo project
LOG:
2022-12-08 16:32:13.092 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.Iterable.1 : | onNext(256)
2022-12-08 16:32:13.092 DEBUG 17932 --- [actor-tcp-nio-1] o.s.r2dbc.core.DefaultDatabaseClient : Executing SQL statement [INSERT INTO foo_bar (foo) VALUES ($1)]
2022-12-08 16:32:13.114 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.FlatMap.2 : onNext(FooBar(id=258, foo=test-1))
2022-12-08 16:32:13.143 DEBUG 17932 --- [actor-tcp-nio-1] o.s.r2dbc.core.DefaultDatabaseClient : Executing SQL statement [SELECT foo_bar.* FROM foo_bar]
2022-12-08 16:32:13.143 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.Iterable.1 : | request(1)
2022-12-08 16:32:13.143 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.Iterable.1 : | onNext(257)
2022-12-08 16:32:13.144 DEBUG 17932 --- [actor-tcp-nio-1] o.s.r2dbc.core.DefaultDatabaseClient : Executing SQL statement [INSERT INTO foo_bar (foo) VALUES ($1)]
2022-12-08 16:32:13.149 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.Iterable.1 : | onComplete()
2022-12-08 16:32:13.149 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.Iterable.1 : | cancel()
2022-12-08 16:32:13.160 ERROR 17932 --- [actor-tcp-nio-1] reactor.Flux.FlatMap.2 : onError(org.springframework.dao.TransientDataAccessResourceException: executeMany; SQL [INSERT INTO foo_bar (foo) VALUES ($1)]; Cannot exchange messages because the request queue limit is exceeded; nested exception is io.r2dbc.postgresql.client.ReactorNettyClient$RequestQueueException: [08006] Cannot exchange messages because the request queue limit is exceeded)
2022-12-08 16:32:13.167 ERROR 17932 --- [actor-tcp-nio-1] reactor.Flux.FlatMap.2 :
org.springframework.dao.TransientDataAccessResourceException: executeMany; SQL [INSERT INTO foo_bar (foo) VALUES ($1)]; Cannot exchange messages because the request queue limit is exceeded; nested exception is io.r2dbc.postgresql.client.ReactorNettyClient$RequestQueueException: [08006] Cannot exchange messages because the request queue limit is exceeded
at org.springframework.r2dbc.connection.ConnectionFactoryUtils.convertR2dbcException(ConnectionFactoryUtils.java:215) ~[spring-r2dbc-5.3.24.jar:5.3.24]
at org.springframework.r2dbc.core.DefaultDatabaseClient.lambda$inConnectionMany$8(DefaultDatabaseClient.java:147) ~[spring-r2dbc-5.3.24.jar:5.3.24]
at reactor.core.publisher.Flux.lambda$onErrorMap$29(Flux.java:7105) ~[reactor-core-3.4.25.jar:3.4.25]
at reactor.core.publisher.Flux.lambda$onErrorResume$30(Flux.java:7158) ~[reactor-core-3.4.25.jar:3.4.25]
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94) ~[reactor-core-3.4.25.jar:3.4.25]
I have tried to change the Queues.SMALL_BUFFER_SIZE, and also tried to add a concurrency value to the flatmap. It works when I reduced the value to 255 but I think it is not a good solution.
Given the setup https://gist.github.com/gel-hidden/0a8627cf93f5396d6b73c2a6e71aad3e, I would expect when I send a message that the ServiceActivator would be called with a delay of 10 000 between messages.
The first channel takes in a list, then split the messages and then call another QueueChannel. But for some reason each pull polls all the split messages. I know I am missing something stupid, or I'm just too stupid to understand whats happening.
Related test case: https://gist.github.com/gel-hidden/de7975fffd0853ec8ce49f9d6fa6531d
Output:
2022-10-26 15:22:02.708 INFO 78647 --- [ scheduling-1] com.example.demo.DemoApplicationTests : Received message Hello
2022-10-26 15:22:02.708 INFO 78647 --- [ scheduling-1] com.example.demo.UpdateLocationFlow : Doing some work for model with id 2
2022-10-26 15:22:03.009 INFO 78647 --- [ scheduling-1] com.example.demo.UpdateLocationFlow : Completed some work for model with id 2
2022-10-26 15:22:03.017 INFO 78647 --- [ scheduling-1] com.example.demo.DemoApplicationTests : Received message World
2022-10-26 15:22:03.018 INFO 78647 --- [ scheduling-1] com.example.demo.UpdateLocationFlow : Doing some work for model with id 3
2022-10-26 15:22:03.319 INFO 78647 --- [ scheduling-1] com.example.demo.UpdateLocationFlow : Completed some work for model with id 3
2022-10-26 15:22:04.322 INFO 78647 --- [ scheduling-1] o.s.i.a.AggregatingMessageHandler : Expiring MessageGroup with correlationKey[1]
My thoughts is that the messages should be something like:
00:01 Doing some work for model with id 2
00:02 Completed some work for model with id 2
00:12 Doing some work for model with id 3
00:13 Completed some work for model it id 3
So, it is a bug in the Spring Integration around lifecycle management for the IntegrationFlowAdapter management. It just starts twice.
As a workaround I suggest to pull your #ServiceActivator handle() into an individual component with its own #Poller configuration and an inputChannel and outputChannel. In other words int must go outside of your UpdateLocationFlow. This way the IntegrationFlowAdapter won't have a control for its lifecycle and won't start it twice.
Meanwhile I'm looking how to fix it.
Thank you for reporting this!
I have spent days on this simple issue , I am giving up and finally posting this issue which I am facing locally. I am trying to set up a microservices flow in my local for my hand itching learning purpose. This is no brainer. I have Eureka , Zuul Gateway , Simple Microservice. When I try to reach to the underlying service with the "url route" its working. But when I try to do serviceId look up its not working. Guys help me fixing it.
Git hub link is Git hub source code link
I have also raised an issue Git hut Issue link
Eureka Screenshot
Zuul Gateway logs
2019-10-06 11:11:24.611 INFO 26980 --- [nio-2020-exec-4] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2019-10-06 11:11:24.611 INFO 26980 --- [nio-2020-exec-4] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2019-10-06 11:11:24.633 INFO 26980 --- [nio-2020-exec-4] o.s.web.servlet.DispatcherServlet : Completed initialization in 22 ms
2019-10-06 11:11:25.103 INFO 26980 --- [nio-2020-exec-4] c.netflix.config.ChainedDynamicProperty : Flipping property: CHECKOUT-SERVICE.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647
2019-10-06 11:11:25.157 INFO 26980 --- [nio-2020-exec-4] c.n.u.concurrent.ShutdownEnabledTimer : Shutdown hook installed for: NFLoadBalancer-PingTimer-CHECKOUT-SERVICE
2019-10-06 11:11:25.157 INFO 26980 --- [nio-2020-exec-4] c.netflix.loadbalancer.BaseLoadBalancer : Client: CHECKOUT-SERVICE instantiated a LoadBalancer: DynamicServerListLoadBalancer:{NFLoadBalancer:name=CHECKOUT-SERVICE,current list of Servers=[],Load balancer stats=Zone stats: {},Server stats: []}ServerList:null
2019-10-06 11:11:25.167 INFO 26980 --- [nio-2020-exec-4] c.n.l.DynamicServerListLoadBalancer : Using serverListUpdater PollingServerListUpdater
2019-10-06 11:11:25.215 INFO 26980 --- [nio-2020-exec-4] c.netflix.config.ChainedDynamicProperty : Flipping property: CHECKOUT-SERVICE.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647
2019-10-06 11:11:25.218 INFO 26980 --- [nio-2020-exec-4] c.n.l.DynamicServerListLoadBalancer : DynamicServerListLoadBalancer for client CHECKOUT-SERVICE initialized: DynamicServerListLoadBalancer:{NFLoadBalancer:name=CHECKOUT-SERVICE,current list of Servers=[192.168.0.6:8098],Load balancer stats=Zone stats: {defaultzone=[Zone:defaultzone; Instance count:1; Active connections count: 0; Circuit breaker tripped count: 0; Active connections per server: 0.0;]
},Server stats: [[Server:192.168.0.6:8098; Zone:defaultZone; Total Requests:0; Successive connection failure:0; Total blackout seconds:0; Last connection made:Wed Dec 31 19:00:00 EST 1969; First connection made: Wed Dec 31 19:00:00 EST 1969; Active Connections:0; total failure count in last (1000) msecs:0; average resp time:0.0; 90 percentile resp time:0.0; 95 percentile resp time:0.0; min resp time:0.0; max resp time:0.0; stddev resp time:0.0]
]}ServerList:org.springframework.cloud.netflix.ribbon.eureka.DomainExtractingServerList#6f7f7ca0
2019-10-06 11:11:26.177 INFO 26980 --- [erListUpdater-0] c.netflix.config.ChainedDynamicProperty : Flipping property: CHECKOUT-SERVICE.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647
Never mind guys it was a mistake from my side in resolving the API path
guys !
I notices that from time to time, my response is generated about approx. 10 sec. which is not normal. This normally happens when i submit a post form, but not always. I'm using Spring Boot 1.5.6 with Thymeleaf 3.0.2.
2017-10-16 09:09:21.737 TRACE 19366 --- [https-jsse-nio-8443-exec-1] o.t.s.e.LinkExpression : [THYMELEAF][https-jsse-nio-8443-exec-1] Evaluating link: "#{'/resources/js/lightbox.js'}"
2017-10-16 09:09:21.737 TRACE 19366 --- [https-jsse-nio-8443-exec-1] o.t.s.e.TextLiteralExpression : [THYMELEAF][https-jsse-nio-8443-exec-1] Evaluating text literal: "'/resources/js/lightbox.js'"
2017-10-16 09:09:21.737 TRACE 19366 --- [https-jsse-nio-8443-exec-1] o.t.TemplateEngine : [THYMELEAF][https-jsse-nio-8443-exec-1] FINISHED PROCESS AND OUTPUT OF TEMPLATE "distributor/createDistributor" WITH LOCALE en_US
2017-10-16 09:09:21.737 TRACE 19366 --- [https-jsse-nio-8443-exec-1] o.t.T.TIMER : [THYMELEAF][https-jsse-nio-8443-exec-1][distributor/createDistributor][en_US][8336337999][8336] TEMPLATE "distributor/createDistributor" WITH LOCALE en_US PROCESSED IN 8336337999 nanoseconds (approx. 8336ms)
2017-10-16 09:09:21.738 TRACE 19366 --- [https-jsse-nio-8443-exec-1] ationConfigEmbeddedWebApplicationContext : Publishing event in org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext#1c6b6478: ServletRequestHandledEvent: url=[/distributor/add]; client=[192........]; method=[GET]; servlet=[dispatcherServlet]; session=[F2E366C8DA7EC3C5BD2026FA388F3437]; user=[admin]; time=[8338ms]; status=[OK]
On line 4 ( TIMER ) is showing that processing time is time=[8338ms]
Can somebody help me how to debug this ?
Hello good people interested in writing spring apps in Kotlin. I am playing with Spring Boot 2.0.0 snapshot and spring-webflux. This piece of code:
#Component
class TestRouter() : RouterFunction<ServerResponse> {
override fun route(request: ServerRequest) = route(request) {
"/".route {
GET("/hello") { ServerResponse.ok().body(BodyInserters.fromObject("World")) }
"/{id}".route {
GET("/hello") { ServerResponse.ok().body(BodyInserters.fromObject("World ${request.pathVariable("id")}")) }
}
}
}
}
does not work as expected (at least as I would expect:))
➜ ~ curl -i http://localhost:8080/hello
HTTP/1.1 200 OK
transfer-encoding: chunked
Content-Type: text/plain;charset=UTF-8
World
but:
➜ ~ curl -i http://localhost:8080/1/hello
HTTP/1.1 404 Not Found
content-length: 0
Working case trace:
2017-03-03 00:58:03.865 TRACE 7666 --- [ctor-http-nio-4] o.s.w.r.f.server.RequestPredicates : Pattern "//**" matches against value "/hello"
2017-03-03 00:58:03.865 DEBUG 7666 --- [ctor-http-nio-4] o.s.w.r.function.server.RouterFunctions : Nested predicate "//**" matches against "GET /hello"
2017-03-03 00:58:03.865 TRACE 7666 --- [ctor-http-nio-4] o.s.w.r.f.server.RequestPredicates : Method "GET" matches against value "GET"
2017-03-03 00:58:03.866 TRACE 7666 --- [ctor-http-nio-4] o.s.w.r.f.server.RequestPredicates : Pattern "/hello" matches against value "/hello"
2017-03-03 00:58:03.866 DEBUG 7666 --- [ctor-http-nio-4] o.s.w.r.function.server.RouterFunctions : Predicate "(GET && /hello)" matches against "GET /hello"
Not working case trace:
2017-03-03 00:59:26.958 TRACE 7666 --- [ctor-http-nio-1] o.s.w.r.f.server.RequestPredicates : Pattern "//**" matches against value "/1/hello"
2017-03-03 00:59:26.958 DEBUG 7666 --- [ctor-http-nio-1] o.s.w.r.function.server.RouterFunctions : Nested predicate "//**" matches against "GET /1/hello"
2017-03-03 00:59:26.958 TRACE 7666 --- [ctor-http-nio-1] o.s.w.r.f.server.RequestPredicates : Method "GET" matches against value "GET"
2017-03-03 00:59:26.958 TRACE 7666 --- [ctor-http-nio-1] o.s.w.r.f.server.RequestPredicates : Pattern "/hello" does not match against value "/1/hello"
2017-03-03 00:59:26.959 TRACE 7666 --- [ctor-http-nio-1] o.s.w.r.f.server.RequestPredicates : Pattern "/{id}/**" matches against value "/1/hello"
2017-03-03 00:59:26.959 DEBUG 7666 --- [ctor-http-nio-1] o.s.w.r.function.server.RouterFunctions : Nested predicate "/{id}/**" matches against "GET /1/hello"
2017-03-03 00:59:26.959 TRACE 7666 --- [ctor-http-nio-1] o.s.w.r.f.server.RequestPredicates : Method "GET" matches against value "GET"
2017-03-03 00:59:26.959 TRACE 7666 --- [ctor-http-nio-1] o.s.w.r.f.server.RequestPredicates : Pattern "/hello" does not match against value "/1/hello"
This seems like a bug (as the "/{id}".route {...} supposedly is using RouterFunctions.nest), but I could be wrong. Your thoughts and help are welcome.
I obviously know that I can make /1/hello work by just writing GET("/{id}/hello") { ... }, but I am interested in the nested .route { ...} variant as it supports my use case of adding nested routes from another location (like a map, etc).
This issue has been fixed in SPR-15310.