during deploying k8bernates pod on server getting Graceful shutdown complete - spring-boot

I am trying to implement a spring actuator for readiness and liveness,
so if I deploy it with enabled probe, a graceful shutdown occurs.
However, for other services with the same settings and dependencies it works well, as well as if I run service with actuator on localhost it works well. What should I check?
my configs
probes:
enabled: true
ready:
path: /actuator/health/readiness
port: 8080
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
successThreshold: 3
failureThreshold: 15
live:
path: /actuator/health/liveness
port: 8080
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
successThreshold: 3
failureThreshold: 15
I have tried different settings for actuator
18:19:33,559 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
18:19:33,560 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
18:19:33,561 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [jar:file:/app.jar!/BOOT-INF/lib/sbol2-lib-commons2-1.0.0-SNAPSHOT.jar!/logback.xml]
18:19:33,578 |-INFO in ch.qos.logback.core.joran.spi.ConfigurationWatchList#30c7da1e - URL [jar:file:/app.jar!/BOOT-INF/lib/sbol2-lib-commons2-1.0.0-SNAPSHOT.jar!/logback.xml] is not of type file
18:19:33,889 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
18:19:33,894 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [consoleAppender]
18:19:34,072 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.LoggingEventJsonProviders] for [providers] property
18:19:34,173 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.LoggingEventFormattedTimestampJsonProvider] for [timestamp] property
18:19:34,175 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.LogLevelJsonProvider] for [logLevel] property
18:19:34,176 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.ThreadNameJsonProvider] for [threadName] property
18:19:34,178 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.LoggerNameJsonProvider] for [loggerName] property
18:19:34,180 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.MdcJsonProvider] for [mdc] property
18:19:34,183 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.StackTraceJsonProvider] for [stackTrace] property
18:19:34,186 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.StackHashJsonProvider] for [stackHash] property
18:19:34,189 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.LoggingEventPatternJsonProvider] for [pattern] property
18:19:34,770 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to ALL
18:19:34,770 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [consoleAppender] to Logger[ROOT]
18:19:34,771 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
18:19:34,772 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator#3ecd23d9 - Registering current configuration as safe fallback point
{"#timestamp":"2022-10-20T18:19:35.182+06:00","level":"TRACE","thread_name":"main","logger_name":"o.s.c.i.s.SpringFactoriesLoader","message":"Loaded [org.springframework.boot.logging.LoggingSystemFactory] names: [org.springframework.boot.logging.logback.LogbackLoggingSystem.Factory, org.springframework.boot.logging.log4j2.Log4J2LoggingSystem.Factory, org.springframework.boot.logging.java.JavaLoggingSystem.Factory]"}
18:19:36,597 |-INFO in ch.qos.logback.core.joran.spi.ConfigurationWatchList#bd4dc25 - URL [jar:file:/app.jar!/BOOT-INF/lib/sbol2-lib-commons2-1.0.0-SNAPSHOT.jar!/logback.xml] is not of type file
18:19:36,658 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
18:19:36,659 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [consoleAppender]
18:19:36,665 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.LoggingEventJsonProviders] for [providers] property
18:19:36,665 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.LoggingEventFormattedTimestampJsonProvider] for [timestamp] property
18:19:36,665 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.LogLevelJsonProvider] for [logLevel] property
18:19:36,666 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.ThreadNameJsonProvider] for [threadName] property
18:19:36,666 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.LoggerNameJsonProvider] for [loggerName] property
18:19:36,666 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.MdcJsonProvider] for [mdc] property
18:19:36,667 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.StackTraceJsonProvider] for [stackTrace] property
18:19:36,667 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.StackHashJsonProvider] for [stackHash] property
18:19:36,668 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [net.logstash.logback.composite.loggingevent.LoggingEventPatternJsonProvider] for [pattern] property
18:19:36,670 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to ALL
18:19:36,670 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#f736069 - Propagating ALL level on Logger[ROOT] onto the JUL framework
18:19:36,671 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [consoleAppender] to Logger[ROOT]
18:19:36,671 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
18:19:36,671 |-INFO in org.springframework.boot.logging.logback.SpringBootJoranConfigurator#4229bb3f - Registering current configuration as safe fallback point
{"#timestamp":"2022-10-20T18:19:36.677+06:00","level":"TRACE","thread_name":"main","logger_name":"o.s.beans.BeanUtils","message":"No property editor [org.springframework.boot.logging.LogLevelEditor] found for type org.springframework.boot.logging.LogLevel according to 'Editor' suffix convention"}
18:19:36,680 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#f736069 - Propagating INFO level on Logger[ROOT] onto the JUL framework
{"#timestamp":"2022-10-20T18:19:37.157+06:00","level":"INFO","thread_name":"main","logger_name":"k.s.s.a.l.LimitsApplication","message":"Starting LimitsApplication v1.0.13 using Java 15.0.2 on limits-5c859d4dbb-w642r with PID 1 (/app.jar started by 1000630000 in /)"}
{"#timestamp":"2022-10-20T18:19:37.159+06:00","level":"INFO","thread_name":"main","logger_name":"k.s.s.a.l.LimitsApplication","message":"The following profiles are active: ift"}
{"#timestamp":"2022-10-20T18:19:40.777+06:00","level":"INFO","thread_name":"main","logger_name":"o.s.d.r.c.RepositoryConfigurationDelegate","message":"Bootstrapping Spring Data JPA repositories in DEFAULT mode."}
{"#timestamp":"2022-10-20T18:19:41.484+06:00","level":"INFO","thread_name":"main","logger_name":"o.s.d.r.c.RepositoryConfigurationDelegate","message":"Finished Spring Data repository scanning in 694 ms. Found 9 JPA repository interfaces."}
{"#timestamp":"2022-10-20T18:19:44.194+06:00","level":"INFO","thread_name":"main","logger_name":"c.u.j.c.EnableEncryptablePropertiesBeanFactoryPostProcessor","message":"Post-processing PropertySource instances"}
{"#timestamp":"2022-10-20T18:19:44.296+06:00","level":"INFO","thread_name":"main","logger_name":"c.u.j.EncryptablePropertySourceConverter","message":"Converting PropertySource configurationProperties [org.springframework.boot.context.properties.source.ConfigurationPropertySourcesPropertySource] to AOP Proxy"}
{"#timestamp":"2022-10-20T18:19:44.298+06:00","level":"INFO","thread_name":"main","logger_name":"c.u.j.EncryptablePropertySourceConverter","message":"Converting PropertySource systemProperties [org.springframework.core.env.PropertiesPropertySource] to EncryptableMapPropertySourceWrapper"}
{"#timestamp":"2022-10-20T18:19:44.298+06:00","level":"INFO","thread_name":"main","logger_name":"c.u.j.EncryptablePropertySourceConverter","message":"Converting PropertySource systemEnvironment [org.springframework.boot.env.SystemEnvironmentPropertySourceEnvironmentPostProcessor$OriginAwareSystemEnvironmentPropertySource] to EncryptableSystemEnvironmentPropertySourceWrapper"}
{"#timestamp":"2022-10-20T18:19:44.299+06:00","level":"INFO","thread_name":"main","logger_name":"c.u.j.EncryptablePropertySourceConverter","message":"Converting PropertySource random [org.springframework.boot.env.RandomValuePropertySource] to EncryptablePropertySourceWrapper"}
{"#timestamp":"2022-10-20T18:19:44.299+06:00","level":"INFO","thread_name":"main","logger_name":"c.u.j.EncryptablePropertySourceConverter","message":"Converting PropertySource Config resource 'class path resource [application-ift.yaml]' via location 'optional:classpath:/' [org.springframework.boot.env.OriginTrackedMapPropertySource] to EncryptableMapPropertySourceWrapper"}
{"#timestamp":"2022-10-20T18:19:44.299+06:00","level":"INFO","thread_name":"main","logger_name":"c.u.j.EncryptablePropertySourceConverter","message":"Converting PropertySource Config resource 'class path resource [application.yaml]' via location 'optional:classpath:/' [org.springframework.boot.env.OriginTrackedMapPropertySource] to EncryptableMapPropertySourceWrapper"}
{"#timestamp":"2022-10-20T18:19:44.483+06:00","level":"INFO","thread_name":"main","logger_name":"c.u.j.f.DefaultLazyPropertyFilter","message":"Property Filter custom Bean not found with name 'encryptablePropertyFilter'. Initializing Default Property Filter"}
{"#timestamp":"2022-10-20T18:19:45.901+06:00","level":"INFO","thread_name":"main","logger_name":"c.u.j.r.DefaultLazyPropertyResolver","message":"Property Resolver custom Bean not found with name 'encryptablePropertyResolver'. Initializing Default Property Resolver"}
{"#timestamp":"2022-10-20T18:19:45.974+06:00","level":"INFO","thread_name":"main","logger_name":"c.u.j.d.DefaultLazyPropertyDetector","message":"Property Detector custom Bean not found with name 'encryptablePropertyDetector'. Initializing Default Property Detector"}
{"#timestamp":"2022-10-20T18:19:46.597+06:00","level":"INFO","thread_name":"main","logger_name":"o.s.b.w.e.t.TomcatWebServer","message":"Tomcat initialized with port(s): 9090 (http)"}
{"#timestamp":"2022-10-20T18:19:46.662+06:00","level":"INFO","thread_name":"main","logger_name":"o.a.c.http11.Http11NioProtocol","message":"Initializing ProtocolHandler [\"http-nio-9090\"]"}
{"#timestamp":"2022-10-20T18:19:46.662+06:00","level":"INFO","thread_name":"main","logger_name":"o.a.c.core.StandardService","message":"Starting service [Tomcat]"}
{"#timestamp":"2022-10-20T18:19:46.663+06:00","level":"INFO","thread_name":"main","logger_name":"o.a.c.core.StandardEngine","message":"Starting Servlet engine: [Apache Tomcat/9.0.48]"}
{"#timestamp":"2022-10-20T18:19:47.993+06:00","level":"INFO","thread_name":"main","logger_name":"o.h.j.internal.util.LogHelper","message":"HHH000204: Processing PersistenceUnitInfo [name: default]"}
{"#timestamp":"2022-10-20T18:19:48.185+06:00","level":"INFO","thread_name":"main","logger_name":"org.hibernate.Version","message":"HHH000412: Hibernate ORM core version 5.4.32.Final"}
{"#timestamp":"2022-10-20T18:19:48.506+06:00","level":"INFO","thread_name":"main","logger_name":"o.h.annotations.common.Version","message":"HCANN000001: Hibernate Commons Annotations {5.1.2.Final}"}
{"#timestamp":"2022-10-20T18:19:48.786+06:00","level":"WARN","thread_name":"main","logger_name":"com.zaxxer.hikari.HikariConfig","message":"HikariPool-1 - leakDetectionThreshold is less than 2000ms or more than maxLifetime, disabling it."}
{"#timestamp":"2022-10-20T18:19:48.787+06:00","level":"WARN","thread_name":"main","logger_name":"com.zaxxer.hikari.HikariConfig","message":"HikariPool-1 - idleTimeout has been set but has no effect because the pool is operating as a fixed size pool."}
{"#timestamp":"2022-10-20T18:19:48.787+06:00","level":"INFO","thread_name":"main","logger_name":"c.z.hikari.HikariDataSource","message":"HikariPool-1 - Starting..."}
{"#timestamp":"2022-10-20T18:19:49.595+06:00","level":"INFO","thread_name":"main","logger_name":"c.z.hikari.HikariDataSource","message":"HikariPool-1 - Start completed."}
{"#timestamp":"2022-10-20T18:19:49.806+06:00","level":"INFO","thread_name":"main","logger_name":"org.hibernate.dialect.Dialect","message":"HHH000400: Using dialect: org.hibernate.dialect.Oracle12cDialect"}
{"#timestamp":"2022-10-20T18:19:50.771+06:00","level":"INFO","thread_name":"main","logger_name":"o.h.e.b.i.EnversServiceImpl","message":"Envers integration enabled? : true"}
{"#timestamp":"2022-10-20T18:19:52.074+06:00","level":"INFO","thread_name":"main","logger_name":"o.h.v.internal.util.Version","message":"HV000001: Hibernate Validator 6.2.0.Final"}
{"#timestamp":"2022-10-20T18:19:55.159+06:00","level":"INFO","thread_name":"main","logger_name":"o.h.e.t.j.p.i.JtaPlatformInitiator","message":"HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]"}
{"#timestamp":"2022-10-20T18:19:55.173+06:00","level":"INFO","thread_name":"main","logger_name":"o.s.o.j.LocalContainerEntityManagerFactoryBean","message":"Initialized JPA EntityManagerFactory for persistence unit 'default'"}
{"#timestamp":"2022-10-20T18:20:08.260+06:00","level":"INFO","thread_name":"main","logger_name":"o.s.b.w.e.t.TomcatWebServer","message":"Tomcat started on port(s): 9090 (http) with context path ''"}
{"#timestamp":"2022-10-20T18:20:08.470+06:00","level":"INFO","thread_name":"main","logger_name":"o.s.b.w.e.t.TomcatWebServer","message":"Tomcat initialized with port(s): 8080 (http)"}
{"#timestamp":"2022-10-20T18:20:08.502+06:00","level":"INFO","thread_name":"main","logger_name":"o.s.b.a.e.w.EndpointLinksResolver","message":"Exposing 2 endpoint(s) beneath base path '/actuator'"}
{"#timestamp":"2022-10-20T18:20:08.703+06:00","level":"INFO","thread_name":"main","logger_name":"o.s.b.w.e.t.TomcatWebServer","message":"Tomcat started on port(s): 8080 (http) with context path ''"}
{"#timestamp":"2022-10-20T18:20:10.382+06:00","level":"INFO","thread_name":"main","logger_name":"k.s.s.a.l.LimitsApplication","message":"Started LimitsApplication in 35.283 seconds (JVM running for 37.72)"}
{"#timestamp":"2022-10-20T18:20:10.472+06:00","level":"INFO","thread_name":"main","logger_name":"k.sber.sbol2.app.limits.Runner","message":"Limits is running..."}
{"#timestamp":"2022-10-20T18:20:10.475+06:00","level":"INFO","thread_name":"SpringApplicationShutdownHook","logger_name":"o.s.b.w.e.t.GracefulShutdown","message":"Commencing graceful shutdown. Waiting for active requests to complete"}
{"#timestamp":"2022-10-20T18:20:10.487+06:00","level":"INFO","thread_name":"tomcat-shutdown","logger_name":"o.s.b.w.e.t.GracefulShutdown","message":"Graceful shutdown complete"}
{"#timestamp":"2022-10-20T18:20:10.563+06:00","level":"INFO","thread_name":"SpringApplicationShutdownHook","logger_name":"o.s.b.w.e.t.GracefulShutdown","message":"Commencing graceful shutdown. Waiting for active requests to complete"}
{"#timestamp":"2022-10-20T18:20:10.570+06:00","level":"INFO","thread_name":"tomcat-shutdown","logger_name":"o.s.b.w.e.t.GracefulShutdown","message":"Graceful shutdown complete"}
{"#timestamp":"2022-10-20T18:20:12.778+06:00","level":"INFO","thread_name":"SpringApplicationShutdownHook","logger_name":"o.s.o.j.LocalContainerEntityManagerFactoryBean","message":"Closing JPA EntityManagerFactory for persistence unit 'default'"}
{"#timestamp":"2022-10-20T18:20:12.781+06:00","level":"INFO","thread_name":"SpringApplicationShutdownHook","logger_name":"c.z.hikari.HikariDataSource","message":"HikariPool-1 - Shutdown initiated..."}
{"#timestamp":"2022-10-20T18:20:13.013+06:00","level":"INFO","thread_name":"SpringApplicationShutdownHook","logger_name":"c.z.hikari.HikariDataSource","message":"HikariPool-1 - Shutdown completed."}
Normal Scheduled 8m42s default-scheduler Successfully assigned x/limits-5c859d4dbb-p5jgt to ocp-t-w002
Normal AddedInterface 8m41s multus Add eth0 [10.111.6.127/24] from openshift-sdn
Normal Pulled 8m38s kubelet Successfully pulled image "nexus.x.x/x/limits:1.0.13" in 2.961535874s
Normal Killing 8m13s kubelet Container limits failed liveness probe, will be restarted
Warning Unhealthy 8m13s (x3 over 8m33s) kubelet Liveness probe failed: Get "http://10.111.6.127:8080/actuator/health/liveness": dial tcp 10.111.6.127:8080: connect: connection refused
Normal Pulling 7m43s (x2 over 8m41s) kubelet Pulling image "nexus.x.x:x/x/limits:1.0.13"
Normal Pulled 7m43s kubelet Successfully pulled image "nexus.x:x/x/limits:1.0.13" in 23.406574ms
Normal Created 7m42s (x2 over 8m38s) kubelet Created container limits
Normal Started 7m42s (x2 over 8m37s) kubelet Started container limits
Warning Unhealthy 3m41s (x51 over 8m36s) kubelet Readiness probe failed: Get "http://10.111.6.127:8080/actuator/health/readiness": dial tcp 10.111.6.127:8080:

Related

Thingsboard performance tests

I try to execute the thingsboard performance test
Thingsboard performance tests
$ sudo docker run -it --env-file .env --name tb-perf-test thingsboard/tb-performance-test
but I get this result, anyone can help, please
Starting ThingsBoard Performance Tests...
16:57:11,071 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
16:57:11,071 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
16:57:11,071 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/usr/share/tb-performance-tests/conf/logback.xml]
16:57:11,085 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs multiple times on the classpath.
16:57:11,085 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [file:/usr/share/tb-performance-tests/conf/logback.xml]
16:57:11,085 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/usr/share/tb-performance-tests/bin/tb-performance-tests.jar!/BOOT-INF/classes!/logback.xml]
16:57:11,502 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
16:57:11,579 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
16:57:11,588 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
16:57:11,692 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - This appender no longer admits a layout as a sub-component, set an encoder instead.
16:57:11,692 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - To ensure compatibility, wrapping your layout in LayoutWrappingEncoder.
16:57:11,692 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - See also http://logback.qos.ch/codes.html#layoutInsteadOfEncoder for details
16:57:11,693 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.thingsboard] to INFO
16:57:11,693 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
16:57:11,693 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
16:57:11,694 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
16:57:11,708 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator#5387f9e0 - Registering current configuration as safe fallback point
Error starting ApplicationContext. To display the auto-configuration report re-run your application with 'debug' enabled.
16:57:21.391 [main] ERROR o.s.boot.SpringApplication - Application startup failed
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'testExecutor': Unsatisfied dependency expressed through field 'deviceAPITest'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'deviceMqttAPITest': Invocation of init method failed; nested exception is org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://localhost:8080/api/auth/login": Connection refused (Connection refused); nested exception is java.net.ConnectException: Connection refused (Connection refused)
Maybe you are using port 9090, while your deployment uses 8080.

Configuring logback with Storm

I have a storm project for which I have added some custom logback configurations using logback.xml (added configs to write logs to an index). When I build the jar and deploy the same on the cluster, I get the multiple bindings errors for this class StaticLoggerBinder.class due to it being present both in my jar and storm's own dependency list. Below is the error
SLF4J: Found binding in [jar:file:/usr/lib/storm/apache-storm-1.2.2/lib/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/app/test/test-storm-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
This causes none of my custom configs to be picked up.
When I remove the conflicting jar from Storm's lib folder, my logback.xml configs are reflected and my logs are written to the index with no issues. I also see these logs in the beginning when I deploy the topology showing the configs I have added
22:05:13,818 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [jar:file:/app/test/test-storm-0.0.55-SNAPSHOT-jar-with-dependencies.jar!/logback.xml]
22:05:13,839 |-INFO in ch.qos.logback.core.joran.spi.ConfigurationWatchList#19bb089b - URL [jar:file:/app/test/test-storm-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/logback.xml] is not of type file
22:05:14,094 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
22:05:14,097 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [CONSOLE]
22:05:14,102 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
22:05:14,125 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [com.splunk.logging.HttpEventCollectorLogbackAppender]
22:05:14,126 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [SPLUNK]
22:05:14,132 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [com.test] to INFO
22:05:14,132 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [com.test] to false
22:05:14,132 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [SPLUNK] to Logger[com.test]
22:05:14,133 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [CONSOLE] to Logger[com.test]
22:05:14,133 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
22:05:14,133 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [CONSOLE] to Logger[ROOT]
22:05:14,133 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
However, I do not want to take this route since it cannot be maintained so I decided use the maven-shade-plugin to relocate the org.slf4j.impl to org.shaded.slf4j.impl. Now when I deploy the topology there is no multiple bindings anymore, however my logback.xml is not honored and none of my configs are reflected nor do I see the above messages. It probably falls back to using a different logger.
How can I make my configs to be reflected when I deploy the topology? Please help. (Storm Version 1.2.2)
Storm switched from Logback to Log4j2 a few years ago. You can configure Log4j2 via the storm/log4j2/worker.xml and storm/log4j2/cluster.xml files. The former configures logging for your topology, as well as the worker JVM.
You should exclude logback from your fat jar.
Please also see this answer https://stackoverflow.com/a/50229035/8845188.

Spring-boot Events Missing Using Logback ServerSocketReceiver

Symptom: spring-boot basic app (no web, no jpa, no addons) defines a receiver in logback-spring.xml, but remote appender events never reach the local appenders.
logback-spring.xml:
<configuration debug="true">
<receiver name="logsink" class="ch.qos.logback.classic.net.server.ServerSocketReceiver">
<port>6004</port>
</receiver>
<appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss} %-5level %logger{25} - %X{client} %msg %n</pattern>
</encoder>
</appender>
<appender name="crap" class="org.oclc.hadoop.logging.logbacksinksb.service.CrapAppender"/>
<root level="DEBUG">
<appender-ref ref="stdout"/>
<appender-ref ref="crap"/>
</root>
</configuration>
The crapAppender is stripped to bare-bones for this problem:
public class CrapAppender extends AppenderBase<ILoggingEvent> {
#Override
protected void append(ILoggingEvent eventObject) {
System.out.println("got event!");
}
}
On startup, the log says my components are being instantiated:
16:39:12,103 |-INFO in ch.qos.logback.classic.joran.action.ReceiverAction - About to instantiate receiver of type [ch.qos.logback.classic.net.server.ServerSocketReceiver]
16:39:12,119 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
16:39:12,119 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [stdout]
16:39:12,119 |-INFO in ch.qos.logback.classic.net.server.RemoteAppenderServerRunner#228dc826 - listening on 0.0.0.0:6004
16:39:12,120 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
16:39:12,135 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [org.oclc.hadoop.logging.logbacksinksb.service.CrapAppender]
16:39:12,135 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [crap]
16:39:12,136 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to DEBUG
16:39:12,136 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [stdout] to Logger[ROOT]
16:39:12,136 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [crap] to Logger[ROOT]
16:39:12,136 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
16:39:12,136 |-INFO in org.springframework.boot.logging.logback.SpringBootJoranConfigurator#1dd02175 - Registering current configuration as safe fallback point
16:39:12,138 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#23bb8443 - Propagating INFO level on Logger[org.springframework] onto the JUL framework
I can even see that the remote appender has connected to the receiver port (6004):
java 5935 jamiesoh 32u IPv6 0xaf52811569ae62a3 0t0 TCP localhost:54160->localhost:6004 (ESTABLISHED)
java 6642 jamiesoh 53u IPv6 0xaf52811569ae5d63 0t0 TCP *:6004 (LISTEN)
java 6642 jamiesoh 54u IPv6 0xaf5281155d876d63 0t0 TCP localhost:6004->localhost:54160 (ESTABLISHED)
The local appenders only emit local events; the remote appender never reaches the local crap appender.
Any one have any ideas on what is preventing the remote events from passing thru the local appenders?
Just in case this bites someone else, I found the cause. One of the other artifacts had a dependency on logback-classic:1.2.3; the spring-boot version is 1.1.11. Excluding this transitive dependency corrected the problem.
Doctor, heal thyself...

spring JNDI logging before logback is initialized

I've got a spring boot app that is outputing around 1500 lines of JNDI debug logging to stdout at startup before logback is initialized. Does anyone know what's causing this and if there is a way to get logback initialized before this? I've got -Dlogback.debug=true but it doesn't help much.
10:44:56.453 [main] DEBUG org.springframework.jndi.JndiTemplate - Looking up JNDI object with name [java:comp/env/logging.exception-conversion-word]
10:44:56.457 [main] DEBUG jndi - InitialContextFactory.getInitialContext()
10:44:56.457 [main] DEBUG jndi - Created initial context delegate for local namespace:org.eclipse.jetty.jndi.local.localContextRoot#564fabc8
10:44:56.457 [main] DEBUG org.eclipse.jetty.jndi.java.javaURLContextFactory - >>> new root context requested
10:44:56.457 [main] DEBUG jndi - Looking up name="comp/env/logging.exception-conversion-word"
10:44:56.457 [main] DEBUG jndi - Trying thread context classloader
10:44:56.457 [main] DEBUG jndi - Looking up name="env/logging.exception-conversion-word"
10:44:56.458 [main] DEBUG org.springframework.jndi.JndiLocatorDelegate - Converted JNDI name [java:comp/env/logging.exception-conversion-word] not found - trying original name [logging.exception-conversion-word]. javax.naming.NameNotFoundException; remaining name 'env/logging.exception-conversion-word'
10:44:56.458 [main] DEBUG org.springframework.jndi.JndiTemplate - Looking up JNDI object with name [logging.exception-conversion-word]
...
10:44:56.487 [main] DEBUG jndi - Looking up name="env/LOGGING_PATTERNLEVEL"
10:44:56.487 [main] DEBUG org.springframework.jndi.JndiLocatorDelegate - Converted JNDI name [java:comp/env/LOGGING_PATTERNLEVEL] not found - trying original name [LOGGING_PATTERNLEVEL]. javax.naming.NameNotFoundException; remaining name 'env/LOGGING_PATTERNLEVEL'
10:44:56.487 [main] DEBUG org.springframework.jndi.JndiTemplate - Looking up JNDI object with name [LOGGING_PATTERNLEVEL]
10:44:56.487 [main] DEBUG jndi - InitialContextFactory.getInitialContext()
10:44:56.487 [main] DEBUG jndi - Created initial context delegate for local namespace:org.eclipse.jetty.jndi.local.localContextRoot#bcec361
10:44:56.487 [main] DEBUG jndi - Looking up name="LOGGING_PATTERNLEVEL"
10:44:56.487 [main] DEBUG org.springframework.jndi.JndiPropertySource - JNDI lookup for name [LOGGING_PATTERNLEVEL] threw NamingException with message: null. Returning null.
10:44:56,536 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
10:44:56,539 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
10:44:56,543 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
10:44:56,561 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
10:44:56,564 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
10:44:56,565 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
10:44:56,571 |-INFO in my.log.TimestampRollingPolicy#24fcf36f - Will use the pattern /tmp/logs/qs.%d{yyyyMMdd-HHmmss}.log to archive files
10:44:56,574 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: /tmp/logs/qs.log
10:44:56,574 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [/tmp/logs/qs.log]
10:44:56,576 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
10:44:56,576 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#4a7f959b - Propagating WARN level on Logger[ROOT] onto the JUL framework
10:44:56,576 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
10:44:56,576 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
10:44:56,576 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
10:44:56,577 |-INFO in org.springframework.boot.logging.logback.SpringBootJoranConfigurator#32b260fa - Registering current configuration as safe fallback point
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v1.4.4.RELEASE)
10:44:56,750 |-INFO in c.q.l.co.rolling.helper.RenameUtil - Renaming file [/tmp/logs/qs.log] to [/tmp/logs/qs.20170222-173130.log]
10:44:56 [main] INFO my.Application - Starting ...
I also saw this with Spring Boot 1.5.1 when using a custom logback-spring.xml. The debug statements don't appear if logback-spring.xml doesn't exist.
I was able to work around it by disabling the JNDIPropertySource. Create a file, spring.properties, in your classpath root:
# Disable the JNDIPropertySource. This avoids dozens of spurious DEBUG messages
# when starting up with a custom logback-spring.xml. We're not alone:
# http://stackoverflow.com/questions/42446013/spring-jndi-logging-before-logback-is-initialized
spring.jndi.ignore=true
spring.jndi.ignore was added in https://jira.spring.io/browse/SPR-14026.
This approach feels squicky, but as I'm not running in a container where JNDI would be useful, it doesn't break anything for me.
I also had the same problem, but I couldn't turn off jndi, because my application is using it on startup. So after some research, I ended up with the solution:
1) rename logback-spring.xml to some other name, for example logback-config.xml
2) put logging.config property into application.properties file.
As the result you configuration will be applied only on spring startup, and this debug logging will be skipped. Also in my case the first solution wasn't working because I needed jndi, in this case you also leave jndi enabled.
Update: this helped only for local environment, to make the same work on remote tomcat and remove this debug log from catalina.log file, I ended up with renaming this logback file to logback.xml, thus it's found on the startup, before jndi lookup.
you can avoid this by add this line to your logback xml file
<logger name="org.springframework.jndi" level="INFO" />

Including spark in spring boot project

I have java spring boot project(for web server), and it works find.
After that, I added spark depencies on this project, and it build without error.
In gradle
compile group: 'org.apache.spark', name: 'spark-core_2.11', version: '2.0.1'
compile group: 'org.apache.spark', name: 'spark-sql_2.11', version: '2.0.1'
BUT everytime spring web server got request, web server kept re-compiling itself.
some request output
|------------|------------|-----|------------|
// WHEN REQEUST GOT INTO, SPRING IS CLOSED AND RESTARTED ITSELF. WHY?????
17:26:19.052 INFO o.s.b.c.e.AnnotationConfigEmbeddedWebApplicationContext [] - [ : ]Closing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext#722b17b0: startup date [Mon Nov 21 17:26:02 KST 2016]; root of context hierarchy
17:26:19.055 INFO o.s.j.e.a.AnnotationMBeanExporter [] - [ : ]Unregistering JMX-exposed beans on shutdown
17:26:19,607 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
17:26:19,607 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [SQLFileAppender]
17:26:19,615 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - No compression will be used
17:26:19,615 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - Will use the pattern log/sql.%d{yyyy-MM-dd}.log for the active file
17:26:19,616 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is 'yyyy-MM-dd' from file name pattern 'log/sql.%d{yyyy-MM-dd}.log'.
17:26:19,616 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
17:26:19,616 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Mon Nov 21 17:26:19 KST 2016
17:26:19,618 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
17:26:19,622 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[SQLFileAppender] - Active log file name: log/sql.2016-11-21.log
17:26:19,623 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[SQLFileAppender] - File property is set to [null]
17:26:19,623 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
17:26:19,623 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [ControllerFileAppender]
17:26:19,623 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - No compression will be used
17:26:19,624 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - Will use the pattern log/controller.%d{yyyy-MM-dd}.log for the active file
17:26:19,624 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is 'yyyy-MM-dd' from file name pattern 'log/controller.%d{yyyy-MM-dd}.log'.
17:26:19,624 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
17:26:19,624 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Mon Nov 21 17:26:19 KST 2016
17:26:19,624 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
17:26:19,625 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ControllerFileAppender] - Active log file name: log/controller.2016-11-21.log
17:26:19,625 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ControllerFileAppender] - File property is set to [null]
17:26:19,625 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
17:26:19,625 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [console]
17:26:19,628 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
17:26:19,629 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#be00188 - Propagating INFO level on Logger[ROOT] onto the JUL framework
17:26:19,630 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [console] to Logger[ROOT]
17:26:19,630 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [jdbc] to OFF
17:26:19,630 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#be00188 - Propagating OFF level on Logger[jdbc] onto the JUL framework
17:26:19,630 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [jdbc.sqltiming] to INFO
17:26:19,630 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#be00188 - Propagating INFO level on Logger[jdbc.sqltiming] onto the JUL framework
17:26:19,630 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [SQLFileAppender] to Logger[jdbc.sqltiming]
17:26:19,630 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [jdbc.resultsettable] to INFO
17:26:19,630 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#be00188 - Propagating INFO level on Logger[jdbc.resultsettable] onto the JUL framework
17:26:19,630 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [Controller] to INFO
17:26:19,630 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#be00188 - Propagating INFO level on Logger[Controller] onto the JUL framework
17:26:19,630 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [ControllerFileAppender] to Logger[Controller]
17:26:19,630 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
17:26:19,630 |-INFO in org.springframework.boot.logging.logback.SpringBootJoranConfigurator#6917246 - Registering current configuration as safe fallback point
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v1.3.2.RELEASE)

Resources