-Dlogback.configurationFile=logback.xml ignored when running Spring-Boot - spring

We have a spring-boot 1.0.RC1 application, with logback configured for logging and a logback.xml file in src/test/resources. When we run spring-boot from maven, the logback file seems to be seen and respected. (That's our required logging format after the spring-boot logo)
D:\Users\myName\My Projects\Work\my-co\my-app\src\my-app\my-app-camel>mvn clean spring-boot:run -Dlogback.statusListenerClass=ch.qos.logback.core.status.OnConsoleStatusListener -Dlogback.configurationFile=src/test/resources/logback.xml
<snip/>
[INFO] --- spring-boot-maven-plugin:0.5.0.M7:run (default-cli) # my-app-camel ---
06:41:56,011 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [src/test/resources/logback.xml] at [file:/D:/Users/myName/My%20Projects/Work/my-co/my-app/src/my-app/my-app-camel/src/test/resources/logback.xml]
06:41:56,150 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
06:41:56,170 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Setting ReconfigureOnChangeFilter scanning period to 5 seconds
06:41:56,171 |-INFO in ReconfigureOnChangeFilter{invocationCounter=0} - Will scan for changes in [[D:\Users\myName\My Projects\Work\my-co\my-app\src\my-app\my-app-camel\src\test\resources\logback.xml]] every 5 seconds.
06:41:56,172 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Adding ReconfigureOnChangeFilter as a turbo filter
06:41:56,223 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
06:41:56,233 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
06:41:56,276 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
06:41:56,389 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [me.moocar.logbackgelf.GelfAppender]
06:41:56,398 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [GELF]
06:41:56,567 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [com.my-co.my-app.] to DEBUG
06:41:56,568 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [com.my-co.my-app.] to false
06:41:56,569 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[com.my-co.my-app.]
06:41:56,571 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.] to DEBUG
06:41:56,572 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [org.] to false
06:41:56,572 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[org.]
06:41:56,573 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
06:41:56,574 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
06:41:56,574 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
06:41:56,578 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator#4ffdcca5 - Registering current configuration as safe fallback point
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v1.0.0.RC1)
06:41:57.436 [com.my-co.my-app.Application.main()] INFO com.my-co.my-app.Application - Starting Application on MY-LAPTOP-HOSTNAME with PID 7812 (started by myName)
06:41:57.952 [com.my-co.my-app.Application.main()] INFO o.s.b.f.xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [META-INF/spring/camel-server.xml]
<etc./>
However, when we remove a rogue logback.xml file which snuck into src/java/resources and re-run with everything else unchanged (making sure we clean to remove the one in target too):
D:\Users\myName\My Projects\Work\my-co\my-app\src\my-app\my-app-camel>mvn clean spring-boot:run -Dlogback.statusListenerClass=ch.qos.logback.core.status.OnConsoleStatusListener -Dlogback.configurationFile=src/test/resources/logback.xml
<snip/>
[INFO]
[INFO] <<< spring-boot-maven-plugin:1.0.0.RC1:run (default-cli) # my-app-camel <<<
[INFO]
[INFO] --- spring-boot-maven-plugin:1.0.0.RC1:run (default-cli) # my-app-camel ---
06:53:59,598 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [src/test/resources/logback.xml] at [file:/D:/Users/myName/My%20Projects/Work/my-co/my-app/src/my-app/my-app-camel/src/test/resources/logback.xml]
06:53:59,736 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
06:53:59,755 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Setting ReconfigureOnChangeFilter scanning period to 5 seconds
06:53:59,756 |-INFO in ReconfigureOnChangeFilter{invocationCounter=0} - Will scan for changes in [[D:\Users\myName\My Projects\Work\my-co\my-app\src\my-app\my-app-camel\src\test\resources\logback.xml]] every 5 seconds.
06:53:59,757 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Adding ReconfigureOnChangeFilter as a turbo filter
06:53:59,806 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
06:53:59,817 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
06:53:59,858 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
06:53:59,971 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [me.moocar.logbackgelf.GelfAppender]
06:53:59,980 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [GELF]
06:54:00,144 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [com.my-co.my-app.] to DEBUG
06:54:00,144 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [com.my-co.my-app.] to false
06:54:00,145 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[com.my-co.my-app.]
06:54:00,148 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.] to DEBUG
06:54:00,148 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [org.] to false
06:54:00,149 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[org.]
06:54:00,150 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to DEBUG
06:54:00,150 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
06:54:00,151 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
06:54:00,154 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator#54c1f582 - Registering current configuration as safe fallback point
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v1.0.0.RC1)
2014-02-12 06:54:01.023 INFO 5240 --- [lication.main()] com.my-co.my-app.Application : Starting Application on MY-LAPTOP-HOSTNAME with PID 5240 (started by myName)
2014-02-12 06:54:01.564 INFO 5240 --- [lication.main()] o.s.b.f.xml.XmlBeanDefinitionReader : Loading XML bean definitions from class path resource [META-INF/spring/camel-server.xml]
<etc./>
You'll notice that the logging format after spring-boot starts is different from the first output. However, logback (prior to Spring-Boot starting) reports it's seen the logback.xml in src/test/resources, just like it did the first time.
For interest, our logback.xml (which was in both src/java/resources and src/test/resources is as follows:
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" scanPeriod="5 seconds">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{40} - %msg%n</pattern>
</encoder>
</appender>
<appender name="GELF" class="me.moocar.logbackgelf.GelfAppender">
<facility>logback-gelf-test</facility>
<graylog2ServerHost>XXX.XXX.XXX.XXX</graylog2ServerHost>
<graylog2ServerPort>NNNNN</graylog2ServerPort>
<useLoggerName>true</useLoggerName>
<useThreadName>true</useThreadName>
<useMarker>true</useMarker>
<graylog2ServerVersion>0.9.6</graylog2ServerVersion>
<chunkThreshold>1000</chunkThreshold>
<messagePattern>%m%rEx</messagePattern>
<shortMessagePattern>%.-100(%m%rEx)</shortMessagePattern>
<additionalField>ipAddress:_ip_address</additionalField>
<additionalField>requestId:_request_id</additionalField>
<staticAdditionalField>_node_name:www013</staticAdditionalField>
<includeFullMDC>true</includeFullMDC>
</appender>
<logger name="com.my-co.my-app." level="DEBUG" additivity="false">
<appender-ref ref="STDOUT" />
</logger>
<logger name="org." level="DEBUG" additivity="false">
<appender-ref ref="STDOUT" />
</logger>
<root level="INFO">
<appender-ref ref="STDOUT" />
<!--<appender-ref ref="GELF" />-->
</root>
</configuration>
So in summary, it seems (though I might be wrong) that when a logback.xml is bundled with my spring-boot app it is seen and used (i.e. in src/java/resources), but when we don't bundle, then despite logback's best efforts it's ignored.
As for fixing this, we're stumped. We're guessing that there is something somewhere in our spring-boot jar which is somehow taking precedence, or turning off logback, or stepping in and telling logback to ignore what it's just been told and to use another default.
Alternatively we're being dumb, but can't see it. All help gratefully received.

The default strategy assumes that if you don't have a logback.xml (or one of the other standard file names) in the classpath then you must be happy with the defaults (see LogbackLoggingSystem for details). Spring Boot tries to unify the external configuration switches for common logging sytems, so it looks in logging.config (it didn't know about logback.configurationFile). You could use that instead (i.e. logging.config=file:./src/test/resources/logback.xml), or make sure your config file is on the classpath.

I know this post is a bit old but adding the following to application.properties is a quick workaround based on Dave Syer's answer:
logging.config=${logback.configurationFile}

According to Spring Boot documentation, you could run like this:
java -jar app.jar --logging.config=./src/test/resources/logback.xml
Additionally, Spring Boot tries to unify the properties name of the different log startegies. As a result, the JVM parameter Dlogback.configurationFile is not used for Spring Boot.

If logback xml is under resources folder, you can add following to VM option:
java -jar App.jar -Dlogging.config=classpath:logback-local.xml

Add logback file path in application.properties file . Example test folder contains app.jar and config folder as below
test
|- app.jar
|- config
|--- application.properties
|--- logback.xml
Then add logging.config:config\\logback.xml. Then try , it will work

Assuming you have spring logback configurations in config directory, the command below will add external file by overriding the existing file (if any) inside the application:
java -jar -Dlogging.config=./config/logback-spring.xml app.jar
--spring.config.location=./config/application.properties

Not quite right. logback.configurationFile is not managed by Spring (i.e. env var replacement or so), but logback will still pick it up. At least that's my understanding of the docs and my practical experience.

Related

Spring boot app running in docker can't access external servers under test profile

I wrote a spring boot app that I am dockerizing. My app communicates with a mongodb (running locally on my box) and with a rabbitMq server (running in a docker container locally on my box). Running under my dev profile (having replaced localhost or 127.0.0.1 with *host.docker.internal) everything works.
Here is my Dockerfile
# Filename: Dockerfile
# It is assumed that you are in the root directory for this project
FROM openjdk:8
ADD /target/monitor-msg-consumer-0.0.1-SNAPSHOT.jar monitor-msg-consumer-0.0.1-SNAPSHOT.jar
EXPOSE 11371
## Profile: dev
ENTRYPOINT ["java","-jar", "-Dlog_home=/tmp","monitor-msg-consumer-0.0.1-SNAPSHOT.jar"]
I create an image, then run that image using the following command:
docker run --network=host -p 11371:11371 -e "SPRING_PROFILES_ACTIVE=dev" --name monitor-msg-consumer monitor-msg-consumer-img:latest
And as stated the app runs and I can interact with it under the dev profile.
Now, if I delete that container and run it with my test profile using the command:
docker run --network=host -p 11371:11371 -e "SPRING_PROFILES_ACTIVE=test" --name monitor-msg-consumer monitor-msg-consumer-img:latest
the app can't find an external rabbitMq server running in our test environment.
What am I missing?
I am new to docker. So, go easy on me!
Here are the logs from the docker container:
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.1.7.RELEASE)
2019-09-30T12:46:43.344207600Z
12:46:44,049 |-INFO in ch.qos.logback.core.joran.spi.ConfigurationWatchList#62fdb4a6 - URL [jar:file:/monitor-msg-consumer-0.0.1-SNAPSHOT.jar!/BOOT-INF/classes!/logback.xml] is not of type file
12:46:44,053 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Will scan for changes in [jar:file:/monitor-msg-consumer-0.0.1-SNAPSHOT.jar!/BOOT-INF/classes!/logback.xml]
12:46:44,053 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Setting ReconfigureOnChangeTask scanning period to 30 seconds
12:46:44,054 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
12:46:44,054 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
12:46:44,055 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
12:46:44,056 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
12:46:44,057 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
12:46:44,059 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy#500179317 - No compression will be used
12:46:44,061 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy#500179317 - Will use the pattern /tmp/archived/monitor_msg_consumer.%d{yyyy-MM-dd}.%i.log for the active file
12:46:44,062 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP#31206beb - The date pattern is 'yyyy-MM-dd' from file name pattern '/tmp/archived/monitor_msg_consumer.%d{yyyy-MM-dd}.%i.log'.
12:46:44,067 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP#31206beb - Roll-over at midnight.
12:46:44,067 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP#31206beb - Setting initial period to Mon Sep 30 12:46:42 UTC 2019
12:46:44,068 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP#31206beb - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
12:46:44,068 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP#31206beb - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
12:46:44,068 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: /tmp/monitor_msg_consumer.log
12:46:44,068 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [/tmp/monitor_msg_consumer.log]
12:46:44,069 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [gov.utah.dts.monitor] to DEBUG
12:46:44,069 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#3e77a1ed - Propagating DEBUG level on Logger[gov.utah.dts.monitor] onto the JUL framework
12:46:44,069 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to DEBUG
12:46:44,069 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
12:46:44,069 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
12:46:44,070 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
12:46:44,070 |-INFO in org.springframework.boot.logging.logback.SpringBootJoranConfigurator#3ffcd140 - Registering current configuration as safe fallback point
2019-09-30 12:46:44 | ERROR | [main] o.s.c.l.LoggingRebinder:83 - Cannot set level: WARN #(Options: DEBUG, ERROR, FATAL, INFO, WARN) for 'org.springframework'
12:46:44,074 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#3e77a1ed - Propagating WARN level on Logger[ROOT] onto the JUL framework
2019-09-30 12:46:44 | ERROR | [main] o.s.c.l.LoggingRebinder:83 - Cannot set level: DEBUG #(Options: DEBUG, ERROR, FATAL, INFO, WARN) for 'gov.utah.dts.monitor'
2019-09-30 12:46:44 | ERROR | [main] o.s.c.l.LoggingRebinder:83 - Cannot set level: DEBUG #(Options: ALL, DEBUG, ERROR, FATAL, INFO, WARN) for 'gov.utah.dts'
2019-09-30 12:46:44 | INFO | [main] g.u.d.m.MainApp:651 - The following profiles are active: test
2019-09-30 12:46:54 | INFO | [main] g.u.d.m.MainApp:59 - Started MainApp in 13.437 seconds (JVM running for 14.251)
12:47:14,053 |-INFO in ReconfigureOnChangeTask(born:1569847604053) - Empty watch file list. Disabling
12:47:44,053 |-INFO in ReconfigureOnChangeTask(born:1569847604053) - Empty watch file list. Disabling
12:48:14,053 |-INFO in ReconfigureOnChangeTask(born:1569847604053) - Empty watch file list. Disabling
12:48:44,054 |-INFO in ReconfigureOnChangeTask(born:1569847604053) - Empty watch file list. Disabling
12:49:14,053 |-INFO in ReconfigureOnChangeTask(born:1569847604053) - Empty watch file list. Disabling
12:49:44,052 |-INFO in ReconfigureOnChangeTask(born:1569847604053) - Empty watch file list. Disabling
This code would make one think that things are running smoothly, but what I do not see on the rabbitMq server, itself, is a consumer that should be this application. That is why I believe it is not working.
When I run this application non-dockerized, then I see the consumer for the specified queue.
Here are application-test.properties:
#Port Range 11370 - 11379
server:
port: 11371
spring:
## Rabbit
rabbitmq:
host: 172.23.2.12
port: 5672
username: guest
password: guest
## MongoDB
data:
mongodb:
host: 172.23.1.242
port: 20000
database: **********
username: **********
password: **********
# Only required if you're hitting a local database w/o username and pswd
uri: mongodb://**********:**********#***.***.***.242:20000/**********
rabbitmq:
queue: erepMonitorQueue
durable: true
exchange: erepMonitorExchg
route: erepMonitorRoute
logging:
level:
root: WARN
# Only print warning messages and above from external libraries
gov.utah.dts: DEBUG #(Options: ALL, DEBUG, ERROR, FATAL, INFO, WARN)
gov.utah.dts.monitor: DEBUG #(Options: DEBUG, ERROR, FATAL, INFO, WARN)
org.springframework: INFO #(Options: DEBUG, ERROR, FATAL, INFO, WARN)

Configuring logback with Storm

I have a storm project for which I have added some custom logback configurations using logback.xml (added configs to write logs to an index). When I build the jar and deploy the same on the cluster, I get the multiple bindings errors for this class StaticLoggerBinder.class due to it being present both in my jar and storm's own dependency list. Below is the error
SLF4J: Found binding in [jar:file:/usr/lib/storm/apache-storm-1.2.2/lib/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/app/test/test-storm-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
This causes none of my custom configs to be picked up.
When I remove the conflicting jar from Storm's lib folder, my logback.xml configs are reflected and my logs are written to the index with no issues. I also see these logs in the beginning when I deploy the topology showing the configs I have added
22:05:13,818 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [jar:file:/app/test/test-storm-0.0.55-SNAPSHOT-jar-with-dependencies.jar!/logback.xml]
22:05:13,839 |-INFO in ch.qos.logback.core.joran.spi.ConfigurationWatchList#19bb089b - URL [jar:file:/app/test/test-storm-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/logback.xml] is not of type file
22:05:14,094 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
22:05:14,097 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [CONSOLE]
22:05:14,102 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
22:05:14,125 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [com.splunk.logging.HttpEventCollectorLogbackAppender]
22:05:14,126 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [SPLUNK]
22:05:14,132 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [com.test] to INFO
22:05:14,132 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [com.test] to false
22:05:14,132 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [SPLUNK] to Logger[com.test]
22:05:14,133 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [CONSOLE] to Logger[com.test]
22:05:14,133 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
22:05:14,133 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [CONSOLE] to Logger[ROOT]
22:05:14,133 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
However, I do not want to take this route since it cannot be maintained so I decided use the maven-shade-plugin to relocate the org.slf4j.impl to org.shaded.slf4j.impl. Now when I deploy the topology there is no multiple bindings anymore, however my logback.xml is not honored and none of my configs are reflected nor do I see the above messages. It probably falls back to using a different logger.
How can I make my configs to be reflected when I deploy the topology? Please help. (Storm Version 1.2.2)
Storm switched from Logback to Log4j2 a few years ago. You can configure Log4j2 via the storm/log4j2/worker.xml and storm/log4j2/cluster.xml files. The former configures logging for your topology, as well as the worker JVM.
You should exclude logback from your fat jar.
Please also see this answer https://stackoverflow.com/a/50229035/8845188.

Spring-boot Events Missing Using Logback ServerSocketReceiver

Symptom: spring-boot basic app (no web, no jpa, no addons) defines a receiver in logback-spring.xml, but remote appender events never reach the local appenders.
logback-spring.xml:
<configuration debug="true">
<receiver name="logsink" class="ch.qos.logback.classic.net.server.ServerSocketReceiver">
<port>6004</port>
</receiver>
<appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss} %-5level %logger{25} - %X{client} %msg %n</pattern>
</encoder>
</appender>
<appender name="crap" class="org.oclc.hadoop.logging.logbacksinksb.service.CrapAppender"/>
<root level="DEBUG">
<appender-ref ref="stdout"/>
<appender-ref ref="crap"/>
</root>
</configuration>
The crapAppender is stripped to bare-bones for this problem:
public class CrapAppender extends AppenderBase<ILoggingEvent> {
#Override
protected void append(ILoggingEvent eventObject) {
System.out.println("got event!");
}
}
On startup, the log says my components are being instantiated:
16:39:12,103 |-INFO in ch.qos.logback.classic.joran.action.ReceiverAction - About to instantiate receiver of type [ch.qos.logback.classic.net.server.ServerSocketReceiver]
16:39:12,119 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
16:39:12,119 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [stdout]
16:39:12,119 |-INFO in ch.qos.logback.classic.net.server.RemoteAppenderServerRunner#228dc826 - listening on 0.0.0.0:6004
16:39:12,120 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
16:39:12,135 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [org.oclc.hadoop.logging.logbacksinksb.service.CrapAppender]
16:39:12,135 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [crap]
16:39:12,136 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to DEBUG
16:39:12,136 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [stdout] to Logger[ROOT]
16:39:12,136 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [crap] to Logger[ROOT]
16:39:12,136 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
16:39:12,136 |-INFO in org.springframework.boot.logging.logback.SpringBootJoranConfigurator#1dd02175 - Registering current configuration as safe fallback point
16:39:12,138 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#23bb8443 - Propagating INFO level on Logger[org.springframework] onto the JUL framework
I can even see that the remote appender has connected to the receiver port (6004):
java 5935 jamiesoh 32u IPv6 0xaf52811569ae62a3 0t0 TCP localhost:54160->localhost:6004 (ESTABLISHED)
java 6642 jamiesoh 53u IPv6 0xaf52811569ae5d63 0t0 TCP *:6004 (LISTEN)
java 6642 jamiesoh 54u IPv6 0xaf5281155d876d63 0t0 TCP localhost:6004->localhost:54160 (ESTABLISHED)
The local appenders only emit local events; the remote appender never reaches the local crap appender.
Any one have any ideas on what is preventing the remote events from passing thru the local appenders?
Just in case this bites someone else, I found the cause. One of the other artifacts had a dependency on logback-classic:1.2.3; the spring-boot version is 1.1.11. Excluding this transitive dependency corrected the problem.
Doctor, heal thyself...

spring JNDI logging before logback is initialized

I've got a spring boot app that is outputing around 1500 lines of JNDI debug logging to stdout at startup before logback is initialized. Does anyone know what's causing this and if there is a way to get logback initialized before this? I've got -Dlogback.debug=true but it doesn't help much.
10:44:56.453 [main] DEBUG org.springframework.jndi.JndiTemplate - Looking up JNDI object with name [java:comp/env/logging.exception-conversion-word]
10:44:56.457 [main] DEBUG jndi - InitialContextFactory.getInitialContext()
10:44:56.457 [main] DEBUG jndi - Created initial context delegate for local namespace:org.eclipse.jetty.jndi.local.localContextRoot#564fabc8
10:44:56.457 [main] DEBUG org.eclipse.jetty.jndi.java.javaURLContextFactory - >>> new root context requested
10:44:56.457 [main] DEBUG jndi - Looking up name="comp/env/logging.exception-conversion-word"
10:44:56.457 [main] DEBUG jndi - Trying thread context classloader
10:44:56.457 [main] DEBUG jndi - Looking up name="env/logging.exception-conversion-word"
10:44:56.458 [main] DEBUG org.springframework.jndi.JndiLocatorDelegate - Converted JNDI name [java:comp/env/logging.exception-conversion-word] not found - trying original name [logging.exception-conversion-word]. javax.naming.NameNotFoundException; remaining name 'env/logging.exception-conversion-word'
10:44:56.458 [main] DEBUG org.springframework.jndi.JndiTemplate - Looking up JNDI object with name [logging.exception-conversion-word]
...
10:44:56.487 [main] DEBUG jndi - Looking up name="env/LOGGING_PATTERNLEVEL"
10:44:56.487 [main] DEBUG org.springframework.jndi.JndiLocatorDelegate - Converted JNDI name [java:comp/env/LOGGING_PATTERNLEVEL] not found - trying original name [LOGGING_PATTERNLEVEL]. javax.naming.NameNotFoundException; remaining name 'env/LOGGING_PATTERNLEVEL'
10:44:56.487 [main] DEBUG org.springframework.jndi.JndiTemplate - Looking up JNDI object with name [LOGGING_PATTERNLEVEL]
10:44:56.487 [main] DEBUG jndi - InitialContextFactory.getInitialContext()
10:44:56.487 [main] DEBUG jndi - Created initial context delegate for local namespace:org.eclipse.jetty.jndi.local.localContextRoot#bcec361
10:44:56.487 [main] DEBUG jndi - Looking up name="LOGGING_PATTERNLEVEL"
10:44:56.487 [main] DEBUG org.springframework.jndi.JndiPropertySource - JNDI lookup for name [LOGGING_PATTERNLEVEL] threw NamingException with message: null. Returning null.
10:44:56,536 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
10:44:56,539 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
10:44:56,543 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
10:44:56,561 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
10:44:56,564 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
10:44:56,565 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
10:44:56,571 |-INFO in my.log.TimestampRollingPolicy#24fcf36f - Will use the pattern /tmp/logs/qs.%d{yyyyMMdd-HHmmss}.log to archive files
10:44:56,574 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: /tmp/logs/qs.log
10:44:56,574 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [/tmp/logs/qs.log]
10:44:56,576 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
10:44:56,576 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#4a7f959b - Propagating WARN level on Logger[ROOT] onto the JUL framework
10:44:56,576 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
10:44:56,576 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
10:44:56,576 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
10:44:56,577 |-INFO in org.springframework.boot.logging.logback.SpringBootJoranConfigurator#32b260fa - Registering current configuration as safe fallback point
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v1.4.4.RELEASE)
10:44:56,750 |-INFO in c.q.l.co.rolling.helper.RenameUtil - Renaming file [/tmp/logs/qs.log] to [/tmp/logs/qs.20170222-173130.log]
10:44:56 [main] INFO my.Application - Starting ...
I also saw this with Spring Boot 1.5.1 when using a custom logback-spring.xml. The debug statements don't appear if logback-spring.xml doesn't exist.
I was able to work around it by disabling the JNDIPropertySource. Create a file, spring.properties, in your classpath root:
# Disable the JNDIPropertySource. This avoids dozens of spurious DEBUG messages
# when starting up with a custom logback-spring.xml. We're not alone:
# http://stackoverflow.com/questions/42446013/spring-jndi-logging-before-logback-is-initialized
spring.jndi.ignore=true
spring.jndi.ignore was added in https://jira.spring.io/browse/SPR-14026.
This approach feels squicky, but as I'm not running in a container where JNDI would be useful, it doesn't break anything for me.
I also had the same problem, but I couldn't turn off jndi, because my application is using it on startup. So after some research, I ended up with the solution:
1) rename logback-spring.xml to some other name, for example logback-config.xml
2) put logging.config property into application.properties file.
As the result you configuration will be applied only on spring startup, and this debug logging will be skipped. Also in my case the first solution wasn't working because I needed jndi, in this case you also leave jndi enabled.
Update: this helped only for local environment, to make the same work on remote tomcat and remove this debug log from catalina.log file, I ended up with renaming this logback file to logback.xml, thus it's found on the startup, before jndi lookup.
you can avoid this by add this line to your logback xml file
<logger name="org.springframework.jndi" level="INFO" />

Including spark in spring boot project

I have java spring boot project(for web server), and it works find.
After that, I added spark depencies on this project, and it build without error.
In gradle
compile group: 'org.apache.spark', name: 'spark-core_2.11', version: '2.0.1'
compile group: 'org.apache.spark', name: 'spark-sql_2.11', version: '2.0.1'
BUT everytime spring web server got request, web server kept re-compiling itself.
some request output
|------------|------------|-----|------------|
// WHEN REQEUST GOT INTO, SPRING IS CLOSED AND RESTARTED ITSELF. WHY?????
17:26:19.052 INFO o.s.b.c.e.AnnotationConfigEmbeddedWebApplicationContext [] - [ : ]Closing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext#722b17b0: startup date [Mon Nov 21 17:26:02 KST 2016]; root of context hierarchy
17:26:19.055 INFO o.s.j.e.a.AnnotationMBeanExporter [] - [ : ]Unregistering JMX-exposed beans on shutdown
17:26:19,607 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
17:26:19,607 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [SQLFileAppender]
17:26:19,615 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - No compression will be used
17:26:19,615 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - Will use the pattern log/sql.%d{yyyy-MM-dd}.log for the active file
17:26:19,616 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is 'yyyy-MM-dd' from file name pattern 'log/sql.%d{yyyy-MM-dd}.log'.
17:26:19,616 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
17:26:19,616 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Mon Nov 21 17:26:19 KST 2016
17:26:19,618 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
17:26:19,622 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[SQLFileAppender] - Active log file name: log/sql.2016-11-21.log
17:26:19,623 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[SQLFileAppender] - File property is set to [null]
17:26:19,623 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
17:26:19,623 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [ControllerFileAppender]
17:26:19,623 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - No compression will be used
17:26:19,624 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - Will use the pattern log/controller.%d{yyyy-MM-dd}.log for the active file
17:26:19,624 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is 'yyyy-MM-dd' from file name pattern 'log/controller.%d{yyyy-MM-dd}.log'.
17:26:19,624 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
17:26:19,624 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Mon Nov 21 17:26:19 KST 2016
17:26:19,624 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
17:26:19,625 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ControllerFileAppender] - Active log file name: log/controller.2016-11-21.log
17:26:19,625 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ControllerFileAppender] - File property is set to [null]
17:26:19,625 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
17:26:19,625 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [console]
17:26:19,628 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
17:26:19,629 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#be00188 - Propagating INFO level on Logger[ROOT] onto the JUL framework
17:26:19,630 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [console] to Logger[ROOT]
17:26:19,630 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [jdbc] to OFF
17:26:19,630 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#be00188 - Propagating OFF level on Logger[jdbc] onto the JUL framework
17:26:19,630 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [jdbc.sqltiming] to INFO
17:26:19,630 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#be00188 - Propagating INFO level on Logger[jdbc.sqltiming] onto the JUL framework
17:26:19,630 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [SQLFileAppender] to Logger[jdbc.sqltiming]
17:26:19,630 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [jdbc.resultsettable] to INFO
17:26:19,630 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#be00188 - Propagating INFO level on Logger[jdbc.resultsettable] onto the JUL framework
17:26:19,630 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [Controller] to INFO
17:26:19,630 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator#be00188 - Propagating INFO level on Logger[Controller] onto the JUL framework
17:26:19,630 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [ControllerFileAppender] to Logger[Controller]
17:26:19,630 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
17:26:19,630 |-INFO in org.springframework.boot.logging.logback.SpringBootJoranConfigurator#6917246 - Registering current configuration as safe fallback point
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v1.3.2.RELEASE)

Resources