how to include xml in log-back.xml - spring-boot

I am writing a custom spring boot starter project for logging. My logback-spring.xml become large and want to put the appenders in separate file (console-appender.xml and file-appender.xml) and want to include in the logback-spring.xml file like below. I place all the three xmls in src/main/resources folder in my custom starter project.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<springProperty scope="context" name="JSONLOGSENABLED" source="spring.logs.json.enabled" defaultValue="true" />
<springProperty scope="context" name="JSONLOGSAPPENDER" source="spring.logs.json.appender" defaultValue="console" />
<if condition='property("JSONLOGSENABLED").equalsIgnoreCase("true")'>
<then>
<include file="{path to file}/console-appender.xml"/>
<include file="{path to file}/file-appender.xml"/>
<root level="INFO">
<if condition='property("JSONLOGSAPPENDER").equalsIgnoreCase("file")'>
<then>
<appender-ref ref="FILEJSON" />
</then>
<else>
<appender-ref ref="CONSOLEJSON" />
</else>
</if>
</root>
</then>
</if>
<logger
name="org.springframework.web.filter.CommonsRequestLoggingFilter">
<level value="DEBUG" />
</logger>
</configuration>
file-appender.xml
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>testFile.log</file>
<append>true</append>
<!-- set immediateFlush to false for much higher logging throughput -->
<immediateFlush>true</immediateFlush>
<!-- encoders are assigned the type
ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
</appender>
But when use this starter in my spring boot application, it is not able to resolve the console and file appender files.
I have tried putting following paths :
<include file="/src/main/resources/console-appender.xml"/>
<include file="./console-appender.xml"/>
<include resource="console-appender.xml"/>
How to include files properly in this case ?

Yes it's possible. Logback uses Joran so you have to follow the Joran rules.
For more info see documentation.
Anyway this is the preferred way to do what you're looking for.
For instance I define two files, then you adapt it as you prefer.
logback-spring.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="console-appender.xml"/>
<root level="debug">
<appender-ref ref="CONSOLE"/>
</root>
</configuration>
console-appender.xml
<included>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%date{"yyyy-MM-dd HH:mm:ss.SSS", Europe/Rome} [%mdc{id}] [%-11thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
</included>
Please note the <included> tag is mandatory

Related

logback-spring. is unable to read property value from application.yml

My logback-spring.xml reads fine from application.properties but not from application.yml. In my project, we have been asked to only use the YAML format as this format is being used in other microservices within the same project, so I cannot add propertes file. please help me why my application.yml is not read in logback.xml
I have tried variety of ways and searched similar questions on stackoverflow but no question has correct answer**. please dont mark this as duplicate**.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<include resource="org/springframework/boot/logging/logback/console-appender.xml"/>
<property resource ="application.yml"/>
<property name="LOGS" value="./logs" />
<appender name="Console"
class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>
%black(%d{ISO8601}) %highlight(%-5level) [%blue(%t)] %yellow(%C{1.}): %msg%n%throwable
</Pattern>
</layout>
</appender>
<appender name="RollingFile"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOGS}/${spring.application.name}.log</file>
<encoder
class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>%d %p %C{1.} [%t] %m%n</Pattern>
</encoder>
<rollingPolicy
class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily and when the file reaches 10 MegaBytes -->
<fileNamePattern>${LOGS}/archived/spring-boot-logger-%d{yyyy-MM-dd}.%i.log
</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>10MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
<!-- LOG everything at INFO level -->
<root level="info">
<appender-ref ref="RollingFile" />
<appender-ref ref="Console" />
</root>
<!-- LOG "com.baeldung*" at TRACE level -->
<logger name="com.ms" level="trace" additivity="false">
<appender-ref ref="RollingFile" />
<appender-ref ref="Console" />
</logger>
</configuration>
Above is my logback-spring.xml. Please refer my application.yml below:-
spring:
application:
name: Logbacking-service
You can use below in your logback file as explained in the docs here
<springProperty name = "appname" source= "spring.application.name"/>
and then use this elsewhere
<file>${LOGS}/${appname}.log</file>
I tested the exact code you used and it was indeed giving issue so the above solution should work as it work for me as well. Earlier with your code the log file name that was getting generated was "appname_IS_UNDEFINED.log", and post the above change the name was "Logbacking-service.log".
If you enable trace logging level in your logback for "org.springframework.core.env.PropertySourcesPropertyResolver" you will see the location of application.yml from where it reads the properties. This will help you understand from where the spring boot is trying to find configuration. Something like below
Searching for key 'spring.profiles.active' in PropertySource 'applicationConfig: [classpath:/application.yml]

console pattern not recognized in logback-spring.xml

I already looked up on stackoverflow but I did not find something comparable.
I use spring boot with following dependencies:
spring-boot 2.2.1 with
spring-boot-starter-logging 2.2.1
logback-core 1.2.3
logback-classic 1.2.3
I want to use a custom pattern for my log output to print my mdc.variables on console. Which works via application.properties
logging.level.com.mypackage=TRACE
logging.pattern.level=%d{yyyy-MM-dd HH:mm:ss} [%t] [myMdc:%X{myMdcVariable}] %-5level %logger{36} - %m%n
But I get double entries but I can not set the additivity property via application.properties. I have to use logback-spring.xml. Which looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<include resource="org/springframework/boot/logging/logback/console-appender.xml"/>
<root level="INFO">
<appender-ref ref="CONSOLE"/>
</root>
<logger name="com.mypackage" level="TRACE" additivity="false">
<appender-ref ref="CONSOLE"/>
</logger>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%d{yyyy-MM-dd HH:mm:ss} [%t] [myMdc:%X{myMdcVariable}] %-5level %logger{36} - %m%n
</Pattern>
</encoder>
</appender>
</configuration>
This solves the double entries. But the pattern is not use for the logoutput. I can have either double entries or no pattern because it is not possible to combine configuration via application.properties and logback-spring.xml. What am I doing wrong?
I finally found out what is wrong. I checked the defaults.xml and the ConsoleAppender.xml on github e.g.
https://github.com/spring-projects/spring-boot/blob/master/spring-boot-project/spring-boot/src/main/resources/org/springframework/boot/logging/logback/defaults.xml
https://github.com/spring-projects/spring-boot/blob/master/spring-boot-project/spring-boot/src/main/resources/org/springframework/boot/logging/logback/console-appender.xml
The solution was to define the constant CONSOLE_LOG_PATTERN. And to define it before importing the defaults.xml and console-appender.xml
My working config file looks now like this:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="CONSOLE_LOG_PATTERN" value="%d{yyyy-MM-dd HH:mm:ss} [%t] [myMdc:%X{myMdcVariable] %-5level %logger{36} - %m%n"/>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<include resource="org/springframework/boot/logging/logback/console-appender.xml"/>
<root level="INFO">
<appender-ref ref="CONSOLE"/>
</root>
<logger name="com.mypackage" level="TRACE" additivity="false">
<appender-ref ref="CONSOLE"/>
</logger>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>
${CONSOLE_LOG_PATTERN}
</pattern>
</encoder>
</appender>
</configuration>

Logback I want File do not create when the appender is never used. [RollingFileAppender]

I'm creating a logback-common config file for my applis.
I'm defining a RollingFileAppender in it, that for all my applis generates the same log format in a file (if we need it).
Sometimes I want to use this appender, and sometimes not (when we test for example).
So we configure our specific logback-{profile}.xml depending what we want.
But when we do not use the FILE appender, the file gets created and I would like not.
I have:
logback-common.xml >> with all appenders definition (FILE and COMMON)
appli_one
resources/logback.xml >> call logback-common and config/logback-{profile}.xml
resources/config/logback-{profile}.xml >> specific appli/profile logback configuration.
To configure we can do in logback-{profile}.xml
<root level="WARN">
<appender-ref ref="FILE"/> <!-- For File Log when we need it -->
<appender-ref ref="CONSOLE"/>
</root>
<root level="WARN">
<!-- <appender-ref ref="FILE"/> --> <!-- in comment when we do not need if > BUT create a empty file -->
<appender-ref ref="CONSOLE"/>
</root>
logback-common.xml
<included>
<!-- The FILE and ASYNC appenders are here as examples for a production
configuration -->
<appender name="FILE"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>log/${spring.application.name}.log</file>
<rollingPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>
log/${spring.application.name}.%d{yyyy-MM-dd}.%i.log.zip
</fileNamePattern>
<maxHistory>90</maxHistory>
<maxFileSize>10MB</maxFileSize>
</rollingPolicy>
<encoder>
<charset>utf-8</charset>
<Pattern>%d %-5level [%thread] %logger{0}: %msg%n</Pattern>
</encoder>
</appender>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>"%d{yyyy-MM-dd} [%thread] %-5level %45logger{45} - %msg%n"</pattern>
</encoder>
</appender>
</included>
logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" packagingData="true">
<property name="spring_profiles_active" value="${spring.profiles.active}" />
<property resource="application.properties"/>
<include resource="config/log/logback-common.xml"/>
<include resource="config/log/logback-${spring_profiles_active}.xml"/>
</configuration>
Not a core feature of logback but there are workarounds to achieve lazy file initialization.
See more here
Logback - do not create empty log files at startup

Overriding logback.xml with Spring Boot 1.5.x application

I'm trying to write logback configuration in Spring Boot(1.5.x) application.
Problem -
The problem is that it doesn't create any log file/folder (or may be not in correct path).
CODE -
logback.xml
<configuration>
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<property name="LOG_PATH" value="logs" />
<appender name="ROLLING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_PATH}/mylog.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${LOG_PATH}/mylog-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<!-- each file should be at most 100MB, keep 60 days worth of history, but at most 20GB -->
<maxFileSize>100MB</maxFileSize>
<maxHistory>60</maxHistory>
<totalSizeCap>20GB</totalSizeCap>
</rollingPolicy>
<encoder>
<pattern>%msg%n</pattern>
</encoder>
</appender>
</configuration>
Either pass the LOG_PATH while variable while running the application or define it in the same file as follows <property name="LOG_PATH" value="logs"/>
And also add
<root level="INFO">
<appender-ref ref="ROLLING"/>
</root>
Add this at the end before </configuration>
<root level="INFO">
<appender-ref ref="ROLLING"/>
</root>

How to avoid duplication of log message when I want to change log format?

I want to change log format which will be written to console.
I'm using spring boot, so I'm including base.xml in logback.xml.
<include resource="org/springframework/boot/logging/logback/base.xml"/>
But I realised that if I use the code above and use appender="CONSOLE" to specify log format, my log message will be duplicate.
I know that if I comment the line above out, I can stop duplication, but I must specify all the settings which has been settle in the base.xml in that case.
Could someone give me advice how should I specify log format as I use base.xml?
Here is my logbook.xml:
<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="false">
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<logger name="jdbc" level="OFF" />
<logger name="jdbc.sqltiming" level="WARN" />
<logger name="jdbc.sqlonly" level="INFO" />
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%date{yyyy-MM-dd HH:mm:ss} [%thread] %level %logger{0} -
%msg \(%file:%line\)%n</pattern>
</encoder>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
</appender>
<appender name="FILE"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>c:/tmp/app.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>logs/app.%d{yyyy-MM-dd}.log.tar.gz</fileNamePattern>
<maxHistory>7</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd'T'HH:mm:ss'Z'} - %m%n</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="FILE" />
<appender-ref ref="CONSOLE" />
</root>
<logger name="FILE" additivity="false">
<level value="INFO" />
<appender-ref ref="FILE" />
</logger>
</configuration>
As of now You cant do that as a configuration. Please check the link. The feature has been requested but not yet implemented.

Resources