I have a Springboot application that creates multiple log files using slf4j(logback.xml) throughout day and along with it my application archives some necessary files throughout the day in same logs/archived-YYYY-MM-DD folder.
I am able to create a directory under logs/archived-YYYY-MM-DD Eg: logs/archived-2022-08-26 using logback.xml(below). Is it possible via slf4j when the next day folder is created the previous day is zipped(archived-2022-08-26.zip) and rolledover for only kept for last 3 days.
Here is the snippet of my current logback.xml
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<property resource="application.properties" />
<property name="APP_LOG_PATTERN" value="${logback.log.app.pattern}"/>
<property name="APP_LOG_ROOT" value="${logback.app.log.root}"/>
<appender name="FILE-LOGGING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${APP_LOG_ROOT}/application.log</file>
<encoder>
<pattern>${APP_LOG_PATTERN}</pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>logs/archived-%d{yyyy-MM-dd}/application.%i.log</fileNamePattern>
<maxFileSize>10MB</maxFileSize>
<totalSizeCap>2GB</totalSizeCap>
<maxHistory>3</maxHistory>
</rollingPolicy>
</appender>
<appender name="SERVICEA-LOGGING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${APP_LOG_ROOT}/serviceA.log</file>
<encoder>
<pattern>${APP_LOG_PATTERN}</pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>logs/archived-%d{yyyy-MM-dd}/serviceA.%i.log</fileNamePattern>
<maxFileSize>10MB</maxFileSize>
<totalSizeCap>2GB</totalSizeCap>
<maxHistory>3</maxHistory>
</rollingPolicy>
</appender>
<root name="application-log" level="info" additivity="false">
<appender-ref ref="FILE-LOGGING"/>
</root>
<logger name="token-log" level="info" additivity="false">
<appender-ref ref="SERVICEA-LOGGING" />
</logger>
</configuration>
Related
I have a spring boot microservice application. logback.xml is configured to write the logs to a location on the server. In addition to this, I want to write logs from a particular java class to Syslog. I do not want logs from all other classes to be written to syslog but to a rolling file appender. Is there a way to configure this using logback.
below is my logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="DEV_HOME" value="logs"></property>
<appender name="Console"
class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>%black(%d{ISO8601}) %highlight(%-5level) [%blue(%t)] %yellow(%C{1.}): %msg%n%throwable
</Pattern>
</layout>
</appender>
<appender name="RollingFile"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>/data/storage/log/cms.log</file>
<encoder
class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>%d %p %C{1.} [%t] %m%n</Pattern>
</encoder>
<rollingPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<!-- roll over daily and when the file reaches 10 MB, max of 7 days or 3GB threshold -->
<fileNamePattern>/data/storage/log/cms.%d{yyyy-MM-dd}.%i.Logs.gz
</fileNamePattern>
<maxFileSize>100MB</maxFileSize>
<maxHistory>7</maxHistory>
<totalSizeCap>3GB</totalSizeCap>
</rollingPolicy>
</appender>
<appender name="SPECIFIC_CLASS" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${DEV_HOME}/sizeTimeOutputlogFile.log</file>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>[%d{yyyy-MM-dd HH:mm:ss}] %p %c{1.} [%t]- %m%n</pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${DEV_HOME}/archived/sizeTimeOutputlogFile.%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<!--Max Size of file to start Archive -->
<maxFileSize>10KB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<!-- Days till log history to keep -->
<maxHistory>3</maxHistory>
</rollingPolicy>
</appender>
<!--syslog appender-->
<appender name="SYSLOG" class="ch.qos.logback.classic.net.SyslogAppender">
<syslogHost>localhost</syslogHost>
<facility>LOCAL0</facility>
<port>514</port>
<throwableExcluded>true</throwableExcluded>
<suffixPattern>%thread: %-5level %logger{36} - %msg%n</suffixPattern>
</appender>
<!-- LOG everything at ERROR level -->
<root level="ERROR">
<appender-ref ref="RollingFile"/>
<appender-ref ref="Console"/>
</root>
<!-- LOG "com.myproject*" at ERROR level -->
<logger name="com.myproject" level="ERROR" additivity="false">
<appender-ref ref="RollingFile"/>
<appender-ref ref="Console"/>
</logger>
<logger name="com.myproject.ActivityLogAspect" level="DEBUG" additivity="false">
<appender-ref ref="SYSLOG"/>
<appender-ref ref="SPECIFIC_CLASS"/>
</logger>
</configuration>
From the above XML, SYSLOG and SPECIFIC_CLASS appenders are used to write logs from ActivityLogAspect class. When I run my application I see that only SPECIFIC_CLASS is working. I don't see any logs written to /var/log/messages. But if I add the SYSLOG appender along with the console appender, I see the messages written to /var/log/messages.
I do not want logs from all classes written to syslog but only from one particular class.
Please advise.
I use logback to print log in my SpringBoot application. When I check out the log file, I found that, all log message print twice! It is so strange .
Yeah, I have found some answer similar to my issue. But maybe they are not what I want.
here is my logback.xml:
<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="false">
<property name="LOG_HOME" value="${user.home}/app/logs"/>
<appender name="Console" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS}-[%thread]-%-5level-%logger{50}: %msg%n</pattern>
</encoder>
</appender>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${LOG_HOME}/cloud-sync-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<!-- each file should be at most 10MB, keep 3 days worth of history, but at most 1GB -->
<maxFileSize>10MB</maxFileSize>
<maxHistory>3</maxHistory>
<totalSizeCap>1GB</totalSizeCap>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
</encoder>
</appender>
<appender name ="ASYNC" class= "ch.qos.logback.classic.AsyncAppender">
<discardingThreshold >0</discardingThreshold>
<queueSize>256</queueSize>
<includeCallerData>true</includeCallerData>
<appender-ref ref ="FILE"/>
</appender>
<root level="INFO">
<appender-ref ref="Console"/>
<appender-ref ref="FILE"/>
<appender-ref ref="ASYNC"/>
</root>
</configuration>
How can I make sure log messages appear only once. Thank you.
It is because you have added ASYNC and FILE for INFO.
Please remove one of them based on your requirement.
Any help or hint would be greatly appreciated!!
My SpringBoot application is logging only 1 day. I want it to log more than 30 days.
logback-spring.xml
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<charset>UTF-8</charset>
<pattern>${CONSOLE_LOG_PATTERN}</pattern>
</encoder>
</appender>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<encoder>
<charset>UTF-8</charset>
<pattern>${FILE_LOG_PATTERN}</pattern>
</encoder>
<file>${LOG_FILE}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${LOG_FILE}.%i</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>10</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<MaxFileSize>10MB</MaxFileSize>
</triggeringPolicy>
</appender>
<logger name="ca.test.hub" level="INFO" additivity="false">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="FILE"/>
</logger>
<logger name="org.apache.cxf" level="INFO" additivity="false">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="FILE"/>
</logger>
<logger name="org.hibernate" level="DEBUG" additivity="false">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="FILE"/>
</logger>
<logger name="org.springframework" level="INFO" additivity="false">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="FILE"/>
</logger>
application.yml:
logging:
file: logs/test.log
pattern:
console: "%d{dd-MM-yyyy HH:mm:ss.SSS} [%thread] %-5level %logger{36}.%M - %msg%n"
file: "%d{dd-MM-yyyy HH:mm:ss.SSS} [%thread] %-5level %logger{36}.%M - %msg%n"
level:
root: INFO
If you want to keep log files for more than 30 days your best bet is not to use FixedWindowRollingPolicy (documentation), which is used if you only want to limit the amount of log files that are kept on the file system.
A better appender suited for your requirements would be SizeAndTimeBasedRollingPolicy which, as stated in the documentation,
archive[s] files essentially by date but at the same time limit the size
of each log file
My suggested edit to your current FILE appender would be something like this
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<encoder>
<charset>UTF-8</charset>
<pattern>${FILE_LOG_PATTERN}</pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${LOG_FILE}-%d{yyyy-MM-dd}.%i</fileNamePattern>
<maxFileSize>10MB</maxFileSize>
<maxHistory>30</maxHistory>
</rollingPolicy>
</appender>
This SizeAndTimeBasedRollingPolicy stores files with the fileNamePattern LOG_FILE-2019-05-02.0 (LOG_FILE would of course be the value for the ${LOG_FILE} variable). This file will roll over every time it reaches a size larger that 10MB - and then a new file, LOG_FILE-2019-05-02.1, will be created and written to.
First log to be written after midnight would then create a new file, LOG_FILE-2019-05-03.0, and write to it. Log files older than 30 days will be deleted. Adjust the configuration to best suit your needs.
I'd recommend you read the documentation for this appender to get a bit more details.
I have been researching for this for a long time and doesn't get a solution at all. My requirement is to create a log file if it is not there at the place.
Below is my logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="LOG_PATH" value="/home/logs" />
<appender name="FILE-AUDIT"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_PATH}/debug.log</file>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%p\t%d{dd MMM yyyy HH:mm:ss,SSS}\t%r\t%c\t[--%t--]\t%m%n
</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${LOG_PATH}/debug.%d{yyyy-MM-dd}.%i.log
</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>10MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
<logger name="com.example" level="debug"
additivity="false">
<appender-ref ref="FILE-AUDIT" />
</logger>
<root level="debug">
<appender-ref ref="FILE-AUDIT" />
</root>
</configuration>
And I have tried changing the configuration to configuration debug="true", but no luck.
What is the right way to implement this ?
Please don't write to check on file permissions or ask me to use Log4j, I changed my application to use slf4j instead of log4j. If i delete the file also, it should create
You should use ch.qos.logback.core.FileAppender for file appending.
<appender name="File-Appender" class="ch.qos.logback.core.FileAppender">
<file>${LOG_PATH}/logfile-${timestamp-by-second}.log</file>
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
Since you've said that you worked with Log4j previously make sure you didn't exclude "spring-boot-starter-logging" from your .pom in tryings to avoid conflicts between Log4j and slf4j.
I'm using the same settings in logback.xml and everything is working fine
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="consoleAppender" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<Pattern>%d{HH:mm:ss} [%thread] %-5level %logger{36} - %msg %n</Pattern>
</encoder>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>TRACE</level>
</filter>
</appender>
<appender name="testServiceFileAppender" class="ch.qos.logback.core.rolling.RollingFileAppender">
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<FileNamePattern>logs/test_service_%d{yyyy-MM-dd}.log</FileNamePattern>
</rollingPolicy>
<encoder>
<Pattern>%-5level %logger{35} - %msg %n</Pattern>
</encoder>
</appender>
<logger name="com.mycompany.test" additivity="false">
<level value="INFO"/>
<appender-ref ref="testServiceFileAppender"/>
</logger>
<root>
<level value="INFO"/>
<appender-ref ref="consoleAppender"/>
</root>
</configuration>
in pom:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
</dependencies>
I want to change log format which will be written to console.
I'm using spring boot, so I'm including base.xml in logback.xml.
<include resource="org/springframework/boot/logging/logback/base.xml"/>
But I realised that if I use the code above and use appender="CONSOLE" to specify log format, my log message will be duplicate.
I know that if I comment the line above out, I can stop duplication, but I must specify all the settings which has been settle in the base.xml in that case.
Could someone give me advice how should I specify log format as I use base.xml?
Here is my logbook.xml:
<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="false">
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<logger name="jdbc" level="OFF" />
<logger name="jdbc.sqltiming" level="WARN" />
<logger name="jdbc.sqlonly" level="INFO" />
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%date{yyyy-MM-dd HH:mm:ss} [%thread] %level %logger{0} -
%msg \(%file:%line\)%n</pattern>
</encoder>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
</appender>
<appender name="FILE"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>c:/tmp/app.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>logs/app.%d{yyyy-MM-dd}.log.tar.gz</fileNamePattern>
<maxHistory>7</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd'T'HH:mm:ss'Z'} - %m%n</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="FILE" />
<appender-ref ref="CONSOLE" />
</root>
<logger name="FILE" additivity="false">
<level value="INFO" />
<appender-ref ref="FILE" />
</logger>
</configuration>
As of now You cant do that as a configuration. Please check the link. The feature has been requested but not yet implemented.