I have been using the default spring logging configuration where I only specify filename in the application.properties file.
logging.file.name=app.log
But this by default appends logs when I start the application from cmd line "java -jar abc.jar"
I tried to search for the property which clears the file before starting application every time but couldn't find it. How should I clear the log file before starting the app?
Spring Boot uses Logback framework as as a default Logger.
You can use environnement variable via application.properties file to set some logging properties but logback xml configuration provides more powerfull features.
When a file in the classpath has one of the following names, Spring Boot will automatically load it over the default configuration:
logback-spring.xml
logback.xml
logback-spring.groovy
logback.groovy
So You can just put the code snippet below in src/main/resources/logback-spring.xml file
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="Console"
class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>
%black(%d{ISO8601}) %highlight(%-5level) [%blue(%t)] %yellow(%C{1.}): %msg%n%throwable
</Pattern>
</layout>
</appender>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>app.log</file>
<append>false</append>
<encoder>
<pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
</appender>
<!-- LOG everything at INFO level -->
<root level="info">
<appender-ref ref="FILE" />
<appender-ref ref="Console" />
</root>
</configuration>
The line <append>false</append> does the job.
if you wanna log more information than those avaible with the encoder pattern <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern> please check logback documention https://logback.qos.ch/manual/layouts.html#logback-access
I prepared a custom RollingFileAppender configuration in the logback-spring.xml file and application.properties files. The log file soduncu.log created under path /var/app/sefa/logs. This is expected behavior for the configuration but there is an unexpected situation that soduncu.log created under default Linux log path /var/log/. I tried to prevent this unexpected log file creation but I couldn't. What is the situation here and what I did wrong?
logback-spring.xml:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<springProperty name="LOG_DIR" source="logging.path" defaultValue="/var/app/sefa/logs">
</springProperty>
<appender name="ROLLING"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_DIR}/soduncu.log</file>
<rollingPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${LOG_DIR}/soduncu-%d{yyyy-MM-dd}.%i.log
</fileNamePattern>
<maxFileSize>10MB</maxFileSize>
<maxHistory>60</maxHistory>
<totalSizeCap>20GB</totalSizeCap>
</rollingPolicy>
<encoder>
<pattern>%date %-4relative [%thread] %-5level %logger{35} - %msg %n</pattern>
</encoder>
</appender>
<root level="WARN">
<appender-ref ref="ROLLING" />
</root>
</configuration>
application.properties:
logging.path=/var/app/sefa/logs
logging.file=/var/app/sefa/logs/soduncu.log
logging.level.root=INFO
logging.level.org.springframework=ERROR
logging.level.com.nuvia=DEBUG
Here is the one Linux process id link to a log file under two different path:
/proc/14368/fd/1 -> /var/log/soduncu.log
/proc/14368/fd/2 -> /var/log/soduncu.log
/proc/14368/fd/6 -> /var/app/sefa/logs/soduncu.log
I am unable to replicate this behavior with your code.
Try to change
name="LOG_DIR" source="logging.path"
to
name="LOG_DIR_PATH" source="log.dir.path"
and see what happens
make sure to delete soduncu.log from var/log first.
I hope it helps.
I have some logs that I want to write to a file in a spring-boot microservice.
In my "application.yml" I have the below line which is working fine. But I also want to add date and time to the file name. How do I do that?
logging:
file: logs/${spring.application.name}.log
Ideally, I want the filename to be something like this:
logs/spring-boot-application_YYYY_MM_DD_HH_MM
A similar question has been answered here: How to include date in log file's name with Spring Boot / slf4j?
Basically if you want more control over the logging setup, you should have your own logback.xml under src/main/resources and configure the naming format similar to:
<FileNamePattern>LogFile.%d{yyyy-MM-dd}.log</FileNamePattern>
If you are using spring-boot , you could easily specify it in application.yml or application.properties and configuring logback xml to enable Rolling File Appender as :
spring:
logging:
file: logs/dev_app.log
pattern:
console: "%d %-5level %logger : %msg%n"
file: "%d %-5level [%thread] %logger : %msg%n"
level:
org.springframework.web: DEBUG
guru.springframework.controllers: DEBUG
org.hibernate: DEBUG
and specify the time based rolling policy for the file in :
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<appender name="ROLLIN" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_FILE}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>${LOG_FILE}.%d{yyyy-MM-dd}.log</fileNamePattern>
</rollingPolicy>
</appender>
<root level="INFO">
<appender-ref ref="ROLLIN" />
</root>
<logger name="org.springframework.web" level="INFO"/>
</configuration>
Doc
Official Spring Doc
I am trying to get log output in JSON format. I achieved it by configuring logback.xml file. Is it possible to achieve same by YML file ?
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
<jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
<prettyPrint>true</prettyPrint>
</jsonFormatter>
<timestampFormat>yyyy-MM-dd' 'HH:mm:ss.SSS</timestampFormat>
</layout>
</appender>
If your goal is to have different configurations for different environments, you can use spring's profile dependent configuration feature
From my logback-spring.xml:
...
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<springProfile name="dev">
<encoder>
<pattern>${CONSOLE_LOG_PATTERN}</pattern>
<charset>utf8</charset>
</encoder>
</springProfile>
<springProfile name="qa,prod">
<encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
</springProfile>
</appender>
...
You should also check the paragraph about the Environment Properties
As mentioned above in comment It is not possible to configure only in YML file for JSON Layout in logback. My requirement was to set log level ERROR/INFO from YML file. I came to know We can override logback log level by YML file in spring boot. For JSON format configuration one can use logback.xml and for setting log level use YML file
Using spring boot 2.1.1.RELEASE one can seemingly format logs as JSON by providing a logback-spring.xml file as follows:
<appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
<timestampFormat>yyyy-MM-dd'T'HH:mm:ss.SSSX</timestampFormat>
<timestampFormatTimezoneId>Etc/UTC</timestampFormatTimezoneId>
<jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
<prettyPrint>true</prettyPrint>
</jsonFormatter>
</layout>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="stdout" />
</root>
and adding to the pom.xml
<dependency>
<groupId>ch.qos.logback.contrib</groupId>
<artifactId>logback-json-classic</artifactId>
<version>0.1.5</version>
</dependency>
<dependency>
<groupId>ch.qos.logback.contrib</groupId>
<artifactId>logback-jackson</artifactId>
<version>0.1.5</version>
</dependency>
indeed leading to messages like:
{
"timestamp" : "2018-12-11T18:20:25.641Z",
"level" : "INFO",
"thread" : "main",
"logger" : "com.netflix.config.sources.URLConfigurationSource",
"message" : "To enable URLs as dynamic configuration sources, define System property archaius.configurationSource.additionalUrls or make config.properties available on classpath.",
"context" : "default"
}
Why?
I'm trialing logz.io which appears to behave more favourably when logs are JSON formatted, some o the shippers struggle with multiline logs like we see in java stack traces and when formatting in JSON it can automatically parse fields like level and message and if there is MDC data it automatically gets that.
I had some not so great experiences with a few of the methods of shipping logs to logzio, like their docker image and using rsyslog without using JSON formatted log messages.
Issues With This Approach
It works ok for console appending, but spring boot provides like logging.file=test.log, logging.level.com.example=WARN, logging.pattern.console. I can indeed import the managed configuration from spring-boot-2.1.1.RELEASE.jar!/org/springframework/boot/logging/logback/base.xml which in turn imports a console-appender.xml andfile-appender.xml`.
An example of the console-appender
<included>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>${CONSOLE_LOG_PATTERN}</pattern>
</encoder>
</appender>
</included>
An example of the file appender
<included>
<appender name="FILE"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<encoder>
<pattern>${FILE_LOG_PATTERN}</pattern>
</encoder>
<file>${LOG_FILE}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${LOG_FILE}.%d{yyyy-MM-dd}.%i.gz</fileNamePattern>
<maxFileSize>${LOG_FILE_MAX_SIZE:-10MB}</maxFileSize>
<maxHistory>${LOG_FILE_MAX_HISTORY:-0}</maxHistory>
</rollingPolicy>
</appender>
</included>
These two are exactly what I need to support spring configuration of the properties, but they don't include the encoder/layout I'd need.
It appears in my initial tests that I can't simple name my appender the same as those and provide my layouts. For example:
<configuration>
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
<timestampFormat>yyyy-MM-dd'T'HH:mm:ss.SSSX</timestampFormat>
<timestampFormatTimezoneId>Etc/UTC</timestampFormatTimezoneId>
<jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
<prettyPrint>true</prettyPrint>
</jsonFormatter>
</layout>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="CONSOLE" />
</root>
</configuration>
leads to the message being logged in both JSON and plain text format.
I can indeed just copy and paste the contents of these 3 files into my custom config rather than import them at all. Then I may override what I want to customise.
However, as spring evolves and new releases are made which may add features, I'd be forever forcing myself to keep up, copy and paste the new files and make my changes and test them.
Is there any better way that I can either:
Just make additive changes to the appenders rather than entirely redefine them, e.g. keep the config from spring but provide my own encoder or layout to be used by those appenders.
Configure spring to JSON log via properties entirely without any config - I doubt this :S
Footnote: logzio do provide a dependency one can import, but I dislike the idea of coupling the logging provider into the code directly. I feel that if the servoce happens to produce JSON logs to stdout or a file, it's easy for any provider to process those and ship them to some destination.
I am not using any dependency.
Simply, doing it via application.yml, that's all.
This solution solves, multiline log issue, too.
logging:
pattern:
console: "{\"time\": \"%d\", \"level\": \"%p\", \"correlation-id\": \"%X{X-Correlation-Id}\", \"source\": \"%logger{63}:%L\", \"message\": \"%replace(%m%wEx{6}){'[\r\n]+', '\\n'}%nopex\"}%n"
I use something like the following, has always worked fine.
Spring Boot recommendation is to name the file logback-spring.xml and place it under src/main/resources/, this enables us to use spring profiles in logback.
So in the file below you will see that for LOCAL profile you can log in the standard fashion but for the deployments on the server or a container you can you a different logging strategy.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %5p [YourApp:%thread:%X{X-B3-TraceId}:%X{X-B3-SpanId}] %logger{40} - %msg%n
</pattern>
</encoder>
</appender>
<appender name="jsonstdout" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LogstashEncoder">
<providers>
<timestamp>
<timeZone>EST</timeZone>
</timestamp>
<pattern>
<pattern>
{
"level": "%level",
"service": "YourApp",
"traceId": "%X{X-B3-TraceId:-}",
"spanId": "%X{X-B3-SpanId:-}",
"thread": "%thread",
"class": "%logger{40}",
"message": "%message"
}
</pattern>
</pattern>
<stackTrace>
<throwableConverter class="net.logstash.logback.stacktrace.ShortenedThrowableConverter">
<maxDepthPerThrowable>30</maxDepthPerThrowable>
<maxLength>2048</maxLength>
<shortenedClassNameLength>20</shortenedClassNameLength>
<rootCauseFirst>true</rootCauseFirst>
</throwableConverter>
</stackTrace>
</providers>
</encoder>
</appender>
<root level="info">
<springProfile name="LOCAL">
<appender-ref ref="stdout" />
</springProfile>
<springProfile name="!LOCAL">
<appender-ref ref="jsonstdout" />
</springProfile>
</root>
</configuration>
Sounds like you need to copy paste with modifications 3 out of 4 files from here https://github.com/spring-projects/spring-boot/tree/v2.1.1.RELEASE/spring-boot-project/spring-boot/src/main/resources/org/springframework/boot/logging/logback into your configuration.
The good news is that you don't need to copy paste https://github.com/spring-projects/spring-boot/blob/v2.1.1.RELEASE/spring-boot-project/spring-boot/src/main/resources/org/springframework/boot/logging/logback/defaults.xml
That can be included like so <include resource="org/springframework/boot/logging/logback/default.xml"/>
That will get you some of spring's default config without copying and pasting
If you switch to log4j2 using the method specified in the Spring Boot documentation
implementation "org.springframework.boot:spring-boot-starter-log4j2"
modules {
module("org.springframework.boot:spring-boot-starter-logging") {
replacedBy("org.springframework.boot:spring-boot-starter-log4j2", "Use Log4j2 instead of Logback")
}
}
You can have a file log4j2.properties that contains the following.
appender.stdout.type=Console
appender.stdout.name=json
appender.stdout.json.type=JsonTemplateLayout
appender.stdout.json.eventTemplateUri=classpath:LogstashJsonEventLayoutV1.json
appender.console.type=Console
appender.console.name=console
rootLogger.appenderRef.stdout.ref=${env:LOGGING_APPENDER:-json}
I used LOGGING_APPENDER to match my environment level overrides.
Note one odd flaw though... this does not work when you're on JDK17.