Springboot sending logs to fluentd not working - spring

I need some help for the following problem.
I have a spring boot application and I would like to configure a fluentd appender using logback.
I've created a file called logback.xml in my src/main/resources with the following content:
<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="true">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%date - %level - [%thread] - %logger - [%file:%line] - %msg%n</pattern>
</encoder>
</appender>
<appender name="FLUENT_TEXT" class="ch.qos.logback.more.appenders.DataFluentAppender">
<tag>dab</tag>
<label>normal</label>
<remoteHost>localhost</remoteHost>
<port>24224</port>
<maxQueueSize>20</maxQueueSize>
</appender>
<logger name="org.com" level="DEBUG"/>
<root level="DEBUG">
<appender-ref ref="STDOUT" />
<appender-ref ref="FLUENT_TEXT" />
</root>
</configuration>
In my build.gradle I have :
compile 'org.fluentd:fluent-logger:0.3.1'
compile 'com.sndyuk:logback-more-appenders:1.1.0'
When I launch the app using gradle bootRun I have the following message:
10:56:33,020 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - Attempted to append to non started appender [STDOUT].
10:56:33,020 |-WARN in ch.qos.logback.more.appenders.DataFluentAppender[FLUENT_TEXT] - Attempted to append to non started appender [FLUENT_TEXT].
10:56:33,028 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - Attempted to append to non started appender [STDOUT].
Exception in thread "main" 10:56:33,028 |-WARN in ch.qos.logback.more.appenders.DataFluentAppender[FLUENT_TEXT] - Attempted to append to non started appender [FLUENT_TEXT].
java.lang.NullPointerException
at ch.qos.logback.more.appenders.DataFluentAppender$FluentDaemonAppender.close(DataFluentAppender.java:72)
I've found here https://github.com/spring-projects/spring-boot/blob/master/spring-boot-docs/src/main/asciidoc/spring-boot-features.adoc something saying that logback.xml is loaded too early so I need to use a file called logback-spring.xml.
I've did it and it's like the file is never loaded, no error but nothing gets to my fluetd socket.
Any idea how to solve it ?
Thanks.
C.C.

When running your springboot application, load a 'spring' profile.
One way of doing it would be via the command line, see below.
-Dspring.profiles.active=spring

Related

How to clear log file before executing spring boot app?

I have been using the default spring logging configuration where I only specify filename in the application.properties file.
logging.file.name=app.log
But this by default appends logs when I start the application from cmd line "java -jar abc.jar"
I tried to search for the property which clears the file before starting application every time but couldn't find it. How should I clear the log file before starting the app?
Spring Boot uses Logback framework as as a default Logger.
You can use environnement variable via application.properties file to set some logging properties but logback xml configuration provides more powerfull features.
When a file in the classpath has one of the following names, Spring Boot will automatically load it over the default configuration:
logback-spring.xml
logback.xml
logback-spring.groovy
logback.groovy
So You can just put the code snippet below in src/main/resources/logback-spring.xml file
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="Console"
class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>
%black(%d{ISO8601}) %highlight(%-5level) [%blue(%t)] %yellow(%C{1.}): %msg%n%throwable
</Pattern>
</layout>
</appender>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>app.log</file>
<append>false</append>
<encoder>
<pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
</appender>
<!-- LOG everything at INFO level -->
<root level="info">
<appender-ref ref="FILE" />
<appender-ref ref="Console" />
</root>
</configuration>
The line <append>false</append> does the job.
if you wanna log more information than those avaible with the encoder pattern <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern> please check logback documention https://logback.qos.ch/manual/layouts.html#logback-access

SpringBoot Logback configuration error - Empty or null pattern

One of my spring boot applications suddenly stopped working, after server restart. The application is failing to start, these are the messages from log file
ERROR o.s.boot.SpringApplication - Application startup failed
java.lang.IllegalStateException: Logback configuration error detected:
ERROR in ch.qos.logback.classic.PatternLayout("") - Empty or null pattern.
Server: WebSphere Application Server
Spring Boot version: 1.5.21.RELEASE
logback-spring.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<include resource="org/springframework/boot/logging/logback/console-appender.xml"/>
<appender name="ROLLING-FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
<file>app.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>app.log.%d{yyyy-MM-dd}.log</fileNamePattern>
</rollingPolicy>
</appender>
<root level="INFO">
<appender-ref ref="ROLLING-FILE"/>
</root>
</configuration>
I tried setting the log parameters in the application properties file but still doesn't work, properties like logging.path, logging.file, logging.pattern.file
The application used to work fine, no code changes were made for last few months, but after server restart it stopped working, application works fine in my local machine.
For people that might still be looking for an answer. We were able to solve this by editing our logback-spring xml by removing:
<include resource="org/springframework/boot/logging/logback/console-appender.xml"/>
and if present (it was in our file but not in the code posted in the question) remove:
<appender-ref ref="CONSOLE"/>

Add datetime to log file name

I have some logs that I want to write to a file in a spring-boot microservice.
In my "application.yml" I have the below line which is working fine. But I also want to add date and time to the file name. How do I do that?
logging:
file: logs/${spring.application.name}.log
Ideally, I want the filename to be something like this:
logs/spring-boot-application_YYYY_MM_DD_HH_MM
A similar question has been answered here: How to include date in log file's name with Spring Boot / slf4j?
Basically if you want more control over the logging setup, you should have your own logback.xml under src/main/resources and configure the naming format similar to:
<FileNamePattern>LogFile.%d{yyyy-MM-dd}.log</FileNamePattern>
If you are using spring-boot , you could easily specify it in application.yml or application.properties and configuring logback xml to enable Rolling File Appender as :
spring:
logging:
file: logs/dev_app.log
pattern:
console: "%d %-5level %logger : %msg%n"
file: "%d %-5level [%thread] %logger : %msg%n"
level:
org.springframework.web: DEBUG
guru.springframework.controllers: DEBUG
org.hibernate: DEBUG
and specify the time based rolling policy for the file in :
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<appender name="ROLLIN" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_FILE}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>${LOG_FILE}.%d{yyyy-MM-dd}.log</fileNamePattern>
</rollingPolicy>
</appender>
<root level="INFO">
<appender-ref ref="ROLLIN" />
</root>
<logger name="org.springframework.web" level="INFO"/>
</configuration>
Doc
Official Spring Doc

How to ignore logback RollingFileAppender FileNotFoundException

I am using logback in my spring boot project, but logback log file must locate in /home/xxx/logs folder.
spring boot cannot start caused by RollingFileAppender's FileNotFoundException exception in my MacOS machine, because of MacOS cannot create folder /home/xxx/logs.
How to ignore this Exception in my spring boot?
As much I know logging framework don't provide any exception handling capability. It's not their job. Either you Correct log location and syntax or just remove config which you don't need.
Include a file logback-spring.xml in your classpath or resources folder and then you will be able to explicitly configure the location of the log file or you may have it print to the console instead. The config below would do just that and override the default config that comes with spring boot.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{dd-MM-yyyy HH:mm:ss.SSS} %magenta([%thread]) %highlight(%-5level) %logger{36}.%M - %msg%n</pattern>
</encoder>
</appender>
<root level="info">
<appender-ref ref="STDOUT" />
</root>
</configuration>

How to implement logging in custom module in Spring XD?

It's not so easy to debug custom module deployed to the Spring XD runtime (version 1.3.1-RELEASE).
I'm aware of log sink, however it's something different that I want to achieve.
I'd like to add my own log messages to the XD log (ideally to the STDOUT alongside it's own logs). These log messages are generated in my custom module (processor in this case) using slf4j API.
I've added:
org.slf4j.Logger#info invocation to the processor class
logback-classic dependency to the pom.xml (w/o a version, as it's managed by spring-xd-module-parent dependencyManagement
logback.xml to the resources directory
logback-test.xml to the test resources directory
Log messages are logged into STDOUT during integration test invocation (via SingleNodeIntegrationTestSupport), however they don't appear in the XD log when module is uploaded or stream using it is deployed.
logback.xml contents (identical for -test):
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
<charset>UTF-8</charset>
</encoder>
</appender>
<logger name="com.maxromanovsky" level="debug" />
<logger name="org.springframework" level="warn" />
<logger name="org.apache.zookeeper" level="error" />
<root level="warn">
<appender-ref ref="STDOUT" />
</root>
The container logback configuration files can be found in xd/config (xd-container-logback.groovy and xd-singlenode-logback.groovy).
You need to add your custom logger configuration there.

Resources