I have to put all the log data (i.e., debug, info, error) into mysql database instead of to file/console.
I read the spring boot documentation but I didn't see any configuration related to database for logging.
https://docs.spring.io/spring-boot/docs/current/reference/html/howto-logging.html
Also tried the following link but its also not working.
https://www.tutorialspoint.com/log4j/log4j_logging_database.htm
Can anyone help me to do this.
Thanks.
I read the spring boot documentation but I didn't see any
configuration related to database for logging.
Because spring boot hands off that functionality to logging framework (slf4j + logback/log4j etc). So you need to configure your logging framework accordingly using it's configuration file (eg: logback.xml, logback-spring.xml, logback.groovy etc). Default logging frameworks in Spring boot is slf4j+logback. So checkout how you can use DBAppender.
For Logback
https://logback.qos.ch/manual/appenders.html#DBAppender
http://learningviacode.blogspot.com/2014/01/writing-logs-to-database.html
Log to database with LogBack
https://medium.com/#chakrar27/storing-log-data-in-postgresql-using-logback-db-appender-292891a9918
1. create logback.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n
</pattern>
</encoder>
</appender>
<appender name="db" class="ch.qos.logback.classic.db.DBAppender">
<connectionSource
class="ch.qos.logback.core.db.DriverManagerConnectionSource">
<driverClass>org.postgresql.Driver</driverClass>
<url>jdbc:postgresql://localhost:5432/simple</url>
<user>postgres</user>
<password>root</password> <!-- no password -->
</connectionSource>
</appender>
<!-- the level of the root level is set to DEBUG by default. -->
<root level="TRACE">
<appender-ref ref="stdout" />
<appender-ref ref="db" />
</root>
</configuration>
2. Create the 3 tables
logging_event
logging_event_property
logging_event_exception
They must exist before DBAppender can be used
For Log4J
https://logging.apache.org/log4j/2.x/manual/appenders.html#JDBCAppender
For Log4J2
http://smasue.github.io/log4j2-spring-database-appender
Related
Hi I am getting confused with setting up old version of Sentry(ver 1.7.30) with current Spring Boot project(ver 1.5.7).
Here's my part of Gradle:
dependencies {
compile("org.springframework.boot:spring-boot-starter-web")
compile("org.springframework.boot:spring-boot-starter-logging")
compile("org.springframework.boot:spring-boot-starter-actuator")
...
implementation 'io.sentry:sentry-spring:1.7.30'
implementation 'io.sentry:sentry-logback:1.7.30'
}
application-local.yml
sentry:
dsn: https://dsnPath
logback-spring.xml
<?xml version="1.0" encoding="UTF-8"?>
<!-- sentry -->
<configuration>
<appender name="Sentry" class="io.sentry.logback.SentryAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>WARN</level>
</filter>
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="info">
<appender-ref ref="FILE"/>
<appender-ref ref="Sentry" />
</root>
</configuration>
When I run the project on local laptop, I can see some exception logs on console but they don't show up on Sentry. One thing I'm sure is..there's log4j2-spring.xml before I created logback-spring.xml file only for Sentry and this might be a problem?
(I followed the instruction on official page - https://docs.sentry.io/platforms/java/legacy/spring/ - and it didn't work. Maybe I made mistake regarding resolver.)
Thanks in advance!
I have been using the default spring logging configuration where I only specify filename in the application.properties file.
logging.file.name=app.log
But this by default appends logs when I start the application from cmd line "java -jar abc.jar"
I tried to search for the property which clears the file before starting application every time but couldn't find it. How should I clear the log file before starting the app?
Spring Boot uses Logback framework as as a default Logger.
You can use environnement variable via application.properties file to set some logging properties but logback xml configuration provides more powerfull features.
When a file in the classpath has one of the following names, Spring Boot will automatically load it over the default configuration:
logback-spring.xml
logback.xml
logback-spring.groovy
logback.groovy
So You can just put the code snippet below in src/main/resources/logback-spring.xml file
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="Console"
class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>
%black(%d{ISO8601}) %highlight(%-5level) [%blue(%t)] %yellow(%C{1.}): %msg%n%throwable
</Pattern>
</layout>
</appender>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>app.log</file>
<append>false</append>
<encoder>
<pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
</appender>
<!-- LOG everything at INFO level -->
<root level="info">
<appender-ref ref="FILE" />
<appender-ref ref="Console" />
</root>
</configuration>
The line <append>false</append> does the job.
if you wanna log more information than those avaible with the encoder pattern <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern> please check logback documention https://logback.qos.ch/manual/layouts.html#logback-access
I am using logback in my spring boot project, but logback log file must locate in /home/xxx/logs folder.
spring boot cannot start caused by RollingFileAppender's FileNotFoundException exception in my MacOS machine, because of MacOS cannot create folder /home/xxx/logs.
How to ignore this Exception in my spring boot?
As much I know logging framework don't provide any exception handling capability. It's not their job. Either you Correct log location and syntax or just remove config which you don't need.
Include a file logback-spring.xml in your classpath or resources folder and then you will be able to explicitly configure the location of the log file or you may have it print to the console instead. The config below would do just that and override the default config that comes with spring boot.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{dd-MM-yyyy HH:mm:ss.SSS} %magenta([%thread]) %highlight(%-5level) %logger{36}.%M - %msg%n</pattern>
</encoder>
</appender>
<root level="info">
<appender-ref ref="STDOUT" />
</root>
</configuration>
When logging errors to stackdriver, every message is logged as INFO, even when using log.error or log.warn, etc., but the payload is correct.
I'd like to be able to filter by severity and get email on error.
I'm using Spring Boot and Logback. The app has been deployed on a Kubernetes Cluster on GCP.
Here is my logback-spring.xml
<configuration>
<include resource="org/springframework/cloud/gcp/autoconfigure/logging/logback-appender.xml" />
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type
ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern>%d{HH:mm:ss, UTC} %-5level %logger{35} - %msg %n</pattern>
</encoder>
</appender>
<springProfile name="prod,qa">
<root level="WARN">
<appender-ref ref="STACKDRIVER" />
</root>
</springProfile>
</configuration>
And here is the dep added in Maven
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-gcp-starter-logging</artifactId>
</dependency>
Spring Boot version: 2.1.3.RELEASE
Spring Cloud version: Greenwich.RELEASE
What is wrong with this config? Is there any other solution?
EDIT: Just realized that the STACKDRIVER appender above is not the one logging to Stackdriver, but STDOUT is enough (maybe bc it's a Kubernetes cluster?), but the issue persists
The Stackdriver logging agent configuration for Kubernetes defaults to INFO for any logs written to the container's stdout and ERROR for logs written to stderr. If you want finer-grained control over severity, you can configure Spring to log as single-line JSON (e.g., via JsonLayout1) and let the logging agent pick up the severity from the JSON object (see https://cloud.google.com/logging/docs/agent/configuration#process-payload).
1By default, JsonLayout will use "level" for the log level, while the Stackdriver logging agent recognizes "severity", so you may have to override addCustomDataToJsonMap.
See also GKE & Stackdriver: Java logback logging format?
Directly using google cloud logging logback appender takes the severity from the log level on each case:
In Maven:
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-logging-logback</artifactId>
<version>0.116.0-alpha</version>
</dependency>
In logback.xml:
<appender name="Cloud" class="com.google.cloud.logging.logback.LoggingAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
<log>YOUR_LOG_NAME</log>
<resourceType>container</resourceType>
<flushLevel>INFO</flushLevel>
</appender>
...
<logger name="org.springframework" level="WARN" additivity="true">
<appender-ref ref="Cloud"/>
</logger>
It is not the spring cloud component but solves the problem.
I want to use Graylog/RabbitMQ for logging with my spring boot microservices. As per my understanding I have to send my logs to RabbitMQ and have to integrate it with Graylog. I want to know the workflow and how to implement it like how to send the logs to RabbitMQ, do I need to use any other logging framework.
You can use Logback appender to send logs from spring-boot app. Add following dependency to your pom.xml
<dependency>
<groupId>de.siegmar</groupId>
<artifactId>logback-gelf</artifactId>
<version>1.1.0</version>
</dependency>
Then you need to add a logback configuration file to your classpath.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/base.xml" />
<appender name="GELF" class="de.siegmar.logbackgelf.GelfUdpAppender">
<graylogHost>localhost</graylogHost>
<graylogPort>12201</graylogPort>
<maxChunkSize>508</maxChunkSize>
<useCompression>true</useCompression>
<layout class="de.siegmar.logbackgelf.GelfLayout">
<originHost>localhost</originHost>
<includeRawMessage>false</includeRawMessage>
<includeMarker>true</includeMarker>
<includeMdcData>true</includeMdcData>
<includeCallerData>false</includeCallerData>
<includeRootCauseData>false</includeRootCauseData>
<includeLevelName>false</includeLevelName>
<shortPatternLayout class="ch.qos.logback.classic.PatternLayout">
<pattern>%m%nopex</pattern>
</shortPatternLayout>
<fullPatternLayout class="ch.qos.logback.classic.PatternLayout">
<pattern>%m</pattern>
</fullPatternLayout>
<staticField>app_name:backend</staticField>
<staticField>os_arch:${os.arch}</staticField>
<staticField>os_name:${os.name}</staticField>
<staticField>os_version:${os.version}</staticField>
</layout>
</appender>
<root level="debug">
<appender-ref ref="GELF" />
<appender-ref ref="CONSOLE" />
</root>
</configuration>
For more information: logback-gelf