Spring-boot log rotations issue with logback - spring

I am working on a spring-boot project and I am using logback and slf4j for logging. Application is deployed on external tomcat and I am using logback 1.1.6 .
I have created an appender with marker , it working fine but rotation policy is not working.
here is the entry from logback.xml
<appender name="FAILED-QUERY"
class="com.xyz.config.HourlyFileAppender">
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator class="ch.qos.logback.classic.boolex.OnMarkerEvaluator">
<marker>FAILEDQUERY</marker>
</evaluator>
<onMismatch>DENY</onMismatch>
<onMatch>NEUTRAL</onMatch>
</filter>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%d{yyyy-MM-dd HH:mm:ss} - %msg%n
</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${DEV_HOME}/xyz_otp_logs_failed_query.%d{yyyy-MM-dd-HH-mm}.log
</fileNamePattern>
<!-- <timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
</timeBasedFileNamingAndTriggeringPolicy> -->
</rollingPolicy>
</appender>
<logger name="com.xyz" level="info"
additivity="false">
<appender-ref ref="FAILED-QUERY" />
<!-- <appender-ref ref="STDOUT" /> -->
</logger>
The code for rotating policy is below
public class HourlyFileAppender<E> extends RollingFileAppender<E> {
private static long start = System.currentTimeMillis(); // minutes
private int rollOverTimeInSecond = 60*60;
#Override
public void rollover()
{
long currentTime = System.currentTimeMillis();
int maxIntervalSinceLastLoggingInMillis = rollOverTimeInSecond * 1000;
if ((currentTime - start) >= maxIntervalSinceLastLoggingInMillis)
{
super.rollover();
start = System.currentTimeMillis();
}
}
}
though the same settings for other appender is working fine.

Related

spring logback how to dinamically retrieve class and application name

I have the following logback-spring.xml configuration file:
<configuration scan="true">
<property name="LOG_PATH" value="/logs/application/"/>
<timestamp key="bySecond" datePattern="yyyyMMdd'T'HHmmss"/>
<appender name="ALL" class="ch.qos.logback.core.rolling.RollingFileAppender">
<append>true</append>
<encoder>
<pattern>[%d{yyyy-MM-dd_HH:mm:ss.SSS}] %-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily and when the file reaches 15 MegaBytes -->
<fileNamePattern>${catalina_home}/${LOG_PATH}/general-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>15MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<maxHistory>30</maxHistory>
<cleanHistoryOnStart>true</cleanHistoryOnStart>
</rollingPolicy>
</appender>
<root level="INFO">
<appender-ref ref="ALL" />
</root>
</configuration>
Now, I have my springboot application "Application" and within the class "MyClass" I call:
LoggerFactory.getLogger(MyClass.class);
I want to configure the log file name by changing the static name "general" in "Application-MyClass" (whatever is the application name). Any tips?

How to show sql Queries in console in springboot azure (Springboot + Azure Cosmos)

i am using SpringBoot with azurecosmos db
i want to see the query for below repository code.
findByFirstNameAndLastName(String firstName,String lastName);
For me adding following configuration to application.yaml worked.
logging:
config: classpath:logback-local-spring.xml
level:
com.azure.cosmos.implementation.SqlQuerySpecLogger: debug
You can try adding the below lines inside application.properties to get all the query generated in console
spring.jpa.show-sql=true
spring.jpa.generate-ddl=true
logging.level.org.hibernate.SQL = DEBUG
logging.level.org.hibernate.type = TRACE
logging.level.org.springframework.web = DEBUG
You can use also logback. Here is one example:
<configuration>
<include resource="/org/springframework/boot/logging/logback/base.xml"/>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n
</pattern>
</encoder>
</appender>
<root level="info">
<appender-ref ref="STDOUT"/>
</root>
<logger name="com.azure.cosmos" level="error"/>
<logger name="org.springframework" level="error"/>
<logger name="io.netty" level="error"/>
<logger name="com.azure.cosmos.implementation.SqlQuerySpecLogger" level="DEBUG"/>

AWS CloudWatch logging with Spring Boot

What should be the approach to store logs of multiple spring boot application(s) in cloud watch?
Sample spring-boot - logback-spring.xml configuration file is below.
<?xml version="1.0" encoding="UTF-8"?>
<property name="LOGS" value="/logs/abc/" />
<appender name="Console" class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern> %black(%d{ISO8601}) %highlight(%-5level) [%blue(%t)] %yellow(%C{1.}): %msg%n%throwable </Pattern>
</layout>
</appender>
<appender name="RollingFile" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOGS}/abc-log.log</file>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>%d %p %C{1.} [%t] %m%n</Pattern> </encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily and when the file reaches 10 MegaBytes -->
<fileNamePattern>${LOGS}/archived/abc-log-%d{yyyy-MM-dd}.%i.log
</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>5MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<maxHistory>2</maxHistory>
<totalSizeCap>10MB</totalSizeCap>
</rollingPolicy>
</appender>
<!-- LOG everything at INFO level -->
<root level="info">
<appender-ref ref="RollingFile" />
<appender-ref ref="Console" />
</root>
<logger name="com.abc" level="trace" additivity="false">
<appender-ref ref="RollingFile" />
<appender-ref ref="Console" />
</logger>
I was able to achieve the same using Cloudwatch log agent.
Step 1 - Create awslogs.conf file to point to the log location.
[/logs/abcd/8080-abcd.log]
datetime_format = %Y-%m-%d %H:%M:%S
file = /logs/abcd/8080-abcd.log
buffer_duration = 5000
log_stream_name = {hostname}
initial_position = start_of_file
log_group_name = ABCD Group Name
[/logs/defg/8081-defg.log]
datetime_format = %Y-%m-%d %H:%M:%S
file = /logs/abcd/8081-defg.log
buffer_duration = 5000
log_stream_name = {hostname}
initial_position = start_of_file
log_group_name = DEFG Group Name
Step 2 - Install cloud watch log agent as part of UserData/Bootstrap script for ec2.
yum install wget -y
wget https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py
python ./awslogs-agent-setup.py --region $aws_region --non-interactive --configfile=/configlocation/awslogs.conf
Step 3 - Need a role for ec2 machine to create logs in CloudWatch.

How to log all requests and responses for Play Framework 2.5.x?

I want to log requests and responses for my application. I expect they should be present in access.log file. However, my access.log file is empty. I have enables access logging in configuration.
Here is my prod-logger.xml :
<configuration>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${user.dir}/web/logs/application.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- Daily rollover with compression -->
<fileNamePattern>application-log-%d{yyyy-MM-dd}.gz</fileNamePattern>
<!-- keep 30 days worth of history -->
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%date{yyyy-MM-dd HH:mm:ss ZZZZ} [%level] from %logger in %thread - %message%n%xException</pattern>
</encoder>
</appender>
<appender name="ACCESS_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${user.dir}/web/logs/access.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover with compression -->
<fileNamePattern>access-log-%d{yyyy-MM-dd}.gz</fileNamePattern>
<!-- keep 1 week worth of history -->
<maxHistory>7</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%date{yyyy-MM-dd HH:mm:ss ZZZZ} %message%n</pattern>
<!-- this quadruples logging throughput -->
<immediateFlush>false</immediateFlush>
</encoder>
</appender>
<!-- additivity=false ensures access log data only goes to the access log -->
<logger name="access" level="INFO" additivity="false">
<appender-ref ref="ACCESS_FILE"/>
</logger>
<root level="INFO">
<appender-ref ref="FILE"/>
</root>

How to use request param as log identifier in logback

In my controller I get an id as request param. Is there a way to use this id to identify all the log entries of this particular request?
#ResponseBody
#RequestMapping(value="/anyRequest", method = RequestMethod.GET)
public String doAnything(#RequestParam(value="Id", required = true) long id) {
logger.info(id);
return "";
}
Means anywhere of this entry should be the id value:
2017-02-28 08:30:41.035 INFO 23050 --- [http-bio-1084-exec-20] AnyServiceImpl ...
Im using logback and have this configuration:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<springProfile name="dev">
<property name="FILE_PATH" value="C:\\DATA\\temp" />
</springProfile>
<appender name="FILE-AUDIT"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${FILE_PATH}/service.log</file>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%d{yyyy-MM-dd HH:mm:ss.SSS} ${LOG_LEVEL_PATTERN:-%5p} ${PID:- } --- [%t] %-40.40logger{39} : %m%n
</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>service.%d{yyyy-MM-dd}.%i.log
</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>25MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<maxHistory>90</maxHistory>
</rollingPolicy>
</appender>
<appender name="consoleAppender" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<charset>UTF-8</charset>
<Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} ${LOG_LEVEL_PATTERN:-%5p} ${PID:- } --- [%t] %-40.40logger{39} : %m%n</Pattern>
</encoder>
</appender>
<logger name="Service" level="info"
additivity="false">
<appender-ref ref="FILE-AUDIT" />
<appender-ref ref="consoleAppender" />
</logger>
<root level="info">
<appender-ref ref="FILE-AUDIT" />
<appender-ref ref="consoleAppender" />
</root>
</configuration>
And I use the logger in this way:
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
private final Logger logger = LoggerFactory.getLogger(this.getClass());
Is there any Spring out of the box magic?
Use MDC to achieve this.
Controller Method:
#ResponseBody
#RequestMapping(value="/anyRequest", method = RequestMethod.GET)
public String doAnything(#RequestParam(value="Id", required = true) long id) {
try {
MDC.put("id", id);
logger.info(id);
return "";
} finally {
org.slf4j.MDC.clear();
}
}
Logback.xml:
<Pattern>[%X{id}] %d{yyyy-MM-dd HH:mm:ss.SSS} ${LOG_LEVEL_PATTERN:-%5p} ${PID:- } --- [%t] %-40.40logger{39} : %m%n</Pattern>

Resources