Problem with Log4j2 Configurator initialize in spring boot - spring-boot

I'm working on a spring boot project, which use log4j2 for logging. The log4j2.xml is stored in DB, so I have to retrieve it before using.
It looks like below:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<!-- setting -->
<Properties>
<Property name="merchantdb">/syslog/translog/merchantdb/</Property>
<Property name="pattern1">%d{yyyy-MM-dd HH:mm:ss}[%5p][%C{2}][%t] %m%n</Property>
<Property name="pattern2">%d{yyyy-MM-dd HH:mm:ss}[%5p][%C{2}.%M()] %m%n</Property>
</Properties>
<!-- appender setting -->
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="${pattern1}"/>
</Console>
<RollingFile name="merchantdb" filePattern="${merchantdb}/merchantdb.log.%d{yyyy-MM-dd}">
<PatternLayout pattern="${pattern1}"/>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
</Policies>
<DirectWriteRolloverStrategy/>
</RollingFile>
<RollingFile name="merchantdb_io" filePattern="${merchantdb}/merchantdb_io.log.%d{yyyy-MM-dd}">
<PatternLayout pattern="${pattern2}"/>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
</Policies>
<DirectWriteRolloverStrategy/>
</RollingFile>
<RollingFile name="merchantdb_i" filePattern="${merchantdb}/merchantdb_i.log.%d{yyyy-MM-dd}">
<PatternLayout pattern="${pattern2}"/>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
</Policies>
<DirectWriteRolloverStrategy/>
</RollingFile>
<RollingFile name="merchantdb_o" filePattern="${merchantdb}/merchantdb_o.log.%d{yyyy-MM-dd}">
<PatternLayout pattern="${pattern2}"/>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
</Policies>
<DirectWriteRolloverStrategy/>
</RollingFile>
<RollingFile name="rmds" filePattern="${merchantdb}/rmds.log.%d{yyyy-MM-dd}">
<PatternLayout pattern="${pattern2}"/>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
</Policies>
<DirectWriteRolloverStrategy/>
</RollingFile>
<Socket name="socket" host="10.77.249.230" port="20002" reconnectionDelay="5000" immediateFlush="true">
<JsonLayout compact="true" includeStacktrace="true" charset="UTF-8" eventEol="true" locationInfo="true" objectMessageAsJsonObject="true" stacktraceAsString="true">
<KeyValuePair key="hostName" value="$${hostName}"/>
</JsonLayout>
</Socket>
</Appenders>
<Loggers>
<Logger name="merchantdb" level="all" additivity="false">
<AppenderRef ref="merchantdb"/>
<AppenderRef ref="Console"/>
<AppenderRef ref="socket"/>
</Logger>
<Logger name="merchantdb_io" level="all" additivity="false">
<AppenderRef ref="merchantdb_io"/>
<AppenderRef ref="Console"/>
<AppenderRef ref="socket"/>
</Logger>
<Logger name="merchantdb_i" level="all" additivity="false">
<AppenderRef ref="merchantdb_i"/>
<AppenderRef ref="Console"/>
<AppenderRef ref="socket"/>
</Logger>
<Logger name="merchantdb_o" level="all" additivity="false">
<AppenderRef ref="merchantdb_o"/>
<AppenderRef ref="Console"/>
<AppenderRef ref="socket"/>
</Logger>
<Logger name="com.alibaba.druid.pool" level="all" additivity="false">
<AppenderRef ref="rmds"/>
</Logger>
</Loggers>
</Configuration>
I know there are multiple ways to initialize *.xml. It did work when using URI to initialize. However, I found myself having problem with initialize with ConfigurationSource. It seems the config doesn't set properly when using ConfigurationSource, logger still print in default format.
Below is my code, is there anything that I miss?
Thanks a lot.
#Configuration
public class ConfigLoader {
public static Logger logger;
#Bean
#DependsOn("ds_settingsDB")
public Logger log4j2() throws IOException {
System.out.println("loading Log4j2");
FileWriter fw = null;
LoggerContext context = null;
try {
String fieldvalue = SettingsDB.get("Log4J2.xml");
//initBySource(fieldvalue); //this doesn't works
initByFilePath(fieldvalue); //this works
logger = LogManager.getFormatterLogger("merchantdb");
logger.info("log4j2 load complete");
return logger;
}catch(Exception e) {
throw new EipException(e);
} finally {
if(fw!=null){
fw.close();
}
}
}
private LoggerContext initBySource(String fieldvalue) throws Exception {
InputStream in = new ByteArrayInputStream(fieldvalue.getBytes("utf-8"));
ConfigurationSource source = new ConfigurationSource(in);
LoggerContext ctx = Configurator.initialize(null, source);
return ctx;
}
private LoggerContext initByFilePath(String fieldvalue) throws Exception {
FileWriter fw = null;
LoggerContext ctx = null;
String filePath = "C:\\apconf\\log4j2zzz.xml";
fw = new FileWriter(filePath);
fw.write(fieldvalue);
fw.flush();
File file = new File(filePath);
if(file.exists()){
ctx = Configurator.initialize(null, filePath);
}
return ctx;
}
}

Related

Application not writing logs on when log file rotates

My app is using spring log4j2 and uses slf4j api to write log to separate log file "application.log". This app gets deployed to tomcat v8 along with other apps. All app share common log4j2 configuration and writes to common logfile 'application.log'. We have a log rotation policy of 250 mb and when the log file rotates the logs are not written to the logfile surprising only one app among all the app is able to write to the log file. I'm able to reproduce this locally too. Can you please help in fixing this issue.
The other
Please find the log4j2.xml config below.
JAR VERSIONS
slf4j-api 1.7.21
log4j-slf4j-impl 2.5
log4j-api 2.5
log4j-core 2.5
log4j-web 2.5
Log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration name="api-config" status="trace" monitorInterval="30">
<Properties>
<Property name="logdir">/Users/kramesan/microservices-config/logs</Property>
</Properties>
<Appenders>
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="%X{_requestId} %X{authToken} %X{urlEmployeeId} %X{urlCompanyId} [%X{authEmplIds}] [%X{authCompanyIds}] %d{yyy
y-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n"/>
</Console>
<RollingFile name="ApplicationLogRollingFile" fileName="${logdir}/application.log"
filePattern="${logdir}/$${date:yyyy-MM}/app-%d{MM-dd-yyyy}-%i.log.gz">
<JSONLayout locationInfo="true" complete="true" compact="true" eventEol="true" properties="true" />
<Policies>
<TimeBasedTriggeringPolicy />
<SizeBasedTriggeringPolicy size="250 MB" />
</Policies>
<DefaultRolloverStrategy max="20" />
</RollingFile>
<RollingFile name="AuditLogRollingFile" fileName="${logdir}/audit/api-audit.log"
filePattern="${logdir}/audit/$${date:yyyy-MM}/api-audit-%d{MM-dd-yyyy}-%i.log.gz">
<PatternLayout>
<Pattern>%X{_requestId} %X{authToken} %X{urlEmployeeId} %X{urlCompanyId} [%X{authEmplIds}] [%X{authCompanyIds}] %m%n</Pattern>
</PatternLayout>
<!-- JSONLayout locationInfo="true" complete="true" compact="true" eventEol="true" properties="true" -->
<Policies>
<TimeBasedTriggeringPolicy />
<SizeBasedTriggeringPolicy size="250 MB" />
</Policies>
<DefaultRolloverStrategy max="20" />
</RollingFile>
<Async name="ApplicationLogAsync" bufferSize="262144">
<AppenderRef ref="ApplicationLogRollingFile"/>
</Async>
<Async name="AuditLogAsync" bufferSize="262144">
<AppenderRef ref="AuditLogRollingFile"/>
</Async>
</Appenders>
<Loggers>
<!-- All the 3rd Party frameworks -->
<Logger name="org.springframework" level="warn" />
<Logger name="org.hibernate" level="warn" />
<!-- common package name for all the business application level code -->
<Logger name="com.trinet" level="info" />
<!-- Audit Loggger This is used for spring aspect to log before and after execution -->
<Logger name="AuditLogger" level="info">
<AppenderRef ref="AuditLogAsync" />
</Logger>
<Root level="info">
<AppenderRef ref="ApplicationLogAsync" />
</Root>
</Loggers>
</Configuration>
Please, edit your Logger name. Logger name property needs package path.
AS-IS
...
<Logger name="AuditLogger" level="info">
...
TO-BE
...
<Logger name="com.foo.bar.AuditLogger" level="info" additivity="false">
...
or
...
<Logger name="com.foo.bar.*" level="info" additivity="false">
...
I think you'd better write property additivity Because your Logger works two times com.foo.bar.AuditLogger Logger and Root Logger. so you additivity to false then it works each.
reference link : Additivity

Sentry Log4j2 Kotlin Spring Boot Webflux - No issues published to Sentry

Having trouble getting sentry issues to publish.
Here's my setup.
I have exported my DSN as SENTRY_DSN.
src/main/kotlin/resources/log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration status="warn" packages="org.apache.logging.log4j.core,io.sentry.log4j2">
<appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" />
</Console>
<Sentry name="Sentry" />
</appenders>
<loggers>
<root level="INFO">
<appender-ref ref="Console" />
<!-- Note that the Sentry logging threshold is overridden to the WARN level -->
<appender-ref ref="Sentry" level="WARN" />
</root>
</loggers>
</configuration>
build.gradle.kts
...
dependencies {
compile("io.sentry:sentry-log4j2:1.7.22")
...
}
Controller.kt
#RestController
#RequestMapping("/users")
class AuthenticationController(private val exampleService: ExampleService) {
private val logger = LogManager.getLogger(AuthenticationService::class)
#DeleteMapping("/session")
fun logout(): Mono<Response> {
logger.error("this is a error")
return exampleService.returnMono()
}
}
I expect this logger.error call to send a message to sentry.
Sentry SDK for Java recently got support for Webflux:
https://github.com/getsentry/sentry-java/pull/1529
That was on version 5.1.0-beta3 and GA is about to land in a few days/weeks:
https://github.com/getsentry/sentry-java/blob/main/CHANGELOG.md#510-beta3

How to set log4j log level in spring MVC web project

Please find below for the log4j.xml and controller class. I have added lines in log4j.xml to change the log level from error to debug. But there is no effect seen after adding these lines.
I need this setup to debug "HTTP 400: The request sent by the client was syntactically incorrect."
Could anyone help me on this? Thanks.
log4j.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration PUBLIC "-//log4j/log4j Configuration//EN" "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<appender name="Appender1" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-7p %d [%t] %c %x - %m%n"/>
</layout>
</appender>
<appender name="Appender2" class="org.apache.log4j.FileAppender">
<param name="File" value="./Logs/SpringMVC2.log" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-7p %d [%t] %c %x - %m%n"/>
</layout>
</appender>
<!-- no effect of these lines -->
<!-- <logger name="org.springframework.web" additivity="false"> -->
<logger name="org.springframework" additivity="false">
<priority value ="DEBUG" />
<level value="DEBUG"/>
<appender-ref ref="CONSOLE"/>
</logger>
<root>
<priority value ="DEBUG" />
<level value="DEBUG"/>
<appender-ref ref="CONSOLE"/>
</root>
<!-- /no effect of these lines -->
</log4j:configuration>
Controller class
#EnableWebMvc
#Controller
#RequestMapping(value="/home")
public class HomeController {
private Logger logger= Logger.getLogger(HomeController.class);
#RequestMapping(method=RequestMethod.GET)
public String home() {
logger.info("sample info log");
logger.debug("sample debug log");
logger.trace("sample trace log");
logger.error("sample error log");
return "home";
}
}
You define appender with name <appender name="Appender1" class="org.apache.log4j.ConsoleAppender"> but you don't use it.
Instead of use <appender-ref ref="CONSOLE"/> use the one you define :
<logger name="org.springframework" additivity="false">
<priority value ="DEBUG" />
<level value="DEBUG"/>
<appender-ref ref="Appender1"/>
</logger>

How to use request param as log identifier in logback

In my controller I get an id as request param. Is there a way to use this id to identify all the log entries of this particular request?
#ResponseBody
#RequestMapping(value="/anyRequest", method = RequestMethod.GET)
public String doAnything(#RequestParam(value="Id", required = true) long id) {
logger.info(id);
return "";
}
Means anywhere of this entry should be the id value:
2017-02-28 08:30:41.035 INFO 23050 --- [http-bio-1084-exec-20] AnyServiceImpl ...
Im using logback and have this configuration:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<springProfile name="dev">
<property name="FILE_PATH" value="C:\\DATA\\temp" />
</springProfile>
<appender name="FILE-AUDIT"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${FILE_PATH}/service.log</file>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%d{yyyy-MM-dd HH:mm:ss.SSS} ${LOG_LEVEL_PATTERN:-%5p} ${PID:- } --- [%t] %-40.40logger{39} : %m%n
</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>service.%d{yyyy-MM-dd}.%i.log
</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>25MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<maxHistory>90</maxHistory>
</rollingPolicy>
</appender>
<appender name="consoleAppender" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<charset>UTF-8</charset>
<Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} ${LOG_LEVEL_PATTERN:-%5p} ${PID:- } --- [%t] %-40.40logger{39} : %m%n</Pattern>
</encoder>
</appender>
<logger name="Service" level="info"
additivity="false">
<appender-ref ref="FILE-AUDIT" />
<appender-ref ref="consoleAppender" />
</logger>
<root level="info">
<appender-ref ref="FILE-AUDIT" />
<appender-ref ref="consoleAppender" />
</root>
</configuration>
And I use the logger in this way:
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
private final Logger logger = LoggerFactory.getLogger(this.getClass());
Is there any Spring out of the box magic?
Use MDC to achieve this.
Controller Method:
#ResponseBody
#RequestMapping(value="/anyRequest", method = RequestMethod.GET)
public String doAnything(#RequestParam(value="Id", required = true) long id) {
try {
MDC.put("id", id);
logger.info(id);
return "";
} finally {
org.slf4j.MDC.clear();
}
}
Logback.xml:
<Pattern>[%X{id}] %d{yyyy-MM-dd HH:mm:ss.SSS} ${LOG_LEVEL_PATTERN:-%5p} ${PID:- } --- [%t] %-40.40logger{39} : %m%n</Pattern>

Redirecting feign & ribbon logs to log4j2

I currently use spring cloud netflix with log4j2. The log4j2 configuration comes from the xml in the classpath. When I run the app, I see that the feign & ribbon logs are not being redirected to the logger specified in the configuration. I have configured log for com.netflix.ribbon & feign packages to be logged at debug level.
However, log configured for spring is properly redirecting to the specified appender, ribbon & feign are not.
I am using gradle with spring-boot-starter-logging ignored & added spring-boot-starter-log4j2 in as part of my build.
I see that feign has a way by which we can configure slf4j, but since we use annotation driven feign support, I cant configure the feign to use slf4j for logging.
Any help is appreciated.
My log4j2.xml looks some what like
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Properties>
<Property name="log-path">logs</Property>
<Property name="log-fileName">test</Property>
</Properties>
<Appenders>
<Console name="console-log" target="SYSTEM_OUT">
<PatternLayout pattern="[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n" />
</Console>
<RollingFile name="trace-log" fileName="${log-path}/${log-fileName}-trace.log" filePattern="${log-path}/${log-fileName}_trace-%d{yyyy-MM-dd}.log">
<PatternLayout pattern="[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n" />
<Policies>
<TimeBasedTriggeringPolicy interval="1" modulate="true" />
</Policies>
</RollingFile>
<RollingFile name="error-log" fileName="${log-path}/${log-fileName}-error.log" filePattern="${log-path}/${log-fileName}_error-%d{yyyy-MM-dd}.log">
<PatternLayout pattern="[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n" />
<Policies>
<TimeBasedTriggeringPolicy interval="1" modulate="true" />
</Policies>
</RollingFile>
</Appenders>
<Loggers>
<logger name="org.springframework" level="trace" additivity="false">
<AppenderRef ref="trace-log" />
</logger>
<logger name="feign" level="trace" additivity="false">
<AppenderRef ref="trace-log" />
</logger>
<logger name="com.netflix.ribbon" level="trace" additivity="false">
<AppenderRef ref="trace-log" />
</logger>
<Root level="info">
<AppenderRef ref="console-log"></AppenderRef>
<AppenderRef ref="error-log" level="ERROR"/>
</Root>
</Loggers>
</Configuration>
PS: The reason for debugging feign/ribbon is to understand a weird feign behavior between two different machines in our micro services setup
Looking at Spring Cloud's FeignClientFactoryBean shows that you can optionally autowire a bean of type feign.Logger.Level. Try registering such a bean in your #Configurationusing
#Bean
public feign.Logger.Level feignLoggerLevel() {
return feign.Logger.Level.FULL;
}
#jensfischerhh's answer would fix many cases but looks mistakenly missed one thig.
You need to config feign generated class's logger level with feignLoggerLevel Bean.
Both config must be exist together.
related doccument in spring-cloud-netflix
bean config ( in #Configuration annotated class )
#Bean
public feign.Logger.Level feignLoggerLevel() {
return feign.Logger.Level.FULL;
}
log config
</Configuration>
<Loggers>
<logger name="your.feign-interface-package" level="trace">
</Loggers>
</Configuration>

Resources