Log4j2 pattern layout not appending the correlation Id - spring

I'm using HandlerInterceptorAdapter to generate a correlation id for each service request. Here is the configuration I'm using. I'm trying to append the correlation id on my console log as shown in the expected output. But somehow it was not generating the log pattern as shown below.
log4j2.xml
<?xml version="1.0" encoding="UTF-8" ?>
<Configuration packages="com.microsoft.applicationinsights.log4j.v2">
<Properties>
<Property name="LOG_PATTERN">
%d{yyyy-MM-dd HH:mm:ss.SSS} %5p ${hostName} --- [%15.15t] %-40.40c{1.} : %m%n%ex
</Property>
</Properties>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %X{correlationId} %-5level %logger{36} - %msg%n"/>
</Console>
<ApplicationInsightsAppender name="aiAppender"/>
</Appenders>
<Loggers>
<Root level="trace">
<AppenderRef ref="Console" level = "info" />
<AppenderRef ref="aiAppender" level= "info" />
</Root>
</Loggers>
</Configuration>
CorrelationInterceptor.java
public class CorrelationInterceptor extends HandlerInterceptorAdapter{
private static final String CORRELATION_ID_LOG_VAR_NAME = "correlationId";
private static final String CORRELATION_ID_HEADER_NAME = "Correlation-Id";
#Override
public boolean preHandle(final HttpServletRequest request, final HttpServletResponse response,
final Object handler) throws Exception {
final String correlationId = getCorrelationIdFromHeader(request);
System.out.println("correlationId:{}"+correlationId);
MDC.put(CORRELATION_ID_LOG_VAR_NAME, correlationId);
System.out.println("correlationId:{}"+MDC.get(CORRELATION_ID_LOG_VAR_NAME));
return true;
}
#Override
public void afterCompletion(final HttpServletRequest request, final HttpServletResponse response,
final Object handler, final Exception ex) throws Exception {
System.out.println("correlationId:{}"+MDC.get(CORRELATION_ID_LOG_VAR_NAME));
MDC.remove(CORRELATION_ID_LOG_VAR_NAME);
System.out.println("correlationId:{}"+MDC.get(CORRELATION_ID_LOG_VAR_NAME));
}
private String getCorrelationIdFromHeader(final HttpServletRequest request) {
String correlationId = request.getHeader(CORRELATION_ID_HEADER_NAME);
if (IpUtil.isNullOrEmpty(correlationId)) {
correlationId = generateUniqueCorrelationId();
}
return correlationId;
}
private String generateUniqueCorrelationId() {
return UUID.randomUUID().toString();
}
}
Output:
2020-12-29 17:59:42.026 INFO 31140 --- [nio-8080-exec-1] filter.RequestHeaderFilter : No correlationId found In Header. Generated :29e624b7-3ab6-4a1c-8c6a-4da342de4858
2020-12-29 17:59:42.027 INFO 31140 --- [nio-8080-exec-1] filter.RequestHeaderFilter : Found coorelationId in Header : 29e624b7-3ab6-4a1c-8c6a-4da342de4858
Expected Output:
2020-12-30 13:59:01.352 [http-nio-8080-exec-3] ea7aa31a-1e26-42e0-b7a6-ef0562ba5c4f INFO helper.GraphServiceHelper - CATEGORIES : [13], SUB_CATEGORIES : [0]
2020-12-30 13:59:01.353 [http-nio-8080-exec-3] ea7aa31a-1e26-42e0-b7a6-ef0562ba5c4f INFO helper.GraphServiceHelper - locationType :club

Related

Log4j2 plugin not loaded in Spring 4

While Spring 4 Rest application loading , i am getting the logger information ( log file name / path / archival days) from database and passing this value to Logj plugin , so that i can retrieve the value from log4j.xml.
My application log file not created and plugin not called ! not getting any error in console as well.
What should i do , for Logj plugin load and log file creation ?
Spring:4.3.15.RELEASE
log4j:2.4.1
java:1.8
SpringWebInitializer.java
public class SpringWebInitializer implements WebApplicationInitializer{
#Override
public void onStartup(ServletContext ctx) throws ServletException {
AnnotationConfigWebApplicationContext webCtx = new AnnotationConfigWebApplicationContext();
webCtx.register(WebConfiguration.class);
webCtx.setServletContext(ctx);
ServletRegistration.Dynamic servlet = ctx.addServlet("dispatcher", new DispatcherServlet(webCtx));
servlet.setLoadOnStartup(1);
servlet.addMapping("/");
//Add Listener.
ctx.addListener(new MyAppContextListner());
}
}
WebConfiguration.java
#EnableWebMvc
#Configuration
#ComponentScan(basePackages ="org.vasa.ws.myapp")
public class WebConfiguration {
}
MyAppContextListner.java
public class MyAppContextListner implements ServletContextListener{
private static final Logger LOGGER = LogManager.getLogger(BenefitsCompositeContextListner.class);
private WebApplicationContext webApplicationContext = null;
#Override
public void contextInitialized(ServletContextEvent event) {
// Database connectivity and get the logger information from db.
}
#Override
public void contextDestroyed(ServletContextEvent event) {
// TODO Auto-generated method stub
}
}
Given below Log4j Plugin i am using
MyLog4JConfigDatabaseLookup.java
#Plugin(name = "MyAppdbLookup", category = StrLookup.CATEGORY)
public class MyLog4JConfigDatabaseLookup extends AbstractLookup {
public String lookup(final LogEvent event, final String key) {
System.out.println("Lookup......");
}
}
log4j2.xml
<Configuration packages="org.vasa.ws.myapp">
<Properties>
<Property name="app-name">MyAppdbLookup</Property>
<Property name="file-level">${MyAppdbLookup:logLevel}</Property>
<Property name="log-file">${MyAppdbLookup:logFile}</Property>
<Property name="log-file-level">${MyAppdbLookup:logLevel}</Property>
<Property name="log-path">${MyAppdbLookup:logPath}</Property>
<Property name="archive-days">${MyAppdbLookup:archive-days}</Property>
</Properties>
<Appenders>
<Routing name="route-log">
<Routes pattern="framework">
<Route key="benefitCompositeWS">
<RollingFile name="message-log" fileName="${log-path}/myapp.log"
filePattern="${log-path}/$${date:yyyy-MM}/myapp.%d{MM-dd-yyyy}-%i.log.gz" append="true">
<PatternLayout
pattern="%d{MM/dd/yyyy HH:mm:ss.SSS z} [%t] %-5level %logger{36} - %msg%n" />
<Policies>
<TimeBasedTriggeringPolicy />
<SizeBasedTriggeringPolicy size="150 MB" />
</Policies>
<DefaultRolloverStrategy max="1000">
<Delete basePath="${log-path}" maxDepth="2">
<IfFileName glob="*/myapp*.log.gz" />
<IfLastModified age="${archive-days}" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile >
</Route>
</Routes>
</Routing>
<Loggers>
<Root level="${file-level}" additivity="false">
<AppenderRef ref="route-log" />
<AppenderRef ref="STDOUT" />
</Root>
</Loggers>
</Configuration>
I have changed my plugin name as lower case and now it's working fine as expected:
#Plugin(name="myappdblookup")

How to load application.properties into Log4j2 LogEventPatternConverter class?

I am working on a task that I want to mask sensitive data using Log4j2 LogEventPatternConverter Class.
#Plugin(name="SensitiveDataLog", category = "Converter")
#ConverterKeys({"sense"})
public class SensitiveDataLog extends LogEventPatternConverter {
#Value("${ssn}")
private String ssn;
public SensitiveDataLog(String name, String style) {
super(name, style);
}
public static SensitiveDataLog newInstance(String[] options) {
return new SensitiveDataLog("sense","sense");
}
#Override
public void format(LogEvent logEvent, StringBuilder outputMsg) {
String message = logEvent.getMessage().getFormattedMessage();
Matcher matcher = SSN_PATTERN.matcher(message);
if (matcher.find()) {
String maskedMessage = matcher.replaceAll("***-**-****");
outputMsg.append(maskedMessage);
} else {
outputMsg.append(message);
}
}
}
Suppose I want to keep pattern in application.properties, But problem here is we cannot load property value ssn. Always its null.
Here is my log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="info" monitorInterval="30"
packages="com.virtusa.xlab.fw.logging.component"
xmlns="http://logging.apache.org/log4j/2.0/config">
<Properties>
<Property name="basePath">logs/log4j2</Property>
</Properties>
<Appenders>
<!-- File Appender -->
<RollingFile name="FILE"
fileName="${basePath}/logfile.log" filePattern="${basePath}/logfile.%d{yyyy-MM-dd}-%i.log" append="true">
<PatternLayout
pattern="%-5p | %d{yyyy-MM-dd HH:mm:ss} | [%t] %C{2} (%F:%L) - %sense%n" />
<Policies>
<SizeBasedTriggeringPolicy size="1 KB" />
</Policies>
<DefaultRolloverStrategy max="4" />
</RollingFile>
<!-- Console Appender -->
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout
pattern="%-5p | %d{yyyy-MM-dd HH:mm:ss} | [%t] %C{2} (%F:%L) - %sense%n" />
</Console>
</Appenders>
<Loggers>
<Logger name="com.virtusa.xlab.fw" level="info" />
<Root level="info">
<AppenderRef ref="STDOUT" />
<AppenderRef ref="FILE" />
</Root>
</Loggers>
</Configuration>
Can anyone help me out here?
Thanks.
The problem is that SensitiveDataLog is created via static method newInstance(). Obviously, field ssn is not initialized at that moment. What you can do is to init the field later, e.g. when refreshing Spring context.
Here is my snippet:
private static XmlMaskPatternConverter INSTANCE = new XmlMaskPatternConverter();
public XmlMaskPatternConverter() {
super(NAME, NAME);
}
public static XmlMaskPatternConverter newInstance() {
return INSTANCE;
}
Now you can call static method getInstance() somewhere in your Spring Configuration (I do it in #Bean method) and set the ssn value there. Ofc, you need to create a setter for this field.
P.S. Hope it helps. I faced this problem too, so decided to leave my solution here. My first post on SO btw)

log4j2 how to filter out messages having not ThreadContext key

In my application, sometimes I have a threadContext key 'jobId' when logging, sometimes not.
So I want to pass log messages to my jdbc log appender (which requires the field 'jobId') only if the key 'jobId' is present in the threadContext.
I can not manage it to configure the DynamicThresholdFilter so that log message are skipped
<JDBC name="databaseAppender" tableName="LOG_MESSAGE">
<DynamicThresholdFilter key="jobId" defaultThreshold="INFO" onMatch="ACCEPT" onMismatch="DENY">
</DynamicThresholdFilter>
<DataSource jndiName="java:jboss/datasources/myDS" />
<Column name="MESSAGE" pattern="%message" />
<Column name="Job_KEY_" pattern="%X{jobId}" />
</JDBC>
You can use RoutingAppender to do this, similar to an example on the log4j2 FAQ page.
Here's a Java class that does some logging:
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;
public class SomeClass {
private static final Logger log = LogManager.getLogger();
public static void main(String[] args){
//These should appear in the console only
log.debug("No jobId, This is some debug!");
log.info("No jobId, Here's some info!");
log.error("No jobId, Some error happened!");
//These should route to a simulated JDBC appender only
ThreadContext.put("jobId", "1234");
log.debug("With jobId, This is some debug!");
log.info("With jobId, Here's some info!");
log.error("With jobId, Some error happened!");
//Once again these will go to console only
ThreadContext.remove("jobId");
log.debug("No jobId, This is some debug!");
log.info("No jobId, Here's some info!");
log.error("No jobId, Some error happened!");
}
}
Here's a sample log4j2.xml configuration file:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" />
</Console>
<File name="simulatedJdbc" fileName="logs/simulatedJdbc.txt"
immediateFlush="false" append="true">
<PatternLayout
pattern="%d{yyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} %X{jobId} - %msg%n" />
</File>
<Routing name="Routing">
<Routes pattern="$${ctx:jobId}">
<!-- This route is chosen if ThreadContext has no value for key jobId. -->
<Route key="$${ctx:jobId}" ref="Console" />
<!-- This is the default route when no others match -->
<Route ref = "simulatedJdbc"/>
</Routes>
</Routing>
</Appenders>
<Loggers>
<Root level="debug">
<AppenderRef ref="Routing" />
</Root>
</Loggers>
</Configuration>
The result of running the above is the following output:
Console output:
17:49:41.659 [main] DEBUG example.SomeClass - No jobId, This is some debug!
17:49:41.663 [main] INFO example.SomeClass - No jobId, Here's some info!
17:49:41.664 [main] ERROR example.SomeClass - No jobId, Some error happened!
17:49:41.664 [main] DEBUG example.SomeClass - No jobId, This is some debug!
17:49:41.664 [main] INFO example.SomeClass - No jobId, Here's some info!
17:49:41.664 [main] ERROR example.SomeClass - No jobId, Some error happened!
simulatedJdbc.txt output:
2019-02-08 17:49:41.664 [main] DEBUG example.SomeClass 1234 - With jobId, This is some debug!
2019-02-08 17:49:41.664 [main] INFO example.SomeClass 1234 - With jobId, Here's some info!
2019-02-08 17:49:41.664 [main] ERROR example.SomeClass 1234 - With jobId, Some error happened!
As you can see, when the ThreadContext contains jobId the logs are routed to the simulated JDBC appender (just a file appender in my example). When the jobId is missing logs are routed to the console appender instead.
Hope this helps!

Print intLevel in log4j2

You can see in https://logging.apache.org/log4j/2.x/manual/customloglevels.html the numerical values corresponding to built-in log4j2 logging levels, for example INFO->400. How can you refer to it in patternlayout resp. in JDBC Logger configuration?
I have an old log4j 1.x config for JDBC where it was referred as %iprio.
A workaround is to use
level{OFF=0,FATAL=100,ERROR=200,WARN=300,INFO=400,DEBUG=500,TRACE=600,ALL=1000}
but I am not very happy about that.
It sounds like you want to log the integer value of the log level instead of the name and you don't want to use the labeling feature of the PatternLayout's level parameter.
One possible solution would be to create a custom Lookup, here is some sample code:
package example;
import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.lookup.StrLookup;
#Plugin(name = "level", category = "Lookup")
public class LevelLookup implements StrLookup{
/**
* Lookup the value for the key.
* #param key the key to be looked up, may be null
* #return The value for the key.
*/
public String lookup(String key) {
return null;
}
/**
* Lookup the value for the key using the data in the LogEvent.
* #param event The current LogEvent.
* #param key the key to be looked up, may be null
* #return The value associated with the key.
*/
public String lookup(LogEvent event, String key) {
return String.valueOf(event.getLevel().intLevel());
}
}
Next, here is a sample configuration using the new lookup - note the ${level:}:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] ${level:} %logger{36} - %msg%n" />
</Console>
</Appenders>
<Loggers>
<Root level="debug">
<AppenderRef ref="Console" />
</Root>
</Loggers>
</Configuration>
Here is some sample code that performs some logging:
package example;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
public class SomeClass {
private static final Logger log = LogManager.getLogger();
public static void main(String[] args){
log.debug("This is some debug!");
log.info("Here's some info!");
log.error("Some erorr happened!");
}
}
and finally here is the output:
22:49:29.438 [main] 500 example.SomeClass - This is some debug!
22:49:29.440 [main] 400 example.SomeClass - Here's some info!
22:49:29.440 [main] 200 example.SomeClass - Some erorr happened!
Hope this helps!

Spring AOP Controller Executing twice

My applicationContext is as follows
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="..">
<mvc:annotation-driven/>
<task:annotation-driven/>
<mvc:resources mapping="/resources/**" location="/resources/"/>
<aop:aspectj-autoproxy />
<context:component-scan base-package="com.abc">
<context:include-filter type="aspectj" expression="com.abc.aspects.LogControllerAspect"/>
</context:component-scan>
<context:annotation-config />
</beans>
Have 2 Aspect Java classes, LogControllerAspect (for logging all the calls to Spring Controllers) and LogDAOAspect (for logging all the calls to DB).
#Aspect
#Service
public class LogDAOAspect {
private Logger logger = LoggerFactory.getLogger(this.getClass());
#Around("execution(* com.*.*DAOImpl.*(..))")
public Object logAround(ProceedingJoinPoint joinPoint) throws Throwable {
String methodId = joinPoint.getTarget().getClass().getSimpleName()+" : "+joinPoint.getSignature().getName() + " : " + ((joinPoint.getArgs()==null||joinPoint.getArgs().length<1)?"":(Arrays.toString(joinPoint.getArgs())));
Object returnVal = null;
StopWatch sw = new StopWatch(methodId);
try {
sw.start();
returnVal= joinPoint.proceed(joinPoint.getArgs());
sw.stop();
} catch (Throwable e) {
logger.error(methodId+"\n"+e);
throw e;
}
logger.debug(methodId + ":" +sw.getTotalTimeMillis());
return returnVal;
}
}
#Aspect
public class LogControllerAspect {
private Logger logger = LoggerFactory.getLogger(this.getClass());
#Around("execution(* com.*.*Controller.*(..))")
public Object logAround(ProceedingJoinPoint joinPoint) throws Throwable {
String methodId = joinPoint.getTarget().getClass().getSimpleName()+" : "+joinPoint.getSignature().getName() + " : " + ((joinPoint.getArgs()==null||joinPoint.getArgs().length<1)?"":(Arrays.toString(joinPoint.getArgs())));
Object returnVal = null;
StopWatch sw = new StopWatch(methodId);
try {
sw.start();
returnVal= joinPoint.proceed(joinPoint.getArgs());
sw.stop();
} catch (Throwable e) {
logger.error(methodId+"\n"+e);
throw e;
}
logger.debug(methodId + ":" +sw.getTotalTimeMillis());
return returnVal;
}
}
LogDAOAspect is fine, but LogControllerAspect is logging twice (logAround method is executing twice) when I request some page. I can understand that that aspect is getting proxied twice, but am not sure how to avoid this. Help appreciated.
It was a silly mistake. It was not even a spring related issue!
I had setup log4j as follows !!!???!!!
<appender name="AppAppender" class="org.apache.log4j.DailyRollingFileAppender">
<param name="DatePattern" value=".yyyy-MM-dd.HH"/>
<param name="File" value="logs/logfile.log"/>
<param name="Append" value="true"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-5p : %d{ISO8601} : %m%n"/>
</layout>
</appender>
<logger name="com.abc.aspects.LogControllerAspect">
<priority value="debug"></priority>
<appender-ref ref="AppAppender"/>
</logger>
<root>
<priority value="error" />
<appender-ref ref="AppAppender" />
</root>
Thank you #erencan. You question really helped me look closely what is really happening inside that method.
Changed is at follows, and it work fine. Should log4j to this question's tags!
<logger name="com.abc.aspects.LogControllerAspect" additivity="false">
<priority value="debug"></priority>
<appender-ref ref="AppAppender"/>
</logger>

Resources