You can see in https://logging.apache.org/log4j/2.x/manual/customloglevels.html the numerical values corresponding to built-in log4j2 logging levels, for example INFO->400. How can you refer to it in patternlayout resp. in JDBC Logger configuration?
I have an old log4j 1.x config for JDBC where it was referred as %iprio.
A workaround is to use
level{OFF=0,FATAL=100,ERROR=200,WARN=300,INFO=400,DEBUG=500,TRACE=600,ALL=1000}
but I am not very happy about that.
It sounds like you want to log the integer value of the log level instead of the name and you don't want to use the labeling feature of the PatternLayout's level parameter.
One possible solution would be to create a custom Lookup, here is some sample code:
package example;
import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.lookup.StrLookup;
#Plugin(name = "level", category = "Lookup")
public class LevelLookup implements StrLookup{
/**
* Lookup the value for the key.
* #param key the key to be looked up, may be null
* #return The value for the key.
*/
public String lookup(String key) {
return null;
}
/**
* Lookup the value for the key using the data in the LogEvent.
* #param event The current LogEvent.
* #param key the key to be looked up, may be null
* #return The value associated with the key.
*/
public String lookup(LogEvent event, String key) {
return String.valueOf(event.getLevel().intLevel());
}
}
Next, here is a sample configuration using the new lookup - note the ${level:}:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] ${level:} %logger{36} - %msg%n" />
</Console>
</Appenders>
<Loggers>
<Root level="debug">
<AppenderRef ref="Console" />
</Root>
</Loggers>
</Configuration>
Here is some sample code that performs some logging:
package example;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
public class SomeClass {
private static final Logger log = LogManager.getLogger();
public static void main(String[] args){
log.debug("This is some debug!");
log.info("Here's some info!");
log.error("Some erorr happened!");
}
}
and finally here is the output:
22:49:29.438 [main] 500 example.SomeClass - This is some debug!
22:49:29.440 [main] 400 example.SomeClass - Here's some info!
22:49:29.440 [main] 200 example.SomeClass - Some erorr happened!
Hope this helps!
Related
I would like to reduce the size of certain logs using logback (I attempted to use Pattern layout with with a pattern of %.100000msg to limit the max size to one hundred thousand but had no luck), the large(over 1 million characters caused by a few REST GET calls) logs are causing Elastic to slow down when searching for certain information.
What would be the best practice to overcome this?
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<appender name="Console"
class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>
%.30msg
</Pattern>
</layout>
</appender>
<logger name="" level="INFO"/>
I solved this by refering to this documentation to create a custom converter. The property was an extra, it was implemented to allow for configuration within the xml file.
<property scope="context" name="max-message-length" value="100000"/>
<conversionRule conversionWord="boundedMsg"
converterClass="za.co.ksdc.qadee.config.TruncationCustomLogConverter" />
The following java class was implemented to truncate the log message:
public class TruncationCustomLogConverter extends ClassicConverter {
#Override
public String convert(ILoggingEvent event) {
int maxMessageLength = Integer.parseInt(getContext().getProperty("max-
message-length"));
String formattedMessage = event.getFormattedMessage();
if (formattedMessage == null ||
formattedMessage.length() < maxMessageLength) {
return formattedMessage;
}
return new StringBuilder(maxMessageLength)
.append(formattedMessage, 0, maxMessageLength)
.toString();
}
}
I am working on a task that I want to mask sensitive data using Log4j2 LogEventPatternConverter Class.
#Plugin(name="SensitiveDataLog", category = "Converter")
#ConverterKeys({"sense"})
public class SensitiveDataLog extends LogEventPatternConverter {
#Value("${ssn}")
private String ssn;
public SensitiveDataLog(String name, String style) {
super(name, style);
}
public static SensitiveDataLog newInstance(String[] options) {
return new SensitiveDataLog("sense","sense");
}
#Override
public void format(LogEvent logEvent, StringBuilder outputMsg) {
String message = logEvent.getMessage().getFormattedMessage();
Matcher matcher = SSN_PATTERN.matcher(message);
if (matcher.find()) {
String maskedMessage = matcher.replaceAll("***-**-****");
outputMsg.append(maskedMessage);
} else {
outputMsg.append(message);
}
}
}
Suppose I want to keep pattern in application.properties, But problem here is we cannot load property value ssn. Always its null.
Here is my log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="info" monitorInterval="30"
packages="com.virtusa.xlab.fw.logging.component"
xmlns="http://logging.apache.org/log4j/2.0/config">
<Properties>
<Property name="basePath">logs/log4j2</Property>
</Properties>
<Appenders>
<!-- File Appender -->
<RollingFile name="FILE"
fileName="${basePath}/logfile.log" filePattern="${basePath}/logfile.%d{yyyy-MM-dd}-%i.log" append="true">
<PatternLayout
pattern="%-5p | %d{yyyy-MM-dd HH:mm:ss} | [%t] %C{2} (%F:%L) - %sense%n" />
<Policies>
<SizeBasedTriggeringPolicy size="1 KB" />
</Policies>
<DefaultRolloverStrategy max="4" />
</RollingFile>
<!-- Console Appender -->
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout
pattern="%-5p | %d{yyyy-MM-dd HH:mm:ss} | [%t] %C{2} (%F:%L) - %sense%n" />
</Console>
</Appenders>
<Loggers>
<Logger name="com.virtusa.xlab.fw" level="info" />
<Root level="info">
<AppenderRef ref="STDOUT" />
<AppenderRef ref="FILE" />
</Root>
</Loggers>
</Configuration>
Can anyone help me out here?
Thanks.
The problem is that SensitiveDataLog is created via static method newInstance(). Obviously, field ssn is not initialized at that moment. What you can do is to init the field later, e.g. when refreshing Spring context.
Here is my snippet:
private static XmlMaskPatternConverter INSTANCE = new XmlMaskPatternConverter();
public XmlMaskPatternConverter() {
super(NAME, NAME);
}
public static XmlMaskPatternConverter newInstance() {
return INSTANCE;
}
Now you can call static method getInstance() somewhere in your Spring Configuration (I do it in #Bean method) and set the ssn value there. Ofc, you need to create a setter for this field.
P.S. Hope it helps. I faced this problem too, so decided to leave my solution here. My first post on SO btw)
I'm using HandlerInterceptorAdapter to generate a correlation id for each service request. Here is the configuration I'm using. I'm trying to append the correlation id on my console log as shown in the expected output. But somehow it was not generating the log pattern as shown below.
log4j2.xml
<?xml version="1.0" encoding="UTF-8" ?>
<Configuration packages="com.microsoft.applicationinsights.log4j.v2">
<Properties>
<Property name="LOG_PATTERN">
%d{yyyy-MM-dd HH:mm:ss.SSS} %5p ${hostName} --- [%15.15t] %-40.40c{1.} : %m%n%ex
</Property>
</Properties>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %X{correlationId} %-5level %logger{36} - %msg%n"/>
</Console>
<ApplicationInsightsAppender name="aiAppender"/>
</Appenders>
<Loggers>
<Root level="trace">
<AppenderRef ref="Console" level = "info" />
<AppenderRef ref="aiAppender" level= "info" />
</Root>
</Loggers>
</Configuration>
CorrelationInterceptor.java
public class CorrelationInterceptor extends HandlerInterceptorAdapter{
private static final String CORRELATION_ID_LOG_VAR_NAME = "correlationId";
private static final String CORRELATION_ID_HEADER_NAME = "Correlation-Id";
#Override
public boolean preHandle(final HttpServletRequest request, final HttpServletResponse response,
final Object handler) throws Exception {
final String correlationId = getCorrelationIdFromHeader(request);
System.out.println("correlationId:{}"+correlationId);
MDC.put(CORRELATION_ID_LOG_VAR_NAME, correlationId);
System.out.println("correlationId:{}"+MDC.get(CORRELATION_ID_LOG_VAR_NAME));
return true;
}
#Override
public void afterCompletion(final HttpServletRequest request, final HttpServletResponse response,
final Object handler, final Exception ex) throws Exception {
System.out.println("correlationId:{}"+MDC.get(CORRELATION_ID_LOG_VAR_NAME));
MDC.remove(CORRELATION_ID_LOG_VAR_NAME);
System.out.println("correlationId:{}"+MDC.get(CORRELATION_ID_LOG_VAR_NAME));
}
private String getCorrelationIdFromHeader(final HttpServletRequest request) {
String correlationId = request.getHeader(CORRELATION_ID_HEADER_NAME);
if (IpUtil.isNullOrEmpty(correlationId)) {
correlationId = generateUniqueCorrelationId();
}
return correlationId;
}
private String generateUniqueCorrelationId() {
return UUID.randomUUID().toString();
}
}
Output:
2020-12-29 17:59:42.026 INFO 31140 --- [nio-8080-exec-1] filter.RequestHeaderFilter : No correlationId found In Header. Generated :29e624b7-3ab6-4a1c-8c6a-4da342de4858
2020-12-29 17:59:42.027 INFO 31140 --- [nio-8080-exec-1] filter.RequestHeaderFilter : Found coorelationId in Header : 29e624b7-3ab6-4a1c-8c6a-4da342de4858
Expected Output:
2020-12-30 13:59:01.352 [http-nio-8080-exec-3] ea7aa31a-1e26-42e0-b7a6-ef0562ba5c4f INFO helper.GraphServiceHelper - CATEGORIES : [13], SUB_CATEGORIES : [0]
2020-12-30 13:59:01.353 [http-nio-8080-exec-3] ea7aa31a-1e26-42e0-b7a6-ef0562ba5c4f INFO helper.GraphServiceHelper - locationType :club
How to add MDC variables in the json log generated by JsonLayout of log4j2. I've used KeyValuePair tag to add properties like host name into the log, but I didn't found any way to add MDC variables into it. In pattern layout I used %X{traceId} but I'm sure JsonLayout can't parse those conversion chars(As far as I know conversion chars are used by pattern layout only). I went into source code of JsonLayout but didn't found function which actually puts all of the data into the log message.
Thank you.
What you're looking for is a log4j2 lookup. It sounds like you're interested specifically in the Context Map Lookup as you mentioned MDC (which is now called ThreadContext in log4j2 by the way).
Here is a simple example:
package example;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;
public class ThreadContextExample {
private static final Logger log = LogManager.getLogger();
public static void main(String[] args) {
ThreadContext.put("myKey", "myValue");
log.info("Here's a message!");
}
}
Here is the log4j2.xml configuration:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<JsonLayout compact="false" eventEol="false" stacktraceAsString="true">
<KeyValuePair key="myJsonKey" value="${ctx:myKey}"/>
</JsonLayout>
</Console>
</Appenders>
<Loggers>
<Root level="debug">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
and finally some sample output (shortened for readability):
{
"thread" : "main",
"level" : "INFO",
"loggerName" : "example.ThreadContextExample",
"message" : "Here's a message!",
...
"myJsonKey" : "myValue"
}
Is there easy way to get development access logs on my console with Play 2.5?
Something I could read as "GET /foo/123 routed to FooController's show action with id=123"?
I've found how to get netty access log ( btw, option play.server.netty.log.wire=true in application.conf doesn't work for me for some reason, but -Dplay.server.netty.log.wire=true does ), but it's too low-level.
You can create a logger.xml file in the conf directory. It should follow logback's file format.
for instance a default configuration could look like :
<configuration scan="true" scanPeriod="5 seconds">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%level %logger{15} - %message%n%xException{5}</pattern>
</encoder>
</appender>
<logger name="play" level="INFO" />
<logger name="application" level="INFO" />
<root level="ERROR">
<appender-ref ref="STDOUT" />
</root>
</configuration>
You can then enable loglevels : FATAL, ERROR, WARNING, INFO, DEBUG, TRACE selectively for any package or as a default on the root leve.
This is described in the playframework configuration configuring logging
Note that this behaviour was changed in 2.4.x from the previous versions where you could configure loggers through application.conf
Once you have logging working you can use the sample logging filter provided in the documentation to log all requests to your server.
import javax.inject.Inject
import akka.stream.Materializer
import play.api.Logger
import play.api.mvc._
import scala.concurrent.{ExecutionContext, Future}
class LoggingFilter #Inject() (implicit val mat: Materializer, ec: ExecutionContext) extends Filter {
def apply(nextFilter: RequestHeader => Future[Result])
(requestHeader: RequestHeader): Future[Result] = {
val startTime = System.currentTimeMillis
nextFilter(requestHeader).map { result =>
val endTime = System.currentTimeMillis
val requestTime = endTime - startTime
Logger.info(s"${requestHeader.method} ${requestHeader.uri} took ${requestTime}ms and returned ${result.header.status}")
result.withHeaders("Request-Time" -> requestTime.toString)
}
}
}
you will have to activate it by setting the play.http.filters:
play.http.filters=com.example.LoggingFilter