Play! 2.5 log access to stdout in development - playframework-2.5

Is there easy way to get development access logs on my console with Play 2.5?
Something I could read as "GET /foo/123 routed to FooController's show action with id=123"?
I've found how to get netty access log ( btw, option play.server.netty.log.wire=true in application.conf doesn't work for me for some reason, but -Dplay.server.netty.log.wire=true does ), but it's too low-level.

You can create a logger.xml file in the conf directory. It should follow logback's file format.
for instance a default configuration could look like :
<configuration scan="true" scanPeriod="5 seconds">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%level %logger{15} - %message%n%xException{5}</pattern>
</encoder>
</appender>
<logger name="play" level="INFO" />
<logger name="application" level="INFO" />
<root level="ERROR">
<appender-ref ref="STDOUT" />
</root>
</configuration>
You can then enable loglevels : FATAL, ERROR, WARNING, INFO, DEBUG, TRACE selectively for any package or as a default on the root leve.
This is described in the playframework configuration configuring logging
Note that this behaviour was changed in 2.4.x from the previous versions where you could configure loggers through application.conf
Once you have logging working you can use the sample logging filter provided in the documentation to log all requests to your server.
import javax.inject.Inject
import akka.stream.Materializer
import play.api.Logger
import play.api.mvc._
import scala.concurrent.{ExecutionContext, Future}
class LoggingFilter #Inject() (implicit val mat: Materializer, ec: ExecutionContext) extends Filter {
def apply(nextFilter: RequestHeader => Future[Result])
(requestHeader: RequestHeader): Future[Result] = {
val startTime = System.currentTimeMillis
nextFilter(requestHeader).map { result =>
val endTime = System.currentTimeMillis
val requestTime = endTime - startTime
Logger.info(s"${requestHeader.method} ${requestHeader.uri} took ${requestTime}ms and returned ${result.header.status}")
result.withHeaders("Request-Time" -> requestTime.toString)
}
}
}
you will have to activate it by setting the play.http.filters:
play.http.filters=com.example.LoggingFilter

Related

Log4j log file not getting created at specified location springboot

I am trying to configure log4j2 in springboot.I have removed(excluded) the logback dependency already from pom.xml.I am using this xml under resource folder named log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="DEBUG">
<Appenders>
<Console name="LogToConsole" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
</Console>
<File name="LogToFile" fileName="logs/app.log">
<PatternLayout>
<Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
</PatternLayout>
</File>
</Appenders>
<Loggers>
<Logger name="com.ashish" level="debug" additivity="false">
<AppenderRef ref="LogToFile"/>
<AppenderRef ref="LogToConsole"/>
</Logger>
<Logger name="org.springframework.boot" level="error" additivity="false">
<AppenderRef ref="LogToConsole"/>
</Logger>
<Root level="error">
<AppenderRef ref="LogToFile"/>
<AppenderRef ref="LogToConsole"/>
</Root>
</Loggers>
</Configuration>
This is my controller class.
package com.ashish;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.GetMapping;
import java.util.Arrays;
import java.util.List;
#Controller
public class HelloController {
private static final Logger logger = LogManager.getLogger(HelloController.class);
private List<Integer> num = Arrays.asList(1, 2, 3, 4, 5);
#GetMapping("/")
public String main(Model model) {
if (logger.isDebugEnabled()) {
logger.debug("Hello from Log4j 2 - num : {}", num);
}
logger.debug("Hello from Log4j 2 - num : {}", () -> num);
model.addAttribute("tasks", num);
return "welcome";
}
private int getNum() {
return 100;
}
}
Am i missing anything here?I tried to set it using application.properties too using latest version of Log4j2.But still it's not getting created.When i run the application i can't see any log file getting dynamically created at the path specified in xml.
First, you have status="DEBUG" specified in your configuration. So you will see Log4j configure itself on the console (or wherever system.out is getting routed to). If you do not then you aren't really using Log4j.
If you do see the output then check the debug lines. I have a suspicion your log file is either not getting created due to a permissions problem or it isn't being written where you expect it.
Your configuration specifies a relative directory named "logs". Whatever directory is the working directory when the app is started should contain your logs directory. Frequently on Linux that will end up being "/". You almost certainly won't have permission to create a logs directory there so configuration will fail.
I can't comment on the content of the log4j2 config file itself, probably it will make sense to print on console first and make sure that its driven by log4j2 indeed.
However, I'll refer to the beginning of the question:
I am trying to configure log4j2 in springboot.I have removed(excluded) the logback dependency already from pom.xml
You don't present the pom.xml but in general in order to switch spring boot to work with log4j2 you should:
"Exclude" the default logging mechanism of spring boot:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
Add a log4j2 starter that will in turn (transitively) add log4j2 dependency of the versions compatible with your spring boot version:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
Quick googling has revealed this tutorial that contains all the steps including end-to-end example of such an integration.

Logback and JsonLayout: can't pass custom fields

I have this configuration in my logback.xml into a Spring Web Application (NO Spring Boot).
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="Console" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
<timestampFormat>yyyy-MM-dd'T'HH:mm:ss.SSSX</timestampFormat>
<timestampFormatTimezoneId>Etc/UTC</timestampFormatTimezoneId>
<jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
<prettyPrint>true</prettyPrint>
</jsonFormatter>
</layout>
<customFields>{"appname":"foobar"}</customFields>
</encoder>
</appender>
<!-- LOG everything at INFO level -->
<root level="INFO">
<appender-ref ref="Console" />
</root>
</configuration>
The JSON layout works fine but custom fields as "appname": "foobar" are not printed:
{
"timestamp" : "2020-06-10T14:55:25.534Z",
"level" : "INFO",
"thread" : "Catalina-utility-1",
"logger" : "org.springframework.web.servlet.DispatcherServlet",
"message" : "FrameworkServlet 'dispatcher': initialization completed in 72 ms",
"context" : "default"
}
What am I doing wrong?
SOLUTION
I was using the wrong libraries for my needs:
logback-jackson
logback-json-classic
Because of the fact that I need to process logs through Logstash I've corrected my configuration like this:
pom.xml
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.2.3</version>
</dependency>
<dependency>
<groupId>net.logstash.logback</groupId>
<artifactId>logstash-logback-encoder</artifactId>
<version>6.4</version>
</dependency>
logback.xml
<appender name="Console" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LogstashEncoder">
<customFields>{"customer":"X", "appname":"Y", "environment":"dev"}</customFields>
</encoder>
</appender>
and now It works fine.
I just stumbled this question because I had the same problem, and I found a solution, with logback-jackson and logback-json-classic.
Option 1: Per-Thread via Mapped Diagnostic Context (MDC)
SLF4j's Mapped Diagnostic Context is a per-thread key-value store that we can use to write custom structured data to the log output.
MDC.put("customKey", "customValue");
Logback's JsonLayout will automatically print this value under a special mdc JSON object without any further configuration.
{ [...], "mdc": {"customKey", "customValue"}}
Note that the MDC is constructed per thread and if it is empty, no mdc field is printed to the log output.
Option 2: Global (for all threads)
If you want custom fields to appear at the JSON output's root, you need to create a custom, but simple Layout class that extends JsonLayout. JsonLayout provides us with a addCustomDataToJsonMap we can override.
package com.mypackage;
import ch.qos.logback.contrib.json.classic.JsonLayout;
public class CustomJsonLayout extends JsonLayout {
#Override
protected void addCustomDataToJsonMap(Map<String, Object> map, ILoggingEvent event) {
map.put("customKey", "customValue");
}
}
Now, you just need to tell Logback to use CustomJsonLayout instead of JsonLayout in your logback.xml file and keep the rest the same.
<layout class="com.mypackage.CustomJsonLayout">
...
</layout>
Now, any log message will have the following output:
{ ..., "customKey": "customValue"}

How to enable Logback-access in Spring Boot?

I'm trying to enable logback-access in a spring boot app to log all http requests that hit the application.
I've tried implementing this using: https://github.com/akihyro/logback-access-spring-boot-starter
Adding the XML file shown in the example doesn't do anything, is there anything more that needs to be added to enable?
Any other suggestions to achieve the same result would be welcomed :)
you still need to wire a bean to your application...like this code snippet wires the filter to your application:
#SpringBootApplication
public class MyApp {
public static void main(String[] args) {
SpringApplication.run(MyApp.class, args);
}
// ... your other methods here
#Bean
public CommonsRequestLoggingFilter logFilter() {
CommonsRequestLoggingFilter filter = new CommonsRequestLoggingFilter();
filter.setIncludeQueryString(true);
return filter;
}
}
I am pretty sure you are talking about the logback logger for SpringBoot. If I am not wrong, this is how you can do this
a. Add the dependency in your POM
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
</dependency>
b. Now there are may ways you can ask spring to configure logback. For example
In the application.properties file
In the logback.xml file
The advantage of using logback.xml file is that, you might have different xml file for different build profile. But in the application.properties, you don’t have this freedom.
Sample entries in the application.properties file from one of my project
logging.level.org.springframework.web = INFO
logging.level.com.company.app = DEBUG
#logging.level.org.hibernate=ERROR
logging.file=logs/spring-boot-logging.log
## Hibernate Logging
logging.level.org.hibernate.SQL = DEBUG
If you are using XML, configuration probably will look like this
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n</Pattern>
</layout>
</appender>
<logger name="org.springframework" level="error" additivity="false">
<appender-ref ref="STDOUT" />
</logger>
<logger name="org.springframework" level="info" additivity="false">
<appender-ref ref="STDOUT" />
</logger>
<logger name="org.springframework" level="warn" additivity="false">
<appender-ref ref="STDOUT" />
</logger>
<logger name="com.memorynotfound" level="debug" additivity="false">
<appender-ref ref="STDOUT" />
</logger>
<configuration scan="true"/>
</configuration>
I recommend you to do some Googling for better understating.
Good luck!

Debug logging of ResponseBody by Springframework

I have a scenario where I send some sensitive data in the ResponseBody of a spring controller. Spring logs the entire response body in DEBUG mode, where I can see the sensitive data also. Now if I configure my app to print only INFO logs or higher, this won't be displayed, but that is not a fool proof method, since its possible that DEBUG mode could be turned on accidentally.
Is there anyway I can disable certain fields in the response body from being logged by Spring?
Thanks
My suggestion would be that you implement a custom converter for logback (if that is your logging library)
http://logback.qos.ch/manual/layouts.html#customConversionSpecifier
It enables you to convert your logging data and blur out any data that should be removed from the message.
From the documentation it is pretty straight forward
public class MySampleConverter extends ClassicConverter {
long start = System.nanoTime();
#Override
public String convert(ILoggingEvent event) {
long nowInNanos = System.nanoTime();
return Long.toString(nowInNanos-start);
}
}
And the config
<configuration>
<conversionRule conversionWord="nanos"
converterClass="chapters.layouts.MySampleConverter" />
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%-6nanos [%thread] - %msg%n</pattern>
</encoder>
</appender>
<root level="DEBUG">
<appender-ref ref="STDOUT" />
</root>
</configuration>
If you're using Log4j you can implement a custom filter which would mask the data per your layout
http://vozis.blogspot.sg/2012/02/log4j-filter-to-mask-payment-card.html
Edit
The final easier solution turned out to be to override toString of the return object. Spring uses toString to log the return message

Querydsl logging queries with bindings

logQuery is called in prepareStatementAndSetParameters mehtod - SQLInsertClause class
protected void logQuery(Logger logger, String queryString, Collection<Object> parameters) {
String normalizedQuery = queryString.replace('\n', ' ');
MDC.put(QueryBase.MDC_QUERY, normalizedQuery);
MDC.put(QueryBase.MDC_PARAMETERS, String.valueOf(parameters));
if (logger.isDebugEnabled()) {
logger.debug(normalizedQuery);
}
}
how can I set debug level to logger ?
That logger there is from SLF4J API. Depending on the logger you have behind the API you use facilities of that underlying logging implementation.
For instance we use Logback Classic (dependency ch.qos.logback:logback-classic) and I can explicitly override what configuration file to use with -Dlogback.configurationFile=devel-logback.xml in JVM parameters. Default mechanism is documented here. My file looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%date %level [%.60thread] %logger{1} %msg%n</pattern>
</encoder>
</appender>
<logger name="com.mysema.query.jpa.impl.JPAQuery" level="DEBUG"/>
<!-- more loggers -->
<root level="DEBUG">
<appender-ref ref="CONSOLE"/>
</root>
</configuration>
Also adding -Dlogback.debug=true to JVM arguments adds some debug output when logback is being initialized.

Resources