Spring boot embedded tomcat logs - spring

i'm using spring boot embedded tomcat with spring boot 1.5.9 ,
im also using Log4j2.
recently i exerience problems during load, so i want to understand better the tomcat logs [Not the access Logs] , i tried (in application.properties) :
logging.level.org.apache.tomcat: INFO
logging.level.org.apache.catalina: INFO
but none of the above worked. is there any other way to achieve it ?

Found it !! You are now able to see the internal Logs of Embedded Tomcat in your App's Log4j log file with 3 easy steps:
1] add to your pom:
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-jul</artifactId>
</dependency>
2] add to your running arg a new JVM param , e.g:
java -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -jar target/demo-0.0.1-SNAPSHOT.jar
3] add to your application.properties:
logging.level.org.apache=DEBUG
Enjoy Life ! :)
Explaination:
the problem is because Log4j log levels is not propagated into JUL (which is the actual Logging way Embedded tomcat use) so the above achieves this connection with JUL and Log4j log levels.
Reference:
After reading the Spring boot 1.5.10 release notes (which is not required for the solution) i saw the new documentation that shed light how to achive it and explaination about it:
https://github.com/spring-projects/spring-boot/issues/2923#issuecomment-358451260

I struggled a lot,and didnt find anything of my help.Utlimately I had build "WAR" out of my spring boot application.Deploy it to tomcat instance and
followed below steps,which redirected all the internal tomcat logs(JULI) logs to my application log file.
Delete existing JULI library (CATALINA_HOME/bin/tomcat-juli.jar file) and the existing Tomcat Java Logging configuration file (CATALINA_HOME/conf/logging.properties).
Download JULI Log4j Tomcat library (tomcat-juli.jar) from the Tomcat downloads’ Extras section (http://tomcat.apache.org/download-70.cgi). Place the downloaded file to CATALINA_HOME/bin directory.
Download Tomcat JULI adapters library (tomcat-juli-adapters.jar) from the Tomcat downloads’ Extras section. Place this file in the CATALINA_HOME/lib directory.
Download Log4j (version 1.2 or later), and place the downloaded library file to CATALINA_HOME/lib directory.
Create the Log4j configuration file at the following location: CATALINA_HOME/lib/log4j.properties. Check below log4j configuration matching the default Java Logging configuration.
Restart Tomcat.
Log4j configuration File Matching the Default Tomcat Logging Settings:
log4j.rootLogger=INFO, CATALINA
//Define all the appenders log4j.appender.CATALINA=org.apache.log4j.DailyRollingFileAppender
log4j.appender.CATALINA.File=${catalina.base}/logs/catalina.
log4j.appender.CATALINA.Append=true log4j.appender.CATALINA.Encoding=UTF-8
//Roll-over the log once per day
log4j.appender.CATALINA.DatePattern='.'yyyy-MM-dd'.log'
log4j.appender.CATALINA.layout = org.apache.log4j.PatternLayout
log4j.appender.CATALINA.layout.ConversionPattern = %d [%t] %-5p %c- %m%n
log4j.appender.LOCALHOST=org.apache.log4j.DailyRollingFileAppender
log4j.appender.LOCALHOST.File=${catalina.base}/logs/localhost.
log4j.appender.LOCALHOST.Append=true log4j.appender.LOCALHOST.Encoding=UTF-8
log4j.appender.LOCALHOST.DatePattern='.'yyyy-MM-dd'.log'
log4j.appender.LOCALHOST.layout = org.apache.log4j.PatternLayout
log4j.appender.LOCALHOST.layout.ConversionPattern = %d [%t] %-5p %c- %m%n
log4j.appender.MANAGER=org.apache.log4j.DailyRollingFileAppender
log4j.appender.MANAGER.File=${catalina.base}/logs/manager.
log4j.appender.MANAGER.Append=true log4j.appender.MANAGER.Encoding=UTF-8
log4j.appender.MANAGER.DatePattern='.'yyyy-MM-dd'.log'
log4j.appender.MANAGER.layout = org.apache.log4j.PatternLayout
log4j.appender.MANAGER.layout.ConversionPattern = %d [%t] %-5p %c- %m%n
log4j.appender.HOST-MANAGER=org.apache.log4j.DailyRollingFileAppender
log4j.appender.HOST-MANAGER.File=${catalina.base}/logs/host-manager.
log4j.appender.HOST-MANAGER.Append=true log4j.appender.HOST-MANAGER.Encoding=UTF-8
log4j.appender.HOST-MANAGER.DatePattern='.'yyyy-MM-dd'.log'
log4j.appender.HOST-MANAGER.layout = org.apache.log4j.PatternLayout
log4j.appender.HOST-MANAGER.layout.ConversionPattern = %d [%t] %-5p %c- %m%n
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.Encoding=UTF-8
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern = %d [%t] %-5p %c- %m%n
//Configure which loggers log to which appenders
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].
[localhost]=INFO,
LOCALHOST
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].
[localhost].[/manager]=INFO,MANAGER
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].
[localhost].[/host-manager]=
INFO, HOST-
MANAGER
You can also check a adapter avaiable on GIT # link
In your spring boot application,you can make changes like adding and removing jars,folder from embedded Tomcat server Or even adding custom config files to it using TomcatEmbeddedServletContainerFactory.class ,of spring boot.

For slf4j and Spring Boot 2 hide exceptions from Tomcat and handle them by yourself:
Add to pom:
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jul-to-slf4j</artifactId>
<version>${slf4j.version}</version>
</dependency>
Add to config:
#PostConstruct
void postConstruct() {
SLF4JBridgeHandler.install();
}
Add to application.yaml
logging:
level:
org.apache.catalina: off
Handle exception in ErrorController
#Controller
#Slf4j
public class ErrorController implements
org.springframework.boot.web.servlet.error.ErrorController {
private static final String ERROR_PATH = "/error";
#Autowired
private ErrorAttributes errorAttributes;
#Override
public String getErrorPath() {
return ERROR_PATH;
}
#RequestMapping(ERROR_PATH)
public ModelAndView error(HttpServletRequest request) {
return processException(errorAttributes.getError(new ServletWebRequest(request)));
}
}

Default configurations are provided for Java Util Logging, Log4J, Log4J2 and Logback. In each case loggers are pre-configured to use console output with optional file output also available
refer this link : https://stackoverflow.com/questions/31939849/spring-boot-default-log-location/31939886
The embedded tomcat in spring boot internally echoes logs to console.
The default log configuration will echo messages to the console as they are written. So until you explicitly specify a file as you described, it stays in the Console.
From the spring boot logging doc.
You can custmize the logging as per your need.

A log file, generated by org.apache.catalina.valves.AccessLogValve, usually named something like localhost_access_log can be configured like this:
#Configuration
public class EmbeddedTomcatConfig {
#Bean
public TomcatEmbeddedServletContainerFactory containerFactory() {
TomcatEmbeddedServletContainerFactory tomcatEmbeddedServletContainerFactory = new TomcatEmbeddedServletContainerFactory();
AccessLogValve accessLogValve = new AccessLogValve();
// set desired properties like
accessLogValve.setDirectory(...);
tomcatEmbeddedServletContainerFactory.addEngineValves(accessLogValve);
return tomcatEmbeddedServletContainerFactory;
}
}
Or, much better with Spring Boot 2:
#Bean
public WebServerFactoryCustomizer<TomcatServletWebServerFactory> customizer() {
return container -> {
AccessLogValve accessLogValve = new AccessLogValve();
// set desired properties like
accessLogValve.setDirectory("...");
container.addEngineValves(accessLogValve);
};
}

Related

Can I make Log4j show logging from Spring classes (specifically Spring Security)?

I have a Spring MVC web app that uses Spring Security 3.2.
It's configured to authenticate users against a MySQL database but users cannot log in. I suspect Spring Security is not connecting to the MySQL database, but there are no log messages displayed.
What do I add to my log4j.properties file to make the Spring Security logs appear?
I tried adding this to the file ...
log4j.category.org.springframework=DEBUG
But that didn't work. Here's my current log4j.properties file ...
status = error
name = PropertiesConfig
filters = threshold
filter.threshold.type = ThresholdFilter
filter.threshold.level = debug
appenders = console
appender.console.type = Console
appender.console.name = STDOUT
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = %d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n
rootLogger.level = info
rootLogger.appenderRefs = stdout
rootLogger.appenderRef.stdout.ref = STDOUT
log4j.category.org.springframework=DEBUG <<<--- adding this didn't work
Any ideas?
Try this :)
log4j.logger.org.springframework=DEBUG

AspectJ LTW (weaving) not working with Spring Boot

I'm on Spring Boot 2.1.2.RELEASE - Java 11 - Fat JAR
Following the documentation, I have:
added the required dependencies to the Gradle build
implementation 'org.springframework.boot:spring-boot-starter-aop'
implementation 'org.aspectj:aspectjrt:1.9.2'
implementation 'org.aspectj:aspectjweaver:1.9.2'
enabled LoadTimeWeaving
#SpringBootApplication
#EnableLoadTimeWeaving
public class MyApplication { ... }
provided the aop.xml under META-INF
<!DOCTYPE aspectj PUBLIC "-//AspectJ//DTD//EN" "http://www.eclipse.org/aspectj/dtd/aspectj.dtd">
<aspectj>
<weaver>
<!-- only weave classes in our application-specific packages -->
<include within="my.base.package.*" />
</weaver>
<aspects>
<!-- weave in just this aspect -->
<aspect name="my.base.package.spring.aop.MyAspects" />
</aspects>
</aspectj>
created the new #Aspect class
#Aspect
public class MyAspects {
#Around("methodsToBeProfiled()")
public Object profile(final ProceedingJoinPoint pjp) throws Throwable {
final var sw = new StopWatch(getClass().getSimpleName());
try {
sw.start(pjp.getSignature().getName());
return pjp.proceed();
} finally {
sw.stop();
System.out.println(sw.prettyPrint());
}
}
#Pointcut("execution(* my.base.package.other.MyClass.*(..))")
public void methodsToBeProfiled() {}
}
added the Jar for instrumentation
-javaagent:/home/myuser/spring-instrument-5.1.5.RELEASE.jar
Log, which as you see, show MyAspects as recognized
[AppClassLoader#2c13da15] info AspectJ Weaver Version 1.9.2 built on Wednesday Oct 24, 2018 at 15:43:33 GMT
[AppClassLoader#2c13da15] info register classloader jdk.internal.loader.ClassLoaders$AppClassLoader#2c13da15
[AppClassLoader#2c13da15] info using configuration /home/edoardo/IdeaProjects/scheduler/scheduler-engine/out/production/resources/META-INF/aop.xml
[AppClassLoader#2c13da15] info using configuration file:/home/edoardo/.gradle/caches/modules-2/files-2.1/org.springframework/spring-aspects/5.1.5.RELEASE/3bb95e05b646ef93e2a4cf0b600924c2979fc723/spring-aspects-5.1.5.RELEASE.jar!/META-INF/aop.xml
[AppClassLoader#2c13da15] info register aspect my.base.package.spring.aop.MyAspects
[AppClassLoader#2c13da15] info register aspect org.springframework.beans.factory.aspectj.AnnotationBeanConfigurerAspect
[AppClassLoader#2c13da15] info register aspect org.springframework.scheduling.aspectj.AnnotationAsyncExecutionAspect
[AppClassLoader#2c13da15] info register aspect org.springframework.transaction.aspectj.AnnotationTransactionAspect
[AppClassLoader#2c13da15] info register aspect org.springframework.transaction.aspectj.JtaAnnotationTransactionAspect
[AppClassLoader#2c13da15] info deactivating aspect 'org.springframework.transaction.aspectj.JtaAnnotationTransactionAspect' as it requires type 'javax.transaction.Transactional' which cannot be found on the classpath
[AppClassLoader#2c13da15] info register aspect org.springframework.cache.aspectj.AnnotationCacheAspect
[AppClassLoader#2c13da15] info register aspect org.springframework.cache.aspectj.JCacheCacheAspect
[AppClassLoader#2c13da15] info deactivating aspect 'org.springframework.cache.aspectj.JCacheCacheAspect' as it requires type 'org.springframework.cache.jcache.interceptor.JCacheAspectSupport' which cannot be found on the classpath
[AppClassLoader#2c13da15] info deactivating aspect 'org.springframework.cache.aspectj.JCacheCacheAspect' as it requires type 'javax.cache.annotation.CacheResult' which cannot be found on the classpath
[AppClassLoader#2c13da15] warning javax.* types are not being woven because the weaver option '-Xset:weaveJavaxPackages=true' has not been specified
However MyAspects is never instantiated (so no debug), and methods are not being weaved with my aspect code.
Did I miss something?
Edit: seems both Jars, aspectjweaver and spring-instrument are required as agents. Is this normal?
You do need the aspectjweaver agent jar. You don't need spring-instrument (ever for a fat jar - AFAIK it was used in app servers to work around some historical issues with their class loaders). You also don't need to #EnableLoadTimeWeaving (also an app server feature, redundant if you control the agent). Also (minor niggle) aspectjrt is a transitive dependency of aspectjweaver, so you don't need both. So you have it working it seems, even though you have done more work than you needed. Sample Spring Boot apps with various weaving options: here.

Spring Boot : LOGBack custom fields in every log statement

Thought of sharing the below work as I struggled to find this info:
Requirement is to put the business correlation id(AccCode and Mac) in every log statement. This can be done by using Mapped Diagnostic Context(MDC)..
No need to understand how LOGBack appender->Encoder->Layout work.
Spring boot log4j.properties has below pattern:
LOG_LEVEL_PATTERN=%5p
LOG_PATTERN=[%d{yyyy-MM-dd HH:mm:ss.SSS}] boot%X{context} - ${PID} ${LOG_LEVEL_PATTERN} [%t] --- %c{1}: %m%n
# CONSOLE is set to be a ConsoleAppender using a PatternLayout.
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=${LOG_PATTERN}
Override LOG_LEVEL_PATTERN from application.properties with custom fields:
logging.pattern.level=AccCode:%X{AccCode} Mac:%X{Mac} %5p
Please note, now we have two extra fields defined into the layout. AccCode and Mac. Value for these new fields need to be provided using the MDC:
MDC.put("AccCode", "46014" );
MDC.put("Mac", "2025b9bcf");
Then all the log statements would have AccCode and Mac as below:
2016-09-12 09:11:37.574 **AccCode:46014** **Mac:2025b9bcf** DEBUG 9724 --- [main] c.c.p.sph.http.client.SphApAdminClient : Loging out from Sph

Spring Boot and Log4j Issue

my log4j setup is following. When I run as Boot App, the logs are correctly written in console, debug.log and dump.log. Below is what I do in my program to write log in console and debug.log
static final Logger LOG = Logger.getLogger(EnvironmentLoader.class);
LOG.info("blah blah!");
Below is what I do in my program to write log in dump.log
private static final Logger DUMP_LOG = Logger.getLogger("dumpLogger");
DUMP_LOG.info("blah blah!");
Both works fine if I run as Spring Boot App. If I package it as war and run in tomcat, DUMP_LOG writes correctly in dump.log but LOG is not writing in console or debug.log. I wonder why.
log4j.rootLogger=INFO, stdout, debugLog
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d %-5p %c.%M:%L - %m%n
log4j.appender.stdout.Target=System.out
log4j.appender.debugLog=org.apache.log4j.DailyRollingFileAppender
log4j.appender.debugLog.Append=true
log4j.appender.debugLog.DatePattern='.'dd-MM-yyyy
log4j.appender.debugLog.File=${catalina.base}/logs/debug.log
log4j.appender.debugLog.MaxFileSize=10MB
log4j.appender.debugLog.encoding=UTF-8
log4j.appender.debugLog.layout=org.apache.log4j.PatternLayout
log4j.appender.debugLog.layout.ConversionPattern=%d %-5p %c.%M:%L - %m%n
log4j.category.debugLogger=DEBUG, debugLog
log4j.additivity.debugLogger=false
log4j.appender.dumpLog=org.apache.log4j.DailyRollingFileAppender
log4j.appender.dumpLog.Append=true
log4j.appender.dumpLog.DatePattern='.'dd-MM-yyyy
log4j.appender.dumpLog.File=${catalina.base}/logs/dump.log
log4j.appender.dumpLog.MaxFileSize=10MB
log4j.appender.dumpLog.encoding=UTF-8
log4j.appender.dumpLog.layout=org.apache.log4j.PatternLayout
log4j.appender.dumpLog.layout.ConversionPattern=%d - %m%n
log4j.category.dumpLogger=DEBUG, dumpLog
log4j.additivity.dumpLogger=false
The following dependency resolved the issue.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j</artifactId>
</dependency>

How to turn off Spring 3 debug logging?

I would like to turn off log4j logging for Spring 3.1 while leaving things on debug for my own code.
I tried sticking this line into my log4j.properties:
log4j.category.org.springframework = WARN
To get this:
# Root Logger Setup: Includes the level of reporting and appenders -> where
# log messages get sent
log4j.rootLogger = DEBUG,ca,fa
log4j.category.org.springframework = WARN
#ca - Console Appender - Send messages to the console
log4j.appender.ca = org.apache.log4j.ConsoleAppender
log4j.appender.ca.layout = org.apache.log4j.PatternLayout
log4j.appender.ca.layout.ConversionPattern = [acme]: [%-5p] - %d{yyyy-MMM-dd HH:mm:ss} - %c{1}:%M(): %m%n
#fa - File Appender - Send messages to a log file
log4j.appender.fa = org.apache.log4j.RollingFileAppender
log4j.appender.fa.File = acme.log
log4j.appender.fa.MaxFileSize = 100KB
log4j.appender.fa.MaxBackupIndex = 10
log4j.appender.fa.Threshold = DEBUG
log4j.appender.fa.layout = org.apache.log4j.PatternLayout
log4j.appender.fa.layout.ConversionPattern = [%-5p] - %d{yyyy-MMM-dd HH:mm:ss} - %c{2}:%M(): %m%n
No luck in shutting off the debug output from Spring though.
Thanks in advance for any help
Steve
Are all your dependencies in place?
1.3.2.3 Using Log4J
Many people use Log4j as a logging framework for configuration and management purposes. It's efficient and well-established, and in fact it's what we use at runtime when we build and test Spring. Spring also provides some utilities for configuring and initializing Log4j, so it has an optional compile-time dependency on Log4j in some modules.
To make Log4j work with the default JCL dependency (commons-logging) all you need to do is put Log4j on the classpath, and provide it with a configuration file (log4j.properties or log4j.xml in the root of the classpath). So for Maven users this is your dependency declaration:
<dependencies>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>3.0.0.RELEASE</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.14</version>
<scope>runtime</scope>
</dependency>
</dependencies>
And here's a sample log4j.properties for logging to the console:
log4j.rootCategory=INFO, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %t %c{2}:%L - %m%n
log4j.category.org.springframework.beans.factory=DEBUG
I figured this out.
In my classpath I have a directory C:\Classes for convenience when I am experimenting with things. I had another log4.properties file there, which was superseding the one packaged in my WAR making it look like the technique below wasn't working. I renamed the log4.properties in my C:\Classes and all is well.
Thanks to everyone who took a look and thanks to the people at Spring Source who made doing this necessary. It is good to know that an extensive level of debugging is easily available to me when I want it instead of just getting a black box.

Resources