log4j2 not generating logs on Staging machine - spring

I have upgraded log4j 1 to log4j2 in our Spring project. Logs works fine on Dev and QA machines. But when we deploy it on Staging, logs are not being produced. When we turn on debug under there are logs on startup. But it gets silent after that without error. This is my configuration file.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn" name="my-project" packages="">
<Appenders>
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="%d{ISO8601} %-5p (%x) [%t] %c{1} - %m%n"/>
</Console>
<!-- Debug Logger -->
<RollingFile name="DEBUG_LOG" fileName="/var/log/tomcat/my-project-backoffice-debug.log"
filePattern="/var/log/tomcat/my-project-backoffice-debug.log.%d{yyyy-MM-dd}.gz">
<ThresholdFilter level="DEBUG"/>
<PatternLayout >
<pattern>%d{ISO8601} %-5p (%x) [%t] %c{1} - %m%n</pattern>
</PatternLayout>
<TimeBasedTriggeringPolicy />
</RollingFile>
<!-- Info Logger -->
<RollingFile name="INFO_LOG" fileName="/var/log/tomcat/my-project-backoffice-info.log"
filePattern="/var/log/tomcat/my-project-backoffice-info.log.%d{yyyy-MM-dd}.gz">
<ThresholdFilter level="INFO"/>
<PatternLayout >
<pattern>%d{ISO8601} %-5p (%x) [%t] %c{1} - %m%n</pattern>
</PatternLayout>
<TimeBasedTriggeringPolicy />
</RollingFile>
<!-- Warn Logger -->
<RollingFile name="WARN_LOG" fileName="/var/log/tomcat/my-project-backoffice-warn.log"
filePattern="/var/log/tomcat/my-project-backoffice-warn.log.%d{yyyy-MM-dd}.gz">
<ThresholdFilter level="WARN"/>
<PatternLayout >
<pattern>%d{ISO8601} %-5p (%x) [%t] %c{1} - %m%n</pattern>
</PatternLayout>
<TimeBasedTriggeringPolicy />
</RollingFile>
<!-- Error Logger -->
<RollingFile name="ERROR_LOG" fileName="/var/log/tomcat/my-project-backoffice-error.log"
filePattern="/var/log/tomcat/my-project-backoffice-error.log.%d{yyyy-MM-dd}.gz">
<ThresholdFilter level="ERROR"/>
<PatternLayout >
<pattern>%d{ISO8601} %-5p (%x) [%t] %c{1} - %m%n</pattern>
</PatternLayout>
<TimeBasedTriggeringPolicy />
</RollingFile>
</Appenders>
<Loggers>
<Logger name="org.springframework" level="INFO"/>
<Root level="info">
<AppenderRef ref="DEBUG_LOG"/>
<AppenderRef ref="INFO_LOG"/>
<AppenderRef ref="WARN_LOG"/>
<AppenderRef ref="ERROR_LOG"/>
</Root>
</Loggers>
Thanks.
Update: The Solution:
One of older log4j dependency (transitive) was being loaded on Stage machine. Which is weird because the package got generated by Jenkins worked fine on Dev and QA machines. So I made sure to remove all transitive dependencies and it worked fine after that. These are the transitive dependencies I deleted.
commons-logging-1.1.1.jar
log4j-slf4j-impl-2.11.2.jar
log4j-1.2.17.jar

Related

console pattern not recognized in logback-spring.xml

I already looked up on stackoverflow but I did not find something comparable.
I use spring boot with following dependencies:
spring-boot 2.2.1 with
spring-boot-starter-logging 2.2.1
logback-core 1.2.3
logback-classic 1.2.3
I want to use a custom pattern for my log output to print my mdc.variables on console. Which works via application.properties
logging.level.com.mypackage=TRACE
logging.pattern.level=%d{yyyy-MM-dd HH:mm:ss} [%t] [myMdc:%X{myMdcVariable}] %-5level %logger{36} - %m%n
But I get double entries but I can not set the additivity property via application.properties. I have to use logback-spring.xml. Which looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<include resource="org/springframework/boot/logging/logback/console-appender.xml"/>
<root level="INFO">
<appender-ref ref="CONSOLE"/>
</root>
<logger name="com.mypackage" level="TRACE" additivity="false">
<appender-ref ref="CONSOLE"/>
</logger>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%d{yyyy-MM-dd HH:mm:ss} [%t] [myMdc:%X{myMdcVariable}] %-5level %logger{36} - %m%n
</Pattern>
</encoder>
</appender>
</configuration>
This solves the double entries. But the pattern is not use for the logoutput. I can have either double entries or no pattern because it is not possible to combine configuration via application.properties and logback-spring.xml. What am I doing wrong?
I finally found out what is wrong. I checked the defaults.xml and the ConsoleAppender.xml on github e.g.
https://github.com/spring-projects/spring-boot/blob/master/spring-boot-project/spring-boot/src/main/resources/org/springframework/boot/logging/logback/defaults.xml
https://github.com/spring-projects/spring-boot/blob/master/spring-boot-project/spring-boot/src/main/resources/org/springframework/boot/logging/logback/console-appender.xml
The solution was to define the constant CONSOLE_LOG_PATTERN. And to define it before importing the defaults.xml and console-appender.xml
My working config file looks now like this:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="CONSOLE_LOG_PATTERN" value="%d{yyyy-MM-dd HH:mm:ss} [%t] [myMdc:%X{myMdcVariable] %-5level %logger{36} - %m%n"/>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<include resource="org/springframework/boot/logging/logback/console-appender.xml"/>
<root level="INFO">
<appender-ref ref="CONSOLE"/>
</root>
<logger name="com.mypackage" level="TRACE" additivity="false">
<appender-ref ref="CONSOLE"/>
</logger>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>
${CONSOLE_LOG_PATTERN}
</pattern>
</encoder>
</appender>
</configuration>

Different developement and production environment production logs when they shouldn't

I am working on a 9 yo Spring application with log generation done with Logback.
Logging is working fine in development environment, but when we switch to our production environment, logs are still being written until the "Application started in ...ms". After that, nothing else will ever been written.
After some digging, I noticed that our logback is not built in our war but is set into Tomcat libs. It made me think that we could have override properties disabling our logging settings. Problem is, the developer who worked on that is not in my company anymore, so I'm basically searching blindly.
Do you think I am right with my first assumption? Where should I search for some conf files overriding my application.properties?
Here is our application.properties logging settings:
logging.config= classpath:./extranet_config/logback.xml
logging.level.com.sun.mail= trace
logging.exception-conversion-word=
logging.pattern.console=%d{HH:mm:ss.SSS} %-5level [%thread] %logger - %msg %n
logging.pattern.file=.%d{HH:mm:ss.SSS} [%thread] %-5level - %msg %n
logging.pattern.level=%5p
And the logback:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="LOGS_FOLDER" value="C://log/Extranet/logs" />
<property name="LOGS_TO_COLLECT_FOLDER" value="C://log/Extranet/logs-to-collect" />
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>TRACE</level>
</filter>
<encoder>
<Pattern>%d{HH:mm:ss.SSS} | %-5level | %-22thread | %-12logger | %msg %n</Pattern>
</encoder>
</appender>
<appender name="log-file" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>TRACE</level>
</filter>
<file>${LOGS_FOLDER}/extranet.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOGS_TO_COLLECT_FOLDER}/extranet.%d{yyyyMMdd}.log
</fileNamePattern>
</rollingPolicy>
<encoder>
<pattern>%d{HH:mm:ss.SSS} | %-5level | %-22thread | %-12logger | %m %throwable{0}%n</pattern>
</encoder>
</appender>
<appender name="troubleshooting-file" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>ERROR</level>
</filter>
<file>${LOGS_FOLDER}/extranet-troubleshooting.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${LOGS_FOLDER}/extranet-troubleshooting.%i.log
</fileNamePattern>
<maxIndex>10</maxIndex>
</rollingPolicy>
<triggeringPolicy
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>10MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{HH:mm:ss.SSS} | %-5level | %-22thread | %-12logger | %msg %n</pattern>
</encoder>
</appender>
<logger name="org.springframework" level="INFO"/>
<logger name="org.hibernate" level="ERROR"/>
<logger name="org.springframework.boot.web.support.ErrorPageFilter" level="OFF" />
<logger name="Application" level="DEBUG"/>
<logger name="QueryLogger" level="DEBUG"/>
<logger name="AOPLogger" level="TRACE"/>
<root>
<appender-ref ref="console" />
<appender-ref ref="log-file" />
<appender-ref ref="troubleshooting-file" />
</root>
</configuration>
Note: those two files are identical in prod and in development environment.
For some reason, our logback classpath was not read the same way on both servers, we changed them and everything is working properly now.

log4j2 not writing to log file specified in xml file

I'm running into an issue with log4j2 on my springboot api. We're using ubuntu on an EC2 instance and can't get the log4j2.xml to create a log file where we want it to, nor write to a log file that we manually create in that location.
We've checked the xml file multiple times, and as far as we can tell the issue isn't with the file itself. We've also messed around with permissions for writability of the log file and we don't think that's the problem either; we're both super new to linux though so we could be wrong.
<Configuration>
<Properties>
<Properties>
<Property name="filename">log/api.log</Property>
<Property name="LOG_PATTERN">%d{yyyy-MM-dd'T'HH:mm:ss.SSSZ} %p %m%n</Property>
</Properties>
</Properties>
<Appenders>
<Console name="ConsoleAppender" target="SYSTEM_OUT">
<PatternLayout
pattern="[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n"/>
</Console>
<File name="MyFile" fileName="log/api.log">
<PatternLayout
pattern="[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n"/>
</File>
<RollingFile name="appLog"
fileName="api.log"
filePattern="application-%d{yyyy-MM-dd}-%i.log">
<PatternLayout pattern="${LOG_PATTERN}" />
<Policies>
<SizeBasedTriggeringPolicy size="19500KB" />
</Policies>
<DefaultRolloverStrategy max="1" />
</RollingFile>
</Appenders>
<Loggers>
<Logger name="org.hibernate.SQL" level="debug" additivity="false">
<AppenderRef ref="ConsoleAppender"/>
<AppenderRef ref="MyFile"/>
<AppenderRef ref="appLog"/>
</Logger>
<Logger name="org.hibernate.type.descriptor.sql" level="trace" additivity="false">
<AppenderRef ref="ConsoleAppender"/>
<AppenderRef ref="MyFile"/>
<AppenderRef ref="appLog"/>
</Logger>
<Root level="info">
<AppenderRef ref="ConsoleAppender"/>
<AppenderRef ref="MyFile"/>
<AppenderRef ref="appLog"/>
</Root>
</Loggers>
</Configuration>
Any thoughts?
You have two files specified; log/api.log and api.log. Log4j will create these in whatever the current working directory is when the application is started. Since you haven't stated where the files are actually being created not much more can be said.

Logback I want File do not create when the appender is never used. [RollingFileAppender]

I'm creating a logback-common config file for my applis.
I'm defining a RollingFileAppender in it, that for all my applis generates the same log format in a file (if we need it).
Sometimes I want to use this appender, and sometimes not (when we test for example).
So we configure our specific logback-{profile}.xml depending what we want.
But when we do not use the FILE appender, the file gets created and I would like not.
I have:
logback-common.xml >> with all appenders definition (FILE and COMMON)
appli_one
resources/logback.xml >> call logback-common and config/logback-{profile}.xml
resources/config/logback-{profile}.xml >> specific appli/profile logback configuration.
To configure we can do in logback-{profile}.xml
<root level="WARN">
<appender-ref ref="FILE"/> <!-- For File Log when we need it -->
<appender-ref ref="CONSOLE"/>
</root>
<root level="WARN">
<!-- <appender-ref ref="FILE"/> --> <!-- in comment when we do not need if > BUT create a empty file -->
<appender-ref ref="CONSOLE"/>
</root>
logback-common.xml
<included>
<!-- The FILE and ASYNC appenders are here as examples for a production
configuration -->
<appender name="FILE"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>log/${spring.application.name}.log</file>
<rollingPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>
log/${spring.application.name}.%d{yyyy-MM-dd}.%i.log.zip
</fileNamePattern>
<maxHistory>90</maxHistory>
<maxFileSize>10MB</maxFileSize>
</rollingPolicy>
<encoder>
<charset>utf-8</charset>
<Pattern>%d %-5level [%thread] %logger{0}: %msg%n</Pattern>
</encoder>
</appender>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>"%d{yyyy-MM-dd} [%thread] %-5level %45logger{45} - %msg%n"</pattern>
</encoder>
</appender>
</included>
logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" packagingData="true">
<property name="spring_profiles_active" value="${spring.profiles.active}" />
<property resource="application.properties"/>
<include resource="config/log/logback-common.xml"/>
<include resource="config/log/logback-${spring_profiles_active}.xml"/>
</configuration>
Not a core feature of logback but there are workarounds to achieve lazy file initialization.
See more here
Logback - do not create empty log files at startup

Can't get a RollingFile working with log4j2

I'm trying to configure log4j2 to write a rolling log file to disk, but I can't get it to work. No log file appears at the given path and the Glassfish server.log doesn't show any Spring logging at all. I've read a lot of similar questions on SO, but none of the proposed solutions have worked in this case. Can anyone help me? I'm using Spring 3.0 on a Glassfish 3.1 application server.
From my pom.xml:
<properties>
<junit.version>4.11</junit.version>
<tiles.version>3.0.3</tiles.version>
<slf4j.version>1.7.5</slf4j.version>
<log4j.version>2.0-beta9</log4j.version>
</properties>
<dependencies>
<dependency>
<artifactId>jcl-over-slf4j</artifactId>
<groupId>org.slf4j</groupId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>${log4j.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>${log4j.version}</version>
</dependency>
</dependencies>
And here's my log4j.xml:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO">
<Properties>
<Property name="fileName" value="C:/temp/rolling-file.log"/>
<Property name="fileNamePattern" value="C:/temp/rolling-file-$d{dd-MM-yyyy}-%i.log"/>
<Property name="logPattern" value="%d{dd-MM-yyyy HH:mm:ss,SSS} [%t] %-5p %c - %m%n"/>
</Properties>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="${logPattern}"/>
</Console>
<RollingFile name="RollingFile" fileName="${fileName}" filePattern="${fileNamePattern}">
<PatternLayout pattern="${logPattern}"/>
<Policies>
<TimeBasedTriggeringPolicy/>
<SizeBasedTriggeringPolicy size="20MB"/>
</Policies>
<DefaultRolloverStrategy max="10"/>
</RollingFile>
</Appenders>
<Loggers>
<Logger name="my.root.package" level="info" additivity="false">
<AppenderRef ref="Console"/>
<AppenderRef ref="RollingFile"/>
</Logger>
<Root level="info">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
Besides the excellent point made by M. Deinum, it turns out my log4j2.xml needed to be changed to the following:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="DEBUG">
<Properties>
<Property name="fileName">C:/temp/rolling-file.log</Property>
<Property name="fileNamePattern">C:/temp/rolling-file-%d{dd-MM-yyyy}-%i.log</Property>
<Property name="logPattern">%d{dd-MM-yyyy HH:mm:ss,SSS} [%t] %-5p %c - %m%n</Property>
</Properties>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="${logPattern}"/>
</Console>
<RollingFile name="MyRollingFile" fileName="${fileName}" filePattern="${fileNamePattern}">
<PatternLayout pattern="${logPattern}"/>
<Policies>
<TimeBasedTriggeringPolicy/>
<SizeBasedTriggeringPolicy size="20MB"/>
</Policies>
<DefaultRolloverStrategy max="10"/>
</RollingFile>
</Appenders>
<Loggers>
<Logger name="my.root.package" level="info" additivity="false">
<AppenderRef ref="Console"/>
<AppenderRef ref="MyRollingFile"/>
</Logger>
<Root level="info">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
Performed changes:
The value attributes of the Property elements were not parsed correctly. Specifying them as body of the Property element seems to solve this.
There was a mistake in the value of the fileNamePattern property. I used $d to denote the date, but it should be %d.
Apparently log4j2 doesn't like a RollingFile element with the name "RollingFile". After I changed it, logging started working.
You might need to include log4j-api to the pom dependencies (http://logging.apache.org/log4j/2.x/faq.html#which_jars ).
About the properties: if the problem goes away by putting the values in the config directly:
<RollingFile name="RollingFile" fileName="C:/temp/rolling-file.log"
filePattern="C:/temp/rolling-file-$d{dd-MM-yyyy}-%i.log">
<PatternLayout pattern="%d{dd-MM-yyyy HH:mm:ss,SSS} [%t] %-5p %c - %m%n"/>
then you may have found a bug. In that case, could your raise this issue in the log4j2 Jira issue tracker?

Resources