Different developement and production environment production logs when they shouldn't - spring

I am working on a 9 yo Spring application with log generation done with Logback.
Logging is working fine in development environment, but when we switch to our production environment, logs are still being written until the "Application started in ...ms". After that, nothing else will ever been written.
After some digging, I noticed that our logback is not built in our war but is set into Tomcat libs. It made me think that we could have override properties disabling our logging settings. Problem is, the developer who worked on that is not in my company anymore, so I'm basically searching blindly.
Do you think I am right with my first assumption? Where should I search for some conf files overriding my application.properties?
Here is our application.properties logging settings:
logging.config= classpath:./extranet_config/logback.xml
logging.level.com.sun.mail= trace
logging.exception-conversion-word=
logging.pattern.console=%d{HH:mm:ss.SSS} %-5level [%thread] %logger - %msg %n
logging.pattern.file=.%d{HH:mm:ss.SSS} [%thread] %-5level - %msg %n
logging.pattern.level=%5p
And the logback:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="LOGS_FOLDER" value="C://log/Extranet/logs" />
<property name="LOGS_TO_COLLECT_FOLDER" value="C://log/Extranet/logs-to-collect" />
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>TRACE</level>
</filter>
<encoder>
<Pattern>%d{HH:mm:ss.SSS} | %-5level | %-22thread | %-12logger | %msg %n</Pattern>
</encoder>
</appender>
<appender name="log-file" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>TRACE</level>
</filter>
<file>${LOGS_FOLDER}/extranet.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOGS_TO_COLLECT_FOLDER}/extranet.%d{yyyyMMdd}.log
</fileNamePattern>
</rollingPolicy>
<encoder>
<pattern>%d{HH:mm:ss.SSS} | %-5level | %-22thread | %-12logger | %m %throwable{0}%n</pattern>
</encoder>
</appender>
<appender name="troubleshooting-file" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>ERROR</level>
</filter>
<file>${LOGS_FOLDER}/extranet-troubleshooting.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${LOGS_FOLDER}/extranet-troubleshooting.%i.log
</fileNamePattern>
<maxIndex>10</maxIndex>
</rollingPolicy>
<triggeringPolicy
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>10MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{HH:mm:ss.SSS} | %-5level | %-22thread | %-12logger | %msg %n</pattern>
</encoder>
</appender>
<logger name="org.springframework" level="INFO"/>
<logger name="org.hibernate" level="ERROR"/>
<logger name="org.springframework.boot.web.support.ErrorPageFilter" level="OFF" />
<logger name="Application" level="DEBUG"/>
<logger name="QueryLogger" level="DEBUG"/>
<logger name="AOPLogger" level="TRACE"/>
<root>
<appender-ref ref="console" />
<appender-ref ref="log-file" />
<appender-ref ref="troubleshooting-file" />
</root>
</configuration>
Note: those two files are identical in prod and in development environment.

For some reason, our logback classpath was not read the same way on both servers, we changed them and everything is working properly now.

Related

Write specfic logs to syslog using logback in springboot

I have a spring boot microservice application. logback.xml is configured to write the logs to a location on the server. In addition to this, I want to write logs from a particular java class to Syslog. I do not want logs from all other classes to be written to syslog but to a rolling file appender. Is there a way to configure this using logback.
below is my logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="DEV_HOME" value="logs"></property>
<appender name="Console"
class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>%black(%d{ISO8601}) %highlight(%-5level) [%blue(%t)] %yellow(%C{1.}): %msg%n%throwable
</Pattern>
</layout>
</appender>
<appender name="RollingFile"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>/data/storage/log/cms.log</file>
<encoder
class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>%d %p %C{1.} [%t] %m%n</Pattern>
</encoder>
<rollingPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<!-- roll over daily and when the file reaches 10 MB, max of 7 days or 3GB threshold -->
<fileNamePattern>/data/storage/log/cms.%d{yyyy-MM-dd}.%i.Logs.gz
</fileNamePattern>
<maxFileSize>100MB</maxFileSize>
<maxHistory>7</maxHistory>
<totalSizeCap>3GB</totalSizeCap>
</rollingPolicy>
</appender>
<appender name="SPECIFIC_CLASS" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${DEV_HOME}/sizeTimeOutputlogFile.log</file>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>[%d{yyyy-MM-dd HH:mm:ss}] %p %c{1.} [%t]- %m%n</pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${DEV_HOME}/archived/sizeTimeOutputlogFile.%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<!--Max Size of file to start Archive -->
<maxFileSize>10KB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<!-- Days till log history to keep -->
<maxHistory>3</maxHistory>
</rollingPolicy>
</appender>
<!--syslog appender-->
<appender name="SYSLOG" class="ch.qos.logback.classic.net.SyslogAppender">
<syslogHost>localhost</syslogHost>
<facility>LOCAL0</facility>
<port>514</port>
<throwableExcluded>true</throwableExcluded>
<suffixPattern>%thread: %-5level %logger{36} - %msg%n</suffixPattern>
</appender>
<!-- LOG everything at ERROR level -->
<root level="ERROR">
<appender-ref ref="RollingFile"/>
<appender-ref ref="Console"/>
</root>
<!-- LOG "com.myproject*" at ERROR level -->
<logger name="com.myproject" level="ERROR" additivity="false">
<appender-ref ref="RollingFile"/>
<appender-ref ref="Console"/>
</logger>
<logger name="com.myproject.ActivityLogAspect" level="DEBUG" additivity="false">
<appender-ref ref="SYSLOG"/>
<appender-ref ref="SPECIFIC_CLASS"/>
</logger>
</configuration>
From the above XML, SYSLOG and SPECIFIC_CLASS appenders are used to write logs from ActivityLogAspect class. When I run my application I see that only SPECIFIC_CLASS is working. I don't see any logs written to /var/log/messages. But if I add the SYSLOG appender along with the console appender, I see the messages written to /var/log/messages.
I do not want logs from all classes written to syslog but only from one particular class.
Please advise.

Writing log in file and display it on console at the same time

here is my logback-spring.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="DEBUG_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>DEBUG</level>
</filter>
<file>./target/log/app-debug.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>./log/target/app-debug.log-%d.gz</fileNamePattern>
<maxHistory>7</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%-30(%d{HH:mm:ss.SSS} [%thread]) %-5level %logger{32} - %msg%n</pattern>
</encoder>
</appender>
<appender name="ERROR_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>ERROR</level>
</filter>
<file>./target/log/app-error.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>./target/log/app-error.log-%d.gz</fileNamePattern>
<maxHistory>7</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%-30(%d{HH:mm:ss.SSS} [%thread]) %-5level %logger{32} - %msg%n</pattern>
</encoder>
</appender>
<root level="DEBUG">
<appender-ref ref="DEBUG_FILE"/>
</root>
<root level="ERROR">
<appender-ref ref="ERROR_FILE"/>
</root>
</configuration>
Now everything log correctly but I am unable to see INFO level on my console while running the app. I still have
logging:
level:
root: INFO
file:
name: ./log/app.log
in my .yml file but there is no whatsoever logging on console.
Can I write the log in file and display it on the console at the same time?
You can use ConsoleAppender in your existing logback config. Below is a small snippet for your reference,
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>
%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n
</Pattern>
</layout>
</appender>
And you can control the root log level and also include the reference of your appender "DEBUG_FILE" as well to print in the console in addition to the implementation which you have done to write in a file. Below snippet is set to info to match your use case,
<root level="info">
<appender-ref ref="STDOUT" />
<appender-ref ref="DEBUG_FILE" />
</root>

Logback created file but not able to append

I am using spring boot and logback. I am able to log into file in my local environment with IDE (IntelliJ), but it fails to log on AWS EC2 redhat linux with a JBoss EAP 7.1. I tried to set the log file permission to 777 after it is created, but still not working. The way I start up the server is
sudo sh /home/ec2-user/jboss-eap-7.1/bin/standalone.sh
my logback-spring.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<!--Local-->
<!--<property name="LOG_FOLDER" value="${user.dir}/log" />-->
<!--Server-->
<property name="LOG_FOLDER" value="/home/ec2-user/jboss-eap-7.1/applications/springboottest/log"/>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>[%d{HH:mm:ss.SSS}] THREAD=[%6.-6X{threadID}] %-5level [%30.-30logger{0}] - %msg%n</Pattern>
</layout>
</appender>
<appender name="ALL" class="ch.qos.logback.core.rolling.RollingFileAppender">
<append>true</append>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_FOLDER}/log.all.%d{yyyy-MM-dd}.log</fileNamePattern>
<maxHistory>60</maxHistory>
</rollingPolicy>
<encoder>
<pattern>[%d{HH:mm:ss.SSS}] THREAD=[%6.-6X{threadID}] %-5level [%30.-30logger{0}] - %msg%n</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="STDOUT"/>
<appender-ref ref="ALL"/>
</root>
</configuration>
Does anyone know why it cannot append to the file?
Any help is appreciated. Thanks

how to specify specific logger level for each of file writing and console writing

I want to maintain a log of only WARN, ERROR and FATAL in my logfile.log whereas want all above INFO to be written to console in my spring boot application. I see that when I do log.level.root=WARN, in application.properties file it sets a log level of WARN for the whole application. What can I do for displaying all log messages in console but writing only ones above WARN in the file?
Also!!!!
Is there a way to set level to INFO and write only certain specified logs in the file. For eg. I want to trace each and every logins/logouts/filechanges (say business logic related) in log with INFO level but don't want to see the default INFO messages that pop up during the start of the application like
2018-07-11 10:19:00.554 INFO 11190 --- [ main] c.v.guruji.GurujiApplication : Started GurujiApplication in 10.307 seconds (JVM running for 11.029)
.
The answer depends on which actual logging framework are you planning to use,
spring boot has integrations with most of them.
In general, if you opt for "zero configuration", specify the level of events in spring boot configuration files (application.properties / yaml) with the following line:
logging.level.com.myorg=DEBUG
logging.level.root=INFO
If you want to write to files, or in general provide more flexible configurations, you'll need to configure something called appender, but it's not a spring boot notion, but a logging framework term.
So the actual configuration will depend on a specific logging framework, usually, people put an XML configuration file (like logback.xml if you use logback) into src/main/resources folder and it works. Another modern option is log4j2 framework.
From the question it seems like you better first understand how to use these frameworks and then try to integrate them into spring boot, each of them is really flexible and will provide a solution you're looking for
Here is a link to the tutorial on how to use each of those frameworks with spring boot
Here is a logging chapter of official documentation of spring boot
Well , I went through some logback documentations and pretty much figured out what to do.
I'm posting this code for others who'll be having such problems as me for quick reference.
But I'll suggest to go through a proper documentation at first to understand it clearly.
logback.xml(for logging each level log in a separate file and to show all at console)
<?xml version="1.0" encoding="UTF-8"?>
<property name="DEV_HOME" value="logs" />
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>
%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n
</Pattern>
</layout>
</appender>
<appender name="FILE-ERROR"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${DEV_HOME}/error.log</file>
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>ERROR</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n
</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${DEV_HOME}/archived/error.%d{yyyy-MM-dd}.%i.log
</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>10MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
<appender name="FILE-INFO"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${DEV_HOME}/info.log</file>
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>INFO</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n
</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${DEV_HOME}/archived/info.%d{yyyy-MM-dd}.%i.log
</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>10MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
<appender name="FILE-FATAL"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${DEV_HOME}/fatal.log</file>
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>FATAL</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n
</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${DEV_HOME}/archived/fatal.%d{yyyy-MM-dd}.%i.log
</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>10MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
<appender name="FILE-WARN"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${DEV_HOME}/warn.log</file>
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>WARN</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n
</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${DEV_HOME}/archived/warn.%d{yyyy-MM-dd}.%i.log
</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>10MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
<!-- Send logs to both console and file audit -->
<logger name="com.vaidiksanatansewa.guruji" level="fatal"
additivity="false">
<appender-ref ref="FILE-FATAL" />
</logger>
<logger name="com.vaidiksanatansewa.guruji" level="error"
additivity="false">
<appender-ref ref="FILE-ERROR" />
</logger>
<logger name="com.vaidiksanatansewa.guruji" level="warn"
additivity="false">
<appender-ref ref="FILE-WARN" />
</logger>
<logger name="com.vaidiksanatansewa.guruji" level="info"
additivity="false">
<appender-ref ref="FILE-INFO" />
</logger>
<root level="INFO">
<appender-ref ref="STDOUT" />
</root>

SLF4J+Logback to create a file if doesn't exists/ if deleted

I have been researching for this for a long time and doesn't get a solution at all. My requirement is to create a log file if it is not there at the place.
Below is my logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="LOG_PATH" value="/home/logs" />
<appender name="FILE-AUDIT"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_PATH}/debug.log</file>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%p\t%d{dd MMM yyyy HH:mm:ss,SSS}\t%r\t%c\t[--%t--]\t%m%n
</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${LOG_PATH}/debug.%d{yyyy-MM-dd}.%i.log
</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>10MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
<logger name="com.example" level="debug"
additivity="false">
<appender-ref ref="FILE-AUDIT" />
</logger>
<root level="debug">
<appender-ref ref="FILE-AUDIT" />
</root>
</configuration>
And I have tried changing the configuration to configuration debug="true", but no luck.
What is the right way to implement this ?
Please don't write to check on file permissions or ask me to use Log4j, I changed my application to use slf4j instead of log4j. If i delete the file also, it should create
You should use ch.qos.logback.core.FileAppender for file appending.
<appender name="File-Appender" class="ch.qos.logback.core.FileAppender">
<file>${LOG_PATH}/logfile-${timestamp-by-second}.log</file>
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
Since you've said that you worked with Log4j previously make sure you didn't exclude "spring-boot-starter-logging" from your .pom in tryings to avoid conflicts between Log4j and slf4j.
I'm using the same settings in logback.xml and everything is working fine
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="consoleAppender" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<Pattern>%d{HH:mm:ss} [%thread] %-5level %logger{36} - %msg %n</Pattern>
</encoder>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>TRACE</level>
</filter>
</appender>
<appender name="testServiceFileAppender" class="ch.qos.logback.core.rolling.RollingFileAppender">
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<FileNamePattern>logs/test_service_%d{yyyy-MM-dd}.log</FileNamePattern>
</rollingPolicy>
<encoder>
<Pattern>%-5level %logger{35} - %msg %n</Pattern>
</encoder>
</appender>
<logger name="com.mycompany.test" additivity="false">
<level value="INFO"/>
<appender-ref ref="testServiceFileAppender"/>
</logger>
<root>
<level value="INFO"/>
<appender-ref ref="consoleAppender"/>
</root>
</configuration>
in pom:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
</dependencies>

Resources