Spring boot sending mails with log4j2 - spring-boot

I just want to send logs with ERROR level via email in my spring boot application using log4j2.
After reading infinite tutorials about this, im unable to reach this.
I have a Docker SMTP server in localhost, and here are my configuration files:
pom.xml
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.2.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
</dependency>
<!-- Add Log4j2 Dependency -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
<!-- Needed for SMTP appender -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-mail</artifactId>
</dependency>
...
...
log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="trace">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout
pattern="%style{%d{ISO8601}}{black} %highlight{%-5level }[%style{%t}{bright,blue}] %style{%C}{bright,yellow}: %msg%n%throwable" />
</Console>
<SMTP name="MailAppender"
subject="error log"
to="test#monitor.com"
from="monitor#monitor.com"
smtpHost="localhost"
smtpPort="25"
bufferSize="10"
ignoreExceptions="false"
smtpDebug="true"
>
<ThresholdFilter level="ERROR" onMatch="ACCEPT"/>
<PatternLayout>
<pattern>%d %p [%C] [%t] %m%n</pattern>
</PatternLayout>
</SMTP>
</Appenders>
<Loggers>
<Root level="all">
<AppenderRef ref="MailAppender" level="debug" />
</Root>
</Loggers>
</Configuration>
application.yml
spring:
mail:
host: localhost
port: 25
auth: false
properties:
mail:
transport:
protocol: smtp
smtp:
auth: false
host: localhost
port: 25
With this configuration, log4j2 isnt able to send any mail. Aparently, log4j2 takes configuration correctly on startup:
2020-09-24 18:31:32,780 restartedMain DEBUG createAppender(Configuration(/XXX/target/classes/log4j2.xml), name="MailAppender", to="test#monitor.com", cc="null", bcc="null", from="monitor#monitor.com", replyTo="null", subject="error log", smtpProtocol="null", smtpHost="localhost", smtpPort="25", smtpUsername="null", smtpPassword="c0323768f8f5dc63ac2d877d8e65de66", smtpDebug="true", bufferSize="10", PatternLayout(%d %p [%C] [%t] %m%n), ThresholdFilter(ERROR), ignoreExceptions="false")
but then it cannot send any email.
The funniest thing is that I can use spring mail to send mails (using JavaMailSender API) with same configuration, so I think that there is a problem with log4j2 configuration. Maybe log4j2 overrides mail configuration for any reason?
Could anyone help me?

Ok, I finally solved it. It have been working everytime, but it doesnt recognizes my application logger, so it only works with spring framework log names, for example: org.springframework.boot
I solved adding a new logger referencing my SMTPAppender:
<Logger name="com.myapp" level="all" additivity="false">
<AppenderRef ref="AsyncMailer" level="debug" />
</Logger>
And on a log.error("XXX"), it sends a mail with the log info.

Related

log4j2.xml configurations are not reflected in spring boot application

I am trying to setup a project with spring boot. Although I have defined all the dependencies for log4j, and added the log4j2.xml configuration file, the logs printed doesn't have the pattern defined in xml file. I checked the external libraries pulled by maven, and I see logback dependencies, which are not in my pom, I have even added exclusions. I am using spring boot 3 with Java 17. I have also tried excluding spring-boot-starter-logging. Nothing seems to work.
Here is the pom.xml file
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>3.0.0</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.geek8080</groupId>
<artifactId>db_service</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>war</packaging>
<name>db_service</name>
<description>DB Service for web app</description>
<properties>
<java.version>17</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
<exclusion>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
<exclusion>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<excludes>
<exclude>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
Here is the log4j2.xml file, I have placed this in src/main/resources,
<?xml version="1.0" encoding="UTF-8" ?>
<Configuration>
<Properties>
<Property name="logpath-location">/app/logs</Property>
<Property name="logfile-name">db_service.log</Property>
<Property name="archive">${logpath-location}/archive/dbservice</Property>
<Property name="interval">10</Property>
</Properties>
<Appenders>
<Console name="Console">
<PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss,SSS} [%t] %-5p %C.%M():%L %X - %m%n"/>
</Console>
<RollingFile name="RollingFileAppender" fileName="${logpath-location}/${logfile-name}"
filePattern="${archive}/${logfile-name}.%d{yyyy-MM-dd-HH}.gz">
<PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss,SSS} [%t] %-5p %c.%M ():%L %X - %m%n"/>
<Policies>
<TimeBasedTriggeringPolicy/>
</Policies>
</RollingFile>
</Appenders>
<Loggers>
<Logger name="com.indiantraditionalsnacks.db_service" level="DEBUG" additivity="false">
<AppenderRef ref="Console" level="INFO"/>
<AppenderRef ref="RollingFileAppender" level="DEBUG"/>
</Logger>
<Root level="INFO" includeLocation="true">
<AppenderRef ref="Console" level="INFO"/>
<AppenderRef ref="RollingFileAppender" level="DEBUG"/>
</Root>
</Loggers>
I have noticed that your "pom.xml" doesn't include org.springframework.boot:spring-boot-starter dependency, which is the core Spring Boot Starter.
Try adding it and add exclusion to "org.springframework.boot:spring-boot-starter-logging" from that dependency

Splunk HTTP Event Collector Log4j2 Spring boot unable to send data to Splunk cloud

I have followed all the tutorials on youtube and I have read all the available documentation on Splunk Cloud HEC setup in Log4j2 Spring boot.
Below is my log4j2-spring.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="info" name="LoggingTesting" packages="">
<Appenders>
<Console name="console" target="SYSTEM_OUT">
<PatternLayout
pattern="event:{%style{%d{ISO8601}} %highlight{%-5level }[%style{%t}{bright,blue}] %style{%C{10}}{bright,yellow}: %msg%n%throwable}" />
</Console>
<SplunkHttp
name="splunkhttp"
url="https://prd-p-vy06a.splunkcloud.com:8088/services/collector/event"
token="TOKEN********************"
host="wa-blog-service"
index="wa_blog_service_dev_index"
sourcetype="_json"
disableCertificateValidation="true" >
<PatternLayout
pattern="event:{%style{%d{ISO8601}} %highlight{%-5level }[%style{%t}{bright,blue}] %style{%C{10}}{bright,yellow}: %msg%n%throwable}" />
</SplunkHttp>
<File name="wa-blog-application" fileName="logs/waBlogApplication.log">
<PatternLayout pattern="%d{yyyy-mm-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
</File>
</Appenders>
<Loggers>
<!-- LOG everything at INFO level -->
<Root level="INFO">
<appender-ref ref="wa-blog-application"/>
<AppenderRef ref="console" />
<AppenderRef ref="splunkhttp" />
</Root>
</Loggers>
</Configuration>
Here is my POM.XML
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.0</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.worldastrologers</groupId>
<artifactId>blog</artifactId>
<version>1.0</version>
<name>wa-qa-service</name>
<description>WorldAstrologers Service.</description>
<properties>
<java.version>11</java.version>
</properties>
<repositories>
<repository>
<id>splunk-artifactory</id>
<name>Splunk Releases</name>
<url>https://splunk.jfrog.io/splunk/ext-releases-local</url>
</repository>
</repositories>
<dependencies>
<!-- https://mvnrepository.com/artifact/com.splunk.logging/splunk-library-javalogging -->
<dependency>
<groupId>com.splunk.logging</groupId>
<artifactId>splunk-library-javalogging</artifactId>
<version>1.8.0</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<artifactId>spring-boot-starter-logging</artifactId>
<groupId>org.springframework.boot</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
<version>2.7.4</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.postgresql/postgresql -->
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.13.3</version>
</dependency>
<dependency>
<groupId>jakarta.json</groupId>
<artifactId>jakarta.json-api</artifactId>
<version>2.1.0</version>
</dependency>
</dependencies>
<build>
<finalName>wa-blog-service</finalName>
</build>
</project>
What am I missing ???
I can't see data on my Splunk cloud account. I have checked all the indices.
I am able to see data in Splunk cloud when I send a request from Postman.
The URL in the log4j2-spring.xml file is incorrect.
The URL should be
url="https://prd-p-vy06a.splunkcloud.com:8088"

How can I get AWS Lambda and Spring Boot 2.1.5 to use Log4j2.xml and log to Cloudwatch?

In order to log to cloudwatch in a lambda, one must use this log4j2.xml:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration packages="com.amazonaws.services.lambda.runtime.log4j2">
<Appenders>
<Lambda name="Lambda">
<PatternLayout>
<pattern>%d{yyyy-MM-dd HH:mm:ss} %X{AWSRequestId} %-5p %c{1}:%L - %m%n</pattern>
</PatternLayout>
</Lambda>
</Appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="Lambda" />
</Root>
</Loggers>
</Configuration>
Also the following items must be in the pom.xml:
<parent>
<artifactId>spring-boot-starter-parent</artifactId>
<groupId>org.springframework.boot</groupId>
<version>2.1.5.RELEASE</version>
</parent>
...
<dependency>
<artifactId>aws-lambda-java-log4j2</artifactId>
<groupId>com.amazonaws</groupId>
<version>1.1.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.11.2</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.11.2</version>
</dependency>
Of course none of this works for me. It seems Spring uses Logback and I have tried shutting that off using these instructions. Nothing seems to work.
Any tips?
Got it! I read a post/rant where a guy noticed that AWS used a "5 year old jar" for the Cloudwatch/Lambda logging (can't find it now). So I found a Log4j2 jar that is compatible with 1.2 (log4j-1.2-api) and it worked. I had to shut off Logback in Spring and activate Log4j2. Below should take you all the way.
<dependency>
<artifactId>spring-boot-starter</artifactId>
<groupId>org.springframework.boot</groupId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
<dependency>
<artifactId>aws-lambda-java-log4j2</artifactId>
<groupId>com.amazonaws</groupId>
<version>1.1.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.11.2</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-1.2-api</artifactId>
<version>2.11.2</version>
</dependency>
...
PS. You might need to fix this as well: log4j2 ERROR StatusLogger Unrecognized conversion specifier

How to integrate SLF4J logging with Spring 4?

Couldn't find any good answer why slf4j logging not working with spring 4. Thanks to this post I've made particularly all work, but the author didn't mention that:
The mandatory logging dependency in Spring is the Jakarta Commons Logging API (JCL)
(http://docs.spring.io/spring/docs/4.1.x/spring-framework-reference/htmlsingle/).
It means that if we want slf4j to work WE MUST EXCLUDE commons-logging from our classpath (or pom.xml if using maven). While using maven spring-core automatically picks commons-logging. (I understood this when executed the command "mvn dependency:tree").
STEPS TO REPRODUCE:
Exclude commons-logging from spring-core
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>${spring.version}</version>
<exclusions>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
Add to the pom.xml next dependencies:
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.19</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.5</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.5</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.5</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<version>1.7.19</version>
</dependency>
Create log4j2.xml file like this (if you want log4j2 logging through slf4j).
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="CONSOLE" target="SYSTEM_OUT">
<PatternLayout
pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" />
</Console>
</Appenders>
<Loggers>
<Root level="error">
<AppenderRef ref="CONSOLE" />
</Root>
<logger level="info" additivity="false" name="org.springframework.security">
<AppenderRef ref="CONSOLE" />
</logger>
<logger level="info" additivity="false" name="org.springframework.web">
<AppenderRef ref="CONSOLE" />
</logger>
<logger level="warn" additivity="false" name="controllers">
<AppenderRef ref="CONSOLE" />
</logger>
</Loggers>
</Configuration>
write log messages in your classes:
private final Logger logger =LoggerFactory.getLogger(this.getClass());
logger.debug("This is a debug message");
logger.info("This is an info message");
You do not need to exclude commons-logging. If you use log4j-jcl instead of jcl-over-slf4j the Spring logging will be routed to log4j directly instead of going through slf4j.
Now, thanx to #rgoers my logging part of pom.xml looks like:
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.19</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.5</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.5</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.5</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-jcl</artifactId>
<version>2.5</version>
</dependency>

Why am I not getting logging from Spring Security?

I am trying to test my #PreAuthorize tags that are placed on the methods in my Spring Controller. I am not getting any logging information as to the decisions being made by Spring on whether to allow or disallow a call. Currently I am being allowed into a method and I want to debug why I am being allowed even though I don't have the role.
I am using Spring Framework 3.2.3, Spring Security 3.1.4, Logback 1.0.13, Logback Spring 0.1.1.
Here is my logback file:
...
<appender name="MAIN" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>/logs/main_log.txt</file>
<append>true</append>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>TRACE</level>
</filter>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} | %p | %c %M | %m%n</pattern>
</encoder>
<rollingPolicy>
<fileNamePattern>/logs/main_log.txt.%d</fileNamePattern>
</rollingPolicy>
</appender>
<logger name="org.springframework" level="TRACE"/>
<logger name="my.package" level="TRACE"/>
<root level="WARN">
<appender-ref="MAIN"/>
</root>
I have the listener set up properly in my web.xml file. I know this because I am getting log messages from the package my.package as well as WARN messages from 3rd party software.
Any ideas as to how I can see the decisions being made by the #PreAuthorize annotation?
Your configuration seeems fine, but Spring unfortunately uses commons logging. Follow the official documentation on how to use SLF4J (+ log4j or logback).
http://docs.spring.io/spring/docs/3.2.x/spring-framework-reference/html/overview.html#overview-not-using-commons-logging
Just don't add commons-logging jar or if you use Maven exlude it like shown in the documentation. Then you need a bridge between commons-logging and SLF4J (jcl-over-slf4j.jar).
Cut out from my (working) configuration:
<logger name="org.springframework" level="DEBUG" additivity="false">
<appender-ref ref="STDOUT" />
<appender-ref ref="FILE" />
</logger>
pom.xml:
<!-- Logging -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.5</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<version>1.7.5</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.0.13</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-core</artifactId>
<version>1.0.13</version>
<scope>runtime</scope>
</dependency>
<!-- Spring -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>${spring-version}</version>
<exclusions>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
In order to use Pre-PostAuthorize annotations, it's necessary to enable their use in spring configuration, as they are off by default.
This is how to enable these annotations when using the global-method-security XML element (see here for further details):
<global-method-security pre-post-annotations="enabled"/>

Resources