Configuring spring boot to use log4j2 and the ECS was simple enough. All we needed to do was include the following dependencies in our pom.xml:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
<dependency>
<groupId>co.elastic.logging</groupId>
<artifactId>log4j2-ecs-layout</artifactId>
<version>1.0.1</version>
</dependency>
Then in the log4j2.xml file we just needed to include something like this:
<Console name="prod" target="SYSTEM_OUT">
<EcsLayout serviceName="${service-name}" stackTraceAsArray="true" includeOrigin="true" includeMarkers="true" />
</Console>
However, when we configure spring boot to log the application access logs in application.yml we do this:
server:
tomcat:
accesslog:
enabled: true
encoding: UTF-8
directory: /dev
prefix: stdout
buffered: false
suffix:
file-date-format:
pattern: "ACCESS_LOG: %h %l %u \"%{yyyy-MM-dd HH:mm:ss.SSS}t\" \"%r\" status=%s size=%b \"user-agent=%{User-Agent}i\""
The access logs are not logged as json with the EcsLayout.
How can I configure a spring boot application to log tomcat access logs as json to the console using the elastic common schema (json) format?
Related
I am trying to add prometheus into my spring boot project. I am using spring boot actuator to expose the metrics endpoint . Did everything following tutorials but I keep on getting a 404 error. I also tried every single solution given in:
Unable to access Spring Boot Actuator "/actuator" endpoint
and
Spring Boot 2 - Actuator Metrics Endpoint not working
My dependecies:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-core</artifactId>
<version>1.7.0</version>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
<version>1.7.0</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-rest-hal-explorer</artifactId>
<version>3.5.1</version>
</dependency>
context.xml (external properties):
<Parameter name="management.endpoints.web.exposure.include" value="*"/>
<Parameter name="management.security.enabled" value="false"/>
<Parameter name="management.endpoints.beans.enabled" value="false"/>
<Parameter name="endpoints.actuator.enabled" value="true"/>
<Parameter name="management.endpoints.web.base-path" value="/actuator"/>
It looks like you missing configuration in your application.yml file (If you use the application.properties file try to change code format):
management:
endpoint:
health:
show-details: always
endpoints:
web:
exposure:
include: '*'
metrics:
export:
prometheus:
enabled: false
I hope that's help you
I have a Spring Boot application which runs a Spring Batch job at regular intervals from a Quartz Scheduler. Every time the job runs information about it is always logged as shown below:
Sep 16, 2019 5:01:56 PM org.springframework.batch.core.launch.support.SimpleJobLauncher$1 run
INFO: Job: [SimpleJob: [name=testJob]] launched with the following parameters: [{jobRunTime=1568649716469}]
Sep 16, 2019 5:01:56 PM org.springframework.batch.core.job.SimpleStepHandler handleStep
INFO: Executing step: [publishRequests]
Sep 16, 2019 5:01:56 PM org.springframework.batch.core.launch.support.SimpleJobLauncher$1 run
INFO: Job: [SimpleJob: [name=testJob]] completed with the following parameters: [{jobRunTime=1568649716469}] and the following status: [COMPLETED]
How can I prevent this from being logged? I am using slf4j with log4j2 for logging. I tried adding this line to my logging properties file so only error messages are logged but this did not work:
logging.level.org.springframework.batch=ERROR
I am using Spring Boot 2.1.2 with Spring Batch 4.1.1.
Here are the dependencies in my log file:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<version>${spring-boot.version}</version>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
<version>${spring-boot.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-jcl</artifactId>
<version>${slf4j.version}</version>
<exclusions>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
I use the following properties for my logging in a log file called logging.properties:
handlers= java.util.logging.FileHandler
.level= INFO
java.util.logging.FileHandler.pattern = myApp.log
java.util.logging.FileHandler.limit = 10000000
java.util.logging.FileHandler.count = 20
java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
java.util.logging.ConsoleHandler.level = SEVERE
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
java.util.logging.SimpleFormatter.format=[%1$tc] %4$s %2$s: %5$s %n
logging.level.org.springframework.batch=ERROR
By default, it uses logback.
Maybe your configuration doesn't consider your log4j2.
To fix it: Create a logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<logger name="org.springframework.batch" level="ERROR"/>
</configuration>
I managed to get this working by specifying the property in my logging.properties file as
org.springframework.batch.level=ERROR
You need to adjust logging level like that:
logging.level.root=ERROR
Written simple POC to prove and test Spring Boot and log4j2 compatibility. Once successful I will move it to real application.
Please refer my POC source code:
https://github.com/Dennyss/SpringBootLog4j2POC
I know/read about Spring version and log4j2 compatibility from:
How to set up Spring Boot and log4j2 properly?
Found and tried recommendations described here:
Spring-Boot logging with log4j2?
But still not working. The problem is that both application logs and Spring logs are printing to console only.
Please refer maven dependencies below (from POC):
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.6.2</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.6.2</version>
</dependency>
</dependencies>
If I don't exclude Spring's logback and don't add boot-starter-log4j2 then application logs are printing to application file but Spring logs are not printing at all. I feel the problem somewhere with dependencies. Appreciate any help.
According to the Spring Boot Logging documentation, the location of the logging configuration file can be specified using the logging.config property. I noticed that your start.sh script is passing -Dlog4j.configurationFile. Normally, this would be sufficient for direct Log4J 2 integration, but Spring Boot uses logging.config.
After switching to this, it writes to the log files:
java -Dlogging.config=/share/LogPOC/log4j2.xml -cp poc.jar:libs/* com.poc.logger.Application
The application is deployed in pivotal web services and when I hit the application with the link https://webstore.cfapps.io/, the login.jsp page is not rendered.
The same application runs successfully in the local server. I tried for searching the same issue but none of them solve the problem. I used both STS deployment and also tried cf command line for deploying separately but couldn't get the result.
I am giving you the details.
The error shown.
Whitelabel Error Page
This application has no explicit mapping for /error, so you are seeing this as a fallback.
Sat Jan 14 04:21:40 UTC 2017
There was an unexpected error (type=Not Found, status=404).
/WEB-INF/jsp/login.jsp
My pom file is
http://maven.apache.org/xsd/maven-4.0.0.xsd">
4.0.0
ecommerce.com
webstore
0.0.1-SNAPSHOT
jar
ecommerce-webstore
Demo project for Spring Boot
org.springframework.boot
spring-boot-starter-parent
1.4.2.RELEASE
UTF-8
UTF-8
1.8
org.springframework.boot</groupId> spring-boot-starter-security</artifactId>
</dependency> -->
org.springframework.boot
spring-boot-starter-web
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.apache.tomcat.embed</groupId>
<artifactId>tomcat-embed-jasper</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
</dependency>
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
org.springframework.boot
spring-boot-maven-plugin
3.MY application.properties file is
spring.mvc.view.prefix:/WEB-INF/jsp/
spring.mvc.view.suffix: .jsp
spring.datasource.url= jdbc:mysql://localhost:3306/ecommercestore
spring.datasource.username=root
spring.datasource.password=root
spring.jpa.hibernate.ddl-auto=update
3.The manifest.yml generated after deployment is
applications:
name: webstore
memory: 1024M
host: webstore
domain: cfapps.io
buildpack: standard buildpack provided in git hub. I am not writing link
services:
cleardb
The result after giving the following command from cmd is
cf push webstore -p target/webstore-0.0.1-SNAPSHOT.jar --no-start
......................................................................
C:\Users\santosh dahal\Desktop\excellerant\ecommerce-webstore>cf push webstore -p target/webstore-0.0.1-SNAPSHOT.jar --no-start
Creating app webstore in org santosh-org / space Myspace as santoshdahal2072#gmail.com...
OK
Using route webstore.cfapps.io
Binding webstore.cfapps.io to webstore...
OK
Uploading webstore...
Uploading app files from: C:\Users\SANTOS~1\AppData\Local\Temp\unzipped-app904639435
Uploading 478.9K, 140 files
Done uploading
OK
6. I went to myapp in pivotal and started the application after binding the database cleardb to the same instant and application, the application runs successfully with the Running Status
7. the folder arrangement is as follows:
I have kept jsp pages in
src/main/webapp/WEB-INF/jsp/login.jsp
While the application.properties is in src/main/resources.
I will provide more details if needed. the github link for the code is
here
you are packaging it as a jar not as a war . In that case put your jsp files into static folder under resources or follow the below the link to understand better on the packaging aspect. But I feel it is better to package it as a war
Package a spring boot application including JSPs and static resources.
Also please check the jar generated before pushing the app to PWS whether jsp is avaibale at the requested path.
How to load Custom Log4j.properties file in Spring boot
My code in application.properties is here
logging.file=E:/Apps_Tek/apps-webservices-log/apps-webservices.log
logging.level.*=INFO
logging.config=log4j.properties
My code in log4j.properties is here
log4j.rootLogger=INFO,ConsoleAppender,FileAppender
log4j.appender.ConsoleAppender=org.apache.log4j.ConsoleAppender
log4j.appender.ConsoleAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.ConsoleAppender.layout.ConversionPattern=%-7p %d [%t] %c [%X{userName}] [%X{accessToken}] - %m%n
log4j.appender.FileAppender=org.apache.log4j.RollingFileAppender
log4j.appender.FileAppender.File=E:/Apps_Tek/apps-webservices-log/apps-webservices.log
log4j.appender.FileAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.FileAppender.layout.ConversionPattern=%-7p %d [%t] %c [%X{userName}] [%X{accessToken}] - %m%n
But i am not getting any expected output i.e., spring boot is not loading log4j.properties file. Spring boot is having its own default logging.
log4j.properties file is in src/main/resources
My question is how to map log4j.properties file with logging.config property in application.properties if it is in src/main/resources.
Please suggest all the required changes.
Thanks for any help in advance.
If you want spring boot to use log4j instead of its own default logging (logback) then you have to exclude default logging and include the log4j dependency for spring boot in your pom.xml:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j</artifactId>
</dependency>
that way is going to look for your log4j.properties file located in src/main/resources.
Also if you wish to use properties defined in you application.properties inside the log4j.properties file
for example, you want to have log.file.path defined in your application.properties and use it on log4j.properties
Then you have to define filtering inside your pom.xml:
<filters>
<filter>src/main/resources/application.properties</filter>
</filters>
<resources>
<resource>
<directory>src/main/resources</directory>
<includes>
<include>log4j.properties</include>
</includes>
<filtering>true</filtering>
</resource>
</resources>
That way you can have:
log4j.appender.file.File=${log.file.path}/${project.artifactId}.log
in your log4j.properties file
To exclude default logging and include the log4j dependency for spring boot in your pom.xml:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
if you wish to use log4j properties defined in you application.properties inside the log4j.properties file need to add below property in application.properties file.
logging.config = src/main/resources/log4j2.properties
Then spring boot will read the properties from log4j2.properties file
In log4j2.properties file add the below properties
name=PropertiesConfig
appenders = console, file
appender.console.type = Console
appender.console.name = ConsoleAppender
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = %d{MM:dd HH:mm:ss.SSS} [%t] [%level] [%logger{36}] - %msg%n
appender.file.type = File
appender.file.name = FileAppender
appender.file.fileName=/home/ubuntu/application.log
appender.file.layout.type=PatternLayout
appender.file.layout.pattern= %d{MM:dd HH:mm:ss.SSS} [%t] [%level] [%logger{36}] - %msg%n
loggers=file
logger.file.name=com.project
logger.file.level = debug
logger.file.appenderRefs = file
logger.file.appenderRef.file.ref = FileAppender
rootLogger.level = debug
rootLogger.appenderRefs = stdout
rootLogger.appenderRef.stdout.ref = ConsoleAppender
Is there any other work around which doesnot require to specify
'logging.config = src/main/resources/log4j2.properties' inside application.properties file..?
Note : Spring boot version i am using is 2.1.3.RELEASE
Reference : https://docs.spring.io/spring-boot/docs/current/reference/html/howto-logging.html
My guess is that your pom.xml file is not set up to include the correct log4j dependency. If you do that it should automatically work. See their example at spring-boot-sample-actuator-log4j. Their project includes the artifact spring-boot-starter-log4j which should then allow your log4j.properties to be picked up from the current src/main/resources location and you should no longer need the logging.* entries in your properties. It may also be possible that you need to exclude their spring-boot-starter-logging as I see they are doing that as well. If this does not seem to resolve it please post your POM file in a location I can see it. Please let me know if this works for you.
In case someone faces the same problem, I did not have the spring-boot-starter dependency so I had to add the to the spring-boot-starter-web dependency, this is how it looked:
<!-- Spring Boot Starter Web -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<!-- Excluding logback dependencies to use l4j2 -->
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
Then you add the Log4j2 Dependency:
<!-- Add Log4j2 Dependency -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
Your log4j2.properties file will be used for logging configurations from now on.