While Spring 4 Rest application loading , i am getting the logger information ( log file name / path / archival days) from database and passing this value to Logj plugin , so that i can retrieve the value from log4j.xml.
My application log file not created and plugin not called ! not getting any error in console as well.
What should i do , for Logj plugin load and log file creation ?
Spring:4.3.15.RELEASE
log4j:2.4.1
java:1.8
SpringWebInitializer.java
public class SpringWebInitializer implements WebApplicationInitializer{
#Override
public void onStartup(ServletContext ctx) throws ServletException {
AnnotationConfigWebApplicationContext webCtx = new AnnotationConfigWebApplicationContext();
webCtx.register(WebConfiguration.class);
webCtx.setServletContext(ctx);
ServletRegistration.Dynamic servlet = ctx.addServlet("dispatcher", new DispatcherServlet(webCtx));
servlet.setLoadOnStartup(1);
servlet.addMapping("/");
//Add Listener.
ctx.addListener(new MyAppContextListner());
}
}
WebConfiguration.java
#EnableWebMvc
#Configuration
#ComponentScan(basePackages ="org.vasa.ws.myapp")
public class WebConfiguration {
}
MyAppContextListner.java
public class MyAppContextListner implements ServletContextListener{
private static final Logger LOGGER = LogManager.getLogger(BenefitsCompositeContextListner.class);
private WebApplicationContext webApplicationContext = null;
#Override
public void contextInitialized(ServletContextEvent event) {
// Database connectivity and get the logger information from db.
}
#Override
public void contextDestroyed(ServletContextEvent event) {
// TODO Auto-generated method stub
}
}
Given below Log4j Plugin i am using
MyLog4JConfigDatabaseLookup.java
#Plugin(name = "MyAppdbLookup", category = StrLookup.CATEGORY)
public class MyLog4JConfigDatabaseLookup extends AbstractLookup {
public String lookup(final LogEvent event, final String key) {
System.out.println("Lookup......");
}
}
log4j2.xml
<Configuration packages="org.vasa.ws.myapp">
<Properties>
<Property name="app-name">MyAppdbLookup</Property>
<Property name="file-level">${MyAppdbLookup:logLevel}</Property>
<Property name="log-file">${MyAppdbLookup:logFile}</Property>
<Property name="log-file-level">${MyAppdbLookup:logLevel}</Property>
<Property name="log-path">${MyAppdbLookup:logPath}</Property>
<Property name="archive-days">${MyAppdbLookup:archive-days}</Property>
</Properties>
<Appenders>
<Routing name="route-log">
<Routes pattern="framework">
<Route key="benefitCompositeWS">
<RollingFile name="message-log" fileName="${log-path}/myapp.log"
filePattern="${log-path}/$${date:yyyy-MM}/myapp.%d{MM-dd-yyyy}-%i.log.gz" append="true">
<PatternLayout
pattern="%d{MM/dd/yyyy HH:mm:ss.SSS z} [%t] %-5level %logger{36} - %msg%n" />
<Policies>
<TimeBasedTriggeringPolicy />
<SizeBasedTriggeringPolicy size="150 MB" />
</Policies>
<DefaultRolloverStrategy max="1000">
<Delete basePath="${log-path}" maxDepth="2">
<IfFileName glob="*/myapp*.log.gz" />
<IfLastModified age="${archive-days}" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile >
</Route>
</Routes>
</Routing>
<Loggers>
<Root level="${file-level}" additivity="false">
<AppenderRef ref="route-log" />
<AppenderRef ref="STDOUT" />
</Root>
</Loggers>
</Configuration>
I have changed my plugin name as lower case and now it's working fine as expected:
#Plugin(name="myappdblookup")
Related
I am working on a task that I want to mask sensitive data using Log4j2 LogEventPatternConverter Class.
#Plugin(name="SensitiveDataLog", category = "Converter")
#ConverterKeys({"sense"})
public class SensitiveDataLog extends LogEventPatternConverter {
#Value("${ssn}")
private String ssn;
public SensitiveDataLog(String name, String style) {
super(name, style);
}
public static SensitiveDataLog newInstance(String[] options) {
return new SensitiveDataLog("sense","sense");
}
#Override
public void format(LogEvent logEvent, StringBuilder outputMsg) {
String message = logEvent.getMessage().getFormattedMessage();
Matcher matcher = SSN_PATTERN.matcher(message);
if (matcher.find()) {
String maskedMessage = matcher.replaceAll("***-**-****");
outputMsg.append(maskedMessage);
} else {
outputMsg.append(message);
}
}
}
Suppose I want to keep pattern in application.properties, But problem here is we cannot load property value ssn. Always its null.
Here is my log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="info" monitorInterval="30"
packages="com.virtusa.xlab.fw.logging.component"
xmlns="http://logging.apache.org/log4j/2.0/config">
<Properties>
<Property name="basePath">logs/log4j2</Property>
</Properties>
<Appenders>
<!-- File Appender -->
<RollingFile name="FILE"
fileName="${basePath}/logfile.log" filePattern="${basePath}/logfile.%d{yyyy-MM-dd}-%i.log" append="true">
<PatternLayout
pattern="%-5p | %d{yyyy-MM-dd HH:mm:ss} | [%t] %C{2} (%F:%L) - %sense%n" />
<Policies>
<SizeBasedTriggeringPolicy size="1 KB" />
</Policies>
<DefaultRolloverStrategy max="4" />
</RollingFile>
<!-- Console Appender -->
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout
pattern="%-5p | %d{yyyy-MM-dd HH:mm:ss} | [%t] %C{2} (%F:%L) - %sense%n" />
</Console>
</Appenders>
<Loggers>
<Logger name="com.virtusa.xlab.fw" level="info" />
<Root level="info">
<AppenderRef ref="STDOUT" />
<AppenderRef ref="FILE" />
</Root>
</Loggers>
</Configuration>
Can anyone help me out here?
Thanks.
The problem is that SensitiveDataLog is created via static method newInstance(). Obviously, field ssn is not initialized at that moment. What you can do is to init the field later, e.g. when refreshing Spring context.
Here is my snippet:
private static XmlMaskPatternConverter INSTANCE = new XmlMaskPatternConverter();
public XmlMaskPatternConverter() {
super(NAME, NAME);
}
public static XmlMaskPatternConverter newInstance() {
return INSTANCE;
}
Now you can call static method getInstance() somewhere in your Spring Configuration (I do it in #Bean method) and set the ssn value there. Ofc, you need to create a setter for this field.
P.S. Hope it helps. I faced this problem too, so decided to leave my solution here. My first post on SO btw)
Good day,
I have a Spring batch, that using org.springframework.batch.item.support.CompositeItemWriter to write data into db.
The following is the writer code in my xml:
<bean id="iReturnCompositeTableWriter" class="org.springframework.batch.item.support.CompositeItemWriter">
<property name="delegates">
<list>
<ref bean="iReturnTableWriter5" />
</list>
</property>
</bean>
<bean id="iReturnTableWriter5" class="com.batch.job.writer.IReturnTableWriter">
<property name="assertUpdates" value="false" />
<property name="itemSqlParameterSourceProvider">
<!-- Our Mapper for table writer -->
<bean class="com.batch.job.writer.mapper.IReturnWriterTableMapper" />
</property>
<!-- Put in your INSERT, UPDATE, DELETE SQL here -->
<property name="sql"
value="UPDATE ADM.TRX
SET STATUS = 'FAIL'
WHERE INTERNALREFERENCENO = :theirTraceNum" />
<property name="dataSource" ref="dataSource" />
</bean>
And the following is code of IReturnTableWriter class:
public class IReturnTableWriter extends JdbcBatchItemWriter< TrxVO >
implements StepExecutionListener, ItemWriteListener< TrxVO > {
public void afterWrite(List<? extends BulkPaymentVO> arg0) {
}
public void beforeWrite(List<? extends BulkPaymentVO> arg0) {
}
public void onWriteError(Exception arg0, List<? extends BulkPaymentVO> arg1) {
}
public ExitStatus afterStep(StepExecution arg0) {
return null;
}
public void beforeStep(StepExecution arg0) {
}
}
And the following is the code of IReturnWriterTableMapper class:
public class IReturnWriterTableMapper implements
ItemSqlParameterSourceProvider< IbgReturnVO > {
public SqlParameterSource createSqlParameterSource(TrxVO trxVO) {
return new BeanPropertySqlParameterSource(trxVO);
}
}
The batch log show this batch run successful without any error. The successful count is match with my input file row count. In the log, I also saw that its being read by the reader and store into trxVO object.
However, there is 1 of the record did not updated to database in production, even the success count is correct. I grab the input file and run in SIT, UAT and its working fine. From the log, I cant see it log about the update query.
I actually already enable log level to trace for org.springframework.batch and org.springframework.jdbc in logback.xml:
<logger name="org.springframework.batch">
<level value="TRACE" />
</logger>
<logger name="org.springframework.jdbc">
<level value="TRACE" />
</logger>
And in log, I can saw its already take effect:
15:44:19,173 |-INFO in ch.qos.logback.classic.joran.action.LevelAction - org.springframework.batch level set to TRACE
15:44:19,173 |-INFO in ch.qos.logback.classic.joran.action.LevelAction - org.springframework.jdbc level set to TRACE
I would like to ask the way on how to enable more log for the writer, so that I can troubleshoot what is happening.
Or, if you have more useful idea, or suggestion, please advise.
I would like to ask the way on how to enable more log for the writer
The JdbcBatchItemWriter uses spring-jdbc APIs behind the scene, so you can set the log level to debug for org.springframework.jdbc package.
I would also enable debug logs for org.springframework.batch as well.
I have multi-tenant application followed by microservices created in spring boot version 1.4.3, i have generated tenant wise logs in different folders inside logs folder of tomcat.
A Problem which i am facing is after some time/hours/days/weeks my logs will stop but application work in backgroud. I am trying to find root cause of the same but failed.
In java code some classes using Slf4j and some classes using log4j to print logs.
As mentined i am using SiftingAppender and changing logFolder variable from Filter/Intercepter based on the tenant.
previously i was doing MDC.clear() to remove logFolder from MDC, then i have moved to MDC.remove("logFolder") but that also not worked
logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml" />
<property name="LOG_FILE" value="${LOG_FILE:-${LOG_PATH:-${LOG_TEMP:-${java.io.tmpdir:-/tmp}}/}}" />
<include resource="org/springframework/boot/logging/logback/console-appender.xml" />
<!-- This is MDC value -->
<!-- We will assign a value to 'logFileName' via Java code -->
<appender name="FILE-THREAD" class="ch.qos.logback.classic.sift.SiftingAppender">
<!-- This is MDC value -->
<!-- We will assign a value to 'logFileName' via Java code -->
<discriminator>
<key>logFolder</key>
<defaultValue>main</defaultValue>
</discriminator>
<sift>
<appender name="FILE-${logFileFolder}" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_FILE}/${logFolder}/enquiryengine.log</file>
<encoder>
<pattern>${FILE_LOG_PATTERN}</pattern>
</encoder>
<!-- <file>${LOG_FILE}</file> -->
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_FILE}/${logFolder}/enquiryengine.log.%d{yyyy-MM-dd}.%i.gz</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
<maxHistory>15</maxHistory>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
</sift>
</appender>
<springProfile name="development">
<root level="INFO">
<appender-ref ref="CONSOLE" />
<appender-ref ref="FILE-THREAD" />
</root>
</springProfile>
<springProfile name="production">
<root level="INFO">
<appender-ref ref="FILE-THREAD" />
</root>
</springProfile>
<springProfile name="staging">
<root level="INFO">
<appender-ref ref="FILE-THREAD" />
</root>
</springProfile>
<springProfile name="testing3">
<root level="INFO">
<appender-ref ref="FILE-THREAD" />
</root>
</springProfile>
<springProfile name="testing">
<root level="INFO">
<appender-ref ref="FILE-THREAD" />
</root>
</springProfile>
<springProfile name="testing2">
<root level="INFO">
<appender-ref ref="FILE-THREAD" />
</root>
</springProfile>
<jmxConfigurator />
</configuration>
public class MyClass extends HandlerInterceptorAdapter {
#Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler)
throws Exception {
MDC.put(UtilConstant.LOG_FOLDER_NAME, "dynamicTenantFolderName");
return true;
}
#Override
public void afterConcurrentHandlingStarted(HttpServletRequest request, HttpServletResponse response, Object handler)
throws Exception {
MDC.remove(UtilConstant.LOG_FOLDER_NAME);
}
#Override
public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex)
throws Exception {
MDC.remove(UtilConstant.LOG_FOLDER_NAME);
}
}
public class TenantRequestFilter implements Filter {
#Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
throws IOException, ServletException {
MDC.remove(UtilConstant.LOG_FOLDER_NAME);
MDC.put(UtilConstant.LOG_FOLDER_NAME, "dynamicLogFolderName");
// Goes to default servlet
chain.doFilter(requestWrapper, response);
}
#Override
public void init(FilterConfig filterConfig) throws ServletException {
// TODO Auto-generated method stub
}
#Override
public void destroy() {
MDC.remove(UtilConstant.LOG_FOLDER_NAME);
}
}
As logs are not working so i will not get any stack trace, Expected Result is logs generation should not stop.
The default Timeout for Siftappender is 30 Minutes,
If no logs are written in 30mins then the siftappender gets timeout and no further logs on that MDC will be processed.
To overcome this,
1. You can increase the time limit as per the application's requirement
2. Have a scheduler to print log after particular interval, so that it never gets timeout
Refer Document: http://logback.qos.ch/manual/appenders.html
I am running spring boot as War on tomcat with logback to console and file.
as long as i run as Java application it is fine i can see logs in console and file.
but i dont see logs printed to file when run on server.
I tried setting logger manager also, didnt work. was wondering to know if some one has faced similar issue.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml" />
<include resource="org/springframework/boot/logging/logback/file-appender.xml" />
<property name="LOG_FILE"
value="${LOG_FILE:-${LOG_PATH:-${LOG_TEMP:-${java.io.tmpdir:-/tmp}}/}app.log}"/>
<property name="LOG_FILE_MAX_SIZE" value="10MB" />
<property name="LOG_TOTAL_SIZE_CAP" value="100MB" />
<property name="LOG_FILE_MAX_HISTORY" value="20" />
<root level="INFO">
<appender-ref ref="FILE" />
</root>
</configuration>
Verify that you have the following dependency :
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
or even if you have spring-boot-starter-web dependency added, logging should work.
and have the following in yml or properties file :
logging.path=logs
logging.file=${logging.path}/log.log
logging.pattern.file=%d{dd-MM-yyyy HH:mm:ss.SSS} [%thread] %-5level %logger{36}.%M - %msg%n
and you can also have a logback.xml and use the spring default base.xml in that so that all default spring configurations apply for your logging as well :
logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<logger name="org.springframework.web" level="DEBUG"/>
</configuration>
Here is my logback-spring.xml which i have
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml" />
<include resource="org/springframework/boot/logging/logback/file-appender.xml" />
<property name="LOG_FILE"
value="${LOG_FILE:-${LOG_PATH:-${LOG_TEMP:-${java.io.tmpdir:-/tmp}}/}app.log}"/>
<property name="LOG_FILE_MAX_SIZE" value="10MB" />
<property name="LOG_TOTAL_SIZE_CAP" value="100MB" />
<property name="LOG_FILE_MAX_HISTORY" value="20" />
<root level="INFO">
<appender-ref ref="FILE" />
</root>
</configuration>
I could fix it by writing the logs to a different folder looks application doesn't have write access the path, however i need to make some changes to springboot main class for loading application props based profile, please find the class below. not sure if others had to the same.
anyway i am glad it is working finally:)
public class Application extends SpringBootServletInitializer{
public String PROFILE = null;
private static String CONFIG_LOCATION = null;
#Override
public void onStartup(ServletContext servletContext) throws ServletException {
//Grab the active profile from the servlet conext
PROFILE = servletContext.getInitParameter("spring.profiles.active");
CONFIG_LOCATION = servletContext.getInitParameter("spring.config.path");
super.onStartup(servletContext);
}
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
//...and pass it to the boot application
application.application().setAdditionalProfiles(PROFILE);
return application.sources(Application.class).properties(getProperties());
}
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
//For Loading config from server
static Properties getProperties() {
Properties props = new Properties();
props.put("spring.config.location", CONFIG_LOCATION);
return props;
}
}
web.xml
<context-param>
<param-name>spring.profiles.active</param-name>
<param-value>dev</param-value>
</context-param>
<context-param>
<param-name>spring.config.path</param-name>
<param-value>classpath:app.config/</param-value>
</context-param>
My applicationContext is as follows
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="..">
<mvc:annotation-driven/>
<task:annotation-driven/>
<mvc:resources mapping="/resources/**" location="/resources/"/>
<aop:aspectj-autoproxy />
<context:component-scan base-package="com.abc">
<context:include-filter type="aspectj" expression="com.abc.aspects.LogControllerAspect"/>
</context:component-scan>
<context:annotation-config />
</beans>
Have 2 Aspect Java classes, LogControllerAspect (for logging all the calls to Spring Controllers) and LogDAOAspect (for logging all the calls to DB).
#Aspect
#Service
public class LogDAOAspect {
private Logger logger = LoggerFactory.getLogger(this.getClass());
#Around("execution(* com.*.*DAOImpl.*(..))")
public Object logAround(ProceedingJoinPoint joinPoint) throws Throwable {
String methodId = joinPoint.getTarget().getClass().getSimpleName()+" : "+joinPoint.getSignature().getName() + " : " + ((joinPoint.getArgs()==null||joinPoint.getArgs().length<1)?"":(Arrays.toString(joinPoint.getArgs())));
Object returnVal = null;
StopWatch sw = new StopWatch(methodId);
try {
sw.start();
returnVal= joinPoint.proceed(joinPoint.getArgs());
sw.stop();
} catch (Throwable e) {
logger.error(methodId+"\n"+e);
throw e;
}
logger.debug(methodId + ":" +sw.getTotalTimeMillis());
return returnVal;
}
}
#Aspect
public class LogControllerAspect {
private Logger logger = LoggerFactory.getLogger(this.getClass());
#Around("execution(* com.*.*Controller.*(..))")
public Object logAround(ProceedingJoinPoint joinPoint) throws Throwable {
String methodId = joinPoint.getTarget().getClass().getSimpleName()+" : "+joinPoint.getSignature().getName() + " : " + ((joinPoint.getArgs()==null||joinPoint.getArgs().length<1)?"":(Arrays.toString(joinPoint.getArgs())));
Object returnVal = null;
StopWatch sw = new StopWatch(methodId);
try {
sw.start();
returnVal= joinPoint.proceed(joinPoint.getArgs());
sw.stop();
} catch (Throwable e) {
logger.error(methodId+"\n"+e);
throw e;
}
logger.debug(methodId + ":" +sw.getTotalTimeMillis());
return returnVal;
}
}
LogDAOAspect is fine, but LogControllerAspect is logging twice (logAround method is executing twice) when I request some page. I can understand that that aspect is getting proxied twice, but am not sure how to avoid this. Help appreciated.
It was a silly mistake. It was not even a spring related issue!
I had setup log4j as follows !!!???!!!
<appender name="AppAppender" class="org.apache.log4j.DailyRollingFileAppender">
<param name="DatePattern" value=".yyyy-MM-dd.HH"/>
<param name="File" value="logs/logfile.log"/>
<param name="Append" value="true"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-5p : %d{ISO8601} : %m%n"/>
</layout>
</appender>
<logger name="com.abc.aspects.LogControllerAspect">
<priority value="debug"></priority>
<appender-ref ref="AppAppender"/>
</logger>
<root>
<priority value="error" />
<appender-ref ref="AppAppender" />
</root>
Thank you #erencan. You question really helped me look closely what is really happening inside that method.
Changed is at follows, and it work fine. Should log4j to this question's tags!
<logger name="com.abc.aspects.LogControllerAspect" additivity="false">
<priority value="debug"></priority>
<appender-ref ref="AppAppender"/>
</logger>