Spring 4, JPA, Turn off console debug messages - spring

I have a basic Spring 4 JPA application. I use all Java configurations (no XML at all). I want to turn off the console debug messages. I have seen many questions on this and tried the solutions but still I see all the messages.
The console messages look like this:
14:58:29.301 [main] DEBUG o.h.loader.entity.plan.EntityLoader....
14:58:29.328 [main] INFO o.h.tool.hbm2ddl.SchemaUpdate....
....
14:58:29.905 [main] DEBUG o.h.h.i.ast.QueryTranslatorImpl - --- HQL AST ---
\-[QUERY] Node: 'query'
\-[SELECT_FROM] Node: 'SELECT_FROM'
+-[FROM] Node: 'from'
| \-[RANGE] Node: 'RANGE'
| +-[DOT] Node: '.'
| | +-[IDENT] Node: 'hello'
| | \-[IDENT] Node: 'Customer'
| \-[ALIAS] Node: 'generatedAlias0'
\-[SELECT] Node: 'select'
\-[IDENT] Node: 'generatedAlias0'
....
Hundreds of lines more....
I tried to set show_sql both in the HibernateJpaVendorAdapter and in the LocalContainerEntityManagerFactoryBean as shown below:
#Bean
public EntityManagerFactory entityManagerFactory() throws SQLException {
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
vendorAdapter.setGenerateDdl(true);
vendorAdapter.setShowSql(false);
LocalContainerEntityManagerFactoryBean factory = new
LocalContainerEntityManagerFactoryBean();
factory.setJpaVendorAdapter(vendorAdapter);
factory.setPackagesToScan("hello");
factory.setDataSource(dataSource());
Properties jpaProperties = new Properties();
jpaProperties.put( "hibernate.dialect", "org.hibernate.dialect.MySQLDialect" );
jpaProperties.put( "hibernate.show_sql", false );
jpaProperties.put( "show_sql", false );
jpaProperties.put( "hibernate.generate_statistics", false );
factory.setJpaProperties(jpaProperties);
factory.afterPropertiesSet();
return factory.getObject();
}
Thank you for any ideas on this!
-- Edit -- more info
What I do is create a Spring Starter Project with Spring Tools Suite and just select JPA. I then add MySQL to my pom.xml.
As a test I have a basic Customer and CustomerRepository class and the JPA configuration I noted above.
My Application class:
#Configuration
#EnableAutoConfiguration
public class Application {
public static void main(String[] args) {
AbstractApplicationContext context = new
AnnotationConfigApplicationContext(JPAConfiguration.class);
CustomerRepository repository = context.getBean(CustomerRepository.class);
//use the repository.....
((AbstractApplicationContext) context).close();
context.close();
}
}
So that is it -- a very basic JPA Spring Starter Project created with Spring Tools Suite. If I could figure out how to deal with logging in that I could translate that info to my actual project.
-- Edit -- even more info -- and it's fixed!
OK this is interesting...
I changed my Application class and the problem goes away.
So using the this Application (vs. the one posted above) and the logging problem is now OK -- anyone who could comment why this works like this?
#Configuration
#ComponentScan
#EnableAutoConfiguration
public class Application {
public static void main(String[] args) {
ConfigurableApplicationContext context =
SpringApplication.run(Application.class);
CustomerRepository repository = context.getBean(CustomerRepository.class);
//use the repository.....
context.close();
}
}
Note the solution provided below by Alan Hay also works great regardless of how I do the Application class!
Note with either way you can still set setJpaProperties as shown my configuration (see Bean above) to control if you want to see Hibernate's SQL, etc.

I encountered the same issue recently. It appears Spring 4 uses the logback library for logging and my app only had a log4j config file. Adding an additional logging configuration file for logback solved the issue.
If you have an existing log4j config file there is a tool to convert this to logback format here:
http://logback.qos.ch/translator/
If not try adding a file named logback.xml to the root of your classpath:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="A1" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{dd MMM yyyy HH:mm:ss} %-4r [%t] %-5p %c %x - %m%n</pattern>
</encoder>
</appender>
<logger name="org.springframework" level="ERROR"/>
<logger name="org.hibernate" level="ERROR"/>
<root level="DEBUG">
<appender-ref ref="A1"/>
</root>
</configuration>

Related

Logback.xml configuration by environment

I have logback configuration which is like bellow, as you can see i have an appender that appends all the logs to the standalone logging service.
<appender name="digitalGelfAppender"
class="de.siegmar.logbackgelf.GelfUdpAppender">
<graylogHost>testserver.loggingservice.com</graylogHost>
<graylogPort>1234</graylogPort>
<useCompression>true</useCompression>
<layout class="de.siegmar.logbackgelf.GelfLayout">
<originHost>originHost</originHost>
<includeRawMessage>false</includeRawMessage>
<includeMarker>true</includeMarker>
<includeMdcData>true</includeMdcData>
<includeCallerData>false</includeCallerData>
<includeRootCauseData>false</includeRootCauseData>
<includeLevelName>false</includeLevelName>
<shortPatternLayout class="ch.qos.logback.classic.PatternLayout">
<pattern>${log.pattern}</pattern>
</shortPatternLayout>
<fullPatternLayout class="ch.qos.logback.classic.PatternLayout">
<pattern>${log.pattern}</pattern>
</fullPatternLayout>
<staticField>application:DAP</staticField>
<staticField>environment:UAT</staticField>
</layout>
</appender>
<root level="#logback.loglevel#">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="digitalGelfAppender"/>
</root>
My question is that i need to change
<graylogHost>testserver.loggingservice.com</graylogHost>
<graylogPort>1234</graylogPort>
by environment value. So it will be like bellow on production server.
<graylogHost>prodserver.loggingservice.com</graylogHost>
<graylogPort>prodPort</graylogPort>
How can i pass the environment value to the logback.xml file by environment?
I tried to add these variables and use it like property inside xml. Its not working at all.
<property name="GRAYLOG_SERVER" value="${graylog-hostname}" />
<property name="GRAYLOG_SERVER_PORT" value="{graylog-port}" />
Usage
<graylogHost>${GRAYLOG_SERVER}</graylogHost>
<graylogPort>${GRAYLOG_SERVER_PORT}</graylogPort>
Hi i solve my problem by implementing listener that retrieve data from System parameter of environment;
#Controller
public class LoggerContextListener extends ContextAwareBase implements
LoggerContextListener, LifeCycle {
#Override
public void start() {
Map<String, String> property = System.getenv();
Context context = getContext();
context.putProperty("GRAYLOG_SERVER", property.get("graylog-hostname"));
context.putProperty("GRAYLOG_SERVER_PORT", property.get("graylog-port"));
}
After that i add that listener as a contextListener to the lockback.xml
<contextListener class="com.anadolusigorta.dap.config.DapLoggerContextListener"/>
Then problem solved! Now my xml can use these parameter that set from listener;
<graylogHost>${GRAYLOG_SERVER}</graylogHost>
<graylogPort>${GRAYLOG_SERVER_PORT}</graylogPort>
You don't need to create the listener. It is enough that use the environment variables as follows.
<graylogHost>${graylog-hostname}</graylogHost>
<graylogPort>${graylog-port}</graylogPort>

Access to request parameter in custom logback appender

I have custom logback appender that is used in my project. The project is based on REST controllers.
Is there a possibility to have access to current request parameters in the logback appender that is used by logger in spring controller? I need to write something to logs in basis on the request parameter.
What I can see is that logback is initialized before spring boot beans which is not good for me.
Use the org.slf4j.MDC: You can put values in the MDC at any time. Before anything else.
Example :
#ResponseBody
#RequestMapping(value="/myRequest", method = RequestMethod.GET)
public String doAnyAction(#RequestParam(value="Key", required = true) long key)
{
MDC.put("key", key);
logger.info(key);
}
And change your Logback.xml appender to add the "key" as well :
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<layout>
<Pattern>%X{key} - %m%n</Pattern>
</layout>
</appender>

Spring boot application with embedded tomcat, send access logs to syslog

I have a spring boot application, which runs on AWS EC2 instances.
The deployable is just a jar that has tomcat embedded in it.
I am trying to get the application to send its access log to syslog so that I can use AWS CloudWatch to collect logs from different EC2 instances.
I have tried https://logback.qos.ch/access.html, but it does not work with syslog due to this issue: https://jira.qos.ch/browse/LOGBACK-317
Then I tried CommonsRequestLoggingFilter, this does not seem to send its logs to syslog.
I guess trying to send logs to syslog with Spring boot application is not a rare desire.
Is there any comprehensive way to do this?
I just implemented this in my application. (For Spring boot 1.5.6)
Make sure your tomcat access logging is on, below is the way access logging is enabled in application.yml -
server:
tomcat:
accesslog:
enabled: true
pattern: "<APPNAME> %h %l %u %t \"%r\" %s %b %D"
prefix: "localhost_access_log"
suffix: .txt
basedir: /opt/tomcat
The above will output the access logs in your instances.
Now have your logging defined. For example in application.yml file
logging:
level:
org.springframework.web: INFO
com.project.path : DEBUG
config: <path to your config xxml>/logback-springboot.xml
The above will get your logback config file from the desired location.
Now, in your code, you need to define your own tomcat access config. For example -
#Configuration
public class AccessValveConfig extends WebMvcConfigurerAdapter implements
EmbeddedServletContainerCustomizer {
private static XLogger logger = XLoggerFactory
.getXLogger(AccessValveConfig.class);
#Override
public void customize(ConfigurableEmbeddedServletContainer container) {
if (container instanceof TomcatEmbeddedServletContainerFactory) {
TomcatEmbeddedServletContainerFactory factory =
(TomcatEmbeddedServletContainerFactory) container;
AbstractAccessLogValve accessLogValve = new ApiAccessLogValve();
accessLogValve.setPattern("common");
factory.addContextValves(accessLogValve);
} else {
logger.error("WARNING! this customizer does not support your
configured container");
}
}
}
public class ApiAccessLogValve extends AbstractAccessLogValve {
private static XLogger logger = XLoggerFactory
.getXLogger(ApiAccessLogValve.class);
public ApiAccessLogValve(){
super();
}
#Override
protected void log(CharArrayWriter message) {
synchronized (this) {
logger.info(message.toString());
}
}
}
Now make the necessary configs in your logback xml.
<!--This will print app logs in user facility-->
<appender name="SYSLOG" class="ch.qos.logback.classic.net.SyslogAppender">
<syslogHost><<your host name>></syslogHost>
<port>514</port>
<facility>USER</facility>
<suffixPattern><<your pattern>></suffixPattern>
</appender>
<!-- This will print your access logs in local0 facility-->
<appender name="LOCAL0" class="ch.qos.logback.classic.net.SyslogAppender">
<syslogHost><<your host name>></syslogHost>
<port>514</port>
<facility>LOCAL0</facility>
<suffixPattern><<pattern>></suffixPattern>
</appender>
<Logger name="package path to ApiAccessLogValve" level="INFO"
additivity="false">
<appender-ref ref="LOCAL0" />
</Logger>
Now you are all set.

How to get Environment properties from application.properties into logback.groovy in Spring Boot project?

Trying to inject properties defined in application.properties/application.yml into logback.groovy script in Spring Boot project.
I cannot Inject Environment or ApplicationContext into groovy scripts.
Are there any workarounds?
I am not looking for solutions like System.getProperty('spring.profiles.active')
src/main/resources/logback.groovy
import org.springframework.core.env.Environment
#Inject private Environment env; //this is not working. how to get env here?
println "spring.profiles.active : ${env.getProperty('spring.profiles.active')}"
appender("STDOUT", ConsoleAppender) {
encoder(PatternLayoutEncoder) {
pattern = "%green(%d{HH:mm:ss.SSS}) [%thread] %highlight(%-5level) %cyan(%logger{36}) - %msg%n"
}
}
if(System.getProperty("spring.profiles.active")?.equalsIgnoreCase("prod")) {
root INFO, ["STDOUT", "FILE"]
} else {
root INFO, ["STDOUT"]
}
src/main/resources/application.yml
---
spring:
profiles:
active: development
logback.groovy needs to be evaluated very early because otherwise the code for loading the spring configuration, instantiating beans, etc. could not log anything. That's why #Inject can't work.
I see 2 options:
You could easily load application.properties in logback.groovy, but it's far from trivial to take all the configuration override mechanisms that spring provides into account
Create a spring bean that changes the logback configuration programmaticaly
when initialized
A different approach is to externalize the logback configuration on production with -Dlogback.configurationFile=/path/to/config.groovy. Putting the config file in a well known location (like /etc/my-app) makes it easy to change log levels in production without re-deploying (or even re-starting).
Sorry to resurrect this thread but while since this is thread is what I found while looking for a solution, I wanted to share a workaround.
I have used a Custom converter and conversion rule to pull out properties in classpath resource (i.e. application.properties):
In logback.groovy:
conversionRule('springApplicationName', CustomSpringApplicationNameConverter)
def patternExpression = '%green([%d{YYYY-MM-dd HH:mm:ss.SSS}]) [%t] %highlight(%-5level) %magenta([%springApplicationName]) - %cyan(%logger{36}) -- %msg%n'
and then 'patternExpression' used in desired appender
and my custom converter class (in groovy):
class CustomSpringApplicationNameConverter extends ClassicConverter {
#Override
String convert(ILoggingEvent event) {
ClassPathResource classPathResource = new ClassPathResource('application.properties')
Properties applicationProperties = new Properties()
applicationProperties.load(classPathResource.inputStream)
String springApplicationName = applicationProperties.getProperty('spring.application.name')
if (!springApplicationName) {
System.err.println('Could not find entry for \'spring.application.name\' in \'application.properties\'')
}
springApplicationName
}
}

perf4j #Profiled annotation not working

I have gone thru the below link from perf4J site and have done same to same: http://perf4j.codehaus.org/devguide.html#Using_Spring_AOP_to_Integrate_Timing_Aspects
Added the following in my spring.xml.
<aop:aspectj-autoproxy/>
<bean id="timingAspect" class="org.perf4j.log4j.aop.TimingAspect"/>
<bean id="wscClientBase" class="com.xyz.csa.core.common.WscClientBase"/>
In class WscClientBase I have the following method with #Profiled annotation.
#Profiled(tag = "SOAPCALLTEST")
public Object sendMessage(Object message) {
String msg = message.toString();
if (msg.indexOf(' ') > 1) {
msg = msg.substring(1, msg.indexOf(' '));
}
try {
Object ret = marshalSendAndReceive(message);
return ret;
} catch (RuntimeException ex) {
throw ex;
}
}
I dont see the perf4j TimingLogger statements in the application log. However if I use it obtrusively (without annotation) as below, I see the log statements successfully.
public Object sendMessage(Object message) {
String msg = message.toString();
if (msg.indexOf(' ') > 1) {
msg = msg.substring(1, msg.indexOf(' '));
}
StopWatch stopWatch = new Slf4JStopWatch();
try {
Object ret = marshalSendAndReceive(message);
stopWatch.stop("PERF_SUCCESS_TAG", msg);
return ret;
} catch (RuntimeException ex) {
stopWatch.stop("PERF_FAILURE_TAG", msg);
throw ex;
}
}
Am I missing something?
Perf4j
This is a performance analysis and checking plugin for application. It can be integrated with spring using spring AOP. It creates a log file that is give to a parser to analyse and produce relevant information. It can provide average,mean, std deviation by default.
For more general information please check http://perf4j.codehaus.org/index.html
How to setup Perf4j.
For normal setup you just need to add perf4j jar and create StopWatch instance for every code sniplet you want to monitor.
StopWatch stopWatch= new StopWatch(“snipletTagName”)
…
//{your code sniplet}
…
stopwatch.stop();
This will create perf4j monitor and you will get logg information on the console.
Main purpose of this documentation is to have a setup by setup understanding of integrating perf4j with spring.
1.Add all of the below Jar files.
1.perf4j-0.9.16-slf4jonly.jar
2.aspectjweaver-1.6.12.jar
3.aopalliance-1.0.jar
4.commons-logging-1.1.1.jar
5.logback-classic-1.0.7.jar
6.logback-core-1.0.7.jar
7.slf4j-api-1.7.1.jar
8.perf4j-0.9.16.jar
9.aspectjrt-1.6.1.jar
10.commons-jexl-1.1.jar
11.asm-1.5.3.jar
12.cglib-2.1_3.jar
Make sure you have all these jars in your classpath along with spring libraries.
2.create your own logback.xml that will be used by perf4j implicitly
the content of the logback.xml will be
<configuration>
<appender name="perf4jFileAppender"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<File>logs/perf4j.log</File>
<encoder>
<Pattern>%date %-5level [%thread] %logger{36} [%file:%line] %msg%n
</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<FileNamePattern>logs/perf4j.%d{yyyy-MM-dd}.log</FileNamePattern>
</rollingPolicy>
</appender>
<appender name="CoalescingStatistics"
class="org.perf4j.logback.AsyncCoalescingStatisticsAppender">
<param name="TimeSlice" value="1" />
<appender-ref ref="perf4jFileAppender" />
</appender>
<appender name="RootConsoleAppender" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>debug</level>
</filter>
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>%date %-5level [%thread] %logger{36} [%file:%line] %msg%n
</pattern>
</layout>
</appender>
<!-- Loggers -->
<!-- The Perf4J logger. Note that org.perf4j.TimingLogger is the value of
the org.perf4j.StopWatch.DEFAULT_LOGGER_NAME constant. Also, note that additivity
is set to false, which is usually what is desired - this means that timing
statements will only be sent to this logger and NOT to upstream loggers. -->
<logger name="org.perf4j.TimingLogger" additivity="false">
<level value="DEBUG" />
<appender-ref ref="CoalescingStatistics" />
<appender-ref ref="perf4jFileAppender" />
<appender-ref ref="RootConsoleAppender" />
</logger>
</configuration>
3.In your spring configuration file you need to add aspectj tag that will enable #Profiled annotation of perf4j.
(Note: What is #Profiled annotation?: you will add this tag to all the
methods in all the classes that are called from spring instance or use
dependency injection. The object basically should be spring context
registered and the method should be invoked by the object that are
registered in spring context. I wasted one day thinking why my method
was not logged then I realized that the object I tested was not part
of spring context.
OK, the code that you need to add to spring configuration xml is
<!-- this is my spring-context.xml -->
<beans>
<aop:aspectj-autoproxy>
<aop:include name="timingAspect" />
</aop:aspectj-autoproxy>
<bean id="timingAspect" class="org.perf4j.slf4j.aop.TimingAspect" />
<!-- this is the class that will be registered with the spring and now we can get this class and call the method that we need to monitor-->
<bean class="com.perf4jexample.Test" />
</beans>
4.Create the Test class that will implement #Profiled annotation.
public class Test {
private String testVal;
public Test() {
// TODO Auto-generated constructor stub
}
#Profiled
public void testing() {
System.out.println("testt" );
}
public String getTestVal() {
return testVal;
}
public void setTestVal(String testVal) {
this.testVal = testVal;
}
}
5.Ok now you have setup every thing just thing remains is test class that will start spring context and with it load perf4j.
public class Test(){
public static void main(){
AbstractApplicationContext context = new ClassPathXmlApplicationContext(
"spring-context.xml");
context.start();
Test bean = context.getBean(Test.class);
bean.testing();
}
I hope by following these setups you should be able to perf4j console appender to display one line on console.
Perf4j Monitoring command on the log:
For Generating Performance statistical information execute on you logger path
java -jar perf4j-0.9.16.jar myLogger.log
For Generating Graphs
java -jar perf4j-0.9.16.jar --graph perfGraphs.out myLogger.log
I hope this tutorial helps you to integrated Spring, perf4j, logback with Profiled annotation.
Try adding <aop:include name="timingAspect"/> inside the <aop:aspectj-autoproxy/>.
Can you also confirm that you are invoking sendMessage on an object that is retrieved from the spring application context (using getBean or injected as a dependency).
Here I have two ways to make perf4j #Profiled work on spring boot project.
Precondition is adding below dependencies
"org.aspectj:aspectjweaver",
"org.perf4j:perf4j:0.9.16",
"commons-jexl:commons-jexl:1.1",
"cglib:cglib:3.2.1",
For normal spring project, probably need to add little more dependencies like spring-aop, aopalliance... those looks included in spring-boot-starter-parent
1. Java Configuration
This is simplest way and mostly works, but I found that somehow not working on spring-data JpaRepository method. It just provides org.perf4j.log4j.aop.TimingAspect bean and do aspectj autoproxing. Same way like xml configuration provided by other people above
#Configuration
#EnableAspectJAutoProxy
public class PerformanceConfig {
#Bean
public TimingAspect timingAspect() {
return new TimingAspect();
}
}
2. Provide Your own Aspect
In this way, the #Profiled annotated spring-data repository interface methods also work fine. But the downside of this is ignoring the tag given in #Profiled(tag='some tag') and using the joinPoint method name as tag.
#Aspect
#Component
public class PerformanceTracker {
#Around(value="execution(#org.perf4j.aop.Profiled * com.mypackage..*(..))")
public Object checkPerformance(ProceedingJoinPoint pjp) throws Throwable {
StopWatch stopWatch = new Log4JStopWatch(pjp.getSignature().toShortString());
Object result = pjp.proceed();
stopWatch.stop();
return result;
}
}
The reason why it could not work is that the profiled method is in the parent class of the Spring-bean. I can tell it by looking at your bean name: WscClientBase. I assume that this is base class from which you have many children classes.
After some research time I found very important note in Spring documentation about #Transactional and #Cacheable.
http://docs.spring.io/spring-framework/docs/current/spring-framework-reference/html/transaction.html#transaction-declarative-annotations. Check “Method visibility and #Transactional” block.
http://docs.spring.io/spring/docs/current/spring-framework-reference/html/cache.html#cache-annotation-enable. Check “Method visibility and cache annotations”
In both cases they say things like this:
“When using proxies, you should apply the <> annotations only to
methods with public visibility. If you do annotate protected, private
or package-visible methods with these annotations, no error is raised,
but the annotated method does not exhibit the configured caching
settings. Consider the use of AspectJ (see below) if you need to
annotate non-public methods as it changes the bytecode itself.”
And below…
“Spring recommends that you only annotate concrete classes (and
methods of concrete classes) with the #Cache* annotation, as opposed
to annotating interfaces. You certainly can place the #Cache*
annotation on an interface (or an interface method), but this works
only as you would expect it to if you are using interface-based
proxies. The fact that Java annotations are not inherited from
interfaces means that if you are using class-based proxies
(proxy-target-class="true") or the weaving-based aspect (
mode="aspectj"), then the caching settings are not recognized by the
proxying and weaving infrastructure, and the object will not be
wrapped in a caching proxy, which would be decidedly bad.”
I assume that #Profiled use similar weaving mechanism therefore you can't put #Profiled on any method in parent classes.
As a matter of fact I had similar issue in my application: I had #Profiled and #Cacheable in parent class and none of those worked: I didn't see any records in my Perf4J log and cache wasn't updated. When I moved #Profiled into methods of children-classes I started seeing the records in the perf4j.log.
put config below in your "servlet-context-config.xml" .have fun!
<aop:aspectj-autoproxy/>
<bean id="timingAspect" class="org.perf4j.log4j.aop.TimingAspect"/>
<bean id="wscClientBase" class="com.xyz.csa.core.common.WscClientBase"/>
For people who have this type of problem they can check that there is not in spring logs (level info), messages like it " is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)".

Resources