How to avoid C3P0 logging framework from generation lots of String objects - performance

During a performance test with c3p0, I found that com.mchange.v2.resourcepool.BasicResourcePool.trace() is creating a lots of string objects. Although the strings output are suppressed by the slf4j log instance (as the log level is warn), this contributed significantly on the object allocation rate. Do we have any configuration of avoiding this? Thanks in advance.
logback.xml
<logger name="com.mchange" level="WARN" additivity="false">
<appender-ref ref="ROLLING" />
</logger>
pom.xml
<logback.version>1.1.3</logback.version>
<slf4j.version>1.7.12</slf4j.version>
<!-- Logging -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-core</artifactId>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
</dependency>

So BasicResourcePool.trace() is properly guarded to test the logger for FINEST (which should map to TRACE before allocating Strings. The question then is why, given your config, are the Strings being allocated?
The answer turns out to be... I screwed up. The guard methods in the bridge library to slf4j logging improperly reported that loggers were logging at levels that they were not (in the mchange-commons-java library on which c3p0 depends). Pretty bad. I'll try to publish a fix tonight (and notify you with a comment).

Related

Why is my Log4j logging not displaying to the console?

I have a web app built in Eclipse/STS with Spring MVC and Maven.
I want to add logging, so I added SLF4 and Log4J to the pom.xml like this ..
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-api -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.30</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-simple -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.30</version>
<scope>test</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-log4j12 -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.30</version>
<scope>test</scope>
</dependency>
I have a simple log4j.properties file in the project/src/main/resources folder, like this ...
# Root logger option
log4j.rootLogger=DEBUG, stdout, file
# Redirect log messages to console
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n
I created a Logger class in my main WebController, like this ...
private static Logger logger = LoggerFactory.getLogger(WebController.class);
And in my "showMain" method I'm doing some logging stuff, like this ...
#RequestMapping(value={"", "/", "showhome"}, method={RequestMethod.POST,RequestMethod.GET})
public ModelAndView showHome(Model model) {
logger.info("########### TEST LOG INFO");
logger.error("########### TEST LOG ERROR");
logger.warn("########### TEST LOG WARN");
logger.debug("########### TEST LOG DEBUG");
/* ... */
}
But when I run the application, I don't see any logging output in the console.
I also don't see anything in the console output to indicate that it is even using the logging framework. And there's no "can't find log4j.properties" message or anything.
I tried putting the log4j.properties in different places in the project, but nothing.
I must be missing something simple? What have I missed?
In your pom.xml is missing the logging engine, you could use log4j2.
Please Consider to use the latest version of log4j2 instead of log4j (1.2.x) because you could take advance of:
Lazy log: log4j(1.2.x) build a string also if a level is not activated
lambda in order to avoid evaluation of expensive methods
A lot of appenders for modern platform
More easy way to configure a lot of parameters (reload configuration, appenders, ...)
pom.xml
<dependencies>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.13.3</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.13.3</version>
</dependency>
</dependencies>
You stated that you added log4j in your pom.xml but I do not see it, are you sure this is your newest version of pom.xml?
<!-- https://mvnrepository.com/artifact/log4j/log4j -->
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>

Direct Spring DEBUG logging to Console

I have a Maven project and I want to direct DEBUG Spring logging to the console.
My log interface is slf4j and I'm binding to Log4j2.
In my pom.xml I have excluded commons-logging in favour or Slf4j:
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>${org.springframework-version}</version>
<exclusions>
<!-- Exclude Commons Logging in favor of SLF4j -->
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
I believe slf4j is wired up correctly because I have this in my log4j2.xml:
<Logger name="com.myApp" level="info" additivity="false">
<AppenderRef ref="Console" />
</Logger>
This in my test class constructor:
logger.info("constructor");
logger.debug("constructor");
and if i flip the log level in log4j2.xml from info/debug I get one or two log lines in the console.
I also have this in my log4j2.xml:
<Logger name="org.springframework" level="all" additivity="false">
<AppenderRef ref="Console" />
</Logger>
But it has zero effect. What am I missing? Why doesn't Spring log detailed debug lines to the console?
Found it if it helps anyone else. This dependency was missing:
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-jcl</artifactId>
<version>2.1</version>
</dependency>
So logging looks like its working, but Spring logging is not.

Disabling Spring log, to have readable logs

How can I disable Spring logs to have log outputs that I can easily read or someone else can read.
An answer to a similar question at, how to disable spring bean loading log suggested to comment out all lines having org.springframework substring in log4j.properties file. In my case there no such lines.
Here is log4j.properties
# Define the root logger with appender file
log4j.rootLogger = DEBUG, stdout
# Define the file appender
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
# Set the name of the logs destination
log4j.appender.stdout.target=System.out
# Set the immediate flush to true (default)
log4j.appender.stdout.ImmediateFlush=true
# Set the threshold to debug mode
log4j.appender.stdout.Threshold=debug
# Set the append to false, overwrite
log4j.appender.stdout.Append=false
# Define the layout for appender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.conversionPattern=%d{yyyy-MM-dd}:%m%n
Your default logging, for everything that isn't explictily specified, is DEBUG. So everything is logged at that level (judging from your configuration), basically you are flooding your logs. You should not remove loggers for org.springframework you should add them and set another level for them.
log4j.logger.org.springframework=INFO
or whatever log level level you like.
Before you do, you should have get some knowledge of :
1.How to add maven dependency
2.Where to put log4j configuration file
OK, return to the question.The top answer is not working for spring 4.x, if you are using spring4.x try following 3 steps:
remove common-logging from spring-core
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>4.3.4.RELEASE</version>
<exclusions>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
Without this step, no matter what you put in log4j configuration file is not working, cause spring is using common-logging my boy!
PS:Within lots of spring modules, spring-core is the the only module that explicitly depends on commons-logging.
Add SLF4J and log4j
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<version>1.5.8</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.5.8</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.5.8</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.14</version>
</dependency>
configure log4j.properties(You can also use xml file)
log4j.rootCategory=INFO, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %t
%c{2}:%L - %m%n
log4j.category.org.springframework.beans.factory=INFO
Now, the annoying spring debug log is going away.Enjoy coding!
The answer is from spring.io doc,for origin click here
All the answers gave examples with configuration in log4j.properties, which is what was asked. I had the same problem but I was using configuration in log4j2.xml and answers here lead me to a solution.
In case someone is on the same boat as me, here is what I did: I added a node Logger with name org.springframework and level WARN as shown in the following sample:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
</Console>
</Appenders>
<Loggers>
<Logger name="org.springframework" level="WARN"/>
<Root level="info">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
I'm using Spring Boot and the exclusion I'm making is logback-classic as shown in the following snippet:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<exclusions>
<exclusion>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
</exclusion>
</exclusions>
</dependency>
You can specify the required package name as can be seen in the following example:
log4j.logger.com.foo=WARN
Now you can see only WARN, ERROR and FATAL logs in console.
I was also facing this same issue. Springframework logging was not getting removed even after log4j configuration. Then I found that its logging depends on commons-logging.
You have to disable commons-logging from the dependency in pom.xml file of the web app.
Even after removing commons-logging from pom.xml please check the dependency hierarchy available in Eclipse or STS IDE. This will help in knowing if somehow its getting added because of some other dependency management which we may not be knowing.
After removing dependency just add log4j.logger.org.springframework=ERROR to your log4j configuration. This will help.

Onejar-maven-plugin to set the order of jar loading

I've been using the library apache-log4j-extras for logging. It contains class org.apache.log4j.Logger.
Now I had to reference some 3rd party library, that uses logback and has among its dependencies log4j-over-slf4j (jar). Unfortunately, latter jar also contains class org.apache.log4j.Logger.
Looks like the latter class is preferred by the onejar classloader...
I don't need logback and log4j-over-slf4j. Just want my org.apache.log4j.Logger from apache-log4j-extras back. What are my options with the Onejar-maven-plugin?
EDIT: It appeared to be an issue with Debug mode in IDEA, not with onejar. However the question is still relevant: how can I ensure that I load the requried class with Onejar?
EDIT2: E.g. in C# it could be easily resolved with "extern alias" feature.
why don't you just exclude it?
<dependency>
<groupId>my.naughty.thirdparty</groupId>
<artifactId>thirdparty-with-log4j-over-slf4j</artifactId>
<version>${thirdparty.version}</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>log4j-over-slf4j</artifactId>
</exclusion>
</exclusions>
</dependency>
Looks like (for Debug mode in IDEA) just switching depenencies order solved the issue.
How it was:
<dependency>
<groupId>my.naughty.thirdparty</groupId>
<artifactId>thirdparty-ref-log4j-over-slf4j</artifactId>
<version>0.50</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>apache-log4j-extras</artifactId>
<version>1.1</version>
</dependency>
Changed to:
<dependency>
<groupId>log4j</groupId>
<artifactId>apache-log4j-extras</artifactId>
<version>1.1</version>
</dependency>
<dependency>
<groupId>my.naughty.thirdparty</groupId>
<artifactId>thirdparty-ref-log4j-over-slf4j</artifactId>
<version>0.50</version>
</dependency>
But surely I can't consider it as a robust solution, especially for Onejar.

How does simply adding slf4j to the pom.xml wrap log4j?

From what I have seen in example spring pom.xml files is that they add a few entries for slf4j and log4j and somehow when you use log4j in your spring application it will be wrapped by slf4j library.
Can someone explain to me how this magically happens?
Spring still uses commons-logging for all the internal logging (backwards compatibility).
If you wish to use some other logging framework (log4j) then you need to bridge the calls from commons logging to your framework of choice. Otherwise you will have to maintain multiple logging configurations.
slf4j acts as a simple facade for various logging frameworks (jul, log4j, jcl, logback) and allows you to plug in the desired logging framework at deployment time.
Instead of using the logging framework implementation that is imposed by the third party framework you provide the slf4j's bridge implementation that acts like the real thing but really just forwards the logging calls to slf4j or its concrete binding.
Logging section of Maven pom.xml usually looks like this:
<!-- remove the real commons-logging from classpath -->
<!-- declare as provided or exclude from spring jars -->
<dependency>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
<version>1.0</version>
<scope>provided</scope>
</dependency>
<!-- add slf4j interfaces to classpath -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.6.4</version>
<scope>compile</scope>
</dependency>
<!-- add commons logging to slf4j bridge to classpath -->
<!-- acts as jcl but routes commons-logging calls to slf4j -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<version>1.6.4</version>
<scope>runtime</scope>
</dependency>
<!-- add log4j binding to classpath -->
<!-- routes slf4j calls to log4j -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.6.4</version>
<scope>runtime</scope>
</dependency>
<!-- add log4j to classpath -->
<!-- does the logging -->
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.16</version>
</dependency>
This has nothing to do with the Spring container nor dependency injection, it is pure classpath, classloader stuff...
Please see the following links for further details.
slf4j is a logging API, which doesn't do anything, just bunch of interfaces. log4j is a logging system, with concrete classes. there is a slf4j-log4j library which uses log4j as a backend for the slf4j API.
Some projects explicitly depend on log4j, they call concrete classes. So, you cannot use another backend (e.g. logback or j.u.l or apache commons or whatever) for your project which you wisely made using the slf4j API only.
There is a trick to substitute log4j classes by a mock implementation (the bridge) which just simply redirects all calls to the sl4j. In maven you just declare a dependency with very high version number and this mock considered as ultra-modern log4j library.
try with adding :
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>

Resources