I installed HBase 0.92 on Hadoop 1.0.0 and it works fine in full-distributed mode, but an annoying warning keeps appearing. How can I get rid of it?
.......
hbase(main):001:0> status
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/opt/hbase-0.92.0/lib/slf4j-log4j12-1.5.8.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/hadoop-1.0.0/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
3 servers, 0 dead, 0.6667 average load
.......
P.S. I did not set the $CLASSPATH variable in hbase-env.sh. I run Hadoop with start-all.sh, then start HBase with start-hbase.sh.
I removed the slf4j-log4j12-1.5.8.jar in ${hase}/lib/ then the warning never shown again. It should be due to a duplicated class loaded, both hadoop and hbase use the same jar in the same jvm.
You can give it a try.
The warning emitted by SLF4J is just that, a warning. Even when multiple bindings are present, SLF4J will pick one logging framework/implementation and bind with it. The way SLF4J picks a binding is determined by the JVM and for all practical purposes should be considered random. As of version 1.6.6, SLF4J will name the framework/implementation class it is actually bound to.
Embedded components such as libraries or frameworks should not declare a dependency on any SLF4J binding but only depend on slf4j-api. When a library declares a compile-time dependency on a SLF4J binding, it imposes that binding on the end-user, thus negating SLF4J's purpose. When you come across an embedded component declaring a compile-time dependency on any SLF4J binding, please take the time to contact the authors of said component/library and kindly ask them to mend their ways.
Related
Background
I am creating a library for internal use in Spring-Boot projects. The library I am writing, using Gradle, has an unavoidable dependency on log4j2 since it is a logging library. Here is the dependency in question:
implementation group: 'org.apache.logging.log4j', name: 'log4j-slf4j-impl', version: '2.14.1'
Spring-Boot has its own implicit dependency on Logback. When importing my library into a Spring project, it creates the following error:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/alexandre.meddin/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-slf4j-impl/2.14.1/9a40554b8dab7ac9606089c87ae8a5ba914ec932/log4j-slf4j-impl-2.14.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/alexandre.meddin/.gradle/caches/modules-2/files-2.1/ch.qos.logback/logback-classic/1.2.3/7c4f3c474fb2c041d8028740440937705ebb473a/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
It is possible to subdue this error with the following Gradle configuration in my Spring project's build.gradle:
configurations.all {
resolutionStrategy.dependencySubstitution {
substitute module('org.apache.logging.log4j:log4j-slf4j-impl') using module('org.slf4j:slf4j-api:0.0.0')
}
}
Question
Is it possible to solve this conflict inside the library? That is, use the required dependency but not expose it to the importing project? This way the conflict will be solved for all future users of the library, instead of needing to be debugged anew every time someone unfamiliar uses it.
compileOnly
Use compileOnly instead of implementation. This would have been solved in a heartbeat if I'd read the documentation more closesly. Problem solved.
https://blog.gradle.org/introducing-compile-only-dependencies
Think to Simple Logging Facade (SLF4J) for Java. As a background, it provides an API by means of a simple facade pattern in a way that the underlying logging backend is determined at runtime by adding the desired binding to the classpath. It may be the standard java.util.logging, log4j, logback or tinylog.
The neat separation of the client application from the logging backend reduces the coupling between the specific application and any particular logging framework. This can make it easier to integrate a newly implemented client with existing code of other projects that have already made a choice of logging backend.
So considering the logging API SLF4J, for compilation you need only the slf4j-api and you shall avoid including any specific binding like slf4j-log4j12 as a compile dependency.
As result, slf4j-log4j12 is a good candidate for being a runtime scoped dependency instead of a compile dependency, because this will allow you to switch among slf4j bindings at runtime without having to recompile the application.
Question: where to set the switching among logging frameworks bindings at runtime in a Maven project? Any example?
Disclaimer: I am the author of SLF4J
Even though the documentation talks about changing the logging back-end at runtime, SLF4J allows to switch logging back-ends at build time but not at runtime.
I see that when starting a NodeManager，
the -Dyarn.log.file=yarn-hadoop-nodemanager-hostname1.log parameter is passed to the NodeManager's main method,
but I can't find where this yarn.log.file is used in java code so that log message can write into the yarn.log.file
wish for some help, thanks
Hadoop uses log4j behind the scenes. Log4j supports different configurable appenders, but when the logging system is configured, you will not see any reference at the file that is just one of many possible appenders ( ie output for the log ). You will probably dig on various log4j configuration file in the hadoop sources looking for *log4j.properties and you will eventually find your referenced file.
I am having problems integrating perf4j in an existing maven application.
I tried several approaches, but none of them seemed to work, so I was wondering if anyone has some insight into how this is done.
What I want to do is use the AOP part of perf4j on some methods and log them into a different file than the one used for app logging. Thanks
You might be suffering from the same declaration order issue as mentioned in another perf4j/logback related question. In short, it is always a good idea to enable printing of logback's internal status messages by setting the debug attribute to true within the configuration element. Also do not forget that any referenced appender must be declared beforehand.
I have a following question. We use log4j in our two projects, that are hosted on the same GlassFish server. Each project has inside log4j.properties file, that points to the files, that are based in different catalogs (let's name them Project1 and Project2).
Now, for some unclear reasons sometimes the info messages of the first project are written to Project2 log files, and the reverse is also true. I checked the log4j.properties files for both of the projects, there is nothing pointing in them to the log of the other project.
The suspicion is that log4j is not actually thread-safe, therefore if two users are working in two systems in the same time, the messages of the loggers can get mixed. Is this suspicion correct?
Yes, log4j is thread safe:
Yes, log4j is thread-safe. Log4j components are designed to be used in
heavily multithreaded systems.
Ref.
What you are describing sounds more like a config mistake, rather than a cross process/threading issue.
Yes, log4j is thread safe. The reason is the method AppenderSkeleton.doAppend() is synchronized. But be carefull configuring programmatically!
For example you can't use the same instance of TTCCLayout in different appenders(read javadoc)! Take a look at PatternLayout method format(). It changes instance field(StringBuffer sbuf), so if you use same PatternLayout instance in different appenders you are to face race conditions. EnhancedPatternLayout is better, cause they modified format method.