My system currently uses a gemfire cache and uses the io.pivotal.gemfire dependencies. Two of the dependencies, geode-core and geode-management both use org.jgroups (jgroups) as a runtime dependency. Currently I am using version 9.10.13 of the io.pivotal.gemfire dependencies (geode-core, geode-wan, geode-management etc. (see code snippet below)) and version 3.6.14.Final of jgroups.
Due to a vulnerability in the current version of jgroups, I want to upgrade to a newer version (any in version 4 would be sufficient but the newer the better (versions)). The problem I am having is that when I attempt to upgrade to a newer version (3.6.17 or newer), I get the following error when attempting to start-up the cache server:
java.lang.NoSuchMethodError: org.jgroups.View.<init>(Lorg/jgroups/ViewId;Ljava/util/List;)V
at org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger.installView(JGroupsMessenger.java:459) ~[geode-membership-9.10.13.jar:?]
When I actually go to investigate the issue, it looks as though it is referencing a line where the View class constructor was changed to take in a "Collection" instead of a "List", which seemed like a strange error since a List is a Collection.
I have tried using different combinations of jgroups and gemfire going up to 4/5 and 9.10.17 respectively without success. If anyone has any recommendations on how to move past (solve) this jgroups vulnerability while using a gemfire cache it would be much appreciated. Also, is there a table that lists gemifre versions vs. supported versions of jgroups?
Was able to get in contact with VMware Tanzu. All 9.* versions of gemfire require jgroups 3.6.14.Final. The new version of gemfire (10.*) which is currently in beta, is supposed to remove all dependencies on jgroups.
Related
We are currently facing a little conundrum with Spring Boot that's actually not a rare situation:
Spring Security OAuth2 Client has a critical vulnerability that our production systems might be vulnerable to; the vulnerability is fixed in the latest patch release of Spring Security. Naturally, we want to update our production systems ASAP, but this means we need to override the Spring Boot (Gradle) dependency management system if we don't want to wait until the next Spring Boot patch release.
I know that this can be done quite easily, in this case e.g. by setting something like this in gradle.properties:
spring-security-oauth2-client.version=5.7.5
The problem with this is that this dependency is now pinned to a specific version; I need to remember to remove this property as soon as a Spring Boot patch release is available. This means extra coordination effort because we need to document this in our backlog, and even with good documentation on our part there is a risk that we forget to do it, which means the dependency will eventually be outdated - which is the exact opposite of what we wanted to achieve in the first place.
What I'd rather do is specify a minimum version of the dependency, that gets ignored if it is older than what the Spring Boot dependency management plugin's default version.
Can this be done? Or is there a better strategy to handle a situation like this?
This is possible using gradle's dynamic versions.
For instance, you can have:
dependencies {
implementation 'org.springframework.security:spring-security-oauth2-client:5.+'
}
But keep in mind that dynamic versions add nondeterminism to your build and can introduce unexpected behaviour changes to the system.
Using dynamic versions in a build bears the risk of potentially
breaking it. As soon as a new version of the dependency is released
that contains an incompatible API change your source code might stop
compiling.
References:
Version ranges in gradle
I have a web application that used to run fine on many web servers (tomcat, jboss, weblogic and websphere). Now, however, it has an error when deploying on WebSphere 9.
The app contains the jar javax.transaction-api-1.2. Some of its classes, e.g., javax.transaction.xa.XAResource, are also included in Java SE, but not all of them. Some are specific to Java EE and are required by some 3rd-party libraries in my app. The app is always deploying with child-first (parent-last) classloader.
WebSphere 9 throws this error during startup when the app tries to load the Oracle JDBC driver:
java.lang.LinkageError: loading constraint violation: loader "com/ibm/ws/classloader/CompoundClassLoader#7157be44" previously initiated loading for a different type with name
"javax/transaction/xa/XAResource" defined by loader "com/ibm/oti/vm/BootstrapClassLoader#422c7b1b"
Note that we aren't actually using XA transactions in the app, we are using regular transactions.
On other servers, and previous versions of WebSphere, it was never a problem. The server didn't care that we load XAResource from inside the war, even if it was previously loaded somewhere in the server. Now WebSphere 9 is different, it says that the app classloader already loaded this class from the server, but I don't know why or when did this happen.
Any idea how to solve this?
Remove the transaction API from your application. JTA 1.2 is already included in the server and provides no value in your applications. It's always risky to bring Java EE/SE APIs in a parent-last class loader unless you are 100% certain that they are technically necessary, because they can lead to issues like this one.
I can't say how this worked in previous server versions (there have been some Java-level changes in enforcing linkage issues like this), but the solution is reasonably straightforward.
At the end we did two things to solve this problem.
1) We upgraded the jta jar to version 1.3 (link here). This jar solves the problem by avoiding duplicate classes - it contains only J2EE classes and omits the J2SE classes that are already included in the JVM.
2) We upgraded WebSphere server from 9.0.0.7 to 9.0.0.11.
At the time, I suspected just upgrading the jar should suffice, but our QA had some issues with it and they also upgraded the server. Due to lack of time, we didn't investigate it further and just decided to do both.
I've downloaded the drivers via Maven Central (org.apache.derby).
derby-10.15.1.3.jar
derbyclient-10.15.1.3.jar
derbynet-10.15.1.3.jar
derbyshared-10.15.1.3.jar
So what am I missing ? None of these JARs contains the package "org.apache.derby.jdbc", which used to contain the ClientDriver and EmbeddedDriver in the past?
Indeed, to use the Derby client driver with Derby 10.15, you now need all three of: derbyclient.jar, derbyshared.jar, and derbytools.jar. This is (weakly) documented here:
A new jar file (derbyshared.jar) has been added. All Derby
configurations require it. In addition, the derbytools.jar library is
now required when running the network server and/or when using Derby
DataSources.
Since you are running the network server, you now require derbytools.jar (as well as the new derbyshared.jar when running the client software.
I think it would be worth suggesting to the Derby community that the release note could make this stand out more clearly (you could file an improvement request with the Derby project, e.g.)
It looks like derbytools depends on derbyshared, so you don't have to list derbyshared as a dependency in your pom.xml (just derbytools).
However, this seems to work counter to how every other jdbc client works for every other database. Rather than updating documentation to say you have to include extra dependencies, Derby should make the derbyclient stand alone (better solution) or have the maven derbyclient depend on derbytools (so that when this dependency problem is fixed, people won't have to go back and update their pom.xmls to remove unneeded dependencies).
In my project I am forced to use these packages:
com.sparkjava:spark-core:2.3, which ends up using jetty-server:9.3.2.v20150730
org.apache.spark:spark-core_2.10:1.2.0, which ends up using jetty-server:8.1.14.v20131031
Note that com.sparkjava and org.apache.spark have nothing to do with each other. They are called both spark funnily.
The issue here is that both jetty versions are incompatible, so if I force jetty 8.X the system crashes, if I force jetty 9.X the system crashes again, I get java.lang.NoClassDefFoundError: org/eclipse/jetty/server/ServerConnector in one case and java.lang.NoClassDefFoundError: org/eclipse/jetty/server/bio/SocketConnector in the other.
What I am expected to do in such a situation ?
Note: I've tried to shadow jetty, but the dependency manager resolves just one (9.X by default, or 8.X if I force it) and then it shadows it, so it's really not helping.
It would be exceedingly difficult to resolve this situation.
Jetty 8.1 is about 4 major version behind Jetty 9.3, which represents many hundreds of releases of difference.
Note: Jetty versioning is [servlet_support].[major_ver].[minor_ver].
Jetty 8.x is Servlet 3.0, while Jetty 9.x is Servlet 3.1
The architecture of the connectors has evolved tremendously in that time frame, from being old school blocking Sockets in Jetty 8 to no blocking connectors at all in Jetty 9, with Jetty 9 needing to evolve the connectors to support features in TLS/1.2, and ALPN in order to properly support HTTP/2, and the internal I/O handling to support the new Servlet 3.1 Async I/O feature set.
Solution #1:
You won't be able to have both versions running in the same VM without some sort of classloader isolation, and careful configuration to ensure they don't claim the same resources (listening ports, temp files, etc)
Solution #2:
Upgrade (or downgrade) one or the other spark dependency till you hit a common jetty version. (Spark_2.11 / 2.0.0 seems to support Jetty 9.2.x)
Solution #3:
Apache Spark is open source, go submit a patch that upgrades its use of Jetty to 9.3 (this might be difficult as Apache Spark isn't ready to use Java 8 yet, which is a requirement for Jetty 9.3)
I'm having a project using both HBase 1.0.0 (Cloudera version) and Elasticsearch. With the upgrade to ES 2.0 I'm experiencing a problem with guava version. ES 2.0 requires guava version 18.0, but Cloudera requires guava 14.0.1.
No matter what version I define in my dependency management in my parent pom one of the two won't work.
Looking around I see that this problem occurs quite some time (e.g. http://gbif.blogspot.co.at/2014/11/upgrading-our-cluster-from-cdh4-to-cdh5.html)
1) Any ideas on how to solve this problem without any complex re-design of my application?
If not, I'm thinking of doing all the ES-stuff in a separate application. Communicating via messaging (already using AMQ) for indexing. Not sure though how to communicate for search/filter requests (at the moment implemented via Java API).
2) Any other ideas?
3) Any ideas/hints on how to solve the communication issue?
I found this blog post when googling in combination with maven-shade plug-in, so this might be another option.
https://www.elastic.co/blog/to-shade-or-not-to-shade