Storm-Kafka : java.lang.NoClassDefFoundError: com/google/common/base/Strings - maven

I am trying to run a storm LocalCluster using storm-core 1.0.2 and kafka-spout 1.0.2.
My pom file looks like below
<dependencies>
<!-- Storm Dependency -->
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-core</artifactId>
<version>1.0.2</version>
<scope>provided</scope>
</dependency>
<!-- Storm Kafka Dependency -->
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-kafka</artifactId>
<version>1.0.2</version>
<exclusions>
<exclusion>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
</exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
</exclusions>
</dependency>
.....
</dependencies>
<build>
<resources>
<resource>
<directory>src/main/resources</directory>
</resource>
</resources>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>com.walmartlabs.midas.storm.Topology</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.2.1</version>
<executions>
<execution>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
<configuration>
<executable>java</executable>
<includeProjectDependencies>true</includeProjectDependencies>
<includePluginDependencies>false</includePluginDependencies>
<classpathScope>compile</classpathScope>
<mainClass>${storm.topology}</mainClass>
</configuration>
</plugin>
</plugins>
</build>
I use following command to run storm cluster locally.
mvn package
storm jar target/myproject-0.1-SNAPSHOT-jar-with-dependencies.jar com.company.project.storm.Topology
I get following exception
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/base/Strings
at org.apache.storm.kafka.KafkaSpout.declareOutputFields(KafkaSpout.java:184)
at org.apache.storm.topology.TopologyBuilder.getComponentCommon(TopologyBuilder.java:431)
at org.apache.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:135)
at com.walmartlabs.midas.storm.Topology.submitLocalTopology(Topology.java:50)
at com.walmartlabs.midas.storm.Topology.main(Topology.java:75)
Caused by: java.lang.ClassNotFoundException: com.google.common.base.Strings
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
The google common dependency is that of storm-kafka artifact. And i have set the scope of storm-kafka as default so i believe all its dependency should be build in my jar.
If i replace KafkaSpout with a simple spout that is just reading some file, everything works fine.
I believe while packaging the jar i am not packaging all the dependencies of storm-kafka artifact.What is wrong here?

https://github.com/apache/storm/tree/v1.0.2/external/storm-kafka#using-storm-kafka-with-different-versions-of-kafka
Storm 1.0.2 changed the scope of artifact "kafka_[scala_version]" to "provided", which means that you need to add this as dependency. This is for having flexibility of combination on scala version and/or kafka version.

I get this problem,and sovled by adding the dependency like below (ps: guava.jar
version is 16.0.1 used by storm1.0.3) .By the way,we can reference the guava version used by storm-starter.
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>${guava.version}</version>
<!--<version>16.0.1</version>-->
</dependency>

Related

How to solve a Camel3 fat-jar creation exception?

I use Apache Camel 3.18.2 witch camel-main. To create a fat-jar I configured the following two plugins within my pom.xml:
<plugin>
<groupId>org.apache.camel</groupId>
<artifactId>camel-maven-plugin</artifactId>
<version>3.18.2</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>prepare-fatjar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>de.powerstat.camel.homeautomation.MainApp</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
As dependecies I have:
<dependencies>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-core</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-main</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-netty</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-stream</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-paho</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-quartz</artifactId>
</dependency>
<dependency>
<groupId>de.powerstat.camel.component</groupId>
<artifactId>camel-fbaha</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>de.powerstat.camel.component</groupId>
<artifactId>camel-fbtr64</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.openmuc</groupId>
<artifactId>jsml</artifactId>
<version>1.1.2</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.18.0</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.18.0</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.18.0</version>
<scope>runtime</scope>
</dependency>
</dependencies>
But durin running mvn clean install I got:
[INFO] --- camel-maven-plugin:3.18.2:prepare-fatjar (default) # myhome ---
[INFO] Found 5 Camel type converter loaders from project classpath
[INFO]
[INFO] --- maven-assembly-plugin:2.2-beta-5:single (make-assembly) # myhome ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.2-beta-5:single (make-assembly) on project myhome: Error reading assemblies: No assembly descriptors found. -> [Help 1]
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Exception in thread "ivy-httpclient-shutdown-handler" java.lang.NoClassDefFoundError:
org/apache/http/impl/conn/PoolingHttpClientConnectionManager$2
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.shutdown(PoolingHttpClientConnectionManager.java:413)
at org.apache.http.impl.client.HttpClientBuilder$2.close(HttpClientBuilder.java:1244)
at org.apache.http.impl.client.InternalHttpClient.close(InternalHttpClient.java:201)
at org.apache.ivy.util.url.HttpClientHandler.close(HttpClientHandler.java:357)
at org.apache.ivy.util.url.HttpClientHandler$1.run(HttpClientHandler.java:84)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.ClassNotFoundException: org.apache.http.impl.conn.PoolingHttpClientConnectionManager$2
at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:271)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:247)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:239)
... 6 more
That looks to me like this is an implicit dependency that have not been added by the camel-maven-plugin.
So my question is how to solve this problem?
I also tried to use the maven-shade-plugin instead of maven-assembly-plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.4.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<createDependencyReducedPom>true</createDependencyReducedPom>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
Which then results in a fat-jar with the following exception during executing it:
[ERROR] 2022-09-30T22:04:48,426 15612/22 org.apache.camel.processor.errorhandler.DefaultErrorHandler Failed delivery for (MessageId:
08B251E5AD78E72-0000000000000000 on ExchangeId:
08B251E5AD78E72-0000000000000000). Exhausted after
delivery attempt: 1 caught:
org.apache.camel.InvalidPayloadException: No body available of type: java.lang.Integer but has type:
java.lang.String on: Message. Caused by: No type converter available to convert from type: java.lang.String to the required type: java.lang.Integer. Exchange[]. Caused by:
[org.apache.camel.NoTypeConversionAvailableException - No type converter available to convert from type:
java.lang.String to the required type: java.lang.Integer]
So it looks the shade-plugin creates a fat-jar but missed something of camels dependencies.
Maybe this could be solved in one or the other way?
Last but not least the main question is how to create a Camel3 fat jar that is complete and works correctly?
Based on the error message
Error reading assemblies: No assembly descriptors found.
It looks like you are missing your assembly descriptor. You'll need to define a descriptor or descriptorRef
descriptor example:
<project>
[...]
<build>
[...]
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.4.2</version>
<configuration>
<descriptors>
<descriptor>src/assembly/src.xml</descriptor>
</descriptors>
</configuration>
[...]
</project>
descriptorRef example:
<project>
[...]
<build>
[...]
<plugins>
<plugin>
<!-- NOTE: We don't need a groupId specification because the group is
org.apache.maven.plugins ...which is assumed by default.
-->
<artifactId>maven-assembly-plugin</artifactId>
<version>3.4.2</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
[...]
</project>
If defining a descriptor, see the this about the descriptor format.

Using Provided Artifact As Maven Plugin Dependency

This seems like it should be a simple question, but I can't seem to find any information about it. When a maven plugin has a required dependency, is it possible to tell it to use an artifact defined elsewhere in the section of the pom?
As an example, I'm trying to add the 'maven-processor-plugin' to my build. That plugin has a dependency on 'hibernate-jpamodelgen'. I'm working with wildfly, so I already have that jar as a dependency of the project. I want to ensure I am using the same version for both. Is what I'm trying to do even possible?
Some code snippets:
<dependencies>
<dependency>
<groupId>org.wildfly</groupId>
<artifactId>wildfly-ejb3</artifactId>
<version>${version.server.bom}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-jpamodelgen</artifactId>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<compilerArguments>
<processor>-proc:none</processor>
</compilerArguments>
</configuration>
</plugin>
<plugin>
<groupId>org.bsc.maven</groupId>
<artifactId>maven-processor-plugin</artifactId>
<version>4.5</version>
<executions>
<execution>
<id>process</id>
<goals>
<goal>process</goal>
</goals>
<phase>generate-sources</phase>
<configuration>
<outputDirectory>${project.build.directory}/generated-sources/java/jpametamodel</outputDirectory>
<processors>
<processor>org.hibernate.jpamodelgen.JPAMetaModelEntityProcessor</processor>
</processors>
<overwrite>true</overwrite>
</configuration>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-jpamodelgen</artifactId>
<!-- How do I handle this without hard coding the version? -->
<!-- <version>???</version> -->
</dependency>
</dependencies>
</plugin>
</build>
Define a property like <hibernate-jpamodelgen.version> in the <properties> section of the POM.
Then use that property for the version like ${hibernate-jpamodelgen.version}.

Can't build working Java app with gRPC, Protobuf and BoringSSL to JAR using Maven

I have problem with building my Java app to jar file using Maven.
Application is using gRPC and Protobuf.
When I start my app in IntelliJ everything work just fine, problem is when I want to build jar with Maven... I don't have much experience with creating pom files.
I tried to find some solution but nothing works and I ended up with pom.xml as below:
[...]
<properties>
<grpc.version>1.17.1</grpc.version>
<protoc.version>3.5.1-1</protoc.version>
<netty.tcnative.version>2.0.13.Final</netty.tcnative.version>
[...]
</properties>
<dependencies>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-protobuf</artifactId>
<version>${grpc.version}</version>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-stub</artifactId>
<version>${grpc.version}</version>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-netty</artifactId>
<version>${grpc.version}</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-tcnative-boringssl-static</artifactId>
<version>${netty.tcnative.version}</version>
<scope>runtime</scope>
</dependency>
[...]
</dependencies>
<build>
<extensions>
<extension>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
<version>1.5.0.Final</version>
</extension>
</extensions>
<plugins>
<plugin>
<groupId>org.xolstice.maven.plugins</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<version>0.5.1</version>
<configuration>
<protocArtifact>com.google.protobuf:protoc:${protoc.version}:exe:${os.detected.classifier}
</protocArtifact>
<pluginId>grpc-java</pluginId>
<pluginArtifact>io.grpc:protoc-gen-grpc-java:${grpc.version}:exe:${os.detected.classifier}
</pluginArtifact>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>compile-custom</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<version>1.4.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
<executions>
<execution>
<id>enforce</id>
<goals>
<goal>enforce</goal>
</goals>
<configuration>
<rules>
<requireUpperBoundDeps/>
</rules>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>pl.test.Main</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
</plugins>
</build>
And this is the exception when I try to start gRPC server with SSL context:
Exception in thread "Thread-2" java.lang.UnsatisfiedLinkError: failed to load the required native library
at io.netty.handler.ssl.OpenSsl.ensureAvailability(OpenSsl.java:346)
at io.netty.handler.ssl.ReferenceCountedOpenSslContext.<init>(ReferenceCountedOpenSslContext.java:202)
at io.netty.handler.ssl.OpenSslContext.<init>(OpenSslContext.java:43)
at io.netty.handler.ssl.OpenSslServerContext.<init>(OpenSslServerContext.java:347)
at io.netty.handler.ssl.OpenSslServerContext.<init>(OpenSslServerContext.java:335)
at io.netty.handler.ssl.SslContext.newServerContextInternal(SslContext.java:422)
at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:447)
at pl.test.grpc.GrpcServer.start(GrpcServer.java:80)
at pl.test.app.Main.lambda$new$0(Main.java:80)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSL
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:85)
at io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:194)
at pl.test.grpc.GrpcServer.getSslContextBuilder(GrpcServer.java:72)
... 3 more
I'm building it using command:
mvn clean compile assembly:single
Can someone help with creating working pom file? The result doesn't have to be single jar file, it might be multiple jars.
I found solution, maybe it will help someone deal with the same problem.
We have to add dependency io.netty.netty-handler and set compatible versions of io.grpc.grpc-netty, io.netty.netty-tcnative-boringssl-static and io.netty.netty-handler as it is described in table over here https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty.
Here's my current pom.xml
[...]
<properties>
<grpc.version>1.17.1</grpc.version>
<protoc.version>3.5.1-1</protoc.version>
<netty.tcnative.version>2.0.17.Final</netty.tcnative.version>
<netty.handler.version>4.1.30.Final</netty.handler.version>
[...]
</properties>
<dependencies>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-protobuf</artifactId>
<version>${grpc.version}</version>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-stub</artifactId>
<version>${grpc.version}</version>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-netty</artifactId>
<version>${grpc.version}</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-handler</artifactId>
<version>${netty.handler.version}</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-tcnative-boringssl-static</artifactId>
<version>${netty.tcnative.version}</version>
<scope>runtime</scope>
</dependency>
[...]
</dependencies>
<build>
<extensions>
<extension>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
<version>1.5.0.Final</version>
</extension>
</extensions>
<plugins>
<plugin>
<groupId>org.xolstice.maven.plugins</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<version>0.5.1</version>
<configuration>
<protocArtifact>com.google.protobuf:protoc:${protoc.version}:exe:${os.detected.classifier}
</protocArtifact>
<pluginId>grpc-java</pluginId>
<pluginArtifact>io.grpc:protoc-gen-grpc-java:${grpc.version}:exe:${os.detected.classifier}
</pluginArtifact>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>compile-custom</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<version>1.4.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
<executions>
<execution>
<id>enforce</id>
<goals>
<goal>enforce</goal>
</goals>
<configuration>
<rules>
<requireUpperBoundDeps/>
</rules>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>pl.test.Main</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
</plugins>
</build>
And now I can build jar using command:
mvn clean compile assembly:assembly

Spring Boot jar executable - How to generate files to different directory similar as maven-dependency-plugin does

As straich question, why spring-boot-maven-plugin works but maven-assembly-plugin not in the pom below?
I have googled few hours and I red somewhere that we don't need maven-assembly-plugin if we use spring-boot-maven-plugin. It doesn't answer my question since I want to have more control in which folder my files are genereated. Along that, if I "mvn clean compile package assembly:single" I get two executable jars, one outcome of spring-boot-maven-plugin and another outcome of maven-assembly-plugin in my project target folder. The one generated by spring-boot-maven-plugin runs as expected when I start via command line (java -jar aws.scheduller-1.jar. On the opposite, if I try java -jar aws.scheduller-1-jar-with-dependencies.jar I get a message complaining that it wasn't possible to load ...config.BootApp (I googled a bit and I guess it is something related to certain default profile in Spring Boot).
That said, how can I use maven to package Spring-boot dependency in different folder than default target directory?
Main Class
#SpringBootApplication
#EnableScheduling
#ComponentScan({ "br.com.mycompany.tasks","br.com.mycompany.utils" })
#PropertySource("file:C:/temp/application.properties")
//#PropertySource("file:/home/ec2-user/JOBs/application.properties")
public class BootApp {
public static void main(String[] args) {
SpringApplication.run(new Object[] { BootApp.class }, args);
}
}
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>grpAwsScheduller</groupId>
<artifactId>aws.scheduller</artifactId>
<version>1</version>
<packaging>jar</packaging>
<properties>
<project.build.sourceencoding>UTF-8</project.build.sourceencoding>
<logback.version>1.1.9</logback.version>
<logstash-logback-encode.version>4.8</logstash-logback-encode.version>
<spring-boot-starter.version>1.4.3.RELEASE</spring-boot-starter.version>
<aws-java-sdk.version>1.11.73</aws-java-sdk.version>
<jdk.version.version>1.7</jdk.version.version>
<project.build.directory>C:/temp</project.build.directory>
<!-- Maven versions -->
<!-- https://maven.apache.org/plugins/ -->
<maven-compiler-plugin.version>3.6.1</maven-compiler-plugin.version>
<maven-dependency-plugin.version>3.0.0</maven-dependency-plugin.version>
</properties>
<dependencies>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>${aws-java-sdk.version}</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-core</artifactId>
<version>1.11.73</version>
</dependency>
<!-- <dependency> <groupId>com.amazonaws</groupId> <artifactId>aws-java-sdk-sqs</artifactId>
<version>${aws-java-sdk.version}</version> </dependency> <dependency> <groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-sns</artifactId> <version>${aws-java-sdk.version}</version>
</dependency> -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<version>${spring-boot-starter.version}</version>
</dependency>
<dependency>
<groupId>net.logstash.logback</groupId>
<artifactId>logstash-logback-encoder</artifactId>
<version>${logstash-logback-encode.version}</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>${logback.version}</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-core</artifactId>
<version>${logback.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<fork>true</fork>
<mainClass>br.com.MyCompany.config.BootApp</mainClass>
</configuration>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.0.0</version>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>br.com.MyCompany.config.BootApp</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>assemble-all</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<!-- <excludes> <exclude>**/log4j.properties</exclude> </excludes> -->
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>br.com.MyCompany.config.BootApp</mainClass>
<classpathPrefix>dependency-jars/</classpathPrefix>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>${maven-dependency-plugin.version}</version>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<includeScope>runtime</includeScope>
<outputDirectory>${project.build.directory}/dependency-jars/</outputDirectory>
<overWriteReleases>false</overWriteReleases>
<overWriteSnapshots>false</overWriteSnapshots>
<overWriteIfNewer>true</overWriteIfNewer>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

Maven uses different dependencies depending on which machine it runs?

I have a project that is building properly on my local machine. In my company we use a server machine to do the continuous integration but that machine has no connection to the internet. So I have set up artifactory and copied my whole repository to it. Then connected jenkins with this artifactory. So jenkins should do the build without any problem no? As it has the same setup as my local project.
After a few seconds of building a message pops up that he can't find a dependency:
[ERROR] Failed to execute goal on project crs-data: Could not resolve
dependencies for project com.ing.crs:crs-data:jar:1.1.3-SNAPSHOT:
Failed to collect dependencies for
[com.ing.crs:crs-framework:jar:1.1.3-SNAPSHOT (compile),
org.apache.openjpa:openjpa-all:jar:2.3.0 (provided),
ibm.websphere:j2ee6:jar:8.5.0 (provided),
ibm.websphere:jpaThinClient:jar:8.5.0 (provided),
com.google.guava:guava:jar:14.0.1 (compile),
org.hamcrest:hamcrest-all:jar:1.3 (test), junit:junit:jar:4.11 (test),
mockito-all:mockito-all:jar:1.8.4 (test), oracle.jdbc:ojdbc6:jar:11.2
(test), com.h2database:h2:jar:1.3.167 (test),
ibm.websphere:embeddedEJBContainer:jar:8.5.0 (test),
com.ing.be:bbllib.DeploymentInfo:jar:2.3.2 (provided)]: Failed to
read artifact descriptor for commons-dbcp:commons-dbcp:jar:1.4: Could
not transfer artifact commons-dbcp:commons-dbcp:pom:1.4 from/to
crs.maven.repo
(http://sdbeapp00433.devbe.development:8082/artifactory/repo): Access
denied to:
http://sdbeapp00433.devbe.development:8082/artifactory/repo/commons-dbcp/commons-dbcp/1.4/commons-dbcp-1.4.pom
-> [Help 1]
But locally the build doesn't even use commons-dbcp:commons-dbcp:pom:1.4. How is that possible????
The pom:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupId>com.ing.crs</groupId>
<artifactId>crs-parent-pom</artifactId>
<version>1.1.3-SNAPSHOT</version>
<relativePath>../crs-parent-pom/pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>crs-data</artifactId>
<version>${project.crs-data.version}</version>
<profiles>
<profile>
<id>default</id>
<properties>
<dbDictionary>disableAlterSeqenceIncrementBy=true</dbDictionary>
</properties>
</profile>
<profile>
<id>stubs</id>
<properties>
<dbDictionary></dbDictionary>
</properties>
</profile>
</profiles>
<dependencies>
<!-- CRS -->
<dependency>
<groupId>com.ing.crs</groupId>
<artifactId>crs-framework</artifactId>
<version>${project.crs-framework.version}</version>
</dependency>
<!-- Other -->
<dependency>
<groupId>org.apache.openjpa</groupId>
<artifactId>openjpa-all</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>ibm.websphere</groupId>
<artifactId>j2ee6</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>ibm.websphere</groupId>
<artifactId>jpaThinClient</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>
<!-- test -->
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-all</artifactId>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</dependency>
<dependency>
<groupId>mockito-all</groupId>
<artifactId>mockito-all</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>oracle.jdbc</groupId>
<artifactId>ojdbc6</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>1.3.167</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>ibm.websphere</groupId>
<artifactId>embeddedEJBContainer</artifactId>
<version>8.5.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.ing.be</groupId>
<artifactId>bbllib.DeploymentInfo</artifactId>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<finalName>crs-data</finalName>
<outputDirectory>target/crs-data</outputDirectory>
<plugins>
<plugin>
<groupId>org.bsc.maven</groupId>
<artifactId>maven-processor-plugin</artifactId>
<version>2.2.4</version>
<executions>
<execution>
<id>process</id>
<goals>
<goal>process</goal>
</goals>
<phase>generate-sources</phase>
<configuration>
<compilerArguments>-Aopenjpa.source=7 -Aopenjpa.metamodel=true</compilerArguments>
<processors>
<processor>org.apache.openjpa.persistence.meta.AnnotationProcessor6</processor>
</processors>
<outputDirectory>target/generated-sources/metamodel</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.8</version>
<executions>
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>target/generated-sources/metamodel</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-ejb-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.openjpa</groupId>
<artifactId>openjpa-maven-plugin</artifactId>
<version>2.3.0</version>
<executions>
<execution>
<id>enhancer_for_test</id>
<configuration>
<includes>**/entities/**/*.class</includes>
<includes>**/data/**/*.class</includes>
<addDefaultConstructor>true</addDefaultConstructor>
<enforcePropertyRestrictions>true</enforcePropertyRestrictions>
</configuration>
<phase>compile</phase>
<goals>
<goal>enhance</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>org.apache.openjpa</groupId>
<artifactId>openjpa</artifactId>
<version>2.3.0</version>
</dependency>
</dependencies>
</plugin>
</plugins>
<pluginManagement>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.3</version>
<executions>
<execution>
<id>copy-test-persistence</id>
<phase>process-test-resources</phase>
<configuration>
<tasks>
<!--backup the "proper" persistence.xml-->
<copy file="${project.build.outputDirectory}/META-INF/persistence.xml" tofile="${project.build.outputDirectory}/META-INF/persistence.xml.proper"/>
<!--replace the "proper" persistence.xml with the "test" version-->
<!-- copy file="${project.build.testOutputDirectory}/META-INF/persistence.xml" tofile="${project.build.outputDirectory}/META-INF/persistence.xml"/-->
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
<execution>
<id>restore-persistence</id>
<phase>prepare-package</phase>
<configuration>
<tasks>
<!--restore the "proper" persistence.xml-->
<copy file="${project.build.outputDirectory}/META-INF/persistence.xml.proper" tofile="${project.build.outputDirectory}/META-INF/persistence.xml"/>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathLayoutType>custom</classpathLayoutType>
<customClasspathLayout>${artifact.artifactId}.${artifact.extension}</customClasspathLayout>
</manifest>
<manifestEntries>
<Implementation-Version>${project.version}</Implementation-Version>
<Build-Time>${timestamp}</Build-Time>
</manifestEntries>
</archive>
</configuration>
</plugin>
<!--This plugin's configuration is used to store Eclipse m2e settings
only. It has no influence on the Maven build itself. -->
<plugin>
<groupId>org.eclipse.m2e</groupId>
<artifactId>lifecycle-mapping</artifactId>
<version>1.1.1</version>
<configuration>
<lifecycleMappingMetadata>
<pluginExecutions>
<pluginExecution>
<pluginExecutionFilter>
<groupId>
org.codehaus.mojo
</groupId>
<artifactId>
openjpa-maven-plugin
</artifactId>
<versionRange>
[1.0,)
</versionRange>
<goals>
<goal>enhance</goal>
</goals>
</pluginExecutionFilter>
<action>
<execute>
<runOnIncremental>false</runOnIncremental>
</execute>
</action>
</pluginExecution>
<pluginExecution>
<pluginExecutionFilter>
<groupId>
org.apache.openjpa
</groupId>
<artifactId>
openjpa-maven-plugin
</artifactId>
<versionRange>
[2.2.1,)
</versionRange>
<goals>
<goal>enhance</goal>
</goals>
</pluginExecutionFilter>
<action>
<ignore />
</action>
</pluginExecution>
</pluginExecutions>
</lifecycleMappingMetadata>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<resources>
<resource>
<directory>src/main/resources/</directory>
<filtering>true</filtering>
</resource>
</resources>
</build>
Your build is failing since Maven is failing to download dependencies from Artifactory due to an authentication issue:
Access denied to: http://sdbeapp00433.devbe.development:8082/artifactory/repo/commons-dbcp/commons-dbcp/1.4/commons-dbcp-1.4.pom
If you will look at the Artifactory access.log you should see a matching log entry about a denied download.
If you are working with the Artifactory Jenkins plugin, you can configure the resolver credentials as described here.
If you are not using the Artifactory Jenkins plugin, you should configure the Maven authentication as described in "Working With Maven". This requires adding the correct credentials to the Maven settings.xml file.
A third option is allowing anonymous access to Artifactory as described here. This will allow resolving dependencies from Artifactory without authentication.
commons-dbcp is probably a transitive dependency. If you want to find its origin, you should use the dependency:tree goal of the Maven dependency plugin which displays the dependency tree for this project.
As a side note, you should not use the "repo" repository. This is a default global virtual repository which effectively aggregates all other repositories. By configuring Maven with this URL, any request for an artifact will go through Artifactory which will search through all of the local and remote repositories defined in the system. It is better to work with a virtual repository which aggregates only the relevant repositories for your build.

Resources