Maven could not resolve dependencies spark - maven

I try to build a simple java program: JavaWordCount for spark-1.1.0.
I get this error: Building JavaWordCount 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.279 s
[INFO] Finished at: 2014-10-23T11:28:30-04:00
[INFO] Final Memory: 9M/156M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project JavaWordCount: Could not resolve dependencies for project spark.examples:JavaWordCount:jar:1.0-SNAPSHOT: Failure to find org.apache.spark:spark-assembly_2.10:jar:1.1.0 in http://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
Here are my dependencies from pom.xml
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-assembly_2.10</artifactId>
<version>1.1.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-examples_2.10</artifactId>
<version>1.1.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.1.0</version>
</dependency>
It includes spark assembly.
Any ideas would be really appreciated.
Thank you!

The problem is that the dependency:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-assembly_2.10</artifactId>
<version>1.1.0</version>
</dependency>
is not a jar its a pom file only which means you can't define it like this. You can see it in the error message:
Failure to find org.apache.spark:spark-assembly_2.10:jar:1.1.0
which shows that Maven will try to download a jar file. The means you have to define it like this:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-assembly_2.10</artifactId>
<version>1.1.0</version>
<type>pom</type>
</dependency>
But i'm not sure if this will solve all problems. You should take a deep look into the documentation if this is the right path.
Update:
You can also use that as BOM via:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-assembly_2.10</artifactId>
<version>1.1.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>

Related

Maven - command line to update dependency by groupId

I would like to update dependencies just of the "groupA". How can I do this using command line?
<dependency>
<groupId>group-A</groupId>
<artifactId>artifact-1</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>group-A</groupId>
<artifactId>artifact-2</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>group-B</groupId>
<artifactId>artifact-1</artifactId>
<version>1.0.0</version>
</dependency>
The versions plugin has a goal "use-latest-versions" that seems to do what you need.
Assume you have something like:
<dependencies>
<dependency>
<groupId>group-B</groupId>
<artifactId>artifact-1</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.1</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-compress</artifactId>
<version>1.9</version>
</dependency>
</dependencies>
Then running: mvn versions:use-latest-versions -D"includes=org.apache.commons:*:*:*:*"
would update those two dependencies in that group:
[INFO] --- versions-maven-plugin:2.7:use-latest-versions (default-cli) # artifact-id ---
[INFO] Major version changes allowed
[INFO] Updated org.apache.commons:commons-lang3:jar:3.1 to version 3.9
[INFO] artifact org.apache.commons:commons-compress: checking for updates from central
[INFO] Updated org.apache.commons:commons-compress:jar:1.9 to version 1.18
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
Assuming you just want the latest version that can be found in a repository.
The versions plugin has some other things to offer as well. For dependency work the dependency plugin is also often handy.

Run "mvn test" failed: Unable to load the mojo 'test' (or one of its required components)

I try to run command like "mvn -B -f ./pom.xml test" but got error message like below:
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.676 s
[INFO] Finished at: 2019-06-01T20:27:24+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.smartbear.soapui:soapui-pro-maven-plugin:5.1.2:test (default) on project apitest-hfa: Execution default of goal com.smartbear.soapui:soapui-pro-maven-plugin:5.1.2:test failed: Unable to load the mojo 'test' (or one of its required components) from the plugin 'com.smartbear.soapui:soapui-pro-maven-plugin:5.1.2': java.util.NoSuchElementException
[ERROR] role: org.apache.maven.plugin.Mojo
[ERROR] roleHint: com.smartbear.soapui:soapui-pro-maven-plugin:5.1.2:test
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginContainerException
But this command can work well on my another machine, and it also all pass when I try it with SoapUIPro software but not command, I can't figure out what's wrong with my pom.xml, is there anything I have missed in my pom dependencies?
Really need some help, thanks a lot.
Screenshots like below:
enter image description here
Try to add the following which are missing from your dependency list -
<dependencies>
<dependency>
<groupId>org.reflections</groupId>
<artifactId>reflections</artifactId>
<version>0.9.9-RC1</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>3.10-FINAL</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
<version>2.5.6</version>
<type>pom</type>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.8.1</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-utils</artifactId>
<version>3.0.5</version>
</dependency>
<dependency>
<groupId>com.jgoodies</groupId>
<artifactId>forms</artifactId>
<version>1.2.1</version>
</dependency>
</dependencies>
org.codehaus.plexus can be solved the issue.
If Possible, can you share your pom file, that would be easier to identify!!!

Error executing selenium scripts through maven

I am getting the following error when executing 'install' command for maven
**[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] Copying 0 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.0:testCompile (default-testCompile) # com.amazon.hybrid ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is platform dependent!
[INFO] Compiling 6 source files to G:\My Learnings\Framework\target\test-classes
[INFO] -------------------------------------------------------------**
**[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] No compiler is provided in this environment. Perhaps you are running on a JRE rather than a JDK?
[INFO] 1 error**
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.483s
[INFO] Finished at: Thu Jun 22 23:27:00 IST 2017
[INFO] Final Memory: 9M/153M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.0:testCompile (default-testCompile) on project com.amazon.hybrid: Compilation failure
[ERROR] No compiler is provided in this environment. Perhaps you are running on a JRE rather than a JDK?
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Below is my pom.xml structure
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.amazon.hybrid</groupId>
<artifactId>com.amazon.hybrid</artifactId>
<version>0.0.1-SNAPSHOT</version>
<description>This is used to order items from Amazon.in</description>
<dependencies>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.53.0</version>
</dependency>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.7</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.aventstack</groupId>
<artifactId>extentreports</artifactId>
<version>3.0.6</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi</artifactId>
<version>3.14</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>3.14</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.0</version>
<configuration>
<compilerVersion>1.8</compilerVersion>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
</plugins>
</build>
</project>
Note:
I have changed the environment from JRE to JDK by Window -> Preferences -> Java -> Installed JREs -> Added JDK and selected JDK
2. Updated Project
Even after doing the previous steps, the issue remains unresolved.
Any help in solving the issue is appreciated.
Regards,
ANUP J
You are using very old versions of almost everything present in your pom.xml
you can download the latest versions from here ---> Maven repository link to get latest dependencies
I'm mentioning few of them :
Selenium
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>3.4.0</version>
</dependency>
TestNG
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.11</version>
</dependency>
Similarly, you can find rest of the dependencies from the above given link.
Let me know if it helps. :)

mvn fails to find HTMLUnitDriver dependency during project install

maven suggests:
Starting with 2.53.0 you need to explicitly include HtmlUnitDriver as a dependency to include it.
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-htmlunit-driver</artifactId>
<version>2.20</version>
</dependency>
However, build fails not being able to resolve the dependency.
>
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12.648 s
[INFO] Finished at: 2016-03-24T14:49:18+02:00
[INFO] Final Memory: 7M/118M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project maven_selenium_project_sample: Could not resolve dependencies for project com.samplecompany:maven_selenium_project_sample:jar:1.0-SNAPSHOT: Could not find artifact org.seleniumhq.selenium:selenium-htmlunit-driver:jar:2.20 in central (https://repo.maven.apache.org/maven2) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal on project maven_selenium_project_sample: Could not resolve dependencies for project com.samplecompany:maven_selenium_project_sample:jar:1.0-SNAPSHOT: Could not find artifact org.seleniumhq.selenium:selenium-htmlunit-driver:jar:2.20 in central (https://repo.maven.apache.org/maven2)
What am I missing?
Thanks
You're missing a 0 in the version like this :
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-htmlunit-driver</artifactId>
<version>2.20.0</version>
</dependency>
As per the error: what you are asking for does not exist at Maven Central. You need either:
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>htmlunit-driver</artifactId>
<version>2.20</version>
</dependency>
or:
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-htmlunit-driver</artifactId>
<version>2.52.0</version>
</dependency>
Or one of the other possible versions from here.

Cannot download Maven Resources Plugin when building in Jenkins

I have created Selenium IDE test cases and have tested them in Eclipse. The project is a Maven java project and the tests will run fine in Eclipse however, when I add the project to Jenkins and try to build my project, there is a error and a build failure.
The error:
Started by user anonymous
Building in workspace C:\Program Files (x86)\Jenkins\workspace\Selenium IDE Recon
Parsing POMs
Discovered a new module Test:SeleniumTest SeleniumTest
Modules changed, recalculating dependency graph
[Selenium IDE Recon] $ C:\java/bin/java -cp "C:\Program Files (x86)\Jenkins\plugins\maven-plugin\WEB-INF\lib\maven3-agent-1.5.jar;C:\Users\MC44948\Software\springsource\springsource\apache-maven-3.0.4\boot\plexus-classworlds-2.4.jar" org.jvnet.hudson.maven3.agent.Maven3Main C:\Users\MC44948\Software\springsource\springsource\apache-maven-3.0.4 "C:\Program Files (x86)\Jenkins\war\WEB-INF\lib\remoting-2.45.jar" "C:\Program Files (x86)\Jenkins\plugins\maven-plugin\WEB-INF\lib\maven3-interceptor-1.5.jar" "C:\Program Files (x86)\Jenkins\plugins\maven-plugin\WEB-INF\lib\maven3-interceptor-commons-1.5.jar" 53032
<===[JENKINS REMOTING CAPACITY]===>channel started
Executing Maven: -B -f C:\Users\MC44948\Spring_Workspace\Selenium IDE Recon\pom.xml install
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building SeleniumTest 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: http://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.5/maven-resources-plugin-2.5.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.671s
[INFO] Finished at: Mon Sep 29 11:46:35 BST 2014
[INFO] Final Memory: 4M/8M
[INFO] ------------------------------------------------------------------------
Waiting for Jenkins to finish collecting data
[ERROR] Plugin org.apache.maven.plugins:maven-resources-plugin:2.5 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-resources-plugin:jar:2.5: Could not transfer artifact org.apache.maven.plugins:maven-resources-plugin:pom:2.5 from/to central (http://repo.maven.apache.org/maven2): repo.maven.apache.org: Unknown host repo.maven.apache.org -> [Help 1]
[JENKINS] Archiving C:\Users\MC44948\Spring_Workspace\Selenium IDE Recon\pom.xml to Test/SeleniumTest/0.0.1-SNAPSHOT/SeleniumTest-0.0.1-SNAPSHOT.pom
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
channel stopped
Finished: FAILURE
My pom:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>Test</groupId>
<artifactId>SeleniumTest</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>SeleniumTest</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.6</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.43.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.opera</groupId>
<artifactId>operadriver</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-chrome-driver</artifactId>
<version>2.43.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-server</artifactId>
<version>2.6.0</version>
<scope>test</scope>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.opera</groupId>
<artifactId>operadriver</artifactId>
<version>0.16</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-remote-driver</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
</dependencyManagement>
</project>
This seems to be related to Internet connectivity. Did you notice the error: "Unknown host repo.maven.apache.org"?
Try wget http://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.5/maven-resources-plugin-2.5.pom and see if you can fetch data.
For more details, you can refer this link.
You can try also this...
My Java project was building successfully when building from command line, but I was getting an error "Unknown host repo.maven.apache.org" when building it with Jenkins.
The problem was is the custom maven settings file which our company was using.
So what you can do is: Under Job configuration, Build section, click "Advanced...".
Set the "Settings file" to "Settings file in filesystem" and give the path to the setting file from your maven installation location. For example
C:\Program Files\apache-maven-3.1.1\conf\settings.xml
Set the "Global Settings file" to "Global settings file on filesystem" and give the path to the "${user.home}/.m2/settings.xml".
You might be behind cooperate proxy (or any other proxy\firewall).
Make sure you can access maven central from the slave you're running your build.
If not - configure your http_proxy and https_proxy on your server.

Resources