I am using Gitlab_ci to trigger a maven script when code is pushed.
The script executes:
mvn compile
[...]
[INFO] project ............................................ SUCCESS [1.312s]
[INFO] project-api ........................................ SUCCESS [1.416s]
[INFO] project-api-impl ................................... SUCCESS [0.329s]
[INFO] project-webapp ..................................... SUCCESS [0.192s]
[INFO] project-webapp-exec ................................ SUCCESS [0.026s]
[INFO] project-webapp-it .................................. SUCCESS [2.052s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
mvn tomcat7:deploy
[INFO] project............................................ SUCCESS [1.840s]
[INFO] project-api ........................................ SUCCESS [2.524s]
[INFO] project-api-impl ................................... FAILURE [0.257s]
[INFO] project-webapp ..................................... SKIPPED
[INFO] project-webapp-exec ................................ SKIPPED
[INFO] project-webapp-it .................................. SKIPPED
And I get the following error:
[ERROR] Failed to execute goal on project project-api-impl: Could not resolve dependencies for project eu.project:project-api-impl:jar:0.6-DEVELOPMENT: Failure to find eu.project:project-api:jar:0.6-DEVELOPMENT in http://repo.maven.apache.org/maven2
Since the project builds successfuly (the WAR is created), I can't understand why when executing deploy, build fails.
[Also, I am not sure whether to mark this as homework. The assignment's goal is to create a tomcat web application and manually deploy to a server to test. Since maven / continuous integration wasn't a requisit, I believe this should not be marked as a homework question]
Related
I'm using maven to manage the build of a large project that it's divided in several modules that are built from the root. So, in this case, maven will use the reactor functionality to build each module in the correct order.Something like this:
root/pom.xml
sp1/pom.xml
sp2/pom.xml
sp3/pom.xml
sp4/pom.xml
I build the project with maven from root directory using the command
mvn clean install
If the build is OK, maven will print the reactor summary:
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Root Project ....................................... SUCCESS [ 18.271 s]
[INFO] sp1 ................................................ SUCCESS [ 2.034 s]
[INFO] sp2 ................................................ SUCCESS [ 22.770 s]
[INFO] sp3 ................................................ SUCCESS [03:39 min]
[INFO] sp4 ................................................ SUCCESS [04:39 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 46:34 min
[INFO] Finished at: 2017-10-04T17:08:11+00:00
[INFO] Final Memory: 256M/1599M
[INFO] ------------------------------------------------------------------------
But, What happens if one of build phase of one of the subproject fails?. For example, some tests in the sp2 subproject fail and I fix them and relaunch the build from the sp2 with this command
mvn clean install -rf :sp2
The project is built correctly. Then, maven prints the reactor, but not the full reactor, only from sp2. Something like this:
```
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] sp2 ................................................ SUCCESS [ 22.770 s]
[INFO] sp3 ................................................ SUCCESS [03:39 min]
[INFO] sp4 ................................................ SUCCESS [04:39 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 46:34 min
[INFO] Finished at: 2017-10-04T17:08:11+00:00
[INFO] Final Memory: 256M/1599M
[INFO] ------------------------------------------------------------------------
My question is: Is it possible to obtain the full reactor summary (including the modules that have been built in the previous execution?)
Reactor Build Order is resolved during Reactor Sorting which leads into a deterministic sequence of execution for the list of projects. The command --resume-from (-rf) as used in:
mvn clean install -rf :sp2
resumes reactor build from the specified project, where the order is similar to as was already been decided, it simply traverses the remaining of the projects in the list which is the updated Reactor Build Order and hence the Reactor Summary corresponds to the new build order.
I am building apache drill 1.1.0 from its source. I got the following error:
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Drill Root POM ............................. SUCCESS [8.234s]
[INFO] Drill Protocol .................................... SUCCESS [6.639s]
[INFO] Common (Logical Plan, Base expressions) ........... SUCCESS [50.152s]
[INFO] contrib/Parent Pom ................................ SUCCESS [0.717s]
[INFO] contrib/data/Parent Pom ........................... SUCCESS [0.365s]
[INFO] contrib/data/tpch-sample-data ..................... SUCCESS [4.968s]
[INFO] exec/Parent Pom ................................... SUCCESS [0.545s]
[INFO] exec/Java Execution Engine ........................ FAILURE [38.896s]
[INFO] exec/JDBC Driver using dependencies ............... SKIPPED
[INFO] JDBC JAR with all dependencies .................... SKIPPED
[INFO] contrib/mongo-storage-plugin ...................... SKIPPED
[INFO] contrib/hbase-storage-plugin ...................... SKIPPED
[INFO] contrib/hive-storage-plugin/Parent Pom ............ SKIPPED
[INFO] contrib/hive-storage-plugin/hive-exec-shaded ...... SKIPPED
[INFO] contrib/hive-storage-plugin/core .................. SKIPPED
[INFO] Packaging and Distribution Assembly ............... SKIPPED
[INFO] contrib/sqlline ................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:51.105s
[INFO] Finished at: Mon Aug 10 07:24:19 IST 2015
[INFO] Final Memory: 67M/325M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project drill-java-exec: Could not
resolve dependencies for project
org.apache.drill.exec:drill-java-exec:jar:1.2.0-SNAPSHOT: Could not
transfer artifact net.hydromatic:foodmart-data-json:jar:0.4 from/to
central (http://repo.maven.apache.org/maven2): GET request of:
net/hydromatic/foodmart-data-json/0.4/foodmart-data-json-0.4.jar from
central failed: Premature end of Content-Length delimited message body
(expected: 18196233; received: 17400 -> [Help 1]
Edit:
I checked the jar in .m2/repository. It contains foodmart-data-json-0.4.jar.lastUpdated, foodmart-data-json-0.4.pom.lastUpdated, foodmart-data-json-0.4.pom, foodmart-data-json-0.4.pom.sha1, _maven.repositories. When I opened foodmart-data-json-0.4.jar.lastUpdated in text editor. I found:
#NOTE: This is an internal implementation file, its format can be changed without prior notice.
#Mon Aug 10 08:44:18 IST 2015
http\://repository.mapr.com/nexus/content/repositories/drill/.lastUpdated=1439176430716
default-http\://repo.maven.apache.org/maven2/.lastUpdated=1439176458509
http\://conjars.org/repo/.lastUpdated=1439176430238
http\://repository.mapr.com/nexus/content/repositories/drill/.error=
http\://repo.maven.apache.org/maven2/.error=Could not transfer artifact net.hydromatic\:foodmart-data-json\:jar\:0.4 from/to central (http\://repo.maven.apache.org/maven2)\: GET request of\: net/hydromatic/foodmart-data-json/0.4/foodmart-data-json-0.4.jar from central failed
http\://conjars.org/repo/.error=
How to resolve this error?
I am trying to build Apache Tez (Both 0.6.1 and 0.7.0 version) for hadoop-2.6.0 in windows using below command
mvn clean package -Dhadoop.version=2.6.0 -DskipTests -Dmaven.javadoc.skip
But i am getting below exception
[INFO]
[INFO] --- exec-maven-plugin:1.3.2:exec (Bower install) # tez-ui ---bower FileSaver.js#24b303f49213b905ec9062b708f7cd43d56a5dde ENOGIT git is not installed or not in the PATH
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] tez ................................................ SUCCESS [ 0.924 s]
[INFO] tez-api ............................................ SUCCESS [ 11.585 s]
[INFO] tez-common ......................................... SUCCESS [ 1.421 s]
[INFO] tez-runtime-internals .............................. SUCCESS [ 2.029 s]
[INFO] tez-runtime-library ................................ SUCCESS [ 4.751 s]
[INFO] tez-mapreduce ...................................... SUCCESS [ 2.553 s]
[INFO] tez-examples ....................................... SUCCESS [ 0.862 s]
[INFO] tez-dag ............................................ SUCCESS [ 8.363 s]
[INFO] tez-tests .......................................... SUCCESS [ 2.044 s]
[INFO] tez-ui ............................................. FAILURE [ 3.105 s]
[INFO] tez-plugins ........................................ SKIPPED
[INFO] tez-yarn-timeline-history .......................... SKIPPED
[INFO] tez-yarn-timeline-history-with-acls ................ SKIPPED
[INFO] tez-mbeans-resource-calculator ..................... SKIPPED
[INFO] tez-tools .......................................... SKIPPED
[INFO] tez-dist ........................................... SKIPPED
[INFO] Tez ................................................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 38.169 s
[INFO] Finished at: 2015-07-13T15:07:23+05:30
[INFO] Final Memory: 76M/1049M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.3.2:exec (Bower install) on project tez-ui: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :tez-ui
How to solve this?
Based on the error, it seems like you need git installed:
ENOGIT git is not installed or not in the PATH
Also, this might be helpful for other errors: https://cwiki.apache.org/confluence/display/TEZ/Build+errors+and+solutions
Once look at this... this may help you.
Reason: You are running as root or with superuser permissions.
Solutions:
Recommended: Run without superuser permission, or as a non-root user.
Alternate: If you want to continue as root, add --allow-root to arguments tag of exec-maven-plugin in tez-ui/pom.xml.
Thank you.
Like hadoop_user proposed, I would firstly check this website and try all the fixes but with one remark: in 3rd fix (clearing folder node_tmp) - if you cannot find that folder try clearing the following folder: $TEZ_PATH/tez-X.Y.Z/ tez-ui/src/main/webapp/node_modules/
I tried deleting that folder once I tried everything else and then it worked.
Install git, node & npn. This worked in my case.
Installing git, node and npm worked for me as well.Before building install all the three and try to run the mvn package -DskipTests.This works.
I am trying to build Spark 1.2 with Maven. My goal is to use PySpark with YARN on Hadoop 2.2.
I saw that this was only possible by building Spark with Maven. First, is this true?
If it is true, what is the problem in the log below? How do I correct this?
C:\Spark\spark-1.2.0>mvn -Pyarn -Phadoop-2.2 -Dhadoop.version=2.2.0 -DskipTests
clean package
Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] Spark Project Parent POM
[INFO] Spark Project Networking
[INFO] Spark Project Shuffle Streaming Service
[INFO] Spark Project Core
[INFO] Spark Project Bagel
[INFO] Spark Project GraphX
[INFO] Spark Project Streaming
[INFO] Spark Project Catalyst
[INFO] Spark Project SQL
[INFO] Spark Project ML Library
[INFO] Spark Project Tools
[INFO] Spark Project Hive
[INFO] Spark Project REPL
[INFO] Spark Project YARN Parent POM
[INFO] Spark Project YARN Stable API
[INFO] Spark Project Assembly
[INFO] Spark Project External Twitter
[INFO] Spark Project External Flume Sink
[INFO] Spark Project External Flume
[INFO] Spark Project External MQTT
[INFO] Spark Project External ZeroMQ
[INFO] Spark Project External Kafka
[INFO] Spark Project Examples
[INFO] Spark Project YARN Shuffle Service
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Spark Project Parent POM 1.2.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) # spark-parent ---
[INFO] Deleting C:\Spark\spark-1.2.0\target
[INFO]
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-versions) # spark-parent
---
[INFO]
[INFO] --- build-helper-maven-plugin:1.8:add-source (add-scala-sources) # spark-
parent ---
[INFO] Source directory: C:\Spark\spark-1.2.0\src\main\scala added.
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) # spark-parent --
-
[INFO]
[INFO] --- scala-maven-plugin:3.2.0:compile (scala-compile-first) # spark-parent
---
[INFO] No sources to compile
[INFO]
[INFO] --- build-helper-maven-plugin:1.8:add-test-source (add-scala-test-sources
) # spark-parent ---
[INFO] Test Source directory: C:\Spark\spark-1.2.0\src\test\scala added.
[INFO]
[INFO] --- scala-maven-plugin:3.2.0:testCompile (scala-test-compile-first) # spa
rk-parent ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-dependency-plugin:2.9:build-classpath (default) # spark-parent
---
[INFO] Wrote classpath file 'C:\Spark\spark-1.2.0\target\spark-test-classpath.tx
t'.
[INFO]
[INFO] --- gmavenplus-plugin:1.2:execute (default) # spark-parent ---
[INFO] Using Groovy 2.3.7 to perform execute.
[INFO]
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) # spark-p
arent ---
[INFO]
[INFO] --- maven-shade-plugin:2.2:shade (default) # spark-parent ---
[INFO] Including org.spark-project.spark:unused:jar:1.0.0 in the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO]
[INFO] --- maven-source-plugin:2.2.1:jar-no-fork (create-source-jar) # spark-par
ent ---
[INFO]
[INFO] --- scalastyle-maven-plugin:0.4.0:check (default) # spark-parent ---
[WARNING] sourceDirectory is not specified or does not exist value=C:\Spark\spar
k-1.2.0\src\main\scala
Saving to outputFile=C:\Spark\spark-1.2.0\scalastyle-output.xml
Processed 0 file(s)
Found 0 errors
Found 0 warnings
Found 0 infos
Finished in 32 ms
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Spark Project Networking 1.2.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) # spark-network-common_2
.10 ---
[INFO] Deleting C:\Spark\spark-1.2.0\network\common\target
[INFO]
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-versions) # spark-networ
k-common_2.10 ---
[INFO]
[INFO] --- build-helper-maven-plugin:1.8:add-source (add-scala-sources) # spark-
network-common_2.10 ---
[INFO] Source directory: C:\Spark\spark-1.2.0\network\common\src\main\scala adde
d.
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) # spark-network-c
ommon_2.10 ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # spark-netw
ork-common_2.10 ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory C:\Spark\spark-1.2.0\network\common\s
rc\main\resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- scala-maven-plugin:3.2.0:compile (scala-compile-first) # spark-networ
k-common_2.10 ---
[WARNING] Zinc server is not available at port 3030 - reverting to normal increm
ental compile
[INFO] Using incremental compilation
[INFO] compiler plugin: BasicArtifact(org.scalamacros,paradise_2.10.4,2.0.1,null
)
[INFO] Compiling 42 Java sources to C:\Spark\spark-1.2.0\network\common\target\s
cala-2.10\classes...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 5.267 s]
[INFO] Spark Project Networking ........................... FAILURE [ 1.922 s]
[INFO] Spark Project Shuffle Streaming Service ............ SKIPPED
[INFO] Spark Project Core ................................. SKIPPED
[INFO] Spark Project Bagel ................................ SKIPPED
[INFO] Spark Project GraphX ............................... SKIPPED
[INFO] Spark Project Streaming ............................ SKIPPED
[INFO] Spark Project Catalyst ............................. SKIPPED
[INFO] Spark Project SQL .................................. SKIPPED
[INFO] Spark Project ML Library ........................... SKIPPED
[INFO] Spark Project Tools ................................ SKIPPED
[INFO] Spark Project Hive ................................. SKIPPED
[INFO] Spark Project REPL ................................. SKIPPED
[INFO] Spark Project YARN Parent POM ...................... SKIPPED
[INFO] Spark Project YARN Stable API ...................... SKIPPED
[INFO] Spark Project Assembly ............................. SKIPPED
[INFO] Spark Project External Twitter ..................... SKIPPED
[INFO] Spark Project External Flume Sink .................. SKIPPED
[INFO] Spark Project External Flume ....................... SKIPPED
[INFO] Spark Project External MQTT ........................ SKIPPED
[INFO] Spark Project External ZeroMQ ...................... SKIPPED
[INFO] Spark Project External Kafka ....................... SKIPPED
[INFO] Spark Project Examples ............................. SKIPPED
[INFO] Spark Project YARN Shuffle Service ................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.048 s
[INFO] Finished at: 2015-02-09T10:17:47+08:00
[INFO] Final Memory: 49M/331M
[INFO] ------------------------------------------------------------------------
[**ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.0:compi
le (scala-compile-first) on project spark-network-common_2.10: wrap: java.io.IOE
xception: Cannot run program "javac": CreateProcess error=2, The system cannot f
ind the file specified -> [Help 1]**
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e swit
ch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please rea
d the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionE
xception
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :spark-network-common_2.10
I had first installed JRE instead of JDK. My environment variables still referenced the JRE folder, and so it couldn't find the javac.exe binary.
A 'quirk' with Spark builds is that it can download its own version of Maven if it determines it is required.
When you run ./build/mvn clean package you are not running Maven directly, you are running a Spark proprietary script. The first thing that script does is check if your mvn --version is new enough for the version that the project determines it needs (which is set in the pom.xml file).
This is an important point because if you're running an old version of maven, Spark may download an additional maven version and install it and use that instead.
Some key things:
When you run ./build/mvn clean package, check which version of maven it is using
When maven runs it does its own traversal to figure out which JAVA_HOME is used
Before trying to run the spark build, check JAVA_HOME is set as a variable
Check that the JAVA_HOME version is a full jdk, not just a jre
Update your Maven to the latest version (or check it is at least as new as the version set by in the pom.xml in the root directory
Thanks
For this problem you need to set your java environment path correctly in .bashrc file. Then you need to build maven correct on set maven path for that check mvn -version.
Then it will build automaticaly without error.
Context: I want to compile and test all modules in a multi-module project but if any fail either compilation or tests I want the overall build to fail.
Default configurations either stop on the first failure or skip modules after a test failure
Running:
mvn clean install
stops at the first failing module.
If you add:
mvn clean install -fae //fail at end
then all modules are run, but if tests fail then any dependent modules are skpped:
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] Module A ............................................. SUCCESS [15.210s]
[INFO] Module B ............................................. SUCCESS [10.923s]
[INFO] Module C ............................................. FAILED [1.731s]
[INFO] Module D ............................................. SUCCESS [3.791s]
[INFO] Module E ............................................. SUCCESS [1.488s]
[INFO] Module F ............................................. SKIPPED (dependency build failed or was skipped)
[INFO] Module G ............................................. SKIPPED (dependency build failed or was skipped)
[INFO] Module H ............................................. SKIPPED (dependency build failed or was skipped)
[INFO] Module I ............................................. SUCCESS [1.690s]
[INFO] -----------------------------------------
Another option to force all modules to compile is:
mvn clean install -fn //fail never
but this results in the build passing when tests fail
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] Module A ............................................. SUCCESS [15.210s]
[INFO] Module B ............................................. SUCCESS [10.923s]
[INFO] Module C ............................................. FAILED [1.731s]
[INFO] Module D ............................................. SUCCESS [3.791s]
[INFO] Module E ............................................. SUCCESS [1.488s]
[INFO] Module F ............................................. SUCCESS [9.062s]
[INFO] Module G ............................................. SUCCESS [16.324s]
[INFO] Module H ............................................. SUCCESS [4.032s]
[INFO] Module I ............................................. SUCCESS [1.690s]
[INFO] ------------------------------------------------------------------------
[INFO] Error for project: Module C (during install)
[INFO] ------------------------------------------------------------------------
[INFO] There are test failures.
Please refer to C:\MavenBuildDir\ModuleC\surefire-reports for the
individual test results.
[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] + Ignoring failures
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30 minutes 38 seconds
[INFO] Finished at: Fri May 23 16:42:08 BST 2014
[INFO] Final Memory: 39M/185M
Can anyone advise a set of options to achieve the following:
compile all modules
run tests on all modules
If a module's tests fail but the code compiles dependent modules still get compiled and tested
Responses much appreciated - otherwise we have to run the tests repeatedly on the build server if there are multiple issues - burning a lot of time.
I would suggest to use:
mvn -Dmaven.test.failure.ignore=true --fail-at-end clean install
I would suggest to split it into two mvn calls:
mvn clean compile
mvn -fae install
The first call will fail, if there are compile errors. The second call will reuse the compiled .class-files, since "clean" is omitted. It will fail at the end, if there are test failures. But compilation has already been finished for ALL modules.
Here is a different approach: parse maven output. So either you
pipe it into some grep like command:
mvn clean install -fn | perl -pe 'END { exit $status } $status=1 if /BUILD FAILURE/;'
See https://github.com/backuitist/maven-fae-seriously
configure your CI server to do it:
See https://stackoverflow.com/a/28683700/303737