Caching Maven with Docker and Kotlin - maven

I am trying to warm up my Docker + Maven cache before building a Kotlin project.
As suggested by many Maven/Docker threads, my docker file looks like this:
COPY pom.xml .
RUN mvn dependency:go-offline
COPY ./src/ src/
RUN mvn package
The thought is that if I change a file in the ./src directory, I want docker cache to skip maven's lengthy dependency download page.
My problem is that the mvn package command still downloads alot of files.
I tried to use mvn -o package (maven offline flag) to diagnose what dependencies are missing, but it just complains that it cannot download dependencies. But I would have expected that the dependencies would allready be downloaded in the previous step. Here are the errors that I get with the "-o" flag:
Step 8/13 : RUN mvn dependency:go-offline
---> Using cache
---> 0334facb9cc9
Step 9/13 : COPY ./src/ src/
---> Using cache
---> 27149a191017
Step 10/13 : RUN mvn -o package
---> Running in 5183eced32ca
Warning: JAVA_HOME environment variable is not set.
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building auth 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # auth ---
[WARNING] The POM for org.apache.maven:maven-core:jar:2.0.6 is missing, no dependency information available
[WARNING] The POM for org.apache.maven:maven-monitor:jar:2.0.6 is missing, no dependency information available
[WARNING] The POM for org.codehaus.plexus:plexus-utils:jar:2.0.5 is missing, no dependency information available
[WARNING] The POM for org.apache.maven.shared:maven-filtering:jar:1.1 is missing, no dependency information available
[WARNING] The POM for org.codehaus.plexus:plexus-interpolation:jar:1.13 is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.882 s
[INFO] Finished at: 2019-07-12T07:02:23+00:00
[INFO] Final Memory: 13M/174M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:2.6:resources (default-resources) on project auth: Execution default-resources of goal org.apache.maven.plugins:maven-resources-plugin:2.6:resources failed: Plugin org.apache.maven.plugins:maven-resources-plugin:2.6 or one of its dependencies could not be resolved: The following artifacts could not be resolved: org.apache.maven:maven-plugin-api:jar:2.0.6, org.apache.maven:maven-project:jar:2.0.6, org.apache.maven:maven-profile:jar:2.0.6, org.apache.maven:maven-artifact-manager:jar:2.0.6, org.apache.maven:maven-repository-metadata:jar:2.0.6, org.apache.maven:maven-plugin-registry:jar:2.0.6, org.apache.maven:maven-core:jar:2.0.6, org.apache.maven:maven-artifact:jar:2.0.6, org.apache.maven:maven-settings:jar:2.0.6, org.apache.maven:maven-model:jar:2.0.6, org.apache.maven:maven-monitor:jar:2.0.6, classworlds:classworlds:jar:1.1-alpha-2, org.codehaus.plexus:plexus-utils:jar:2.0.5, org.apache.maven.shared:maven-filtering:jar:1.1, org.codehaus.plexus:plexus-interpolation:jar:1.13: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.apache.maven:maven-plugin-api:jar:2.0.6 has not been downloaded from it before. -> [Help 1

Try following mvn commands, it saved me from the maven error.
RUN mvn --batch-mode --errors --strict-checksums --threads 1C \
org.apache.maven.plugins:maven-dependency-plugin:3.0.2:go-offline
RUN mvn --batch-mode --errors --offline package

From what I gathered from various sources go-offline not always handles caching packages in a correct way, leaving some of them outside of the local repository (see pull request on GitHub).
In my case, the standard mvn dependency:go-offline used go-offline 2.8 and returned errors. I then tried running mvn org.apache.maven.plugins:maven-dependency-plugin:3.2.0:go-offline but I still had some issues with different packages.
What worked for me was using a different "go-offline" plugin available at GitHub.
Using
mvn de.qaware.maven:go-offline-maven-plugin:resolve-dependencies
finally did the job.

Related

Bamboo: maven-resources-plugin:2.6 or one of its dependencies could not be resolved

I am trying to build a sample Java image using bamboo via a Dockerfile. But I'm getting this error. We have added the proxy settings in settings.xml in ~/.m2 folder.
I have referred to similar questions, and most of the solutions don't suit my situation.
Sending build context to Docker daemon 22.53kB
Step 1/8 : FROM maven:3.5.2-jdk-8-alpine AS MAVEN_TOOL_CHAIN
---> 293423a981a7
Step 2/8 : COPY pom.xml /tmp/
---> Using cache
---> b77d358f80b0
Step 3/8 : COPY src /tmp/src/
---> Using cache
---> d448db75d074
Step 4/8 : WORKDIR /tmp/
---> Using cache
---> 7c7a16ec65ab
Step 5/8 : RUN mvn package
---> Running in cc37fcc7488d
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building my-app 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.453 s
[INFO] Finished at: 2018-08-13T11:02:11Z
[INFO] Final Memory: 10M/240M
[INFO] ------------------------------------------------------------------------
[ERROR] Plugin org.apache.maven.plugins:maven-resources-plugin:2.6 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-resources-plugin:jar:2.6: Could not transfer artifact org.apache.maven.plugins:maven-resources-plugin:pom:2.6 from/to central (https://repo.maven.apache.org/maven2): repo.maven.apache.org: Try again: Unknown host repo.maven.apache.org: Try again -> [Help 1]
Removing intermediate container cc37fcc7488d
The command '/bin/sh -c mvn package' returned a non-zero code: 1
Failing task since return code of [/usr/bin/docker build --force-rm=true --tag=ackris/java-sample /app/bamboosoftware-home/xml-data/build-dir/CIB-CID-JOB1] was 1 while expected 0
It looks like a clear proxy related issue. As aforementioned, we have already appended proxy related information in ~/.m2 folder. Is there something needs to be done on bamboo level which I am forgetting?
PS: I am testing this out inside a corporate network, and we are behind a proxy. I am getting a value of 200 OK when I tried to telnet central Maven repo.

maven plugin sonarqube doesn't resolve it's dependencies

When I try to run the mvn sonar:sonar target maven are unable to run sonar, here is the relevant part of the mvn output:
[INFO] Execute: org.codehaus.sonar:sonar-maven-plugin:3.6:sonar
[WARNING] While downloading javax.xml:jaxrpc:1.1
This artifact has been relocated to javax.xml:jaxrpc-api:1.1.
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Can not execute Sonar
Embedded error: Unable to load the mojo 'org.codehaus.sonar:sonar-maven-plugin:3.6:sonar' in the plugin 'org.codehaus.sonar:sonar-maven-plugin'. A required class is missing: Lorg/apache/maven/shared/dependency/tree/DependencyTreeBuilder;
org.apache.maven.shared.dependency.tree.DependencyTreeBuilder
It seems like maven doesn't download the required dependency for some reason.
I have a local nexus configured that contains the jar, but i get the same result regardless if I have that profile active in my settings.xml or not.
How should I configure maven in order for it to try to download the dependency?
This was resolved by running:
mvn sonar:sonar -U
Apparently there was some corruption in the .m2 directory.

Does maven need Internet connection?

I try to build a project by maven.I tried
$ mvn archetype:generate -DgroupId=org.thoughtworks.app -DartifactId=mvntest -DarchetypeArtifactId=maven-archetype-quickstart
It worked.And then I tried
$ mvn archetype:generate -DgroupId=org.thoughtworks.app -DartifactId=mvntest
I mean,I delete archetypeArtifactId parameter.It fail,I didn't supply it with Internet.the error message is here.Is this parameter made it need Internet?Forget about other depedency.
[INFO] artifact org.apache.maven.archetypes:maven-archetype-quickstart: checking for updates from central
[WARNING] repository metadata for: 'artifact org.apache.maven.archetypes:maven-archetype-quickstart' could not be retrieved from repository: central due to an error: Error transferring file: repo1.maven.org
[INFO] Repository 'central' will be blacklisted
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] The desired archetype does not exist (org.apache.maven.archetypes:maven-archetype-quickstart:RELEASE)
After you have all the dependencies downloaded on your local repository, then you can run mvn command with -o (offline) option and it should work, unless you don't need any other dependency from the remote repository (when the internet connection is needed).
If you have a repository with all the dependencies needed in your network you can reach on without the outside internet connection, just edit your settings.xml for your intranet dependencies.

maven deploy-file failed with 503: cannot find maven-metadata.xml

I am using Maven to integrate our project with others, maybe in a unpopular way and encountered issue.
We have a project that used to compile with Ant. Ant script is big and awesome, so when we are using Maven for integration, it is decided to keep compiling with Ant.
Now let's say Ant compile output is res-1.0-SNAPSHOT.tar.gz. (any filename can be possible but it IS tar.gz) And I am deploying the file to a nexus-hosted snapshot repository called "snapshots".
I tried to deploy with this command:
mvn deploy:deploy-file \
-DgroupId="com.my-company" \
-DartifactId="res" \
-Dversion="1.0-SNAPSHOT" \
-Dpackaging="tar.gz" \
-Dfile="res-1.0-SNAPSHOT.tar.gz" \
-Durl="http://our-nexus-ip/nexus/content/repositories/snapshots" \
-DrepositoryId="snapshots"
I have a simple settings.xml in ~/.m2 with proxy and server settings. However server settings is not being used in current progress yet, wrong passwords don't get errors.
The output is like this:
[[root#cnbi maven]# ./run.sh
+ mvn deploy:deploy-file -DgroupId=com.my-company -DartifactId=res -Dversion=1.0-SNAPSHOT -Dpackaging=tar.gz -Dfile=res-1.0-SNAPSHOT.tar.gz -Durl=http://135.252.234.142:8081/nexus/content/repositories/snapshots -DrepositoryId=snapshots
Warning: JAVA_HOME environment variable is not set.
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'deploy'.
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Default Project
[INFO] task-segment: [deploy:deploy-file] (aggregator-style)
[INFO] ------------------------------------------------------------------------
[INFO] [deploy:deploy-file]
[INFO] Retrieving previous build number from snapshots
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error retrieving previous build number for artifact 'com.my-company:res:tar.gz': repository metadata for: 'snapshot com.my-company:res:1.0-SNAPSHOT' could not be retrieved from repository: snapshots due to an error: Error transferring file
Server returned HTTP response code: 503 for URL: http://135.252.234.142:8081/nexus/content/repositories/snapshots/com/my-company/res/1.0-SNAPSHOT/maven-metadata.xml
[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 51 seconds
[INFO] Finished at: Tue Jun 12 08:44:13 CST 2012
[INFO] Final Memory: 7M/209M
[INFO] ------------------------------------------------------------------------
You see, it fails to find maven-metadata.xml. It is for certain, the file and its folder "com/my-company" do not exist at all in the repository.
Besides any misuse of Maven you may find, my questions are:
1) am I using Maven in the right way? (deploying tar.gz, using deploy:deploy-file...)
2) are there incorrect parameters?
3) what is maven-metadata.xml for? It is not there but Maven insists to find it -- I guess it is generated, am I missing some steps?
The solution might be stupid, I am really not familiar with Maven. Unfortunately it has to be done... Please, help me out of this.
Maven version is 2.0.11. Let me know if you want to know more.
I had the same problem and there was a bug in our nexus.
Using maven3 (with same settings.xml, pom.xml and .m2 repo) solved our problem and its easier than upgrading nexus.
You can check this bug also.
If you're still stuck with using Ant, I would recommend you at least consider adding Ivy to the picture, as your dependency manager. If I recall correctly, it was able to update maven-metadata.xml files in the repository.
Have a look at this example.

What is Maven dependency:purge-local-repository supposed to do?

I'm trying to purge the local repository of a project dependency before launching releasing it in order to make sure every dependency required is on the central repository and is downloaded from it.
In the project folder (containing the pom.xml), I launch the following command:
mvn clean dependency:purge-local-repository -DreResolve=false -Dverbose=true
The project's POM is very simple and just have a declared dependency to junit:junit:3.8.1
The command's output give me:
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building corelib-api 0.1.2-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) # corelib-api ---
[INFO] Deleting d:\Users\fpaillard\git-repositories\TEST_CORELIB\corelib-api\target
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building corelib-api 0.1.2-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.1:purge-local-repository (default-cli) # corelib-api ---
[WARNING] Missing POM for junit:junit:jar:3.8.1
[INFO] Skipping: corelib-api. It cannot be resolved.
[INFO] Nothing to do for project: test:corelib-api:jar:0.1.2-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.953s
[INFO] Finished at: Mon May 14 11:34:40 CEST 2012
[INFO] Final Memory: 6M/15M
[INFO] ------------------------------------------------------------------------
When I look in the local repository (path checked with mvn help:effetive-settings), junit JARs nor POMs are still in .m2/repository/junit/junit/3.8.1 folder.
Isn't dependency:purge-local-repository supposed to delete it?
I don't understand the WARNING of the output above. Why is junit:junit:jar:3.8.1 POM missing? It is still present at .m2/repository/junit/junit/3.8.1/junit-3.8.1.pom
Is the problem related to the INFO line Skipping: corelib-api. It cannot be resolved.? corelib-api is the artifact name of the project I ran mvn dependency:purge-local-repository against.
I know this is old, but I had the same issue, and adding -DactTransitively=false to the command line fixed this issue. I'm unable to tell why it helped, but it did...
I hope this helps.
Looking at the documentation, disabling the actTransitively option causes the purge goal to only purge the dependencies that are named directly by your pom.xml. When it is time for the build, Maven automatically pulls not only your direct dependencies, but all of the TRANSITIVE dependencies down into your local repo as well.
When the purge goal is looking for what to delete, if it finds other dependencies in the dependencies' poms, it transverses those dependencies to figure out the entire tree in your local repository that can be purged. To do this, it at least needs the transitive project's pom.xml. If it cannot find it in the local repo, or if it thinks there might be a more recent version to analyze, it will go to the external repositories to find it.
I don't think it actually tries to download full project content before it starts purging. But since it at least pulls down the projects' pom.xml files, it will complain if it can't find one just like it would if it were resolving dependencies for an actual build.
Besides just preventing Maven from accessing external repositories while purging, another practical reason would be if you have two projects that have the same transitive dependency, and you don't want the purge from one to affect the performance of the other (since the latter will have to download any missing dependencies again).
On the other hand, something to carefully consider is that if you do NOT allow the purge to consider all of the transitive dependencies possible, you stand to leave a set of downstream dependencies sitting in your local repository that you would otherwise have wanted to remove.
I could make a case for saying that the output you are getting is either unnecessary or preventable with another flag like "reportInaccessibleDependencies=false". But unless it is killing your build, I wouldn't say it is anything to worry about.

Resources