Run (Docker) Test Container in gitlab with Maven - maven

I am working in a gitlab ci/cd pipeline. This pipeline executes all of its commands (excluding the deployments) with maven and docker. In this case, I am trying to run integration tests (that are kicked off by maven), which use a test container (for the mysql database). These tests work fine when running locally. However, I am running into issues when I try to run them from gitlab. I am fairly certain my issue is I don't have access to docker (however, in this case, I need my image to have both docker and maven).
Here is the applicable section of gitlab file:
#This phase is only run when merging (to master)
merge_tests:
image: maven:latest
stage: mvn_build_and_test
#TODO can remove services if this does not work without docker image as base image
services:
- docker:stable-dind
#set variables for use later when running maven in script section
variables:
MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode"
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
script:
#ONLY RUN WHEN MERGE REQUEST IS TO MASTER BRANCH
#if ["$CI_MERGE_REQUEST_TARGET_BRANCH_NAME" == "master"]; then
#check docker is installed and running
#- docker --version #this fails if uncommented
- mvn $MAVEN_CLI_OPTS clean install -Pintegration-test-profile
#; fi
#only:
#- merge_requests
Here is the error stack trace on gitlab:
org.testcontainers.containers.ContainerLaunchException: Container startup failed
1958 Caused by: org.testcontainers.containers.ContainerFetchException: Can't get Docker image: RemoteDockerImage(imageNameFuture=java.util.concurrent.CompletableFuture#21baa903[Completed normally], imagePullPolicy=DefaultPullPolicy(), dockerClient=LazyDockerClient.INSTANCE)
1959 Caused by: java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
Before anyone suggests it, this testing works locally (and has worked locally for a long time). Here is part of the sample output:
[INFO] --- maven-failsafe-plugin:2.22.1:integration-test (integration-tests) # reading-comprehension-api ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.connor.retrieving.GetByIsbnIT
-----content ommitted for brevity----------------------------
Tests run: 3, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 10.451 s - in com.connor.adding.AddAssessmentIT
--content ommitted, but test containers start
[INFO] --- maven-failsafe-plugin:2.22.1:verify (integration-tests) # reading-comprehension-api ---
[INFO]
[INFO] --- maven-cucumber-reporting:2.8.0:generate (addCucumberReport) # reading-comprehension-api ---
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
[INFO] About to generate Cucumber report.
[INFO]
[INFO] --- maven-cucumber-reporting:2.8.0:generate (getCucumberReport) # reading-comprehension-api ---
[INFO] About to generate Cucumber report.
[INFO]
[INFO] --- maven-cucumber-reporting:2.8.0:generate (updateCucumberReport) # reading-comprehension-api ---
[INFO] About to generate Cucumber report.
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) # reading-comprehension-api ---
[INFO] Installing /home/connor/Desktop/code/reading-comprehension-api/target/reading-comprehension-api-0.0.1-SNAPSHOT.jar to /home/connor/.m2/repository/com/connor/reading-comprehension-api/0.0.1-SNAPSHOT/reading-comprehension-api-0.0.1-SNAPSHOT.jar
[INFO] Installing /home/connor/Desktop/code/reading-comprehension-api/pom.xml to /home/connor/.m2/repository/com/connor/reading-comprehension-api/0.0.1-SNAPSHOT/reading-comprehension-api-0.0.1-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 38.282 s
[INFO] Finished at: 2019-12-04T22:31:23-05:00
How should i fix this? Should i create my own base image which has maven and docker installed? Can I easily modify my container to use apt get in my container? Any suggestions are welcome, but I would like to have both docker and maven, since they are in my technology stack and working--just a gitlab config issue.
Here is the project location on gitlab: https://gitlab.com/connorbutch/reading-comprehension-api
While i certainly welcome comments on other parts of the project, those should be directed to me via email,in order to keep this page clean (don't put them here)

For anyone who's wondering: You can fix this by adding a few variables to your CI/CD configuration.
The ones I added were:
MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode"
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
#these were added to try and get test containers to work -- if it doesnt, can remove below two
DOCKER_HOST: "tcp://docker:2375"
DOCKER_DRIVER: "overlay2"

For anyone who's are running kubernetes runners the runner must be configured privileged to run dind service:
runners:
## Run all containers with the privileged flag enabled
## This will allow the docker:stable-dind image to run if you need to run Docker
## commands. Please read the docs before turning this on:
## ref: https://docs.gitlab.com/runner/executors/kubernetes.html#using-docker-dind
##
privileged: true
https://docs.gitlab.com/runner/install/kubernetes.html#running-privileged-containers-for-the-runners
Example from values.yaml

Related

How to get CodenameOne Maven Project Running in Netbeans

Thx to Steve for the CN1 ant to maven migration tool and video here. - Has anyone been able to run the Kitchen Sink project in Netbeans after migration? I follow the video instructions with the migration tool and migrate successfully and it runs from command line with ./run.sh but when I go to run it in NetBeans, I had a lot of 501 (https needed) errors. Added umer's code from here and that solved the 501 errors but now get:
Invalid POM for com.codenameone:codenameone-javase:jar:7.0.23, and
No implementation for org.codehaus.plexus.languages.java.jpms.LocationManager was bound.
while locating org.apache.maven.plugin.surefire.SurefirePlugin.
Note invalid POM warning...
------------------------------------------------------------------------
Building kitchensink-common 1.0-SNAPSHOT
------------------------------------------------------------------------
--- properties-maven-plugin:1.0.0:read-project-properties (default) # kitchensink-common ---
--- codenameone-maven-plugin:7.0.23:generate-gui-sources (generate-gui-sources) # kitchensink-common ---
The POM for com.codenameone:codenameone-javase:jar:7.0.23 is invalid, transitive dependencies (if any) will not be available, enable debug logging for more details
Generating GUI sources
No GUI Entries available
Running from command line, invalid POM warning disappears:
[INFO] -----------< com.codename1.demos.kitchen:kitchensink-common >-----------
[INFO] Building kitchensink-common 1.0-SNAPSHOT [4/5]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- properties-maven-plugin:1.0.0:read-project-properties (default) # kitchensink-common ---
[INFO]
[INFO] --- codenameone-maven-plugin:7.0.23:generate-gui-sources (generate-gui-sources) # kitchensink-common ---
Generating GUI sources
No GUI Entries available
Running on apache-maven-3.8.1 on MacOSX
Java version: 1.8.0_172
Suggestions welcomed.
Thx, Mike
I think perhaps the method I'm using to detect the latest version of cn1 has issues. It picked 7.0.23 which was released this morning. But then it was unable to find some of the jars.
Probably it will work if you just try running the project again as it should be fully propagated to maven central now.
Problem was solved updating from Netbeans 8.2 to latest Netbeans version (currently 12.3).

Travis CI SonarCloud error "Project was never analyzed. A regular analysis is required before a branch analysis"

Hi I am automating code coverage on SonarCloud using Travis CI for a Maven application.
Now, running the sonar:sonar command locally submits the report on SoundCloud and I can see it as shown below with Branch master
Now I have also configured .travis.yml file to auto push the report to SoundCloud on each build as below
But when build is triggered by Travis CI it fails with following error
[INFO] Load project branches
[INFO] Load project branches (done) | time=114ms
[INFO] Load project pull requests
[INFO] Load project pull requests (done) | time=116ms
[INFO] Load branch configuration
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 27.884 s
[INFO] Finished at: 2019-05-19T16:47:23Z
[INFO] Final Memory: 93M/496M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar (default-cli) on project safenest-java-server:
Project was never analyzed. A regular analysis is required before a branch analysis`
I am new to SonarCloud and couldn't find much documentation to get help with this. Can somebody explain what exactly A regular analysis is required mean? And how to fix this?
Thanks in advance.
You should check following parameters:
Project key generated during maven build and generated on SonarCloud (or make sure that you set sonar.projectKey property that was generated on SonarCloud).
Check if provided Token value matching the one on SonarCloud (you can provide your own value on project setup page.
The error message means:
Please analyze main branch, before you will analyze other branches
You have to build main repository branch (usually master) with this configuration on Travis. After that you should be able to analyze all other branches. I hit the same problem when I was trying to analyze a feature branch which integrates my project with SonarCloud for the first time. I just merged my feature to master, pushed, and my project has been analyzed successfully. My configuration is similar to yours:
language: java
jdk: openjdk8
env: MVN_VERSION='3.6.0'
addons:
sonarcloud:
organization: $SONAR_ORGANIZATION
token:
secure: $SONAR_TOKEN
before_install:
- wget https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/$MVN_VERSION/apache-maven-$MVN_VERSION-bin.zip
- unzip -qq apache-maven-$MVN_VERSION-bin.zip
- export M2_HOME=$PWD/apache-maven-$MVN_VERSION
- export PATH=$M2_HOME/bin:$PATH
script:
- mvn -B -e verify site
- if [ -n "$SONAR_TOKEN" ]; then
mvn -B -e sonar:sonar -Dsonar.sources=pom.xml,src/main;
fi

Build step 'Invoke top-level Maven targets' marked build as failure

I am getting the below error while i was trigger build job in jenkins .there is no change from last build to this build but i am getting this error .
please help me
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) # avsbe-web-ca ---
[INFO] Deleting /root/.jenkins/workspace/CA-16_PORTAL_BE-NG-WEB/16-PORTAL_BE/avsbe-web-ca/target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # avsbe-web-ca ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 4 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.0:compile (default-compile) # avsbe-web-ca ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 40 source files to /root/.jenkins/workspace/CA-16_PORTAL_BE-NG-WEB/16-PORTAL_BE/avsbe-web-ca/target/classes
Build step 'Invoke top-level Maven targets' marked build as failure
Finished: FAILURE
Follow below steps to solve problem. Its work for me.
Uninstall you maven from running environment.
Install Maven form :
wget http://www-eu.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
Extract tar file using:
tar -zxvf apache-maven-3.3.9-bin.tar.gz
Now set M2_HOME in environment. Open vi /etc/profile
End of file add below lines and save file.
export M2_HOME=/opt/apache-maven-3.3.9/
export M2="$M2_HOME"bin
export PATH=$PATH:$M2
Compile this file using source /etc/profile command.
Your Maven home path is set. Yo can check using echo $M2_HOME command.
Not set /opt/apache-maven-3.3.9/ as a Maven path on jenkins.
Run your project from jenkis.

how to get a jar build in docker maven to my local repository

I use a docker image of maven to build this custom Scribe, the simple OAuth Java lib
Scribe OAuth Java lib
[INFO] Installing /usr/src/app/target/scribe-1.2.3.oauth.jar to /root/.m2/repository/org/scribe/scribe/1.2.3.oauth/scribe-1.2.3.oauth.jar
[INFO] Installing /usr/src/app/pom.xml to /root/.m2/repository/org/scribe/scribe/1.2.3.oauth/scribe-1.2.3.oauth.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17.309 s
[INFO] Finished at: 2017-01-26T15:06:12+00:00
[INFO] Final Memory: 16M/41M
[INFO] ------------------------------------------------------------------------
using this
docker build -t my-maven .
The image build creates the scribe-1.2.3.oauth.jar perfectly. But i need this jar file to build the following project
Alfresco Share OAuth SSO Support
How can i get the jar file from the first build and use it in a local repository for my next build where i again want to use a docker maven to build that project
Left the jar build process out of the docker image build.
Why?
Because a better way to do this is making the maven docker image act as a single command:
docker run -it --rm --name my-maven-project -v "$PWD":/usr/src/mymaven -w /usr/src/mymaven maven:3.2-jdk-7 mvn clean install
I've taken the above example from the maven docker image documentation.
The -v flag attach a local directory to docker image which act as a mount point inside the docker image.
The -w flag indicates to maven where outputs the compiled jar project.

Error cobertura in sonar "Too small initial heap"

I'm using sonar and cobertura (for code coverage). I'm baffled with the problem I'm facing. If I run
mvn sonar:sonar
I saw the following error during executions
[INFO]
[INFO] <<< cobertura-maven-plugin:2.5.1:cobertura (default-cli) # hss-core <<<
[INFO]
[INFO] --- cobertura-maven-plugin:2.5.1:cobertura (default-cli) # hss-core ---
[INFO] Error occurred during initialization of VM
Too small initial heap
[ERROR] Error in Cobertura Report generation: Unable to generate Cobertura Report for project.
org.apache.maven.plugin.MojoExecutionException: Unable to generate Cobertura Report for project.
at org.codehaus.mojo.cobertura.tasks.ReportTask.execute(ReportTask.java:93)
if I run the cobertura plugin directly using the following
mvn cobertura:cobertura
There is no error as above, and display the following output, which I assume was the expected output.
[INFO]
[INFO] <<< cobertura-maven-plugin:2.5.1:cobertura (default-cli) # hss-core <<<
[INFO]
[INFO] --- cobertura-maven-plugin:2.5.1:cobertura (default-cli) # hss-core ---
[INFO] Cobertura 1.9.4.1 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
Cobertura: Loaded information on 39 classes.
Report time: 1035ms
I have set the MAVEN_OPTS (MAVEN_OPTS=-Xms512m) environment variable, but still getting the error.
Can anyone tell me what went wrong, and how to fix this?
It almost feel that you omitted 'm' in your -Xms parameter:
$java -Xms512 -version
Error occurred during initialization of VM
Too small initial heap
Won't hurt to check values for other environment variables to see if there is anything Java-related defined ('set' command on Windows should print them all) - check _JAVA_OPTS, MAVEN_OPTS etc.
Sonar Cobertura Plugin had a specific parameter to increase the memory allocated to the Cobertura process: sonar.cobertura.maxmem
For instance, you can run: mvn sonar:sonar -Dsonar.cobertura.maxmem=512m
See http://docs.codehaus.org/display/SONAR/Code+coverage+plugins#Cobertura

Resources