Using Fabric8 to Deploy a Java Project - maven

I am trying to use Fabric8 to build an image for my Java application. However, I am new and this could be a duplicate question.
I have docker installed and the fabric8 library added via maven.
Below is my initial setup for the fabric maven plugin.
<plugin>
<groupId>io.fabric8</groupId>
<artifactId>docker-maven-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>build</goal>
</goals>
<configuration>
<dockerHost>/var/run/docker.sock</dockerHost>
<images>
<image>
<alias>${project.artifactId}</alias>
<name></name>
<build>
<from>java:8</from>
<maintainer>${project.maintainer}</maintainer>
<dockerFile>${project.basedir}/Dockerfile</dockerFile>
<dockerHost>/var/run/docker.sock</dockerHost>
<ports>
<port>8080</port>
<port>8081</port>
</ports>
</build>
</image>
</images>
</configuration>
</execution>
</executions>
</plugin>
Below is the error message I am getting.
Unable to parse configuration of mojo io.fabric8:docker-maven-plugin:0.30.0:build for parameter dockerHost: Cannot find 'dockerHost' in class io.fabric8.maven.docker.config.BuildImageConfiguration

Try to remove the dockerHost element from the image build configuration. There is no such option for the build configuration.
dockerHost specifies the connection to the Docker host, i.e the machine where the image is going to be built and eventually run. This option is not actually needed
unless the plugin cannot determine it by itself. The discovery sequence is detailed in the Global Configuration section of the docs.
If you build with maven on the machine where the docker daemon runs, you normally don't need this configuration. The plugin will connect to the unix socket /var/run/docker.sock which is the default URL of the docker daemon.
If the requirement is to run the image on a remote host, then you either specify the dockerHost option or the DOCKER_HOST environment variable. On the host, the docker daemon must be configured for remote access.
I hope this helps.

Related

Paketo Buildpack for OpenTelemetry not working with maven spring boot 2.5.12 app

I'm trying to add OpenTelemetry automated instrumentation to our spring boot app but I can't get it working.
The app is deployed as a docker image and the image is created via the spring-boot-maven-plugin.
I'm following these instructions: https://github.com/paketo-buildpacks/opentelemetry
I've added an env section to the spring-boot-maven-plugin config in the pom.xml:
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
<configuration>
<image>
<name>appname</name>
<env>
<BP_OPENTELEMETRY_ENABLED>true</BP_OPENTELEMETRY_ENABLED>
</env>
</image>
</configuration>
</plugin>
I'm not sure this is the correct way to enable it but I'm having a hard time determining whether that step is working or not.
The docker image is created and run with:
mvn clean spring-boot:build-image
docker compose -f app.yml up
I've added environment variables to app.yml file (hostname replaced with XXXXXX):
services:
appname:
image: appname:latest
environment:
- SPRING_PROFILES_ACTIVE=docker-local
- BPE_OTEL_TRACES_EXPORTER=zipkin
- BPE_OTEL_EXPORTER_ZIPKIN_ENDPOINT=http://XXXXXX:9411/api/v2/spans
- BPE_OTEL_SERVICE_NAME=appname
- BPE_OTEL_JAVAAGENT_ENABLED=true
I don't think these environment variables are being set, however because I don't see them when I run:
docker run --entrypoint launcher -it appname:latest bash -c set
I don't see any traces going to zipkin and I don't see anything in the logs.
Without docker, I have everything working fine.
I tried to figure out if I just need to use a more recent version of spring boot but I couldn't find a way to determine that.
I couldn't find any examples of apps that have this working.
Edit to include working solution:
Martin Theiss's solution is correct. Here is the section of the pom.xml that does everything:
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
<configuration>
<image>
<name>appname</name>
<buildpacks>
<buildpack>paketo-buildpacks/java</buildpack>
<buildpack>gcr.io/paketo-buildpacks/opentelemetry</buildpack>
</buildpacks>
<env>
<BP_OPENTELEMETRY_ENABLED>true</BP_OPENTELEMETRY_ENABLED>
<BPE_OTEL_JAVAAGENT_ENABLED>true</BPE_OTEL_JAVAAGENT_ENABLED>
<BPE_OTEL_TRACES_EXPORTER>zipkin</BPE_OTEL_TRACES_EXPORTER>
<BPE_OTEL_EXPORTER_ZIPKIN_ENDPOINT>http://zipkinhost:9411/api/v2/spans</BPE_OTEL_EXPORTER_ZIPKIN_ENDPOINT>
<BPE_OTEL_SERVICE_NAME>appname</BPE_OTEL_SERVICE_NAME>
</env>
</image>
</configuration>
</plugin>
Note that this is sending traces directly to zipkin. Eventually I'll be sending traces to an opentelemetry collector.
Note also that I was wrong to try to put environment variables in the spring app.yml config file. These should be put in the pom.xml as per above.
OpenTelemetry buildpack is not contained in the buildpacks/java. You have to specify it additionally.
<image>
<buildpacks>
<buildpack>paketo-buildpacks/java</buildpack>
<buildpack>gcr.io/paketo-buildpacks/opentelemetry</buildpack>
</buildpacks>
<env>
<BP_OPENTELEMETRY_ENABLED>true</BP_OPENTELEMETRY_ENABLED>
</env>
</image>

Spring Boot Maven Plugin > 2.4.x build image publish on GitLab registry

I'm currently developing a GitLab CI/CD pipeline which compiles, tests and builds a standard Spring Boot application.
I want to package it in a docker image and publish that to the GitLab registry to use it later on.
Spring Boot recently added the build-image goal to its maven plugin which also has the ability to publish the image to a registry.
My problem is, that I can't get the auth to work.
I'm using a maven:3.6.3-jdk-11-slim image for the job with the docker:dind service to have access to a docker daemon.
Building the image runs fine, but publishing fails.
I configured the maven plugin in the project pom to use properties for auth, which will be overwritten by the CLI in my CI/CD Job as follows:
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<docker>
<publishRegistry>
<username>${CI_REGISTRY_USER}</username>
<password>${CI_REGISTRY_PASSWORD}</password>
<url>${CI_REGISTRY}</url>
</publishRegistry>
</docker>
</configuration>
</plugin>
Properties defined in the POM with no value (Will be filled in by CLI call) :
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>11</java.version>
<CI_REGISTRY/>
<CI_REGISTRY_USER/>
<CI_REGISTRY_PASSWORD/>
</properties>
My maven CLI call in the Pipeline/Job uses the GitLab registry variables :
docker image job:
stage: Build
image: maven:3.6.3-jdk-11-slim
services:
- docker:dind
script:
- echo "java.runtime.version=11" > system.properties
- mvn spring-boot:build-image -DCI_REGISTRY=$CI_REGISTRY -DCI_REGISTRY_USER=$CI_REGISTRY_USER -DCI_REGISTRY_PASSWORD=$CI_REGISTRY_PASSWORD -Dspring-boot.build-image.imageName=SpringBootImage_${CI_JOB_ID} -Dspring-boot.build-image.publish=true
I was following the instructions via GitLab and Spring Boot documentation, but cant seem to identify my problem.
GitLab Registry Auth documentation
Spring Boot Maven Plugin image publishing documentation
I know it's been a while but in case others are trying to accomplish this
This works well
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<mainClass>com.foo.Main</mainClass>
</configuration>
<executions>
<execution>
<goals>
<goal>build-image</goal>
</goals>
<phase>deploy</phase>
<configuration>
<image>
<name>${env.CI_REGISTRY_IMAGE}/${project.artifactId}:${project.version} </name>
<publish>true</publish>
</image>
<docker>
<publishRegistry>
<username>${env.CI_REGISTRY_USER}</username>
<password>${env.CI_REGISTRY_PASSWORD}</password>
</publishRegistry>
</docker>
</configuration>
</execution>
</executions>
</plugin>
It uses gitlab-ci built-in env vars to configure everything

How to setup AWS ECS + dockerfile-maven-plugin?

I am trying to setup my project's pom.xml and Maven's settings.xml to automate the process of generating a Docker image and pushing it to my AWS ECS private Docker repository.
In my pom.xml, I added the dockerfile-maven-plugin and configured it as follows:
<plugin>
<groupId>com.spotify</groupId>
<artifactId>dockerfile-maven-plugin</artifactId>
<version>1.3.6</version>
<executions>
<execution>
<id>default</id>
<goals>
<goal>build</goal>
<goal>push</goal>
</goals>
</execution>
</executions>
<configuration>
<finalName>myproject/server</finalName>
<repository>137037344249.dkr.ecr.us-east-2.amazonaws.com/myproject/server</repository>
<tag>${docker.image.tag}</tag>
<serverId>ecs-docker</serverId>
<useMavenSettingsForAuth>true</useMavenSettingsForAuth>
<buildArgs>
<VERSION>${project.version}</VERSION>
<BUILD_NUMBER>${buildNumber}</BUILD_NUMBER>
<WAR_FILE>${project.build.finalName}.war</WAR_FILE>
</buildArgs>
</configuration>
</plugin>
Per the instructions given by dockerfile-maven-plugin, I need to add configurations for my ECS server authentication, but I don't know what username / password I need to provide. I doubt it's my AWS login user/pass.
<servers>
<server>
<id>ecs-docker</id>
<username>where_to_get_this</username>
<password>where_to_get_this</password>
</server>
</servers>
Also, any suggestions to automate this Docker image generation / pushing to my repo in a better way are welcome.
To build the docker image and push it to AWS ECR with Spotify dockerfile-maven-plugin you should:
Install amazon-ecr-credential-helper
go get -u github.com/awslabs/amazon-ecr-credential-helper/ecr-login/cli/docker-credential-ecr-login
Move it to some folder that is already in the execution PATH:
mv ~/go/bin/docker-credential-ecr-login ~/bin/
Add credHelpers section to ~/.docker/config.json file for your Amazon ECR docker repo ID:
{
"credHelpers": {
"<ecr-id>.dkr.ecr.<aws-region>.amazonaws.com": "ecr-login"
},
//...
}
(on Windows remove line "credsStore": "wincred",, if it exists, from this file)
Check that ~/.aws/config has your region
[default]
region = <aws-region>
and ~/.aws/credentials has your keys
[ecr-push-user]
aws_access_key_id = <id>
aws_secret_access_key = <secret>
(More info...)
Add Spotify dockerfile-maven-plugin to your pom.xml:
<properties>
<docker.image.prefix>xxxxxxxxxxxx.dkr.ecr.rrrrrrr.amazonaws.com</docker.image.prefix>
<docker.image.name>${project.artifactId}</docker.image.name>
<docker.image.tag>${project.version}</docker.image.tag>
<docker.file>Dockerfile</docker.file>
</properties>
<build>
<finalName>service</finalName>
<plugins>
<!-- Docker image mastering -->
<plugin>
<groupId>com.spotify</groupId>
<artifactId>dockerfile-maven-plugin</artifactId>
<version>1.4.10</version>
<configuration>
<repository>${docker.image.prefix}/${docker.image.name}</repository>
<tag>${docker.image.tag}</tag>
<dockerfile>${docker.file}</dockerfile>
</configuration>
<executions>
<execution>
<id>default</id>
<phase>package</phase>
<goals>
<goal>build</goal>
<goal>push</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
Make sure that Dockerfile exists, for example:
FROM openjdk:11-jre-slim
VOLUME /tmp
WORKDIR /service
COPY target/service.jar service.jar
ENTRYPOINT exec java -server \
-Djava.security.egd=file:/dev/./urandom \
$JAVA_OPTS \
-jar service.jar
Build and push the image with one command:
mvn package
To login on ECR, you must use the AWS command-line to generate a docker login command, and then login your docker daemon with it. I don't think this use case is handled by any docker maven plugin.
What I do on my project is login my docker daemon before doing the push :
logstring=`aws --profile my-aws-profile ecr get-login --registry-ids my-registry-id`
`$logstring`
This manual step is required in my case because we have a single AWS account that is secured with a hardware token that generate one time use codes, but it is not a problem, since we only need to do it once a day (ECR login lasts for 12 hours), on the days we deploy to ECR (as opposed to those where we only test locally).
So the solutions:
Login manually to ECR, so that your docker pushes work without needing to login from maven.
Add a login step that scripts the external login directly in your pom
Try AWS CodePipeline to build your code directly when you commit, and deploy to ECR (what I recommend if you are not otherwise restricted)
Have fun!
I did not configure anything in my maven settings file.
I usually login using below command
$(aws ecr get-login --no-include-email --region my-region)
then I run the maven commands (docker commands are embedded as a part of maven goals) and it works fine.
For your reference , This is my pom file setup using docker plugin
<plugin>
<groupId>com.spotify</groupId>
<artifactId>docker-maven-plugin</artifactId>
<version>1.1.1</version>
<configuration>
<imageName>${docker.image.prefix}/${project.artifactId}:${project.version}</imageName>
<dockerDirectory>docker</dockerDirectory>
<!-- <serverId>docker-hub</serverId> -->
<registryUrl>https://${docker.image.prefix}</registryUrl>
<forceTags>true</forceTags>
<resources>
<resource>
<targetPath>/</targetPath>
<directory>${project.build.directory}</directory>
<include>${project.build.finalName}.jar</include>
</resource>
</resources>
</configuration>
<executions>
<execution>
<id>tag-image</id>
<phase>package</phase>
<goals>
<goal>build</goal>
</goals>
</execution>
<execution>
<id>push-image</id>
<phase>deploy</phase>
<goals>
<goal>push</goal>
</goals>
<configuration>
<imageName>${docker.image.prefix}/${project.artifactId}:${project.version}</imageName>
</configuration>
</execution>
</executions>
</plugin>

Sprint-boot executable same sources multiple jars

I did the following steps in order to run 2 instances of my spring-boot application on the same host:
build jar with spring-boot maven plugin repackage goal
copy application.jar to application-0.jar on my remote server
created a application-0.conf file with APP_NAME=application-0
start application-0.jar as a service
copy application.jar to application-1.jar on my remote server
created a application-1.conf file with APP_NAME=application-1
start application-1.jar as a service
Then I got the following error upon application-1 start: service application already provided!
So I realized that:
linux service name is hard coded in the jar so no matter how many times you copy the jar en rename it, the service will remain the same (in our case application service)
APP_NAME has no effect on the service name. The APP_NAME in the .conf files were respectively set to application-0 and application-1. I still got the issue.
I read the spring-boot plugin documentation and tried an unsuccessful compination of attach, classifier and embeddedLaunchScriptProperties.initInfoProvides options on different spring-boot maven plugin executions of the same build.
I also tried to encapsulate different executions on separate maven profiles. I still got one original jar file and one single spring-boot jar.
So if anyone knows how to achieve my goal (same source, multiple well configured linux services since I run on the same host) I would really appreciate the help.
Below an example of unsuccessful configuration:
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<id>classic</id>
<goals>
<goal>repackage</goal>
</goals>
<configuration>
<executable>true</executable>
</configuration>
</execution>
<execution>
<id>instance-0</id>
<goals>
<goal>repackage</goal>
</goals>
<configuration>
<finalName>${project.artifactId}-0</finalName>
<executable>true</executable>
<classifier>exec</classifier>
<attach>false</attach>
</configuration>
</execution>
<execution>
<id>instance-1</id>
<goals>
<goal>repackage</goal>
</goals>
<configuration>
<finalName>${project.artifactId}-1</finalName>
<executable>true</executable>
<classifier>exec</classifier>
<attach>false</attach>
</configuration>
</execution>
</executions>
</plugin>
Thank you
I solved the issue by using systemd:
build single executable jar (no multiple spring boot plugin execution, no maven profile, just regular repackage executable goal)
make it available at an absolute path (/var/xxx/application.jar)
create any /etc/systemd/system/application-{i}.service you wish with the following config
[Unit]
Description=application-{i}
After=syslog.target
[Service]
ExecStart=/var/xxx/application.jar --server.port=4500{i} -- logging.file=/var/log/xxx/application-{i}.log
[Install]
WantedBy=multi-user.target
Note the place holders {i} to avoid conflicts on service name, linstening ports and log file.
Systemd solved my issue.
Note: you are only concerned with my issue if you run same instances of your service on the same host. All this config is not needed if you run one service instance per host (common microservice approach).
Hope this helps anyone with the same issue.

Cannot push to remote repo using the (spotify) Docker maven plugin

I've got a Spring Boot project which I want to built an image from and push to a Docker private registry. I've followed Spring Boot official docs for that, using the Spotify Docker Maven plugin. That worked well when publishing my image to a docker local instance using Boot2Docker.
Now I'm trying to do the same with a remote private repo (an unsecured one) and I'm struggling with it. I've followed the plugin documentation for that and that's how my configuration looks like:
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>com.spotify</groupId>
<artifactId>docker-maven-plugin</artifactId>
<version>0.4.9</version>
<configuration>
<imageName>service-discovery</imageName>
<dockerDirectory>src/main/docker</dockerDirectory>
<resources>
<resource>
<targetPath>/</targetPath>
<directory>${project.build.directory}</directory>
<include>${project.build.finalName}.war</include>
</resource>
</resources>
</configuration>
<executions>
<execution>
<id>build-image</id>
<phase>package</phase>
<goals>
<goal>build</goal>
</goals>
</execution>
<execution>
<id>tag-image</id>
<phase>package</phase>
<goals>
<goal>tag</goal>
</goals>
<configuration>
<image>service-discovery</image>
<newName>develop01.mycompany.com:5000/service-discovery</newName>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
My registry is accessible through http://develop01.mycompany.com:5000, but the docker plugin seems not to be able to find it.
Executing mvn clean install throws:
[ERROR] Failed to execute goal com.spotify:docker-maven-plugin:0.4.9:build (buil
d-image) on project service-discovery: Exception caught: java.util.concurrent.Ex
ecutionException: com.spotify.docker.client.shaded.javax.ws.rs.ProcessingExcepti
on: org.apache.http.conn.HttpHostConnectException: Connect to localhost:2375 [lo
calhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused: connec
t -> [Help 1]
Executing mvn docker:tag -DpushImage throws:
[ERROR] Failed to execute goal com.spotify:docker-maven-plugin:0.4.9:tag (defaul
t-cli) on project service-discovery: The parameters 'image', 'newName' for goal
com.spotify:docker-maven-plugin:0.4.9:tag are missing or invalid -> [Help 1]
So the plugin seems not to recognize the parameters into the image goal (in fact, it looks like it's trying to connect to a local Docker instance when executing mvn clean install).
Software:
docker version: Server: 1.11.0
docker-maven-plugin version: 0.4.9
maven version: 3.2.3
A link to the GitHub issue I created
Update
I have set my DOCKER_HOST environment variable to tcp://develop01.mycompany.com:5000 and now the plugin seems to pick it. Still don't understand why I have to do it through an environment variable, when I'm declaring the server name in the execution configuration. All in all, that's the error I'm getting right now:
[ERROR] Failed to execute goal
com.spotify:docker-maven-plugin:0.4.9:build (buil d-image) on project
service-discovery: Exception caught: Request error: GET http
://develop01.mycompany.com:5000/version: 404: HTTP 404 Not Found ->
[Help 1]
In fact, the /version endpoint of my registry returns 404. The /v2 endpoint however, returns an empty JSON. Is the plugin trying to deal with a previous docker version API?

Resources