Install package in Docker image created by Spring Boot Maven plugin - spring-boot

My Spring Boot project contains the Spring Boot Maven Plugin which I use for building a Docker image by running mvn spring-boot:build-image.
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>build-image</goal>
</goals>
</execution>
</executions>
</plugin>
When deploying this image to a Docker stack I need to run a healthcheck using the curl command but unfortunately curl is not installed by the default buildpack.
Is it possible to further tweak the image building process so that curl gets installed into the iamge? I couldn't find the necessary information

TLDR;
Install curl into the build image with:
docker run --user="root" --entrypoint launcher my-app:0.0.1-SNAPSHOT "apt-get update && apt-get install curl -y"
Crab container id of the stopped container with docker ps -a:
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2ff7db32825f my-app:0.0.1-SNAPSHOT "launcher 'apt-get u…" 44 minutes ago Exited (0) 44 minutes ago reverent_swanson
Create a new container image based on the one we installed curl into with:
docker commit 2ff7db32825f my-app-with-curl
Fire up a new container defining the correct ENTRYPOINT to start Spring Boot app:
docker run --rm -p 8080:8080 --user="cnb" --entrypoint /cnb/process/web my-app-with-curl
Now curl should be ready inside your container.
Details of the solution:
The reasoning behind Cloud Native Buildpacks (CNBs) & Paketo.io, which are basically abstracted away by the spring-boot-maven-plugins build-image goal, is to free us from the need to write/maintain our own Dockerfiles. So the inversion of this is: It's easy to configure the build process, but it is not easy to change things like installed packages.
The reason is, that those packages are maintained in a so called stack, that manages the used build-time and run-time images. And if the stack doesn't define a Mixin for your OS-level dependency, then you can't simply add another package. It would also not suffice to create your own simple buildpack (I tried this approach). And creating your own stacks, buildpacks and/or builders would also negate the huge benefits that Cloud Native Buildpacks provide! Amongst other things we would be also forced to keep the images updated ourselves...
But there's another solution. As we don't want to create our own stacks/buildpacks, we can tweak the container image which has been created by CNBs/spring-boot-maven-plugin. Because the official docs show us how to hook into the startup process of the produced containers and run shell scripts for example. Let's assume our mvn spring-boot:build-image command produced a container image called my-app:0.0.1-SNAPSHOT.
Then first we install curl into the image with:
docker run --user="root" --entrypoint launcher my-app:0.0.1-SNAPSHOT "apt-get update && apt-get install curl -y"
We need to use --user="root" here in order that the command apt-get update && apt-get install curl -y will run successfully (otherwise we would run into errors like List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied)). This will install curl, but we shouldn't use the resulting container in production. Because our Spring Boot app would run using the root user, which would introduce a variety of security problems. Also we've overwritten the ENTRYPOINT of our container, so it wouldn't be able to start our app.
Therefore we simply start this stopped container with a new command, entrypoint & user! Simply crab the container id of the stopped container with docker ps -a:
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2ff7db32825f my-app:0.0.1-SNAPSHOT "launcher 'apt-get u…" 44 minutes ago Exited (0) 44 minutes ago reverent_swanson
And create a new container image based on the one we installed curl into with:
docker commit 2ff7db32825f my-app-with-curl
Finally fire up a new container based on this new image, defining the correct ENTRYPOINT to start our Spring Boot app and also using the cnb user again (as defined in the Cloud Native Buildpacks):
docker run --rm -p 8080:8080 --user="cnb" --entrypoint /cnb/process/web my-app-with-curl
Off topic but relevant:
There are ongoing discussions if it is desired to install curl in a production container. See this post for example.

I ended up using a mounted volume (/utils for instance) with a static compiled curl (https://github.com/moparisthebest/static-curl) and configured health checker like
/utils/curl http://localhost:8080/actuator/health

Related

Docker image for executing gradle bootBuildImage command

I'm looking for a docker image to build my gradle project which also need a docker engine to execute gradle bootBuildImage command. Any recommandation ?
Thanks,
Dan
If you use the Gradle wrapper scripts (which you should), you can use any image you like as long as it has Java on it. OpenJDK is a good match.
If you don't use the wrapper scripts, you need to have an image with Gradle installed. The official Gradle image should do.
But I think what you are really asking is how to build a docker image inside a container. The bootBuildImage task doesn't need the local Docker cli tools, and only needs to connect to a daemon. That daemon could be running on a remote host, but you can also make it connect to your local host outside the container. To do this, mount the local docker socket.
Here is an example that mounts the current directory inside a container and builds a Docker image in it through the Spring Boot plugin for Gradle:
docker run --rm \
-v gradle-cache:/home/gradle/.gradle \
-v /var/run/docker.sock:/var/run/docker.sock \
-v "$PWD":/home/gradle/project \
-w /home/gradle/project \
gradle:6.7.0-jdk11 \
gradle --no-daemon bootBuildImage
Note that it persists the Gradle home directory in a volume, which means you can't run this command concurrently. Delete the volume when no longer needed with docker volume rm gradle-cache.
Also note that it executes the build as root.

Unable to find docker image locally

I was following this post - the reference code is on GitHub. I have cloned the repository on my local.
The project has got a react app inside it. I'm trying to run it on my local following step 7 on the same post:
docker run -p 8080:80 shakyshane/cra-docker
This returns:
Unable to find image 'shakyshane/cra-docker:latest' locally
docker: Error response from daemon: pull access denied for shakyshane/cra-docker, repository does not exist or may require 'docker login'.
See 'docker run --help'.
I tried login to docker again but looks like since it belongs to #shakyShane I cannot access it.
I idiotically tried npm start too but it's not a simple react app running on node - it's in the container and containers are not controlled by npm
Looks like docker pull shakyshane/cra-docker:latest throws this:
Error response from daemon: pull access denied for shakyshane/cra-docker, repository does not exist or may require 'docker login'
So the question is how do I run this docker image on my local mac machine?
Well this is illogical but still sharing so future people like me don't get stuck.
The problem was that I was trying to run a docker image which doesn't exist.
I needed to build the image:
docker build . -t xameeramir/cra-docker
And then run it:
docker run -p 8080:80 xameeramir/cra-docker
In my case, my image had TAG specified with it and I was not using it.
REPOSITORY TAG IMAGE ID CREATED SIZE
testimage testtag 189b7354c60a 13 hours ago 88.3MB
Unable to find image 'testimage:latest' locally for this command docker run testimage
So specifying tag like this - docker run testimage:testtag worked for me
Posting my solution since non of the above worked.
Working on macbook M1 pro.
The issue I had is that the image was built as arm/64. And I was running the command:
docker run --platform=linux/amd64 ...
So I had to build the image for amd/64 platform in order to run it.
Command below:
docker buildx build --platform=linux/amd64 ...
In conclusion your docker image platform and docker run platform needs to be the same from what I experienced.
In my case, the docker image did exist on the system and still I couldn't run the container locally, so I used the exact image ID instead of image name and tag, like this:
docker run myContainer c29150c8588e
I received this error message when I typed the name/character wrong. That is, "name1\name2" instead of "name1/name2" (wrong slash).
In my case, I saw this error when I had logged in to the dockerhub in my docker desktop. The repo I was pulling was local to my enterprise. Once i logged out of dockerhub, the pull worked.
This just happened to me because my local docker vm on macos ran out of disk space.
I just deleted some old images using docker image prune and it started working correctly again.
shakyshane/cra-docker Does not exist in that user's repo https://hub.docker.com/u/shakyshane/
The problem is you are trying to run an imagen that does not exists. If you are executing a Dockerfile, the image was not created until Dockerfile pass with no errors; so when Dockerfile tries to run the image, it can't find it. Be sure you have no errors in the execution of your scripts.
The simplest answer can be the correct one!.. make sure you have permissions to execute the command, use:
sudo docker run -p 8080:80 shakyshane/cra-docker
In my case, I didn't realise there was a difference between docker run and docker start, and I kept using the run command when I should've been using the start command.
FYI, run is for building and creating the docker container, start is to just start a stopped container
Use -d
sudo docker run -d -p 8000:8000 rasa/duckling
learn about -d here
sudo docker run --help
At first, i build image on mac-m1-pro with this command docker build -t hello_k8s_world:0.0.1 ., when is run this image the issue appear.
After read Master Yi's answer, i realize the crux of the matter and rebuild my images like this docker build --platform=arm64 -t hello_k8s_world:0.0.1 .
Finally,it worked.

Jenkins tutorial maven project (with Docker) fails at Build stage

I'm using the current Jenkins Maven Project tutorial using Docker:
https://jenkins.io/doc/tutorials/build-a-java-app-with-maven/
I keep getting this error at the Build stage:
[simple-java-maven-app] Running shell script
sh: can't create
/var/jenkins_home/workspace/simple-java-maven-app#tmp/durable-bae402a9/jenkins-log.txt:
nonexistent directory
sh: can't create
/var/jenkins_home/workspace/simple-java-maven-app#tmp/durable-bae402a9/jenkins-result.txt:
nonexistent directory
I've tried setting least restrictive permissions with chmod -R 777, chown -R nobody and chown -R 1000 on the listed directories, but nothing seems to work.
This is happening with the jenkins image on Docker version 17.12.0-ce, build c97c6d6 on Windows 10 Professional.
As this is happening with the Maven project tutorial on the Jenkins site, I'm wondering how many others have run into this issue.
I had also the same problem on MacOSX.
After few hours of research, I have finally find the solution.
To solve the problem, it's important to understand that Jenkins is inside a container and when the docker agent inside this container talk to your docker engine, it give path to mount volume matching inner the container. But your docker engine is outer. So to allow to work correctly path inner the container must match the same path outer the container in your host.
To allow working correctly, you need to change 2 things.
docker run arguments
Jenkinsfile docker agent arguments
For my own usage, I used this
docker run -d \
--env "JENKINS_HOME=$HOME/Library/Jenkins" \
--restart always \
--name jenkins \
-u root \
-p 8080:8080 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v $HOME/Library/Jenkins:$HOME/Library/Jenkins \
-v "$HOME":/home \
jenkinsci/blueocean
In the Jenkinsfile
Replace the agent part
agent {
docker {
image 'maven:3-alpine'
args '-v /root/.m2:/root/.m2'
}
by
agent {
docker {
image 'maven:3-alpine'
args '-v <host_home_path>/.m2:/root/.m2'
}
It's quite likely that this issue resulted from a recent change in Docker behaviour, which was no longer being handled correctly by the Docker Pipeline plugin in Jenkins.
Without going into too much detail, the issue was causing Jenkins to no longer be able to identify the container it was running in, which results in the errors (above) that you encountered with these tutorials.
A new version (1.15) of the Docker Pipeline plugin was released yesterday (https://plugins.jenkins.io/docker-workflow).
If you upgrade this plugin on your Jenkins (in Docker) instance (via Manage Jenkins > Manage Plugins), you'll find that these tutorials should start working again (as documented).
The error message means that the directory durable-bae402a9 was not created.
Walk back through the tutorial to find the step that should have created that directory, and make whatever changes are needed to make sure it succeeds.

Build docker image without docker installed

Is it somehow possible to build images without having docker installed. On maven build of my project I'd like to produce docker image, but I don't want to force others to install docker on their machines.
I can think of some virtual box image with docker installed, but it is kind of heavy solution. Is there some way to build the image with some maven plugin only, some Go code or already prepared virtual box image for exactly this purpose?
It boils down to question how to use docker without forcing users to install anything. Either just for build or even for running docker images.
UPDATE
There are some, not really up to date, maven plugins for virtual machine provisioning with vagrant or with vbox. I have found article about building docker images without docker on basel
So far I see two options either I can somehow build the images only or run some VM with docker daemon inside(which can be used not only for builds, but even for integration tests)
We can create Docker image without Docker being installed.
Jib Maven and Gradle Plugins
Google has an open source tool called Jib that is relatively new, but
quite interesting for a number of reasons. Probably the most interesting
thing is that you don’t need docker to run it - it builds the image using
the same standard output as you get from docker build but doesn’t use
docker unless you ask it to - so it works in environments where docker is
not installed (not uncommon in build servers). You also don’t need a
Dockerfile (it would be ignored anyway), or anything in your pom.xml to
get an image built in Maven (Gradle would require you to at least install
the plugin in build.gradle).
Another interesting feature of Jib is that it is opinionated about
layers, and it optimizes them in a slightly different way than the multi-
layer Dockerfile created above. Just like in the fat jar, Jib separates
local application resources from dependencies, but it goes a step further
and also puts snapshot dependencies into a separate layer, since they are
more likely to change. There are configuration options for customizing the
layout further.
Pls refer this link https://cloud.google.com/blog/products/gcp/introducing-jib-build-java-docker-images-better
For example with Spring Boot refer https://spring.io/blog/2018/11/08/spring-boot-in-a-container
Have a look at the following tools:
Fabric8-maven-plugin - http://maven.fabric8.io/ - good maven integration, uses a remote docker (openshift) cluster for the builds.
Buildah - https://github.com/containers/buildah - builds without a docker daemon but does have other pre-requisites.
Fabric8-maven-plugin
The fabric8-maven-plugin brings your Java applications on to Kubernetes and OpenShift. It provides a tight integration into Maven and benefits from the build configuration already provided. This plugin focus on two tasks: Building Docker images and creating Kubernetes and OpenShift resource descriptors.
fabric8-maven-plugin seems particularly appropriate if you have a Kubernetes / Openshift cluster available. It uses the Openshift APIs to build and optionally deploy an image directly to your cluster.
I was able to build and deploy their zero-config spring-boot example extremely quickly, no Dockerfile necessary, just write your application code and it takes care of all the boilerplate.
Assuming you have the basic setup to connect to OpenShift from your desktop already, it will package up the project .jar in a container and start it on Openshift. The minimum maven configuration is to add the plugin to your pom.xml build/plugins section:
<plugin>
<groupId>io.fabric8</groupId>
<artifactId>fabric8-maven-plugin</artifactId>
<version>3.5.41</version>
</plugin>
then build+deploy using
$ mvn fabric8:deploy
If you require more control and prefer to manage your own Dockerfile, it can handle this too, this is shown in samples/secret-config.
Buildah
Buildah is a tool that facilitates building Open Container Initiative (OCI) container images. The package provides a command line tool that can be used to:
create a working container, either from scratch or using an image as a starting point
create an image, either from a working container or via the instructions in a Dockerfile
images can be built in either the OCI image format or the traditional upstream docker image format
mount a working container's root filesystem for manipulation
unmount a working container's root filesystem
use the updated contents of a container's root filesystem as a filesystem layer to create a new image
delete a working container or an image
rename a local container
I don't want to force others to install docker on their machines.
If by "without Docker installed" you mean without having to install Docker locally on every machine running the build, you can leverage the Docker Engine API which allow you to call a Docker Daemon from a distant host.
The Docker Engine API is a RESTful API accessed by an HTTP client such
as wget or curl, or the HTTP library which is part of most modern
programming languages.
For example, the Fabric8 Docker Maven Plugin does just that using the DOCKER_HOST parameter. You'll need a recent Docker version and you'll have to configure at least one Docker Daemon properly so it can securely accept remote requests (there are lot of resources on this subject, such as the official doc, here or here). From then on, your Docker build can be done remotely without having to install Docker locally.
Google has released Kaniko for this purpose. It should be run as a container, whether in Kubernetes, Docker or gVisor.
I was running into the same problems, and I did not find any solution, thus i developed odagrun, it's a runner for Gitlab with integrated registry api, update DockerHub, Microbadger etc.
OpenSource and has a MIT license.
Ideal to create a docker image on the fly, without the need of a docker daemon nor the need of a root account, or any image at all (image: scratch will do), currrently still in development, but i use it every day.
Requirements
project repository on Gitlab
an openshift cluster (an openshift-online-starter will do for most medium/small
extract how the docker image for this project was created:
# create and push image to ImageStream:
build_rootfs:
image: centos
stage: build-image
dependencies:
- build
before_script:
- mkdir -pv rootfs
- cp -v output/oc-* rootfs/
- mkdir -pv rootfs/etc/pki/tls/certs
- mkdir -pv rootfs/bin-runner
- cp -v /etc/pki/tls/certs/ca-bundle.crt rootfs/etc/pki/tls/certs/ca-bundle.crt
- chmod -Rv 777 rootfs
tags:
- oc-runner-shared
script:
- registry_push --rootfs --name=test-$CI_PIPELINE_ID --ISR --config

What is happening when docker-maven plugin tries to build image?

I am running Jenkins in a docker container and Jenkins tries to run my maven build. As part of the build, the docker maven plugin instructs it to build a docker image.
That part of the POM is below.
<plugin>
<groupId>com.spotify</groupId>
<artifactId>docker-maven-plugin</artifactId>
<version>0.3.8</version>
<configuration>
<imageName>example</imageName>
<baseImage>java:latest</baseImage>
<skipDockerBuild>false</skipDockerBuild>
<cmd>["java", "-jar", "myLogThread-jar-with-dependencies.jar"]</cmd>
<resources>
<resource>
<directory>target/</directory>
<include>config.properties</include>
</resource>
<resource>
<directory>${project.build.directory}</directory>
<include>myLogThread-jar-with-dependencies.jar</include>
</resource>
</resources>
</configuration>
</plugin>
The maven build runs until it attempts to build the image, at which point the following error message is spat out:
[INFO] Building image example
[INFO] I/O exception (java.io.IOException) caught when processing request to {}->unix://localhost:80: Permission denied
I can go into the correct directory and the Dockerfile is there.
I can also run sudo docker build . and it will build the image with no issues.
Why is the maven build failing? What request is being made to localhost:80? How can I correct this so that maven can build my image?
Note: I have mounted the docker socket and binary in this container
As mentioned above by Rajith Delantha, this solved the problem for me:
Add: DOCKER_OPTS=' -G jenkins' directly in /etc/default/docker.
Then restart docker service by sudo service docker restart.
This can be resolved by adding DOCKER_HOST environment variable in Jenkins.
Setup your docker daemon like this:
[/etc/sysconfig/docker]
OPTIONS="-H tcp://127.0.0.1:4243"
Jenkins Jobs (Inject environment variables):
DOCKER_HOST=tcp://127.0.0.1:4243
I had the same problem, but in my local machine.
I've got it after reading this comment in Github thread: https://github.com/docker/compose/issues/1214#issuecomment-256774629
It says:
Solution (from
https://docs.docker.com/engine/installation/linux/debian/, does not
only work with Debian):
Add the docker group if it doesn't already exist.
sudo groupadd docker
Add the connected user "${USER}" to the docker group. Change the user
name to match your preferred user. You may have to logout and log back
in again for this to take effect.
sudo gpasswd -a ${USER} docker
Restart the Docker daemon.
sudo service docker restart
after making sure docker ps works from the same user that runs mvn I still had the same problem.
it looks like a bug due to special characters in the image name. I resolved it by removing the dash sign (-) (or any special characters) from the docker image name.
try set the repository to deferent name and check.
<configuration>
<repository>somename</repository>
</configuration>
I was able to solve the problem by combining elements of both upvoted answers.
Set options to use different port in /etc/default/docker.
DOCKER_OPTS="-H tcp://127.0.0.1:4243"
Restart the Docker daemon.
sudo service docker restart
Then build your package.
export DOCKER_HOST=tcp://127.0.0.1:4243
mvn clean package docker:build
I met the issue at bamboo-agent, but I assume the same holds for jenkins.
Add the user running maven to the docker group. Then restart docker AND the service running maven. Group changes are not loaded while the services are running. So in my case:
sudo groupadd docker # if it does not exist
sudo usermod -a -G docker bamboo-user
sudo systemctl restart docker.service
sudo systemctl restart bamboo-agent.service
I had a similar issue when I didn't have the docker daemon running - restart the Docker toolbox and it looks much happier now
I had the same issue with Jenkins.
Added jenkins to docker group
sudo usermod -aG docker jenkins
Then restrat jenkins.

Resources