How to set JVM arguments when running mvn spring-boot:build-image command in Spring Boot 2.3.0 to generate a docker image? - spring

I am trying to build a docker image using the latest version of Spring Boot (2.3.0) . All I have to create an image now is run the command mvn:spring-boot:build-image . This will create a docker image for me. How do I set the JVM arguments (Max , Min heap sizes) in this case?

As of today, you can't set JVM arguments in spring-boot:build-image.
Spring boot build image uses Packeto internally and it accepts following 4 environment variables as mentioned in bellsoft-liberica GitHub.
"BP_JVM_VERSION" : "13.0.1",
"BPL_JVM_HEAD_ROOM" : "10",
"BPL_JVM_LOADED_CLASS_COUNT" : "35",
"BPL_JVM_THREAD_COUNT" : "10"
As alternate option, you can pass JVM arguments when you run the image.
docker run -p 8080:8080 --env JAVA_OPTS="-Xmx300m -Xms200m" -t youImageName
If using Kubernetes, you can configure JVM options at deployment level.
spec:
containers:
- name: yourapp
image: image path
ports:
- containerPort: 8080
env:
- name: SPRING_PROFILES_ACTIVE
value: "prod"
- name: BPL_JVM_HEAD_ROOM
value: "2"
- name: BPL_JVM_LOADED_CLASS_COUNT
value: "35"
- name: BPL_JVM_THREAD_COUNT
value: "10"
- name: JAVA_OPTS
value: >-
-XX:ReservedCodeCacheSize=40M
-XX:MaxMetaspaceSize=60M
-Xlog:gc
-Xms34m
-Xmx40m
-Xss256k
-XX:MaxRAM=150M
There's a GitHub issue open in Spring boot repository discussing this Failed to change JVM arguments for buildpacked image

Build Back Environment Variables
I tried setting JVM arguments when I started using Spring Boot 2.3 & BuildPacks in November 2020, got nowhere and gave up / put it to one side.
Two weeks ago I picked it up again and purely by chance found this: https://github.com/paketo-buildpacks/environment-variables
Basically, you prefix your environment variable with BPE_APPEND_ and this triggers the Environment Variables Build Pack to append your value to the environment variable.
NB: JAVA_TOOL_OPTIONS is what you want here, not JAVA_OPTS.
I needed to attach a Java Agent to monitor our microservices and something like this build.gradle snippet was what worked:
bootBuildImage {
environment = [
'BPE_DELIM_JAVA_TOOL_OPTIONS' : ' ',
'BPE_APPEND_JAVA_TOOL_OPTIONS' : '-javaagent:my-java-agent.jar'
]
}
I used BPE_DELIM_JAVA_TOOL_OPTIONS to make sure a space was added to the existing value of JAVA_TOOL_OPTIONS before my value was appended (the buildpack also allows you to override or prepend to existing value - see their README).
PS: my value was more like '-javaagent:my-java-agent-${some-dynamic-version}.jar', so I needed double quotes, but that made it a Gradle String which didn't work so I had to write this instead "-javaagent:my-java-agent-${some-dynamic-version}.jar".toString().

None of these answers quite get it right. Here's the full situation at the time of writing.
It's important to understand that there are two different times where code runs: at build time and at runtime.
You can set env variables at build time using Spring Boot in your Maven or Gradle config. You just give it a list of env variables to set. See here https://docs.spring.io/spring-boot/docs/2.3.x/maven-plugin/reference/html/#build-image-customization. Those will only be set at build time though. Not what you want to set things like -Xmx.
If you want env variables set at runtime, then it's not something Spring Boot or the buildpacks can control. It's not a failing of either, it's just that neither are involved in actually running your app image/containers.
To pass env variables to your app image/containers at runtime, you need to use the facilities of whatever is running your app image, like Docker or Kubernetes. For Docker, it's just docker run -e foo=bar .... For Kubernetes, it's in the pod spec.
FYI. Any env variable that is prefixed with BPL_ means it only applies at run time (the L is for launch, which is what buildpacks term the runtime environment). Thus it would only make sense to set those when your app runs.
The environment variables buildpack, mentioned in the other answers, is a mix of the two. If you set env variables using option #1 above but prefixed in a certain way, this buildpack will see them and turn them into env variables that are embedded into the image. This will have the effect of setting them at runtime.
This works well if you want to set default values on your images. Then users of your image won't need to set them unless they need to deviate from the default values you, the image author set. Having said that, you would never want to use this for anything that is sensitive or secret. This is because the env variable and value end up in the image and thus anyone with access to the image can see that value.
To answer the question:
How do I set the JVM arguments (Max , Min heap sizes) in this case?
If you want to set default values that your image users can override, if needed, then you want the environment variables buildpack.
Ex:
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<image>
<env>
<BPE_APPEND_JAVA_TOOL_OPTIONS>-Xss256k</BPE_APPEND_JAVA_TOOL_OPTIONS>
<BPE_DELIM_JAVA_TOOL_OPTIONS> </BPE_DELIM_JAVA_TOOL_OPTIONS>
</env>
</image>
<layers>
<enabled>true</enabled>
</layers>
</configuration>
</plugin>
You'll see output like this:
[INFO] [creator] Paketo Environment Variables Buildpack 3.1.1
[INFO] [creator] https://github.com/paketo-buildpacks/environment-variables
[INFO] [creator] Launch Configuration:
[INFO] [creator] $BPE_<NAME> prepend value to $NAME, delimiting with OS path list separator
[INFO] [creator] $BPE_APPEND_<NAME> append value to $NAME
[INFO] [creator] $BPE_DEFAULT_<NAME> set default value for $NAME
[INFO] [creator] $BPE_DELIM_<NAME> set delimeter to use when appending or prepending to $NAME
[INFO] [creator] $BPE_OVERRIDE_<NAME> set $NAME to value
[INFO] [creator] $BPE_PREPEND_<NAME> prepend value to $NAME
[INFO] [creator] Environment Variables: Contributing to layer
[INFO] [creator] Writing env.launch/JAVA_TOOL_OPTIONS.append
When you run an instance of your image, the values you set should get passed through to the container, unless the user overrides them.
> docker run -it apps/maven
Setting Active Processor Count to 6
Calculating JVM memory based on 9057276K available memory
Calculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx8699636K -XX:MaxMetaspaceSize=88839K -XX:ReservedCodeCacheSize=240M (Total Memory: 9057276K, Thread Count: 50, Loaded Class Count: 13271, Headroom: 0%)
Adding 129 container CA certificates to JVM truststore
Spring Cloud Bindings Enabled
Picked up JAVA_TOOL_OPTIONS: -Djava.security.properties=/layers/paketo-buildpacks_bellsoft-liberica/java-security-properties/java-security.properties -agentpath:/layers/paketo-buildpacks_bellsoft-liberica/jvmkill/jvmkill-1.16.0-RELEASE.so=printHeapHistogram=1 -Xss256k -XX:ActiveProcessorCount=6 -XX:MaxDirectMemorySize=10M -Xmx8699636K -XX:MaxMetaspaceSize=88839K -XX:ReservedCodeCacheSize=240M -Dorg.springframework.cloud.bindings.boot.enable=true
If you want to set the values, then just set them at runtime. For example: docker run -e JAVA_TOOL_OPTIONS '-Xss256k' ... or in your Kubernetes pod spec like this.
apiVersion: v1
kind: Pod
metadata:
name: my-cool-app
labels:
purpose: my-cool-app-label
spec:
containers:
- name: my-cool-app
image: my-cool-image
env:
- name: JAVA_TOOL_OPTIONS
value: "-Xss256k"
You'll see similar output when the container runs and Picked up JAVA_TOOL_OPTIONS: will show the values you set, -Xss256k from this example.

As #jeremyt suggests, you can override the value of this property using BPE_OVERRIDE_BPL_JVM_THREAD_COUNT build variable. Example:
bootBuildImage {
environment('BPE_OVERRIDE_BPL_JVM_THREAD_COUNT', '150')
}
Sadly, as of now, BPE_DEFAULT_BPL_JVM_THREAD_COUNT does not work (the value is ignored). The downside of BPE_OVERRIDE_BPL_JVM_THREAD_COUNT is that it ignores the environment variable set for runtime (in docker run).

Related

Google cloud build with pack and secrets manager not accessing environment variables

I'm using a standard gcr.io/k8s-skaffold/pack build function to build my app for google cloud run using google cloud build.
In my cloudbuild.yaml I load 2 secrets from google secrets manager and pass it to the build function. The google cloud build has access to those secrets, otherwise I would get an error message for this (I got this kind of error at the beginning when setting up the build, now it seems to have access).
However, it seems like the environment variables don't get set.
I think that it might be a syntactical problem of how I try to pass the variables.
This is the stripped down cloudbuild.yaml
steps:
- name: gcr.io/k8s-skaffold/pack
args:
- build
- '$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA'
- '--builder=gcr.io/buildpacks/builder:v1'
- '--network=cloudbuild'
- '--path=.'
- '--env=SEC_A=$$SEC_A'
- '--env=SEC_B=$$SEC_B'
secretEnv: ['SEC_A', 'SEC_B']
id: Buildpack
entrypoint: pack
availableSecrets:
secretManager:
- versionName: projects/<pid>/secrets/SEC_A/versions/latest
env: SEC_A
- versionName: projects/<pid>/secrets/SEC_B/versions/latest
env: SEC_B
An Error message that I hacked into the build for checking shows me that the env var is empty during this build step.
I tried using $, $$ (as seen above), &&, ${...}, for substitution. But maybe the problem lies somewhere else.
Yes, it's a common issue and a trap on Cloud Build. In fact, your secrets can't be read if you use the args[] arrays to pass argument. you have to use the script mode, like that
steps:
- name: gcr.io/k8s-skaffold/pack
entrypoint: bash
args:
- -c
- |
pack build $_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA --builder=gcr.io/buildpacks/builder:v1 --network=cloudbuild --path=. --env=SEC_A=$$SEC_A --env=SEC_B=$$SEC_B
secretEnv: ['SEC_A', 'SEC_B']
id: Buildpack

How can I create several executors for a job in Circle CI orb?

NOTE: The actual problem I am trying to solve is run testcontainers in Circle CI.
To make it reusable, I decided to extend the existing orb in my organisation.
The question, how can I create several executors for a job? I was able to create the executor itself.
Executor ubuntu.yml:
description: >
The executor to run testcontainers without extra setup in Circle CI builds.
parameters:
# https://circleci.com/docs/2.0/configuration-reference/#resource_class
resource-class:
type: enum
default: medium
enum: [medium, large, xlarge, 2xlarge]
tag:
type: string
default: ubuntu-2004:202010-01
resource_class: <<parameters.resource-class>>
machine:
image: <<parameters.tag>>
One of the jobs itself:
parameters:
executor:
type: executor
default: openjdk
resource-class:
type: enum
default: medium
enum: [small, medium, medium+, large, xlarge]
executor: << parameters.executor >>
resource_class: << parameters.resource-class >>
environment:
# Customize the JVM maximum heap limit
MAVEN_OPTS: -Xmx3200m
steps:
# Instead of checking out code, just grab it the way it is
- attach_workspace:
at: .
# Guessing this is still necessary (we only attach the project folder)
- configure-maven-settings
- cloudwheel/fetch-and-update-maven-cache
- run:
name: "Deploy to Nexus without running tests"
command: mvn clean deploy -DskipTests
I couldn't find a good example of adding several executors, and I assume that I will need to add ubuntu and openjdk for every job. Am I right?
I continue looking into other orbs and documentation but cannot find a similar case to mine.
As it is stated in Circle CI documentation, executors can be defined like this:
executors:
my-executor:
machine: true
my-openjdk:
docker:
- image: openjdk:11
Side note, there can be many executors of any type such as docker, machine (Linux), macos, win.
See, StackOverflow question how to invoke executors from CircleCI orbs.

How to properly set up CircleCI plugin

I am trying to integrate CircleCI plugin with my spring-pet-clinic project. I was following the instruction on CircleCI web page. I have created .circleci folder inside my project root folder.
Inside .circleci I have added config.yml file and copy-pasted config from CircleCI page
My config was like this:
# Use the latest 2.1 version of CircleCI pipeline processing engine,
see https://circleci.com/docs/2.0/configuration-reference/
version: 2.1
# Use a package of configuration called an orb, see
https://circleci.com/docs/2.0/orb-intro/
orbs:
# Declare a dependency on the welcome-orb
welcome: circleci/welcome-orb#0.3.1
# Orchestrate or schedule a set of jobs, see
https://circleci.com/docs/2.0/workflows/
workflows:
# Name the workflow "Welcome"
Welcome:
# Run the welcome/run job in its own container
jobs:
- welcome/run
After I ran the project CircleCI has thrown an error. Especially this one: "Config Processing Error: Don't rerun"
$#!/bin/sh -eo pipefail
# No configuration was found in your project. Please refer to
https://circleci.com/docs/2.0/ to get started with your
configuration.
#
# -------
# Warning: This configuration was auto-generated to show you
the
message above.
# Don't rerun this job. Rerunning will have no effect.
false
Exited with code 1
Spin Up Environment looks like this
Build-agent version 1.0.10572-3ce00c85 (2019-04-
15T22:09:28+0000)
Docker Engine Version: 17.05.0-ce
Kernel Version: Linux b0a81c56acff 4.4.0-144-generic
#170~14.04.1-
Ubuntu SMP Mon Mar 18 15:02:05 UTC 2019 x86_64 Linux
Starting container bash:4.4.19
using image
bash#sha256:9f0a4aa3c9931bd5fdda51b1b2b74a0398a8eabeaf9519d807e010b9d9d41993
Using build environment variables
BASH_ENV=/tmp/.bash_env-5cbebf83d4b030000849b60f-0-build
CI=true
CIRCLECI=true
CIRCLE_BRANCH=master
CIRCLE_BUILD_NUM=5
CIRCLE_BUILD_URL=https://circleci.com/gh/sajmon2325/Spring-
Pet-
Clinic/5
CIRCLE_COMPARE_URL=
CIRCLE_JOB=Build Error
CIRCLE_NODE_INDEX=0
CIRCLE_NODE_TOTAL=1
CIRCLE_PREVIOUS_BUILD_NUM=4
CIRCLE_PROJECT_REPONAME=Spring-Pet-Clinic
CIRCLE_PROJECT_USERNAME=sajmon2325
CIRCLE_REPOSITORY_URL=git#github.com:sajmon2325/Spring-Pet-
Clinic.git
CIRCLE_SHA1=48f6db114b41c338e606de32d8648c64ba5119fd
CIRCLE_SHELL_ENV=/tmp/.bash_env-5cbebf83d4b030000849b60f-0-
build
CIRCLE_STAGE=Build Error
CIRCLE_USERNAME=sajmon2325
CIRCLE_WORKFLOW_ID=2789d93e-f1e4-4c81-93f1-846f7d38c107
CIRCLE_WORKFLOW_JOB_ID=670105ca-617e-445e-9b5e-6ac57f6af8da
CIRCLE_WORKFLOW_UPSTREAM_JOB_IDS=
CIRCLE_WORKFLOW_WORKSPACE_ID=2789d93e-f1e4-4c81-93f1-
846f7d38c107
CIRCLE_WORKING_DIRECTORY=~/project
Using environment variables from project settings and/or
contexts
CIRCLE_JOB=**REDACTED**
So at first I thought that I have only a skeleton of CircleCI configuration, that's why I have edited my config.yml file to look like this (the actual version)
# Java Maven CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-java/ for more details
#
version: 2
jobs:
build:
docker:
# specify the version you desire here
- image: circleci/openjdk:11-browsers-legacy
# Specify service dependencies here if necessary
# CircleCI maintains a library of pre-built images
# documented at https://circleci.com/docs/2.0/circleci-images/
# - image: circleci/postgres:9.4
working_directory: ~/repo
environment:
# Customize the JVM maximum heap limit
MAVEN_OPTS: -Xmx3200m
steps:
- checkout
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "pom.xml" }}
# fallback to using the latest cache if no exact match
is
found
- v1-dependencies-
- run: mvn install -DskipTests
- run: mvn dependency:go-offline
- save_cache:
paths:
- ~/.m2
key: v1-dependencies-{{ checksum "pom.xml" }}
# run tests!
- run: mvn integration-test
But even this is not working. I still have the same error:
$#!/bin/sh -eo pipefail
# No configuration was found in your project. Please refer to https://circleci.com/docs/2.0/ to get started with your configuration.
#
# -------
# Warning: This configuration was auto-generated to show you the message above.
# Don't rerun this job. Rerunning will have no effect.
false
Exited with code 1
I just need to successfully integrate CircleCi plugin with my project. If you need to see my repo, here is the link: https://github.com/sajmon2325/Spring-Pet-Clinic.git
The problem is that .circleci is not in the root of the repository. It is currently in sfg-pet-clinic/, and the CircleCI build process won't find it there.

Configurable Replica Number in Template using ValueFrom

I have a template.yml file that is used when deploying to any OpenShift project. Each project has specific project-props configMap to be used, this is part of our CICD pipeline, so each project has a unique project.props available to it
I would like to be able to control the number of replicas and CPU/Memory limits based on what project I am deploying to. For example a branch testing OpenShift project vs Performance testing OpenShift project would have a different CPU request and limit than an ephemeral OpenShift project.
My template.yml file looks something like this:
// <snip>
spec:
replicas: "${OS_REPLICAS}"
// <snip>
resources:
limits:
cpu: "${OS_CPU_LIMIT}"
memory: "${OS_MEMORY_LIMIT}"
requests:
cpu: "${OS_CPU_REQUEST}"
memory: "${OS_MEMORY_REQUEST}"
// <snip>
parameters:
- name: OS_REPLICAS
displayName: OS Number of Replicas
valueFrom:
configMapKeyRef:
name: project-props
key: os.replicas
// rest of params are similar
My relevant project-props section is:
os.replicas=2
os.cpu.limit=2
os.cpu.request=250m
os.memory.limit=1Gi
os.memory.request=1Gi
When I try to deploy this I get the following error:
quantities must match the regular expression '^([+-]?[0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$'
If I change template.yml to have a parameter defined it works fine
parameters:
- name: OS_REPLICAS
displayName: OS Number of Replicas
value: 2
It seems that valueFrom vs value has a different behavior. Is this impossible to do using valueFrom? Is there another way I can dynamically change spec and resources using a configMap?
The alternative is to deploy and then use oc scale dc <deploy_config_name> --replicas=<number> but it's not very elegant.
Where you have:
spec:
replicas: "${OS_REPLICAS}"
you should have:
spec:
replicas: "${{OS_REPLICAS}}"
With template parameter of:
parameters:
- name: OS_REPLICAS
displayName: OS Number of Replicas
value: 2
See:
https://docs.openshift.org/latest/dev_guide/templates.html#writing-parameters
for use of "${{}}".
What it does is interpret the contents of the parameter as JSON/YAML, rather than a string value. This allows you to supply an integer, which replicas requires.
So you don't need valueFrom, which wouldn't work anyway as that is only usable for environment variables and not arbitrary fields like replicas.
As to trying to set a default for memory and CPU for pods deployed in a project, you should look at having a LimitRange resource defined against the project and set a default.
https://docs.openshift.com/container-platform/3.5/dev_guide/compute_resources.html#dev-limit-ranges
I figured out the answer, it's does not read the values from the file but at least they can be dynamic.
OpenShift has an oc process command that you can be run when using a template.
So this works by doing:
oc process -f <template_name>.yaml -v <param_name>=<param_value>
This will over write the parameter value with the one being inserted by -v.
An actual example would be
oc process -f ./src/main/openshift/service.template.yaml -v OS_REPLICAS=2
You can read more about it OpenShift template documentation
It seems that the OS Origin team does not want to support using files for parameter insertion. You can read more about it here:
https://github.com/openshift/origin/pull/10952
https://github.com/openshift/origin/issues/10687

spring.profiles.include does not work from command line parameter

I made trivial Spring Boot application and packaged it as JAR.
When I call
java -jar target/boot-active-include-0.0.1-SNAPSHOT.jar
I got
No active profile set, falling back to default profiles: default
When I call
echo spring.profiles.include=B > application.properties
java -jar target/boot-active-include-0.0.1-SNAPSHOT.jar
I got
The following profiles are active: B
When I call
rm application.properties
java -Dspring.profiles.include=B -jar target/boot-active-include-0.0.1-SNAPSHOT.jar
I got
No active profile set, falling back to default profiles: default
But I expect The following profiles are active: B
Can I include Spring profile from command line?
Version: 1.5.1.RELEASE
PS. spring.profiles.active works as expected. But I don't want replace active profiles.
I see what's happening now, and I think it's arguably something I missed when making this change.
Prior to that change, spring.profiles.include was only considered when it was used in a configuration file. Following that change, spring.profiles.include is now considered from any source (System properties, command line arguments, etc) as long as spring.profiles.active is also set.
You can see this by running your sample with both properties set:
java -Dspring.profiles.include=alpha -Dspring.profiles.active=bravo -jar target/boot-active-include-0.0.1-SNAPSHOT.jar
Both the alpha and bravo profiles are active:
2017-02-09 11:30:49.467 INFO 40409 --- [ main] my.Example : The following profiles are active: alpha,bravo
I've opened an issue so that we can straighten this out.

Resources