Jib - How to use environmental variables from base image - maven

I have a base java image with some pre defined java_opts as a environmental variable. How can I use them in plugin?
<plugin>
<groupId>com.google.cloud.tools</groupId>
<artifactId>jib-maven-plugin</artifactId>
<version>1.8.0</version>
<configuration>
<from>
<image>${docker.registry}java:11</image>
</from>
<to>
<image>${docker.registry}portal-backend:${dockerfile.tag}</image>
</to>
<container>
<jvmFlags>
# This will fail
<jvmFlag>$JAVA_OPTS</jvmFlag>
</jvmFlags>
</container>
</configuration>
</plugin>

(Before I start: even if $JAVA_OPTS were expanded when running a Maven build at compile time (it isn't expanded, obviously), <jvmFlag>$JAVA_OPTS<jvmFlag> would still fail, because the entire string value of $JAVA_OPTS containing multiple JVM flags would be passed as a single argument to the java binary. For example, -Xms1024m -Xmx2048m should be passed as two separate flags. The entire string including the whitespace as a single argument is not a valid JVM flag.)
If possible, have the base image define JAVA_TOOL_OPTIONS (note not JAVA_TOOL_OPTS nor JAVA_OPTS). Most JVMs will honor JAVA_TOOL_OPTIONS. See https://stackoverflow.com/a/58715040/1701388 for details. (Also note that, container runtimes (docker, Kubernetes, etc.) can always provide environment variables (and/or override whatever variables defined at build time as container configuration) at runtime. That is, you can dynamically set arguments at runtime.)
Another option is to define your own <entrypoint> to use a shell. (Therefore, you need a base image that includes a shell binary (such as /bin/bash). Note that the default base image prior to Jib 3.0 was Distroless and did not include a shell program. OTOH, Jib 3.0+ doesn't use Distroless.)
In this method, you'll need to know the right Java runtime classpath and the main class to use in your JVM launch command. To help this, starting with Jib >= 3.1, Jib creates two JVM argument files inside a built image; they will hold, respectively, the classpath and the main class inside a built image.
Knowing the entrypoint, you can write a shell script (my-entrypoint.sh):
#!/bin/sh
# Assumes `java` is on PATH in the base image.
exec java $JAVA_OPTS \
-cp $( cat /app/jib-classpath-file ) \
$( cat /app/jib-main-class-file )
Alternatively, if you are on Java 9+, you can leverage the #-argument file:
exec java $JAVA_OPTS -cp #/app/jib-classpath-file #/app/jib-main-class-file
Place my-entrypoint.sh under <project root>/src/main/jib. This is the default directory for Jib's <extraDirectories> feature, and Jib will place src/main/jib/my-entrypoint.sh at the root directory in the container image. Then set the default <entrypoint> to this script:
<container>
<!-- Assumes you have /bin/sh as specified at the top of /my-entrypoint.sh. -->
<entrypoint>/my-entrypoint.sh</entrypoint>
</container>
<!-- You also need to make the script executable. -->
<extraDirectories>
<permissions>
<permission>
<file>/my-entrypoint.sh</file>
<mode>755</mode>
</permission>
</permissions>
</extraDirectories>
Alternatively, if you invoke /bin/sh as below, you don't have to configure <extraDirectories> to make the file executable. This may not look customary; you would normally make the script executable and run it directly. But this is perfectly valid, and there is no difference in terms of actual execution (as long as the shebang of /entrypoint.sh is the same #!/bin/sh).
<container>
<entrypoint>
<arg>/bin/sh</arg>
<arg>/my-entrypoint.sh</arg>
</entrypoint>
</container>
It's also possible to do this without creating a script (basically embedding the entire script in pom.xml and passing it to a shell program). In this case, you don't need to configure <extraDirectories>.
<container>
<entrypoint>
<arg>/bin/sh</arg>
<arg>-c</arg>
<arg>exec java $JAVA_OPTS -cp $( cat /app/jib-classpath-file ) $( cat /app/jib-main-class-file )</arg>
</entrypoint>
</container>

Variables defined as below
Option 1: Java System Properties (VM Arguments)
It's important that the -D parameters are before your application.jar otherwise they are not recognized.
java -jar -Dspring.profiles.active=prod application.jar
Option 2: Program arguments
java -jar application.jar --spring.profiles.active=prod --spring.config.location=c:\config
POM changes :
When using jib as maven plugin - to change the loading of spring config file location : then entryPoint to be passed inside container, but seems jib plugin didn't pick that up . so below changes needs to be done in pom for the argument access for the location :
<plugin>
<groupId>com.google.cloud.tools</groupId>
<artifactId>jib-maven-plugin</artifactId>
<version>2.2.0</version>
<configuration>
<to>
<image>image-url</image>
</to>
<container>
<creationTime>${​​​​​​maven.build.timestamp}​​​​​​</creationTime>
<mainClass>com.package.SpringBootMainClass</mainClass>
<args>
<arg>--spring.config.location=/demo/location/application.yml</arg>
</args>
</container>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>build</goal>
</goals>
</execution>
</executions>
</plugin>
Jib-maven plugin in pom how to pass the arguments , just shown a way through pom , jib don't pick up arguments from entrypoint for spring app, that's why thought of providing another way for the same. as above answer doesn't have it.

My solution is to remove JVM memory parameters from jib-maven-plugin configuration at all. Instead I define JAVA_TOOL_OPTIONS environment variable (example: JAVA_TOOL_OPTIONS='-Xss=512k') for the container (for example: in docker-compose configuration file).
Please note that this environment variable DOESN'T NEED to be defined in a base image at all.

Related

No instances of ... are allowed in the image heap as this class should be initialized at image runtime

I am running a simple Java application (just a REST endpoint, a "Hello" response - exactly the example generated by the Quarkus maven archetype, without modification) with the following stack:
Quarkus (MicroProfile)
JDK 1.8 HotSpot 1.8.0_231-b11
GraalVM 19.3.0 (native-image)
MacOS Catalina
The first error it is (although it does'nt seems to be the main problem):
[code-with-quarkus-1.0.0-SNAPSHOT-runner:1868] setup: 8,524.65 ms
java.lang.NoSuchMethodException: com.oracle.svm.core.jdk.LocalizationSupport.addBundleToCache(java.lang.String)
at java.lang.Class.getDeclaredMethod(Class.java:2130)
at io.quarkus.runner.AutoFeature.beforeAnalysis(AutoFeature.zig:744)
at com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$7(NativeImageGenerator.java:669)
at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:63)
at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:669)
at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:530)
at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:445)
at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
20:17:42,508 INFO [org.jbo.threads] JBoss Threads version 3.0.0.Final
[code-with-quarkus-1.0.0-SNAPSHOT-runner:1868] (typeflow): 34,257.18 ms
[code-with-quarkus-1.0.0-SNAPSHOT-runner:1868] (objects): 19,361.86 ms
[code-with-quarkus-1.0.0-SNAPSHOT-runner:1868] (features): 866.06 ms
[code-with-quarkus-1.0.0-SNAPSHOT-runner:1868] analysis: 56,364.44 ms
The real problem (I think) it is here:
Error: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: No instances of java.net.Inet4Address are allowed in the image heap as this class should be initialized at image runtime. To see how this object got instantiated use -H:+TraceClassInitialization.
Detailed message:
Trace:
at parsing org.wildfly.common.net.CidrAddress.<clinit>(CidrAddress.java:46)
Call path from entry point to org.wildfly.common.net.CidrAddress.<clinit>():
no path found from entry point to target method
So, the first thing I tried was to instruct that the class org.wildfly.common.net.CidrAddress don't be initialized at Image BuildTime, rather initialized at the Image RunTime, with the configuration:
quarkus.native.additional-build-args=--initialize-at-run-time=org.wildfly.common.net.CidrAddress
quarkus.native.enable-jni=true
But, nothing changes. I also tried to activate (as instructed) the parameter -H:+TraceClassInitialization, to find out what class is being initialized that could be causing the problem. No effect! It doesn't make any difference with this parameter (no extra info).
Did someone else go through this scenario? Any hints, ideas?
Thanks!
GraalVM 19.3.0 is not supported by Quarkus yet, it was just released and it will need some changes.
In particular you're hitting this problem:
- https://github.com/quarkusio/quarkus/pull/5353
I expect GraalVM 19.3 to be supported by Quarkus 1.1, but I'm not sure.
I had the same problem with my multistage build:
First I modified my pom.xml:
<build>
<plugins>
<plugin>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-maven-plugin</artifactId>
<version>${quarkus.version}</version>
<executions>
<execution>
<goals>
<goal>native-image</goal>
</goals>
<configuration>
<enableHttpUrlHandler>true</enableHttpUrlHandler>
<enableHttpsUrlHandler>true</enableHttpsUrlHandler>
<enableJni>true</enableJni>
<additionalBuildArgs>--initialize-at-run-time=java.net.Inet6Address -H:+TraceClassInitialization</additionalBuildArgs>
</configuration>
</execution>
</executions>
</plugin>
Still the same error...
Then I changed the base image of my Dockerfile from FROM quay.io/quarkus/centos-quarkus-maven:19.3.0-java8 AS build to FROM quay.io/quarkus/centos-quarkus-maven:19.2.1 AS build Here is my complete Dockerfile.multistage:
FROM quay.io/quarkus/centos-quarkus-maven:19.2.1 AS build
COPY src /usr/src/app/src
COPY pom.xml /usr/src/app/pom.xml
USER root
RUN chown -R quarkus /usr/src/app
USER quarkus
RUN mvn -f /usr/src/app/pom.xml -Pnative clean package
## Stage 2 : create the docker final image
FROM registry.access.redhat.com/ubi8/ubi-minimal
WORKDIR /work/
COPY --from=build /usr/src/app/target/*-runner /work/application
RUN chmod 775 /work
EXPOSE 8080
CMD ["./application", "-Dquarkus.http.host=0.0.0.0"]

How do I access environment variables in xquery?

My program in xquery has a few variables that are based on the environment where the function is being run. For example, dev points to "devserver", test to "testserver", prod to "server", etc.
How do I set this up in my application.xml file and how do I reference these in my .xqy functions?
"workaround" solution 1
use "switch" to determine host:
switch (xdmp:host-name(xdmp:host()))
case <dev environment as string> return "devserver"
case <test environment as string> return "testserver"
.
.
.
default return fn:error(xs:QName("ERROR"), "Unknown host: " || xdmp:host-name(xdmp:host()))
"workaround" solution 2
create an xml file in your project for each host, update your application.xml to place the xml file in a directory depending on the environment name, then refer to the document now built on installation.
application.xml:
<db dbName="!mydata-database">
<dir name="/config/" permissionsMode="set">
<uriPrivilege roles="*_uberuser"/>
<permissions roles="*_uberuser" access="riu"/>
<load env="DEV" root="/config/DEV/" merge="true" include="*.xml"/>
<load env="TEST" root="/config/TEST/" merge="true" include="*.xml"/>
documents located in project directory /config//environment.xml
<environment>
<services>
<damApi>https://stage.mydam.org</damApi>
<dimeApi>https://stage.mydime.org/api/Services</dimeApi>
</services>
</environment>
usage when I need to get the value
fn:doc("/config/environment.xml")/environment/services/damApi/fn:string()
neither solution seems the best to me.
If you use ml-gradle to deploy your project, it can do substitutions in your code. That means you can set up an XQuery library with code like this:
declare variable $ENV = "%%environmentName%%";
You can then import that library wherever you need.
Don't know if MarkLogic supports it, but XQuery 3.1 has functions available-environment-variables() and environment-variable().
You can consider using the tiny “properties” library at https://github.com/marklogic-community/commons/tree/master/properties
We wrote it long, long ago for MarkMail.org with the belief we didn’t want to put the configuration into a database document because configuration should be separate from data. Data get backed up, restored somewhere else, and the new location may not be the same environment as the old.
So instead we did a little hack and put config into the static namespace context (which each group and app server has). The configured prefix is the property name. The configured value is the property value (including type information). Here’s a screen shot from a MarkMail deployment showing it’s a production server, to email on errors, what static file version to serve, and what domain to output as its base.
This approach lets you configure properties administratively (via the Red GUI or REST) and they’re kept separate from the data. They’re statically available to the execution context without extra cost. You can configure them at the Group level or App Server level or both. The library is a convenience wrapper to pull the typed values.
Maybe by now there’s a better way like the XQuery 3.1 functions, but this one has been working well for 10+ years.
Not yet using gradle in our project, I managed to work out how to use maven profiles to find/replace the values I needed based on the environment it was deployed into. I just have to add to the proper profile the plugin to include, the files to update, and what to replace.
pom.xml:
<plugin>
<groupId>com.google.code.maven-replacer-plugin</groupId>
<artifactId>replacer</artifactId>
<version>1.5.2</version>
<executions>
<execution>
<phase>prepare-package</phase>
<goals>
<goal>replace</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<include>**/modules/runTasks.xqy</include>
<include>**/imports/resetKey.xqy</include>
</includes>
<replacements>
<replacement>
<token>https://stage.mydime.org/api/Services</token>
<value>https://www.mydime.org/api/Services</value>
</replacement>
</replacements>
</configuration>
</plugin>

Ant propertyregex task to replace special character in string

My string is C:\tools\jenkins\HOME\workspace\MAL1793_Driver_DIO
I want to replace windows-style directory path "\" with UNIX style path "/"
I have used an Ant propertyregex task in my pom file to achieve this as below.
<execution>
<id>ReplaceWSPath</id>
<phase>process-resources</phase>
<configuration>
<tasks>
<echo>"Updating workspace path"</echo>
<propertyregex
property="WSPath"
input="C:\tools\jenkins\HOME\workspace\MAL1793_Driver_DIO"
regexp="\"
replace="/"
global="true" />
<echo>"workspace Path = ${WSPath}"</echo>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
But after execution I am getting this error:
Problem: failed to create task or type propertyregex
[ERROR] Cause: The name is undefined.
[ERROR] Action: Check the spelling.
[ERROR] Action: Check that any custom tasks/types have been declared.
[ERROR] Action: Check that any <presetdef>/<macrodef> declarations have taken place.
I am using Ant version 1.7. Are there any settings missing?
The <propertyregex> task is not part of Ant, it's part of the third-party Ant-Contrib collection of Ant tasks. The error message you quote indicates that you're at least missing the <taskdef> needed to use Ant-Contrib in your buildfile.
Instructions on how to set up and use Ant-Contrib are available at SourceForge:
First you must install Apache Ant itself, most of the Ant-Contrib
tasks require Ant 1.5 or higher to work properly. You can download Ant
from Apache.
Ant-contrib releases are available at the downloads page. Mailing
lists, CVS and bug trackers can be accessed from the project page.
See the cc tasks for installation instructions for cpptasks. To
install ant-contrib:
Copy ant-contrib-0.3.jar to the lib directory of your Ant
installation. If you want to use one of the tasks in your own project,
add the lines
<taskdef resource="net/sf/antcontrib/antcontrib.properties"/>
to your build file.
Keep ant-contrib-0.3.jar in a separate location. You now have to
tell Ant explicitly where to find it (say in /usr/share/java/lib):
<taskdef resource="net/sf/antcontrib/antcontrib.properties">
<classpath>
<pathelement location="/usr/share/java/lib/ant-contrib-0.3.jar"/>
</classpath>
</taskdef>
You might consider using the built-in Ant <pathconvert> task as an alternative to the <propertyregex> if you don't already have Ant-Contrib or need it for anything else in your build.
I feel using ant script-javascript for this is much simpler
<property name="wsPath" value="C:\tools\jenkins\HOME\workspace\MAL1793_Driver_DIO" />
<script language="javascript">
var wsPath_BackSlash = project.getProperty("wsPath");
println("before: " + wsPath_BackSlash);
var wsPath_FrwdSlash= wsPath_BackSlash.replace("\\", "/");
println("wsPath_FrwdSlash: "+wsPath_FrwdSlash);
project.setProperty("wsPath", wsPath_FrwdSlash);
</script>
<echo message="${wsPath}" />
note: that naming your variable same as argument e.g var wsPath may give error, it gave to me!
courtesy: https://stackoverflow.com/a/16099717/4979331

Maven appassembler program properties

I'm using the maven appassembler plugin to create an executable script that I can deploy to a server. I see that the plugin supports commandLineArguments, but how any thoughts on how I'd set -D properties? Ideally, I'd be able to specify different -D values for each <program> entry. e.g.
<programs>
<program>
<id>one</id>
<mainClass>com.foo.One</mainClass>
<propOne>one</propOne>
</program>
<program>
<id>prime-image-cache</id>
<mainClass>com.foo.Two</mainClass>
<propOne>foo</propOne>
<propTwo>bar</propTwo>
</program>
</programs>
Ah ha! Not obvious, but seems to work.
<program>
<mainClass>com.mycompany.app.App</mainClass>
<id>app</id>
<jvmSettings>
<extraArguments>
<extraArgument>-Dtest=false</extraArgument>
<extraArgument>-Dlog4j.properties=false</extraArgument>
</extraArguments>
</jvmSettings>
</program>

Custom values in Maven pom.properties file

I'm trying to add custom values in the pom.properties file that Maven generates in the META-INF/maven/${groupId}/${artifactId} location
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.4</version>
<configuration>
<archive>
<manifestEntries>
<build>${BUILD_TAG}</build>
</manifestEntries>
<addMavenDescriptor>true</addMavenDescriptor>
<pomPropertiesFile>${project.build.directory}\interface.properties</pomPropertiesFile>
</archive>
</configuration>
</plugin>
The content of the interface.properties files is
# Build Properties
buildId=746
Using the documentation I have pointed the pomPropertiesFile element to an external properties, but the generated pom.properties file still has the default content after running mvn install
What's the correct usage of the pomPropertiesFile element?
EDIT
I believe that the problem lies in org.apache.maven.archiver.PomPropertiesUtil. If you look at the method sameContents in the source it returns true if the properties in the external file are the same as the defaults and false if different. If the result of sameContents is false, then the contents of the external file are ignored.
Sure enough, this has already been logged as a bug
I think you need to place a file under src/main/resources/META-INF/${groupId}/${artifactId}/interface.properties and let maven do the filtering job (configure the filtering). The file will automatically be copied to target/META-INF/maven/${groupId}/${artifactId}/ location.
See https://issues.apache.org/jira/browse/MNG-4998
Maven 3 will resolve property placeholders eagerly when reading pom.xml for all properties values that are available at this time. Modifying these properties at a later time will not affect the values that are already resolved in pom.xml.
However, if property value is not available (there's no default), then placeholder will not be replaced by the value and it can still be processed later as a placeholder. For example, if a plugin will generate some property during the build, or if placeholder is read and processed by a plugin during some build step.

Resources