Quarkus native executable build: high memory consumption - quarkus

I'm building a Quarkus native executable with a multi-stage Docker build as described in Quarkus - Building a Native Executable
My project just includes the Hello World-Example with some added ORM-functionality (so not really a lot of dependencies). The build works fine, but my problem is, that it consumes a lot of memory during build time. That means up to 6 GiB. Build time is also very long in my opinion (~4-6 minutes in total).
The problem starts when I'm building on our CI/CD-infrastructure. We don't have that much memory there and so the build fails with Error: Image build request failed with exit status 137.
Am I doing something wrong or is this just the normal behaviour? Is there a possibility to reduce at least the memory consumption?

Thanks to Ken and Luca Burgazzoli! So, it is normal for GraalVM to use >4GiB of RAM and to take more than 3 minutes.
One can limit memory consumption by specifiying -J-Xmx2G as an additionalBuildArgs-param for the quarkus-maven-plugin. But this may increase build time.

#ben answer is correct but maybe it is useful to be more precise. You have to edit the pom.xml in the getting-started dir and edit the native profile and adding <additionalBuildArgs>-J-Xmx2G</additionalBuildArgs> like this:
<profile>
<id>native</id>
<activation>
<property>
<name>native</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-maven-plugin</artifactId>
<version>${quarkus.version}</version>
<executions>
<execution>
<goals>
<goal>native-image</goal>
</goals>
<configuration>
<enableHttpUrlHandler>true</enableHttpUrlHandler>
<additionalBuildArgs>-J-Xmx2G</additionalBuildArgs>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
...
</plugin>
</plugins>
</build>
</profile>

Now, you can limit memory usage from Quarkus:
In your src/main/resources/application.properties file, just set:
quarkus.native.native-image-xmx=2G
Or just pass this option to maven:
mvn package -Dnative -Dquarkus.native.native-image-xmx=2G

If you are using gradle, edit the build.gradle like this:
.
.
.
compileJava {
options.compilerArgs << '-parameters'
}
buildNative {
additionalBuildArgs = [
'-J-Xmx2G'
]
}
So you can limit the memory usage when building with gradle.

With camel quarkus example using maven, I configured as follows to make it work:
<profiles>
<profile>
<id>native</id>
<activation>
<property>
<name>native</name>
</property>
</activation>
<properties>
<quarkus.package.type>native</quarkus.package.type>
<quarkus.native.additional-build-args>-J-Xmx5G</quarkus.native.additional-build-args>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
Using the option <quarkus.native.additional-build-args>-J-Xmx5G</quarkus.native.additional-build-args>

Related

How can I specify the path in tomee-maven-plugin such that both deploy and undeploy works?

I'm trying to build a pom.xml to handle the deployment and undeployment to a remote TomEE (PluME 7.0.3) server. However, I cannot figure out the correct value to use in the path configuration tag. Here's a working plugin configuration for deploying my application (full-blown EAR, for educational purposes) to the remote TomEE:
<build>
<plugins>
<plugin>
<groupId>org.apache.tomee.maven</groupId>
<artifactId>tomee-maven-plugin</artifactId>
<version>7.0.3</version>
<configuration>
<context>someear</context>
<tomeeClassifier>plus</tomeeClassifier>
<tomeeHost>192.168.100.100</tomeeHost>
<debugPort>8000</debugPort>
<tomeeAjpPort>8009</tomeeAjpPort>
<tomeeHttpPort>8080</tomeeHttpPort>
<tomeeShutdownPort>8005</tomeeShutdownPort>
<path>target/someear-1.0-SNAPSHOT.ear</path>
<useBinaries>true</useBinaries>
</configuration>
</plugin>
</plugins>
</build>
Whereas to undeploy, I would have to configure the plugin like this:
<build>
<plugins>
<plugin>
<groupId>org.apache.tomee.maven</groupId>
<artifactId>tomee-maven-plugin</artifactId>
<version>7.0.3</version>
<configuration>
<context>someear</context>
<tomeeClassifier>plus</tomeeClassifier>
<tomeeHost>192.168.100.100</tomeeHost>
<debugPort>8000</debugPort>
<tomeeAjpPort>8009</tomeeAjpPort>
<tomeeHttpPort>8080</tomeeHttpPort>
<tomeeShutdownPort>8005</tomeeShutdownPort>
<path>someear-1.0-SNAPSHOT</path>
<useBinaries>true</useBinaries>
</configuration>
</plugin>
</plugins>
</build>
Notice the difference in the path configuration. From the command line, this behaves similarly; when the path tag in the pom.xml is omitted, I can deploy and undeploy like this:
mvn tomee:deploy -Dtomee-plugin.archive=target/someear-1.0-SNAPSHOT.ear
mvn tomee:undeploy -Dtomee-plugin.archive=someear-1.0-SNAPSHOT
Has anybody experienced the same behaviour, and found a way to mitigate this? I'd like to have both deploy and undeploy configured completely in the pom.xml without specifying additional parameters when calling mvn. But, as of now, I cannot do that, since tomee:deploy seems to expect a different path than tomee:undeploy.
EDIT
Ok, based on #Old School's edited answer, I can do something like this:
<profiles>
<profile>
<id>deploy</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<my-tomee-maven-plugin-path>target/someear-1.0-SNAPSHOT.ear</tomee-maven-plugin-path>
</properties>
</profile>
<profile>
<id>undeploy</id>
<properties>
<my-tomee-maven-plugin-path>someear-1.0-SNAPSHOT</tomee-maven-plugin-path>
</properties>
</profile>
</profiles>
Then, specify path in tomee-maven-plugin's configuration section like this:
...
<path>${my-tomee-maven-plugin-path}</path>
...
Then, execute maven like this:
mvn tomee:deploy
mvn tomee:undeploy -Pundeploy
Which I consider more convenient than specifying some -D parameters at execution time (YMMV).
However, the perfect solution in my opinion would be if you could configure path in tomee-maven-plugin's configuration such that both tomee:deploy and tomee:undeploy work without further ado.
EDIT2
I discovered another possibility without profiles which I was seemingly unable to find before (based on: http://tomee-openejb.979440.n4.nabble.com/Deployment-to-TomEE-7-0-0-using-tomee-maven-plugin-error-Cannot-open-input-stream-to-server-tp4679142p4679282.html):
<plugin>
<groupId>org.apache.tomee.maven</groupId>
<artifactId>tomee-maven-plugin</artifactId>
<version>7.0.3</version>
<configuration>
<context>${project.artifactId}</context>
<tomeeClassifier>plus</tomeeClassifier>
<context>someear</context>
<tomeeClassifier>plus</tomeeClassifier>
<tomeeHost>192.168.100.100</tomeeHost>
<debugPort>8000</debugPort>
<tomeeAjpPort>8009</tomeeAjpPort>
<tomeeHttpPort>8080</tomeeHttpPort>
<tomeeShutdownPort>8005</tomeeShutdownPort>
<!-- no <path> tag -->
<useBinaries>true</useBinaries>
</configuration>
<executions>
<execution>
<id>deploy-it</id>
<phase>none</phase>
<goals>
<goal>deploy</goal>
</goals>
<configuration>
<path>target/someear-1.0-SNAPSHOT.ear</path>
</configuration>
</execution>
<execution>
<id>undeploy-it</id>
<phase>none</phase>
<goals>
<goal>undeploy</goal>
</goals>
<configuration>
<path>someear-1.0-SNAPSHOT</path>
</configuration>
</execution>
</executions>
</plugin>
Usage:
mvn tomee:deploy#deploy-it
mvn tomee:undeploy#undeploy-it
I use glassfish, not tomcat, but I think the same principles apply here. In GF, deploy requires the full path and the undeploy requires only the package name. For example, glassfish goes something like:
asadmin deploy C:\Projects\Java\helloworld\helloworld.war
and
asadmin undeploy helloworld
There is no path required on undeploy because the file is on the server and once you name it, glassfish/tomcat knows where it is.
EDIT based off comments: My apologies for the glassfish stuff, I wasn't paying attention to you mentioning tomcat and I've edited a bit to reflect that somewhat.
I do understand now, I think, what you're looking for. Check out:
Maven Resource Filtering
It's another SO answer where he does a fantastic job of explaining exactly what you're looking for, I think. I hope that works.

How can I set path in this Maven plugin?

I'm new to Maven plugins, and I need to get this plugin working to run the sencha cmd tool to minify our JavaScript app as part of the daily build process.
Currently the executable tag has a hard coded path, but I'm wondering if I can specify the path as an environment variable, and then access that environment variable in the code below so it can be run on any machine?
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.2.1</version>
<executions>
<execution>
<id>sencha-compile</id>
<phase>compile</phase>
<goals>
<goal>exec</goal>
</goals>
<configuration>
<executable>C:\Sencha\Sencha\Cmd\4.0.2.67\sencha.exe</executable>
<arguments>
<argument>app</argument>
<argument>build</argument>
</arguments>
</configuration>
</execution>
</executions>
</plugin>
Check my Sencha ExtJS 5 + Sencha Cmd 5 + Maven integration example at:
https://github.com/dobromyslov/sencha-extjs-maven
You have to set environment variable:
Export it in console via:
$ export SENCHA_CMD="/path/to/your/Sencha/Cmd/5.0.0.116/sencha"
Also you can add this export statement to your ~/.bashrc or /etc/profile file in order to make it permanent.
Or add new environment variable on Windows.
Set Sencha Cmd build environment:
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<!-- Default build environment -->
<sencha.env>production</sencha.env>
</properties>
<profiles>
<!-- Development profile -->
<profile>
<id>dev</id>
<activation>
<property>
<name>env</name>
<value>development</value>
</property>
</activation>
<properties>
<sencha.env>testing</sencha.env>
</properties>
</profile>
<!-- Production profile -->
<profile>
<id>prod</id>
<activation>
<property>
<name>env</name>
<value>production</value>
</property>
</activation>
<properties>
<sencha.env>production</sencha.env>
</properties>
</profile>
</profiles>
Then use the following Maven plugin:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.2.1</version>
<executions>
<execution>
<id>sencha-compile</id>
<phase>compile</phase>
<goals>
<goal>exec</goal>
</goals>
<configuration>
<!-- Set path to your Sencha Cmd executable-->
<executable>${env.SENCHA_CMD}</executable>
<arguments>
<argument>-sdk</argument>
<argument>${basedir}/src/main/webapp</argument>
<argument>app</argument>
<argument>build</argument>
<argument>--clean</argument>
<argument>--environment</argument>
<argument>${sencha.env}</argument>
<argument>--destination</argument>
<argument>${basedir}/src/main/webapp/build</argument>
</arguments>
</configuration>
</execution>
</executions>
</plugin>
And run
$ mvn compile
To answer your question, you can reference system environment variables from within a maven pom file with this syntax:
${env.NAME_OF_VARIABLE}
See this link for more details:
https://maven.apache.org/pom.html#Properties
If you named the environment variable PATH_TO_SENCHA_EXE you could reference it like this:
<executable>${env.PATH_TO_SENCHA_EXE}sencha.exe</executable>
As an alternative to environment variables, you could consider creating a property in your pom to contain this path. Then you could change the value used for different environments by passing a new value for the property on the command line or by loading a properties file in your pom that may contain this property. There are many options here.
Edit:
I found the latter suggestion has been covered on SO at the following link (and likely other places):
Reading properties file from Maven POM file

Maven: How to activate profile based on whether child module contains a file?

I’m using Maven 3.1.1 on Mac 10.9.1. I want to activate a profile based on whether certain child modules contain a file. I have tried the following
<profile>
<id>deploy-war-to-jboss</id>
<activation>
<file>
<exists>${session.executionRootDirectory}/src/main/webapp/WEB-INF/web.xml</exists>
</file>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.jboss.as.plugins</groupId>
<artifactId>jboss-as-maven-plugin</artifactId>
<version>${jboss.maven.plugin.version}</version>
<configuration>
<hostname>${jboss.remote.hostname}</hostname>
<port>${jboss.remote.port}</port>
</configuration>
<executions>
<execution>
<id>deploy-to-jboss</id>
<phase>install</phase>
<goals>
<goal>deploy</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
but this profile doesn’t get activated when running “mvm clean install” despite the fact I’ve verified that the child modules in question contain the file. I also tried ${project.basedir} without luck. Any ideas how I make this happen?
Since you define the profile in the parent POM (I guess) you should try this
${basedir}/name-of-your-child-module//src/main/webapp/WEB-INF/web.xml
Furthermore, you should try running Maven in debug mode to get more information at runtime:
mvm -X clean install

How to configure maven jar plugin to exclude properties on profile

I have this configuration in my pom
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.2</version>
<configuration>
<excludes>
<exclude>**/logging/*</exclude>
<exclude>**/config/*</exclude>
</excludes>
</configuration>
</plugin>
I use profiles to handle different behaviour from local environment to production environment.
Is it possible not activate the exclusions when executing mvn install with local profile?
I tried to set a blank properties on local environment like this
but the plugin complains.
This is a workaround solution, maybe a better one exists. I think the easiest you could do is to let your DEV environment free from any config of the jar plugin. And then place your PROD config in a dedicated profile :
<profiles>
<profile>
<id>PROD</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.2</version>
<configuration>
<excludes>
<exclude>**/logging/*</exclude>
<exclude>**/config/*</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
When you need to build the production jar, launch :
mvn clean install -PPROD
Are the logging and config files resources for testing only? If yes, put them in ${basedir}/src/test/resources. They'll be on the classpath for your tests but will not end up in the final jar, and you won't need specific jar plugin config.
The best workaround I found is to use an invalid value to filter on when executing on DEV environment.
<profiles>
<profile>
<id>env-local</id>
<activation>
<property>
<name>env</name>
<value>local</value>
</property>
</activation>
<properties>
<jndi.iban0>cont0Data</jndi.iban0>
<config.file.path>classpath:config</config.file.path>
<logging.file.path>classpath:logging</logging.file.path>
<exclude.logging>none</exclude.logging>
<exclude.config>none</exclude.config>
</properties>
</profile>
</profiles>

Using DB unit to load DEMO data

I'm using the Maven DBUnit plugin to load test data for unit tests. I'd also like to use it for loading a different set of data to a different db for demo purposes. The dbunit plugin only allows for a single execution. Should I just create a separate pom or is there some trick that I can do to make this happen. (I.e., I'd like to simply do something like mvn load-demo-data).
you can execute a plugin several times if you like it depends just on the configuration you give in the pom:
<plugin>
<artifactId>maven-whatever-plugin</artifactId>
<version>1.0</version>
<executions>
<execution>
<id>execution1</id>
<phase>test</phase>
<configuration>
....
</configuration>
<goals>
<goal>WhatEverGoalYouHave</goal>
</goals>
<phase>...</phase>
</execution>
<execution>
<id>execution2</id>
<configuration>
....
</configuration>
<goals>
<goal>WhatEverGoalYouHave</goal>
</goals>
<phase>...</phase>
</execution>
</executions>
</plugin>
You can also define a profile to control the execution which might the right choice for your demo data. The call you described mvn load-demo-data is not possible with maven, cause maven calls only a goal or lifecycle.
Use properties when configuring the database credentials
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>dbunit-maven-plugin</artifactId>
<version>${dbunit.plugin.version}</version>
<configuration>
<url>${db.url}</url>
<driver>${db.driver}</driver>
<username>${db.username}</username>
<password>${db.password}</password>
</configuration>
..
Then add a profiles section to your POM, controling the setting of these properties:
<profiles>
<profile>
<id>db1</id>
<properties>
<db.url>jdbc:h2:target/db1;AUTO_SERVER=TRUE</db.url>
<db.driver>org.h2.Driver</db.driver>
<db.username>user</db.username>
<db.password>pass</db.password>
</properties>
<profile>
<profile>
<id>db2</id>
<properties>
<db.url>jdbc:h2:target/db2;AUTO_SERVER=TRUE</db.url>
<db.driver>org.h2.Driver</db.driver>
<db.username>user</db.username>
<db.password>pass</db.password>
</properties>
<profile>
..
The profile cane be activated as follows to update two different databases
mvn -Pdb1 clean test
mvn -Pdb2 clean test

Resources