docker build fails inside gitlab-runner but works locally : spring boot native compilation with GraalVm - spring-boot
I made a Dockerfile to build my spring boot project with GraalVm natively; everything went correctly.
Here is the Dockerfile
FROM ghcr.io/graalvm/graalvm-ce:22.3.1 AS buildnative
WORKDIR /app
COPY mvnw pom.xml ./
COPY .mvn/ .mvn
COPY src ./src
RUN ./mvnw clean package -Pnative
FROM ubuntu:23.04
EXPOSE 8080
COPY --from=buildnative /app/target/spring-boot-project /build/app
CMD ["/build/app"]
This runs perfectly locally, but in the GitLab runner, I always have the same error.
JAVA_HOME is not defined correctly.
We cannot execute /opt/graalvm-ce-java17-22.3.1/bin/java
The command '/bin/sh -c ./mvnw clean package -Pnative' returned a non-zero code: 1
So I decided to add some logs within the maven wrapper, and here is what I have :
Step 7/11 : RUN ./mvnw clean package -Pnative ---> Running in 81e0558130f3 ------------> /opt/graalvm-ce-java17-22.3.1/bin/java ------------> JAVA_HOME is /opt/graalvm-ce-java17-22.3.1 Error: JAVA_HOME is not defined correctly. We cannot execute /opt/graalvm-ce-java17-22.3.1/bin/java The command '/bin/sh -c ./mvnw clean package -Pnative' returned a non-zero code: 1 Cleaning up project directory and file based variables
Step 7/11 : RUN ./mvnw clean package -Pnative
---> Running in 81e0558130f3
------------> /opt/graalvm-ce-java17-22.3.1/bin/java
------------> JAVA_HOME is /opt/graalvm-ce-java17-22.3.1
Error: JAVA_HOME is not defined correctly.
We cannot execute /opt/graalvm-ce-java17-22.3.1/bin/java
The command '/bin/sh -c ./mvnw clean package -Pnative' returned a non-zero code: 1
Cleaning up project directory and file based variables
In the log I have added, we can see JAVA_HOME is defined and is adequately defined. It is the same as locally, where everything works perfectly.
I tried to add this line: RUN chmod +x mvnw before running it, but it did not change anything.
I need more ideas. Is there anyone have an idea of what is happening?
Edit:
I decided to dive deeper into the issue. I have added logs to know why it does not work. I modified the mvnw script to know what was happening.
I have added this to mvnw
if [ -e "$JAVACMD" ] ; then
echo "------------> THE FILE EXIST" >&2
else
echo "------------> THE FILE DOES NOT EXIST" >&2
fi
if [ -x "$JAVACMD" ] ; then
echo "------------> THE FILE IS EXECUTABLE" >&2
else
echo "------------> THE FILE IS NOT EXECUTABLE" >&2
fi
Results:
Here is in local:
------------> JAVACMD /opt/graalvm-ce-java17-22.3.1/bin/java
------------> THE FILE EXIST
------------> THE FILE IS EXECUTABLE
Here is in the gitlab-runner:
------------> JAVACMD /opt/graalvm-ce-java17-22.3.1/bin/java
------------> THE FILE EXIST
------------> THE FILE IS NOT EXECUTABLE
Makes no sense to me
Is your GitLab runner configured to use a non-root user when executing the Dockerfile?
As #jilliss pointed out, it seems that it's the Java binary that needs execute permission, but maybe only root has the permission (which is why it works locally as by default you will be running it as root).
If the Ops team have tried to run the Dockerfile as another user, then it could explain why /opt/graalvm-ce-java17-22.3.1/bin/java is no longer executable.
Try adding a whoami log and see which user is running when it runs in GL.
Correct compile command is
mvn -Pnative native:compile
You can see more details and full doc here. After that you will see graalvm build result in the logs. So you need to change your docker file like below
FROM ghcr.io/graalvm/graalvm-ce:22.3.1 AS buildnative
WORKDIR /app
COPY mvnw pom.xml ./
COPY .mvn/ .mvn
COPY src ./src
RUN ./mvnw native:compile -Pnative
FROM ubuntu:23.04
EXPOSE 8080
COPY --from=buildnative /app/target/spring-boot-project /build/app
CMD ["/build/app"]
example build log from my local build
[INFO] --- native-maven-plugin:0.9.19:compile (default-cli) # demo ---
Downloading: Component catalog from www.graalvm.org
Processing Component: Native Image
Downloading: Component native-image: Native Image from github.com
Installing new component: Native Image (org.graalvm.native-image, version 22.3.1)
[INFO] Found GraalVM installation from JAVA_HOME variable.
[INFO] [graalvm reachability metadata repository for ch.qos.logback:logback-classic:1.4.5]: Configuration directory not found. Trying latest version.
[INFO] [graalvm reachability metadata repository for ch.qos.logback:logback-classic:1.4.5]: Configuration directory is ch.qos.logback/logback-classic/1.4.1
[INFO] Executing: /opt/graalvm-ce-java17-22.3.1/bin/native-image -cp /app/target/classes:/root/.m2/repository/org/springframework/spring-aop/6.0.4/spring-aop-6.0.4.jar:/root/.m2/repository/org/springframework/boot/spring-boot-starter-logging/3.0.2/spring-boot-starter-logging-3.0.2.jar:/root/.m2/repository/org/springframework/spring-context/6.0.4/spring-context-6.0.4.jar:/root/.m2/repository/org/springframework/spring-core/6.0.4/spring-core-6.0.4.jar:/root/.m2/repository/org/apache/logging/log4j/log4j-api/2.19.0/log4j-api-2.19.0.jar:/root/.m2/repository/org/springframework/spring-expression/6.0.4/spring-expression-6.0.4.jar:/root/.m2/repository/org/apache/logging/log4j/log4j-to-slf4j/2.19.0/log4j-to-slf4j-2.19.0.jar:/root/.m2/repository/ch/qos/logback/logback-core/1.4.5/logback-core-1.4.5.jar:/root/.m2/repository/jakarta/annotation/jakarta.annotation-api/2.1.1/jakarta.annotation-api-2.1.1.jar:/root/.m2/repository/org/springframework/spring-beans/6.0.4/spring-beans-6.0.4.jar:/root/.m2/repository/ch/qos/logback/logback-classic/1.4.5/logback-classic-1.4.5.jar:/root/.m2/repository/org/springframework/boot/spring-boot-starter/3.0.2/spring-boot-starter-3.0.2.jar:/root/.m2/repository/org/springframework/spring-jcl/6.0.4/spring-jcl-6.0.4.jar:/root/.m2/repository/org/springframework/boot/spring-boot-autoconfigure/3.0.2/spring-boot-autoconfigure-3.0.2.jar:/root/.m2/repository/org/slf4j/jul-to-slf4j/2.0.6/jul-to-slf4j-2.0.6.jar:/root/.m2/repository/org/yaml/snakeyaml/1.33/snakeyaml-1.33.jar:/root/.m2/repository/org/springframework/boot/spring-boot/3.0.2/spring-boot-3.0.2.jar:/root/.m2/repository/org/slf4j/slf4j-api/2.0.6/slf4j-api-2.0.6.jar --no-fallback -H:Path=/app/target -H:Name=demo -H:ConfigurationFileDirectories=/app/target/graalvm-reachability-metadata/160481799c4b6c37cde925c9aebf513c32245dcf/ch.qos.logback/logback-classic/1.4.1
========================================================================================================================
GraalVM Native Image: Generating 'demo' (executable)...
========================================================================================================================
[1/7] Initializing... (6.6s # 0.23GB)
Version info: 'GraalVM 22.3.1 Java 17 CE'
Java version info: '17.0.6+10-jvmci-22.3-b13'
C compiler: gcc (redhat, x86_64, 11.3.1)
Garbage collector: Serial GC
1 user-specific feature(s)
- org.springframework.aot.nativex.feature.PreComputeFieldFeature
Field org.apache.commons.logging.LogAdapter#log4jSpiPresent set to true at build time
Field org.apache.commons.logging.LogAdapter#log4jSlf4jProviderPresent set to true at build time
Field org.apache.commons.logging.LogAdapter#slf4jSpiPresent set to true at build time
Field org.apache.commons.logging.LogAdapter#slf4jApiPresent set to true at build time
Field org.springframework.core.NativeDetector#imageCode set to true at build time
Field org.springframework.format.support.DefaultFormattingConversionService#jsr354Present set to false at build time
Field org.springframework.core.KotlinDetector#kotlinPresent set to false at build time
Field org.springframework.core.KotlinDetector#kotlinReflectPresent set to false at build time
Field org.springframework.cglib.core.AbstractClassGenerator#imageCode set to true at build time
Field org.springframework.boot.logging.log4j2.Log4J2LoggingSystem$Factory#PRESENT set to false at build time
Field org.springframework.boot.logging.java.JavaLoggingSystem$Factory#PRESENT set to true at build time
Field org.springframework.boot.logging.logback.LogbackLoggingSystem$Factory#PRESENT set to true at build time
Field org.springframework.boot.logging.logback.LogbackLoggingSystemProperties#JBOSS_LOGGING_PRESENT set to false at build time
Field org.springframework.context.event.ApplicationListenerMethodAdapter#reactiveStreamsPresent set to false at build time
[2/7] Performing analysis... [*******] (60.8s # 2.15GB)
8,903 (88.31%) of 10,082 classes reachable
13,147 (64.27%) of 20,456 fields reachable
40,485 (56.88%) of 71,181 methods reachable
365 classes, 115 fields, and 1,191 methods registered for reflection
64 classes, 70 fields, and 55 methods registered for JNI access
4 native libraries: dl, pthread, rt, z
[3/7] Building universe... (8.9s # 2.03GB)
[4/7] Parsing methods... [***] (9.6s # 0.82GB)
[5/7] Inlining methods... [***] (3.7s # 2.16GB)
[6/7] Compiling methods... [*******] (48.2s # 1.97GB)
[7/7] Creating image... (5.6s # 1.60GB)
17.53MB (49.48%) for code area: 25,460 compilation units
17.60MB (49.66%) for image heap: 215,829 objects and 25 resources
312.40KB ( 0.86%) for other data
35.44MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 packages in code area: Top 10 object types in image heap:
936.87KB java.util 3.72MB byte[] for code metadata
594.27KB java.lang.invoke 2.07MB java.lang.String
469.21KB c.s.org.apache.xerces.internal.impl.xs.traversers 2.06MB java.lang.Class
455.78KB java.lang 1.68MB byte[] for general heap data
423.05KB com.sun.org.apache.xerces.internal.impl 1.60MB byte[] for java.lang.String
407.79KB com.sun.crypto.provider 765.10KB com.oracle.svm.core.hub.DynamicHubCompanion
375.00KB org.springframework.beans.factory.support 576.38KB java.util.HashMap$Node
371.71KB java.io 513.09KB int[][]
354.25KB java.util.concurrent 394.88KB java.lang.String[]
328.57KB java.text 394.65KB byte[] for reflection metadata
12.74MB for 405 more packages 3.23MB for 1771 more object types
------------------------------------------------------------------------------------------------------------------------
12.0s (8.0% of total time) in 35 GCs | Peak RSS: 3.04GB | CPU load: 5.08
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
/app/target/demo (executable)
/app/target/demo.build_artifacts.txt (txt)
========================================================================================================================
Finished generating 'demo' in 2m 29s.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:11 min
[INFO] Finished at: 2023-01-27T18:05:23Z
[INFO] ------------------------------------------------------------------------
Never tried gitlab runner myself, but have you tried to force the JAVA_HOME env path before maven command?
ENV JAVA_HOME=/opt/graalvm-ce-java17-22.3.1
RUN ./mvnw clean package -Pnative
hope it helps
Related
Build Docker Images with Bazel targets Built
I have a (1) Dockerfile and a (2) C++ project with Bazel I want to create a docker image that has Bazel targets pre-built within the image, so as to when I power up new containers the Bazel targets are pre-built and I just do Bazel run //hello:hello_world from the container bash. Dockerfile # Copy my project with Bazel files to a Docker image, and the ... RUN bazel --output_user_root=/tmp/hello_project/bazel build //... ... Within the execution of the Dockerfile, I get the following output which is expected Loading: Loading: 0 packages loaded Analyzing: 2 targets (1 packages loaded, 0 targets configured) Analyzing: 2 targets (11 packages loaded, 18 targets configured) INFO: Analyzed 2 targets (15 packages loaded, 60 targets configured). INFO: Found 2 targets... [0 / 11] [Prepa] BazelWorkspaceStatusAction stable-status.txt INFO: Elapsed time: 6.333s, Critical Path: 0.37s INFO: 11 processes: 6 internal, 5 processwrapper-sandbox. INFO: Build completed successfully, 11 total actions INFO: Build completed successfully, 11 total actions When I run a new container form the Docker Image built previously and then on the container I run bazel run //hello:hello_world Instead of using existing pre-built targets it re-builds the targets, which is not required. Result I expect (not get): Everything is pre-built and just needs to run INFO: Analyzed target //hello:hello_world (0 packages loaded, 0 targets configured). INFO: Found 1 target... Target //hello:hello_world up-to-date: bazel-bin/hello/hello_world INFO: Elapsed time: 0.163s, Critical Path: 0.01s INFO: 1 process: 1 internal. INFO: Build completed successfully, 1 total action INFO: Build completed successfully, 1 total action Hello World! Result I get: I re-builds the binaries [root#4a6bdb57fd79 test-rc]# bazel run //hello:hello_world Extracting Bazel installation... Starting local Bazel server and connecting to it... INFO: Analyzed target //hello:hello_world (15 packages loaded, 60 targets configured). INFO: Found 1 target... Target //hello:hello_world up-to-date: bazel-bin/hello/hello_world INFO: Elapsed time: 6.255s, Critical Path: 0.38s INFO: 7 processes: 4 internal, 3 processwrapper-sandbox. INFO: Build completed successfully, 7 total actions INFO: Build completed successfully, 7 total actions Hello World! How can I make sure, the the bazel run uses same pre-built targets and not build them again before run.
This sounds like non-determinism - in the second execution of Bazel, something is different causing a cache miss. Some things that can cause it: different options passed to bazel build vs. bazel test - check your .bazelrc file actions that include VCS info or a timestamp - make sure you have rules_docker 0.22.0 or later to pick up https://github.com/bazelbuild/rules_docker/commit/2b35b2dd56f0be6cc6b8df957332a31435f6b3ce One step to diagnose is to use the --explain=log.txt and --verbose_explanations flags to Bazel, then the log file will say why it was rebuilt. However it doesn't have much detail, just something like "a source file has changed". If you want the power tool for this, there is a way to find out exactly why Bazel didn't get a cache hit - read https://georgi.hristozov.net/til/2020/04/20/compare-bazel-execlogs-to-find-non-deterministic-parts-of-the-build
Cannot build go_proto_library with gRPC
I'm getting started with bazel and trying to generate the protobuf code for golang for an RPC service. When I try to build it I get the following error: bazel-out/k8-fastbuild/bin/examples/grpc/protos/helloworld_go_proto_/examples/grpc/protos/helloworld.pb.go:229:7: undefined: grpc.ClientConnInterface bazel-out/k8-fastbuild/bin/examples/grpc/protos/helloworld_go_proto_/examples/grpc/protos/helloworld.pb.go:233:11: undefined: grpc.SupportPackageIsVersion6 bazel-out/k8-fastbuild/bin/examples/grpc/protos/helloworld_go_proto_/examples/grpc/protos/helloworld.pb.go:243:5: undefined: grpc.ClientConnInterface bazel-out/k8-fastbuild/bin/examples/grpc/protos/helloworld_go_proto_/examples/grpc/protos/helloworld.pb.go:246:26: undefined: grpc.ClientConnInterface Full build log: https://app.buildbuddy.io/invocation/c3773978-22dd-44c8-b977-13967a1953b7 Here is the code: https://github.com/juanique/example-go-grpc. I'm trying to include the least possible amount of code to make that target work. Since the BUILD file was generated by gazelle I suspect the issue is in the WORKSPACE file: https://raw.githubusercontent.com/juanique/example-go-grpc/main/WORKSPACE. I'm just doing what I found in https://github.com/bazelbuild/rules_go
UPDATE: it looks surely a version issue: https://github.com/grpc/grpc-go#compiling-error-undefined-grpcsupportpackageisversion Not a bazel user, but after hours' test, I found your rpc version should be higher: go_repository( name = "org_golang_google_grpc", - build_file_proto_mode = "disable", importpath = "google.golang.org/grpc", - sum = "h1:J0UbZOIrCAl+fpTOf8YLs4dJo8L/owV4LYVtAXQoPkw=", - version = "v1.22.0", + sum = "h1:f+PlOh7QV4iIJkPrx5NQ7qaNGFQ3OTse67yaDHfju4E=", + version = "v1.41.0", ) Then you can build it: ~/code/example-go-grpc (main*) [09:45:55] p1gd0g$ bazel build //examples/grpc/protos:helloworld_go_proto INFO: Analyzed target //examples/grpc/protos:helloworld_go_proto (49 packages loaded, 306 targets configured). INFO: Found 1 target... Target //examples/grpc/protos:helloworld_go_proto up-to-date: bazel-bin/examples/grpc/protos/helloworld_go_proto.a INFO: Elapsed time: 41.180s, Critical Path: 1.20s INFO: 29 processes: 1 internal, 28 linux-sandbox. INFO: Build completed successfully, 29 total actions And I recommend using gazelle with go.mod to generate WORKSPACE. A demo: https://github.com/p1gd0g/helloworld
Execute selenise runner on Firefox
I am using selenise runner to run script in different browsers. I am using this config file: firefox: C://Users/Desktop/geckodriver.exe highlight: Yes baseurl: https://example.com/ set-speed: 1000 timeout: 30000 driver: firefox xml-result: C://Users/Desktop/Selenise/XMLReport html-result: C://Users/Desktop/Selenise/HTMLReport and this command to execute the execute the jars and reports: set webdriver.gecko.driver=C:\Users\Desktop\geckodriver.exe set path=%path%;webdriver.gecko.driver; java -jar cmdJars.jar TestCases\TestCase1.html --config config.txt --driver firefox --firefox "C://Program Files/Mozilla Firefox/firefox.exe" I am getting this error log: [2017-01-17 10:34:38.139 +05:30] [INFO] Start: Selenese Runner 3.0.0 [2017-01-17 10:34:38.357 +05:30] [INFO] Firefox binary: C://Program Files/Mozill a Firefox/firefox.exe java.lang.IllegalStateException: The path to the driver executable must be set b y the webdriver.gecko.driver system property; for more information, see https:// github.com/mozilla/geckodriver. The latest version can be downloaded from https: //github.com/mozilla/geckodriver/releases at com.google.common.base.Preconditions.checkState(Preconditions.java:19 9) at org.openqa.selenium.remote.service.DriverService.findExecutable(Drive rService.java:109) at org.openqa.selenium.firefox.GeckoDriverService.access$000(GeckoDriver Service.java:37) at org.openqa.selenium.firefox.GeckoDriverService$Builder.findDefaultExe cutable(GeckoDriverService.java:95) at org.openqa.selenium.remote.service.DriverService$Builder.build(Driver Service.java:296) at org.openqa.selenium.firefox.FirefoxDriver.createCommandExecutor(Firef oxDriver.java:277) at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:2 47) at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:2 42) at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:1 35) at jp.vmi.selenium.webdriver.FirefoxDriverFactory.newInstance(FirefoxDri verFactory.java:173) at jp.vmi.selenium.webdriver.WebDriverManager$Builder.build(WebDriverMan ager.java:77) at jp.vmi.selenium.webdriver.WebDriverManager.get(WebDriverManager.java: 266) at jp.vmi.selenium.selenese.Main.setupRunner(Main.java:139) at jp.vmi.selenium.selenese.Main.run(Main.java:82) at jp.vmi.selenium.selenese.Main.main(Main.java:221) [2017-01-17 10:34:38.434 +05:30] [INFO] Exit code: 1 Also i have set webdriver.gecko.driver in the system environment variable. why am I getting this error even after setting the path of gecko driver. Version: Gecko driver : geckodriver 0.13.0, Selenise runner : Selenese Runner 3.0.0
Are you sure the webdriver.gecko.driver System Property is set? You can pass this as a JVM argument (I had to on Mac since it doesn't really like dots in environment variables): java -Dwebdriver.gecko.driver=/path/to/geckodriver -jar cmdJars.jar TestCases\TestCase1.html --config config.txt --driver firefox --firefox "C://Program Files/Mozilla Firefox/firefox.exe" Please make sure -D appears before -jar, as above.
Maven+TestNG testing through Jenkins on Ubuntu
I have built a maven+testng testing framework project on eclipse. I installed Jenkins on my local host and ran my project in there till now without any problem. My OS is Windows 10. I also ran my project on different Windows OS machines without any problem.
However I wanted cross platform testing, so I tried running my project in an Ubuntu machine. I did all the setup processes. I configured Firefox binary path and also installed X server, i.e xvfb on that Ubuntu machine.
I did some changes in my webdriver java file
package com.mednet.webdriver;
import java.io.File;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxBinary;
import org.openqa.selenium.firefox.FirefoxDriver;
public class WebdriverSetUp {
public static WebDriver driver = null;
public static WebDriver startdriver() {
FirefoxBinary binary = new FirefoxBinary(new File("/opt/firefox/firefox"));
binary.setEnvironmentProperty("DISPLAY", System.getProperty("lmportal.xvfb.id", ":99"));
return driver = new FirefoxDriver(binary, null);
}
}
Then I used following commands in terminal of Ubuntu to install firefox and xvfb.
to install firefox: sudo apt-get install firefox
to install xvfb: sudo apt-get xvfb & export DISPLAY=:99 & xvfb 99 -ac
Here I used DISPLAY=:99
Now when I start to build in Jenkins.
Following is my console output
Results :
Failed tests: testInvestigation(com.mednet.executor.diagnostics.ExecTest1): Timed out after 50 seconds waiting for visibility of element located by By.id: lineItemForCounterBilling(..)
testInvestigation(com.mednet.executor.diagnostics.ExecTest6): Timed out after 50 seconds waiting for visibility of element located by By.id: lineItemForCounterBilling(..)
Tests run: 2, Failures: 2, Errors: 0, Skipped: 0
[ERROR] There are test failures.
Please refer to/var/lib/jenkins/workspace/MednetTestingFramework/target/surefire-reports for the individual test results.
[JENKINS] Recording test results
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:19 min
[INFO] Finished at: 2016-08-25T11:06:08+05:30
[INFO] Final Memory: 34M/381M
[INFO] ------------------------------------------------------------------------
[JENKINS] Archiving /var/lib/jenkins/workspace/MednetTestingFramework/pom.xml to com.mednet/MednetTestingFrameworkV1/0.0.1-SNAPSHOT/MednetTestingFrameworkV1-0.0.1-SNAPSHOT.pom
channel stopped
Finished: UNSTABLE
Test stops after login steps that I have put in. There is nothing wrong with my Element as it locates without any problem in my Windows machine.
I searched on internet about the problem. In many sources I found out about Xvfb screen resolution is by default usually not large i suppose. So I am not able to
increase screen size
I tried many commands such as xvfb :99 -screen 0 1024*720 or 1440*900
But of no use it show invalid screen resolution.
Need help in solving like any important step that I may have missed out.
Thanks in advance
Custom gradle task for rpmbuild
We're currently in the process of moving away from scons to gradle to build the java portions of our software. One thing we need is to build rpm from the spec file. So we tried to write custom task to execute rpmbuild to build rpm from spec file. The code boils down to this:
def rpmPath = 'sample-master-5.0-128000-final.x86_64.rpm'
def rpmArgs = ['--quiet', '-bb', 'SPECS/sample-rpm-spec.spec', ' --define "rpmdir artifacts"', ' --define "sourcedir ."', ' --define "version 5.0"', ' --define "targetdir build"', ' --define "name sample"', ' --define "pathname sample"', ' --define "arch x86_64"', ' --define "revision 128000"', ' --define "branchName master"']
project.exec {
workingDir '.'
commandLine 'rpmbuild'
args rpmArgs
}
and here is the spec file:
# --define 'rpmdir dir' to specify the rpm destination directory
# --define 'version x.y.z' to define a build version
# --define 'sourcedir dir' to specify the location of the source directory
%define _builddir .
%define _rpmdir %{?rpmdir}%{!?rpmdir:RPMS}
%define _rpmfilename %{pathname}-%%{VERSION}-%%{RELEASE}%{?final}%{!?final:%{nil}}.%%{ARCH}.rpm
%define xyzapp /opt/xyzapp
Summary: Sample Spec File
Name: %{name}
Version: %{version}
Release: %{revision}
License: Commercial
Group: Applications/Imaging
Vendor: Sample Company Inc.
BuildRoot: %{_tmppath}/%{pathname}-buildroot
BuildArchitectures: %{arch}
Requires: first-rpm%{?branchName:-%{branchName}}%{!?branchName:} >= 2.10.0
Requires: second-rpm%{?branchName:-%{branchName}}%{!?branchName:} >= 2.10.0
%description
Sample Spec description
%build
echo building with rpmdir=%{_rpmdir} version=%{version} sourcedir=%{sourcedir}
# ensure the build root is clean
if [ %{buildroot} != "/" ]; then
rm -rf %{buildroot}
fi
mkdir -p %{_rpmdir}
mkdir -p %{buildroot}/tmp/classes
mkdir -p %{buildroot}%{xyzapp}
mkdir -p %{buildroot}%{xyzapp}/webapp/WEB-INF/lib/
if [ -f %{targetdir}/distributions/SampleZipArtifact.zip ]; then
unzip %{targetdir}/distributions/SampleZipArtifact.zip -d %{buildroot}%{xyzapp}/
fi
# remove the server classes that should not be included with this plugin
rm -rf %{buildroot}/tmp
find %{buildroot} -name .svn -o -name .classpath -o -name .project | xargs rm -rf
%clean
if [ %{buildroot} != "/" ]; then
rm -rf %{buildroot}
fi
%pre
%post
%preun
%postun
%files
%defattr(-,xyzapp,users,-)
%doc
%{xyzapp}/webapp
Essentially, by executing gradle buildSampleRpm
We got the error:
[ERROR] [system.err] error: No compatible architectures found for build
Here's is the full debug logs to give you more details:
14:23:16.462 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter] Executing actions for task ':buildSampleRpm'.
14:23:16.490 [INFO] [org.gradle.process.internal.DefaultExecHandle] Starting process 'command 'rpmbuild''. Working directory: /mnt/hgfs/CSI/GradleRpmSample Command: rpmbuild --quiet -bb SPECS/sample-rpm-spec.spec --define "rpmdir artifacts" --define "sourcedir ." --define "version 5.0" --define "targetdir ." --define "name sample" --define "pathname sample" --define "arch x86_64" --define "revision 128000" --define "branchName master"
14:23:16.490 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Environment for process 'command 'rpmbuild'': {TERM=xterm, JAVA_HOME=/usr/lib/jvm/java, FLEX_SDK=4.11, SESSION_MANAGER=local/unix:#/tmp/.ICE-unix/2695,unix/unix:/tmp/.ICE-unix/2695, GNOME_DESKTOP_SESSION_ID=this-is-deprecated, MAIL=/var/spool/mail/trungvo, GDMSESSION=gnome, XDG_SESSION_COOKIE=51806fc4a6f192828593c2df00000009-1428348402.461525-264542834, PWD=/mnt/hgfs/CSI/GradleRpmSample, HOSTNAME=tonycsicentos6, CVS_RSH=ssh, GIO_LAUNCHED_DESKTOP_FILE_PID=16285, G_BROKEN_FILENAMES=1, NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat, GNOME_KEYRING_SOCKET=/tmp/keyring-LiVncy/socket, CSI_DATA=/mnt/hgfs/CSI/Data/trunk, GDM_KEYBOARD_LAYOUT=us, HISTSIZE=1000, EDITOR=vi, PATH=/mnt/hgfs/CSI/Library/rhel5-64/ant/bin:/mnt/hgfs/CSI/Library/rhel5-64/Tools/flex_sdk_4.11/bin:/home/trungvo/gradle-2.3/bin:/mnt/hgfs/CSI/Library/rhel5-64/ant/bin:/mnt/hgfs/CSI/Library/rhel5-64/Tools/flex_sdk_4.11/bin:/home/trungvo/gradle-2.3/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/apache-ant-1.8.1/bin:/home/trungvo/bin:/opt/apache-ant-1.8.1/bin, GRADLE_HOME=/home/trungvo/gradle-2.3, QTLIB=/usr/lib64/qt-3.3/lib, GDCM_DATA_ROOT=/mnt/hgfs/CSI/Data/trunk/CSI/Rendering/gdcmData, GDM_LANG=en_US.utf8, XAUTHORITY=/var/run/gdm/auth-for-trungvo-dyz7I2/database, WINDOWPATH=1, FLEX_HOME=/mnt/hgfs/CSI/Library/rhel5-64/Tools/flex_sdk_4.11, USERNAME=trungvo, SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass, SHLVL=2, XFILESEARCHPATH=/usr/dt/app-defaults/%L/Dt, COLORTERM=gnome-terminal, CSI_LIB64=/mnt/hgfs/CSI/Library/rhel5-64, GIO_LAUNCHED_DESKTOP_FILE=/usr/share/applications/gnome-terminal.desktop, WINDOWID=44040195, LOGNAME=trungvo, QTDIR=/usr/lib64/qt-3.3, SSH_AUTH_SOCK=/tmp/keyring-LiVncy/socket.ssh, OLDPWD=/home/trungvo/gradle-2.3, LIBGL_DRIVERS_PATH=/usr/local/lib/gallium, DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-9FkHHSyF0Z,guid=7c6139ab66bf88999c60bf4d0000013e, SHELL=/bin/bash, GNOME_KEYRING_PID=2685, GTK_RC_FILES=/etc/gtk/gtkrc:/home/trungvo/.gtkrc-1.2-gnome2, DESKTOP_SESSION=gnome, FLEX_HOME_4=/mnt/hgfs/CSI/Library/rhel5-64/Tools/flex_sdk_4.11, QTINC=/usr/lib64/qt-3.3/include, DISPLAY=:0.0, USER=trungvo, ANT_HOME=/mnt/hgfs/CSI/Library/rhel5-64/ant, GRADLE_OPTS=-Dorg.gradle.native=false, MESA_GLSL=opt,nopvert,nopfrag, HOME=/home/trungvo, HISTCONTROL=ignoredups, LESSOPEN=|/usr/bin/lesspipe.sh %s, MALLOC_TRIM_THRESHOLD_=128*1024, LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31:*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:, ORBIT_SOCKETDIR=/tmp/orbit-trungvo, LANG=en_US.utf8}
14:23:16.499 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Changing state to: STARTING
14:23:16.506 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Waiting until process started: command 'rpmbuild'.
14:23:16.516 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Changing state to: STARTED
14:23:16.517 [DEBUG] [org.gradle.process.internal.ExecHandleRunner] waiting until streams are handled...
14:23:16.517 [INFO] [org.gradle.process.internal.DefaultExecHandle] Successfully started process 'command 'rpmbuild''
14:23:16.545 [ERROR] [system.err] error: No compatible architectures found for build
14:23:16.569 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Changing state to: FAILED
14:23:16.569 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Process 'command 'rpmbuild'' finished with exit value 1 (state: FAILED)
14:23:16.570 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter] Finished executing task ':buildSampleRpm'
14:23:16.571 [LIFECYCLE] [class org.gradle.TaskExecutionLogger] :buildSampleRpm FAILED
Did we miss to pass in the architecture in the command?
Please note that we do pass in --define 'arch x86_64'
We did try to pass in the option --buildarch x86_64, but apparently, it's the unknown option for our version of rpmbuild
If we execute the rpmbuild command directly from the terminal, it works just fine, however, as soon as we execute the rpmbuild in the gradle shell, things starting to break. That makes me to think that it's properly we missed to passed in some configuration into the Gradle.project.exec
You can download the full sample source code from here:
https://dl.dropboxusercontent.com/u/59660821/GradleRpmSample.zip
I'm running gradle on Centos 6.4
I did think about using nebula.os-package however, according to their API they use Gradle DSL to construct rpm packages. It doesn't offer a way for us to construct rpm package from spec files. We have a huge amount of spec files, converting spec files logic to Gradle DSL is not an option for us at the moment.
I looked around and apparently scons/python is a preferred choice for building rpm packages from spec files, so I'm trying to use gradle to drive scons to build rpm from spec files with rpmbuild command
Little bit history, we're mainly C/C++ shops so we're currently using scons to build our system. However, we run into many problems with specifying dynamically generated dependency in scons for Java side. So we're thinking of using Gradle as orchestration tool where it would drive scons to build C++ side, build Java side and dependency management.
It would be very nice if we could use gradle to build rpm packages from spec files through rpmbuild command line as our company have huge amount of spec files. Furthermore, it would also drive more Gradle adoption as I think most people use spec files to build rpm packages.
Let me know if there's anything else I can provide. Thank you very much guys
in spec file, have you tried to replace BuildArchitectures with BuildArch, maybe a typo? ref. https://fedoraproject.org/wiki/How_to_create_an_RPM_package#SPEC_file_overview rpmbuild -vv gives debug. kind regards
Thanks Joern, I got it working by split the arguments '--define blah' into separate arguments and remove any white spaces and it seems to work just fine.
You can try the SetupBuilder plugin. It build also an rpm installer.