Cloud Build fails to build the the simple build step with maven - maven

Testing the cloud-build
Part of my cloudbuild.yaml
- name: 'gcr.io/cloud-builders/mvn'
args: ['dockerfile:build']
dockerfile:build perfectly works in bitbucket pipeline, no problem. I use
<plugin>
<groupId>com.spotify</groupId>
<artifactId>dockerfile-maven-plugin</artifactId>
<version>${dockerfile-maven-version}</version>
<executions>
<execution>
<id>default</id>
<goals>
<goal>build</goal>
<goal>push</goal>
</goals>
</execution>
</executions>
<configuration>
<repository>gcr.io/my-project-id/${project.artifactId}</repository>
<tag>${project.version}</tag>
<buildArgs>
<JAR_FILE>${project.build.finalName}.jar</JAR_FILE>
</buildArgs>
</configuration>
</plugin>
But with the cloud-build for this single step I get the error:
[INFO] Step 14/15 : ARG JAR_FILE
[INFO]
[INFO] ---> Using cache
[INFO] ---> 55793de4bb9f
[INFO] [INFO] Step 15/15 : ADD target/${JAR_FILE} /usr/share/$SERVCE_FOLDER_NAME/app.jar
[INFO]
[ERROR] ADD failed: stat /mnt/cache/docker/tmp/docker-builder589658449/target/myappname-0.0.1-SNAPSHOT.jar: no such file or directory
(the JAR_FILE is passed from the maven dockerfile plugin
no such file or directory
Why?.. In the end of the day I juse call dockerfile:build and expect it to be the same as it is when I build it from another pipeline.
My Dockerfile:
FROM openjdk:8-jdk
ENV GOOGLE_APPLICATION_CREDENTIALS=/app/credentials.json
ARG ACTIVE_PROFILES=dev
ENV ACTIVE_PROFILES=$ACTIVE_PROFILES
ARG CREDENTIALS
ARG SERVCE_FOLDER_NAME=myappname-service
ENV SERVCE_FOLDER_NAME=$SERVCE_FOLDER_NAME
#ENTRYPOINT ["/usr/bin/java", "-jar", "/usr/share/$SERVCE_FOLDER_NAME/app.jar"]
ENTRYPOINT ["./entrypoint.sh" ]
WORKDIR /app
EXPOSE 8080
COPY ./.gcloud/credentials.json credentials.json
COPY entrypoint.sh .
#Add Maven dependencies (not shaded into the artifact; Docker-cached)
#ADD target/lib /usr/share/$SERVCE_FOLDER_NAME/lib
ARG JAR_FILE
ADD target/${JAR_FILE} /usr/share/$SERVCE_FOLDER_NAME/app.jar
EntryPoint script is (that is what is mentioned on step 15/15 in the log):
java -Djava.security.egd=file:/dev/./urandom -jar /usr/share/$SERVCE_FOLDER_NAME/app.jar --spring.profiles.active=$ACTIVE_PROFILES
(I did try to pass hard-coded values to $SERVCE_FOLDER_NAME, $ACTIVE_PROFILES - same [it works in bitbucket pipeline])

A few things come to mind,
how are you triggering the builds?
manually with gcloud or api? or automatically with build triggers or the github app?
it seems that the target/ directory might not be present in the remote workspace-- are you ignoring target/ or .jar files anywhere?
the remote workspace might not be getting the target/ directory or .jar files if they are in your .gitignore or .gcloudignore
try making an empty .gcloudignore or temporarily removing target/ and .jar files from .gitignore
relevant links: https://cloud.google.com/sdk/gcloud/reference/topic/gcloudignore, https://github.com/GoogleCloudPlatform/cloud-builders/issues/40
have you tried debugging with cloud-build-local? it allows you to write and explore the workspace locally
https://cloud.google.com/cloud-build/docs/build-debug-locally
https://github.com/GoogleCloudPlatform/cloud-build-local

Related

dockerizing Spring Boot Application error: Error: Invalid or corrupt jarfile /app.jar

I'm starting working with docker and I'm trying to dockerize my Spring boot Application. I generated this using spring initializr. I'm working with maven. My DockerFile looks like this:
FROM java:8-jdk-alpine
COPY ${JAR_FILE} app.jar
WORKDIR /usr/app
EXPOSE 8080
ENTRYPOINT ["java","-jar","/app.jar"]
It's pretty simple.
I executed the next commands:
docker build -t shortenurl .
I can see the image in the console with docker images:
But when I try to run the image with the command: docker run -p 8080:8080 shortenurl I got this error message: Error: Invalid or corrupt jarfile /app.jar
I've tried different configurations but not pretty sure. Any advice?
Thanks.
This is my plugin pom.xml plugins section:
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>com.spotify</groupId>
<artifactId>docker-maven-plugin</artifactId>
<version>0.2.3</version>
<configuration>
<imageName>${docker.image.prefix}/${project.artifactId}</imageName>
<dockerDirectory>src/main/docker</dockerDirectory>
<resources>
<resource>
<directory>${project.build.directory}</directory>
<include>${project.build.finalName}.jar</include>
</resource>
</resources>
</configuration>
</plugin>
</plugins>
This is my log after the command:docker build -t shortenurl .
$ docker build -t shortenurl .
Sending build context to Docker daemon 32.89MB
Step 1/5 : FROM java:8-jdk-alpine
---> 3fd9dd82815c
Step 2/5 : COPY ${JAR_FILE} app.jar
---> 248aa4be697d
Step 3/5 : WORKDIR /usr/app
---> Running in 5fcd71c548af
Removing intermediate container 5fcd71c548af
---> bc344b970b11
Step 4/5 : EXPOSE 8080
---> Running in 46b4e4f8e9b7
Removing intermediate container 46b4e4f8e9b7
---> 6b4f55a86a23
Step 5/5 : ENTRYPOINT ["java","-jar","/app.jar"]
---> Running in 838a6052f4c8
Removing intermediate container 838a6052f4c8
---> 7d7d272ea42d
Successfully built 7d7d272ea42d
Successfully tagged shortenurl:latest
SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. All files and directories added to build context will have '-rwxr-xr-x' permissions. It is recommended to double check and reset permissions for sensitive files and directories.
You're building with spring-boot-maven-plugin but you're not generating an executable jar. Update your plugin's configuration to:
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<executable>true</executable>
</configuration>
</plugin>
</plugins>
</build>
and you should be fine.

How to automatic deploy spring boot in jenkin and put images to docker

I using jenkins spring boot and docker in my app. I want deploy spring boot in jenkin and put jar when deploy into docker and start it automatic. I reference to here :
https://denisdbell.wordpress.com/2017/08/26/automated-deployment-jenkins-docker-spring-boot/
But i have a problem. It deploy success but don't run jar in docker. When i type docker - ps it show me jar and i must run it manually. I want when jenkin deploy it automatic put images into docker and run it automatic.
I have two question:
- In my project spring boot, i need put docker in root project ?
- How to jenkin deploy and put images to docker and run automatic jar when jenkin deploy
- When i have mutiple modules, how to i specific copy only jar modules i need deploy and copy to docker and run it.
Thanks you so much help me
Question 1:- Yes you need to add it to the root folder.
Question 2:- You have to use the dockerfile-maven-plugin to push/pull images and use the shell commands from Jenkins to run the same.
Add the SCM GIT, Stash, Hg as this
<scm>
<connection>scm:git:ssh://git#guthub.com/yourproject.git</connection>
<developerConnection>scm:git:ssh://git#guthub.com/yourproject.git[push=]scm:git:ssh://git#guthub.com/yourproject.git</developerConnection>
<tag>HEAD</tag>
</scm>
Add the plugin as
<plugin>
<groupId>com.spotify</groupId>
<artifactId>dockerfile-maven-plugin</artifactId>
<version>1.4.3</version>
<executions>
<execution>
<id>default</id>
<goals>
<goal>build</goal>
<goal>push</goal>
</goals>
</execution>
</executions>
<configuration>
<repository>${docker.image.prefix}/yourproject-${project.artifactId}
</repository>
<tag>${project.version}</tag>
</configuration>
In this while building your jar pass the arguments
mvn clean install -DskiptTests -Ddocker.image.prefix=<your docker repo url>
This command would automatically push the latest project version docker image to the repository.
Use the Shell commands to fetch this image and start the docker from the shell.

Add Confguration in pom.xml at runtime in dockerfile

I am trying to build docker image.
Steps in dockerfile
1.pull code from gitlab
2.Maven Build code from gitlab**
However maven build is failing, when i am building docker image and reason is surefile couldn't fork
Below is printed in logs
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test
(default-test) on project DFDMWeb: Execution default-test of goal
org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test failed: The
forked VM terminated without properly saying goodbye. VM crash or
System.exit called?
However I know the workaround for above i.e to add below line in pom.xml in configuration for surefire plugin
false
pom.xml contents for surefire plugin is like below after that
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<useSystemClassLoader>false</useSystemClassLoader>
</configuration>
</plugin>
I have tested the above workaround by manually copying source code in aws EC2 instance and modifying the pom.xml file with useSystemClassLoader as false in pom.xml for surefire plugin
After doing so when i ran below command
docker build -t test -f Dockerfile .
Image was created successfully.
But now the problem is i am cloning repo from gitlab and I don't know how can I add false in surefire plugin configuration in pom.xml at run time using dockerfile ?
Anyone having any leads would be helpful.
Thanks in advance.
Try to set the useSystemClassLoader parameter with mvn command option -Dsurefire.useSystemClassLoader=false i.e. change your dockerfile to run mvn command like this mvn install -Dsurefire.useSystemClassLoader=false
The option is mentioned as a "User Property" in surfire plugin's docs
https://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#useSystemClassLoader

Have Maven write to file in WAR package

What is the best approach to have my maven package-deploy project write to a properties file inside of a WAR file?
I currently have three separate maven projects that create their own packages:
a.war, b.zip, and c.tar.gz
Inside of the WAR file (a.war), there is a properties file that contains the following:
buildDate=2018-01-25 16:11:49 PST
aUiNumber=2.1.0-SNAPSHOT.5
buildNumber=2.1.0-SNAPSHOT.${deploy.number}
The file is located here (inside of the WAR file):
WEB-INF/classes/a-version.properties
On a Jenkins server, I have a job that uses maven to do the following:
Pull the latest a.war, b.zip, c.tar.gz from nexus
Package these into app-assets.zip
Deploy app-assets.zip
I would like to have this maven job populate the ${deploy.number} in the a-version.properties file with the Jenkins job number. What is the best approach for this? Is there a way to do it without unpacking the WAR file?
I attempted this by adding the a.war/WEB-INF/classes to the <directory> section of the war file. As expected, the build did not fail; however, it did not populate the variable as well:
mvn -U -f ./PackageDeployPom.xml resources:resources -Ddeploy.number=${BUILD_NUMBER}
[INFO] skip non existing resourceDirectory /home/jenkins/app-assets/apache-tomcat-8.0.41/webapps/a.war/WEB-INF/classes
Not sure if this is the best way to do this but here is how I did it:
I used the exec-maven-plugin.
The maven project now follows this procedure:
Pull the latest a.war, b.zip and c.tar from nexus (place in to-be-packaged-directory).
Use exec-maven-plugin to call a bash script. This script will:
i. Copy the a.war file to a temporary workspace directory, unpack it here.
ii. Populate the variables in temp-workspace/WEB-INF/classes/a-version.properties. (Using sed).
iii. Use jar to update the file in the war file.
Package the app-assets.zip
Deploy app-assets.zip
Here is what I added to the pom file for the packaging job:
<plugin>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<groupId>org.codehaus.mojo</groupId>
<executions>
<execution><!-- For Adding deploy Number -->
<id>Renaming build artifacts</id>
<phase>package</phase>
<goals>
<goal>exec</goal>
</goals>
<configuration>
<executable>bash</executable>
<commandlineArgs>scripts/BuildNumber.sh -b ${deploy.number}</commandlineArgs>
</configuration>
</execution>
</executions>
</plugin>
Here is the working portion of the Bash script that gets executed with by the maven plugin:
#Cleaning the TempWorkspace
TempWs=${Workspace}/a-war-temp;
if [[ ! -d ${TempWs} ]];then
mkdir ${TempWs};
else
rm -r ${TempWs}/;
mkdir ${TempWs};
fi
#--- end cleaning temp workspace ---
#Copying the war file to the temp workspace
cp ${Workspace}/app-assets/${ApacheTomcat}/webapps/a.war ${TempWs}/a-old.war;
#Unpacking the war file, using sed to make changes to variable(s)
cd ${TempWs}
jar xvf aw-old.war;
sed -i "s/\${deploy.number}/${BuildNumber}/g" ${TempWs}/WEB-INF/classes/am-version.properties
cd ${Workspace}; #Going back to the Workspace
#--- end populating variable(s) ---
#Updating the war file with the new properties file
jar -uf ${Workspace}/aw-emr/${ApacheTomcat}/webapps/aw.war -C ${TempWs} WEB-INF/classes/am-version.properties
Maven is run with this command:
mvn -U -B -Ddeploy.number=${BUILD_NUMBER} -f ./App-Asset-deploy.xml clean package

Why isn't mvn resources:resources picking up buildNumber?

I've got a maven project that uses the buildnumber-maven-plugin. If I run mvn validate I see it's working:
[INFO] --- buildnumber-maven-plugin:1.3:create (default) # myproject ---
[INFO] Executing: /bin/sh -c cd /Users/rob/Workspace/myproject && git rev-parse --verify HEAD
[INFO] Storing buildNumber: 5d315d8d1a43c3289fbf114c379fa1a3d3787044 at timestamp: 1477059166424
But if I run mvn resources:resources the filtered file does not pick it up:
[INFO] --- maven-resources-plugin:2.6:resources (default-cli) # myproject ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
The pom.xml has:
<build>
...
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
<includes>
<include>version.txt</include>
</includes>
</resource>
version.txt has:
${buildNumber}
But after maven runs, no filtering:
> cat target/classes/version.txt
${buildNumber}
The build number config in pom.xml:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>buildnumber-maven-plugin</artifactId>
<version>1.3</version>
<executions>
<execution>
<phase>validate</phase>
<goals><goal>create</goal></goals>
</execution>
I don't know enough Maven. Shouldn't running the resources "goal" also get the buildNumber property?
There is a difference in the commands that you execute:
mvn validate executes the maven phase "valdate": meaning all phases that come before (in this case none)
mvn resources:resources is a shortcut for executing the goal "resources" on the resources plugin. Actually its a shortcut for executing: org.apache.maven.plugins:maven-resources-plugin:3.0.1:resources. These short names are resolved by maven and very typical for plugins in the Apache namespace.
As you can see on the maven life-cycle page the goal you may look for is: "mvn process-resources". That phase has a default plugin binding to "resources:resources" which will run the resource plugin. Since you execute a phase all phases before that will be run too, including the build number plugin.
The ":" indicates the difference for the maven command line.

Resources