Cloud Run, Secret Manager and Spring Boot - spring-boot

I want to load whole .properties file from GCP Secret Manager to my Spring Boot application.
Secret is "mounted as volume" in Cloud Run (whole .properties file, in path /secrets/secret.properties), but I cannot manage to load it to Spring Boot using spring.config.import
I was trying:
spring.config.import=optional:configtree:/secrets/
spring.config.import=optional:classpath:/secrets/secret.properties
spring.config.import=optional:/secrets/secret.properties
but nothing works. Values are not visible in Spring Boot application.
(java.lang.IllegalArgumentException: Could not resolve placeholder 'x' in value "${x}")
When putting file on resource classpath, everything works.

Here the set up that I have tested and which works (don't forget to grant the correct permissions)
JIB configuration
...
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>com.google.cloud.tools</groupId>
<artifactId>jib-maven-plugin</artifactId>
<version>3.1.1</version>
<configuration>
<container>
<args>--spring.config.location=file:///secret/my.properties</args>
</container>
<to>
<image>gcr.io/<PROJECT_ID>/springboot</image>
<credHelper>gcr</credHelper>
</to>
</configuration>
</plugin>
...
My Cloud Run deployment
gcloud beta run deploy --image=gcr.io/<PROJECT_ID>/springboot \
--region=us-central1 --allow-unauthenticated --platform=managed \
--set-secrets=/secret/my.properties=projects/<PROJECT_Number>/secrets/springboot:1 \
secret-springboot

Related

Hot deployment failure with Maven jetty plugin on Windows

I have configured the Jetty Maven plugin to run my compiled war.
Here is the relevant part of my pom.xml.
<plugin>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<version>9.4.50.v20221201</version>
<configuration>
<war>${jway.webapps.dir}/myapp.war</war>
<scanIntervalSeconds>2</scanIntervalSeconds>
</configuration>
</plugin>
If I execute mvn jetty:run-war, my war is build and Jetty serves the app as expected.
I have configured scanIntervalSeconds to allow hot redeploy.
However, if I rebuild using mvn install, I get the following error during redeployment:
java.lang.IllegalStateException: Failed to delete temp dir F:\...\myproject\target\tmp
at org.eclipse.jetty.webapp.WebInfConfiguration.configureTempDirectory (WebInfConfiguration.java:532)
at org.eclipse.jetty.webapp.WebInfConfiguration.resolveTempDirectory (WebInfConfiguration.java:424)
at org.eclipse.jetty.webapp.WebInfConfiguration.preConfigure (WebInfConfiguration.java:140)
at org.eclipse.jetty.webapp.WebAppContext.preConfigure (WebAppContext.java:488)
at org.eclipse.jetty.webapp.WebAppContext.doStart (WebAppContext.java:523)
at org.eclipse.jetty.maven.plugin.JettyWebAppContext.doStart (JettyWebAppContext.java:397)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start (AbstractLifeCycle.java:73)
at org.eclipse.jetty.maven.plugin.JettyRunWarMojo.restartWebApp (JettyRunWarMojo.java:113)
at org.eclipse.jetty.maven.plugin.AbstractJettyMojo$1.filesChanged (AbstractJettyMojo.java:472)
at org.eclipse.jetty.util.Scanner.reportBulkChanges (Scanner.java:848)
at org.eclipse.jetty.util.Scanner.reportDifferences (Scanner.java:765)
at org.eclipse.jetty.util.Scanner.scan (Scanner.java:641)
at org.eclipse.jetty.util.Scanner$1.run (Scanner.java:558)
at java.util.TimerThread.mainLoop (Timer.java:555)
at java.util.TimerThread.run (Timer.java:505)
It seems that Jetty wants to delete the file, but Windows locks the file. In the plugin documentation, I have not found any configuration which seems to be helpful. Furthermore I have nothing found on Google. Is there any way to solve this issue?
I don't know if its relevant, but I do not use the jetty:run goal, because my war is build using a third party tool and I do not have a standard directory structure.
The jetty documentation contains a section about Troubleshooting Locked Files on Windows.
So I updated my plugin config according to the documentation:
<plugin>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<version>9.4.50.v20221201</version>
<configuration>
<war>${jway.webapps.dir}/myapp.war</war>
<scanIntervalSeconds>2</scanIntervalSeconds>
<webApp>
<_initParams>
<org.eclipse.jetty.servlet.Default.useFileMappedBuffer>false</org.eclipse.jetty.servlet.Default.useFileMappedBuffer>
</_initParams>
</webApp>
</configuration>
</plugin>

Automatic update spring boot docker app using ansible

We plan to release spring boot application to customer. In corrent process, every time we release a new version, we have to uninstall the rpm and re install using rpm.
We want to move towards auto update of our application whenever there is a new version available. Essentially customer does not have to deal with updating the applications.
Instead of rpm, we are thinking about docker image, packaged with all dependencies.
. How do achieve this functionality?
Based on what I have read so far, if we package we use ansible + docker to deploy the application, , ansible (using watch tower or custom code) will ping an IP and when discovers a new version, will stop the application, pull the application from destination and then deploy and start.
Is my understanding correct?
Steps to automate deployment
Docker
Create application and dockerise it with any JDK docker image. We have maven package to dockerise application.
Add Plugin to pom.xml
<plugin>
<groupId>com.spotify</groupId>
<artifactId>docker-maven-plugin</artifactId>
<version>0.4.10</version>
<configuration>
<imageName>job-demand-service-analytics-processor</imageName>
<dockerDirectory>src/main/docker</dockerDirectory>
<resources>
<resource>
<targetPath>/</targetPath>
<directory>${project.build.directory}</directory>
<include>${project.build.finalName}.jar</include>
</resource>
</resources>
</configuration>
</plugin>
OR
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>com.google.cloud.tools</groupId>
<artifactId>jib-maven-plugin</artifactId>
<version>${jib-maven-plugin.version}</version>
<configuration>
<from>
<image>adoptopenjdk:11-jre-hotspot</image>
</from>
<to>
<image>websockethandler:latest</image>
</to>
<container>
<entrypoint>
<shell>sh</shell>
<option>-c</option>
<arg>chmod +x /entrypoint.sh && sync && /entrypoint.sh</arg>
</entrypoint>
<ports>
<port>8090</port>
</ports>
<environment>
<SPRING_OUTPUT_ANSI_ENABLED>ALWAYS</SPRING_OUTPUT_ANSI_ENABLED>
<APP_SLEEP>0</APP_SLEEP>
<JWT_SECRET_KEY_BASE64>""</JWT_SECRET_KEY_BASE64>
..........
</environment>
<useCurrentTimestamp>true</useCurrentTimestamp>
</container>
</configuration>
</plugin>
</plugins>
There are many options to dockerise your Spring application
Git
Push your code to git server like Bitbucket, Gitlabs, Github and so on.
Create a master branch for production releases and develop branch for development.
Jenkins
Trigger a jenkins job to create a docker image when you commit or merge changes from develop to master when feature is commited to develop.This is based on the hooks we add at git server.
If the docker image created successfully trigger the job which deploys.
Reverse Proxy or Load Balancer
To achive zero downtime, use load balnacer(reverse proxy like NGINX with upstream) running your container in multiple replicas of the container.
This applies for any application. If you have special requirement, update your question with the basic example code and will update the answer with Dockerising and automation steps.

Best way to run spring boot 2 rest services

I am writing an app that exposes some functions via REST service.
To do this I am using Spring Boot 2, but what is the best way to put it on production environment?
Is a good idea run the jar using java?
Short answer
yes it is a good idea.
Long answer
Spring Boot features a plugin that prepends a service script (Unix-compatible) in the JAR file itself. That makes the JAR file executable in Unix/Linux environments and you can easily install it as a service. Excerpt from https://docs.spring.io/spring-boot/docs/current/reference/html/deployment-install.html follows:
To create a "fully executable" jar with Maven, use the following plugin configuration:
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<executable>true</executable>
</configuration>
</plugin>
</plugins>
If packaging for Windows, the startup script isn't of much use and can be omitted. You would need to run using java -jar ... on windows, or install a service wrapper. Another excerpt from the Spring Boot doco:
A Spring Boot application can be started as a Windows service by using winsw.
A (separately maintained sample) describes step-by-step how you can create a Windows service for your Spring Boot application.
You can make it fully executable with below code in your pom.xml. You can run with shell script or as systemv or initd service[Spring Boot DOC]. This is the best tutorial link I have found explaining the multiple run as service options. You might want to take a look at spring doc for production ready features.
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<executable>true</executable>
</configuration>
</plugin>
</plugins>
</build>

Spring Contract Maven Plugin pushStubsToScm does not work

I tried to use Spring Contract Maven Plugin in a producer side to upload the stubs jar and share it with consumers.
I am using Spring Cloud Contract 2.0.0.
I configured the spring-cloud-contract-maven-plugin in my project, the codes is uploaded to Github.
<plugin>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-contract-maven-plugin</artifactId>
<version>${spring-cloud-contract.version}</version>
<extensions>true</extensions>
<configuration>
<basePackageForTests>com.example.demo</basePackageForTests>
<baseClassMappings>
<baseClassMapping>
<contractPackageRegex>.*rest.*</contractPackageRegex>
<baseClassFQN>com.example.demo.RestVerifierBase</baseClassFQN>
</baseClassMapping>
</baseClassMappings>
<!-- We want to pick contracts from a Git repository -->
<!--<contractsRepositoryUrl>git://file://${project.basedir}/target/contract_git/</contractsRepositoryUrl>-->
<!-- Example of URL via git protocol -->
<!--<contractsRepositoryUrl>git://git#github.com:spring-cloud-samples/spring-cloud-contract-samples.git</contractsRepositoryUrl>-->
<!-- Example of URL via http protocol -->
<!--<contractsRepositoryUrl>git://https://github.com/spring-cloud-samples/spring-cloud-contract-samples.git</contractsRepositoryUrl>-->
<contractsRepositoryUrl>git://https://github.com/hantsy/contracts-git.git</contractsRepositoryUrl>
<!-- We reuse the contract dependency section to set up the path
to the folder that contains the contract definitions. In our case the
path will be /groupId/artifactId/version/contracts -->
<contractDependency>
<groupId>${project.groupId}</groupId>
<artifactId>${project.artifactId}</artifactId>
<version>${project.version}</version>
</contractDependency>
<!-- The mode can't be classpath -->
<contractsMode>REMOTE</contractsMode>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<!-- By default we will not push the stubs back to SCM,
you have to explicitly add it as a goal -->
<goal>pushStubsToScm</goal>
</goals>
</execution>
</executions>
</plugin>
When I ran command mvn clean install -DskipTests -DcontractsRepositoryUsername=hantsy -DcontractsRepositoryPassword=mypassword and got the failure.
Caused by: org.eclipse.jgit.errors.TransportException: https://github.com/hantsy/contracts-git.git: Authentication is required but no CredentialsProvider has been registered
Update: If I set contractsMode to LOCAL, it will fail with downloading stubs jar error, maybe similar with this issue, in the new version 2.0.0, Spring Cloud contracts can not resolve jars from Local maven repos as expected when stubsMode is set to LOCAL.
Update 2: I also tried to add contractsRepositoryUsername and contractsRepositoryPassword in spring contract maven plugin config, it does not work.
contractsMode needs to be REMOTE. I've taken your example and it works well with:
<contractsRepositoryUrl>git://git#github.com:marcingrzejszczak/contracts-git.git</contractsRepositoryUrl>
<contractsMode>REMOTE</contractsMode>
with passing the credentials manually I see sth absolutely bizarre...
I see [INFO] Passed username and password - will set a custom credentials provider which means that username and password got passed and I'm using them. But actually what I get is Authentication is required but no CredentialsProvider has been registered, which makes no sense. Can you file another bug for that? The workaround is to use the agent.
UPDATE:
The issue got fixed here https://github.com/spring-cloud/spring-cloud-contract/issues/678

Why is my ${user.home} variable resoled at build time

I have a JHipster generated application with an YAML property file that looks like this:
storage:
location: ${user.home}/my/folder
My problem is that the variable ${user.home} is resolved at build time, when I run mvn package (on Jenkins). So the property is already resolved in the resulting artifact, hence when I deploy on on my server, that path contains the resolved home of the user Jenkins.
Anybody know who is doing this and why? I was expecting that the variable would be resolved at runtime.
Thanks.
Valentin
I'm not totally sure of how JHipster builds on top of Spring Boot, but my guess would be that it's Maven's resource filtering that's expanding ${user.home} at build time. It's enabled by default by spring-boot-starter-parent for application.properties and application.yaml in src/main/resources.
This Spring Boot issue contains some more information, along with details of a configuration change that you may like to make so that ${…} entries are no longer filtered:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>2.6</version>
<configuration>
<encoding>UTF-8</encoding>
<delimiters>
<delimiter>#</delimiter>
</delimiters>
<useDefaultDelimiters>false</useDefaultDelimiters>
</configuration>
</plugin>

Resources