GCloud secrets not resolving in spring properties - spring

TL;DR: the GCP secrets are not resolved in bootstrap file but the sql starter requires an instance connection name and database name on bootstrap
I'm trying to incorporate GCP Secretmanager in a Spring Boot application that is running on Google App Engine and using the GCP SQL.
However the ${sm:// prefix doesn't seem to be resolved at bootstrap time.
For reference, this is my part of my pom. (I'm using the com.google.cloud dependencies) And I enable the spring profile "gcp"
<parent>
<artifactId>spring-boot-starter-parent</artifactId>
<groupId>org.springframework.boot</groupId>
<relativePath/>
<version>2.4.2</version> <!-- lookup parent from repository -->
</parent>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>spring-cloud-gcp-dependencies</artifactId>
<version>2.0.1</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<!-- ... -->
<!-- cloud -->
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>spring-cloud-gcp-starter</artifactId>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>spring-cloud-gcp-starter-sql-postgresql</artifactId>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>spring-cloud-gcp-starter-secretmanager</artifactId>
</dependency>
And in my bootstrap (for example)
spring:
cloud:
gcp:
sql:
database-name: ${sm://some-fancy-db-secret}
instance-connection-name: ${sm://some-cool-connection-name}
When deploying I get an exception stating that a database-name needs to be defined.
If I fill in the plain properties it works just fine.
Even the ${sm://db-username} works inside my application-gcp.yml file.
When I move the property from the bootstrap file it also fails.
It seems it needs the connection when bootstrapping. (I'm a bit in the dark about that)
There's no fancy multi-project going on, and yes the secret exists.
I have a feeling I'm missing something stupid here or there's a version mismatch somewhere.
(The Codelab doesn't seem to be mentioning anything special at all.)
I also checked this question. However the proposed answer doesn't seem to be valid anymore. The com.google.cloud.spring.autoconfigure.secretmanager.GcpSecretManagerProperties don't even contain a prefix property, besides it works just fine in my normal properties file.

Has been fixed in the latest release.

Related

Determine compatible Versions of Spring Modules and other libraries

Which is the best way to determine the versions for my Spring modules and other libraries in a Spring Boot project?
Versions for Spring Modules
Versions for Third Party libraries
There are a lot of question when you search for "Compatible versions in Spring".
But all these questions are regarding specific libraries.
I would like to have general rules of how to determine compatible versions for my project.
You may checkout my post on this A comprehensive list of dependencies managed by latest Spring-Boot 2.3.2.RELEASE (as a custom parent)
That's is just a reference for how it should be, but you may discard the version in as they will be managed indirectly by spring itself.
Ok so, this is something almost every spring developer stumble upon. Let me try to explain this how was I able to resolve all the managed and third party libraries.
Let's suppose you want to build spring-boot microservices with a centralized configuration server.
So we can take it as following modules:
A company project starter: acts as a parent, managing the dependency
A config-server
A config-client
and let's suppose you chose spring-boot 2.3.2 version, which I used and find more stable. You would expect all the managed ones are using this spring-boot 2.3.2 version directly or indirectly.
I would highly recommend using https://mvnrepository.com
artifact: my_company-boot-starter-parent
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.3.2.RELEASE</version>
</parent>
<groupId>com.my_company</groupId>
<artifactId>my_company-boot-starter-parent</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>pom</packaging>
<name>my_company starter-parent</name>
<dependencyManagement>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<!-- ... -->
<!-- ... -->
<!-- other dependencies -->
<!-- ... -->
<!-- ... -->
</dependencyManagement>
If you got to Spring Boot Starter Parent » 2.3.2.RELEASE, under Managed Dependencies you can see the comprehensive list of all the dependencies with default version that would be referenced implicitly with boot version 2.3.2, see column Version.
The Updates column indicate that these managed dependencies are having their newest updates as this, but you are not required to override the dependency version of managed ones. If you intent to use more recent version, you have to chose more recent version of spring-boot-starter-parent. So let the spring download all the managed one itself.
With spring-boot-starter-parent 2.3.2 , they do not mention which spring-cloud-dependencies verion you should use, and this is where we get stuck and we need to figure it out.
Let us got to spring-cloud-dependencies . Hee you can see numerous version but which one to chose, it's like verifying which latest version uses spring-boot 2.3.2 indirectly.
You need to follow the managed dependency and go along with it until you find your parent version.
For example If you go for :
Hoxton.SR6
Spring Cloud Dependencies(Hoxton.SR6) -> Spring Cloud Config Dependencies( 2.2.3.RELEASE) -> Spring Cloud Starter Config(2.2.3.RELEASE) -> Spring Cloud Starter(2.2.3.RELEASE) -> Spring Boot Starter(2.3.0.RELEASE)
Here you can see, we end up using Spring Boot Starter(2.3.0.RELEASE) which is not what we expected it to be.
Hoxton.SR7
Spring Cloud Dependencies(Hoxton.SR7) -> Spring Cloud Config Dependencies(2.2.4.RELEASE) -> Spring Cloud Starter Config(2.2.4.RELEASE) -> Spring Cloud Starter(2.2.4.RELEASE) -> Spring Boot Starter(2.3.2.RELEASE)
Here we end up using same boot version 2.3.2. So in your parent pom.xml, you can set the spring cloud version as :
<properties>
<java.version>15</java.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<lombok.version>1.18.20</lombok.version>
<spring-cloud.version>Hoxton.SR7</spring-cloud.version>
</properties>
And in child poms (jars), you can just use the dependencies justby mentioning the group and artifact, skipping the version.
sample:
<parent>
<groupId>com.my_company</groupId>
<artifactId>my_company-boot-starter-parent</artifactId>
<version>1.0-SNAPSHOT</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-server</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>com.google.googlejavaformat</groupId>
<artifactId>google-java-format</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
</dependency>
</dependencies>
See, we have not provided the any version here, so spring-cloud-config-server version would be auto managed and it would be 2.2.4.RELEASE which again uses spring-boot-starter-web and spring-boot-starter both of 2.3.2 version.
Third party libraries
This is somewhat based on language version. You might want to use the latest third-party libs which is most recent till your language version supports it.
Like lombok: 1.18.20
Hope this might have helped you and others and provides an approach towards version compatibility.
Tips: Never forget to check the Managed Dependency Coordinates in Aappendix of all the spring boot release page as they keep their managed dependencies & version there. Like this one Appendix F: Dependency versions
1. Use Spring Initializr
Select all Spring modules you need in spring initializr and generate your code: https://start.spring.io/
You do not need to use the full generated code. But you should copy the library versions out of the generated pom.xml.
2. Watch out for dependency pom.xml
If available, import dependency pom.xml in your dependencyManagement.
Use the versions provided by these dependency poms.
E.g.
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
Spring Dependency Version Documentation
You can also check the Spring Dependency Version Documentation. But I prefer the initilizr, because it's easier to handle.

Spring cloud stater client not working in existing project

I was working on a project and then add eureka client dependency in my project but it showing following error.
Cannot resolve org.springframework.cloud:spring-cloud-starter-netflix-eureka-client:unknown
Spring version is: 2.3.1
Maven:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
But when a create a new project from initializer with eureka client dependency its working fine.
now i am at a lost what to do .
My guess is that what's going on here is that for your existing project, you have not included anything in your pom.xml stream that defines the version of the spring-cloud-starter-netflix-eureka-client package you want to use. Notice the unknown at the end of the error message. But when you create a new project with Initializr, you are getting a pom.xml setup that does define a version number for this package.
There is likely some other dependency, possibly spring-cloud-starter-parent or spring-cloud-dependencies, that you need to add to your pom.xml to define version numbers for all the Spring Cloud packages.
I found out that
following setting in pom.xml solves the error
<properties>
<java.version>11</java.version>
<spring-cloud.version>Hoxton.SR8</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
Follow the steps:
If you have installed spring boot in your eclipse or you have STS IDE then click on your existing project.
Select spring and then edit starter.
Now spring initializer window will open and here you have to select dependencies as per your need.
Select spring and then edit starter screen
spring initializer window

Spring boot 2.1.7 having tomcat-embed-core conflit

I am migrating existing Spring project into Spring boot.unable to run spring boot application its showing following error.
The error log says there is a conflict on tomcat-embed-core.
In eclipse Dependency hierarchy of porm.xml is given below
i exclude the maven architect ,and try to run the application its showing following error
porm.xml
<modelVersion>4.0.0</modelVersion>
<artifactId>MyService</artifactId>
<packaging>jar</packaging>
<properties>
<java.version>1.8</java.version>
<!-- 2.1.3.RELEASE -->
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<springframework.boot.version>2.1.7.RELEASE</springframework.boot.version>
</properties>
<name>MyService</name>
<url>http://maven.apache.org</url>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>com.sybase.jdbc3.jdbc</groupId>
<artifactId>jconn3</artifactId>
<version>${jconn3.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.tomcat</groupId>
<artifactId>tomcat-juli</artifactId>
<version>${tomcat.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.tomcat</groupId>
<artifactId>tomcat-jdbc</artifactId>
<version>${tomcat.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
what was wrong in this porm.xml
Where is
${tomcat.version}
defined?
That version probably does not match the tomcat version that auto magically is included with spring boot items.
And thus the conflict.
Go here:
https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-web/2.1.7.RELEASE
And start following the COMPILE dependencies, and you'll find the versions that are auto included with 2.1.7.RELEASE. and you have to alter the other includes that are overwriting the springboot auto include tomcat versions.
Again, follow the COMPILED dependency trail.
So below is what you should find by crawling the COMPILED dependency trail (from immediately above in my answer)
https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-tomcat/2.1.7.RELEASE
And you'll find you need to set
tomcat.version to
9.0.22
By defining tomcat.version as 8.x, you are breaking it.
Another way to put it
You have to go ~way~ back to springboot 1.5.2.RELEASE or 1.5.3.RELEASE
https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-tomcat/1.5.2.RELEASE
https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-tomcat/1.5.3.RELEASE
(Again, in the two above links, looked at the COMPILE dependencies)
To find a version of tomcat (that is auto included with springboot) that gets close to tomcat 8.5.x (where 8.5.x is the one you are attempting to use)
That's pretty old.
The principal you are missing is that springboot auto includes dependencies. And anything else you import has to play nice with everything springboot auto includes.
And your current value for tomcat.version is NOT playing nice with everything springboot 2.1.7.RELEASE is auto including.
And now that you've been through all of that. You'll find you'll make your life easier if you engage the springboot world more completely.
Alot of times, springboot will have a (sub)package that will bring in the thing you really desire.
spring-boot-starter-jdbc
https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-jdbc/2.1.7.RELEASE
You would probably be better off bringing that package in, vs hand-picking ones. Aka, get rid of your "tomcat-jdbc" include and see if the spring-boot-starter-jdbc can give you what you want.
The curse/blessing of spring-boot is that it is its own universe. But if you engage, you probably want to play by its rules more often than not.
PS
It is pom.xml, not porm.xml
Try adding spring-boot-starter-tomcat as a dependency:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
<scope>provided</scope>
</dependency>
Remove tomcat-juli and tomcat-jdbc dependencies. If you need JDBC support, add the corresponding starter:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
If you use JSP views, you will probably need the following dependencies as well:
<dependency>
<groupId>org.apache.tomcat.embed</groupId>
<artifactId>tomcat-embed-jasper</artifactId>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
</dependency>
Also, pay attention to your dependencies versions. Spring Boot's parent POM defines version management for many common artifacts so you don't need to set the <version></version> for these libraries. See https://docs.spring.io/spring-boot/docs/2.1.7.RELEASE/reference/htmlsingle/#appendix-dependency-versions

PCF Unable to use Config Server and Service Registry together in an Application

I am trying to bind the PCF Config Server and PCF Service registry in the same Application and push the app . However I get an error while the application is starting . The problem is due to the application unable to send the Hertbeat information and it fails with unable to refresh cache.
If i bind config server alone and Service registry alone it works but when both the services are bound it fails.
Any help with this is appreciated
The dependencies are as follows
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.5.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<dependency>
<groupId>io.pivotal.spring.cloud</groupId>
<artifactId>spring-cloud-services-starter-service-registry</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
<dependency>
<groupId>io.pivotal.spring.cloud</groupId>
<artifactId>spring-cloud-services-starter-config-client</artifactId>
</dependency>
This was resolved after gave the Service registry to take the path using the following configuration
spring.cloud.services.registrationMethod=route

Neither Spring Boot Actuator + Hal-Browser nor DevTools working

Updated: Attached pom.xml and application.property and some more error / warning msgs.
Hi I am very new to Spring Boot and just taking some classes on Udemy for it.
While following through a class, I create a starter project via spring.io and added actuator and Hal browser dependencies including Dev tools.
Just ran my app and tried to go to localhost:8080/application & /browser as well but I get 404.
What am I doing wrong?
I wrote a simple bean which returns a hard coded value and then print that, I changed the value to test Dev tools and it didn’t restart the resource , I had to kill and restart the app to reflect the new changes ..
How can I check what the issue ?
I can provide console grab if needed.
Please help.
Update: I don't know the significance of the following so putting it in here.
in the XML editor hal is red underlined with the following msg on hover:
The managed version is 3.0.5.RELEASE The artifact is managed in org.springframework.data:spring-data-releasetrain:Kay-SR5
in the XML editor devtools is red underlined with the following msg on hover:
The managed version is 2.0.0.RELEASE The artifact is managed in org.springframework.boot:spring-boot-dependencies:2.0.0.RELEASE
pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.myoxigen.training.springboot</groupId>
<artifactId>library</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>library</name>
<description>Demo project for Spring Boot</description>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.0.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-rest-hal-browser</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
application.properties file:
logging.level.org.springframework = DEBUG
management.security.enabled=false
Note: This answer is based on Spring Boot 2 (paths and properties have changed from version 1).
Your pom.xml looks fine to me. To get the "HAL Browser" to work with actuator you should have starters: web, actuator, and Rest Repositories HAL Browser. I recommend Spring Initializr as a nice way to construct an initial valid project structure.
The default path for the actuator is /actuator. Health, for example, is at /actuator/health.
To view the actuator endpoints in the HAL Browser, go to /browser/index.html#/actuator.
You can change the actuator path in application.properties by setting the following.
management.endpoints.web.base-path=/actuator
To run your server, use the following console command:
./mvnw clean spring-boot:run
If you have DevTools in your project, then changes to files in your classpath will restart the server. See my further comments on how to take advantage of DevTools here.
Here's the relevant documentation on a single page for easy searching:
https://docs.spring.io/spring-boot/docs/2.0.0.RELEASE/reference/htmlsingle
https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle
Add this Dependency:
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-rest-hal-browser</artifactId>
<version>3.3.5.RELEASE</version>
</dependency>`enter code here`
And then go to link:
http://localhost:8080/browser/index.html#
Spring Boot 2.2 apps should use spring-data-rest-hal-explorer rather than spring-data-rest-hal-browser
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-rest-hal-explorer</artifactId>
</dependency>
For enabling actuator, add this into the property file (application.properties)
management.endpoints.web.exposure.include=*
Restart the Spring boot application and access the actuator endpoints using:
http://localhost:8080/actuator/
the health of the application can also be monitored using:
http://localhost:8080/actuator/health
dependency required:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
Ain't you were studying a course by "in28minutes"? =) Anyway when I studied on a Spring Boot for Beginners in 10 Steps course, I faced the same issue with accessing http://localhost:8080/application.
For http://localhost:8080/browser I got a plain json:
{"cause":null,"message":"Fragment must not be empty"}
In pom.xml I had the same dependencies as you outlined in question.
Some googling brought me to this article. In section named "II. Hal Browser" you can see a screenshot, where in browser a link is http://localhost:8080/browser/index.html#. If use this link, then Hal Browser should magically appear.
So /index.html# was the only puzzle piece you missed.
Note: above approach worked with Spring Boot version 2.1.7
EDIT: despite the fact that Hal Browser itself works with above approach, it still doesn't show actuator endpoints. And DevTools also ain't working for me.

Resources