How to read liquibase.properties dynamically from password hashicorp vault - spring-boot

In my Spring Boot project, I am trying to setup liquibase and use it between dev, test and production databases. Everything seems to be working fine, except passing credentials to liquibase.properties file from HashiCorp Vault. I am able to access credentials in application.properties without any issues, but I can't in liquibase.properties file. I have the following file and I would like pass URLs and credentials dynamically from password vault.
liquibase.properties
changeLogFile=src/main/resources/liquibase-changeLog.xml
url=jdbc:mysql://localhost:3306/oauth_reddit
username=tutorialuser
password=tutorialmy5ql
driver=com.mysql.jdbc.Driver
referenceUrl=hibernate:spring:org.baeldung.persistence.model
?dialect=org.hibernate.dialect.MySQLDialect
diffChangeLogFile=src/main/resources/liquibase-diff-changeLog.xml

liquibase.properties is used by liquibase directly. I'm not sure that spring is somehow modifying the liquibase.properties, it's probably used only by maven plugin. So you will need to create some additional parser in liquibase which is able to use Vault or just forget about liquibase.properties and use spring's properties.

Below code fetches the db details from vault injects into data source, this datasource is used by liquibase to connect and execute the scripts
build.gradle.kts
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
plugins {
id("org.springframework.boot") version "2.4.4"
id("io.spring.dependency-management") version "1.0.11.RELEASE"
kotlin("jvm") version "1.4.31"
kotlin("plugin.spring") version "1.4.31"
}
group = "com.db"
version = "0.0.1-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_1_8
configurations {
compileOnly {
extendsFrom(configurations.annotationProcessor.get())
}
}
repositories {
mavenCentral()
}
dependencies {
implementation("org.springframework.boot:spring-boot-starter-data-jpa")
implementation("org.springframework.cloud:spring-cloud-starter-bootstrap:3.0.2")
implementation("org.jetbrains.kotlin:kotlin-reflect")
implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
implementation("org.liquibase:liquibase-core:4.3.2")
implementation(files("libs/ojdbc6.jar"))
implementation("org.springframework.cloud:spring-cloud-starter-vault-config:3.0.2")
}
tasks.withType<KotlinCompile> {
kotlinOptions {
freeCompilerArgs = listOf("-Xjsr305=strict")
jvmTarget = "1.8"
}
}
tasks.withType<Test> {
useJUnitPlatform()
}
bootstrap.properties
spring.cloud.vault.application-name=database-config
spring.cloud.vault.token=XXXXX
spring.cloud.vault.scheme=http
spring.cloud.vault.kv.enabled=true
spring.cloud.vault.host=localhost
spring.cloud.vault.port=8200
application.properties
logging.level.liquibase=DEBUG
spring.liquibase.change-log=classpath:db/changelog.xml
spring.liquibase.enabled=true
VaultDBConfig
import org.springframework.boot.context.properties.ConfigurationProperties
#ConfigurationProperties("db")
class VaultDBConfig {
var username: String? = null
var password: String? = null
var url: String? = null
}
DatabaseConfig
import oracle.jdbc.pool.OracleDataSource
import java.sql.SQLException
import org.slf4j.Logger
import org.slf4j.LoggerFactory
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.context.annotation.Primary
import org.springframework.context.annotation.Profile
import org.springframework.core.env.Environment
import javax.sql.DataSource
#Configuration
class DatabaseConfig(private val dbDetails: VaultDBConfig, private val environment: Environment) {
val logger: Logger = LoggerFactory.getLogger(DatabaseConfig::class.java)
#Primary
#Bean
#Throws(SQLException::class)
fun dataSource(): DataSource? {
val oracleDataSource = OracleDataSource()
oracleDataSource.setURL(dbDetails.url)
oracleDataSource.setUser(dbDetails.username)
oracleDataSource.setPassword(dbDetails.password)
return oracleDataSource
}
}
Enable config properties in application.kt
#SpringBootApplication
#EnableConfigurationProperties(VaultDBConfig::class)
class ConfigApplication
fun main(args: Array<String>) {
runApplication<ConfigApplication>(*args)
}
Vault insert
vault kv put secret/database-config db.username=xxx db.password=xxx dp.url=xxx

Related

MapStruct not injected in Kotlin project

I am trying to create a project with current Kotlin, MapStruct and Java using Spring-Boot folowing some online examples, as I am new to MapStruct, however I am not able to inject the mapper into my service. Both Idea and Gradle (in build task test) complain that no bean has been found (UnsatisfiedDependencyException). Googling didn't help. What am I missing?
MWE:
build.gradle.kts
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
plugins {
id("org.springframework.boot") version "2.7.4"
id("io.spring.dependency-management") version "1.0.14.RELEASE"
kotlin("jvm") version "1.7.20"
kotlin("plugin.spring") version "1.7.20"
kotlin("plugin.jpa") version "1.7.20"
}
group = "com.example"
version = "0.0.1-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_17
configurations {
compileOnly {
extendsFrom(configurations.annotationProcessor.get())
}
}
repositories {
mavenCentral()
}
dependencies {
implementation("org.springframework.boot:spring-boot-starter-data-jpa")
implementation("org.springframework.boot:spring-boot-starter-web")
implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
implementation("org.jetbrains.kotlin:kotlin-reflect")
implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
annotationProcessor("org.springframework.boot:spring-boot-configuration-processor")
testImplementation("org.springframework.boot:spring-boot-starter-test")
implementation("org.mapstruct:mapstruct:1.5.3.Final")
annotationProcessor("org.mapstruct:mapstruct-processor:1.5.3.Final")
}
tasks.withType<KotlinCompile> {
kotlinOptions {
freeCompilerArgs = listOf("-Xjsr305=strict")
jvmTarget = "17"
}
}
tasks.withType<Test> {
useJUnitPlatform()
}
DemoAppllication.kt
package com.example.demo
import org.springframework.boot.autoconfigure.SpringBootApplication
import org.springframework.boot.runApplication
#SpringBootApplication
class DemoApplication
fun main(args: Array<String>) {
runApplication<DemoApplication>(*args)
}
App.kt
package com.example.demo
import org.mapstruct.Mapper
import org.springframework.data.jpa.repository.JpaRepository
import org.springframework.stereotype.Repository
import org.springframework.stereotype.Service
import javax.persistence.*
#Entity
#Table(name = "items")
class Item(#Id var id: Int = 0)
data class ItemDto(val id: Int)
#Repository
interface ItemRepo : JpaRepository<Item, Int>
#Mapper(componentModel = "spring")
interface ItemMapper {
fun entToDto(item: Item) : ItemDto
fun entsToDtos(items: List<Item>) : List<ItemDto>
fun dtoToEnt(itemDto: ItemDto) : Item
}
#Service
class Srvc(private val itemMapper: ItemMapper, // XXX: no bean found for this one
private val repo: ItemRepo)
{
fun items() = itemMapper.entsToDtos(repo.findAll())
}
// controller skipped
Kapt is in maintenance mode! See example: https://github.com/mapstruct/mapstruct-examples/tree/main/mapstruct-kotlin
plugin {
// ...
kotlin("kapt")
// ...
}
dependencies {
// ...
kapt("org.mapstruct:mapstruct-processor:$version")
implementation("org.mapstruct:mapstruct:$version")
// ...
}
EDIT (by mpts.cz — for future me or anyone as new to Mapstruct and/or Gradle as I am):
use plugin kotlin("kapt")
replace annotationProcessor("org.mapstruct:mapstruct-processor:$version")
with kapt("org.mapstruct:mapstruct-processor:$version")
put the previous line before implementation("org.mapstruct:mapstruct:$version")
With these changes all seems to work smoothly. Thanks Numichi!

How to fix unresolvable circular reference when using Sentry with Spring Boot 2.6.x instead of 2.5.x?

With org.springframework.boot version 2.5.9, thinks work fine, but with 2.6.0 (2.6.1, 2.6.2, 2.6.3), I get the following error:
org.springframework.beans.factory.BeanCurrentlyInCreationException: Error creating bean with name 'sentryOptions': Requested bean is currently in creation: Is there an unresolvable circular reference?
(full log: https://gist.github.com/Dobiasd/be7810282a06b538ccee0078ab2267aa)
Here is my minimal example to reproduce the issue:
Application.kt:
package com.acme.foo.bar
import io.sentry.spring.EnableSentry
import org.springframework.boot.autoconfigure.SpringBootApplication
import org.springframework.boot.runApplication
import org.springframework.context.annotation.Configuration
#EnableSentry
#Configuration
class SentryConfiguration
#SpringBootApplication
class Application
fun main(args: Array<String>) {
runApplication<Application>(*args)
}
IntegrationTest.kt:
package com.acme.foo.bar.integration
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.extension.ExtendWith
import org.springframework.boot.test.context.SpringBootTest
import org.springframework.test.context.ActiveProfiles
import org.springframework.test.context.junit.jupiter.SpringExtension
#ExtendWith(SpringExtension::class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class FullStackTest {
#Test
fun init_context() {
}
}
build.gradle.kts
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
plugins {
id("org.springframework.boot") version "2.6.3"
id("io.spring.dependency-management") version "1.0.11.RELEASE"
kotlin("jvm") version "1.6.10"
kotlin("plugin.spring") version "1.6.10"
}
group = "com.acme"
version = "1.0.0-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_11
repositories {
mavenCentral()
}
dependencies {
implementation(group = "org.springframework.boot", name = "spring-boot-starter-web")
implementation(group = "io.sentry", name = "sentry-spring", version = "5.5.3")
testImplementation(group = "org.springframework.boot", name = "spring-boot-starter-test")
testImplementation(group = "org.jetbrains.kotlin", name = "kotlin-test-junit")
}
tasks {
withType<KotlinCompile> {
kotlinOptions {
freeCompilerArgs = listOf("-Xjsr305=strict")
jvmTarget = "11"
}
}
withType<Test> {
testLogging.exceptionFormat = org.gradle.api.tasks.testing.logging.TestExceptionFormat.FULL
useJUnitPlatform()
}
}
Any ideas what I'm doing wrong? Or maybe sentry-spring just does work with Spring Boot 2.6.x (yet)?
Found the solution. Instead of io.sentry:sentry-spring one has to use io.sentry:sentry-spring-boot-starter in the dependencies and then remove the following from the code:
#EnableSentry
#Configuration
class SentryConfiguration
I've just tested in my actual project, and logged errors are still sent to Sentry correctly.

How to listen to two RabbitMQ queues with spring-cloud-stream

I've got a working application that listens to a single RabbitMQ queue.
However, when I add another bean that consumes messages and try to bind that to another queue, neither of the queues are created in RabbitMQ and when creating them manually no messages are consumed from these queues.
Small kotlin project I created to demonstrate the issue:
#SpringBootApplication
class SpringCloudStreamTwoRabbitConsumersApplication
fun main(args: Array<String>) {
runApplication<SpringCloudStreamTwoRabbitConsumersApplication>(*args)
}
package com.example.springcloudstreamtworabbitconsumers
import org.springframework.context.annotation.Bean
import org.springframework.messaging.Message
import org.springframework.stereotype.Component
import java.util.function.Consumer
#Component
class Listener1Config {
#Bean
fun listener1(): Consumer<Message<String>> {
return Consumer { input -> println(input) }
}
}
package com.example.springcloudstreamtworabbitconsumers
import org.springframework.context.annotation.Bean
import org.springframework.messaging.Message
import org.springframework.stereotype.Component
import java.util.function.Consumer
#Component
class Listener2Config {
#Bean
fun listener2(): Consumer<Message<String>> {
return Consumer { input -> println(input) }
}
}
application.properties:
# Rabbit properties
spring.rabbitmq.host=localhost
spring.rabbitmq.port=5672
spring.rabbitmq.username=guest
spring.rabbitmq.password=guest
# Listener 1
spring.cloud.stream.bindings.listener1-in-0.destination=exchange1
spring.cloud.stream.bindings.listener1-in-0.group=exchange1-queue
spring.cloud.stream.rabbit.bindings.listener1-in-0.consumer.queueNameGroupOnly=true
spring.cloud.stream.rabbit.bindings.listener1-in-0.consumer.binding-routing-key-delimiter=,
spring.cloud.stream.rabbit.bindings.listener1-in-0.consumer.bindingRoutingKey=binding.key.1,binding.key.1.1
spring.cloud.stream.rabbit.bindings.listener1-in-0.consumer.exchangeType=topic
spring.cloud.stream.rabbit.bindings.listener1-in-0.consumer.autoBindDlq=true
# Listener 2
spring.cloud.stream.bindings.listener2-in-0.destination=exchange2
spring.cloud.stream.bindings.listener2-in-0.group=exchange2-queue
spring.cloud.stream.rabbit.bindings.listener2-in-0.consumer.queueNameGroupOnly=true
spring.cloud.stream.rabbit.bindings.listener2-in-0.consumer.binding-routing-key-delimiter=,
spring.cloud.stream.rabbit.bindings.listener2-in-0.consumer.bindingRoutingKey=binding.key.2,binding.key.2.1
spring.cloud.stream.rabbit.bindings.listener2-in-0.consumer.exchangeType=topic
spring.cloud.stream.rabbit.bindings.listener2-in-0.consumer.autoBindDlq=true
build.gradle.kts:
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
plugins {
id("org.springframework.boot") version "2.4.3"
id("io.spring.dependency-management") version "1.0.11.RELEASE"
kotlin("jvm") version "1.4.30"
kotlin("plugin.spring") version "1.4.30"
}
group = "com.example"
version = "0.0.1-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_1_8
repositories {
mavenCentral()
}
extra["springCloudVersion"] = "2020.0.1"
dependencies {
implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
implementation("org.springframework.boot:spring-boot-starter-webflux")
implementation("org.jetbrains.kotlin:kotlin-reflect")
implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
implementation("org.springframework.cloud:spring-cloud-stream")
implementation("org.springframework.cloud:spring-cloud-stream-binder-rabbit")
testImplementation("org.springframework.boot:spring-boot-starter-test")
}
dependencyManagement {
imports {
mavenBom("org.springframework.cloud:spring-cloud-dependencies:${property("springCloudVersion")}")
}
}
tasks.withType<KotlinCompile> {
kotlinOptions {
freeCompilerArgs = listOf("-Xjsr305=strict")
jvmTarget = "1.8"
}
}
tasks.withType<Test> {
useJUnitPlatform()
}
When I comment out one of the listener beans the other one works as expected. However with both beans active, no queue is created in RabbitMQ, nor are messages read from the queues if I create them manually and send messages to the exchange.
What am I doing wrong here?
The framework can only detect a single function. When you have multiple, you need to specify:
spring.cloud.function.definition=listener1;listener2
https://docs.spring.io/spring-cloud-stream/docs/3.1.1/reference/html/spring-cloud-stream.html#spring_cloud_function
In the event you only have single bean of type java.util.function.[Supplier/Function/Consumer], you can skip the spring.cloud.function.definition property, since such functional bean will be auto-discovered. However, it is considered best practice to use such property to avoid any confusion.

Default profile values are not returned when specific profile selected in Spring Cloud Config

I'm moving from a local application.yml configuration file to a config server managed configuration.
My application.yml file contains 2 (or more) profiles in the same file, in the format:
spring.application.name: config-client
app.myvar1: "Var 1 in default profile"
app.myvar2: "Var 2 in default profile"
---
spring.config.activate.on-profile: docker
app.myvar1: "Var 1 in docker profile"
When I test this configuration file in a NOT config-server environment, I the result reading from the specific profile, and if not found, reading from default. Sample
Correct result when I test without config-server
Profile: docker
MyVar1=Var 1 in docker profile
MyVar2=Var 2 in default profile
For testing I'm using a simple program:
package com.bthinking.configclient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.core.env.Environment;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
#SpringBootApplication
#RestController
public class ConfigClientApplication {
#Autowired
private Environment environment;
#Value("${app.myvar1:MyVar1 not found}")
String myVar1;
#Value("${app.myvar2:MyVar2 not found}")
String myVar2;
public static void main(String[] args) {
SpringApplication.run(ConfigClientApplication.class, args);
}
#RequestMapping("/")
public String index() {
StringBuffer b = new StringBuffer();
for (String profile : environment.getActiveProfiles()) {
b.append(String.format("Profile: %s</br>", profile));
}
b.append(String.format("MyVar1=%s</br>", myVar1));
b.append(String.format("MyVar2=%s</br>", myVar2));
return b.toString();
}
}
And I start the program with (I use gradle):
gradle :config-client:bootRun --args='--spring.profiles.active=docker'
But, when I migrate to config server, moving the file to a config-repo (I'm using file based repor), I get the invalid result (it's unable to read the variable in the default section). I have also tried with --spring.profiles.active=docker,default with no change
Profile: docker
MyVar1=Var 1 in docker profile
MyVar2=MyVar2 not found
For reference, my config-server has the following configuration:
server.port: 8888
spring.application.name: config-server
spring.cloud.config.server.native.searchLocations: file:${PWD}/config-repo
# spring.cloud.config.server.native.searchLocations: file:/Users/jmalbarran/Projects/BTH/BTH/SPB_SpringBoot/bugs/config/config-repo
logging.level:
root: debug
---
spring.config.activate.on-profile: docker
spring.cloud.config.server.native.searchLocations: file:/config-repo
The main class
package com.bthinking.configserver;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.config.server.EnableConfigServer;
#EnableConfigServer
#SpringBootApplication
public class ConfigServerApplication {
public static void main(String[] args) {
SpringApplication.run(ConfigServerApplication.class, args);
}
}
And the build.gradle
plugins {
id 'org.springframework.boot' version '2.4.1'
id 'io.spring.dependency-management' version '1.0.10.RELEASE'
id 'java'
}
group = 'com.b-thinking'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '1.8'
repositories {
mavenCentral()
maven { url 'https://repo.spring.io/milestone' }
}
ext {
set('springCloudVersion', "2020.0.0")
}
dependencies {
implementation 'org.springframework.cloud:spring-cloud-config-server'
testImplementation 'org.springframework.boot:spring-boot-starter-test'
}
dependencyManagement {
imports {
mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}"
}
}
test {
useJUnitPlatform()
}
And I start it with:
gradle :config-server:bootRun --args='--spring.profiles.active=native'
NOTE: As I think this a bug, I have also created an issue in github. Check in Issue
After the #spencergibb (Thanks!) comment, I tried with the version 3.0.1, and it solves the problem.
This is now my build.gradle
plugins {
id 'org.springframework.boot' version '2.4.1'
id 'io.spring.dependency-management' version '1.0.10.RELEASE'
id 'java'
}
group = 'com.b-thinking'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '1.8'
repositories {
mavenCentral()
maven { url 'https://repo.spring.io/milestone' }
}
ext {
set('springCloudVersion', "2020.0.0")
}
dependencies {
implementation 'org.springframework.cloud:spring-cloud-config-server:3.0.1'
testImplementation 'org.springframework.boot:spring-boot-starter-test'
}
dependencyManagement {
imports {
mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}"
}
}
test {
useJUnitPlatform()
}
It's amazing, because the version 3.0.1 is reported solving the opposite bug (the default configuration override the specific), but I imagine that the review solved both.
For references, this were the related issues
Issue#1777 Profile not correct with SpringBoot 2.4.1 + Ilford 2020.0.0 (working with 2.4.1 + Ilford 2020.0.0-RC1)
Issue#1778 multidocument files from config server have the same property name

Test EntityManager with Junit5 Kotlin

I am trying to introduce to Spring JPA and I have difficulties running up my tests.
My gradle.build.kts looks like the following
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
plugins {
id("org.springframework.boot") version "2.4.0"
id("io.spring.dependency-management") version "1.0.10.RELEASE"
kotlin("jvm") version "1.4.10"
kotlin("plugin.spring") version "1.4.10"
kotlin("plugin.jpa") version "1.4.10"
}
group = "com.pluralsight"
version = "0.0.1-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_1_8
repositories {
mavenCentral()
jcenter()
google()
}
dependencies {
implementation("org.springframework.boot:spring-boot-starter-data-jpa")
implementation("org.jetbrains.kotlin:kotlin-reflect")
implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
runtimeOnly("com.h2database:h2")
testImplementation(platform("org.junit:junit-bom:5.7.0"))
testImplementation("org.junit.jupiter:junit-jupiter")
}
tasks.withType<KotlinCompile> {
kotlinOptions {
freeCompilerArgs = listOf("-Xjsr305=strict")
jvmTarget = "1.8"
}
}
tasks.withType<Test> {
useJUnitPlatform()
}
I am using Junit5 for my test framework. And what first I need to test is that my Flight Entity is created correctly. I am not using #SpringRunner since we are on Junit5 so I do the following:
package com.pluralsight.springdataoverview
import com.pluralsight.springdataoverview.entity.Flight
import org.junit.jupiter.api.Assertions
import org.junit.jupiter.api.Test
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.boot.test.autoconfigure.orm.jpa.DataJpaTest
import org.springframework.boot.test.context.SpringBootTest
import java.time.LocalDateTime
import javax.persistence.EntityManager
#SpringBootTest
#DataJpaTest
class SpringDataOverviewApplicationTests {
#Autowired
private val entityManager: EntityManager? = null
#Test
fun verifyFlighTCanBeSaved() {
var flight = Flight()
flight.origin = "London"
flight.destination = "New York"
flight.scheduledAt = LocalDateTime.parse("2011-12-13T12:12:00")
entityManager!!.persist(flight)
val flights = entityManager
.createQuery("SELECT f FROM Flight f", Flight::class.java)
.resultList
Assertions.assertEquals(flights.first(), flight)
}
}
And I have the following in red
What dependency I am missing ?
You have multiple declarations of configuration (that's what your error message is saying).
It's because you are using #SpringBootTest and #DataJpaTest in the same configuration class i.e SpringDataOverviewApplicationTests.
Use either of it and it should be fine.

Resources