I came across a problematic to which I can't find any nice solution. Some context: we work with several micro-services, most of which use rest clients. We found out that a lot of them will use similar configurations for similar issues (i.e. resiliency). Naturally, we want to extract common, heavily duplicated, non business code into a library. But here is the thing: How can I extract a #ConstructorBinding #ConfigurationProperties data class in a library (especially if there could be several instances of these classes in the code base that uses the library)?
Here is some example code:
#ConstructorBinding
#ConfigurationProperties(prefix = "rest.client")
data class MyDuplicatedRestClientProperties(
val host: String,
val someOtherField: Int,
val someFieldWithDefaultValue: String = "default value"
)
I would like to import this in a project to configure 2 different REST clients. I tried:
Creating an abstract class my ClientProperties would extend. Sadly, I need to expose all the fields of the parent class which doesn't really help with duplication:
abstract class MyAbstractClient(
val host: String,
val someOtherField: Int,
val someFieldWithDefaultValue: String = "default value"
)
#ConstructorBinding
#ConfigurationProperties(prefix = "rest.client")
class MyImplematationClient(
val host: String,
val someOtherField: Int,
val someFieldWithDefaultValue: String = "default value"
): MyAbstractClient(
host,
someOtherField,
someFieldWithDefaultValue
)
Instantiating the properties as a #Bean method with the #ConfigurationProperties but this doesn't work well either as it forces me to put fields with #Value in the #Configuration class:
#Configuration
class MyConfigurationClass {
#Value("${my.client.host}")
lateinit var host: String
#Value("${my.client.someOtherField}")
lateinit var someOtherField: Int
#Value("${my.client.someFieldWithDefaultValue:default value}")
lateinit var someFieldWithDefaultValue: String
#Bean
#ConfigurationProperties
fun myClient() = MyDuplicatedRestClientProperties(
host,
someOtherField,
someFieldWithDefaultValue
)
}
From my experience, you're on a wrong road. Why?
Duplication in microservices is allowed. Code is not too large, it's decoupled and can be easily changed.
From Distributed Systems theory, sharing classes between multiple components it's a bad thing. Why? Because doing this will couple the components via those classes.
A better approach will be to encapsulate all the integration into a specific library such as a REST client. For example, accessing Service A can be done via a service-a-client.jar which will contain the configuration and the integration that is necessary in order to call the Service A and will expose one or multiple interfaces that can be used as Spring Beans.
Putting the configuration into a library gives you no advantage, configurations are not business related, they are somehow synthetic objects and have no value in the architecture.
Related
i'm adding property validation to an existing big project. It has hundrets of webservices and there are some that have simple numbers as names.
Now im trying to write a data class using #Validated, #ConstructorBinding and #ConfigurationProperties.
So imagine a property
dummy.941=http:...
The name of the variable would need to be 941 now, as far as i can tell, but kotlin/java dont allow variable names starting with numbers.
#Validated
#ConstructorBinding
#ConfigurationProperties(value = "dummy", ignoreUnknownFields = false)
data class DummyProperties(
val abc: Abc = Abc(), ....
val 941: Ws941: Ws941()
)
Is there any workaround, some annotation, that says which property is meant? It is not possible to change the name of the property, since the same property database is in use different working systems and people told me thats off the table.
Thanks for any help!
EDIT:
I found a way, spring offers a #Name annotation (org.springframework.boot.context.properties.bind)
#Valid
#Name(value = "703")
val s703: S703 = S703(),
Works like a charm:)
Some time ago, I had a similar issue. You can solve it, at least for Java, by using a custom setter. I have no idea about Kotlin, but I assume it works in the same way for Spring Kotlin.
#ConfigurationProperties(value = "dummy", ignoreUnknownFields = false)
public class DummyProperties {
private Ws941 _941;
public void set941(Ws941 _941) {
this._941 = _941;
}
public Ws941 get941() {
return this._941;
}
}
Spring can map using the setter, so the variable can have a different name.
I am trying to set up my application's Spring Cloud config objects using a Kotlin data class. I've been successful with using #ConstructorBinding when defining classes that exist once at a specific node in the config but have run into a roadblock with objects that only appear in a child list.
Sample config:
toolbox:
tools:
- id: 1
description: hammer
- id: 2
description: screwdriver
What I would like to be able to do is this:
#ConstructorBinding
#ConfigurationProperties("toolbox")
data class ToolBox(val tools: List<Tool>) {
#ConstructorBinding
data class Tool (val id: String, val description: String)
}
But when I do this I receive errors like this:
Property: toolbox.tools[0].description
Value: hammer
Origin: "toolbox.tools[0].description" from property source "bootstrapProperties-https:/some-config.yml"
Reason: The elements [toolbox.tools[0].description,toolbox.tools[0].id,toolbox.tools[1].description,toolbox.tools[1].id] were left unbound.
My application is configured with #ConfigurationPropertiesScan and other uses of #ConstructorBinding work for me. For example, the following alternative configuration works:
#ConstructorBinding
#ConfigurationProperties("toolbox")
data class ToolBox(val tools: List<Tool>) {
class Tool {
lateinit var id: String
lateinit var description: String
}
}
I've tried adding #ConfigurationProperties to the Tool class but that annotation requires a prefix and I don't know of a way to define a prefix that addresses into a list like I am trying to use here.
Is there a way to use #ConstructorBinding with a configuration class that only appears within a list?
My scenario:
I'm building an app that uses Kotlin and SpringBoot 2.0.3. I'm trying to write all my unit tests in JUnit5. All 3 of these are new to me, so I'm struggling a bit.
I'm using a #ConfigurationProperties class (instead of #Value) to inject values from my application.yml into my Spring context.
#Configuration
#ConfigurationProperties(prefix = "amazon.aws.s3")
class AmazonS3Config {
val s3Enabled: Boolean = false
val region: String = ""
val accessKeyId: String = ""
val secretAccessKey: String = ""
val bucketName: String = ""
}
I then have a Kotlin class that is utilizing these properties, following Kotlin/Spring best practice to define the injected class as a constructor parameter.
class VqsS3FileReader(val amazonS3Config: AmazonS3Config) : VqsFileReader {
companion object: mu.KLogging()
override fun getInputStream(filePath: String): InputStream {
val region: String = amazonS3Config.region
val accessKeyId: String = amazonS3Config.accessKeyId
val secretAccessKey: String = amazonS3Config.secretAccessKey
val bucketName: String = amazonS3Config.bucketName
logger.debug { "The configured s3Enabled is: $s3Enabled" }
logger.debug { "The configured region is: $region" }
logger.debug { "The configured accessKeyId is: $accessKeyId" }
logger.debug { "The configured secretAccessKey is: $secretAccessKey" }
logger.debug { "The configured bucketName is: $bucketName" }
val file: File? = File(filePath)
//This method is not yet implemented, just read a file from local disk for now
return file?.inputStream() ?: throw FileNotFoundException("File at $filePath is null")
}
}
I have not completed this implementation, as I'm trying to get the unit test working first. So for the moment, this method doesn't actually reach out to S3, just streams a local file.
My unit test is where I'm getting stuck. I don't know how to inject the properties from my application.yml into the test context. Since the ConfigProperty class is passed as a construction parameter, I have to pass it when I establish my service in my unit test. I've tried various solutions that don't work. I found this piece of info, which was helpful:
If Spring Boot is being used, then #ConfigurationProperties instead of #Value annotations can be used, but currently this only works with lateinit or nullable var properties (the former is recommended) since immutable classes initialized by constructors are not yet supported.
So this means I cannot use class VqsS3FileReaderTest(amazonS3Config: AmazonS3Config): TestBase() { ... } and then pass the config to my service.
This is what I have currently:
#ActiveProfiles("test")
#TestInstance(TestInstance.Lifecycle.PER_CLASS)
#ExtendWith(SpringExtension::class)
#ContextConfiguration(classes = [AmazonS3Config::class, VqsS3FileReader::class])
class VqsS3FileReaderTest(): TestBase() {
#Autowired
private lateinit var amazonS3Config: AmazonS3Config
#Autowired
private lateinit var fileReader: VqsS3FileReader
val filePath: String = "/fileio/sampleLocalFile.txt"
#Test
fun `can get input stream from a valid file path` () {
fileReader = VqsS3FileReader(amazonS3Config)
val sampleLocalFile: File? = getFile(filePath) //getFile is defined in the TestBase class, it just gets a file in my "resources" dir
if (sampleLocalFile != null) {
val inStream: InputStream = fileReader.getInputStream(sampleLocalFile.absolutePath)
val content: String = inStream.readBytes().toString(Charset.defaultCharset())
assert.that(content, startsWith("Lorem Ipsum"))
} else {
fail { "The file at $filePath was not found." }
}
}
}
With this, my test runs, and my context seems to setup properly, but the properties from my application.yml are not being injected. For my debug output, I see the following:
08:46:43.111 [main] DEBUG com.ilmn.vqs.fileio.VqsS3FileReader - The configured s3Enabled is: false
08:46:43.111 [main] DEBUG com.ilmn.vqs.fileio.VqsS3FileReader - The configured region is:
08:46:43.112 [main] DEBUG com.ilmn.vqs.fileio.VqsS3FileReader - The configured accessKeyId is:
08:46:43.112 [main] DEBUG com.ilmn.vqs.fileio.VqsS3FileReader - The configured secretAccessKey is:
08:46:43.112 [main] DEBUG com.ilmn.vqs.fileio.VqsS3FileReader - The configured bucketName is:
All empty strings, which is the default values. Not the values I have in my application.yml:
amazon.aws.s3:
s3Enabled: true
region: us-west-2
accessKeyId: unknown-at-this-time
secretAccessKey: unknown-at-this-time
bucketName: test-bucket
I see mistake in the following line:
#ContextConfiguration(classes = [AmazonS3Config::class, VqsS3FileReader::class])
Please put configuration classes here (instead of just beans).
Short - hot to fix test
Create class (if missing) like VqsS3Configration in the main module (e.g. in the module, where you have production code)
Create class like VqsS3TestConfigration in the same package with your tests. Content on this file:
#org.springframework.context.annotation.Configuration // mark, that this is configuration class
#org.springframework.context.annotation.Import(VqsS3Configration::class) // it references production configuration from test configuration
#org.springframework.context.annotation.ComponentScan // ask Spring to autoload all files from the package with VqsS3TestConfigration and all child packages
class VqsS3TestConfigration {
/*put test-related beans here in future*/
}
Then go to test and change declaration:
#ContextConfiguration(classes = [VqsS3TestConfigration ::class]) // we ask Spring to load configuration here
I created sample application here: https://github.com/imanushin/spring-boot2-junit5-and-kotlin-integration
Please execude line .\gradlew.bat test or gradlew.bat bootRun in the src folder. Test will check, that we able to read properties. bootRun will print auto-loaded properties
Boring theory
First of all - Spring has Configuration classes - they are needed to load and initialize other classes. Instead of Service or Comonent classes, main purpose of Configuration classes - just create services, components, etc.
If we will simplify algorithm of the Spring application load, then it will be like this:
Find Configuration classes
Read annotation of them, understand list of classes (e.g. reference tree), which should be loaded (and in addition - how they should be loaded)
Load classes with different ways:
3.1. For classes, which are annotated with #ConfigurationProperties - put configuration items here
3.2. For classes, which are annotated with #RestController - register them as rest controllers
3.N. etc...
How does Spring understand, what configuration should be loaded?
Formally is it done by Spring Boot, however I will name it as Spring
Understand several initial configurations - they can be put into the class SpringApplicationBuilder, into the test annotations (see above), into the XML context, etc. For our case we use test annotation and #ContextConfiguration attribute
Recursive get all imported configuration (e.g. Spring reads #Import annotation, then it get children, then it check their imports, etc.)
Use Spring Factories to get configuration automatically from jar
Therefore, in our case, Spring will do actions like this:
Get configuration from test annotation
Get all other configurations by recursive way
Load all classes into the contet
Start test
Okay, it took me all day, but I finally got my application properties to load into my unit test context. I made 2 changes:
First, I added the #Service annotation to my VqsS3FileReader service - which I had originally forgotten. Also, while I had updated my Test to not inject the AmazonS3Config via the constructor, I had neglected to update my service to do the same. So I changed
this:
class VqsS3FileReader(val amazonS3Config: AmazonS3Config) : VqsFileReader {
companion object: mu.KLogging()
...
to this:
#Service
class VqsS3FileReader : VqsFileReader {
companion object: mu.KLogging()
#Resource
private lateinit var amazonS3Config: AmazonS3Config
...
Finally, I modified my Spring annotations on my test.
from this:
#ActiveProfiles("test")
#TestInstance(TestInstance.Lifecycle.PER_CLASS)
#ExtendWith(SpringExtension::class)
#ContextConfiguration(classes = [AmazonS3Config::class, VqsS3FileReader::class])
class VqsS3FileReaderTest(): TestBase() {
...
to this:
#ActiveProfiles("test")
#SpringBootTest
#ComponentScan("com.ilmn.*")
#TestInstance(TestInstance.Lifecycle.PER_CLASS)
#ExtendWith(SpringExtension::class)
#EnableAutoConfiguration
#SpringJUnitConfig(SpringBootContextLoader::class)
class VqsS3FileReaderTest(): TestBase() {
...
It seems like I have an unordinary amount of annotations on my test now... so I will be looking carefully at what each of them really do, and see if I can reduce it. But at least my properties are being injected into my test context now.
I have two REST services using Spring Boot running on two different servers. I am using REST Template for this communication.
There are some models that are shared by these two services. All these models are of type 'IDataToTransferred' .
'IDataToTransferred' is a marker Interface implemented by various Model Beans.
I need to write a common logic for passing a list of these models between these REST services.
Hence I wrote a logic which uses parameters
List<? extends IDataToTransferred> from Sender service to Receiver Service.
Update: With Some Code
IDataToTransferred.java is a marker Interface
DataToBeSent.java
DataToBeSent Implements IDataToTransferred{
//Simple Pojo
}
SenderService.java
sendData(List<? extends IDataToTransferred> uploadDataObjectList){
//Some Code with REST Template
//restTemplate.postForEntity
}
IDataToTransferred Interface is shared between communicating webservices.
DataToBeReceived.java
DataToBeReceived Implements IDataToTransferred{
//Simple Pojo
}
ReceiverService.java
receiveData(List<? extends IDataToTransferred> uploadDataObjectList){
//Some Code to convert uploadDataObjectList to DataToBeReceived
}
Note In REST service I was always getting 415 error. Unsupported Media type. when I use the same List<? extends IDataToTransferred> on Receiver.
When I changed this to List<? super IDataToTransferred> on Receiver side, now it works, I am guessing because of Producer extends Consumer super rules.
But the problem is that now I can't typecast to the IDataToTransferred type on Receiver Side. Inside the list I am getting all linkedHashmap, the json got converted to linked HashMap between these services.
How can I get DataToBeReceived class object in ReceiverService?
For simplicity sake I have removed Controllers. Assume that they have the same signature as the services.
If I had known better terms to search, I would have found answer before Posting. But alas.
In any case I found the answer in stackoverflow page here together with a this blog ofcourse.
The examples are with abstract classes. I have used with interfaces.
As mentioned in the link. I Introduced below annotation in the marker interface IDataToTransferred:
#JsonTypeInfo(
use = JsonTypeInfo.Id.NAME,
include = JsonTypeInfo.As.PROPERTY,
property = "type")
#JsonSubTypes({
#Type(value = DataToBeSent.class, name = "datatransfer")})
The property type is introduced in the bean DataToBeSent as a property. This type param is used as information for conversion into implementing type from interface type. One can use a different variable than one named "type". In JsonSubTypes annotation , we mention the classes that are implementing this interface.
DataToBeSent Implements IDataToTransferred{
//Simple Pojo
// Some Properties with getter and setter
String type = "datatransfer";
//with getter and setter
}
The same exercise needs to be implemented on the Receiver Side also. Hence, we will have annotation as below:
#JsonTypeInfo(
use = JsonTypeInfo.Id.NAME,
include = JsonTypeInfo.As.PROPERTY,
property = "type")
#JsonSubTypes({
#Type(value = DataToBeReceived.class, name = "datatransfer")})
Here, we have DataToBeReceived class as implementing the IDataToTransferred interface. Ofcourse you need to add type as property to DataToBeReceived class also as below:
DataToBeReceived Implements IDataToTransferred{
//Simple Pojo
// Some Properties with getter and setter
String type = "datatransfer";
//with getter and setter
}
Hope this helps.
I am facing some problems with kotlin in conjunction with spring.
I have a controller bean (without an interface btw) which has an auto-wired service bean via the primary constructor.
It works perfectly unless I use caching annotations for the controller. Apparently springs caching generates a proxy class under the hood which deals with the caching.
My code looks like this:
#RestController
#RequestMapping("/regions/")
open class RegionController #Autowired constructor(val service: RegionService) {
#RequestMapping("{id}", method = arrayOf(RequestMethod.GET))
#Cacheable(cacheNames = arrayOf("regions"))
fun get(#PathVariable id: Long): RegionResource {
return this.service.get(id)
}
}
The problem now is a null pointer exception when the method is executed, actually this.service is null which technically is not possible as it is a nonnull variable in kotlin.
I assume that class proxies generated by spring initialize the class with null values instead of the autowired bean. This must be a common pitfall using kotlin and spring. How did you circumvent this problem?
In Kotlin both classes and members are final by default.
For the proxying library (CGLIB, javaassist) to be able to proxy a method it has to be declared non final and in a non final class (since those libraries implement proxying by subclassing). Change your controller method to:
#RequestMapping("{id}", method = arrayOf(RequestMethod.GET))
#Cacheable(cacheNames = arrayOf("regions"))
open fun get(#PathVariable id: Long): RegionResource {
return this.service.get(id)
}
You probably see a warning in console regarding RegionController methods not being subject to proxying.
The Kotlin compiler plugin
The Kotlin team has acknowledged this difficulty and created a plugin that marks the standard AOP proxy candidates e.g. #Component with open.
You can enable the plugin by in your build.gradle:
plugins {
id "org.jetbrains.kotlin.plugin.spring" version "1.1.60"
}
Soon this might not be a problem any longer.
There is work in progress that any lib (including spring for example) can specify a list of annotations a file in META-INF. Once a class is annotated with one of these, it will default to open for the class itself and all its functions. This is also true for classes inheriting from an annotated class.
For more details, have a look at https://github.com/Kotlin/KEEP/pull/40#issuecomment-250773204