Spring Cacheable - Ignore Feign exceptions, and instead return the last known good result - spring

I have a service, ItemService, that uses a Feign client to get data from an external service. I want to cache the results of the external service and refresh them every X minutes, unless the external service fails, in which case I want to keep using the previous result (or an empty list if the initial call fails).
I've tried using Spring's cache abstraction, catching all exceptions, and setting 'null' to be disregarded using unless, but this doesn't work and instead getItems() returns null!
#Service
class ItemService(
private val itemFeignClient: ItemFeignClient,
) {
#Cacheable(
cacheManager = "itemClientCacheManager",
cacheNames = ["get_items"],
unless = "#result == null", // this does not work
)
fun getItems(): List<String>? {
return try {
itemFeignClient.getItems()
} catch (e: Exception) {
// if the Feign client fails for any reason, I don't want the cache to update
null
}
}
}
I using the Spring cache abstraction in other parts of the code, so I would like to keep using it - but I'm not tied to it. Maybe there's an option for Feign to ignore errors?
Here is a summary of the test I am using
#SpringBootTest(
classes = [
ClientConfig::class,
ItemService::class,
ItemFeignClient::class,
]
)
#EnableFeignClients(
clients = [ItemFeignClient::class]
)
class ItemServiceTest {
#Autowired
lateinit var itemService: ItemService
#Test
fun `when a valid result is cached, expect it is not replaced after an error`() {
stubItemServiceResponse(
httpCode = 200,
responseBody = """
[ "item 1", "item 2"]
""".trimIndent()
)
val responseBeforeError = itemService.getItems()
verifyItemServiceCalled(expectedCount = 1) // assertion succeeds ✅
assertEquals(listOf("item 1", "item 2"), responseBeforeError) // assertion succeeds ✅
// mock an exception from the Feign client
stubItemServiceResponse(
httpCode = 500
)
val responseAfterError = itemService.getItems()
verifyItemServiceCalled(expectedCount = 2) // assertion succeeds ✅
assertEquals(listOf("item 1", "item 2"), responseAfterError) // assertion fails ❌
// assertion failure:
// expected ["item 1", "item 2"], actual: null
}
fun stubItemServiceResponse(
httpCode: Int,
responseBody: String? = null,
) {
// (stubbed using Wiremock)
}
fun verifyItemServiceCalled(
expectedCount: Int
) {
// (stubbed using Wiremock)
}
}
I am certain that the caching is working in the test, because this test succeeds:
#Test
fun `expect multiple calls are cached`() {
stubItemServiceResponse(
httpCode = 200,
responseBody = """
[ "item 1", "item 2"]
""".trimIndent()
)
itemService.getItems()
itemService.getItems()
itemService.getItems()
verifyItemServiceCalled(expectedCount = 1) // assertion succeeds ✅
}
Here is the cache config. I am using Caffeine Cache.
#Configuration
#EnableCaching
class ClientConfig {
#Bean
#Qualifier("itemClientCacheManager")
fun itemClientCacheManager(): CaffeineCacheManager {
val manager = CaffeineCacheManager(
"get_items",
)
manager.setCaffeine(
Caffeine.newBuilder()
.maximumSize(10)
.expireAfterWrite(5, TimeUnit.MINUTES)
)
return manager
}
}
Versions:
Spring Boot 2.7.0
Caffeine 3.1.1
Spring Cloud 2021.0.3
OpenFeign 3.1.3
Kotlin 1.7.0

Related

Why am I getting error when I publish to config topic in Spring MQTT?

I'm sending a message to "config" topic from my Spring boot backend application
Here's my mqtt setup
final String mqttServerAddress =
String.format("ssl://%s:%s", options.mqttBridgeHostname, options.mqttBridgePort);
// Create our MQTT client. The mqttClientId is a unique string that identifies this device. For
// Google Cloud IoT Core, it must be in the format below.
final String mqttClientId =
String.format(
"projects/%s/locations/%s/registries/%s/devices/%s",
options.projectId, options.cloudRegion, options.registryId, options.deviceId);
MqttConnectOptions connectOptions = new MqttConnectOptions();
// Note that the Google Cloud IoT Core only supports MQTT 3.1.1, and Paho requires that we
// explictly set this. If you don't set MQTT version, the server will immediately close its
// connection to your device.
connectOptions.setMqttVersion(MqttConnectOptions.MQTT_VERSION_3_1_1);
Properties sslProps = new Properties();
sslProps.setProperty("com.ibm.ssl.protocol", "TLSv1.2");
connectOptions.setSSLProperties(sslProps);
// With Google Cloud IoT Core, the username field is ignored, however it must be set for the
// Paho client library to send the password field. The password field is used to transmit a JWT
// to authorize the device.
connectOptions.setUserName(options.userName);
DateTime iat = new DateTime();
if ("RS256".equals(options.algorithm)) {
connectOptions.setPassword(
createJwtRsa(options.projectId, options.privateKeyFile).toCharArray());
} else if ("ES256".equals(options.algorithm)) {
connectOptions.setPassword(
createJwtEs(options.projectId, options.privateKeyFileEC).toCharArray());
} else {
throw new IllegalArgumentException(
"Invalid algorithm " + options.algorithm + ". Should be one of 'RS256' or 'ES256'.");
}
// [START iot_mqtt_publish]
// Create a client, and connect to the Google MQTT bridge.
MqttClient client = new MqttClient(mqttServerAddress, mqttClientId, new MemoryPersistence());
// Both connect and publish operations may fail. If they do, allow retries but with an
// exponential backoff time period.
long initialConnectIntervalMillis = 500L;
long maxConnectIntervalMillis = 6000L;
long maxConnectRetryTimeElapsedMillis = 900000L;
float intervalMultiplier = 1.5f;
long retryIntervalMs = initialConnectIntervalMillis;
long totalRetryTimeMs = 0;
while ((totalRetryTimeMs < maxConnectRetryTimeElapsedMillis) && !client.isConnected()) {
try {
client.connect(connectOptions);
} catch (MqttException e) {
int reason = e.getReasonCode();
// If the connection is lost or if the server cannot be connected, allow retries, but with
// exponential backoff.
System.out.println("An error occurred: " + e.getMessage());
if (reason == MqttException.REASON_CODE_CONNECTION_LOST
|| reason == MqttException.REASON_CODE_SERVER_CONNECT_ERROR) {
System.out.println("Retrying in " + retryIntervalMs / 1000.0 + " seconds.");
Thread.sleep(retryIntervalMs);
totalRetryTimeMs += retryIntervalMs;
retryIntervalMs *= intervalMultiplier;
if (retryIntervalMs > maxConnectIntervalMillis) {
retryIntervalMs = maxConnectIntervalMillis;
}
} else {
throw e;
}
}
}
attachCallback(client, options.deviceId);
// The MQTT topic that this device will publish telemetry data to. The MQTT topic name is
// required to be in the format below. Note that this is not the same as the device registry's
// Cloud Pub/Sub topic.
String mqttTopic = String.format("/devices/%s/%s", options.deviceId, options.messageType);
long secsSinceRefresh = ((new DateTime()).getMillis() - iat.getMillis()) / 1000;
if (secsSinceRefresh > (options.tokenExpMins * MINUTES_PER_HOUR)) {
System.out.format("\tRefreshing token after: %d seconds%n", secsSinceRefresh);
iat = new DateTime();
if ("RS256".equals(options.algorithm)) {
connectOptions.setPassword(
createJwtRsa(options.projectId, options.privateKeyFile).toCharArray());
} else if ("ES256".equals(options.algorithm)) {
connectOptions.setPassword(
createJwtEs(options.projectId, options.privateKeyFileEC).toCharArray());
} else {
throw new IllegalArgumentException(
"Invalid algorithm " + options.algorithm + ". Should be one of 'RS256' or 'ES256'.");
}
client.disconnect();
client.connect(connectOptions);
attachCallback(client, options.deviceId);
}
MqttMessage message = new MqttMessage(data.getBytes(StandardCharsets.UTF_8.name()));
message.setQos(1);
client.publish(mqttTopic, message);
here's the options class
public class MqttExampleOptions {
String mqttBridgeHostname = "mqtt.googleapis.com";
short mqttBridgePort = 8883;
String projectId =
String cloudRegion = "europe-west1";
String userName = "unused";
String registryId = <I don't want to show>
String gatewayId = <I don't want to show>
String algorithm = "RS256";
String command = "demo";
String deviceId = <I don't want to show>
String privateKeyFile = "rsa_private_pkcs8";
String privateKeyFileEC = "ec_private_pkcs8";
int numMessages = 100;
int tokenExpMins = 20;
String telemetryData = "Specify with -telemetry_data";
String messageType = "config";
int waitTime = 120
}
When I try to publish message to topic "config" I get this error
ERROR 12556 --- [nio-8080-exec-1] o.a.c.c.C.[.[.[.[dispatcherServlet] : Servlet.service()
for servlet [dispatcherServlet] in context with path [/iot-admin] threw exception [Request processing failed; nested exception is Connection Lost (32109) - java.io.EOFException] with root cause
java.io.EOFException: null
at java.base/java.io.DataInputStream.readByte(DataInputStream.java:273) ~[na:na]
at org.eclipse.paho.client.mqttv3.internal.wire.MqttInputStream.readMqttWireMessage(MqttInputStream.java:92) ~[org.eclipse.paho.client.mqttv3-1.2.5.jar:na]
at org.eclipse.paho.client.mqttv3.internal.CommsReceiver.run(CommsReceiver.java:137) ~[org.eclipse.paho.client.mqttv3-1.2.5.jar:na]
this is the message I am sending
{
"Led": {
"id": "e36b5877-2579-4db1-b595-0e06410bde11",
"rgbColors": [{
"id": "1488acfe-baa7-4de8-b4a2-4e01b9f89fc5",
"configName": "Terminal",
"rgbColor": [150, 150, 150]
}, {
"id": "b8ce6a35-4219-4dba-a8de-a9070f17f1d2",
"configName": "PayZone",
"rgbColor": [150, 150, 150]
}, {
"id": "bf62cef4-8e22-4804-a7d8-a0996bef392e",
"configName": "PayfreeLogo",
"rgbColor": [150, 150, 150]
}, {
"id": "c62d25a4-678b-4833-9123-fe3836863400",
"configName": "BagDetection",
"rgbColor": [200, 200, 200]
}, {
"id": "e19e1ff3-327e-4132-9661-073f853cf913",
"configName": "PersonDetection",
"rgbColor": [150, 150, 150]
}]
}
}
How can I properly send a message to a config topic without getting this error? I am able to send message to "state" topic, but not to "config" topic.

Spring boot Neo4j - query depth not working correctly

TL;DR: #Depth(value = -1) throws nullpointer and other values above 1 are ignored
In my Spring Boot with Neo4j project I have 3 simple entities with relationships:
#NodeEntity
data class Metric(
#Id #GeneratedValue val id: Long = -1,
val name: String = "",
val description: String = "",
#Relationship(type = "CALCULATES")
val calculates: MutableSet<Calculable> = mutableSetOf()
) {
fun calculates(calculable: Calculus) = calculates.add(calculable)
fun calculate() = calculates.map { c -> c.calculate() }.sum()
}
interface Calculable {
fun calculate(): Double
}
#NodeEntity
data class Calculus(
#Id #GeneratedValue val id: Long = -1,
val name: String = "",
#Relationship(type = "LEFT")
var left: Calculable? = null,
#Relationship(type = "RIGHT")
var right: Calculable? = null,
var operator: Operator? = null
) : Calculable {
override fun calculate(): Double =
operator!!.apply(left!!.calculate(), right!!.calculate())
}
#NodeEntity
data class Value(
#Id #GeneratedValue val id: Long = -1,
val name: String = "",
var value: Double = 0.0
) : Calculable {
override fun calculate(): Double = value
}
enum class Operator : BinaryOperator<Double>, DoubleBinaryOperator {//not relevant}
I create a simple graph like this one:
With the following repositories:
#Repository
interface MetricRepository : Neo4jRepository<Metric, Long>{
#Depth(value = 2)
fun findByName(name: String): Metric?
}
#Repository
interface CalculusRepository : Neo4jRepository<Calculus, Long>{
fun findByName(name: String): Calculus?
}
#Repository
interface ValueRepository : Neo4jRepository<Value, Long>{
fun findByName(name: String): Value?
}
And the following code:
// calculus
val five = valueRepository.save(Value(
name = "5",
value = 5.0
))
val two = valueRepository.save(Value(
name = "2",
value = 2.0
))
val fiveTimesTwo = calculusRepository.save(Calculus(
name = "5 * 2",
operator = Operator.TIMES,
left = five,
right = two
))
println("---")
println(fiveTimesTwo)
val fromRepository = calculusRepository.findByName("5 * 2")!!
println(fromRepository) // sometimes has different id than fiveTimesTwo
println("5 * 2 = ${fromRepository.calculate()}")
println("--- \n")
// metric
val metric = metricRepository.save(Metric(
name = "Metric 1",
description = "Measures a calculus",
calculates = mutableSetOf(fromRepository)
))
metricRepository.save(metric)
println("---")
println(metric)
val metricFromRepository = metricRepository.findByName("Metric 1")!!
println(metricFromRepository) // calculates node is partially empty
println("--- \n")
To retrieve the same graph as shown in the picture above (taken from the actual neo4j dashboard), I do metricRepository.findByName("Metric 1") which has #Depth(value = 2) and then print the saved metric and the retrieved metric:
Metric(id=9, name=Metric 1, description=Measures a calculus, calculates=[Calculus(id=2, name=5 * 2, left=Value(id=18, name=5, value=5.0), right=Value(id=1, name=2, value=2.0), operator=TIMES)])
Metric(id=9, name=Metric 1, description=Measures a calculus, calculates=[Calculus(id=2, name=5 * 2, left=null, right=null, operator=TIMES)])
No matter the value of the depth, I can't get the Metric node with all his children nodes, it retrieves one level deep max and returns null on the leaf nodes.
I've read in the docs that depth=-1 retrieves the fully-resolved node but doing so causes the findByName() method to fail with a null pointer: kotlin.KotlinNullPointerException: null
Here is a list of resources I've consulted and a working GitHub repository with the full code:
GitHub Repo
Spring Data Neo4j Reference Documentation
Neo4j-OGM Docs
Final notes:
The entities all have default parameters because Kotlin then makes an empty constructor, I think the OGM needs it
I've also tried making custom queries but couldn't specify the depth value because there are different relationships and can be at different levels
To use the GitHub repository I linked you must have Neo4j installed, the repo has a stackoverflow-question branch with all the code.
Versions:
Spring boot: 2.3.0.BUILD-SNAPSHOT
spring-boot-starter-data-neo4j: 2.3.0.BUILD-SNAPSHOT
Thank you for helping and all feedback is welcomed!
The problem is not with the query depth but with the model. The Metric entity has a relation with Calculable, but Calculable itself has no relationships defined. Spring Data cannot scan all possible implementations of the Calculable interface for their relationships. If you changed Metrics.calculates type to MutableSet<Calculus>, it would work as expected.
To see Cypher requests send to the server you can add logging.level.org.neo4j.ogm.drivers.bolt=DEBUG to the application.properties
Request before the change:
MATCH (n:`Metric`) WHERE n.`name` = $`name_0` WITH n RETURN n,[ [ (n)->[r_c1:`CALCULATES`]->(x1) | [ r_c1, x1 ] ] ], ID(n) with params {name_0=Metric 1}
Request after the change:
MATCH (n:`Metric`) WHERE n.`name` = $`name_0` WITH n RETURN n,[ [ (n)->[r_c1:`CALCULATES`]->(c1:`Calculus`) | [ r_c1, c1, [ [ (c1)-[r_l2:`LEFT`]-(v2:`Value`) | [ r_l2, v2 ] ], [ (c1)-[r_r2:`RIGHT`]-(v2:`Value`) | [ r_r2, v2 ] ] ] ] ] ], ID(n) with params {name_0=Metric 1}

Kotlin iterator to check if payload list have an id/projectId or not, returning false when there is no attributes?

I have an issue where i have a method where i am checking the payload has the attributes or not. When i am sending my payload i want to check that the user dont have inserted attributes which not allowed in the payload.
My entity class:
#Entity
data class ProjectAssociated(
#Id
#GeneratedValue(generator = "uuid2")
#GenericGenerator(name = "uuid2", strategy = "uuid2")
#Column(columnDefinition = "BINARY(16)")
var id: UUID? = null,
#Column(columnDefinition = "BINARY(16)")
var projectId: UUID? = null,
#Column(columnDefinition = "BINARY(16)")
var associatedProjectId: UUID? = null
)
My Service class:
fun addAssociatedProjectByProjectId(
projectId: UUID,
projectAssociatedList: MutableList<ProjectAssociated>
): MutableList<ProjectAssociated> {
if (projectAssociatedList.isNotEmpty()) {
println(projectAssociatedList)
if (!projectAssociatedList.map { it.id }.isNullOrEmpty()) {
val errorMessage = "Not allowed to provide parameter 'id' in this request"
throw UserInputValidationException(errorMessage)
}
if (!projectAssociatedList.map { it.projectId }.isNullOrEmpty()) {
val errorMessage = "Not allowed to provide parameter 'projectId' in this request"
throw UserInputValidationException(errorMessage)
}
val checkIds = projectAssociatedList.map {
projectRepository.existsById(it.associatedProjectId)
}
if (checkIds.contains(false)) {
val errorMessage = "One or more ID 'associatedProjectId' not exists"
throw UserInputValidationException(errorMessage)
}
}
return projectAssociatedList.map {
projectAssociatedRepository.save(
ProjectAssociated(
null,
projectId,
it.associatedProjectId
)
)
}.toMutableList()
}
My Controller class:
#ApiOperation("Add associated Projects to a specific Project")
#PostMapping(path = ["/project-associated"], consumes = [MediaType.APPLICATION_JSON_VALUE])
fun createAssociatedProjectList(
#ApiParam("The id of the Project", required = true)
#RequestParam("id")
id: UUID,
#ApiParam("JSON object representing the ProjectAssociated")
#RequestBody projectAssociated: MutableList<ProjectAssociated>
): ResponseEntity<WrappedResponse<MutableList<ProjectAssociated>>> {
val createdProjectAssociatedList = projectService.addAssociatedProjectByProjectId(id, projectAssociated)
return ResponseEntity
.status(201)
.location(URI.create("$id/project-associated"))
.body(
ResponseDto(
code = 201,
data = PageDto(list = mutableListOf(createdProjectAssociatedList))
).validated()
)
}
But when i try to send this payload with the project id in #RequestParam:
[
{
"associatedProjectId": "7fe40f90-5178-11ea-9136-1b65a920a5d9"
},
{
"associatedProjectId": "7fe8aaaa-5178-11ea-9136-1b65a920a5d9"
}
]
I have a custom exception where i tell the user if projectId or the id is in the payload that is now allowed to have it in the payload. When i try to POST the payload example above it tells me that projectId or id is in the request? How can that be?
I also printed out the list before if checks:
[ProjectAssociated(id=null, projectId=null, associatedProjectId=7fe40f90-5178-11ea-9136-1b65a920a5d9), ProjectAssociated(id=null, projectId=null, associatedProjectId=7fe8aaaa-5178-11ea-9136-1b65a920a5d9)]
What am I doing wrong?
Thanks for the help!
In the block projectAssociatedList.map { it.id } you are mapping your list to something like [null, null] and it is not null or empty.
So, the complete condition !projectAssociatedList.map { it.id }.isNullOrEmpty() returns true.
If you want to continue using the same logic, you should use !projectAssociatedList.mapNotNull { it.id }.isNullOrEmpty() instead.
The mapNotNull function will filter the null values and output a list just with the not null values. If there is only null values, the list will be empty.
But, a simpler and expressive way to check if there is any not null attribute in a list of objects could be projectAssociatedList.any { it.id != null }

How do fetch the state with custome query? Corda application using Spring boot webserver- error while fetching the result

I have created the IOU in corda applicatiion, the IOU has ID,xml payload in body, partyName. NOW, i want to fetch the state with custome query that is basis on ID. NOTE- i am not using linearID.
Below is my API call- which gives me syntax error on. Can someone please correct me, what is the wrong thing that i am doing.
#GetMapping(value = ["getIous"],produces = [ MediaType.APPLICATION_JSON_VALUE])
private fun getTransactionOne(#RequestParam(value = "payloadId") payloadId: String): ResponseEntity<List<IOUState>> {
val generalCriteria = QueryCriteria.VaultQueryCriteria(Vault.StateStatus.ALL)
val results = builder { IOUState::iouId.equal(payloadId)
val customCriteria = QueryCriteria.VaultCustomQueryCriteria(results)}
val criteria = customCriteria.and(customCriteria)
val res = proxy.vaultQueryBy<IOUState>(criteria)
return ResponseEntity.ok(res)
}
I think the issue is because VaultCustomQueryCriteria is applicable only to StatePersistable objects. So you should use PersistentIOU instead of IOUState. Also, I could see incorrect use of brackets. Here is how your code should look like:
#GetMapping(value = ["getIous"],produces = [ MediaType.APPLICATION_JSON_VALUE])
private fun getTransactionOne(#RequestParam(value = "payloadId") payloadId: String): ResponseEntity<List<IOUState>> {
val generalCriteria = QueryCriteria.VaultQueryCriteria(Vault.StateStatus.ALL)
val results = builder {
val idx = IOUSchemaV1.PersistentIOU::iouId.equal(payloadId);
val customCriteria = QueryCriteria.VaultCustomQueryCriteria(idx)
val criteria = generalCriteria.and(customCriteria)
proxy.vaultQueryBy<IOUState>(criteria);
}
return ResponseEntity.ok(results)
}

Referencing value and calling methods in generic class types

I'm new to Kotlin coming from C#. Currently I am trying to setup a class that takes in a couple of interchangeable generic types, the internal code of this class is a spring service end-point.
I have started with something like below, however I seem to have trouble with the syntax to reference the parameters of the request body as well as calling a method, which are of the types passed in through the class constructor. Syntax of generics and reflection does not seem that straight forward and most of the Kotlin examples I have been digging up has not seem to covered precisely what I am trying to do (if even possible). The object instance of type1 will be passed in through the body parameter and the object instance of type2 should be passed in through the constructor (syntax is probably not right).
Planning to use this as a template to setup several end-points based on the same base code but with different requests and services classes.
Any help is greatly appreciated.
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.http.HttpStatus
import org.springframework.http.ResponseEntity
import org.springframework.web.bind.annotation.RequestBody
import org.springframework.web.bind.annotation.RequestMapping
import org.springframework.web.bind.annotation.RequestMethod
import javax.validation.Valid
open class Base <T1,T2>(t1: Class<T1>, t2: Class<T2>) {
#Autowired
var type1 = t1
#Autowired
var type2 = t2
#ApiOperation(value = "API 1", response = myResponse::class)
#ApiResponses(value = *arrayOf(
ApiResponse(code = 200, message = "successful", response = CcdaResponse::class),
ApiResponse(code = 405, message = "Invalid", response = Void::class)))
#RequestMapping(
value = "/myEndPoint",
produces = arrayOf("application/json"),
consumes = arrayOf("application/json"),
method = arrayOf(RequestMethod.POST)
)
fun endpoint(
#ApiParam(value = "Options", required = true)
#Valid
#RequestBody
body: Class<T1>
): ResponseEntity<myResponse> {
val r = myResponse()
val response: ResponseEntity<myResponse>
response = ResponseEntity(r, HttpStatus.OK)
try {
//payload
val parameters = Parameters().apply {
Id1 = type1::body.Id1.get()
Id2 = type1::body.Id2.get()
Id3 = type1::body.Id3.get()
Id4 = type1::body.Id4.get()
v = type1::body.v.get()
}
//Do stuff like calling method in class of type2 passed in
val d = type2.getViewModel(parameters)
r.status = "ok"
} catch (e: Exception) {
r.message = e.toString()
r.status = "error"
} finally {
}
return response
}
}
The types of the parameters are passed in through the type arguments when creating an instance (the same as Java). So you do need to pass in the types themselves, adding a Class parameter just isn't the correct syntax.
I believe this is what you are looking for (omitted some code for brevity).
open class Base<T1, T2> (#Autowired var t2: T2) {
#Autowired var type1: T1? = null
fun endpoint(
#ApiParam(value = "Options", required = true) #Valid #RequestBody
body: T1
): ResponseEntity<MyResponse> {
type1 = body
}
}
Then, for instance, you can create an instance of this class with the types Int and String (for T1 and T2 respectively) in the following manner.
val t2 = "t2"
val base = Base<Int, String>(t2)
Or you can subclass the Base class with any (or none) of the types specified.
class SubBase(t2: String): Base<Int, String>(t2)

Resources