Reactive Redis does not continually publish changes to the Flux - spring-boot

I am trying to get live updates on my redis ordered list without success.
It seems like it fetches all the items and just ends on the last item.
I would like the client to keep get updates upon a new order in my ordered list.
What am I missing?
This is my code:
#RestController
class LiveOrderController {
#Autowired
lateinit var redisOperations: ReactiveRedisOperations<String, LiveOrder>
#GetMapping(produces = [MediaType.TEXT_EVENT_STREAM_VALUE], value = "/orders")
fun getLiveOrders(): Flux<LiveOrder> {
val zops = redisOperations?.opsForZSet()
return zops?.rangeByScore("orders", Range.unbounded())
}
}

There is no such feature in Redis. First, reactive retrieval of a sorted set is just getting a snapshot, but your calls are going in a reactive fashion. So you need a subscription instead.
If you opt in for keyspace notifications like this (K - enable keyspace notifications, z - include zset commands) :
config set notify-keyspace-events Kz
And subscribe to them in your service like this:
ReactiveRedisMessageListenerContainer reactiveRedisMessages;
// ...
reactiveRedisMessages.receive(new PatternTopic("__keyspace#0__:orders"))
.map(m -> {
System.out.println(m);
return m;
})
<further processing>
You would see messages like this: PatternMessage{channel=__keyspace#0__:orders, pattern=__keyspace#0__:orders, message=zadd}. It will notify you that something has been added. And you can react on this somehow - get the full set again, or only some part (head/tail). You might even remember the previous set, get the new one and send the diff.
But what I would really suggest is rearchitecting the flow in some way to use Redis Pub/Sub functionality directly. For example: publisher service instead of directly calling zadd will call eval, which will issue 2 commands: zadd orders 1 x and publish orders "1:x" (any custom message you want, maybe JSON).
Then in your code you will subscribe to your custom topic like this:
return reactiveRedisMessages.receive(new PatternTopic("orders"))
.map(LiveOrder::fromNotification);

Related

How to segregate data from Flux without the underlying call again?

I am working with reactive spring-boot framework and I have a flow like this in my application, (psuedo code):
Flux<String> keys; // getting flux of keys from some part of code
Flux<RedisData> response = getDataFromRedisForKeys(keys);
Flux<RedisData> responseTypeA = filterATypeResponseFromRedisDataFlux(response);
Flux<RedisData> responseTypeB = filterBTypeResponseFromRedisDataFlux(response);
Flux<RedisData> responseTypeC = filterCTypeResponseFromRedisDataFlux(response);
Flux<RedisData> responseTypeD = filterDTypeResponseFromRedisDataFlux(response);
Now, when I am trying to do flatMap ops on the 4 fluxes after the filter, I am seeing that data from redis is being get 4 times, what I want is that we get it once reactively and segregate it out without blocking.
getDataFromRedisForAllKeys is according to your incoming key returns the corresponding redis data needs to be yourself
You just need to search through redis for the key you need first and then filter or subscribe
You can try using the subscribe method
And process data in it
Flux result = getDataFromRedisForAllKeys(keys);
result.subscribe(data -> {
switch (data.key){
case "A":
// do something
case "B":
// do something
case "C":
// do something
case "D":
// do something
}
});
Or use the Flux filter method
Flux<RedisData> responseTypeA = result.filter(redisData -> "A".equals(redisData.key));

Spring Webflux: Extract value from Mono

I am new to spring webflux and am trying to perform some arithmetic on the values of two monos. I have a product service that retrieves account information by calling an account service via webClient. I want to determine if the current balance of the account is greater than or equal to the price of the product.
Mono<Account> account = webClientBuilder.build().get().uri("http://account-service/user/accounts/{userId}/",userId)
.retrieve().bodyToMono(Account.class);
//productId is a path variable on method
Mono<Product> product =this.productService.findById(productId);
When I try to block the stream I get an error
block()/blockFirst()/blockLast() are blocking, which is not supported in thread reactor-http-nio-2
//Causes Error
Double accountBalance = account.map(a->a.getBalance()).block():
Double productPrice = product.map(p->p.getPrice()).block();
///Find difference, send response accordingly....
Is this the correct approach of there is another, better way to achieve this? I was also thinking something along the lines of:
Mono<Double> accountBalance = account.map(a->a.getBalance()):
Mono<Double> productPrice = product.map(p->p.getPrice());
Mono<Double> res = accountBalance.zipWith(productPrice,(b,p)-> b-p);
//Something after this.....
You can't use block method on main reactor thread. This is forbidden. block may work when publish mono on some other thread but it's not a case.
Basically your approach with zipping two monos is correct. You can create some helper method to do calculation on them. In your case it may look like:
public boolean isAccountBalanceGreater(Account acc, Product prd) {
return acc.getBalance() >= prd.getPrice();
}
And then in your Mono stream you can pass method reference and make it more readable.
Mono<Boolean> result = account.zipWith(productPrice, this::isAccountBalanceGreater)
The question is what you want to do with that information later. If you want return to your controller just true or false that's fine. Otherwise you may need some other mappings, zippings etc.
Update
return account.zipWith(productPrice, this::createResponse);
...
ResponseEntity createResponse(Account acc, Product prd) {
int responseCode = isAccountBalanceGreater(acc, prd) ? 200 : 500;
return ResponseEntity.status(responseCode).body(prd);
}

querying data with micrometer

We have this fancy monitoring system to which our spring-boot services are posting metrics to an influx DB with micrometer. There's a nice grafana frontend, but the problem is that we're now at a stage where we have to have some of these metrics available in other services to reason on.
The whole system was set up by my predecessor, and my current understanding of it is practically zero. I can add and post new metrics, but I can't for the life of me get anything out of it.
Here's a short example:
Our gateway increments the counter for each image that a camera posts to it. The definition of the counter looks like this:
private val imageCounters = mutableMapOf<String, Counter>()
private val imageCounter = { camera: String ->
imageCounters.getOrPut(camera) {
registry.counter("gateway.image.counter", "camera", camera)
}
And the counter is incremented in the code like this:
imageCounter("placeholder-id").increment()
Now we're improving our billing, and the billing service needs to know how many images for a certain camera went through the gateway. So naturally the first thing I try looks like this:
class MonitoringService(val metrics: MeterRegistry) {
private val log = logger()
private val imageCounters = mutableMapOf<String, Counter>()
private val imageCounter = { camera: String ->
imageCounters.getOrPut(camera) {
metrics.counter("gateway.image.counter", "camera", camera)
}
}
fun test() {
val test = imageCounter("16004").count()
val bugme = true
log.info("influx test: $test")
}
}
There's two problems with this: First off it always returns zero, so obviously I'm doing it wrong. I just can't figure out what it is.
Second, even if it would return a reasonable value, I don't see a way to limit this by time (I'll usually need the number of images uploaded during the current month).
What worries me is that while I can find a lot of documentation on how to post data with micrometer, I can't seem to find any documentation on how to query. Is Micrometer only designed to post monitoring data, but not query it? the .getOrPut() method would indicate it can do both, but since querying data seems undocumented as far as I can tell, that might be a misconception on my part.
There is an influx-db client for Java, which I'll try next, but at the end of the day I don't want multiple components in my application doing the same thing just because I'm not familiar with the tools I inherited.
InfluxMeterRegistry is a StepMeterRegistry, so the created Counter from it is a StepCounter. StepCounter.increment() increments the count in the current step but StepCounter.count() will return the count in the previous step. That's why you're seeing 0 with count() although you've already invoked increment() several times. You can see it in the next step and the default step is 1 minute, so you have to wait for 1 minute to see it.
See the following test to get an idea on how it works: https://github.com/izeye/sample-micrometer-spring-boot/blob/influx/src/test/java/com/izeye/sample/InfluxMeterRegistryTests.java

spring statemachine set multiple initial state

i have a serial state of order like
public enum orderStateEnum {
STATE_UNUSED("UNUSED"),
STATE_ORDERED("ORDERED"),
STATE_ASSIGNED("ASSIGNED"),
STATE_ASSIGN_EXCEPTION("ASSIGN_EXCEPTION"),
STATE_PACKED("PACKED"),
//and so on
}
  and i want to use spring.statemachine(or other state machine implementation) to manage the transition like from STATE_UNUSED to STATE_ORDERED STATE_ORDERED to STATE_ASSIGNED STATE_ORDERED to STATE_ASSIGN_EXCEPTION STATE_ASSIGNED to STATE_PACKED   however all the order data is stored in database,so in my case, if i have an order with STATE_ASSIGNED state, i fetch the order state from the database,but in spring.statemachine, i have to ``` StateMachine stateMachine = new StateMachine(); stateMachine.createEvent(Event_take_order);
  when i new a instance of stateMachine, it's inital state is STATE_UNUSED,however i want the inital state to be the state i fetch from the database which is STATE_ASSIGNED,how can i achieve that? i've read [https://docs.spring.io/spring-statemachine/docs/1.0.0.BUILD-SNAPSHOT/reference/htmlsingle/] but i can't find any solution in it.
When you create a new StateMachine you can get StateMachineAccessor using stateMachine.getStateMachineAccessor()
StateMachineAccessor is:-
Functional interface for StateMachine to allow more programmaticaccess to underlying functionality. Functions prefixed "doWith" will expose StateMachineAccess via StateMachineFunction for better functionalaccess with jdk7. Functions prefixed "with" is better suitable for lambdas.(From Java Docs)
StateMachineAccessor has a method called doWithAllRegions where you can provide implementation of StateMachineFunction (interface) and doWithAllRegions will execute given StateMachineFunction with all recursive regions.
So, to achieve what you are trying to do the code will look like this:-
StateMachine<orderStateEnum, Events> stateMachine = smFactory.getStateMachine();
stateMachine.getStateMachineAccessor().doWithAllRegions(access -> access
.resetStateMachine(new DefaultStateMachineContext<>(STATE_ASSIGNED, null, null, null)));
I have provided the implementation of the interfaces using lambdas.

Spring data Neo4j Affected row count

Considering a Spring Boot, neo4j environment with Spring-Data-neo4j-4 I want to make a delete and get an error message when it fails to delete.
My problem is since the Repository.delete() returns void I have no ideia if the delete modified anything or not.
First question: is there any way to get the last query affected lines? for example in plsql I could do SQL%ROWCOUNT
So anyway, I tried the following code:
public void deletesomething(Long somethingId) {
somethingRepository.delete(getExistingsomething(somethingId).getId());
}
private something getExistingsomething(Long somethingId, int depth) {
return Optional.ofNullable(somethingRepository.findOne(somethingId, depth))
.orElseThrow(() -> new somethingNotFoundException(somethingId));
}
In the code above I query the database to check if the value exist before I delete it.
Second question: do you recommend any different approach?
So now, just to add some complexity, I have a cluster database and db1 can only Create, Update and Delete, and db2 and db3 can only Read (this is ensured by the cluster sockets). db2 and db3 will receive the data from db1 from the replication process.
For what I seen so far replication can take up to 90s and that means that up to 90s the database will have a different state.
Looking again to the code above:
public void deletesomething(Long somethingId) {
somethingRepository.delete(getExistingsomething(somethingId).getId());
}
in debug that means:
getExistingsomething(somethingId).getId() // will hit db2
somethingRepository.delete(...) // will hit db1
and so if replication has not inserted the value in db2 this code wil throw the exception.
the second question is: without changing those sockets is there any way for me to delete and give the correct response?
This is not currently supported in Spring Data Neo4j, if you wish please open a feature request.
In the meantime, perhaps the easiest work around is to fall down to the OGM level of abstraction.
Create a class that is injected with org.neo4j.ogm.session.Session
Use the following method on Session
Example: (example is in Kotlin, which was on hand)
fun deleteProfilesByColor(color : String)
{
var query = """
MATCH (n:Profile {color: {color}})
DETACH DELETE n;
"""
val params = mutableMapOf(
"color" to color
)
val result = session.query(query, params)
val statistics = result.queryStatistics() //Use these!
}

Resources