We are using Spring Cloud Steam with RabbitMQ binder.
I'm currently in the Process of setting up a RabbitMQ Cluster and want to use quorum queues. The options to set the queue type must be sent with the creation of the queue. In version 3.0.0 of Spring Cloud Stream, the option is added, but we are on an old version of String Boot and String Cloud Stream and don't want to update, because our live release will be in the next few weeks.
I've looked up the PR link. but the options are all private and I don't see any listeners or interceptors.
Is there any way to inject additional arguments into the queue creation call?
I would recommend upgrading to a newer version.
The only option with older versions is to provision and bind the queue manually and disable automatic provisioning of the queues as discussed in the binder reference.
If you have an existing exchange/queue that you wish to use, you can completely disable automatic provisioning as follows, assuming the exchange is named myExchange and the queue is named myQueue:
spring.cloud.stream.binding.<binding name>.destination=myExhange
spring.cloud.stream.binding.<binding name>.group=myQueue
spring.cloud.stream.rabbit.bindings.<binding name>.consumer.bindQueue=false
spring.cloud.stream.rabbit.bindings.<binding name>.consumer.declareExchange=false
spring.cloud.stream.rabbit.bindings.<binding name>.consumer.queueNameGroupOnly=true
Related
We are currently writing a library that consumes rabbitmq events with spring-amqp.
This library needs to be used from some applications that themselves use rabbitmq with spring-amqp.
Is it possible to isolate the separate RabbitMQ Configurations from each other, so that the configurations form within the library dont interfere with the existing ones in the applications?
both would connect to the same rabbitmq cluster.
I looked through the documentation of spring-amqp but only found a way to split the rabbit configuration for consuming and producing events.
Since spring-amqp 2.3 there's a Multiple Broker (or Cluster) Support which could be used to create multiple connections to the same broker. You can find a sample config at this link.
Also, you can take a look at the spring-multirabbit library (https://github.com/freenowtech/spring-multirabbit) which is actually the ancestor to that feature in spring-amqp and can be used to add multiple RabbitMq connections support to a service that already has a Spring-configured connection in a non-intrusive way.
I have a set of microservices using springboot rest. These microservices will be deployed in a autoscaled and load balanced environment. One of these services is responsible for managing the system's configuration. When other microservices startup, they obtain the configuration from this service. If and when the configuration is updated, I need to inform all currently running microservices instances to update their cached configuration.
I am considering using RabbitMQ with a fanout exchange. In this solution, each instance at startup will create its queue and bind that queue to the exchange. When there is a configuration change, the configuration service will publish an update to all queues currently bound to that exchange.
However, as service instances are deleted, I cannot figure out how would I delete the queue specific to that instance. I googled but could not find a complete working example of a solution.
Any help or advise?
The idea and solution is correct. What you just miss that those queues, created by your consumer services could be declared as auto-delete=true: https://www.rabbitmq.com/queues.html. As long as your service is UP, the queue is there as well. You stop your service, its consumers are stopped and unsubscribed. At the moment the last consumer is unsubscribed the queue is deleted from the broker.
On the other hand I would suggest to look into Spring Cloud Bus project which really is aimed for tasks like this: https://spring.io/projects/spring-cloud-bus.
We have several micro-services using Spring Boot and Spring Cloud Stream Kafka binder to communicate between them.
Occasionally, we observe bursts of duplicate messages received by a consumer - often several days after it was first consumed and processed (successfully).
While I understand that Kafka does not guarantee exactly-once delivery, it still looks very strange, given that there were no rebalancing events or any 'suspicious' activity in the logs of either the brokers nor the services. Since the consumer is interacting with external APIs, it is a bit difficult to make it idempotent.
Any hints what might be the cause of duplication? What should I be looking for to figure this out?
We are using Kafka broker 1.0.0, and this particular consumer uses Spring Cloud Stream Binder Kafka 2.0.0, which is based on kafka-client 1.0.2 (version of the other services might be a bit different).
You should show your configuration when asking questions like this.
Best guess is the broker's offsets.retention.minutes.
With modern broker versions (since 2.0), it defaults to 1 week; with older versions it was only one day.
Our java spring boot application is creating/declaring queues (if they do not exist) after successful connection to a certain exchange / topic.
Is it possible (from rabbitmq admin panel) to prohibit certain users (in this case the one used by this spring boot app) from creating/declaring a queue if it does no exist?
Thanks!
You can configure the permissions of the user a Spring-Boot app uses to connect to the broker.
This is achieved by supplying 3 regex (configuration, write, read), if you let the first one empty ("^$"), the user will not be able to delare any queue as mentionned in the complete documentation
You can also disable the RabbitAdmin bean by adding the following property to the app configuration file spring.rabbitmq.dynamic=false, so Spring will not try to declare anything.
I'm trying to aggregate different Sink and Source spring boot applications using the AggregateApplicationBuilder as described here: http://docs.spring.io/spring-cloud-stream/docs/current-SNAPSHOT/reference/htmlsingle/#_aggregation
Since I expect in process communication, I don't want to setup kafka or rabbitmq binder. How to configure a local one? I found that a spring-cloud-stream-binder-local exists but it's in M2 since a long time and is not embedded with a release train.
How I can use the AggregateApplicationBuilder with no external system dependency?
Thanks
With AggregateApplicationBuilder you don't have to configure the binder for the in-process communication of the directly bound channels within the aggregated application. The binder is required only if you need the aggregate application itself consumes messages from broker or produces messages to broker. If the aggregated application itself is self-contained, then there is no need for the binder at all.