I am trying to find some examples of Spring Cloud Data Flow and Azure Service Bus setup. .
I have found https://github.com/microsoft/spring-cloud-azure/tree/master/spring-cloud-azure-stream-binder/spring-cloud-azure-servicebus-topic-stream-binder but it is still in RC and I do not see any examples (which cover Spring Cloud Data Flow) there.
Could you please help me to understand if I can use Spring Cloud Data Flow and Azure Service Bus together?
I was able to run examples with Kafka and RabbitMQ, but I cannot find anything about Azure Service Bus that can be used as the integration solution for Spring Cloud Data Flow
Spring Cloud Data Flow doesn't necessarily need to know what messaging layer you choose to run your applications on. Hence, your question is more likely related to how one can run the application with Azure Service Bus using Spring Cloud Stream.
This is one such a sample: https://learn.microsoft.com/en-us/java/azure/spring-framework/configure-spring-cloud-stream-binder-java-app-azure-event-hub?view=azure-java-stable
Once the application is built using Spring Cloud Stream (using the Azure event hub binder), you can then manage these applications using Spring Cloud Data Flow as Streaming applications.
Related
Following documentation, there are multiple ways to integrate Google Cloud PubSub events with Spring application:
Spring Cloud GCP has several modules for sending messages to Pub/Sub
topics and receiving messages from Pub/Sub subscriptions using the
Spring Framework. You can use these modules independently or combine
them for different use cases:
Spring Cloud GCP Pub/Sub Starter lets you send and receive messages
using helper classes and call the Pub/Sub Java client library for more
advanced scenarios.
Spring Integration Channel Adapters for Pub/Sub
let you connect Spring Integration Message Channels with Pub/Sub.
Spring Cloud Stream Binder for Pub/Sub lets you use Pub/Sub as
messaging middleware in Spring Cloud Stream applications.
I don't have full understanding - what are those different use cases mentioned, and how to determine, which module is best for which use case?
Application (Dockerized Spring Boot app, deployed to Kubernetes in GCP) I am working on is rather simple, it is expected to act upon received PubSub event, it is not going to publish any events itself.
Spring Cloud GCP Pub/Sub Starter module contains the java client classes for pub sub which will be used by your spring application to perform administrative and functional operations (ie. sending and receiving messages).
Spring Integration Channel Adapters for Pub/Sub module is utilized when your spring application uses Message Channels. This module will help routing message between message channel and pub/sub using channel adapters.
Spring Cloud Stream Binder for Pub/Sub module is used in Spring Cloud Stream Applications in order to utilize cloud Pub/Sub API.
Since, your application requirements are basic you can easily go for Spring Cloud GCP Pub/Sub Starter module. For more information you can refer to this Google documentation.
I found these two docs in Spring Azure Documentation
https://learn.microsoft.com/en-us/azure/developer/java/spring-framework/configure-spring-cloud-stream-binder-java-app-with-service-bus
https://learn.microsoft.com/en-us/azure/developer/java/spring-framework/configure-spring-boot-starter-java-app-with-azure-service-bus
What is the difference between them? If there are other options, please suggest them!
I haven't seen any example of combining the two, even though it makes sense they'll work together (because of being both subprojects of Spring Cloud). I want to use Spring Cloud Stream (Reactive Processing) for reading from Kafka and writing to MongoDB with the Reactive Mongo driver. Can I use Spring Cloud Config and Spring Cloud Kubernetes for a remote git configuration server, even though the application is an event-driven application and not a requests-based API?
It's pretty unclear how to plug these Kubernetes and config projects into Spring Cloud Stream, and in general, it's unclear to me if all of the other Spring Cloud projects can work with Spring Cloud Stream reactively. For instance, I couldn't also find a reference for using Spring Cloud Sleuth & Zipkin with Spring Cloud Stream. Can someone make it clearer for me and reference code examples if exists?
we are developing microservices using spring boot. we want to track the progress of microservices using Spring Cloud Sleuth and wants to show that logs using Splunk on some dashboard etc.
For demo purpose we have are building three services, Service A calls service B which in turns call Service C. we want to trace their calls using slueth and consolidate them in splunk.
how to implement distributed tracing with Sleuth and Splunk?
any help would be greatly appreciated.
I have worked with spring cloud data flow and rabbitmq and kafka but i would like to know if it would be possible to install scdf with google pub/sub.
I don't want to create a stream (new app spring cloud stream) with source or sink to gcp i want google pub/sub over spring cloud data flow server to use as an intermediate messaging broker.
Any suggestions?
Yes, you can use Spring dataflow and GCP Pub Sub link.
Through the use of one of many Spring Cloud Stream Binders, many
different messaging middleware products can be used. The following
popular platforms are among the list supported by Spring Cloud Data
Flow:
1.Kafka Streams
2.Amazon Kinesis
3.Google Pub/Sub
4.Solace PubSub+
5.Azure Event
Hubs You can find more information like the the current list of Spring
Cloud Stream Binders here library as a dependency to the application.
Spring Cloud Stream supports Google PubSub (partner maintained) binder implementation link.
Here you can find a related SO question Spring dataflow and GCP Pub Sub.