I have an application with file-supplier Srping Cloud module included. My workflow is like track file creating/modifying in particular directory->push events to Kafka topic if there are such events. FileSupplierConfiguration is used to configure this file supplier. But now I have to track one more directory and push events to another relevant Kafka topic. So, there is an issue, because there is no possibility to include multiple FileSupplierConfiguration in project for configuration another file supplier. I remember that one of the main principles of microservices for which spring-cloud-stream was designed for is you do one thing and do it well without affecting others, but it still the same microservice with same tracking/pushing functionality but for another directory and topic. Is there any possibility to add one more file supplier with relevant configuration with file-supplier module? Or is the best solution for this issue to run one more application instance with another configuration?
Yes, the best way is to have another instance of this application, but with its own specific configuration properties. This is really how these functions have been designed: microservice based on the convention on configuration. What you are asking really contradicts with Spring Boot expectations. Imaging you'd need to connect to several data bases. So, you only can have a single JdbcTemplate auto-configured. The rest is only possible manually. But better to have the same code base which relies on the auto-configuration and may apply different props.
Related
Let me preface this with the fact that I am still very new to Apache Camel. I'm still trying to understand how it all works, and what needs to be done (and HOW to do it) to achieve a particular effect.
I am trying to develop a Spring Boot application that will use Apache Camel to handle the transmission (and possibly also receipt) of data to/from a number of possible sources and destinations. The purpose of the application is to provide a means to produce/generate network traffic, at the network application level, that will be fed into another Spring Boot application - let's call this the target. We are trying to observe and measure the effects various network loads have on the target.
We would like to be able to transmit data via a number of protocols, including: ftp, http/s, file systems (nfs), various mail protocols (smtp, pop) and data streaming protocols for voice and video. There may be other protocols added at a later time. The data itself is irrelevant, we just need to be able to transmit data via various protocols with various loads.
These applications/services will be running in a containerized environment (Docker) that will be run within our local development and test environment, as well as possibly in a cloud environment, such as AWS. We have used Docker, Ansible, Terraform and are currently working towards using Kubernetes and Istio to manage the configuration, deployment, and operation of these applications.
We need to be able to provide specific configurations of Camel routes for particular deployments.
It would appear that the preferred method to configure Camel routes is via Java DSL, rather than XML DSL. The Camel documentation and nearly every other source of information I've found have a strong bias towards using Java DSL. Examples of XML DSL route configuration are far and few.
My initial impression is that going the Java DSL route (excuse the pun), would not work well with our need to be able to deploy a Camel application with a specific route configuration. It seems like you are required to have Java DSL defined route configurations hardwired into the code.
We think that it will be easier to provide a specific route configuration via an XML file that can be included in a deployment, hence why I've been trying to investigate and experiment with XML DSL. Perhaps we are mistaken in this regard.
My question to the community is: Considering what I've described above, can the Java DSL approach be used to meet the requirements as I've described them? Can we use Java DSL in a way that allows for dynamic route configuration? Keep in mind we would not be attempting to change configuration during operation, just in the course of performing a deployment.
If Java DSL could be used for this purpose, it would be very much appreciated if pointers to documentation, examples, etc. could be provided.
For your use cases you could use XML DSL also. Anyhow below book covers most aspects Camel development with examples. In this book authors describes XML DSL use for most of java DSL examples.
https://www.manning.com/books/camel-in-action-second-edition
In below github repository you can find the source code for all the examples listed in above book.
https://github.com/camelinaction/camelinaction2
Simple tutorial and github repository for Apache Camel using Spring boot.
https://www.baeldung.com/apache-camel-spring-boot
https://github.com/eugenp/tutorials/tree/master/spring-boot-modules/spring-boot-camel
Maven Plugin for build and deployment of spring boot container application into Kubernetes cluster
https://maven.fabric8.io/
In case if your company can afford some funding for your effort look at below link which provides commercial offerings around Camel.
https://camel.apache.org/manual/latest/commercial-camel-offerings.html
Thanks
Madhu Gupta
Our team has a few projects which use the Java DSL for building routes. In order to make them dynamic, there are control structures for iterating and setting endpoints based off configurations. That works for us because the routes are basically all the same, just with different sources and sinks.
If you could dynamically add/change the XML DSL files in a way that doesn't involve redeploying your application, that might be a viable route to follow. One might, for example, change the camel.springboot.xml-routes property to point to a folder which changes as needed.
I have externalized properties defined in separate jar. I want to read this in Spring cloud config server and expose this as service to connecting app. These properties would vary depending on environment, again using Profile, one can get relevant properties.
I see there are many ways to do this, using PropertySource One with PropertySourcesPlaceholderConfigurer. Few approaches are using Environment
What are pros and cons between each approach. This might be community question. I see plenty of questions on this area, but there isn't a uniformed, guideline on what approach we need to use for the different scenarios? One of the youtube video talks about having 15 different ways to pick properties. I would like to know more around this area and so that this might be useful for other developers
I have a use case where I would like build a common interface or service which can update entities of application. Example case is shown as below:
Now every application has to handle update functionality of entities. Rather than implementing update functionality in n application module. I would like to build a common interface or server in spring boot.
Service will be like below:
My question is how to design service/interface which can used for above scenario. Any api or tool which can help me to achieve this. I dont want to write code for update in every application module.
Thanks in advance.
Last year I was thinking about the similar concept to yours, but in Apache Camel framework context. I haven't got enough time and motivation to do so, but your post encouraged me to give it a try - perhaps mostly because I've found your concept very similar to mine.
This is how I see it:
So basically I considered an environment with application that might uses N modules/plugins that enriches application's features, i.e. processing feature etc. Application uses module/plugin when it is available in the classpath - considering Java background. When the module is not available application works without its functionality like it was never there. Moreover I wanted to implement it purely using framework capabilities - in this case Spring - without ugly hacks/ifs in the source code.
Three solutions come to my mind:
- using request/response interceptors and modifying(#ControllerAdvice)
- using Spring AOP to intercept method invocations in *Service proxy classes
- using Apache Camel framework to create a routes for processing entities
Here's the brief overview of POC that I implemented:
I've chosen Spring AOP because I've never been using it before on my own.
simple EmployeeService that simulates saving employee - EmployeeEntity
3 processors that simulates Processing Modules that could be located outside the application. These three modules change properties of EmployeeEntity in some way.
one Aspect that intercepts "save" method in EmployeeService and handles invocation of available processors
In the next steps I'd like to externalize these Processors so these are some kind of pluggable jar files.
I'm wondering if this is something that you wanted to achieve?
link to Spring AOP introduction here: https://docs.spring.io/spring/docs/5.0.5.RELEASE/spring-framework-reference/core.html#aop
link to repository of mentioned POC: https://github.com/bkpawlowski/spring-aop
I have a Spring Integration project that has several flows (some where between 10-15). I would like to keep my namespace clean since several flows might have similar sounding components (for ex - several flows might have a channel named fileValidatorChannel). I think I have a couple of different options to keep names from colliding with each other:
A. Preface every component name with the flow that it belongs to. For ex - flowAFileValidatorChannel, flowBFileValidatorChannel, etc
B. Create a context hierarchy where every flow is it's own context and every flow inheriting from a master context where all the common beans/sub-flows are.
What's the better approach? Is there are better way to keep my name space clean?
To be honest your problem isn't clear.
Any Spring Integration component is a bean finally. So, their ids are just to distinguish them from other bean.
Let's imaging if you don't have Spring Integration in your application. So, you would worry about some clean naming strategy for all your beans anyway?
From other side consider to use Spring Integration Flow project:
The goal is to support these, and potentially other semantics while providing better encapsulation and configuration options. Configuration is provided via properties and or referenced bean definitions. Each flow is initialized in a child application context. This allows you to configure multiple instances of the same flow differently.
I have been playing with Spring Cloud Config and like many of the ideas I see there. I would like to better understand how its creators intended on it being used though.
Lets say that I have several services that support a larger API. Because these services are independent from each other, their source is managed in separate repositories. This allows us to version them and deploy them separately from one another. Today, their properties are managed individually.
I like the idea of having a single config server provide all of the configuration information for the individual applications/services that support this larger API. Looking at the default implementation of EnvironmentRepository (which is GIT based), I would have to have a single repository with all of my application config files in it. Because they all live within the same repository, they would all be managed/versioned together in a single place.
How do I make both models work with each other? Would it be better to have a repository per application instead of one for all applications? What are your thoughts?
-Joshua
It might just be a detail of the implementation of the EnvironmentRepository. See here for some discussion on how and when that might happen.