I am working on a project that is trying to use the polled consumer API. However, existing documentation, blog posts and sample code seems to use deprecated annotations (such as org.springframework.cloud.stream.annotation.Input). This seems to be because they are relying on the older style of Spring Cloud stream applications rather than using Java functional api (e.g., java.util.function.Function), as shown in other examples such as this one, given in the same repo.
Is there a way to use functional style with polled consumers in Spring Cloud Stream?
You are using outdated documentation. The most current is available from the project site - https://spring.io/projects/spring-cloud-stream#learn.
The section you are looking for is - https://docs.spring.io/spring-cloud-stream/docs/3.1.5/reference/html/spring-cloud-stream.html#spring-cloud-streams-overview-using-polled-consumers
Related
Let me preface this with the fact that I am still very new to Apache Camel. I'm still trying to understand how it all works, and what needs to be done (and HOW to do it) to achieve a particular effect.
I am trying to develop a Spring Boot application that will use Apache Camel to handle the transmission (and possibly also receipt) of data to/from a number of possible sources and destinations. The purpose of the application is to provide a means to produce/generate network traffic, at the network application level, that will be fed into another Spring Boot application - let's call this the target. We are trying to observe and measure the effects various network loads have on the target.
We would like to be able to transmit data via a number of protocols, including: ftp, http/s, file systems (nfs), various mail protocols (smtp, pop) and data streaming protocols for voice and video. There may be other protocols added at a later time. The data itself is irrelevant, we just need to be able to transmit data via various protocols with various loads.
These applications/services will be running in a containerized environment (Docker) that will be run within our local development and test environment, as well as possibly in a cloud environment, such as AWS. We have used Docker, Ansible, Terraform and are currently working towards using Kubernetes and Istio to manage the configuration, deployment, and operation of these applications.
We need to be able to provide specific configurations of Camel routes for particular deployments.
It would appear that the preferred method to configure Camel routes is via Java DSL, rather than XML DSL. The Camel documentation and nearly every other source of information I've found have a strong bias towards using Java DSL. Examples of XML DSL route configuration are far and few.
My initial impression is that going the Java DSL route (excuse the pun), would not work well with our need to be able to deploy a Camel application with a specific route configuration. It seems like you are required to have Java DSL defined route configurations hardwired into the code.
We think that it will be easier to provide a specific route configuration via an XML file that can be included in a deployment, hence why I've been trying to investigate and experiment with XML DSL. Perhaps we are mistaken in this regard.
My question to the community is: Considering what I've described above, can the Java DSL approach be used to meet the requirements as I've described them? Can we use Java DSL in a way that allows for dynamic route configuration? Keep in mind we would not be attempting to change configuration during operation, just in the course of performing a deployment.
If Java DSL could be used for this purpose, it would be very much appreciated if pointers to documentation, examples, etc. could be provided.
For your use cases you could use XML DSL also. Anyhow below book covers most aspects Camel development with examples. In this book authors describes XML DSL use for most of java DSL examples.
https://www.manning.com/books/camel-in-action-second-edition
In below github repository you can find the source code for all the examples listed in above book.
https://github.com/camelinaction/camelinaction2
Simple tutorial and github repository for Apache Camel using Spring boot.
https://www.baeldung.com/apache-camel-spring-boot
https://github.com/eugenp/tutorials/tree/master/spring-boot-modules/spring-boot-camel
Maven Plugin for build and deployment of spring boot container application into Kubernetes cluster
https://maven.fabric8.io/
In case if your company can afford some funding for your effort look at below link which provides commercial offerings around Camel.
https://camel.apache.org/manual/latest/commercial-camel-offerings.html
Thanks
Madhu Gupta
Our team has a few projects which use the Java DSL for building routes. In order to make them dynamic, there are control structures for iterating and setting endpoints based off configurations. That works for us because the routes are basically all the same, just with different sources and sinks.
If you could dynamically add/change the XML DSL files in a way that doesn't involve redeploying your application, that might be a viable route to follow. One might, for example, change the camel.springboot.xml-routes property to point to a folder which changes as needed.
I'm tasked with the requirement of using Spring Cloud Stream (with Kafka bindings). Going through Spring Cloud Stream's Quick Start and judging from the Config file it uses, LogSinkConfiguration, it seems the recommendation here is to use Spring Integration patterns, e.g. #ServiceActivator(inputChannel = Sink.INPUT) to connect to the input channel. However all the tutorials I've found are using a different set of annotations from the Spring Cloud Stream library (vs Spring Integration), i.e. #StreamListener(Processor.INPUT) in this walk-through.
So which is better/newer/preferred (i.e. what's the "best practice"). Should I use the S.I. way of configuring the sink using #ServiceActivator(inputChannel = Sink.INPUT) or the alternative #StreamListener(Processor.INPUT). Are these two methods of hooking up the sink virtually the same, or are there advantages to using one over the other?
With 2.0 we introduced spring-cloud-function support and with the imminent release of 3.0 we are now fully committed to functional support over any annotation driven (Spring Integration included).
So, to answer your question, functional support is the preferred way. Here is the link to more info and feel free to follow up with more concrete questions.
After some researching, I have discovered that there is a spring actuator project which is used to monitor and log information about the application. Here is the link that suggests how to put log in to my own custom repository for post processing. [http://www.sedooe.com/2017/08/tracing-requests-and-responses-with-spring-boot-actuator/ ]
However, the total in/out byte counts are not included. I know that I can try to wrap HttpServeletRequest and HttpServeletResponse (in the filter, pass them to the filter chain so that it can count in/out throughput whenever they are consumed or written out) which will return custom SerlvetInputStream and ServletOutputStream that can count the number of bytes.
This seems to be a common problem in business application and there should already be a solution so I wonder if there is any easier way to achieve this.
From my understanding the Spring-Boot 1.5 actuator implementation doesn't provide such a functionality. (Assuming you are talking about Spring-Boot 1.5.)
What you could do is having a look at the Micrometer project. It's the new actuator metrics "backend" for Spring-Boot 2.0. In case you are still using Spring-Boot 1.5 you can use the micrometer-spring-legacy library in your app.
Key thing to understand here is that for Spring-Boot 1.5 this is a metric backend which is living next to the "old" actuator.
What the project provides is instrumentation for embedded Jetty and Tomcat server. Allthough only the latter does provide in and out traffic metrics.
Assuming Tomcat, you'll get tomcat_global_received_bytes_total and tomcat_global_sent_bytes_total metrics over which you can reason about in your favorite monitoring system.
Is there anyway to extract Facebook data using Spring XD?
I am now able to extract Facebook data using Graph api, Json, Java Program, but the question is if could I integrate this this to be executed automatically through Spring XD.
You can create a custom source module.
If it is general enough, consider contributing it to spring-xd-modules. The README has information about contributing.
I was advised to develop Neo4j extension because of poor performance due to the many calls to the Neo4j REST interface introduced by Spring Data Neo4j. For the good start I was able to run simply Neo4j extension from Neo4j documentation:
http://docs.neo4j.org/chunked/stable/server-unmanaged-extensions.html
Now I have to move some logic from my web service to Neo4j extension to make communication between them as small as possible. However, my web service is based on Spring Data Neo4j as well as Spring Data JPA, so to reuse my actual code I would like to use also Spring Data in my extension. I have found this documentation:
http://docs.spring.io/spring-data/data-neo4j/docs/current/reference/html/reference_neo4j-server.html
but it is not clear for me and there is no working example. Is there any sample code with such integration?
There is a great example on using SDN in an unmanage extension, see http://inserpio.wordpress.com/2014/04/30/extending-the-neo4j-server-with-spring-data-neo4j/.