I want to implement a serverless function that polls an HTTP endpoint and publishes the response to a messaging queue.
My initial thought is to build a spring boot application using spring integration gateway and adapters for HTTP polling and publishing to queue (and deploy as lambda). Is there a better option in the spring stack?
I looked at spring cloud function, spring cloud stream, spring cloud task. Any suggestions?
In Spring Cloud Stream this type of microservice is called source. So, you need to have a Supplier bean based on Spring Integration Java DSL to build a simple flow to let Spring Cloud Stream to poll it periodically and produce a result into bound destination.
Something like this:
#Bean
public IntegrationFlow pollingHttpFlow() {
return IntegrationFlows
.from(Supplier.class, gateway -> gateway.beanName("httpSupplier"))
.handle(Http.outboundGateway("http://somehost"))
.get();
}
See a blog post about this kind of interoperability: https://spring.io/blog/2019/10/25/spring-cloud-stream-and-spring-integration
Related
With Spring Cloud Stream - is it possible to use spring cloud function("function" specifically) to expose as rest endpoint and publish to a topic - something like below-
#Bean
public Function<String,String> postLoginEvent() {
return valFrmRest -> valFrmRest;
}
Currently if I try this, the endpoint isn't exposed and I get 404. As a workaround, I used an EmitterProcessor to get input from rest endpoint and separately have a supplier return this processor to publish to a topic. Not sure if what I am asking is technically possible, but it might be a pretty common usecase and it will be nice to have out-of-the-box functionality available. Any thoughts? Thank you in advance for sharing your inputs.
You can have it as a REST endpoint and then use StreamBridge API to publish it programmatically. See the details here.
Here is a sample using StreamBridge.
I have been trying to use the Spring Integration's #Router with Spring Cloud Stream with Kafka binding. My understanding is that when you return a List of channel names from the method annotated with #Router they should be produced to the destined Kafka topics. But I don't see the messages being produced.
Does Spring Integration's #Router work with Spring Cloud Stream. If not, what's the alternative and how do I programmatically route to channels selected at runtime?
What makes you believe that Spring Integration #Router has any effect in Spring Cloud Stream? It's a completely different framework.
Yes there are mechanisms to route FROM and TO in Spring Cloud Stream and they are described here.
I think the specific section of interest to your use case is Routing FROM, but consider reading the full section to understand the differences and mechanisms used.
I have to invoke a couple of rest web service from my spring boot application. I am planning to use the Camel to configure the flow and other EIP use cases. Some of the endpoints are using oAuth2 authentication. I am planning to use the Spring oAuthResttempalte. All the examples on the internet are either using restlet, CXF or camel-http.
Camel Rest Consmer
I am not able to find a single example with just spring resttemplate. Did anyone implement Camel Rest consumer using Spring Resttemplate?
Some of the Examples on the internet use a jetty server to consume a rest endpoint. Why do you need a jetty server for simple rest consumer?
Did anyone implement Camel Rest consumer using Spring Resttemplate?
I'm not aware of that and it's unlikely to found something in that direction because Camel already have bult-in components to consume rest endpoints.
Some of the Examples on the internet use a jetty server to consume a rest endpoint. Why do you need a jetty server for simple rest consumer?
I believe that jetty was used as a consumer not a producer endpoint. So you won't need the "server". Or maybe you saw an example using jetty acting as a server to serve an OAuth endpoint?
If you excuse my approach, I'd suggest to remain with Camel HTTP/Rest capabilities to consume REST APIs using OAuth. I've found this example on Gist:
from("direct:authService").tracing()
.setHeader(Exchange.HTTP_PATH)
.simple("<auth service context>/oauth2/token")
.setHeader("CamelHttpMethod")
.simple("POST")
.setHeader("Content-Type")
.simple("application/x-www-form-urlencoded")
.setHeader("Accept")
.simple("application/json")
.setBody()
.constant("grant_type=client_credentials&client_id=<client id>&client_secret=<client sec>")
.to("https4://<remote auth service url>")
.convertBodyTo(String.class)
.log("response from API: " + body())
.choice()
.when().simple("${header.CamelHttpResponseCode} == 200")
.unmarshal().json(JsonLibrary.Jackson, AccessResponseToken.class)
.setHeader("jwt").simple("${body.access_token}")
.to("direct:<some direct route>")
.otherwise()
.log("Not Authenticated!!!");
If you want to stick into OAuthRestTemplate you may implement a Processor or a bean to wrap those calls and return to your route the authorization token.
I have a spring webapp that communicates with external service over kafka. IS it possible to somehow test contract between those services?
Yes you can. Spring Cloud Contract supports CDC with messaging. If you're using Spring Cloud Stream - the work to be done is trivial. If not then you'll have to implement your own as presented in this issue - Spring Cloud Contract and plain Spring AMQP . Summing it up it's enough for both consumer and producer to implement a custom org.springframework.cloud.contract.verifier.messaging.MessageVerifier bean that will be responsible for receiving and sending of messages via Kafka
I want to integration a restful interface with Spring Integration. I'm fairly new to Spring Integration and don't really know where to start. The goal is that the restful interface is polled regularly and then a Spring Batch job is automatically started with the new data. Is there somewhere a good example of how to do that? Where is a good point to start? I would like to implement this with Spring DSL.
IntegrationFlows.from(// MessageSource to generate request //
, e -> e.poller(...))
.handle(Http(outboundGateway(...))
.transform(// to JobLaunchRequest //)
.handle(// JobLaunchingRequestHandler //)
.handle(// JobExecutionResult //)
.get();
Refer to the Spring Integration Documentation for information about configuring the HTTP gateway and concepts of transformers, message handlers, and the spring-batch-integration project (part of spring-batch) for JobLaunchingRequestHandler etc.