Spring cloud stream messaging system(RabbitMQ) implementation using Rest Controller(Restful API) - spring

From past few days i'm trying to implement the Spring cloud stream messaging system using RestController, but it is not happening through the current implementation.
For this sample code i'm going to add RestController
#EnableBinding(Source.class)
#EnableConfigurationProperties(TimeSourceOptionsMetadata.class)
public class TimeSource {
#Autowired
private TimeSourceOptionsMetadata options;
#InboundChannelAdapter(value = Source.OUTPUT)
public String timerMessageSource() {
return new SimpleDateFormat(this.options.getFormat()).format(new Date());
}
}
But the #InboundChannelAdapter cannot accept any parameters from RequestMapping Get Method URL.At the end what i need is to add message to the broker using Restful API Get method from api call. which is the best way to do it?, I couldn't figure out any best process from internet.

spring cloud team already provided a source application that listens for HTTP requests and emits the body as a message payload. If the Content-Type matches text/* or application/json, the payload will be a String, otherwise the payload will be a byte array.
github link
You can go with this or if you want to write it yourself, you can do it like below:
#RestController
#EnableBinding(Source.class)
public class RestSource {
#Autowired
private Source channels;
#RequestMapping(path = "/", method = POST, consumes = {"application/json" })
#ResponseStatus(HttpStatus.ACCEPTED)
public void handleRequest(#RequestBody String body, #RequestHeader(HttpHeaders.CONTENT_TYPE) Object contentType) {
sendMessage(body, contentType);
}
private void sendMessage(Object body, Object contentType) {
channels.output().send(MessageBuilder.createMessage(body,
new MessageHeaders(Collections.singletonMap(MessageHeaders.CONTENT_TYPE, contentType))));
}
}

Related

Resolving POST /** request URL to full request URL using micrometer

With the micro-service architecture I have written a generic POST request handler which is consumed by all the micro-services. The post mapping in spring look like this:
#RestController
#RequestMapping(value = "/v1/", consumes = {MediaType.APPLICATION_JSON_VALUE}, produces = {MediaType.APPLICATION_JSON_VALUE})
public class V1Controller {
#PostMapping(path = "/**")
public #ResponseBody Json post () {}
}
Now while I am consuming the metrics for this endpoint using micrometer I am only getting /v1/ as the endpoint in the metrics while I am sending the full URL like /v1/demo/foo from the calling service. I tried lot of the combination but it is not working. I have also added the WebMvcTagsProvider where I am listing to request and resolving the POST api calls.
#Bean
#SuppressWarnings("unchecked")
public WebMvcTagsProvider webMvcTagsProvider(ObjectMapper objectMapper) {
return new DefaultWebMvcTagsProvider() {
public Iterable<Tag> getTags(HttpServletRequest request, HttpServletResponse response, Object handler, Throwable exception) {
if ("POST".equals(request.getMethod())) {
Tag uriTag = Tag.of("uri", String.valueOf(request.getRequestURI()));
return Tags.of(WebMvcTags.method(request), uriTag, WebMvcTags.exception(exception), WebMvcTags.status(response));
}
return Tags.of(WebMvcTags.method(request), WebMvcTags.uri(request, response), WebMvcTags.exception(exception), WebMvcTags.status(response));
}
};
}
Still it is resolving to /v1/ URL in the metrics. I tried googling alot but didn't find any solution. Thanks in advance.
The build in Spring Boot RequestMapping based metrics match on the annotations and add those as tags.
This is to avoid a tag explosion. Imagine a #RequestMapping for a path like user/{userId}, you would want to group all those calls together (user/1, user/2, user/3).
You'll want to create your own Timer in your post method that set that url tags, etc there.
If you decide to reuse the same metric name as the built in Spring Boot metric, you'll want to disable that one as well, so you don't double count those requests.

Limiting Access to Endpoints by Method in Spring Boot

We are implementing a simple health check API our load balancers can call to help with the routing of requests. If the status of the application is "standby", requests should not be sent to it. Only the admins can set the state to "up" or "standby", but anyone (including the load balancers) can get the status of the application.
We are trying this with Spring Boot 2, but are having problems configuring security to grant anonymous access to just one of the routes. Consider the following controller:
#RestController
public class AppStatusController {
private static final String STATUS = "status";
String state = "standby";
private String getState() {
return state;
}
private Map<String, String> getStatusMap() {
Map<String, String> retval = new HashMap<>();
retval.put(STATUS, getState());
return retval;
}
// GET calls are public, all others require AuthN & AuthZ
#GetMapping(path = "/appstatus", produces = "application/json")
public Map<String, String> getStatus() {
return getStatusMap();
}
// Only those with the ADMIN role can POST to this endpoint
#PostMapping(path = "/appstatus", consumes = "application/json", produces = "application/json")
public Map<String, String> setStatus(#RequestBody Map state) {
// Validate and update the state
return getStatusMap();
}
}
There is only one endpoint, /appstatus, but one method is called with an HTTP GET and the other with an HTTP POST. We want calls to getStatus to be public, but allow Spring Security to control access to setStatus. One might expect an annotation such as #Anonymous or something similar to be applied to the getStatus() method but we can't seem to find one.
Some have suggested using a separate #Configuration class and setting up antMatchers but it's not clear how we can match on the HTTP method.
Does anyone have suggestions on how to configure Spring Security to allow public access to GET method requests but control access to other methods?
EDIT: We are trying to avoid any authentication on the getStatus() call. We can't store auth credentials in the health check probe and can't perform a login exchange. This is a simple GET request to see if the application is up and ready for operation.
Have you tried using Method Security Expressions?
It looks like this will do what you want:
// GET calls are public, all others require AuthN & AuthZ
#GetMapping(path = "/appstatus", produces = "application/json")
#PreAuthorize("permitAll")
public Map<String, String> getStatus() {
return getStatusMap();
}
// Only those with the ADMIN role can POST to this endpoint
#PostMapping(path = "/appstatus", consumes = "application/json", produces = "application/json")
#PreAuthorize("hasRole('ADMIN')")
public Map<String, String> setStatus(#RequestBody Map state) {
// Validate and update the state
return getStatusMap();
}
Note: I don't know what roles your admins have, so I used 'ADMIN' as a placeholder.

Can I use Spring WebFlux to implement REST services which get data through Kafka request/response topics?

I'm developing REST service which, in turn, will query slow legacy system so response time will be measured in seconds. We also expect massive load so I was thinking about asynchronous/non-blocking approaches to avoid hundreds of "servlet" threads blocked on calls to slow system.
As I see this can be implemented using AsyncContext which is present in new servlet API specs. I even developed small prototype and it seems to be working.
On the other hand it looks like I can achieve the same using Spring WebFlux.
Unfortunately I did not find any example where custom "backend" calls are wrapped with Mono/Flux. Most of the examples just reuse already-prepared reactive connectors, like ReactiveCassandraOperations.java, etc.
My data flow is the following:
JS client --> Spring RestController --> send request to Kafka topic --> read response from Kafka reply topic --> return data to client
Can I wrap Kafka steps into Mono/Flux and how to do this?
How my RestController method should look like?
Here is my simple implementation which achieves the same using Servlet 3.1 API
//took the idea from some Jetty examples
public class AsyncRestServlet extends HttpServlet {
...
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
String result = (String) req.getAttribute(RESULTS_ATTR);
if (result == null) { //data not ready yet: schedule async processing
final AsyncContext async = req.startAsync();
//generate some unique request ID
String uid = "req-" + String.valueOf(req.hashCode());
//share it to Kafka receive together with AsyncContext
//when Kafka receiver will get the response it will put it in Servlet request attribute and call async.dispatch()
//This doGet() method will be called again and it will send the response to client
receiver.rememberKey(uid, async);
//send request to Kafka
sender.send(uid, param);
//data is not ready yet so we are releasing Servlet thread
return;
}
//return result as html response
resp.setContentType("text/html");
PrintWriter out = resp.getWriter();
out.println(result);
out.close();
}
Here's a short example - Not the WebFlux client you probably had in mind, but at least it would enable you to utilize Flux and Mono for asynchronous processing, which I interpreted to be the point of your question. The web objects should work without additional configurations, but of course you will need to configure Kafka as the KafkaTemplate object will not work on its own.
#Bean // Using org.springframework.web.reactive.function.server.RouterFunction<ServerResponse>
public RouterFunction<ServerResponse> sendMessageToTopic(KafkaController kafkaController){
return RouterFunctions.route(RequestPredicates.POST("/endpoint"), kafkaController::sendMessage);
}
#Component
public class ResponseHandler {
public getServerResponse() {
return ServerResponse.ok().body(Mono.just(Status.SUCCESS), String.class);
}
}
#Component
public class KafkaController {
public Mono<ServerResponse> auditInvalidTransaction(ServerRequest request) {
return request.bodyToMono(TopicMsgMap.class)
// your HTTP call may not return immediately without this
.subscribeOn(Schedulers.single()) // for a single worker thread
.flatMap(topicMsgMap -> {
MyKafkaPublisher.sendMessages(topicMsgMap);
}.flatMap(responseHandler::getServerResponse);
}
}
#Data // model class just to easily convert the ServerRequest (from json, for ex.)
// + ~#constructors
public class TopicMsgMap() {
private Map<String, String> topicMsgMap;
}
#Service // Using org.springframework.kafka.core.KafkaTemplate<String, String>
public class MyKafkaPublisher {
#Autowired
private KafkaTemplate<String, String> template;
#Value("${topic1}")
private String topic1;
#Value("${topic2}")
private String topic2;
public void sendMessages(Map<String, String> topicMsgMap){
topicMsgMap.forEach((top, msg) -> {
if (topic.equals("topic1") kafkaTemplate.send(topic1, message);
if (topic.equals("topic2") kafkaTemplate.send(topic2, message);
});
}
}
Guessing this isn't the use-case you had in mind, but hope you find this general structure useful.
There is several approaches including KafkaReplyingRestTemplate for this problem but continuing your approach in servlet api's the solution will be something like this in spring Webflux.
Your Controller method looks like this:
#RequestMapping(path = "/completable-future", method = RequestMethod.POST)
Mono<Response> asyncTransaction(#RequestBody RequestDto requestDto, #RequestHeader Map<String, String> requestHeaders) {
String internalTransactionId = UUID.randomUUID().toString();
kafkaSender.send(Request.builder()
.transactionId(requestHeaders.get("transactionId"))
.internalTransactionId(internalTransactionId)
.sourceIban(requestDto.getSourceIban())
.destIban(requestDto.getDestIban())
.build());
CompletableFuture<Response> completableFuture = new CompletableFuture();
taskHolder.pushTask(completableFuture, internalTransactionId);
return Mono.fromFuture(completableFuture);
}
Your taskHolder component will be something like this:
#Component
public class TaskHolder {
private Map<String, CompletableFuture> taskHolder = new ConcurrentHashMap();
public void pushTask(CompletableFuture<Response> task, String transactionId) {
this.taskHolder.put(transactionId, task);
}
public Optional<CompletableFuture> remove(String transactionId) {
return Optional.ofNullable(this.taskHolder.remove(transactionId));
}
}
And finally your Kafka ResponseListener looks like this:
#Component
public class ResponseListener {
#Autowired
TaskHolder taskHolder;
#KafkaListener(topics = "reactive-response-topic", groupId = "test")
public void listen(Response response) {
taskHolder.remove(response.getInternalTransactionId()).orElse(
new CompletableFuture()).complete(response);
}
}
In this example I used internalTransactionId as CorrelationId but you can use "kafka_correlationId" that is a known kafka header.

Response MIME type for Spring Boot actuator endpoints

I have updated a Spring Boot application from 1.4.x to 1.5.1 and the Spring Actuator endpoints return a different MIME type now:
For example, /health is now application/vnd.spring-boot.actuator.v1+json instead simply application/json.
How can I change this back?
The endpoints return a content type that honours what the client's request says it can accept. You will get an application/json response if the client send an Accept header that asks for it:
Accept: application/json
In response to the comment of https://stackoverflow.com/users/2952093/kap (my reputation is to low to create a comment): when using Firefox to check endpoints that return JSON I use the Add-on JSONView. In the settings there is an option to specify alternate JSON content types, just add application/vnd.spring-boot.actuator.v1+jsonand you'll see the returned JSON in pretty print inside your browser.
As you noticed the content type for actuators have changed in 1.5.x.
If you in put "application/json" in the "Accept:" header you should get the usual content-type.
But if you don't have any way of modifying the clients, this snippet returns health (without details) and original content-type (the 1.4.x way).
#RestController
#RequestMapping(value = "/health", produces = MediaType.APPLICATION_JSON_VALUE)
public class HealthController {
#Inject
HealthEndpoint healthEndpoint;
#RequestMapping(method = RequestMethod.GET)
public ResponseEntity<Health > health() throws IOException {
Health health = healthEndpoint.health();
Health nonSensitiveHealthResult = Health.status(health.getStatus()).build();
if (health.getStatus().equals(Status.UP)) {
return ResponseEntity.status(HttpStatus.OK).body(nonSensitiveHealthResult);
} else {
return ResponseEntity.status(HttpStatus.BAD_REQUEST).body(nonSensitiveHealthResult);
}
}
}
Configuration (move away existing health)
endpoints.health.path: internal/health
Based on the code in https://github.com/spring-projects/spring-boot/issues/2449 (which also works fine but completely removes the new type) I came up with
#Component
public class ActuatorCustomizer implements EndpointHandlerMappingCustomizer {
static class Fix extends HandlerInterceptorAdapter {
#Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler)
throws Exception {
Object attribute = request.getAttribute(HandlerMapping.PRODUCIBLE_MEDIA_TYPES_ATTRIBUTE);
if (attribute instanceof LinkedHashSet) {
#SuppressWarnings("unchecked")
LinkedHashSet<MediaType> lhs = (LinkedHashSet<MediaType>) attribute;
if (lhs.remove(ActuatorMediaTypes.APPLICATION_ACTUATOR_V1_JSON)) {
lhs.add(ActuatorMediaTypes.APPLICATION_ACTUATOR_V1_JSON);
}
}
return true;
}
}
#Override
public void customize(EndpointHandlerMapping mapping) {
mapping.setInterceptors(new Object[] {new Fix()});
}
}
which puts the new vendor-mediatype last so that it will use application/json for all actuator endpoints when nothing is specified.
Tested with spring-boot 1.5.3
Since SpringBoot 2.0.x the suggested solution in implementing the EndpointHandlerMappingCustomizer doesn't work any longer.
The good news is, the solution is simpler now.
The Bean EndpointMediaTypes needs to be provided. It is provided by the SpringBoot class WebEndpointAutoConfiguration by default.
Providing your own could look like this:
#Configuration
public class ActuatorEndpointConfig {
private static final List<String> MEDIA_TYPES = Arrays
.asList("application/json", ActuatorMediaType.V2_JSON);
#Bean
public EndpointMediaTypes endpointMediaTypes() {
return new EndpointMediaTypes(MEDIA_TYPES, MEDIA_TYPES);
}
}
To support application/vnd.spring-boot.actuator.v1+json in Firefox's built in JSON viewer, you can install this addon: json-content-type-override. It will convert content types that contain "json" to "application/json".
Update: Firefox 58+ has built-in support for these mime types, and no addon is needed anymore. See https://bugzilla.mozilla.org/show_bug.cgi?id=1388335

Spring Cloud : Using routing type filter in Zuul

I have 2 micro-services (Service A and Service B) built using Spring Boot, which gets routed through a Zuul Proxy also built as a Spring Boot app and I have checked that the Zuul proxy works just fine. However, what I am trying to do is to write a custom routing type ZuulFilter which should first route to Service A when a request comes in for Service B. Here is what I need assistance for:
I would like to know an example of how a routing filter looks like as I do not see anything after searching the internet. What I get are some examples of pre-filter and Netflix's documentation doesn't help much as well on that aspect.
Whether writing a custom route filter would mess up the original routing behavior of Zuul
I would construct a Feign client in the Zuul filter and make the call to service A using it. Feign will populate a ribbon load balancer to make the call in just the same way that Zuul does when proxying.
I had the same issue and this is what I came up with.
public class ServicesLegacyRouteFilter extends ZuulFilter {
private ServiceB serviceB;
public ServiceLegacyRouteFilter(ServiceB serviceB) {
this.serviceB = serviceB;
}
#Override
public String filterType() {
return ROUTE_TYPE;
}
#Override
public int filterOrder() {
return 10;
}
#Override
public boolean shouldFilter() {
RequestContext ctx = RequestContext.getCurrentContext();
if ("serviceA".equals(ctx.get("serviceId"))) {
//call Service B here and use return type to set
//the final destination service
String destination = serviceB.routeWhere();
ctx.set("serviceId", destination);
return true;
}
return false;
}
#Override
public Object run() {
RequestContext ctx = RequestContext.getCurrentContext();
// Or call ServiceB here to make your determination on
// the final destination.
String destination = serviceB.routeWhere();
ctx.set("serviceId", destination);
return null;
}
}
My actual production use case was more complicated on the routing of course, but this is the basics of how I was able to change routes based on what was coming in and how to take advantage of Zuul to get it out to the correct service.

Resources