I'm using Spring Cloud Stream 3.x in Spring Boot 2.x application to consume messages from a Kafka topic.
I want to have a listener that consumes messages conditionally on some custom header value, as per doc:
#StreamListener(value = "someTopic", condition = "headers['SomeHeader']=='SomeHeaderValue'")
public void onMessage(Message<?> message) {
LOGGER.info("Received: {}", message);
}
However listener never gets notified, and if condition is removed I see the following in the log:
Received: ... SomeHeader: [B#1055e4af ...
It turns out that custom headers are left in Kafka byte array raw format, making them not eligible for condition evaluation.
Is some additional configuration needed or am I missing something?
After some digging in sources and stackoveflow I have found the following:
Spring Cloud Stream delegates to Spring Kafka message and headers conversion (KafkaMessageChannelBinder ~ getHeaderMapper)
Headers are left in raw format by default headers conversion implementation (BinderHeaderMapper)
Spring Cloud Stream allows customization of headers mapping and particularly conversion of headers from byte array to String (How can I map incoming headers as String instead of byte[] in my Spring Cloud Stream project?)
So I added my custom header mapper bean (bean name is important, it allows to omit additional configuration property), which maps my custom header to String:
#Bean
public KafkaHeaderMapper kafkaBinderHeaderMapper() {
SimpleKafkaHeaderMapper headerMapper = new SimpleKafkaHeaderMapper();
headerMapper.setRawMappedHeaders(Map.of(
"SomeHeader", true
));
return headerMapper;
}
That fixed the problem:
Received: ... SomeHeader: SomeHeaderValue ...
P.S. It seems like a bug in Spring Cloud Stream:
It introduces its own implementation of header mapper (BinderHeaderMapper), but the latter doesn't respect conditional routing feature.
Header mapper is subclassed in KafkaMessageChannelBinder, this added behaviour is non-obvious and will be lost if custom header mapper is provided.
Related
Let's say I have a very simple HTTP endpoint using Spring Webflux:
#GetMapping
fun greeting(#RequestParam("msg") val message : String) = Mono.just(Greeting(message))
where Greeting is a simple DTO serialized as JSON. How can I instruct Spring Webflux to return the response compressed as GZIP? I'm using the Netty implementation if that matters.
What you are looking for is server compression properties
server.compression.enabled=true
server.compression.min-response-size=1024
Adding to above-accepted answer, it's always better to give min-response-size as well to remove server overhead of compressing all responses and also the the mime-types.
server.compression.enabled=true
server.compression.mime-types=text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json,application/xml
server.compression.min-response-size=2KB
I have a Spring Boot based REST API application with the following endpoint (Written in Kotlin)
#RequestMapping(value = ["/search"], method = [RequestMethod.GET])
#ApiOperation("Check whether any of the map values exists. Returns string 'true' if stamp exists, else 'false'")
fun checkExists(
#ApiParam("information about the stamp as key-value pairs (example: ds=2017-11-34&hh=05)", required = true)
#RequestParam searchValues: Map<String, String>
): Boolean {
return service.checkExists(searchValues)
}
And I know Spring supports sending a dynamic map of key value pairs as documented here.
I am also using Swagger to document the API definitions, and further more, I am using swagger-codegen-cli to generate the client library using which someone can connect with this REST API.
Now, the issue is, I am not able to send a map of values from the swagger generated client to the Spring REST API (even though Spring supports it). Starting from Swagger OpenAPI 3, they've added support for Object types in the specification. But this works in a different way than I need. For example with just Spring and RequestParam of type Map
http://localhost:8080/search?foo=A&bar=B
is parsed as a map of key value pairs
key="foo",value="A"
key="bar",value="B"
But, When I send a Map Object from the swagger client with the same key-value pairs
Map<String, String> values = new HashMap<>();
values.put("foo","A");
values.put("bar","B");
return out = clientApi.checkExistsUsingGET(values);
This sends a request to the REST API in form of
http://localhost:8080/search?searchValues={foo=A,bar=B}
and the map in Spring side ends up as
key="searchValues",value="{foo=A,bar=B}"
I've been struggling to get the swagger client api to send the request in a way the Spring API is intended to work with Map of values, but I am not able to figure a solution.
Am I doing using the client API in a wrong way?, or this just can't be done with swagger?
Any suggestions/opinions appreciated!
This is not yet supported by swagger-ui. See this issue https://github.com/swagger-api/swagger-ui/issues/2241
What is the easiest way to add the HTTP.outboundGateway header in my program?
What I want to do is:
I first do the HTTP GET for the URL
http://localhost:8050/session
then I get the JSON
{
"session": "session8050"
}
I extract the value of the session variable and add that to the next HTTP GETas the session header variable.
Currently I have working code, but I was thinking could I do this easier? My implementation
Extracts the session variable from the JSON with the jsonPath method
Then the implementation adds the session variable to the integration flow message header with the enrichHeaders method
Then the implementation adds the session variable to the HTTP call header with the HeaderMapper class
My implementation is
integrationFlowBuilder
.transform(p -> authenticationJson)
.enrichHeaders(h -> h.header("Content-Type", "application/json"))
.handle(Http.outboundGateway("http://localhost:8050/session").httpMethod(HttpMethod.POST)
.expectedResponseType(String.class))
.enrichHeaders(
h -> h.headerExpression("session", "#jsonPath(payload, '$.session')", true)
.handle(Http
.outboundGateway(completeFromUrl)
.httpMethod(HttpMethod.GET).mappedRequestHeaders("session").headerMapper(headerMapper())
.expectedResponseType(String.class))
My headerMapper is
#Bean
HeaderMapper headerMapper() {
final DefaultHttpHeaderMapper headerMapper = new DefaultHttpHeaderMapper();
final String[] headerNames = { "session" };
headerMapper.setOutboundHeaderNames(headerNames);
headerMapper.setUserDefinedHeaderPrefix("");
return headerMapper;
}
Is it possible to extract the session variable from the JSON and add it straight to the HTTP headers??
Why the HeaderMapper must be used? Why the integration flow message headers don't go straight to the HTTP.outboundGateway call as the payload goes?
First of all you need to understand that main goal of Spring Integration as any other EIP solution is to make components in the flow as isolated as possible, so in the future you can add some intermediate steps or remove without big impact for the whole solution and other components in your integration flow. This should be an answer to your questions about why HeaderMapper must be used.
As you see the contract of the HeaderMapper to remap MessageHeaders to the target protocol headers representation. There is nothing about payload, hence you need to map the value from the payload into the headers, first of all. And then say Http.outboundGateway() what should be remapped from the MessageHeaders into the HttpHeaders.
By default the DefaultHttpHeaderMapper (it is present there in the Http.outboundGateway()) maps only standard HTTP headers suitable for the HTTP request.
If you need to include some custom header, like in your case with that session, you really can use a custom configuration for the DefaultHttpHeaderMapper, or just configure a convenient option on the Http.outboundGateway():
.mappedRequestHeaders("session")
The setUserDefinedHeaderPrefix("") is not necessary from version 5.0. It is empty string by default now, since there is no requirements in the prefixes for custom headers in the HTTP protocol.
I am looking at this example - https://github.com/spring-cloud/spring-cloud-stream-samples/blob/master/kafka-streams-samples/kafka-streams-product-tracker/src/main/java/kafka/streams/product/tracker/KafkaStreamsProductTrackerApplication.java
Trying to do something similar, but for me it is not working. How does product json string is received as Product Object ?
By default, the deserialization on the inbound KStream is done by Spring Cloud Stream. The default content-type used is application/json (equivalent to providing the property: spring.cloud.stream.bindings.input.contentType: application/json). This is why the product json string is properly converted.
You can disable the framework level conversion and let Kafka do that in which case you need to provide the Serdes through properties. In order to enable native deserialization, you can set the property - spring.cloud.stream.bindings.input.consumer.useNativeDecoding: true. Then you need to provide the appropriate Serdes. More on all these are here: https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.BUILD-SNAPSHOT/reference/htmlsingle/#_message_conversion
I'm experimenting with Spring 4 WebSocket STOMP application. Is there a way to explicitly specify content type of the returned message produced by a handler? By default the handler below produces application/json and processed by corresponding message converter.
#Controller
public class ProductController {
#MessageMapping("/products/{id}")
public String getProduct(#DestinationVariable int id) {
return getProductById(id);
}
}
I'm looking for something like #RequestMapping(produces = "text/xml") in Spring MVC.
UPDATE (reply to Rossen's answer):
Ideally I would like to be able to return both formats depending on what the user asks for. But if I have to choose, I would say XML and almost never JSON (XML is just an example, we use binary format). I went the second way you suggested - configuring custom converters instead of the default ones.
I have implemented custom MessageConverter extending AbstractMessageConverter. In the constructor I've registered appropriate supported MimeType.
Then I've registered my custom converter by overriding WebSocketMessageBrokerConfigurer's configureMessageConverters, and return false from the method to not add default converters.
As soon as my controller returns the value I get NPE in SendToMethodReturnValueHandler's postProcessMessage. This happens because CompositeMessageConverter contains only a single converter - my custom one. But my converter fails AbstractMessageConverter's supportsMimeType check and AbstractMessageConverter's toMessage returns null. This null causes the exception in postProcessMessage.
As the workaround I can register additional default MimeType application/json for my custom converter. But it looks like too dirty to me.
There is nothing like the produces condition. The only way to do this right now is to inject SimpMessagingTemplate and then use the convertAndSend method variation that takes a MessagePostProcessor. It could be made simpler. What's your use case? Are you using JSON primarily and need to use XML in a few places? Or do you need to use XML primarily? If it is the latter you can simply configure the converters to use instead of the default ones.
I see that you can change the headers changing the Message converter.
For example:
SockJsClient sockJsClient = new SockJsClient(transports);
WebSocketStompClient stompClient = new WebSocketStompClient(sockJsClient);
..
stompClient.setMessageConverter(new StringMessageConverter());
or
stompClient.setMessageConverter(new SimpleMessageConverter());
when you use StringMessageConverter then the Java client add an content-type header.
GenericMessage [payload=byte[33], headers={simpMessageType=MESSAGE, stompCommand=SEND, nativeHeaders={destination=[/app/chatchannel], content-type=[text/plain;charset=UTF-8], content-length=[33]}, simpSessionAttributes={ip=/127.0.0.1:59629}, simpHeartbeat=[J#147a2bf, contentType=text/plain;charset=UTF-8, lookupDestination=/chatchannel, simpSessionId=79431feb8b5f4a9497492ccc64f8965f, simpDestination=/app/chatchannel}]
But if you use SimpleMessageConverter, the content-type header is not add.
this is a clue about what you want to do?
Check this question, I put some code that could help you:
How configure the Spring Sockjs Java Client message converters