Spring cloud stream IntegrationFlow with Rabbitmq messaging, The consumer giving ASCII numbers as message payload - spring-boot

I am using spring cloud stream for messaging. In the consumer part, I used IntegrationFlow to listen to the queue. It is listening and printing the message from producer side. But the format is different that's the problem I am facing now. The content-type of the producer is application/json and the IntegrationFLow message payload showing ASCII numbers. The code is written for the consumer is given below
#EnableBinding(UserOperationConsume.class)
public class ConsumerController {
#Bean
IntegrationFlow consumerIntgrationFlow(UserOperationConsume u) {
return IntegrationFlows
.from(u.userRegistraionProduces())
//.transform(Transformers.toJson()) // not working as expected
//.transform(Transformers.fromJson(UserDTO.class))
.handle(String.class, (payload, headers) -> {
System.out.println(payload.toString()); // here the output is 123,34,105,100,34,58,49,44,34,110,97,109,101,34,58,34,86,105,115,104,110,117,34,44,34,101,109,97,105,108,34,58,34,118...
return null;
}).get();
}
}
The input interface is,
public interface UserOperationConsume {
#Input
public SubscribableChannel userRegistraionProduces();
}
And the consumer yml configuration is,
server:
port: 8181
spring:
application:
name: nets-alert-service
---
spring:
cloud:
config:
name: notification-service
uri: http://localhost:8888
---
spring:
rabbitmq:
host: localhost
port: 5672
username: guest
password: guest
---
spring:
cloud:
stream:
bindings:
userRegistraionProduces:
destination: userOperations
input:
content-type: application/json
I have tried Sink.class binding, That time I got an exact message from the queue. So please let me know if there is any mistake in this IntegrationFlow configuration. Because I am a newbie in spring cloud stream and IntegrationFlow. is there any way to transform this ascii to exact string?
Thanks in advance

Using IntegrationFlows.from(channel) provides no conversion hints so you just get the raw byte[] payload (containing JSON). It's not clear why you then use a toJson() transformer.
Your .handle(String.class, (payload, headers) -> {... is causing a simple ArrayToStringConverter to be used which is why you are seeing each byte value.
In any case, you are not using the framework properly. Use...
#StreamListener("userRegistraionProduces")
public void listen(UserDTO dto) {
System.out.println(dto);
}
...and the framework will take care of the conversion for you. Or...
#StreamListener("userRegistraionProduces")
public void listen(Message<UserDTO> dtoMessage) {
System.out.println(dtoMessage);
}
if your producer conveys additional information in headers.
EDIT
If you prefer to do the conversion yourself, this works fine...
#Bean
IntegrationFlow consumerIntgrationFlow(UserOperationConsume u) {
return IntegrationFlows.from(u.userRegistraionProduces())
.transform(Transformers.fromJson(UserDTO.class))
.handle((payload, headers) -> {
System.out.println(payload.toString());
return null;
}).get();
}
...since the Json transformer can read a byte[].

Related

Spring Cloud Stream Kafka Custom Error Handler

I have Kafka Consumer using spring cloud stream
spring:
cloud:
stream:
function:
definition: myConsumer
bindings:
myConsumer-in-0:
destination:myDest
binder: kafka
group: myGroup
content-type: application/json
I was able to catch errors through global error channel
#ServiceActivator(inputChannel = IntegrationContextUtils.ERROR_CHANNEL_BEAN_NAME)
public void processError(Message<MessageHandlingException> message) {
log.info("error had happend {}", message);
}
Now I have two questions
How can we redirect the error to a specific internal channel by a consumer?
Will the message be acknowledged automatically even for the error scenario?

Spring Integration Mqtt : DestinationResolutionException: no output-channel or replyChannel header available

Please can someone help me to understand where is the probleme in this config:
Versions :
org.springframework.integration:spring-integration-mqtt:5.5.2
org.springframework.boot:spring-boot-starter:2.5.3
org.eclipse.paho:org.eclipse.paho.client.mqttv3:1.2.5
#Configuration
public class MqttConfig {
#Bean
public MqttPahoClientFactory mqttClientFactory() {
DefaultMqttPahoClientFactory factory = new DefaultMqttPahoClientFactory();
MqttConnectOptions options = new MqttConnectOptions();
options.setServerURIs(new String[] { "tcp://localhost:1883" });
return factory;
}
#Bean
public MqttPahoMessageDrivenChannelAdapter inboundAdapter(MqttPahoClientFactory clientFactory) {
return new MqttPahoMessageDrivenChannelAdapter("MyApp", clientFactory, "ReplyTopic");
}
#Bean
IntegrationFlow inboundFlow(MqttPahoMessageDrivenChannelAdapter inboundAdapter) {
return IntegrationFlows.from(inboundAdapter)
.bridge()
.channel("replyChannel")
.get();
}
#Bean
public MessageChannel replyChannel() {
return MessageChannels.publishSubscribe().get();;
}
#Bean
public MqttPahoMessageHandler outboundAdapter(MqttPahoClientFactory clientFactory) {
return new MqttPahoMessageHandler("MyApp", clientFactory);
}
#Bean
public IntegrationFlow outboundFlow(MqttPahoMessageHandler outboundAdapter) {
return IntegrationFlows.from("requestChannel")
.handle(outboundAdapter).get()
}
#MessagingGateway
public interface MyGateway {
#Gateway(requestChannel = "requestChannel", replyChannel = "replyChannel")
String send(String request, #Header(MqttHeaders.TOPIC) String requestTopic);
}
}
Client code
#RestController
public class MyController {
#Autowired
private MyGateway myGateway;
#GetMapping("/sendRequest")
public String sendRequest() {
var response = myGateway.send("Hello", "MyTopic");
return response;
}
}
Usage:
curl http://localhost:8080/sendRequest
manual response from the mqtt broker (HiveMQ)
docker exec -it hivemq mqtt pub -t ReplyTopic -m "World" --debug
CLIENT mqttClient-MQTT_5_0-9ecded84-8416-4baa-a8f3-d593c692bc65: acknowledged PUBLISH: 'World' for PUBLISH to Topic: ReplyTopic
But I dont know why i have this message on the Spring application output
2022-10-25 18:04:33.171 ERROR 17069 --- [T Call: MyApp] .m.i.MqttPahoMessageDrivenChannelAdapter : Unhandled exception for GenericMessage [payload=World, headers={mqtt_receivedRetained=false, mqtt_id=0, mqtt_duplicate=false, id=9dbd5e14-66ed-5dc8-6cea-6d04ef19c6cc, mqtt_receivedTopic=ReplyTopic, mqtt_receivedQos=0, timestamp=1666713873170}]
org.springframework.messaging.MessageHandlingException: error occurred in message handler [org.springframework.integration.handler.BridgeHandler#6f63903c]; nested exception is org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available
Please can someone explain why i have this ?
no output-channel or replyChannel header available
I think the problem you are facing is not related to your bridge() configuration.
This comes from the MessagingGatewaySupport and its replyMessageCorrelator feature which is activated by your replyChannel = "replyChannel".
The real problem that you are trying to do what is not possible with MQTT v3. There is just no headers transferring over MQTT broker to carry on a required for gateway initiator a correlation key - the TemporaryReplyChannel. See more in docs about gateway: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#gateway.
In other words: independently of the replyChannel configuration on gateway, the replyChannel header must be present in the reply message. This is the way how gateway correlates requests with replies.
You have to look into an aggregator to send the request message in parallel and to preserve the mentioned TemporaryReplyChannel header. Then when you receive a reply (inboundAdapter) you send it to this aggregator. You need to ensure some correlation key from a request and reply payload, so they can match and fulfill group for reply to be sent back to the gateway.
See more info in docs: https://docs.spring.io/spring-integration/docs/current/reference/html/message-routing.html#aggregator

Async RabbitMQ communcation using Spring Integration

I have two spring boot services that communicate using RabbitMQ.
Service1 sends request for session creation to Service2.
Service2 handles request and should return response.
Service1 should handle the response.
Service1 method for requesting session:
public void startSession()
{
ListenableFuture<SessionCreationResponseDTO> sessionCreationResponse = sessionGateway.requestNewSession();
sessionCreationResponse.addCallback(response -> {
//handle success
}, ex -> {
// handle exception
});
}
On Service1 I have defined AsyncOutboundGateway, like:
#Bean
public IntegrationFlow requestSessionFlow(MessageChannel requestNewSessionChannel,
AsyncRabbitTemplate amqpTemplate,
SessionProperties sessionProperties)
{
return flow -> flow.channel(requestNewSessionChannel)
.handle(Amqp.asyncOutboundGateway(amqpTemplate)
.exchangeName(sessionProperties.getRequestSession().getExchangeName())
.routingKey(sessionProperties.getRequestSession().getRoutingKey()));
}
On Service2, I have flow for receiving these messages:
#Bean
public IntegrationFlow requestNewSessionFlow(ConnectionFactory connectionFactory,
SessionProperties sessionProperties,
MessageConverter messageConverter,
RequestNewSessionHandler requestNewSessionHandler)
{
return IntegrationFlows.from(Amqp.inboundGateway(connectionFactory,
sessionProperties.requestSessionProperties().queueName())
.handle(requestNewSessionHandler)
.get();
Service2 handles there requests:
#ServiceActivator(async = "true")
public ListenableFuture<SessionCreationResponseDTO> handleRequestNewSession()
{
SettableListenableFuture<SessionCreationResponseDTO> settableListenableFuture = new SettableListenableFuture<>();
// Goes through asynchronous process of creating session and sets value in listenable future
return settableListenableFuture;
}
Problem is that Service2 immediately returns ListenableFuture to Service1 as message payload, instead of waiting for result of future and sending back result.
If I understood documentation correctly Docs by setting async parameter in #ServiceActivator to true, successful result should be returned and in case of exception, error channel would be used.
Probably I misunderstood documentation, so that I need to unpack ListenableFuture in flow of Service2 before returning it as response, but I am not sure how to achieve that.
I tried something with publishSubscribeChannel but without much luck.
Your problem is here:
.handle(requestNewSessionHandler)
Such a configuration doesn't see your #ServiceActivator(async = "true") and uses it as a regular blocking service-activator.
Let's see if this helps you:
.handle(requestNewSessionHandler, "handleRequestNewSession", e -> e.async(true))
It is better to think about it like: or only annotation configuration. or only programmatic, via Java DSL.

How to retry an external service when a call to an internal service fails using spring cloud gateway?

I'm implementing a service that mocks another service available on the web.
I'm trying to configure the spring cloud gateway to reroute requests to the public service when a call to my implementation fails.
I tried using the Hystrix filter the following way:
spring:
cloud:
gateway:
routes:
- id: foo
uri: http://my-internal-service/
filters:
- name: Hystrix
args:
name: fallbackcmd
fallbackUri: https://the.public.service/
Unfortunately, like the documentation says:
Currently, only forward: schemed URIs are supported. If the fallback is called, the request will be forwarded to the controller matched by the URI.
Therefore, I can't use fallbackUri: https://....
Is there any plan to support this feature soon?
Otherwise, what are my options for this particular use case?
I ended up with a kind of hacky workaround that seems to work for my particular use case (i.e. a GET request):
Create my own fallback controller in the Gateway application
Configure the hystrix fallback to point to that controller
Use the WebClient to call my public service
This is what the end result looks like:
application.yml
spring:
cloud:
gateway:
default-filters:
- name: AddResponseHeader
args:
name: X-Data-Origin
value: My internal service
routes:
- id: foo
uri: http://my-internal-service/
filters:
- name: Hystrix
args:
name: local-service-fallback
fallbackUri: forward:/fallback/foo
FallbackController.java
#RestController
#RequestMapping(path = "/fallback")
public class FallbackController {
private static final String fallbackUri = "https://the.public.service";
WebClient webClient;
public FallbackController() {
webClient = WebClient.create(fallbackUri);
}
#GetMapping("/foo")
Mono<MyResponse> foo(ServerWebExchange failedExchange) {
failedExchange.getResponse().getHeaders().remove("X-Data-Origin");
failedExchange.getResponse().getHeaders().add("X-Data-Origin", "The public service");
// Now call the public service using the same GET request
UriComponents uriComponents = UriComponentsBuilder.newInstance()
.uri(URI.create(fallbackUri))
.path("/path/to/service")
.queryParams(failedExchange.getRequest().getQueryParams())
.build();
return WebClient.create(uriComponents.toUriString())
.get()
.accept(MediaType.TEXT_XML)
.exchange()
.doOnSuccess(clientResponse -> {
// Copy the headers from the public service's response back to our exchange's response
failedExchange.getResponse().getHeaders()
.addAll(clientResponse.headers().asHttpHeaders());
})
.flatMap(clientResponse -> {
log.info("Data origin: {}",
failedExchange.getResponse().getHeaders().get("X-Data-Origin"));
return clientResponse.bodyToMono(MyResponse.class);
});
}
}
I had similar problem to solve.
I added new route for fallback and it worked.
.route(p -> p .path("/fallback/foo").uri("https://example.com"))

Kafka Spring Integration: Headers not coming for kafka consumer

I am using Kafka Spring Integration for publishing and consuming messages using kafka. I see Payload is properly passed from producer to consumer, but the header information is getting overridden somewhere.
#ServiceActivator(inputChannel = "fromKafka")
public void processMessage(Message<?> message) throws InterruptedException,
ExecutionException {
try {
System.out.println("Headers :" + message.getHeaders().toString());
}
} catch (Exception e) {
e.printStackTrace();
}
}
I get following headers:
Headers :{timestamp=1440013920609, id=f8c645f7-677b-ec32-dad0-a7b79082ef81}
I am constructing the message at producer end like this:
Message<FeelDBMessage> message = MessageBuilder
.withPayload(samplePayloadObj)
.setHeader(KafkaHeaders.MESSAGE_KEY, "key")
.setHeader(KafkaHeaders.TOPIC, "sampleTopic").build();
// publish the message
publisher.publishMessage(message);
and below is the header info at producer:
headers={timestamp=1440013914085, id=c4159c1c-2c67-634b-ef8d-3fb026b1172e, kafka_messageKey=key, kafka_topic=sampleTopic}
Any idea why the Headers are overridden by a different value?
Just because by default Framework uses the immutable GenericMessage.
Any manipulation to the existing message (e.g. MessageBuilder.withPayload) will produce a new GenericMessage instance.
From other side Kafka doesn't support any headers abstraction like JMS or AMQP. That's why KafkaProducerMessageHandler just do this when it publishes a message to Kafka:
this.kafkaProducerContext.send(topic, partitionId, messageKey, message.getPayload());
As you see it doesn't send headers at all. So, other side (consumer) just deals with only message from the topic as a payload and some system options as headers like topic, partition, messageKey.
In two words: we don't transfer headers over Kafka because it doesn't support them.

Resources