private Flux<Record> consumeRecord() {
return reactiveKafkaConsumerTemplate
.receive()
.doOnNext(consumerRecord -> {
Record record = consumerRecord.value();
recordWorkflowService.handleRecord(record);
}
)
.map(ConsumerRecord::value)
.doOnError(throwable -> {
log.error("something bad happened while consuming : {}", throwable.getMessage());
});
}
Currently this is the code I have in my consumer. When a record comes in I do see that my recordWorflowService.handleRecord is called and the record is processed successfully, however I cannot get the error case to trigger.
I have a use case where I am consumer records from a kafka topic and do some processing on them. However, if any part of that processing fails I do not want the kafka record to be committed so that it can get reprocessed. So if any error occurs in the recordWorkflowService I want .doOnError() to be triggered and to not commit the offset (So it can be reprocessed).
Am I on the right path here? I have tried manually throwing an exception within handleRecord() but .doOnError() never seems to get triggered.
So I've been trying to use Spring Cloud Contract to test RabbitListener.
So far I have found out that by defining "triggeredBy" in contract, the generated test will call the method provided there and so we will need to provide the actual implementation of what that method do in the TestBase.
Another thing is "outputMessage", where we can verify whether the method call before have correctly resulting on some message body sent to certain exchange.
Source: documentation and sample
My question is, is there any way to produce the input message from the contract, instead of triggering own custom method?
Perhaps something similar like Spring Integration or Spring Cloud Stream example in the documentation:
Contract.make {
name("Book Success")
label("book_success")
input {
messageFrom 'input.exchange.and.maybe.route'
messageHeaders {
header('contentType': 'application/json')
header('otherMessageHeader': '1')
}
messageBody ([
bookData: someData
])
}
outputMessage {
sentTo 'output.exchange.and.maybe.route'
headers {
header('contentType': 'application/json')
header('otherMessageHeader': '2')
}
body([
bookResult: true
])
}
}
I couldn't find any examples in their sample project that show how to do this.
Having used spring cloud contract to document and test rest api services, if possible I would like to stay consistent by defining both the input and expected output in contract files for event based services.
Never mind, actually its already supported.
For unknown reason the documentation in "Stub Runner Spring AMQP" does not list the scenario like others previous sample.
Here is how I make it works:
Contract.make {
name("Amqp Contract")
label("amqp_contract")
input {
messageFrom 'my.exchange'
messageHeaders {
header('contentType': 'text/plain')
header('amqp_receivedRoutingKey' : 'my.routing.key')
}
messageBody(file('request.json'))
}
outputMessage {
sentTo 'your.exchange'
headers {
header('contentType': 'text/plain')
header('amqp_receivedRoutingKey' : 'your.routing.key')
}
body(file('response.json'))
}
}
This will create a test that will call your listener based on "my.exchange" and "my.routing.key" triggering the handler method.
It will then capture the message and routing key on your RabbitTemplate call to "your.exchange".
verify(this.rabbitTemplate, atLeastOnce()).send(eq(destination), routingKeyCaptor.capture(),
messageCaptor.capture(), any(CorrelationData.class));
Both message and routing key then will be asserted.
I am using Spring Cloud SQS messaging for listening to a specified queue. Hence using #SqsListener annotation as below:
#SqsListener(value = "${QUEUE}", deletionPolicy = SqsMessageDeletionPolicy.ALWAYS )
public void receive(#Headers Map<String, String> header, #Payload String message) {
try {
logger.logInfo("Message payload is: "+message);
logger.logInfo("Header from SQS is: "+header);
if(<Some condition>){
//Dequeue the message once message is processed successfully
awsSQSAsync.deleteMessage(header.get(LOOKUP_DESTINATION), header.get(RECEIPT_HANDLE));
}else{
logger.logInfo("Message with header: " + header + " FAILED to process");
logger.logError(FLEX_TH_SQS001);
}
} catch (Exception e) {
logger.logError(FLEX_TH_SQS001, e);
}
}
I am able to connect the specified queue successfully and read the message as well. I am setting a message attribute as "Key1" = "Value1" along with message in aws console before sending the message. Following is the message body:
{
"service": "ecsservice"
}
I am expecting "header" to receive a Map of all the message attributes along with the one i.e. Key1 and Value1. But what I am receiving is:
{service=ecsservice} as the populated map.
That means payload/body of message is coming as part of header, although body is coming correctly.
I wonder what mistake I am doing due to which #Header header is not getting correct message attributes.
Seeking expert advice.
-PC
I faced the same issue in one of my spring projects.
The issue for me was, SQS configuration of QueueMessageHandlerFactory with Setting setArgumentResolvers.
By default, the first argument resolver in spring is PayloadArgumentResolver.
with following behavior
#Override
public boolean supportsParameter(MethodParameter parameter) {
return (parameter.hasParameterAnnotation(Payload.class) || this.useDefaultResolution);
}
Here, this.useDefaultResolution is by default set to true – which means any parameter can be converted to Payload.
And Spring tries to match your method actual parameters with one of the resolvers, (first is PayloadArgumentResolver) - Indeed it will try to convert all the parameters to Payload.
Source code from Spring:
#Nullable
private HandlerMethodArgumentResolver getArgumentResolver(MethodParameter parameter) {
HandlerMethodArgumentResolver result = this.argumentResolverCache.get(parameter);
if (result == null) {
for (HandlerMethodArgumentResolver resolver : this.argumentResolvers) {
if (resolver.supportsParameter(parameter)) {
result = resolver;
this.argumentResolverCache.put(parameter, result);
break;
}
}
}
return result;
}
How I solved this,
The overriding default behavior of Spring resolver
factory.setArgumentResolvers(
listOf(
new PayloadArgumentResolver(converter, null, false),
new HeaderMethodArgumentResolver(null, null)
)
)
Where I set, default flag to false and spring will try to convert to payload only if there is annotation on parameter.
Hope this will help.
Apart from #SqsListener, you need to add #MessageMapping to the method. This annotation will helps to resolve method arguments.
I had this issue working out of a rather large codebase. It turned out that a HandlerMethodArgumentResolver was being added to the list of resolvers that are used to basically parse the message into the parameters. In my case it was the PayloadArgumentResolver, which usually always resolves an argument to be the payload regardless of the annotation. It seems by default it's supposed to come last in the list but because of the code I didn't know about, it ended up being added to the front.
Anyway, if you're not sure take a look around your code and see if you're doing anything regarding spring's QueueMessageHandler or HandlerMethodArgumentResolver.
It helped me to use a debugger and look at HandlerMethodArgumentResolver.resolveArgument method to start tracing what happens.
P.S. I think your #SqsListener code looks fine except that I think #Headers is supposed to technically resolve to a Map of < String, Object >", but I'm not sure that would cause the issue you're seeing.
I have my service declared this way:
public interface BlogQueryService extends Service {
public ServiceCall<String, Source<String, ?>> tick(int interval);
public ServiceCall<String, Source<String, ?>> tock();
public ServiceCall<NotUsed, Source<PostSummary, ?>> newPosts();
public ServiceCall<String, Source<PostSummary, ?>> getPostSummaries();
#Override
default Descriptor descriptor() {
return named("blog-query").with(
//pathCall("/api/bloggie/tick/:interval", this::tick),
pathCall("/api/bloggie/tock", tock())
//pathCall("/api/bloggie/newPosts", this::newPosts),
//pathCall("/api/bloggie/postSummaries", this::getPostSummaries)
).withAutoAcl(true);
}
}
The tick works. The tock doesn't.
When I invoke it using websocket client (to ws://localhost:9000/api/bloggie/tock ) , I got "undefined" as response, indicating no mapping found for that URL.
After some experimentings, found out why: tick works because it has url param (the :interval). Tick doesn't work because it doesn't have url param. Seriously pathCall requires you to have param in your URL? So I checked the API of Service: http://www.lagomframework.com/documentation/1.0.x/api/java/com/lightbend/lagom/javadsl/api/Service.html
There are several overloaded declarations of pathCall. Apparently the tick uses this one:
static <Request,Response,A> Descriptor.Call<Request,Response> pathCall(String pathPattern, akka.japi.function.Function<A,ServiceCall<Request,Response>> methodRef)
So from the signature, yes it requires the method to take a parameter. So, if the method (such is tock) doesn't take a param, the binding will fail at runtime. So I guess I need to use this one instead:
static <Request,Response> Descriptor.Call<Request,Response> pathCall(String pathPattern, akka.japi.function.Creator<ServiceCall<Request,Response>> methodRef)
The problem is... I don't know how. I haven't seen any example of the use of akka.japi.function.Creator in pathCall.
I tried this:
default Descriptor descriptor() {
return named("blog-query").with(
pathCall("/api/bloggie/tick/:interval", this::tick),
pathCall("/api/bloggie/tock", new Creator<ServiceCall<String, Source<String, ?>>> () {
public ServiceCall<String, Source<String, ?>> create() {
return tock();
}
})
//pathCall("/api/bloggie/newPosts", this::newPosts),
//pathCall("/api/bloggie/postSummaries", this::getPostSummaries)
).withAutoAcl(true);
}
It compiles. But it throws an error at runtime:
com.google.inject.CreationException: Unable to create injector, see the following errors:
1) Error in custom provider, java.lang.IllegalStateException: Unable to resolve method for service call with ID PathCallId{pathPattern='/api/bloggie/tock'}. Ensure that the you have passed a method reference (ie, this::someMethod). Passing anything else, for example lambdas, anonymous classes or actual implementation classes, is forbidden in declaring a service descriptor.
at com.lightbend.lagom.javadsl.server.ServiceGuiceSupport.bindServices(ServiceGuiceSupport.java:43) (via modules: com.google.inject.util.Modules$OverrideModule -> sample.bloggie.impl.BlogServiceModule)
while locating com.lightbend.lagom.internal.server.ResolvedServices
Thanks in advance!
I just did some experiments... All compiled, but none of them works....
namedCall("/api/bloggie/tock", this::tock)
Result: Compile success. Runtime: path unknown (no binding (?)).
Then I tried
pathCall("/api/bloggie/tock", () -> this.tock())
Result: exception.
com.google.inject.CreationException: Unable to create injector, see the following errors:
1) Error in custom provider, scala.MatchError: Request (of class sun.reflect.generics.reflectiveObjects.TypeVariableImpl)
at com.lightbend.lagom.javadsl.server.ServiceGuiceSupport.bindServices(ServiceGuiceSupport.java:43) (via modules: com.google.inject.util.Modules$OverrideModule -> sample.bloggie.impl.BlogServiceModule)
while locating com.lightbend.lagom.internal.server.ResolvedServices
for parameter 1 at com.lightbend.lagom.internal.server.ServiceRegistrationModule$RegisterWithServiceRegistry.<init>(ServiceRegistrationModule.scala:55)
at com.lightbend.lagom.internal.server.ServiceRegistrationModule.bindings(ServiceRegistrationModule.scala:29):
Binding(class com.lightbend.lagom.internal.server.ServiceRegistrationModule$RegisterWithServiceRegistry to self eagerly) (via modules: com.google.inject.util.Modules$OverrideModule -> play.api.inject.guice.GuiceableModuleConversions$$anon$1)
while locating com.lightbend.lagom.internal.server.ServiceRegistrationModule$RegisterWithServiceRegistry
Then I tried:
public ServiceCall<NotUsed, Source<String, ?>> tock(Void x);
Result: exception
com.google.inject.CreationException: Unable to create injector, see the following errors:
1) Error in custom provider, java.lang.IllegalArgumentException: Don't know how to serialize ID class java.lang.Void
at com.lightbend.lagom.javadsl.server.ServiceGuiceSupport.bindServices(ServiceGuiceSupport.java:43) (via modules: com.google.inject.util.Modules$OverrideModule -> sample.bloggie.impl.BlogServiceModule)
Update: "Solved" (partially). Figured out that this one works:
pathCall("/tock", this::tock)
I can open it using this URL: ws://localhost:9000/tock
So..., I can't have nicely structured URL for those functions that returns stream, when those functions need no param? At least for now (?).
UPDATE: seems like this problem is happening not only with pathCall. I encountered the same problem with rest call. This one doesn't work (no binding):
public ServiceCall<NotUsed, PSequence<PostSummary>> getPostSummaries();
...
restCall(Method.GET, "/api/bloggie/postSummaries", this::getPostSummaries)
This one works:
public ServiceCall<NotUsed, PSequence<PostSummary>> getPostSummaries();
...
restCall(Method.GET, "/postSummaries", this::getPostSummaries)
Thanks!
So firstly, namedCall should only be used if you don't care about the path. You are invoking the service call directly, which means you do care about the path, so you have to use pathCall or restCall.
This should work:
pathCall("/api/bloggie/tock", this::tock)
Also, I think you're not pasting the full errors. Make sure you check right to the bottom of the list of Guice errors, that should explain exactly what the problem is, in many of the cases above, the problem is that you're not passing a method reference, you're passing a lambda, and the error message should say that.
I have a channel name in a header of the message. I want to send the message (with all the headers on the message) to that channel. I am trying to define an IntegrationFlow for the same. For example, the name of channel I want to submit the message to is set in the channelName header on the message and my flow definition is as follows:
public IntegrationFlow someFlowDefinition(){
return IntegrationFlows.from("channelA")
.channel("headers['channelName']")
.get();
}
The problem here is .channel() does not understand spel expressions. So it thinks the name of the channel is "headers['channelName']"
I also tried to use the SpelExpressionParser, but couldnt get set the right context (shown as ??? below).
public IntegrationFlow someFlowDefinition(){
return IntegrationFlows.from("channelA")
.channel(new SpelExpressionParser().parseExpression("headers['channelName']").getValue(???))
.get();
}
As I understand, I need to set the Message as the context, but not sure how to get a reference to the Message in the flow definition
Any pointers?
.channel() is static; you need a Router.
Use .route("headers['channelName']").