Aws integration spring: Extend Visibility Timeout - spring

Is it possible to extend the visibility time out of a message that is in flight.
See:
http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/AboutVT.html.
Section: Changing a Message's Visibility Timeout.
http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/sqs/AmazonSQSClient.html#changeMessageVisibility-com.amazonaws.services.sqs.model.ChangeMessageVisibilityRequest-
In summary I want to be able to extend the first set visibility timeout for a given message that is in flight.
Example if 15secs have passed I then want to extend the timeout by another 20secs. Better example in java docs above.
From my understanding in the links above you can do this on the amazon side.
Below are my current settings;
SqsMessageDrivenChannelAdapter adapter =
new SqsMessageDrivenChannelAdapter(queue);
adapter.setMessageDeletionPolicy(SqsMessageDeletionPolicy.ON_SUCCESS);
adapter.setMaxNumberOfMessages(1);
adapter.setSendTimeout(2000);
adapter.setVisibilityTimeout(200);
adapter.setWaitTimeOut(20);
Is it possible to extend this timeout?

Spring Cloud AWS supports this starting with Version 2.0. Injecting a Visiblity parameter in your SQS listener method does the trick:
#SqsListener(value = "my-sqs-queue")
void onMessageReceived(#Payload String payload, Visibility visibility) {
...
var extension = visibility.extend(20);
...
}
Note, that extend will work asynchronously and will return a Future. So if you want to be sure further down the processing, that the visibility of the message is really extended at the AWS side of things, either block on the Future using extension.get() or query the Future with extension.isDone()

OK. Looks like I see your point.
We can change visibility for particular message using API:
AmazonSQS.changeMessageVisibility(String queueUrl, String receiptHandle, Integer visibilityTimeout)
For this purpose in downstream flow you have to get access to (inject) AmazonSQS bean and extract special headers from the Message:
#Autowired
AmazonSQS amazonSqs;
#Autowired
ResourceIdResolver resourceIdResolver;
...
MessageHeaders headers = message.getHeaders();
DestinationResolver destinationResolver = new DynamicQueueUrlDestinationResolver(this.amazonSqs, this.resourceIdResolver);
String queueUrl = destinationResolver.resolveDestination(headers.get(AwsHeaders.QUEUE));
String receiptHandle = headers.get(AwsHeaders.RECEIPT_HANDLE);
amazonSqs.changeMessageVisibility(queueUrl, receiptHandle, YOUR_DESIRED_VISIBILITY_TIMEOUT);
But eh, I agree that we should provide something on the matter as out-of-the-box feature. That may be even something similar to QueueMessageAcknowledgment as a new header. Or even just one more changeMessageVisibility method to this one.
Please, raise a GH issue for Spring Cloud AWS project on the matter with link to this SO topic.

Related

Can I store sensitive data in a Vert.x context in a Quarkus application?

I am looking for a place to store some request scoped attributes such as user id using a Quarkus request filter. I later want to retrieve these attributes in a Log handler and put them in the MDC logging context.
Is Vertx.currentContext() the right place to put such request attributes? Or can the properties I set on this context be read by other requests?
If this is not the right place to store such data, where would be the right place?
Yes ... and no :-D
Vertx.currentContext() can provide two type of objects:
root context shared between all the concurrent processing executed on this event loop (so do NOT share data)
duplicated contexts, which are local to the processing and its continuation (you can share in these)
In Quarkus 2.7.2, we have done a lot of work to improve our support of duplicated context. While before, they were only used for HTTP, they are now used for gRPC and #ConsumeEvent. Support for Kafka and AMQP is coming in Quarkus 2.8.
Also, in Quarkus 2.7.2, we introduced two new features that could be useful:
you cannot store data in a root context. We detect that for you and throw an UnsupportedOperationException. The reason is safety.
we introduced a new utility class ( io.smallrye.common.vertx.ContextLocals to access the context locals.
Here is a simple example:
AtomicInteger counter = new AtomicInteger();
public Uni<String> invoke() {
Context context = Vertx.currentContext();
ContextLocals.put("message", "hello");
ContextLocals.put("id", counter.incrementAndGet());
return invokeRemoteService()
// Switch back to our duplicated context:
.emitOn(runnable -> context.runOnContext(runnable))
.map(res -> {
// Can still access the context local data
String msg = ContextLocals.<String>get("message").orElseThrow();
Integer id = ContextLocals.<Integer>get("id").orElseThrow();
return "%s - %s - %d".formatted(res, msg, id);
});
}

Provision custom-named SQS Queue with PCF Service Broker

I'm trying to create a new queue, but when using
cf create-service aws-sqs standard my-q
the name of the queue in AWS is automatically assigned and is just an id composed of random letters and numbers.
This is fine when using the normal java client. However, we want to use spring-cloud-aws-messaging (#SqsListener annotation), because it offers us deletion policies out of the box, and a way to extend visibility, so that we can implement retries easily.
#SqsListener(value = "my-q", deletionPolicy = SqsMessageDeletionPolicy.ON_SUCCESS)
public void listen(TestItem item, Visibility visibility) {
log.info("received message: " + item);
//do business logic
//if call fails
visibility.extend(1000);
//throw exception
//if no failure, message will be dropped
}
The queue name on the annotation is declared, so we can't change it dynamically after reading the VCAP_SERVICE environment variable injected by PCF on the application.
The only alternative we can think of is use reflection to set accessibility on value of the annotation, and set the value to the name on the VCAP_SERVICE, but that's just nasty, and we'd like to avoid it if possible.
Is there any way to change the name of the queue to something specific on creation? This suggests that it's possible, as seen below:
cf create-service aws-sqs standard my-q -c '{ "CreateQueue": { "QueueName": “my-q”, "Attributes": { "MaximumMessageSize": "1024"} } }'
However, this doesn't work. It returns:
Incorrect Usage: Invalid configuration provided for -c flag. Please
provide a valid JSON object or path to a file containing a valid JSON
object.
How do I set the name on creation of the queue? Or the only way to achieve my end goal is to use reflection?
EDIT: As pointed out by Daniel Mikusa, the double quotes were not real double quotes, and that was causing the error. The command is successful now, however it doesn't create the queue with the intended name. I'm now wondering if this name needs to be set on bind-service instead. The command has a -c option too but I cannot find any documentation to support which parameters are available for a aws-sqs service.

How to create Apis on Spring, that are using up to date environment values

I want to create API, which takes part of its input from environment (urls etc..). Basically a Jar file.
I also wan't the values being auto updated when application.properties changes. When there is change, this is called:
org.springframework.cloud.endpoint.RefreshEndpoint#refresh
However, I consider it bad practice to have magic env variable keys like 'server.x.url' in the Api contract between client application and the jar. (Problem A)
That's why I'd like to use Api like this. But there's problem of old values.
public class MyC {
TheAPI theApi=null;
void MyC(){
theApi = new TheApi();
theApi.setUrl( env.get("server.x.url") );
}
doStuff() {
theApi.doStuff(); // fails as theApi has obsolete value of server.x.url, Problem B
}
So either I have ugly API contract or I get obsolete values in the API calls.
I'm sure there must be Spring way of solving this, but I cant get it to my head just now.

Spring expression language (SpEL) with jms selector

We have a requirement to segregate the incoming messages based on environment property defined in the application properties. I have come up with following configuration but that is not working, any suggestion?
#JmsListener(id = "queueListener", destination = "${request.jms.queue}", containerFactory = "requestJmsListenerContainerFactory", selector = "ENV=${jms.env}")
public void onMessage(final Message message, MessageHeaders headers)
Every message has ENV property and "jms.env" application property can have value e.g. DEV, UAT etc. We are using Spring 4 and Websphere JMS
This started working after we changed selector expression to add quotes around value part i.e. "ENV='${jms.env}'"

Is it Possible to have more than one messages file in Play framework

We have a site which will be used for two different clients. During first request the user will be asked to choose a client. Based on that text,labels and site content should be displayed.
Is it possible to have two messages file in Play framework and during session startup the messages file would be decided
As of my research we can have more than a file for each Locale, the messages will be get based on locale in the request.
No, it is not supported at the moment.
You can easily do that either in a plugin(Look at MessagesPlugin ) or even using a bootstrap job with the #onApplicationStartup annotation
// From MessagesPlugin.java
//default languange messages
VirtualFile appDM = Play.getVirtualFile("conf/messages");
if(appDM != null && appDM.exists()) {
Messages.defaults.putAll(read(appDM));
}
static Properties read(VirtualFile vf) {
if (vf != null) {
return IO.readUtf8Properties(vf.inputstream());
}
return null;
}
You can wrote you own PlayPlugin and handle implement play.PlayPlugin.getMessage(String, Object, Object...). Then you could choose the right file. The class play.i18n.Messages can be used as inspiration how to implement the method.
Solved this problem with below solution,
Created a class MessagesPlugIn which extends play.i18n.MessagesPlugin
Created a class Messages as like play.i18n.Messages
Had a static Map messaagesByClientID in Messages.java
Overridden onApplicationStart() in MessagesPlugIn
Loaded the Properties in messaagesByClientID as locales loaded in play.i18n.MessagesPlugin
Had a method get() in Messages.java, retrieve the property from messaagesByClientID based ClientId in the session. If the property is not available call get() in play.i18n.Messages
7.Created a Custom tag il8nTag and its used in HTML templates. il8nTag will invoke the methos in Messages.get().
Create your own Module based on play.api.i18n.I18nModule, but bound to your own implementation of MessagesApi, based on Play's DefaultMessagesApi (here is the part defining the files to load)
Then in your application.conf, disable Play's play.api.i18n.I18nModule and enable your own module.

Resources