I have a Spring Cloud Function running on AWS Lambda handling SNS Events.
For some error cases I would like to trigger automatic Lambda retries or triggerthe retry capabilities of the SNS Service. SNS Retry Policies are in default configuration.
I tried to return a JSON with {"statusCode":500}, which is working when we make a test invokation in the aws console.
Anyway when we send this status, no retry invokation of the Function is triggered.
We use the SpringBootRequestHandler
public class CustomerUpdatePersonHandler extends SpringBootRequestHandler<SNSEvent, Response> {
}
#Component
public class CustomerUpdatePerson implements Function<SNSEvent, Response> {
#Override
public Response apply(final SNSEvent snsEvent) {
//when something goes wrong return 500 and trigger a retry
return new Response(500)
}
}
public class Response{
private int statusCode;
public Response(int code){
this.statusCode = code;
}
public int getStatusCode(){
retrun statusCode;
}
}
We currently don't provide support for retry, but given that every function is transformed to reactive function anyway you can certainly do it yourself if you declare your function using reactor API. Basically Function<Flux<SNSEvent>, Flux<Response>> and then you can use one of the retry operations available (e.g., retryBackoff).
Related
I have an aggregation function that aggregates events published into output-channel. I have subscribed to the flux generated by the function like below:
#Component
public class EventsAggregator {
#Autowired
private Sinks.Many<Message<?>> eventsPublisher; // Used to publish events from different places in the code
private final Function<Flux<Message<?>>, Flux<Message<?>>> aggregatorFunction;
#PostConstruct
public void aggregate() {
Flux<Message<?>> output = aggregatorFunction.apply(eventsPublisher.asFlux());
output.subscribe(this::aggregate);
}
public void aggregate(Message<?> aggregatedEventsMessage) {
if (...) {
//some code
} else {
throw new RuntimeException();
}
}
}
If the RuntimeException is thrown, the aggregation function does not work, and I get this message The [bean 'outputChannel'; defined in: 'class path resource [org/springframework/cloud/fn/aggregator/AggregatorFunctionConfiguration.class]'; from source: 'org.springframework.cloud.fn.aggregator.AggregatorFunctionConfiguration.outputChannel()'] doesn't have subscribers to accept messages at org.springframework.util.Assert.state(Assert.java:97)
Is there any way to subscribe to the flux generated by the aggregation function in a safe way?
That's correct. That's how Reactive Streams work: if an exception is thrown, the subscriber is cancelled and no new data can be send to that subscriber anymore.
Consider to not throw that exception up to the stream.
See more in docs: https://docs.spring.io/spring-cloud-stream/docs/4.0.0-SNAPSHOT/reference/html/spring-cloud-stream.html#spring-cloud-stream-overview-error-handling
I consume messages from spring-cloud-stream through a Consumer<MyMessage> Implementation. As part of the message handling I need to access methods that are protected with #PreAuthorize security-checks. By default the Consumer run unauthenticated so message-handling fails.
Consumer:
#Bean
public Consumer<MyMessage> exampleMessageConsumer(MyMessageConsumer consumer) {
return consumer::handleMessage;
}
Secured Method:
#PreAuthorize("hasAuthority('ROLE_ADMIN') or hasAuthority('ROLE_USER')")
public void doSomething() { ... }
I dont just want to bypass security, so what is the easiest way to authenticate my Consumer so it passes the check?
EDIT: we are using google pubsub as a binder
For the Kafka binder:
Add an #EventListener to listen for ConsumerStartedEvents; you can then add the authentication to the security context via the SecurityContextHolder; this binds it to the thread; the same thread is used to call the listener.
I found two possible solutions to my problem
use springs RunAs support (baeldung) to add permissions to a security context for a specific method. If i do this i need to add ROLE_RUN_AS_USER to my secured methods. At scale this would complicated annotations a lot.
Manually change the security context before executing the handler method and return it to its original state afterwards.
I went with the second option. I would have liked a transparent solution but there does not appear to be one.
To make this work i created a class that wraps a functional interface with the changing code and returns it.
public class RunAs {
#FunctionalInterface
public interface RunAsMethod {
void runWithException() throws Throwable;
}
public static <T> Consumer<T> createWriteConsumer(Consumer<T> originalConsumer) {
return message -> runWithWritePermission(() -> originalConsumer.accept(message));
}
public static void runWithWritePermission(final RunAsMethod func) {
final Authentication originalAuthentication = SecurityContextHolder.getContext().getAuthentication();
final AnonymousAuthenticationToken token = new AnonymousAuthenticationToken(
"system",
originalAuthentication != null ? originalAuthentication.getPrincipal() : "system",
AuthorityUtils.createAuthorityList("ROLE_ADMIN", "SCOPE_write")
);
SecurityContextHolder.getContext().setAuthentication(token);
try {
func.runWithException();
} catch (Throwable e) {
throw new RuntimeException("exception during method with altered permissions", e);
} finally {
SecurityContextHolder.getContext().setAuthentication(originalAuthentication);
}
}
}
I need help with Java AWS-Lambda.
I am working on Lambda to update the database. There is cloudwatch event trigger and invoke Lambda for an update database based on some criteria. I want to send notification as soon as DB update. I want to invoke SNS services from Lambda.
I am using SNS account which running with multi-factor authentication. Please suggest how to create/setup lambda to send SNS.
Note: I am not looking at "how to trigger Lambda from SNS". I am looking at how to send SNS from the Lambda function IN JAVA.
This is a simple way of publishing a message to an SNS topic using aws java sdk.
To publish message with attributes use this java code sample Publish a Message with Attributes to an Amazon SNS Topic
You basically have to use the same code in your lambda handler.
you need to grant your Lambda IAM role permissions to publish to your SNS topic using appropriate IAM Policy.
// Publish a message to an Amazon SNS topic.
final String msg = "If you receive this message, publishing a message to an Amazon SNS topic works.";
final PublishRequest publishRequest = new PublishRequest(topicArn, msg);
final PublishResult publishResponse = snsClient.publish(publishRequest);
// Print the MessageId of the message.
System.out.println("MessageId: " + publishResponse.getMessageId());
This is how i have done in my project created a helperservice and used in my lambdahandler.
// my lambda handler
public ApiGatewayResponse handleRequest(ApiGatewayRequest request, Context context) {
try {
// call to the helperservice
PublishResult publishResult = helperService.publishMessage("Message to be published to sns topic");
//return response
return buildResponse(HttpStatus.SC_OK);
} catch(Exception e) {
this.logger.error(e.getMessage(), e);
return buildResponse(e.getMessage(), HttpStatus.SC_INTERNAL_SERVER_ERROR);
}
}
// HerperService class class
public class HelperService {
private final AmazonSNS snsClient;
private final Configuration config;
#Inject
public HelperService(
AmazonSNS snsClient,
Configuration config,
) {
this.snsClient = snsClient;
this.config = config;
}
public PublishResult publishMessage(String message) {
PublishRequest publishRequest = new PublishRequest(config.getAuditSNSTopicARN(), message);
return snsClient.publish(publishRequest);
}
}
I am trying to create a event based system for communicating between services using Apache Kafka as Messaging system and Spring Cloud Stream Kafka.
I have written my Receiver class methods as below,
#StreamListener(target = Sink.INPUT, condition = "headers['eventType']=='EmployeeCreatedEvent'")
public void handleEmployeeCreatedEvent(#Payload String payload) {
logger.info("Received EmployeeCreatedEvent: " + payload);
}
This method is specifically to catch for messages or events related to EmployeeCreatedEvent.
#StreamListener(target = Sink.INPUT, condition = "headers['eventType']=='EmployeeTransferredEvent'")
public void handleEmployeeTransferredEvent(#Payload String payload) {
logger.info("Received EmployeeTransferredEvent: " + payload);
}
This method is specifically to catch for messages or events related to EmployeeTransferredEvent.
#StreamListener(target = Sink.INPUT)
public void handleDefaultEvent(#Payload String payload) {
logger.info("Received payload: " + payload);
}
This is the default method.
When I run the application, I am not able to see the methods annoated with condition attribute being called. I only see the handleDefaultEvent method being called.
I am sending a message to this Receiver Application from the Sending/Source App using the below CustomMessageSource class as below,
#Component
#EnableBinding(Source.class)
public class CustomMessageSource {
#Autowired
private Source source;
public void sendMessage(String payload,String eventType) {
Message<String> myMessage = MessageBuilder.withPayload(payload)
.setHeader("eventType", eventType)
.build();
source.output().send(myMessage);
}
}
I am calling the method from my controller in Source App as below,
customMessageSource.sendMessage("Hello","EmployeeCreatedEvent");
The customMessageSource instance is autowired as below,
#Autowired
CustomMessageSource customMessageSource;
Basicaly, I would like to filter messages received by the Sink/Receiver application and handle them accordingly.
For this I have used the #StreamListener annotation with condition attribute to simulate the behaviour of handling different events.
I am using Spring Cloud Stream Chelsea.SR2 version.
Can someone help me in resolving this issue.
It seems like the headers are not propagated. Make sure you include the custom headers in spring.cloud.stream.kafka.binder.headers http://docs.spring.io/autorepo/docs/spring-cloud-stream-docs/Chelsea.SR2/reference/htmlsingle/#_kafka_binder_properties .
I am connection through SockJS over STOMP to my Spring backend. Everything work fine, the configuration works well for all browsers etc. However, I cannot find a way to send an initial message. The scenario would be as follows:
The client connects to the topic
function connect() {
var socket = new SockJS('http://localhost:8080/myEndpoint');
stompClient = Stomp.over(socket);
stompClient.connect({}, function(frame) {
setConnected(true);
console.log('Connected: ' + frame);
stompClient.subscribe('/topic/notify', function(message){
showMessage(JSON.parse(message.body).content);
});
});
}
and the backend config looks more or less like this:
#Configuration
#EnableWebSocketMessageBroker
public class WebSocketAppConfig extends AbstractWebSocketMessageBrokerConfigurer {
...
#Override
public void registerStompEndpoints(final StompEndpointRegistry registry) {
registry.addEndpoint("/myEndpoint").withSockJS();
}
I want to send to the client an automatic reply from the backend (on the connection event) so that I can already provide him with some dataset (e.g. read sth from the db) without the need for him (the client) to send a GET request (or any other). So to sum up, I just want to send him a message on the topic with the SimMessagingTemplate object just after he connected.
Usually I do it the following way, e.g. in a REST controller, when the template is already autowired:
#Autowired
private SimpMessagingTemplate template;
...
template.convertAndSend(TOPIC, new Message("it works!"));
How to achieve this on connect event?
UPDATE
I have managed to make it work. However, I am still a bit confused with the configuration. I will show here 2 configurations how the initial message can be sent:
1) First solution
JS part
stompClient.subscribe('/app/pending', function(message){
showMessage(JSON.parse(message.body).content);
});
stompClient.subscribe('/topic/incoming', function(message){
showMessage(JSON.parse(message.body).content);
});
Java part
#Controller
public class WebSocketBusController {
#SubscribeMapping("/pending")
Configuration
#Override
public void configureMessageBroker(final MessageBrokerRegistry config) {
config.enableSimpleBroker("/topic");
config.setApplicationDestinationPrefixes("/app");
}
... and other calls
template.convertAndSend("/topic/incoming", outgoingMessage);
2) Second solution
JS part
stompClient.subscribe('/topic/incoming', function(message){
showMessage(JSON.parse(message.body).content);
})
Java part
#Controller
public class WebSocketBusController {
#SubscribeMapping("/topic/incoming")
Configuration
#Override
public void configureMessageBroker(final MessageBrokerRegistry config) {
config.enableSimpleBroker("/topic");
// NO APPLICATION PREFIX HERE
}
... and other calls
template.convertAndSend("/topic/incoming", outgoingMessage);
SUMMARY:
The first case uses two subscriptions - this I wanted to avoid and thought this can be managed with one only.
The second one however has no prefix for application. But at least I can have a single subscription to listen on the provided topic as well as send initial message.
If you just want to send a message to the client upon connection, use an appropriate ApplicationListener:
#Component
public class StompConnectedEvent implements ApplicationListener<SessionConnectedEvent> {
private static final Logger log = Logger.getLogger(StompConnectedEvent.class);
#Autowired
private Controller controller;
#Override
public void onApplicationEvent(SessionConnectedEvent event) {
log.debug("Client connected.");
// you can use a controller to send your msg here
}
}
You can't do that on connect, however the #SubscribeMapping does the stuff in that case.
You just need to mark the service method with that annotation and it returns a result to the subscribe function.
From Spring Reference Manual:
An #SubscribeMapping annotation can also be used to map subscription requests to #Controller methods. It is supported on the method level, but can also be combined with a type level #MessageMapping annotation that expresses shared mappings across all message handling methods within the same controller.
By default the return value from an #SubscribeMapping method is sent as a message directly back to the connected client and does not pass through the broker. This is useful for implementing request-reply message interactions; for example, to fetch application data when the application UI is being initialized. Or alternatively an #SubscribeMapping method can be annotated with #SendTo in which case the resulting message is sent to the "brokerChannel" using the specified target destination.
UPDATE
Referring to this example: https://github.com/revelfire/spring4Test how would that be possible to send anything when the line 24 of the index.html is invoked: stompClient.subscribe('/user/queue/socket/responses' ... from the spring controllers?
Well, look like this:
#SubscribeMapping("/queue/socket/responses")
public List<Employee> list() {
return getEmployees();
}
The Stomp client part remains the same.