We are trying to use spring-cloud-aws to receive messages from AWS SQS
We would like to receive messages using annotation. In spring documentation, it is confusing.
Below, they stated to use MessageMapping and QueueMessageHandler annotation.
Annotation-driven listener endpoints are the easiest way for listening
on SQS messages. Simply annotate methods with MessageMapping and the
QueueMessageHandler will route the messages to the annotated methods.
But in the sample, #SQSListener is used.
#SqsListener("queueName")
public void queueListener(Person person) {
// ...
}
I searched for #SqsListener and found that it is being used in test classes like here . So we tried to import, org.springframework.cloud.aws.messaging.listener.annotation.SqsListener. Unfortunately this annotation class is not available in latest release.
Is the org.springframework.cloud.aws.messaging.listener.annotation.SqsListener that I am using is proper one? Or it is not yet present in released version? If not released can I use #MessageMapping to receive messages from SQS?
It appears to not be included in the 1.0.4 release of Spring Cloud AWS however I was able to successfully import SqsListener when using 1.1.0.RC1
You need to add:
dependencyManagement {
imports {
mavenBom 'org.springframework.cloud:spring-cloud-aws:1.1.0.RC1'
mavenBom "org.springframework.boot:spring-boot-starter-parent:1.3.3.RELEASE"
}
Additionally the messaging dependency needs to be added (and I've got actuator included too):
dependencies {
compile("org.springframework.cloud:spring-cloud-starter-aws")
compile("org.springframework.cloud:spring-cloud-aws-messaging")
compile("org.springframework.boot:spring-boot-starter-actuator")
}
Note, I haven't tested it to see if it can actually consume a message of SQS but at least the dependency is resolving.
I am using the 1.1.0.RELEASE, that's the dependencies I have:
compile("org.springframework.boot:spring-boot-starter:1.3.5.RELEASE")
compile("org.springframework.cloud:spring-cloud-starter-aws-messaging:1.1.0.RELEASE")
I tried both annotations #SqsListener and #MessageMapping both work fine. The SqsListener is a specialization of the MessageMapping annotation which is adding an additional property, the deletion policy.
I am guessing the documentation has to be updated, I got confused as well.
Now #SqsListener available with 1.1.0.RELEASE.
Related
This question asks for some specifics about more general topic regarding modularization of bean validation I asked before.
In question linked above, following this documentation and this post I split annotation and ConstraintValidator definition into 2 java modules, and linked them together using ServiceLoader as shown in documentation here. Works, mostly. But there is one unsolved issue, that it does not work for validation defined via XML, which I did according to documentation again. What does not work: The pairing between annotation and ConstraintValidator is not set, the service loader stuff is not used at all.
To recap: I have working setup using this ServiceLoader approach and it works when validating stuff coming through rest layer. All paired correctly.
BUT! We are getting these DTOs also through kafka. And here we have two different flows. There is some initialization of common ConstraintValidators on startup, and then:
if we first get REST message, ServiceLoader stuff is discovered only at this request time, some next initialization is done seemignly, and after that even kafka messages works, meaning pairing for custom validator is available everywhere. (Great!)
if kafka message arrives first though(typical), no service loader stuff is consulted and somehow it 'destroys' the configuration in way, that even if later rest request comes it won't work either, saying, that there is no ConstraintValidator for given annotation. The initialization is completed somehow defectively.
validation.xml is as easy as:
<validation-config
xmlns="http://xmlns.jcp.org/xml/ns/validation/configuration"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/validation/configuration validation-configuration-2.0.xsd"
version="2.0">
<constraint-mapping>/META-INF/validation-constraints.xml</constraint-mapping>
</validation-config>
notes:
2.0 version is because of hibernate-validator 6.2.0 which comes from spring dependency management.
Why not use annotation and dump this xml stuff altogether? Not mine file, unmodifiable.
If there is some trivial newbie mistake, please advise. Maybe there is some way how to kick in service loader functionality into action in validation.xml file, I'm not aware of and cannot find anywhere.
EDITS/suggestions:
A: try to inject validator on startup to make sure it's loaded:
#Autowired
private Validator validator;
#EventListener(ApplicationReadyEvent.class)
public void logReady() {
System.out.println(validator.toString());
}
did print initialized validator, did not help though.
We use Spring Kafka together with Spring Boot (all latest versions). We switched handling of Kafka messages into #KafkaHandler annotated methods and expected that #Valid/#Validated together with #Payload will ensure payload validation, but that did not happen. This feature is working for #KafkaListener, should it be also working for #KafkaHandler?
#KafkaListener(...)
#Component
public class NotificationListener {
#KafkaHandler
public void handleV1(#Payload #Valid NotificationV1 notification) {
Thank you.
The Validator is not applied in this case because we just don't reach the PayloadMethodArgumentResolver for that purpose.
The target payload for the multi-method #KafkaListener is resolved before we call the method because we definitely need to know which method to call. Such a logic is done in the InvocableHandlerMethod.getMethodArgumentValues():
args[i] = findProvidedArgument(parameter, providedArgs);
if (args[i] != null) {
continue;
}
...
try {
args[i] = this.resolvers.resolveArgument(parameter, message);
}
The Validator functionality is done in those resolvers. The findProvidedArgument() gives us the payload converted before for execution and here we just don't check any annotations on parameters.
We probably need to poll validation logic into the DelegatingInvocableHandler when we have selected a handler and before its invocation...
Feel free to raise a GitHub issue so we don't forget that this is needed to be addressed somehow.
The reason that why #KafkaListener works with #Valid annotation is that it just like a restful controller endpoint, which is the entrance of the service. The team works to add support for validation working on these situations, and it can be found that this validation mechanism is added in 2018.
As for #KafkaHandler, I'm not that familiar with spring-kafka, but if the validation just not work, it just means that the team doesn't add support for this situation. I recommend you to use the Spring Boot Method Validation Feature, which works fine for all spring managed beans and all the standard validation annotations such as #Size. One last thing, be careful about the exception thrown while validation fails.
I would like to disable the rabbit health check in my default RabbitMockConfiguration.
We have a Configuration that is imported via #Import. Unfortunately the Configuration does not prevent the health check from being added to the health indicator as that happens once spring-rabbit is in the classpath.
We have the workaround, that we add a properties file in every service using that Configuration, which disables the property management.health.rabbit.enabled, but for us it would be much nicer to be able to disable that heathcheck on configuration level.
I thought about the tests with #TestPropertySource(properties = ["management.health.rabbit.enabled=false"]), but I could not find an equivalent to use for the a #Configuration, as #PropertySource expects a location for a properties file and does not accept single properties.
Any idea what we can do?
Spring boot version: 2.2.4
Spring amqp version: 2.2.3
Spring Version: 5.2.3
If you want to change the behaviour of the health check, I'd rather override the health check so that it states Rabbit is in mock mode.
To do so, just create a HealthIndicator bean named rabbitHealthIndicator:
#Bean
public HealthIndicator rabbitHealthIndicator() {
return () -> Health.up().withDetail("version", "mock").build();
}
This has the effect of switching the production one and exposes the fact the app is running with a mock.
I guess you should add 'ApplicationListener' and add the implementation to 'src/main/resources/META-INF/spring.factories' to your module with MockReddisConfiguration. This is described in more detail here
When using a XStreamMarshaller with spring batch, I get the following message:
Security framework of XStream not initialized, XStream is probably vulnerable.
First try: According to the documentation, I've tried to reset all permissions, but I still have the same message. Besides, I have no security error when parsing XML files... So I think that this code just doen't work. Here's a sample of code:
XStreamMarshaller marshaller = new XStreamMarshaller();
marshaller.getXStream().addPermission(NoTypePermission.NONE);
Second try: I have also tried with the setSupportedClasses method, but it doesn't work either (I still get the vulnerability message and not supported classes are still unmarshelled correctly):
XStreamMarshaller marshaller = new XStreamMarshaller();
marshaller.setSupportedClasses(FooBar.class);
How can I set security permissions with XStreamMarshaller?
Note: according to this thread, the Security Framework was introduced with 1.4.7 and it is still not mandatory.... But it will be mandatory for XStream 1.5.0!
Version of XStream used: 1.4.10
Version of Spring Batch used: 4.0.1
For information, I'm using Spring Boot (but I'm not sure it's relevant here)
Solution for the 'First Try':
The reason why it didn't work is that XStreamMarshaller instantiates a xstream object with afterPropertiesSet without checking if one have already been created, so we can't use getXStream() in a #Bean method. To make this work, we can for example set security config while injecting the marshaller in another bean:
#Configuration
public class JobSecurityConfig {
public JobSecurityConfig(XStreamMarshaller marshaller) {
XStream xstream = marshaller.getXStream();
XStream.setupDefaultSecurity(xstream);
xstream.allowTypes(new Class[]{Bar.class});
}
}
Another solution: extend XSreamMarshaller
You can also extend XStreamMarshaller and override only the customizeXStream() method to set security configuration.
#Override
protected void customizeXStream(XStream xstream) {
XStream.setupDefaultSecurity(xstream);
xstream.allowTypes(new Class[]{Bar.class});
}
Why the 'Second Try' doesn't work:
setSupportedClasses is only used on marshalling!!.. StaxEventItemReader doesn't care about supported classes!
Xstream website have provided details about the Security Framework Security Framework.
below method are provided to set Security permissions
XStream.addPermission(TypePermission);
XStream.allowTypes(Class[]);
XStream.allowTypes(String[]);
XStream.allowTypesByRegExp(String[]);
XStream.allowTypesByRegExp(Pattern[]);
XStream.allowTypesByWildcard(String[]);
XStream.allowTypeHierary(Class);
XStream.denyPermission(TypePermission);
XStream.denyTypes(Class[]);
XStream.denyTypes(String[]);
XStream.denyTypesByRegExp(String[]);
XStream.denyTypesByRegExp(Pattern[]);
XStream.denyTypesByWildcard(String[]);
XStream.denyTypeHierary(Class);
You can also refer this Tutorial
I hope this helps
From the official spring docs:
By default, XStream allows for arbitrary classes to be unmarshalled,
which can lead to unsafe Java serialization effects. As such, it is
not recommended to use the XStreamMarshaller to unmarshal XML from
external sources (i.e. the Web), as this can result in security
vulnerabilities.
You're using Spring's abstraction XStreamMarshaller to interface with the XStream library. By default the library can marshall/unmarshall arbitrary classes (including from external web source).
If you are not doing that (working with classes from external web sources) you can simply ignore the message.
If you want to remove the message follow what's recommended in Spring's official doc (linked above) and XStream website (security config example).
It boils down to setting up supported classes to make sure only the registered classes are eligible for unmarshalling.
This property is empty by default, which means - support all classes - hence the warning message you're getting.
I am trying to integrate the Brave MySql Instrumentation into my Spring Boot 2.x service to automatically let its interceptor enrich my traces with spans concerning MySql-Queries.
The current Gradle-Dependencies are the following
compile 'io.zipkin.zipkin2:zipkin:2.4.5'
compile('io.zipkin.reporter2:zipkin-sender-okhttp3:2.3.1')
compile('io.zipkin.brave:brave-instrumentation-mysql:4.14.3')
compile('org.springframework.cloud:spring-cloud-starter-zipkin:2.0.0.M5')
I already configured Sleuth successfully to send traces concerning HTTP-Request to my Zipkin-Server and now I wanted to add some spans for each MySql-Query the service does.
The TracingConfiguration it this:
#Configuration
public class TracingConfiguration {
/** Configuration for how to send spans to Zipkin */
#Bean
Sender sender() {
return OkHttpSender.create("https://myzipkinserver.com/api/v2/spans");
}
/** Configuration for how to buffer spans into messages for Zipkin */
#Bean AsyncReporter<Span> spanReporter() {
return AsyncReporter.create(sender());
}
#Bean Tracing tracing(Reporter<Span> spanListener) {
return Tracing.newBuilder()
.spanReporter(spanReporter())
.build();
}
}
The Query-Interceptor works properly, but my problem now is that the spans are not added to the existing trace but each are added to a new one.
I guess its because of the creation of a new sender/reporter in the configuration, but I have not been able to reuse the existing one created by the Spring Boot Autoconfiguration.
That would moreover remove the necessity to redundantly define the Zipkin-Url (because it is already defined for Zipkin in my application.yml).
I already tried autowiring the Zipkin-Reporter to my Bean, but all I got is a SpanReporter - but the Brave-Tracer-Builder requries a Reporter<Span>
Do you have any advice for me how to properly wire things up?
Please use latest snapshots. Sleuth in latest snapshots uses brave internally so integration will be extremely simple.