If the message payload passed from one "filter" to the next "filter" in Spring XD stream is a custom Java class instance, I suppose some kind of serialization mechanism is required if the "pipe" in between is a remote transport.
What kind of "serialization"/"transformation" is available in Spring XD for this case?
Will Java serialization work for this case? And if the custom class is serializable, will Spring XD automatically serialise/deserialize the object, or we still need to give some hints in the stream definition/module definition?
Thanks,
Simon
XD uses Kryo serialization with remote transports. Java.io.serialization would work in theory, however we don't want to assume that payload types implement java.io.Serializable. Also, I personally don't see any advantage in choosing Java serialization automatically over Kryo if the payload is Serializable. Java serialization is supported via Spring XD's type conversion.
You should be able to create a stream containing something like:
filter1 --outputType=--application/x-java-serialized-object | filter2 --input-type=my.custom.SerializablePayloadType
In this case, the type conversion will use Java serialization before hitting the transport. The message bus will detect that the payload is a byte array and will send it directly to the next module as is. The message containing the bytes will set the content-type header to the declared outputType so that it can be deserialized using Java serialization by the inbound converter.
The above requires that the payload implements Serializable. Also custom payload types must be included in Spring XD's class path, i.e., add a jar to xd/lib on each installed container.
Related
I think I'm not grasping some basic concepts of Kafka here, so I'm hoping Stack maybe able to the help.
I've been trying to learn Kafka with Spring boot by following this GIT repo here:
I understand how to without avro take a Java class from one Microservice, send it to Kafka and consume / serialise it on another Microservice...however I hate that idea. As it means I must have an identical class on the other Microservice in terms of package location / name etc
So overall I've two questions here I guess.
I want to understand how I can share message across my spring boot microservices and map them to classes without copying said classes from one service to the other
I want to be able to consume from my Spring Kafka listeners messages created from another language say C#
Where I'm currently at is, I have the avro example from the repo above up and running along with my local kafka and Schema registry instance.
However if I create a duplicate class and call it UserTest (For example) and have it identical to the User class consumed here I get stacktraces like the following:
Caused by: org.springframework.messaging.converter.MessageConversionException: Cannot convert from [io.confluent.developer.User] to [io.confluent.developer.kafkaworkshop.streams.User] for GenericMessage [payload={"name": "vik", "age": 33}, headers={kafka_offset=6, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer#54200a0e, kafka_timestampType=CREATE_TIME, kafka_receivedMessageKey=vik, kafka_receivedPartitionId=1, kafka_receivedTopic=users12, kafka_receivedTimestamp=1611278455840, kafka_groupId=simple-consumer}]
Am I missing something exceptionally basic here? I thought that once the message was send in Avro format that it could be consume and mapped to another object which had the same fields...that way if the object was created in c#, the spring service would be able to interpret it no?
If anyone can help me that would be great....
Thanks!
I have 2 spring boot micro-services core and web:
The core service reacts to some event (EmployeeCreatedEvent) which is triggered by web.
The core service is using jackson serializer to serialize commands, queries, events and messages whereas the web service is using xstream serializer.
i am getting below error in core while handling EmployeeCreatedEvent triggered by web:
Caused by: com.fasterxml.jackson.core.JsonParseException: Unexpected character (’<’ (code 60)):
expected a valid value (JSON String, Number, Array, Object or token ‘null’, ‘true’ or ‘false’)
i am using below properties (jackson for core and default for web):
axon.serializer.general = jackson/default
axon.serializer.events = jackson/default
axon.serializer.messages = jackson/default
can someone suggest whether it is ok to use different serializer for same event in different services.
I agree with #Augusto here and you should make a decision about which serialization format you are going to use across all your services.
I am assuming you started with the default serializer (which is XStream and XML) and later on decided to move to Jackson (which is JSON).
In that case, there are 2 advices I can share with you:
You can write a Custom Serializer which have both implementations and try with both of them and see which one works, for example trying with XML and fallbacking to JSON.
Or you can have a Component which will listen to all Events from your EventStore, deserialize them using XStream and write them back to another EventStore using Jackson. In this case, for this migration period, you will have this component using 2 Event Streams (one for each EventStore) but after the migration is done your whole EventStore will be in JSON. This requires some work but is the best approach in my opinion and will save you a lot of time and pain in the future.
You can look more about configuring 2 sources here.
I'm using Java 8 Date/Time APIs in my DTO objects sent to and from clients to the server. I'm using Spring Boot 1.3.2 and its default Jackson 2. I added the com.fasterxml.jackson.datatype:jackson-datatype-jsr310 module as a dependency. Still, the Instant instances are not serialized using the InstantSerializer, they are written as a nested JSON structure with two fields, epochSeconds and nano. As these two methods are discoverable as getters in the Instant class, I guess the default bean serializer takes care of this.
What I would like is to have Instant serialized as longs, just as a Date would have been. I'm not doing any extra customization of the ObjectMapper, so I kind of expected this to work without much fuss.
We currently use Spring's PayloadRootAnnotationMethodEndpointMapping to map incoming messages to the appropriate Java endpoints. We're going to have a new version of WSDL (with enhanced business functionality) to be made available to the user community while we're also required to continue to support the existing version of WSDL for backward compatibility.
The versioning information will be embedded in the namespaces' URNs, for examples:
urn:mycompany:myproject:mymodule:messages:1.0urn:mycompany:myproject:mymodule:messages:1.1
Since there's only a small fraction of Java methods that have changed between the old version and the new version, I was wondering what would be the best way to handle those methods that have NOT changed between the two versions in terms of endpoint mapping. In other words, how can I route the incoming messages of both versions to the same Java endpoint method?
One option I was thinking of was to write a custom Spring-ws endpoint mapping class (possibly by extending PayloadRootAnnotationMethodEndpointMapping class. But before I write any code, I'd like to check with you guys to see:
1) Are there some best practices with respect to supporting multiple versions of WSDLs by a single server side implementation?
2) Does Spring-ws have any out-of-box solutions for this type of the situations?
Thanks,
As of Spring-WS 2.2, there is a #PayloadRoots annotation which allows you to map multiple payloads to one method, like so:
#PayloadRoots({
#PayloadRoot(localPart = "Request1", namespace = "http://springframework.org/spring-ws"),
#PayloadRoot(localPart = "Request2", namespace = "http://springframework.org/spring-ws")
})
public void doIt(#RequestPayload Source payload) {
...
}
I'd also like to point you to the PayloadTransformingInterceptor, which transforms the payload of the SOAP message using XSLT stylesheet. Depending on the differences between the two versions of the WSDL, you could transform the "old" requests to the new format with one XSLT, thus letting them be handled by the "new" endpoint. In turn, the "new" responses can be transformed to the old format again with another XSLT.
I am asking because I have only seen java beans used with a framework like struts or JSF.
Is it possible to send and access a java bean over an AJAX request?
Servlet myServlet creates and fills a java bean instance, and sets it in the request scope. An already loaded jsp/html page uses AJAX to request data from myServlet. Can this bean be accessed in any way? After thinking for a while, I have come to accept that this cannot be done.
If it can't be done, what would be the best practice when trying to transmit data from a model (i.e. user information from a database) asynchronously to a client when using Tomcat/Servlets and JSP?
It's technically possible if you serialize the javabean to a byte array or even a base64 encoded string using the usual Java Serialization API.
But how would it ever make sense to use a proprietary format to transfer data around? How would non-Java clients (e.g. JavaScript!) ever be able to consume the serialized Java object? These days XML, JSON and even CSV are much more widely supported and accepted. Practically every self-respected programming language has tools to easily convert between XML/JSON/CSV and the model as definied in the programming language in question. E.g. Java has JAX-RS API to easily convert between javabeans and XML or JSON. JavaScript has —obviously— builtin support for JSON (guess what "JS" in JSON stands for).
To learn and play around with the basic concept, head to this answer: How to use Servlets and Ajax?
To learn about the advantages of JAX-RS over servlet, head to this answer: Servlet vs RESTful
You can still use struts or jsf as you would normally to construct markup(html). And then consume the markup that was constructed via ajax and then append to the dom. If you are familiar with jQuery, something like jQuery('#selector').load('actionUrl.action');
But if you are looking to examine a java bean, then you will have to serialize it to xml or json. If you are using a web framework like struts2 or spring, there is likely a mechanism for doing this serialization for you. If you want to edit the bean you will have to serialize, then edit the serialized bean, and then deserialize back to the java bean.