We have a set of microservices communicating with each other and some external downstream services. Spring Cloud Contract is used for integration tests to check the service interfaces. I'm able to test the contracts for communication between the microservices.
I'd like to know how to write the contract tests for a producer which
doesn't provide any contracts (cannot access source code)
SOAP-based
Can I impose the contracts from the consumer itself?
Couldn't find the SCC documentation helpful in this regard. Any helpful pointers are appreciated. Thanks!
If it's SOAP based then it's a standard XML based communication. You can generate XML based contracts (an example is here https://docs.spring.io/spring-cloud-contract/docs/current-SNAPSHOT/reference/htmlsingle/#contract-dsl-xml ). Now as for the one where you don't have access to source code, what you can is the following:
You can create a test that will go via a proxy (e.g. WireMock proxy) to the real application and that way you will store that communication in a form of stubs (https://www.youtube.com/watch?v=ZyHG-VOzPZg), you can also check the code (https://github.com/spring-cloud-samples/the-legacy-app/tree/master/stubs_with_proxy). The problem here is that the producer might not be idempotent or may require a lot of concrete setup on the input to get proper response (e.g. a precisely defined date).
You can create contracts on the consumer side, tell stubrunner where they lay (https://docs.spring.io/spring-cloud-contract/docs/current-SNAPSHOT/reference/htmlsingle/#features-stub-runner-stubs-protocol) and turn the generateStubs feature (https://docs.spring.io/spring-cloud-contract/docs/current-SNAPSHOT/reference/htmlsingle/#features-stub-runner-generate-stubs-at-runtime), example of code (https://github.com/spring-cloud-samples/spring-cloud-contract-samples/blob/master/consumer_with_latest_2_2_features/src/test/java/com/example/GenerateStubsTest.java). The problem here is that as a consumer you can write whatever you want and it might not have anything to do with reality. On the other hand you already have the contracts that you could upload to the producer side with a request that they implement their part one day.
It's up to you to decide which one you prefer, bearing in mind that each of the solutions might have its issues.
As for Pact, Pact follows the second approach since it's the consumer that defines how the interaction should look like. That again might have nothing to do with reality.
Related
I wrote some micro-services using Quarkus that communicate via Artemis. Now I want to add OpenTelemetry for tracing purpose.
What I already tried is to call service B from service A using HTTP/REST. Here the trace id from service A is automatically added to the header of the HTTP request and used in service B. So this works fine. In Jaeger I can see the correlation.
But how can this be achieved using Artemis as messaging system? Do I have to (manually) add the trace id from service A into the message and read it in service B to setup somehow the context (don't know whether this is possible)? Or is there possibly an automatism like for HTTP requests?
I would appreciate any assistance.
I have to mention at this point that I have little experience with tracing so far.
There is no quarkus, quarkiverse extension or smallrye lib that provides integration with Artemis and OpenTelemetry, yet.
Also, OpenTelemetry massaging spec is being worked at the moment, because the correct way to correlate sent, received messages and services is under definition at the OTel spec level.
However, I had exactly the same problem as you and did a manual instrumentation that you can use as inspiration: quarkus-observability-demo-activemq
It will correlate the sent service as parent of receiving end.
I have an application separated in various OSGI bundles which run on a single Apache Karaf instance. However, I want to migrate to a microservice framework because
Apache Karaf is pretty tough to set up due its dependency mechanism and
I want to be able to bring the application later to the cloud (AWS, GCloud, whatever)
I did some research, had a look at various frameworks and concluded that Quarkus might be the right choice due to its container-based approach, the performance and possible cloud integration opportunities.
Now, I am struggeling at one point and I didn't find a solution so far, but maybe I also might have a misunderstanding here: my plan is to migrate almost every OSGI bundle of my application into a separate microservice. In that way, I would be able to scale horizontally only the services for which this is necessary and I could also update/deploy them separately without having to restart the whole application. Thus, I assume that every service needs to run in a separate Quarkus instance. However, Quarkus does not not seem to support this out of the box?!? Instead I would need to create a separate configuration for each Quarkus instance.
Is this really the way to go? How can the services discover each other? And is there a way that a service A can communicate with a service B not only via REST calls but also use objects of classes and methods of service B incorporating a dependency to service B for service A?
Thanks a lot for any ideas on this!
I think you are mixing some points between microservices and osgi-based applications. With microservices you usually have a independent process running each microservice which can be deployed in the same o other machines. Because of that you can scale as you said and gain benefits. But the communication model is not process to process. It has to use a different approach and its highly recommended that you use a standard integration mechanism, you can use REST, you can use Json RPC, SOAP, or queues or topics to use a event-driven communication. By this mechanisms you invoke the 'other' service operations as you do in osgi, but you are just using a different interface, instead of a local invocation you do a remote invocation.
Service discovery is something that you can do with just Virtual IP's accessing other services through a common dns name and a load balancer, or using kubernetes DNS, if you go for kubernetes as platform. You could use also a central configuration service or let each service register itself in a central registry. There are already plenty different flavours of solutions to tackle this complexity.
Also more importantly, you will have to be aware of your new complexities, but some you already have.
Contract versioning and design
Synchronous or asynchronous communication between services.
How to deal with security in the boundary of the services / Do i even need security in most of my services or i just need information about the user identity.
Increased maintenance cost and redundant side code for common features (here quarkus helps you a lot with its extensions and also you have microprofile compatibility).
...
Deciding to go with microservices is not an easy decision and not one that should be taken in a single step. My recommendation is that you analyse your application domain and try to check if your design is ok to go with microservices (in terms of separation of concenrs and model cohesion) and extract small parts of your osgi platform into microservices, otherwise you mostly will be force to make changes in your service interfaces which would be more difficult to do due to the service to service contract dependency than change a method and some invocations.
The case for event-driven microservices such as Spring Cloud Stream is their asynchronous nature, which I do agree it makes them more scalable
But I have an issue regarding how to code it in a way where I don't lose certain key features that I have access to using synchronous services
In a servlet-based MS, I make full use of servlet context variables and servlet-based Spring autowiring functions
For e.g., I leverage heavily on HTTP headers to carry metadata between microservices without having to impact the payload. But in Spring Cloud Stream using Kafka, Kafka doesn't support message headers of any kind! I lose that immediately if I use SCS. Putting them into the payload causes all sort of changes in my model classes if I define the attributes clearly. Yes, I can use a simple Hashmap to simulate the HTTP header object but it really seems like reinventing the wheel to me.
On the auto-wiring side: I maintain an audit log record per request, which I implement by declaring a request-scoped Hashmap bean and autowiring it into any methods in the Servlet's call stack that needs to append data to the audit log. Basically it's just a global variable to hold some data within a single request. But in SCS, again, I lose that cos bean scopes that leverage on servlets are not available.
So far, there seems to be a lot of trade-offs that I have to make just to make Spring Cloud Stream work for me.
I thought about an alternative approach where I use SCS just to create an entry point but the Source method would just get the event, use a Processor to construct a HTTP request and send the request along to a HTTP endpoint. But, why go through all that trouble then?
Hoping that some more experienced devs would be able to shed some light on how they leverage on SCS.
#feicipet Thanks for the detailed question. let me try to address some of your concerns in the order you have listed them:
+1
+1
I am not sure why you are referring to it as servlet-based instead of Spring-based? Those are features provided by Spring, but read on. . .
Spring Cloud Stream doesn't use Kafka, the end user does while Spring Cloud Stream provides Kafka binder allowing Spring Cloud Stream to integrate with Kafka. Further more, while Kafka indeed did not support headers prior to version 0.11, Spring Cloud Stream always supported and will continue support headers even with Kafka pre-0.11, embedding them in the Message and then extracting them in the consumer side into the proper Message headers completely transparent to the end user. In other words one would assume that Kafka did support headers by simply using Spring Cloud Stream. With Kafka 0.11+ headers are supported natively and we have adjusted to that with the same level of transparency.
So, you don't need to put anything in the payload. Just create an appropriate Message<payload, headers> and SCSt will take care of the rest regardless of the broker (Kafka, Rabbit, Foo etc.).
Yes you do simply due to the fact that as you eluded earlier SCSt promotes an asynchronous and stateless architecture. However, I do not agree that what you are trying to accomplish is un-accomplishable. Rather it is accomplishable the way you are describing, but there are other way to maintain context and I would be more then glad to discuss it as a separate topic.
I would not call them trade-offs, rather difference in the architecture, that has its benefits, but it is a not one-size-fits-all architecture and therefore its viability should be discussed within the context of a concrete use case.
+1. You don't have to separate it as Source and Processor. You can simply create a custom Source app with exposed REST endpoint and custom processing logic. However we are currently working on enhancements i the framework to ensure that you could do the same with the existing starter apps.
Obviously we have touched on many points here and some of them would probably need to be debated further, but I hope this clears up some of your concerns.
Cheers
the technology stack in our company are:
Java, Spring MVC, Spring Boot, Jaxws etc..
and we provide webservices for the client to querying our services.
in terms of securing the SOAP service. some of the webservices uses spring OAuth security and some of them uses the Spring Basic Auth
recently one of the client flooded our server by sending huge amount of request in the short period of time.
we are going to implement something to provent this to happen. ideally a
per client based calling interval. which can recognize the high calling frequency. then ban the client or force the client to wait
before we code this from the scratch, I wonder if there are libraries we can reuse. Spring normally very good at providing solutions for most of the enterprise issues. but so far I have't found any thing. any hint, ideally a working sample. would be great!
EDIT1: ideally we want to implement this instead of fully rely on the HTTP server e.g tomcat or apache to handle this. because our own implementation would offer more fine grained rules, such as how long the interval should be,
what kind of customised message we can return, more important we can implement our own monitoring mechanism, and treating different client with different traffic allowance etc...
We have around 6 individual REST services which we would be calling in our orchestration layer. for example "Calling service-1 to check if a bank account is of certain type, if yes, then call service-2, else call service-3".
Is Spring Integration messaging framework is a right tool for orchestrating calls to those services or its an overkill? Please suggest if there is a better way to accomplish the same.
I can't suggest you better tool, because Spring Integration was the best one for me in the past and now it is my full time job. So, try to find other answers somewhere else.
Now about the point of orchestration with the Spring Integration. To be honest it's just a word which tries to describe the business logic in one application when it calls other external application according some conditions.
So, from big height having HTTP adapters in the Spring Integration, Transformers, Splitters, Routers and Gateways between them, we can say with confidence that messaging layer on Spring Integration fits your requirements and really can help you to distinguish business logic from the orchestration logic.
Not sure what to say else, but I can recommend take a look to the Spring Integration Reference Manual and pay attention to the Routing Slip and Scatter-Gather patterns.