Lets say I have multiple domain services for book, review and customer with a database each and a BFF as proxy/mediator for the client. The BFF brings all data together and communicates with non-domain-services. There is also an external API where some data is fetched from or sent to:
I am facing some issues here:
isolation: The book model is distributed over several services which also implies a lot of request/response mapping. Ergo: If the book domain model changes I have to change a lot of stuff, although, according domain modelling or microservices should rather prevent developers from adjusting multiple services/interfaces.
external dependencies: Unfortunately, in the real life scenario it is even worse, because the book service is implicitly highly coupled to the external API, meaning that, if the book model of the external API changes, the book service will probably change too.
validation: AFAIK validation should belong to the domain service. In the real life scenario however, I have to access author, customer and book at once, in order to do business validation.
Can you give me some direction how to deal with these issues?
Related
I have application which acts as a proxy between different systems without own database. There are few possible use cases which are covered by the application:
Display data from specific system or systems
Store data to specific system or systems
Actually this application has their own front-end and back-end (with sping boot and angular stack). And back-end is responsible to get/put data from/to external systems and front-end communicates with the back-end and it does not know anything about external systems. Also, the back-end follows hexagonal architecture and has their own defined domain models.
Currently there are requirements to cover auditing for business use cases related to the application. For instance, if user goes to some feature related to the application and make some changes there, it should be audited.
I've googled this topic on the internet but I only found entity based auditing like this https://docs.spring.io/spring-data/jpa/docs/1.7.0.DATAJPA-580-SNAPSHOT/reference/html/auditing.html. For my case I would need something similar but based on domain models rather then on entities.
Could you please recommend some direction to cover this? Actually which library or so can be used for such use case to use state of domain model to prepare audit events. I've found something like this https://logging.apache.org/log4j-audit/latest/gettingStarted.html, but I am really not sure if it is rigth way to go
I would say you can build your own auditing strategy based on events.
Let us take the example you gave: "if user goes to some feature related to the application and make some changes there, it should be audited.".
I assume you have a service that handles these requests from a REST API or something similar. That same service would not only communicate with the external systems but would also publish an event with let's say the information about the user and the performed changes or updated (here you can rely on Redis for example, but there are other options like RabbitMQ or even Kafka, depending on how reliable you want your auditing feature to be).
Then you would have another component of your app listening for these events so that you can store them in a Database (I guess that is the purpose). Or you can even have a separated micro-service only for this purpose, depending on how complex this auditing system is meant to be.
If you want something more "magical" and automated you can try to take a look at Spring Boot Data Audit code to see how it is implemented, but you might end up building an overengineered solution.
I'm new with this but in case if we have fronted + few different microservices I just don't get why do we need any of them to communicate with each other if we can manipulate between their data via axios on frontend. What is the purpose of event bus and event-driven architecture in the case if we use both frontend and backend microservices?
Okay, for my example I'm using 5 microservices. There are 2 of them:
Shopping cart
Posts
And I want to access posts microservice directly, pass their data through the event bus, so the shopping cart microservice would have its information. The reason is that posts and shopping cart both have different data bases, so is a good example doing this that way, or just through frontend with axios service?
What you are suggesting could be true for a very simple application, which hardly even needs an architecture such as microservice. It is clear why services need communication:
some services are not even accessable (for various reasons such as security) in client, so a change in them must be initiated in other backend services with such priviledge
some changes are raised in backend services and not client, e.g. a cronjob for doing some task
it would question reusability as you must consider the service to be used by not only client, but in any environment
what would happen if you want your services to be used by public, what if they do not implement part of the needed logic intentionally or by mistake
making client do everything could be so complex and would reduce flexibility
some services such as authentication are acting as a supporting mechanism to ensure safety (or anything other than main logic), these should be communicated directly by the service needing them
As for second part of your question, it depends on several factors like your business needs & models, desired scalability, performance, availability, etc. so the right answer or in another say, the answer that fits would be different.
For your problem, using event bus which is async would not be a good solution as it would hurt consistency in your services. Instead synchronous ways like a simple API call in your posts service would be a better idea.
I try to work with go-kit (gokit.io) and to build real-work application with it.
I look through examples. These examples are great. But I do not understand how to do services to service communications / data transfers in go-kit framework.
I can see "real-world" shipping app, but I do not understand how it could be "real world" micro-services. I can see in sources that, for example, they build the booking service just passing foreign repositories into service
type service struct {
cargoRepository cargo.Repository
locationRepository location.Repository
routingService routing.Service
handlingEventRepository cargo.HandlingEventRepository
}
and later they get data from repositories (this repository belongs to foreign micro-service) just calling the method:
locationRepository.Find(...)
Could someone please explain me:
how to build micro-service to micro-service communications in go-kit framework? Just show me the way / pattern, please. I do not understand it at all.
I see it as they just share direct access to data. But in real world micro-services, I expected that micro-services will communicate to each other to get needed data. And I do not understand how to do it in go-kit framework.
I'm the author of the shipping example. Sorry for not seeing your question earlier.
This particular example needs a bit of explanation. It is an example based on tactical patterns from Domain Driven Design, which means that we need to understand what we are talking about when we're referring to a service.
There are application services that deal with the use cases offered by the application, e.g. booking.Service. There are domain services that reside in the domain layer and provides your domain with concepts that aren't necessarily bound to a domain object. In the shipping example, routing.Service is a domain service whose implementation actually queries another application, in this case it talks to this routing service.
Application and domain services are merely ways of organizing our code. Putting it differently, these services communicate within a process, while microservices typically communicate over a network using some form of common transport, e.g. JSON, gRPC and so on.
Coming back to your question, what I believe you are looking for is the implementation of the routing.Service which you can find here.
The proxy service used here is explained under Client-side endpoints on this page, and is used to make requests from your application to another.
If you want more detail, I wrote a blog post on the subject a while ago.
I am designing an review analysis platform in microservices architecture.
Application is works like below;
all product reviews retrieved from ecommerce-site-a ( site-a ) as an excel file
reviews are uploaded to system with excel
Analysis agent can list all reviews, edit some of them, delete or approve
Analysis agent can export all reviews for site-a
Automated regexp based checks are applied for each review on upload and editing.
I have 3 microservices.
Reviews: Responsible for Review Crud operations plus operations similar to approve/reject..
Validations: Responsible for defining and applying validation rules on review.
Export/Import: Export service exports huge files given site name (like site-a)
The problem is at some point, validation service requires to get all reviews for site-a, apply validation rules and generate errors if is there any. I know sharing database schema's and entities breaks micro-services architecture.
One possible solution is
Whenever validation service requires reviews for a site, it requests gateway, gateway redirects request to Reviews service and response taken.
Two possible drawbacks of this approach is
validation service knows about gateway? Is it brings a dependency?
in case I have 1b reviews for a site, getting all reviews via rest request may be a problem. ( or not, I can make paginated requests from validation service to gateway..)
So what is the best practice for sharing huge data between micro-services without
sharing entity
and dublicating data
I read lot about using messaging queues but I think in my case it is not good to use messaging queue to share gigabytes of data.
edit 1: Instead of sharing entity, using data stores with rest API can be a solution? Assume I am using mongodb, instead of sharing my entity object between microservices, I can use rest interface of mongo (http://restheart.org/) and query data whenever possible.
Your problem here is not "sharing huge data", but rather the boundaries you choose to separate your micro services based on.
I can tell from your requirements that the 3 micro services you chose to separate (Reviews, Validations, Import/Export) are actually operating on the same context and business domain .. which is Reviews.
I would encourage you to reconsider your design decision and consider Reviews, as a single micro service, that handles all reviews operations and logic as a black box.
I assume that reviews are independent from each other and that validating a review therefore requires only that review, and no other reviews.
You don't want to share entities, which rules out things like shared databases, Hadoop clusters or data stores like Redis. You also don't want to duplicate data, thereby ruling out plain file copies or trigger-based replication on database level.
In summary, I'd say your aim should be a stream. Let the Validator request everything from Reviews about Site A, but not in one bulk, but in a stream of single or small packages of reviews.
The Validator can now process the reviews one after the other, at constant memory and processor consumption. To gain performance, you can make multiple instances of the Validator who pull different, disjunct pieces of the stream at the same time. Similarly, you can create multiple instances of the Reviews microservice if one alone wouldn't be able to answer the pull fast enough.
The Validator does not persist this stream, it produces only the errors and stores or sends them somewhere; this should fulfill your no-sharing no-duplication requirements.
Currently our service is implemented using a multilayer architecture dividing the whole service into three:
API
Business
Persistence
However this introduces a lot of redundancy within our system. A common adage in the industry is "DRY" (Don't Repeat Yourself). The redundancy has increased the development time, and made the system more fragile and cluttered our code with "copy" methods.
To give a better idea, say we have a Person service. This would require the following:
Person entity - JPA annotated class for ORM
Repository service request - contains field values to be persisted of the Person domain object with additional persistence options
Repository service response - contains field values of the Person entity
Person - class with business logic, domain fields and computed fields
Domain service request - contains field values of Person resource and additional business options
Domain service response - contains field values of Person business object excluding those that shouldn't be visible to API users
Person resource - class representing what should be viewable to the API users
And things get worse when taking nested objects into consideration.
The current design facilitates difference between concerns (business, API, persistence), however:
Currently, the differences are very small. This is causing us to have
very similar classes with only minor differences
Services returning
service response objects with fields instead of just the objects
itself hampers other services from depending on other services
Questions:
Is it worth it to go through with this design?
What are our alternatives?
What could we change to improve our situation?
I know where you're coming from. My shortest advice would be: read "Domain Driven Desing - Tackling Complexity Inside the Heart of Software" by Eric Evans.
A central part of the DDD is the domain: POJOs containing majority of the business logic.
The building blocks are more or less what you've already mentioned.
There are three kinds of services:
Application Services that are responsible for orchestration, transaction management and authorization
Domain Services contain business logic that doesn't fit other domain building blocks: entities, policies, factories, value objects. Create them only if you can't use other domain mechanisms.
Infrastructure Services. The most common are repositories which are responsible for persistence of root aggregates (this role play some of the entities), and only them. This is contrast with DAOs which are created for any entity. Other infrastructure services might be for instance clients of Web Services that are being used by the application.
This richness of different kinds of services together with the idea of pushing the logic down as far as possible, because the logic in the domain is the most easily reusable, gives the developers tools they need to build comprehensive and maintainable complex software. Note that DDD might be too heavy for simple CRUD apps.
The entry points to the system are either Web Services endpoints or controllers (for Web apps where UI is generated on the backend like in case of JSPs of JSFs).
For the middle sized systems I like to use approach inspired by CQRS, that is, in order to avoid inevitable slowness when loading multiple root aggregates for displaying purposes (read side) I write dedicated query services that return DTOs straight from the DB, in case of JPA using select new mechanism.