I have a question about microservice implementation. right now I am using an api gateway to process all get request to my individual services and using kafka to handle asynchronous post put and delete request. Is this a good way of handling of handling request in a microservice architecture?
Your question is too unspecific to give a good answer. What is a good architecture totally depends on the details of your use cases. Are you serving web pages, streaming media, amass data for analysis, or something completely different? We would also need to know what are you requirements in terms of concurrency, consistency and scalability? What are the constraints for budget/size of development teams, ease of development, dev skills, etc?
For example the decisions you have taken may be considered good if you have strong requirements for a highly scalable input of large data sets and very frequent data collection as well as the team to support it. But it may be considered bad if you have a small team only and are trying to get a quick and cheap MVP for a new service that has limited scalability requirements (because the complexity of the solution slows down your development unnecessarily).
It may be good because the development team is familiar with those technologies and can effectively develop with those. Or it may be bad because your team does not know anything about those and the investment in learning those will not be justifiable by long term gains.
Don't forget that one of the ideas of the microservices architectural style is that each service can be owned by a distinct team that makes its own decisions about what technology to use for implementation (for whatever reason: ease of development, business reasons etc). So in other words the microservices style embraces the old wisdom architecture follows organization.
Here a link to a recommended further read.
Related
As a mechanism to connect microservices together and make them work it is usually suggested to use APIs and Service Discovery. But they usually work as their own microservices, but these ones should apparently be "hard-coded" into others, as every microservice is supposed to register with them and query for other microservices' locations. Doesn't this break the idea of loose coupling, since a loss of a discovery service implies others being unable to communicate?
Pretty much yes. If one microservice "knows" about another microservice - it means that they are highly coupled. It doesn't matter where specifically this knowledge about other service is coming from: hardcoded, config file or maybe some service discovery, it still leads to high coupling.
What's important to understand is that many microservice evangelists are just preaching about how to build monolith apps on top of Web APIs. Some of them probably think that the more buzz words they use - the better ... not quite sure why this happens. It is probably easier to fake a language and become an "expert" by using buzzword salad instead of really building fault tolerant and horizontally scalable app.
But there is another way to look at service discovery: when it is used by service clients like SPA application or an API Gateway it may be very useful. Clients and gateways should know about service APIs, otherwise, the whole thing doesn't make sense. But they can use a registry to make it more flexible/dynamic.
So, to summarize:
if services are using discovery to get more information about each other - this is probably a bad thing and design flaw (pretty sure there are corner cases where this may be a valid scenario, please post a comment if you know some)
if discovery is used by other parts of the app, like SPA or API Gateway, this may be useful, but not necessarily it always is.
PS: to avoid high coupling, consider reading series of articles by Jeppe Cramon that illustrate the problem and possible solutions very nicely.
In a distributed system, you will always have some amount of coupling, what you want to do is to reduce all aspects of coupling to a minimum.
I'd argue that is does matter how you design your service location. if your code knows of the other service i.e. OrderService.Send(SubmitOrderMessage); (where 'OrderService' is an instance of the other service's proxy)
as opposed to transportAgent.Send(SubmitOrderMessage); (where 'transportAgent' is an instance of the transport's proxy i.e. the queuing service/agent and the actual address of the queue can be in your config), this reduces the coupling and your business logic code (Service) and delegates the routing to your infrastructure.
Make Sense?
Each microservice is expected to be functionally independent. For interacting with other microservices, it should rely on rest api calls only. Service Discovery plays a role to keep the services fairly loose coupled with each other. Also due to dynamic nature of the service urls, the hard dependencies are removed. Hope this helps
Reduced coupling or fairly loose coupling still has one thing in common; coupling. In my opinion, coupling to any degree will always create rigid communication patterns that are difficult to maintain and difficult to troubleshoot as a platform grows into a large distributed platform. Isn't the idea behind microservices to allow consumers to engage in "Permissionless innovation?" I would suggest that this is only possible by decomposing to microservices that have high cohesion and low coupling and then let the consumer decide how to route, orchestrate or aggregate.
To anyone with real world experience breaking a monolith into separate modules and services.
I am asking this question having already read the MonolithFirst blog entry by Martin Fowler. When taking a monolith and breaking it into microservices the "size" element of the equation is the one that I ponder over the most. Specifically, how to approach breaking a monolith application (we're talking 2001: A Space Oddessy; as in it is that old and that large) into micro services without getting overly fine grained or staying too monolithic. The end goal is creating separate modules that can be upgraded indepenently and scaled independently.
What are some recommended best practices based on personal experience of breaking a monolith into microservices?
The rule of thumb is breaking the monolith based on bounded context . The most common way of defining the bounded context is using BU ( Business Unit) . For example the module which does actual payment is mostly a separate BU .
The second thing to consider is the overhead micro-services bring. You should analyse the hardware , monitoring , infra pieces before completely breaking the service. What I have seen is people taking smaller microservices out of monolith instead of going and writing say 10 new service and depreciating the monolith.
My advice will be have an incremental approach . Take the first BU which is being worked upon out of monolith. This will also give a goos learning curve for the whole team.
You should clearly distinguish sub-domain areas (bounded contexts) from you domain.
Usually (if everything is fine with your architecture) you already have some separate components in your monolith application which responsible for each sub-domain. These components interact with each other in one process
(in monolith application) and you should to think about how to put them into separate processes. Of course you need to produce a lot of refactoring when moving one by one parts of the monolith to microservices.
Always remember that every microservice is responsible for some sub-domain.
I strongly recommend you to learn Domain Driven Design.
Domain-Driven Design: Tackling Complexity in the Heart of Software by Eric Evans
Implementing Domain-Driven Design by Vaughn Vernon
Also learn CQRS pattern
At the beginning you also should decide how your micservices will interact with each other.
There are several options:
Direct calls from one service to another
Send messages through some dispatcher service
which abstracts the client service from the knowledge where the called (destination) services are located.
This approach is similar to how proxy server like NGINX works.
Interact through some messaging bus (middleware), like RabbitMQ
You can combine these options, for example Query requests can be processed through Dispatcher Service, Commands and Events through message bus.
From my experience the biggest problem will be to go away from a single database,
which monolith applications is usually used.
In addition some good practices:
Put each microservice in own repository - this isolates from the ability to directly use the code of one micro service in another.
You also get faster checkouts and builds of each microservice on CI.
Interactions with any service should occur only through its public contracts.
It is necessary to aspire that each microservice has its own database
Example of the sub-domains (bounded contexts) for some Tourism Industry application.
Each bounded context can be serviced by a microservice.
We also started our journey some time back and i started writing a blog series for exactly the same thing: https://dzone.com/articles/how-i-started-my-journey-in-micro-services-and-how
Basically what i understood is to break my problem in diff. microservices, i need a design framework which Domain Driven Design gives(Domain Driven Design Distilled Book by Vaugh Vernon).
Then to implement the design (using CQRS and Event Sourcing and ...) i need a framework which provides all the above support.
I found Lagom good for this.(Eventuate , Spring Microservices are some other choices).
Sample Microservices Domain analysis using Domain Driven Design by Microsoft: https://learn.microsoft.com/en-us/azure/architecture/microservices/domain-analysis
One more analysis is: http://cqrs.nu/tutorial/cs/01-design
After reading on Domain Driven Design i think lagom and above links will help you to build a end to end application. If still any doubts , please raise :)
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
This post was edited and submitted for review 1 year ago and failed to reopen the post:
Opinion-based Update the question so it can be answered with facts and citations by editing this post.
Improve this question
What are advantages and disadvantages of microservices and monolithic architecture?
When to chose microservice architecture or monolithic architecture?
This is a very important question because a few people get lured by all the buzz around microservices, and there are tradeoffs to consider. So, what are the benefits and challenges of microservices (when compared with the monolithic model)?
Benefits
Deployability: more agility to roll out new versions of a service due to shorter build+test+deploy cycles. Also, flexibility to employ service-specific security, replication, persistence, and monitoring configurations.
Reliability: a microservice fault affects that microservice alone and its consumers, whereas in the monolithic model a service fault may bring down the entire monolith.
Availability: rolling out a new version of a microservice requires little downtime, whereas rolling out a new version of a service in the monolith requires a typically slower restart of the entire monolith.
Scalability: each microservice can be scaled independently using pools, clusters, grids. The deployment characteristics make microservices a great match for the elasticity of the cloud.
Modifiability: more flexibility to use new frameworks, libraries, datasources, and other resources. Also, microservices are loosely-coupled, modular components only accessible via their contracts, and hence less prone to turn into a big ball of mud.
Management: the application development effort is divided across teams that are smaller and work more independently.
Design autonomy: the team has freedom to employ different technologies, frameworks, and patterns to design and implement each microservice, and can change and redeploy each microservice independently
Challenges
Deployability: there are far more deployment units, so there are more complex jobs, scripts, transfer areas, and config files to oversee for deployment. (For that reason, continuous delivery and DevOps are highly desirable for microservice projects.)
Performance: services more likely need to communicate over the network, whereas services within the monolith may benefit from local calls. (For that reason, the design should avoid "chatty" microservices.)
Modifiability: changes to the contract are more likely to impact consumers deployed elsewhere, whereas in the monolithic model consumers are more likely to be within the monolith and will be rolled out in lockstep with the service. Also, mechanisms to improve autonomy, such as eventual consistency and asynchronous calls, add complexity to microservices.
Testability: integration tests are harder to setup and run because they may span different microservices on different runtime environments.
Management: the effort to manage operations increases because there are more runtime components, log files, and point-to-point interactions to oversee.
Memory use: several classes and libraries are often replicated in each microservice bundle and the overall memory footprint increases.
Runtime autonomy: in the monolith the overall business logic is collocated. With microservices the logic is spread across microservices. So, all else being equal, it's more likely that a microservice will interact with other microservices over the network--that interaction decreases autonomy. If the interaction between microservices involves changing data, the need for a transactional boundary further compromises autonomy. The good news is that to avoid runtime autonomy issues, we can employ techniques such as eventual consistency, event-driven architecture, CQRS, cache (data replication), and aligning microservices with DDD bounded contexts. These techniques are not inherent to microservices, but have been suggested by virtually every author I've read.
Once we understand these tradeoffs, there's one more thing we need to know to answer the other question: which is better, microservices or monolith? We need to know the non-functional requirements (quality attribute requirements) of the application. Once you understand how important is performance vs scalability, for example, you can weigh the tradeoffs and make an educated design decision.
While I'm relatively new to the microservices world, I'll try to answer your question as complete as possible.
When you use the microservices architecture, you will have increased decoupling and separation of concerns. Since you are litteraly splitting up your application.
This results into that your codebase will be easier to manage (each application is independent of the other applications to stay up and running). Therefore, if you do this right, it will be easier in the future to add new features to your application. Whereas with a monolithic architecture, it might become a very hard thing to do if your application is big (and you can assume at some point in time it will be).
Also deploying the application is easier, since you are building the independent microservices separately and deploying them on separate servers. This means that you can build and deploy services whenever you like without having to rebuild the rest of your application.
Since the different services are small and deployed separately, it's obvious easier to scale them, with the advantage that you can scale specific services of your application (with a monolithic you scale the complete "thing", even if it's just a specific part within the application that is getting an excessive load).
However, for applications that are not intended to become too big to manage in the future. It is better to keep it at the monolithic architecture. Since the microservices architecture has some serious difficulties involved. I stated that it is easier to deploy microservices, but this is only true in comparison with big monoliths. Using microservices you have the added complexity of distributing the services to different servers at different locations and you need to find a way to manage all of that. Building microservices will help you in the long-run if your application gets big, but for smaller applications, it is just easier to stay monolithic.
#Luxo is spot on. I'd just like to offer a slight variation and bring about the organizational perspective of it. Not only does microservices allow the applications to be decoupled but it may also help on an organizational level. The organization for example would be able to divide into multiple teams where each may develop on a set of microservices that the team may provide.
For example, in larger shops like Amazon, you might have a personalization team, ecommerce team, infrastructure services team, etc. If you'd like to get into microservices, Amazon is a very good example of it. Jeff Bezos made it a mandate for teams to communicate to another team's services if they needed access to a shared functionality. See here for a brief description.
In addition, engineers from Etsy and Netflix also had a small debate back in the day of microservices vs monolith on Twitter. The debate is a little less technical but can offer a few insights as well.
Where do you get info about 'how to build scalable, high perfomance web app'? I mean architecture, best practice ets. regardless of platform and language: .net, php, java ...
Did you get your own 'epic fails' in your project and then refactor your system in a few nights or get info from internet?
Is there any communities where I could share my own expirience and get some response?
Yeah, I know that every project is individual.
You can read the High Scalability blog. If you have questions about architechture and scalability, you can always use StackOverflow unless the question is subjective.
It is not easy to answer this question. Language and Platform takes a secondary place when thinking about scalability.
"Scalability is actually a property of a system, not an individual layer of that system, infrastructure. Even with the best, sexiest, most automatic scaling layer, you can easily write code that just doesn't scale. - glyph"
How ever, you can immerse into a very good collection of resources on this issue at
http://www.royans.net/arch/library/
Just focusing on the web part of scalable web architecture, you might want to take a look at these 7 reasons why you should be using XMPP instead of AJAX especially if your web app needs to scale with lots of real-time social features.
What are the advantages and disadvantages of DTOs from a website performance perspective? (I'm talking in the case where the database is accessed on a different app server to the web server - and the web server could access the database directly.)
DTO's aren't a performance concern. I think what you are asking about is the performance implications of tiering. In particular, using an application tier between your web tier (web server) and data tier (database server).
Generally, the implications are that latency is increased (you have extra network roundtrips), but you gain some additional capacity by splitting the load across machines.
Another common reason (again, non-performance) that people would do that is to allow them to place the web server in the DMZ while keeping the application and database servers inside the firewall.
Another potential reason (non-performance) is the ability to plug multiple UIs on top of a single application. I've done this on past projects with great results (where the business required it).
Also, don't underestimate the work required to maintain an architecture of that nature. It's more work than a non-tiered solution, so only use it if you anticipate needing it.
That being said, the use of DTOs does not necessitate the use of Tiering.
The best description I've found of tiering comes from Martin Fowler's book, Analysis Patterns. There's a small section in the back on application facades and tiering.
Just to reiterate the previous answer, DTOs aren't a performance concern. It's just a class without methods used to provide isolation between various parts of your application.
I'd also suggest picking up Martin's other book, Patterns of Enterprise Application Architecture. The DTO "pattern" is documented there.