Currently, I deal with microservices on a daily basis at my 9-5. Most everything that I touch is written in PHP, and as only a software engineer, SysOps manages everything that has to do with apps running, etc. I have a little familiarity in how the infrastructure and build pipeline is setup, but I still am not a SysOps or DevOps guy.
With that said, I love Golang and for a side project, I am creating a fairly large web application with a lot of moving parts. Writing and designing the code is easy as I have learned a lot from my day job, but deploying and managing Golang web apps (as they are executables) is quite different than updating files for apache to serve.
I have researched a lot on how I would build and deploy my microservice apps, but I keep on thinking of more problems that will need to be solved along the way. I have tinkered with the idea of using Docker for all of this, but I would rather not have the added complexity of learning that and managing storage for all of the images as that could be large.
Is there a best practice or a good way to manage Golang applications after they have been deployed? I would need a way to keep track of all the microservice processes to be able to see if they are still up and to be able to stop them when a new build is going to be deployed.
As for the setup, just assume that all the microservices will be run on the server, not in a container or in a VM. They will all need to be managed, but also able to act upon independently. Jenkins will be used for building and deploying. I will be using Consul for service discovery and possibly configuration, and most likely health checks on the services. I'm thinking of having each microservice register itself to consul when started and deregister when stopping.
Again, I am looking for a solution that is hopefully not just "Docker". I also had thoughts into creating a deploy service that manages the services (add and remove), as well as registering them in Consul. So if I cannot find a better solution, I might go that path. Any help is appreciated.
** Sorry if my question was confusing, but since a couple people answered on the wrong topic at hand, I will try to clarify. I don't need any help making the microservices, or even know anything about them. I brought that point up as to why I need to ask my question. Basically what I need is just the ability to manage the running go processes of all my microservices so I can do deployments and be able to stop and start processes to update the code. It is easier when you have to worry about one app, but when you can have up to 10-15 difference microservices they become harder to keep track of. After my own research, it seems that Supervisord is what I am looking for, but I'm not sure. That is the direction I am going in with this question. Thanks.
Golang is great to use for microservices, but I would say there is not so big difference of managing golang or other languages microservices.
What I would say is golang specific:
you don't need to install anything on servers since golang is compiled to single library
you can take advantage of std lib golang rpc package and gob binary decoding, instead of usin 3rd party solution (gorpc, protocol buffer etc)
Other than that you need to use your own judgement. There is plenty of ways of doing one things in microservices world; one day you will implement solution A but when after 3 month you will see that its better to do B, do that.
In internet, there is so much reading about microservices. I will recommend you 2 good resurces: https://books.google.co.uk/books/about/Building_Microservices.html?id=RDl4BgAAQBAJ&source=kp_cover&redir_esc=y&hl=en
And article: http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html
Remember, microservices are not a golden bullet, they often can help making application easier to maintain and grow, but from the other side require lot of additional work, consequence in specifying API contracts and strong devops culture.
Related
I'm a mobile/front-end developer and need help with the architecture on the back-end where I'm totally green. I'm building web and mobile front in Flutter that will communicate with the server written in GO. Based on the config file attached the Flutter front I will create few separate apps, but for every single app I need a separate instance of the back-end services or at least separate database.
My question is about what architecture I should use in terms of future scaling to lower the server maintenance costs while having the best performance. Correct me if I'm wrong because what I will write is the image of my understanding of the structure but based on what I wrote above - am I correct that I should use some load balancer with the business logic spread across Kubernetes instances and only have separate database for every single Flutter app? Or is there any other solution I'm unaware about? Any help or guides that will at least lead me to more knowledge I can learn would be much appreciated.
I don't know yet whether it's a perfect solution but I will leave it if someone in future will be looking for it. My friend who codes in PHP introduced me to the multi tenant architecture pattern and after I've researched it I find it a good solution to what I've been looking for.
referred to some posts and videos for Microservices that how it works and what is the benefit of it but none of the examples or videos satisfy my confusion. can anyone help and can share some live examples which can be related to the App development methodology.
It isn´t essential but it is necessary according to the problem you are trying to solve. The fact that microservices becomes a popular way to build application does not mean it solve all the problems.
There are cases where you should rethink the use of microservice and to build a monolith application can be good enough for your scenarios or you will end with a distrbuted big ball of mud.
The Law of the instrument says something like:
If the only tool you have is a hammer, everything looks like a nail.
So, we can not make the assumption that microservice is essential for all scenarios in application development.
I have 28 micro-services, some of which communicates with each other. All of them are built with SpringBoot 2x and they use their own resources (database, rabbitmq etc.). They are deployed in PCF.
I need to identify weakness of the overall system. That's when I resorted to Chaos Engineering. As this is my first time, I can use some help regarding how to design the effort, what metrics should I collect, tools that I can use, how long should I run such tests etc.
TIA
Done with my research. Posting it here so that someone else might find it useful.
Very good introductions including how to start your first chaos experiment: https://www.gremlin.com/community/tutorials/chaos-engineering-the-history-principles-and-practice/
Good summary of people, tools, companies doing chaos experiments:
https://coggle.it/diagram/WiKceGDAwgABrmyv/t/chaos-engineeringcompanies%2C-people%2C-tools-practices/0a2d4968c94723e48e1256e67df51d0f4217027143924b23517832f53c536e62
Tools:
Spinnaker: https://www.spinnaker.io/. Netflix Chaos Monkey does not support deployments that are managed by anything other than Spinnaker. That makes it pretty hard to use Chaos Monkey from Netflix.
ChaosMonkey for SpringBoot: https://docs.chaostoolkit.org/drivers/cloudfoundry/. Very easy to follow instructions. Easy to turn on/off using Spring profile.
Chaos Toolkit - https://docs.chaostoolkit.org/drivers/cloudfoundry/. This tool is particularly helpful to my situation since my applications are deployed in Cloud Foundry and this tool has a CloudFoundry extension. Pretty elaborate, but easy to follow instructions. My preferred tool so far.
Chaos Lemur - https://content.pivotal.io/blog/chaos-lemur-testing-high-availability-on-pivotal-cloud-foundry. This tool has promise but network admin won't share AWS credentials for me to muck with Pivotal cells.
You already found Chaos Toolkit, which is what I would recommend for the actual experimenting, especially when starting out.
The basis of Chaos is that you need to plan it out and analyse properly before starting out. And then you need to find out what kind of information you will need to monitor the results of the experiment. What happens to service X if we implement latency? Monitor requests, errors etc. Is there a functioning degraded state or is it just dead?
Your question is very broad and as such it's difficult to answer.
Make sure to try to prevent cascading failures whenever you start an experiment and always have stop that terminates the experiment and rolls everything back should something break really bad.
In the microservice architecture, the main idea is that, every service is a small program, that does one thing and does it well (like unix philosophy)
Some things are common to a variety of projects, eg. user management, contacts, companies, clients, products, payments, etc.
I want to know if there is some kind of repository for microservices that solve there common tasks, so that I don't must reinvent the wheel again.
On github it is hard to find something like this and on docker hub are most repos not documented.
So you use some 3th party microservices? Or you create everything yourself?
We use some of the stuff StdLib provides and they also have a registry for others to submit their own stuff. You should check it out.
Usually I create all but the concept of micro is opinionated, there are a lot of authentication framework ready in different languages for example this is in PHP
https://github.com/thephpleague/oauth2-server
but there is also something in golang, you can use this libraries to make your work simple.
I tried to search something ready to help you but I have the same feeling there aren't around.
We have a set of Microservices collaborating with each other in the eco system. We used to have occasional problems where one or more of these Microservices would go down accidentally. Thankfully, we have some monitoring built around which would realize this and take corrective action.
Now, we would like to have redundancy built around each of those Microservices. I'm thinking more like a master / slave approach where a slave is always on stand by and when the master goes off, the slave picks it up.
Should we consider using any framework that we could use as service registry, where we register each of those Microservices and allow them to be controlled? Any other suggestions on how to achieve the kind of master / slave architecture with the Microservices that would enable us to have failover redundancy?
I thought about this for a couple of minutes and this is what I currently think is the best method, based on experience.
There are a couple of problems you will face with availability. First is always having at least one endpoint up. This is easy enough to do by installing on multiple servers. In the enterprise space, you would use a name for the endpoint and then have it resolve to multiple servers (virtual or hardware). You would also load balance it.
The second is registry. This is a very easy problem with API management software. The really good software in this space is not cheap, so this is not a weekend hobbyist type of software. But there are open source API Management solutions out there. As I work in the Enterprise space, I am very familiar with options like Apigee, CA, Mashery, etc. so I cannot recommend an open source option and feel good about myself.
You could build your own registry, if you desire. Just be careful how you design it, as a "registry of all interface points" leads to a service that becomes more tightly coupled.