I'm working on a micro-services project. For this, I'd like to have one Go package per service, all included in a parent package for the project. It looks like this:
.
└── github.com
└── username
└── project
├── service1
└── service2
I think this structure allows to comply with the Go conventions on package names and paths. A consequence of this is that all my microservices end on the same repository on Github, as the repository will be at depth 3 in the URL. I think this may become an issue in the future if the codebase becomes large. It may also add complexity for the CI/CD pipeline, for example a change to one service would trigger a build for all other services and the code to clone would be unnecessarily large.
Is there a way to avoid this conflict between the Go conventions and the way Github works ? Or this a problem that have to be solved during the CI/CD work ?
What you're talking about is popularly called "monorepo" these days. While I personally like having all my projects in the own independent repositories (including microservices and everything else), there are a number of proponents of having all code for a company in a single repository. Interestingly, both Google and Facebook use monorepos though it must be said they have built a lot of fancy tooling to make that work for them.
One important thing to note is that your repository is a separate thing from your architecture. There is not necessarily any correlation between them. You can have microservices all in a single repo and you can have a monolith divided up into several repos; the repository is only a tool to store and document your code base, nothing more.
While researching the topic, here are some of the advantages and disadvantages taken from a number of articles across the web:
Monorepo Advantages
Ease of sharing modules between projects (even in microservices, there are often cross-cutting concerns)
One single place to see and know what code exists - especially useful in large companies with lots of code
Simplifies automated and manual code review processes
Simplifies documentation rather than pulling from multiple, disconnected repos
Monorepo Disadvantages
Massive codebase can be challenging/slow to check in/out to local
Without very clear, strict guidelines it can be easy to cause tight coupling between products
Requires (slightly) more complex CI/CD tooling to partial-release
Depending on repository platform, very large codebases can affect performance
And here's a good discussion on the pros and cons of monorepos, and here's one specifically related to switching TO a monorepo with microservices architecture. Here's one more with lots of links both pro- and against-monorepo.
Like so many other things in programming and especially in SOA, the right solution for you depends on a number of factors that only you can determine. The main takeaway is that big and small companies have been successful with both options and many in between, so choose carefully but don't worry too much about it.
Versioning sub-packages that a Go project depends on can be track by git tagging. So using Go-modules one is encouraged to move sub-packages into their own git repos.
If the majority of your solution will be written in go, I'd suggest leveraging go modules.
This blog post explains how to manage Go-modules' go.mod with regard to package dependencies and their associated version git tags:
https://blog.golang.org/migrating-to-go-modules
There is a good way to work with separate repositories and Go microservices: Go plugins. In short:
Build a base image that implements shared functionality.
Have that base image look for a Go plugin somewhere in the container
In derived images, compile the functionality as a Go plugin and place it where the base image can find it.
For Go/gRPC, I have put a base image that does this here.
Related
I wonder how people are handling the repository of the documentation of their modules (I am mostly doing python/sphinx/git).
Facing two choices:
include the doc sources in the module repository
has the advantage of having a all in one unique place for the project, but the inconvenience of bringing non-vital data with the code (for instance images for the doc)
make a separate repository for the documentation
has the advantage of having a separate life cycle between the doc and the module code. When correcting a doc typo for instance one would like to avoid a new repository version for the code. The drawback is to maintain two repositories with maybe equivalent branching.
Are there clear guidelines somewhere? Am I missing some important points?
The question is tied more to CI/CD practices and infrastructure. In the release we follow, we club a set of microservices docker image tags as a single release, and do CI/CD pipeline and promote that version.yaml to staging and production - say a sort of Mono-release pattern. The problem with this is that at one point we need to serialize and other changes have to wait, till a mono-release is tested and tagged as ready for the next stage.A little more description regarding this here.
An alternate would be the micro-release strategy, where each microservice release in parallel through production through the CI/CD pipeline. But then would this mean that there would be as many pipelines as there are microservices? An alternate could have a single pipeline, but parallel test cases and a polling CD - sort of like GitOps way which takes the latest production tagged Docker images.
There seems precious little information regarding the way MS is released. Most talk about interface level or API level versioning and releasing, which is not really what I am after.
Assuming your organization is developing services in microservices architecture and is deploying in a kubernetes cluster, you must use some CD tool (continuous delivery tool) to release new microservices services, or even update a microservice.
Take a look in tools like Jenkins (https://www.jenkins.io), DroneIO (https://drone.io)... Some organizations use Python scripts, or Go and so on... I, personally, do not like this approch, I think the best solution is to pick a tool from CNCF Landscape (https://landscape.cncf.io/zoom=150) in Continuous Integration & Delivery group, these are tools test and used in the market.
An alternate would be the micro-release strategy, where each microservice release in parallel through production through the CI/CD pipeline. But then would this mean that there would be as many pipelines as there are microservices?
It's ok in some tools you have a parameterized pipeline thats build projects based in received parameters, but I think the best solution is to have one pipeline per service, and some parameterized pipelines to deploy, or apply specific tests, archive assets and so on... Like you say micro-release strategy
Agreed, there is little information about this out there. From all I understand the approach to keep one pipeline per service sounds reasonable. With a growing amount of microservices you will run into several problems:
how do you keep track of changes in the configuration
how do you test your services efficiently with regression and integration tests
how do you efficiently setup environments
The key here is most probably that you make better use of parameterized environment variables that you then look to version in an efficient manner. This will allow you to keep track of the changes in an efficient manner. To achieve this make sure to a.) strictly paramterize all variables in the container configs and the code and b.) organize the config variables in a way that allows you to inject them at runtime. This is a piece of content that I found helpful in regard to my point a.);
As for point b.) this is slightly more tricky. As it looks you are using Kubernetes so you might just want to pick something like helm-charts. The question is how you structure your config files and you have two options:
Use something like Kustomize which is a configuration management tool that will allow you to version to a certain degree following a GitOps approach. This comes (in my biased opinion) with a good amount of flaws. Git is ultimately not meant for configuration management, it's hard to follow changes, to build diffs, to identify the relevant history if you handle that amount of services.
You use a Continuous Delivery API (I work for one so make sure you question this sufficiently). CDAPIs connect to all your systems (CI pipelines, clusters, image registries, external resources (DBs, file storage), internal resources (elastic, redis) etc. They dynamically inject environment variables at run-time and create the manifests with each deployment. They cache these as so called "deployment sets". Deployment Sets are the representation of the state of an environment at deployment time. This approach has several advantages: It allows you to share, version, diff and relaunch any state any service and application were in at any given point in time. It provides a very clear and bullet proof audit auf anything in the setup. QA environments or test-feature environments can be spun of through the API or UI allowing for fully featured regression and integration tests.
I have question about microservices and repositories. We are a small team (5 people) and we creating new project in microservices. Expected microservice applications in our project is between 10-15.
We are thinking about one repository for all microservices in structure like that:
-/
--/app1
--/app2
--/app3
-./script.sh
-./script.bat
What do you think about this design? Can you recommend something better? We think if we will have repository per app it will be overkill for that small project in one team. As our applications you can imagine spring boot or spa applications in angular. Thank you in advice.
In general you can have all your micro-services in one repository but I think while the code grows for each of them it can be difficult to manage that.
Here are some things that you might want to consider before deciding to put all your micro-services in one repository:
Developer discipline:
Be careful with coupling of code. Since the code for all your micro-services is in one repository you don't have a real physical boundary between them, so developers can just use some code from other micro-services like adding a reference or similar. Having all micro-services in one repository will require some discipline and rules for developers not to cross boundaries and misuse them.
Come into temptation to create and misuse shared code.
This is not a bad thing if you do it in a proper and structured way. Again this leaves a lot of space for doing it the wrong way. If people just start using the same shared jar or similar that could lead to a lot of problems. In order to have something shared it should be isolated and packaged and ideally should have some versioning with support for backwards compatibility. This way each micro-service when this library is updated would still have a working code with the previous version. Still it is doable in the same repository but as with the 1. point above it requires planing and managing.
Git considerations:
Managing a lot of pull requests and branches in one repository can be challenging and can lead to the situation: "I am blocked by someone else". Also as possibly more people are going to work on the project and will commit to your source branch you will have to do rebase and/or merge source branch to your development or feature branch much more often(even if you do not need the changes from other services). Email notifications configured for the repository can be very annoying as you will receive Emails for things which are not in your micro-service code. In this case you need to create some Filters/Rules in your Email clients to avoid the Emails that you are not interested in.
Number of micro-services grow even further then your initial 10-15. The number can grow? If not, all fine. But if it does, at some point you could maybe consider to split each micro-service in a dedicated repository. Doing this at the point where you are in later stage of project can be challenging and could require some work and in worst case you will find out that there are some couplings that people made over time which you will have to resolve at this stage.
CI pipelines considerations:
If you use something like Jenkins to build, test and/or deploy your code
you could encounter some small configuration difficulties like the integration between Jenkins and GitHub. You would need to configure a pipeline which would only build/test a specific part of the code (or one micro-service) if someone creates a merge/pull request against that micro-service. I never tried to do such a thing but I guess you will have to figure out how to do it (script and automate this). It is doable I guess but will required some work to achieve it.
Conclusion
Still all or most of these points can be resolved with some extra management and configuration but it is still worth knowing what additional effort you could encounter. I guess there are some other points to be taken into considerations as well but my general advice would be to use separate repositories for each micro-service if you can (Private Repository pricing and similar reasons). This is a decision which is made project by project.
In my company, we have a system organized with microservices with a dedicated git repository per service. We would like to introduce gRPC and we were wondering how to share protobuf files and build libs for our various languages. Based on some examples we collected, we decided at the end to go for a single repository with all our protobuf inside, it seems the most common way of doing it and it seems easier to maintain and use.
I would like to know if you have some examples on your side ?
Do you have some counter examples of companies doing the exact opposite, meaning hosting protobuf in a distributed way ?
We have a distinct repo for protofiles (called schema) and multiple repos for every microservice. Also we never store generated code. Server and client files are generated from scratch by protoc during every build on CI.
Actually this approach works and fits our needs well. But there are two potential pitfalls:
Inconsistency between schema and microservice repositories. Commits to two different git repos are not atomic, so, at the time of schema updates, there is always a little time period when schema is updated, while microservice's repo is not yet.
In case if you use Go, there is a potential problem of moving to Go modules introduced in Go 1.11. We didn't make a comprehensive research on it yet.
Each of our microservices has it's own API (protobuf or several protobuf files). For each API we have separate repository. Also we have CI job which build protoclasses into jar (and not only for Java but for another language too) and publish it into our central repository. Than you just add dependencies to API you need.
For example, we have microservice A, we also have repository a-api (contains only protofiles) which build by job into jar (and to another languages) com.api.a-service.<version>
I'm starting to work on a new serverless project using AWS Lambda and API gateway.
What is the best way to organize my project, without being locked into one framework such as the serverless framework or chalice?
Here's what I'm using so far.
project-dir/
serverless.yaml (config file)
functions/
function1.py
function2.py
lib/
common_helper_functions.py
tests/
unit/
test1.py
test2.py
functional/
test1.py
test2.py
migrations
resources
cloudformation.templates.json
Do any of you recommend a better way to organize my project? Does each micro-service get a separate git repo? Am I missing other important folders?
Your structure looks good if a bit flat. I like putting code flows together. There are usually multiple functions to get to a result. Those should be grouped. Common functions that cross flows but don't cross projects go into a common folder in project. I base my repo organization on overall ideas. If lambdas cross projects they go in a common repo. Project specific stay in their repo.
Many times the hardest part of using a serverless architecture is finding the code being called. With a good logical grouping you will save yourself many headaches later.