Newbie to GitLab CI/CD here.
I'd like to use a linux base container as the Runner. Reason being I only have access to a Windows VM, and I'd rather not have to write PowerShell scripts build/test/deploy. Also, the build is using Spring Boot + maven, and I'm doubtful that the build scripts provided by Spring will run on Windows.
To further the problem, the build done by maven+spring spins up a container to execute the build. In other words, my build would run a container in a container, which seems feasible based on this blog.
Any feedback is much appreciated.
Edit based on #JoSSte 's feedback:
Are there any better approaches to setup a runner besides needing a container inside a container? For instance, if WSL enables running bash scripts, can the Windows VM act as the build server?
And to satisfy #JoSSte's request to make this question less opinion based, are there any best practices to approach a problem like this?
I have a crazy idea to run integration tests (xUnit in .Net) in the Jenkins pipeline by using Docker Compose. The goal is to create testing environment ad-hoc and run integration tests form Jenkins (and Visual Studio) wthout using DBs etc. on physical server. In my previous project sometimes there was a case, when two builds override test data from the second build and I would like to avoid it.
The plan is the following:
Add dockerfile for each test project
Add references in the docker compose file (with creation of DBs on docker)
Add step in the Jenkins that will run integration tests
I have no long experience with contenerization, so I cannot predict what problems can appear.
The questions are:
Does it have any sence?
Is it possible?
Can it be done simpler?
I suppose that Visual Sutio test runner won't be able to get results from the docker images. I am right?
It looks that development of tests will be more difficult, because test will be run on the docker. I am right?
Thanks for all your suggestions.
Depends very much on the details. In a small project - no, in a big project with multiple micro services and many devs - sure.
Absolutely. Anything that can be done with shell commands can be automated with Jenkins
Yes, just have a test DB running somewhere. Or just run it locally with a simple script. Automation and containerization is the opposite of simple, you would only do it if the overhead is worth it in the long run
Normally it wouldn't even run on the same machine, so that could be tricky. I am no VS Code expert though
The goal of containers is to make it simpler because the environment does not change, but they add configuration overhead. Most days it shouldn't make a difference but whenever you make a big change it will cost some time.
I'd say running a Jenkins on your local machine is rarelly worth it, you could just use docker locally with scripts (bash or WSL).
I'm quite new in docker, and i'm wondering that when using spring-boot, we can easily build, ship and deploy the application with maven or gradle plugin; and we can easily add Load Balance feature. So, what is the main reason to use docker in this case? Is containerized really needed in everywhere? Thanks for the reply!
Containers helps you to get the software to run reliably when moved from one computing environment to another. Docker consists of an entire runtime environment: your application and all its dependencies, libraries and other binaries and configuration files needed for its execution.
It also simplifies your deployment process, reducing a hell lot of mess to just one file.
Once you are done with your code, you can simply build and push the image on docker hub. All you need to do now on other systems is to pull the image and run container. It will take care of all the dependencies and everything.
I am using tools like Puppet/Chef/Ansible to set up and config development environments and production servers.
Whenever I update the configuration, I run the tool against my development environment and log in to check manually if things works as expected.
But this is tedious to do, and I can't test everything every time, so is there any way I can automate the testing?
There are Infrastructure Testing Frameworks for this:
ServerSpec / InSpec - ruby-based. Famous, big community, nice looking and best in his class.
BATS - Bash Automated Testing System, which is a bit easier.
TestInfra - Python-based infra testing framework. Still pretty young, very small community. Intro.
Goss - Fast (written in Go), small tool for validating server/infra configuration. Test scenarios are written in yaml.
Automation:
There is interesting Molecule project - some automation for testing Ansible roles, designed by Cisco. Never tried it yet.
Step further would be using TestKitchen which handles automation to spin up Vagrant or Docker or even AWS instance and test Puppet/Chef/Ansible with Rspec/BATS against just spinned up machines.
So what you need - pick up framework, write tests and run your playbooks/recipes & tests against mock VMs.
Ideally is to keep your "infra as code" in vcs and configure ci like TravisCI to run your tests for every PR once you bring new changes in your repository.
You can even follow tdd here: write tests first, make them fail, then write actual implementation in your favorite configuration management tool and see if that change makes tests green/passed.
MOAR Infrastructure Testing & Automation!
If you can let us know, what you want to test. We can help better.
But,
Did you check the dry-run mode? I think, Puppet and Ansible supports it, you can have a cron or some automated script which runs all the puppet/ansible modules against a single(test) node.
More info:
1. http://docs.ansible.com/ansible/playbooks_checkmode.html
2. Check the noop mode in https://docs.puppet.com/puppet/latest/reference/man/agent.html
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
I've never used CI tools before, but from what I've read, I'm not sure it would provide any benefit to a solo developer that isn't writing code every day.
First - what benefits does CI provide to any project?
Second - who should use CI? Does it benefit all developers?
The basic concept of CI is that you have a system that builds the code and runs automated tests everytime someone makes a commit to the version control system. These tests would include unit and functional tests, or even behavior driven tests.
The benefit is that you know - immediately - when someone has broken the build.
This means either:
A. They committed code that prevents compilation, which would screw any one up
B. They committed code that broke some tests, which either means they introduced a bug that needs to be fixed, or the tests need to be updated to reflect the change in the code.
If you are a solo developer, CI isn't quite as useful if you are in a good habit of running your tests before a commit, which is what you should be doing. That being said, you could develop a bad habit of letting the CI do your tests for you.
As a solo programmer, it mainly comes down to discipline. Using CI is a useful skill to have, but you want to avoid developing any bad habits that wouldn't translate to a team environment.
As other people have noted, CI does have advantages for a solo developer. But the question you have to ask yourself is; is it worth the overhead? If you're like me, it will probably take an hour or two to set up a CI system for a project, just because I'll have to allocate a server, set up all the networking, and install the software. Remember that the CI system will only be saving you a few seconds at a time. For a solo developer, these times aren't likely to add up to more than the time it took to do the CI setup.
However, if you've never set up a CI system before, I recommend doing it just for the sake of learning how to do it. It doesn't take so long that it isn't worth the learning experience.
The benefit of CI lies in the ability to discover early when a check in has broken the build. You can also run your suite of automated tests against the build, as well as run any kind of tools to give you metrics and such.
Obviously, this is very valuable when you have a team of commiters, not all of whom are diligent to check for breaking changes. As a solo developer, it is not quite as valuable. Presumably, you run your unit tests, and even maybe integration tests. However, I have seen a number of occasions where the developer forgets to checkin a file out of a set.
The CI build can also be thought of as your "release" build. The environment should be stable, and unaffected by whatever development gizmo you just add to your machine. It should allow you to always reproduce a build.
This can be valuable if you add a new dependency to your project, and forget to setup the release build environment to take that into account.
If you need to support multiple compilers then it's handy to have a CI build system to do all of that whilst you just develop in one IDE. My code builds with Vc6 through VS2008 in x86 and x64 builds on VS2005 & 8, so that's 7 builds per project per project configuration... Having a CI system means that I can develop in one IDE and let the CI system prove that all of the compilers that I support still build.
Likewise, if you are building libs that are used by multiple projects then CI will make sure they work with ALL of the projects rather than just the one that you're working with right now...
The truth is, that continuous integration makes most sense in teams. Single developers can also get some advantages, you must decide yourself if they are enough to counter the time you invest into setting a CI-system up.
If you forgot to checkin some needed file, the repository contains a broken version, even if it works on your machine. CI would detect that case.
If your CI-server runs on a different machine, it can indicate dependencies on your build-environment. Means, the build and all tests can work on your dev-box, but on another machine some dependencies aren't fulfilled and the build breaks.
Daily builds can indicate, that your older software doesn't work with the newest upgrade of the OS/compiler/library...
If your CI-system has an archive of build-artifacts you can easy get an distribution of an older version of your software.
Some CI have a nice interface to show you metrics about your build, have links to automatic generated documentation and stuff like that.
We use our CI system to do Release builds (as well as the usual automatic "on-commit" builds).
Being able to click a button that kicks off a Release build that steps through all the processes to release a setup is:
fast (I can go straight on with other things, and it runs on a separate machine so it is not slowing me down);
repetitive (it doesn't forget anything, including copying the setup to the release folder and notifying everyone who needs to know)
dependable (no mistakes, unlike a human!).
In an Agile environment, where you expect to be delivering working software every 2-4 weeks, this is definitely worth having, even in a team of 1.
CI benefits a solo developer in the sense that you're aware if you forgot to check something in (because the build will be broken). The integration value of it is diminished when there are no other developers, though.