I wonder how long it would usually take for:
Professional
Average
Beginner
to setup and configure CI for a new project?
I have never set up CI before, which puts me squarely in your "Beginner" category. Your question nudged me to try and setup a CI system for my projects; something which I've always avoided, because I thought it would cost me a lot of effort and time.
It took me all of 20 minutes.
I used a fantastic project called CInABox (Continuous Integration in a Box). It consists of two simple scripts which download and compile Ruby and download, install and configure CruiseControl.rb for Ubuntu 8.04.
In just 20 minutes, I downloaded Ubuntu JeOS 8.04, configured a VirtualBox VM, installed Ubuntu in that VM, setup networking, installed Ruby, installed CruiseControl.rb, added my first project to CC.rb and watched the light go green! The most time was actually spent downloading Ubuntu, downloading Ruby and installing Ubuntu. The actual CI setup took less than 5 minutes.
Don't let the name fool you: CC.rb is written in Ruby, but you can build anything with it. In the default configuration, it assumes that you are using rake to build your project, but by setting just one configuration option, you can just as well use a shell script.
It depends on how much other infrastructure you already have in place and whether you have issues tying everything together. Even with that in mind, you should be able to get TeamCity and all the infrastructure up and and running within a day or so if you have a decent idea of what you're doing. The documentation is pretty good for TeamCity and should get you past any bumps.
It depends on may factors:
What features of CI do you want to use.
Have you project installed in your CI environment already.
What type of project. How easily it can be installed on fresh environment.
just to say a few.
I think that if project is not a trivial, then all this time spent for the CI environment is worth the price. Whether it is 20 minutes or 3 days.
CI Factory
TeamCity
CC.NET sample configs
Try.
Related
We are currently using bitbucket cloud to host our grails-app repository. We want to set up some pipelines to do things like run unit tests and make sure the app compiles before being able to merge a branch to master.
I know this can pretty easily be done by letting them host the pipeline and committing a well written pipe file, however there is a problem standing that our app is very large, and even on brand new macbook pros takes 20 minutes to compile, on some older ones it can take 2 hours or more. Grails, thankfully, only compiles files that have changes in them from the last compilation. However, this can't be used on a bitbucket pipe that's working off a fresh pull of the app every time it runs.
My solution to this was wanting to set up a pipeline to run for us internally so that it can already have the app pulled, and just switch to the desired branch and run from there. This still might take time if switching between 2 very diverged branches, but it's better than compiling from fresh every time.
I can't seem to find any documentation on hosting a pipeline internally with bitbucket cloud, does anyone know if this is possible, and if so where there is documentation for it?
It would also be acceptable to find a solution to the long compilation problem itself with bitbucket hosted pipelines.
A few weeks ago, self hosted runners was made available as a public beta. Here are the details: https://community.atlassian.com/t5/Bitbucket-Pipelines-articles/Bitbucket-Pipelines-Runners-is-now-in-open-beta/ba-p/1691022
Additionally, if you're looking to retain some of your files from one build to the next to save doing the same work over and over again, have a look at caches: https://support.atlassian.com/bitbucket-cloud/docs/cache-dependencies/ there are some built ones that you could use, but you can define your own custom ones as well. Essentially it's just a way of preserving the contents of a directory for a future build.
I need solution to speed up maven build process.
The project is based on Adobe cq5 otherwise AEM and i have more than 10 modules in my project where the build happens in linear way.
Currently Build process takes more than 10 min to compile.
is any other specific tool available or other way to speed up the process?
Thanks
I have one suggestion,If you have 10 modules than make separate profile for each module,and build only that part in which you are changing and modifying,no need to deploy 10 modules each time unnecessarily.It will not speed up maven project build but can help you to save your time.It is a workaround but will be helpful .
Try mvn -T 4 clean install #Builds with 4 threads
Its a multi-threaded mode to run Maven and is faster. Apache documentation here.
To add to the other answers:
1) Give more memory resources to the server where the AEM instance is installed, content creation involves a lot of disk access so using SSD is a must.
2) Having a clean AEM instance helps to speed up the process. As you may know the AEM repository grows because of revisions so each time you deploy the repository size grows and it becomes slower. if it's a production environment use maintenance tools like revision clean up and compaction.
revision clean up
how to maintain repository
As per my knowledge there are no such mechanism to speed up.
better to make a build of particular module, it will deploy faster rather waiting for all 10 module to be happened.
Thanks
You can try using the suggested answers to build modules in parallel. It should speed up the build in theory.
But really there is no magic answer. You have to find the bottleneck in your build, it could be the number of dependencies, it could be a specific slow plugin, it could be hardware related, and it could be something else.
Try this: https://github.com/lptr/maven-time-tracker
It may help you find the bottleneck.
I would like to answer this question, knowing it was posted a really long time back.
Currently I am using AEM 6.3 and to recompile and deploy CORE module changes there is a simple maven trick -
This command tells us to -
run 1 thread/core
compile just the core module of the list of projects
and of course, zip package and send it to running AEM instance.
In my observation, this reduced the turnaround time by a huge margin.
At work where we do LOB .NET/MSSQL developement, many projects we have are 2 person or even 1 person projects that have development life cycles of 1-3 months. The developers serve as business analyst/project managers/QA so things get done fast with minimal 'BS time' spent. We do get the bigger projects that can take 6 months and have a team of 5 devs on it, but these are more uncommon.
We're doing a push to initiate everyone doing TDD going forward (my most recent project has full code coverage and was developed solely), and I was doing research on the architecture required to take maximum benefit of it. It seems that most people doing TDD are doing CI, have a build server and are doing automated builds and have some kind of automated client build tool (FinalBuilder or nAnt) etc.
So my questions - I see the obvious benefits on the uncommon large projects where you have 5 people working on the same codebase at once - but will we see much benefit from doing CI on the small 2 man projects? What about on a 1 man project - for those is it just a complete waste since you're really not 'integrating' with anyone? And, how would you pitch CI / automated builds / build server to management?
Having an automated/repeatable build process, and being able to prove that the current build passes all tests and runs in a server environment is worth the effort on any size project IMHO.
I'd pitch it this way: manual builds are manual. Things can get mixed up even on small projects . An automated build solves this problem. The amount of time it takes to set up the build script will be made up many times over during the lifecycle of the application.
As far as CI with test runs etc... goes: It's a constant health check on the quality of the code base. It's good to know as soon as possible when one thing inadvertently breaks another thing.
On a small project, you don't need most of the infrastructure to do CI, especially the build server. What you do need is the tests, the build automation, revision control, and a controlled build environment. You can just as well have your build and test servers be virtual machine images you run on your workstations... just so long as the images are under revision control like the rest of the project.
Two parts to this questions.
1) As part of our Continuous Integration build process i would like to install everything as-if it were a virgin machine. Martin folwler paper: http://martinfowler.com/articles/continuousIntegration.html
Does he mean that we take each (integration) build (clean machine) and installing ALL the necessary software to make the build work? I'm guess this is what he meant by "Single Command" build.
2) Which leads me nicely onto the next question. Is it possible to install programs using Powershell/Dos all through the command line? For example how would I install WinRar and possibly MySQL? (Winrar being a easy example, MySql complex).
Anyways, I am interested to hear from real-world practitioners of CI and how they approach their build processes.
In the latest CI environment I built, I installed and configured the toolchains and SDKs under a single directory tree and then created an ImageX WIM image of the tree. Each clean build would then mount the image, checkout sources from version control, build them, run tests etc. When unmounting, just remember to not commit the changes back to the image so that the image file stays clean.
For each of our builds with Zed we ensure a completely clean working environment, but assume that the entire tool-chain and utility applications are already installed on the machine.
If you really want to go to the level of virgin machine, then I would agree with laalto and look into VM. Setup your VM library to represent the different build environments/configurations that you will need for your product set, and load/start them on demand as you require builds for different products.
I think it is very important to always build from a clean working directory, but I'd question the real value of always trying to start with a bare OS and install everything from scratch for every build.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
I've never used CI tools before, but from what I've read, I'm not sure it would provide any benefit to a solo developer that isn't writing code every day.
First - what benefits does CI provide to any project?
Second - who should use CI? Does it benefit all developers?
The basic concept of CI is that you have a system that builds the code and runs automated tests everytime someone makes a commit to the version control system. These tests would include unit and functional tests, or even behavior driven tests.
The benefit is that you know - immediately - when someone has broken the build.
This means either:
A. They committed code that prevents compilation, which would screw any one up
B. They committed code that broke some tests, which either means they introduced a bug that needs to be fixed, or the tests need to be updated to reflect the change in the code.
If you are a solo developer, CI isn't quite as useful if you are in a good habit of running your tests before a commit, which is what you should be doing. That being said, you could develop a bad habit of letting the CI do your tests for you.
As a solo programmer, it mainly comes down to discipline. Using CI is a useful skill to have, but you want to avoid developing any bad habits that wouldn't translate to a team environment.
As other people have noted, CI does have advantages for a solo developer. But the question you have to ask yourself is; is it worth the overhead? If you're like me, it will probably take an hour or two to set up a CI system for a project, just because I'll have to allocate a server, set up all the networking, and install the software. Remember that the CI system will only be saving you a few seconds at a time. For a solo developer, these times aren't likely to add up to more than the time it took to do the CI setup.
However, if you've never set up a CI system before, I recommend doing it just for the sake of learning how to do it. It doesn't take so long that it isn't worth the learning experience.
The benefit of CI lies in the ability to discover early when a check in has broken the build. You can also run your suite of automated tests against the build, as well as run any kind of tools to give you metrics and such.
Obviously, this is very valuable when you have a team of commiters, not all of whom are diligent to check for breaking changes. As a solo developer, it is not quite as valuable. Presumably, you run your unit tests, and even maybe integration tests. However, I have seen a number of occasions where the developer forgets to checkin a file out of a set.
The CI build can also be thought of as your "release" build. The environment should be stable, and unaffected by whatever development gizmo you just add to your machine. It should allow you to always reproduce a build.
This can be valuable if you add a new dependency to your project, and forget to setup the release build environment to take that into account.
If you need to support multiple compilers then it's handy to have a CI build system to do all of that whilst you just develop in one IDE. My code builds with Vc6 through VS2008 in x86 and x64 builds on VS2005 & 8, so that's 7 builds per project per project configuration... Having a CI system means that I can develop in one IDE and let the CI system prove that all of the compilers that I support still build.
Likewise, if you are building libs that are used by multiple projects then CI will make sure they work with ALL of the projects rather than just the one that you're working with right now...
The truth is, that continuous integration makes most sense in teams. Single developers can also get some advantages, you must decide yourself if they are enough to counter the time you invest into setting a CI-system up.
If you forgot to checkin some needed file, the repository contains a broken version, even if it works on your machine. CI would detect that case.
If your CI-server runs on a different machine, it can indicate dependencies on your build-environment. Means, the build and all tests can work on your dev-box, but on another machine some dependencies aren't fulfilled and the build breaks.
Daily builds can indicate, that your older software doesn't work with the newest upgrade of the OS/compiler/library...
If your CI-system has an archive of build-artifacts you can easy get an distribution of an older version of your software.
Some CI have a nice interface to show you metrics about your build, have links to automatic generated documentation and stuff like that.
We use our CI system to do Release builds (as well as the usual automatic "on-commit" builds).
Being able to click a button that kicks off a Release build that steps through all the processes to release a setup is:
fast (I can go straight on with other things, and it runs on a separate machine so it is not slowing me down);
repetitive (it doesn't forget anything, including copying the setup to the release folder and notifying everyone who needs to know)
dependable (no mistakes, unlike a human!).
In an Agile environment, where you expect to be delivering working software every 2-4 weeks, this is definitely worth having, even in a team of 1.
CI benefits a solo developer in the sense that you're aware if you forgot to check something in (because the build will be broken). The integration value of it is diminished when there are no other developers, though.