We are a small software development shop, small meaning one perm. software engineer and two contractors. I am considering using a continuous integration and release management process since our product needs to be certified by the FDA. Our development environment is WinCE 6.0, ARM 7, both using C/C++. I am looking for recommendations for open source tools to improve our process.
We are using Subversion for our repository and in the process of setting up Bugzilla as well. Would TeamCity be a good addition to this suite of tools?
Good luck with the FDA support; my company has been moving towards FDA certification and it is a crazy process.
I would definitely recommend setting up an automated build process as soon as possible in your development process; even if it is not a CI build process. I have had experience setting up an automated build process in both new and existing code bases and it is much easier to implement a build process in a new code base. If you are planning on a build process from the beginning, you can make changes to the structures of your code\project files (I work with C#) so that it will make setting up a build process easier.
As for build servers, yes, I would recommend TeamCity. I have been using it at my company for about a year to run a CI, nightly and QA build process. Overall, it is a very easy tool to configure, use, and maintain.
I also know that Jenkins (Hudson) is a very popular build server. However, I do not have any experience using it, so I cannot give you a good comparison.
Hope this information helps.
Related
Im doing research regarding continuous integration tools and there benefits. For my research im looking at the following tools:
GitLab CI
Jenkins
Bamboo
GoCD
TeamCity
Now I wont bother you with all the requirements and benefits. But so far im not finding so many differences between the tools except for these:
Fan-in fan-out support GoCD
Community size, Jenkins and GitLab seem to have most contributors
Costs
Open source or not
Amount of plugins available
I was wondering if some people who have had to choose a continuous integration tool aswell could share there experience and why they chose that tool and if there are certain differences that are worth thinking about before choosing which I didn't cover.
Now im leaning towards GoCD because of fan-in fan-out support and the visualisation of the continuous delivery pipeline does anybody have experience with the support on issues for this tool?
Thanks in regard,
Disclaimer: I was an active contributor to GoCD before previous Fall.
I haven't used GitLab CI so won't talk about that :) Also, I haven't used any of these tools in the past one year.
I think TeamCity is a good CI tool. It integrates very well with IDE if you want to debug some failures. The test reports are brilliant. But I don't think they are that advanced in CD space and in my opinion you need both. But if you are interested only in CI, you might want to give it a look. However, you will miss on some of the good features of GoCD I've mentioned below.
Jenkins has a huge community but Jenkins has its own disadvantages. Many a times one plugin doesn't work due to another plugin for some compatibility issues for instance.
GoCD has Fan-in/Fan-out support which avoids many unnecessary builds saving a lot of build time and resources. The value stream map is intuitive and helps to get a better picture of the build stage from a developer's, QA's or even Deliver Manager's point of view. The pipeline modeling in GoCD is also very good. If you read Jez Humble and David Farley's book on Continuous Delivery, you will see the power behind such a build design.
Now, to your second question:
Now im leaning towards GoCD because of fan-in fan-out support and the
visualisation of the continuous delivery pipeline does anybody have
experience with the support on issues for this tool?
Good to hear that :P I love GoCD. The support is good. If you choose to go the Open Source way, the mailing list is pretty active. You can expect a reply from the GoCD team within a day or two. Of course, your questions have to be genuine and specific. Looking through the forums before posting a question helps :)
You can also choose to buy support for GoCD from ThoughtWorks. They used to offer multiple support tiers, not sure of the current support model. You might face issues only when your DB grows too huge (~5-7 GB) when you might want to go for the proprietary Postgres DB support from ThoughtWorks. I've seen very few users of GoCD with that DB size.
I have a lot of experience with Teamcity and some with Gocd. If you are interested in fan-in/fan-out it's also possible to do the same in Teamcity -- it's called Build Chains.
Also there is a good post about this topic on official blog.
If I could choose I would prefer Teamcity. It's more mature and more feature rich product suitable for use in corporate environment.
Taking a indepth look at CI and a question rose up. Is a agile development process a pre-requisite to be able to work with Continuous Integration?
Would it be possible to implement a CI process in a traditional, team based
development process?
Gut feeling says me that agility is more or less a pre-requisite, but "gut feeling" is not an argument when talking to management... :-)
And is there any documentation out there about this? All I found take it for granted
that you already work agile.
I would argue that continuous integration is good practice in almost all development teams, whether you are following an agile process or not (along with source control and free coffee). I've used it in agile teams, traditional teams and when I am coding alone - it has always added value.
For any development process, CI gives you:
Immediate feedback on any build errors (e.g. when a developer has forgotten to add or check in a file)
Immediate feedback on unit test failures (if you have written unit tests, which again are a good idea whether you are following an agile process or not)
Your QA team having up to date binaries to test with
Automating the build process (which greatly reduces the chances of error when you release your software)
Have a look at Jenkins - it's free and pretty easy to set up.
CI is not really related to agile or not-agile methodology (although some state to require it, while others just indirectly imply it or not mention at all)
CI is the only tool (yeah suppose it like a keyboard) which helps you during development to eliminate some bugs ASAP
actually the only thing you need to do CI is configure version control system with some build tool (like post-commit hook), and ask all developers to commit/fetch code as soon they pretty sure that it will compile - this will be enough to start continuous integration, then of course you can add unit test etc
so, the answer - agile is not requirement and you can implement CI in any process, without implementing XP, Scrum, Whatever methodology
We know this is good to have, but I find myself justifying it to my employer. Please pitch in on why a development team needs a build server.
There are multiple reasons to use build servers. In no particular order and off the top of my head:
You simplify the developers' workflow and reduce the chance of mistakes. Your build server can take care of multiple steps such as checking out latest code, having required software installed, etc. There's no chance of a developer having some stray DLLs on their machine that can cause the build to pass or fail seemingly at random.
Your build server can replicate your target environment (operating system, etc.) and there's less of a chance of something working on developers' desktops and breaking in production.
While it's a good practice for developers to test everything they check in, sometimes they just don't. Then it's good to have the build server there to catch test errors and let the team know the product is broken.
Centralized builds provide easy access to code metrics -- which tests passed, which failed, how often, how well is your code covered by your tests, etc. Having a solid understanding of the quality state of the codebase reduces maintenance and testing costs by providing timely feedback that allows errors to be fixed quickly and easily.
Product deployment is simplified -- the developer or QA doesn't have to remember multiple manual steps. It can be easily automated.
The link between developers and QA is simplified. QA personnel can go to a known location to grab latest, propertly versioned builds.
It's easy to set up builds for release branches, providing an extra safety net for products in their release stage, when making code changes must be done with extra care.
To avoid the "but it works on my box" issue.
Have a consistent, known environment where the software is built to avoid dependencies on local dev boxes.
You can use a virtual server to avoid (much) extra cost if you need to.
ASAP knowledge on what unit tests are currently working and which do not; furthermore, you'll also know if a once passing unit tests starts to fail.
This should sum up why it is critical to have a build server:
http://www.codinghorror.com/blog/2006/10/the-build-server-your-projects-heart-monitor.html
It's a continuous quality test dashboard; it shows you statistics about how the quality of your software is doing, and it shows them to you now. (JUnit, Cobertura)
It makes sure developers aren't hamstrung by other developers breaking the build, and encourages developers to write better code. (FindBugs, PMD)
It saves you time and money throughout the year by getting better code from developers the first time - less money on testing and retesting - and by getting more code from the same developers, because they're less likely to trip each other up.
Two main reasons that non technical people can relate to:
It improves the productivity of the dev team because problems are identified earlier.
It makes the state of the project very obvious. I've shown my management the build status dashboard an now they look at it all the time.
One more thing. Something like Hudson is very simple to set up - you might want to simply run it somewhere in a corner for a while and then show it later.
This is my principal argument:
all official releases must be build in a controlled environment. No exception.
simply because you never know how the developers create their personal releases.
You also don't need to talk about build server as in "blade that costs an arm a a leg". The first build server I set up was a desktop machine that sat unplugged in a corner. It served us very well for more than 3 years.
One you have your build machine, you can start adding some features (Hudson is great) and implement everything that the other posters mentioned.
Once your build machine becomes indispensable to your organization (and everyone sees its benefits), you will be able to ask for a shiny new blade if you wish :-)
The simplest thing you can do to convince your your employer to have a build server is to tell them that they will be able to release faster and with better quality.
Faster releases come from the immediate feedback about quality of the build. If someone breaks the build, he or she can fix the broken build immediately thus avoiding a delay in the build and release schedule. Without a build server the team will have to spend time trying to find what and when happened and how to fix it.
Better quality is achieved by the build server running bug detection tools automatically every time someone check is changes into a version control system. You don't mention what is the main development language in your organization, but such tools, advanced but commercial and simple but free, exist practically for all languages. Lint, FxCop, FindBugs and PMD come to mind.
You may also check this presentation on benefits of continuous integration for a more extensive discussion.
Is a CI server required for continous integration?
In order to facilitate continous integration you need to automate the build, distribution, and deploy processes. Each of these steps is possible without any specialized CI-Server. Coordinating these activities can be done through file notifications and other low level mechanisms; however, a database driven backend (a CI-Server) coordinating these steps greatly enhances the reliability, scalability, and maintainability of your systems.
You don't need a dedicated server, but a build machine of some kind is invaluable, otherwise there is no single central place where the code is always being built and tested. Although you can mimic this affect using a developer machine, there's the risk of overlap with the code that is being changed on that machine.
BTW I use Hudson, which is pretty light weight - doesn't need much to get it going.
It's important to use a dedicated machine so that you get independent verification, without corruption.
For small projects, it can be a pretty basic machine, so don't let hardware costs get you down. You probably have an old machine in a closet that is good enough.
You can also avoid dedicated hardware by using a virtual machine. Best bet is to find a server that is doing something else but is underloaded, and put the VM on it.
Before I ever heard the term "continuous-integration" (This was back in 2002 or 2003) I wrote a nightly build script that connected to cvs, grabbed a clean copy of the main project and the five smaller sub-projects, built all the jars via ant then built and redeployed a WAR file via a second ant script that used the tomcat ant tasks.
It ran via cron at 7pm and sent email with a bunch of attached output files. We used it for the entire 7 months of the project and it stayed in use for the next 20 months of maintenance and improvements.
It worked fine but I would prefer hudson over bash scripts, cron and ant.
A separate machine is really necessary if you have more than one developer on the project.
If you're using the .NET technology stack here's some pointers:
CruiseControl.Net is fairly lightweight. That's what we use. You could probably run it on your development machine without too much trouble.
You don't need to install or run Visual Studio unless you have Visual Studio Setup Projects. Instead, you can use a free command line build tool called MSBuild.
I am preparing to use continuous integration for the first time. I will be using Nant for the automated build and testing tasks, and am trying to find the appropriate CI tool to poll the repository and execute the Nant script.
I have so far loosely researched the following:
Hudson
Cruise Control
Draco
My sense so far is that since most of the work will be done by Nant, Draco would be fine because it is supposed to be easy to install/setup. Cruise Control and Hudson, on the other hand might provide better reporting options.
What is the best way to perform Continuous Integration when using Nant for the build and test tasks?
We use CruiseControl.NET as our CI server along with our full build system being in NAnt. It has worked exceptionally well over the past 5 years. I've since looked into other alternatives and have not been able to find any compelling reason(s) to switch.
CruiseControl.NET has amazing support for different source control systems. Also, I enjoy how extensible their application is as we've built a few systems that integrate into it.
Have you looked into TeamCity? I use it for my personal projects.
I can't speak for the applications listed, but I'm a huge fan of buildbot: http://buildbot.net/trac
It will build/test on any platform that runs Python (so if your software is supposed to work on multiple OS's each can be covered), it is very easy to use, and extremely extensible.
CruiseControl.NET rather than normal CruiseControl is what I'd recommend, although I have limited experience here.
I won't venture to say which one is best, however CruiseControl.NET certainly works very well, especially with NAnt-based builds. Personally speaking, though, it's the only one I've used.
Cruise Control seems to be the de facto standard out there (though I have no data to back that up).
Another option you might consider (though I haven't tried it) is ThoughtWorks' new Cruise product (see http://studios.thoughtworks.com/cruise-continuous-integration). ThoughtWorks developers were the fathers/mothers of Cruise Control years ago. It's free for up to two "agents".
We have started using TeamCity in my company and I must say that I am really impressed by this product. We tried CruiseControl.NET before, but TeamCity is much easier to configure and the configuration is much more transparent since you can control almost everything from the Web GUI.
As some other people already suggested, I will say TeamCity.
You may still wait until the end of this week, as their new version (TC 4.0) will be released (maybe today, maybe tomorrow)...
We use CruiseControl.NET and take advantage of it's preprocessors.
For the NAnt part, we use UppercuT. UppercuT uses NAnt to build and it is the insanely easy to use Build Framework.
http://code.google.com/p/uppercut/
Some good explanations here: UppercuT
Our company uses Hudson in coupled with NAnt. From my perspective, it works nicely.