I've been tasked with setting up a continuous integration server for our project and am weighing up the options.
I would like to use cc.rb rather than cc for the simple fact that I'd rather be writing Ruby than XML. From what I can see cc is a more mature product and comes with more features out of the box.
So I would like to know if you were setting up an integration server with the following requirements, which system would you choose and how would you go about it?
Mercurial support
Run selenium test suite on multiple platforms
x10 support for build lights (not essential)
Any guidance would be appreciated.
Regards,
Chris
yup I agree, hudson is much more easier to manage and has a variety of nifty plugins that make it flexible
It does have mercurial support
http://wiki.hudson-ci.org/display/HUDSON/Mercurial+Plugin
Selenium support is also available
http://thinkvitamin.com/dev/easy-automated-web-application-testing-with-hudson-and-selenium/
Have you considered using hudson. It has support for multiple languages and pluins and it generally very flexible.
I believe it has mercurial support and you could prob hook in selenium test somehow.
Karl
Well personally I would use the original CruiseControl, but that's probably because I helped write it. ;)
There are lots of tools available to do what you want (CC, cc.rb, Hudson, etc.). If you just care about running those tests then most of them will do. If you want a more polished experience and a few more features (like IDE integration) then tools like Zutubi Pulse (or Bamboo or Team City) start to look good. If you want to get into the full lifecycle (CI->test->deployment) and reporting from multiple tools then something higher end like AnthillPro might be the way to go.
So really a question of what else you think you might want to do.
Related
Im doing research regarding continuous integration tools and there benefits. For my research im looking at the following tools:
GitLab CI
Jenkins
Bamboo
GoCD
TeamCity
Now I wont bother you with all the requirements and benefits. But so far im not finding so many differences between the tools except for these:
Fan-in fan-out support GoCD
Community size, Jenkins and GitLab seem to have most contributors
Costs
Open source or not
Amount of plugins available
I was wondering if some people who have had to choose a continuous integration tool aswell could share there experience and why they chose that tool and if there are certain differences that are worth thinking about before choosing which I didn't cover.
Now im leaning towards GoCD because of fan-in fan-out support and the visualisation of the continuous delivery pipeline does anybody have experience with the support on issues for this tool?
Thanks in regard,
Disclaimer: I was an active contributor to GoCD before previous Fall.
I haven't used GitLab CI so won't talk about that :) Also, I haven't used any of these tools in the past one year.
I think TeamCity is a good CI tool. It integrates very well with IDE if you want to debug some failures. The test reports are brilliant. But I don't think they are that advanced in CD space and in my opinion you need both. But if you are interested only in CI, you might want to give it a look. However, you will miss on some of the good features of GoCD I've mentioned below.
Jenkins has a huge community but Jenkins has its own disadvantages. Many a times one plugin doesn't work due to another plugin for some compatibility issues for instance.
GoCD has Fan-in/Fan-out support which avoids many unnecessary builds saving a lot of build time and resources. The value stream map is intuitive and helps to get a better picture of the build stage from a developer's, QA's or even Deliver Manager's point of view. The pipeline modeling in GoCD is also very good. If you read Jez Humble and David Farley's book on Continuous Delivery, you will see the power behind such a build design.
Now, to your second question:
Now im leaning towards GoCD because of fan-in fan-out support and the
visualisation of the continuous delivery pipeline does anybody have
experience with the support on issues for this tool?
Good to hear that :P I love GoCD. The support is good. If you choose to go the Open Source way, the mailing list is pretty active. You can expect a reply from the GoCD team within a day or two. Of course, your questions have to be genuine and specific. Looking through the forums before posting a question helps :)
You can also choose to buy support for GoCD from ThoughtWorks. They used to offer multiple support tiers, not sure of the current support model. You might face issues only when your DB grows too huge (~5-7 GB) when you might want to go for the proprietary Postgres DB support from ThoughtWorks. I've seen very few users of GoCD with that DB size.
I have a lot of experience with Teamcity and some with Gocd. If you are interested in fan-in/fan-out it's also possible to do the same in Teamcity -- it's called Build Chains.
Also there is a good post about this topic on official blog.
If I could choose I would prefer Teamcity. It's more mature and more feature rich product suitable for use in corporate environment.
I am new to load testing.
So please help in learning gatling and Apache Jmeter for stress testing.
Please help in installing both on Windows and Linux.
How to implement them in my application?
Which one is better for stress testing?
You are asking very generic questions in terms of Stress/Load testing. I think it would be best if you take a look at their documentation then formulate a more specific question.
Installation documentation is best served from the creators of the software.
Implementing these load/stress testing tools into your application isn't really a thing. If you are looking for unit testing (test to utilize in validating your functions/classes/etc work then look at your languages specific go-to libraries - ie. Java is junit/jboss, Nodejs is Karma/Protractor, Python is TestCase/Nose, etc). These tools (jmeter/gatling) are used for stressing your application outside of your build process so they should be treated as end-users (meaning you run the stress testing from remote machines if it is a web service).
Either are best for the right scenario. I think jmeter clusters easier (built-in, where gatling is more manual) but gatling is more programatic and can be manipulated more.
These are opinions and shouldn't be taken as fact or the best so your milage may vary
I strongly doubt that you need them both, if you want a piece of advice in regards which one to choose take a look at Open Source Load Testing Tools: Which One Should You Use? guide.
Once you have clear vision on what tool better suits the needs - you could start ramping up on the selected tool and ask questions in its community communication channels.
I am starting a maintenance project where I need to work with legacy code and also create new ones. New ones I can create proper interface based development and I can use Moq to do proper unit testing. I can't use Moq against the legacy code since they do not have proper coding to mock out the objects. Based on the reading, Typemock looks like an excellent fit since I can isolate the object and return what we want when a method is called on legacy objects. I really like the simplicity and ease of development. I would like to know is there any other tools out there like Typemock I should look at before committing to it since I have to pay for it.
Thanks
PS: ours is Microsoft shop we work with C#/ASP/ASp.Net/Silverlight and VB.Net
** I just found that Infragistics have one mocking tool as well.
What you're looking for is a mocking framework that uses the Profiler API. These are the ones that can "mock anything" because they sort of run "underneath" the code being tested.
When considering one of these, you'll want to think about things like:
Integration support: Does it work with the tools you're using? For example, your build server, coverage tool, unit testing tool, etc. Are there any additional tools it comes with (e.g., Visual Studio plugins) that might help you work with it?
Technical support: If something goes wrong or you find a bug in the product, can you ask someone about it?
Syntax/patterns: Does it support AAA testing or is it only record/replay? How easy is it to start working with?
Community and learning resources: How's the documentation? Are there people out there who can answer your more general usage questions or are you "on your own?"
There are three primary mocking frameworks to consider that I'm aware of:
Typemock Isolator - The most mature with a free license for open source projects, but otherwise not free.
Telerik JustMock - New to the scene, but good features. There is a free/limited version.
Moles - Free from Microsoft, though somewhat less easy to integrate with tools outside the VSTS realm.
Full disclosure: I'm a Typemock MVP and I've been using Isolator for a while. It has thus far been worth its weight in gold, particularly in Brownfield development. I will admit in some cases, where it's more Greenfield, I use Rhino or Moq. It really depends on what I'm doing.
In any case, I recommend evaluating each of the tools yourself to see which works best in your environment. Be sure to see how it integrates, how easy it is to get going, etc. for each one, and then make your decision based on your experience in your setting.
I am preparing to use continuous integration for the first time. I will be using Nant for the automated build and testing tasks, and am trying to find the appropriate CI tool to poll the repository and execute the Nant script.
I have so far loosely researched the following:
Hudson
Cruise Control
Draco
My sense so far is that since most of the work will be done by Nant, Draco would be fine because it is supposed to be easy to install/setup. Cruise Control and Hudson, on the other hand might provide better reporting options.
What is the best way to perform Continuous Integration when using Nant for the build and test tasks?
We use CruiseControl.NET as our CI server along with our full build system being in NAnt. It has worked exceptionally well over the past 5 years. I've since looked into other alternatives and have not been able to find any compelling reason(s) to switch.
CruiseControl.NET has amazing support for different source control systems. Also, I enjoy how extensible their application is as we've built a few systems that integrate into it.
Have you looked into TeamCity? I use it for my personal projects.
I can't speak for the applications listed, but I'm a huge fan of buildbot: http://buildbot.net/trac
It will build/test on any platform that runs Python (so if your software is supposed to work on multiple OS's each can be covered), it is very easy to use, and extremely extensible.
CruiseControl.NET rather than normal CruiseControl is what I'd recommend, although I have limited experience here.
I won't venture to say which one is best, however CruiseControl.NET certainly works very well, especially with NAnt-based builds. Personally speaking, though, it's the only one I've used.
Cruise Control seems to be the de facto standard out there (though I have no data to back that up).
Another option you might consider (though I haven't tried it) is ThoughtWorks' new Cruise product (see http://studios.thoughtworks.com/cruise-continuous-integration). ThoughtWorks developers were the fathers/mothers of Cruise Control years ago. It's free for up to two "agents".
We have started using TeamCity in my company and I must say that I am really impressed by this product. We tried CruiseControl.NET before, but TeamCity is much easier to configure and the configuration is much more transparent since you can control almost everything from the Web GUI.
As some other people already suggested, I will say TeamCity.
You may still wait until the end of this week, as their new version (TC 4.0) will be released (maybe today, maybe tomorrow)...
We use CruiseControl.NET and take advantage of it's preprocessors.
For the NAnt part, we use UppercuT. UppercuT uses NAnt to build and it is the insanely easy to use Build Framework.
http://code.google.com/p/uppercut/
Some good explanations here: UppercuT
Our company uses Hudson in coupled with NAnt. From my perspective, it works nicely.
We are about to get a canned package in that has been modified to our needs. I'm part of the team setup to prepare tests for it. It has an Oracle back end and I believe it's written in C++ .NET.
My question is what free or open source testing tools would you recommend.
Thanks
Ken
For regression testing of our applications I use a free tool called AutoHotKey http://www.autohotkey.com/. It is simple, batch configurable, and can work for virtually any application you have. Not exactly designed for black box testing, but a good free tool to add to your toolbox.
While there are a few good commercial applications for black box testing of applications (HoloDeck http://www.sisecure.com/holodeck/index.shtml, Cenzic Hailstorm http://www.cenzic.com/), the only open source applications that I know about only test network security (Spike http://www.immunitysec.com/resources-freesoftware.shtml, OWASP WebScarab http://www.owasp.org/index.php/Category:OWASP_WebScarab_Project, and Nikto http://www.cirt.net/nikto2)
Value checking. See if only valid dates are exempted, number fields except the full range, ect.
What do you expect from such a tool? I don't know of any tool that can arbitrarily test any piece of software.
For what is sounds like you already know what it is that you want to check. Being a custom application your best bet would be to devise a test plan and manually test the values that you would like to validate.
Agree with the others - since the application has been modified to your needs, you should make sure that it actually is modified to your needs.
If the assembly isn't obfuscated, you can use FxCop to analyze the binaries and see if there are any critical bugs (note - if you're not familiar with fxcop and static analysis, find someone who is before reporting a ton of bugs that won't be fixed).
Beyond that, you're looking at more techniques vs. tools to get the job done.
Testing, either functional or non-functional, without reference to the
internal structure of the component or system.