I am not sure what is the best practice for running unit tests.
I suppose every developers should pass the unit tests locally before pushing the code to the GIT repo. And then the CI server (Jenkins) would pick up the new changes and run the tests again.
Why do we want to do it twice? Should we?
If the unit tests take a lot of time to run, do we expect the developer only picks the tests related to the change or runs every tests (even outside the scope of his projects), assuming we have a big maven multi-module POM.
Also consider we usually have a powerful hardware for CI server and relatively less powerful workstation for developers.
If the unit tests take a lot of time to run, do we expect the
developer only picks the tests related to the change or runs every
tests (even outside the scope of his projects), assuming we have a big
maven multi-module POM.
As a developer changes a class, modifies the database structure or makes any change that could have side effects, he/she will not/cannot know all potential side effects on the whole application.
And he/she never has to try to be too clever by saying : "I know that it may have be broken with my changes, so I will run just this test".
Unit testing ensures a some degree of code quality : don't make it less helpful
The unit tests are also non-regression tests. To not play all non regression tests before commit and push is taking the risk to introduce flawed code in the source content management.
You will never do it.
Unit tests have to be fast executed
If units test are so long to be executed as it hurts the developer velocity , it generally means that they are bad designed or maybe that they are not real unit tests. A unit test is designed to be run fast.
If you write tests that are long to be run because they require to start a server, to load/clean data in a database, to load/unload some containers, and so for... it means you didn't write unit test but integration tests. These tests are not designed to be executed regularly and automatically on the local development machine but on a CI tool.
The CI has to run all tests
Do we need to run tests in CI server if every developers run the tests
before push?
As explained, integration tests have to be executed by the CI tool.
About unit testing, sparing their execution in the CI side is not a good idea either.
Of course developers have to run the tests before pushing to the SCM but actually you don't have the guarantee that it will always be done.
Besides, even in a perfect world where developers execute all tests before pushing, you could fall into cases where the tests are successful on the developer machine but fail on the CI or the other developer machines.
For example, the developer could introduce in the base code some absolute paths specific to its machine or he/she could also forget to replicate a modification on the database used in the CI environment.
So running all tests (unit and integration tests) has not to be an option for the CI.
Yes, they shall be run twice. Because if some developers don't, then they run never. Developers should run the tests locally to make sure that their code works correctly.
Your CI system is however your reference, so there's no chance of one person arguing that it "works on my machine", but fails for others. Looking forward to continuous delivery, knowing this state on the CI/CD system becomes even more important.
You might hope that always and forever, every commit has been tested successfully locally (and all workstations are the same and identical with production systems...), but hope is a bad strategy.
We usually do smoke tests to check critical functionalities whenever we receive a new build. After executing the smoke tests, we are sure to go to next stage (next level of testing). I heard from my colleagues that smoke tests are really useful when your team employs Continuous Integration and DevOps. Smoke tests are always beneficial, but how it will be more beneficial with the combination of CI and DevOps?
Testing is interesting and every time a new challenge for QA which requires higher level of efforts in the final deployment of product. This consist of continuous delivery in continuous integration environment. In this continuous deployment process, requires testing to be followed in parallel in order to keep the process moving.
I've usually heard smoke testing used to refer to manual testing that you run to sanity-check builds. This article defines smoke testing as follows:
Smoke Testing, also known as “Build Verification Testing”, is a type
of software testing that comprises of a non-exhaustive set of tests
that aim at ensuring that the most important functions work. The
results of this testing is used to decide if a build is stable enough
to proceed with further testing.
First, I would certainly hope that people are doing this whenever they check code into the main branch to ensure that their changes didn't break the software in some obvious way. That holds whether you're doing continuous integration or not. (One of my personal pet peeves has always been people who check in code and then leave for the day without checking to make sure that it worked).
Also, keep in mind that in a typical CI cycle nowadays a build will often occur with every checkin to the main branch (or, at a minimum, there will be a nightly automated build; at my current company we have both), so you don't really have time to manually run your entire test suite for every build. One of the main purposes of CI is to have integration (and, as an extension, builds) occur much more frequently than is typical in other kinds of development cycles.
As one final comment: if you're doing continuous integration, I'd strongly encourage you to have some kind of automated testing (e.g. coded UI tests, unit tests, etc.) as part of that. Those can provide basic smoke/sanity testing and regression testing and reduce the burden of having to do all of it manually for every build.
What is the difference between these three terms? My university provides the following definitions:
Continuous Integration basically just means that the developer's working copies are synchronized with a shared mainline several times a day.
Continuous Delivery is described as the logical evolution of continuous integration: Always be able to put a product into production!
Continuous Deployment is described as the logical next step after continuous delivery: Automatically deploy the product into production whenever it passes QA!
They also provide a warning: Sometimes the term "Continuous Deployment" is also used if you are able to continuously deploy to the test system.
All this leaves me confused. Any explaination that is a little more detailed (or comes with an example) is appreciated!
Continuous Integration
I Agree with your university's definition. Continuous Integration is a strategy for how a developer can integrate code to the mainline continuously - as opposed to frequently.
You might claim that it's merely a branching strategy in your version control system.
It has to do with the size of the tasks you assign to a developer; If a task is estimated to take 4-5 man-days then the developer will have no incitement to deliver anything for the next 4-5 days, because he's not done with anything - yet.
So size matters:
small task = continuous integration
big task = frequent integration
The ideal task size is not bigger than a day's work. This way a developer will naturally have at least one integration per day.
Continuous Delivery
There are basically three schools within Continuous Delivery:
Continuous Delivery is a natural extension of Continuous Integration
This school, looks at the Addison-Wesley "Martin Fowler" signature series and makes the assumption that since the 2007 release was called "Continuous Integration" and the one that followed in 2011 was called "Continuous Delivery" they are probably volume 1+2 of the same conceptual idea that has to do with continuous something.
Continuous Delivery has to do with Agile Software Development
This school takes off-set in the idea that Continuous Delivery is all about being able to support the principles in the agile movement, not just as a conceptual idea or a letter of intent but for real - in real life.
Taking offset in the first principle in the Agile Manifesto where the term "continuous delivery" is actually used for the first time:
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
This school claims that "Continuous Delivery" is a paradigm that embraces everything required to implement an automated verification of your "definition of done".
This school accepts that "Continuous Delivery" and the buzz word or megatrend "DevOps" are flip sides of the same coin, in the sense that they both try to embrace or encapsulate this new paradigm or approach and not just a technique.
Continuous Delivery is a synonym to Continuous Deployment
The third school advocates that Continuous Deployment and Continuous Delivery can be used interchangeably to mean the same thing.
When something is ready in the hands of the developers, it's immediately delivered to the end-users, which in most cases will mean that it should be deployed to the production environment. Hence "Deploy" and "Deliver" means the same.
Which school to join
Your university clearly joined the first school and claims that we're referring to volume 1+2 of the same publication series. My opinion is that this is a misuse of the term Continuous Delivery.
I personally advocate for the understanding that Continuous Delivery is related to implementing a real-life support for the ideas and concepts stated by the agile movement. So I joined the school that says the term embraces a whole paradigm - like "DevOps".
The school that uses delivery as a synonym to deploy is mostly advocated by tool vendors who create deployment consoles, trying to get a bit of hype from the more widespread use of the term Continuous Delivery.
Continuous Deployment
The focus on Continuous Deployment is mostly relevant in domains where the end user's access to software updates relies on the update of some centralized source for this information and where this centralized source is not always easy to update because it's monolithic or has (too) high coherence by nature (web, SOA, Databases etc.).
For a lot of domains that produces software where there is no centralized source of information (devices, consumer products, client installations etc.) or where the centralized source for information is easy to update (app stores artifact management systems, Open Source repositories etc.), there is almost no hype about the term Continuous Deployment at all. They just deploy; it's not a big thing - it's not a pain that requires special focus.
The fact that Continuous Deployment is not something that is generically interesting to everyone is also an argument that the school that claims that "delivery" and "deploy" are synonyms got it all wrong. Because Continuous Delivery actually makes perfectly good sense to everyone - even if you are doing embedded software in devices or releasing Open Source plugins for a framework.
Your university's definition that Continuous Deployment is a natural next step of Continuous Delivery implicitly assumes that every delivery that is QA'ed should go become available to the end-users immediately, is closer to the definition that my tribe use to describe the term "Continous Release", which, in turn, is another concept that doesn't generically makes sense to everyone either.
A release can be a very strategic or political thing and there is no reason to assume that everybody would want to do this all the time (unless they are an online bookstore a streaming service type of company). Nevertheless, companies that don't blindly release everything all the time may have any number of reasons why they would want to be masters of deployment anyway, so they too do Continuous Deployment. Not of release to production, but of release-candidates to production-like environments.
Again I believe your university got it wrong. They are mistaking "Continuous Deployment" for "Continuous Release".
Continuous deployment is simply the discipline of continuously being able to move the result of a development process to a production-like environment where functional testing can be executed in full scale.
The Continuous Delivery Storyline
In the picture it all comes alive:
The Continuous Integration process is the first two actions in the state-transition diagram. Which - if successful - kicks off the Continuous Delivery pipeline that implements the definition of done. Deployment is just one of the many actions that will have to be done continuously in this pipeline. Ideally, the process is automated from the point where the developer commits to the VCS to the point where the pipeline has confirmed that we have a valid release candidate.
Neither the question nor the answers really fit my simple way of thinking about it. I'm a consultant and have synchronized these definitions with a number of Dev teams and DevOps people, but am curious about how it matches with the industry at large:
Basically I think of the agile practice of continuous delivery like a continuum:
Not continuous (everything manual) 0% ----> 100% Continuous Delivery of Value (everything automated)
Steps towards continuous delivery:
Zero. Nothing is automated when devs check in code... You're lucky if they have compiled, run, or performed any testing prior to check-in.
Continuous Build: automated build on every check-in, which is the first step, but does nothing to prove functional integration of new code.
Continuous Integration (CI): automated build and execution of at least unit tests to prove integration of new code with existing code, but preferably integration tests (end-to-end).
Continuous Deployment (CD): automated deployment when code passes CI at least into a test environment, preferably into higher environments when quality is proven either via CI or by marking a lower environment as PASSED after manual testing. I.E., testing may be manual in some cases, but promoting to next environment is automatic.
Continuous Delivery: automated publication and release of the system into production. This is CD into production plus any other configuration changes like setup for A/B testing, notification to users of new features, notifying support of new version and change notes, etc.
EDIT: I would like to point out that there's a difference between the concept of "continuous delivery" as referenced in the first principle of the Agile Manifesto (http://agilemanifesto.org/principles.html) and the practice of Continuous Delivery, as seems to be referenced by the context of the question. The principle of continuous delivery is that of striving to reduce the Inventory waste as described in Lean thinking (http://www.miconleansixsigma.com/8-wastes.html). The practice of Continuous Delivery (CD) by agile teams has emerged in the many years since the Agile Manifesto was written in 2001. This agile practice directly addresses the principle, although they are different things and apparently easily confused.
I think amazon definition is straight and simple to understand.
"Continuous delivery is a software development methodology where the release process is automated. Every software change is automatically built, tested, and deployed to production. Before the final push to production, a person, an automated test, or a business rule decides when the final push should occur. Although every successful software change can be immediately released to production with continuous delivery, not all changes need to be released right away.
Continuous integration is a software development practice where members of a team use a version control system and integrate their work frequently to the same location, such as a master branch. Each change is built and verified by tests and other verifications in order to detect any integration errors as quickly as possible. Continuous integration is focused on automatically building and testing code, as compared to continuous delivery, which automates the entire software release process up to production."
Please check out http://docs.aws.amazon.com/codepipeline/latest/userguide/concepts.html
Atlassian posted a good explanation about Continuous integration vs. continuous delivery vs. continuous deployment.
In a nutshell:
Continuous Integration - is
an automation to build and test application whenever new commits are pushed into the branch.
Continuous Delivery - is Continuous Integration + Deploy application to production by "clicking on a button" (Release to customers is often, but on demand).
Continuous Deployment - is
Continuous Delivery but without human intervention (Release to customers is on-going).
Continuous Integration basically just means that the developer's working copies are synchronized with a shared mainline several times a day.
Or more than several times per day. As often as any given discrete task is completed, basically. Consider for example a team of developers working on a single business application. In many environments, the following may happen:
One or two developers keep local changes for a few days because "it's not ready yet".
One or two developers create branches in the source control so they can work on their feature(s) "without being bothered by other people's changes".
These can lead to problems. Poor code/task organization leads to branching, branching leads to merging, merging... leads to suffering. Continuous integration as a practice addresses this by encouraging everybody to work from the same shared source. Individual work items should be discrete enough to be completed in a short amount of time (hours at most).
Basically the general idea is that integrating a small change in a small amount of work. Integrating a large change is a disproportionately large amount of work. The aggregate of integration work is smaller if done in constant small steps. This allows developers to spend more time working on business-visible features instead of development process overhead.
Continuous Delivery is described as the logical evolution of continuous integration: Always be able to put a product into production!
This follows the same idea of discrete, well defined work items. If there's a single master codebase which is only ever adjusted in small increments by complete, tested, known working features then that codebase is always stable. Automated testing is key here to be able to prove that stability at the push of a button.
The less stabilization work that needs to be done (which, again, is development process overhead and should be eliminated), the more often that codebase can be pushed to any given environment. In a lot of companies a deployment can be a pretty grueling process. Even a week-long all-hands-on-deck operation. This is expensive and produces no business value. By employing good work item definitions, effective automated testing, and continuous integration a team can be in a position to automate the codebase's delivery to any given environment.
Continuous Deployment is described as the logical next step after continuous delivery: Automatically deploy the product into production whenever it passes QA!
You'll rarely see this happen in a business environment, and it's quite a joy when it's encountered. If the codebase can be automatically tested and automatically deployed to any given environment then, well, production is an environment like any other. So if the team has built up to this point then there's a potential for significant value to the business by always being able to deploy updates to production.
Defect fixes are sent to customers faster, new features reach the market faster, new ideas are tested against the market in smaller increments to allow for redirection of priorities, etc.
For example, let's say a company has a big idea for a new feature in their software-based product or service. They've done some research, they know the market, and they believe this idea will result in a strong new line of revenue. Now consider two options for delivering that feature:
Spend months developing the whole thing in a one-off branch. Spend weeks integrating it back into the main codebase. Spend days testing it. Spend a day deploying it. And only then start tracking actual revenue in the production system.
Implement small parts of the feature, one at a time. Each week release a new piece of it. Each week get more data on actual revenue.
In the first scenario, if the feature doesn't have the desired market effect then a lot of money is wasted on something customers don't actually want. In the second scenario the fact that customers don't want it is determined much, much earlier and the rest of the work is de-prioritized.
Ultimately these "continuous things" are all about removing development process overhead. If a company's line of revenue is a particular service offering then ideally all of their costs should go into that offering. Development process overhead (merging code, re-testing the same features after a merge, manual deployment tasks, etc.) don't actually contribute to the value of the service, so these concepts seek to remove those costs from the process.
One graph can replace many words:
Enjoy! :-)
# I have updated the correct image...
Difference between Continuous Integration, Continuous delivery and continuous deployment
I think we're over analyzing and maybe complicating a bit the "continuous" suite of words. In this context continuous means automation. For the other words attached to "continuous" use the English language as your translation guide and please don't try to complicate things!
In "continuous build" we automatically build (write/compile/link/etc) our application into something that's executable for a specific platform/container/runtime/etc.
"Continuous integration" means that your new functionality tests and performs as intended when interacting with another entity. Obviously, before integration takes place, the build must happen and thorough testing would also be used to validate the integration. So, in "continuous integration" one uses automation to add value to an existing bucket of functionality in a way that doesn't negatively disrupt the existing functionality but rather integrates nicely with it, adding a perceived value to the whole.
Integration implies, by its mere English definition, that things jive harmoniously so in code-talk my add compiles, links, tests and runs perfectly within the whole. You wouldn't call something integrated if it failed the end product, would you?!
In our context "Continuous deployment" is synonymous with "continuos delivery" since at the end of the day we've provided functionality to our customers. However, by overanalyzing this, I could argue that deploy is a subset of delivery because deploying something doesn't necessarily mean that we delivered. We deployed the code but because we haven't effectively communicated to our stakeholders, we failed to deliver from a business perspective! We deployed the troops but we haven't delivered the promised water and food to the nearby town.
What if I were to add the "continuous transition" term, would it have its own merit? After all, maybe it's better suited to describe the movement of code through environments since it has the connotation of "from/to" more so than deployment or delivery which could imply one location only, in perpetuity! This is what we get if we don't apply common sense.
In conclusion, this is simple stuff to describe (doing it is a bit more ...complicated!), just use common sense, the English language and you'll be fine.
Continuous Integration: The practice of merging the development work with the main branch constantly so that the code has been tested as often as possible to catch issues early.
Continuous Delivery: Continuous delivery of code to an environment once the code is ready to ship. This could be staging or production. The idea is the product is delivered to a user base, which can be QA's or customers for review and inspection.
Unit test during the Continuous Integration phase can not catch all the bugs and business logic, particularly design issues that is why we need QA, or staging environment for testing.
Continuous Deployment: The deployment or release of code as soon as it's ready. Continuous Deployment requires Continuous Integration and Continuous Delivery otherwise the code quality won't be guarantee in a release.
Continuous Deployment ~~ Continuous Integration + Continuous Delivery
Continuous Integration
Automated(building of check ins + unit test)
Continuous Delivery
Continuous Integration
Automated(deployment to test environment + load testing + integration test)
Manual(deployment to production)
Continuous Deployment
Continuous Delivery but automated(deployment to production)
CI/CD is a journey. Not a destination.
These stages are suggestions. You can adapt the stages based on your
business need. Some stages can be repeated for multiple types of
testing, security, and performance. Depending on the complexity of
your project and the structure of your teams, some stages can be
repeated several times at different levels. For example, the end
product of one team can become a dependency in the project of the next
team. This means that the first team’s end product is subsequently
staged as an artifact in the next team’s project.
Footnote :
Practicing Continuous
Integration and Continuous
Delivery on AWS
Source: https://thenucleargeeks.com/2020/01/21/continuous-integration-vs-continuous-delivery-vs-continuous-deployment/
What is Continuous Integration
Continuous Integration is a process or a development practice of automated build and automated test i.e. A developer is required to commit his code multiple times into a shared repository where each integration is verified by automated build and test.
If the build fails/success it is notified to a developer and then he can take relevant actions.
What is Continuous Delivery
Continuous Delivery is the practise where we keep our code deployable at any point which has passed all the test and has all the required configuration to push the code to production but hasn’t been deployed yet.
What is Continuous Deployment
With the help of CI we have created s build for our application and is ready to push to production. In this step our build is ready and with CD we can deploy our application directly to QA environment and if everything goes well we can deploy the same build to production.
So basically, Continuous deployment is one step further than continuous delivery. With this practice, every change which passes all stages of your production pipeline is released to your customers.
Continuous Deployment is a combination of Configuration Management and Containerization.
Configuration Management: CM is all about maintaining the configuration of server which will be compatible to application requirement.
Containerization: Containerization is a set of tolls which will maintain consistency across the environment.
Img source: https://www.atlassian.com/
From what I've learned with Alex Cowan in the course Continuous Delivery & DevOps, CI and CD is part of a product pipeline that consists in the time it goes from an Observations to a Released Product.
From Observations to Designs the goal is to get high quality testable ideas. This part of the process is considered Continuous Design.
What happens after, when we go from the Code onwards, it's considered a Continuous Delivery capability whose aim is to execute the ideas and release to the customer very fast (you can read Jez Humble's book Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation for more details). The following pipeline explains which steps Continuous Integration (CI) and Continuous Delivery (CD) consist of.
Continuous Integration, as Mattias Petter Johansson explains,
is when a software team has habit of doing multiple merges per day and
they have an automated verification system in place to check those
merges for problems.
(you can watch the following two videos for a more pratical overview using CircleCI - Getting started with CircleCI - Continuous Integration P2 and Running CircleCI on Pull Request).
One can specify the CI/CD pipeline as following, that goes from New Code to a released Product.
The first three steps have to do with Tests, extending the boundary of what's being tested.
Continuous Deployment, on the other hand, is to handle the Deployment automatically. So, any code commit that passes the automated testing phase is automatically released into the production.
Note: This isn't necessarily what your pipelines should look like, yet they can serve as reference.
DevOps is a combination of 3C's - continuous, communication, collaboration and this lead to prime focus in various industries.
In an IoT connected devices world, multiple scrum features like product owner, web, mobile and QA working in an agile manner in a scrum of scrum cycle to deliver a product to end customer.
Continuous integration: Multiple scrum feature working simultanrouly in multiple endpoints
Continuous delivery: With integration and deployment, delivery of product to multiple customers to be handled at the same time.
Continuous deployment: multiple products deployed to multiple customers at multiple platform.
Watch this to know how DevOps enabling IoT connected world: https://youtu.be/nAfZt2t4HqA
Continuous Integration : is the practice where developers merge the changes to the code base to the main branch as often as possible. These changes are validated by creating a build and then running automated tests against the build. If these tests don’t pass, the changes aren’t merged, and developers avoid integration challenges that can happen.
Continuous Delivery : is an extension of CI since it enables automation to deploy all the code changes to an environment (dev, qa, stage, prod, etc) after the changes have been merged. The artifact may be built as part of CI or as part of this process since the source of truth (your repository) is reliable given your CI process. In simple terms, this means that there is an automated release process on top of the automated testing process and that developers can deploy their changes at any time by simply clicking a button or at the completion of CI.
Continuous Deployment : takes the process one step further than continuous delivery. Here, all changes that pass the verification steps at each stage in the pipeline are released to production. This process is completely automated and only a failed verification step will prevent pushing the changes to production.
lets keep it short :
CI:
A software development practice where members of a team integrate their work at least daily. Each integration is verified by automated build (include tests)to detect error as quick as possible.
CD:
CD Builds on CI, where you build software in such a way that the software can be released to production at any time.
I've been wondering how you go about doing CI-style testing when you're dealing with physical devices.
I imagine you have a suite of tests, and a pool of devices against which they can be run.
Additionally:
Some tests may require specific device models.
Some tests may require the use of more than one device.
What CI servers have support for this?
I'm still interested in those which have partial support, either natively or through plugins, as I'm interested in how it's done.
Continuous Integration enables a team integrate & test their work frequently. Automated builds are meant to compile, link and run unit-tests. You want your CI to run fast, especially if you run it at every check-in. That is why you would want to restrict CI activities to simple confirmation of the build and unit tests alone. What you're asking seems more-along the lines of quality assurance (QA) testing...and having QA failures mixed into your CI efforts would detract development efforts from progressing.
As such, I'm more under the impression activities associated with CI are not dependent upon the final physical machine said work may eventually be migrated to.
Now...this doesn't mean you CAN'T take the CI-compiled package and run it against some final target-machine....but again...that is really considered a seperate activity.
This seems to be re-enforced in the following article by Martin Fowler.
Notice he doesn't talk about the final targeted devices...only the build machine.
I can suggest Test Manager that is part of the TFS suite of Microsoft. I have not tried it with many different environments apart from windows based though I know there are many connectors. For windows based environments I believe it will satisfy most needs.
I use it for nightly builds to perform smoke tests (Turn it on, see if any smoke comes out) but you have to be careful to keep tests small in order to have them finished in a matter of hours and not days, if you want it to be part of your CI.
Then when you have a good enough quality you can proceed onto regression tests and integration tests if needed.
I wouldn't get too caught up in what a CI system is supposed to do or not do. Instead I would focus on the problem you are trying to solve. It sounds like that problem is to facilitate development on multiple platforms. You can use the concept of Continuous Integration and add to it successfully address the issue. I know, because I've done it in the past.
I implemented a build system for code that needed to compile and test successfully on 4 different platforms (nt, wince, linux-arm, linux-x86). The CI server would:
Used a linux and winnt build server compilation (and cross compilation)
The compiled tests and supporting libs would then be copied to the appropriate devices and an automated test run executed.
After the test suite was completed the log would be copied back, (or it was written to a network mounted fs)
If the test suite was successful we would tag the source, and package the libs and executables.
This same platform was reused for developer verification before commits. Developers would run a partial build and test (only updated source would be recompiled and those tests rerun). The CI would execute a full build (from scratch).
Our build were pretty fast because we had a proper DAG for build dependencies. This allowed for concurrent compilation within a platform build. Each platform build was also concurrent. As a result partial builds took a few seconds, full builds took ~30 minutes. Our build servers were quite beefy (optimized for fast compiles) and the codebase was of moderate size (I don't remember the stats).
I've been exploring different strategies for running integration tests within some Nant build scripts. Typically a number of different scripts are chained in one monolithic build that has separate targets: staging (build a staging version, like build), build (just build the stuff), integration (build the stuff and run the integration tests). This works reasonably well, the build target takes about a third of the time to run as the integration target and it's not painfully long so I don't find myself disinclined to run it frequently.
The integration target on the other hand takes long enough that I don't want to do it very often - ideally just before I'm ready to do a deploy. Does this seem like a reasonable strategy? IOW, am I doing it right?
The plan is to eventually move this project to Continuous Integration. I'm new to the whole Continuous Integration thing but I think I understand the concept of "breaking the build" so I'm wondering what are some good practices to pick up in order to make the most of it?
Any good sources of reading on this subject would be appreciated as well. Thanks!
Yes, you are on the right track. What you need to do now is to hook up your nant target to an automated process. I recommend using either Team City or Cruise Control for as your CI tool. Once you have your automated server setup you can run your build and unit tests on each check in (Continuous Integration). Your integration tests could then run at night or over the weekend since they typically take longer to run. If your integration tests are successful, you can then have a job that will deploy to some QA or other server.
Sounds like you're 99% of the way there. My advice is to just dive in and start doing it. You'll learn a lot more by actually taking the plunge and doing it than by thinking about whether you're doing it right.
My company is currently using CruiseControl and I personally think it's great.
See this related thread What is a good CI build process?
You are on the right track. If you're using a decent CI tool, you should be able to set each setup up as a separate project that triggers the next step in the chain... i.e. sucessfull build triggers tests which trigger deployment which triggers integration etc
This way your ealiest "break" stops the line so to speak.
We use CruiseControl to build, unit-test, configure and deploy, run integration tests and code coverage, run acceptance tests, and package for release. This is with a system of 8 or so web services, and a dozen or so databases, all with interralated configuration and deployment dependencies with across multiple environments with different configurations (anythin from single boxes to redundent boxes for each component)