This question mixes a bit of project management as well as development. I understand the [major].[minor].[patch] scheming for version numbering of a project. With my clients' projects I use these numbering primarily for internal purposes so instead of refering to a project by the features involved, the team can say "what is the progress of v1.3.2?".
However, sometimes our clients have multiple minor releases at once. Each minor release contains a set of independent features (working with different departments of the client company), but both could launch different times. So if we label them as v1.3.3 and v1.3.4, the v1.3.4 version could release earlier than v1.3.3 and then the whole naming scheme is invalid.
How do you refer internally to these various builds if you don't know which will release first (due to waiting for client approval, or other external scheduling conflicts)?
Thanks!
Pretty simple -- we don't assign version numbers until we release. Problem solved!
That may sound flippant, but it's the truth. Sure, we would have internal projects dubbed e.g. "v5.5", but those were separate and independent from current work on the next iteration of v5.4.x, which would receive the next value of 'x' only upon completion and release. When v5.5 is ready, work on 5.4 ceases, we merge any changes made to 5.4 into 5.5, and then we release 5.5.0.
If you have separate builds for different clients (departments in your case), you can employ a modified versioning scheme. What we did is use [major].[minor].[client].[patch], e.g. 5.4.client1.4. The [patch] would be independent and only meaningful to that particular client, whereas [major].[minor] would correspond to the [major].[minor] version of the main codebase from which we forked off. For example, we might have simultaneous work on 5.5, 5.4.x, and 5.4.client1.x. When 5.5 is ready, 5.4.x merges into that, and then both projects fold into 5.5.x, but the client project might not be ready to merge all those changes and thus it would remain 5.4.client1.x until it was brought up-to-date with 5.5, then becoming 5.5.client1.x.
This may sound confusing, but it actually worked very well for us. We previously employed a variation of this scheme, where the client name was appended to the complete version number, i.e. [major].[minor].[patch]_[client]; again, however, [major].[minor] corresponds to the "core" [major].[minor] from whence it was forked/last merged, and the [patch] is wholly independent from the other versions and only meaningful to that client (this is why we later swapped the relative positions of [client] and [patch], to make it clear that e.g. 5.4.7 might actually have more fixes/be more "current" than 5.4.12.client1, and to better communicate that independence.
When a client-specific project merges back, of course, you drop it and increment to the next [patch], or perhaps make the jump up to the next [minor] or even [major] version, depending on the nature of the work. This does occasionally lead to some temporary confusion when a client project merges into the 5.4.x project and then we release from that version 6.0, then remember to rename the internal 5.5 project to 6.1, but it worked nonetheless.
As an alternative for your environment, internally refer to your current projects simply by client (department) name, e.g. the Accounting project, the HR project, etc. Don't use version numbers internally for this kind of thing, because as you see it just leads to confusion like version 5.4.6 coming out after 5.4.7 but before 5.4.9; meanwhile 5.4.8 never gets released because it was canceled. That's just a mess, so stay away from that. Simply call your projects by client name, and assign the next number
Related
A brief note before I start: there is a lot of explanation required to "set the stage", and it may seem like this is more of a design question than a question about a programming problem. The question is actually about SVN branching and merging, so please read to the end.
Scenario:
I have a large Visual Studio solution with quite a few projects. I'm using SVN, so of course the trunk has my production line of development. This consists of a core DLL assembly, a "main" UI user client, and a handful of "plugin" assemblies that operate by implementing interfaces on the core assembly in order to provide functionality within the UI, and also by utilizing a set of service methods which provide common functionality to all of the plugins (such as persistence logic operations, storage operations for a centralized file store architecture, etc.)
There are also external utilities that I have built over time which must duplicate a lot of the business logic in the plugins. I won't go into much detail because it will ultimately distract from my main question, but just picture, for example, a scheduled service on a server that handles centralized maintenance operations related to a particular plugin's data.
When I initially built this application, I (stupidly) didn't anticipate the need for centralized service tiers, so I architected the core assembly (for better or worse), as shown above, to be tightly integrated with the presentation layer of the application. In other words, the UI presentation logic needed to integrate the plugins with the user interface and the business logic needed by the plugins to perform common plugin logic operations is all part of the one "core" assembly. Therefore, much of the "shared" logic that exists between the plugins and the centralized services has resulted in duplicated code.
I decided to undertake the major refactoring initiative to pull out the common logic -- that which is not related to the presentation -- into a "shared" assembly. For this, I created a branch off the trunk. I reorganized common code into a "shared" assembly, and I re-pointed everything in the client application (plugins, etc.) and the external service applications to utilize the shared assembly. In many cases, I also had to rename classes in order to fit their more-general purpose going forward. The core assembly remained in place only to broker presentation-layer responsibilities between the plugins and the UI.
Problem:
Now that I have successfully completed the refactoring, I want to reintegrate the branch back into the trunk. Merging is tricky business even in simple cases, but what I'm facing here is a lot of tree conflicts to put it mildly. Also, in addition to residing in an entirely new project, the folder structure in the "shared" project is quite a bit different from what it was in the "core" project. Classes are, in many cases, located in different places due to the new mechanisms for using the shared assembly.
I want to maintain the version history of every class from its old home in the core assembly to its new home in the shared assembly. Furthermore, I want to guarantee that the merge is successful. That seems obvious, but in testing a miniature version of this whole scenario, I was never able to get the conflicts to resolve in such a way where my branch features remained entirely intact. Furthermore, the fact that I have renamed some of the classes, as I stated earlier, to suit their more-general roles, makes it very tricky to maintain the version history.
I will note that I am using AnkhSVN which helps in "normal" cases when you rename files to repair the moves, but it doesn't seem to work in these major tree-conflict cases. Also, I know there is a difference in how merges work between different versions of SVN -- I believe it's pre-SVN 1.5 and post-SVN 1.5. I'm using SVN 1.9.3.
I have been trying to figure this out for a few weeks now. I've been pouring through the SVN book, TortoiseSVN resources like this, and anything I could find from google searches, like this, this, and this -- among many, many, many others. I feel like I'm going crazy and I think advanced SVN (and Tortoise) are impossible to learn with the traditional teach-yourself, learn-from-the-web-and-books approach. At any rate, I would greatly appreciate any insight that is out there.
What is the proper methodology when you create a feature branch using SVN and plan on making major tree changes and "moves" (i.e. renames) so that you can reintegrate those changes with the trunk without losing anything?
Congratulations to stepping on the most "popular" rake in SVN - "Merge Hell after refactoring"!
There are (at least) two simple rules for your case, produced by the bitter experience:
Never perform refactoring in SVN
If you'll ignore rule 1: in the name of all that is holy and good in the world don't touch ANYTHING in trunk during refactoring in branch
If you reject these the righteous covenants you still have a ways to salvation
Pure SVN-way, long and dirty
Merge all and every subtree, which is source of Tree Conflict, determining by hands every source and target like
svn merge NEW_PATH/NEW_NAME old_path/old_name
and finalize this the bloody work by full merge
Tricky Mercurial-way (or Git-way, but I just hate Git)
Preface: such merges aren't problem at all for modern DVCSes, they have "bridges" to SVN-repos, thus - you can delegate this job of merging to external VCS of choice and return results back (with some limitations and warnings)
I'm too lazy to speak about all DVCSes and will explain only about Mercurial (considering that with SVN-background it will be the least painful migration).
With HGSubversion Mercurial can read (pull) and write (push) to Subversion repositories, but - it can't push to Subversion results of it's own merges, thus: it will be multi-stage operation with the substitution of WC of Subversion in the process
A brief synopsis
Install Mercurial (TortoiseHG) and HGSubversion extension
Clone the whole SVN-repository to Mercurial into some temporary location (not current Subversion WC)
Merge branch to mainline (SVN's trunk become default branch), resolve (possible) context-conflicts (not tree)
Test (?) results
Perform the full replacement of Subversion Working Copy (WC of trunk, obviously) by the content of Mercurial Working Directory (beware of .svn and .hg folders respectively)
Commit WC to trunk
For the beauty and compliance with all rules "cheat" mergeinfo data of trunk (committed in step 6 must me known later as mergeset, although it is not true formally)
HTH
PS - migration to Mercurial with HGVS doesn't seems as totally crazy idea for now
Here's the situation:
A while ago, I created a back-end interface for adding/modifying/deleting items in a DB. It was created as an individual project, and we are still using that particular instance of the code. I will call this "version 1".
A little while after I completed this, I duplicated the code and integrated it into a much larger project. I copied all of the DB tables that were used in the original project's DB into the DB that this project was already using. I also had to duplicate the front-end here, but it's the back-end that I'm concerned with. The requirements for this interface (I will call this "version 2") were a bit more complex, so I had to keep working on it and also make some modifications to the DB tables.
It is essentially 2 branched versions of the same interface. However, version 1 has not received much attention since version 2's conception. Some of the changes I made in version 2 were not just new features, but improvements/bug fixes that are relevant to version 1. I should have made these changes on version 1 as well, but unfortunately I didn't. Going forward I will need to make improvements to both versions, and some of these improvements will apply to both of them.
I would really like to merge the 2 interfaces somehow, as I'm quite sure that it will reduce my workload in the long run. Frankly, I would like to abandon version 1 and adapt the version 2 UI to version 1's DB.
My first instinct is to transport everything to a user control that has a "version" attribute. The version would then be used to hide/show the controls that are or aren't relevant to that version, and it would also be used to determine what DB queries/stored procedures to use. The issue with this is that this whole interface spans across 5 pages. A few of these pages are opened up as dialogs from another one of the 5 pages. I suppose that I could simply have one div per interface, and then when !IsPostBack, I can set the Visible property of each div according to a querystring variable. However, this seems like it would be awfully messy and it would be a LOT of code and markup on one page. I guess alternatively I could have 5 user controls, but this just seems like an odd design.
Are there any other viable solutions?
i've been maintaining 4 products with nearly identical interfaces:
they started from one project
got cloned and branched
i've been slowly re-merging them
The best suggestion i have is try to some up with a UI that works for both, and switch both to using that.
After that i have a project-local settings file, which defines which features should not exist in one or the other.
in our company we use tfs for source control of sql database version,when developers change the database they generate Equivalent script and put it in sql tfs project and checked in it with related workItem.after build we generate patch with this script for clients,but before pacth we need to some one decide on priority of checked in script,now i want to this decition become automatic and my question is how could specified priority in the moment of check in?
Sorry for my bad english,if you want more informationn to answer let me know.thanks.
Version handling of databases seems to be a never-ending problem. At a previous client, we gave the databases version properties, and then stored patch scripts in folders for each version, e.g. "Patches/2.0.10", "Patches/2.1.0". The patch scripts could then be executed in the same order as they were checked in (creation date).
Upon release, we ended up generating a complete patch script consisting of all those separate patches merged together (since the patches often affected the same data, they could be optimized) along with a new version number, allowing us to record what version any given databes instance had.
I am looking for some advice on how to allow easy customisation and extension of a core product on a per client basis. I know it is probably too big a question. However we really need to get some ideas as if we get the setup of this wrong it could cause us problems for years. I don't have a lot of experience in customising and extending existing products.
We have a core product that we usually bespoke on a per client basis. We have recently rewritten the the product in C# 4 with an MVC3 frontend. We have refactored and now have 3 projects that compose the solution:
Core domain project (namespace - projectname.domain.*) - consisting of domain models (for use by EF), domain service interfaces etc (repository interfaces)
Domain infrastructure project (namespace -projectname.infrastructure.*) - that implements the domain service-EF Context, Repository implementation, File upload/download interface implementations etc.
MVC3 (namespace - projectname.web.*)-project that consists of controllers, viewmodels, CSS, content,scripts etc. It also has IOC (Ninject) handling DI for the project.
This solution works fine as a standalone product. Our problem is extending and customising the product on a per client basis. Our clients usually want the core product version given to them very quickly (usually within a couple of days of signing a contract) with branded CSS and styling. However 70% of the clients then want customisations to change the way it functions. Some customisations are small such as additional properties on domain model, viewmodel and view etc. Others are more significant and require entirely new domain models and controllers etc.
Some customisations appear to be useful to all clients, so periodically we would like to change them from being customisations and add them to the core.
We are presently storing the source code in TFS. To start a project we usually manually copy the source into a new Team Project. Change the namespace to reflect the clients name and start customising the basic parts and then deploy to Azure. This obviously results in an entirely duplicated code base and I’m sure isn’t the right way to go about it. I think we probably should be having something that provides the core features and extends/overrides where required. However I am really not sure how to go about this.
So I am looking for any advice on the best project configuration that would allow:
Rapid deployment of the code – so easy to start off a new client to
allow for branding/minor changes
Prevent the need for copying and pasting of code
Use of as much DI as possible to keep it loosely coupled
Allow for bespoking of the code on a
per client basis
The ability to extend the core product in a single
place and have all clients gain that functionality if we get the
latest version of the core and re-deploy
Any help/advice is greatly appreciated. Happy to add more information that anyone thinks will help.
I may not answer to this completly, but here some advices:
Don't copy your code, ever, whatever the reason is.
Don't rename the namespace to identify a given client version. Use the branches and continuous integration for that.
Choose a branching model like the following: a root branch called "Main", then create one branch from Main per major version of your product, then one branch per client. When you develop something, target from the start in which branch you'll develop depending on what you're doing (a client specific feature will go in the client branch, a global version in the version branch or client branch if you want to prototype it at first, etc.)
Try the best to rely on Work Item to track features you develop to know in which branch it's implemented to ease merge across branches.
Targeting the right branch for you dev is the most crucial thing, you don't have to necessary define some hard rules of "what to do in which occasion", but try to be consistant.
I've worked on a big 10 years project with more than 75 versions and what we usually did was:
Next major version: create a new branch from Main, dev Inside
Next minor version: dev in the current major branch, use Labels to mark each minor versions Inside your branch.
Some complex functionnal features was developped in the branch of the client that asked for it, then reversed integrated in the version branch when we succeeded in "unbranded" it.
Bug fixes in client branch, then reported in other branches when needed. (you have to use the Work Item for that or you'll get easily lost).
It's my take on that, other may have different point of view, I relied a lot on the Work Item for traceability of the code, which helped a lot for the delivery and reporting of code.
EDIT
Ok, I add some thought/feedback about branches:
In Software Configuration Management (SCM) you have two features to help you for versionning: branches and labels. Each one is not better nor worst than the other, it depends on what you need:
A Label is used to mark a point in time, using a label, for you to later be able to go back to that point if needed.
A Branch is used to "duplicate" your code to be able to work on two versions at the same time.
So using branches only depends on what you want to be able to do. If you have to work one many different versions (say one per client) at the same time: there's no other way to deal with it than using branches.
To limit the number of branches you have to decide what will be a new branch or what will be marked by a label for: Client Specific Versions, Major Version, Minor Version, Service Pack, etc.
Using branches for Client versions looks to be a no brainer.
Using one branch for each Major version may be the toughest choice for you to make. If you choose to use only one branch for all major versions, then you won't have the flexibility to work on different major versions at the same time, but your number of branches will be the lowest possible.
Finally, Jemery Thompson has a good point when he says that not all your code should be client dependent, there are some libraries (typically the lowest level ones) that shouldn't be customized per client. What we do usually is using a separated branch tree (which is not per client) for Framework, cross-cutting, low level services libraries. Then reference these projects in the per client version projects.
My advice for you is using Nuget for these libraries and create nuget package for them, as it's the best way to define versionned dependencies. Defining a Nuget package is really easy, as well as setting up a local Nuget server.
I just worried that with 30 or 40 versions (most of which aren't that different) branching was adding complexity.
+1 Great question, its more of a business decision you'll have to make:
Do I want a neat code-base where maintenance is easy and features and fixes get rolled out quickly to all our customers
or do I want a plethora of instances of one codebase split up, each with tiny tweaks that is hard (EDIT: unless your a ALM MVP who can "unbrand" things) to merged into a trunk.
I agree with almost everthing #Nockawa mentioned except IMHO dont substitute extending your code architecture with branches.
Definitely use a branch/trunk strategy but as you mentioned too many branches makes it harder to quickly roll-out site wide features and hinder project-wide continuous integration. If you wish to prevent copy/pasting limit the number of branches.
In terms of a coding solution here is what I believe you are looking for:
Modules/Plug-ins, Interfaces and DI is right on target!
Deriving custom classes off base ones (extending the DSL per customer, Assembly.Load())
Custom reporting solution (instead of new pages a lot of custom requests could be reports)
Pages with spreadsheets (hehe I know - but funnily enough it works!)
Great examples of the module/plugin point are CMS's such as DotNetNuke or Kentico. Other idea's could be gained by looking at Facebook's add-in architecture, plugin's for audio and video editing, 3D modeling apps (like 3DMax) and games that let you build your own levels.
The ideal solution would be a admin app that you can choose your
modules (DLL's), tailor the CSS (skin), script the dB, and auto-deploy
the solution upto Azure. To acheive this goal plugin's would make so
much more sense, the codebase wont be split up. Also when an
enhancement is done to a module - you can roll it out to all your
clients.
You could easily do small customisations such as additional properties on domain model, viewmodel and view etc with user controls, derived classes and function overrides.
Do it really generically, say a customer says I want to a label that tally's everyone's age in the system, make a function called int SumOfField(string dBFieldName, string whereClause) and then for that customers site have a label that binds to the function. Then say another customer wants a function to count the number of product purchases by customer, you can re-use it: SumOfField("product.itemCount","CustomerID=1").
More significant changes that require entirely new domain models and controllers etc would fit the plug-in architecture. An example might be a customer needs a second address field, you would tweak your current Address user-control to be a plug-in to any page, it would have settings to know which dB table and fields it can implement its interface to CRUD operations.
If the functionality is customised per client in 30-40 branches
maintainability will become so hard as I get the feeling you wont be
able to merge them together (easily). If there is a chance this will
get really big you dont want to manage 275 branches. However, if its
that specialised you have to go down to the User-Control level for
each client and "users cant design their own pages" then having
Nockawa 's branching strategy for the front-end is perfectly
reasonable.
If we need to issue a bug patch that does not include current development that has been committed, or any changes from their current version, what should be done to make the process safer and with lower overhead?
We are currently using Subversion for our source control in a small (3 developers) team primarily developing in Visual Studio 2008. We anticipate that the team may group to 8 developers over the next year, and for any previous release support to become more complicated. While most customers are on the current release, some are further behind.
Source control can handle this pretty easily, and was designed for this.
When you reach the stabilization period of your release a branch should be done. It is important that you do not start any work on the next release before this is done.
Any bug fixes for that release should be done in that branch. This prevents new code from an upcoming release from polluting the bug fix. Once the bug fix is done then you can merge that change down to the trunk, and any other releases as necessary.
Don't forget to put the bug number in the comment, as this will make keeping track of commits easier.
How about: a branch per major version, with bug fixes applied to the branch(es) as needed, and also applied (or merged) back to trunk.
Where I work, we have several projects working simultaneously. In order to avoid this issue, we have several variants of the source code. For example, the first release is Variant 1.0. We create a branch off of this release, say, Variant 2.0, for all future development. If we need to do a bug fix, we do it on the main Variant, which is currently 1.0 and can release that. When Variant 2.0 is ready to go to production, we merge it with whatever is on the main branch (in this case, 1.1) and that becomes the new main trunk. At one point, we had 4 branches running at the same time.
Merging the code can be time consuming, and you have to be careful not to introduce new bugs during the merge, but if you have a decent code comparison tool available, then it shouldn't be too bad. We did a merge a while back using Beyond Compare on a 10,000 file source directory, and it took a single morning.