How to properly reference a library in nuget-based architecture - visual-studio

I'm trying to create couple nuget packages for internal company use:
OurCompany.Infrastructure.Logging.Interfaces and just OurCompany.Infrastructure.Logging.
So I created two class library projects in the same solution, provided an implementation for OurCompany.Infrastructure.Logging.Interfaces and created a package.
Now I have a question: how to properly reference the OurCompany.Infrastructure.Logging.Interfaces library in OurCompany.Infrastructure.Logging project?
Should I add it as a nuget package or simply add as a project reference?

There is no perfect solution.
When you use a NuGet package but need to make a change in the library while working on an app that uses it, you need to make the change in the library, publish a new version of the package, update your reference in your app, then hope the change you made in the library worked as needed. Particularly when working on a new feature this is very slow because your software design often changes multiple times between when you first start to when you finish, and debugging is more difficult. If your company doesn't have tooling to check for updates and remind teams that updates are available, then you can get into a fragmentation situation with some teams still using very old versions of the library and if they try to upgrade their app breaks and need extra time to adapt to the breaking changes in the shaed library.
On the other hand, if you use project references, debugging and implementing new features is easier, and breaking changes are detected immeidately, assuming you have adequite automated tests, but it requires all the code to be in the same repository, or to look like it does (maybe using git submodules, or have a convention where multiple repos are always clonsed in the same relative path).
Both Google and Microsoft had to implement their own source control system tools to deal with massive repos that are too big for a single developer machine to work with, plus speed up compile times both of dev boxes and on CI agents, because you don't want changing 1 line of code in one app causing every single app the company has being rebuilt and tested. But unless your company wants to dedicate engineers to working on the build system and not contributing to customer applications, this is infeasible.
So, the submodule, or multiple-repos with relative path conventions sounds appealing, but it still requires the library to be recompiled on every CI build that uses the library. Not a big deal, but if the frequency of changing the shared library is very low, there's little benefit to making code changes to the library easy. Plus when someone else updates the library, how will you know that you need to update the submodule? The NuGet tooling is probably better for checking for updates and actually updating.
Personally, I always use project references references for projects that are already in the same solution and nuget references for everything else. What I've done in the past is when I need to either bug fix or add a new feature to the packaged library, I clone both repos side, temporarily add the library's project(s) to my app's solution and change the nuget references to project references. Then I develop as if they're all in one repo. Finally, I need to manually undo the solution/project reference changes and increment the package reference version number before check in. Since it's manual it sometimes goes wrong. But it's the best balance I've found so far. You need to decide on what's best for yor own situation.

The best solution is: Do Both!
With the Choose directive, you can set conditional referencing:
<Choose>
<When Condition="'$(Configuration)'=='Debug'">
<ItemGroup>
<ProjectReference Include="Path\To\Your.Awesome.Package.csproj" />
</ItemGroup>
</When>
<Otherwise>
<ItemGroup>
<PackageReference Include="Your.Awesome.Package" Version="1.0.0" />
</ItemGroup>
</Otherwise>
</Choose>
In this example, your project will use the direct project reference while running in Debug mode, then, the rest of the time it will use the NuGet package reference.
Even though you are still responsible for maintaining the proper NuGet package version, nothing is perfect.
I hope this would help somebody.

Related

Include un-referenced projects into nuget package

I'm having issues creating a nuget package for a repo I've recently taken over (with no helpful input from the previous owner). The problem I'm having, is the VS nuget package creation doesn't seem to have any clear way to include other projects into the package. The way the package was built previously, was there's a base nuget package (package), that also included several other dlls from package.iOS, package.Droid, etc... But I can't figure out how to built it that way. The only other solution I can think of (which if I'm reading the nuget documentation on Microsoft correctly, it's the more correct way) is to create a separate nuget package for each of the projects, instead of including them all in the base package.
Please advise.
I am fully comfortable with creating multiple more specific nuget packages, but the people using this library are used to only downloading the base one, and it having everything they need.
I've looked into the documentation, and as far as I was able to tell, I could include referenced projects, but the base project doesn't rely on any of the other ones.
Just for reference, the project structure is similar to:
project
project.forms
project.iOS
project.Droid
project.macOS
project.Uwp
Where previous versions of the nuget built from project also included project.iOS, project.Droid, project.macOS, and project.Uwp, which in turn, depend on project.
Logically, to me, it seems that each of the project.{}s should probably have their own package, as they all may not be needed in someone's utilization of the package. Just wanted to see if there was an easy way to continue existing patterns without a load of work.
Scratch that
Turns out I can't make individual packages out of the project.{} projects through VS.
Okay, found it, just had to make a .nuspec file, and reference that file in the nuget pack command.
In the .nuspec file, you need a <files> node with a <file src="...thing.dll" target="lib\..." /> for each of the dll's you're needing in your project. Simple as that. Just wish there were a way to do it in the VS interface, instead of having to use a .nuspec file.

How to make successive version of a project in Microsoft Visual Studio 2010

Maybe I just can't figure out the right keywords to get an answer out of Google, but here goes.
Say I have a Project I'm working on named "Project." For it's first version, I have it stored in the folder "Project_Version1", and have the name of it's solution, project, build exe, etc as "Project_Version1".
Now I want to make the next version of the Project, and call it "Project_Version2". To do this currently, I copy the original folder, and rename it to "Project_Version2", and I want to rename all of the other internal stuff to that as well. Currently, I have to do that with a combination of changing the names in windows explorer, and some of the various properties pages in my solution.
There has to be a better way to do this. How does somebody make a second version of their project, and store it's files separately from the first version? Is there a way to also rename appropriate files that contain version numbers?
Indeed, there is a better way to do this, it is called version control.
If you check your source into TFS (or git or any other SCM software) all versions will be tracked over time.
If you then need to maintain multiple versions simultaneously (e.g. for a customer base where customers slowly migrate to the bleeding edge and patches are required to fix critical issues in older, supported versions) you can keep these versions as branches and selectively merge changes.
If you do decide source control is a good fit for managing your versions (It brings a lot of other benefits as well), consider reading some posts such as this git tutorial or why use version control.
If you REALLY can't use source control, for some reason, I suggest trying to reference as much as you can using language constructs which can be easily changed (e.g. with C# and a reasonable IDE, a quick refactor, or with other languages some rigorous de-duplication of version-specific data).
Ideally you would have just one definition of the version for your code along with a version definition in the project files (which could be set as a build variable to change quickly).
In this case you would copy the project, change the code version and the version variable for the project and, assuming good use of relative paths etc. (no hardcoded versions!), a quick rename of a couple of variables should get you going.
Ideally, the bulk of of the file-names would not reference the version. The only thing referencing the version should probably be the containing directory, the project/solution files and a few isolated references in your code.
You can do some pretty neat tricks with MSBuild. Let's say, for instance, that this is your project.
Program.cs:
namespace MyCustomBuild
{
class Program
{
static void Main(string[] args)
{
System.Console.WriteLine("Hello World");
}
}
}
Create your MSBuild file as shown. build.msbuild:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<MajorVersion>1</MajorVersion>
<MinorVersion>0</MinorVersion>
</PropertyGroup>
<Target Name="Default">
<CombinePath BasePath="$(MSBuildProjectDirectory)"
Paths="Program.$(MajorVersion).$(MinorVersion)">
<Output TaskParameter="CombinedPaths" PropertyName="OutputDir"/>
</CombinePath>
<MakeDir Directories="$(OutputDir)"/>
<Csc Sources="program.cs"
OutputAssembly="$(OutputDir)\Program.$(MajorVersion).$(MinorVersion).exe"/>
</Target>
</Project>
After running msbuild build.msbuild, this is the result:
Ways to make what I posted even better:
Use the MSBuild Extension Pack AssemblyInfo. This will allow you to change the version info if you have it defined inside an AssemblyInfo.cs file. Or you can look into the GenerateApplicationManifest Task, which can set assembly versions, but as of right now the file version of the executable will always be 1.0.0.0
Implement some sort of autoincrement system like this one
You can copy all your source files into the versioned directory using the Copy Task
It's a lot to take in, especially if you aren't familiar with MSBuild, but it's a powerful program that allows customization that Visual Studio can't even imagine! Note that a solution file (.sln) is a MSBuild file, even though it's not xml.
Hopefully this is what you are looking for. I do also recommend that you use an official version control, as others have mentioned, but you seemed to have a specific task you wanted automated, which is MSBuild's forte.
Ideally, versioning your project should be done within a source control system since Visual Studio in and of itself is just an IDE.
In the situation of not using a source control system, unfortunately I'm not aware of a better solution than what you are doing currently.

Visual Studio 2008, MSBuild: "replacement" projects

My solution has a library project which needs a special environment to be built (lots of external libraries and tools)... but it is not vital to our application. We'd like to avoid installing these tools when not necessary (most of our developers work on other parts of code).
We have created another project which has the same API, but has an empty implementation and is compilable without those external tools. I'd like to be able to easily switch between those projects and still get all the references in other projects correct.
I don't know VS/MSBuild very well, but willing to learn whatever is necessary. Is it possible? I am looking for ideas... We're using Subversion, and solutions involving some hacks inside VCS are also welcome.
It sounds as if your library project is one that can be separated from your primary solution, taking the tool baggage with it. Doing that, you could build the speciality solution separately, an link the compiled assembly from the main solution.
Create another build-configuration for your project.
So you will have at least 2 build-configurations e.g. Debug_SpecialNeeds and Debug.
For discussion, I'll assume you have a project directory containing your solution file, a "RealLibrary\RealLibrary.csproj" project file (your "real" library, with the dependencies), and a "MockLibrary\MockLibrary.csproj" file (your "mock" library, with the empty implementations).
If I understand correctly, you want to easily "swap" the MockLibrary for the RealLibrary in your solution, and vice-versa.
The easiest/hackiest way to do this, assuming your solution (and dependent projects) are configured to look for the "RealLibrary.csproj" project, is to rename the "RealLibrary" directory (it doesn't matter to what), and rename the "MockLibrary" directory to "RealLibrary" and rename "MockLibrary.csproj" to "RealLibrary.csproj". This will effectively "trick" your solution and dependent projects into loading the "mock library" even though they are referencing the "real library".
A slightly more complex (and perhaps cleaner) solution is to actually modify your "sln" and "csproj" files to reference "MockLibrary.csproj" instead of "RealLibrary.csproj". In the "sln" file, you'll need to change the path to the project in the section near the top:
Microsoft Visual Studio Solution File, Format Version 10.00
# Visual Studio 2008
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "RealLibrary", "RealLibrary\RealLibrary.csproj", "{E1714F9A-E1D9-4132-A561-AE2B4919391C}"
EndProject
You need to change that path "RealLibrary\RealLibrary.csproj" to "MockLibrary\MockLibrary.csproj". If you're going for completeness, you can change the name as well (or perhaps just use a generic name like "Library" for the name).
Likewise, in the dependent csproj files, you'll need to find all instances of the "ProjectReference" node where you reference "RealLibrary.csproj" and modify the path. These sections look like this:
<ProjectReference Include="..\RealLibrary\RealLibrary.csproj">
<Project>{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</Project>
<Name>RealLibrary</Name>
</ProjectReference>
You could relatively easily write some scripts to perform this swap. However, I think there's a deeper problem here that can be addressed more directly. I'll post that as a separate answer, but I wanted you to have the actual answer you were looking for first.
The deeper problem I see here is that your library "needs a special environment to be built", specifically because it depends on "lots of external libraries and tools". I would suggest that you NOT go down the path of creating the mock library, but instead focus on getting the library to build correctly without a special environment. You can achieve this by including all of those dependencies in source control along with your project, and reference those dependencies via relative paths inside your working copy. In my build environments, I try to avoid static environmental dependencies as much as possible (ideally limiting it just to the .NET framework itself).
To get the dependencies into source control, you can either check them directly into the project itself, or you can check them into a different location and then "reference" them in your project via svn:external definitions. In my environment, I have a separate "bin" repository used just for these kind of third party library dependencies, and then many dependent projects can pull them in via externals.
If you can eliminate your library's build-time environmental dependencies, your build will be much more robust and it will be much easier for developers to work with the project.

How do you share external dependencies between Visual Studio solutions?

I have a Java background so I’m used to having Maven handle all problem around downloading and keeping dependencies up to date. But in the .NET environment I have not yet found a good way to manage all these external dependencies.
The main problem here is that I mass produce solutions and they all tend to depend on the same third party dll’s. But I don’t want to maintain separate copies of each component under each solution. So I need a way of linking all the different solutions to the same set of dll’s.
I realized that one solution might be to include the external libraries in a ”library project” that is included in all solutions and let the other projects references them through it. (Or just make sure to reference the external dll’s from the same place for all projects.)
But are there any better ways to do this?
(Preferably using some sort of plug-in for Visual Studio.)
I’ve looked at the Visual Studio Dependency Manager and it seems like a perfect match but have anyone tried it for real? I’ve also seen the .NET ports of Maven, but unfortunately I was not too impressed by the status of those. (But please go ahead and recommend them anyone if you think I should give them another try.)
So what would be the smartest way to tackle this problem?
Update:
I realized that I needed to explain what I meant with linking to the same set of dll’s.
One of the things I'm trying to achieve here is to avoid that the different solutions are referencing different versions of each component. If I update a component to a new version, it should be updated for all solutions upon next build. This would force me to make sure all solutions are up to date with the latest components.
Update 2:
Note that this is an old question asked before tools like NuGet or OpenWrap existed. If anyone is willing to provide a more up-to-date, please go ahead and I will change the accepted answer.
Find some place to store the assemblies. For example, I store the .Net core assemblies like so:
<branch>\NetFX\2.0527\*
<branch>\NetFX\3.0\*
<branch>\NetFX\3.5\*
<branch>\NetFX\Silverlight 2\*
<branch>\NetFX\Silverlight 3\*
Use the ReferencePath property in MSBuild (or AdditionalReferencePath in Team Build) to point your projects at the appropriate paths. For simplicity and easy maintenance, I have 1 *.targets file that knows about every such directory; all of my projects Import that file.
Make sure your version control strategy (branching, merging, local<->server mappings) keeps the relative paths between your projects & your reference paths constant.
EDIT
In response to the update in the question, let me add one more step:
4) Make sure every assembly reference in every project file uses the full .Net strong name and nothing else.
Bad:
<Reference Include="Microsoft.SqlServer.Smo">
<SpecificVersion`>False</SpecificVersion>
<HintPath>..\..\..\..\..\..\..\Program Files (x86)\Microsoft SQL Server\100\Shared\Microsoft.SqlServer.Smo.dll</HintPath>
</Reference>
Good:
<Reference Include="Microsoft.SqlServer.Smo, Version=10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91, processorArchitecture=MSIL" />
Advantages of the latter format:
Using a HintPath in a collaborative development environment will inevitably lead to situations where "it works for me" but not others. Especially your build server. Omitting it forces you to get your reference paths correct or it won't compile.
Using a weak name invites the possibility of "DLL hell." Once you use strong names then it's safe to have multiple versions of the same assembly in your reference paths because the linker will only load ones that match every criterion. In addition, if you decide to update some assemblies in place (instead of adding copies), then you'll be notified of any breaking changes at compile time instead of whenever the bugs start coming in.
Adding to what everybody else is saying, it basically comes down to two things:
Making sure that all developers have the same versions of external libraries
Making sure that all developers have the external libraries located in the same place (at least, relative to the source code)
As Richard Berg points out, you can use ReferencePath and/or AdditionalReferencePath to help solve #2. If you're using msbuild in your build process (in our case, we're using CruiseControl instead of MS Team Build), you can also pass ReferencePath to it on the command line. To solve #1, I've found svn:externals to be useful (if you're using SVN).
My experience with Maven is that it's way overkill for most purposes.
I usually have a separate folder structure on the source control for extrenal or Internal dependencies, and these filders have the assemblies according to build or version number for example
public\External\libraries\Nunit\2.6\
or
Public\Internal\libraries\Logger\5.4.312\
and inside the solutions all the projects that need to use any of the dependencies just adds a reference to that assemblies in the public internal or extrenal folders.

Structuring projects & dependencies of large winforms applications in C#

UPDATE:
This is one of my most-visited questions, and yet I still haven't really found a satisfactory solution for my project. One idea I read in an answer to another question is to create a tool which can build solutions 'on the fly' for projects that you pick from a list. I have yet to try that though.
How do you structure a very large application?
Multiple smallish projects/assemblies in one big solution?
A few big projects?
One solution per project?
And how do you manage dependencies in the case where you don't have one solution.
Note: I'm looking for advice based on experience, not answers you found on Google (I can do that myself).
I'm currently working on an application which has upward of 80 dlls, each in its own solution. Managing the dependencies is almost a full time job. There is a custom in-house 'source control' with added functionality for copying dependency dlls all over the place. Seems like a sub-optimum solution to me, but is there a better way? Working on a solution with 80 projects would be pretty rough in practice, I fear.
(Context: winforms, not web)
EDIT: (If you think this is a different question, leave me a comment)
It seems to me that there are interdependencies between:
Project/Solution structure for an application
Folder/File structure
Branch structure for source control (if you use branching)
But I have great difficulty separating these out to consider them individually, if that is even possible.
I have asked another related question here.
Source Control
We have 20 or 30 projects being built into 4 or 5 discrete solutions. We are using Subversion for SCM.
1) We have one tree in SVN containing all the projects organised logically by namespace and project name. There is a .sln at the root that will build them all, but that is not a requirement.
2) For each actual solution we have a new trunks folder in SVN with SVN:External references to all the required projects so that they get updated from their locations under the main tree.
3) In each solution is the .sln file plus a few other required files, plus any code that is unique to that solution and not shared across solutions.
Having many smaller projects is a bit of a pain at times (for example the TortoiseSVN update messages get messy with all those external links) but does have the huge advantage that dependancies are not allowed to be circular, so our UI projects depend on the BO projects but the BO projects cannot reference the UI (and nor should they!).
Architecture
We have completely switched over to using MS SCSF and CAB enterprise pattern to manage the way our various projects combine and interact in a Win Forms interface. I am unsure if you have the same problems (multiple modules need to share space in a common forms environment) but if you do then this may well bring some sanity and convention to how you architect and assemble your solutions.
I mention that because SCSF tends to merge BO and UI type functions into the same module, whereas previously we maintained a strict 3 level policy:
FW - Framework code. Code whose function relates to software concerns.
BO - Business Objects. Code whose function relates to problem domain concerns.
UI - Code which relates to the UI.
In that scenario dependancies are strictly UI -> BO -> FW
We have found that we can maintain that structure even while using SCSF generated modules so all is good in the world :-)
To manage dependencies, whatever the number of assemblies/namespaces/projects you have, you can have a glance at the tool NDepend.
Personnaly, I foster few large projects, within one or several solutions if needed. I wrote about my motivations to do so here: Benefit from the C# and VB.NET compilers perf
I think it's quite important that you have a solution that contains all your 80 projects, even if most developers use other solutions most of the time. In my experience, I tend to work with one large solution, but to avoid the pain of rebuilding all the projects each time I hit F5, I go to Solution Explorer, right-click on the projects I'm not interested in right now, and do "Unload Project". That way, the project stays in the solution but it doesn't cost me anything.
Having said that, 80 is a large number. Depending on how well those 80 break down into dicrete subsystems, I might also create other solution files that each contain a meaningful subset. That would save me the effort of lots of right-click/Unload operations. Nevertheless, the fact that you'd have one big solution means there's always a definitive view of their inter-dependencies.
In all the source control systems that I've worked with, their VS integration chooses to put the .sln file in source control, and many don't work properly unless that .sln file is in source control. I find that intriguing, since the .sln file used to be considered a personal thing, rather than a project-wide thing. I think the only kind of .sln file that definitely merits source control is the "one-big-solution" that contains all projects. You can use it for automated builds, for example. As I said, individuals might create their own solutions for convenience, and I'm not against those going into source control, but they're more meaningful to individuals than to the project.
I think the best solution is to break it in to smaller solutions. At the company I currently work for, we have the same problem; 80 projects++ in on solution. What we have done, is to split into several smaller solutions with projects belonging together. Dependent dll's from other projects are built and linked in to the project and checked in to the source control system together with the project. It uses more disk space, but disk is cheap. Doing it this way, we can stay with version 1 of a project until upgrading to version 1.5 is absolutely necessary. You still have the job with adding dll's when deciding to upgrade to a other version of the dll though. There is a project on google code called TreeFrog that shows how to structure the solution and development tree. It doesn't contain mush documentation yet, but I guess you can get a idea of how to do it by looking at the structure.
A method that i've seen work well is having one big solution which contains all the projects, for allowing a project wide build to be tested (No one really used this to build on though as it was too big.), and then having smaller projects for developers to use which had various related projects grouped together.
These did have depencies on other projects but, unless the interfaces changed, or they needed to update the version of the dll they were using, they could continue to use the smaller projects without worrying about everything else.
Thus they could check-in projects while they were working on them, and then pin them (after changing the version number), when other users should start using them.
Finally once or twice a week or even more frequently the entire solution was rebuild using pinned code only, thus checking if the integration was working correctly, and giving testers a good build to test against.
We often found that huge sections of code didn't change frequently, so it was pointless loading it all the time. (When you're working on the smaller projects.)
Another advantage of using this approach is in certain cases we had pieces of functionality which took months to complete, by using the above approach meant this could continue without interrupting other streams of work.
I guess one key criteria for this is not having lots of cross dependencies all over your solutions, if you do, this approach might not be appropriate, if however the dependencies are more limited, then this might be the way to go.
For a couple of systems I've worked on we had different solutions for different components. Each solution had a common Output folder (with Debug and Release sub-folders)
We used project references within a solution and file references between them. Each project used Reference Paths to locate the assemblies from other solutions. We had to manually edit the .csproj.user files to add a $(Configuration) msbuild variable to the reference paths as VS insists on validating the path.
For builds outside of VS I've written msbuild scripts that recursively identify project dependencies, fetch them from subversion and build them.
I gave up on project references (although your macros sound wonderful) for the following reasons:
It wasn't easy to switch between different solutions where sometimes dependency projects existed and sometimes didn't.
Needed to be able to open the project by itself and build it, and deploy it independently from other projects. If built with project references, this sometimes caused issues with deployment, because a project reference caused it to look for a specific version or higher, or something like that. It limited the mix and match ability to swap in and out different versions of dependencies.
Also, I had projects pointing to different .NET Framework versions, and so a true project reference wasn't always happening anyways.
(FYI, everything I have done is for VB.NET, so not sure if any subtle difference in behavior for C#)
So, I:
I build against any project that is open in the solution, and those that aren't, from a global folder, like C:\GlobalAssemblies
My continuous integration server keeps this up to date on a network share, and I have a batch file to sync anything new to my local folder.
I have another local folder like C:\GlobalAssembliesDebug where each project has a post build step that copies its bin folder's contents to this debug folder, only when in DEBUG mode.
Each project has these two global folders added to their reference paths. (First the C:\GlobalAssembliesDebug, and then C:\GlobalAssemblies). I have to manually add this reference paths to the .vbproj files, because Visual Studio's UI addes them to the .vbprojuser file instead.
I have a pre-build step that, if in RELEASE mode, deletes the contents from C:\GlobalAssembliesDebug.
In any project that is the host project, if there are non dlls that I need to copy (text files outputted to other project's bin folders that I need), then I put a prebuild step on that project to copy them into the host project.
I have to manually specify the project dependencies in the solution properties, to get them to build in the correct order.
So, what this does is:
Allows me to use projects in any solution without messing around with project references.
Visual Studio still lets me step into dependency projects that are open in the solution.
In DEBUG mode, it builds against open loaded projects. So, first it looks to the C:\GlobalAssembliesDebug, then if not there, to C:\GlobalAssemblies
In RELEASE mode, since it deletes everything from C:\GlobalAssembliesDebug, it only looks to C:\GlobalAssemblies. The reason I want this is so that released builds aren't built against anything that was temporarily changed in my solution.
It is easy to load and unload projects without much effort.
Of course, it isn't perfect. The debugging experience is not as nice as a project reference. (Can't do things like "go to definition" and have it work right), and some other little quirky things.
Anyways, that's where I am on my attempt to make things work for the best for us.
We have one gigantic solution on the source control, on the main branch.
But, every developer/team working on the smaller part of the project, has its own branch which contains one solution with only few projects which are needed. In that way, that solution is small enough to be easily maintenaced, and do not influence on the other projects/dlls in the larger solution.
However, there is one condition for this: there shouldn't be too much interconnected projects within solution.
OK, having digested this information, and also answers to this question about project references, I'm currently working with this configuration, which seems to 'work for me':
One big solution, containing the application project and all the dependency assembly projects
I've kept all project references, with some extra tweaking of manual dependencies (right click on project) for some dynamically instantiated assemblies.
I've got three Solution folders (_Working, Synchronised and Xternal) - given that my source control isn't integrated with VS (sob), this allows me to quickly drag and drop projects between _Working and Synchronised so I don't lose track of changes. The XTernal folder is for assemblies that 'belong' to colleagues.
I've created myself a 'WorkingSetOnly' configuration (last option in Debug/Release drop-down), which allows me to limit the projects which are rebuilt on F5/F6.
As far as disk is concerned, I have all my projects folders in just one of a few folders (so just one level of categorisation above projects)
All projects build (dll, pdb & xml) to the same output folder, and have the same folder as a reference path. (And all references are set to Don't copy) - this leaves me the choice of dropping a project from my solution and easily switching to file reference (I've got a macro for that).
At the same level as my 'Projects' folder, I have a 'Solutions' folder, where I maintain individual solutions for some assemblies - together with Test code (for example) and documentation/design etc specific to the assembly.
This configuration seems to be working ok for me at the moment, but the big test will be trying to sell it to my colleagues, and seeing if it will fly as a team setup.
Currently unresolved drawbacks:
I still have a problem with the individual assembly solutions, as I don't always want to include all the dependent projects. This creates a conflict with the 'master' solution. I've worked around this with (again) a macro which converts broken project references to file references, and restores file references to project references if the project is added back.
There's unfortunately no way (that I've found so far) of linking Build Configuration to Solution Folders - it would be useful to be able to say 'build everything in this folder' - as it stands, I have to update this by hand (painful, and easy to forget). (You can right click on a Solution Folder to build, but that doesn't handle the F5 scenario)
There is a (minor) bug in the Solution folder implementation which means that when you re-open a solution, the projects are shown in the order they were added, and not in alphabetical order. (I've opened a bug with MS, apparently now corrected, but I guess for VS2010)
I had to uninstall the CodeRushXPress add-in, because it was choking on all that code, but this was before having modified the build config, so I'm going to give it another try.
Summary - things I didn't know before asking this question which have proved useful:
Use of solution folders to organise solutions without messing with disk
Creation of build configurations to exclude some projects
Being able to manually define dependencies between projects, even if they are using file references
This is my most popular question, so I hope this answer helps readers. I'm still very interested in further feedback from other users.

Resources