Related
We have a few hundred visual studio project files that I need to assemble into a solution for building. We currently have a custom ruby script, that uses rake, to do this. But is fragile, and only allows a few visual studio macros ( $(TargetDir),$(TargetName), etc...) through, and failing on the rest. Plus the grammar of Ruby rubs me like Perl: The wrong way.
So my question is, given a directory is there a tool that will recursively find all all the .vcxproj and .csproj files and generate a solution file with dependencies? When I say 'with dependencies' it means that some projects need to be built before others. I found some other posts here on stack overflow that pointed to a tool that generates solution files: but it doesn't generate dependencies. Therefore without dependencies any solution creation tool is completely useless. Does anyone know of something that will do this?
If not a solution file, does anyone know of something that will just emit a dependency list?
P.S.
And before anyone asks: creating a solution file manually is completely out of the question. We simply have way too many project files.
So my question is, given a directory
is there a tool that will recursively
find all all the .vcxproj and .csproj
files and generate a solution file
with dependencies?
No.
What you're asking for is very reasonable; your approach to the problem is quite rational. Unfortunately, the tools haven't kept up with you. (We had the same problem.)
You're going to have to script that yourself, or otherwise customize tools. That's what we did. Successful approaches I've seen include:
Generate the *.vcproj/*.sln from
"reference project definitions",
using tools like CMake, QMake, Scons, or
Gyp. Our main system currently sits
on Scons, with our custom Python
code to navigate these dependencies,
generate solutions based on projects
(spidering dependencies). By
default, we generate a "complete"
solution for each project (including
all required supporting projects),
plus a "Master All Projects"
solution. It works very well. But,
it was custom work that took effort,
and we extended Scons somewhat to
describe our projects (but we simply
rely on the Scons generation of
*.sln and *.vcproj).
Write a custom tool to "find" these dependencies by
parsing all the *.vcproj files in
your workspace. This is work, but can be done. Those files can be "tricky" to navigate, but you might be fine with a "good enough" solution that uses the GUIDs as hash keys to generate those dependencies.
I totally agree with you: This type of stuff (project dependencies) is prohibitively difficult to maintain manually when you move beyond "simple" (e.g., many dozens of projects, yes, we also have hundreds).
Sorry. MSVS is a pretty good IDE (intended for iterative development), and a terrible build configuration management system, and not designed to do what we're talking about.
Because I care about your sanity and Your Everlasting Soul, please Please PLEASE do not attempt to write your custom solution in MSBuild.
On a side note, having hundreds of VS projects is a bad idea, it will kill VS performances, see the two white-books:
Partitioning code base through .NET assemblies and Visual Studio projects (8 pages)
Defining .NET Components with Namespaces (7 pages)
I've inherited some C99 code that I'm planing on reusing in a C++-centric solution. Unfortunately, even Microsoft's latest compiler has virtually no support for non-trivial C99 features.
The code in question has been tested to death and I'd rather not go through the trouble of rewriting it in C++. This means that in order for me to reuse the code verbatim I'll have to rely on a conforming 3rd-party compiler.
After looking around, it appears that the nicest way for me to integrate this code is by adding a "Makefile Project" to my solution. Only one problem. It seems like it is now my responsibility to keep the "Build Command Line" property synchronized with the files that I add to the project through Visual Studio.
At first glance, I couldn't find a way to get a list of files in my project through the usual Visual Studio $()-style macros. I could always write a shell script that would enumerate *.c files in my source tree and pass their paths to the 3rd-party compiler. However, I kind of expected that Visual Studio would do at least that part of the work for me since it already has this information in the relevant *.vc[x]proj file.
It is very unlikely that I'll need to add any new source files to this project, but still, manual synchronization (i.e., without a script) of this sort seems rather fragile to me.
What are my options besides writing a helper script?
From Hans Passant:
"Makefile project" means what it says,
there needs to be another 'agent'
that's responsible for the
dependencies. Like a make file. Rule
files can help you select another
build tool but that's kinda broken
right now in VS2010. Leverage the
original tool that built this C99
code, run it from the makefile
project.
I have a C# project which includes one exe and 11 library files. The exe references all the libraries, and lib1 may reference lib2, lib3, lib4, etc.
If I make a change to a class in lib1 and built the solution, I assumed that only lib1 and the exe would need to be changed. However, all dll's and the exe are being built if I want to run the solution.
Is there a way that I can stop the dependencies from being built if they have not been changed?
Is the key this phrase? "However, all dll's and the exe are being built if I want to run the solution"
Visual Studio will always try to build everything when you run a single project, even if that project doesn't depend on everything. This choice can be changed, however. Go to Tools|Options|Projects and Solutions|Build and Run and check the box "Only build startup projects and dependencies on Run". Then when you hit F5, VS will only build your startup project and the DLLs it depends on.
I just "fixed" the same problem with my VS project. Visual Studio did always a rebuild, even if didn't change anything. My Solution: One cs-File had a future timestamp (Year 2015, this was my fault). I opened the file, saved it and my problem was solved!!!
I am not sure if there is a way to avoid dependencies from being built. You can find some info here like setting copylocal to false and putting the dlls in a common directory.
Optimizing Visual Studio solution build - where to put DLL files?
We had a similar problem at work. In post-build events we were manually embedding manifests into the outputs in the bin directory. Visual Studio was copying project references from the obj dir (which weren't modified). The timestamp difference triggered unnecessary rebuilds.
If your post-build events modify project outputs then either modify the outputs in the bin and obj dir OR copy the modified outputs in the bin dir on top of those in the obj dir.
You can uncheck the build option for specified projects in your Solution configuration:
(source: microsoft.com)
You can can create your own solution configurations to build specific project configurations...
(source: microsoft.com)
We actually had this problem on my current project, in our scenario even running unit tests (without any code changes) was causing a recompile. Check your build configuration's "Platform".
If you are using "Any CPU" then for some reason it rebuilds all projects regardless of changes. Try using processor specific builds, i.e. x86 or x64 (use the platform which is specific to the machine architecture of your machine). Worked for us for x86 builds.
(source: episerver.com)
Now, after I say this, some propeller-head is going to come along and contradict me, but there is no way to do what you want to do from Visual Studio. There is a way of doing it outside of VS, but first, I have a question:
Why on earth would you want to do this? Maybe you're trying to save CPU cycles, or save compile time, but if you do what you're suggesting you will suddenly find yourself in a marvelous position to shoot yourself in the foot. If you have a library 1 that depends upon library 2, and only library 2 changes, you may think you're OK to only build the changed library, but one of these days you are going to make a change to library 2 that will break library 1, and without a build of library 2 you will not catch it in the compilation. So in my humble opinion, DON'T DO IT.
The reason this won't work in VS2005 and 2008 is because VS uses MSBuild. MSBuild runs against project files, and it will examine the project's references and build all referenced projects first, if their source has changed, before building the target project. You can test this yourself by running MSBuild from the command line against one project that has not changed but with a referenced project that has changed. Example:
msbuild ClassLibrary4.csproj
where ClassLibrary4 has not changed, but it references ClassLibrary5, which has changed. MSBuild will build lib 5 first, before it builds 4, even though you didn't mention 5.
The only way to get around all these failsafes is to use the compiler directly instead of going through MSBuild. Ugly, ugly, but that's it. You will basically be reduced to re-implementing MSBuild in some form in order to do what you want to do.
It isn't worth it.
Check out the following site for more detailed information on when a project is built as well as the differences between build and rebuild.
I had this problem too, and noticed these warning messages when building on Windows 7 x64, VS2008 SP1:
cl : Command line warning D9038 : /ZI is not supported on this platform; enabling /Zi instead
cl : Command line warning D9007 : '/Gm' requires '/Zi'; option ignored
I changed my project properties to:
C/C++ -> General -> Debug Information Format = /Zi
C/C++ -> Code Generation -> Enable Minimal Build = No
After rebuilding I switched them both back and dependencies work fine again. But prior to that no amount of cleaning, rebuilding, or completely deleting the output directory would fix it.
I don't think there's away for you to do it out of the box in VS. You need this add-in
http://workspacewhiz.com/
It's not free but you can evaluate it before you buy.
Yes, exclude the non-changing bits from the solution. I say this with a caveat, as you can compile in a way where a change in build number for the changed lib can cause the non built pieces to break. This should not be the case, as long as you do not break interface, but it is quite common because most devs do not understand interface in the .NET world. It comes from not having to write IDL. :-)
As for X projcts in a solution, NO, you can't stop them from building, as the system sees a dependency has changed.
BTW, you should look at your project and figure out why your UI project (assume it is UI) references the same library as everything else. A good Dependency Model will show the class(es) that should be broken out as data objects or domain objects (I have made an assumption that the common dependency is some sort of data object or domain object, of course, but that is quite common). If the common dependency is not a domain/data object, then I would rethink my architecture in most cases. In general, you should be able to create a path from UI to data without common dependencies other than non-behavioral objects.
Not sure of an awesome way to handle this, but in the past if I had a project or two that kept getting rebuilt, and assuming I wouldn't be working in them, I would turn the build process off for them.
Right click on the sln, select configuration manager and uncheck the check boxes. Not perfect, but works when Visual Studio isn't behaving.
If you continue to experience this problem, it may be due to a missing or out of date calculated dependency (like a header) that is listed in your project, but does not exist.
This happens to me especially common after migrating to a new version (for example: from 2012 to 2013) because VS may have recalculated dependencies in the conversion, or you are migrating to a new location.
A quick check is to double-click every file in offending project from solution explorer. If you discover a file does not exist, that is your problem.
Failing a simple missing file: You may have a more complicated build date relationship between source and target. You can use a utility to find out what front-end test is triggering the build. To get that information you can enable verbose CPS logging. See: Andrew Arnott - Enable C++ and Javascript project system tracing (http://blogs.msdn.com/b/vsproject/archive/2009/07/21/enable-c-project-system-logging.aspx). I use the DebugView option. Invaluable tool when you need it.
(this is a C# specific question, but a different post was merged as identical)
I know the ideal way to build projects is without requiring IDE based project files, since it theoretically causes all sort of trouble with automation and what not. But I've yet to work on a project that compiles on Windows that doesn't depend on the VisualStudio project (Ok, obviously some Open Source stuff gets done with Cygwin, but I'm being general here).
On the other hand if we just use VS to run a makefile, we loose all the benefits of the compile options window, and it becomes a pain to maintain the external makefile.
So how do people that use VS actually handle external makefiles? I have yet to find a painless system to do this...
Or in reality most people don't do this, although its preached as good practice?
Take a look at MSBuild!
MSBuild can work with the sln/csproj files from VS, so for simple projects you can just call them directly.
if you need more control, wrap the projects in your own build process, add your own tasks etc. - it is very extensible!
(I wanted to add a sample but this edior totally messed up the XML... sorry)
Ideally perhaps, in practice no.
Makefiles would be my preference as the build master, however, the developers spend all their time inside the visual studio IDE and when they make a change, it's to the vcproj file, not the makefile. So if I'm doing the global builds with makefiles, it's too easily put out of synch with the project/solution files in play by 8 or 10 others.
The only way I can stay in step with the whole team is to run devenv.exe on the solution files directly in my build process scripts.
There are very few makefiles in my builds, where there are they are in the pre-build or custom build sections or a separate utility project.
One possibility is to use CMake - you describe with a script how you project is to be built, and CMake generates the Visual Studio solution/project files for you.
And if you need to build your project from the command line, or in a continuous integration tool, you use CMake to generate a Makefile for NMake.
And if you project is a cross-platform one - you can run CMake to generate the makefiles for the toolchain of your choice.
A simple CMake script looks like this:
project(hello)
add_executable(hello hello.cpp)
Compare these two lines with a makefile or the way you setup a simple project in your favorite IDE.
In a nutshell CMake does not only cross-platform-enables your project it also makes it cross-IDE. If you like to just test your project with eclipse or KDevelop, or codeblocks, just run CMake to generate the corresponding project files.
Well, in practice it is no always so easy, but the CMake idea just rocks.
For example, if you consider using CMake with Visual Studio there is some tweaking required to obtain the familiar VS project feeling, main obstacle is to organize your header and source files, but it is possible - check the CMake wiki (and by writting a short script you might even simplify this task).
We use a NAnt script, which at the compile step calls MSBuild. Using NAnt allows us to perform both pre- and post-build tasks, such as setting version numbers to match source control revision numbers, collating code coverage information, assembling and zipping deployment sources. But still, at the heart of it, it's MSBuild that's actually doing the compiling.
You can integrate a NAnt build as a custom tool into the IDE, so that it can be used both on a build or continuous integration server and by the developers in the same way.
Personally, I use Rake to call msbuild on my solution or project. For regular development I use the IDE and all the benefits that provides.
Rake is set up so that I can just compile, compile and run tests or compile run tests and create deployable artifacts.
Once you have a build script it is really easy to start doing things like setting up continuous integration and using it to automate deployment.
You can also use most build tools from within the IDE if you follow these steps to set it up.
We use the devenv.exe (same exe that launches the IDE) to build our projects from the build scripts (or the command line). When specifying the /Build option the IDE is not displayed and everything is written back to the console (or the log file if you specify the /Out option)
See http://msdn.microsoft.com/en-us/library/xee0c8y7(VS.80).aspx for more information
Example:
devenv.exe [solution-file-name] /Build [project-name] /Rebuild "Release|Win32" /Out solution.log
where "Release|Win32" is the configuration as defined in the solution and solution.log is the file that gets the compiler output (which is quite handy when you need to figure out what went wrong in the compile)
We have a program that parses the vcproj files and generates makefile fragments from that. (These include the list of files and the #defines, and there is some limited support for custom build steps.) These fragments are then included by a master makefile which does the usual GNU make stuff.
(This is all for one of the systems we target; its tools have no native support for Visual Studio.)
This didn't require a huge amount of work. A day to set it up, then maybe a day or two in total to beat out some problems that weren't obvious immediately. And it works fairly well: the compiler settings are controlled by the master makefile (no more fiddling with those tiny text boxes), and yet anybody can add new files and defines to the build in the usual way.
That said, the combinatorical problems inherent to Visual Studio's treatment of build configurations remain.
Why would you want to have project that "compiles on Windows that doesn't depend on the VisualStudio project"? You already have a solution file - you can just use it with console build.
I'd advise you to use msbuild with conjunction with makefile, nant or even simple batch file if your build system is not as convoluted as ours...
Is there something I'm missing?
How about this code?
public TRunner CleanOutput()
{
ScriptExecutionEnvironment.LogTaskStarted("Cleaning solution outputs");
solution.ForEachProject(
delegate (VSProjectInfo projectInfo)
{
string projectOutputPath = GetProjectOutputPath(projectInfo.ProjectName);
if (projectOutputPath == null)
return;
projectOutputPath = Path.Combine(projectInfo.ProjectDirectoryPath, projectOutputPath);
DeleteDirectory(projectOutputPath, false);
string projectObjPath = String.Format(
CultureInfo.InvariantCulture,
#"{0}\obj\{1}",
projectInfo.ProjectName,
buildConfiguration);
projectObjPath = Path.Combine(productRootDir, projectObjPath);
DeleteDirectory(projectObjPath, false);
});
ScriptExecutionEnvironment.LogTaskFinished();
return ReturnThisTRunner();
}
public TRunner CompileSolution()
{
ScriptExecutionEnvironment.LogTaskStarted ("Compiling the solution");
ProgramRunner
.AddArgument(MakePathFromRootDir(productId) + ".sln")
.AddArgument("/p:Configuration={0}", buildConfiguration)
.AddArgument("/p:Platform=Any CPU")
.AddArgument("/consoleloggerparameters:NoSummary")
.Run(#"C:\Windows\Microsoft.NET\Framework\v3.5\msbuild.exe");
ScriptExecutionEnvironment.LogTaskFinished ();
return ReturnThisTRunner ();
}
You can find the rest of it here: http://code.google.com/p/projectpilot/source/browse/trunk/Flubu/Builds/BuildRunner.cs
I haven't tried it myself yet, but Microsoft has a Make implementation called NMake which seems to have a Visual Studio integration:
NMake
Creating NMake Projects
Visual Studio since VS2005, uses "msbuild" to define and run builds. When you fiddle with project settings in the Visual Studio designer - let's say you turn XML doc generation on or off, or you add a new dependency, or you add a new project or Assembly reference - Visual Studio will update the .csproj (or .vbproj, etc) file, which is an msbuild file.
Like Java's ant or Nant before it, msbuild uses an XML schema to describe the project and build. It is run from VS when you do a "F6" build, and you can also run it from the command line, without ever opening VS or running devenv.exe.
So, use the VS tool for development and command-line msbuild for automated builds - same build, and same project structure.
I've got a solution containing multiple projects. I'm only changing the code in one of them, but every time I hit Ctrl+Shift+B, Visual Studio rebuilds all of the others.
I want it to build the other projects, so this is good. What's not good is that, normally, it would see that there was nothing to do. I have a wonky dependency somewhere, so this isn't working.
Is there a tool or macro (or switch) that'll explore the dependency tree and tell me which files are missing or out-of-date, so that I can get it to stop?
I know that I can solve this specific case, by (e.g.) touching all of the project files.
Unfortunately, I've often seen this situation when a file is configured to produce an output file (e.g. an IDL file is configured to output a typelibrary, but doesn't contain a 'library' block, so it'll never create a TLB).
This wouldn't be resolved by touching all of the project files, so I'm looking for something more general to add to my personal toolbox that'll easily tell me why a file is being rebuilt, whether it be because it's older than a dependency, or because the project is misconfigured to expect an output file that will never be produced.
In Options / Projects and Solutions / Build and Run turn up the MSBuild project build output verbosity to Detailed. It should give you an idea of why it is rebuilding all the projects.
If I understand you right, you might solve this by touching all your project's files. It may be caused by a source-file having a last-modified-time that's in the future.
Edit:
I know that I can solve this specific case, by (e.g.) touching all of the project files, but I'd like to add something to my personal box of tricks that I can use in the future, in the general case.
I'm confused - what's the 'general case' of this problem?
Not that I've found. If you know that a project is not going to change often, you can tell the Configuration Manager not to build it. (Right-click on the Solution, and select Configuration Management)
As far as I know ctrl + shift + b is by default bound to BuildSolution, so that would be why all your projects are being build. i'm not really sure what else you could use except for rightclicking the project and pressing build :)
You might want to check in Tools>Option>Projects and Solutions and check if your option is set to Only Build startup project and dependencies instead of all the solution.
Or instead of using ctrl+shirt+b you should simply press F6 on the project you want to build :)
You can use shift+F6 to build just the current project.
While not directly answering my question: "is there a tool that'll work this out for me?", I found the specific problem by using SysInternals Process Monitor:
The project was configured with /analyze, which requires Visual Studio Team Edition, but the version on this PC is Visual Studio Professional, which doesn't support it. Unfortunately, there appears to be a bug in Visual Studio, where it thinks that the .pchast file should be created, even though it has no way to do so. I've raised this on Connect.
I think I might write a macro for Visual Studio Professional that, if /analyze is turned on, simply creates an empty .pchast file at the end of the build...