Does anyone have a method to overcome the 260 character limit of the MSBuild tool for building Visual Studio projects and solutions from the command line? I'm trying to get the build automated using CruiseControl (CruiseControl.NET isn't an option, so I'm trying to tie it into normal ant scripts) and I keep on running into problems with the length of the paths. To clarify, the problem is in the length of paths of projects referenced in the solution file, as the tool doesn't collapse paths down properly :(
I've also tried using DevEnv which sometimes works and sometimes throws an exception, which isn't good for an automated build on a separate machine. So please don't suggest using this as a replacement.
And to top it all, the project builds fine when using Visual Studio through the normal IDE.
It seems that it is limitation of the MSBuild. We had the same problem, and in the end, we had to get paths shortened, because did not find any other solution that worked properly.
The SUBST command stills seems to exist so remapping the root of your build folder to a drive letter may save some characters if Judah Himango's solution is no good.
I solved similar issue by adjusting CSPROJ-file:
<BaseIntermediateOutputPath>$([System.IO.Path]::GetFullPath('$(MSBuildProjectDirectory)\..\..\..\Intermediate\$(AssemblyName)_$(ProjectGuid)\'))</BaseIntermediateOutputPath>
As the result during compilation CSC.EXE receives full path instead of relative one.
Thanks to harrydev for clue on how CSC.EXE operates with the paths.
There are two kinds of long path problems relevant to build. One is paths that aren't really too long, but have lots of "..\" in them. Typically, these are references' HintPath values. MSBuild should normalize these paths down to below the max limit, so that they work.
The other kind of path is just plain too long. Sorry, but these just won't work. After looking at it a fair bit, the problem is that there just isn't sufficient API support for long paths. The BCL team (see their blog) had similar problems. Only some of the Win32 API's support the \?\ format. Arbitrary build tools, and probably 98% of apps out there, don't; and worse would probably behave badly (think of all the buffers sized for MAX_PATH).
We came to the conclusion that until there's a big ecosystem effort to make long paths work, or Windows comes up with some ingenious way to make them work anyway (like the short paths mangling?) long paths just aren't possible for MSBuild to support. Workarounds include subst, as you found; but if your tree just is simply too deep, your only options are to build it in fragments, or to shorten the folder names. Sorry.
Dan/MSBuild
I found the problem to be that when the C# compiler (csc.exe) is called it uses the projects directory path PROJECTDIRECTORY together with the output path OUTPUTPATH by simply appending them as:
PROJECTDIRECTORY+OUTPUTPATH
However, if the OUTPUTPATH is relative i.e. "..\..\Build\ProjectName\AnyCPU_Debug_Bin\" and the project directory is pretty long then the total length is longer than 259 characters since the path will be:
PROJECTPATH+"..\..\Build\ProjectName\AnyCPU_Debug_Bin\"
instead of an absolute path.
If csc.exe would make an absolute path before calling Win32 functions this would work. Since in our case the absolute path length is less than 160 characters.
For some reason the call to csc.exe from visual studio is then different from MSBuild than it is from visual studio. Do not know why.
In any case, the problem can be resolved by changing either or both PROJECTDIRECTORY and/or OUTPUTPATH paths.
Have you tried DOS paths? Or the \\?\ prefix? The .NET BCL team blog has more info.
If the path length is 260, then there is warning resolving reference, for 259 or 261 of this error does not occur. I think there is msbuild bug.
I know there is already an accepted answer, but I had a different problem while using msbuild that gave me the same error output, and led me on a circular wild-goose chase. So, for future googlers, here goes:
We have a batch file that calls msbuild, but as the build machine can build for multiple versions of Visual Studio, each batch file calls vcvarsall.bat before it runs msbuild. This has the nasty side effect of stuffing the path completely full of the same thing over and over again. When it fills up, you get the error shown in the question above: The input line is too long. A simple Google search could make you think your paths are suddenly too long for msbuild.
In my case, it was as simple as killing the session of cmd.exe and restarting, as this reverted the environment variables to their native state.
Related
I am trying to get a C# Visual Studio 2019/MSBuild job to build on a Jenkins build server. I know that my file paths are too long, so I have enabled Long File Paths in the Group Policy Editor (and verified that it has persisted in the registry editor after a server restart).
However, now I am getting the following error "ALINK: fatal error AL1065: File name ... is too long or invalid".
A quick google search led me to this page for Alchemy Software. However I have no idea what Alchemy Software is and why it is being used in the build process and why it is failing. (Although for the last point, I'm guessing that the Alchemy Software .dll is not "Long Address Aware", which I believe is necessary for an application to take advantage of Long Paths in Windows. But since I can't locate any .dll or .exe associated with this software, I can't be certain.)
Does anyone know why my build is still failing with this error, what Alchemy Software is, and how to get it to take advantage of Long Paths in Windows?
P.S. And please, no comments about how I should restructure my file paths to be shorter. I have tried doing that but it's impractical for this application. And anyway, it keeps popping up and is becoming a whack-a-mole situation, so I'd really rather fix the root cause rather than constantly putting band-aids everywhere.
I am going to set this as an answer since, after Hans Passant's very helpful comments and subsequent research, I think it's pretty definitive that this can mostly only be worked around, not resolved. (As possible exception will be discussed at the end of this answer.)
As stated in those comments, this error originates from a linker module called al.exe that is utilized by MSBuild for certain project configurations (more on that later). This linker can be found in a couple places, but for me it was being called from C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools.
MSBuild can handle long file paths as long as Windows is configured to do so (by enabling long paths in the Group Policy Editor or by modifying the registry directly). However this al.exe module cannot. And as far as I can tell, there's no way to force it to do so. So if your build tool chain requires this al.exe module to be used, you're kind of SOL.
For my particular situation, for a job in Jenkins that was failing because my paths were too long, I worked around this by changing the workspace of my job. So now instead of the default of something like D:\Jenkins\workspaces\[product]\Releases\[product_version], I changed it to D:\j\dd0_1c, which is an encoding that makes sense to me. This shortening of the folder path avoids any subsequent file paths from exceeding the limit of 260 characters. It's not a satisfying solution, but it works for my particular situation.
I did mention that there was a possible exception to all of this: if you can get MSBuild to avoid using al.exe altogether, then you can avoid this error.
I don't know all of the scenarios or workflows in which MSBuild utilizes this module, but I do know that it does get utilized when your application has localized resources, and somehow, some way, MSBuild uses al.exe when it is generating those resources. This was exactly my scenario, and I found this page and this page describing how you can reconfigure your localization projects such that MSBuild does not utilize al.exe. I did try the steps described in these pages and was able to verify that I no longer got this ALINK error from al.exe. However I never got my project to fully build since this reconfiguration caused other build errors to crop up. So in the end, for the sake of expediency (and because it was cleaner than performing a major refactor of my code), I went with the Jenkins workspace workaround.
However, it is interesting to note that you can get MSBuild to avoid using al.exe as long as your project is conducive to the solution given in those two links. So hopefully, if someone runs into this same issue, they might have more success than I in utilizing this method.
As suggested by https://www.visualstudio.com/en-us/docs/tfvc/share-your-code-in-tfvc-xcode I am putting my existing code to TFVC using git-tf. But my paths are deep. During check-in I am getting,
git-tf: TF205022: The following path contains more than the allowed 259 characters:
/Users/VeryLongPath
Even though the path length limitation is really annoying, the most effective and easy way still is spending some time tweaking your file/folder structure to make this work.
For example: instead of \xx\Build\Drop\ProjectName, just use \xx\Build\Drop (or \xx\Builds) since the project name is also in the build name.
For the long path issue in TFS, there had been a related uservoice and now completed. However, it's still a problem in some areas.
Fix 260 character file name length limitation
We’ve removed the limitation from the BCL for the basic file
manipulation functionality (CRUD). You can find more details here:
https://blogs.msdn.microsoft.com/dotnet/2016/08/02/announcing-net-framework-4-6-2/
Immo Landwerth Program Manager .NET
Also suggest you to take a look at this great blog-TFS Path Too Long problems for how to deal with the issue.
In short currently its the TFVC limitation, from https://www.visualstudio.com/en-us/docs/reference/naming-restrictions#version-control-paths
Must not contain more than 259 Unicode characters for a single folder or file name.
I've got a relatively simple project that is under source control (svn), and I wanted to create an installer. I know that I could (should) use WiX, but as I'm new to creating installers I thought it'd be easier to just use the built-in Visual Studio (2010) Setup and Deployment Wizard.
Unfortunately, it seems that files including external (non-project maintained) documentation, configuration files, and "Content" files are added with absolute paths. This, of course, is suboptimal. I searched the web, but found only the same question, without an answer. Another stackoverflow user seems to have asked a similar question, but the only answer, which suggests ClickOnce, seems off-base (I'd like to have an MSI that I distribute not a web-based installation).
Does anyone know how (or whether) this can be fixed?
With VS2005, sometimes the paths stored in the vdproj file were absolutes, and sometimes relatives. In my case, it seemed to be related to whether the files were accessed via the canonical path or not. Here's a concrete example:
Source is on C:\Views\builddir, open solution C:\Views\builddir\solution.sln and add files from C:\Views\builddir\.. and VS2005 would add relative paths into the vdproj file. However, if you map that builddir to a letter drive, for example, make a subst from C:\Views\builddir to s:, open the solution via S:\solution.sln, and then add files by navigating to S:\.., VS2005 would insert absolute paths into the vdproj files. Whether VS2005 displayed paths as absolutes or relatives had no relation to what it stored in the vdproj files.
So, it may well be that the problem comes down to what path you're using to open that solution.. opening \\server\shareddir\solution.sln might get different behavior than mapping \\server\shareddir to W: and opening w:\solution.sln.
You can always add the files, then use a text editor (e.g. notepad) to change the absolute paths in the vdproj file to relative ones. You'll be fine until you change that project again.
MS doesn't seem to really fix minor bugs like this so much as rewrite the code to introduce an entirely different set of bugs, so VS2010 might still act this way.
FYI, why would one want to map an absolute path to your builddir? It was a holdover from the bad old days when VS didn't do anything correct with relative paths.
As tzerb mentioned, the main source of confusion might be that paths show up as absolute under the property window inside VS, but when you look into the actual VDPROJ file you should see the paths show up as relative. However, as patbob mentioned, I believe the paths ARE stored as absolute when they come from a different letter drive.
It might be easier now but when you start bumping into the limitations of the tool it's going to get real hard. Let's not even talk about the bad practices it will encourage which could end up being real hard for the poor end user installing your product. You've got Visual Studio 2010 so InstallShield LE ( free ) would be a better choice.
Otherwise, to answer your question, it will only use absolute paths if it can't caculate a relative path. ( for example c:\foo\foo.vdproj consuming d:\foo.txt consuming c:\test\foo.txt should automatically be ....\test\foo.txt )
BTW, if you decide to check out WiX and want some "easy" check out my IsWiX project on CodePlex. I'm trying to bridge the feature gap between InstallShield and WiX.
Sorry for what seems like a silly question: But I've never, ever worked with boost, until tonight, and I'm finding that getting it configured seems to be harder to use than it should be.
I wanted experiment with it tonight. So I downloaded the zip file, and unzipped it to a directory here:
F:/boost_1_44_0
Then I created an empty c++ project in visual studio 2010 (not using pch either). So all I wanted to do was to include a header file. But not even a silly thing like that seems to work. Now I've been using visual studio for years, though at work we are still stuck on vs 2008 (That is another story). So usually what you do is set an include directory, and then you can include files in at will right?
So I set the global include directory to include the boost root. i.e. Property Manager -> My configuration (debug|win32) -> Microsoft.Cpp.Win32.user -> Common Properties -> C++ Directories -> Include Directories. There I added my path to f:/boost_1_44_0.
I also went to the project properties and set the C++ include directory for the project to point to the boost root like in vs 2008.
I then added a silly include declaration like so:
#include <boost/lambda/lambda.hpp>
But, amazingly it fails to compile!!! with the following error:
Error 1 error C1083: Cannot open include file: 'boost/type_traits/transform_traits.hpp': No such file or directory f:\boost_1_44_0\boost\lambda\core.hpp 25 1 test_boost
Which when I double click it, it opens up in f:\boost_1_44_0\boost\lambda\core.hpp, and takes me to this line:
#include "boost/type_traits/transform_traits.hpp"
So I have no idea what's happening. Is visual studio just not delivering up my global include paths that I set? It seems also that the include directive in core.hpp should be using angle brackets and not quotes.
If I'm doing something wrong what?
EDIT:
!! SOLVED !!
Before I didn't have all the files unzipped. I don't know what happened. So I re-downloaded the zip file, and unzipped it again. This time the zip file took much longer to unzip, and it extracted much more files: Including the missing files.
Problem solved, my hello world app compiles just fine now.
The behaviour of compilers in locating header files is implementation defined for both the <> and "" variants.
However, based on this page for VC2010, it appears the quoted form searches a superset of the angle bracket form so I'm not sure that's the problem.
I suppose it would be a silly question to ask if the following file actually existed?
f:\boost_1_44_0\boost\type_traits\transform_traits.hpp
So, a couple of investigative jobs:
Make sure that f:\boost_1_44_0\boost\type_traits\transform_traits.hpp exists.
Try changing your top-level include to use quotes.
Try changing the include in f:\boost_1_44_0\boost\lambda\core.hpp to use angle brackets.
Make sure you try all four possibilities for those last two.
Is f: a network-mounted drive? What happens if you put it all on c:?
That last one is just in case Windows is doing some shenanigans under the covers :-)
While it's a bit overkill for this, learning to use SysInternals' Process Monitor will pay off over time. It will show you what files are actually opened, and which attempts failed. Look where Visual Studio tries to read transform_traits.hpp from, and you'll probably have the answer.
We were thinking about organizing our BIG project this way:
\trunk
[CompanyName]
[Product1]
[Project1]
CompanyName.Product1.Project1.csproj
[Project2]
CompanyName.Product1.Project2.csproj
CompanyName.Product1.sln
[Product2]
We were trying to follow Microsoft's recommendation that namespace names follow folder structure, but are there any drawbacks for making it this way?
What is the naming convention for solutions and projects that you apply?
That looks pretty good if you ask me. Especially naming your projects by their full name including full name space part. I've found this helpful when there are numerous projects, especially if there happens to be similar projects across different products.
How and whether you split on product and project (where I assume project is more like an application than a solution project) is very much down to the size of your organisation, it's make-up and your preferences.
An alternative that I've used is to have all my solution files in the same directory.
\trunk
[CompanyName]
CompanyName.Product1.sln
CompanyName.Product2.sln
[Product1]
[Project1]
CompanyName.Product1.Project1.csproj
[Project2]
CompanyName.Product1.Project2.csproj
[Product2]
[Project3]
CompanyName.Product2.Project3.csproj
With respect to file names - I prefer that my project file name match the output assembly name because it makes it much easier to know what produces what. Doing a directory listing is much faster than searching the csproj files in a tree for the one that produces the assembly I care about.
I don't get worked up about solution files because they don't influence our build environment so I end up making my own to have the exact scope I want (and the specific per-solution items, like test metadata, that I want).
With respect to folder structure - I don't worry too much if the folders leading down the project files match the namespaces. I want my code to sit on disk in a way that makes the most sense for the project. Sometimes this means the test code and product code are in sibling directories - sometimes it means they are much further apart. Sometimes there is a namespace that is contributed to by multiple teams (not advocating that design, just a reality) - but those teams want to life in their own folders for whatever reason.
Don't forget the importance of version control branching strategies in your overall project design. The Company and Product boundaries may be branches and therefore would not necessarily need to represented as directories on disk.
Don't let this be a source of analysis paralysis though. Make a reasonable choice. Use version control. You can always change later if you are wrong.
Looks like taken from the school book. That's usually how my solutions get set up, and I have found it to work quite well over the years.
Looks good to me.
It's a point to note that by default, the default namespace in a Visual Studio project is just the project name. Surely that indicates that naming your projects like your namespaces is "the Visual Studio Way".
Solutions are most naturally named after the product/project. Like you indicate.
I consider this is better
\trunk
[CompanyName]
[Product1]
CompanyName.Product1.sln
[Main] --Optional
CompanyName.Product1.csproj
[Project1]
CompanyName.Product1.Project1.csproj
[Project2]
CompanyName.Product1.Project2.csproj
[Product2]
CompanyName.Product2.sln
[Main] --Optional
CompanyName.Product2.csproj
[Project1]
CompanyName.Product2.Project3.csproj
[Project2]
CompanyName.Product2.Project2.csproj
[Project3]
CompanyName.Product2.Project2.csproj
Why? Because when you get the code from repository you get, by example, "Product1" directory, it contains all you need to work. The [Main] directory contains the default base namespace, usually the exe or main project. It is optional.
Downsides to naming projects / solutions with name spacing are that:
1. Your solution project will not build due to Windows ( my version and previous ) imposing a max limits on the length of path ( not a problem on <cough><cough> mac ). Two solutions are:
1. Changing an operating system setting.
2. Configure build to shorten path.
2. Various apps important to you like say, SourceTree, may encounter problems. I figure this is due to the assumption as to filepath length. This problem is detailed in this Stack Overflow question:Filename too long in Git for Windows
Regarding Problem (1)
Visual Studio might warn you
I don't yet know how to enact (2).
If you choose (1), then this will have an affect on your downstream environments: build OS, dev OS, qa build OS, qa OS, production OS as you will either need to make changes to the OS. Stack overflow question with answer describing change to operating system: Could not write lines to file "obj\Debug\net5.0\SolutionName.GeneratedMSBuildEditorConfig.editorconfig exceeds the OS max path limit
Code MSB3491
https://www.google.com/search?q=Code+MSB3491+max+path+limit
Could not write lines to file "obj\Debug\net5.0\------------my namespace------------..GeneratedMSBuildEditorConfig.editorconfig". Path: obj\Debug\net5.0\------------my namespace------------.GeneratedMSBuildEditorConfig.editorconfig exceeds the OS max path limit. The fully qualified file name must be less than 260 characters.
-------------my namespace--------------------------------
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\Roslyn\Microsoft.Managed.Core.targets
Regarding Problem (2)
To fix, possible solutions are:
(1) Tell SourceTree to use the git installed on the system. I only had Atlassian's version of git with which to work.
(2) Tell Atlassian's git to work with long file paths. Run cmd as administrator, then run this command:
c:\Users\{theuser}\AppData\Local\Atlassian\SourceTree\git_local\cmd\git.exe config --system core.longpaths true
If you choose either of these resolutions, then you should expect to make changes to systems / processes down the line from your development machine.