I have a custom MSBuild target included in my C++ project that produces a data file in the $(OutDir) folder for each item of a given item type. I have the item type hooked up with a property page schema so you can select it on files in the solution explorer and my target declares input and outputs so incremental builds work. I have also added my target to the $(BuildDependsOn) property so it is automatically evaluated during the Build target Visual Studio invokes.
Everything seems to work except for one thing: If I delete one of my output data files in the $(OutDir) and then build Visual Studio does nothing and says my project is up to date. If I delete the exe file the project produces or touch the modified time of one of the MSBuild scripts Visual Studio re-evaluates the targts and finds the output file is missing, causing it to be re-built using my target.
From the MSBuild diagnostic logging it seems like Visual Studio is internally maintaining some list of output files and input files that it checks to avoid evaluating the MSBuild script at all. How do I add my output files to this list?
MsBuild/VS indeed have a mechanism to determine what is up-to-date with respect to the input files, it revolves around an executable tracker.exe which scans .tlog files to figure out what a project's output files are. There might be more to it, and if you look around on the internet you can probably get more info about this.
But the thing is you don't really need to understand every single detail of it: you can find a simple usage example for it when inspecting how the built-in CustomBuildStep works and apply that to your case. I'll briefly explain how I got to this because I think it might be useful for you as well in dealing with msbuild questions like these.
If you add
<ItemDefinitionGroup>
<CustomBuildStep>
<Command>echo foo > $(OutDir)\foo.txt</Command>
<Outputs>$(OutDir)\foo.txt</Outputs>
</CustomBuildStep>
</ItemDefinitionGroup>
either manually or via the project's property pages for Custom Build Step you'll see the beahviour is eactly what you need: if foo.txt is deleted a build will start, while a build is marked up-to-date if it is not (well, and when the rest of the outputs are also up-to-date).
Hence the key is to do what CustomBuildStep does under the hood, and figuring that out is just a matter of using your tool of choice to search all occurrences of CustomBuildStep in all files under C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120 (adjust path for platform/VS version used).
This leads us to Microsoft.CppCommon.Targets where the target named CustomBuildStep (mind you, that's the same name as the entry in the ItemDefinitionGroup above) invokes the actual CustomBuildStep command. It also has this particularily interesting bit:
<!-- Appended tlog to track custom build events -->
<WriteLinesToFile Encoding="Unicode"
File="$(TLogLocation)$(ProjectName).write.1u.tlog"
Lines="#(CustomBuildStep->'^%(Identity)');#(CustomBuildStep->MetaData('Outputs')->FullPath()->Distinct())"/>
So this writes the path of the Outputs to a .tlog file in the directory used by the tracker and makes it work as desired. Also see here for more information about the format.
tl;dr Use WriteLinesToFile to append full paths of your targets' outputs to a file like $(TLogLocation)$(ProjectName).write.1u.tlog. I'm saying like because write.tlog, write.u.tlog etc also work.
Visual Studio uses something called Visual Studio Common Project System (CPS) (https://github.com/Microsoft/VSProjectSystem) (VS 2017)
to manage projects, including build process.
Within CPS anything that implements IBuildUpToDateCheckProvider interface can be used
as a 'UpToDateChecker' for a project.
'UpToDateChecker' is invoked before invoking MsBuild. Its main purpose is to determine whether or not invoke MsBuild to build project, or to mark project as 'Up To Date' and skip msbuild all along.
This 'UpToDateChecker' is exactly what prints into diagnostic build output:
1>------ Up-To-Date check: Project: "ProjectName", Configuration:
Debug x86 ------ Project is not up-to-date: build input 'header.h' was
modified after build output 'a.out'. Input time: 12/27/2018 4:43:08
PM, Output time: 1/1/0001 2:00:00 AM
As for C++ Projects, for VS 2017 its default 'UpToDateChecker' is VCProjectBuildUpToDateCheck
( Microsoft.VisualStudio.Project.VisualC.VCProjectEngine.dll ).
As starter, it looks into tlogs directory ( usually something like Debug\x86\.tlog) for these files:
.lastbuildstate
unsuccessfulbuild
all '.read..tlog' - input files, marked as 'build input' in diagnostic build output
all '.write..tlog' - output files, marked as 'build output' in diagnostic build output
There's actually more checks, but most fails occur when checking these 4 types
The original question here relates to C++ projects, but for anyone finding this while searching for information about modern (SDK-style) C#/VB/F# projects, you can customise Visual Studio's fast up-to-date check as described in this document:
https://github.com/dotnet/project-system/blob/master/docs/up-to-date-check.md
In a nutshell, you specify inputs and outputs as items:
UpToDateCheckInput — Describes an input file that MSBuild would not otherwise know about
UpToDateCheckBuilt — Describes an output file that MSBuild would not otherwise know about
It can be very helpful to increase the diagnostic logging level for the up-to-date check via this setting:
You can find out why a project is being rebuilt by enabling the verbosity of the fast up to date checker in the registry key:
New-ItemProperty `
-Name U2DCheckVerbosity `
-PropertyType DWORD -Value 1 `
-Path HKCU:\Software\Microsoft\VisualStudio\14.0\General -Force
You should be able to see in the build log messages like
Project 'Caliburn.Micro.Silverlight.Extensions' is not up to date. Project item 'C:\dev\projects\Caliburn.Micro.Silverlight.Extensions\NavigationBootstrapperSample.cs.pp' has 'Copy to Output Directory' attribute set to 'Copy always'.
[1] https://blogs.msdn.microsoft.com/kirillosenkov/2014/08/04/how-to-investigate-rebuilding-in-visual-studio-when-nothing-has-changed/
To enable logging for old-style projects (i.e. non-SDK-style projects, common in the .NET Framework era):
Open a "Developer Command Prompt" for the particular version of Visual Studio you are using.
Enter command:
vsregedit set "%cd%" HKCU General U2DCheckVerbosity dword 1
The message Set value for U2DCheckVerbosity should be displayed.
Run the same command with a 0 instead of a 1 to disable this logging.
More information at: https://github.com/dotnet/project-system/blob/main/docs/up-to-date-check.md#net-framework-projects
Related
I've been a linux/make guy and recently I'm learning to build UE5 engine from VS 2022. I need to figure out a CLI way to build it.
For example, I right click on one of the modules (not sure if it's the most proper name) and choose 'Build' then the build will start. I want to automate the procedure using CLI.
How can I find the corresponding CLI command for this manual operation?
I don't have access to the Unreal Engine source code and I don't know if Epic has done anything highly unconventional.
From your start menu launch the "Developer Command Prompt for VS2022". This is a shortcut file for launching the Windows command line with a batch file run to set up the PATH and other environment variables for the Visual Studio development tools.
Visual Studio project files (.csproj for C# and .vcxproj for C++ for example) are MSBuild files. (MSBuild was inspired by Ant, if that helps.)
Solution files (.sln) are a completely different format but MSBuild can build a solution file.
From the screenshot in the question I can see that the solution is UE5 which will be UE5.sln. I can also see that you want to build a C++ project. I'm guessing the project may be named BenchmarkTest (BenchmarkTest.vcxproj)?
MSBuild has a notion of targets. A target always has a name and it groups a set of tasks to be performed. (It's like a makefile rule in some respects but it's not the same.)
Solutions and projects created with Visual Studio support some standard targets. The 'Build', 'Rebuild', and 'Clean' menu items map directly to some of these targets.
Visual Studio solutions and projects support Configurations and Platforms. The standard Configurations are Debug and Release. The screenshot shows a non-standard configuration of Develop. The screenshot also shows a platform of Win64.
In the Developer Command Prompt, msbuild should be in the PATH. Try the following command:
msbuild --version
To build the solution with the default target (which is 'build') and the default configuration and platform:
msbuild UE5.sln
To run a 'clean':
msbuild UE5.sln -target:clean
The target switch can be shortened to -t.
The configuration and platform are passed as properties using the -property switch. The short form is -p. Multiple property switches can be provided and multiple properties, delimited by ';', can be provided in one property switch.
msbuild UE5.sln -t:rebuild -p:Configuration=Develop -p:Platform=Win64
or
msbuild UE5.sln -t:rebuild -p:Configuration=Develop;Platform=Win64
To build the BenchmarkTest project, specify the project file:
msbuild BenchmarkTest.vcxproj -t:build -p:Configuration=Develop;Platform=Win64
So I've look at a few questions here which popped up after a search, but I still haven't managed to get this to work.
My project has two files, main.cpp and shader.comp.
The shader needs to be compiled before the main program is ran and I have a small .bat script that does just that. It's set to run as a pre-build event.
However, if I edit shader.comp and leave main.cpp unmodified since I last ran the project, there is no need to rebuild the project (according to VS anyway), so there is no need to run any pre-build events and my shader doesn't get compiled.
Is there a way to tell VS2017 (or VS2019) that if some file is modified, then run something, or at least a way to add an arbitrary file to list of files that VS checks against when deciding whether to run the build or not?
I've tried setting "Exclude from build" to "No" in the file properties, but no matter what "Item type" I choose, editing just the shader won't trigger the rebuild.
It's possible to define the shader that needs to be compiled as Custom Build Tool in the properties of the file (as Item Type). This will open another menu in the properties where cmd script and similar can be written.
In this particular case, value for Command Line was:
glslangValidator.exe -V -o "%(RootDir)%(Directory)%(Filename).spv" "%(FullPath)"
And the Outputs:
%(RootDir)%(Directory)%(Filename).spv
In short, if file defined in Outputs doesn't exists or is older than the owner of this property (the file that needs to be compiled), the Command Line argument will be ran in cmd.
Official documentation has more info on this.
Maybe you can get some help from this issue. You can specify the files for VS Up-To-Date check.
<Project...>
...
<ItemGroup>
<UpToDateCheckInput Include="shader.comp" /> <!--when this file is in the project folder-->
<!--<UpToDateCheckInput Include="path/shader.comp" />-->
</ItemGroup>
</Project>
And this script into your project file. Then if there's any change to shader.comp file,VS will build your project.
Note: In this way, if you only change the shader.comp file , but not change the source file(xx,cpp), the build will start but vs will skip the compile target of C++ source code. Only when you modify the source code main.cpp, then VS will run the pre-build event and compile the code. Let me know if it's what you want, hope i didn't misunderstand anything :)
I use CMake generated solution for Visual Studio 2010.
In my solution I have several lib projects and one exe project.
For Debug build configuration I use output names like lib1_d.lib, lib2_d.lib etc...
For Release build configuration I use lib1.lib, lib2.lib ...
thanks to CMake I have one extra build configuration I use - RelWithDebugInfo. I use same output names for this build configuration as for Release.
Now here is the problem:
Assuming everything is cleaned.
I hit F5 (run / start debugging) RelWithDebugInfo. All project are built (exe is depending on them) and project runs successfully.
I switch to Release and hit F5 again. All project are built and project runs successfully. (libraries in output directory are overwritten)
I switch back to RelWithDebugInfo and hit F5. VS quickly goes through and gives All outputs are up-to-date. ... Build succeeded. And DOES NOT overwrite lib files in output directory. So application crashes because it uses libraries for other build configuration.
This problem occurs for both ordering Release->RelWithDebugInfo and RelWithDebugInfo->Release
I haven't find a solution, how to add other prefix to RelWithDebugLibraries my SO question
Is there a way, to force Visual Studio 2010 to always overwrite outputs? Preferably by some flag which I can provide from CMake.
The VS build system solves this problem by using different build directories for different configurations. By default, 32-bit Debug output goes to the Debug directory, Release output goes to the Release directory, 64-bit Debug output goes to x64\Debug directory, etcetera. This way different configurations never step on each other's output files.
Looks to me like the mistake you made with your added RelwithDebugInfo configuration is that you didn't modify the output file names. So the build system sees an up-to-date output file from another configuration and doesn't think it necessary to rebuild them.
Coming up with variations of build output file names does get to be impractical once you go past two, do consider the VS-way to do this.
I do have few config files and dll files that are needed in order to run the final product and I want to instruct visual studio build to pick them up.
So far it was easy to add them to the project but I wasn't able to identify the correct mix of properties that I need to setup for these files, in order to get the files inside the bin directory (target).
Note: the term deploy here doed not refer to the after build deployment, that would be a custom task anyway.
So far I see these options:
Excluded from Build: No
Content: Yes
Item Type: contains a list of ~20 options but none of them sounds like something that would work. I already tried most of them.
Select the project that requires the configuration file and right click on it and then select Properties.
Go to the Build Events tab and type in the Post-Build event command line something like the following
copy pathfiletocopy $(ProjectDir)$(OutDir)
now you could check the Copy Only on Successful Build option.
Of course you need to adapt the command line to your liking.
You can also press the button Edit Post-build and see a list of predefined macros that refers to specific informations related to your projects and solution. You could also insert other CMD shell commands like IF $(ConfigurationName) == "Release" to execute commands only when you compile for release instead of debug (for example running an obfuscator)
I am using Visual Studio Express 2012. Where is the location of the log file? I have searched in the folder where my solution and projects are stored, but cannot find any .log file.
This is the configuration for logging:
Log file from Visual Studio is only supported for C++ projects. You just have to work with the output window for others.
See this similar thread: VS2010: minimal build log in output and detailed log in log file
And in case you happen to do this for a C++ project, the file is at:
... build log in the intermediate files directory
... The path and name of the build log is represented by the MSBuild macro
expression, $(IntDir)\$(MSBuildProjectName).log.
Use build output instead of logging to file. Instead of copy/paste, simply click somewhere in the output and press CTRL + S to save. Visual Studio will prompt you for a location (tested with Visual Studio 2017, but I'm assuming this works in earlier versions too).
The msdn documentation is pretty clear about this (And you ain't gonna like it!):
https://msdn.microsoft.com/en-us/library/jj651643.aspx
Where it says:
To create a build log file for a managed-code project On the menu bar,
choose Build, Build Solution.
In the Output window, highlight the
information from the build, and then copy it to the Clipboard.
Open a
text editor, such as Notepad, paste the information into the file, and
then save it.
While it's true that VS doesn't allow this directly, it is still possible to build with MSBuild "inside" VS2015 and get both the build window output and the log file, as follows: (Arguably this is a bit of a hack.)
In your VS Managed solution, add a new project (Let's call it 'Make').
a. The project type you want is Visual C++/NMake project.
Define the MSBuild commands you need on the command line (see below).
Change the solution configuration to build the NMake project instead of the normal managed projects.
This will create a project that has Build, Rebuild, and Clean command lines where you can execute MSBuild directly. For example:
Rebuild: MSBuild.exe /ds /v:diag /property:Configuration=Debug ..\BuildTest\BuildTest.csproj /t:Clean,Build
Build: MSBuild.exe /ds /v:diag /property:Configuration=Debug ..\BuildTest\BuildTest.csproj /t:Build
Clean: MSBuild.exe /ds /v:diag /property:Configuration=Debug ..\BuildTest\BuildTest.csproj /t:Clean
You can also specify multiple MSBuild.EXE command lines in order to build multiple projects. For the usual build-the-entire-solution outcome you can target only the final end assemblies and let the dependency graph generate the individual targets.
This will produce a .log file, where NAME is the name of the NMake project you used. In the example above, the log would be make.log.
A working example is available on GitHub:
https://github.com/bitblitz/VS_MsbuildExample
(Tested with VS2015)
Note that building individual projects directly will still build with the normal VS behavior, but you can build the full solution inside VS and get the build logs.