Qt MOC multiple files in parallel with MSBuild - visual-studio-2010

Good morning!
I know there is already this question here: Qt Moc'ing multiple files in parallel under msbuild but I would not show up this old question.
I work under Visual Studio 2010 and I've got to speed up the compilation time of my app. I use all flags like /MP with MSBuild -j with Make etc... The last step of the optimization is to parallelize MOC'ing steps. They are painfully slow and I googled a lot and I didn't find a solution.
I know that jom exists but it uses nmake and I've got to use MSBuild.
If someone has already heard a solution, it should be really cool!
Have a nice day!

If you generate VC project file with qmake from qt *.pro , it generates it in a way that mocables are compiled in one thread. The only way I know to workaround this behavior is to explicitly call jom for moc pre-processing.
I have only VS2012 (win32-msvc2012), but I used to do the similar thing for VS2010 (win32-msvc2010 in your case)
To do so you should automate the following steps:
Create VC project from pro file through qmake:
qmake -spec win32-msvc2012 -tp vc -o ${path-to-target}/${your-project}.vcxproj ${path-to-source}/${your-qt-pro}.pro
Create makefile from pro file through qmake:
qmake -spec win32-msvc2012 CONFIG+=release -o ${path-to-target}/Makefile', ${path-to-source}/${your-qt-pro}.pro
Create the following .bat file next to vcproj file (put %VS100COMNTOOLS% for vc2010 and x86/x64 for arch):
call "%VS110COMNTOOLS%\..\..\VC\vcvarsall.bat" ${arch}
md build\release\generated
${environment.dir}\bin\jom.exe -j 16 /F Makefile.release mocables
To do debug build change 'release' to 'debug' (or introduce a variable)
Now it is necessary to edit VC project file. Here is what you need to find/replace (using regular expressions):
1) For all includes (tags Project->ItemGroup->CustomBuild Include that contains *.h files:
find: </CustomBuild>
replace to: <ExcludedFromBuild>true</ExcludedFromBuild>
</CustomBuild>
2) for Project->ItemDefinitionGroup:
find: </Link>
replace to: </Link>
<PreBuildEvent>
<Command>build_moc.bat</Command>
</PreBuildEvent>
3) for Project->ItemDefinitionGroup:
- find: <ItemDefinitionGroup>
- replace to: <Target Name="BeforeClean">
<Message Text="Cleaning moc generated files"/>
<Exec Command="del \$\(ProjectDir\)..\\\$(Platform\)\\build\\${arch}\\generated\\moc_*.* /F /Q" />
</Target>
<ItemDefinitionGroup>
I automate it with Maven, so here is the code snippet for the reference:
build_moc.bat:
cd %1
md build\%2\generated
c:\\develop\\buildenv\bin\jom.exe -j 16 /F Makefile.%2 mocables
maven script that does replacements (maven-replacer-plugin config):
<!-- Disabling moc preprocessor steps, since we do it with jom -->
<replacement>
<xpath>//Project/ItemGroup/CustomBuild[contains(#Include,'.h')]</xpath>
<token><![CDATA[</CustomBuild>]]></token>
<value><![CDATA[
<ExcludedFromBuild>true</ExcludedFromBuild>
</CustomBuild>
]]></value>
</replacement>
<!-- Adding moc preprocessor steps with jom -->
<replacement>
<xpath>//Project/ItemDefinitionGroup[not(#*)]</xpath>
<token><![CDATA[</Link>]]></token>
<value><![CDATA[
</Link>
<PreBuildEvent>
<Command>\$\(ProjectDir\)../${arch}/build_moc.bat \$\(ProjectDir\)../${arch} \$\(Configuration\)</Command>
</PreBuildEvent>
]]></value>
</replacement>
<!-- Cleaning moc files -->
<replacement>
<token><![CDATA[<ItemDefinitionGroup>]]></token>
<value><![CDATA[
<Target Name="BeforeClean">
<Message Text="Cleaning moc generated files"/>
<Exec Command="del \$\(ProjectDir\)..\\\$\(Platform\)\\build\\${arch}\\generated\\moc_*.* /F /Q" />
</Target>
<ItemDefinitionGroup>
]]></value>
</replacement>
I hope this helps

Related

Set `AfterClean` to csproj file generated by CMake

I'm generating MSBuild files (.csproj) using CMake.
At some time, I need to open the generated .csproj using Visual Studio, to do some debug and rebuild, etc...
I need my CMakeLists file to insert the following lines to the .csproj file:
<Target Name="AfterClean">
<Exec Command="..\my_clean_script.sh" />
</Target>
So, my_clean_script.sh is executed when I clean the project from Visual Studio to clean some external dependencies files (not part of the project) created by my CMakeLists file.
Any way to do that?
Thanks
try to use file(APPEND <filename> <content>...) ?
ref: https://cmake.org/cmake/help/latest/command/file.html#writing ?

pass global options such as /Zm through msbuild (14.0)

How can I pass the annoying /Zm500 (500% virtual memory, because MS compiler is to stupid and even 32bit) through "msbuild.exe" such that when compiling a solution with it uses this option for every "cl.exe" invocation?
How can I pass the annoying /Zm500 through "msbuild.exe"
We could not pass the global option /Zm via the MSBuild command line directly. Because the PreprocessorDefinitions of CLCompile, which is not a PropertyGroup.
<ClCompile>
<AdditionalOptions>/bigobj /Zm500 %(AdditionalOptions)</AdditionalOptions>
<DisableSpecificWarnings>4453;28204</DisableSpecificWarnings>
</ClCompile>
As a workaround for this question, you can add a target invoke MSBuild to pass an external parameter into the project file by MSBuild command line:
First, change the fixed values of “/Zm500” with $(Zm) in the project file:
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<ClCompile>
<AdditionalOptions>/bigobj $(Zm) %(AdditionalOptions)</AdditionalOptions>
<DisableSpecificWarnings>4453;28204</DisableSpecificWarnings>
</ClCompile>
Second, add a target in to the project file:
<Target Name="TestBuild" Returns="#(ManagedTargetPath)">
<MSBuild Projects="YourProjectName.xxproj" Targets="NormalBuild" Properties="Zm=/Zm500"/>
</Target>
Third, use the MSBuild command line with the properties /Zm:
msbuild.exe "$(ProjectPath)\.xxproj" /p:Zm=/Zm500

Determine whether it's a build or rebuild in .cmd script called in prelink step inside Visual Studio

How can a .cmd script run from within a Visual Studio (2005, 2008, 2010, 2012 and 2013 respectively) project's pre-link stage determine whether this is a full rebuild (Build.RebuildSolution/Build.RebuildOnlyProject) or "ordinary" build (Build.BuildSolution/Build.BuildOnlyProject)?
This is an external script (LuaJIT, if you must know) and I don't want to rebuild the library every single build of the project. Instead I'd like to limit the complete rebuild to situations where I choose exactly that option.
How can a .cmd script run from within a Visual Studio (2005, 2008, 2010, 2012 and 2013 respectively) project's pre-link stage determine whether this is a full rebuild ... or "ordinary" build ... ?
I do not know if the exact thing that you are asking can be done - perhaps someone else knows how to do it. I will, however, suggest an alternate approach.
My approach is to remove the build of the Lua library from the pre-link step to a separate Visual Studio NMake project. If you create an NMake project, you will be able to know which type of build (build or rebuild) is occurring.
Note that later versions of Visual Studio simply refer to the project type as "Make". For discussion purposes here, I will refer to the project type as "NMake". I believe this is just a naming difference, and that the underlying build project remains the same between the two versions.
As a simple test, I created two Visual Studio applications: 1) an NMake project that calls a batch file to create a static library, and 2) a console application that consumes the library from step 1.
The NMake Project
In Visual Studio, if you create a new NMake project, you will see a dialog that allows you to provide MS-DOS commands:
As you can see, there are commands for: Build, Clean, Rebuild, and others. I don't have a screen shot of the above dialog with my commands, but here is my NMake project's properties:
My Build command just checks for the existence of the output file (lua.lib). If it does not exist, then it calls the rebuild.bat batch file. My Rebuild command always calls the batch file. My Clean command just deletes the output. I am not really sure what the Output command is used for, but I just filled in the path to the build output (lua.lib).
Now if you do a build, the lua.lib file will only be created if it is not there. If it is already there, nothing is done. If you do a rebuild, then a new lua.lib file is created.
The Console Application
In my console application, I added a reference to the NMake project - this way the NMake project is built prior to the console application. Here is the console application's reference page:
I also added the lua.lib file as an input during the application's link stage:
When the console application is built (during a build), it will build the NMake project if needed, and use the output (lua.lib) during the linker stage. When the console application is rebuilt (during a rebuild), it will also rebuild the NMake project.
Other Thoughts
My screen shots above only show the debug version of the properties. Your projects will have to account for the release version. There probably is a VS macro to handle this, but I am not sure since it has been ages since I've done anything with C/ C++.
In my testing above I use a single build batch file for both the build and rebuild. Obviously, you could do the same or you could use different batch files.
It may be a bit of a hack, but in .csproj file there are sections
<Target Name="BeforeBuild">
</Target>
<Target Name="AfterBuild">
</Target>
You can set an variable from BeforeBuild and retrieve it from cmd script. Later on reset this variable in AfterBuild and you should be good to go.
Ok, this is going to be a long one.
First of all - do not take my code 'as is' - it is terrible one with lots of hacks, I had no idea msbuild is so broken by default (it seems at work I have access to waaaay more commands that make life easier). And another thing - it seems vcxproj is broken at some poin - I was not able to integrate the way I wanted with only BeforeRebuild and AfterRebuild targets - I had to redefine hole Rebuild target (it is located in C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets)
So, the idea is the following: when a Rebuild is happening we create an anchor. Then, during PreLink stage we execute cmd which is able to use created anchor. If the anchor is in place - we deal with Rebuild, if there is no anchor - it is a simple Build. After Rebuild is done - we delete the anchor.
modifications in vcxproj file:
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
....
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
....
<PreLinkEventUseInBuild>true</PreLinkEventUseInBuild>
....
</PropertyGroup>
....
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
.....
<PreLinkEvent>
<Command>D:\PreLink\b.cmd</Command>
</PreLinkEvent>
.....
</ItemDefinitionGroup>
.....
<Target Name="BeforeRebuild">
<Exec Command="echo 2 > D:\PreLink\2.txt" />
</Target>
<Target Name="AfterRebuild">
<Exec Command="del D:\PreLink\2.txt" />
</Target>
<!-- This was copied from MS file -->
<PropertyGroup>
<_ProjectDefaultTargets Condition="'$(MSBuildProjectDefaultTargets)' != ''">$(MSBuildProjectDefaultTargets)</_ProjectDefaultTargets>
<_ProjectDefaultTargets Condition="'$(MSBuildProjectDefaultTargets)' == ''">Build</_ProjectDefaultTargets>
<RebuildDependsOn>
BeforeRebuild;
Clean;
$(_ProjectDefaultTargets);
AfterRebuild;
</RebuildDependsOn>
<RebuildDependsOn Condition=" '$(MSBuildProjectDefaultTargets)' == 'Rebuild' " >
BeforeRebuild;
Clean;
Build;
AfterRebuild;
</RebuildDependsOn>
</PropertyGroup>
<Target
Name="Rebuild"
Condition=" '$(_InvalidConfigurationWarning)' != 'true' "
DependsOnTargets="$(RebuildDependsOn)"
Returns="$(TargetPath)"/>
<!-- End of copy -->
</Project>
And the cmd looks like this:
if exist 2.txt (
echo Rebuild818181
) else (
echo Build12312312
)
The output from Output window:
1>Task "Exec" (TaskId:41)
1> Task Parameter:Command=D:\PreLink\b.cmd
1> :VCEnd (TaskId:41)
1> Build12312312 (TaskId:41)
Things to improve:
Use normal variables instead of external file (it seems MsBuild extension pack should do it)
Probably find a way to override only BeforeRebuild and AfterRebuild instead of the hole Rebuild part
It is much easier. Just add the following target to your build file or visual Studio Project
<Target Name="AfterRebuild">
<Message Text="AFTER REBUILD" Importance="High" />
<!--
Do whatever Needs to be done on Rebuild - as the message shows in VS Output
window it is only executed when an explicit rebuild is triggered
-->
</Target>
If you want a two step solution use this as a template:
<PropertyGroup>
<IsRebuild>false</IsRebuild>
</PropertyGroup>
<Target Name="BeforeRebuild">
<Message Text="BEFORE REBUILD" Importance="High" />
<PropertyGroup>
<IsRebuild>true</IsRebuild>
</PropertyGroup>
</Target>
<Target Name="BeforeBuild">
<Message Text="BEFORE BUILD: IsRebuild: $(IsRebuild)" Importance="High" />
</Target>

How do I use an MSBuild file from Visual Studio 2012?

I have a simple MSBuild file that I'm learning with.
Here it is:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Clean" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup Label="Globals">
<ProjectGuid>{D5A16164-962E-4A6D-9382-240F31AB6C50}</ProjectGuid>
</PropertyGroup>
<Target Name="Clean">
<ItemGroup>
<BinFiles Include="bin\*.*" />
<fff Include="f\*.*" />
</ItemGroup>
<Delete Files="#(BinFiles)" />
<Delete Files="#(fff)" />
</Target>
</Project>
Now I want to include this in a Visual Studio solution and be able to run the "clean" target from Visual Studio 2012. I tried naming it testproject.msbuildproj like the internet seems to suggest "works", but it doesn't work. When I run the clean command I just get "unexpected error".
If I rename the project to testproject.csproj, it does some unintuitive things like creating compilation directories, but it does actually run my clean command properly. However, this is undesireable because it creates obj and bin/x86/debug type directories. It also looks goofy in Visual Studio because it still gives the References drop down.
How can I use just a plain vanilla MSBuild project from Visual Studio without random errors or false assumptions?
Note I only am having a problem with this from Visual Studio. Using msbuild from the command line it works perfectly
Visual Studio creates bin / obj folders when it opens csproj file. When you click Build / Rebuild / Clean it just uses appropriate targets from the project file.
You cannot stop VS from creating these folders, but you can ask it to create them in say temp folder by setting appropriate properties - refer this MSDN article for details.
So the steps are to rename your project to csproj, and add the following lines into project:
<PropertyGroup>
<OutputPath>$(Temp)\bin</OutputPath>
<IntermediateOutputPath>$(Temp)\obj</IntermediateOutputPath>
</PropertyGroup>
I usually use a bit different approach to work with MSBUILD files from VS:
I use regular csproj file with removed Import ... CSharp.targets part as pure container for my Build projects.
I add actual build files with targets and logic, and all properties, necessary artifacts like XSLT etc using "Include into project", so I can manage hierarchy and change any file from within VS.Net.
I redefine Build / Rebuild targets in csproj file for whatever I need, for example Build may contain minimum output, and while rebuild diagnostic one.
Like this:
<Target Name="Build">
<Exec Command="%windir%\Microsoft.NET\Framework64\v4.0.30319\msbuild.exe Builds\build.proj /t:Build /v:m" />
</Target>
<Target Name="Rebuild">
<Exec Command="%windir%\Microsoft.NET\Framework64\v4.0.30319\msbuild.exe Builds\build.proj /t:Build /v:d" />
</Target>

Use different pre-build events for different build configurations in Visual Studio

Is it possible to use different pre-build events for different build configurations in Visual Studio?
For example, I'd like both a release configuration for a beta & live system and have the relevant app.[type].config get copied to app.config before it is compiled.
At the moment the configuration settings are baked into the .settings file, using the settings from the default app.config file.
Or just put the Condition on your target ... eg.,
Condition="'$(Configuration)' == 'Debug'"
.. or on your task.
If you're using Visual Studio VB/C# simple post build events, you can hand-edit the project file to put such conditions on the PreBuildEvent/PostBuildEvent property tags; and repeat the tags for Release.
Dan (msbuild dev)
You can do this in a couple of ways, depending on your exact situation:
Option 1: Check the $(ConfigurationName) variable in your pre-build script, like so:
IF EXISTS $(ProjectDir)app.$(ConfigurationName).config
COPY $(ProjectDir)app.$(ConfigurationName).config $(ProjectDir)app.config
Option 2: Add a "BeforeCompile" MSBuild target to your project file:
<Target Name="BeforeBuild">
<!-- MSBuild Script here -->
</Target>
Option 3: Use configuration file transformations; this VSIX plug-in adds the web.config transform features to non-web projects. These are XSLT files that let you rewrite your config files on build (unlike web projects, where it happens on publish.)
To use different build events for different configuration in visual studio, open the cs proj file of the project. in the pre build section
<Target Name="PreBuild" BeforeTargets="PreBuildEvent">
<Exec Condition="'$(Configuration)'=='Release'" Command="echo Release" />
<Exec Condition="'$(Configuration)'=='Debug'" Command="echo Debug" />
</Target>
The command in "Command" parameter will only execute if this condition is met.

Resources