Octopack: Additional files copied in a pre-build event - octopus-deploy

I am trying to get Octopack to include some additional dll's that are not part of my Visual Studio project.
Is there a way to get Octopack to use my csproj files (as normal) and then specify just the additional files I need, rather than using the section in the nuspec. (Because then I have to maintain a list of all project files in my nuspec)
I am currently copying some shared libraries to a folder "/providers" in a pre-build event. Is there a way to extend on this to copy them somewhere that Octopack will see and package them?

You don't need to maintain a list of all your project files in your nuspec file, you can do this:
<files>
<file src="MyProject.Website\**\*.*" target="" />
</files>
This will grab everything in your project (all files and sub directories). If that's too much for you, you can take advantage of exclusion rules:
<files>
<file src="MyProject.Website\**\*.*" target="" exclude="MyProject.Website\docs\admin.txt" />
</files>
Hope that helps.

The downside of the answer of grabbing everything inside your project directory via the nuspec is that this could include files you don't want (see the answer's subsequent exclusion example).
A probably better way to do this is to ensure in your project that you have your files defined as "Copy to bin" or "Copy as content" and then build via msbuild w/Octopack using an additional command line argument of /p:OctoPackEnforceAddingFiles=true which tells OctoPack to combine the csproj and the nuspec, so you don't have to call out ALL the possible paths with a wildcard, you can only include files in your nuspec that aren't picked up from the regular build process. This is supported as of version 2.1.3 of OctoPack.
You could also use regular Nuget.exe and MSBuild by just specifying your csproj file, and if a nuspec exists with specific paths it should join them together.
See these reference links for better examples.
https://getadigital.com/no/blogg/setting-up-your-project-and-teamcity-octopack-for-front-end-builds/
https://octopus.com/docs/packaging-applications/nuget-packages/using-octopack/octopack-to-include-buildevent-files

Related

Prevent duplicating files in NuGet content and contentFiles folders

My NuGet package needs to deliver some rather large files to build output directory.
In an old NuGet model, such files have to be stored in content folder of the .nupkg. While in a new model introduced in NuGet 3.3, such files have to be stored in contentFiles folder.
To maintain a compatibility with older versions of NuGet and mainly with Package.config package management format, I need to duplicate the files into both folders. That unfortunately almost doubles a size of the package.
Is there a way to prevent that? Can I somehow link contentFiles to content folder?
Found updated documentation describing this in detail at
MSBuild targets for NuGet.
By default, everything gets added to the root of the content and contentFiles\any\<target_framework> folder within a package and preserves the relative folder structure, unless you specify a package path:
<Content Include="..\win7-x64\libuv.txt">
<Pack>true</Pack>
<PackagePath>content\myfiles\</PackagePath>
</Content>
If you want to copy all your content to only a specific root folder(s) (instead of content and contentFiles both), you can use the MSBuild property ContentTargetFolders, which defaults to "content;contentFiles" but can be set to any other folder names. Note that just specifying "contentFiles" in ContentTargetFolders puts files under contentFiles\any\<target_framework> or contentFiles\<language>\<target_framework> based on buildAction.
If you only want to output the file to the build output (content only copies the file to the output directory but does cause it to be set as copy to output directory item), you can use a completely different approach by creating an msbuild file that will be included in the project.
You can do this by putting both the file - say test.jpg into the tools folder (you could also use build) and add a Your.Package.Id.targets file to the build folder (the name being the package id of your package with .targets as extension) with the following content:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Content Include="$(MSBuildThisFileDirectory)..\tools\test.jpg">
<Link>test.jpg</Link>
<Visible>false</Visible>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
</Content>
</ItemGroup>
</Project>
This target will be automatically imported into the project files regardless of which "style" of NuGet reference is used (packages.config, PackageReference) and should be backwards compatible to older versions of VS as long as they support NuGet and ToolsVersion 4.0.
The Link metadata denotes where in the output / publish directories the file will end up. You could set it to e.g. defaultContent\images\foo.jpg to create a nested structure and rename the file. (you could even use MSBulid variables to use some of the referencing project's configuration). The Visible metadata prevents the solution explorer from showing the full relative path to the file, which could end up in lots of nested .. nodes. The CopyToPublishDirectory applies to .NET Core / ASP.NET Core apps or SDK-based projects using the Publish target for publishing.
Note that you can set the Inclue-path to anything depending on where in your package the file is. You can also use wildcards (but then set Link to %(Filename)%(Extension))

How to get Octopack to use custom .nuspec file?

The documentation seems to be really incomplete. All it says is that you can use your own .nuspec file, but it makes no mention of where you're supposed to put it, or how to get octopack to use it.
http://docs.octopusdeploy.com/display/OD/Using+OctoPack
I've tried naming the .nuspec file the same thing that my solution is named and having it in the same directory. That didn't work.
I've tried modifying the .nuspec file that Octopack generates, but those changes just get overwritten every time I run it.
Everything else I try is just a shot in the dark.
Has anyone gotten this to work?
If you want to use a custom nuspec file (something different than your.project.name.nuspec, there is a msbuild property OctoPackNuSpecFileName, through wich you can specify your nuspec file, like this:
msbuild yoursolution.sln /p:RunOctoPack=true p:OctoPackNuSpecFileName=Dev.nuspec
Another way to use a custom or conditional .nuspec file with OctoPack is by adding a PropertyGroup to your .csproj file, like this:
<PropertyGroup>
<RunOctoPack>true</RunOctoPack>
</PropertyGroup>
<PropertyGroup>
<OctoPackNuSpecFileName Condition="'$(Configuration)' == 'Release'">MyApp.nuspec</OctoPackNuSpecFileName>
<OctoPackNuSpecFileName Condition="'$(Configuration)' != 'Release'">MyApp.Debug.nuspec</OctoPackNuSpecFileName>
</PropertyGroup>
Put the files MyApp.nuspec and MyApp.Debug.nuspec in the same directory as your .csproj file and you are good to go.
blerg.
I'm a dummy.
The .nuspec file needs to be in the same directory as the .csproj file. This actually makes sense because it allows you to have a different nuspec file for each project in your solution.
Hopefully this post helps someone else.
Note, you don't have to have NuSpec as the OctoPack will generate one on the fly (based off the project file) if it's not there - that's what you were experiencing.
Granted, if you want to customize the NuGet package, the NuSpec is required. Check out the OctoPack source here: https://github.com/OctopusDeploy/OctoPack and you can see for yourself how the OctoPack works with the NuSpec.

What is incremental clean in msbuild and when is it triggered?

I am debugging a bug in my build process that happens occasionally but I can't directly reproduce it. I'm using msbuild with teamcity.
I have a dependency hierarchy like this:
Some.Interop.dll
Dependency-> SharedDllABC.dll
SomeService.exe
Depenendcy-> Some.Interop
Usually the final service exectuable gets in its release directory:
Some.Interop
SharedDllABC.Dll
ServiceExectuable.exe
However I can see in our msbuild logs that sometimes the tertiary dependency gets deleted during an Incremental Clean after everything is built resulting in:
Some.Interop
ServiceExectuable.exe
You can see it here in the msbuild log:
[src\SomeService\SomeService.csproj] _TimeStampAfterCompile
[12:32:43]: [src\SomeService\SomeService.csproj] Compile
// some other targets
[12:32:43]: [src\SomeService\SomeService.csproj] _CopyFilesMarkedCopyLocal
[12:32:43]: [_CopyFilesMarkedCopyLocal] Copy
[12:32:43]: [Copy] Copying file from "C:Projects\trunk\src\Some.Interop\bin\Release\Some.Interop.dll" to "bin\Release\Some.Interop.dll".
// some other targets
[src\Project\SomeService\SomeService.csproj] IncrementalClean
[18:54:42]: [IncrementalClean] Delete
[18:54:42]: [Delete] Deleting file "C:\Projects\trunk\src\Project\SomeService\bin\Release\SharedDllABC.dll".
[18:54:42]: [Delete] Deleting file "C:\Projects\trunk\src\Project\SomeServiceService\bin\Release\SharedDllABC.pdb".
[18:54:42]: [src\Project\SomeService\SomeService.csproj] CoreBuild
[18:54:42]: [src\Project\SomeService\SomeService.csproj] AfterBuild
[18:54:42]: [src\Project\SomeService\SomeService.csproj] Build
This is my direct msbuild output, I just changed the project names/dll names to match my example. By the time this Incremental Clean has occurred the SomeService.csproj has already been built. You can see that its not getting copied. However in other msbuild logs it does properly get copied and then the incremental clean doesn't delete it.
I think incrementeal clean from this post is supposed to clean dll's that were created from previous builds, but that doesn't explain how this dll didn't get built when most of the time it does. In visual studio this always works as well.
I guess I just want to know what exactly is Incremental clean, what causes it to kick in, and maybe what things I should look for when debugging a situation like this (assembly versions, timestamps, etc?)
Try the following:
Add:
<Target Name="IncrementalClean" />
to a .targets file that's included in all projects.
From --> https://github.com/Microsoft/msbuild/issues/1054
#Kebabbi recommends a good fix by editing a csproj file. As of MSBuild 15, there is a simple way to make this apply to all CSPROJ files, instead of editing each csproj file.
https://learn.microsoft.com/en-us/visualstudio/msbuild/customize-your-build?view=vs-2017
Directory.Build.props and Directory.Build.targets
Prior to MSBuild version 15, if you wanted to provide a new, custom property to projects in your solution, you had to manually add a reference to that property to every project file in the solution. Or, you had to define the property in a .props file and then explicitly import the .props file in every project in the solution, among other things.
However, now you can add a new property to every project in one step by defining it in a single file called Directory.Build.props in the root folder that contains your source. When MSBuild runs, Microsoft.Common.props searches your directory structure for the Directory.Build.props file (and Microsoft.Common.targets looks for Directory.Build.targets). If it finds one, it imports the property. Directory.Build.props is a user-defined file that provides customizations to projects under a directory.
Create a file Directory.Build.props, and place it adjacent to the SLN file.
<Project>
<Target
Name="ForceAssignProjectConfigurationBeforeSplitProjectReferencesByFileExistence_KLUDGE"
BeforeTargets="_SplitProjectReferencesByFileExistence"
DependsOnTargets="AssignProjectConfiguration" />
</Project>
This could be caused by a bug in MsBuild: https://github.com/Microsoft/msbuild/issues/1054.
A fix is proposed in the comments: https://github.com/Microsoft/msbuild/issues/1054#issuecomment-406438561
When MsBuild determines which items to copy from referenced projects, it should do this recursively but does not properly do this.
As a workaround the following can be added to each csproj.
<Target
Name="ForceAssignProjectConfigurationBeforeSplitProjectReferencesByFileExistence_KLUDGE"
BeforeTargets="_SplitProjectReferencesByFileExistence"
DependsOnTargets="AssignProjectConfiguration"
/>
I just spent a few days trying to figure this out with a similar pattern. In our case it was nuget files that were being removed from the output folder.
NugetPackage (that drops files in x86/x64 subfolders in output folder)
LibraryA.dll
Dependency-> NugetPackage
LibraryB.dll
Dependency-> LibraryA.dll
In our case, we have a number of solution files that are built as part of an msbuild script in a certain order.
The problem was that LibraryB.csproj was included in two solution files.
Solution1 builds and output files are all present.
Solution2 builds and sees that LibraryB.dll is present and up to date, so for some reason triggers the IncrementalClean that removes the NugetPackage files from the output folder.
Once I removed the LibraryB.csproj from solution 2, the problem is solved and the files are present in the output folder.

Project file with just files and no built output

How can I make a project file (VS 2008) that just has some data files in and has no built output?
I can make an empty project and add my data files to it (which get copied to the output folder
), but it produces an EmptyProject.dll after I do a build. I want just my data files in the output directory and not some empty DLL or EXE.
I want the data files to be the only thing in this project as the project will be shared in a couple of solutions.
Our application is C#. All of our normal code projects are C#.
The data files are schemas (XSD). I want these schemas to be in the output folder, but I don't want them included with an existing project. I would like a project named "Schemas" that has nothing in except the XSD files and does nothing except copy the XSD files to the output folder. I would like this in a project file so that the same schemas project can be referenced in multiple solutions.
I don't know of a way to suppress the creation of the .dll file. BUT... here's an easy workaround. In the project properties, Build Events tab, write a Post-build event command line that will delete the file. Something like:
del path\filename.dll
Expanding on Scott's answer:
Create a new project of type Empty project
In Properties->Application, change Output type to Class Library
In Properties->Build->Advanced, change Debug Info to None
In Properties->Build Events, set the Post-build event command line to del $(TargetPath)
That way, the project creates only a DLL, which gets deleted. At the same time, the "copy to output directory" settings on your data files is respected.
Possibly another way is editing the csproj file by replacing this:
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
with this:
<Target Name="Build" />
<Target Name="Rebuild" />
Then builds don't create anything. It worked for me.
Same general idea should work for any xxproj file. Just replace the <Import Project...> tags with the <Target...> tags.
I'd be interested in knowing if this causes any issues or doesn't work for anyone.
What do you need a project for if you're not building it?
You can use solution folders to "store" files...
Why not just disable building this project for all configurations (use the Configuration Manager) - that way it won't build.
Great stuff. Expanding on Scott > Daniel's answer:
Safe to remove all References and Properties (AssemblyInfo.cs)
If it is a node/grunt/gulp project then you can invoke it in your Build Events > *Post-build event command line * eg: gulp build or gulp clean
Perhaps you can add removal or obj and bin output folders to your node/grunt/gulp clean scripts mitigating the need for del $(TargetPath)

Checking a file out (TFS) for a pre-build action

I've added a pre-build action for an ASP.NET web control (server control) project, that runs jsmin.exe on a set of Javascript files. These output files are part of the source control tree and are embedded into the assembly.
The problem is when the pre-build runs, jsmin can't write the file as it's readonly. Is it possible to check the file out before hand? Or am I forced to set the file's attributes in the command line.
Any improved solution to the problem is welcome.
Update
One small issue with Mehmet's answer -you need to prepend the VS directory:
"$(DevEnvDir)tf" checkout /lock:none "$(ProjectDir)myfile"
If you're using Team Foundation Server, you can use team foundation command line utility (tf.exe) to check out the file(s) during pre-build and then check them back in during post-build. If you're using something else for source control, you can check if they have a command line tool like tf.exe.
If you do not want to check the files in as part of the build (which you normally wouldn't for this sort of thing) then I would simply set the attributes of the .js files before running jsmin on them. The easiest way of setting the files read-writeable is to use the the Attrib task provided by the MSBuild community extensions. The same community extensions also provide a JSCompress task for easily calling JSMin from MSBuild.
Therefore you'd have something like the following (not tested):
<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets" />
<!-- rest of TFSBuild.proj file -->
<Target Name="AfterGet">
<Message Text="Compressing Javascript files under "$(SolutionRoot)"." />
<CreateItem Include="$(SolutionRoot)\**\*.js">
<Output TaskParameter="Include" ItemName="JsFiles"/>
</CreateItem>
<Attrib Files="#(JsFiles)" ReadOnly="false"/>
<JSCompress Files="#(JsFiles)" />
</Target>
Note that by modifying the files after getting them may well cause issues if you tried to move to an incremental build.

Resources