The problem:
I want to manage my database(s) using one SSDT database project. in which I want to centralize database development and automate deployments (mainly stored procedures).
In a multi-tenant environment, where object names are preceded by company names,
example :
[dbo].[spu_COMPANY_NAME$Stored Procedure Name]
we have a central database in which we do our development and every time we publish, we do a 'Replace All' to the company name.
The SQLCMD variables won't do because they cannot be included inside object names.
Is there a way I can build so that for every build configuration I get tailored stored procedures during build/publish, I get a folder structure like this :
--Database.Project/
--bin/
--CompanyA(build.companyA.congig)/
--CompanyB(build.companyB.congig)/
As #PeterSchott mentioned in a previous comment, the solution is easily doable through Powershell scripts and Pre/Post build events in Visual Studio.
I'll post the answer here in case someone needed it for reference (or maybe in case I needed it)
First:
Prebuild powershell script, this one takes the project directory, and replaces all strings with 'COMPANY_NAME' to the actual company name depending on the build configuration, for all files with .sql extension in them.
Next:
The powershell script:
if ($args[0] -eq "Release"){
$company_name = "ReleaseCorp";
}
$dboDir = $args[1] + 'dbo\';
$sqlFiles = Get-ChildItem $dboDir -rec | Where-Object {($_.Extension -eq ".sql")}
foreach ($file in $sqlFiles)
{
echo $file;
(Get-Content $file.PSPath) |
Foreach-Object { $_ -replace "COMPANY_NAME", $company_name } |
Set-Content $file.PSPath
}
Post-build event does the inverse, only that this time it takes the actual company name for the configuration and puts it back to the token, for upcoming builds.
Worth noting that for Visual Studio to be able to run Powershell (x86) scripts, the following step is mandatory :
Run Windows Powershell (x86) in administrator mode, and run :
set-executionpolicy unrestricted
More info about this to be found in David Frette's informative blog post :
Creating Powershell pre-build and post-build events for Visual Studio projects
Related
I am working on a large scale application which uses multiple small-small project/solutions. Each solution is built in their respective agents. Similar to below screenshot
Now the problem is that all the projects/solutions are compiled even if a single project file is changed.
To reduce CI build time, I want to add conditional expression in the build which would build the project only if their respective source is changed.
I know this can be achieved in Azure DevOps via Custom condition using variable expressions. But I am not sure how can I check if the respective source code is changed
Does anyone know what variable expression I need to write here?
Well as I see your are using the Git Repo, you can do this in the below way
As a very first step you need to find whether which project/solution is modified.
You can find the solution from my answer here,where you can see I've used a simple powershell script to pull out the modified files, and enables the corresponding variables.
Sample Powershell script to pull-out the modified files
$files=$(git diff HEAD HEAD~ --name-only)
$temp=$files -split ' '
$count=$temp.Length
echo "Total changed $count files"
For ($i=0; $i -lt $temp.Length; $i++)
{
$name=$temp[$i]
echo "this is $name file"
if ($name -like "SubFolderA/*")
{
Write-Host "##vso[task.setvariable variable=MicroserviceAUpdated]True"
}
}
you cant achieve that with condition. condition are for executing tasks inside build. you are looking for build triggers in yaml files:
trigger:
branches:
include:
- master
paths:
include:
- path\to\app\*
this build will only trigger when anything under `path\to\app\ folder was changed. that way you create a build per app and can isolate those builds to specific files changed.
https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema#pipeline
I have post build events set up already for copying files for me dependent on ConfigurationName and want to be able to set an "environment variable" in a config (js) file on client side of an angular application to allow debug info to be visible or not dependent upon environment running application in.
To this end I've created a powershell script (ReplaceText.ps1):
function replace-file-content([String] $path, [String] $replace, [String] $replaceWith)
{
(Get-Content $path) | Foreach-Object {$_ -replace $replace,$replaceWith} | Out-File $path
}
and added this line to the post build event of my web project.
if "$(ConfigurationName)"=="LIVE" (powershell -File "$(ProjectDir)Tools\ReplaceText.ps1" "$(ProjectDir)app\application.config.js" "DEBUG" "LIVE")
which I was hoping to change the word "DEBUG" to "LIVE" when built against LIVE build configuration in my application.config.js file which contains this line:
$provide.constant('currentEnv', 'DEBUG');
Build succeeds but no changes occur on my file. Can anyone identify where I'm going wrong?
I do know I could do this sort of stuff with Gulp or another task runner BTW, but was trying to do it without bringing in another dependency and just use VS post build events & PS. :)
Cheers
Your PS code only defines a function but there's nothing that invokes it.
Use param as the first statement in the script to convert command line into the script's parameters:
param([String] $path, [String] $replace, [String] $replaceWith)
(Get-Content -literalPath $path -raw) -replace $replace, $replaceWith |
Out-File $path -encoding UTF8
-literalPath - correctly handles paths with [] symbols otherwise interpreted as a wildcard;
-raw - reads the entire file as one string for speedup, available since PowerShell 3.
I am using a visual studio c# library project to contain static resources that are needed as deployment artifacts. (in my case SQL files that are run with a combination of RoundhousE and Octopus deploy). By convention all files in the project must have their properties set so that the "Build action" is "Content" and "Copy to output directory" is "Copy always".
If someone on the team adds a file but forgets to set these properties we see deployment errors. This is usually picked up in an internal environment, but I was hoping to find a way to enforce this in the CI build.
So is there a way to either fail the build or better still override these properties during the build with an MS Build task? Am I tackling this the wrong way? Any suggestions welcomed.
You are going to have to parse the project files and check for Content without CopyToOutputDirectory set to Always, I doubt there is another way.
That can be done using whatever scripting language you want, or you could even write a small C# tool that uses the classes from the Microsoft.Build.Evaluation namespace. Here is a possible PowerShell implementation - the hardest part is getting the regexes right. First one checks for Content without any metadata, second one for Content where CopyToOutputDirectory does not start with "A" (which I assume should be "Always", no idea how to match that whole word).
FindBadContentNodes.ps1 :
param([String]$inputDir)
Function FindBadContent()
{
$lines = Get-Content $input
$text = [string]::Join( "`n", $lines )
if( $text -match "<Content Include.*/>" -Or
$text -match "<Content Include.*`n\s*<CopyToOutputDirectory>[^A]\w*<.*" )
{
"Found file with bad content node"
exit 1
}
}
Get-ChildItem -Recurse -Include *.csproj -Path $inputDir | FindBadContent
Call this from MsBuild:
<Target Name="FindBadContentNodes">
<Exec Command="Powershell FindBadContentNodes.ps1 -inputDir path\to\sourceDir"/>
</Target>
Note you mention or better still override these properties during the build. I'd stay away from such a solution: you're just burying the problem and relying on the CI to produce correct builds, so local builds using just VS would not be the same. Imo making the build fail is better, especially since most CI systems have a way of notifying the developper that is responsible anyway so the fix should be applied quickly.
Another possibility would be to have the CI apply the fix and then commit the changes so at least everyone has the correct version.
IIRC there is a way in Visual Studio to set a file extension to do certain things on default, much like .config files will always set to content and copy to output directory.
So one could do the same with .sql files (and other files that they would want to be set up this way). A quick search brought me to this: http://blog.andreloker.de/post/2010/07/02/Visual-Studio-default-build-action-for-non-default-file-types.aspx
The relevant parts:
The default build action of a file type can be configured in the
registry. However, instead of hacking the registry manually, we use a
much better approach: pkgdef files (a good article about pkgdef
files). In essence, pkdef are configuration files similar to .reg
files that define registry keys and values that are automatically
merged into the correct location in the real registry. If the pkgfile
is removed, the changes are automatically undone. Thus, you can safely
modify the registry without the danger of breaking anything – or at
least, it’s easy to undo the damage.
Finally, here’s an example of how to change the default build action
of a file type:
1: [$RootKey$\Projects{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}\FileExtensions.spark]
2: "DefaultBuildAction"="Content" The Guid in the key refers to project type. In this case, “{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}” means “C# projects”. A rather comprehensive list of project type guids can be found here. Although it does not cover Visual Studio 2010 explicitly, the Guids apply to the current version as well. By the way, we can use C# as the project type here, because C# based MVC projects are in fact C# projects (and web application projects). For Visual Basic, you’d use “{F184B08F-C81C-45F6-A57F-5ABD9991F28F}” instead.
$RootKey$ is in abstraction of the real registry key that Visual
Studio stores the configuration under:
HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\10.0_Config (Note:
Do not try to manually edit anything under this key as it can be
overwritten at any time by Visual Studio).
The rest should be self explanatory: this option sets the default
build action of .spark files to “Content”, so those files are included
in the publishing process.
All you need to do now is to put this piece of text into a file with
the extension pkgdef, put it somewhere under
%PROGRAMFILES(x86)%\Microsoft Visual Studio
10.0\Common7\IDE\Extensions (on 64-bit systems) or %PROGRAMFILES(x86)%\Microsoft Visual Studio
10.0\Common7\IDE\Extensions (on 32-bit systems) and Visual Studio will load and apply the settings automatically the next time it starts. To
undo the changes, simply remove the files.
Finally, I’ve attached a bunch of pkgdef files that are use in
production that define the “Content” default Build Action for C# and
VB projects for .spark, .brail, .brailjs and .less files respectively.
Download them, save them somewhere in the Extensions folder and you’re
good to go.
The author also says that he built a utility to help do all of this for you:
http://tools.andreloker.de/dbag
Expanding on #stijn answer, instead of using regex it is far easier to use native xml parsing.
Here is my proposed file, it also supports the ability to customize which files are evaluated by using a regex on the filename only.
param([String]$Path, [string]$IncludeMatch, [switch]$AllowPreserve)
Function Test-BadContentExists
{
param (
[parameter(Mandatory=$true,ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)]
[Alias("FullName")]
[string[]]$Path,
[string]$IncludeMatch,
[switch]$AllowPreserve
)
[xml]$proj = Get-Content -Path $Path
$ContentNodes = ($proj | Select-Xml "//Content|//n:Content" -Namespace #{n='http://schemas.microsoft.com/developer/msbuild/2003'}).Node
if (![string]::IsNullOrEmpty($IncludeMatch)) {
$ContentNodes = $ContentNodes | Where-Object -Property Include -Match $IncludeMatch
}
#remove the always nodes
$ContentNodes = $ContentNodes | Where-Object -Property CopyToOutputDirectory -ne 'Always'
#optionally remove the preserve nodes
if ($AllowPreserve) {
$ContentNodes = $ContentNodes | Where-Object -Property CopyToOutputDirectory -ne 'PreserveNewest'
}
if($ContentNodes)
{
write-output "Found file with bad content node:"
write-output ($ContentNodes | Select-Object Include,CopyToOutputDirectory | sort Include | Out-String)
exit 1
}
}
[hashtable]$Options = $PSBoundParameters
[void]$Options.Remove("Path")
Get-ChildItem -Recurse -Include *.csproj -Path $Path | Test-BadContentExists #Options
and calling it, with parameter:
<Target Name="FindBadContentNodes">
<Exec Command="Powershell FindBadContentNodes.ps1 -inputDir path\to\sourceDir -IncludeMatch '^Upgrade.*\.(sql|xml)$'"/>
</Target>
I ended up using a pre-build event instead and put this ps1 file in my solution directory so i could use it with multiple projects.
echo "Build Dir: %cd%"
echo "Sol Dir: $(SolutionDir)"
echo "Proj Dir: '$(ProjectDir)"
echo.
Powershell -NoProfile -Command "& '$(SolutionDir)\FindBadContentNodes.ps1' -Path '$(ProjectDir)' -IncludeMatch '^Upgrade.*\.(sql|xml)$'"
example build output:
1> "Build Dir: C:\Source\RPS\MRM BI\MRMBI-Setup\MRMBI-Schema\bin\Debug"
1> "Sol Dir: C:\Source\RPS\MRM BI\MRMBI-Setup\"
1> "Proj Dir: 'C:\Source\RPS\MRM BI\MRMBI-Setup\MRMBI-Schema\"
1>
1> Found file with bad content node:
1>
1> Include CopyToOutputDirectory
1> ------- ----------------------
1> Upgrades\V17.09\myfile1.sql
1> Upgrades\V20.05\myfile2.sql PreserveNewest
1>
During nuget install I give the user a command they can run. This command basically scans some files and creates some code templates and then inserts them into the current project. This works just fine - except for the fact that Solution Explorer does not update its tree view with the new files. I know this works because I can unload and reload the project file and the files are there.
In case it helps, here is the code I use to add the files to the project - the second function is what the user actually calls.
function add-to-project ($itemType, $project)
{
process
{
$bogus = $project.Xml.AddItem($itemType, $_)
}
}
# Parse a file
function Write-TTree-MetaData ($Path = $(throw "-Path must be supplied"))
{
$p = Get-Project
Write-Host "Inserting the results of the parsing into project" $p.Name
$ms = Get-MSBuildProject
$destDir = ([System.IO.FileInfo] $p.FullName).Directory
# Run the parse now
CmdTFileParser -d $destDir.FullName $Path
# Now, attempt to insert them all into the project
$allFiles = Get-ChildItem -Path $destDir.FullName
$allFiles | ? {$_.Extension -eq ".ntup"} | add-to-project "TTreeGroupSpec" $ms
$allFiles | ? {$_.Extension -eq ".ntupom"} | add-to-project "ROOTFileDataModel" $ms
# Make sure everything is saved!
$ms.Save()
$p.Save()
}
This code causes a funny dialog to pop up - "The project has been modified on disk - please reload" - and hopefully the user will reload, and then the files show up correctly... But it would be nice to avoid that and just have the script cause whatever is needed to happen. Perhaps I have to figure out how to unload and re-load the project?
What do I need to do to force solution explorer to update? Many thanks!
By using the MSBuild project you are bypassing Visual Studio and directly updating the MSBuild project file on disk. The easiest way to get Visual Studio to update the Solutions Explorer window is to use the Visual Studio project object instead which you get from the Get-Project command.
Below is a simple example which adds a file to the solution and changes its ItemType to be ROOTFileDataModel. The example assumes you have a packages.config file in your project's root directory which is not currently added to the project so it is not showing in Solution Explorer initially.
# Get project's root directory.
$p = Get-Project
$projectDir = [System.IO.Path]::GetDirectoryName($p.FileName)
# Add packages.config file to project. Should appear in Solution Explorer
$newFile = [System.IO.Path]::Combine($projectDir, "packages.config")
$projectItem = $p.ProjectItems.AddFromFile($newFile)
# Change file's ItemType to ROOTFileDataModel
$itemType = $projectItem.Properties.Item("ItemType")
$itemType.Value = "ROOTFileDataModel"
# Save the project.
$p.Save()
The main Visual Studio objects being used here are the Project, ProjectItem and the ProjectItems objects. Hopefully the above code can be adapted to your specific requirements.
I'm trying to configure Windows Powershell to work with Visual Studio. Nothing fancy, just get things set so I can cl & nmake. I think all I need to do is edit the path setting(but I don't know how to set that in WPSH).
After much digging around, I created a directory in My Documents named WindowsPowerShell, created a file named Microsoft.PowerShell_profile.ps1, and(after a few iterations), inserted the following code in it, and it works.
function Get-Batchfile($file)
{
$theCmd = "`"$file`" & set"
cmd /c $theCmd | Foreach-Object {
$thePath, $theValue = $_.split('=')
Set-Item -path env:$thePath -value $theValue
}
}
Get-Batchfile("C:\Program Files\Microsoft Visual Studio 9.0\VC\vcvarsall.bat")
Thanks everyone for their kind help. :-)
You might want to check out this post -- it seems to address your question fairly well (it worked for what I wanted, anyway).