How To set Azure pipeline variable from PowerShell - continuous-integration

I am trying to set the Azure pipeline variable value in PowerShell. I have created one variable winversion in the Azure pipeline. Now, in a PowerShell task, I want to assign some values to the winversion variable.
My simple question is how can I change the value of an Azure PipeLine variable at run time?
Write-Host "Main value is $winversion"
$env:WINVERSION="abhinav";
Write-Host "Modified value is $env:WINVERSION"
Write-Host "Main value is $(winversion)"
Firstline print: original value is 123
Thirdline Print: Modified value is abhinav
Fourth Line print: 123
I want when I change the value of winversion from "123" to "abhinav" so it actually changes the pipeline variable value to abhinav.
I want to update this variable through Powershell. I am using one PowerShell script calling the API and trying to update its variable but getting the page not found error:-
param(
[string]$winVersion
)
$body = "{ 'definition' : { 'id' :85}
}"
$valueName="Winver"
$definitionId=85
$User=""
$Password=""
$base64authinfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $User, $Password)))
$Uri = "https://Muac.visualstudio.com/OSGCXE/_apis/release/releases?api-version=2.0"
$urlDef = "https://Muac.visualstudio.com/OSGCXE/_apis/release/definitions/" + $definitionId + "?api-version=2.0"
$definition = Invoke-RestMethod -Headers #{Authorization=("Basic {0}" -f $base64authInfo)} -Method Get -Uri $urlDef
#Write-Host $definition
$definition.variables.$valueName.Value = "$winVersion"
$definitionJson = $definition | ConvertTo-Json -Depth 50 -Compress
#Write-Host (ConvertTo-Json $definition -Depth 100)
$update=Invoke-RestMethod -Headers #{Authorization=("Basic {0}" -f $base64authInfo)} -Method Put -Uri $urlDef -Body $definitionJson -ContentType "application/json"
#Write-Host "$update"
#$buildresponse = Invoke-RestMethod -Method Post -ContentType application/json -Uri $Uri -Headers #{Authorization=("Basic {0}" -f $base64authinfo)} -Body $body
#write-Host $buildresponse.status

How To set azure pipeline variable from PowerShell
There is a bit of confusion here, you use the variable $winversion in the powershell scripts, but the variable is set testvar in the pipeline variable.
Anyway, no matter we overwrite the pipeline variable value directly like you, or use the script "##vso[task.setvariable variable=testvar;]testvalue" to overwrite it, the overwrite value only work for current build pipeline. When you use the $(winversion) to get the value, it will still pull the value from pipeline variable value. To get the current value, you need use $env:WINVERSION.
Besides, you said:
I want when I change the value of winversion from "123" to "abhinav"
so it actually changes the pipeline variable value to abhinav.
If you mean you want change the pipeline variable value on the web portal, you need the REST API (Definitions - Update) to update the value of the build pipeline definition variable from a build task.
There is a very similar thread, you can check the answer for the details:
How to modify Azure DevOps release definition variable from a release task?
Note:Change the API to the build definitions:
PUT https://dev.azure.com/{organization}/{project}/_apis/build/definitions/{definitionId}?api-version=5.0
Hope this helps.

I found this link helpful: https://learn.microsoft.com/en-us/azure/devops/pipelines/scripts/logging-commands?view=azure-devops&tabs=powershell
This has the complete options of what you can do:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch
You can reuse set variable from task to task, and also job to job.
I couldn't find anything on stage to stage.
In summary:
jobs:
# Set an output variable from job A
- job: A
pool:
vmImage: 'vs2017-win2016'
steps:
- powershell: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the value"
name: setvarStep
- script: echo $(setvarStep.myOutputVar)
name: echovar
# Map the variable into job B
- job: B
dependsOn: A
pool:
vmImage: 'ubuntu-16.04'
variables:
myVarFromJobA: $[ dependencies.A.outputs['setvarStep.myOutputVar'] ] # map in the variable
# remember, expressions require single quotes
steps:
- script: echo $(myVarFromJobA)
name: echovar

Related

Assigning a variable to a variable in Github action variables

I'm new to yaml & Github & actions and trying to figure out how to assign x=y vs. x=value. Is this possible? In the example below, I'm trying to assign CertificatePath as a global variable to a concatenation of several directories.
env:
Solution_Name: WpfApp3.sln
Test_Project_Path: TestProject1\TestProject1.csproj
Wap_Project_Directory: WapProjTemplate1
Wap_Project_Path: WapProjTemplate1\WapProjTemplate1.wapproj
SigningCertificate: GitHubActionsDemo.pfx
- name: SetCertPath
run:
$currentDirectory = Get-Location
tempcertificatePath= Join-Path -Path $currentDirectory -ChildPath $env:Wap_Project_Directory -AdditionalChildPath $env:SigningCertificate
echo "CertificatePath=${{ tempcertificatePath }}" >> $GITHUB_ENV
Thanks!
If you want to run multiple lines as part of a run step, you need to use a |. Furthermore, when assigning a variable, you don't need to use the $ on the left hand side.
Example:
- run: |
myVar="foo"
mySecondVar="${myVar}bar"
echo $mySecondVar

Azure build fail if branch doesn't have latest master

I have the following bash script written in Azure pipeline:
if [[ "$(Build.Tag)" == "blabla" && "$(Build.SourceBranch)" != *"blabla"* ]];
then
run some git command to get the diff between current branch and master
if branch doesnt have the lastest master fail the build
else
echo "Passed"
fi
I want to fail the build of the current branch if it does't have the latest changes from master.
Is there any way to do it ?
I want to fail the build of the current branch if it does't have the latest changes from master. Is there any way to do it ?
Yes. Here is an example:
if [ -z "$(git diff origin/bran1 origin/bran2)"]
then
echo "ERROR!"
echo "ERROR" > logfile.log
exit 125
fi
Check this REST API Get Changes Between Builds, it will get the changes made to the repository between two given builds. if the build commit ID is same, the response body count value is 0.
Add task power shell and enter below script:
$PAT="$(pat)"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($PAT)"))
#Get the previous build commit id
$GetPreviousURL = "https://dev.azure.com/{Org name}/{Project name}/_apis/build/builds?definitions={Build definition ID}&api-version=6.1-preview.6"
$GetPreviousBuild = Invoke-RestMethod -Uri $GetPreviousURL -Headers #{Authorization = "Basic {0}" -f $base64AuthInfo} -Method get
$PreviousBuildID=$GetPreviousBuild.value.id[1]
#Write-Host $PreviousBuildID
#Get the changes made to the repository between two given builds
$CompareBuildURL= "https://dev.azure.com/{Org name}/{Project name}/_apis/build/changes?fromBuildId=$($PreviousBuildID)&toBuildId=$($env:BUILD_BUILDNUMBER)&api-version=6.1-preview.2"
#Write-Host $CompareBuildURL
$CompareBuildResult = Invoke-RestMethod -Uri $CompareBuildURL -Headers #{Authorization = "Basic {0}" -f $base64AuthInfo} -Method get
$Count = $CompareBuildResult.count
#Write-Host $Count
#If the count is 0, fail this build
if ($Count.equals(0)) {
throw 'it does not have the latest changes from master (exit 1)'
}
Result:
it does not have the latest changes.
it have the latest changes.

azure devops pipeline | bash script to run dotnet test on multiple projects

I'm trying to run the dotnet test command on defined test projects in a MS Azure devops pipeline.
This is working:
- script: |
dotnet test "./Project One/Project One Unit Tests.fsproj" -c Release --no-build --filter "TestCategory!=SKIP_ON_DEPLOY & TestCategory!=REQUIRES_API_KEY"
dotnet test "./Project Two/Project Two Unit Tests.fsproj" -c Release --no-build --filter "TestCategory!=SKIP_ON_DEPLOY & TestCategory!=REQUIRES_API_KEY"
displayName: 'run tests in cascade'
I don't want to specify the project names, only some rule (project name has to end with "UnitTests", "Unit Tests" or "Unit_Tests") but dotnet test <PROJECT> does not allows wildcards.
Seems like wildcards works with dotnet test <DLL> (./**/*Unit?Tests.dll) but it fails because it does not find the deps.json file on the obj folder.
My solution is to loop trough a filtered list of project files:
for proj in ./**/*Unit?Tests.*proj
do
dotnet test "$proj" -c Release --no-build --filter "TestCategory!=SKIP_ON_DEPLOY & TestCategory!=REQUIRES_API_KEY"
done
It's running the tests but differently from the cascade calls, here when a project test fails it does not fail the step so the pipeline does not stop!
I tried to get the result form the run, but unsuccessfully (this does not work):
for proj in ./**/*Unit?Tests.*proj
do
result=$(dotnet test "$proj" -c Release --no-build --filter "TestCategory!=SKIP_ON_DEPLOY & TestCategory!=REQUIRES_API_KEY")
if result = 1
then
exit 1
fi
done
Any suggestion?
Why when dotnet test fails (exit 1) the step does not fail?
I tried to put in the loop only the failing test project and it fails in that case.
[UPDATE]
Full pipeline.yaml:
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
variables:
project file: "Alex75.MySolution/Alex75.MyProject.fsproj"
steps:
- script: dotnet build -c Release
displayName: 'Build'
- script: |
dotnet test "./ProjectTwo Unit Tests/ProjectTwo Unit Tests.fsproj" -c Release --no-build --filter "TestCategory!=SKIP_ON_DEPLOY & TestCategory!=REQUIRES_API_KEY"
dotnet test "./ProjectOne Tests/ProjectOne Unit Tests.fsproj" -c Release --no-build --filter "TestCategory!=SKIP_ON_DEPLOY & TestCategory!=REQUIRES_API_KEY"
displayName: 'Test'
condition: false
- script: |
for proj in ./**/*Unit?Tests.*proj
do
echo "run tests in $proj"
dotnet test "$proj" -c Release --no-build --filter "TestCategory!=SKIP_ON_DEPLOY & TestCategory!=REQUIRES_API_KEY"
done
displayName: 'Test (Unit Test projects)'
condition: true
- powershell: |
$URL = "$(System.CollectionUri)/$(System.TeamProject)/_apis/build/builds/$(Build.BuildId)/logs?api-version=5.1"
Write-Host "URL = $URL"
$logs = Invoke-RestMethod -Uri $URL -Headers #{authorization = "Basic $(PAT)"} -Method Get
$lastLogId = $Logs.value[$Logs.value.count-1].id
Write-Host "lastLogId = $lastLogId"
$URL = "$(System.CollectionUri)/$(System.TeamProject)/_apis/build/builds/$(Build.BuildId)/logs/$lastLogId?api-version=5.1"
$result = Invoke-RestMethod -Uri $URL -Headers #{authorization = "Basic $(PAT)"} -Method Get
Write-Host $result
Write-Host "Start Check result..."
$lines = $result.Split([Environment]::NewLine)
foreach($line in $lines) {
if($line -match "Failed!")
{
throw 'dotnet test fails ($line)'
}
}
Write-Host "Test result check completed."
displayName: 'Check tests result'
PowerShell script
Using the example of #vito-liu-msft I tried to check the test logs to check the error.
Here the powershell script alone:
$URL = "$(System.CollectionUri)/$(System.TeamProject)/_apis/build/builds/$(Build.BuildId)/logs?api-version=5.1"
Write-Host "URL = $URL"
$logs = Invoke-RestMethod -Uri $URL -Headers #{authorization = "Basic $(PAT)"} -Method Get
$lastLogId = $Logs.value[$Logs.value.count-1].id
Write-Host "lastLogId = $lastLogId"
$URL = "$(System.CollectionUri)/$(System.TeamProject)/_apis/build/builds/$(Build.BuildId)/logs/$lastLogId?api-version=5.1"
$result = Invoke-RestMethod -Uri $URL -Headers #{authorization = "Basic $(PAT)"} -Method Get
Write-Host $result
Write-Host "Start Check result..."
$lines = $result.Split([Environment]::NewLine)
foreach($line in $lines) {
if($line -match "Failed!")
{
throw 'dotnet test fails ($line)'
}
}
Write-Host "Test result check completed."
PAT is the Personal Access Token as it is created, no need to be transformed
before using it in the HTTP request.
The LogId is not available so there is a request to get the logs collection and than a second request to get the specific last log (the response seems ordered by date, maybe it's worth to double check).
Note that if the second request returns a 404, 500 or other result (not a failure) the error check will not find a evenctual errors! A proper check of the response is required.
I tested the -match "Failed!" in PowerShell but I haven't tested the throw command in the pipeline because...
dotnet test command in the loop is working!
After spending so much time trying to figure out how to read and check the logs throw a separate PowerShell script I find out that the initial simple solution works!
Yes, the build fails because of the test failure (it also shows the precise error pointing to the problem) and the step fails.
(so the next step, the check of the test results, is skipped!)
I think I should have made some error before on filtering the test projects so that the failing project was not running (and not raising any error) so I thoutght it was not "intercepting" the error, and the build didn't stop.
It could be that I mixed 2 different pipelines files while I was changing it.
Anyway this is the incriminated step (script):
for proj in ./**/*Unit?Tests.*proj
do
echo "run tests in $proj"
dotnet test "$proj" -c Release --no-build --filter "TestCategory!=SKIP_ON_DEPLOY & TestCategory!=REQUIRES_API_KEY"
done
At list with the echo it is possible to see which test projects are used.
when a project test fails it does not fail the step so the pipeline does not stop!
Why when dotnet test fails (exit 1) the step does not fail?
As a workaround, we could get the dotnet test task log id via this rest api, add the task power shell and enter below script to analyze dotnet test log. we need to enter match code, such as fails (exit 1), If dotnet test fails it will stop the pipeline.
We should add PAT to variable and set it to secret, then use it in the script
$connectionToken="{PAT}"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($connectionToken)"))
$URL = "https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}/logs/{logId}?api-version=6.1-preview.2"
$Result = Invoke-RestMethod -Uri $URL -Headers #{authorization = "Basic $base64AuthInfo"} -Method Get
Write-Host $result
$lines = $result.Split([Environment]::NewLine)
$passed = 0;
$failed = 0;
foreach($line in $lines) {
if ($line -match "{match sentence}") {
throw 'dotnet test fails (exit 1)'
}
}
Update1
how can I find the "logId" ?
We could use this REST API to check logId
GET https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}/logs?api-version=6.1-preview.2
Result:
And I really have to use a api "preview" version?
We could also use other version, such as 5.1, we could switch the REST API version in the doc, you could check the pic below.

Use data from variable in bash

This is what I am trying to do in bash. Here is a PowerShell example:
$metaData = Invoke-RestMethod -Headers #{"Metadata"="true"} -URI http://169.254.169.254/metadata/instance?api-version=2019-04-30 -Method GET
$azEnv = $metadata.compute.azEnvironment
The above command gives me the value of azEnviroment from the Json output and saves that as a variable called $azEnv.
How can I do this in bash?
You can do something like this,
PC_VMSIZE=$(curl -s -H Metadata:true "http://169.254.169.254/metadata/instance?api-version=2018-10-01" | jq -r '.compute.vmSize')
export AZHPC_VMSIZE=${AZHPC_VMSIZE,,}
#echo "Running on $PC_VMSIZE"

How call invoke-expression to pass arguments when calling a remote script in powershell? [duplicate]

This question already has answers here:
What is shortest possible way to download script from HTTP and run it with parameters using Powershell?
(2 answers)
Closed 1 year ago.
My script content:
param (
[string]$arg1 = "2.2.2",
[string]$arg2 = "master"
)
& {
Write-Host "show $arg1 and $arg2 .."
}
Then I want to call this script in remote machine via http.
Invoke-Expression (Invoke-Webrequest "https://x.x.x.x/myscript.ps1" -UseBasicParsing).Content
But I don't know how to pass parameters to the script. Like this?
Invoke-Expression (Invoke-Webrequest "https://x.x.x.x/myscript.ps1" -UseBasicParsing).Content -arg1 2.2.1 -arg2 dev
How can help me? thanks!
If I do not pass arguments, the following commands are working fine.
Invoke-Expression (Invoke-Webrequest "https://x.x.x.x/myscript.ps1" -UseBasicParsing).Content
Download the code, create a scriptblock from it, and provide the arguments when calling Invoke() on the scriptblock.
$script = Invoke-RestMethod -Uri 'https://gist.githubusercontent.com/gittorta/75838602b2712d49c44e85ea0bc179e9/raw/8392f437943f66d47731423b674b81bd74378e4b/myscript.ps1'
$sb = [scriptblock]::Create($script)
$arg1 = "test1"
$arg2 = "test2"
$sb.Invoke($arg1, $arg2)
Output
show test1 and test2 ..
Same command condensed into a one-liner
([scriptblock]::Create((Invoke-RestMethod -Uri 'https://gist.githubusercontent.com/gittorta/75838602b2712d49c44e85ea0bc179e9/raw/8392f437943f66d47731423b674b81bd74378e4b/myscript.ps1'))).Invoke("test1", "test2")

Resources