I have created SPFX library module in which there is code to be shared with multiple webparts similar to the one here. I used npm link <lib name> command to link it. It works pretty well in the local environment as well as when I deploy it manually in SharePoint online app catalog. But if I deploy it using Azure Pipelines (YAML script) it always throws and error for library not found.
I have made it sure that the library is built first and then other webparts are built in pipeline (by introducting stages) but still it doesn't find the library. Is there a way to run npm link as a pipeline task?
If the library is first built in a different stage or job, the library will not be located in the same agent machine with the other webparts.
Let's say the library is built in Stage A and the other webparts are built in Stage B. As workaround, you will need to publish the library as artifacts to azure devops server using Publish Build Artifacts task in stage A. And then download the library to the agent which builds the other webparts in Stage B using Download Build Artifacts task.
For below example:
The library is built in stage A and published to azure devops server as artifacts named library.
In Stage B, the library artifacts is downloaded from azure devops server to folder $(Build.ArtifactStagingDirectory). Then you can refer to the library by path $(Build.ArtifactStagingDirectory)/library for the following tasks in Stage B.
If multiple webparts are built in different jobs in Stage B, each job in Stage B will need to download the library artifacts, for different jobs run on different agent virtual machines.
stages:
- stage: A
jobs:
- job: Library
pool:
vmImage: "windows-latest"
steps:
...
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: {path to the library}
ArtifactName: library
- stage: B
dependsOn: A
jobs:
- job: WebParts1
pool:
vmImage: "windows-latest"
steps:
- task: DownloadBuildArtifacts#0
inputs:
downloadPath: $(Build.ArtifactStagingDirectory)
buildType: current
artifactName: library
...
- job: WebParts2
pool:
vmImage: "windows-latest"
steps:
- task: DownloadBuildArtifacts#0
inputs:
downloadPath: $(Build.ArtifactStagingDirectory)
buildType: current
artifactName: library
...
After trying the solutions posted above it still didn't work so I published it as NPM package using npm publish command. This solved the problem.
We can also use this command in YAML to automatically publish the package everytime we build but that was not required in my case so I didn't use it.
Related
My appveyor.yml looks like below:
environment:
matrix:
- solution_name: Src/App1.sln
- solution_name: Src/App2.sln
before_build:
- cmd: dotnet restore %solution_name%
build:
project: '%solution_name%'
publish_nuget: true
after_build:
- dotnet pack Src\App1.csproj
deploy:
- provider: NuGet
api_key:
secure: [my api key]
on:
branch: master
artifact: /.*(\.|\.s)nupkg/
artifacts:
- path: Out\
name: App1
- path: '**\*.nupkg'
It works well with one catch: it seems deploy twice during the build process. Below is part of the AppVeyor log:
Collecting artifacts...
Found artifact 'Out' matching 'Out' path
Found artifact 'Out\App1\Release\App1.6.0.3.nupkg' matching '**\*.nupkg' path
Uploading artifacts...
[1/2] App1.6.0.3.nupkg (164,127 bytes)...100%
[2/2] Out\App1\Release\App1.6.0.3.nupkg (164,127 bytes)...100%
Deploying using NuGet provider
Publishing App1.6.0.3.nupkg to https://www.nuget.org/api/v2/package...OK
Publishing App1.6.0.3.nupkg to https://www.nuget.org/api/v2/package...Skipped (A package with ID 'App1' and version '6.0.3' already exists and cannot be modified.)
No packages were pushed.
So I login NuGet.org and I can see App1.6.0.3 is published successfull. However, in the above log, the same package is published twice, hence the second publish is skipped, and end up with message "No packages we pushed." But actually there was one package pushed.
Maybe it is because I have a matrix configuration under environment section, so AppVeyor runs deploy twice? If so, how can tell AppVeyor to deploy only once?
After some experiments, I found a fix, that is removing publish_nuget: true under build section. This line will cause AppVeyor to create a nuget package, and dotnet pack Src\App1.csproj command under after_build section will create a nuget package, too. So there will be two identical .nupkg files generated in different location, and AppVeyor will find both files and push them to nuget server.
Result:
build:
project: '%solution_name%'
after_build:
- dotnet pack Src\App1.csproj
We have a complex build system, with a many to many relationship between our libraries and our applications. We put each library and application in it's own repository, and use the output of the library builds in our application builds.
On our old Jenkins server, we simply set up a custom workspace and checked out the projects into standardized relative paths so they could find each other. Post build steps assured that only successful builds copied to the central bin folder at the expected relative path.
On our Bamboo server, our repository was fetched to a Checkout Directory at the expected relative path, and we could fetch artifacts from other builds and put them in the central bin folder at the expected relative path.
Now I'm trying to set up some Bitbucket Pipelines builds, and I can't see an obvious way to do a similar thing. The working folder is set automatically by pipelines, I can't push that repository into a subfolder that is relative to other build outputs. I can create artifacts, but I can't seem to import them into other pipelines. I can create caches, but again I can't seem to import them into other pipelines.
Library bitbucket-pipelines.yml
image: mcr.microsoft.com/dotnet/sdk:5.0
pipelines:
branches:
master:
eCRF2:
- step:
name: Build and Test
caches:
- dotnetcore
- platform2
script:
- dotnet restore ./NET5/Platform2.sln
- dotnet build ./NET5/Platform2.sln --no-restore --configuration Release
artifacts:
- NET5/Platform2/bin/**
definitions:
caches:
platform2: NET5/Platform2/bin
App bitbucket-pipelines.yml
image: mcr.microsoft.com/dotnet/sdk:5.0
pipelines:
default:
- step:
name: Build and Test
caches:
- dotnetcore
- platform2
script:
- export PROJECT_NAME=./PlatformDataService.sln
- dotnet restore ${PROJECT_NAME}
- dotnet build ${PROJECT_NAME} --no-restore --configuration Release
artifacts:
- PlatformDataService/bin/**
https://support.atlassian.com/bitbucket-cloud/docs/deploy-build-artifacts-to-bitbucket-downloads/ did get me to upload a file to the Downloads section of the repository, but how do I pull it into the other pipeline?
Is there a way to solve this within bitbucket pipelines itself or do I have to get a nuget server that's available outside my VPN?
Can you please give an example the way to specify in build.yaml if needs to do dotnet publish a test project in .Net Core 2.1. referred this couldn't figure out exactly the way.
It says below, what should be the yaml way
For this task to work, you must have already published the output of your build to this directory by using the dotnet publish --output $(Build.ArtifactStagingDirectory) command. To copy additional files to this directory before publishing,
As example how below represent in build.yaml in class application (API Solution)
dotnet publish ~/projects/app1/app1.csproj
Tried below
- task: DotNetCoreCLI#2
displayName: "Prepare Publish Files"
inputs:
command: publish
projects: ""
arguments: --output $(Build.ArtifactStagingDirectory)/src/xxx.EndToEnd.Integration.Tests/xxx.EndToEnd.Integration.Tests.csproj
But get below error, in fact mine is API solution where I am trying to publish end2end tests and run them after deployment.
##[error]No web project was found in the repository. Web projects are identified by presence of either a web.config file or wwwroot folder in the directory.
##[error]Project file(s) matching the specified pattern were not found.
With below YAML code I was able to run the build
- task: DotNetCoreCLI#2
displayName: "dotnet e2e tests"
inputs:
command: publish
publishWebProjects: false
projects: '**/*.csproj'
arguments: --output $(Build.ArtifactStagingDirectory)/src/xxx.EndToEnd.Integration.Tests
zipAfterPublish: false
Explanation:
publishWebProjects: false - Since this is a API solution.
The easy way is to create task in designer wizard and see the YAML file there (Separate tab to show yaml file)
I'm trying to setup a CI/CD without much luck. My aim is to build a .net web project trough VSTS and deploy it to a AWS Beanstalk app.
Where am I so far?
Created a vsts-ci.yml file as following:
# ASP.NET
# Build and test ASP.NET web applications.
# Add steps that publish symbols, save build artifacts, deploy, and more:
# https://learn.microsoft.com/vsts/pipelines/apps/aspnet/build-aspnet-4
pool:
vmImage: 'VS2017-Win2016'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#0
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
- task: VSBuild#1
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(Build.artifactStagingDirectory)"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
And I have a build definition like below:
With this my first step build runs correctly. Btw Agent pool is Hosted VS2017.
Then I have created a release pipeline like below:
And as the Stage 1 tasks (the part that runs after artifact step), I have following:
I have hooked the build to a master branch commit, and hooked the release to a successful build, these triggerings works correctly. Though it fails. The problem is caused by not found artifact files probably because of wrong path inputs. But I don't know the correct versions.
After build finishes, it claims it created a .zip file containing the published version:
But after this, the release fails, only by saying no such file. I have tried many paths at Web Deploy Archive field, but none of them could find the zip file.
The powershell task you see, I created it to check the paths, and what they have in them, and they are mostly empty.
One interesting thing I figured is, build puts the files in D:\a\1\a but release tasks try to look into D:\a\r1\a when I suffixed the Web Deploy Archive field with $(System.defaultWorkingDirectory) and $(Build.artifactStagingDirectory).
Another odd thing is that on release Download artifact stage, it says Linked artifact count: 0. Which I would expect to be something else.
What am I doing wrong here? If someone can guide me trough, I would appreciate.
Update: I've added "Publish Artifact: drop"
When Path to publish is "$(System.DefaultWorkingDirectory)/project-2-codes.zip"
Publishing build artifacts failed with an error: Not found PathtoPublish: D:\a\r1\a\project-2-codes.zip
When Path to publish is "$(System.DefaultWorkingDirectory)"
Publishing build artifacts failed with an error: Not found PathtoPublish: D:\a\r1\a\$(Build.ArtifactStagingDirectory)
After a lot of head banging to the walls, I've found out the problem.
I was creating the build definition all wrong. I have used the default wizard thing, without noticing the custom designer (I don't think the text next to the link and the name of the link makes any sence).
And it was creating an almost empty build definition. I didn't even understood what it was doing. Now, using the visual designer I can actually select a buil definition template with all the required build tasks in it. After creating the build correctly, everything worked just nice.
Here is the build definition I came up with:
And here is the beanstalk deployment task:
You need to publish your build outputs as an artifact. Use the Publish Artifact task.
Currently, I have two main project.
1-) Vue Project which contains (webviews for IOS and Android, websites, and renderer for our Electron ) they are sharing components & API's.
2-) Electron Project which builds desktop app for (windows, darwin, linux)
i would like to automate our building, releasing process. my current setup..
before_script:
- apt-get update
- apt-get install zip unzip
- rm -rf vue-project
- git clone vue-project
- cd vue-project
- git checkout dev
- git pull
- sed -i "/\b\(areaCode\|inline-svg-loader\)\b/d" ./packages/devtool/package.json
- yarn install
- ln -s vue-project/packages/desktop/ web
- npm install
build_darwin:
stage: build
script:
- npm run package -- darwin --deploy
cache:
paths:
- vue-project/node_modules
- node_modules
which basically before bundling electron project it's cloning vue-project install dependencies and bundling electron-renderer then when it's finish. i'm running package.
I would like to separate this two different job from each other. is there anyway i could use artifacts from different project gitlab-CI pipelines ?
any help would be an appreciated.
Gitlab has a API for do a lot of tricks.
curl --header "PRIVATE-TOKEN:YOURPRIVATETOKEN" "https://gitlab.example.com/api/v4/projects/1/jobs/artifacts/master/download?job=test"
for download it as a file.
curl --header "PRIVATE-TOKEN:YOURPRIVATETOKEN" -o artifacts.zip "http://gitlab.example.net/api/v4/projects/<projectnumber>/jobs/artifacts/master/download?job=build_desktop
Gitlab can certainly support this. To accomplish this, follow these steps:
ARTIFACT GENERATION
In your Vue Project, modify your job(s) of interest to store artifacts relevant to the Electron project. Each job's artifacts are defined using Gitlab Job Artifacts notation, and are uploaded at job completion to Gitlab, and stored associated to your Project, Branch, and Job.
Note: Branch is often overlooked, and matters when you want to retrieve your artifacts, more on this later.
Illustrating:
Vue Project .gitlab_ci.yml
stages:
- stage1
- ...
vue-job1:
stage: stage1
script:
- echo "vue-job1 artifact1" > ./artifact1
- echo "vue-job1 artifact2" > ./artifact2
artifacts:
when: always
paths:
- ./artifact1
- ./artifact2
expire_in: 90 days
vue-job2:
stage: stage1
script:
# error, would overwrite job1's artifacts since working
# directory is a global space shared by all pipeline jobs
# - echo "vue-job2 artifact1" > ./artifact1
- echo "vue-job2 artifact1" > ./artifact3
artifacts:
when: always
paths:
- ./artifact3
expire_in: 90 days
The artifacts generated above are written to the working directory, which is a clone of your project's repo. So be careful with filename conflicts. To be safe, put your artifacts in a subdirectory (eg: cat "foo" > ./subdir/artifact) and reference them in paths the same way (paths: - ./subdir/artifact). You can use 'ls' in your script to view the working directory.
When your job completes, you can confirm the artifacts stored in Gitlb by using the Gitlab UI. View the job output, and use the Browse button under Job Artifacts on the right panel.
ARTIFACT RETRIEVAL
In your Electron Project, modify your job(s) of interest to retrieve artifacts stored in the Vue Project using the Gitlab Job Artifacts API and curl. In order to access the Vue artifacts, you will need the Vue Project, Branch, and Job that the artifacts were created under.
Project: For Project, use the Project ID displayed in the Gitlab UI Project Details screen.
Branch: Usually master, but depends on the branch your pipeline executes against. Although this is not relevant for your problem, if you are generating and consuming artifacts across executions of the same pipeline, use the Gitlab variable $CI_COMMIT_BRANCH for the branch.
Job: Generally the Job Name that generated the artifacts for your Project. But if you need artifacts produced by a specific Job, then use the Job Number and the corresponding retrieval API.
Illustrating:
Electron Project .gitlab_ci.yml
stages:
- stage1
- ...
electron-job1:
stage: stage1
script:
- curl -o ./artifact1 -H "PRIVATE-TOKEN:$TOKEN" https://gitlab.example.com/api/v4/projects/$VUE_PROJECT_ID/jobs/artifacts/$BRANCH/raw/artifact1?job=vue-job1
- curl -o ./artifact2 -H "PRIVATE-TOKEN:$TOKEN" https://gitlab.example.com/api/v4/projects/$VUE_PROJECT_ID/jobs/artifacts/$BRANCH/raw/artifact2?job=vue-job1
- curl -o ./artifact3 -H "PRIVATE-TOKEN:$TOKEN" https://gitlab.example.com/api/v4/projects/$VUE_PROJECT_ID/jobs/artifacts/$BRANCH/raw/artifact3?job=vue-job2
This script retrieves artifacts individually to the working directory of your Electron Project. There are also options for retrieving all artifacts for your job at once as a zip archive.
MISCELLANEOUS
Although this is not in the problem posed, it is worth noting that you can use artifacts both within the lifespan of a single pipeline execution to pass information between jobs. You can also use this to pass information across pipeline executions within the same project.
With recent versions of gitlab, this can be achieved simply by using either the multi-project pipeline feature (then starting a pipeline in one project triggers a build from the other project): see the documentation
Or you can also use the "needs:project" mechanism which allows one job to download artifacts from other pipelines (see the documentation Use needs:project to download artifacts from up to five jobs in other pipelines.)