So I try to run a bash script in my pipeline on Azure Devops. Here is my code for it:
- task: Bash#3
inputs:
filePath: '../marvel-lcg-companion/hooks/az-emulator'
But as you can see I received this error when I run the pipeline.
##[error]ENOENT: no such file or directory, stat '/Users/runner/work/1/marvel-lcg-companion/hooks/az-emulator'
So for me it is not clear how to format the file path in my YAML file. Can you guys point me in the right direction? I also tried the glob version without any success
**/hooks/az-emulator
UPDATE: my root folder is marvel-lcg-companion
First of please ensure what you have in your working directory by adding this:
- script: ls '$(System.DefaultWorkingDirectory)'
but if marvel-lcg-companion is folder in root of your repo (and you use sinfle repo) you should try:
- task: Bash#3
inputs:
filePath: '$(System.DefaultWorkingDirectory)/marvel-lcg-companion/hooks/az-emulator'
However if marvel-lcg-companion is name of your repo than rather this:
- task: Bash#3
inputs:
filePath: '$(System.DefaultWorkingDirectory)/hooks/az-emulator'
Related
I'm defining a github action script that's referencing to another yaml file, hoping to put the configuration into a more organised way.
Here is my job file, named as deploy.yml in the path of ./.github/workflows/, where the first . is the root folder of my project.
....
jobs:
UnitTest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- uses: ./.github/workflows/unittest.yml
In the same ./.github/workflows/ folder, I created another file called unittest.yml as below:
name: "UnitTest"
description: "Perform Unit Test"
runs:
# using: "composite"
- name: Dependency
run: |
echo "Dependency setup commands go here"
- name: UnitTest
run: make test.unit
However, when I tried to test the script locally using act with command act --secret-file .secrets --container-architecture linux/amd64, I received the following error:
[Deploy/UnitTest] ✅ Success - Main actions/checkout#v3
[Deploy/UnitTest] ⭐ Run Main ./.github/workflows/unittest.yml
[Deploy/UnitTest] ❌ Failure - Main ./.github/workflows/unittest.yml
[Deploy/UnitTest] file does not exist
[Deploy/UnitTest] 🏁 Job failed
I have tried to put just the file name unittest.yml or ./unittest.yml or myrepo_name/.github/workflows/unittest.yml or put the file into a subfolder like step 2 of this document illustrated, but all no luck.
Based on examples of runs for composition actions, I would imagine this should work.
Would anyone please advise?
P.S. You might have noticed the commented line of using: "composite" in the unittest.yml. If I uncomment the line, I'll receive error:
Error: yaml: line 3: did not find expected key
Composite actions are not referenced by YAML file, but a folder. In that folder, you are expected to have an action.yml describing the action.
This is why you're getting the error with using: composite, you're defining a workflow (because it's in ./github/workflows), but you are using action syntax.
I would advise this folder structure:
.github/
|-- workflows/
| -- deploy.yml
unittest-action/
|-- action.yml
With this structure, you should be able to reference the action with
- uses: actions/checkout#v3
- uses: ./unittest-action
Please see the docs for more information.
Depending on your use-case and setup, you might also want to consider reusable workflows.
You can define a reusable workflow in your .github/workflows directory like so:
# unittest.yml
on: workflow_call
jobs:
deploy:
# ...
and then call it like so:
jobs:
UnitTest:
uses: ./.github/workflows/unittest.yml
Note how the reusable workflow is an entire job. This means, you can't do the checkout from the outside and then just run the unit test in the reusable job. The reusable job (unittest.yml) needs to do the checkout first.
Which one to pick?
Here's a blog post summarising some of the differences between composite actions and reusable workflows, like:
reusable workflows can contain several jobs, composite actions only contain steps
reusable workflows have better support for using secrets
composite actions can be nested, but as of Jul '22, reusable workflows can't call other reusable workflows
I want to prune our node_modules folder of unnecessary bloat before it is packaged as a vsix extension.
This tool works well node-prune. Running locally I installed it with chocolatey - but trying to install it as part of our CI pipeline with Azure Dev ops - I get node-prune not found.
- task: GoTool#0
displayName: Install Go
inputs:
version: '1.10'
- task: Go#0
displayName: Install Node-Prune
inputs:
command: 'get'
arguments: 'github.com/tj/node-prune'
- task: Go#0
displayName: Prune Node Modules
inputs:
command: 'custom'
customCommand: 'node-prune'
arguments: '$(projectDirectory)'
Set the GO variables for the pipeline:
variables:
GOBIN: '$(GOPATH)/bin' # Go binaries path
GOROOT: '/usr/local/go1.10' # Go installation path
GOPATH: '$(system.defaultWorkingDirectory)/gopath' # Go workspace path
It seems that node-prune is not found in the current PATH, you shared a part of the YAML file.
node-prune must work in your local since the GOPATH is already defined, and new Go binaries exist under GOPATH/bin
I created a YML pipeline using terraform .
It uses a script task and returns in output the web app name
steps:
- script: |
[......]
terraform apply -input=false -auto-approve
# Get the App Service name for the dev environment.
WebAppNameDev=$(terraform output appservice_name_dev)
# Write the WebAppNameDev variable to the pipeline.
echo "##vso[task.setvariable variable=WebAppNameDev;isOutput=true]$WebAppNameDev"
name: 'RunTerraform'
The task works fine but when i deploy the webapp it crashes because seems variable $WebAppNameDev has double quotes.
- task: AzureWebApp#1
displayName: 'Azure App Service Deploy: website'
inputs:
azureSubscription: 'MySubscription'
appName: $(WebAppNameDev)
package: '$(Pipeline.Workspace)/drop/*.zip'
The error looks like:
Got service connection details for Azure App Service:'"spikeapp-dev-6128"'
##[error]Error: Resource '"spikeapp-dev-6128"' doesn't exist. Resource should exist before deployment.
How can i remove double quotes or fix the terraform output?
I solved by adding --raw parameter to terraform output.
WebAppNameDev=$(terraform output --raw appservice_name_dev)
ref. https://www.terraform.io/docs/cli/commands/output.html
I would like to save gradle-based project's version into a variable in gitlab-ci script. In my build.gradle I have:
tasks.register('version') {
doLast {
println(version)
}
}
It reads version from gradle.properties (let's say version=0.1) and returns it.
I execute it as gradlew version -q so I get only result, with no unnecessary output. When using unix-style variable creation of command result, that is: version=$(./gradlew version -q), the runner ends script. Is it possible to save the output into a variable for script?
My .gitlab-ci.yml:
image: gradle:jdk11
cache: &wrapper
paths:
- .gradle/wrapper
- .gradle/caches
before_script:
- export GRADLE_USER_HOME=`pwd`/.gradle
- chmod a+x gradlew
stages:
- prepare
- build
- deploy
wrapper:
stage: prepare
script:
- gradle wrapper
compile:
stage: build
script:
- ./gradlew assemble
artifacts:
paths:
- build/classes/**
- build/libs/*.jar
cache:
<<: *wrapper
policy: pull
properties:
stage: deploy
script:
- eval version=$(./gradlew version -q)
- echo $version # not even called
I also tried to omit eval to have version=$(./gradlew version -q) in script, but nothing changes.
CI output:
$ export GRADLE_USER_HOME=`pwd`/.gradle
$ chmod a+x gradlew
$ version=$(./gradlew version -q)
Cleaning up file based variables
Ok, I've found the solution. I mustn't use variable, simply. It's needed to directly pass evaluation to another command, using double quotes, like:
.gitlab-ci.yml (part):
properties:
stage: deploy
script:
- echo "$(./gradlew version -q)"
It started to work
I have many Gitlab project followed the same CI template. Whenever there is a small change in the CI script, I have to manually modify the CI script in each project. Is there a way you can store your CI script in a central location and have your project called that CI script with some environment variable substitution? For instance,
gitlab-ci.yml in each project
/bin/bash -c "$(curl -fsSL <link_to_the_central_location>.sh)"
gitlab-ci.yml in the central location
stages:
- build
- test
build-code-job:
stage: build
script:
- echo "Check the ruby version, then build some Ruby project files:"
- ruby -v
- rake
test-code-job1:
stage: test
script:
- echo "If the files are built successfully, test some files with one command:"
- rake test1
test-code-job2:
stage: test
script:
- echo "If the files are built successfully, test other files with a different command:"
- rake test2
You do not need curl, actually gitlab supports this via the include directive.
you need a repository, where you store your general yml files. (you can choose if it is a whole ci file, or just parts. For this example lets call this repository CI and assume your gitlab runs at example.com - so the project url would be example.com/ci. we create two files in there just to show the possibilities.
is a whole CI definition, ready to use - lets call the file ci.yml. This approach is not really flexible
stages:
- build
- test
build-code-job:
stage: build
script:
- echo "Check the ruby version, then build some Ruby project files:"
- ruby -v
- rake
test-code-job1:
stage: test
script:
- echo "If the files are built successfully, test some files with one command:"
- rake test1
test-code-job2:
stage: test
script:
- echo "If the files are built successfully, test other files with a different command:"
- rake test2
is a partly CI definition, which is more extendable. lets call the files includes.yml
.build:
stage: build
script:
- echo "Check the ruby version, then build some Ruby project files:"
- ruby -v
- rake
.test:
stage: test
script:
- echo "this script tag will be overwritten"
There is even the option to use template string from yaml. please reference the gitlab documentation but it is similar to 2.
we do have our project which wants to use such definitions. so either
For the whole CI file
include:
- project: 'ci'
ref: master # think about tagging if you need it
file: 'ci.yml'
as you can see now we are referencing one yml file, with all the cahnges.
with partial extends
include:
- project: 'ci'
ref: master # think about tagging if you need it
file: 'includes.yml'
stages:
- build
- test
build-code-job:
extends: .build
job1:
extends: .test
script:
- rake test1
job2:
extends: .test
script:
- rake test2
As you see, you can easily use the includes, to have a way more granular setup. Additionally you could define at job1 and job2 variables, eg for the test target, and move the script block into the includes.yml
Futhermore you can also use anchors for the script parts. Which looks like this
includes.yml
.build-scirpt: &build
- echo "Check the ruby version, then build some Ruby project files:"
- ruby -v
- rake
.build:
stage: build
script:
- *build
and you can use also the script anchor within your configuration
For a deeper explanation you can also take a look at https://docs.gitlab.com/ee/ci/yaml/includes.html