Gitlab CI/CD pipeline run/extend other jobs in sequence - continuous-integration

Question

So I have a job which is triggered with specific rules - creating a new tag app-prod-1.0.0 or app-dev-1.0.0. Whenever new tag is created I call the job, which in return extends other jobs
image: node:lts-alpine
stages:
- install
- build
- deploy
.install-packages:
stage: install
script:
- echo "INSTALL-PACKAGES"
- yarn install --cache-folder .yarn-cache
artifacts:
paths:
- node_modules
cache:
- key:
files:
- yarn.lock
paths:
- .yarn-cache/
.build-project:
stage: build
script:
- echo "BUILD-PROJECT"
- echo $ENVIRONMENT
- yarn build
artifacts:
paths:
- build
.deploy-project:
stage: deploy
script:
- echo "DEPLOY-PROJECT"
- ls -la build
build_prod:
variables:
PACKAGE: '/app/prod'
ENVIRONMENT: 'prod'
extends:
- .install-packages
- .build-project
- .deploy-project
rules:
- if: '$CI_COMMIT_TAG =~ /^app-prod-[0-9]+\.[0-9]+\.[0-9]+$/'
build_dev:
variables:
PACKAGE: '/app/dev'
ENVIRONMENT: 'dev'
extends:
- .install-packages
- .build-project
- .deploy-project
rules:
- if: '$CI_COMMIT_TAG =~ /^app-dev-[0-9]+\.[0-9]+\.[0-9]+$/'
My thought was that jobs will be called in the order I have described inside the job: .install-packages, .build-project, .deploy-project. But that's not happening it seems that it just jumps to the last job .deploy-project, without installing and building and thus breaking my pipeline.
How to run/extend jobs in sequence?

Answer 1

This is the behaviour for which I didn't use multiple extends so far in my work with GitLab.
GitLab, attempts to merge the code from parent job.
Now all your parent jobs defines the script tag and in your job for e.g. build_prod the extends happening in below order
extends:
- .install-packages
- .build-project
- .deploy-project
the script code from .deploy-project is overwriting the other job's script tag.
It works differently for the variables. It will merge all the variables and overwrites if same variable is used.
See your own example updated with variables.
image: node:lts-alpine
stages:
- install
- build
- deploy
.install-packages:
stage: install
variables:
PACKAGE: 'install'
INSTALL: 'install'
script:
- echo "INSTALL-PACKAGES"
- yarn install --cache-folder .yarn-cache
artifacts:
paths:
- node_modules
cache:
- key:
files:
- yarn.lock
paths:
- .yarn-cache/
.build-project:
stage: build
variables:
PACKAGE: 'build'
BUILD: 'build'
script:
- echo "BUILD-PROJECT"
- echo $ENVIRONMENT
- yarn build
artifacts:
paths:
- build
.deploy-project:
stage: deploy
variables:
PACKAGE: 'deploy'
DEPLOY: 'from deploy'
script:
- echo "DEPLOY-PROJECT"
- ls -la build
build_prod:
variables:
PACKAGE: '/app/prod'
ENVIRONMENT: 'prod'
extends:
- .install-packages
- .build-project
- .deploy-project
rules:
- if: '$CI_COMMIT_TAG =~ /^app-prod-[0-9]+\.[0-9]+\.[0-9]+$/'
build_dev:
variables:
PACKAGE: '/app/dev'
ENVIRONMENT: 'dev'
extends:
- .install-packages
- .build-project
- .deploy-project
rules:
- if: '$CI_COMMIT_TAG =~ /^app-dev-[0-9]+\.[0-9]+\.[0-9]+$/'
And now notice how PACKAGE variable is overwritten with the final value of '/app/prod' which comes from build-prod job itself. At the same time other variables from individual parent jobs are merged to look like below
variables:
PACKAGE: "/app/prod"
INSTALL: install
BUILD: build
DEPLOY: from deploy
ENVIRONMENT: prod
I really found View merged YAML feature best to understand how my yml file will be evaluated.
Its available in CI/CD -> Editor

Answer 2

It's not actually "jumps to the last job", but simply executes a single job you have provided - that is build_prod or build_dev, depending on commit tag.
As per docs when you are calling extends, you are basically just merging everything inside all the template jobs that you specified, so the last stage keyword, which comes from .deploy-project template job wins.
You should specify each job separately for each stage, and maybe even put your rules to a separate template job, i.e.
.dev:
variables:
PACKAGE: '/app/dev'
ENVIRONMENT: 'dev'
rules:
- if: '$CI_COMMIT_TAG =~ /^app-dev-[0-9]+\.[0-9]+\.[0-9]+$/'
install-dev:
extends:
- .dev
- .install-packages
build-dev:
extends:
- .dev
- .build-project
deploy-dev:
extends:
- .dev
- .deploy-project
You should create similar jobs for prod env, define template job .prod, and create install-prod, build-prod, deploy-prod jobs