variables substitution (or overriding) when extends jobs from gitlab templates unsing include - continuous-integration

Question

Using gitlab ci, I have a repo where all my templates are created.
For example, I have a sonar scanner job template named .sonar-scanner.yml:
sonar-analysis:
stage: quality
image:
name: sonar-scanner-ci:latest
entrypoint: [""]
script:
- sonar-scanner
-D"sonar.projectKey=${SONAR_PROJECT_NAME}"
-D"sonar.login=${SONAR_LOGIN}"
-D"sonar.host.url=${SONAR_SERVER}"
-D"sonar.projectVersion=${CI_COMMIT_SHORT_SHA}"
-D"sonar.projectBaseDir=${CI_PROJECT_DIR}"
I have include this template as a project like this in main gitlab ci file:
include:
- project: 'organization/group/ci-template'
ref: master
file: '.sonar-scanner.yml'
So as you can understand I have a repo named ci-templates where all my templates are created. And in another repo, I extends using include these templates.
Finally, in a repo, when a new merge request is created, my job for sonar is running under another file in my project test/quality.yml:
sonar:
stage: quality
extends:
- sonar-analysis
allow_failure: true
only:
refs:
- merge_requests
All is working well except the substitution or the overriding of my env var. Indeed of my template. I have many sonar server or project names. I would like to know how to override these variables SONAR_SERVER and SONAR_PROJECT_NAME when I extend the job from a template.
In my main .gitlab-ci.yml file I have a variables section declaration, and when I override these variables in, it works.
But it's not really what I want. Using many stages and many micro service it is possible to reuse the same extending job in a different way. That I really want to do is to override these variables directly in the file test/quality.yml.
This, for example does not work:
sonar:
stage: quality
extends:
- sonar-analysis
variables:
SONAR_PROJECT_NAME: foo
SONAR_SERVER: bar
allow_failure: true
only:
refs:
- merge_requests
This not work too:
variables:
SONAR_PROJECT_NAME: foo
SONAR_SERVER: bar
sonar:
stage: quality
extends:
- sonar-analysis
allow_failure: true
only:
refs:
- merge_requests
What is the best way to make this work ?

Answer 1

Since this question was asked in Feburary 2020, a new MR Use non-predefined variables inside CI include blocks has been merged into Gitlab 14.2, which resolves the issue for the overridden job.
The project that does the include can redefine the variables when extending a job:
include:
- project: 'organization/group/ci-template'
ref: master
file: '.sonar-scanner.yml'
sonar:
stage: quality
extends:
- sonar-analysis
variables:
SONAR_PROJECT_NAME: foo
SONAR_SERVER: bar
allow_failure: true
But in this case you probably want the job in the template to start with a dot .sonar-analysis instead of sonar-analysis to not create a real sonar-analysis job in the template (see hidden jobs).
Or you can also directly set the variables values (to redefine them) of an existing job in the project that does the include:
include:
- project: 'organization/group/ci-template'
ref: master
file: '.sonar-scanner.yml'
sonar-analysis:
variables:
SONAR_PROJECT_NAME: foo
SONAR_SERVER: bar
I verified this with a test project, which includes a template from a peer test project, and when it runs, results in two jobs. This is the job output for the overriding job:
$ echo sonar.projectKey=${SONAR_PROJECT_NAME}
sonar.projectKey=foo
$ echo sonar.login=${SONAR_LOGIN}
sonar.login=bob
$ echo sonar.host.url=${SONAR_SERVER}
sonar.host.url=bar