Use different Azure Subscription ID per environment in a Gitlab CI pipeline - bash

Question

We have a gitlab pipeline which I am trying to configure to use a different Azure subscription per environment without much luck.
Basically what I need to be able to do is set the environment variables ARM_CLIENT_ID, ARM_CLIENT_SECRET, ARM_SUBSCRIPTION_ID, ARM_TENANT_ID to different values depending on the environment being built.
In the cicd settings I have variables set for development_ARM_SUBSCRIPTION_ID, test_ARM_SUBSCRIPTION_ID etc with the idea being I assign the values from these variables to the ARM_CLIENT_ID, ARM_CLIENT_SECRET, ARM_SUBSCRIPTION_ID, ARM_TENANT_ID variables in the pipeline.
This is what my pipeline looks like
stages:
- infrastructure-validate
- infrastructure-deploy
- infrastructure-destroy
variables:
DESTROY_INFRA: "false"
development_ARM_SUBSCRIPTION_ID: $development_ARM_SUBSCRIPTION_ID
development_ARM_TENANT_ID: $development_ARM_TENANT_ID
development_ARM_CLIENT_ID: $development_ARM_CLIENT_ID
development_ARM_CLIENT_SECRET: $development_ARM_CLIENT_SECRET
image:
name: hashicorp/terraform:light
entrypoint:
- '/usr/bin/env'
- 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
before_script:
- rm -rf .terraform
- terraform --version
- terraform init
.terraform-validate:
script:
- export ARM_SUB_ID=${CI_ENVIRONMENT_NAME}_ARM_SUBSCRIPTION_ID
- export ARM_SUBSCRIPTION_ID=${!ARM_SUB_ID}
- export ARM_CLI_ID=${CI_ENVIRONMENT_NAME}_ARM_CLIENT_ID
- export ARM_CLIENT_ID=${!ARM_CLI_ID}
- export ARM_TEN=${CI_ENVIRONMENT_NAME}_ARM_TENANT_ID
- export ARM_TENANT_ID=${!ARM_TEN_ID}
- export ARM_CLI_SECRET=${CI_ENVIRONMENT_NAME}_ARM_CLIENT_SECRET
- export ARM_CLIENT_SECRET=${!ARM_CLI_SECRET")
- echo $development_ARM_SUBSCRIPTION_ID
- echo ${ARM_SUBSCRIPTION_ID}
- terraform workspace select ${CI_ENVIRONMENT_NAME}
- terraform validate
- terraform plan -out "terraform-plan-file"
only:
variables:
- $DESTROY_INFRA != "true"
development-validate-and-plan-terraform:
stage: infrastructure-validate
environment: development
extends: .terraform-validate
only:
refs:
- main
- develop
artifacts:
paths:
- terraform-plan-file
The variable substitution works fine when I test locally but in the pipeline it fails with
/bin/sh: eval: $ export ARM_SUBSCRIPTION_ID=${!ARM_SUB_ID}
line 139: syntax error: bad substitution
I think the problem is the terraform image does not have bash available, only sh but I can't for the life of me work out how I do the same substitution in sh. If anyone has any suggestions, or knows a better way of using different Azure subscriptions for different environments in the pipeline I would really appreciate it.

Answer 1

I would define different jobs for each environment that extend your main .terraform-validate job template, and define the environment variables on that job. This way you don't have to do the indirect substitution that seems to be giving you trouble. That would look something like this:
.terraform-validate:
stage: infrastructure-validate
script:
- echo ${ARM_SUBSCRIPTION_ID}
- terraform workspace select ${CI_ENVIRONMENT_NAME}
- terraform validate
- terraform plan -out "terraform-plan-file"
only:
variables:
- $DESTROY_INFRA != "true"
artifacts:
paths:
- terraform-plan-file
development-validate-and-plan-terraform:
extends: .terraform-validate
environment: development
only:
refs:
- main
- develop
variables:
ARM_SUBSCRIPTION_ID: $development_ARM_SUBSCRIPTION_ID
ARM_TENANT_ID: $development_ARM_TENANT_ID
ARM_CLIENT_ID: $development_ARM_CLIENT_ID
ARM_CLIENT_SECRET: $development_ARM_CLIENT_SECRET
production-validate-and-plan-terraform:
extends: .terraform-validate
environment: production
only:
refs:
- main
variables:
ARM_SUBSCRIPTION_ID: $production_ARM_SUBSCRIPTION_ID
ARM_TENANT_ID: $production_ARM_TENANT_ID
ARM_CLIENT_ID: $production_ARM_CLIENT_ID
ARM_CLIENT_SECRET: $production_ARM_CLIENT_SECRET
Then you define all the development_* and production_* vars in the GitLab CI/CD settings.
Note that I also moved the stage: infrastructure-validate and artifacts: ... directives to the template since I'd imagine they're the same for all environments.