Fail to active conda in Azure DevOps pipeline - yaml

Question

Testing the azure devops pipeline on a python project build by conda
jobs:
- job: pre_build_setup
displayName: Pre Build Setup
pool:
vmImage: 'ubuntu-18.04'
steps:
- bash: echo "##vso[task.prependpath]$CONDA/bin"
displayName: Add conda to PATH
- job: build_environment
displayName: Build Environment
dependsOn: pre_build_setup
steps:
- script: conda env create --file environment.yml --name build_env
displayName: Create Anaconda environment
- script: conda env list
displayName: environment installation verification
- job: unit_tests
displayName: Unit Tests
dependsOn: build_environment
strategy:
maxParallel: 2
steps:
- bash: conda activate build_env
The last step - bash: conda activate build_env fails on me with the following error
Script contents:
conda activate build_env
========================== Starting Command Output ===========================
/bin/bash --noprofile --norc /home/vsts/work/_temp/d5af1b5c-9135-4984-ab16-72b82c91c329.sh
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
See 'conda init --help' for more information and options.
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
##[error]Bash exited with code '1'.
Finishing: Bash
How can I active conda? seems the path is wrong so that it is unable to find conda.

Answer 1

CommandNotFoundError: Your shell has not been properly configured to
use 'conda activate'.
Here the issue is your script is run in a sub-shell, but condahasn't been initialized in this sub-shell.
You need change your active script as：
steps:
- task: Bash#3
inputs:
targetType: 'inline'
script: |
eval "$(conda shell.bash hook)"
conda activate build_env
displayName: Active
In addition, please do not split the Add PATH, create environment and active the environment into different jobs.
For Azure devops pipeline, agent job is the basic unit of the pipeline running process and each agent job has its own independent running environment and work logic.
For more detailed, you were using Hosted agent to apply your scripts in this issue scenario.
While there's one agent job starts to run, our pool system will assign an VM to this agent job. And, this VM will be recycled back once the agent job finished. When next agent job start to run , a completely new VM will be randomly reassign.
dependsOn can only share files and pass variables between jobs. It can not keep the VM continued in next job.
I believe you should be able to guess what problem you are going to encounter. Yes, even you can succeed to apply that activate script, you will continue face another error: Could not find conda environment: build_env. That's because the environment which is using by this activate script is a brand new vm, the VM that previous build_environment job used has been recycled by the system.
So, do not split the create environment and activate into 2 agent jobs:
- job: build_environment
displayName: Build Environment
dependsOn: pre_build_setup
steps:
- script: conda env create --file environment.yml --name build_env
displayName: Create Anaconda environment
- script: conda env list
displayName: environment installation verification
- task: Bash#3
inputs:
targetType: 'inline'
script: |
eval "$(conda shell.bash hook)"
conda activate build_env
displayName: Active

Answer 2

There's one more approach proposed by Microsoft which seems to be more robust.
In every step where you want the environment to be activated, you should run
source $CONDA/bin/activate <myEnv>
or just
source activate <myEnv>
if you've already added $CONDA/bin to the PATH variable. You may check the link above to find examples for Ubuntu, macOS and Windows.
In your case it would look as follows:
steps:
- task: Bash#3
inputs:
targetType: 'inline'
script: source activate build_env
displayName: Active
Important note: as for now, if you pass the name of the environment to activate script, the environment must be created in the same job. However, if you're using prefix (i.e. path to the environment directory), it doesn't matter.