I am using conda 4.6.8 to test a python package in a conda env on Travis CI. I want to replace my old source activate ENVNAME line with the new conda activate ENVNAME command in my Travis CI configuration. If I run this on Travis:
>>> conda update -n base conda
>>> conda init
no change /home/travis/miniconda/condabin/conda
no change /home/travis/miniconda/bin/conda
no change /home/travis/miniconda/bin/conda-env
no change /home/travis/miniconda/bin/activate
no change /home/travis/miniconda/bin/deactivate
no change /home/travis/miniconda/etc/profile.d/conda.sh
no change /home/travis/miniconda/etc/fish/conf.d/conda.fish
no change /home/travis/miniconda/shell/condabin/Conda.psm1
no change /home/travis/miniconda/shell/condabin/conda-hook.ps1
no change /home/travis/miniconda/lib/python3.7/site-packages/xonsh/conda.xsh
no change /home/travis/miniconda/etc/profile.d/conda.csh
modified /home/travis/.bashrc
==> For changes to take effect, close and re-open your current shell. <==
How can I "close and re-open" my shell on Travis? Because otherwise I cannot activate my conda environment:
>>> conda create -n TEST package_names
>>> conda activate TEST
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
See 'conda init --help' for more information and options.
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
The command "conda activate TEST" failed and exited with 1 during .
Your build has been stopped.
Not sure it is currently supported as the official doc still uses source in travis.yml.
What does conda init do?
This new command should harmonize the way users setup their shells to be able to call conda activate.
Actually, if you run conda init --dry-run --verbose you will see that it tries to source conda.sh from your ~/.bashrc (assuming you're running Bash, from info mentioned in your question).
And conda.sh will define a conda() function that will catch a few commands among which activate and deactivate and dispatch to $CONDA_EXE:
conda() {
if [ "$#" -lt 1 ]; then
"$CONDA_EXE"
else
\local cmd="$1"
shift
case "$cmd" in
activate|deactivate)
__conda_activate "$cmd" "$#"
;;
install|update|upgrade|remove|uninstall)
"$CONDA_EXE" "$cmd" "$#" && __conda_reactivate
;;
*) "$CONDA_EXE" "$cmd" "$#" ;;
esac
fi
}
So unless this function is defined in your local shell, you won't be able to call conda activate.
Hint on a solution? (not tested for Travis CI)
The only hint I can suggest is to try source $(conda info --root)/etc/profile.d/conda.sh and then conda activate. This should do roughly the same as conda init assuming you are using Bourne shell derivatives.
For csh there is $(conda info --root)/etc/profile.d/conda.csh, and for fish there is $(conda info --root)/etc/fish/conf.d/conda.fish
Note: although not tested for Travis CI, this solution works for me from bash. Of course, the conda executable should be found in PATH for conda info --root to work properly.
Related
In the part of my bash script I need to switch to specific conda environment to use package that had been installed specifictly in this environment:
#!/bin/bash
home="$PWD"
# activate VINA 1.2.3
conda activate vina
vina --some_in --some_out
# go back to default environment
conda deactivate
While this works fine simply in the terminal (I am using MacOSX) the sh script executing the same commands produces the following errors:
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
I've tried to use
conda init bash && conda deactivate
which works fine in the terminal but did not solve the problem in the sh script
How I could properly switch between different conda environemnts in the SH script ?
I have miniconda 4.8.3 + MacOS Catalina 10.15. I can manually activate the conda environment in the terminal and start a spyder session.
$ ~/miniconda3/bin/conda activate py3
$ ~/miniconda3/bin/conda info | grep "active environment"
$ spyder &
When I put the above in a script, run_spyder.sh it's not working, and it complains about "CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'."
#!/bin/bash
# run_spyder.sh
~/miniconda3/bin/conda activate py3
~/miniconda3/bin/conda info | grep "active environment" # still print base
# spyder &
I tried alternatives like bash -i ./run_spyder.sh, or source ./run_spyder.sh, or adding ~/miniconda3/bin/conda init bash, none of them work.
Shell is still bash, no .bashrc, in .bash_profile this is the script automatically generated by miniconda installation
# .bashrc_profile
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('~/miniconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "~/miniconda3/etc/profile.d/conda.sh" ]; then
. "~/miniconda3/etc/profile.d/conda.sh"
else
export PATH="~/miniconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<
The conda activate function is a shell function that is typically defined in the initialization file for a shell when the session starts (e.g., in the .bash_profile). The conda init function merely adds code to such initialization files, but will not actually source the code it adds. Hopefully, that clarifies the difficulty with what was tried in the question (and comments).
Instead, try directly sourcing the code that Conda uses. Something like:
#!/bin/bash
source ~/miniconda3/etc/profile.d/conda.sh
conda activate py3
conda info | grep "active environment"
spyder &
Another option is to have the bash (or zsh) session launch in login mode (i.e., runs the initialization files for the current user).
#!/usr/bin/env bash -l
conda deactivate # <- may not be needed, but didn't work for me without
conda activate py3
conda info | grep "active environment"
spyder &
However, note that in this latter case I find I need to include a conda deactivate first, in order for the conda activate to properly prioritize the Python in the env on PATH.
I want to use snakemake with fish shell and conda environments in my managed environment (basically I have no root rights and the default shell cannot be changed).
I set up fish as the 'default' shell using this hack inside the .bashrc:
if [ "$REALBASH" != "1" ]; then
case "$-" in
*i*)
export SHELL=/usr/bin/fish
tty > /dev/null && echo "Your ~/.bashrc is switching interactive SHELL to $SHELL"
[ -x $SHELL ] && exec $SHELL "$#"
echo "Apparently $SHELL is not available here. Continuing with bash."
export SHELL=/bin/bash
;;
esac
fi
There is also a command realbash that sets the environment variable REALBASH=1 to bypass this hack.
I managed to get conda to work with fish using this, but it has the disadvantage that within fish the command to activate conda environments is different from bash. In bash, the command is source activate ... and in fish it is conda activate ....
Activating environments works both from bash using source activate ... and from fish using conda activate ....
When I now execute snakemake from fish, I get the following error:
Activating conda environment ...
source: Error encountered while sourcing file “activate”:
source: No such file or directory
If I execute snakemake from bash, the same error occurs.
If I execute snakemake from bash via snakemake --overwrite-shellcmd realbash, I get the same error and end up in the bash shell that was opened by snakemake. Only after typing exit, snakemake completes (but unsuccessfully, of course).
If I execute snakemake from fish via snakemake --overwrite-shellcmd realbash, the same behaviour occurs.
I am confused by the behaviour of --overwrite-shellcmd, is there a way to make this work with my hack?
Otherwise, can I configure snakemake to call conda activate instead of source activate?
Or is there any other solution to this?
Apparently this was a bug in an older version of snakemake. The effects described in the question were produced with snakemake 4.3.1.
Running snakemake from within a conda environment where snakemake 5.17.0 is installed works perfectly fine with the setup as described in the question. No --overwrite-shellcmd or other changes are required.
If I run
xterm -hold
and within the new terminal I type
conda activate my_environment
the conda environment "my_environment" is indeed activated.
However, when passing this command using the -e flag, it does not work:
xterm -hold -e "conda activate my_environment"
It instead returns the following error message:
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
See 'conda init --help' for more information and options.
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
So, how can this be done using xterm? Or should I use another type of external terminal?
Background
The conda activate command is a shell function that gets defined during initialization of the shell. conda init adds code to the initialization file (e.g., .bash_profile) to run the scripts that define the conda activate shell function.
Solutions
Possible fix: xterm options
When using the -c argument with xterm it no longer runs the initialization script. Hence, conda activate never gets defined. For bash there is the -l that tells it to run the init files. I expected xterm's -ls argument to trigger similar behavior, but it didn't work for me. Perhaps someone more familiar can point you to the correct flag.
Manually Run Conda Script
Otherwise, you just run the Conda script yourself (assuming it's the bash version). Either of these will work:
xterm -hold -e ". /path/to/miniconda3/etc/profile.d/conda.sh && conda activate my_environment && which python"
or
xterm -hold -e "$(conda shell.bash hook) && conda activate my_environment && which python"
The which python is only included to show that you're getting the env activated.
Conda Run
Another option is conda run, which automates executing commands under an environment. The follow is equivalent to what I did in the last section, but without having to know what shell I am running in:
xterm -hold -e "conda run -n my_environment which python"
Just be aware that this functionality is still a work-in-progress. Personally, I've found it quite useful for running simple scripts in specific envs and haven't run into problems.
I'm trying to activate my pyenv-virtualenv environment through very simple bash script like this.
set -e
pyenv activate myenv
But I can't activate my env with below error:
Failed to activate virtualenv.
Perhaps pyenv-virtualenv has not been loaded into your shell properly.
Please restart current shell and try again.
I can activate it in command line, so I think it's due to my shell environment.
I use fish shell, but it's launched after $PATH settings read in bash shell.
I show you my settings.
.bash_profile:
export LANG="ja_JP.UTF-8"
export XDG_CONFIG_HOME="$HOME/.config"
export PATH="/sbin"
export PATH="/usr/sbin:$PATH"
export PATH="/bin:$PATH"
export PATH="/usr/bin:$PATH"
export PATH="/usr/local/bin:$PATH"
export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
exec fish
And my config.fish:
status --is-interactive; and source (pyenv init -|psub)
status --is-interactive; and source (pyenv virtualenv-init -|psub)
What's wrong?
I'm using:
OS: OS X 10.13.5
fish: v2.7.1
pyenv: v1.2.5
pyenv-virtualenv: v1.1.3
I believe the cause is that you are initializing PyEnv in the Fish shell environment but then calling PyEnv in the Bash shell environment (via your Bash script). To solve this problem, you could try either of the following potential solutions:
Initialize PyEnv in your Bash environment (in addition to your Fish environment)
Write your script in Fish instead of Bash
Since PyEnv seems so have better support for Bash than Fish, let's focus on option #1. Step 3 of the PyEnv installation guide suggests the following command for adding PyEnv initialization to your ~/.bash_profile:
$ echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bash_profile
After running that command, I suspect your script will function as you expect. If it does not, you could try adding the same PyEnv initialization snippet (that you just added to your ~/.bash_profile) to the top of your Bash script.