Conda Environment doesn't stay activated throughout the bash script - bash

I'm making an installer using the below bash script. After activating the env and later on checking the active env using the conda info, it shows that No env is active. Please refer to the image
Even after installing dependencies, the packages are not installed in the env, instead it's getting installed in the base env.
The script-
#!/usr/bin/bash
set -euo pipefail
conda create --name count python=3.7 <<< 'y'
. $(conda info --base)/etc/profile.d/conda.sh
conda activate count
conda info
#install dependencies
pip install -r requirements_count.txt
Thanks in advance for going through the query!

First, iteratively using ad hoc installs is a bad pattern. Better practice is to predefine everything in the environment with a YAML. That can even include PyPI packages and requirements.txt files.
count.yaml
name: count
channels:
- conda-forge
dependencies:
## Conda deps
- python=3.9 ## change to what you want, but preferable to specify
## PyPI deps
- pip
- pip:
- -r requirements_count.txt
Second, conda activate is intended for interactive shells, not scripts. One can run a bash script interactively by including an -l flag in the shebang. But better practice is to use the conda run method, which is designed for programmatic execution of commands in a specified context.
Pulling this together, the script might be:
script.sh
#!/usr/bin/bash
set -euo pipefail
## create the environment
conda create -n count -f count.yaml
## execute a Python script in the environment
conda run -n count python my_code.py
If the code run in the environment needs to interact with a user or stream output, you may need additional flags (--no-capture-output, --live-stream). See conda run --help.

Related

How to get path of conda env from its name?

When I do conda info --envs, I get list of all envs and their locations like this:
# conda environments:
#
base * /Users/mbp/miniconda3
myenv /Users/mbp/miniconda3/envs/myenv
Is there a way to get the location of myenv environment from command line? May be something like conda info --envs myenv to get
/Users/mbp/miniconda3/envs/myenv
What's the use case?
I want to cache all the environment dependencies in GitHub actions. This has to happen before the environment is activated. If I know the location of the environment, I can cache all the files in it.
conda info --envs | grep -Po 'myenv\K.*' | sed 's: ::g'
This bash command will retrieve all envs from conda and find the line which starts from myenv and give it to the sed command which inturn removes the spaces
surprisingly it worked for me
$(conda info --base)/envs/myenv
Conda's internal function to handle prefix resolution (locate_prefix_by_name) is currently located in conda.base.context, so one could avoid listing all envs with a script like:
conda-locate.py
#!/usr/bin/env conda run -n base python
import sys
from conda.base.context import locate_prefix_by_name
print(locate_prexif_by_name(sys.argv[1]), end='')
Usage
# make executable
chmod +x conda-locate.py
./conda-locate.py jupyter
# /Users/user/miniforge/envs/jupyter
You may want to add a try..catch to handle the conda.exceptions.EnvironmentNameNotFound exception.

conda activate on Travis CI

I am using conda 4.6.8 to test a python package in a conda env on Travis CI. I want to replace my old source activate ENVNAME line with the new conda activate ENVNAME command in my Travis CI configuration. If I run this on Travis:
>>> conda update -n base conda
>>> conda init
no change /home/travis/miniconda/condabin/conda
no change /home/travis/miniconda/bin/conda
no change /home/travis/miniconda/bin/conda-env
no change /home/travis/miniconda/bin/activate
no change /home/travis/miniconda/bin/deactivate
no change /home/travis/miniconda/etc/profile.d/conda.sh
no change /home/travis/miniconda/etc/fish/conf.d/conda.fish
no change /home/travis/miniconda/shell/condabin/Conda.psm1
no change /home/travis/miniconda/shell/condabin/conda-hook.ps1
no change /home/travis/miniconda/lib/python3.7/site-packages/xonsh/conda.xsh
no change /home/travis/miniconda/etc/profile.d/conda.csh
modified /home/travis/.bashrc
==> For changes to take effect, close and re-open your current shell. <==
How can I "close and re-open" my shell on Travis? Because otherwise I cannot activate my conda environment:
>>> conda create -n TEST package_names
>>> conda activate TEST
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
See 'conda init --help' for more information and options.
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
The command "conda activate TEST" failed and exited with 1 during .
Your build has been stopped.
Not sure it is currently supported as the official doc still uses source in travis.yml.
What does conda init do?
This new command should harmonize the way users setup their shells to be able to call conda activate.
Actually, if you run conda init --dry-run --verbose you will see that it tries to source conda.sh from your ~/.bashrc (assuming you're running Bash, from info mentioned in your question).
And conda.sh will define a conda() function that will catch a few commands among which activate and deactivate and dispatch to $CONDA_EXE:
conda() {
if [ "$#" -lt 1 ]; then
"$CONDA_EXE"
else
\local cmd="$1"
shift
case "$cmd" in
activate|deactivate)
__conda_activate "$cmd" "$#"
;;
install|update|upgrade|remove|uninstall)
"$CONDA_EXE" "$cmd" "$#" && __conda_reactivate
;;
*) "$CONDA_EXE" "$cmd" "$#" ;;
esac
fi
}
So unless this function is defined in your local shell, you won't be able to call conda activate.
Hint on a solution? (not tested for Travis CI)
The only hint I can suggest is to try source $(conda info --root)/etc/profile.d/conda.sh and then conda activate. This should do roughly the same as conda init assuming you are using Bourne shell derivatives.
For csh there is $(conda info --root)/etc/profile.d/conda.csh, and for fish there is $(conda info --root)/etc/fish/conf.d/conda.fish
Note: although not tested for Travis CI, this solution works for me from bash. Of course, the conda executable should be found in PATH for conda info --root to work properly.

How To WrapUp the Shell Script with One Line of Command with Argument Passing

I use lots of virtual environment nowadays since the different project parallely going on in my company.
Following is what I usually do for the initial setting of conda creation of new virtual environment
conda install --yes --file requirements.txt
source activate myenv
python -m ipykernel install --user --name myenv --display-name “kernel_name”
Upon the above sequence of code must be ran sequentially while myenv and kernel_name being passed as an manually given argument.
How could I do this at once with wrapped up .sh file? or is this possible without creating .sh file?
You can do it using a shell script. I would do:
#!/usr/bin/env bash
myenv="$1"
kernel_name="$2"
source /path/to/base/conda/installation/etc/profile.d/conda.sh
conda install --yes --file /path/to/requirements.txt
conda activate "$myenv"
python -m ipykernel install --user --name "$myenv" --display-name "$kernel_name"
And run it like: /path/to/script.sh <env-name> <kernel-name>

skip installing confirm('yes' or 'no') in Dockerfile [duplicate]

How do I install the anaconda / miniconda without prompts on Linux command line?
Is there a way to pass -y kind of option to agree to the T&Cs, suggested installation location etc. by default?
can be achieved by bash miniconda.sh -b (thanks #darthbith)
The command line usage for this can only be seen with -h flag but not --help, so I missed it.
To install the anaconda to another place, use the -p option:
bash anaconda.sh -b -p /some/path
AFAIK pyenv let you install anaconda/miniconda
(after successful instalation)
pyenv install --list
pyenv install miniconda3-4.3.30
For a quick installation of miniconda silently I use a wrapper
script script that can be executed from the terminal without
even downloading the script. It takes the installation destination path
as an argument (in this case ~/miniconda) and does some validation too.
curl -s https://gist.githubusercontent.com/mherkazandjian/cce01cf3e15c0b41c1c4321245a99096/raw/03c86dae9a212446cf5b095643854f029b39c921/miniconda_installer.sh | bash -s -- ~/miniconda
Silent installation can be done like this, but it doesn't update the PATH variable so you can't run it after the installation with a short command like conda:
cd /tmp/
curl -LO https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -b -u
Here -b means batch/silent mode, and -u means update the existing installation of Miniconda at that path, rather than failing.
You need to run additional commands to initialize PATH and other shell init scripts, e.g. for Bash:
source ~/miniconda3/bin/activate
conda init bash

Cannot run source activate with conda in Fish-shell

I follow conda_PR_545, conda issues 4221 and still not working on Ubuntu.
After downloading conda.fish from here, and mv it to anaconda3/bin/.
Add "source /home/phejimlin/anaconda3/bin/conda.fish" at the end of ~/.config/fish/config.fish.
conda activate spark_env
Traceback (most recent call last):
File "/home/phejimlin/anaconda3/bin/conda", line 6, in
sys.exit(conda.cli.main())
File "/home/phejimlin/anaconda3/lib/python3.6/site-packages/conda/cli/main.py", line 161, in main
raise CommandNotFoundError(argv1, message)
TypeError: init() takes 2 positional arguments but 3 were given
or
activate spark_env
Error: activate must be sourced. Run 'source activate envname'
instead of 'activate envname'.
Do I miss something?
As of fish 2.6.0 conda 4.3.27: the following steps may change as issue is addressed
update config
Take note of your conda's location
conda info --root
/Users/mstreeter/anaconda # this is my <PATH_TO_ROOT>
Add line to ~/.config/fish/config.fish
source <PATH_TO_ROOT>/etc/fish/conf.d/conda.fish
update convention
Typically you'd run the following from bash
source activate <environment>
source deactivate <environment>
Now you must run the following from fish
conda activate <environment>
conda deactivate <environment>
issues
so after doing this I'm not able to set fish as my default shell and have it still work properly with conda. Currently, I must first enter my default shell, and enter fish and the shell works as expected. I'll update this after I find out how to get it working completely without the need to explicitly choose fish each time I log into my terminal
If you follow https://github.com/conda/conda/issues/2611, the steps are (from start):
[root#6903a8d80f9b ~]# fish
root#6903a8d80f9b ~# echo $FISH_VERSION
2.4.0
root#6903a8d80f9b ~# bash Miniconda2-4.3.11-Linux-x86_64.sh -b -p /conda
root#6903a8d80f9b ~# source /conda/etc/fish/conf.d/conda.fish
root#6903a8d80f9b ~# conda activate root
root#6903a8d80f9b ~# conda create -yn fishtest (root)
Fetching package metadata .........
Solving package specifications:
Package plan for installation in environment /conda/envs/fishtest:
#
# To activate this environment, use:
# > source activate fishtest
#
# To deactivate this environment, use:
# > source deactivate fishtest
#
root#6903a8d80f9b ~# conda activate fishtest (root)
root#6903a8d80f9b ~# (fishtest)
root#6903a8d80f9b ~# conda deactivate fishtest (fishtest)
Adding conda's bin directory to PATH isn't recommended as of conda 4.4.0
https://github.com/conda/conda/blob/master/CHANGELOG.md#440-2017-12-20
All you need to do is adding
source <path-to-anaconda>/etc/fish/conf.d/conda.fish
to config.fish.

Resources