Conda Create Environment - No Compatible Shell found - bash

I have a bash script where i will be creating conda virtual environment and install packages into it.
currently we are using conda version 4.5.12 with python 3.6 in my virtual environment.
Am trying to upgrade conda version to 4.9.2 with python 3.6.
conda --version
4.9.2
This is the command i use inside my script
conda create -y --name virtual_env python=3.6
This Runs and fails during Download and Extracting Packages Step. Below is the Error Report
Traceback (most recent call last):
File "/root/project/miniconda/lib/python3.9/site-packages/conda/exceptions.py", line 1079, in __call__
return func(*args, **kwargs)
File "/root/project/miniconda/lib/python3.9/site-packages/conda/cli/main.py", line 84, in _main
exit_code = do_call(args, p)
File "/root/project/miniconda/lib/python3.9/site-packages/conda/cli/conda_argparse.py", line 83, in do_call
return getattr(module, func_name)(args, parser)
File "/root/project/miniconda/lib/python3.9/site-packages/conda/cli/main_create.py", line 41, in execute
install(args, parser, 'create')
File "/root/project/miniconda/lib/python3.9/site-packages/conda/cli/install.py", line 317, in install
handle_txn(unlink_link_transaction, prefix, args, newenv)
File "/root/project/miniconda/lib/python3.9/site-packages/conda/cli/install.py", line 346, in handle_txn
unlink_link_transaction.execute()
File "/root/project/miniconda/lib/python3.9/site-packages/conda/core/link.py", line 249, in execute
self._execute(tuple(concat(interleave(itervalues(self.prefix_action_groups)))))
File "/root/project/miniconda/lib/python3.9/site-packages/conda/core/link.py", line 712, in _execute
raise CondaMultiError(tuple(concatv(
conda.CondaMultiError: No compatible shell found!
()
Experts Please help.
Adding briefs of the script
#!/bin/bash
set -e
install_conda_for_linux(){
#
# Determine Installation Location for non Windows systems
#
#Get path where miniconda needs to get installed and remove if anything exixsts already#
downloaded_file=$base_dir/$conda_file
output_formatted Removing file: $downloaded_file
rm -f $downloaded_file
#
# Download Miniconda
#
output_formatted Downloading Miniconda from: $conda_url '\n' Saving file in: $base_dir
curl -L $conda_url > $base_dir/$conda_file
#
# Install Miniconda
#
rm -rf $install_dir
bash $base_dir/$conda_file -b -p $install_dir
#
# Modify PATH
#
conda_path=$install_dir/bin
export PATH=$conda_path:\$PATH
conda_version=`conda --version`
}
#
# Variables
#
pyversion=3
python_version="3.6.10"
conda_version="4.9.2"
skip_install=$1
base_url='https://repo.anaconda.com/miniconda'
virtual_env=venv
#conda_file is only specified for use in messages below on Windows, as it is manual install, which must be done before running this script.
declare -A conda_file_map
conda_file_map[Linux]="Miniconda${pyversion}-py39_${conda_version}-Linux-x86_64.sh"
conda_file=${conda_file_map[${os_type}]}
#
# Installation of conda and its dependencies
#
if [ ${skip_install} != 'true' ];then
conda_url=${base_url}/${conda_file}
install_conda_for_linux
#
# Create Environment
#
output_formatted Creating new virtual environment: $virtual_env for python_version $python_version
conda create -y -vv --name $virtual_env python=$python_version

Here's your bug:
conda_path=$install_dir/bin
export PATH=$conda_path:\$PATH
Let's say install_dir=/path/to/install, and the starting value of PATH is PATH=/bin:/usr/bin:/usr/local/bin (which is how which sh finds /bin/sh or /usr/bin/sh).
After you ran this command, because of the backslash, you don't have PATH=/path/to/install/bin:/bin:/usr/bin:/usr/local/bin (which is what you want), but instead you have PATH=/path/to/install/bin:$PATH; the backslash caused the literal string $PATH to be added to your variable, instead of the string that's contained therein.
Thus, /bin and /usr/bin are no longer listed in your PATH variable, so which can't find them.
To fix this, just make it:
conda_path=$install_dir/bin
PATH=$conda_path:$PATH
The big fix is changing \$PATH to just $PATH -- but beyond that, you don't need the export (changes to variables that are already in the environment are automatically re-exported), and having it adds complexity for no good reason.

Related

Poetry config - command: poetry not found, but path added to bashrc and shrc files. export path command caused directory error running script

I am a new linux/python user trying to get poetry set up. Basically I can make the poetry command work by using the export path statement, but that only causes a directory issue when executing script. Ive downloaded poetry using the command:
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
I know that it's installed because if I try and run that again, it says it already downloaded. I run into the error:
command: poetry not found
I used
nano ./bashrc
nano ./shrc
in terminal and I've edited both of those with
export PATH="$HOME/.poetry/bin:$PATH"
then, I'm following instructions to run a set of scripts under directory "thermalproc" that I've been given, and the path is ~/Desktop/VideoProcessing/thermalproc-20220625/thermalproc . I'm to use the command
poetry run thermalproc
the only thing that I have done to make the command work is to use that export line directly in the terminal, and that does allow the command to work, but if i run just the command as so:
joshua#Andrea-G750JZA:~$ poetry run thermalproc
it returns
RuntimeError
Poetry could not find a pyproject.toml file in /home/joshua or its parents
at .poetry/lib/poetry/_vendor/py3.8/poetry/core/factory.py:369 in locate
365│ if poetry_file.exists():
366│ return poetry_file
367│
368│ else:
→ 369│ raise RuntimeError(
370│ "Poetry could not find a pyproject.toml file in {} or its parents".format(
371│ cwd
372│ )
373│ )
and if I run in the directory of the script as I originally thought I should:
joshua#Andrea-G750JZA:~/Desktop/VideoProcessing/thermalproc-20220625/thermalproc$ poetry run thermalproc.
FileNotFoundError
[Errno 2] No such file or directory: b'/snap/bin/thermalproc.'
at /usr/lib/python3.8/os.py:601 in _execvpe
597│ path_list = map(fsencode, path_list)
598│ for dir in path_list:
599│ fullname = path.join(dir, file)
600│ try:
→ 601│ exec_func(fullname, *argrest)
602│ except (FileNotFoundError, NotADirectoryError) as e:
603│ last_exc = e
604│ except OSError as e:
605│ last_exc = e
Thanks for sticking it out this long, any advice would be greatly appreciated!

`command not found` when using Bash Script to Install & Run a Software

I am trying to create a Bash script that downloads a .sh file, runs it to install a program (Anaconda for Python), then run the command conda init to activate the default Anaconda environment base
However, running my bash script as shown
# !#/bin/bash
wget https://repo.anaconda.com/archive/Anaconda3-2021.05-Linux-x86_64.sh
bash Anaconda3-*-Linux-x86_64.sh -b
echo 'export PATH=~/anaconda3/bin:$PATH' >> ~/.bashrc
source ~/.bashrc
conda init
conda env create -f environment.yml
gives the error
./foo.sh: line 6: conda: command not found
./foo.sh: line 7: conda: command not found
Why is conda not found when trying to use it from within the Bash script, but is found if I relogin to the terminal?
How can we fix the Bash script so that using conda works?

Define a function in bash and pass it to a .run script

I'm trying to install VirtualBox on a Linux system. When running the installer, I run into an error due to not having bzip2 on this system:
# bash -i VirtualBox-5.2.16-123759-Linux_x86.run
Verifying archive integrity... All good.
Uncompressing VirtualBox for Linux installation.............
VirtualBox Version 5.2.16 r123759 (2018-07-16T15:17:42Z) installer
Removing previous installation of VirtualBox 5.2.16 r123759 from
/opt/VirtualBox
Installing VirtualBox to /opt/VirtualBox
./install.sh: line 252: bzip2: command not found
Unfortunately for me, I cannot get bzip2 installed on this system. However, I do have tar with the bzip2 support (the "j" flag). I cannot modify the .run file to use tar instead of bzip2, however.
So, my new idea is to use a function in bash, something along these lines:
function bzip2() { ... use tar to extract the bz2 file in a way that's compliant with the use case in the .run file ... }
A simple test, however, shows I cannot get the bzip2 function to be used in the .run file:
# bzip2() { echo $1; }
# bzip2 test
test
# bash -i VirtualBox-5.2.16-123759-Linux_x86.run
Verifying archive integrity... All good.
Uncompressing VirtualBox for Linux installation.............
VirtualBox Version 5.2.16 r123759 (2018-07-16T15:17:42Z) installer
Removing previous installation of VirtualBox 5.2.16 r123759 from /opt/VirtualBox
Installing VirtualBox to /opt/VirtualBox
./install.sh: line 252: bzip2: command not found
Functions are not exported into child processes by default. From Bash Reference Manual:
Functions may be exported so that subshells automatically have them
defined with the -f option to the export builtin.
$ cat script
#!/usr/bin/env bash
greet
$ greet() { echo hello; }
$ ./script
./script: line 2: greet: command not found
$ export -f greet
$ ./script
hello

Cannot run source activate with conda in Fish-shell

I follow conda_PR_545, conda issues 4221 and still not working on Ubuntu.
After downloading conda.fish from here, and mv it to anaconda3/bin/.
Add "source /home/phejimlin/anaconda3/bin/conda.fish" at the end of ~/.config/fish/config.fish.
conda activate spark_env
Traceback (most recent call last):
File "/home/phejimlin/anaconda3/bin/conda", line 6, in
sys.exit(conda.cli.main())
File "/home/phejimlin/anaconda3/lib/python3.6/site-packages/conda/cli/main.py", line 161, in main
raise CommandNotFoundError(argv1, message)
TypeError: init() takes 2 positional arguments but 3 were given
or
activate spark_env
Error: activate must be sourced. Run 'source activate envname'
instead of 'activate envname'.
Do I miss something?
As of fish 2.6.0 conda 4.3.27: the following steps may change as issue is addressed
update config
Take note of your conda's location
conda info --root
/Users/mstreeter/anaconda # this is my <PATH_TO_ROOT>
Add line to ~/.config/fish/config.fish
source <PATH_TO_ROOT>/etc/fish/conf.d/conda.fish
update convention
Typically you'd run the following from bash
source activate <environment>
source deactivate <environment>
Now you must run the following from fish
conda activate <environment>
conda deactivate <environment>
issues
so after doing this I'm not able to set fish as my default shell and have it still work properly with conda. Currently, I must first enter my default shell, and enter fish and the shell works as expected. I'll update this after I find out how to get it working completely without the need to explicitly choose fish each time I log into my terminal
If you follow https://github.com/conda/conda/issues/2611, the steps are (from start):
[root#6903a8d80f9b ~]# fish
root#6903a8d80f9b ~# echo $FISH_VERSION
2.4.0
root#6903a8d80f9b ~# bash Miniconda2-4.3.11-Linux-x86_64.sh -b -p /conda
root#6903a8d80f9b ~# source /conda/etc/fish/conf.d/conda.fish
root#6903a8d80f9b ~# conda activate root
root#6903a8d80f9b ~# conda create -yn fishtest (root)
Fetching package metadata .........
Solving package specifications:
Package plan for installation in environment /conda/envs/fishtest:
#
# To activate this environment, use:
# > source activate fishtest
#
# To deactivate this environment, use:
# > source deactivate fishtest
#
root#6903a8d80f9b ~# conda activate fishtest (root)
root#6903a8d80f9b ~# (fishtest)
root#6903a8d80f9b ~# conda deactivate fishtest (fishtest)
Adding conda's bin directory to PATH isn't recommended as of conda 4.4.0
https://github.com/conda/conda/blob/master/CHANGELOG.md#440-2017-12-20
All you need to do is adding
source <path-to-anaconda>/etc/fish/conf.d/conda.fish
to config.fish.

Attempting to install Portia on OSX or Ubuntu

Could someone help me? I have been over and over installing Portia. All goes well until I get to the point where I am using the twistd command and I get this:
(portia)Matts-Mac-mini:slyd matt$ twistd -n slyd
Traceback (most> recent call last): File "/Users/matt/portia/bin/twistd", line 14, in run() File "/Users/matt/portia/lib/python2.7/site-packages/twisted/scripts/twistd.py", line 27, in run app.run(runApp, ServerOptions) File "/Users/matt/portia/lib/python2.7/site-packages/twisted/application/app.py", line 642, in run runApp(config) File "/Users/matt/portia/lib/python2.7/site-packages/twisted/scripts/twistd.py", line 23, in runApp _SomeApplicationRunner(config).run() File "/Users/matt/portia/lib/python2.7/site-packages/twisted/application/app.py", line 376, in run self.application = self.createOrGetApplication() File "/Users/matt/portia/lib/python2.7/site-packages/twisted/application/app.py", line 436, in createOrGetApplication ser = plg.makeService(self.config.subOptions) File "/Users/matt/portia/portia/slyd/slyd/tap.py", line 74, in makeService root = create_root(config) File "/Users/matt/portia/portia/slyd/slyd/tap.py", line 41, in create_root from .projectspec import create_project_resource File "/Users/matt/portia/portia/slyd/slyd/projectspec.py", line 5, in from slybot.validation.schema import get_schema_validator
ImportError: No module named slybot.validation.schema.
I also noted that when trying to do the 'pip install -r requirements.txt' even though I am in the correct directory( [virtualenv-name]/portia/slyd), the requirements.txt file is not in the slyd directory but in the portia directory.
I am going crazy here and any help is very much appreciated.
Looks like There is a mistake in the installation guide.
The guide should be:
virtualenv ENV_NAME --no-site-packages
source ENV_NAME/bin/activate
cd ENV_NAME
git clone https://github.com/scrapinghub/portia.git
cd portia
pip install -r requirements.txt
pip install -e ./slybot
cd slyd
twistd -n slyd
This worked for me. Hopefully it will work for you too.

Resources