I'm struggling to figure out conda virtual environments on windows. All I want is to be able to have different versions of h2o installed at the same time because of their insane decision to not allow you to be able to load files saved in even the most minor different version.
I created a virtual environment by cloning my base anaconda:
conda create -n h203_14_0_7 --clone base
I then activated the virtual environment like so:
C:\ProgramData\Anaconda3\Scripts\activate h203_14_0_7
Now that I'm in the virtual environment (I see the (h203_14_0_7) at the beginning of the prompt), i want to uninstall the version of h2o in this virtual environment so I tried:
pip uninstall h2o
But this output
which to me looks like it's going to uninstall the global h2o rather than the virtual environment h2o. So I think it's using the global pip instead of the pip it should have cloned off the base. So how to I use the virtual environment pip to uninstall h2o just for my virtual environment and how can I be sure that it's doing the right thing?
I then ran
conda intall pip
and it seems that after that I was able to use pip to uninstall h2o only from the virtual environment (I hope). I then downloaded the older h2o version from here: https://github.com/h2oai/h2o-3/releases/tag/jenkins-rel-weierstrass-7
but when I try install it I get
(h203_14_0_7) C:\ProgramData\Anaconda3\envs\h203_14_0_7>pip install C:\Users\dan25\Downloads\h2o-3-jenkins-rel-weierstrass-7.tar.gz
Processing c:\users\dan25\downloads\h2o-3-jenkins-rel-weierstrass-7.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\ProgramData\Anaconda3\envs\h203_14_0_7\lib\tokenize.py", line 452, in open
buffer = _builtin_open(filename, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\dan25\\AppData\\Local\\Temp\\pip-sf7r_6pm-build\\setup.py'
So what now?
I had trouble (e.g. https://0xdata.atlassian.net/browse/PUBDEV-3370 ) getting that approach to ever work. It felt like some kind of global dependency was in there, somewhere.
So, I personally just uninstall, and install the desired version, as I need to move between versions. (Actually, I am more likely to use a different VirtualBox or AWS image for each.)
However I noticed searching for conda on the H2O jira that there is a lot of activity recently. They might all be pointing out the same bug you have found, but if so it sounds like it is something getting enough attention to get fixed.
Aside: finding old versions (and your edit showing install problems)
To find, e.g. 3.14.0.7, google it with "h2o". The top hit is http://h2o-release.s3.amazonaws.com/h2o/rel-weierstrass/7/index.html
The "rel-weierstrass" represents 3.14.0, and the 7 is in the URL. (I've yet to see a full list of all the rel-XXX names, but google will always find at least one in the series, even if it won't find the exact minor version.)
Download the zip file you find there. Inside you will find both an R package, and a whl package for Python. So unzip it, extract the one you want, then pip install it.
These zip files are always on S3 (AFAIK). The link you showed was a source snapshot, on github.
Install requirements:
pip install requests tabulate numpy scikit-learn
Extract the archive:
zcat h2o-3-jenkins-rel-weierstrass-7.tar.gz | tar xvf -
cd into Python directory and build:
cd h2o-py
../gradlew build
I have this working now. I think the trick is to make sure you do NOT have h2o installed on your base python. I did the following:
pip uninstall h2o
conda create --name h2o-base pip
conda activate h2o-base
conda install numpy
conda install pandas
conda install requests
conda install tabulate
conda install colorama
conda install future
conda install jupyter
python -m pip install ipykernel
conda deactivate
And now to install specific versions of h2o, you need to URL of the .whl file for that version and you can find a list of the URLs of all the old versions here: https://github.com/h2oai/h2o-3/blob/master/Changes.md
So for example to install version 3.18.0.8:
conda create --name h2o-3-18-0-8 --clone h2o-base
conda activate h2o-3-18-0-8
pip install http://h2o-release.s3.amazonaws.com/h2o/rel-wolpert/8/Python/h2o-3.18.0.8-py2.py3-none-any.whl
python -m ipykernel install --user --name h2o-3-18-0-8 --display-name "Python (h2o-3-18-0-8)"
or version 3.20.0.2 (make sure to conda deactivate first):
conda create --name h2o-3-20-0-2 --clone h2o-base
conda activate h2o-3-20-0-2
pip install http://h2o-release.s3.amazonaws.com/h2o/rel-wright/2/Python/h2o-3.20.0.2-py2.py3-none-any.whl
python -m ipykernel install --user --name h2o-3-20-0-2 --display-name "Python (h2o-3-20-0-2)"
This set-up allows me to have multiple versions of h2o installed on the same computer and if I have to use serialized models I just have to run python from the virtual environment with the correct version of h2o installed. I think this is preferable to uninstalling and reinstalling h2o each time.
Here is the environments.yml file if you want to skip all the manual installs above:
name: h2o-base
channels:
- conda-forge
- defaults
dependencies:
- asn1crypto=0.24.0=py37_1003
- backcall=0.1.0=py_0
- bleach=3.0.2=py_0
- ca-certificates=2018.10.15=ha4d7672_0
- certifi=2018.10.15=py37_1000
- cffi=1.11.5=py37hfa6e2cd_1001
- chardet=3.0.4=py37_1003
- colorama=0.4.0=py_0
- cryptography=2.3=py37h74b6da3_0
- cryptography-vectors=2.3.1=py37_1000
- decorator=4.3.0=py_0
- entrypoints=0.2.3=py37_1002
- future=0.16.0=py37_1002
- icu=58.2=vc14_0
- idna=2.7=py37_1002
- ipykernel=5.1.0=pyh24bf2e0_0
- ipython=7.0.1=py37h39e3cac_1000
- ipython_genutils=0.2.0=py_1
- ipywidgets=7.4.2=py_0
- jedi=0.13.1=py37_1000
- jinja2=2.10=py_1
- jpeg=9b=vc14_2
- jsonschema=2.6.0=py37_1002
- jupyter=1.0.0=py_1
- jupyter_client=5.2.3=py_1
- jupyter_console=6.0.0=py_0
- jupyter_core=4.4.0=py_0
- libflang=5.0.0=vc14_20180208
- libpng=1.6.34=vc14_0
- libsodium=1.0.16=vc14_0
- llvm-meta=5.0.0=0
- markupsafe=1.0=py37hfa6e2cd_1001
- mistune=0.8.4=py37hfa6e2cd_1000
- nbconvert=5.3.1=py_1
- nbformat=4.4.0=py_1
- notebook=5.7.0=py37_1000
- openblas=0.2.20=vc14_8
- openmp=5.0.0=vc14_1
- openssl=1.0.2p=hfa6e2cd_1001
- pandas=0.23.4=py37h830ac7b_1000
- pandoc=2.3.1=0
- pandocfilters=1.4.2=py_1
- parso=0.3.1=py_0
- pickleshare=0.7.5=py37_1000
- pip=18.1=py37_1000
- prometheus_client=0.4.2=py_0
- prompt_toolkit=2.0.6=py_0
- pycparser=2.19=py_0
- pygments=2.2.0=py_1
- pyopenssl=18.0.0=py37_1000
- pyqt=5.6.0=py37h764d66f_7
- pysocks=1.6.8=py37_1002
- python=3.7.0=hc182675_1005
- python-dateutil=2.7.3=py_0
- pytz=2018.5=py_0
- pywinpty=0.5.4=py37_1002
- pyzmq=17.1.2=py37hf576995_1001
- qt=5.6.2=vc14_1
- qtconsole=4.4.2=py_1
- requests=2.19.1=py37_1001
- send2trash=1.5.0=py_0
- setuptools=40.4.3=py37_0
- simplegeneric=0.8.1=py_1
- sip=4.18.1=py37h6538335_0
- six=1.11.0=py37_1001
- tabulate=0.8.2=py_0
- terminado=0.8.1=py37_1001
- testpath=0.4.2=py37_1000
- tornado=5.1.1=py37hfa6e2cd_1000
- traitlets=4.3.2=py37_1000
- urllib3=1.23=py37_1001
- vc=14=0
- vs2015_runtime=14.0.25420=0
- wcwidth=0.1.7=py_1
- webencodings=0.5.1=py_1
- wheel=0.32.1=py37_0
- widgetsnbextension=3.4.2=py37_1000
- win_inet_pton=1.0.1=py37_1002
- wincertstore=0.2=py37_1002
- winpty=0.4.3=4
- zeromq=4.2.5=vc14_2
- zlib=1.2.11=vc14_0
- blas=1.0=mkl
- icc_rt=2017.0.4=h97af966_0
- intel-openmp=2019.0=118
- m2w64-gcc-libgfortran=5.3.0=6
- m2w64-gcc-libs=5.3.0=7
- m2w64-gcc-libs-core=5.3.0=7
- m2w64-gmp=6.1.0=2
- m2w64-libwinpthread-git=5.0.0.4634.697f757=2
- mkl=2019.0=118
- mkl_fft=1.0.6=py37hdbbee80_0
- mkl_random=1.0.1=py37h77b88f5_1
- msys2-conda-epoch=20160418=1
- numpy=1.15.2=py37ha559c80_0
- numpy-base=1.15.2=py37h8128ebf_0
Related
I'm reinstalling Conda after a PC factory reset and trying to re-create an old conda environment from a yml file that I created by
conda env export --prefix $path_to_old_env_dir > voice_dep.yml
The resulting yml file looks ok to me, here's what it looks like:
name: voiceeda
channels:
- defaults
- conda-forge
dependencies:
- ca-certificates=2022.12.7=h5b45459_0
- libsqlite=3.40.0=hcfcfb64_0
- openssl=1.1.1s=hcfcfb64_1
- pip=22.3.1=pyhd8ed1ab_0
- python=3.9.13=h6244533_2
- setuptools=66.1.1=pyhd8ed1ab_0
- sqlite=3.40.0=hcfcfb64_0
- tzdata=2022g=h191b570_0
- ucrt=10.0.22621.0=h57928b3_0
- vc=14.3=hb6edc58_10
- vs2015_runtime=14.34.31931=h4c5c07a_10
- wheel=0.38.4=pyhd8ed1ab_0
- pip:
- anyio==3.6.2
- argon2-cffi==21.3.0
- argon2-cffi-bindings==21.2.0
- arrow==1.2.3
...
but when I try to run
conda create -n voiceeda -f voice_dep.yml
The following odd error occurs.
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
- voice_dep.yml
I'd understand if it wasn't finding a particular package, I can remove versions etc. if so, but why is it saying it can't find the yml file itself? I'm very confused, wondering if I missed a crucial setup step during conda installation or smth? I'm on Windows 10, and installed anaconda to a D drive (conda version 23.1.0 & Python 3.9.13.).
Any help would be much appreciated, thank you!
I am trying to install some packages from the Bioconda channel with Conda but even though the channel is added, I get to following error:
C:\Users\matti>conda install -c bioconda pybedtools
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
- pybedtools
Current channels:
- https://conda.anaconda.org/bioconda/win-64
- https://conda.anaconda.org/bioconda/noarch
- https://conda.anaconda.org/conda-forge/win-64
- https://conda.anaconda.org/conda-forge/noarch
- https://repo.continuum.io/pkgs/main/win-64
- https://repo.continuum.io/pkgs/main/noarch
- https://repo.continuum.io/pkgs/free/win-64
- https://repo.continuum.io/pkgs/free/noarch
- https://repo.continuum.io/pkgs/r/win-64
- https://repo.continuum.io/pkgs/r/noarch
- https://repo.continuum.io/pkgs/pro/win-64
- https://repo.continuum.io/pkgs/pro/noarch
- https://repo.continuum.io/pkgs/msys2/win-64
- https://repo.continuum.io/pkgs/msys2/noarch
- https://conda.anaconda.org/daler/win-64
- https://conda.anaconda.org/daler/noarch
- https://conda.anaconda.org/anaconda-fusion/win-64
- https://conda.anaconda.org/anaconda-fusion/noarch
This also happens for other Bioconda packages like bowtie2. I have tried regular pip install and easy_install but those do not work either. Any ideas?
Additional Info
OS: Windows
Anaconda base
From your command line it appears you are on windows. There are several veresions of pybedtools on bioconda, however, if I grep through them, they are all for the linux platform.
If you're on Windows 10, you could consider setting up the 'windows subsystem for linux' (and possibly Xming), installing conda, and then installing pybedtools. This is obviously a long-winded approach, but would open up many bioconda packages to you
I an installing environment.yml file via
conda env create -f environment.yml
But I get
raise ReadTimeoutError(self._pool, None, 'Read timed out.')
pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.
failed
CondaEnvException: Pip failed
My environment.yml has a structure like this
name: relightable-nr
channels:
- pytorch
- defaults
dependencies:
- zlib=1.2.11=h7b6447c_3
- zstd=1.3.7=h0b5b093_0
- pip:
- absl-py==0.8.0
- astor==0.8.0
- astroid==2.3.3
- wrapt==1.11.2
- xarray==0.13.0
prefix: /root/anaconda3/envs/envn
I read How to solve ReadTimeoutError: HTTPSConnectionPool(host='pypi.python.org', port=443) with pip? and Pip Install Timeout Issue
I changed my conda default timeout to 300 but how to change pip timeout in my case here?
Pip will pull configuration options from a pip.conf/pip.inf (Unix/Win) file located in either a global, user, or environment scope, and settings such as timeout can be configured there. See the Pip User Guide section on Config Files.
While that answers the question proper, I would be remiss were I not to mention that all the packages listed in the YAML can come from Conda. A more appropriate solution would be to reconfigure the YAML to not hit PyPI in the first place, e.g.,
name: relightable-nr
channels:
- pytorch
- conda-forge
- defaults
dependencies:
- zlib=1.2.11=h7b6447c_3
- zstd=1.3.7=h0b5b093_0
- absl-py=0.8.0
- astor=0.8.0
- astroid=2.3.3
- wrapt=1.11.2
- xarray=0.13.0
but perhaps you abridged the YAML and left out packages that only have PyPI builds. Still, I would recommend getting everything possible from Conda.
You can use:
sudo pip install --default-timeout=100 <name_of_your_library>
I tried to install dgl(https://github.com/dmlc/dgl)
There were several ways to install it.(https://docs.dgl.ai/install/index.html#install-from-source)
pip
conda
from git source
from whl
and I failed with error message when I tried pip
$ pip install dgl-cu101
ERROR: Could not find a version that satisfies the requirement dgl-cu101 (from versions: none)
ERROR: No matching distribution found for dgl-cu101
even pip search spot the package
$ pip search dgl
dgl (0.4.1) - Deep Graph Library
dgl-bots.py (1.1.0) - A python wrapper for https://bots.discord.gl
dgl-cu100 (0.4.1) - Deep Graph Library
dgl-cu92 (0.4.1) - Deep Graph Library
dgl-cu90 (0.4.1) - Deep Graph Library
dgl-cu101 (0.4.1) - Deep Graph Library
dgl-cu102 (0.5a200108) - Deep Graph Library
conda also does not work
$ conda install -c dglteam dgl-cuda10.1
Solving environment: failed
PackagesNotFoundError:
The following packages are not available from current channels:
- dgl-cuda10.1
Current channels:
- https://conda.anaconda.org/dglteam/linux-ppc64le
- https://conda.anaconda.org/dglteam/noarch
- https://repo.anaconda.com/pkgs/main/linux-ppc64le
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/free/linux-ppc64le
- https://repo.anaconda.com/pkgs/free/noarch
- https://repo.anaconda.com/pkgs/r/linux-ppc64le
- https://repo.anaconda.com/pkgs/r/noarch
- https://repo.anaconda.com/pkgs/pro/linux-ppc64le
- https://repo.anaconda.com/pkgs/pro/noarch
- https://conda.anaconda.org/conda-forge/linux-ppc64le
- https://conda.anaconda.org/conda-forge/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
install with source code is not available option because I am the remote client to the server and has no root access
install with whl seems nice but also occurred error.(https://pypi.org/project/dgl/#files)
$ pip install dgl_cu101-0.4.1-cp37-cp37m-manylinux1_x86_64.whl
ERROR: dgl_cu101-0.4.1-cp37-cp37m-manylinux1_x86_64.whl is not a supported wheel on this platform.
I read almost every articles and most of them said it would be the environment problem,
but as far as I know, they match!
My env server
CentOS 7
python 3.7
64 bit
minsky
4 GPUs
designed for ML
My env client
macos
iterm2
no root access
access from different city
How can I solve this problem?
Please help.
Your conda channel
https://conda.anaconda.org/dglteam/linux-ppc64le
gives the clue. Your system seems to be based on a ppc64le CPU, not the most frequently seen x86_64.
As you can see On the conda page, only linux-64 (i.e. x86_64) is available. Same goes for the pypi project.
So your setup does not match.
install with source code is not available option because I am the remote client to the server and has no root access
You should not need root access to compile the source code. The requirements listed in the guide are
gcc-c++ python3-devel make cmake
which, if not available yet could be installed using conda:
conda install -c conda-forge make cmake libgcc
Why is anaconda choking on common packages, in creating an envionment from a YML file? Anaconda COMES with these packages pre-installed in root (or so I thought?)
YML file:
---
name: rasterenv
channels:
- conda-forge
dependencies:
- gdal>=2.2.3
- rasterio
- cython
- jupyter
- matplotlib
- numpy
- pyproj
- shapely
- rasterio
- pandas
- geopandas
- os
- matplotlib
- seaborn
- fiona
- OSMnx
- pip:
- pygeotools
- pygeoprocessing
Trying to build file with: conda env create -f path/to/file
If I create an enviornment with JUST uncommon packages like rasterio, it appears to work. BUT, I want an environment with all! What gives here?
Error is:
ResolvePackageNotFound:
- os
If I remove os from the list, the error then becomes:
ResolvePackageNotFound:
- matplotlib
As #sinoroc pointed out in the comments, os is part of Python standard library and should not be listed as a dependency. (When you do define it as a dependency, Python is going to look for a package called os on all available repositories [PyPI or anaconda.org in this case] and won't find it.)
You can see which packages are part of the standard library by checking the docs here: https://docs.python.org/3/library/
(Also there have been a few questions on SO on how to find out if a particular package is part of the std lib, e.g. How to check if a module/library/package is part of the python standard library?) When you create a new environment the packages from the std lib are the only ones which are available by default. Anything else needs to be installed.
Additionally there are two packages in your yaml file that are listed twice (rasterio and matplotlib) which makes me think that you manually created that file. You can generate a conda environment file by activating an environment and running conda env export > environment.yml which will create a file called environment.yml with all required dependencies.