NOTE:
I'm duplicating this post because the question has been up on the conda github page for ~6-days with no response. The original link is here:
https://github.com/conda/conda/issues/7938
Current Behavior
When I type try to run conda update -n base conda, conda hung for around 20-minutes on 'Solving environment' and then returned a package plan that did not include an updated version of conda. The package plan that was returned is provided below.
The package plan that was returned is as follows:
environment location: C:\Users\jmatt\Anaconda3
added / updated specs:
- conda
The following packages will be downloaded:
package | build
---------------------------|-----------------
libarchive-3.3.2 | h1d0d21d_1 4.0 MB
lz4-c-1.8.2 | vc14_0 254 KB conda-forge
libcurl-7.61.1 | h7602738_0 249 KB
------------------------------------------------------------
Total: 4.5 MB
The following packages will be UPDATED:
jpeg: 9b-hb83a4c4_2 --> 9b-vc14_2 conda-forge [vc14]
libcurl: 7.61.1-h2a8f88b_0 --> 7.61.1-h7602738_0
libsodium: 1.0.16-h9d3ae62_0 --> 1.0.16-vc14_0 conda-forge [vc14]
libxslt: 1.1.32-hf6f1972_0 --> 1.1.32-vc14_0 conda-forge [vc14]
lz4-c: 1.8.1.2-h2fa13f4_0 --> 1.8.2-vc14_0 conda-forge [vc14]
tk: 8.6.8-hfa6e2cd_0 --> 8.6.8-vc14_0 conda-forge [vc14]
zeromq: 4.2.5-he025d50_1 --> 4.2.5-vc14_2 conda-forge [vc14]
The following packages will be DOWNGRADED:
astropy: 3.0.5-py36he774522_0 --> 3.0.4-py36hfa6e2cd_0
bzip2: 1.0.6-hfa6e2cd_5 --> 1.0.6-vc14_1 conda-forge [vc14]
curl: 7.61.1-h2a8f88b_0 --> 7.60.0-vc14_0 conda-forge [vc14]
cython: 0.29-py36ha925a31_0 --> 0.28.5-py36h6538335_0
freetype: 2.9.1-ha9979f8_1 --> 2.8.1-vc14_0 conda-forge [vc14]
gevent: 1.3.7-py36he774522_1 --> 1.3.6-py36hfa6e2cd_0
hdf5: 1.10.2-hac2f561_1 --> 1.10.2-vc14_0 conda-forge [vc14]
icu: 58.2-ha66f8fd_1 --> 58.2-vc14_0 conda-forge [vc14]
krb5: 1.16.1-h038dc86_6 --> 1.14.6-vc14_0 conda-forge [vc14]
libarchive: 3.3.3-h798a506_0 --> 3.3.2-h1d0d21d_1
libiconv: 1.15-h1df5818_7 --> 1.14-vc14_4 conda-forge [vc14]
libpng: 1.6.35-h2a8f88b_0 --> 1.6.34-vc14_0 conda-forge [vc14]
libtiff: 4.0.9-h36446d0_2 --> 4.0.9-vc14_0 conda-forge [vc14]
libxml2: 2.9.8-hadb2253_1 --> 2.9.5-vc14_1 conda-forge [vc14]
llvmlite: 0.25.0-py36_0 --> 0.24.0-py36h6538335_0
lxml: 4.2.5-py36hef2cd61_0 --> 4.1.1-py36he0adb16_0
lzo: 2.10-h6df0209_2 --> 2.10-vc14_0 conda-forge [vc14]
matplotlib: 3.0.0-py36hd159220_0 --> 2.2.2-py36h153e9ff_0
mistune: 0.8.4-py36he774522_0 --> 0.8.3-py36hfa6e2cd_1
numba: 0.40.0-py36hf9181ef_0 --> 0.39.0-py36h830ac7b_0
pillow: 5.3.0-py36hdc69c19_0 --> 5.1.0-py36h0738816_0
pyqt: 5.9.2-py36h6538335_2 --> 5.6.0-py36_2
pywavelets: 1.0.1-py36h8c2d366_0 --> 1.0.0-py36h452e1ab_0
qt: 5.9.6-vc14h1e9a669_2 --> 5.6.2-vc14_1 conda-forge [vc14]
snappy: 1.1.7-h777316e_3 --> 1.1.7-vc14_1 conda-forge [vc14]
sqlalchemy: 1.2.12-py36he774522_0 --> 1.2.11-py36hfa6e2cd_0
sqlite: 3.25.2-hfa6e2cd_0 --> 3.22.0-vc14_0 conda-forge [vc14]
twisted: 18.9.0-py36he774522_0 --> 18.7.0-py36hfa6e2cd_1
vc: 14.1-h0510ff6_4 --> 14-h0510ff6_3
yaml: 0.1.7-hc54c509_2 --> 0.1.7-vc14_0 conda-forge [vc14]
zlib: 1.2.11-h8395fce_2 --> 1.2.11-vc14_0 conda-forge [vc14]
Proceed ([y]/n)? n
NOTE: the conda version I have installed is 4.5.11 - I'm not sure if this is the most recent version and haven't been able to find a command or resource (other than conda update conda) to check what the most recent version is. I also had a similar problem when trying to conda install websocket-client when websocket client was already installed - I wonder if the current version of conda has trouble when the most recent version of a package is already installed.
NOTE 2: conda update --all solved the environment in a reasonable amount of time (~1 min - I didn't time it precisely).
Steps to Reproduce
conda update -n base conda
As I mentioned above, conda install websocket-client also hung at 'Solving environment' - I already had websocket-client version 0.53.0 installed when I tried to run the install command
Expected Behavior
Conda should either:
1. If the most recent version is installed, conda should promptly inform the user that an update isn't needed.
1. If a newer version is available, I'd expect Conda to solve the environment in a shorter period of time. I think that less than 1-2 minutes would be reasonable - 20+ minutes is too long
Environment Information
The output of: conda info
active environment : base
active env location : C:\Users\jmatt\Anaconda3
shell level : 1
user config file : C:\Users\jmatt\.condarc
populated config files : C:\Users\jmatt\.condarc
conda version : 4.5.11
conda-build version : 3.16.1
python version : 3.6.6.final.0
base environment : C:\Users\jmatt\Anaconda3 (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/win-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/free/win-64
https://repo.anaconda.com/pkgs/free/noarch
https://repo.anaconda.com/pkgs/r/win-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/pro/win-64
https://repo.anaconda.com/pkgs/pro/noarch
https://repo.anaconda.com/pkgs/msys2/win-64
https://repo.anaconda.com/pkgs/msys2/noarch
https://conda.anaconda.org/conda-forge/win-64
https://conda.anaconda.org/conda-forge/noarch
package cache : C:\Users\jmatt\Anaconda3\pkgs
C:\Users\jmatt\AppData\Local\conda\conda\pkgs
envs directories : C:\Users\jmatt\Anaconda3\envs
C:\Users\jmatt\AppData\Local\conda\conda\envs
C:\Users\jmatt\.conda\envs
platform : win-64
user-agent : conda/4.5.11 requests/2.19.1 CPython/3.6.6 Windows/10 Windows/10.0.17134
administrator : False
netrc file : None
offline mode : False
The output of: conda config --show-sources
ssl_verify: True
channels:
- defaults
- conda-forge
The output of: conda list --show-channel-urls
(base) C:\Users\jmatt>conda list --show-channel-urls
packages in environment at C:\Users\jmatt\Anaconda3:
Name Version Build Channel
_ipyw_jlab_nb_ext_conf 0.1.0 py36_0 defaults
alabaster 0.7.12 py36_0 defaults
anaconda custom py36h363777c_0 defaults
anaconda-client 1.7.2 py36_0 defaults
anaconda-navigator 1.9.2 py36_0 defaults
anaconda-project 0.8.2 py36_0 defaults
appdirs 1.4.3 py36h28b3542_0 defaults
asn1crypto 0.24.0 py36_0 defaults
astroid 2.0.4 py36_0 defaults
astropy 3.0.5 py36he774522_0 defaults
atomicwrites 1.2.1 py36_0 defaults
attrs 18.2.0 py36h28b3542_0 defaults
automat 0.7.0 py36_0 defaults
babel 2.6.0 py36_0 defaults
backcall 0.1.0 py36_0 defaults
backports 1.0 py36_1 defaults
backports.os 0.1.1 py36_0 defaults
backports.shutil_get_terminal_size 1.0.0 py36_2 defaults
beautifulsoup4 4.6.3 py36_0 defaults
bitarray 0.8.3 py36hfa6e2cd_0 defaults
bkcharts 0.2 py36h7e685f7_0 defaults
blas 1.0 mkl defaults
blaze 0.11.3 py36_0 defaults
bleach 3.0.2 py36_0 defaults
blosc 1.14.4 he51fdeb_0 defaults
bokeh 0.13.0 py36_0 defaults
boto 2.49.0 py36_0 defaults
bottleneck 1.2.1 py36h452e1ab_1 defaults
bzip2 1.0.6 hfa6e2cd_5 defaults
ca-certificates 2018.03.07 0 defaults
certifi 2018.10.15 py36_0 defaults
cffi 1.11.5 py36h74b6da3_1 defaults
chardet 3.0.4 py36_1 defaults
click 7.0 py36_0 defaults
cloudpickle 0.6.1 py36_0 defaults
clyent 1.2.2 py36_1 defaults
colorama 0.4.0 py36_0 defaults
comtypes 1.1.7 py36_0 defaults
conda 4.5.11 py36_0 defaults
conda-build 3.16.1 py36_0 defaults
conda-env 2.6.0 1 defaults
conda-verify 3.1.1 py36_0 defaults
console_shortcut 0.1.1 3 defaults
constantly 15.1.0 py36h28b3542_0 defaults
contextlib2 0.5.5 py36he5d52c0_0 defaults
cryptography 2.3.1 py36h74b6da3_0 defaults
curl 7.61.1 h2a8f88b_0 defaults
cycler 0.10.0 py36h009560c_0 defaults
cython 0.29 py36ha925a31_0 defaults
cytoolz 0.9.0.1 py36hfa6e2cd_1 defaults
dask 0.19.4 py36_0 defaults
dask-core 0.19.4 py36_0 defaults
datashape 0.5.4 py36_1 defaults
decorator 4.3.0 py36_0 defaults
defusedxml 0.5.0 py36_1 defaults
distributed 1.23.3 py36_0 defaults
docutils 0.14 py36h6012d8f_0 defaults
entrypoints 0.2.3 py36_2 defaults
et_xmlfile 1.0.1 py36h3d2d736_0 defaults
fastcache 1.0.2 py36hfa6e2cd_2 defaults
filelock 3.0.9 py36_0 defaults
flask 1.0.2 py36_1 defaults
flask-cors 3.0.6 py36_0 defaults
freetype 2.9.1 ha9979f8_1 defaults
future 0.16.0 py36_2 defaults
geographiclib 1.49 py_0 conda-forge
geopy 1.17.0 py_0 conda-forge
get_terminal_size 1.0.0 h38e98db_0 defaults
gevent 1.3.7 py36he774522_1 defaults
glob2 0.6 py36_1 defaults
greenlet 0.4.15 py36hfa6e2cd_0 defaults
h5py 2.8.0 py36h3bdd7fb_2 defaults
hdf5 1.10.2 hac2f561_1 defaults
heapdict 1.0.0 py36_2 defaults
html5lib 1.0.1 py36_0 defaults
hyperlink 18.0.0 py36_0 defaults
icc_rt 2017.0.4 h97af966_0 defaults
icu 58.2 ha66f8fd_1 defaults
idna 2.7 py36_0 defaults
imageio 2.4.1 py36_0 defaults
imagesize 1.1.0 py36_0 defaults
importlib_metadata 0.6 py36_0 defaults
incremental 17.5.0 py36_0 defaults
intel-openmp 2019.0 118 defaults
ipykernel 5.1.0 py36h39e3cac_0 defaults
ipython 7.0.1 py36h39e3cac_0 defaults
ipython_genutils 0.2.0 py36h3c5d0ee_0 defaults
ipywidgets 7.4.2 py36_0 defaults
isort 4.3.4 py36_0 defaults
itsdangerous 1.0.0 py36_0 defaults
jdcal 1.4 py36_0 defaults
jedi 0.13.1 py36_0 defaults
jinja2 2.10 py36_0 defaults
jpeg 9b hb83a4c4_2 defaults
jsonschema 2.6.0 py36h7636477_0 defaults
jupyter 1.0.0 py36_7 defaults
jupyter_client 5.2.3 py36_0 defaults
jupyter_console 6.0.0 py36_0 defaults
jupyter_core 4.4.0 py36_0 defaults
jupyterlab 0.35.2 py36_0 defaults
jupyterlab_launcher 0.13.1 py36_0 defaults
jupyterlab_server 0.2.0 py36_0 defaults
keyring 15.1.0 py36_0 defaults
kiwisolver 1.0.1 py36h6538335_0 defaults
krb5 1.16.1 h038dc86_6 defaults
lazy-object-proxy 1.3.1 py36hfa6e2cd_2 defaults
libarchive 3.3.3 h798a506_0 defaults
libcurl 7.61.1 h2a8f88b_0 defaults
libiconv 1.15 h1df5818_7 defaults
libpng 1.6.35 h2a8f88b_0 defaults
libsodium 1.0.16 h9d3ae62_0 defaults
libssh2 1.8.0 hd619d38_4 defaults
libtiff 4.0.9 h36446d0_2 defaults
libxml2 2.9.8 hadb2253_1 defaults
libxslt 1.1.32 hf6f1972_0 defaults
llvmlite 0.25.0 py36_0 defaults
locket 0.2.0 py36hfed976d_1 defaults
lxml 4.2.5 py36hef2cd61_0 defaults
lz4-c 1.8.1.2 h2fa13f4_0 defaults
lzo 2.10 h6df0209_2 defaults
m2w64-gcc-libgfortran 5.3.0 6 defaults
m2w64-gcc-libs 5.3.0 7 defaults
m2w64-gcc-libs-core 5.3.0 7 defaults
m2w64-gmp 6.1.0 2 defaults
m2w64-libwinpthread-git 5.0.0.4634.697f757 2 defaults
markupsafe 1.0 py36hfa6e2cd_1 defaults
matplotlib 3.0.0 py36hd159220_0 defaults
mccabe 0.6.1 py36_1 defaults
menuinst 1.4.14 py36hfa6e2cd_0 defaults
mistune 0.8.4 py36he774522_0 defaults
mkl 2019.0 118 defaults
mkl-service 1.1.2 py36hb217b18_5 defaults
mkl_fft 1.0.6 py36hdbbee80_0 defaults
mkl_random 1.0.1 py36h77b88f5_1 defaults
more-itertools 4.3.0 py36_0 defaults
mpmath 1.0.0 py36_2 defaults
msgpack-python 0.5.6 py36he980bc4_1 defaults
msys2-conda-epoch 20160418 1 defaults
multipledispatch 0.6.0 py36_0 defaults
navigator-updater 0.2.1 py36_0 defaults
nbconvert 5.3.1 py36_0 defaults
nbformat 4.4.0 py36h3a5bc1b_0 defaults
networkx 2.2 py36_1 defaults
nltk 3.3.0 py36_0 defaults
nose 1.3.7 py36_2 defaults
notebook 5.7.0 py36_0 defaults
numba 0.40.0 py36hf9181ef_0 defaults
numexpr 2.6.8 py36h9ef55f4_0 defaults
numpy 1.15.3 py36ha559c80_0 defaults
numpy-base 1.15.3 py36h8128ebf_0 defaults
numpydoc 0.8.0 py36_0 defaults
odo 0.5.1 py36h7560279_0 defaults
olefile 0.46 py36_0 defaults
openpyxl 2.5.9 py36_0 defaults
openssl 1.0.2p hfa6e2cd_0 defaults
packaging 18.0 py36_0 defaults
pandas 0.23.4 py36h830ac7b_0 defaults
pandoc 2.2.3.2 0 defaults
pandocfilters 1.4.2 py36_1 defaults
parso 0.3.1 py36_0 defaults
partd 0.3.9 py36_0 defaults
path.py 11.5.0 py36_0 defaults
pathlib2 2.3.2 py36_0 defaults
patsy 0.5.0 py36_0 defaults
pep8 1.7.1 py36_0 defaults
pickleshare 0.7.5 py36_0 defaults
pillow 5.3.0 py36hdc69c19_0 defaults
pip 10.0.1 py36_0 defaults
pkginfo 1.4.2 py36_1 defaults
pluggy 0.8.0 py36_0 defaults
ply 3.11 py36_0 defaults
prometheus_client 0.4.2 py36_0 defaults
prompt_toolkit 2.0.6 py36_0 defaults
psutil 5.4.7 py36hfa6e2cd_0 defaults
py 1.7.0 py36_0 defaults
pyasn1 0.4.4 py36h28b3542_0 defaults
pyasn1-modules 0.2.2 py36_0 defaults
pycodestyle 2.4.0 py36_0 defaults
pycosat 0.6.3 py36hfa6e2cd_0 defaults
pycparser 2.19 py36_0 defaults
pycrypto 2.6.1 py36hfa6e2cd_9 defaults
pycurl 7.43.0.2 py36h74b6da3_0 defaults
pyflakes 2.0.0 py36_0 defaults
pygments 2.2.0 py36hb010967_0 defaults
pyhamcrest 1.9.0 py36_2 defaults
pylint 2.1.1 py36_0 defaults
pyodbc 4.0.24 py36h6538335_0 defaults
pyopenssl 18.0.0 py36_0 defaults
pyparsing 2.2.2 py36_0 defaults
pyqt 5.9.2 py36h6538335_2 defaults
pysocks 1.6.8 py36_0 defaults
pytables 3.4.4 py36he6f6034_0 defaults
pytest 3.9.1 py36_0 defaults
pytest-arraydiff 0.2 py36h39e3cac_0 defaults
pytest-astropy 0.4.0 py36_0 defaults
pytest-doctestplus 0.1.3 py36_0 defaults
pytest-openfiles 0.3.0 py36_0 defaults
pytest-remotedata 0.3.0 py36_0 defaults
python 3.6.6 hea74fb7_0 defaults
python-dateutil 2.7.3 py36_0 defaults
python-libarchive-c 2.8 py36_6 defaults
pytz 2018.5 py36_0 defaults
pywavelets 1.0.1 py36h8c2d366_0 defaults
pywin32 223 py36hfa6e2cd_1 defaults
pywinpty 0.5.4 py36_0 defaults
pyyaml 3.13 py36hfa6e2cd_0 defaults
pyzmq 17.1.2 py36hfa6e2cd_0 defaults
qt 5.9.6 vc14h1e9a669_2 defaults
qtawesome 0.5.1 py36_1 defaults
qtconsole 4.4.2 py36_0 defaults
qtpy 1.5.2 py36_0 defaults
requests 2.19.1 py36_0 defaults
rope 0.11.0 py36_0 defaults
ruamel_yaml 0.15.46 py36hfa6e2cd_0 defaults
scikit-image 0.14.0 py36h6538335_1 defaults
scikit-learn 0.20.0 py36heebcf9a_1 defaults
scipy 1.1.0 py36h4f6bf74_1 defaults
seaborn 0.9.0 py36_0 defaults
send2trash 1.5.0 py36_0 defaults
service_identity 17.0.0 py36h28b3542_0 defaults
setuptools 40.4.3 py36_0 defaults
simplegeneric 0.8.1 py36_2 defaults
singledispatch 3.4.0.3 py36h17d0c80_0 defaults
sip 4.19.8 py36h6538335_0 defaults
six 1.11.0 py36_1 defaults
snappy 1.1.7 h777316e_3 defaults
snowballstemmer 1.2.1 py36h763602f_0 defaults
sortedcollections 1.0.1 py36_0 defaults
sortedcontainers 2.0.5 py36_0 defaults
sphinx 1.8.1 py36_0 defaults
sphinxcontrib 1.0 py36_1 defaults
sphinxcontrib-websupport 1.1.0 py36_1 defaults
spyder 3.3.1 py36_1 defaults
spyder-kernels 0.2.6 py36_0 defaults
sqlalchemy 1.2.12 py36he774522_0 defaults
sqlite 3.25.2 hfa6e2cd_0 defaults
statsmodels 0.9.0 py36h452e1ab_0 defaults
sympy 1.3 py36_0 defaults
tblib 1.3.2 py36h30f5020_0 defaults
terminado 0.8.1 py36_1 defaults
testpath 0.4.2 py36_0 defaults
tk 8.6.8 hfa6e2cd_0 defaults
toolz 0.9.0 py36_0 defaults
tornado 5.1.1 py36hfa6e2cd_0 defaults
tqdm 4.26.0 py36h28b3542_0 defaults
traitlets 4.3.2 py36h096827d_0 defaults
twisted 18.9.0 py36he774522_0 defaults
typed-ast 1.1.0 py36hfa6e2cd_0 defaults
typing 3.6.4 py36_0 defaults
unicodecsv 0.14.1 py36h6450c06_0 defaults
urllib3 1.23 py36_0 defaults
vc 14.1 h0510ff6_4 defaults
vs2015_runtime 14.15.26706 h3a45250_0 defaults
wcwidth 0.1.7 py36h3d5aa90_0 defaults
webencodings 0.5.1 py36_1 defaults
websocket-client 0.53.0 py36_1000 conda-forge
werkzeug 0.14.1 py36_0 defaults
wheel 0.32.2 py36_0 defaults
widgetsnbextension 3.4.2 py36_0 defaults
win_inet_pton 1.0.1 py36_1 defaults
win_unicode_console 0.5 py36hcdbd4b5_0 defaults
wincertstore 0.2 py36h7fe50ca_0 defaults
winpty 0.4.3 4 defaults
wrapt 1.10.11 py36hfa6e2cd_2 defaults
xlrd 1.1.0 py36_1 defaults
xlsxwriter 1.1.2 py36_0 defaults
xlwings 0.12.1 py36_0 defaults
xlwt 1.3.0 py36h1a4751e_0 defaults
xz 5.2.4 h2fa13f4_4 defaults
yaml 0.1.7 hc54c509_2 defaults
zeromq 4.2.5 he025d50_1 defaults
zict 0.1.3 py36_0 defaults
zlib 1.2.11 h8395fce_2 defaults
zope 1.0 py36_1 defaults
zope.interface 4.5.0 py36hfa6e2cd_0 defaults
A Common Problem: Suboptimal Channel Prioritization
Anaconda distribution is designed and tested to use the anaconda channel (a subset of defaults) as its primary channel. Adding conda-forge in either a higher- (channel_priority: strict) or equal-priority (channel_priority: flexible) configuration opens up many of the packages to be sourced from Conda Forge instead, and this is where Conda struggles to solve.
Including conda-forge both expands the search as well as opens other packages to be subject to channel switching, and since the anaconda package includes dozens of packages, this can be a huge satisfiability problem to solve. This is often most problematic after the first time conda-forge is added into a user's configuration.
Solutions
There are two high-level ways to improve performance: simplify the solving problem or use a faster solver. Of course, these are not mutually exclusive - feel free to be both thoughtful about what you demand of your solver and adopt optimized tools.
Option 1: Optimize Channel Prioritization
When the anaconda metapackage is installed in an environment, keep the defaults channel at highest priority (first channel in .condarc) and set channel_priority: strict. See the documentation on Managing Channels.
Additionally, one can forcefully prioritize the defaults channel with commands like
conda update -n base --override-channels -c defaults conda
Option 2: Mamba
Mamba is a drop-in replacement for the conda CLI that is faster (compiled) and in my experience tends to be more aggressive in pruning. Once installed, it runs similar to conda, e.g.,
mamba update -n base conda
Note on Alternative Configuration
Many users find the coupling of their environment management infrastructure (Conda) to a large working environment (Anaconda) to be less than ideal. A popular alternative configuration is to maintain a minimal base environment, and if Anaconda is ever needed, to create a new environment with the anaconda package installed.
Alternative options for base environments include
Miniconda - minimal base with defaults channel priority
Miniforge - minimal base with conda-forge channel priority
Mambaforge - Miniforge base + Mamba
Upgrade your conda
For particular conda version
conda install conda=4.7.12
For latest conda version
conda update -n base -c defaults conda
Related
This question already has answers here:
How to run pytorch with NVIDIA "cuda toolkit" version instead of the official conda "cudatoolkit" version?
(2 answers)
Closed 2 years ago.
Please feel free to vote "Reopen" at the bottom of this question. The reason is that I have marked this as a duplicate although the answers there are not clear enough for this question.
As soon as the question is reopened, I can add the the following answer: that cuDNN is an integrated part of the conda pytorch cudatoolkit installer and that it is listed together with pytorch, see py3.7_cuda102_cudnn7_0, and not as a stand-alone package that you might perhaps get by using pip or conda to install it explicitly (not recommended! See all answers of How to run pytorch with NVIDIA “cuda toolkit” version instead of the official conda “cudatoolkit” version?). With conda's cudatoolkit, cuDNN is automatically installed together with CUDA just for pytorch, and as a built-in package inside a conda package it does not show up.
I have installed pytorch using conda install pytorch torchvision cudatoolkit=10.2 -c pytorch.
https://github.com/pytorch/pytorch/issues/17445#issuecomment-466838819 says that cudNN is automatically included.
Trying to check the install according to How to get the CUDA version?, I used conda list cudnn to check, and it is not installed, even though cudNN is needed to use your gpu for ML, while conda list cuda works.
Here is the output of anaconda prompt of both commands.
(base) PS C:\Users\Admin> conda list cudnn
# packages in environment at C:\Users\Admin\anaconda3:
#
# Name Version Build Channel
(base) PS C:\Users\Admin> conda list cuda
# packages in environment at C:\Users\Admin\anaconda3:
#
# Name Version Build Channel
cudatoolkit 10.2.89 h74a9793_1
I find a 445 MB cudnn file "cudnn64_7.dll" in C:\Users\Admin\anaconda3\Lib\site-packages\torch\lib, and there are some small files in C:\Users\Admin\anaconda3\Lib\site-packages\torch\backends\cudnn and C:\Users\Admin\anaconda3\Lib\site-packages\torch\include\ATen.
Questions:
Is cudnn installed as a part of pytorch only?
Is there a way to get conda list cudnn to work?
####
Output of just conda list mentions cudnn at
pytorch 1.6.0 py3.7_cuda102_cudnn7_0 pytorch
Full list:
# Name Version Build Channel
_ipyw_jlab_nb_ext_conf 0.1.0 py37_0
alabaster 0.7.12 py37_0
anaconda 2020.07 py37_0
anaconda-client 1.7.2 py37_0
anaconda-navigator 1.9.12 py37_0
anaconda-project 0.8.4 py_0
argh 0.26.2 py37_0
asn1crypto 1.3.0 py37_1
astroid 2.4.2 py37_0
astropy 4.0.1.post1 py37he774522_1
atomicwrites 1.4.0 py_0
attrs 19.3.0 py_0
autopep8 1.5.3 py_0
babel 2.8.0 py_0
backcall 0.2.0 py_0
backports 1.0 py_2
backports.functools_lru_cache 1.6.1 py_0
backports.shutil_get_terminal_size 1.0.0 py37_2
backports.tempfile 1.0 py_1
backports.weakref 1.0.post1 py_1
bcrypt 3.1.7 py37he774522_1
beautifulsoup4 4.9.1 py37_0
bitarray 1.4.0 py37he774522_0
bkcharts 0.2 py37_0
blas 1.0 mkl
bleach 3.1.5 py_0
blosc 1.19.0 h7bd577a_0
bokeh 2.1.1 py37_0
boto 2.49.0 py37_0
bottleneck 1.3.2 py37h2a96729_1
brotlipy 0.7.0 py37he774522_1000
bzip2 1.0.8 he774522_0
ca-certificates 2020.6.24 0
certifi 2020.6.20 py37_0
cffi 1.14.0 py37h7a1dbc1_0
chardet 3.0.4 py37_1003
click 7.1.2 py_0
cloudpickle 1.5.0 py_0
clyent 1.2.2 py37_1
colorama 0.4.3 py_0
comtypes 1.1.7 py37_1001
conda 4.8.3 py37_0
conda-build 3.18.11 py37_0
conda-env 2.6.0 1
conda-package-handling 1.6.1 py37h62dcd97_0
conda-verify 3.4.2 py_1
console_shortcut 0.1.1 4
contextlib2 0.6.0.post1 py_0
cryptography 2.9.2 py37h7a1dbc1_0
cudatoolkit 10.2.89 h74a9793_1
curl 7.71.1 h2a8f88b_1
cycler 0.10.0 py37_0
cython 0.29.21 py37ha925a31_0
cytoolz 0.10.1 py37he774522_0
dask 2.20.0 py_0
dask-core 2.20.0 py_0
decorator 4.4.2 py_0
defusedxml 0.6.0 py_0
diff-match-patch 20200713 py_0
distributed 2.20.0 py37_0
docutils 0.16 py37_1
entrypoints 0.3 py37_0
et_xmlfile 1.0.1 py_1001
fastcache 1.1.0 py37he774522_0
filelock 3.0.12 py_0
flake8 3.8.3 py_0
flask 1.1.2 py_0
freetype 2.10.2 hd328e21_0
fsspec 0.7.4 py_0
future 0.18.2 py37_1
get_terminal_size 1.0.0 h38e98db_0
gevent 20.6.2 py37he774522_0
glob2 0.7 py_0
gmpy2 2.0.8 py37h0964b28_3
greenlet 0.4.16 py37he774522_0
h5py 2.10.0 py37h5e291fa_0
hdf5 1.10.4 h7ebc959_0
heapdict 1.0.1 py_0
html5lib 1.1 py_0
icc_rt 2019.0.0 h0cc432a_1
icu 58.2 ha925a31_3
idna 2.10 py_0
imageio 2.9.0 py_0
imagesize 1.2.0 py_0
importlib-metadata 1.7.0 py37_0
importlib_metadata 1.7.0 0
intel-openmp 2019.4 245
intervaltree 3.0.2 py_1
ipykernel 5.3.2 py37h5ca1d4c_0
ipython 7.16.1 py37h5ca1d4c_0
ipython_genutils 0.2.0 py37_0
ipywidgets 7.5.1 py_0
isort 4.3.21 py37_0
itsdangerous 1.1.0 py37_0
jdcal 1.4.1 py_0
jedi 0.17.1 py37_0
jinja2 2.11.2 py_0
joblib 0.16.0 py_0
jpeg 9b hb83a4c4_2
json5 0.9.5 py_0
jsonschema 3.2.0 py37_1
jupyter 1.0.0 py37_7
jupyter_client 6.1.6 py_0
jupyter_console 6.1.0 py_0
jupyter_core 4.6.3 py37_0
jupyterlab 2.1.5 py_0
jupyterlab_server 1.2.0 py_0
keyring 21.2.1 py37_0
kiwisolver 1.2.0 py37h74a9793_0
krb5 1.18.2 hc04afaa_0
lazy-object-proxy 1.4.3 py37he774522_0
libarchive 3.4.2 h5e25573_0
libcurl 7.71.1 h2a8f88b_1
libiconv 1.15 h1df5818_7
liblief 0.10.1 ha925a31_0
libllvm9 9.0.1 h21ff451_0
libpng 1.6.37 h2a8f88b_0
libsodium 1.0.18 h62dcd97_0
libspatialindex 1.9.3 h33f27b4_0
libssh2 1.9.0 h7a1dbc1_1
libtiff 4.1.0 h56a325e_1
libxml2 2.9.10 h464c3ec_1
libxslt 1.1.34 he774522_0
llvmlite 0.33.0 py37ha925a31_0
locket 0.2.0 py37_1
lxml 4.5.2 py37h1350720_0
lz4-c 1.9.2 h62dcd97_0
lzo 2.10 he774522_2
m2w64-gcc-libgfortran 5.3.0 6
m2w64-gcc-libs 5.3.0 7
m2w64-gcc-libs-core 5.3.0 7
m2w64-gmp 6.1.0 2
m2w64-libwinpthread-git 5.0.0.4634.697f757 2
markupsafe 1.1.1 py37hfa6e2cd_1
matplotlib 3.2.2 0
matplotlib-base 3.2.2 py37h64f37c6_0
mccabe 0.6.1 py37_1
menuinst 1.4.16 py37he774522_1
mistune 0.8.4 py37hfa6e2cd_1001
mkl 2019.4 245
mkl-service 2.3.0 py37hb782905_0
mkl_fft 1.0.14 py37h6288b17_0
mkl_random 1.0.4 py37h343c172_0
mock 4.0.2 py_0
more-itertools 8.4.0 py_0
mpc 1.1.0 h7edee0f_1
mpfr 4.0.2 h62dcd97_1
mpir 3.0.0 hec2e145_1
mpmath 1.1.0 py37_0
msgpack-python 1.0.0 py37h74a9793_1
msys2-conda-epoch 20160418 1
multipledispatch 0.6.0 py37_0
navigator-updater 0.2.1 py37_0
nbconvert 5.6.1 py37_1
nbformat 5.0.7 py_0
networkx 2.4 py_1
ninja 1.9.0 py37h74a9793_0
nltk 3.5 py_0
nose 1.3.7 py37_1004
notebook 6.0.3 py37_0
numba 0.50.1 py37h47e9c7a_0
numexpr 2.7.1 py37h25d0782_0
numpy 1.17.0 py37h19fb1c0_0
numpy-base 1.17.0 py37hc3f5095_0
numpydoc 1.1.0 py_0
olefile 0.46 py37_0
openpyxl 3.0.4 py_0
openssl 1.1.1g he774522_0
packaging 20.4 py_0
pandas 1.0.5 py37h47e9c7a_0
pandoc 2.10 0
pandocfilters 1.4.2 py37_1
paramiko 2.7.1 py_0
parso 0.7.0 py_0
partd 1.1.0 py_0
path 13.1.0 py37_0
path.py 12.4.0 0
pathlib2 2.3.5 py37_1
pathtools 0.1.2 py_1
patsy 0.5.1 py37_0
pep8 1.7.1 py37_0
pexpect 4.8.0 py37_1
pickleshare 0.7.5 py37_1001
pillow 7.2.0 py37hcc1f983_0
pip 20.1.1 py37_1
pkginfo 1.5.0.1 py37_0
pluggy 0.13.1 py37_0
ply 3.11 py37_0
powershell_shortcut 0.0.1 3
prometheus_client 0.8.0 py_0
prompt-toolkit 3.0.5 py_0
prompt_toolkit 3.0.5 0
psutil 5.7.0 py37he774522_0
py 1.9.0 py_0
py-lief 0.10.1 py37ha925a31_0
pybson 0.5.9 pypi_0 pypi
pyclustering 0.9.3.1 pypi_0 pypi
pycodestyle 2.6.0 py_0
pycosat 0.6.3 py37he774522_0
pycparser 2.20 py_2
pycrypto 2.6.1 py37he774522_10
pycurl 7.43.0.5 py37h7a1dbc1_0
pydocstyle 5.0.2 py_0
pyflakes 2.2.0 py_0
pygments 2.6.1 py_0
pylint 2.5.3 py37_0
pymongo 3.9.0 py37ha925a31_0
pynacl 1.4.0 py37h62dcd97_1
pyodbc 4.0.30 py37ha925a31_0
pyopenssl 19.1.0 py_1
pyparsing 2.4.7 py_0
pyqt 5.9.2 py37h6538335_2
pyreadline 2.1 py37_1
pyrsistent 0.16.0 py37he774522_0
pysocks 1.7.1 py37_1
pytables 3.6.1 py37h1da0976_0
pytest 5.4.3 py37_0
python 3.7.7 h81c818b_4
python-dateutil 2.8.1 py_0
python-jsonrpc-server 0.3.4 py_1
python-language-server 0.34.1 py37_0
python-libarchive-c 2.9 py_0
pytorch 1.6.0 py3.7_cuda102_cudnn7_0 pytorch
pytz 2020.1 py_0
pywavelets 1.1.1 py37he774522_0
pywin32 227 py37he774522_1
pywin32-ctypes 0.2.0 py37_1001
pywinpty 0.5.7 py37_0
pyyaml 5.3.1 py37he774522_1
pyzmq 19.0.1 py37ha925a31_1
qdarkstyle 2.8.1 py_0
qt 5.9.7 vc14h73c81de_0
qtawesome 0.7.2 py_0
qtconsole 4.7.5 py_0
qtpy 1.9.0 py_0
regex 2020.6.8 py37he774522_0
requests 2.24.0 py_0
rope 0.17.0 py_0
rtree 0.9.4 py37h21ff451_1
ruamel_yaml 0.15.87 py37he774522_1
scikit-image 0.16.2 py37h47e9c7a_0
scikit-learn 0.23.1 py37h25d0782_0
scipy 1.5.0 py37h9439919_0
seaborn 0.10.1 py_0
send2trash 1.5.0 py37_0
setuptools 49.2.0 py37_0
simplegeneric 0.8.1 py37_2
singledispatch 3.4.0.3 py37_0
sip 4.19.8 py37h6538335_0
six 1.15.0 py_0
snappy 1.1.8 h33f27b4_0
snowballstemmer 2.0.0 py_0
sortedcollections 1.2.1 py_0
sortedcontainers 2.2.2 py_0
soupsieve 2.0.1 py_0
sphinx 3.1.2 py_0
sphinxcontrib 1.0 py37_1
sphinxcontrib-applehelp 1.0.2 py_0
sphinxcontrib-devhelp 1.0.2 py_0
sphinxcontrib-htmlhelp 1.0.3 py_0
sphinxcontrib-jsmath 1.0.1 py_0
sphinxcontrib-qthelp 1.0.3 py_0
sphinxcontrib-serializinghtml 1.1.4 py_0
sphinxcontrib-websupport 1.2.3 py_0
spyder 4.1.4 py37_0
spyder-kernels 1.9.2 py37_0
sqlalchemy 1.3.18 py37he774522_0
sqlite 3.32.3 h2a8f88b_0
statsmodels 0.11.1 py37he774522_0
sympy 1.6.1 py37_0
tbb 2020.0 h74a9793_0
tblib 1.6.0 py_0
terminado 0.8.3 py37_0
testpath 0.4.4 py_0
threadpoolctl 2.1.0 pyh5ca1d4c_0
tk 8.6.10 he774522_0
toml 0.10.1 py_0
toolz 0.10.0 py_0
torchvision 0.7.0 py37_cu102 pytorch
tornado 6.0.4 py37he774522_1
tqdm 4.47.0 py_0
traitlets 4.3.3 py37_0
typed-ast 1.4.1 py37he774522_0
typing_extensions 3.7.4.2 py_0
ujson 1.35 py37hfa6e2cd_0
unicodecsv 0.14.1 py37_0
urllib3 1.25.9 py_0
vc 14.1 h0510ff6_4
vs2015_runtime 14.16.27012 hf0eaf9b_3
watchdog 0.10.3 py37_0
wcwidth 0.2.5 py_0
webencodings 0.5.1 py37_1
werkzeug 1.0.1 py_0
wheel 0.34.2 py37_0
widgetsnbextension 3.5.1 py37_0
win_inet_pton 1.1.0 py37_0
win_unicode_console 0.5 py37_0
wincertstore 0.2 py37_0
winpty 0.4.3 4
wrapt 1.11.2 py37he774522_0
xlrd 1.2.0 py37_0
xlsxwriter 1.2.9 py_0
xlwings 0.19.5 py37_0
xlwt 1.3.0 py37_0
xmltodict 0.12.0 py_0
xz 5.2.5 h62dcd97_0
yaml 0.2.5 he774522_0
yapf 0.30.0 py_0
zeromq 4.3.2 ha925a31_2
zict 2.0.0 py_0
zipp 3.1.0 py_0
zlib 1.2.11 h62dcd97_4
zope 1.0 py37_1
zope.event 4.4 py37_0
zope.interface 4.7.1 py37he774522_0
zstd 1.4.5 ha9fde0e_0
You can try installing CuDNN explicitly by using conda install -c anaconda cudnn. Then conda list cudnn should show CuDNN.
I was trying to run dask-on-yarn with my research group's Hadoop cluster.
I tried each of the following instructions:
dd.read_parquet('hdfs://file.parquet', engine='fastparquet')
dd.read_parquet('hdfs://file.parquet', engine='pyarrow')
dd.read_csv('hdfs://file.csv')
Each time, the following error message occurs:
~/miniconda3/envs/dask/lib/python3.8/site-packages/fsspec/core.py in get_fs_token_paths(urlpath, mode, num, name_function, storage_options, protocol)
521 path = cls._strip_protocol(urlpath)
522 update_storage_options(options, storage_options)
--> 523 fs = cls(**options)
524
525 if "w" in mode:
~/miniconda3/envs/dask/lib/python3.8/site-packages/fsspec/spec.py in __call__(cls, *args, **kwargs)
52 return cls._cache[token]
53 else:
---> 54 obj = super().__call__(*args, **kwargs)
55 # Setting _fs_token here causes some static linters to complain.
56 obj._fs_token_ = token
~/miniconda3/envs/dask/lib/python3.8/site-packages/fsspec/implementations/hdfs.py in __init__(self, host, port, user, kerb_ticket, driver, extra_conf, **kwargs)
42 AbstractFileSystem.__init__(self, **kwargs)
43 self.pars = (host, port, user, kerb_ticket, driver, extra_conf)
---> 44 self.pahdfs = HadoopFileSystem(
45 host=host,
46 port=port,
~/miniconda3/envs/dask/lib/python3.8/site-packages/pyarrow/hdfs.py in __init__(self, host, port, user, kerb_ticket, driver, extra_conf)
38 _maybe_set_hadoop_classpath()
39
---> 40 self._connect(host, port, user, kerb_ticket, extra_conf)
41
42 def __reduce__(self):
~/miniconda3/envs/dask/lib/python3.8/site-packages/pyarrow/io-hdfs.pxi in pyarrow.lib.HadoopFileSystem._connect()
~/miniconda3/envs/dask/lib/python3.8/site-packages/pyarrow/error.pxi in pyarrow.lib.check_status()
OSError: Getting symbol hdfsNewBuilderfailed
How should I resolve this problem?
My Environment
Here are my packages in this conda env:
# Name Version Build Channel
_libgcc_mutex 0.1 main
abseil-cpp 20200225.2 he1b5a44_0 conda-forge
arrow-cpp 0.17.1 py38h1234567_9_cpu conda-forge
attrs 19.3.0 py_0
aws-sdk-cpp 1.7.164 hc831370_1 conda-forge
backcall 0.2.0 py_0
blas 1.0 mkl
bleach 3.1.5 py_0
bokeh 2.1.1 py38_0
boost-cpp 1.72.0 h7b93d67_1 conda-forge
brotli 1.0.7 he6710b0_0
brotlipy 0.7.0 py38h7b6447c_1000
bzip2 1.0.8 h7b6447c_0
c-ares 1.15.0 h7b6447c_1001
ca-certificates 2020.6.24 0
certifi 2020.6.20 py38_0
cffi 1.14.0 py38he30daa8_1
chardet 3.0.4 py38_1003
click 7.1.2 py_0
cloudpickle 1.4.1 py_0
conda-pack 0.4.0 py_0
cryptography 2.9.2 py38h1ba5d50_0
curl 7.71.0 hbc83047_0
cytoolz 0.10.1 py38h7b6447c_0
dask 2.19.0 py_0
dask-core 2.19.0 py_0
dask-yarn 0.8.1 py38h32f6830_0 conda-forge
decorator 4.4.2 py_0
defusedxml 0.6.0 py_0
distributed 2.19.0 py38_0
entrypoints 0.3 py38_0
fastparquet 0.3.2 py38heb32a55_0
freetype 2.10.2 h5ab3b9f_0
fsspec 0.7.4 py_0
gflags 2.2.2 he6710b0_0
glog 0.4.0 he6710b0_0
grpc-cpp 1.30.0 h9ea6770_0 conda-forge
grpcio 1.27.2 py38hf8bcb03_0
heapdict 1.0.1 py_0
icu 67.1 he1b5a44_0 conda-forge
idna 2.10 py_0
importlib-metadata 1.7.0 py38_0
importlib_metadata 1.7.0 0
intel-openmp 2020.1 217
ipykernel 5.3.0 py38h5ca1d4c_0
ipython 7.16.1 py38h5ca1d4c_0
ipython_genutils 0.2.0 py38_0
jedi 0.17.1 py38_0
jinja2 2.11.2 py_0
jpeg 9b h024ee3a_2
json5 0.9.5 py_0
jsonschema 3.2.0 py38_0
jupyter_client 6.1.3 py_0
jupyter_core 4.6.3 py38_0
jupyterlab 2.1.5 py_0
jupyterlab_server 1.1.5 py_0
krb5 1.18.2 h173b8e3_0
ld_impl_linux-64 2.33.1 h53a641e_7
libcurl 7.71.0 h20c2e04_0
libedit 3.1.20191231 h7b6447c_0
libevent 2.1.10 hcdb4288_1 conda-forge
libffi 3.3 he6710b0_1
libgcc-ng 9.1.0 hdf63c60_0
libgfortran-ng 7.3.0 hdf63c60_0
libllvm9 9.0.1 h4a3c616_0
libpng 1.6.37 hbc83047_0
libprotobuf 3.12.3 hd408876_0
libsodium 1.0.18 h7b6447c_0
libssh2 1.9.0 h1ba5d50_1
libstdcxx-ng 9.1.0 hdf63c60_0
libtiff 4.1.0 h2733197_1
llvmlite 0.33.0 py38hd408876_0
locket 0.2.0 py38_1
lz4-c 1.9.2 he6710b0_0
markupsafe 1.1.1 py38h7b6447c_0
mistune 0.8.4 py38h7b6447c_1000
mkl 2020.1 217
mkl-service 2.3.0 py38he904b0f_0
mkl_fft 1.1.0 py38h23d657b_0
mkl_random 1.1.1 py38h0573a6f_0
msgpack-python 1.0.0 py38hfd86e86_1
nbconvert 5.6.1 py38_0
nbformat 5.0.7 py_0
ncurses 6.2 he6710b0_1
notebook 6.0.3 py38_0
numba 0.50.1 py38h0573a6f_0
numpy 1.18.5 py38ha1c710e_0
numpy-base 1.18.5 py38hde5b4d6_0
olefile 0.46 py_0
openssl 1.1.1g h7b6447c_0
packaging 20.4 py_0
pandas 1.0.5 py38h0573a6f_0
pandoc 2.9.2.1 0
pandocfilters 1.4.2 py38_1
parquet-cpp 1.5.1 2 conda-forge
parso 0.7.0 py_0
partd 1.1.0 py_0
pexpect 4.8.0 py38_0
pickleshare 0.7.5 py38_1000
pillow 7.1.2 py38hb39fc2d_0
pip 20.1.1 py38_1
prometheus_client 0.8.0 py_0
prompt-toolkit 3.0.5 py_0
protobuf 3.12.3 py38he6710b0_0
psutil 5.7.0 py38h7b6447c_0
ptyprocess 0.6.0 py38_0
pyarrow 0.17.1 py38h1234567_9_cpu conda-forge
pycparser 2.20 py_0
pygments 2.6.1 py_0
pyopenssl 19.1.0 py38_0
pyparsing 2.4.7 py_0
pyrsistent 0.16.0 py38h7b6447c_0
pysocks 1.7.1 py38_0
python 3.8.3 hcff3b4d_2
python-dateutil 2.8.1 py_0
python_abi 3.8 1_cp38 conda-forge
pytz 2020.1 py_0
pyyaml 5.3.1 py38h7b6447c_1
pyzmq 19.0.1 py38he6710b0_1
re2 2020.07.01 he1b5a44_0 conda-forge
readline 8.0 h7b6447c_0
requests 2.24.0 py_0
send2trash 1.5.0 py38_0
setuptools 47.3.1 py38_0
six 1.15.0 py_0
skein 0.8.0 py38h32f6830_1 conda-forge
snappy 1.1.8 he6710b0_0
sortedcontainers 2.2.2 py_0
sqlite 3.32.3 h62c20be_0
tbb 2020.0 hfd86e86_0
tblib 1.6.0 py_0
terminado 0.8.3 py38_0
testpath 0.4.4 py_0
thrift 0.13.0 py38he6710b0_0
thrift-cpp 0.13.0 h62aa4f2_2 conda-forge
tk 8.6.10 hbc83047_0
toolz 0.10.0 py_0
tornado 6.0.4 py38h7b6447c_1
traitlets 4.3.3 py38_0
typing_extensions 3.7.4.2 py_0
urllib3 1.25.9 py_0
wcwidth 0.2.5 py_0
webencodings 0.5.1 py38_1
wheel 0.34.2 py38_0
xz 5.2.5 h7b6447c_0
yaml 0.2.5 h7b6447c_0
zeromq 4.3.2 he6710b0_2
zict 2.0.0 py_0
zipp 3.1.0 py_0
zlib 1.2.11 h7b6447c_3
zstd 1.4.4 h0b5b093_3
The Hadoop cluster is running version Hadoop 2.7.0-mapr-1607.
The Cluster object is created with:
# Create a cluster where each worker has two cores and eight GiB of memory
cluster = YarnCluster(
environment='conda-env-packed-for-worker-nodes.tar.gz',
worker_env={
# See https://github.com/dask/dask-yarn/pull/30#issuecomment-434001858
'ARROW_LIBHDFS_DIR': '/opt/mapr/hadoop/hadoop-0.20.2/c++/Linux-amd64-64/lib',
},
)
Suspected Cause
I suspect that the version mismatch between the hadoop-0.20.2 in the ARROW_LIBHDFS_DIR environmental variable and the hadoop CLI version Hadoop 2.7.0 might be a cause of the problem.
I had to manually specify pyarrow to use this file (using this setup: https://stackoverflow.com/a/62749053/1147061). The required file, libhdfs.so, is not provided under /opt/mapr/hadoop/hadoop-2.7.0/. Installing libhdfs3 via conda install -c conda-forge libhdfs3 does not resolve the requirement, either.
Might this be the problem?
(a part answer)
To use libhdfs3 (which is poorly maintained these days), you would need to call
dd.read_csv('hdfs://file.csv', storage_options={'driver': 'libhdfs3'})
and, of course, install libhdfs3. This will not help with the hadoop library option, they are independent code paths.
I also suspect that getting the JNI libhdfs (without the "3") working is a case of locating the right .so file.
I'm training a lot of neural networks using hyperopt at the moment. Sometimes it runs perfectly through, sometimes not and I don't understand why. It refers to the validation data, but that's always the same. I don't use any k-fold CV yet. The parameters stored in the dictionary setup 'Monitor', 'Patience', 'MinDelta', 'Epochs' and 'BatchSize' stay unchanged too. As you can see below, shuffle is also set to False. I've just tried to train the network manually with the same hyperparameters and it went through. GPU VRAM should be enough because I trained larger networks without problems (more neurons, higher batch size). Does someone have any suggestions or guesses what could lead to this error?
Here are some relevant code snippets:
from tensorflow.keras.callbacks import EarlyStopping
early_stopping = EarlyStopping(monitor=setup['Monitor'], patience=setup['Patience'],mode='min',
restore_best_weights=True,min_delta=setup['MinDelta'])
history = autoencoder.fit(trainx, trainy, epochs = setup['Epochs'], batch_size = setup['BatchSize'],
validation_data=(valx,valy),callbacks=[early_stopping],verbose=0,shuffle=False)
Used parameters of the last error that was shown:
{'AFunction': 'relu', 'BatchNorm': False, 'BatchSize': 56, 'Bottleneck': 16, 'Dataset': 'ml100k',
'Date': '2020-03-14__22_46_30', 'DecNeurons': 480, 'Decay': 0.0006340241989020302,
'Dropout': 0.0003539460040469268, 'EncNeurons': 256, 'Epochs': 100, 'ID': 3,
'LR': 0.3869023252696237, 'Layers': 4, 'Metric': 'RMSE', 'MinDelta': 0.01,
'Monitor': 'val_root_mean_squared_error', 'MP': True, 'Noise': 0.02, 'Normalize': False,
'Optimizer': 'adam', 'Patience': 25, 'RDigits': 5, 'Split': 'Movie', 'WeightInit': 0,
'Neurons': [480, 408, 328, 256, 16, 256, 328, 408, 480], 'NeuronSum': 2960,
'Loss': <function MMSE at 0x000001DED8DD60D0>, 'IO': 943}
I'm getting the following error:
File "<ipython-input-3-3d2f96200fbe>", line 26, in <module>
best = fmin(fn=ae,space=parameterspace,algo=algo,trials = bayes_trials,max_evals=5000)
File "C:\Users\Admin\Anaconda3\envs\tf2\lib\site-packages\hyperopt\fmin.py", line 482, in fmin
show_progressbar=show_progressbar,
File "C:\Users\Admin\Anaconda3\envs\tf2\lib\site-packages\hyperopt\base.py", line 686, in fmin
show_progressbar=show_progressbar,
File "C:\Users\Admin\Anaconda3\envs\tf2\lib\site-packages\hyperopt\fmin.py", line 509, in fmin
rval.exhaust()
File "C:\Users\Admin\Anaconda3\envs\tf2\lib\site-packages\hyperopt\fmin.py", line 330, in exhaust
self.run(self.max_evals - n_done, block_until_done=self.asynchronous)
File "C:\Users\Admin\Anaconda3\envs\tf2\lib\site-packages\hyperopt\fmin.py", line 286, in run
self.serial_evaluate()
File "C:\Users\Admin\Anaconda3\envs\tf2\lib\site-packages\hyperopt\fmin.py", line 165, in serial_evaluate
result = self.domain.evaluate(spec, ctrl)
File "C:\Users\Admin\Anaconda3\envs\tf2\lib\site-packages\hyperopt\base.py", line 894, in evaluate
rval = self.fn(pyll_rval)
File "<ipython-input-1-5d0cd446e015>", line 229, in ae
validation_data=(valx,valy),callbacks=[early_stopping],verbose=0,shuffle=False)
File "C:\Users\Admin\Anaconda3\envs\tf2\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 728, in fit
use_multiprocessing=use_multiprocessing)
File "C:\Users\Admin\Anaconda3\envs\tf2\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 372, in fit
prefix='val_')
File "C:\Users\Admin\Anaconda3\envs\tf2\lib\contextlib.py", line 88, in __exit__
next(self.gen)
File "C:\Users\Admin\Anaconda3\envs\tf2\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 685, in on_epoch
self.callbacks.on_epoch_end(epoch, epoch_logs)
File "C:\Users\Admin\Anaconda3\envs\tf2\lib\site-packages\tensorflow_core\python\keras\callbacks.py", line 298, in on_epoch_end
callback.on_epoch_end(epoch, logs)
File "C:\Users\Admin\Anaconda3\envs\tf2\lib\site-packages\tensorflow_core\python\keras\callbacks.py", line 1238, in on_epoch_end
self.model.set_weights(self.best_weights)
File "C:\Users\Admin\Anaconda3\envs\tf2\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 1322, in set_weights
if len(params) != len(weights):
TypeError: object of type 'NoneType' has no len()
Used environment:
(tf2) C:\Users\Admin>conda list
# packages in environment at C:\Users\Admin\Anaconda3\envs\tf2:
#
# Name Version Build Channel
_tflow_select 2.1.0 gpu
absl-py 0.8.1 py36_0
alabaster 0.7.12 py36_0
alembic 1.3.2 py_0 conda-forge
appdirs 1.4.3 py_1 conda-forge
asn1crypto 1.3.0 py36_0
astor 0.8.0 py36_0
astroid 2.3.3 py36_0
attrs 19.3.0 py_0
babel 2.8.0 py_0
backcall 0.1.0 py36_0
backports 1.0 py_2 conda-forge
bayesian-optimization 1.0.1 py_0 conda-forge
blas 1.0 mkl
bleach 3.1.0 py36_0
ca-certificates 2020.1.1 0 anaconda
certifi 2019.11.28 py36_0 anaconda
cffi 1.13.2 py36h7a1dbc1_0
chardet 3.0.4 py36_1003
click 7.0 py_0 conda-forge
cloudpickle 1.2.2 py_0
colorama 0.4.3 py_0
configparser 3.7.3 py36_1 conda-forge
cryptography 2.8 py36h7a1dbc1_0
cudatoolkit 10.0.130 0
cudnn 7.6.5 cuda10.0_0
cycler 0.10.0 py36h009560c_0
databricks-cli 0.9.1 py_0 conda-forge
decorator 4.4.1 py_0
defusedxml 0.6.0 py_0
docker-py 4.1.0 py36_0 conda-forge
docker-pycreds 0.4.0 py_0 conda-forge
docutils 0.15.2 py36_0
entrypoints 0.3 py36_0
flask 1.1.1 py_1 conda-forge
floweaver 2.0.0a5 py_0 conda-forge
freetype 2.9.1 ha9979f8_1
future 0.18.2 pypi_0 pypi
gast 0.2.2 py36_0
gitdb2 2.0.6 py_0 conda-forge
gitpython 3.0.5 py_0 conda-forge
google-pasta 0.1.8 py_0
gorilla 0.3.0 py_0 conda-forge
grpcio 1.16.1 py36h351948d_1
h5py 2.9.0 py36h5e291fa_0
hdf5 1.10.4 h7ebc959_0
hyperopt 0.2.3 pypi_0 pypi
icc_rt 2019.0.0 h0cc432a_1
icu 58.2 ha66f8fd_1
idna 2.8 py36_0
imagesize 1.2.0 py_0
importlib_metadata 1.3.0 py36_0
intel-openmp 2019.4 245
ipykernel 5.1.3 py36h39e3cac_1
ipython 7.11.1 py36h39e3cac_0
ipython_genutils 0.2.0 py36_0
ipywidgets 7.5.1 py_0
isort 4.3.21 py36_0
itsdangerous 1.1.0 py_0 conda-forge
jedi 0.15.2 py36_0
jinja2 2.10.3 py_0
joblib 0.14.1 py_0
jpeg 9b hb83a4c4_2
jsonschema 3.2.0 py36_0
jupyter 1.0.0 py36_7
jupyter_client 5.3.4 py36_0
jupyter_console 6.0.0 py36_0
jupyter_core 4.6.1 py36_0
keras 2.3.1 py36h21ff451_0 conda-forge
keras-applications 1.0.8 py_0
keras-preprocessing 1.1.0 py_1
keyring 20.0.0 py36_0
kiwisolver 1.1.0 py36ha925a31_0
lazy-object-proxy 1.4.3 py36he774522_0
libgpuarray 0.7.6 hfa6e2cd_1003 conda-forge
libpng 1.6.37 h2a8f88b_0
libprotobuf 3.11.2 h7bd577a_0
libsodium 1.0.16 h9d3ae62_0
m2w64-gcc-libgfortran 5.3.0 6
m2w64-gcc-libs 5.3.0 7
m2w64-gcc-libs-core 5.3.0 7
m2w64-gmp 6.1.0 2
m2w64-libwinpthread-git 5.0.0.4634.697f757 2
mako 1.1.0 py_0 conda-forge
markdown 3.1.1 py36_0
markupsafe 1.1.1 py36he774522_0
matplotlib 3.1.1 py36hc8f65d3_0
mccabe 0.6.1 py36_1
mistune 0.8.4 py36he774522_0
mkl 2019.4 245
mkl-service 2.3.0 py36hb782905_0
mkl_fft 1.0.15 py36h14836fe_0
mkl_random 1.1.0 py36h675688f_0
mlflow 1.5.0 py36_1 conda-forge
more-itertools 8.0.2 py_0
msys2-conda-epoch 20160418 1
nbconvert 5.6.1 py36_0
nbformat 4.4.0 py36_0
networkx 2.2 pypi_0 pypi
notebook 6.0.2 py36_0
numpy 1.17.4 py36h4320e6b_0
numpy-base 1.17.4 py36hc3f5095_0
numpydoc 0.9.2 py_0
openssl 1.1.1 he774522_0 anaconda
opt_einsum 3.1.0 py_0
packaging 20.0 py_0
palettable 3.3.0 py_0 conda-forge
pandas 0.25.3 py36ha925a31_0
pandoc 2.2.3.2 0
pandocfilters 1.4.2 py36_1
parso 0.5.2 py_0
pickleshare 0.7.5 py36_0
pip 19.3.1 py36_0
plotly 3.10.0 pypi_0 pypi
prometheus_client 0.7.1 py_0
prometheus_flask_exporter 0.12.1 py_0 conda-forge
prompt_toolkit 2.0.10 py_0
protobuf 3.11.2 py36h33f27b4_0
psutil 5.6.7 py36he774522_0
pycodestyle 2.5.0 py36_0
pycparser 2.19 py36_0
pyflakes 2.1.1 py36_0
pygments 2.5.2 py_0
pygpu 0.7.6 py36hc8d92b1_1000 conda-forge
pylint 2.4.4 py36_0
pyopenssl 19.1.0 py36_0
pyparsing 2.4.6 py_0
pyqt 5.9.2 py36h6538335_2
pyreadline 2.1 py36_1
pyrsistent 0.15.6 py36he774522_0
pysocks 1.7.1 py36_0
python 3.6.10 h9f7ef89_0
python-dateutil 2.8.1 py_0
python-editor 1.0.4 py_0 conda-forge
pytz 2019.3 py_0
pywin32 227 py36he774522_1
pywin32-ctypes 0.2.0 py36_0
pywinpty 0.5.7 py36_0
pyyaml 5.3 py36hfa6e2cd_0 conda-forge
pyzmq 18.1.0 py36ha925a31_0
qt 5.9.7 vc14h73c81de_0
qtawesome 0.6.0 py_0
qtconsole 4.6.0 py_1
qtpy 1.9.0 py_0
querystring_parser 1.2.4 py_0 conda-forge
requests 2.22.0 py36_1
retrying 1.3.3 py36_2
rope 0.14.0 py_0
scikit-learn 0.22.1 py36h6288b17_0
scipy 1.3.2 py36h29ff71c_0
seaborn 0.10.0 py_0 anaconda
send2trash 1.5.0 py36_0
setuptools 44.0.0 py36_0
simplejson 3.17.0 py36hfa6e2cd_0 conda-forge
sip 4.19.8 py36h6538335_0
six 1.13.0 py36_0
smmap2 2.0.5 py_0 conda-forge
snowballstemmer 2.0.0 py_0
sphinx 2.3.1 py_0
sphinxcontrib-applehelp 1.0.1 py_0
sphinxcontrib-devhelp 1.0.1 py_0
sphinxcontrib-htmlhelp 1.0.2 py_0
sphinxcontrib-jsmath 1.0.1 py_0
sphinxcontrib-qthelp 1.0.2 py_0
sphinxcontrib-serializinghtml 1.1.3 py_0
spyder 3.3.6 py36_0
spyder-kernels 0.5.2 py36_0
sqlalchemy 1.3.12 py36hfa6e2cd_0 conda-forge
sqlite 3.30.1 he774522_0
sqlparse 0.3.0 py_0 conda-forge
tabulate 0.8.6 py_0 conda-forge
tensorboard 2.0.0 pyhb38c66f_1
tensorflow 2.0.0 gpu_py36hfdd5754_0
tensorflow-base 2.0.0 gpu_py36h390e234_0
tensorflow-estimator 2.0.0 pyh2649769_0
tensorflow-gpu 2.0.0 h0d30ee6_0
termcolor 1.1.0 py36_1
terminado 0.8.3 py36_0
testpath 0.4.4 py_0
theano 1.0.4 py36h6538335_1001 conda-forge
tornado 6.0.3 py36he774522_0
tqdm 4.43.0 pypi_0 pypi
traitlets 4.3.3 py36_0
typed-ast 1.4.0 py36he774522_0
urllib3 1.25.7 py36_0
vc 14.1 h0510ff6_4
vs2015_runtime 14.16.27012 hf0eaf9b_1
vs2015_win-64 14.0.25420 h55c1224_11
waitress 1.4.1 py_0 conda-forge
wcwidth 0.1.7 py36_0
webencodings 0.5.1 py36_1
websocket-client 0.57.0 py36_0 conda-forge
werkzeug 0.16.0 py_0
wheel 0.33.6 py36_0
widgetsnbextension 3.5.1 py36_0
win_inet_pton 1.1.0 py36_0
wincertstore 0.2 py36h7fe50ca_0
winpty 0.4.3 4
wrapt 1.11.2 py36he774522_0
yaml 0.2.2 hfa6e2cd_1 conda-forge
zeromq 4.3.1 h33f27b4_3
zipp 0.6.0 py_0
zlib 1.2.11 h62dcd97_3
This looks like it is cause by a bug (that is yet to be fixed) that happens when the training only sees a worsening performance.
The line self.best_weights (initialized to None (https://github.com/tensorflow/tensorflow/blob/295ad2781683835be974faba0a191528d8079768/tensorflow/python/keras/callbacks.py#L1190) is only overwritten with weight values if at the end of the epoch (https://github.com/tensorflow/tensorflow/blob/295ad2781683835be974faba0a191528d8079768/tensorflow/python/keras/callbacks.py#L1221) if the performance has improved.
If the data/hyperparameter configuration is such that the the performance only worsens, and you've set restore_best_weights=True, then it calls self.model.set_weights(self.best_weights) with None and that's where the error is raised.
I have created an environment but when I try to export it with
conda env export --name ENVNAME > ENVNAME.yml
I get the following error message:
InvalidVersionSpec: Invalid version '(>=': unable to convert to expression tree: ['(']
In response to the question from FlyingTeller, here is what conda list gives (sorry for the long list):
# Name Version Build Channel
altair 3.2.0 py38_0
appdirs 1.4.3 pypi_0 pypi
appnope 0.1.0 py38_0
asn1crypto 1.3.0 py38_0
attrs 19.3.0 py_0
backcall 0.1.0 py38_0
black 19.10b0 pypi_0 pypi
blas 1.0 mkl
bleach 3.1.0 py_0
ca-certificates 2020.1.1 0
certifi 2019.11.28 py38_0
cffi 1.14.0 py38hb5b8e2f_0
chardet 3.0.4 py38_1003
click 7.0 pypi_0 pypi
cloudpickle 1.3.0 py_0
cryptography 2.8 py38ha12b0ac_0
cycler 0.10.0 py38_0
decorator 4.4.1 py_0
defusedxml 0.6.0 py_0
entrypoints 0.3 py38_0
et_xmlfile 1.0.1 py38_0
ezc3d 1.2.4 py38hbf1eeb5_0 conda-forge
freetype 2.9.1 hb4e5f40_0
idna 2.8 py38_1000
importlib_metadata 1.5.0 py38_0
intel-openmp 2020.0 166
ipykernel 5.1.4 py38h39e3cac_0
ipython 7.12.0 py38h5ca1d4c_0
ipython-genutils 0.2.0 pypi_0 pypi
ipython_genutils 0.2.0 py38_0
jdcal 1.4.1 py_0
jedi 0.16.0 py38_0
jinja2 2.11.1 py_0
joblib 0.14.1 py_0
jsonschema 3.2.0 py38_0
jupyter_client 5.3.4 py38_0
jupyter_core 4.6.1 py38_0
kiwisolver 1.1.0 py38ha1b3eb9_0 conda-forge
libcxx 9.0.1 1 conda-forge
libedit 3.1.20181209 hb402a30_0
libffi 3.2.1 h475c297_4
libgfortran 3.0.1 h93005f0_2
libpng 1.6.37 ha441bb4_0
libsodium 1.0.16 h3efe00b_0
littleutils 0.2.2 py_0 conda-forge
llvm-openmp 4.0.1 hcfea43d_1
markupsafe 1.1.1 py38h1de35cc_0
matplotlib 3.1.3 py38_0
matplotlib-base 3.1.3 py38h9aa3819_0
mistune 0.8.4 py38h1de35cc_1000
mkl 2019.4 233
mkl-service 2.3.0 py38hfbe908c_0
mkl_fft 1.0.15 py38h5e564d8_0
mkl_random 1.1.0 py38h6440ff4_0
more-itertools 8.2.0 py_0
mpmath 1.1.0 py38_0
nb-black 1.0.7 pypi_0 pypi
nb_conda_kernels 2.2.2 py38_0 conda-forge
nbconvert 5.6.1 py38_0
nbformat 5.0.4 py_0
ncurses 6.1 h0a44026_1
notebook 6.0.3 py38_0
numpy 1.18.1 py38h7241aed_0
numpy-base 1.18.1 py38h6575580_1
openpyxl 3.0.3 py_0
openssl 1.1.1d h1de35cc_4
outdated 0.2.0 py_0 conda-forge
pandas 1.0.1 py38h6c726b0_0
pandas-flavor 0.2.0 py_0 conda-forge
pandoc 2.2.3.2 0
pandocfilters 1.4.2 py38_1
parso 0.6.1 py_0
pathspec 0.7.0 pypi_0 pypi
patsy 0.5.1 py38_0
peakutils 1.3.2 py_0 conda-forge
pexpect 4.8.0 py38_0
pickleshare 0.7.5 py38_1000
pingouin 0.3.2 py_0 conda-forge
pip 20.0.2 py38_1
prometheus_client 0.7.1 py_0
prompt_toolkit 3.0.3 py_0
ptyprocess 0.6.0 py38_0
pycparser 2.19 py_0
pygments 2.5.2 py_0
pyopenssl 19.1.0 py38_0
pyparsing 2.4.6 py_0
pyrsistent 0.15.7 py38h1de35cc_0
pysocks 1.7.1 py38_0
python 3.8.1 h359304d_1
python-dateutil 2.8.1 py_0
pytz 2019.3 py_0
pyzmq 18.1.1 py38h0a44026_0
readline 7.0 h1de35cc_5
regex 2020.1.8 pypi_0 pypi
requests 2.22.0 py38_1
scikit-learn 0.22.1 py38h27c97d8_0
scipy 1.4.1 py38h44e99c9_0
seaborn 0.10.0 py_0
send2trash 1.5.0 py38_0
setuptools 45.2.0 py38_0
six 1.14.0 py38_0
spyder-kernels 1.8.1 py38_0
sqlite 3.31.1 ha441bb4_0
statsmodels 0.11.0 py38h1de35cc_0
terminado 0.8.3 py38_0
testpath 0.4.4 py_0
tk 8.6.8 ha441bb4_0
toml 0.10.0 pypi_0 pypi
toolz 0.10.0 py_0
tornado 6.0.3 py38h1de35cc_3
traitlets 4.3.3 py38_0
typed-ast 1.4.1 pypi_0 pypi
urllib3 1.25.8 py38_0
wcwidth 0.1.8 py_0
webencodings 0.5.1 py38_1
wheel 0.34.2 py38_0
wurlitzer 2.0.0 py38_0
xarray 0.15.0 py_0
xlrd 1.2.0 py_0
xlwt 1.3.0 py38_0
xz 5.2.4 h1de35cc_4
zeromq 4.3.1 h0a44026_3
zipp 2.2.0 py_0
zlib 1.2.11 h1de35cc_3
I had run into a similar issue: it is due to some (everlasting) incompatibilities between conda and pip. In my case the pip package nb-black was causing the error: running pip uninstall nb-black followed by conda env export > environment.yml solved it: uninstalling black before exporting your environment might help.
I had a similar issue, which was caused by my own package, created with setuptools and distributed as wheel.
I have used a wrong format, with extra trailing dot, to specify required dependency versions in setup.py of my package.
# setup.py
import setuptools
setuptools.setup(
name="mypackage",
version="1.0.0",
python_requires=">=3.7",
install_requires=[
"pandas>=0.25",
"pyodbc>=4.", # Warning! This should be without trailing dot "pyodbc>=4",
"requests>=2.26"
]
)
In the package installed in a conda environment this information gets saved to %CONDA_PREFIX%/Lib/site-packages/mypackage-1.0.0.dist-info/METADATA. This is where command conda env export fails at reading package versions.
On windows you can use this method, suggested by RLashofRegas at
https://github.com/conda/conda/issues/9624#issuecomment-801623523
In Windows PowerShell you input
cd C:\Users\<user>\Anaconda3\envs\<env name>\Lib\site-packages
then
Get-ChildItem -File -Recurse -Filter METADATA | Select-String "4.7.0<4.8.0" | Select-Object -Unique Path
where you put your error inside the quotation marks, e.g. 4.7.0<4.8.0
This gives the path to the package file, that causes all the troubles.
You can either change it's METADATA, or uninstall.
I had a similar issue. I used the export command in debug mode to identify the package causing the problem. Then, uninstalling the package fixed the issue.
conda env export -vv
I installed PySide2 from conda-forge into an virtual environment named "Qt4Python".
It works if I directly type in the Anaconda prompt (in python):
import PySide2
from PySide2.QtWidgets import QApplication, QLabel
However, if I write these 2 lines in a script file named test1.py, then also from Anaconda prompt call:
python test1.py
Then it produces:
AttributeError: module 'PySide2' has no attribute 'QtWidgets'
What I was missing here ? Below is the package list of the environment.
(Qt4Python) C:\Users\XuanPhu\Desktop\ST_project>conda list
# packages in environment at C:\Users\XuanPhu\AppData\Local\conda\conda\envs\Qt4Python:
#
# Name Version Build Channel
backcall 0.1.0 py37_0
ca-certificates 2018.11.29 ha4d7672_0 conda-forge
certifi 2018.11.29 py37_1000 conda-forge
colorama 0.4.1 py37_0
decorator 4.3.0 py37_0
icu 58.2 ha66f8fd_1
ipython 7.2.0 py37h39e3cac_0
ipython_genutils 0.2.0 py37_0
jedi 0.13.2 py37_0
jpeg 9c hfa6e2cd_1001 conda-forge
libiconv 1.15 hfa6e2cd_1004 conda-forge
libpng 1.6.36 h7602738_1000 conda-forge
libxml2 2.9.8 h9ce36c8_1005 conda-forge
libxslt 1.1.32 heafd4d3_1002 conda-forge
openssl 1.0.2p hfa6e2cd_1002 conda-forge
parso 0.3.1 py37_0
pickleshare 0.7.5 py37_0
pip 18.1 py37_1000 conda-forge
prompt_toolkit 2.0.7 py37_0
pygments 2.3.1 py37_0
pyside2 5.6.0a1 py37h20f30f1_1003 conda-forge
python 3.7.1 hc182675_1000 conda-forge
qt 5.6.2 h2639256_8 conda-forge
setuptools 40.6.3 py37_0 conda-forge
shiboken2 5.12.0 <pip>
six 1.12.0 py37_0
sqlite 3.26.0 hfa6e2cd_1000 conda-forge
traitlets 4.3.2 py37_0
vc 14.1 h0510ff6_4
vs2015_runtime 14.15.26706 h3a45250_0
wcwidth 0.1.7 py37_0
wheel 0.32.3 py37_0 conda-forge
wincertstore 0.2 py37_1002 conda-forge
zlib 1.2.11 h2fa13f4_1004 conda-forge
There could be a version mismatch issue.
I can see you have PySide2 from conda-forge which is really outdated (5.6),
since shiboken2 is using the latest available in pypi (5.12.0),
additionally the qt you have there is not the same for the PySide2 version (5.6.2).
At the moment there is no conda-forge PySide2 package that uses the latest version, which is 5.12.0.
I guess you can either contribute to the project and update the package, or move back and use virtualenv.