https://anaconda.org/anaconda/jupyter
I installed Jupyter Notebook with below command to my tutorialEnv
conda install -c anaconda jupyter
Now the version is 6.4.11 but there is a newer verison 6.4.12
so i first tried
conda install -c anaconda jupyter
Nothing changed! it says
All requested packages already installed.
When i use
conda update jupyter
Than there are too much file to install
(base) C:\Users\Messi>activate tutorialEnv
(tutorialEnv) C:\Users\Messi>conda update jupyter
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: C:\Users\Messi\anaconda3\envs\tutorialEnv
added / updated specs:
- jupyter
The following packages will be downloaded:
package | build
---------------------------|-----------------
argon2-cffi-21.3.0 | pyhd3eb1b0_0 15 KB
argon2-cffi-bindings-21.2.0| py39h2bbff1b_0 36 KB
asttokens-2.0.5 | pyhd3eb1b0_0 20 KB
attrs-21.4.0 | pyhd3eb1b0_0 51 KB
backcall-0.2.0 | pyhd3eb1b0_0 13 KB
beautifulsoup4-4.11.1 | py39haa95532_0 190 KB
blas-1.0 | mkl 6 KB
bleach-4.1.0 | pyhd3eb1b0_0 123 KB
certifi-2022.6.15 | py39haa95532_0 153 KB
cffi-1.15.1 | py39h2bbff1b_0 218 KB
colorama-0.4.5 | py39haa95532_0 28 KB
cryptography-37.0.1 | py39h21b164f_0 977 KB
debugpy-1.5.1 | py39hd77b12b_0 2.6 MB
decorator-5.1.1 | pyhd3eb1b0_0 12 KB
defusedxml-0.7.1 | pyhd3eb1b0_0 23 KB
entrypoints-0.4 | py39haa95532_0 17 KB
executing-0.8.3 | pyhd3eb1b0_0 18 KB
glib-2.69.1 | h5dc1a3c_1 1.6 MB
gst-plugins-base-1.18.5 | h9e645db_0 1.7 MB
gstreamer-1.18.5 | hd78058f_0 1.7 MB
intel-openmp-2021.4.0 | haa95532_3556 2.2 MB
ipykernel-6.9.1 | py39haa95532_0 200 KB
ipython-8.4.0 | py39haa95532_0 1009 KB
ipython_genutils-0.2.0 | pyhd3eb1b0_1 27 KB
ipywidgets-7.6.5 | pyhd3eb1b0_1 105 KB
jedi-0.18.1 | py39haa95532_1 982 KB
jinja2-3.0.3 | pyhd3eb1b0_0 106 KB
jpeg-9e | h2bbff1b_0 292 KB
jsonschema-4.4.0 | py39haa95532_0 139 KB
jupyter-1.0.0 | py39haa95532_8 7 KB
jupyter_client-7.2.2 | py39haa95532_0 216 KB
jupyter_console-6.4.3 | pyhd3eb1b0_0 23 KB
jupyter_core-4.10.0 | py39haa95532_0 96 KB
jupyter_server-1.18.1 | py39haa95532_0 372 KB
jupyterlab-3.4.4 | py39haa95532_0 3.8 MB
jupyterlab_pygments-0.1.2 | py_0 8 KB
jupyterlab_server-2.12.0 | py39haa95532_0 83 KB
jupyterlab_widgets-1.0.0 | pyhd3eb1b0_1 109 KB
libclang-12.0.0 |default_h627e005_2 17.8 MB
libffi-3.4.2 | hd77b12b_4 107 KB
libogg-1.3.5 | h2bbff1b_1 33 KB
libpng-1.6.37 | h2a8f88b_0 333 KB
libsodium-1.0.18 | h62dcd97_0 477 KB
libvorbis-1.3.7 | he774522_0 202 KB
libxml2-2.9.14 | h0ad7f3c_0 1.5 MB
libxslt-1.1.35 | h2bbff1b_0 407 KB
markupsafe-2.1.1 | py39h2bbff1b_0 26 KB
matplotlib-inline-0.1.2 | pyhd3eb1b0_2 12 KB
mistune-0.8.4 |py39h2bbff1b_1000 57 KB
mkl-2021.4.0 | haa95532_640 114.9 MB
mkl-service-2.4.0 | py39h2bbff1b_0 51 KB
mkl_fft-1.3.1 | py39h277e83a_0 139 KB
mkl_random-1.2.2 | py39hf11a4ad_0 225 KB
nbclient-0.5.13 | py39haa95532_0 108 KB
nbconvert-6.4.4 | py39haa95532_0 517 KB
nbformat-5.3.0 | py39haa95532_0 146 KB
nest-asyncio-1.5.5 | py39haa95532_0 16 KB
notebook-6.4.12 | py39haa95532_0 4.6 MB
packaging-21.3 | pyhd3eb1b0_0 36 KB
pandocfilters-1.5.0 | pyhd3eb1b0_0 11 KB
parso-0.8.3 | pyhd3eb1b0_0 70 KB
pcre-8.45 | hd77b12b_0 382 KB
pickleshare-0.7.5 | pyhd3eb1b0_1003 13 KB
ply-3.11 | py39haa95532_0 81 KB
prometheus_client-0.14.1 | py39haa95532_0 89 KB
prompt-toolkit-3.0.20 | pyhd3eb1b0_0 259 KB
prompt_toolkit-3.0.20 | hd3eb1b0_0 12 KB
pure_eval-0.2.2 | pyhd3eb1b0_0 14 KB
pycparser-2.21 | pyhd3eb1b0_0 94 KB
pygments-2.11.2 | pyhd3eb1b0_0 759 KB
pyopenssl-22.0.0 | pyhd3eb1b0_0 50 KB
pyparsing-3.0.4 | pyhd3eb1b0_0 81 KB
pyqt-5.15.7 | py39hd77b12b_0 3.7 MB
pyqt5-sip-12.11.0 | py39hd77b12b_0 75 KB
pyrsistent-0.18.0 | py39h196d8e1_0 90 KB
python-dateutil-2.8.2 | pyhd3eb1b0_0 233 KB
python-fastjsonschema-2.15.1| pyhd3eb1b0_0 29 KB
pytz-2022.1 | py39haa95532_0 195 KB
pywin32-302 | py39h2bbff1b_2 5.6 MB
pywinpty-2.0.2 | py39h5da7b33_0 200 KB
pyzmq-23.2.0 | py39hd77b12b_0 404 KB
qt-main-5.15.2 | he8e5bd7_7 50.0 MB
qt-webengine-5.15.9 | hb9a9bb5_4 48.9 MB
qtconsole-5.3.1 | py39haa95532_1 192 KB
qtpy-2.0.1 | pyhd3eb1b0_0 40 KB
qtwebkit-5.212 | h3ad3cdb_4 10.3 MB
requests-2.28.1 | py39haa95532_0 99 KB
send2trash-1.8.0 | pyhd3eb1b0_1 19 KB
setuptools-61.2.0 | py39haa95532_0 1.0 MB
sip-6.6.2 | py39hd77b12b_0 434 KB
six-1.16.0 | pyhd3eb1b0_1 18 KB
soupsieve-2.3.1 | pyhd3eb1b0_0 34 KB
stack_data-0.2.0 | pyhd3eb1b0_0 22 KB
terminado-0.13.1 | py39haa95532_0 31 KB
testpath-0.6.0 | py39haa95532_0 85 KB
tornado-6.1 | py39h2bbff1b_0 598 KB
traitlets-5.1.1 | pyhd3eb1b0_0 84 KB
typing-extensions-4.3.0 | py39haa95532_0 9 KB
typing_extensions-4.3.0 | py39haa95532_0 42 KB
tzdata-2022a | hda174b7_0 109 KB
urllib3-1.26.11 | py39haa95532_0 184 KB
vc-14.2 | h21ff451_1 8 KB
vs2015_runtime-14.27.29016 | h5e58377_2 1007 KB
wcwidth-0.2.5 | pyhd3eb1b0_0 26 KB
webencodings-0.5.1 | py39haa95532_1 20 KB
wheel-0.37.1 | pyhd3eb1b0_0 33 KB
widgetsnbextension-3.5.2 | py39haa95532_0 646 KB
wincertstore-0.2 | py39haa95532_2 15 KB
winpty-0.4.3 | 4 678 KB
zeromq-4.3.4 | hd77b12b_0 4.2 MB
zlib-1.2.12 | h8cc25b3_2 116 KB
zstd-1.5.2 | h19a0ad4_0 509 KB
------------------------------------------------------------
Total: 292.2 MB
The following NEW packages will be INSTALLED:
anyio pkgs/main/win-64::anyio-3.5.0-py39haa95532_0
babel pkgs/main/noarch::babel-2.9.1-pyhd3eb1b0_0
brotlipy pkgs/main/win-64::brotlipy-0.7.0-py39h2bbff1b_1003
charset-normalizer pkgs/main/noarch::charset-normalizer-2.0.4-pyhd3eb1b0
cryptography pkgs/main/win-64::cryptography-37.0.1-py39h21b164f_0
glib pkgs/main/win-64::glib-2.69.1-h5dc1a3c_1
gst-plugins-base pkgs/main/win-64::gst-plugins-base-1.18.5-h9e645db_0
gstreamer pkgs/main/win-64::gstreamer-1.18.5-hd78058f_0
idna pkgs/main/noarch::idna-3.3-pyhd3eb1b0_0
json5 pkgs/main/noarch::json5-0.9.6-pyhd3eb1b0_0
jupyter_server pkgs/main/win-64::jupyter_server-1.18.1-py39haa95532_
jupyterlab pkgs/main/win-64::jupyterlab-3.4.4-py39haa95532_0
jupyterlab_server pkgs/main/win-64::jupyterlab_server-2.12.0-py39haa955
libclang pkgs/main/win-64::libclang-12.0.0-default_h627e005_2
libffi pkgs/main/win-64::libffi-3.4.2-hd77b12b_4
libiconv pkgs/main/win-64::libiconv-1.16-h2bbff1b_2
libogg pkgs/main/win-64::libogg-1.3.5-h2bbff1b_1
libsodium pkgs/main/win-64::libsodium-1.0.18-h62dcd97_0
libvorbis pkgs/main/win-64::libvorbis-1.3.7-he774522_0
libwebp pkgs/main/win-64::libwebp-1.2.2-h2bbff1b_0
libxml2 pkgs/main/win-64::libxml2-2.9.14-h0ad7f3c_0
libxslt pkgs/main/win-64::libxslt-1.1.35-h2bbff1b_0
lz4-c pkgs/main/win-64::lz4-c-1.9.3-h2bbff1b_1
nbclassic pkgs/main/noarch::nbclassic-0.3.5-pyhd3eb1b0_0
pcre pkgs/main/win-64::pcre-8.45-hd77b12b_0
ply pkgs/main/win-64::ply-3.11-py39haa95532_0
pyopenssl pkgs/main/noarch::pyopenssl-22.0.0-pyhd3eb1b0_0
pyqt5-sip pkgs/main/win-64::pyqt5-sip-12.11.0-py39hd77b12b_0
pysocks pkgs/main/win-64::pysocks-1.7.1-py39haa95532_0
pytz pkgs/main/win-64::pytz-2022.1-py39haa95532_0
qt-main pkgs/main/win-64::qt-main-5.15.2-he8e5bd7_7
qt-webengine pkgs/main/win-64::qt-webengine-5.15.9-hb9a9bb5_4
qtwebkit pkgs/main/win-64::qtwebkit-5.212-h3ad3cdb_4
requests pkgs/main/win-64::requests-2.28.1-py39haa95532_0
sniffio pkgs/main/win-64::sniffio-1.2.0-py39haa95532_1
toml pkgs/main/noarch::toml-0.10.2-pyhd3eb1b0_0
urllib3 pkgs/main/win-64::urllib3-1.26.11-py39haa95532_0
websocket-client pkgs/main/win-64::websocket-client-0.58.0-py39haa9553
win_inet_pton pkgs/main/win-64::win_inet_pton-1.1.0-py39haa95532_0
zeromq pkgs/main/win-64::zeromq-4.3.4-hd77b12b_0
zstd pkgs/main/win-64::zstd-1.5.2-h19a0ad4_0
The following packages will be REMOVED:
qt-5.9.7-vc14h73c81de_0
The following packages will be UPDATED:
ca-certificates anaconda::ca-certificates-2022.4.26-h~ --> pkgs/main:
rtificates-2022.07.19-haa95532_0
cffi anaconda::cffi-1.15.0-py39h2bbff1b_1 --> pkgs/main:
1.15.1-py39h2bbff1b_0
colorama anaconda/noarch::colorama-0.4.4-pyhd3~ --> pkgs/main/
::colorama-0.4.5-py39haa95532_0
ipython anaconda::ipython-8.3.0-py39haa95532_0 --> pkgs/main:
on-8.4.0-py39haa95532_0
jupyter anaconda::jupyter-1.0.0-py39haa95532_7 --> pkgs/main:
er-1.0.0-py39haa95532_8
notebook anaconda::notebook-6.4.11-py39haa9553~ --> pkgs/main:
ook-6.4.12-py39haa95532_0
openssl anaconda::openssl-1.1.1o-h2bbff1b_0 --> pkgs/main:
sl-1.1.1q-h2bbff1b_0
pip anaconda::pip-21.2.4-py39haa95532_0 --> pkgs/main:
2.1.2-py39haa95532_0
prometheus_client anaconda/noarch::prometheus_client-0.~ --> pkgs/main/
::prometheus_client-0.14.1-py39haa95532_0
pyqt anaconda::pyqt-5.9.2-py39hd77b12b_6 --> pkgs/main:
5.15.7-py39hd77b12b_0
pyzmq anaconda::pyzmq-22.3.0-py39hd77b12b_2 --> pkgs/main:
-23.2.0-py39hd77b12b_0
qtconsole anaconda/noarch::qtconsole-5.3.0-pyhd~ --> pkgs/main/
::qtconsole-5.3.1-py39haa95532_1
sip anaconda::sip-4.19.13-py39hd77b12b_0 --> pkgs/main:
.6.2-py39hd77b12b_0
sqlite anaconda::sqlite-3.38.5-h2bbff1b_0 --> pkgs/main:
e-3.39.2-h2bbff1b_0
typing-extensions anaconda/noarch::typing-extensions-4.~ --> pkgs/main/
::typing-extensions-4.3.0-py39haa95532_0
typing_extensions anaconda/noarch::typing_extensions-4.~ --> pkgs/main/
::typing_extensions-4.3.0-py39haa95532_0
The following packages will be SUPERSEDED by a higher-priority channel:
argon2-cffi anaconda --> pkgs/main
argon2-cffi-bindi~ anaconda --> pkgs/main
asttokens anaconda --> pkgs/main
attrs anaconda --> pkgs/main
backcall anaconda --> pkgs/main
beautifulsoup4 anaconda --> pkgs/main
blas anaconda --> pkgs/main
bleach anaconda --> pkgs/main
certifi anaconda --> pkgs/main
debugpy anaconda --> pkgs/main
decorator anaconda --> pkgs/main
defusedxml anaconda --> pkgs/main
entrypoints anaconda --> pkgs/main
executing anaconda --> pkgs/main
intel-openmp anaconda --> pkgs/main
ipykernel anaconda --> pkgs/main
ipython_genutils anaconda --> pkgs/main
ipywidgets anaconda --> pkgs/main
jedi anaconda --> pkgs/main
jinja2 anaconda --> pkgs/main
jpeg anaconda --> pkgs/main
jsonschema anaconda --> pkgs/main
jupyter_client anaconda --> pkgs/main
jupyter_console anaconda --> pkgs/main
jupyter_core anaconda --> pkgs/main
jupyterlab_pygmen~ anaconda --> pkgs/main
jupyterlab_widgets anaconda --> pkgs/main
libpng anaconda --> pkgs/main
markupsafe anaconda --> pkgs/main
matplotlib-inline anaconda --> pkgs/main
mistune anaconda --> pkgs/main
mkl anaconda --> pkgs/main
mkl-service anaconda --> pkgs/main
mkl_fft anaconda --> pkgs/main
mkl_random anaconda --> pkgs/main
nbclient anaconda --> pkgs/main
nbconvert anaconda --> pkgs/main
nbformat anaconda --> pkgs/main
nest-asyncio anaconda --> pkgs/main
packaging anaconda --> pkgs/main
pandocfilters anaconda --> pkgs/main
parso anaconda --> pkgs/main
pickleshare anaconda --> pkgs/main
prompt-toolkit anaconda --> pkgs/main
prompt_toolkit anaconda --> pkgs/main
pure_eval anaconda --> pkgs/main
pycparser anaconda --> pkgs/main
pygments anaconda --> pkgs/main
pyparsing anaconda --> pkgs/main
pyrsistent anaconda --> pkgs/main
python-dateutil anaconda --> pkgs/main
python-fastjsonsc~ anaconda --> pkgs/main
pywin32 anaconda --> pkgs/main
pywinpty anaconda --> pkgs/main
qtpy anaconda --> pkgs/main
send2trash anaconda --> pkgs/main
setuptools anaconda --> pkgs/main
six anaconda --> pkgs/main
soupsieve anaconda --> pkgs/main
stack_data anaconda --> pkgs/main
terminado anaconda --> pkgs/main
testpath anaconda --> pkgs/main
tornado anaconda --> pkgs/main
traitlets anaconda --> pkgs/main
tzdata anaconda --> pkgs/main
vc anaconda --> pkgs/main
vs2015_runtime anaconda --> pkgs/main
wcwidth anaconda --> pkgs/main
webencodings anaconda --> pkgs/main
wheel anaconda --> pkgs/main
widgetsnbextension anaconda --> pkgs/main
wincertstore anaconda --> pkgs/main
winpty anaconda --> pkgs/main
zlib anaconda --> pkgs/main
what is the difference between
conda update jupyter and conda update -c anaconda jupyter?
If i use conda update jupyter is this make problem?
I always use conda install -c anaconda package name.
Thanks for comments
The difference here is in the channels used. According to the docs, by default, your anaconda installation is configured to look in the channels
main
r
msys2 (on windows)
free (conda<4.7)
The anaconda channel is only a mirror of main, so it is not needed when doing conda install and can even lead to the strange behavior you are seeing. When multiple channels are specified, conda - by default - has a strict channel priority, meaning that packages will always be installed from the first channel they are available from. See here for more details.
Now let's take a look at the commands you use
conda install -c anaconda jupyter
conda update jupyter
The first command prepends the anaconda channel to your list of channels, making it the highest priority channel. This will result in all packages being installed from the anaconda channel (which you can also see when doing conda list or in the output of the installation).
In the second command, only the default channels are used, making main the highest priority channel. Two things are now happening, which you both see in your output The following packages will be SUPERSEDED by a higher-priority channel: happens because conda now things all packages should be coming from main, even if the same version is already installed from the anaconda channel due to the channel priority. The following packages will be UPDATED: happens either because there actually is a higher version available on the main channel (which was previously ignored because of anaconda having higher priority in the previous command) or the build number differs and conda just treats it as a higher version, e.g. anaconda::jupyter-1.0.0-py39haa95532_7 --> pkgs/main: er-1.0.0-py39haa95532_8
Takeaway
There should be no real need to explicitly state -c anaconda but if you use it, you need to do so on subsequent calls also, otherwise a lot of packages will be reinstalled frequently. Alternatively, you can disable the strict channel priority, but that could slow down the solving step of conda.
As for the difference between install and update: They will both attempt to install the newest version, but behave differently for packages that are not installed. See also this post
Related
I am trying to import seaborn in my IBM cloud (watson) on windows for my final assignment, but somehow it gives me an error. I am trying the following code to import seaborn:
#notice: installing seaborn might takes a few minutes\
!conda install -c anaconda seaborn -y
But then happens this: Please I need help to solve the issue
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: done
Package Plan
environment location: /opt/conda/envs/Python-3.7-main
added / updated specs:
- seaborn
The following packages will be downloaded:
package | build
---------------------------|-----------------
argon2-cffi-20.1.0 | py37h7b6447c_1 49 KB anaconda
defusedxml-0.6.0 | py_0 23 KB anaconda
entrypoints-0.3 | py37_0 12 KB anaconda
fontconfig-2.13.0 | h9420a91_0 291 KB anaconda
icu-58.2 | he6710b0_3 22.7 MB anaconda
jupyter_client-6.1.7 | py_0 76 KB anaconda
libpng-1.6.37 | hbc83047_0 364 KB anaconda
libtiff-4.1.0 | h2733197_1 607 KB anaconda
libuuid-1.0.3 | h1bed415_2 16 KB anaconda
libxcb-1.14 | h7b6447c_0 610 KB anaconda
libxml2-2.9.10 | hb55368b_3 1.3 MB anaconda
lz4-c-1.9.2 | heb0550a_3 203 KB anaconda
markupsafe-1.1.1 | py37h14c3975_1 26 KB anaconda
mistune-0.8.4 |py37h14c3975_1001 53 KB anaconda
ncurses-6.2 | he6710b0_1 1.1 MB anaconda
packaging-20.4 | py_0 35 KB anaconda
pandocfilters-1.4.2 | py37_1 13 KB anaconda
pyjwt-1.7.1 | py37_0 32 KB anaconda
pyparsing-2.4.7 | py_0 64 KB anaconda
webencodings-0.5.1 | py37_1 19 KB anaconda
------------------------------------------------------------
Total: 27.5 MB
Downloading and Extracting Packages
libxml2-2.9.10 | 1.3 MB | ###############################6 | 86% WARNING conda.gateways.disk.delete:unlink_or_rename_to_trash(140): Could not remove or rename /opt/conda/pkgs/libxml2-2.9.10-hb55368b_3/bin/xmllint. Please remove this file manually (you may need to reboot to free file handles)
WARNING conda.gateways.disk.delete:unlink_or_rename_to_trash(140): Could not remove or rename /opt/conda/pkgs/libxml2-2.9.10-hb55368b_3/bin/xml2-config. Please remove this file manually (you may need to reboot to free file handles)
WARNING conda.gateways.disk.delete:unlink_or_rename_to_trash(140): Could not remove or rename /opt/conda/pkgs/libx
If you the notebook is created using the default runtime as shown below, then Seaborn is installed by default.
Run the below command to see Seaborn in the list of packages installed.
!pip list
If you don't see Seaborn in the list, try installing with the below command
!pip install seaborn
If you still see the error, update your question with the region in which you have your project with notebook created and also screenshots of the environment + error helps
I was trying to run dask-on-yarn with my research group's Hadoop cluster.
I tried each of the following instructions:
dd.read_parquet('hdfs://file.parquet', engine='fastparquet')
dd.read_parquet('hdfs://file.parquet', engine='pyarrow')
dd.read_csv('hdfs://file.csv')
Each time, the following error message occurs:
~/miniconda3/envs/dask/lib/python3.8/site-packages/fsspec/core.py in get_fs_token_paths(urlpath, mode, num, name_function, storage_options, protocol)
521 path = cls._strip_protocol(urlpath)
522 update_storage_options(options, storage_options)
--> 523 fs = cls(**options)
524
525 if "w" in mode:
~/miniconda3/envs/dask/lib/python3.8/site-packages/fsspec/spec.py in __call__(cls, *args, **kwargs)
52 return cls._cache[token]
53 else:
---> 54 obj = super().__call__(*args, **kwargs)
55 # Setting _fs_token here causes some static linters to complain.
56 obj._fs_token_ = token
~/miniconda3/envs/dask/lib/python3.8/site-packages/fsspec/implementations/hdfs.py in __init__(self, host, port, user, kerb_ticket, driver, extra_conf, **kwargs)
42 AbstractFileSystem.__init__(self, **kwargs)
43 self.pars = (host, port, user, kerb_ticket, driver, extra_conf)
---> 44 self.pahdfs = HadoopFileSystem(
45 host=host,
46 port=port,
~/miniconda3/envs/dask/lib/python3.8/site-packages/pyarrow/hdfs.py in __init__(self, host, port, user, kerb_ticket, driver, extra_conf)
38 _maybe_set_hadoop_classpath()
39
---> 40 self._connect(host, port, user, kerb_ticket, extra_conf)
41
42 def __reduce__(self):
~/miniconda3/envs/dask/lib/python3.8/site-packages/pyarrow/io-hdfs.pxi in pyarrow.lib.HadoopFileSystem._connect()
~/miniconda3/envs/dask/lib/python3.8/site-packages/pyarrow/error.pxi in pyarrow.lib.check_status()
OSError: Getting symbol hdfsNewBuilderfailed
How should I resolve this problem?
My Environment
Here are my packages in this conda env:
# Name Version Build Channel
_libgcc_mutex 0.1 main
abseil-cpp 20200225.2 he1b5a44_0 conda-forge
arrow-cpp 0.17.1 py38h1234567_9_cpu conda-forge
attrs 19.3.0 py_0
aws-sdk-cpp 1.7.164 hc831370_1 conda-forge
backcall 0.2.0 py_0
blas 1.0 mkl
bleach 3.1.5 py_0
bokeh 2.1.1 py38_0
boost-cpp 1.72.0 h7b93d67_1 conda-forge
brotli 1.0.7 he6710b0_0
brotlipy 0.7.0 py38h7b6447c_1000
bzip2 1.0.8 h7b6447c_0
c-ares 1.15.0 h7b6447c_1001
ca-certificates 2020.6.24 0
certifi 2020.6.20 py38_0
cffi 1.14.0 py38he30daa8_1
chardet 3.0.4 py38_1003
click 7.1.2 py_0
cloudpickle 1.4.1 py_0
conda-pack 0.4.0 py_0
cryptography 2.9.2 py38h1ba5d50_0
curl 7.71.0 hbc83047_0
cytoolz 0.10.1 py38h7b6447c_0
dask 2.19.0 py_0
dask-core 2.19.0 py_0
dask-yarn 0.8.1 py38h32f6830_0 conda-forge
decorator 4.4.2 py_0
defusedxml 0.6.0 py_0
distributed 2.19.0 py38_0
entrypoints 0.3 py38_0
fastparquet 0.3.2 py38heb32a55_0
freetype 2.10.2 h5ab3b9f_0
fsspec 0.7.4 py_0
gflags 2.2.2 he6710b0_0
glog 0.4.0 he6710b0_0
grpc-cpp 1.30.0 h9ea6770_0 conda-forge
grpcio 1.27.2 py38hf8bcb03_0
heapdict 1.0.1 py_0
icu 67.1 he1b5a44_0 conda-forge
idna 2.10 py_0
importlib-metadata 1.7.0 py38_0
importlib_metadata 1.7.0 0
intel-openmp 2020.1 217
ipykernel 5.3.0 py38h5ca1d4c_0
ipython 7.16.1 py38h5ca1d4c_0
ipython_genutils 0.2.0 py38_0
jedi 0.17.1 py38_0
jinja2 2.11.2 py_0
jpeg 9b h024ee3a_2
json5 0.9.5 py_0
jsonschema 3.2.0 py38_0
jupyter_client 6.1.3 py_0
jupyter_core 4.6.3 py38_0
jupyterlab 2.1.5 py_0
jupyterlab_server 1.1.5 py_0
krb5 1.18.2 h173b8e3_0
ld_impl_linux-64 2.33.1 h53a641e_7
libcurl 7.71.0 h20c2e04_0
libedit 3.1.20191231 h7b6447c_0
libevent 2.1.10 hcdb4288_1 conda-forge
libffi 3.3 he6710b0_1
libgcc-ng 9.1.0 hdf63c60_0
libgfortran-ng 7.3.0 hdf63c60_0
libllvm9 9.0.1 h4a3c616_0
libpng 1.6.37 hbc83047_0
libprotobuf 3.12.3 hd408876_0
libsodium 1.0.18 h7b6447c_0
libssh2 1.9.0 h1ba5d50_1
libstdcxx-ng 9.1.0 hdf63c60_0
libtiff 4.1.0 h2733197_1
llvmlite 0.33.0 py38hd408876_0
locket 0.2.0 py38_1
lz4-c 1.9.2 he6710b0_0
markupsafe 1.1.1 py38h7b6447c_0
mistune 0.8.4 py38h7b6447c_1000
mkl 2020.1 217
mkl-service 2.3.0 py38he904b0f_0
mkl_fft 1.1.0 py38h23d657b_0
mkl_random 1.1.1 py38h0573a6f_0
msgpack-python 1.0.0 py38hfd86e86_1
nbconvert 5.6.1 py38_0
nbformat 5.0.7 py_0
ncurses 6.2 he6710b0_1
notebook 6.0.3 py38_0
numba 0.50.1 py38h0573a6f_0
numpy 1.18.5 py38ha1c710e_0
numpy-base 1.18.5 py38hde5b4d6_0
olefile 0.46 py_0
openssl 1.1.1g h7b6447c_0
packaging 20.4 py_0
pandas 1.0.5 py38h0573a6f_0
pandoc 2.9.2.1 0
pandocfilters 1.4.2 py38_1
parquet-cpp 1.5.1 2 conda-forge
parso 0.7.0 py_0
partd 1.1.0 py_0
pexpect 4.8.0 py38_0
pickleshare 0.7.5 py38_1000
pillow 7.1.2 py38hb39fc2d_0
pip 20.1.1 py38_1
prometheus_client 0.8.0 py_0
prompt-toolkit 3.0.5 py_0
protobuf 3.12.3 py38he6710b0_0
psutil 5.7.0 py38h7b6447c_0
ptyprocess 0.6.0 py38_0
pyarrow 0.17.1 py38h1234567_9_cpu conda-forge
pycparser 2.20 py_0
pygments 2.6.1 py_0
pyopenssl 19.1.0 py38_0
pyparsing 2.4.7 py_0
pyrsistent 0.16.0 py38h7b6447c_0
pysocks 1.7.1 py38_0
python 3.8.3 hcff3b4d_2
python-dateutil 2.8.1 py_0
python_abi 3.8 1_cp38 conda-forge
pytz 2020.1 py_0
pyyaml 5.3.1 py38h7b6447c_1
pyzmq 19.0.1 py38he6710b0_1
re2 2020.07.01 he1b5a44_0 conda-forge
readline 8.0 h7b6447c_0
requests 2.24.0 py_0
send2trash 1.5.0 py38_0
setuptools 47.3.1 py38_0
six 1.15.0 py_0
skein 0.8.0 py38h32f6830_1 conda-forge
snappy 1.1.8 he6710b0_0
sortedcontainers 2.2.2 py_0
sqlite 3.32.3 h62c20be_0
tbb 2020.0 hfd86e86_0
tblib 1.6.0 py_0
terminado 0.8.3 py38_0
testpath 0.4.4 py_0
thrift 0.13.0 py38he6710b0_0
thrift-cpp 0.13.0 h62aa4f2_2 conda-forge
tk 8.6.10 hbc83047_0
toolz 0.10.0 py_0
tornado 6.0.4 py38h7b6447c_1
traitlets 4.3.3 py38_0
typing_extensions 3.7.4.2 py_0
urllib3 1.25.9 py_0
wcwidth 0.2.5 py_0
webencodings 0.5.1 py38_1
wheel 0.34.2 py38_0
xz 5.2.5 h7b6447c_0
yaml 0.2.5 h7b6447c_0
zeromq 4.3.2 he6710b0_2
zict 2.0.0 py_0
zipp 3.1.0 py_0
zlib 1.2.11 h7b6447c_3
zstd 1.4.4 h0b5b093_3
The Hadoop cluster is running version Hadoop 2.7.0-mapr-1607.
The Cluster object is created with:
# Create a cluster where each worker has two cores and eight GiB of memory
cluster = YarnCluster(
environment='conda-env-packed-for-worker-nodes.tar.gz',
worker_env={
# See https://github.com/dask/dask-yarn/pull/30#issuecomment-434001858
'ARROW_LIBHDFS_DIR': '/opt/mapr/hadoop/hadoop-0.20.2/c++/Linux-amd64-64/lib',
},
)
Suspected Cause
I suspect that the version mismatch between the hadoop-0.20.2 in the ARROW_LIBHDFS_DIR environmental variable and the hadoop CLI version Hadoop 2.7.0 might be a cause of the problem.
I had to manually specify pyarrow to use this file (using this setup: https://stackoverflow.com/a/62749053/1147061). The required file, libhdfs.so, is not provided under /opt/mapr/hadoop/hadoop-2.7.0/. Installing libhdfs3 via conda install -c conda-forge libhdfs3 does not resolve the requirement, either.
Might this be the problem?
(a part answer)
To use libhdfs3 (which is poorly maintained these days), you would need to call
dd.read_csv('hdfs://file.csv', storage_options={'driver': 'libhdfs3'})
and, of course, install libhdfs3. This will not help with the hadoop library option, they are independent code paths.
I also suspect that getting the JNI libhdfs (without the "3") working is a case of locating the right .so file.
Why do seemingly simple/atomic conda installs result in fairly complex uninstalls??
I recently tried the following conda install
$ conda install -c conda-forge imageio-ffmpeg
Collecting package metadata (current_repodata.json): done
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.7.12
latest version: 4.8.1
Please update conda by running
$ conda update -n base -c defaults conda
## Package Plan ##
environment location: /home/smgutstein/anaconda2/envs/dnn_py3
added / updated specs:
- imageio-ffmpeg
The following packages will be downloaded:
package | build
---------------------------|-----------------
imageio-ffmpeg-0.3.0 | py_0 14 KB conda-forge
------------------------------------------------------------
Total: 14 KB
The following NEW packages will be INSTALLED:
imageio-ffmpeg conda-forge/noarch::imageio-ffmpeg-0.3.0-py_0
Proceed ([y]/n)? y
Downloading and Extracting Packages
imageio-ffmpeg-0.3.0 | 14 KB | ################################################################################################################ | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
This seemed like a fairly atomic install.
However, when I tried to uninstall, things became much more complicated:
$ conda uninstall imageio-ffmpeg
Collecting package metadata (repodata.json): done
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.7.12
latest version: 4.8.1
Please update conda by running
$ conda update -n base -c defaults conda
## Package Plan ##
environment location: /home/smgutstein/anaconda2/envs/dnn_py3
removed specs:
- imageio-ffmpeg
The following packages will be downloaded:
package | build
---------------------------|-----------------
absl-py-0.8.1 | py36_0 164 KB
anaconda-client-1.7.2 | py36_0 147 KB
asn1crypto-1.3.0 | py36_0 164 KB
astor-0.8.0 | py36_0 46 KB
backcall-0.1.0 | py36_0 20 KB
backports-1.0 | py_2 139 KB
bleach-3.1.0 | py36_0 221 KB
c-ares-1.15.0 | h7b6447c_1001 89 KB
ca-certificates-2019.11.27 | 0 124 KB
certifi-2019.11.28 | py36_0 153 KB
chardet-3.0.4 | py36_1003 180 KB
clyent-1.2.2 | py36_1 19 KB
cryptography-2.8 | py36h1ba5d50_0 552 KB
cudnn-7.6.5 | cuda10.0_0 165.0 MB
cycler-0.10.0 | py36_0 13 KB
cytoolz-0.10.1 | py36h7b6447c_0 377 KB
dask-core-2.9.1 | py_0 556 KB
defusedxml-0.6.0 | py_0 23 KB
entrypoints-0.3 | py36_0 12 KB
gast-0.3.2 | py_0 13 KB
giflib-5.1.4 | h14c3975_1 68 KB
idna-2.8 | py36_0 112 KB
imageio-2.6.1 | py36_0 3.2 MB
importlib_metadata-1.3.0 | py36_0 46 KB
ipykernel-5.1.3 | py36h39e3cac_1 173 KB
ipython-7.11.1 | py36h39e3cac_0 988 KB
ipython_genutils-0.2.0 | py36_0 39 KB
jasper-1.900.1 | hd497a04_4 198 KB
jedi-0.15.2 | py36_0 738 KB
joblib-0.14.1 | py_0 201 KB
jsonschema-3.2.0 | py36_0 95 KB
jupyter_client-5.3.4 | py36_0 141 KB
jupyter_console-6.0.0 | py36_0 37 KB
jupyter_core-4.6.1 | py36_0 70 KB
kiwisolver-1.1.0 | py36he6710b0_0 82 KB
lame-3.100 | h7b6447c_0 323 KB
libiconv-1.15 | h63c8f33_5 721 KB
libprotobuf-3.11.2 | hd408876_0 2.9 MB
libsodium-1.0.16 | h1bed415_0 214 KB
libwebp-1.0.1 | h8e7db2f_0 471 KB
markupsafe-1.1.1 | py36h7b6447c_0 29 KB
matplotlib-3.1.1 | py36h5429711_0 5.0 MB
mistune-0.8.4 | py36h7b6447c_0 55 KB
more-itertools-8.0.2 | py_0 39 KB
nbconvert-5.6.1 | py36_0 460 KB
nbformat-4.4.0 | py36_0 128 KB
networkx-2.4 | py_0 1.2 MB
notebook-6.0.2 | py36_0 4.0 MB
numpy-1.17.4 | py36hc1035e2_0 5 KB
numpy-base-1.17.4 | py36hde5b4d6_0 4.2 MB
olefile-0.46 | py36_0 48 KB
openh264-1.8.0 | hd408876_0 659 KB
packaging-20.0 | py_0 35 KB
pandoc-2.2.3.2 | 0 14.0 MB
pandocfilters-1.4.2 | py36_1 13 KB
parso-0.5.2 | py_0 69 KB
pexpect-4.7.0 | py36_0 80 KB
pickleshare-0.7.5 | py36_0 13 KB
pillow-7.0.0 | py36hb39fc2d_0 600 KB
pluggy-0.13.1 | py36_0 33 KB
prometheus_client-0.7.1 | py_0 42 KB
prompt_toolkit-2.0.10 | py_0 227 KB
protobuf-3.11.2 | py36he6710b0_0 635 KB
psutil-5.6.7 | py36h7b6447c_0 318 KB
ptyprocess-0.6.0 | py36_0 23 KB
py-1.8.1 | py_0 71 KB
pygments-2.5.2 | py_0 672 KB
pyopenssl-19.1.0 | py36_0 87 KB
pyparsing-2.4.6 | py_0 64 KB
pyrsistent-0.15.6 | py36h7b6447c_0 93 KB
pysocks-1.7.1 | py36_0 30 KB
pytest-5.3.2 | py36_0 365 KB
pytest-runner-5.2 | py_0 13 KB
python-utils-2.3.0 | py36_0 18 KB
pywavelets-1.1.1 | py36h7b6447c_0 3.5 MB
pyyaml-5.2 | py36h7b6447c_0 180 KB
pyzmq-18.1.0 | py36he6710b0_0 453 KB
qtconsole-4.6.0 | py_1 97 KB
requests-2.22.0 | py36_1 92 KB
scipy-1.3.2 | py36h7c811a0_0 14.0 MB
send2trash-1.5.0 | py36_0 16 KB
setuptools-44.0.0 | py36_0 510 KB
terminado-0.8.3 | py36_0 26 KB
toolz-0.10.0 | py_0 50 KB
tornado-6.0.3 | py36h7b6447c_0 583 KB
tqdm-4.41.1 | py_0 54 KB
traitlets-4.3.3 | py36_0 140 KB
urllib3-1.25.7 | py36_0 169 KB
wcwidth-0.1.7 | py36_0 24 KB
webencodings-0.5.1 | py36_1 19 KB
werkzeug-0.16.0 | py_0 255 KB
widgetsnbextension-3.5.1 | py36_0 862 KB
x264-1!152.20180806 | h7b6447c_0 631 KB
zeromq-4.3.1 | he6710b0_3 496 KB
------------------------------------------------------------
Total: 233.8 MB
The following packages will be REMOVED:
atomicwrites-1.3.0-py_0
gettext-0.19.8.1-hc5be6a0_1002
imageio-ffmpeg-0.3.0-py_0
lz4-c-1.8.3-he1b5a44_1001
pthread-stubs-0.4-h14c3975_1001
xorg-libxau-1.0.9-h14c3975_0
xorg-libxdmcp-1.1.3-h516909a_0
The following packages will be UPDATED:
absl-py 0.7.1-py36_0 --> 0.8.1-py36_0
asn1crypto anaconda::asn1crypto-1.0.1-py36_0 --> pkgs/main::asn1crypto-1.3.0-py36_0
astor 0.7.1-py36_0 --> 0.8.0-py36_0
attrs anaconda/linux-64::attrs-19.1.0-py36_1 --> pkgs/main/noarch::attrs-19.3.0-py_0
c-ares 1.15.0-h7b6447c_1 --> 1.15.0-h7b6447c_1001
cffi anaconda::cffi-1.12.3-py36h2e261b9_0 --> pkgs/main::cffi-1.13.2-py36h2e261b9_0
cloudpickle anaconda::cloudpickle-1.2.1-py_0 --> pkgs/main::cloudpickle-1.2.2-py_0
cryptography anaconda::cryptography-2.7-py36h1ba5d~ --> pkgs/main::cryptography-2.8-py36h1ba5d50_0
cudnn 7.6.0-cuda10.0_0 --> 7.6.5-cuda10.0_0
cytoolz anaconda::cytoolz-0.10.0-py36h7b6447c~ --> pkgs/main::cytoolz-0.10.1-py36h7b6447c_0
dask-core anaconda::dask-core-2.1.0-py_0 --> pkgs/main::dask-core-2.9.1-py_0
dbus conda-forge::dbus-1.13.6-he372182_0 --> pkgs/main::dbus-1.13.12-h746ee38_0
decorator anaconda/linux-64::decorator-4.4.0-py~ --> pkgs/main/noarch::decorator-4.4.1-py_0
expat conda-forge::expat-2.2.5-he1b5a44_1003 --> pkgs/main::expat-2.2.6-he6710b0_0
gast pkgs/main/linux-64::gast-0.2.2-py36_0 --> pkgs/main/noarch::gast-0.3.2-py_0
glib conda-forge::glib-2.58.3-h6f030ca_1002 --> pkgs/main::glib-2.63.1-h5a9c865_0
imageio anaconda::imageio-2.5.0-py36_0 --> pkgs/main::imageio-2.6.1-py36_0
importlib_metadata conda-forge::importlib_metadata-0.23-~ --> pkgs/main::importlib_metadata-1.3.0-py36_0
ipykernel anaconda::ipykernel-5.1.1-py36h39e3ca~ --> pkgs/main::ipykernel-5.1.3-py36h39e3cac_1
ipython anaconda::ipython-7.6.1-py36h39e3cac_0 --> pkgs/main::ipython-7.11.1-py36h39e3cac_0
ipywidgets anaconda::ipywidgets-7.5.0-py_0 --> pkgs/main::ipywidgets-7.5.1-py_0
jedi anaconda::jedi-0.13.3-py36_0 --> pkgs/main::jedi-0.15.2-py36_0
jinja2 anaconda/linux-64::jinja2-2.10.1-py36~ --> pkgs/main/noarch::jinja2-2.10.3-py_0
joblib anaconda::joblib-0.14.0-py_0 --> pkgs/main::joblib-0.14.1-py_0
jsonschema anaconda::jsonschema-3.0.1-py36_0 --> pkgs/main::jsonschema-3.2.0-py36_0
jupyter_client anaconda/noarch::jupyter_client-5.3.1~ --> pkgs/main/linux-64::jupyter_client-5.3.4-py36_0
jupyter_core anaconda/noarch::jupyter_core-4.5.0-p~ --> pkgs/main/linux-64::jupyter_core-4.6.1-py36_0
libprotobuf 3.8.0-hd408876_0 --> 3.11.2-hd408876_0
libtiff conda-forge::libtiff-4.0.10-h57b8799_~ --> pkgs/main::libtiff-4.1.0-h2733197_0
matplotlib anaconda::matplotlib-3.1.0-py36h54297~ --> pkgs/main::matplotlib-3.1.1-py36h5429711_0
mkl_fft 1.0.12-py36ha843d7b_0 --> 1.0.15-py36ha843d7b_0
mkl_random 1.0.2-py36hd81dba3_0 --> 1.1.0-py36hd6b4f25_0
more-itertools conda-forge::more-itertools-7.2.0-py_0 --> pkgs/main::more-itertools-8.0.2-py_0
nbconvert anaconda/noarch::nbconvert-5.5.0-py_0 --> pkgs/main/linux-64::nbconvert-5.6.1-py36_0
networkx anaconda::networkx-2.3-py_0 --> pkgs/main::networkx-2.4-py_0
notebook anaconda::notebook-6.0.0-py36_0 --> pkgs/main::notebook-6.0.2-py36_0
numpy 1.16.4-py36h7e9f1db_0 --> 1.17.4-py36hc1035e2_0
numpy-base 1.16.4-py36hde5b4d6_0 --> 1.17.4-py36hde5b4d6_0
openssl conda-forge::openssl-1.1.1d-h516909a_0 --> pkgs/main::openssl-1.1.1d-h7b6447c_3
packaging pkgs/main/linux-64::packaging-19.0-py~ --> pkgs/main/noarch::packaging-20.0-py_0
parso anaconda::parso-0.5.0-py_0 --> pkgs/main::parso-0.5.2-py_0
pcre conda-forge::pcre-8.41-hf484d3e_1003 --> pkgs/main::pcre-8.43-he6710b0_0
pillow anaconda::pillow-6.1.0-py36h34e0f95_0 --> pkgs/main::pillow-7.0.0-py36hb39fc2d_0
pip 19.1.1-py36_0 --> 19.3.1-py36_0
pluggy conda-forge/noarch::pluggy-0.12.0-py_0 --> pkgs/main/linux-64::pluggy-0.13.1-py36_0
prompt_toolkit anaconda/linux-64::prompt_toolkit-2.0~ --> pkgs/main/noarch::prompt_toolkit-2.0.10-py_0
protobuf 3.8.0-py36he6710b0_0 --> 3.11.2-py36he6710b0_0
psutil anaconda::psutil-5.6.3-py36h7b6447c_0 --> pkgs/main::psutil-5.6.7-py36h7b6447c_0
py conda-forge::py-1.8.0-py_0 --> pkgs/main::py-1.8.1-py_0
pygments anaconda::pygments-2.4.2-py_0 --> pkgs/main::pygments-2.5.2-py_0
pyopenssl anaconda::pyopenssl-19.0.0-py36_0 --> pkgs/main::pyopenssl-19.1.0-py36_0
pyparsing anaconda::pyparsing-2.4.0-py_0 --> pkgs/main::pyparsing-2.4.6-py_0
pyqt anaconda::pyqt-5.9.2-py36h22d08a2_1 --> pkgs/main::pyqt-5.9.2-py36h05f1152_2
pyrsistent anaconda::pyrsistent-0.14.11-py36h7b6~ --> pkgs/main::pyrsistent-0.15.6-py36h7b6447c_0
pytest conda-forge::pytest-5.2.2-py36_0 --> pkgs/main::pytest-5.3.2-py36_0
python-dateutil anaconda/linux-64::python-dateutil-2.~ --> pkgs/main/noarch::python-dateutil-2.8.1-py_0
pytz anaconda::pytz-2019.1-py_0 --> pkgs/main::pytz-2019.3-py_0
pywavelets anaconda::pywavelets-1.0.3-py36hdd077~ --> pkgs/main::pywavelets-1.1.1-py36h7b6447c_0
pyyaml 5.1.1-py36h7b6447c_0 --> 5.2-py36h7b6447c_0
pyzmq anaconda::pyzmq-18.0.0-py36he6710b0_0 --> pkgs/main::pyzmq-18.1.0-py36he6710b0_0
qtconsole anaconda::qtconsole-4.5.2-py_0 --> pkgs/main::qtconsole-4.6.0-py_1
requests anaconda::requests-2.22.0-py36_0 --> pkgs/main::requests-2.22.0-py36_1
scipy 1.2.1-py36h7c811a0_0 --> 1.3.2-py36h7c811a0_0
setuptools 41.0.1-py36_0 --> 44.0.0-py36_0
six 1.12.0-py36_0 --> 1.13.0-py36_0
sqlite 3.28.0-h7b6447c_0 --> 3.30.1-h7b6447c_0
terminado anaconda::terminado-0.8.2-py36_0 --> pkgs/main::terminado-0.8.3-py36_0
testpath anaconda/linux-64::testpath-0.4.2-py3~ --> pkgs/main/noarch::testpath-0.4.4-py_0
traitlets anaconda::traitlets-4.3.2-py36_0 --> pkgs/main::traitlets-4.3.3-py36_0
urllib3 anaconda::urllib3-1.24.2-py36_0 --> pkgs/main::urllib3-1.25.7-py36_0
werkzeug 0.15.4-py_0 --> 0.16.0-py_0
wheel 0.33.4-py36_0 --> 0.33.6-py36_0
widgetsnbextension anaconda::widgetsnbextension-3.5.0-py~ --> pkgs/main::widgetsnbextension-3.5.1-py36_0
The following packages will be SUPERSEDED by a higher-priority channel:
anaconda-client anaconda --> pkgs/main
backcall anaconda --> pkgs/main
backports anaconda --> pkgs/main
bleach anaconda --> pkgs/main
bzip2 conda-forge::bzip2-1.0.8-h516909a_0 --> pkgs/main::bzip2-1.0.8-h7b6447c_0
ca-certificates conda-forge::ca-certificates-2019.11.~ --> pkgs/main::ca-certificates-2019.11.27-0
certifi conda-forge --> pkgs/main
chardet anaconda --> pkgs/main
clyent anaconda --> pkgs/main
cycler anaconda --> pkgs/main
defusedxml anaconda --> pkgs/main
entrypoints anaconda --> pkgs/main
freetype conda-forge::freetype-2.10.0-he983fc9~ --> pkgs/main::freetype-2.9.1-h8a8886c_1
giflib conda-forge::giflib-5.1.9-h516909a_0 --> pkgs/main::giflib-5.1.4-h14c3975_1
gmp conda-forge::gmp-6.1.2-hf484d3e_1000 --> pkgs/main::gmp-6.1.2-h6c8ec71_1
graphite2 conda-forge::graphite2-1.3.13-hf484d3~ --> pkgs/main::graphite2-1.3.13-h23475e2_0
gst-plugins-base conda-forge::gst-plugins-base-1.14.5-~ --> pkgs/main::gst-plugins-base-1.14.0-hbbd80ab_1
gstreamer conda-forge::gstreamer-1.14.5-h36ae1b~ --> pkgs/main::gstreamer-1.14.0-hb453b48_1
icu conda-forge::icu-58.2-hf484d3e_1000 --> pkgs/main::icu-58.2-h9c2bf20_1
idna anaconda --> pkgs/main
ipython_genutils anaconda --> pkgs/main
jasper conda-forge::jasper-1.900.1-h07fcdf6_~ --> pkgs/main::jasper-1.900.1-hd497a04_4
jupyter_console anaconda --> pkgs/main
kiwisolver anaconda --> pkgs/main
lame conda-forge::lame-3.100-h14c3975_1001 --> pkgs/main::lame-3.100-h7b6447c_0
libiconv conda-forge::libiconv-1.15-h516909a_1~ --> pkgs/main::libiconv-1.15-h63c8f33_5
libpng conda-forge::libpng-1.6.37-hed695b0_0 --> pkgs/main::libpng-1.6.37-hbc83047_0
libsodium anaconda --> pkgs/main
libwebp conda-forge::libwebp-1.0.2-h576950b_1 --> pkgs/main::libwebp-1.0.1-h8e7db2f_0
libxcb conda-forge::libxcb-1.13-h14c3975_1002 --> pkgs/main::libxcb-1.13-h1bed415_1
libxml2 conda-forge::libxml2-2.9.9-h13577e0_1 --> pkgs/main::libxml2-2.9.9-hea5a465_1
markupsafe anaconda --> pkgs/main
mistune anaconda --> pkgs/main
mkl-service anaconda --> pkgs/main
nbformat anaconda --> pkgs/main
olefile anaconda --> pkgs/main
openh264 conda-forge::openh264-1.8.0-hdbcaa40_~ --> pkgs/main::openh264-1.8.0-hd408876_0
pandoc anaconda --> pkgs/main
pandocfilters anaconda --> pkgs/main
pexpect anaconda --> pkgs/main
pickleshare anaconda --> pkgs/main
pixman conda-forge::pixman-0.38.0-h516909a_1~ --> pkgs/main::pixman-0.38.0-h7b6447c_0
prometheus_client anaconda --> pkgs/main
ptyprocess anaconda --> pkgs/main
pycparser anaconda --> pkgs/main
pysocks anaconda --> pkgs/main
pytest-runner conda-forge --> pkgs/main
python-utils conda-forge/noarch::python-utils-2.3.~ --> pkgs/main/linux-64::python-utils-2.3.0-py36_0
qt conda-forge::qt-5.9.7-h52cfd70_2 --> pkgs/main::qt-5.9.7-h5867ecd_1
qtpy anaconda --> pkgs/main
send2trash anaconda --> pkgs/main
sip anaconda::sip-4.19.13-py36he6710b0_0 --> pkgs/main::sip-4.19.8-py36hf484d3e_0
toolz anaconda --> pkgs/main
tornado anaconda --> pkgs/main
tqdm conda-forge --> pkgs/main
wcwidth anaconda --> pkgs/main
webencodings anaconda --> pkgs/main
x264 conda-forge::x264-1!152.20180806-h14c~ --> pkgs/main::x264-1!152.20180806-h7b6447c_0
zeromq anaconda --> pkgs/main
zipp conda-forge --> pkgs/main
zstd conda-forge::zstd-1.4.0-h3b9ef0a_0 --> pkgs/main::zstd-1.3.7-h0b5b093_0
Proceed ([y]/n)? n
Why is this? How can I uninstall this version of imageio-ffmpeg without putting my entire environment at risk??
As background, I started trying to install/uninstall this package because I'm trying to use the moviepy package. When I tried to import this package, I got the following error:
RuntimeError: imageio.ffmpeg.download() has been deprecated. Use 'pip install imageio-ffmpeg' instead.'
Instead of following it verbatim, I installed imageio-ffmpeg using conda,, but still received the same error. When I Googled my error, I found a suggested solution of
sudo pip3 install imageio==2.4.1
So, now I want to ensure my conda install doesn't supercede my planned pip3 install.
Asymmetry
In the installation part, Conda runs with an implicit --freeze-installed flag, making it a simple install if all the packages are already there.
In the uninstallation, Conda doesn't have an equivalent simple uninstall. Instead, it will attempt to remove the requested package, plus any of its dependencies that were not explicitly installed or required by other packages. Unfortunately, it appears to accomplish this by trying to solve for an environment that consists of only previously requested packages for the env, and this means that all packages that have superseding versions are subject to being updated.
Your particular case appears to be exacerbated by the fact that you have installed from different channels (e.g., conda-forge), but never explicitly defined those channel priorities in your Conda configuration (globally or in the env). So, most of the changes involve switching back to the defaults channel version of packages.
Alternatives
If you're confident that nothing else has changed, then you could use the --force-remove flag.
Another option, if this was the latest thing you've installed, is to try a revision roll-back, but this may also result in drastic changes. That is, check your revision history:
conda list --revisions
And then attempt installing the penultimate one. I'd definitely dry-run it first:
conda install --revision <your_rev> --dry-run
Pip
Don't use Pip unless you absolutely must, and definitely not in your base env. Generally, using Pip with Conda leads to instability (see Using Pip in a Conda Environment). Also, if you do have to use it, don't use sudo or pip3. Instead, activate the env and use simply pip install.
General (Opinionated) Recommendation
Personally, I've found the most stable way to work with Conda is to treat all envs as immutable. That is, avoid using conda (install|update|remove|uninstall) commands. The only exception to this is when first specifying a new project.
Instead of ad hoc installation and removal of packages, write a YAML file and use that to create a new env (conda env create -f my_env.yml) whenever you wish to edit your environment. If you need to add a new package to the env, edit the YAML and recreate the env from scratch.
Update
Thanks for the hint by #cel below, the command to use is
>sudo conda install sympy=1.4
## Package Plan ##
environment location: /opt/anaconda
added / updated specs:
- sympy=1.4
The following packages will be downloaded:
package | build
---------------------------|-----------------
sympy-1.4 | py37_0 9.7 MB
------------------------------------------------------------
Total: 9.7 MB
The following packages will be REMOVED:
anaconda-2019.03-py37_0
The following packages will be UPDATED:
sympy 1.3-py37_0 --> 1.4-py37_0
Proceed ([y]/n)? y
Verified OK after installation:
>python
Python 3.7.3 (default, Mar 27 2019, 22:11:17)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sympy
>>> sympy.__version__
'1.4'
>>>
I have no idea why other commands did not work. But the above works.
Original question
I am using Linux Manjaro 64 bit. Installed latest Anancoda
>which python
/opt/anaconda/bin/python
>conda list anaconda
# packages in environment at /opt/anaconda:
#
# Name Version Build Channel
anaconda 2019.03 py37_0
anaconda-client 1.7.2 py37_0
anaconda-navigator 1.9.7 py37_0
anaconda-project 0.8.2 py37_0
The problem is that it comes with sympy 1.3, while latest sympy is 1.4 accoding to
https://github.com/sympy/sympy/releases
sympy 1.4 has been out 3 weeks ago.
Now doing
>sudo conda update sympy
Does not update. It says
## Package Plan ##
environment location: /opt/anaconda
added / updated specs:
- sympy
The following packages will be downloaded:
package | build
---------------------------|-----------------
ca-certificates-2019.1.23 | 0 126 KB
certifi-2019.3.9 | py37_0 155 KB
conda-4.6.14 | py37_0 2.1 MB
openssl-1.1.1b | h7b6447c_1 4.0 MB
sympy-1.3 | py37_0 9.5 MB
------------------------------------------------------------
Total: 15.9 MB
But according to https://anaconda.org/anaconda/sympy it says sympy 1.4 is available
I also tried the command above, and it does not update sympy
>sudo conda install -c anaconda sympy
## Package Plan ##
environment location: /opt/anaconda
added / updated specs:
- sympy
The following packages will be downloaded:
package | build
---------------------------|-----------------
ca-certificates-2019.1.23 | 0 126 KB anaconda
certifi-2019.3.9 | py37_0 155 KB anaconda
conda-4.6.14 | py37_0 2.1 MB anaconda
openssl-1.1.1b | h7b6447c_1 4.0 MB anaconda
sympy-1.3 | py37_0 9.5 MB anaconda
------------------------------------------------------------
Total: 15.9 MB
Any one knows why sympy is not being updated? Anything else to try? I could download the tar file from sympy 1.4, but I do not know what to do after that in order to install it in Anancoda.
When conda update does not want to update a package, you can ask conda explicity to install a specific version: conda install sympy=1.4.
NOTE:
I'm duplicating this post because the question has been up on the conda github page for ~6-days with no response. The original link is here:
https://github.com/conda/conda/issues/7938
Current Behavior
When I type try to run conda update -n base conda, conda hung for around 20-minutes on 'Solving environment' and then returned a package plan that did not include an updated version of conda. The package plan that was returned is provided below.
The package plan that was returned is as follows:
environment location: C:\Users\jmatt\Anaconda3
added / updated specs:
- conda
The following packages will be downloaded:
package | build
---------------------------|-----------------
libarchive-3.3.2 | h1d0d21d_1 4.0 MB
lz4-c-1.8.2 | vc14_0 254 KB conda-forge
libcurl-7.61.1 | h7602738_0 249 KB
------------------------------------------------------------
Total: 4.5 MB
The following packages will be UPDATED:
jpeg: 9b-hb83a4c4_2 --> 9b-vc14_2 conda-forge [vc14]
libcurl: 7.61.1-h2a8f88b_0 --> 7.61.1-h7602738_0
libsodium: 1.0.16-h9d3ae62_0 --> 1.0.16-vc14_0 conda-forge [vc14]
libxslt: 1.1.32-hf6f1972_0 --> 1.1.32-vc14_0 conda-forge [vc14]
lz4-c: 1.8.1.2-h2fa13f4_0 --> 1.8.2-vc14_0 conda-forge [vc14]
tk: 8.6.8-hfa6e2cd_0 --> 8.6.8-vc14_0 conda-forge [vc14]
zeromq: 4.2.5-he025d50_1 --> 4.2.5-vc14_2 conda-forge [vc14]
The following packages will be DOWNGRADED:
astropy: 3.0.5-py36he774522_0 --> 3.0.4-py36hfa6e2cd_0
bzip2: 1.0.6-hfa6e2cd_5 --> 1.0.6-vc14_1 conda-forge [vc14]
curl: 7.61.1-h2a8f88b_0 --> 7.60.0-vc14_0 conda-forge [vc14]
cython: 0.29-py36ha925a31_0 --> 0.28.5-py36h6538335_0
freetype: 2.9.1-ha9979f8_1 --> 2.8.1-vc14_0 conda-forge [vc14]
gevent: 1.3.7-py36he774522_1 --> 1.3.6-py36hfa6e2cd_0
hdf5: 1.10.2-hac2f561_1 --> 1.10.2-vc14_0 conda-forge [vc14]
icu: 58.2-ha66f8fd_1 --> 58.2-vc14_0 conda-forge [vc14]
krb5: 1.16.1-h038dc86_6 --> 1.14.6-vc14_0 conda-forge [vc14]
libarchive: 3.3.3-h798a506_0 --> 3.3.2-h1d0d21d_1
libiconv: 1.15-h1df5818_7 --> 1.14-vc14_4 conda-forge [vc14]
libpng: 1.6.35-h2a8f88b_0 --> 1.6.34-vc14_0 conda-forge [vc14]
libtiff: 4.0.9-h36446d0_2 --> 4.0.9-vc14_0 conda-forge [vc14]
libxml2: 2.9.8-hadb2253_1 --> 2.9.5-vc14_1 conda-forge [vc14]
llvmlite: 0.25.0-py36_0 --> 0.24.0-py36h6538335_0
lxml: 4.2.5-py36hef2cd61_0 --> 4.1.1-py36he0adb16_0
lzo: 2.10-h6df0209_2 --> 2.10-vc14_0 conda-forge [vc14]
matplotlib: 3.0.0-py36hd159220_0 --> 2.2.2-py36h153e9ff_0
mistune: 0.8.4-py36he774522_0 --> 0.8.3-py36hfa6e2cd_1
numba: 0.40.0-py36hf9181ef_0 --> 0.39.0-py36h830ac7b_0
pillow: 5.3.0-py36hdc69c19_0 --> 5.1.0-py36h0738816_0
pyqt: 5.9.2-py36h6538335_2 --> 5.6.0-py36_2
pywavelets: 1.0.1-py36h8c2d366_0 --> 1.0.0-py36h452e1ab_0
qt: 5.9.6-vc14h1e9a669_2 --> 5.6.2-vc14_1 conda-forge [vc14]
snappy: 1.1.7-h777316e_3 --> 1.1.7-vc14_1 conda-forge [vc14]
sqlalchemy: 1.2.12-py36he774522_0 --> 1.2.11-py36hfa6e2cd_0
sqlite: 3.25.2-hfa6e2cd_0 --> 3.22.0-vc14_0 conda-forge [vc14]
twisted: 18.9.0-py36he774522_0 --> 18.7.0-py36hfa6e2cd_1
vc: 14.1-h0510ff6_4 --> 14-h0510ff6_3
yaml: 0.1.7-hc54c509_2 --> 0.1.7-vc14_0 conda-forge [vc14]
zlib: 1.2.11-h8395fce_2 --> 1.2.11-vc14_0 conda-forge [vc14]
Proceed ([y]/n)? n
NOTE: the conda version I have installed is 4.5.11 - I'm not sure if this is the most recent version and haven't been able to find a command or resource (other than conda update conda) to check what the most recent version is. I also had a similar problem when trying to conda install websocket-client when websocket client was already installed - I wonder if the current version of conda has trouble when the most recent version of a package is already installed.
NOTE 2: conda update --all solved the environment in a reasonable amount of time (~1 min - I didn't time it precisely).
Steps to Reproduce
conda update -n base conda
As I mentioned above, conda install websocket-client also hung at 'Solving environment' - I already had websocket-client version 0.53.0 installed when I tried to run the install command
Expected Behavior
Conda should either:
1. If the most recent version is installed, conda should promptly inform the user that an update isn't needed.
1. If a newer version is available, I'd expect Conda to solve the environment in a shorter period of time. I think that less than 1-2 minutes would be reasonable - 20+ minutes is too long
Environment Information
The output of: conda info
active environment : base
active env location : C:\Users\jmatt\Anaconda3
shell level : 1
user config file : C:\Users\jmatt\.condarc
populated config files : C:\Users\jmatt\.condarc
conda version : 4.5.11
conda-build version : 3.16.1
python version : 3.6.6.final.0
base environment : C:\Users\jmatt\Anaconda3 (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/win-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/free/win-64
https://repo.anaconda.com/pkgs/free/noarch
https://repo.anaconda.com/pkgs/r/win-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/pro/win-64
https://repo.anaconda.com/pkgs/pro/noarch
https://repo.anaconda.com/pkgs/msys2/win-64
https://repo.anaconda.com/pkgs/msys2/noarch
https://conda.anaconda.org/conda-forge/win-64
https://conda.anaconda.org/conda-forge/noarch
package cache : C:\Users\jmatt\Anaconda3\pkgs
C:\Users\jmatt\AppData\Local\conda\conda\pkgs
envs directories : C:\Users\jmatt\Anaconda3\envs
C:\Users\jmatt\AppData\Local\conda\conda\envs
C:\Users\jmatt\.conda\envs
platform : win-64
user-agent : conda/4.5.11 requests/2.19.1 CPython/3.6.6 Windows/10 Windows/10.0.17134
administrator : False
netrc file : None
offline mode : False
The output of: conda config --show-sources
ssl_verify: True
channels:
- defaults
- conda-forge
The output of: conda list --show-channel-urls
(base) C:\Users\jmatt>conda list --show-channel-urls
packages in environment at C:\Users\jmatt\Anaconda3:
Name Version Build Channel
_ipyw_jlab_nb_ext_conf 0.1.0 py36_0 defaults
alabaster 0.7.12 py36_0 defaults
anaconda custom py36h363777c_0 defaults
anaconda-client 1.7.2 py36_0 defaults
anaconda-navigator 1.9.2 py36_0 defaults
anaconda-project 0.8.2 py36_0 defaults
appdirs 1.4.3 py36h28b3542_0 defaults
asn1crypto 0.24.0 py36_0 defaults
astroid 2.0.4 py36_0 defaults
astropy 3.0.5 py36he774522_0 defaults
atomicwrites 1.2.1 py36_0 defaults
attrs 18.2.0 py36h28b3542_0 defaults
automat 0.7.0 py36_0 defaults
babel 2.6.0 py36_0 defaults
backcall 0.1.0 py36_0 defaults
backports 1.0 py36_1 defaults
backports.os 0.1.1 py36_0 defaults
backports.shutil_get_terminal_size 1.0.0 py36_2 defaults
beautifulsoup4 4.6.3 py36_0 defaults
bitarray 0.8.3 py36hfa6e2cd_0 defaults
bkcharts 0.2 py36h7e685f7_0 defaults
blas 1.0 mkl defaults
blaze 0.11.3 py36_0 defaults
bleach 3.0.2 py36_0 defaults
blosc 1.14.4 he51fdeb_0 defaults
bokeh 0.13.0 py36_0 defaults
boto 2.49.0 py36_0 defaults
bottleneck 1.2.1 py36h452e1ab_1 defaults
bzip2 1.0.6 hfa6e2cd_5 defaults
ca-certificates 2018.03.07 0 defaults
certifi 2018.10.15 py36_0 defaults
cffi 1.11.5 py36h74b6da3_1 defaults
chardet 3.0.4 py36_1 defaults
click 7.0 py36_0 defaults
cloudpickle 0.6.1 py36_0 defaults
clyent 1.2.2 py36_1 defaults
colorama 0.4.0 py36_0 defaults
comtypes 1.1.7 py36_0 defaults
conda 4.5.11 py36_0 defaults
conda-build 3.16.1 py36_0 defaults
conda-env 2.6.0 1 defaults
conda-verify 3.1.1 py36_0 defaults
console_shortcut 0.1.1 3 defaults
constantly 15.1.0 py36h28b3542_0 defaults
contextlib2 0.5.5 py36he5d52c0_0 defaults
cryptography 2.3.1 py36h74b6da3_0 defaults
curl 7.61.1 h2a8f88b_0 defaults
cycler 0.10.0 py36h009560c_0 defaults
cython 0.29 py36ha925a31_0 defaults
cytoolz 0.9.0.1 py36hfa6e2cd_1 defaults
dask 0.19.4 py36_0 defaults
dask-core 0.19.4 py36_0 defaults
datashape 0.5.4 py36_1 defaults
decorator 4.3.0 py36_0 defaults
defusedxml 0.5.0 py36_1 defaults
distributed 1.23.3 py36_0 defaults
docutils 0.14 py36h6012d8f_0 defaults
entrypoints 0.2.3 py36_2 defaults
et_xmlfile 1.0.1 py36h3d2d736_0 defaults
fastcache 1.0.2 py36hfa6e2cd_2 defaults
filelock 3.0.9 py36_0 defaults
flask 1.0.2 py36_1 defaults
flask-cors 3.0.6 py36_0 defaults
freetype 2.9.1 ha9979f8_1 defaults
future 0.16.0 py36_2 defaults
geographiclib 1.49 py_0 conda-forge
geopy 1.17.0 py_0 conda-forge
get_terminal_size 1.0.0 h38e98db_0 defaults
gevent 1.3.7 py36he774522_1 defaults
glob2 0.6 py36_1 defaults
greenlet 0.4.15 py36hfa6e2cd_0 defaults
h5py 2.8.0 py36h3bdd7fb_2 defaults
hdf5 1.10.2 hac2f561_1 defaults
heapdict 1.0.0 py36_2 defaults
html5lib 1.0.1 py36_0 defaults
hyperlink 18.0.0 py36_0 defaults
icc_rt 2017.0.4 h97af966_0 defaults
icu 58.2 ha66f8fd_1 defaults
idna 2.7 py36_0 defaults
imageio 2.4.1 py36_0 defaults
imagesize 1.1.0 py36_0 defaults
importlib_metadata 0.6 py36_0 defaults
incremental 17.5.0 py36_0 defaults
intel-openmp 2019.0 118 defaults
ipykernel 5.1.0 py36h39e3cac_0 defaults
ipython 7.0.1 py36h39e3cac_0 defaults
ipython_genutils 0.2.0 py36h3c5d0ee_0 defaults
ipywidgets 7.4.2 py36_0 defaults
isort 4.3.4 py36_0 defaults
itsdangerous 1.0.0 py36_0 defaults
jdcal 1.4 py36_0 defaults
jedi 0.13.1 py36_0 defaults
jinja2 2.10 py36_0 defaults
jpeg 9b hb83a4c4_2 defaults
jsonschema 2.6.0 py36h7636477_0 defaults
jupyter 1.0.0 py36_7 defaults
jupyter_client 5.2.3 py36_0 defaults
jupyter_console 6.0.0 py36_0 defaults
jupyter_core 4.4.0 py36_0 defaults
jupyterlab 0.35.2 py36_0 defaults
jupyterlab_launcher 0.13.1 py36_0 defaults
jupyterlab_server 0.2.0 py36_0 defaults
keyring 15.1.0 py36_0 defaults
kiwisolver 1.0.1 py36h6538335_0 defaults
krb5 1.16.1 h038dc86_6 defaults
lazy-object-proxy 1.3.1 py36hfa6e2cd_2 defaults
libarchive 3.3.3 h798a506_0 defaults
libcurl 7.61.1 h2a8f88b_0 defaults
libiconv 1.15 h1df5818_7 defaults
libpng 1.6.35 h2a8f88b_0 defaults
libsodium 1.0.16 h9d3ae62_0 defaults
libssh2 1.8.0 hd619d38_4 defaults
libtiff 4.0.9 h36446d0_2 defaults
libxml2 2.9.8 hadb2253_1 defaults
libxslt 1.1.32 hf6f1972_0 defaults
llvmlite 0.25.0 py36_0 defaults
locket 0.2.0 py36hfed976d_1 defaults
lxml 4.2.5 py36hef2cd61_0 defaults
lz4-c 1.8.1.2 h2fa13f4_0 defaults
lzo 2.10 h6df0209_2 defaults
m2w64-gcc-libgfortran 5.3.0 6 defaults
m2w64-gcc-libs 5.3.0 7 defaults
m2w64-gcc-libs-core 5.3.0 7 defaults
m2w64-gmp 6.1.0 2 defaults
m2w64-libwinpthread-git 5.0.0.4634.697f757 2 defaults
markupsafe 1.0 py36hfa6e2cd_1 defaults
matplotlib 3.0.0 py36hd159220_0 defaults
mccabe 0.6.1 py36_1 defaults
menuinst 1.4.14 py36hfa6e2cd_0 defaults
mistune 0.8.4 py36he774522_0 defaults
mkl 2019.0 118 defaults
mkl-service 1.1.2 py36hb217b18_5 defaults
mkl_fft 1.0.6 py36hdbbee80_0 defaults
mkl_random 1.0.1 py36h77b88f5_1 defaults
more-itertools 4.3.0 py36_0 defaults
mpmath 1.0.0 py36_2 defaults
msgpack-python 0.5.6 py36he980bc4_1 defaults
msys2-conda-epoch 20160418 1 defaults
multipledispatch 0.6.0 py36_0 defaults
navigator-updater 0.2.1 py36_0 defaults
nbconvert 5.3.1 py36_0 defaults
nbformat 4.4.0 py36h3a5bc1b_0 defaults
networkx 2.2 py36_1 defaults
nltk 3.3.0 py36_0 defaults
nose 1.3.7 py36_2 defaults
notebook 5.7.0 py36_0 defaults
numba 0.40.0 py36hf9181ef_0 defaults
numexpr 2.6.8 py36h9ef55f4_0 defaults
numpy 1.15.3 py36ha559c80_0 defaults
numpy-base 1.15.3 py36h8128ebf_0 defaults
numpydoc 0.8.0 py36_0 defaults
odo 0.5.1 py36h7560279_0 defaults
olefile 0.46 py36_0 defaults
openpyxl 2.5.9 py36_0 defaults
openssl 1.0.2p hfa6e2cd_0 defaults
packaging 18.0 py36_0 defaults
pandas 0.23.4 py36h830ac7b_0 defaults
pandoc 2.2.3.2 0 defaults
pandocfilters 1.4.2 py36_1 defaults
parso 0.3.1 py36_0 defaults
partd 0.3.9 py36_0 defaults
path.py 11.5.0 py36_0 defaults
pathlib2 2.3.2 py36_0 defaults
patsy 0.5.0 py36_0 defaults
pep8 1.7.1 py36_0 defaults
pickleshare 0.7.5 py36_0 defaults
pillow 5.3.0 py36hdc69c19_0 defaults
pip 10.0.1 py36_0 defaults
pkginfo 1.4.2 py36_1 defaults
pluggy 0.8.0 py36_0 defaults
ply 3.11 py36_0 defaults
prometheus_client 0.4.2 py36_0 defaults
prompt_toolkit 2.0.6 py36_0 defaults
psutil 5.4.7 py36hfa6e2cd_0 defaults
py 1.7.0 py36_0 defaults
pyasn1 0.4.4 py36h28b3542_0 defaults
pyasn1-modules 0.2.2 py36_0 defaults
pycodestyle 2.4.0 py36_0 defaults
pycosat 0.6.3 py36hfa6e2cd_0 defaults
pycparser 2.19 py36_0 defaults
pycrypto 2.6.1 py36hfa6e2cd_9 defaults
pycurl 7.43.0.2 py36h74b6da3_0 defaults
pyflakes 2.0.0 py36_0 defaults
pygments 2.2.0 py36hb010967_0 defaults
pyhamcrest 1.9.0 py36_2 defaults
pylint 2.1.1 py36_0 defaults
pyodbc 4.0.24 py36h6538335_0 defaults
pyopenssl 18.0.0 py36_0 defaults
pyparsing 2.2.2 py36_0 defaults
pyqt 5.9.2 py36h6538335_2 defaults
pysocks 1.6.8 py36_0 defaults
pytables 3.4.4 py36he6f6034_0 defaults
pytest 3.9.1 py36_0 defaults
pytest-arraydiff 0.2 py36h39e3cac_0 defaults
pytest-astropy 0.4.0 py36_0 defaults
pytest-doctestplus 0.1.3 py36_0 defaults
pytest-openfiles 0.3.0 py36_0 defaults
pytest-remotedata 0.3.0 py36_0 defaults
python 3.6.6 hea74fb7_0 defaults
python-dateutil 2.7.3 py36_0 defaults
python-libarchive-c 2.8 py36_6 defaults
pytz 2018.5 py36_0 defaults
pywavelets 1.0.1 py36h8c2d366_0 defaults
pywin32 223 py36hfa6e2cd_1 defaults
pywinpty 0.5.4 py36_0 defaults
pyyaml 3.13 py36hfa6e2cd_0 defaults
pyzmq 17.1.2 py36hfa6e2cd_0 defaults
qt 5.9.6 vc14h1e9a669_2 defaults
qtawesome 0.5.1 py36_1 defaults
qtconsole 4.4.2 py36_0 defaults
qtpy 1.5.2 py36_0 defaults
requests 2.19.1 py36_0 defaults
rope 0.11.0 py36_0 defaults
ruamel_yaml 0.15.46 py36hfa6e2cd_0 defaults
scikit-image 0.14.0 py36h6538335_1 defaults
scikit-learn 0.20.0 py36heebcf9a_1 defaults
scipy 1.1.0 py36h4f6bf74_1 defaults
seaborn 0.9.0 py36_0 defaults
send2trash 1.5.0 py36_0 defaults
service_identity 17.0.0 py36h28b3542_0 defaults
setuptools 40.4.3 py36_0 defaults
simplegeneric 0.8.1 py36_2 defaults
singledispatch 3.4.0.3 py36h17d0c80_0 defaults
sip 4.19.8 py36h6538335_0 defaults
six 1.11.0 py36_1 defaults
snappy 1.1.7 h777316e_3 defaults
snowballstemmer 1.2.1 py36h763602f_0 defaults
sortedcollections 1.0.1 py36_0 defaults
sortedcontainers 2.0.5 py36_0 defaults
sphinx 1.8.1 py36_0 defaults
sphinxcontrib 1.0 py36_1 defaults
sphinxcontrib-websupport 1.1.0 py36_1 defaults
spyder 3.3.1 py36_1 defaults
spyder-kernels 0.2.6 py36_0 defaults
sqlalchemy 1.2.12 py36he774522_0 defaults
sqlite 3.25.2 hfa6e2cd_0 defaults
statsmodels 0.9.0 py36h452e1ab_0 defaults
sympy 1.3 py36_0 defaults
tblib 1.3.2 py36h30f5020_0 defaults
terminado 0.8.1 py36_1 defaults
testpath 0.4.2 py36_0 defaults
tk 8.6.8 hfa6e2cd_0 defaults
toolz 0.9.0 py36_0 defaults
tornado 5.1.1 py36hfa6e2cd_0 defaults
tqdm 4.26.0 py36h28b3542_0 defaults
traitlets 4.3.2 py36h096827d_0 defaults
twisted 18.9.0 py36he774522_0 defaults
typed-ast 1.1.0 py36hfa6e2cd_0 defaults
typing 3.6.4 py36_0 defaults
unicodecsv 0.14.1 py36h6450c06_0 defaults
urllib3 1.23 py36_0 defaults
vc 14.1 h0510ff6_4 defaults
vs2015_runtime 14.15.26706 h3a45250_0 defaults
wcwidth 0.1.7 py36h3d5aa90_0 defaults
webencodings 0.5.1 py36_1 defaults
websocket-client 0.53.0 py36_1000 conda-forge
werkzeug 0.14.1 py36_0 defaults
wheel 0.32.2 py36_0 defaults
widgetsnbextension 3.4.2 py36_0 defaults
win_inet_pton 1.0.1 py36_1 defaults
win_unicode_console 0.5 py36hcdbd4b5_0 defaults
wincertstore 0.2 py36h7fe50ca_0 defaults
winpty 0.4.3 4 defaults
wrapt 1.10.11 py36hfa6e2cd_2 defaults
xlrd 1.1.0 py36_1 defaults
xlsxwriter 1.1.2 py36_0 defaults
xlwings 0.12.1 py36_0 defaults
xlwt 1.3.0 py36h1a4751e_0 defaults
xz 5.2.4 h2fa13f4_4 defaults
yaml 0.1.7 hc54c509_2 defaults
zeromq 4.2.5 he025d50_1 defaults
zict 0.1.3 py36_0 defaults
zlib 1.2.11 h8395fce_2 defaults
zope 1.0 py36_1 defaults
zope.interface 4.5.0 py36hfa6e2cd_0 defaults
A Common Problem: Suboptimal Channel Prioritization
Anaconda distribution is designed and tested to use the anaconda channel (a subset of defaults) as its primary channel. Adding conda-forge in either a higher- (channel_priority: strict) or equal-priority (channel_priority: flexible) configuration opens up many of the packages to be sourced from Conda Forge instead, and this is where Conda struggles to solve.
Including conda-forge both expands the search as well as opens other packages to be subject to channel switching, and since the anaconda package includes dozens of packages, this can be a huge satisfiability problem to solve. This is often most problematic after the first time conda-forge is added into a user's configuration.
Solutions
There are two high-level ways to improve performance: simplify the solving problem or use a faster solver. Of course, these are not mutually exclusive - feel free to be both thoughtful about what you demand of your solver and adopt optimized tools.
Option 1: Optimize Channel Prioritization
When the anaconda metapackage is installed in an environment, keep the defaults channel at highest priority (first channel in .condarc) and set channel_priority: strict. See the documentation on Managing Channels.
Additionally, one can forcefully prioritize the defaults channel with commands like
conda update -n base --override-channels -c defaults conda
Option 2: Mamba
Mamba is a drop-in replacement for the conda CLI that is faster (compiled) and in my experience tends to be more aggressive in pruning. Once installed, it runs similar to conda, e.g.,
mamba update -n base conda
Note on Alternative Configuration
Many users find the coupling of their environment management infrastructure (Conda) to a large working environment (Anaconda) to be less than ideal. A popular alternative configuration is to maintain a minimal base environment, and if Anaconda is ever needed, to create a new environment with the anaconda package installed.
Alternative options for base environments include
Miniconda - minimal base with defaults channel priority
Miniforge - minimal base with conda-forge channel priority
Mambaforge - Miniforge base + Mamba
Upgrade your conda
For particular conda version
conda install conda=4.7.12
For latest conda version
conda update -n base -c defaults conda