I want to activate conda environment and install some packages in github actions. I tried source activate myenv and activate myenv but this step doesn't activate anything
upload_package(){
conda config --set anaconda_upload yes
apt-get update
apt-get install -y build-essential
conda init bash
conda create -n myenv python=3.6
conda activate myenv
echo $PWD
echo "$VIRTUAL_ENV"
conda install --yes pip
conda install --yes numpy cython
conda install --yes -c conda-forge nose mdtraj
anaconda login --username $INPUT_ANACONDAUSERNAME --password $INPUT_ANACONDAPASSWORD
echo $PWD
echo "$VIRTUAL_ENV"
conda build /github/workspace
anaconda logout
}
i tried to check with echo "$VIRTUAL_ENV" but it either gives empty screen (source activate myenv and activate myenv) or just gives an error below. I don't know how can i handle as I don't know how can I restart and close shell as it is on github actions. I would appreciate your help
2020-03-04T10:50:25.2340743Z + conda init bash
2020-03-04T10:50:25.3835655Z no change /opt/conda/condabin/conda
2020-03-04T10:50:25.3836003Z no change /opt/conda/bin/conda
2020-03-04T10:50:25.3836764Z no change /opt/conda/bin/conda-env
2020-03-04T10:50:25.3836919Z no change /opt/conda/bin/activate
2020-03-04T10:50:25.3837068Z no change /opt/conda/bin/deactivate
2020-03-04T10:50:25.3837223Z no change /opt/conda/etc/profile.d/conda.sh
2020-03-04T10:50:25.3837381Z no change /opt/conda/etc/fish/conf.d/conda.fish
2020-03-04T10:50:25.3837539Z no change /opt/conda/shell/condabin/Conda.psm1
2020-03-04T10:50:25.3837904Z no change /opt/conda/shell/condabin/conda-hook.ps1
2020-03-04T10:50:25.3838273Z no change /opt/conda/lib/python3.7/site-packages/xontrib/conda.xsh
2020-03-04T10:50:25.3838453Z no change /opt/conda/etc/profile.d/conda.csh
2020-03-04T10:50:25.3838607Z modified /github/home/.bashrc
2020-03-04T10:50:25.3838685Z
2020-03-04T10:50:25.3839039Z ==> For changes to take effect, close and re-open your current shell. <==
2020-03-04T10:50:25.3839151Z
2020-03-04T10:50:25.4034555Z + conda create -n myenv python=3.6
2020-03-04T10:50:25.9277471Z Collecting package metadata (current_repodata.json): ...working... done
2020-03-04T10:50:25.9642682Z Solving environment: ...working... done
2020-03-04T10:50:26.0490197Z
2020-03-04T10:50:26.0490460Z ## Package Plan ##
2020-03-04T10:50:26.0490535Z
2020-03-04T10:50:26.0490685Z environment location: /opt/conda/envs/myenv
2020-03-04T10:50:26.0490781Z
2020-03-04T10:50:26.0490926Z added / updated specs:
2020-03-04T10:50:26.0491662Z - python=3.6
2020-03-04T10:50:26.0491756Z
2020-03-04T10:50:26.0491824Z
2020-03-04T10:50:26.0491969Z The following packages will be downloaded:
2020-03-04T10:50:26.0492062Z
2020-03-04T10:50:26.0492460Z package | build
2020-03-04T10:50:26.0492845Z ---------------------------|-----------------
2020-03-04T10:50:26.0493211Z _libgcc_mutex-0.1 | main 3 KB
2020-03-04T10:50:26.0493583Z certifi-2019.11.28 | py36_0 153 KB
2020-03-04T10:50:26.0493961Z ld_impl_linux-64-2.33.1 | h53a641e_7 568 KB
2020-03-04T10:50:26.0494323Z libedit-3.1.20181209 | hc058e9b_0 163 KB
2020-03-04T10:50:26.0494679Z libffi-3.2.1 | hd88cf55_4 40 KB
2020-03-04T10:50:26.0495035Z libgcc-ng-9.1.0 | hdf63c60_0 5.1 MB
2020-03-04T10:50:26.0495392Z libstdcxx-ng-9.1.0 | hdf63c60_0 3.1 MB
2020-03-04T10:50:26.0495732Z ncurses-6.2 | he6710b0_0 1.1 MB
2020-03-04T10:50:26.0496087Z pip-20.0.2 | py36_1 1.7 MB
2020-03-04T10:50:26.0496448Z python-3.6.10 | h0371630_0 29.7 MB
2020-03-04T10:50:26.0496812Z readline-7.0 | h7b6447c_5 324 KB
2020-03-04T10:50:26.0497169Z setuptools-45.2.0 | py36_0 520 KB
2020-03-04T10:50:26.0497519Z sqlite-3.31.1 | h7b6447c_0 1.1 MB
2020-03-04T10:50:26.0497879Z tk-8.6.8 | hbc83047_0 2.8 MB
2020-03-04T10:50:26.0498229Z wheel-0.34.2 | py36_0 51 KB
2020-03-04T10:50:26.0498579Z xz-5.2.4 | h14c3975_4 283 KB
2020-03-04T10:50:26.0498931Z zlib-1.2.11 | h7b6447c_3 103 KB
2020-03-04T10:50:26.0499273Z ------------------------------------------------------------
2020-03-04T10:50:26.0499450Z Total: 46.7 MB
2020-03-04T10:50:26.0499541Z
2020-03-04T10:50:26.0499681Z The following NEW packages will be INSTALLED:
2020-03-04T10:50:26.0499758Z
2020-03-04T10:50:26.0500489Z _libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main
2020-03-04T10:50:26.0501110Z ca-certificates pkgs/main/linux-64::ca-certificates-2020.1.1-0
2020-03-04T10:50:26.0501722Z certifi pkgs/main/linux-64::certifi-2019.11.28-py36_0
2020-03-04T10:50:26.0502345Z ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.33.1-h53a641e_7
2020-03-04T10:50:26.0502969Z libedit pkgs/main/linux-64::libedit-3.1.20181209-hc058e9b_0
2020-03-04T10:50:26.0503577Z libffi pkgs/main/linux-64::libffi-3.2.1-hd88cf55_4
2020-03-04T10:50:26.0504179Z libgcc-ng pkgs/main/linux-64::libgcc-ng-9.1.0-hdf63c60_0
2020-03-04T10:50:26.0504791Z libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-9.1.0-hdf63c60_0
2020-03-04T10:50:26.0505394Z ncurses pkgs/main/linux-64::ncurses-6.2-he6710b0_0
2020-03-04T10:50:26.0506278Z openssl pkgs/main/linux-64::openssl-1.1.1d-h7b6447c_4
2020-03-04T10:50:26.0506910Z pip pkgs/main/linux-64::pip-20.0.2-py36_1
2020-03-04T10:50:26.0507509Z python pkgs/main/linux-64::python-3.6.10-h0371630_0
2020-03-04T10:50:26.0508111Z readline pkgs/main/linux-64::readline-7.0-h7b6447c_5
2020-03-04T10:50:26.0508708Z setuptools pkgs/main/linux-64::setuptools-45.2.0-py36_0
2020-03-04T10:50:26.0509475Z sqlite pkgs/main/linux-64::sqlite-3.31.1-h7b6447c_0
2020-03-04T10:50:26.0510365Z tk pkgs/main/linux-64::tk-8.6.8-hbc83047_0
2020-03-04T10:50:26.0510964Z wheel pkgs/main/linux-64::wheel-0.34.2-py36_0
2020-03-04T10:50:26.0511551Z xz pkgs/main/linux-64::xz-5.2.4-h14c3975_4
2020-03-04T10:50:26.0512141Z zlib pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3
2020-03-04T10:50:26.0512229Z
2020-03-04T10:50:26.0512313Z
2020-03-04T10:50:26.0512447Z Proceed ([y]/n)?
2020-03-04T10:50:26.0519068Z
2020-03-04T10:50:26.0519270Z Downloading and Extracting Packages
2020-03-04T10:50:26.0519494Z
2020-03-04T10:50:26.1318915Z libedit-3.1.20181209 | 163 KB | | 0%
2020-03-04T10:50:26.1319506Z libedit-3.1.20181209 | 163 KB | ########## | 100%
2020-03-04T10:50:26.1319646Z
2020-03-04T10:50:26.3010399Z libgcc-ng-9.1.0 | 5.1 MB | | 0%
2020-03-04T10:50:26.3016426Z libgcc-ng-9.1.0 | 5.1 MB | ########## | 100%
2020-03-04T10:50:26.3016551Z
2020-03-04T10:50:26.4178401Z tk-8.6.8 | 2.8 MB | | 0%
2020-03-04T10:50:26.4189777Z tk-8.6.8 | 2.8 MB | ########## | 100%
2020-03-04T10:50:26.4190390Z
2020-03-04T10:50:26.4512708Z zlib-1.2.11 | 103 KB | | 0%
2020-03-04T10:50:26.4513072Z zlib-1.2.11 | 103 KB | ########## | 100%
2020-03-04T10:50:26.4513153Z
2020-03-04T10:50:26.4688184Z _libgcc_mutex-0.1 | 3 KB | | 0%
2020-03-04T10:50:26.4694660Z _libgcc_mutex-0.1 | 3 KB | ########## | 100%
2020-03-04T10:50:26.4694795Z
2020-03-04T10:50:26.8854186Z ncurses-6.2 | 1.1 MB | | 0%
2020-03-04T10:50:26.8854823Z ncurses-6.2 | 1.1 MB | ########## | 100%
2020-03-04T10:50:26.8854969Z
2020-03-04T10:50:26.9238174Z ld_impl_linux-64-2.3 | 568 KB | | 0%
2020-03-04T10:50:26.9238718Z ld_impl_linux-64-2.3 | 568 KB | ########## | 100%
2020-03-04T10:50:26.9238853Z
2020-03-04T10:50:26.9544406Z libffi-3.2.1 | 40 KB | | 0%
2020-03-04T10:50:26.9545028Z libffi-3.2.1 | 40 KB | ########## | 100%
2020-03-04T10:50:26.9545172Z
2020-03-04T10:50:26.9767359Z certifi-2019.11.28 | 153 KB | | 0%
2020-03-04T10:50:26.9773520Z certifi-2019.11.28 | 153 KB | ########## | 100%
2020-03-04T10:50:26.9773720Z
2020-03-04T10:50:27.1019191Z pip-20.0.2 | 1.7 MB | | 0%
2020-03-04T10:50:27.1019873Z pip-20.0.2 | 1.7 MB | ########## | 100%
2020-03-04T10:50:27.1020024Z
2020-03-04T10:50:27.1284184Z wheel-0.34.2 | 51 KB | | 0%
2020-03-04T10:50:27.1284729Z wheel-0.34.2 | 51 KB | ########## | 100%
2020-03-04T10:50:27.1284982Z
2020-03-04T10:50:27.1635023Z xz-5.2.4 | 283 KB | | 0%
2020-03-04T10:50:27.1635574Z xz-5.2.4 | 283 KB | ########## | 100%
2020-03-04T10:50:27.1635690Z
2020-03-04T10:50:27.1957966Z readline-7.0 | 324 KB | | 0%
2020-03-04T10:50:27.1958452Z readline-7.0 | 324 KB | ########## | 100%
2020-03-04T10:50:27.1958592Z
2020-03-04T10:50:27.2419085Z sqlite-3.31.1 | 1.1 MB | | 0%
2020-03-04T10:50:27.2419569Z sqlite-3.31.1 | 1.1 MB | ########## | 100%
2020-03-04T10:50:27.2419671Z
2020-03-04T10:50:27.3421516Z python-3.6.10 | 29.7 MB | | 0%
2020-03-04T10:50:27.4422601Z python-3.6.10 | 29.7 MB | ##4 | 24%
2020-03-04T10:50:27.9656988Z python-3.6.10 | 29.7 MB | ####### | 71%
2020-03-04T10:50:27.9657727Z python-3.6.10 | 29.7 MB | ########## | 100%
2020-03-04T10:50:27.9657877Z
2020-03-04T10:50:28.0623746Z libstdcxx-ng-9.1.0 | 3.1 MB | | 0%
2020-03-04T10:50:28.0624544Z libstdcxx-ng-9.1.0 | 3.1 MB | ########## | 100%
2020-03-04T10:50:28.0624701Z
2020-03-04T10:50:28.1067800Z setuptools-45.2.0 | 520 KB | | 0%
2020-03-04T10:50:28.1068596Z setuptools-45.2.0 | 520 KB | ########## | 100%
2020-03-04T10:50:28.3879641Z Preparing transaction: ...working... done
2020-03-04T10:50:29.1497815Z Verifying transaction: ...working... done
2020-03-04T10:50:29.7835172Z Executing transaction: ...working... done
2020-03-04T10:50:29.7902932Z #
2020-03-04T10:50:29.7903104Z # To activate this environment, use
2020-03-04T10:50:29.7903242Z #
2020-03-04T10:50:29.7903364Z # $ conda activate myenv
2020-03-04T10:50:29.7903502Z #
2020-03-04T10:50:29.7903644Z # To deactivate an active environment, use
2020-03-04T10:50:29.7903785Z #
2020-03-04T10:50:29.7903919Z # $ conda deactivate
2020-03-04T10:50:29.7904010Z
2020-03-04T10:50:29.9029429Z + conda activate myenv
2020-03-04T10:50:30.0185463Z
2020-03-04T10:50:30.0186577Z CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
2020-03-04T10:50:30.0186790Z To initialize your shell, run
2020-03-04T10:50:30.0186898Z
2020-03-04T10:50:30.0187066Z $ conda init <SHELL_NAME>
2020-03-04T10:50:30.0187175Z
2020-03-04T10:50:30.0187334Z Currently supported shells are:
2020-03-04T10:50:30.0187683Z - bash
2020-03-04T10:50:30.0188017Z - fish
2020-03-04T10:50:30.0188343Z - tcsh
2020-03-04T10:50:30.0188668Z - xonsh
2020-03-04T10:50:30.0188995Z - zsh
2020-03-04T10:50:30.0189331Z - powershell
2020-03-04T10:50:30.0189418Z
2020-03-04T10:50:30.0189810Z See 'conda init --help' for more information and options.
2020-03-04T10:50:30.0190158Z
2020-03-04T10:50:30.0190604Z IMPORTANT: You may need to close and restart your shell after running 'conda init'.
I also faced the same issue while trying to activate the conda env. I small change to the command mentioned by #FlyingTeller, is to use something like
conda create -n <YOUR_ENV_NAME> python=3.6
conda info
$CONDA/bin/activate <YOUR_ENV_NAME> # to activate the env
It is an old question without an answer, but anyone like me who faced this issue recently can try the above solution.
Related
I've been using Anaconda as my package manager but was unaware that it is recommended to install additional packages within virtual environments rather than globally. As a result, I've been installing all packages in my base environment by simply opening a terminal window and running commands such as:
pip3 install requests_html
conda install sklearn
This seemed to work fine for a while until I recently decided to try updating my packages. I tried running
conda update --all
which then provided a list of all the packages that will be updated. However, it also said that some packages will be downgraded instead, which after some googling seems to be due to version conflicts which is why environments are recommended.
Sample output:
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /Users/mahuja/opt/anaconda3
The following packages will be downloaded:
package | build
---------------------------|-----------------
_anaconda_depends-2020.07 | py38_0 6 KB
anaconda-custom | py38_1 35 KB
anaconda-navigator-1.10.0 | py38_0 6.2 MB
anaconda-project-0.9.1 | pyhd3eb1b0_1 215 KB
appdirs-1.4.4 | py_0 13 KB
appnope-0.1.2 |py38hecd8cb5_1001 10 KB
argon2-cffi-20.1.0 | py38haf1e3a3_1 44 KB
asn1crypto-1.4.0 | py_0 80 KB
backports-1.0 | pyhd3eb1b0_2 210 KB
backports.functools_lru_cache-1.6.1| pyhd3eb1b0_0 12 KB
backports.shutil_get_terminal_size-1.0.0| pyhd3eb1b0_3 10 KB
backports.tempfile-1.0 | pyhd3eb1b0_1 11 KB
beautifulsoup4-4.9.3 | pyha847dfd_0 86 KB
bitarray-1.6.3 | py38h9ed2024_1 92 KB
black-19.10b0 | py_0 86 KB
bleach-3.2.2 | pyhd3eb1b0_0 113 KB
blosc-1.20.1 | hab81aa3_0 56 KB
bokeh-2.2.3 | py38_0 5.3 MB
brotli-1.0.9 | hb1e8313_2 398 KB
brotlipy-0.7.0 |py38h9ed2024_1003 331 KB
gevent-21.1.1 | py38h9ed2024_1 1.5 MB
giflib-5.1.4 | h1de35cc_1 59 KB
glib-2.66.1 | h9bbe63b_0 2.7 MB
glob2-0.7 | pyhd3eb1b0_0 12 KB
greenlet-0.4.17 | py38haf1e3a3_0 20 KB
idna-2.10 | pyhd3eb1b0_0 52 KB
imagecodecs-2021.1.11 | py38h2ea9c00_1 5.4 MB
importlib-metadata-2.0.0 | py_1 35 KB
importlib_metadata-2.0.0 | 1 11 KB
iniconfig-1.1.1 | pyhd3eb1b0_0 8 KB
intervaltree-3.1.0 | py_0 26 KB
ipykernel-5.3.4 | py38h5ca1d4c_0 180 KB
ipython-7.19.0 | py38h01d92e1_1 991 KB
ipython_genutils-0.2.0 | pyhd3eb1b0_1 27 KB
ipywidgets-7.6.3 | pyhd3eb1b0_1 105 KB
isort-5.6.4 | py_0 78 KB
jedi-0.17.2 | py38hecd8cb5_1 923 KB
jinja2-2.11.2 | pyhd3eb1b0_0 93 KB
joblib-1.0.0 | pyhd3eb1b0_0 208 KB
jsonschema-3.2.0 | py_2 47 KB
jupyter_client-6.1.7 | py_0 77 KB
jupyter_console-6.2.0 | py_0 26 KB
jupyter_core-4.7.0 | py38hecd8cb5_0 68 KB
jupyterlab-2.2.6 | py_0 3.4 MB
jupyterlab_pygments-0.1.2 | py_0 8 KB
jupyterlab_widgets-1.0.0 | pyhd3eb1b0_1 109 KB
jxrlib-1.1 | haf1e3a3_2 206 KB
matplotlib-base-3.3.2 | py38h181983e_0 5.1 MB
mkl_fft-1.2.0 | py38hc64f4ea_0 143 KB
mock-4.0.3 | pyhd3eb1b0_0 29 KB
more-itertools-8.6.0 | pyhd3eb1b0_0 40 KB
msgpack-python-1.0.1 | py38hf7b0b51_0 75 KB
mypy_extensions-0.4.3 | py38_0 9 KB
nbclient-0.5.1 | py_0 58 KB
nbconvert-6.0.7 | py38_0 483 KB
nbformat-5.1.2 | pyhd3eb1b0_1 68 KB
nest-asyncio-1.4.3 | pyhd3eb1b0_0 11 KB
networkx-2.5 | py_0 1.1 MB
nose-1.3.7 | pyhd3eb1b0_1006 128 KB
notebook-6.2.0 | py38hecd8cb5_0 4.1 MB
numba-0.51.2 | py38h6440ff4_1 3.1 MB
numexpr-2.7.2 | py38h16bde0e_0 120 KB
numpy-1.19.2 | py38h456fd55_0 21 KB
numpy-base-1.19.2 | py38hcfb5961_0 4.0 MB
numpydoc-1.1.0 | pyhd3eb1b0_1 42 KB
openjpeg-2.3.0 | hb95cd4c_1 281 KB
openpyxl-3.0.6 | pyhd3eb1b0_0 159 KB
openssl-1.1.1i | h9ed2024_0 2.2 MB
packaging-20.8 | pyhd3eb1b0_0 36 KB
pandas-1.2.1 | py38hb2f4e1b_0 8.3 MB
pandoc-2.11 | h0dc7051_0 10.8 MB
pandocfilters-1.4.3 | py38hecd8cb5_1 14 KB
path-15.0.1 | py38hecd8cb5_0 37 KB
path.py-12.5.0 | 0 4 KB
pathlib2-2.3.5 | py38hecd8cb5_2 38 KB
pathspec-0.7.0 | py_0 26 KB
pexpect-4.8.0 | pyhd3eb1b0_3 53 KB
pickleshare-0.7.5 | pyhd3eb1b0_1003 13 KB
pillow-8.1.0 | py38h5270095_0 590 KB
pip-20.3.3 | py38hecd8cb5_0 1.8 MB
pkginfo-1.7.0 | py38hecd8cb5_0 43 KB
prometheus_client-0.9.0 | pyhd3eb1b0_0 45 KB
prompt-toolkit-3.0.8 | py_0 248 KB
prompt_toolkit-3.0.8 | 0 12 KB
psutil-5.7.2 | py38haf1e3a3_0 341 KB
ptyprocess-0.7.0 | pyhd3eb1b0_2 17 KB
py-1.10.0 | pyhd3eb1b0_0 76 KB
pycurl-7.43.0.6 | py38hddc9c9b_0 66 KB
pydocstyle-5.1.1 | py_0 38 KB
pyerfa-1.7.1.1 | py38h9ed2024_1 296 KB
pygments-2.7.4 | pyhd3eb1b0_0 676 KB
pylint-2.6.0 | py38_0 438 KB
pyls-black-0.4.6 | hd3eb1b0_0 9 KB
pyls-spyder-0.3.0 | pyhd3eb1b0_0 10 KB
pyopenssl-20.0.1 | pyhd3eb1b0_1 49 KB
pyparsing-2.4.7 | pyhd3eb1b0_0 59 KB
pyrsistent-0.17.3 | py38haf1e3a3_0 89 KB
pytest-6.2.1 | py38hecd8cb5_0 427 KB
python-3.8.5 | h26836e1_1 20.6 MB
python-jsonrpc-server-0.4.0| py_0 12 KB
python-language-server-0.36.2| pyhd3eb1b0_0 41 KB
python-libarchive-c-2.9 | pyhd3eb1b0_0 46 KB
python.app-3 | py38h9ed2024_0 20 KB
pytz-2020.5 | pyhd3eb1b0_0 182 KB
pywavelets-1.1.1 | py38haf1e3a3_2 3.4 MB
pyyaml-5.4.1 | py38h9ed2024_1 165 KB
pyzmq-20.0.0 | py38h23ab428_1 405 KB
qtawesome-1.0.1 | py_0 757 KB
qtconsole-4.7.7 | py_0 96 KB
regex-2020.11.13 | py38h9ed2024_0 310 KB
requests-2.25.1 | pyhd3eb1b0_0 52 KB
ripgrep-12.1.1 | 0 1.3 MB
rope-0.18.0 | py_0 126 KB
scikit-image-0.17.2 | py38h81aa140_0 8.9 MB
scikit-learn-0.23.2 | py38h959d312_0 4.7 MB
scipy-1.5.2 | py38h2515648_0 13.3 MB
seaborn-0.11.1 | pyhd3eb1b0_0 212 KB
send2trash-1.5.0 | pyhd3eb1b0_1 14 KB
setuptools-51.3.3 | py38hecd8cb5_4 727 KB
singledispatch-3.4.0.3 | py_1001 12 KB
six-1.15.0 | py38hecd8cb5_0 27 KB
snowballstemmer-2.1.0 | pyhd3eb1b0_0 62 KB
sortedcollections-2.1.0 | pyhd3eb1b0_0 12 KB
sortedcontainers-2.3.0 | pyhd3eb1b0_0 26 KB
soupsieve-2.1 | pyhd3eb1b0_0 31 KB
sphinx-3.4.3 | pyhd3eb1b0_0 1.1 MB
sphinxcontrib-websupport-1.2.4| py_0 34 KB
spyder-4.2.0 | py38hecd8cb5_0 5.7 MB
spyder-kernels-1.10.1 | py38hecd8cb5_0 101 KB
sqlalchemy-1.3.21 | py38h9ed2024_0 1.5 MB
sqlite-3.33.0 | hffcf06c_0 1.3 MB
statsmodels-0.12.1 | py38h9ed2024_0 8.2 MB
sympy-1.7.1 | py38hecd8cb5_0 8.8 MB
tbb-2020.3 | h879752b_0 147 KB
tblib-1.7.0 | py_0 16 KB
terminado-0.9.2 | py38hecd8cb5_0 25 KB
three-merge-0.1.1 | pyhd3eb1b0_0 10 KB
tifffile-2021.1.14 | pyhd3eb1b0_1 126 KB
toolz-0.11.1 | pyhd3eb1b0_0 46 KB
tornado-6.1 | py38h9ed2024_0 585 KB
tqdm-4.55.1 | pyhd3eb1b0_0 77 KB
traitlets-5.0.5 | py_0 83 KB
typed-ast-1.4.2 | py38h9ed2024_1 183 KB
typing_extensions-3.7.4.3 | py_0 28 KB
ujson-4.0.2 | py38h23ab428_0 46 KB
unixodbc-2.3.9 | haf1e3a3_0 251 KB
urllib3-1.26.2 | pyhd3eb1b0_0 105 KB
watchdog-0.10.4 | py38h9ed2024_0 96 KB
wheel-0.36.2 | pyhd3eb1b0_0 33 KB
wrapt-1.11.2 | py38h9ed2024_0 45 KB
xlrd-2.0.1 | pyhd3eb1b0_0 90 KB
xlsxwriter-1.3.7 | py_0 108 KB
xlwings-0.21.4 | py38hecd8cb5_0 446 KB
zeromq-4.3.3 | hb1e8313_3 414 KB
zfp-0.5.5 | h23ab428_4 147 KB
zipp-3.4.0 | pyhd3eb1b0_0 15 KB
zope.event-4.5.0 | py38_0 209 KB
zope.interface-5.2.0 | py38h9ed2024_0 281 KB
------------------------------------------------------------
Total: 201.5 MB
The following NEW packages will be INSTALLED:
_anaconda_depends pkgs/main/osx-64::_anaconda_depends-2020.07-py38_0
appdirs pkgs/main/noarch::appdirs-1.4.4-py_0
argon2-cffi pkgs/main/osx-64::argon2-cffi-20.1.0-py38haf1e3a3_1
async_generator pkgs/main/noarch::async_generator-1.10-py_0
black pkgs/main/noarch::black-19.10b0-py_0
brotli pkgs/main/osx-64::brotli-1.0.9-hb1e8313_2
brunsli pkgs/main/osx-64::brunsli-0.1-h23ab428_0
charls pkgs/main/osx-64::charls-2.1.0-hb1e8313_2
giflib pkgs/main/osx-64::giflib-5.1.4-h1de35cc_1
imagecodecs pkgs/main/osx-64::imagecodecs-2021.1.11-py38h2ea9c00_1
iniconfig pkgs/main/noarch::iniconfig-1.1.1-pyhd3eb1b0_0
jupyterlab_pygmen~ pkgs/main/noarch::jupyterlab_pygments-0.1.2-py_0
jupyterlab_widgets pkgs/main/noarch::jupyterlab_widgets-1.0.0-pyhd3eb1b0_1
jxrlib pkgs/main/osx-64::jxrlib-1.1-haf1e3a3_2
lerc pkgs/main/osx-64::lerc-2.2.1-h23ab428_0
libaec pkgs/main/osx-64::libaec-1.0.4-hb1e8313_1
libdeflate pkgs/main/osx-64::libdeflate-1.7-h9ed2024_5
libllvm10 pkgs/main/osx-64::libllvm10-10.0.1-h76017ad_5
libwebp pkgs/main/osx-64::libwebp-1.0.1-hd73b212_0
libzopfli pkgs/main/osx-64::libzopfli-1.0.3-hb1e8313_0
mypy_extensions pkgs/main/osx-64::mypy_extensions-0.4.3-py38_0
nbclient pkgs/main/noarch::nbclient-0.5.1-py_0
nest-asyncio pkgs/main/noarch::nest-asyncio-1.4.3-pyhd3eb1b0_0
openjpeg pkgs/main/osx-64::openjpeg-2.3.0-hb95cd4c_1
pathspec pkgs/main/noarch::pathspec-0.7.0-py_0
pyerfa pkgs/main/osx-64::pyerfa-1.7.1.1-py38h9ed2024_1
pyls-black pkgs/main/noarch::pyls-black-0.4.6-hd3eb1b0_0
pyls-spyder pkgs/main/noarch::pyls-spyder-0.3.0-pyhd3eb1b0_0
three-merge pkgs/main/noarch::three-merge-0.1.1-pyhd3eb1b0_0
tifffile pkgs/main/noarch::tifffile-2021.1.14-pyhd3eb1b0_1
typed-ast pkgs/main/osx-64::typed-ast-1.4.2-py38h9ed2024_1
zfp pkgs/main/osx-64::zfp-0.5.5-h23ab428_4
The following packages will be UPDATED:
anaconda-navigator 1.9.12-py38_0 --> 1.10.0-py38_0
anaconda-project 0.8.4-py_0 --> 0.9.1-pyhd3eb1b0_1
appnope 0.1.0-py38_1001 --> 0.1.2-py38hecd8cb5_1001
asn1crypto pkgs/main/osx-64::asn1crypto-1.3.0-py~ --> pkgs/main/noarch::asn1crypto-1.4.0-py_0
astropy 4.0.1.post1-py38h01d97ff_1 --> 4.2-py38h9ed2024_0
attrs 19.3.0-py_0 --> 20.3.0-pyhd3eb1b0_0
autopep8 1.5.3-py_0 --> 1.5.4-py_0
babel 2.8.0-py_0 --> 2.9.0-pyhd3eb1b0_0
backports.shutil_~ pkgs/main/osx-64::backports.shutil_ge~ --> pkgs/main/noarch::backports.shutil_get_terminal_size-1.0.0-pyhd3eb1b0_3
beautifulsoup4 pkgs/main/osx-64::beautifulsoup4-4.9.~ --> pkgs/main/noarch::beautifulsoup4-4.9.3-pyha847dfd_0
bitarray 1.4.0-py38haf1e3a3_0 --> 1.6.3-py38h9ed2024_1
bleach 3.1.5-py_0 --> 3.2.2-pyhd3eb1b0_0
blosc 1.19.0-hab81aa3_0 --> 1.20.1-hab81aa3_0
bokeh 2.1.1-py38_0 --> 2.2.3-py38_0
brotlipy 0.7.0-py38haf1e3a3_1000 --> 0.7.0-py38h9ed2024_1003
ca-certificates 2020.6.24-0 --> 2021.1.19-hecd8cb5_0
certifi 2020.6.20-py38_0 --> 2020.12.5-py38hecd8cb5_0
cffi 1.14.0-py38hc512035_1 --> 1.14.4-py38h2125817_0
chardet 3.0.4-py38_1003 --> 4.0.0-py38hecd8cb5_1003
cloudpickle 1.5.0-py_0 --> 1.6.0-py_0
colorama 0.4.3-py_0 --> 0.4.4-pyhd3eb1b0_0
conda-build 3.20.0-py38_0 --> 3.21.4-py38hecd8cb5_0
conda-package-han~ 1.6.1-py38h1de35cc_0 --> 1.7.2-py38h22f3db7_0
cryptography 2.9.2-py38ha12b0ac_0 --> 3.3.1-py38hbcfaee0_0
cytoolz 0.10.1-py38h1de35cc_0 --> 0.11.0-py38haf1e3a3_0
dask 2.20.0-py_0 --> 2020.12.0-pyhd3eb1b0_0
dask-core 2.20.0-py_0 --> 2020.12.0-pyhd3eb1b0_0
dbus 1.13.16-h18a8e69_0 --> 1.13.18-h18a8e69_0
distributed 2.20.0-py38_0 --> 2021.1.0-py38hecd8cb5_0
expat 2.2.9-hb1e8313_2 --> 2.2.10-hb1e8313_2
filelock 3.0.12-py_0 --> 3.0.12-pyhd3eb1b0_1
flake8 3.8.3-py_0 --> 3.8.4-py_0
freetype 2.10.2-ha233b18_0 --> 2.10.4-ha233b18_0
fsspec 0.7.4-py_0 --> 0.8.3-py_0
gevent 20.6.2-py38haf1e3a3_0 --> 21.1.1-py38h9ed2024_1
glib 2.65.0-hc5f4afa_0 --> 2.66.1-h9bbe63b_0
greenlet 0.4.16-py38haf1e3a3_0 --> 0.4.17-py38haf1e3a3_0
importlib-metadata pkgs/main/osx-64::importlib-metadata-~ --> pkgs/main/noarch::importlib-metadata-2.0.0-py_1
importlib_metadata 1.7.0-0 --> 2.0.0-1
intervaltree 3.0.2-py_1 --> 3.1.0-py_0
ipykernel 5.3.2-py38h5ca1d4c_0 --> 5.3.4-py38h5ca1d4c_0
ipython 7.16.1-py38h5ca1d4c_0 --> 7.19.0-py38h01d92e1_1
ipython_genutils pkgs/main/osx-64::ipython_genutils-0.~ --> pkgs/main/noarch::ipython_genutils-0.2.0-pyhd3eb1b0_1
ipywidgets 7.5.1-py_0 --> 7.6.3-pyhd3eb1b0_1
isort pkgs/main/osx-64::isort-4.3.21-py38_0 --> pkgs/main/noarch::isort-5.6.4-py_0
jedi 0.17.1-py38_0 --> 0.17.2-py38hecd8cb5_1
joblib 0.16.0-py_0 --> 1.0.0-pyhd3eb1b0_0
jsonschema pkgs/main/osx-64::jsonschema-3.2.0-py~ --> pkgs/main/noarch::jsonschema-3.2.0-py_2
jupyter_client 6.1.6-py_0 --> 6.1.7-py_0
jupyter_console 6.1.0-py_0 --> 6.2.0-py_0
jupyter_core 4.6.3-py38_0 --> 4.7.0-py38hecd8cb5_0
jupyterlab 2.1.5-py_0 --> 2.2.6-py_0
keyring 21.2.1-py38_0 --> 21.8.0-py38hecd8cb5_0
kiwisolver 1.2.0-py38h04f5b5a_0 --> 1.3.0-py38h23ab428_0
lazy-object-proxy 1.4.3-py38h1de35cc_0 --> 1.4.3-py38h9ed2024_2
libxml2 2.9.10-h3b9e6c8_1 --> 2.9.10-h7cdb67c_3
llvmlite 0.33.0-py38ha11be7d_1 --> 0.34.0-py38h739e7dc_4
locket 0.2.0-py38_1 --> 0.2.1-py38hecd8cb5_1
lxml 4.5.2-py38h63b7cb6_0 --> 4.6.2-py38h26b266a_0
lz4-c 1.9.2-h0a44026_0 --> 1.9.3-h23ab428_0
matplotlib 3.2.2-0 --> 3.3.2-hecd8cb5_0
matplotlib-base 3.2.2-py38h5670ca0_0 --> 3.3.2-py38h181983e_0
mkl_fft 1.1.0-py38hc64f4ea_0 --> 1.2.0-py38hc64f4ea_0
mock 4.0.2-py_0 --> 4.0.3-pyhd3eb1b0_0
more-itertools 8.4.0-py_0 --> 8.6.0-pyhd3eb1b0_0
msgpack-python 1.0.0-py38h04f5b5a_1 --> 1.0.1-py38hf7b0b51_0
nbconvert 5.6.1-py38_1 --> 6.0.7-py38_0
nbformat 5.0.7-py_0 --> 5.1.2-pyhd3eb1b0_1
networkx 2.4-py_1 --> 2.5-py_0
nose pkgs/main/osx-64::nose-1.3.7-py38_1004 --> pkgs/main/noarch::nose-1.3.7-pyhd3eb1b0_1006
notebook 6.0.3-py38_0 --> 6.2.0-py38hecd8cb5_0
numba 0.50.1-py38h959d312_1 --> 0.51.2-py38h6440ff4_1
numexpr 2.7.1-py38hce01a72_0 --> 2.7.2-py38h16bde0e_0
numpy 1.18.5-py38h1da2735_0 --> 1.19.2-py38h456fd55_0
numpy-base 1.18.5-py38h3304bdc_0 --> 1.19.2-py38hcfb5961_0
numpydoc 1.1.0-py_0 --> 1.1.0-pyhd3eb1b0_1
openpyxl 3.0.4-py_0 --> 3.0.6-pyhd3eb1b0_0
openssl 1.1.1g-h1de35cc_0 --> 1.1.1i-h9ed2024_0
packaging 20.4-py_0 --> 20.8-pyhd3eb1b0_0
pandas 1.0.5-py38h959d312_0 --> 1.2.1-py38hb2f4e1b_0
pandoc 2.10-0 --> 2.11-h0dc7051_0
pandocfilters 1.4.2-py38_1 --> 1.4.3-py38hecd8cb5_1
path 13.1.0-py38_0 --> 15.0.1-py38hecd8cb5_0
path.py 12.4.0-0 --> 12.5.0-0
pathlib2 2.3.5-py38_1 --> 2.3.5-py38hecd8cb5_2
pexpect pkgs/main/osx-64::pexpect-4.8.0-py38_1 --> pkgs/main/noarch::pexpect-4.8.0-pyhd3eb1b0_3
pickleshare pkgs/main/osx-64::pickleshare-0.7.5-p~ --> pkgs/main/noarch::pickleshare-0.7.5-pyhd3eb1b0_1003
pillow 7.2.0-py38ha54b6ba_0 --> 8.1.0-py38h5270095_0
pip 20.1.1-py38_1 --> 20.3.3-py38hecd8cb5_0
pkginfo 1.5.0.1-py38_0 --> 1.7.0-py38hecd8cb5_0
prometheus_client 0.8.0-py_0 --> 0.9.0-pyhd3eb1b0_0
prompt-toolkit 3.0.5-py_0 --> 3.0.8-py_0
prompt_toolkit 3.0.5-0 --> 3.0.8-0
psutil 5.7.0-py38h1de35cc_0 --> 5.7.2-py38haf1e3a3_0
ptyprocess pkgs/main/osx-64::ptyprocess-0.6.0-py~ --> pkgs/main/noarch::ptyprocess-0.7.0-pyhd3eb1b0_2
py 1.9.0-py_0 --> 1.10.0-pyhd3eb1b0_0
pycurl 7.43.0.5-py38ha12b0ac_0 --> 7.43.0.6-py38hddc9c9b_0
pydocstyle 5.0.2-py_0 --> 5.1.1-py_0
pygments 2.6.1-py_0 --> 2.7.4-pyhd3eb1b0_0
pylint 2.5.3-py38_0 --> 2.6.0-py38_0
pyopenssl 19.1.0-py_1 --> 20.0.1-pyhd3eb1b0_1
pyrsistent 0.16.0-py38h1de35cc_0 --> 0.17.3-py38haf1e3a3_0
pytest 5.4.3-py38_0 --> 6.2.1-py38hecd8cb5_0
python 3.8.3-h26836e1_2 --> 3.8.5-h26836e1_1
python-jsonrpc-se~ 0.3.4-py_1 --> 0.4.0-py_0
python-language-s~ pkgs/main/osx-64::python-language-ser~ --> pkgs/main/noarch::python-language-server-0.36.2-pyhd3eb1b0_0
python.app 2-py38_10 --> 3-py38h9ed2024_0
pytz 2020.1-py_0 --> 2020.5-pyhd3eb1b0_0
pywavelets 1.1.1-py38h1de35cc_0 --> 1.1.1-py38haf1e3a3_2
pyyaml 5.3.1-py38haf1e3a3_1 --> 5.4.1-py38h9ed2024_1
pyzmq 19.0.1-py38hb1e8313_1 --> 20.0.0-py38h23ab428_1
qtawesome 0.7.2-py_0 --> 1.0.1-py_0
qtconsole 4.7.5-py_0 --> 0.11.1-pyhd3eb1b0_0
send2trash pkgs/main/osx-64::send2trash-1.5.0-py~ --> pkgs/main/noarch::send2trash-1.5.0-pyhd3eb1b0_1
setuptools 49.2.0-py38_0 --> 51.3.3-py38hecd8cb5_4
singledispatch pkgs/main/osx-64::singledispatch-3.4.~ --> pkgs/main/noarch::singledispatch-3.4.0.3-py_1001
snowballstemmer 2.0.0-py_0 --> 2.1.0-pyhd3eb1b0_0
sortedcollections 1.2.1-py_0 --> 2.1.0-pyhd3eb1b0_0
sortedcontainers 2.2.2-py_0 --> 2.3.0-pyhd3eb1b0_0
soupsieve 2.0.1-py_0 --> 2.1-pyhd3eb1b0_0
sphinx 3.1.2-py_0 --> 3.4.3-pyhd3eb1b0_0
sphinxcontrib-web~ 1.2.3-py_0 --> 1.2.4-py_0
spyder 4.1.4-py38_0 --> 4.2.0-py38hecd8cb5_0
spyder-kernels 1.9.2-py38_0 --> 1.10.1-py38hecd8cb5_0
sqlalchemy 1.3.18-py38haf1e3a3_0 --> 1.3.21-py38h9ed2024_0
sqlite 3.32.3-hffcf06c_0 --> 3.33.0-hffcf06c_0
statsmodels 0.11.1-py38haf1e3a3_0 --> 0.12.1-py38h9ed2024_0
sympy 1.6.1-py38_0 --> 1.7.1-py38hecd8cb5_0
tbb 2020.0-h04f5b5a_0 --> 2020.3-h879752b_0
tblib 1.6.0-py_0 --> 1.7.0-py_0
terminado 0.8.3-py38_0 --> 0.9.2-py38hecd8cb5_0
toolz 0.10.0-py_0 --> 0.11.1-pyhd3eb1b0_0
tornado 6.0.4-py38h1de35cc_1 --> 6.1-py38h9ed2024_0
tqdm 4.47.0-py_0 --> 4.55.1-pyhd3eb1b0_0
traitlets pkgs/main/osx-64::traitlets-4.3.3-py3~ --> pkgs/main/noarch::traitlets-5.0.5-py_0
typing_extensions 3.7.4.2-py_0 --> 3.7.4.3-py_0
ujson 1.35-py38h1de35cc_0 --> 4.0.2-py38h23ab428_0
unixodbc 2.3.7-h1de35cc_0 --> 2.3.9-haf1e3a3_0
urllib3 1.25.9-py_0 --> 1.26.2-pyhd3eb1b0_0
watchdog 0.10.3-py38haf1e3a3_0 --> 0.10.4-py38h9ed2024_0
wheel pkgs/main/osx-64::wheel-0.34.2-py38_0 --> pkgs/main/noarch::wheel-0.36.2-pyhd3eb1b0_0
xlrd 1.2.0-py_0 --> 2.0.1-pyhd3eb1b0_0
xlsxwriter 1.2.9-py_0 --> 1.3.7-py_0
xlwings 0.19.5-py38_0 --> 0.21.4-py38hecd8cb5_0
zeromq 4.3.2-hb1e8313_2 --> 4.3.3-hb1e8313_3
zipp 3.1.0-py_0 --> 3.4.0-pyhd3eb1b0_0
zope.event 4.4-py38_0 --> 4.5.0-py38_0
zope.interface 4.7.1-py38h1de35cc_0 --> 5.2.0-py38h9ed2024_0
The following packages will be SUPERSEDED by a higher-priority channel:
six pkgs/main/noarch::six-1.15.0-py_0 --> pkgs/main/osx-64::six-1.15.0-py38hecd8cb5_0
The following packages will be DOWNGRADED:
anaconda 2020.07-py38_0 --> custom-py38_1
backports 1.0-py_2 --> 1.0-pyhd3eb1b0_2
backports.functoo~ 1.6.1-py_0 --> 1.6.1-pyhd3eb1b0_0
backports.tempfile 1.0-py_1 --> 1.0-pyhd3eb1b0_1
click 7.1.2-py_0 --> 7.1.2-pyhd3eb1b0_0
cython 0.29.21-py38hb1e8313_0 --> 0.29.21-py38h23ab428_0
glob2 0.7-py_0 --> 0.7-pyhd3eb1b0_0
idna 2.10-py_0 --> 2.10-pyhd3eb1b0_0
jinja2 2.11.2-py_0 --> 2.11.2-pyhd3eb1b0_0
lzo 2.10-h1de35cc_2 --> 2.10-haf1e3a3_2
mkl-service 2.3.0-py38hfbe908c_0 --> 2.3.0-py38h9ed2024_0
pyparsing 2.4.7-py_0 --> 2.4.7-pyhd3eb1b0_0
python-libarchive~ 2.9-py_0 --> 2.9-pyhd3eb1b0_0
wrapt 1.11.2-py38h1de35cc_0 --> 1.11.2-py38h9ed2024_0
Proceed ([y]/n)?
I was wondering is there any way to easily clean up all the packages I've installed globally so that I can start fresh with virtual environments? Otherwise is there any way I can update everything without the version conflicts?
I have a machine that has 3x 1080 GPU. Below are the code of the training:
dynamic_learning_rate = tf.placeholder(tf.float32, shape=[])
model_version = tf.constant(1, tf.int32)
with tf.device('/cpu:0'):
with tf.name_scope('Input'):
# Input images and labels.
batch_images,\
batch_input_vectors,\
batch_one_hot_labels,\
batch_file_paths,\
batch_labels = self.get_batch()
grads = []
pred = []
cost = []
# Define optimizer
optimizer = tf.train.MomentumOptimizer(learning_rate=dynamic_learning_rate / self.batch_size,
momentum=0.9,
use_nesterov=True)
split_input_image = tf.split(batch_images, self.num_gpus)
split_input_vector = tf.split(batch_input_vectors, self.num_gpus)
split_input_one_hot_label = tf.split(batch_one_hot_labels, self.num_gpus)
for i in range(self.num_gpus):
with tf.device(tf.DeviceSpec(device_type="GPU", device_index=i)):
with tf.variable_scope(tf.get_variable_scope(), reuse=i > 0):
with tf.name_scope('Model'):
# Construct model
with tf.variable_scope("inference"):
tower_pred = self.model(split_input_image[i], split_input_vector[i], is_training=True)
pred.append(tower_pred)
with tf.name_scope('Loss'):
# Define loss and optimizer
softmax_cross_entropy_cost = tf.reduce_mean(
tf.nn.softmax_cross_entropy_with_logits(logits=tower_pred, labels=split_input_one_hot_label[i]))
cost.append(softmax_cross_entropy_cost)
# Concat variables
pred = tf.concat(pred, 0)
cost = tf.reduce_mean(cost)
# L2 regularization
trainable_vars = tf.trainable_variables()
l2_regularization = tf.add_n(
[tf.nn.l2_loss(v) for v in trainable_vars if any(x in v.name for x in ['weights', 'biases'])])
for v in trainable_vars:
if any(x in v.name for x in ['weights', 'biases']):
print(v.name + ' - included for L2 regularization!')
else:
print(v.name)
cost = cost + self.l2_regularization_strength*l2_regularization
with tf.name_scope('Accuracy'):
# Evaluate model
correct_pred = tf.equal(tf.argmax(pred, 1), tf.argmax(batch_one_hot_labels, 1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))
prediction = tf.nn.softmax(pred, name='softmax')
# Creates a variable to hold the global_step.
global_step = tf.Variable(0, trainable=False, name='global_step')
# Minimization
update = optimizer.minimize(cost, global_step=global_step, colocate_gradients_with_ops=True)
After I run the training:
Fri Nov 10 12:28:00 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.90 Driver Version: 384.90 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 1080 Off | 00000000:03:00.0 Off | N/A |
| 42% 65C P2 62W / 198W | 7993MiB / 8114MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
| 1 GeForce GTX 1080 Off | 00000000:04:00.0 Off | N/A |
| 33% 53C P2 150W / 198W | 7886MiB / 8114MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 2 GeForce GTX 1080 Off | 00000000:05:00.0 On | N/A |
| 26% 54C P2 170W / 198W | 7883MiB / 8108MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 23228 C python 7982MiB |
| 1 23228 C python 7875MiB |
| 2 4793 G /usr/lib/xorg/Xorg 40MiB |
| 2 23228 C python 7831MiB |
+-----------------------------------------------------------------------------+
Fri Nov 10 12:28:36 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.90 Driver Version: 384.90 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 1080 Off | 00000000:03:00.0 Off | N/A |
| 42% 59C P2 54W / 198W | 7993MiB / 8114MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 GeForce GTX 1080 Off | 00000000:04:00.0 Off | N/A |
| 33% 57C P2 154W / 198W | 7886MiB / 8114MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
| 2 GeForce GTX 1080 Off | 00000000:05:00.0 On | N/A |
| 27% 55C P2 155W / 198W | 7883MiB / 8108MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 23228 C python 7982MiB |
| 1 23228 C python 7875MiB |
| 2 4793 G /usr/lib/xorg/Xorg 40MiB |
| 2 23228 C python 7831MiB |
+-----------------------------------------------------------------------------+
You see that the whenever the first GPU is running, the other two GPUs will be idle and vice versa. The alternate frequency is about 0.5 second.
For a single GPU, the training speed is around 650 [images/second], with all the 3 GPUs I got only 1050 [images/second].
Any idea of the problem?
You need to make sure that all the trainable variables are on the controller device (usually the CPU) and all the other worker devices (usually GPUs) are using the variables from the CPU in parallel.
We have a production database and we maintain it by using flyway. Recently we cloned our production database to create a UAT database. The UAT database has the same schema and data as that in production. Now we try to run "mvn flyway:migrate" on the UAT database to test new flyway script. However, we got
+---------+-----------------------+---------------------+---------+
| Version | Description | Installed on | State |
+---------+-----------------------+---------------------+---------+
| 0.0.1 | script.1 | | <Baseln |
| 0.0.2 | script.2 | | <Baseln |
| 0.0.3 | script.3 | | <Baseln |
| 0.1.1 | script.4 | | <Baseln |
| 0.1.2 | script.5 | | <Baseln |
| 0.2.0 | script.6 | | <Baseln |
| 0.5.1 | script.7 | | <Baseln |
| 0.5.2 | script.8 | | <Baseln |
| 0.6.0 | script.9 | | <Baseln |
| 0.7.0 | script.10 | | <Baseln |
| 0.8.0 | script.11 | | <Baseln |
| 0.9.0 | script.12 | | <Baseln |
| 0.10.0 | script.13 | | <Baseln |
| 0.11.1 | script.14 | | <Baseln |
| 0.12.0 | script.15 | | <Baseln |
| 0.13.0 | script.16 | | <Baseln |
| 0.14.0 | script.17 | | <Baseln |
| 0.15.0 | script.18 | | <Baseln |
| 0.16.0 | script.19 | | <Baseln |
| 0.16.1 | script.20 | | <Baseln |
| 0.17.0 | script.21 | | <Baseln |
| 0.17.1 | script.22 | | <Baseln |
| 0.18.0 | script.23 | | <Baseln |
| 1 | << Flyway Baseline >> | 2016-11-07 08:11:33 | Baselin |
| 1.16.0 | script.19 | 2017-02-15 10:03:18 | Future |
| 1.16.1 | script.20 | 2017-02-15 10:03:18 | Future |
+---------+-----------------------+---------------------+---------+
The script.23 is a new script. We expect the state is pending.
However, the state of all scripts became Baseln. I searched the relating topics for a day but could not find scenarios closed to my case. Is there any configuration on flyway (maven) I can use to run migrate command on a cloned database? Please help. (My database is SQL Server 2014, flyway version 4.0 ,maven version is 3.5, JDK version 1.7)
Thanks a lot.
Chi-Fu
I think that all versions lower than the baseline are not executed - they are supposed to belong to the baseline.
If script.23 is a new migration it should have a version greater than the last version, typically 1.18.0 (According to standard flyway config, V1_18_0__script.23.sql rather than V0_18_0__script.23.sql).
After having renamed this file, do a repair before trying to migrate again.
I am running Ubuntu 16.04 on an EC2 p2.xlarge shared instance.
The P2.xlarge instance gives access to a single GPU (1/2 of an NVidia K80 GPU?) to my shared instance.
How is this GPU shared between other Amazon EC2 instances on the same physical machine?
I was under the impression that 100% of the GPU was allocated to my instance. But this is clearly not the case. When my instance is running nothing on the GPU:
$ nvidia-smi
Tue Feb 21 00:11:16 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 375.39 Driver Version: 375.39 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K80 Off | 0000:00:1E.0 Off | 0 |
| N/A 39C P0 55W / 149W | 0MiB / 11439MiB | 63% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
A moment later:
| 0 Tesla K80 Off | 0000:00:1E.0 Off | 0 |
| N/A 40C P0 57W / 149W | 0MiB / 11439MiB | 97% Default |
And the next moment:
| 0 Tesla K80 Off | 0000:00:1E.0 Off | 0 |
| N/A 40C P0 56W / 149W | 0MiB / 11439MiB | 100% Default |
And pretty much stays there...
| 0 Tesla K80 Off | 0000:00:1E.0 Off | 0 |
| N/A 41C P0 56W / 149W | 0MiB / 11439MiB | 100% Default |
| 0 Tesla K80 Off | 0000:00:1E.0 Off | 0 |
| N/A 41C P0 56W / 149W | 0MiB / 11439MiB | 99% Default |
What are the rules for GPU allocation for GPU instances?
How can I get the list of GPU cards to which are connected monitors?
Can I get a list with the parameters: pciBusID, pciDeviceID, pciDomainID?
OS: Windows 7
GPUs: nVidia GeForce/Quadro
We can use utility nvidia-smi, which contained in the nVidia Video Drivers, to indicate to which GPU-card display is connected (only for professional GPU-card: Quadro / Tesla):
Windows: C:\Program Files\NVIDIA Corporation\NVSMI\nvidia-smi.exe
Linux: /usr/local/cuda/bin/nvidia-smi
example of nvidia-smi output:
+------------------------------------------------------+
| NVIDIA-SMI 332.88 Driver Version: 332.88 |
|-------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Quadro K4000 WDDM | 0000:01:00.0 Off | N/A |
| 30% 30C P8 9W / 87W | 3027MiB / 3071MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 GeForce GT 640 WDDM | 0000:02:00.0 N/A | N/A |
| 40% 27C N/A N/A / N/A | 2005MiB / 2047MiB | N/A Default |
+-------------------------------+----------------------+----------------------+
| 2 Quadro K4000 WDDM | 0000:03:00.0 On | N/A |
| 30% 34C P8 11W / 87W | 3028MiB / 3071MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
Where Disp.A - Shows the on which GPU-card Display is Active:
Off - display is not connected
On - display is connected
N/A - unknown (for not professional cards: GeForce)
Then we can say, that display is connected to GPU: 2 Quadro K4000 0000:03:00.0.