PackageNotFoundError: Package not found: 'conda' Package 'conda' is not installed in C:\Users\user\AppData\Local\Programs\Python\Python39 - anaconda

my problem started when I tried to use conda create -n nam python3 and got conda.exceptions.CondaRuntimeError:
The following packages will be downloaded:
package | build
---------------------------|-----------------
vs2015_runtime-14.0.25420 | 0 2.0 MB
vc-14 | 0 703 B
python-3.6.2 | 0 31.5 MB
certifi-2016.2.28 | py36_0 214 KB
wheel-0.29.0 | py36_0 129 KB
wincertstore-0.2 | py36_0 14 KB
setuptools-36.4.0 | py36_1 534 KB
pip-9.0.1 | py36_1 1.7 MB
------------------------------------------------------------
Total: 36.0 MB
The following NEW packages will be INSTALLED:
certifi: 2016.2.28-py36_0
pip: 9.0.1-py36_1
python: 3.6.2-0
setuptools: 36.4.0-py36_1
vc: 14-0
vs2015_runtime: 14.0.25420-0
wheel: 0.29.0-py36_0
wincertstore: 0.2-py36_0
Proceed ([y]/n)?
Fetching packages ...
vs2015_runtime 100% |###############################| Time: 0:00:00 3.63 MB/s
vc-14-0.tar.bz 100% |###############################| Time: 0:00:00 140.66 kB/s
python-3.6.2-0 100% |###############################| Time: 0:00:05 5.56 MB/s
certifi-2016.2 100% |###############################| Time: 0:00:00 1.18 MB/s
wheel-0.29.0-p 100% |###############################| Time: 0:00:00 1.02 MB/s
wincertstore-0 100% |###############################| Time: 0:00:00 4.84 MB/s
setuptools-36. 100% |###############################| Time: 0:00:00 1.64 MB/s
pip-9.0.1-py36 100% |###############################| Time: 0:00:01 1.68 MB/s
Extracting packages ...
[ COMPLETE ]|##################################################| 100%
Linking packages ...
An unexpected error has occurred. | 0%
Please consider posting the following information to the
conda GitHub issue tracker at:
https://github.com/conda/conda/issues
Current conda install:
platform : win-64
conda version : 4.2.7
conda is private : False
conda-env version : 4.2.7
conda-build version : not installed
python version : 3.9.12.final.0
requests version : 2.28.1
root environment : C:\Users\user\AppData\Local\Programs\Python\Python39 (writable)
default environment : C:\Users\user\AppData\Local\Programs\Python\Python39
envs directories : C:\Users\user\AppData\Local\Programs\Python\Python39\envs
package cache : C:\Users\user\AppData\Local\Programs\Python\Python39\pkgs
channel URLs : https://repo.continuum.io/pkgs/free/win-64/
https://repo.continuum.io/pkgs/free/noarch/
https://repo.continuum.io/pkgs/pro/win-64/
https://repo.continuum.io/pkgs/pro/noarch/
https://repo.continuum.io/pkgs/msys2/win-64/
https://repo.continuum.io/pkgs/msys2/noarch/
config file : None
offline mode : False
$ C:\Users\user\miniconda3\Scripts\conda-script.py create -n nam python
Traceback (most recent call last):
File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\conda\install.py", line 201, in yield_lines
with open(path) as fh:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\user\\AppData\\Local\\Programs\\Python\\Python39\\pkgs\\vs2015_runtime-14.0.25420-0\\info\\has_prefix'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\conda\install.py", line 209, in yield_lines
raise StopIteration
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\conda\cli\install.py", line 405, in install
execute_actions(actions, index, verbose=not context.quiet)
File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\conda\plan.py", line 643, in execute_actions
inst.execute_instructions(plan, index, verbose)
File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\conda\instructions.py", line 134, in execute_instructions
cmd(state, arg)
File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\conda\instructions.py", line 78, in LINK_CMD
link(state['prefix'], dist, lt, index=state['index'])
File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\conda\install.py", line 943, in link
has_prefix_files = read_has_prefix(join(info_dir, 'has_prefix'))
File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\conda\install.py", line 248, in read_has_prefix
return {pr.filepath: (pr.placeholder, pr.filemode) for pr in parsed_lines}
File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\conda\install.py", line 248, in <dictcomp>
return {pr.filepath: (pr.placeholder, pr.filemode) for pr in parsed_lines}
File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\conda\install.py", line 247, in <genexpr>
parsed_lines = (parse_line(line) for line in yield_lines(path))
RuntimeError: generator raised StopIteration
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\conda\exceptions.py", line 472, in conda_exception_handler
return_value = func(*args, **kwargs)
File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\conda\cli\main.py", line 144, in _main
exit_code = args.func(args, p)
File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\conda\cli\main_create.py", line 68, in execute
install(args, parser, 'create')
File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\conda\cli\install.py", line 420, in install
raise CondaRuntimeError('RuntimeError: %s' % e)
conda.exceptions.CondaRuntimeError: Runtime error: RuntimeError: generator raised StopIteration
my version is conda 4.2 and
I discovered that it's no longer supported, so I used conda update conda and got
PackageNotFoundError: Package not found: 'conda' Package 'conda' is not installed in C:\Users\user\AppData\Local\Programs\Python\Python39
, tried to reinstall anaconda and still got the same error.

"C:\Users\user\AppData\Local\Programs\Python\Python39"
has nothing to do with Anaconda. This the path where standard Python from python.org is installed. Why didn't you mention the 2 Python installations?
Anyhow, before you can use Anaconda you have activate it, preferably using the Anaconda prompt. Only then can you update conda.
And do yourself a favour: Either learn some conda basics from https://conda.io/projects/conda/en/latest/user-guide/getting-started.html
or remove it from your system.

Related

Running CI tests with PyQT

I am setting up a CI Bitbucket pipeline for my team. We are using pytest and pytest-qt to test our software. The tests run locally without any issue, but the build fails using pipeline. Here is the pipeline yml:
image: python:3.9
pipelines:
default:
- parallel:
- step:
name: Test
caches:
- pip
script:
- if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- pip install pytest
- pip install pytest-qt
- pytest -v tools/IntegrationTests.py --junitxml=test-reports/report.xml
The file requirements.txt contains several modules including:
PyQt5==5.15.4
PyQt5-Qt5==5.15.2
And here is the output when trying to build:
+ pytest -v tools/IntegrationTests.py --junitxml=test-reports/report.xml
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 265, in wrap_session
INTERNALERROR> config._do_configure()
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/_pytest/config/__init__.py", line 982, in _do_configure
INTERNALERROR> self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pluggy/hooks.py", line 308, in call_historic
INTERNALERROR> res = self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pluggy/manager.py", line 84, in <lambda>
INTERNALERROR> self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pytestqt/plugin.py", line 203, in pytest_configure
INTERNALERROR> qt_api.set_qt_api(config.getini("qt_api"))
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pytestqt/qt_compat.py", line 104, in set_qt_api
INTERNALERROR> self.QtGui = _import_module("QtGui")
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pytestqt/qt_compat.py", line 100, in _import_module
INTERNALERROR> m = __import__(_root_module, globals(), locals(), [module_name], 0)
INTERNALERROR> ImportError: libGL.so.1: cannot open shared object file: No such file or directory
What I get from that is that Qt is not properly installed remotely, thus the tests cannot run.
My question is: can GUI be tested in CI, and if yes, how?
EDIT:
I have broke down to test only the import of PyQt and it boiled down to this test suite:
def test_import_widgets():
from PyQt5 import QtWidgets
def test_import_core():
from PyQt5 import QtCore
def test_import_gui():
from PyQt5 import QtGui
def test_import_qt():
from PyQt5.QtCore import Qt
Which have resulted in the following results:
2 / 4 tests failed
BasicsTests.test_import_guitools
<1s
ImportError: libGL.so.1: cannot open shared object file: No such file or directory
def test_import_gui():
> from PyQt5 import QtGui
E ImportError: libGL.so.1: cannot open shared object file: No such file or directory
tools/BasicsTests.py:50: ImportError
BasicsTests.test_import_widgetstools
<1s
ImportError: libGL.so.1: cannot open shared object file: No such file or directory
def test_import_widgets():
> from PyQt5 import QtWidgets
E ImportError: libGL.so.1: cannot open shared object file: No such file or directory
tools/BasicsTests.py:42: ImportError
You have to install the Qt dependencies:
image: python:3.9
pipelines:
default:
- parallel:
- step:
name: Test
caches:
- pip
script:
- apt-get update && apt-get autoclean
- apt-get install -y '^libxcb.*-dev' libx11-xcb-dev libglu1-mesa-dev libxrender-dev libxi-dev libxkbcommon-dev libxkbcommon-x11-dev
- if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- pip install pytest
- pip install pytest-qt
- pytest -v tools/IntegrationTests.py --junitxml=test-reports/report.xml

Why is pipenv failing to install dependencies in production environment but not locally?

So I attempting to debug a deployment on AWS EB -> Elastic BeanStalk Django Error in Pipfile
And it seems like the issue is being caused by pipenv
error Command /bin/sh -c /usr/bin/python3.8 -m pipenv install --skip-lock
However, when I run that command locally it works fine, so I dont know what to do to fix this or investigate further on the cause of the issue.
My pipfile is:
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
[packages]
appdirs = "==1.4.4"
asgiref = "==3.3.1"
"backports.entry-points-selectable" = "==1.1.0"
certifi = "==2020.11.8"
chardet = "==3.0.4"
distlib = "==0.3.1"
django-cors-headers = "==3.6.0"
djangorestframework = "==3.12.2"
filelock = "==3.0.12"
idna = "==2.10"
pipenv = "==2020.8.13"
platformdirs = "==2.2.0"
pytz = "==2020.4"
requests = "==2.25.0"
six = "==1.15.0"
sqlparse = "==0.4.1"
stripe = "==2.55.1"
urllib3 = "==1.26.2"
virtualenv = "==20.7.2"
virtualenv-clone = "==0.5.6"
Django = "==3.0.8"
Pillow = "==8.0.1"
[requires]
python_version = "3.8"
And the full error log is:
----------------------------------------
/var/log/eb-engine.log
----------------------------------------
Downloading Pillow-8.0.1-cp38-cp38-manylinux1_x86_64.whl (2.2 MB)
Collecting pipenv==2020.11.15
Downloading pipenv-2020.11.15-py2.py3-none-any.whl (3.9 MB)
Collecting pytz==2020.4
Downloading pytz-2020.4-py2.py3-none-any.whl (509 kB)
Collecting requests==2.25.0
Downloading requests-2.25.0-py2.py3-none-any.whl (61 kB)
Collecting six==1.15.0
Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting sqlparse==0.4.1
Downloading sqlparse-0.4.1-py3-none-any.whl (42 kB)
Collecting stripe==2.55.1
Downloading stripe-2.55.1-py2.py3-none-any.whl (204 kB)
Collecting urllib3==1.26.2
Downloading urllib3-1.26.2-py2.py3-none-any.whl (136 kB)
Collecting virtualenv==20.2.1
Downloading virtualenv-20.2.1-py2.py3-none-any.whl (4.9 MB)
Collecting virtualenv-clone==0.5.4
Downloading virtualenv_clone-0.5.4-py2.py3-none-any.whl (6.6 kB)
Requirement already satisfied: pip>=18.0 in /var/app/venv/staging-LQM1lest/lib/python3.8/site-packages (from pipenv==2020.11.15->-r requirements.txt (line 12)) (21.1.3)
Requirement already satisfied: setuptools>=36.2.1 in /var/app/venv/staging-LQM1lest/lib/python3.8/site-packages (from pipenv==2020.11.15->-r requirements.txt (line 12)) (57.1.0)
Installing collected packages: urllib3, sqlparse, six, pytz, idna, filelock, distlib, chardet, certifi, asgiref, appdirs, virtualenv-clone, virtualenv, requests, Django, stripe, pipenv, Pillow, djangorestframework, django-cors-headers
Successfully installed Django-3.0.8 Pillow-8.0.1 appdirs-1.4.4 asgiref-3.3.1 certifi-2020.11.8 chardet-3.0.4 distlib-0.3.1 django-cors-headers-3.6.0 djangorestframework-3.12.2 filelock-3.0.12 idna-2.10 pipenv-2020.11.15 pytz-2020.4 requests-2.25.0 six-1.15.0 sqlparse-0.4.1 stripe-2.55.1 urllib3-1.26.2 virtualenv-20.2.1 virtualenv-clone-0.5.4
2021/08/13 12:49:39.051999 [INFO] Installing dependencies with Pipfile.lock
2021/08/13 12:49:39.052013 [INFO] Running command /bin/sh -c /usr/bin/python3.8 -m pipenv install --ignore-pipfile
2021/08/13 12:49:40.248089 [INFO] Installing dependencies with Pipfile
2021/08/13 12:49:40.248118 [INFO] Running command /bin/sh -c /usr/bin/python3.8 -m pipenv install --skip-lock
2021/08/13 12:49:40.976863 [INFO] Installing dependencies from Pipfile…
2021/08/13 12:49:40.976902 [ERROR] An error occurred during execution of command [app-deploy] - [InstallDependency]. Stop running the command. Error: fail to install dependencies with Pipfile file with error Command /bin/sh -c /usr/bin/python3.8 -m pipenv install --skip-lock failed with error exit status 1. Stderr:Traceback (most recent call last):
File "/usr/lib64/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib64/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.8/site-packages/pipenv/__main__.py", line 5, in <module>
cli()
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/decorators.py", line 73, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pipenv/cli/command.py", line 232, in install
retcode = do_install(
File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 2052, in do_install
do_init(
File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 1305, in do_init
do_install_dependencies(
File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 827, in do_install_dependencies
lockfile = project.get_or_create_lockfile(from_pipfile=True)
File "/usr/local/lib/python3.8/site-packages/pipenv/project.py", line 783, in get_or_create_lockfile
lockfile_dict.update({"_meta": self.get_lockfile_meta()})
File "/usr/local/lib/python3.8/site-packages/pipenv/project.py", line 817, in get_lockfile_meta
sources = self.lockfile_content.get("_meta", {}).get("sources", [])
File "/usr/local/lib/python3.8/site-packages/pipenv/project.py", line 662, in lockfile_content
return self.load_lockfile()
File "/usr/local/lib/python3.8/site-packages/pipenv/project.py", line 1044, in load_lockfile
j = json.load(lock)
File "/usr/lib64/python3.8/json/__init__.py", line 293, in load
return loads(fp.read(),
File "/usr/lib64/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.8/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 4 column 1 (char 35)
2021/08/13 12:49:40.976920 [INFO] Executing cleanup logic
2021/08/13 12:49:40.977012 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed to install application dependencies. The deployment failed.","timestamp":1628858980,"severity":"ERROR"},{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1628858980,"severity":"ERROR"}]}]}
Additionally, I am operating inside a virtualenv and the command I mentioned gives me the following warning
Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
I have tried to run it outside the virtualenv but can't seem to get the correct version of pipenv I have version 2021.5.29 and I need 2020.8.13 as specified by AWS in their documentation.
There is also a small difference between the python version as I use python3.8.11 and they use 3.8.5 does that make a difference?
Not a python export but the error message explicitly notes.
“Instance deployment failed to install application dependencies”
And reading the stack trace it notes.
“ json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 4 column 1 (char 35)”
Looks like you have malformed JSON in your lock file.

Visualizing decision tree with graphviz: How to solve FileNotFoundError?

I want to visualize a decision tree using graphviz.
I found some example code (https://gist.github.com/WillKoehrsen/ff77f5f308362819805a3defd9495ffd):
from sklearn.datasets import load_iris
iris = load_iris()
# Model (can also use single decision tree)
from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators=10)
# Train
model.fit(iris.data, iris.target)
# Extract single tree
estimator = model.estimators_[5]
from sklearn.tree import export_graphviz
# Export as dot file
export_graphviz(estimator, out_file='tree.dot',
feature_names = iris.feature_names,
class_names = iris.target_names,
rounded = True, proportion = False,
precision = 2, filled = True)
# Convert to png using system command (requires Graphviz)
from subprocess import call
call(['dot', '-Tpng', 'tree.dot', '-o', 'tree.png', '-Gdpi=600'])
# Display in jupyter notebook
from IPython.display import Image
Image(filename = 'tree.png')
What it should output:
When I do this with jupyter notebooks, however, I get a FileNotFoundError:
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
<ipython-input-3-6d9aafea91ef> in <module>()
21 # Convert to png using system command (requires Graphviz)
22 from subprocess import call
---> 23 call(['dot', '-Tpng', 'tree.dot', '-o', 'tree.png', '-Gdpi=600'])
24
25 # Display in jupyter notebook
C:\ProgramData\Anaconda3\lib\subprocess.py in call(timeout, *popenargs, **kwargs)
302 retcode = call(["ls", "-l"])
303 """
--> 304 with Popen(*popenargs, **kwargs) as p:
305 try:
306 return p.wait(timeout=timeout)
C:\ProgramData\Anaconda3\lib\subprocess.py in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, encoding, errors, text)
754 c2pread, c2pwrite,
755 errread, errwrite,
--> 756 restore_signals, start_new_session)
757 except:
758 # Cleanup if the child failed starting.
C:\ProgramData\Anaconda3\lib\subprocess.py in _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_start_new_session)
1153 env,
1154 os.fspath(cwd) if cwd is not None else None,
-> 1155 startupinfo)
1156 finally:
1157 # Child is launched. Close the parent's copy of those pipe
FileNotFoundError: [WinError 2] Das System kann die angegebene Datei nicht finden
(The last one says that my system was not able to find the stated file)
With Anaconda Prompt in admin mode I ran conda install -c anaconda graphviz with no errors:
Collecting package metadata: done
Solving environment: \
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:
- defaults/win-64::anaconda==5.3.1=py37_0
- defaults/win-64::astropy==3.0.4=py37hfa6e2cd_0
- defaults/win-64::bkcharts==0.2=py37_0
- defaults/win-64::blaze==0.11.3=py37_0
- defaults/win-64::bokeh==0.13.0=py37_0
- defaults/win-64::bottleneck==1.2.1=py37h452e1ab_1
- defaults/win-64::dask==0.19.1=py37_0
- defaults/win-64::datashape==0.5.4=py37_1
- defaults/win-64::h5py==2.8.0=py37h3bdd7fb_2
- defaults/win-64::imageio==2.4.1=py37_0
- defaults/win-64::matplotlib==2.2.3=py37hd159220_0
- defaults/win-64::mkl-service==1.1.2=py37hb217b18_5
- defaults/win-64::mkl_fft==1.0.4=py37h1e22a9b_1
- defaults/win-64::mkl_random==1.0.1=py37h77b88f5_1
- defaults/win-64::numba==0.39.0=py37h830ac7b_0
- defaults/win-64::numexpr==2.6.8=py37h9ef55f4_0
- defaults/win-64::numpy==1.15.1=py37ha559c80_0
- defaults/win-64::numpy-base==1.15.1=py37h8128ebf_0
- defaults/win-64::odo==0.5.1=py37_0
- defaults/win-64::pandas==0.23.4=py37h830ac7b_0
- defaults/win-64::patsy==0.5.0=py37_0
- defaults/win-64::pytables==3.4.4=py37he6f6034_0
- defaults/win-64::pytest-arraydiff==0.2=py37h39e3cac_0
- defaults/win-64::pytest-astropy==0.4.0=py37_0
- defaults/win-64::pytest-doctestplus==0.1.3=py37_0
- defaults/win-64::pywavelets==1.0.0=py37h452e1ab_0
- defaults/win-64::scikit-image==0.14.0=py37h6538335_1
- defaults/win-64::scikit-learn==0.19.2=py37heebcf9a_0
- defaults/win-64::scipy==1.1.0=py37h4f6bf74_1
- defaults/win-64::seaborn==0.9.0=py37_0
- defaults/win-64::statsmodels==0.9.0=py37h452e1ab_0
done
## Package Plan ##
environment location: C:\ProgramData\Anaconda3
added / updated specs:
- graphviz
The following packages will be downloaded:
package | build
---------------------------|-----------------
ca-certificates-2019.1.23 | 0 158 KB anaconda
certifi-2019.3.9 | py37_0 155 KB anaconda
conda-4.6.12 | py37_1 2.1 MB anaconda
graphviz-2.38.0 | 4 37.7 MB anaconda
openssl-1.1.1 | he774522_0 5.7 MB anaconda
vc-14.1 | h21ff451_3 5 KB anaconda
vs2015_runtime-15.5.2 | 3 2.2 MB anaconda
------------------------------------------------------------
Total: 48.1 MB
The following packages will be UPDATED:
openssl conda-forge::openssl-1.1.1b-hfa6e2cd_2 --> anaconda::openssl-1.1.1-he774522_0
vs2015_runtime pkgs/main::vs2015_runtime-14.15.26706~ --> anaconda::vs2015_runtime-15.5.2-3
The following packages will be SUPERSEDED by a higher-priority channel:
ca-certificates conda-forge::ca-certificates-2019.3.9~ --> anaconda::ca-certificates-2019.1.23-0
certifi conda-forge --> anaconda
conda conda-forge::conda-4.6.12-py37_2 --> anaconda::conda-4.6.12-py37_1
graphviz conda-forge::graphviz-2.38.0-h6538335~ --> anaconda::graphviz-2.38.0-4
vc pkgs/main::vc-14.1-h0510ff6_4 --> anaconda::vc-14.1-h21ff451_3
Proceed ([y]/n)? y
Downloading and Extracting Packages
certifi-2019.3.9 | 155 KB | ############################################################################ | 100%
conda-4.6.12 | 2.1 MB | ############################################################################ | 100%
graphviz-2.38.0 | 37.7 MB | ############################################################################ | 100%
openssl-1.1.1 | 5.7 MB | ############################################################################ | 100%
vc-14.1 | 5 KB | ############################################################################ | 100%
ca-certificates-2019 | 158 KB | ############################################################################ | 100%
vs2015_runtime-15.5. | 2.2 MB | ############################################################################ | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Suprisingly, it worked just fine when I ran the code on kaggle.com.
Any ideas on what could be the problem and how to solve it?
Just solved this.
The trick was to add the graphviz path to the windows environmental variables, here is a good description: https://bobswift.atlassian.net/wiki/spaces/GVIZ/pages/20971549/How+to+install+Graphviz+software
I restarted my computer after this and voila.

QIIME2 dada2 rlang.so error

I was running the QIIME2 moving picture tutorial, at the dada2 step,
I was running:
qiime dada2 denoise-single \
--i-demultiplexed-seqs demux.qza \
--p-trim-left 0 \
--p-trunc-len 120 \
--o-representative-sequences rep-seqs-dada2.qza \
--o-table table-dada2.qza
and ran into this error:
Plugin error from dada2:
An error was encountered while running DADA2 in R (return code 1),
please inspect stdout and stderr to learn more.
Debug info has been saved to /tmp/qiime2-q2cli-err-52fzrvlu.log.
I then open the file: /tmp/qiime2-q2cli-err-52fzrvlu.log. this is what I found:
Running external command line application(s). This may print messages
to stdout and/or stderr. The command(s) being run are below. These
commands cannot be manually re-run as they will depend on temporary
files that no longer exist.
Command: run_dada_single.R
/tmp/qiime2-archive-pco6y5vm/fe614b44-775f-41b1-9ee3-04319005e830/data
/tmp/tmpda8dnyve/output.tsv.biom /tmp/tmpda8dnyve 120 0 2.0 2
consensus 1.0 1 1000000
R version 3.3.1 (2016-06-21) Loading required package: Rcpp Error in
dyn.load(file, DLLpath = DLLpath, ...) : unable to load shared
object '/home/cao/lib/R/library/rlang/libs/rlang.so':
/home/cao/lib/R/library/rlang/libs/rlang.so: undefined symbol:
R_ExternalPtrAddrFn In addition: Warning message: package ‘Rcpp’ was
built under R version 3.4.1 Error: package or namespace load failed
for ‘dada2’ Execution halted Traceback (most recent call last): File
"/home/cao/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2_dada2/_denoise.py",
line 126, in denoise_single
run_commands([cmd]) File "/home/cao/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2_dada2/_denoise.py",
line 35, in run_commands
subprocess.run(cmd, check=True) File "/home/cao/miniconda3/envs/qiime2-2017.7/lib/python3.5/subprocess.py",
line 398, in run
output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['run_dada_single.R',
'/tmp/qiime2-archive-pco6y5vm/fe614b44-775f-41b1-9ee3-04319005e830/data',
'/tmp/tmpda8dnyve/output.tsv.biom', '/tmp/tmpda8dnyve', '120', '0',
'2.0', '2', 'consensus', '1.0', '1', '1000000']' returned non-zero
exit status 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File
"/home/cao/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2cli/commands.py",
line 222, in __call__
results = action(**arguments) File "<decorator-gen-252>", line 2, in denoise_single File
"/home/cao/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/qiime2/sdk/action.py",
line 201, in callable_wrapper
output_types, provenance) File "/home/cao/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/qiime2/sdk/action.py",
line 334, in _callable_executor_
output_views = callable(**view_args) File "/home/cao/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2_dada2/_denoise.py",
line 137, in denoise_single
" and stderr to learn more." % e.returncode) Exception: An error was encountered while running DADA2 in R (return code 1), please
inspect stdout and stderr to learn more.
I then 'sudo R' and installed the Rcpp and rlang packages, but still got the the same error when I run the same code as I first did:
qiime dada2 denoise-single \
--i-demultiplexed-seqs demux.qza \
--p-trim-left 0 \
--p-trunc-len 120 \
--o-representative-sequences rep-seqs-dada2.qza \
--o-table table-dada2.qza
I figured it out:
it was the R version: i uninstalled R3.4 and install R 3.3 and everythign works

Cloud Init Fails to Install Packages on CentOS 7 in EC2

I have a cloud config file defined on my EC2 instance in the user data. Many of the parts/actions are run by the server properly, but package installation seems to fail invariably:
#cloud-config
package-update: true
package-upgrade: true
# ...
packages:
- puppet3
- lvm2
- btrfs-progs
# ...
I see the following in the logs:
May 20 20:23:39 cloud-init[1252]: util.py[DEBUG]: Package update failed
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py", line 74, in handle
cloud.distro.update_package_sources()
File "/usr/lib/python2.6/site-packages/cloudinit/distros/rhel.py", line 278, in update_package_sources
["makecache"], freq=PER_INSTANCE)
File "/usr/lib/python2.6/site-packages/cloudinit/helpers.py", line 197, in run
results = functor(*args)
File "/usr/lib/python2.6/site-packages/cloudinit/distros/rhel.py", line 274, in package_command
util.subp(cmd, capture=False, pipe_cat=True, close_stdin=True)
File "/usr/lib/python2.6/site-packages/cloudinit/util.py", line 1529, in subp
cmd=args)
ProcessExecutionError: Unexpected error while running command.
Command: ['yum', '-t', '-y', 'makecache']
Exit code: 1
Reason: -
Stdout: ''
Stderr: ''
May 20 20:23:39 cloud-init[1252]: amazon.py[DEBUG]: Upgrade level: security
May 20 20:23:39 cloud-init[1252]: util.py[DEBUG]: Running command ['yum', '-t', '-y', '--exclude=kernel', '--exclude=nvidia*', '--exclude=cudatoolkit', '--security', '--sec-severity=critical', '--sec-severity=important', 'upgrade'] with allowed return codes [0] (shell=False, capture=False)
May 20 20:23:52 cloud-init[1252]: util.py[WARNING]: Package upgrade failed
May 20 20:23:52 cloud-init[1252]: util.py[DEBUG]: Package upgrade failed
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py", line 81, in handle
cloud.distro.upgrade_packages(upgrade_level, upgrade_exclude)
File "/usr/lib/python2.6/site-packages/cloudinit/distros/amazon.py", line 50, in upgrade_packages
return self.package_command('upgrade', args=args)
File "/usr/lib/python2.6/site-packages/cloudinit/distros/rhel.py", line 274, in package_command
util.subp(cmd, capture=False, pipe_cat=True, close_stdin=True)
File "/usr/lib/python2.6/site-packages/cloudinit/util.py", line 1529, in subp
cmd=args)
ProcessExecutionError: Unexpected error while running command.
Command: ['yum', '-t', '-y', '--exclude=kernel', '--exclude=nvidia*', '--exclude=cudatoolkit', '--security', '--sec-severity=critical', '--sec-severity=important', 'upgrade']
Exit code: 1
Reason: -
Stdout: ''
Stderr: ''
May 20 20:23:52 cloud-init[1252]: util.py[DEBUG]: Running command ['yum', '-t', '-y', 'install', 'puppet3', 'lvm2', 'btrfs-progs'] with allowed return codes [0] (shell=False, capture=False)
May 20 20:24:03 cloud-init[1252]: util.py[WARNING]: Failed to install packages: ['puppet3', 'lvm2', 'btrfs-progs']
May 20 20:24:03 cloud-init[1252]: util.py[DEBUG]: Failed to install packages: ['puppet3', 'lvm2', 'btrfs-progs']
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py", line 88, in handle
cloud.distro.install_packages(pkglist)
File "/usr/lib/python2.6/site-packages/cloudinit/distros/rhel.py", line 70, in install_packages
self.package_command('install', pkgs=pkglist)
File "/usr/lib/python2.6/site-packages/cloudinit/distros/rhel.py", line 274, in package_command
util.subp(cmd, capture=False, pipe_cat=True, close_stdin=True)
File "/usr/lib/python2.6/site-packages/cloudinit/util.py", line 1529, in subp
cmd=args)
ProcessExecutionError: Unexpected error while running command.
Command: ['yum', '-t', '-y', 'install', 'puppet3', 'lvm2', 'btrfs-progs']
Exit code: 1
Reason: -
Stdout: ''
Stderr: ''
May 20 20:24:03 cloud-init[1252]: cc_package_update_upgrade_install.py[WARNING]: 3 failed with exceptions, re-raising the last one
May 20 20:24:03 cloud-init[1252]: util.py[WARNING]: Running package-update-upgrade-install (<module 'cloudinit.config.cc_package_update_upgrade_install' from '/usr/lib/python2.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.pyc'>) failed
May 20 20:24:03 cloud-init[1252]: util.py[DEBUG]: Running package-update-upgrade-install (<module 'cloudinit.config.cc_package_update_upgrade_install' from '/usr/lib/python2.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.pyc'>) failed
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/cloudinit/stages.py", line 553, in _run_modules
cc.run(run_name, mod.handle, func_args, freq=freq)
File "/usr/lib/python2.6/site-packages/cloudinit/cloud.py", line 63, in run
return self._runners.run(name, functor, args, freq, clear_on_fail)
File "/usr/lib/python2.6/site-packages/cloudinit/helpers.py", line 197, in run
results = functor(*args)
File "/usr/lib/python2.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py", line 111, in handle
raise errors[-1]
ProcessExecutionError: Unexpected error while running command.
Command: ['yum', '-t', '-y', 'install', 'puppet3', 'lvm2', 'btrfs-progs']
Exit code: 1
Reason: -
Stdout: ''
Stderr: ''
When I actually run the command yum -t -y install puppet3 lvm2 btrfs-progs as root, it just runs fine, but Cloud Init is failing to run it on its own.
Is there something I'm doing wrong here?
Evidently there was a bug fixed with later versions of the image. If this is happening to you, it may be a legitimate bug in Cloud-Init or your servers implementation of it. A package upgrade may fix the problem, also an update to the image will fix it globally.

Resources