QIIME2 dada2 rlang.so error - qiime

I was running the QIIME2 moving picture tutorial, at the dada2 step,
I was running:
qiime dada2 denoise-single \
--i-demultiplexed-seqs demux.qza \
--p-trim-left 0 \
--p-trunc-len 120 \
--o-representative-sequences rep-seqs-dada2.qza \
--o-table table-dada2.qza
and ran into this error:
Plugin error from dada2:
An error was encountered while running DADA2 in R (return code 1),
please inspect stdout and stderr to learn more.
Debug info has been saved to /tmp/qiime2-q2cli-err-52fzrvlu.log.
I then open the file: /tmp/qiime2-q2cli-err-52fzrvlu.log. this is what I found:
Running external command line application(s). This may print messages
to stdout and/or stderr. The command(s) being run are below. These
commands cannot be manually re-run as they will depend on temporary
files that no longer exist.
Command: run_dada_single.R
/tmp/qiime2-archive-pco6y5vm/fe614b44-775f-41b1-9ee3-04319005e830/data
/tmp/tmpda8dnyve/output.tsv.biom /tmp/tmpda8dnyve 120 0 2.0 2
consensus 1.0 1 1000000
R version 3.3.1 (2016-06-21) Loading required package: Rcpp Error in
dyn.load(file, DLLpath = DLLpath, ...) : unable to load shared
object '/home/cao/lib/R/library/rlang/libs/rlang.so':
/home/cao/lib/R/library/rlang/libs/rlang.so: undefined symbol:
R_ExternalPtrAddrFn In addition: Warning message: package ‘Rcpp’ was
built under R version 3.4.1 Error: package or namespace load failed
for ‘dada2’ Execution halted Traceback (most recent call last): File
"/home/cao/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2_dada2/_denoise.py",
line 126, in denoise_single
run_commands([cmd]) File "/home/cao/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2_dada2/_denoise.py",
line 35, in run_commands
subprocess.run(cmd, check=True) File "/home/cao/miniconda3/envs/qiime2-2017.7/lib/python3.5/subprocess.py",
line 398, in run
output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['run_dada_single.R',
'/tmp/qiime2-archive-pco6y5vm/fe614b44-775f-41b1-9ee3-04319005e830/data',
'/tmp/tmpda8dnyve/output.tsv.biom', '/tmp/tmpda8dnyve', '120', '0',
'2.0', '2', 'consensus', '1.0', '1', '1000000']' returned non-zero
exit status 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File
"/home/cao/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2cli/commands.py",
line 222, in __call__
results = action(**arguments) File "<decorator-gen-252>", line 2, in denoise_single File
"/home/cao/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/qiime2/sdk/action.py",
line 201, in callable_wrapper
output_types, provenance) File "/home/cao/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/qiime2/sdk/action.py",
line 334, in _callable_executor_
output_views = callable(**view_args) File "/home/cao/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2_dada2/_denoise.py",
line 137, in denoise_single
" and stderr to learn more." % e.returncode) Exception: An error was encountered while running DADA2 in R (return code 1), please
inspect stdout and stderr to learn more.
I then 'sudo R' and installed the Rcpp and rlang packages, but still got the the same error when I run the same code as I first did:
qiime dada2 denoise-single \
--i-demultiplexed-seqs demux.qza \
--p-trim-left 0 \
--p-trunc-len 120 \
--o-representative-sequences rep-seqs-dada2.qza \
--o-table table-dada2.qza

I figured it out:
it was the R version: i uninstalled R3.4 and install R 3.3 and everythign works

Related

pip install command creates syntax error : Traceback (most recent call last)

I've been trying to install bs4 and selenium from the windows powershell, and here is the error message I get :
ERROR: Exception:
Traceback (most recent call last):
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\cli\base_command.py", line 160, in exc_logging_wrapper
status = run_func(*args)
^^^^^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\cli\req_command.py", line 247, in wrapper
return func(self, options, args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\commands\install.py", line 326, in run
session = self.get_default_session(options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\cli\req_command.py", line 98, in get_default_session
self._session = self.enter_context(self._build_session(options))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\cli\req_command.py", line 125, in _build_session
session = PipSession(
^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\network\session.py", line 343, in __init__
self.headers["User-Agent"] = user_agent()
^^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\network\session.py", line 175, in user_agent
setuptools_dist = get_default_environment().get_distribution("setuptools")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\metadata\importlib\_envs.py", line 188, in get_distribution
return next(matches, None)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\metadata\importlib\_envs.py", line 183, in <genexpr>
matches = (
^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\metadata\base.py", line 612, in iter_all_distributions
for dist in self._iter_distributions():
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\metadata\importlib\_envs.py", line 176, in _iter_distributions
for dist in finder.find_eggs(location):
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\metadata\importlib\_envs.py", line 146, in find_eggs
yield from self._find_eggs_in_zip(location)
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\metadata\importlib\_envs.py", line 123, in _find_eggs_in_zip
from pip._vendor.pkg_resources import find_eggs_in_zip
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3251, in <module>
#_call_aside
^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3235, in _call_aside
f(*args, **kwargs)
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3277, in _initialize_master_working_set
tuple(
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3278, in <genexpr>
dist.activate(replace=False)
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2785, in activate
declare_namespace(pkg)
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2284, in declare_namespace
_handle_ns(packageName, path_item)
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2217, in _handle_ns
loader.load_module(packageName)
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pygoogle\google.py", line 113
print """PyGoogle %(__version__)s
^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)?
I cannot install any pip package in my pc it returns that error
pip version is : pip 22.3.1
python version is : Python 3.11.1
Appreciate Your support .
I tried to install any pip package
You've installed a package PyGoogle which is very old (2006) and works only with Python 2. You need to uninstall it:
pip uninstall -y pygoogle
If that doesn't work — remove the directory C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pygoogle

Airflow commands fail due to typeerror in utils.py when deployed in heroku

Im running airflow version 2.4.1 with python 3.10. I have no issues when running airflow locally in docker, but when I deploy it to heroku using their container registry, the airflow webserver command fails with this trackback:
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
Running the Gunicorn Server with:
Workers: 4 sync
Host: 0.0.0.0:55811
Timeout: 120
Logfiles: - -
Access Logformat:
=================================================================
[2022-11-10 00:19:39 +0000] [43] [INFO] Starting gunicorn 20.1.0
Traceback (most recent call last):
File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.10/site-packages/gunicorn/__main__.py", line 7, in <module>
run()
File "/usr/local/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 67, in run
WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
File "/usr/local/lib/python3.10/site-packages/gunicorn/app/base.py", line 231, in run
super().run()
File "/usr/local/lib/python3.10/site-packages/gunicorn/app/base.py", line 72, in run
Arbiter(self).run()
File "/usr/local/lib/python3.10/site-packages/gunicorn/arbiter.py", line 198, in run
self.start()
File "/usr/local/lib/python3.10/site-packages/gunicorn/arbiter.py", line 138, in start
self.cfg.on_starting(self)
File "/usr/local/lib/python3.10/site-packages/airflow/www/gunicorn_config.py", line 40, in on_starting
ProvidersManager().connection_form_widgets
File "/usr/local/lib/python3.10/site-packages/airflow/providers_manager.py", line 908, in connection_form_widgets
self.initialize_providers_hooks()
File "/usr/local/lib/python3.10/site-packages/airflow/providers_manager.py", line 300, in wrapped_function
func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/airflow/providers_manager.py", line 386, in initialize_providers_hooks
self.initialize_providers_list()
File "/usr/local/lib/python3.10/site-packages/airflow/providers_manager.py", line 300, in wrapped_function
func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/airflow/providers_manager.py", line 363, in initialize_providers_list
self._discover_all_providers_from_packages()
File "/usr/local/lib/python3.10/site-packages/airflow/providers_manager.py", line 429, in _discover_all_providers_from_packages
for entry_point, dist in entry_points_with_dist('apache_airflow_provider'):
File "/usr/local/lib/python3.10/site-packages/airflow/utils/entry_points.py", line 40, in entry_points_with_dist
key = canonicalize_name(dist.metadata["Name"])
File "/usr/local/lib/python3.10/site-packages/packaging/utils.py", line 34, in canonicalize_name
value = _canonicalize_regex.sub("-", name).lower()
TypeError: expected string or bytes-like object
[2022-11-10 00:21:38,935] {webserver_command.py:217} ERROR - No response from gunicorn master within 120 seconds
[2022-11-10 00:21:38,935] {webserver_command.py:218} ERROR - Shutting down webserver
Has anyone run into this error before? Let me know if more information is needed.
I have just solved the problem, in my case I had packages where there weren't NAME field in METADATA.
First you can check it counting number of None:
import importlib_metadata as metadata
import pandas as pd
s = pd.Series([dist.metadata["Name"] for dist in metadata.distributions()])
s.isnull().sum()
After that, you need find what packages are causing your problem and uninstall and install it.
I solve it using vscode debugger with justMycode: false.
I was debugging the next code:
for met in metadata.distributions():
print(met.name)
The point here is to add a if condition and a break point in the next path /PATH/lib/PYTHON_VERSION/site-package/importlib_metadata/__init__.py, for example in my case:
/home/antonio/anaconda3/Aspect-Based-On-Sentiment-Analysis/lib/python3.7/site-package/importlib_metadata/__init__.py
Set if condition and break point in print('now') (line 598):
#property
def name(self):
"""Return the 'Name' metadata for the distribution package."""
if not self.metadata.json:
print('now')
return self.metadata['Name']
You could see on debug window something like that,
The field _normalized_name will indicate you package name that you need to uninstall since the rest of the fields will be None when name is None
May be pip uninstall name_package doesn't work.
I recommend you to search the package in /PATH/lib/PYTHON_VERSION/site-package, check NAME is not in METADATA file or METADATA file doesn't exist and remove the entire package, finally, you can use pip install name-package and check NAME is in METADATA file
Then you check number of packages without NAME in METADATA running:
import importlib_metadata as metadata
import pandas as pd
s = pd.Series([dist.metadata["Name"] for dist in metadata.distributions()])
s.isnull().sum()
You will see it has decreased by one.

Cloud Init Fails to Install Packages on CentOS 7 in EC2

I have a cloud config file defined on my EC2 instance in the user data. Many of the parts/actions are run by the server properly, but package installation seems to fail invariably:
#cloud-config
package-update: true
package-upgrade: true
# ...
packages:
- puppet3
- lvm2
- btrfs-progs
# ...
I see the following in the logs:
May 20 20:23:39 cloud-init[1252]: util.py[DEBUG]: Package update failed
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py", line 74, in handle
cloud.distro.update_package_sources()
File "/usr/lib/python2.6/site-packages/cloudinit/distros/rhel.py", line 278, in update_package_sources
["makecache"], freq=PER_INSTANCE)
File "/usr/lib/python2.6/site-packages/cloudinit/helpers.py", line 197, in run
results = functor(*args)
File "/usr/lib/python2.6/site-packages/cloudinit/distros/rhel.py", line 274, in package_command
util.subp(cmd, capture=False, pipe_cat=True, close_stdin=True)
File "/usr/lib/python2.6/site-packages/cloudinit/util.py", line 1529, in subp
cmd=args)
ProcessExecutionError: Unexpected error while running command.
Command: ['yum', '-t', '-y', 'makecache']
Exit code: 1
Reason: -
Stdout: ''
Stderr: ''
May 20 20:23:39 cloud-init[1252]: amazon.py[DEBUG]: Upgrade level: security
May 20 20:23:39 cloud-init[1252]: util.py[DEBUG]: Running command ['yum', '-t', '-y', '--exclude=kernel', '--exclude=nvidia*', '--exclude=cudatoolkit', '--security', '--sec-severity=critical', '--sec-severity=important', 'upgrade'] with allowed return codes [0] (shell=False, capture=False)
May 20 20:23:52 cloud-init[1252]: util.py[WARNING]: Package upgrade failed
May 20 20:23:52 cloud-init[1252]: util.py[DEBUG]: Package upgrade failed
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py", line 81, in handle
cloud.distro.upgrade_packages(upgrade_level, upgrade_exclude)
File "/usr/lib/python2.6/site-packages/cloudinit/distros/amazon.py", line 50, in upgrade_packages
return self.package_command('upgrade', args=args)
File "/usr/lib/python2.6/site-packages/cloudinit/distros/rhel.py", line 274, in package_command
util.subp(cmd, capture=False, pipe_cat=True, close_stdin=True)
File "/usr/lib/python2.6/site-packages/cloudinit/util.py", line 1529, in subp
cmd=args)
ProcessExecutionError: Unexpected error while running command.
Command: ['yum', '-t', '-y', '--exclude=kernel', '--exclude=nvidia*', '--exclude=cudatoolkit', '--security', '--sec-severity=critical', '--sec-severity=important', 'upgrade']
Exit code: 1
Reason: -
Stdout: ''
Stderr: ''
May 20 20:23:52 cloud-init[1252]: util.py[DEBUG]: Running command ['yum', '-t', '-y', 'install', 'puppet3', 'lvm2', 'btrfs-progs'] with allowed return codes [0] (shell=False, capture=False)
May 20 20:24:03 cloud-init[1252]: util.py[WARNING]: Failed to install packages: ['puppet3', 'lvm2', 'btrfs-progs']
May 20 20:24:03 cloud-init[1252]: util.py[DEBUG]: Failed to install packages: ['puppet3', 'lvm2', 'btrfs-progs']
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py", line 88, in handle
cloud.distro.install_packages(pkglist)
File "/usr/lib/python2.6/site-packages/cloudinit/distros/rhel.py", line 70, in install_packages
self.package_command('install', pkgs=pkglist)
File "/usr/lib/python2.6/site-packages/cloudinit/distros/rhel.py", line 274, in package_command
util.subp(cmd, capture=False, pipe_cat=True, close_stdin=True)
File "/usr/lib/python2.6/site-packages/cloudinit/util.py", line 1529, in subp
cmd=args)
ProcessExecutionError: Unexpected error while running command.
Command: ['yum', '-t', '-y', 'install', 'puppet3', 'lvm2', 'btrfs-progs']
Exit code: 1
Reason: -
Stdout: ''
Stderr: ''
May 20 20:24:03 cloud-init[1252]: cc_package_update_upgrade_install.py[WARNING]: 3 failed with exceptions, re-raising the last one
May 20 20:24:03 cloud-init[1252]: util.py[WARNING]: Running package-update-upgrade-install (<module 'cloudinit.config.cc_package_update_upgrade_install' from '/usr/lib/python2.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.pyc'>) failed
May 20 20:24:03 cloud-init[1252]: util.py[DEBUG]: Running package-update-upgrade-install (<module 'cloudinit.config.cc_package_update_upgrade_install' from '/usr/lib/python2.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.pyc'>) failed
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/cloudinit/stages.py", line 553, in _run_modules
cc.run(run_name, mod.handle, func_args, freq=freq)
File "/usr/lib/python2.6/site-packages/cloudinit/cloud.py", line 63, in run
return self._runners.run(name, functor, args, freq, clear_on_fail)
File "/usr/lib/python2.6/site-packages/cloudinit/helpers.py", line 197, in run
results = functor(*args)
File "/usr/lib/python2.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py", line 111, in handle
raise errors[-1]
ProcessExecutionError: Unexpected error while running command.
Command: ['yum', '-t', '-y', 'install', 'puppet3', 'lvm2', 'btrfs-progs']
Exit code: 1
Reason: -
Stdout: ''
Stderr: ''
When I actually run the command yum -t -y install puppet3 lvm2 btrfs-progs as root, it just runs fine, but Cloud Init is failing to run it on its own.
Is there something I'm doing wrong here?
Evidently there was a bug fixed with later versions of the image. If this is happening to you, it may be a legitimate bug in Cloud-Init or your servers implementation of it. A package upgrade may fix the problem, also an update to the image will fix it globally.

Calling Raspberry Pi temp sensor, invalid syntax when calling driver as bash script

I'm currently putting together a temperature sensor for my Raspberry Pi and ran into this issue. In order to run the driver designated in the code, I had to run it was a shell command. I'm having a difficult time figuring out why my syntax is incorrect.
Code:
def read_dht22( PiPin ):
output = subprocess.check_output([shell=True], ['/home/pi/Adafruit-Raspberry-Pi-Python-Code/Adafruit__DHT_Driver", "2302", str(PiPin)])
Error:
sudo python scr6.py
File "scr6.py", line 31
output = subprocess.check_output([shell=True], ['/home/pi/Adafruit-Raspberry-Pi-Python-Code/Adafruit_DHT_Driver', '2302', str(PiPin)])
^
SyntaxError: invalid syntax
Error Prior to Adding shell=True
sudo python scr5.py
Traceback (most recent call last):
File "scr5.py", line 46, in <module>
temp_c, temp_f = read_dht22(4)
File "scr5.py", line 31, in read_dht22
output = subprocess.check_output(["/home/pi/Adafruit-Raspberry-Pi-Python-Code/Adafruit_DHT_Driver", "2302", str(PiPin)])
File "/usr/lib/python2.7/subprocess.py", line 537, in check_output
process = Popen(stdout=PIPE, *popenargs, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1259, in _execute_child
raise child_exception
OSError: [Errno 13] Permission denied
This should fix your syntax error:
def read_dht22( PiPin ):
output = subprocess.check_output(["/home/pi/Adafruit-Raspberry-Pi-Python-Code/Adafruit__DHT_Driver", "2302", str(PiPin)], shell=True)

Error with DataStax Commuity on Amazon EC2

I got this Error if i want to Setup the Datastax Cassandra Cluster (Community Edition), i have used this:
http://www.datastax.com/docs/1.2/install/install_ami
[EXEC] 06/05/13-10:52:05 sudo rm -rf /etc/motd
[EXEC] 06/05/13-10:52:05 sudo touch /etc/motd
[INFO] Started with user data set to:
[INFO] --clustername myDSCcluster --totalnodes 3 --version community
[INFO] Using instance type: m1.large
[ERROR] Exception seen in ds1_launcher.py:
Traceback (most recent call last):
File "/home/ubuntu/datastax_ami/ds1_launcher.py", line 31, in initial_configurations
ds2_configure.run()
File "/home/ubuntu/datastax_ami/ds2_configure.py", line 932, in run
File "/home/ubuntu/datastax_ami/ds2_configure.py", line 125, in get_ec2_data
File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 406, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 519, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 444, in error
return self._call_chain(*args)
File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 527, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 404: Not Found
It seems as though you may have attempted to launch the cluster within a VPC, which isn't supported.
I did, however, notice that we don't specifically mention that VPC's aren't supported. I'll make sure we get that fixed.

Resources