I am trying to install the following requirements with pip:
coverage>=4.4.0
hypothesis>=4.24.3
ipywidgets>=7.3.0
jupyter
matplotlib>=2.1
pillow>=4.2.1
pycodestyle
pydot
astroid==2.5
pylint==2.7.1
stestr>=2.0.0
PyGithub
wheel
cython>=0.27.1
pylatexenc>=1.4
ddt>=1.2.0,!=1.4.0
seaborn>=0.9.0
reno>=3.2.0
Sphinx>=1.8.3,<3.1.0
qiskit-sphinx-theme>=1.6
sphinx-autodoc-typehints
jupyter-sphinx
sphinx-panels
pygments>=2.4
tweedledum==0.1b0
networkx>=2.2
scikit-learn>=0.20.0
scikit-quant;platform_system != 'Windows'
jax;platform_system != 'Windows'
jaxlib;platform_system != 'Windows'
At SQNomad I get the following error:
Downloading SQNomad-0.1.0.tar.gz (385 kB)
|████████████████████████████████| 385 kB 5.6 MB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... error
ERROR: Command errored out with exit status 1:
command: /Users/jim-felixlobsien/opt/anaconda3/bin/python /Users/jim-felixlobsien/opt/anaconda3/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/tmpta1cj_q6
cwd: /private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-install-pw0tuweh/SQNomad
Complete output (55 lines):
running dist_info
creating /private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-modern-metadata-j_5tehl8/SQNomad.egg-info
writing /private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-modern-metadata-j_5tehl8/SQNomad.egg-info/PKG-INFO
writing dependency_links to /private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-modern-metadata-j_5tehl8/SQNomad.egg-info/dependency_links.txt
writing requirements to /private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-modern-metadata-j_5tehl8/SQNomad.egg-info/requires.txt
writing top-level names to /private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-modern-metadata-j_5tehl8/SQNomad.egg-info/top_level.txt
writing manifest file '/private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-modern-metadata-j_5tehl8/SQNomad.egg-info/SOURCES.txt'
Traceback (most recent call last):
File "/Users/jim-felixlobsien/opt/anaconda3/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
main()
File "/Users/jim-felixlobsien/opt/anaconda3/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/Users/jim-felixlobsien/opt/anaconda3/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 133, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
File "/private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-build-env-8qqwe_v9/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 161, in prepare_metadata_for_build_wheel
self.run_setup()
File "/private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-build-env-8qqwe_v9/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 253, in run_setup
super(_BuildMetaLegacyBackend,
File "/private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-build-env-8qqwe_v9/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 145, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 77, in <module>
setup(
File "/private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-build-env-8qqwe_v9/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/Users/jim-felixlobsien/opt/anaconda3/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/Users/jim-felixlobsien/opt/anaconda3/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/Users/jim-felixlobsien/opt/anaconda3/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-build-env-8qqwe_v9/overlay/lib/python3.8/site-packages/setuptools/command/dist_info.py", line 31, in run
egg_info.run()
File "/private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-build-env-8qqwe_v9/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 299, in run
self.find_sources()
File "/private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-build-env-8qqwe_v9/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 306, in find_sources
mm.run()
File "/private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-build-env-8qqwe_v9/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 541, in run
self.add_defaults()
File "/private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-build-env-8qqwe_v9/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 577, in add_defaults
sdist.add_defaults(self)
File "/Users/jim-felixlobsien/opt/anaconda3/lib/python3.8/distutils/command/sdist.py", line 228, in add_defaults
self._add_defaults_ext()
File "/Users/jim-felixlobsien/opt/anaconda3/lib/python3.8/distutils/command/sdist.py", line 311, in _add_defaults_ext
build_ext = self.get_finalized_command('build_ext')
File "/Users/jim-felixlobsien/opt/anaconda3/lib/python3.8/distutils/cmd.py", line 299, in get_finalized_command
cmd_obj.ensure_finalized()
File "/Users/jim-felixlobsien/opt/anaconda3/lib/python3.8/distutils/cmd.py", line 107, in ensure_finalized
self.finalize_options()
File "/private/var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/pip-build-env-8qqwe_v9/overlay/lib/python3.8/site-packages/numpy/distutils/command/build_ext.py", line 86, in finalize_options
self.set_undefined_options('build',
File "/Users/jim-felixlobsien/opt/anaconda3/lib/python3.8/distutils/cmd.py", line 290, in set_undefined_options
setattr(self, dst_option, getattr(src_cmd_obj, src_option))
File "/Users/jim-felixlobsien/opt/anaconda3/lib/python3.8/distutils/cmd.py", line 103, in __getattr__
raise AttributeError(attr)
AttributeError: cpu_baseline
----------------------------------------
ERROR: Command errored out with exit status 1: /Users/jim-felixlobsien/opt/anaconda3/bin/python /Users/jim-felixlobsien/opt/anaconda3/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/h4/1z5lkc1x4nb2s31y0f6y9r200000gn/T/tmpta1cj_q6 Check the logs for full command output.
Does anybody knows what is happening? I am trying to follow the constructions in the following video:
https://www.youtube.com/watch?v=QjZdvNgYl3s&t=731s
The task is to contribute to qiskit!
Thank you very much in advance
The problem comes from a dependency in scikit-quant. The most direct way to fix this issue is installing the previous (0.7) version of scikit-quant instead of the current one 0.8, as Qiskit seems compatible to it:
pip install 'scikit-quant==0.7'
Fixed with SQNomad 0.2.1; thanks to luciano for reporting.
But it's now also made an optional component of scikit-quant, to install with scikit-quant[NOMAD] if desired. It's quite a large C++ library and needs to be split up between its Python dependent and Python independent parts to prevent creating massive wheels, then the installation will be easier and it's optional status can be reverted.
All optimizers can in any case be installed and used independently (e.g. with python -m pip install SQNomad for NOMAD).
Related
I'm new to poetry and pyenv, and I'm trying to make them work together on my django project.
I first when for poetry with my system python and it all worked well.
poetry init
poetry add django djangorestframework whitenoise gunicorn django-heroku
After that, I tried to install python 3.10.8 since it is the preferred version for Heroku.
And I used pyenv for that:
pyenv install 3.10.8
poetry env use ~/.pyenv/versions/3.10.8/bin/python
After that, I can see two env in the folder:
$ poetry env list
paint-projects-yd5RS8WI-py3.10 (Activated)
paint-projects-yd5RS8WI-py3.8
I removed pyproject.toml and poetry.lock files to start fresh.
Poetry init when well, but when I tried to add dependencies I had problems with psycopg2.
Trying to solve the issue I tried installing sudo apt install libpq-dev but it was already installed.
Adding poetry add psycopg2-binary went well, but I still have issues.
Here is the full error tracer:
$ poetry add django djangorestframework whitenoise gunicorn django-heroku
Using version ^4.1.2 for Django
Using version ^3.14.0 for djangorestframework
Using version ^6.2.0 for whitenoise
Using version ^20.1.0 for gunicorn
Using version ^0.3.1 for django-heroku
Updating dependencies
Resolving dependencies... (0.5s)
Writing lock file
Package operations: 4 installs, 0 updates, 0 removals
• Installing psycopg2 (2.9.5): Failed
CalledProcessError
Command '['/home/miguel/.cache/pypoetry/virtualenvs/paint-projects-yd5RS8WI-py3.10/bin/python', '-m', 'pip', 'install', '--use-pep517', '--disable-pip-version-check', '--prefix', '/home/miguel/.cache/pypoetry/virtualenvs/paint-projects-yd5RS8WI-py3.10', '--no-deps', '/home/miguel/.cache/pypoetry/artifacts/76/35/37/5c9e40406173ab076bef7fe83c7f5488acf226922bd1155bdf857b9b92/psycopg2-2.9.5.tar.gz']' returned non-zero exit status 1.
at /usr/lib/python3.8/subprocess.py:516 in run
512│ # We don't call process.wait() as .__exit__ does that for us.
513│ raise
514│ retcode = process.poll()
515│ if check and retcode:
→ 516│ raise CalledProcessError(retcode, process.args,
517│ output=stdout, stderr=stderr)
518│ return CompletedProcess(process.args, retcode, stdout, stderr)
519│
520│
The following error occurred when trying to handle this error:
EnvCommandError
Command ['/home/miguel/.cache/pypoetry/virtualenvs/paint-projects-yd5RS8WI-py3.10/bin/python', '-m', 'pip', 'install', '--use-pep517', '--disable-pip-version-check', '--prefix', '/home/miguel/.cache/pypoetry/virtualenvs/paint-projects-yd5RS8WI-py3.10', '--no-deps', '/home/miguel/.cache/pypoetry/artifacts/76/35/37/5c9e40406173ab076bef7fe83c7f5488acf226922bd1155bdf857b9b92/psycopg2-2.9.5.tar.gz'] errored with the following return code 1, and output:
Processing /home/miguel/.cache/pypoetry/artifacts/76/35/37/5c9e40406173ab076bef7fe83c7f5488acf226922bd1155bdf857b9b92/psycopg2-2.9.5.tar.gz
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [64 lines of output]
/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
warnings.warn(msg, warning_class)
running dist_info
creating /tmp/pip-modern-metadata-gpkmbva3/psycopg2.egg-info
writing /tmp/pip-modern-metadata-gpkmbva3/psycopg2.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-modern-metadata-gpkmbva3/psycopg2.egg-info/dependency_links.txt
writing top-level names to /tmp/pip-modern-metadata-gpkmbva3/psycopg2.egg-info/top_level.txt
writing manifest file '/tmp/pip-modern-metadata-gpkmbva3/psycopg2.egg-info/SOURCES.txt'
reading manifest file '/tmp/pip-modern-metadata-gpkmbva3/psycopg2.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file '/tmp/pip-modern-metadata-gpkmbva3/psycopg2.egg-info/SOURCES.txt'
creating '/tmp/pip-modern-metadata-gpkmbva3/psycopg2-2.9.5.dist-info'
Traceback (most recent call last):
File "/home/miguel/.cache/pypoetry/virtualenvs/paint-projects-yd5RS8WI-py3.10/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 351, in <module>
main()
File "/home/miguel/.cache/pypoetry/virtualenvs/paint-projects-yd5RS8WI-py3.10/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 333, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/miguel/.cache/pypoetry/virtualenvs/paint-projects-yd5RS8WI-py3.10/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 152, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
File "/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 377, in prepare_metadata_for_build_wheel
self.run_setup()
File "/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 483, in run_setup
super(_BuildMetaLegacyBackend,
File "/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup
exec(code, locals())
File "<string>", line 550, in <module>
File "/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
return distutils.core.setup(**attrs)
File "/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
self.run_command(cmd)
File "/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/setuptools/command/dist_info.py", line 101, in run
bdist_wheel = self.get_finalized_command('bdist_wheel')
File "/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 305, in get_finalized_command
cmd_obj = self.distribution.get_command_obj(command, create)
File "/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 859, in get_command_obj
klass = self.get_command_class(command)
File "/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 954, in get_command_class
self.cmdclass[command] = cmdclass = ep.load()
File "/home/miguel/.pyenv/versions/3.10.8/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
module = import_module(match.group('module'))
File "/home/miguel/.pyenv/versions/3.10.8/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 26, in <module>
from .macosx_libfile import calculate_macosx_platform_tag
File "/tmp/pip-build-env-2ln2ngiu/overlay/lib/python3.10/site-packages/wheel/macosx_libfile.py", line 41, in <module>
import ctypes
File "/home/miguel/.pyenv/versions/3.10.8/lib/python3.10/ctypes/__init__.py", line 8, in <module>
from _ctypes import Union, Structure, Array
ModuleNotFoundError: No module named '_ctypes'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
at ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/env.py:1476 in _run
1472│ output = subprocess.check_output(
1473│ command, stderr=subprocess.STDOUT, env=env, **kwargs
1474│ )
1475│ except CalledProcessError as e:
→ 1476│ raise EnvCommandError(e, input=input_)
1477│
1478│ return decode(output)
1479│
1480│ def execute(self, bin: str, *args: str, **kwargs: Any) -> int:
The following error occurred when trying to handle this error:
PoetryException
Failed to install /home/miguel/.cache/pypoetry/artifacts/76/35/37/5c9e40406173ab076bef7fe83c7f5488acf226922bd1155bdf857b9b92/psycopg2-2.9.5.tar.gz
at ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/pip.py:51 in pip_install
47│
48│ try:
49│ return environment.run_pip(*args)
50│ except EnvCommandError as e:
→ 51│ raise PoetryException(f"Failed to install {path.as_posix()}") from e
52│
Is there anything I need to consider when using poetry and pyenv?
Unable to test rule in elastic, I am running following command in terminal
elastalert-test-rule --config config.yaml example_rules/example_frequency.yaml
File "/usr/local/bin/elastalert-test-rule", line 11, in <module>
load_entry_point('elastalert==0.2.4', 'console_scripts', 'elastalert-test-rule')()
File "/usr/local/lib/python3.6/dist-packages/elastalert-0.2.4-py3.6.egg/elastalert/test_rule.py", line 445, in main
test_instance.run_rule_test()
File "/usr/local/lib/python3.6/dist-packages/elastalert-0.2.4-py3.6.egg/elastalert/test_rule.py", line 437, in run_rule_test
self.run_elastalert(rule_yaml, conf, args)
File "/usr/local/lib/python3.6/dist-packages/elastalert-0.2.4-py3.6.egg/elastalert/test_rule.py", line 307, in run_elastalert
client = ElastAlerter(['--debug'])
File "/usr/local/lib/python3.6/dist-packages/elastalert-0.2.4-py3.6.egg/elastalert/elastalert.py", line 173, in __init__
if not self.init_rule(rule):
File "/usr/local/lib/python3.6/dist-packages/elastalert-0.2.4-py3.6.egg/elastalert/elastalert.py", line 1038, in init_rule
jitter=5)
File "/usr/local/lib/python3.6/dist-packages/APScheduler-3.6.3-py3.6.egg/apscheduler/schedulers/base.py", line 420, in add_job
'trigger': self._create_trigger(trigger, trigger_args),
File "/usr/local/lib/python3.6/dist-packages/APScheduler-3.6.3-py3.6.egg/apscheduler/schedulers/base.py", line 921, in _create_trigger
return self._create_plugin_instance('trigger', trigger, trigger_args)
File "/usr/local/lib/python3.6/dist-packages/APScheduler-3.6.3-py3.6.egg/apscheduler/schedulers/base.py", line 906, in _create_plugin_instance
return plugin_cls(**constructor_kwargs)
File "/usr/local/lib/python3.6/dist-packages/APScheduler-3.6.3-py3.6.egg/apscheduler/triggers/interval.py", line 38, in __init__
self.timezone = astimezone(timezone)
File "/usr/local/lib/python3.6/dist-packages/APScheduler-3.6.3-py3.6.egg/apscheduler/util.py", line 93, in astimezone
raise TypeError('Only timezones from the pytz library are supported')
TypeError: Only timezones from the pytz library are supported
I have done following steps :
sudo apt-get update -y
sudo apt-get install -y python3-tzlocal
Also,
Added 'tzlocal<3.0', to setup.py
But after all this also I am getting the same error.
Please help!
You may try running setup again :
python3 setup.py install
I'm trying to install the transformers library on HPC. I do:
git clone https://github.com/huggingface/transformers.git
cd transformers
pip install -e . --user
All three of these work as expected, with the last output being:
Successfully installed dataclasses-0.7 numpy-1.19.0 tokenizers-0.8.1rc2 transformers
Then, I try python -c "import transformers" but I get the following error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/maths/btech/mt1170727/transformers/src/transformers/__init__.py", line 23, in <module>
from .configuration_albert import ALBERT_PRETRAINED_CONFIG_ARCHIVE_MAP, AlbertConfig
File "/home/maths/btech/mt1170727/transformers/src/transformers/configuration_albert.py", line 18, in <module>
from .configuration_utils import PretrainedConfig
File "/home/maths/btech/mt1170727/transformers/src/transformers/configuration_utils.py", line 25, in <module>
from .file_utils import CONFIG_NAME, cached_path, hf_bucket_url, is_remote_url
File "/home/maths/btech/mt1170727/transformers/src/transformers/file_utils.py", line 37, in <module>
import torch
File "/home/soft/PYTHON/3.6.0/ucs4/gnu/447/lib/python3.6/site-packages/torch/__init__.py", line 125, in <module>
_load_global_deps()
File "/home/soft/PYTHON/3.6.0/ucs4/gnu/447/lib/python3.6/site-packages/torch/__init__.py", line 83, in _load_global_deps
ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
File "/home/soft/PYTHON/3.6.0/ucs4/gnu/447/lib/python3.6/ctypes/__init__.py", line 344, in __init__
self._handle = _dlopen(self._name, mode)
OSError: libnvToolsExt.so.1: cannot open shared object file: No such file or directory
I have done as was written in the documentation, and can't see why I'm facing this error. Any help would be great. Thanks...
Good morning everyone,
First of all, I wanted to make it very clear that I checked if this question had been asked before and if it had been answered I would have solved the question.
I found this:
ModuleNotFoundError: No module named 'Scrapy'
That has been done for more than a year and the answers do not solve any problem related to me as the person who wrote it.
Something is wrong with my MacOs I think. When I wrote this code like on https://scrapy.org it doesn't work at all, after installing python3 and scrapy
Using
brew install python3
and for scrapy
pip install scrapy
Then I use the code from scrapy website:
import scrapy
class BlogSpider(scrapy.Spider):
name = 'blogspider'
start_urls = ['https://fr.wikipedia.org/wiki/Catégorie:Personnage_d%27animation']
def parse(self, response):
for title in response.css('div#mw-pages div.mw-category li'):
yield {'character': title.css('a ::text').extract_first()}
And then I receive 2 type of messages:
If I run the code:
scrapy runspider myspider.py
I receive the message:
bash: scrapy: command not found
And if I open python3 and just write:
import scrapy
I receive this message:
ModuleNotFoundError: No module named 'scrapy'
Someone can help me fixing this, because i don't remember how can I do it I already uninstall both python and scrapy but nothing change.
then i used:
pip3 install scrapy
As DavidG told me told try
but I receive this message:
-bash: pip3: command not found
So after a few more searches, I just tried reinstalling Python3, so I used:
brew install python3
so I wrote:
pip3 --version
and showed this:
pip 10.0.1 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)
Next:
scrapy --version
and received:
Scrapy 1.5.1 - no active project
so I tried to use the command again:
scrapy runspider myspider.py
and received this:
2018-07-18 20:11:49 [scrapy.utils.log] INFO: Scrapy 1.5.1 started (bot: scrapybot)
2018-07-18 20:11:49 [scrapy.utils.log] INFO: Versions: lxml 4.2.3.0, libxml2 2.9.4, cssselect 1.0.3, parsel 1.5.0,w3lib 1.19.0, Twisted 18.7.0, Python 3.7.0 (default, Jun 29 2018, 20:13:13) - [Clang 9.1.0 (clang-902.0.39.2)], pyOpenSSL 18.0.0 (OpenSSL 1.1.0h 27 Mar 2018), cryptography 2.3, Platform Darwin-17.6.0-x86_64-i386-64bit
2018-07-18 20:11:49 [scrapy.crawler] INFO: Overridden settings: {'SPIDER_LOADER_WARN_ONLY': True}
Traceback (most recent call last):
File "/usr/local/bin/scrapy", line 11, in <module>
sys.exit(execute())
File "/usr/local/lib/python3.7/site-packages/scrapy/cmdline.py", line 150, in execute
_run_print_help(parser, _run_command, cmd, args, opts)
File "/usr/local/lib/python3.7/site-packages/scrapy/cmdline.py", line 90, in _run_print_help
func(*a, **kw)
File "/usr/local/lib/python3.7/site-packages/scrapy/cmdline.py", line 157, in _run_command
cmd.run(args, opts)
File "/usr/local/lib/python3.7/site-packages/scrapy/commands/runspider.py", line 88, in run
self.crawler_process.crawl(spidercls, **opts.spargs)
File "/usr/local/lib/python3.7/site-packages/scrapy/crawler.py", line 170, in crawl
crawler = self.create_crawler(crawler_or_spidercls)
File "/usr/local/lib/python3.7/site-packages/scrapy/crawler.py", line 198, in create_crawler
return self._create_crawler(crawler_or_spidercls)
File "/usr/local/lib/python3.7/site-packages/scrapy/crawler.py", line 203, in _create_crawler
return Crawler(spidercls, self.settings)
File "/usr/local/lib/python3.7/site-packages/scrapy/crawler.py", line 55, in __init__
self.extensions = ExtensionManager.from_crawler(self)
File "/usr/local/lib/python3.7/site-packages/scrapy/middleware.py", line 58, in from_crawler
return cls.from_settings(crawler.settings, crawler)
File "/usr/local/lib/python3.7/site-packages/scrapy/middleware.py", line 34, in from_settings
mwcls = load_object(clspath)
File "/usr/local/lib/python3.7/site-packages/scrapy/utils/misc.py", line 44, in load_object
mod = import_module(module)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usr/local/lib/python3.7/site-packages/scrapy/extensions/telnet.py", line 12, in <module>
from twisted.conch import manhole, telnet
File "/usr/local/lib/python3.7/site-packages/twisted/conch/manhole.py", line 154
def write(self, data, async=False):
^
SyntaxError: invalid syntax
Can someone tell me what's going on and how to correct it?
So after the help and explanation of:
#DavidG
#phd
#DanielR.Livingston
I figured it was not a problem with my computer, but it was all a compatibility issue.
It does not solve my problem, because this was an exercise I found in a MOOC, but since it has been done for some time it does not work in the same way, but at least I can move on to a different course since I can not do more anything with the scrapy.
Thank you all
Running DCOS 1.8 on Centos.
I installed the CLI as below:
https://docs.mesosphere.com/1.8/usage/cli/install/
When I try to do a spark install I get the below error. Any ideas?
./dcos package install spark
ip-172-16-6-6.localdomain's username: admin
admin#ip-172-16-6-6.localdomain's password:
Traceback (most recent call last):
File "cli/dcoscli/subcommand.py", line 99, in run_and_capture
File "cli/dcoscli/package/main.py", line 21, in main
File "cli/dcoscli/util.py", line 21, in wrapper
File "cli/dcoscli/package/main.py", line 35, in _main
File "dcos/cmds.py", line 43, in execute
File "cli/dcoscli/package/main.py", line 356, in _install
File "dcos/cosmospackage.py", line 191, in get_package_version
File "dcos/cosmospackage.py", line 366, in __init__
File "cli/env/lib/python3.4/site-packages/requests/models.py", line 826, in json
File "json/__init__.py", line 318, in loads
File "json/decoder.py", line 343, in decode
File "json/decoder.py", line 361, in raw_decode
ValueError: Expecting value: line 1 column 1 (char 0)
I just had this issue to. For me it was because I forgot to install virtualenv. pip install virtualenv.