Error output below:
ronakvora:dtc ronakvora$ pip install apache-airflow
Installing build dependencies ... done
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/pendulum.egg-info
writing requirements to pip-egg-info/pendulum.egg-info/requires.txt
writing pip-egg-info/pendulum.egg-info/PKG-INFO
writing top-level names to pip-egg-info/pendulum.egg-info/top_level.txt
writing dependency_links to pip-egg-info/pendulum.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/pendulum.egg-info/SOURCES.txt'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-install-WFGcOd/pendulum/setup.py", line 50, in <module>
setup(**setup_kwargs)
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-build-env-HZt1xp/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 296, in run
self.find_sources()
File "/private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-build-env-HZt1xp/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 303, in find_sources
mm.run()
File "/private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-build-env-HZt1xp/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 534, in run
self.add_defaults()
File "/private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-build-env-HZt1xp/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 570, in add_defaults
sdist.add_defaults(self)
File "/private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-build-env-HZt1xp/lib/python2.7/site-packages/setuptools/command/py36compat.py", line 36, in add_defaults
self._add_defaults_ext()
File "/private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-build-env-HZt1xp/lib/python2.7/site-packages/setuptools/command/py36compat.py", line 119, in _add_defaults_ext
build_ext = self.get_finalized_command('build_ext')
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command
cmd_obj.ensure_finalized()
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
self.finalize_options()
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/build_ext.py", line 159, in finalize_options
self.include_dirs.append(py_include)
AttributeError: 'unicode' object has no attribute 'append'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-install-WFGcOd/pendulum/
You are using pip version 18.1, however version 19.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
I had recently encountered a similar error on Airflow 1.10.2 and in my case it was related to incorrect version of pendulum
Run pip show pendulum
Name: pendulum
Version: 1.4.4
Summary: Python datetimes made easy.
...
If your version of pendulum is different from v1.4.4 just do a force-reinstall (Airflow 1.10.2 requires pendulum===1.4.4)
pip install --force-reinstall pendulum===1.4.4
References
Find which version of package is installed with pip
Installing specific package versions with pip
I didn't exactly solve the issue above, but just decided to switch to python3 and used pip3 install apache-airflow.
Related
I'm trying to install psychopy within pycharm but keep getting the following error. I have a Mac and I'm using PyCharm 2022.3.2 (Professional Edition).
Defaulting to user installation because normal site-packages is not writeable
Collecting PsychoPy
Using cached PsychoPy-2022.2.5.zip (26.1 MB)
Collecting requests[security]
Using cached requests-2.28.2-py3-none-any.whl (62 kB)
Collecting cryptography
Using cached cryptography-39.0.0.tar.gz (603 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /Library/Developer/CommandLineTools/usr/bin/python3 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /var/folders/93/tfwyd2_n7lj79pnxrfkcn_gw0000gn/T/tmpnxyuxd3l
cwd: /private/var/folders/93/tfwyd2_n7lj79pnxrfkcn_gw0000gn/T/pip-install-xg71migq/cryptography
Complete output (29 lines):
=============================DEBUG ASSISTANCE==========================
If you are seeing an error here please try the following to
successfully install cryptography:
Upgrade to the latest pip and try again. This will fix errors for most
users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
=============================DEBUG ASSISTANCE==========================
Traceback (most recent call last):
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
main()
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 114, in get_requires_for_build_wheel
return hook(config_settings)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/build_meta.py", line 146, in get_requires_for_build_wheel
return self._get_build_requires(
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/build_meta.py", line 127, in _get_build_requires
self.run_setup()
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/build_meta.py", line 142, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 18, in <module>
from setuptools_rust import RustExtension
File "/private/var/folders/93/tfwyd2_n7lj79pnxrfkcn_gw0000gn/T/pip-build-env-g0unt5yo/overlay/lib/python3.8/site-packages/setuptools_rust/__init__.py", line 1, in <module>
from .build import build_rust
File "/private/var/folders/93/tfwyd2_n7lj79pnxrfkcn_gw0000gn/T/pip-build-env-g0unt5yo/overlay/lib/python3.8/site-packages/setuptools_rust/build.py", line 23, in <module>
from setuptools.command.build import build as CommandBuild # type: ignore[import]
ModuleNotFoundError: No module named 'setuptools.command.build'
----------------------------------------
ERROR: Command errored out with exit status 1: /Library/Developer/CommandLineTools/usr/bin/python3 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /var/folders/93/tfwyd2_n7lj79pnxrfkcn_gw0000gn/T/tmpnxyuxd3l Check the logs for full command output.
WARNING: You are using pip version 20.2.3; however, version 22.3.1 is available.
You should consider upgrading via the '/Library/Developer/CommandLineTools/usr/bin/python3 -m pip install --upgrade pip' command.
I would massively appreciate any help, thanks!
I am aware that very similar questions have been asked previously but I have found these have tended to include Docker, which I am not using at this time, nor do I have installed. I am led to believe that WSL2 should be an alternative to Docker regarding running Airflow on Windows.
I am using WSL2 on my Windows 11 laptop and have installed Apache-Airflow from a tutorial from the following link:
https://coding-stream-of-consciousness.com/2018/11/06/apache-airflow-windows-10-install-ubuntu/
On WSL2 I have:
Python version 3.8.10
Pip version 20.0.2
Apache-Airflow 2.4.1 (I believe)
I have run the following commands (as per the tutorial) with no issue:
sudo apt-get install software-properties-common
sudo apt-add-repository universe
sudo apt-get update
sudo apt-get install python-pip
sudo pip install apache-airflow #I had path issues without 'sudo' command
But when I attempt to use the 'airflow' command in the WSL2 Ubuntu terminal, I am greeted with the following error:
$ airflow
Unable to load the config, contains a configuration error.
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/config.py", line 563, in configure
handler = self.configure_handler(handlers[name])
File "/usr/lib/python3.8/logging/config.py", line 744, in configure_handler
result = factory(**kwargs)
File "/usr/local/lib/python3.8/dist-packages/airflow/utils/log/file_processor_handler.py", line 45, in __init__
self.filename_template, self.filename_jinja_template = parse_template_string(filename_template)
File "/usr/local/lib/python3.8/dist-packages/airflow/utils/helpers.py", line 165, in parse_template_string
import jinja2
File "/usr/lib/python3/dist-packages/jinja2/__init__.py", line 33, in <module>
from jinja2.environment import Environment, Template
File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 15, in <module>
from jinja2 import nodes
File "/usr/lib/python3/dist-packages/jinja2/nodes.py", line 23, in <module>
from jinja2.utils import Markup
File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 656, in <module>
from markupsafe import Markup, escape, soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.8/dist-packages/markupsafe/__init__.py)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 5, in <module>
from airflow.__main__ import main
File "/usr/local/lib/python3.8/dist-packages/airflow/__init__.py", line 46, in <module>
settings.initialize()
File "/usr/local/lib/python3.8/dist-packages/airflow/settings.py", line 564, in initialize
LOGGING_CLASS_PATH = configure_logging()
File "/usr/local/lib/python3.8/dist-packages/airflow/logging_config.py", line 74, in configure_logging
raise e
File "/usr/local/lib/python3.8/dist-packages/airflow/logging_config.py", line 69, in configure_logging
dictConfig(logging_config)
File "/usr/lib/python3.8/logging/config.py", line 808, in dictConfig
dictConfigClass(config).configure()
File "/usr/lib/python3.8/logging/config.py", line 570, in configure
raise ValueError('Unable to configure handler '
ValueError: Unable to configure handler 'processor'
I have googled the ValueError extensively and can't find any clear solutions that don't involve Docker.
Any insight into the error would be much appreciated!
You should install airflow using constraints.
https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#installation-tools
This is the only way installation of airflow is guaranteed to work.
When I try to install ftfy here package using command,
pip install ftfy I am getting following error in the terminal:
Collecting ftfy
Using cached https://files.pythonhosted.org/packages/d5/60/7b2d5d41ad6c6ebf48316b1c55ccf7c60d126e07f6306be9f254a139bfd1/ftfy-5.4.1.tar.gz
Complete output from command python setup.py egg_info:
Download error on https://pypi.org/simple/pytest-runner/: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645) -- Some packages may not be found!
Couldn't find index page for 'pytest-runner' (maybe misspelled?)
Download error on https://pypi.org/simple/: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645) -- Some packages may not be found!
No local packages or working download links found for pytest-runner
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/f4/sb0f69y118l8_xvhflgrgsjnrl5c1r/T/pip-install-uxw5kcal/ftfy/setup.py", line 62, in <module>
'Documentation': 'http://ftfy.readthedocs.io',
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/setuptools/__init__.py", line 139, in setup
_install_setup_requires(attrs)
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/setuptools/__init__.py", line 134, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/setuptools/dist.py", line 514, in fetch_build_eggs
replace_conflicting=True,
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/pkg_resources/__init__.py", line 777, in resolve
replace_conflicting=replace_conflicting
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1060, in best_match
return self.obtain(req, installer)
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1072, in obtain
return installer(requirement)
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/setuptools/dist.py", line 581, in fetch_build_egg
return cmd.easy_install(req)
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 670, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pytest-runner')
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/f4/sb0f69y118l8_xvhflgrgsjnrl5c1r/T/pip-install-uxw5kcal/ftfy/
I am using Mac os and Python 3.5
I tried using solution from link here but that could not solve my issue.
The problem got resolved after I update pytest-runner package.
pip3 install pytest-runner --upgrade
Then
pip3 install ftfy
I am having trouble installing GDAL on Google Cloud Datalab. When I run:
!pip install gdal
I get the following error
Collecting gdal
Using cached GDAL-2.2.4.tar.gz
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/GDAL.egg-info
writing pip-egg-info/GDAL.egg-info/PKG-INFO
writing top-level names to pip-egg-info/GDAL.egg-info/top_level.txt
writing dependency_links to pip-egg-info/GDAL.egg-
info/dependency_links.txt
writing manifest file 'pip-egg-info/GDAL.egg-info/SOURCES.txt'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-umhRKK/gdal/setup.py", line 342, in <module>
**extra )
File "/usr/local/lib/python2.7/dist-packages/setuptools/__init__.py",
line 129, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in
run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 278, in run
self.find_sources()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 293, in find_sources
mm.run()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 524, in run
self.add_defaults()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 560, in add_defaults
sdist.add_defaults(self)
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/py36compat.py", line 36, in add_defaults
self._add_defaults_ext()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/py36compat.py", line 119, in _add_defaults_ext
build_ext = self.get_finalized_command('build_ext')
File "/usr/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command
cmd_obj.ensure_finalized()
File "/usr/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
self.finalize_options()
File "/tmp/pip-build-umhRKK/gdal/setup.py", line 217, in finalize_options
self.gdaldir = self.get_gdal_config('prefix')
File "/tmp/pip-build-umhRKK/gdal/setup.py", line 191, in get_gdal_config
return fetch_config(option)
File "/tmp/pip-build-umhRKK/gdal/setup.py", line 144, in fetch_config
raise gdal_config_error, e""")
File "<string>", line 4, in <module>
__main__.gdal_config_error: [Errno 2] No such file or directory
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-umhRKK/gdal/
Same thing goes for :
!pip install gdal==2.2
or:
!pip install python-gdal
and a few other similar commands I have tried. The fix I discovered which was to update the setup-tools still gives me this problem after updating it.
What is the problem?
As per the Datalab documentation on how to add a Python library to Cloud Datalab, there are three approaches that you can follow to perform an installation of a third-party library in Cloud Datalab:
Use !pip install lib-name. This has obviously not worked for you, even by applying some of the troubleshooting fixes you have found after looking around.
Use the code in # Option 2 below to specify the command to run in the startup script of the Datalab instance, then proceed to restart it.
Use your own Docker customized container, inherited from the Datalab Docker container. I know this option is the most complex, but it may be the most appropriate one for packages that require a more flexible customization, such as the gdal one you are having issues with.
Script to be used in option 2:
#Option 2
%%bash
echo "pip install lib-name" >> /content/datalab/.config/startup.sh
cat /content/datalab/.config/startup.sh
Until now, I used the core module of Ansible.
Now I would play also with Extra Module. Let's say that I want to use on my playbook: bigip_partition.
I've seen that it is in https://github.com/F5Networks/f5-ansible/tree/master/library
Could someone tell me how to install them?
I'm on Ubuntu 14LTS, Ansible 2.2
At the moment:
> sudo pip install bigsuds --upgrade pip
> sudo pip install f5-sdk
> sudo pip install git+https://github.com/F5Networks/f5-ansible.git -vvv
Collecting git+https://github.com/F5Networks/f5-ansible.git
Cloning https://github.com/F5Networks/f5-ansible.git to /tmp/pip-NjrWJw-build
Running command git clone -q https://github.com/F5Networks/f5-ansible.git /tmp/pip-NjrWJw-build
Running setup.py (path:/tmp/pip-NjrWJw-build/setup.py) egg_info for package from git+https://github.com/F5Networks/f5-ansible.git
Running command python setup.py egg_info
Traceback (most recent call last):
File "<string>", line 1, in <module>
IOError: [Errno 2] No such file or directory: '/tmp/pip-NjrWJw-build/setup.py'
Cleaning up...
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-NjrWJw-build/
Exception information:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 335, in run
wb.build(autobuilding=True)
File "/usr/local/lib/python2.7/dist-packages/pip/wheel.py", line 749, in build
self.requirement_set.prepare_files(self.finder)
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 634, in _prepare_file
abstract_dist.prep_for_dist()
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 129, in prep_for_dist
self.req_to_install.run_egg_info()
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 439, in run_egg_info
command_desc='python setup.py egg_info')
File "/usr/local/lib/python2.7/dist-packages/pip/utils/__init__.py", line 707, in call_subprocess
% (command_desc, proc.returncode, cwd))
InstallationError: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-NjrWJw-build/
Thanks,
Riccardo
https://github.com/F5Networks/f5-ansible is not a python package, don't try to install it with pip.
To use those modules you just need to put py-files from library to your playbook-path/library.
You'll have a much better experience if you install the 'App Services 3.0' (AS3) declarative interface on you BIG-IP:
https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/3/
Then you just send 'declarations' of your desired state, instead of working with MANY imperative interfaces...
Here are some example declarations (which are easy to generate/manipulate in Ansible using Jinja2 templates):
https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/3/userguide/examples.html