Error while installing Fabric on OSX using recommended pip. Xcode is latest version - macos

I just tried to install Fabric on my Mac, and I was thrown this error after using pip install fabric
Installing collected packages: fabric, paramiko, pycrypto, ecdsa
Cleaning up...
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip-1.5-py2.7.egg/pip/basecommand.py", line 122, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/pip-1.5-py2.7.egg/pip/commands/install.py", line 275, in run
requirement_set.install(install_options, global_options, root=options.root_path)
File "/Library/Python/2.7/site-packages/pip-1.5-py2.7.egg/pip/req.py", line 1371, in install
requirement.install(install_options, global_options, *args, **kwargs)
File "/Library/Python/2.7/site-packages/pip-1.5-py2.7.egg/pip/req.py", line 655, in install
self.move_wheel_files(self.source_dir, root=root)
File "/Library/Python/2.7/site-packages/pip-1.5-py2.7.egg/pip/req.py", line 885, in move_wheel_files
pycompile=self.pycompile,
File "/Library/Python/2.7/site-packages/pip-1.5-py2.7.egg/pip/wheel.py", line 209, in move_wheel_files
clobber(source, lib_dir, True)
File "/Library/Python/2.7/site-packages/pip-1.5-py2.7.egg/pip/wheel.py", line 196, in clobber
os.makedirs(destsubdir)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/fabfile'
Storing debug log for failure in /Users/michaelsnowden/Library/Logs/pip.log
Here is the entire debug log
What does this mean, and why am I getting this error?

For a system wide pip install you need to do a sudo:
sudo pip install fabric
Since the system wide packages are available globally, they are written outside paths that have write permission for a normal user. In that case you need to do a sudo other permission to write to those folders would be denied to you.
When installed globally, you can invoke the Python interpreter anywhere in your system and import fabric would be available to you. Also, it would be available to any user of your system.

Related

Airflow on WSL2 Ubuntu: ValueError: Unable to configure handler 'processor'

I am aware that very similar questions have been asked previously but I have found these have tended to include Docker, which I am not using at this time, nor do I have installed. I am led to believe that WSL2 should be an alternative to Docker regarding running Airflow on Windows.
I am using WSL2 on my Windows 11 laptop and have installed Apache-Airflow from a tutorial from the following link:
https://coding-stream-of-consciousness.com/2018/11/06/apache-airflow-windows-10-install-ubuntu/
On WSL2 I have:
Python version 3.8.10
Pip version 20.0.2
Apache-Airflow 2.4.1 (I believe)
I have run the following commands (as per the tutorial) with no issue:
sudo apt-get install software-properties-common
sudo apt-add-repository universe
sudo apt-get update
sudo apt-get install python-pip
sudo pip install apache-airflow #I had path issues without 'sudo' command
But when I attempt to use the 'airflow' command in the WSL2 Ubuntu terminal, I am greeted with the following error:
$ airflow
Unable to load the config, contains a configuration error.
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/config.py", line 563, in configure
handler = self.configure_handler(handlers[name])
File "/usr/lib/python3.8/logging/config.py", line 744, in configure_handler
result = factory(**kwargs)
File "/usr/local/lib/python3.8/dist-packages/airflow/utils/log/file_processor_handler.py", line 45, in __init__
self.filename_template, self.filename_jinja_template = parse_template_string(filename_template)
File "/usr/local/lib/python3.8/dist-packages/airflow/utils/helpers.py", line 165, in parse_template_string
import jinja2
File "/usr/lib/python3/dist-packages/jinja2/__init__.py", line 33, in <module>
from jinja2.environment import Environment, Template
File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 15, in <module>
from jinja2 import nodes
File "/usr/lib/python3/dist-packages/jinja2/nodes.py", line 23, in <module>
from jinja2.utils import Markup
File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 656, in <module>
from markupsafe import Markup, escape, soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.8/dist-packages/markupsafe/__init__.py)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 5, in <module>
from airflow.__main__ import main
File "/usr/local/lib/python3.8/dist-packages/airflow/__init__.py", line 46, in <module>
settings.initialize()
File "/usr/local/lib/python3.8/dist-packages/airflow/settings.py", line 564, in initialize
LOGGING_CLASS_PATH = configure_logging()
File "/usr/local/lib/python3.8/dist-packages/airflow/logging_config.py", line 74, in configure_logging
raise e
File "/usr/local/lib/python3.8/dist-packages/airflow/logging_config.py", line 69, in configure_logging
dictConfig(logging_config)
File "/usr/lib/python3.8/logging/config.py", line 808, in dictConfig
dictConfigClass(config).configure()
File "/usr/lib/python3.8/logging/config.py", line 570, in configure
raise ValueError('Unable to configure handler '
ValueError: Unable to configure handler 'processor'
I have googled the ValueError extensively and can't find any clear solutions that don't involve Docker.
Any insight into the error would be much appreciated!
You should install airflow using constraints.
https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#installation-tools
This is the only way installation of airflow is guaranteed to work.

Conda info (and other operations) not working

I am trying to install geopandas and have therefore downloaded anaconda. However the anaconda prompt gives back an error when I try the command "conda install geopandas" (Installing geopandas with anaconda prompt causes ImportError). I discovered that conda won't install any package, nor will it create new environments. And, as it happens, will not even work for "conda info" (see below error report).
Looking through help suggestions on the internet I added both "C:\Users\Name\anaconda3" and "C:\Users\Name\anaconda3\Scripts" to the PATH System environment variable but it didn't resolve the issue.
Any help would be greatly appreciated.
>>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<
Traceback (most recent call last):
File "C:\Users\Francesco\anaconda3\lib\site-packages\conda\exceptions.py", line 1079, in __call__
return func(*args, **kwargs)
File "C:\Users\Francesco\anaconda3\lib\site-packages\conda\cli\main.py", line 84, in _main
exit_code = do_call(args, p)
File "C:\Users\Francesco\anaconda3\lib\site-packages\conda\cli\conda_argparse.py", line 83, in do_call
return getattr(module, func_name)(args, parser)
File "C:\Users\Francesco\anaconda3\lib\site-packages\conda\cli\main_info.py", line 316, in execute
info_dict = get_info_dict(args.system)
File "C:\Users\Francesco\anaconda3\lib\site-packages\conda\cli\main_info.py", line 164, in get_info_dict
pkgs_dirs=context.pkgs_dirs,
File "C:\Users\Francesco\anaconda3\lib\site-packages\conda\base\context.py", line 520, in pkgs_dirs
fixed_dirs += user_data_dir(APP_NAME, APP_NAME),
File "C:\Users\Francesco\anaconda3\lib\site-packages\conda\_vendor\appdirs.py", line 67, in user_data_dir
path = os.path.join(_get_win_folder(const), appauthor, appname)
File "C:\Users\Francesco\anaconda3\lib\site-packages\conda\_vendor\appdirs.py", line 284, in _get_win_folder_with_pywin32
from win32com.shell import shellcon, shell
ImportError: DLL load failed while importing shell: The specified module could not be found.
$ C:\Users\Francesco\anaconda3\Scripts\conda-script.py info
'''
I've experienced pretty much the same problem. I was able to narrow down the issue to pywin32 version 302.
Using version 301 worked for me.
Try: pip install pywin32==301 --force
See https://github.com/mhammond/pywin32/issues/1783

Getting error when trying to install apache-airflow on Mac. How can I fix this?

Error output below:
ronakvora:dtc ronakvora$ pip install apache-airflow
Installing build dependencies ... done
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/pendulum.egg-info
writing requirements to pip-egg-info/pendulum.egg-info/requires.txt
writing pip-egg-info/pendulum.egg-info/PKG-INFO
writing top-level names to pip-egg-info/pendulum.egg-info/top_level.txt
writing dependency_links to pip-egg-info/pendulum.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/pendulum.egg-info/SOURCES.txt'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-install-WFGcOd/pendulum/setup.py", line 50, in <module>
setup(**setup_kwargs)
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-build-env-HZt1xp/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 296, in run
self.find_sources()
File "/private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-build-env-HZt1xp/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 303, in find_sources
mm.run()
File "/private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-build-env-HZt1xp/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 534, in run
self.add_defaults()
File "/private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-build-env-HZt1xp/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 570, in add_defaults
sdist.add_defaults(self)
File "/private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-build-env-HZt1xp/lib/python2.7/site-packages/setuptools/command/py36compat.py", line 36, in add_defaults
self._add_defaults_ext()
File "/private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-build-env-HZt1xp/lib/python2.7/site-packages/setuptools/command/py36compat.py", line 119, in _add_defaults_ext
build_ext = self.get_finalized_command('build_ext')
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command
cmd_obj.ensure_finalized()
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
self.finalize_options()
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/build_ext.py", line 159, in finalize_options
self.include_dirs.append(py_include)
AttributeError: 'unicode' object has no attribute 'append'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/6x/xsb52c7936l38mmb9f7s268m0000gn/T/pip-install-WFGcOd/pendulum/
You are using pip version 18.1, however version 19.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
I had recently encountered a similar error on Airflow 1.10.2 and in my case it was related to incorrect version of pendulum
Run pip show pendulum
Name: pendulum
Version: 1.4.4
Summary: Python datetimes made easy.
...
If your version of pendulum is different from v1.4.4 just do a force-reinstall (Airflow 1.10.2 requires pendulum===1.4.4)
pip install --force-reinstall pendulum===1.4.4
References
Find which version of package is installed with pip
Installing specific package versions with pip
I didn't exactly solve the issue above, but just decided to switch to python3 and used pip3 install apache-airflow.

Unable to PIP install Python ftfy package

When I try to install ftfy here package using command,
pip install ftfy I am getting following error in the terminal:
Collecting ftfy
Using cached https://files.pythonhosted.org/packages/d5/60/7b2d5d41ad6c6ebf48316b1c55ccf7c60d126e07f6306be9f254a139bfd1/ftfy-5.4.1.tar.gz
Complete output from command python setup.py egg_info:
Download error on https://pypi.org/simple/pytest-runner/: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645) -- Some packages may not be found!
Couldn't find index page for 'pytest-runner' (maybe misspelled?)
Download error on https://pypi.org/simple/: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645) -- Some packages may not be found!
No local packages or working download links found for pytest-runner
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/f4/sb0f69y118l8_xvhflgrgsjnrl5c1r/T/pip-install-uxw5kcal/ftfy/setup.py", line 62, in <module>
'Documentation': 'http://ftfy.readthedocs.io',
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/setuptools/__init__.py", line 139, in setup
_install_setup_requires(attrs)
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/setuptools/__init__.py", line 134, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/setuptools/dist.py", line 514, in fetch_build_eggs
replace_conflicting=True,
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/pkg_resources/__init__.py", line 777, in resolve
replace_conflicting=replace_conflicting
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1060, in best_match
return self.obtain(req, installer)
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1072, in obtain
return installer(requirement)
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/setuptools/dist.py", line 581, in fetch_build_egg
return cmd.easy_install(req)
File "/Users/raja.charuvil/kattoor/ce-etl/venv/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 670, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pytest-runner')
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/f4/sb0f69y118l8_xvhflgrgsjnrl5c1r/T/pip-install-uxw5kcal/ftfy/
I am using Mac os and Python 3.5
I tried using solution from link here but that could not solve my issue.
The problem got resolved after I update pytest-runner package.
pip3 install pytest-runner --upgrade
Then
pip3 install ftfy

Installing GDAL on Google Cloud Datalab

I am having trouble installing GDAL on Google Cloud Datalab. When I run:
!pip install gdal
I get the following error
Collecting gdal
Using cached GDAL-2.2.4.tar.gz
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/GDAL.egg-info
writing pip-egg-info/GDAL.egg-info/PKG-INFO
writing top-level names to pip-egg-info/GDAL.egg-info/top_level.txt
writing dependency_links to pip-egg-info/GDAL.egg-
info/dependency_links.txt
writing manifest file 'pip-egg-info/GDAL.egg-info/SOURCES.txt'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-umhRKK/gdal/setup.py", line 342, in <module>
**extra )
File "/usr/local/lib/python2.7/dist-packages/setuptools/__init__.py",
line 129, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in
run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 278, in run
self.find_sources()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 293, in find_sources
mm.run()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 524, in run
self.add_defaults()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 560, in add_defaults
sdist.add_defaults(self)
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/py36compat.py", line 36, in add_defaults
self._add_defaults_ext()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/py36compat.py", line 119, in _add_defaults_ext
build_ext = self.get_finalized_command('build_ext')
File "/usr/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command
cmd_obj.ensure_finalized()
File "/usr/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
self.finalize_options()
File "/tmp/pip-build-umhRKK/gdal/setup.py", line 217, in finalize_options
self.gdaldir = self.get_gdal_config('prefix')
File "/tmp/pip-build-umhRKK/gdal/setup.py", line 191, in get_gdal_config
return fetch_config(option)
File "/tmp/pip-build-umhRKK/gdal/setup.py", line 144, in fetch_config
raise gdal_config_error, e""")
File "<string>", line 4, in <module>
__main__.gdal_config_error: [Errno 2] No such file or directory
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-umhRKK/gdal/
Same thing goes for :
!pip install gdal==2.2
or:
!pip install python-gdal
and a few other similar commands I have tried. The fix I discovered which was to update the setup-tools still gives me this problem after updating it.
What is the problem?
As per the Datalab documentation on how to add a Python library to Cloud Datalab, there are three approaches that you can follow to perform an installation of a third-party library in Cloud Datalab:
Use !pip install lib-name. This has obviously not worked for you, even by applying some of the troubleshooting fixes you have found after looking around.
Use the code in # Option 2 below to specify the command to run in the startup script of the Datalab instance, then proceed to restart it.
Use your own Docker customized container, inherited from the Datalab Docker container. I know this option is the most complex, but it may be the most appropriate one for packages that require a more flexible customization, such as the gdal one you are having issues with.
Script to be used in option 2:
#Option 2
%%bash
echo "pip install lib-name" >> /content/datalab/.config/startup.sh
cat /content/datalab/.config/startup.sh

Resources