I am aware that very similar questions have been asked previously but I have found these have tended to include Docker, which I am not using at this time, nor do I have installed. I am led to believe that WSL2 should be an alternative to Docker regarding running Airflow on Windows.
I am using WSL2 on my Windows 11 laptop and have installed Apache-Airflow from a tutorial from the following link:
https://coding-stream-of-consciousness.com/2018/11/06/apache-airflow-windows-10-install-ubuntu/
On WSL2 I have:
Python version 3.8.10
Pip version 20.0.2
Apache-Airflow 2.4.1 (I believe)
I have run the following commands (as per the tutorial) with no issue:
sudo apt-get install software-properties-common
sudo apt-add-repository universe
sudo apt-get update
sudo apt-get install python-pip
sudo pip install apache-airflow #I had path issues without 'sudo' command
But when I attempt to use the 'airflow' command in the WSL2 Ubuntu terminal, I am greeted with the following error:
$ airflow
Unable to load the config, contains a configuration error.
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/config.py", line 563, in configure
handler = self.configure_handler(handlers[name])
File "/usr/lib/python3.8/logging/config.py", line 744, in configure_handler
result = factory(**kwargs)
File "/usr/local/lib/python3.8/dist-packages/airflow/utils/log/file_processor_handler.py", line 45, in __init__
self.filename_template, self.filename_jinja_template = parse_template_string(filename_template)
File "/usr/local/lib/python3.8/dist-packages/airflow/utils/helpers.py", line 165, in parse_template_string
import jinja2
File "/usr/lib/python3/dist-packages/jinja2/__init__.py", line 33, in <module>
from jinja2.environment import Environment, Template
File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 15, in <module>
from jinja2 import nodes
File "/usr/lib/python3/dist-packages/jinja2/nodes.py", line 23, in <module>
from jinja2.utils import Markup
File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 656, in <module>
from markupsafe import Markup, escape, soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.8/dist-packages/markupsafe/__init__.py)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 5, in <module>
from airflow.__main__ import main
File "/usr/local/lib/python3.8/dist-packages/airflow/__init__.py", line 46, in <module>
settings.initialize()
File "/usr/local/lib/python3.8/dist-packages/airflow/settings.py", line 564, in initialize
LOGGING_CLASS_PATH = configure_logging()
File "/usr/local/lib/python3.8/dist-packages/airflow/logging_config.py", line 74, in configure_logging
raise e
File "/usr/local/lib/python3.8/dist-packages/airflow/logging_config.py", line 69, in configure_logging
dictConfig(logging_config)
File "/usr/lib/python3.8/logging/config.py", line 808, in dictConfig
dictConfigClass(config).configure()
File "/usr/lib/python3.8/logging/config.py", line 570, in configure
raise ValueError('Unable to configure handler '
ValueError: Unable to configure handler 'processor'
I have googled the ValueError extensively and can't find any clear solutions that don't involve Docker.
Any insight into the error would be much appreciated!
You should install airflow using constraints.
https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#installation-tools
This is the only way installation of airflow is guaranteed to work.
Related
I am using Ubuntu 16.04 to work. There is a need from my work to upgrade pip.
I was using the command:
pip3 install --upgrade pip
After that, every time I call the command pip3 will get the same error:
File "/usr/bin/pip3", line 11, in <module>
sys.exit(main())
File "/home/my_user/.local/lib/python3.5/site-packages/pip/__init__.py", line 11, in main
from pip._internal.utils.entrypoints import _wrapper
File "/home/my_user/.local/lib/python3.5/site-packages/pip/_internal/utils/entrypoints.py", line 4, in <module>
from pip._internal.cli.main import main
File "/home/my_user/.local/lib/python3.5/site-packages/pip/_internal/cli/main.py", line 57
sys.stderr.write(f"ERROR: {exc}")
^
I've tried to search solution by myself and followed this to try to deal with the issue.
This is the command I tried:
curl -fsSL -o- https://bootstrap.pypa.io/pip/3.5/get-pip.py | python3
However, after I tried the command, I got another issue:
Traceback (most recent call last):
File "<stdin>", line 23974, in <module>
File "<stdin>", line 199, in main
File "<stdin>", line 121, in bootstrap
File "/home/my_user/.local/lib/python3.5/site-packages/setuptools/__init__.py", line 18, in <module>
from setuptools.dist import Distribution
File "/home/my_user/.local/lib/python3.5/site-packages/setuptools/dist.py", line 585
license_files: Optional[List[str]] = self.metadata.license_files
^
SyntaxError: invalid syntax
Is there any other way to fix this issue except remove and install python3.5 again? because the python3.5 is used in the Ubuntu 16.04 OS, if I remove it the system will break.
Thank you very much.
I have went through and installed CUDA, cuDNN, an followed the instructions to the best of my ability. I have added the environment variables I believe I need, but I still seem to have problems.
I have come as far as testing to see if tensorflow has been installed correctly. When pull up a command prompt, type python to use the shell, I type import tensorflow as tf.
I then get this stack trace, something I cannot make sense of to solve the problem myself. This is where I need the communities help:
>>> import tensorflow as tf
Traceback (most recent call last):
File "C:\Users\Troy\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\platform\self_check.py", line 75, in preload_check
ctypes.WinDLL(build_info.cudart_dll_name)
File"C:\Users\Troy\AppData\Local\Programs\Python\Python36\lib\ctypes\__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Troy\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\__init__.py", line 24, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "C:\Users\Troy\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "C:\Users\Troy\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 30, in <module>
self_check.preload_check()
File "C:\Users\Troy\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\platform\self_check.py", line 82, in preload_check
% (build_info.cudart_dll_name, build_info.cuda_version_number))
ImportError: Could not find 'cudart64_90.dll'. TensorFlow requires that this DLL be installed in a directory that is named in your %PATH% environment variable. Download and install CUDA 9.0 from this URL: https://developer.nvidia.com/cuda-toolkit
Please try installing CUDA 9.0. the problem should be with the CUDA 9.1 version. You can download the driver from the following link : CUDA Toolkit 9.0
To uninstall CUDA:
Please run the following commands :
sudo apt-get --purge remove cuda
sudo apt autoremove
These commands should be able to uninstall cuda from your system.
If you have the cuDNN configured to work with GPU:
You can remove them just my deleting the files in the directories that you have copied to, during its setup.
I'm getting the following error trying to run ipython3.
catch_config_error() missing 1 required positional argument: 'app'
I first got it on my Mac (python36). As my system has all sorts of customizations, I tried it again on a Linux system (Centos 7.4). I install ipython using pip3 and everything seemed to install cleanly. (On the same system, if i do this with python2, it seems to work.
Complete error log:
[scott] ipython3
Traceback (most recent call last):
File "/usr/bin/ipython3", line 7, in <module>
from IPython import start_ipython
File "/usr/lib/python3.4/site-packages/IPython/__init__.py", line 54, in <module>
from .core.application import Application
File "/usr/lib/python3.4/site-packages/IPython/core/application.py", line 23, in <module>
from traitlets.config.application import Application, catch_config_error
File "/usr/lib/python3.4/site-packages/traitlets/config/__init__.py", line 6, in <module>
from .application import *
File "/usr/lib/python3.4/site-packages/traitlets/config/application.py", line 120, in <module>
class Application(SingletonConfigurable):
File "/usr/lib/python3.4/site-packages/traitlets/config/application.py", line 291, in Application
def initialize(self, argv=None):
TypeError: catch_config_error() missing 1 required positional argument: 'app'
This is happening because traitlets relies on a package called 'decorator' that was just updated from 4.1.2 to 4.2.0 and broken its backward compatibility.
If you're using a requirements file or pip, install decorator==4.1.2 before installing IPython.
I tried in Ubuntu 16.04.
I also had the same problem (Python 3.6)
I tried again today.
$ sudo -H pip3 install --upgrade ipython
Then, decorator 4.2.1 is newly installed and the problem is solved.
I receive an error when trying to import zmq:
Traceback (most recent call last): File "BasicPub.py", line 1, in <module>
import zmq File "/home/przemek/Enthought/Canopy_32bit/User/lib/python2.7/site-packages/zmq/__init__.py", line 66, in <module>
from zmq import backend File "/home/przemek/Enthought/Canopy_32bit/User/lib/python2.7/site-packages/zmq/backend/__init__.py", line 40, in <module>
reraise(*exc_info) File "/home/przemek/Enthought/Canopy_32bit/User/lib/python2.7/site-packages/zmq/backend/__init__.py", line 27, in <module>
_ns = select_backend(first) File "/home/przemek/Enthought/Canopy_32bit/User/lib/python2.7/site-packages/zmq/backend/select.py", line 27, in select_backend
mod = __import__(name, fromlist=public_api) File "/home/przemek/Enthought/Canopy_32bit/User/lib/python2.7/site-packages/zmq/backend/cython/__init__.py", line 6, in <module>
from . import (constants, error, message, context, ImportError: /home/przemek/Enthought/Canopy_32bit/User/lib/python2.7/site-packages/zmq/backend/cython/message.so: undefined symbol: zmq_msg_gets
Someone has suggested that I should completely remove zmq and reinstall it.
I have tried to do it with pip but it seems that in some config files remain. I have used locate and I manually removed some directories. I thought that it would help but after reinstallation still I have an error. I am using Python2.7 and Ubuntu 14.04.
You can try the following:
sudo rm /usr/local/lib/libzmq*
sudo rm /usr/local/include/zmq.h
Since these are the two locations that ZeroMQ's libzmq installs into.
This link provides a few options on how to uninstall zeromq if you use a Debian operating system.
You would need to use the purge command:
sudo apt-get purge --auto-remove python-zmq
You probably installed it using the source ditribution.
To undo this, cd to the build directory and type:
make uninstall
I've recently installed pylint on a windows 7 and when I run it I get the error:
Error executing PyLint: Command failed with error=None, status=1; stderr:
Traceback (most recent call last):
File "C:\Python27\Scripts\pylint", line 3, in <module>
run_pylint()
File "c:\python27\lib\site-packages\pylint\__init__.py", line 20, in run_pylint
from pylint.lint import Run
File "c:\python27\lib\site-packages\pylint\lint.py", line 30, in <module>
from pylint.checkers import utils #pylint: disable=unused-import
File "c:\python27\lib\site-packages\pylint\checkers__init__.py", line 47, in
from pylint.reporters import diff_string
File "c:\python27\lib\site-packages\pylint\reporters__init__.py", line 20, in
from pylint.utils import MSG_TYPES
File "c:\python27\lib\site-packages\pylint\utils.py", line 32, in
from astroid import nodes, Module
ImportError: No module named astroid
I've followed the install instructions as best I can but can't seem to get past this.
Use this link to install pip properly How do I install pip on Windows?
then do the pip install pylint
then do go through this note http://docs.pylint.org/installation.html#note-for-windows-users