I'm trying to run Apache Airflow using a container CentOS7. I am running using Python 3.6.9. I already install apache airflow using pip but when I try to setup the database using airflow initdb I get the following error:
(app-root) sh-4.2$ airflow version
Traceback (most recent call last):
File "/opt/app-root/bin/airflow", line 5, in <module>
from airflow.__main__ import main
File "/opt/app-root/lib/python3.6/site-packages/airflow/__init__.py", line 34, in <module>
from airflow import settings
File "/opt/app-root/lib/python3.6/site-packages/airflow/settings.py", line 37, in <module>
from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf # NOQA F401
File "/opt/app-root/lib/python3.6/site-packages/airflow/configuration.py", line 1007, in <module>
conf.validate()
File "/opt/app-root/lib/python3.6/site-packages/airflow/configuration.py", line 209, in validate
self._validate_config_dependencies()
File "/opt/app-root/lib/python3.6/site-packages/airflow/configuration.py", line 246, in _validate_config_dependencies
raise AirflowConfigException(f"error: cannot use sqlite version < {min_sqlite_version}")
airflow.exceptions.AirflowConfigException: error: cannot use sqlite version < 3.15.0
What Can I do to solve this problem?
You need to upgrade your sqlite version:
yum update
yum install sqlite
Example with docker run -it centos:latest:
[root#d09e5ee37969 /]# sqlite3 --version
3.26.0 2018-12-01 12:34:55 bf8c1b2b7a5960c282e543b9c293686dccff272512d08865f4600fb58238alt1
Airflow >=2.0.0 only works with Sqlite 3.15.0+
Related
I'm following Drake's installation instructions from here for ubuntu.
I get stuck at this line:
python3 -c 'import pydrake; print(pydrake.__file__)'
with the following error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/drake/lib/python3.6/site-packages/pydrake/__init__.py", line 41, in <module>
from . import common
File "/opt/drake/lib/python3.6/site-packages/pydrake/common/__init__.py", line 7, in <module>
from ._module_py import *
ModuleNotFoundError: No module named 'pydrake.common._module_py'
What is the ._module_py file? I don't see it in:
/opt/drake/lib/python3.6/site-packages/pydrake/common
The python version I've got installed is 3.8.5, but I don't think that's causing any problems.
The instructions you linked to say curl -o drake.tar.gz https://drake-packages.csail.mit.edu/drake/nightly/drake-latest-bionic.tar.gz. The "bionic" there is Ubuntu's codename for 18.04. If you are using Ubuntu 20.04, then the codename is "focal", so you should use start from curl -o drake.tar.gz https://drake-packages.csail.mit.edu/drake/nightly/drake-latest-focal.tar.gz so that you have Python 3.8 support.
Here are Drake's own installation instructions for pydrake, which are usually the most up-to-date reference: https://drake.mit.edu/python_bindings.html#installation
I'm getting below error while trying to run the linux image container on windows 10 in docker desktop.
Traceback (most recent call last):
File "app1.py", line 17, in <module>
import mysql.connector
File "/usr/local/lib/python3.8/site-packages/mysql/connector/__init__.py", line 35, in <module>
__version__ = _version.version
AttributeError: module '_version' has no attribute 'version'
I had the same problem on Ubuntu 16.04 using python3.5 with the package mysql_connector_repackaged installed via pip.
I solved by removing it and installing the package mysql-connector-python
pip3 install mysql-connector-python
Followed through installation tips, but after typing
$ python underactuated/src/double_pendulum/simulate.py
I get
ImportError: dynamic module does not define init function (init_module_py).
If I run python3 I get
ImportError: cannot import name 'FindResource'.
After typing $ python3 -c 'import pydrake; print(pydrake.__file__)' I get /opt/drake/lib/python3.6/site-packages/pydrake/__init__.py
Edit:
Full output of running python3
Traceback (most recent call last):
File "underactuated/src/double_pendulum/simulate.py", line 5, in <module>
from underactuated import FindResource, PlanarSceneGraphVisualizer
ImportError: cannot import name 'FindResource'
And full output of running python (without3):
Traceback (most recent call last):
File "underactuated/src/double_pendulum/simulate.py", line 1, in <module>
from pydrake.all import (AddMultibodyPlantSceneGraph,
File "/opt/drake/lib/python3.6/site-packages/pydrake/__init__.py", line 32, in <module>
from . import common
File "/opt/drake/lib/python3.6/site-packages/pydrake/common/__init__.py", line 3, in <module>
from ._module_py import *
ImportError: dynamic module does not define init function (init_module_py)
Version of python after typing python -V is Python 2.7.15+
The fact that the print test succeeds suggests that drake is installed correctly.
Please post the full output of running simulate.py with python3.
When I try to run the command in my terminal, it raises an error
I'm using anaconda to install my packages and python 2.7.
$ zipline ingest -b quandl
Traceback (most recent call last):
File "/anaconda3/bin/zipline", line 6, in <module>
from pkg_resources import load_entry_point
File "/anaconda3/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3241, in <module>
#_call_aside
File "/anaconda3/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3225, in _call_aside
f(*args, **kwargs)
File "/anaconda3/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3254, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/anaconda3/lib/python2.7/site-packages/pkg_resources/__init__.py", line 583, in _build_master
ws.require(__requires__)
File "/anaconda3/lib/python2.7/site-packages/pkg_resources/__init__.py", line 900, in require
needed = self.resolve(parse_requirements(requirements))
File "/anaconda3/lib/python2.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'mock>=2.0' distribution was not found and is required by tables
It seems you need to install the mock package:
conda install mock
or, if the previous doesn't work:
conda install -c conda-forge mock
I have a docker image with python 2.7. I am trying to install airflow in the image, but during the pip installation of the airflow dependencies I am getting the below issues.
Dockerfile:-
FROM python:2.7-slim
...
...
# Install Airflow and Celery
COPY pip_airflow.txt /pip_airflow.txt
RUN pip install --no-cache-dir -r /pip_airflow.txt
Below is my pip_airflow.txt file.
apache-airflow[all]==1.10.0
celery[redis]>=4.2.1,<4.3.0
flower>=0.7.3,<1.0
kombu>=4.2.0,<5.0
Logs:- The failure is when the pip install for airflow is happening from the dockerfile.
Collecting pendulum==1.4.4 (from apache-airflow[all]==1.10.0->-r /pip_airflow.txt (line 1))
Downloading https://files.pythonhosted.org/packages/85/a5/9fc15751f9725923b170ad37d6c61031fc9e941bafd5288ca6ee51233284/pendulum-1.4.4.tar.gz (74kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'error'
Complete output from command /usr/local/bin/python /usr/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpnYnATX:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 207, in <module>
main()
File "/usr/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 197, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/usr/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 54, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/pip-build-env-OXncUe/overlay/lib/python2.7/site-packages/setuptools/build_meta.py", line 130, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
File "/tmp/pip-build-env-OXncUe/overlay/lib/python2.7/site-packages/setuptools/build_meta.py", line 112, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-OXncUe/overlay/lib/python2.7/site-packages/setuptools/build_meta.py", line 126, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 47, in <module>
from build import *
File "/usr/local/lib/python2.7/site-packages/pip/_vendor/pep517/build.py", line 7, in <module>
from pip._vendor import pytoml
ImportError: No module named pip._vendor
Update pip, e.g. pip install pip===19.0.2.
In my case it worked.