Airflow + MySQL安装躺坑整理

sql_alchemy_conn = mysql://airflow:airflow@hadoop001:3306/airflow

(2021) [root@hadoop001 airflow]# airflow db init
Traceback (most recent call last):
  File "/root/.virtualenvsi/2021/bin/airflow", line 5, in <module>
    from airflow.__main__ import main
  File "/root/.virtualenvsi/2021/lib/python3.9/site-packages/airflow/__init__.py", line 46, in <module>
    settings.initialize()
  File "/root/.virtualenvsi/2021/lib/python3.9/site-packages/airflow/settings.py", line 437, in initialize
    configure_orm()
  File "/root/.virtualenvsi/2021/lib/python3.9/site-packages/airflow/settings.py", line 224, in configure_orm
    engine = create_engine(SQL_ALCHEMY_CONN, connect_args=connect_args, **engine_args)
  File "/root/.virtualenvsi/2021/lib/python3.9/site-packages/sqlalchemy/engine/__init__.py", line 525, in create_engine
    return strategy.create(*args, **kwargs)
  File "/root/.virtualenvsi/2021/lib/python3.9/site-packages/sqlalchemy/engine/strategies.py", line 87, in create
    dbapi = dialect_cls.dbapi(**dbapi_args)
  File "/root/.virtualenvsi/2021/lib/python3.9/site-packages/sqlalchemy/dialects/mysql/mysqldb.py", line 118, in dbapi
    return __import__("MySQLdb")
ModuleNotFoundError: No module named 'MySQLdb'

环境是pyhton3.9 ,比较高，然后需要明显是需要安装mysqlclient
我是用whl 文件没能安装成功
又到pypi.org 下载了mysqlclient-2.0.3.tar.gz
解压，进去，python setup.py install 安装时候又报错

[root@hadoop001 mysqlclient-2.0.3]# python setup.py install
mysql_config --version
/bin/sh: mysql_config: command not found
mariadb_config --version
/bin/sh: mariadb_config: command not found
mysql_config --libs
/bin/sh: mysql_config: command not found
Traceback (most recent call last):
  File "/root/mysqlclient-2.0.3/setup.py", line 15, in <module>
    metadata, options = get_config()
  File "/root/mysqlclient-2.0.3/setup_posix.py", line 70, in get_config
    libs = mysql_config("libs")
  File "/root/mysqlclient-2.0.3/setup_posix.py", line 31, in mysql_config
    raise OSError("{} not found".format(_mysql_config_path))
OSError: mysql_config not found

报错提示：mysql_config not found
centos 安装 yum install libmysqlclient-dev
再次安装，

Installed /root/.virtualenvsi/2021/lib/python3.9/site-packages/mysqlclient-2.0.3-py3.9-linux-x86_64.egg
Processing dependencies for mysqlclient==2.0.3
Finished processing dependencies for mysqlclient==2.0.3

再次： airflow db init

Exception: Global variable explicit_defaults_for_timestamp needs to be on (1) for mysql

根据报错信息，继续解决
vim /etc/my.cnf
在[mysqld]下面添加配置
explicit_defaults_for_timestamp=1

systemctl start mysqld

再次： airflow db init

INFO  [airflow.models.dagbag.DagBag] Filling up the DagBag from /root/airflow/dags
Initialization done

在这里插入图片描述

成功！
创建用户 airflow users create -r Admin -u admin -e [email protected] -f admin -l user -p admin
启动webserver
在这里插入图片描述

进来了
在这里插入图片描述

发现调度服务没有启：
airflow scheduler > /console/scheduler_log.log &

Airflow + MySQL安装躺坑整理

猜你喜欢