Airflow 1.10.12 : webserver gunicorn issue - Arguments to config option are malformed: python:airflow.www.gunicorn_config #11291
Description
Apache Airflow version: 1.10.12
Kubernetes version (if you are using kubernetes) (use kubectl version): not applicable
Environment: AWS EC2
- Cloud provider or hardware configuration: AWS
- OS (e.g. from /etc/os-release): Amazon Linux
- Kernel (e.g.
uname -a): x86_64 x86_64 x86_64 GNU/Linux - Install tools: docker
- Others:
What happened:
We are upgrading airflow to version from 1.10.5 to 1.12.10. We use docker based deployment on AWS infra. Currently we are running airflow on version 1.10.5 which is working fine , but with same dockerfile and airflow version updated to 1.10.12 , we are getting error in gunicorn when starting webserver with command "airflow webserver".
Arguments to config option are malformed: python:airflow.www.gunicorn_config [INFO] Worker exiting (pid: 76)
Use the following form: --config section.key=value
What you expected to happen:
Airflow webserver should start normally
How to reproduce it:
below is the dockerfile to reproduce the issue
FROM debian:stretch
FROM python:3.7-slim-stretch
MAINTAINER DevTeam
ENV DEBIAN_FRONTEND noninteractive
ENV TERM linux
ARG AIRFLOW_VERSION=1.10.12
ARG AIRFLOW_HOME=/usr/local/airflow
ENV LANGUAGE en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LC_ALL en_US.UTF-8
ENV LC_CTYPE en_US.UTF-8
ENV LC_MESSAGES en_US.UTF-8
ENV LC_ALL en_US.UTF-8
RUN set -ex
&& buildDeps='
python3-dev
libkrb5-dev
libssl-dev
libffi-dev
build-essential
libblas-dev
liblapack-dev
default-libmysqlclient-dev
git
'
&& apt-get update -yqq
&& apt-get install -yqq --no-install-recommends
$buildDeps
python3-pip
python3-requests
apt-utils
curl
netcat
locales
libsasl2-dev
&& sed -i 's/^# en_US.UTF-8 UTF-8$/en_US.UTF-8 UTF-8/g' /etc/locale.gen
&& locale-gen
&& update-locale LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8
&& useradd -ms /bin/bash -d ${AIRFLOW_HOME} airflow
&& python3 -m pip install -U pip==9.0.3
&& pip install Cython
&& pip install pytz
&& pip install -U setuptools
&& pip install pyOpenSSL
&& pip install ndg-httpsclient
&& pip install pyasn1
&& pip install Werkzeug==0.16.0
&& pip install JPype1==0.7.1
&& pip install cron-descriptor
&& pip install python-crontab
&& pip install apache-airflow[crypto,celery,mysql,hive,hdfs,jdbc,s3,redis]==$AIRFLOW_VERSION
&& pip install marshmallow-sqlalchemy==0.17.1
&& apt-get install python3-dev
# install mysqlclient version manually due to incopatibility with MySQL-python-1.2.5
&& pip install mysqlclient==1.3.14
&& pip install SQLAlchemy==1.3.15
# ----
&& python3 -m pip install mysql-connector
&& pip install boto
&& pip install botocore==1.16.26
&& pip install boto3==1.13.26
&& pip install watchtower==0.8.0
&& pip install snowflake-connector-python==2.2.6
&& pip install requests===2.23.0
&& apt-get remove --purge -yqq $buildDeps
&& apt-get clean
&& rm -rf
/var/lib/apt/lists/*
/tmp/*
/var/tmp/*
/usr/share/man
/usr/share/doc
/usr/share/doc-base
WORKDIR ${AIRFLOW_HOME}
COPY script script/
RUN /bin/bash script/install_python_deps.sh
RUN rm -rf script
COPY script/entrypoint.sh /entrypoint.sh
COPY config/airflow.cfg airflow.cfg
COPY dags dags
RUN chown -R airflow: ${AIRFLOW_HOME}
EXPOSE 8080 5555 8793
USER airflow
WORKDIR ${AIRFLOW_HOME}
ENTRYPOINT ["/entrypoint.sh"]
In entrypoint.sh file we start webserver with "airflow webserver" command
Anything else we need to know: