You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Following these instructions I failed to install any of the parts (cornflow or ariflow) correctly.
I had to go look for the unit tests of cornflow client to understand the current steps and adapt them to a normal (i.e., outside of github) environment.
here are a few of the issues I found in the two guides above:
apache airflow version is wrong (2.7.1 vs 2.1)
there's no instruction to do cd cornflow-server before installing virtual environments and deploying.
the flask db upgrade instruction lacks an argument.
cornflow-dags dags and requirements need to be copied to the airflow_config directory
Unconsistent references to afvenv and cfvenv (venv) in all documentation/ guied.
A command to install apache airflow providers google was missing and lead to an error. Solved with pip install apache-airflow-providers-google
References to python instead of python3 are not portable to Ubuntu (replacing with python3 worked)
A reference to a cornflow port on launch (5050 vs 5000) was inconsistent.
I'm sure I'm still lacking a few things. I leave below the installation instructions (commands) that worked for me, merging from memory, unit tests and current docs.
I used the installation documentation on the following places:
Following these instructions I failed to install any of the parts (cornflow or ariflow) correctly.
I had to go look for the unit tests of cornflow client to understand the current steps and adapt them to a normal (i.e., outside of github) environment.
here are a few of the issues I found in the two guides above:
cd cornflow-server
before installing virtual environments and deploying.flask db upgrade
instruction lacks an argument.cornflow-dags
dags and requirements need to be copied to theairflow_config
directoryafvenv
andcfvenv
(venv
) in all documentation/ guied.pip install apache-airflow-providers-google
python
instead ofpython3
are not portable to Ubuntu (replacing withpython3
worked)I'm sure I'm still lacking a few things. I leave below the installation instructions (commands) that worked for me, merging from memory, unit tests and current docs.
Cornflow server installation and running
cd cornflow-server
python -m venv cfvenv
source cfvenv/bin/activate
python3 -m pip install -U -r requirements-dev.txt
python3 -m pip install -U -e ../libs/client
export FLASK_APP=cornflow.app
export FLASK_ENV=development
export DATABASE_URL=sqlite:///cornflow.db
export SECRET_KEY=THISNEEDSTOBECHANGED
export AIRFLOW_URL=http://localhost:8080
export AIRFLOW_USER=admin
export AIRFLOW_PWD=admin
flask db upgrade -d cornflow/migrations/
flask access_init
flask register_deployed_dags -r http://127.0.0.1:8080 -u admin -p admin
flask create_admin_user -u admin -e admin@cornflow.org -p Adminpassword1!
flask create_service_user -u airflow -e airflow@cornflow.org -p Airflow_test_password1
flask create_base_user -u user -e user@cornflow.org -p UserPassword1!
flask register_dag_permissions -o 1
flask run -p 5050 &
Airflow server installation
cp -r cornflow-dags/DAG/* cornflow-server/airflow_config/dags/
cp cornflow-dags/requirements.txt cornflow-server/airflow_config/
cd cornflow-server
python3 -m venv afvenv
source afvenv/bin/activate
AIRFLOW_VERSION=2.7.1
PYTHON_VERSION="$(python3 --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
CLIENT_BRANCH="${{ github.head_ref || github.ref_name }}"
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
python3 -m pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
python3 -m pip install -U orloge pulp
python3 -m pip install -U -e ../libs/client
python3 -m pip install -U -r airflow_config/requirements.txt
pip install apache-airflow-providers-google
airflow db init
airflow users create -u admin -f admin -l admin -r Admin -p admin -e admin@example.org
Airflow server running
export AIRFLOW_HOME="$PWD/airflow_config"
export AIRFLOW__CORE__LOAD_EXAMPLES=0
export AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION=0
export AIRFLOW__API__AUTH_BACKEND=airflow.api.auth.backend.basic_auth
export AIRFLOW__WEBSERVER__SECRET_KEY=e9adafa751fd35adfc1fdd3285019be15eea0758f76e38e1e37a1154fb36
export AIRFLOW_CONN_CF_URI=http://airflow:Airflow_test_password1@localhost:5050
airflow webserver -p 8080 &
airflow scheduler &
The text was updated successfully, but these errors were encountered: