psycopg2 UnicodeDecodeError in Docker

[michaltarana]

Apache Airflow version

3.1.7

If "Other Airflow 3 version" selected, which one?

No response

What happened?

Using the official Docker image and docker-compose.yaml, Postgres DB. The database seems to be created correctly (airflow-init runs without any errors). However, the containers seem to have a problem to connect to the DB backend. When I try (inside the apiserver container)
airflow db migrate
I get the following error:
... File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/create.py", line 661, in connect return dialect.connect(*cargs, **cparams) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 630, in connect return self.loaded_dbapi.connect(*cargs, **cparams) # type: ignore[no-any-return] # NOQA: E501 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/psycopg2/__init__.py", line 122, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ UnicodeDecodeError: 'utf-8' codec can't decode byte 0xcb in position 135: invalid continuation byte
The same error appears when I try to test or trigger any dag from the CLI inside the container. All the dags triggered from the UI fail, as well.

What you think should happen instead?

No response

How to reproduce

1. Install Docker version of Apache Airflow 3.1.7 (configure inside docker-compose.yml)
2. Run docker compose up airflow-init
3. Run docker compose --profile server up --wait
4. Run docker compose exec -u airflow airflow-apiserver sh
5. Run airflow db migrate or any other airflow command that makes a DB query
   One should see the error mentioned above.

Operating System

Debian GNU/Linux 12

Versions of Apache Airflow Providers

Only the providers included in the official Docker images.

Deployment

Docker-Compose

Deployment details

Docker Compose version v5.1.0

Anything else?

The problem appeared with the upgrade from Apache Airflow 3.0.3.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[potiuk]

Likely in some of your configuration or env variables you have non-unicode string. That's definitely something in your configuration - maybe a byte injected when copying things from windows.

I think for example vi -b will show binary characters that might otherwise be invisible.,

[michaltarana]

THank you for your reply. It is still very unclear, what was the issue. However, when I removed all the containers, images and volumes, reinstalled everything and reinitialized the database, all the problems disappeared. The connection string and all docker-compose, .env remained unchanged.