Connection pooling settings are misleading when running uWSGI with >1 process

[stevenbal]

Product versie / Product version

latest

Omschrijf het probleem / Describe the bug

Taiga reference: Utrecht 334

The configuration for connection pooling (min_size, max_size, etc.) is misleading when running uWSGI with more than 1 process (we run 4 by default), because each process will instantiate its own connectionpool with these parameters.

In practice this would mean that your value for the max_size envvar should be multiplied by the number of processes uwsgi is running.

TODO:

• Modify the help_text of DB_POOL_ENABLED to state that it is experimental and not recommended for production use (not sure if this is possible, but if we can add a sphinx reference to https://open-api-framework.readthedocs.io/en/latest/connection_pooling.html that would be nice)
• Add this same warning to https://open-api-framework.readthedocs.io/en/latest/connection_pooling.html#connection-pooling
• Make a new release of open-api-framework
• Upgrade OAF in all components and refer to OAF docs from all components with intersphinx (https://open-api-framework.readthedocs.io/en/latest/connection_pooling.html). This reference should be added as a warning in the changelog for the releases that introduced connection pooling
  • Open Zaak ⬆️ [maykinmedia/open-api-framework#171] add changelog warning for connection pooling open-zaak/open-zaak#2255
  • Open Notificaties Upgrade OAF and add changelog warning for connection pooling open-zaak/open-notificaties#349
  • Objects API Upgrade OAF and add changelog warning for connection pooling objects-api#710
  • Objecttypes API Upgrade OAF and add changelog warning for connection pooling objecttypes-api#222
  • Open Klant ✨ [maykinmedia/open-api-framework#171] Upgrade OAF open-klant#535
  • Referentielijsten Upgrade OAF and add changelog warning for connection pooling referentielijsten#122
  • Open Product Upgrade OAF and add changelog warning for connection pooling open-product#255

Stappen om te reproduceren / Steps to reproduce

No response

Verwacht gedrag / Expected behavior

I think this should at least be documented, I'm not sure if there is a way to enfore the min_size and max_size as passed via envvar, because we don't know the number of replicas for example

[sdegroot]

I would expect there is a single connection pool for a single python application. Creating multiple connection pools within the same executable kind of defeats the purpose. Without this, either the connection pooling is unusable or you lose the option to have multiple wsgi processes (I'll have to set that to one)

[stevenbal]

As discussed last tuesday, even with the assumption that each process has its own connection pool, the number of connections still isn't predictable (with a min/max pool size of 3, 10 processes and 2 replicas, we would expect 60 connections, while there were less than 30).

We'll look into this to figure out why this is happening