Using DockerCompose next to dedicated container classes features (like SqlServerContainer)

[g0di]

To test my code against a MS SQL database I spin up a test container using testcontainers and pytest

@pytest.fixture(scope="session")
def mssql_db() -> Iterator[str]:
    with SqlServerContainer("mcr.microsoft.com/mssql/server:2022-latest") as container:
        yield container.get_connection_url()

Using this fixture in my test allow me to spin my database for tests duration and the get the connection URL.

Recently I wanted to switch to docker compose instead so I could define my services in a single place and use that same configuration for both automated tests and running my app in dev mode locally. I got my configuration in a docker-compose.yml in my project root. My first try was the following:

@pytest.fixture(scope="session")
def mssql_db() -> Iterator[str]:
    with DockerCompose(
        "path/to/my/project/root",
        compose_file_name="docker-compose.yml",
        services=["mssql"],
    ):
        yield "<db connection url>"

At this point I got two issues:

1. I cannot get automagically the connection URL anymore
2. I'm loosing ability to wait for the database to be actually ready

Get the connection URL

For this one, my solution is to simply read the connection url I specified directly in my docker compose file for my app. So I add another fixture for getting my docker compose configuration.

class DockerComposeFile(NamedTuple):
    path: Path
    content: dict[str, Any]


@pytest.fixture(scope="session")
def docker_compose() -> DockerComposeFile:
    project_root = Path(__file__).parent.parent
    docker_compose_file_path = project_root.joinpath("docker-compose.yml")
    return DockerComposeFile(
        docker_compose_file_path,
        cast(dict[str, Any], yaml.safe_load(docker_compose_file_path.read_bytes())),
    )


@pytest.fixture(scope="session")
def mssql_db(docker_compose: DockerComposeFile) -> Iterator[str]:
    with DockerCompose(
        docker_compose.path.parent,
        compose_file_name=docker_compose.path.name,
        services=["mssql"],
    ):
        yield docker_compose.content["services"]["app"]["environment"]["DB_CONN_URL"].replace("mssql", "localhost")

Here, no magic anymore but this seems all good to me.

Note that we must replace mssql with localhost in the connection URL because the connection URL I'm specifying in my docker compose is referring to host mssql:1433 which is only available within the compose network. Because I'm running my tests from outside the compose network I can not resolve the name mssql

My suggestion here is maybe we could imagine another api within the DockerCompose object allowing to somehow read the config without having to do that manual stuff.

Loosing specific container classes behaviors

The second issue is that my solution is actually missing the part which wait for the database to be ready. When using SqlServerContainer class, it embeds a _connect() function which is called on start() and which wait for the database to be actually ready by trying several times to run a SQL query.

If I run my tests without this wait, it obviously fail at connecting to the DB.

So I tried a small hack by calling manually the _connect() function passing the compose container as self.

@pytest.fixture(scope="session")
def mssql_db_conn_str(docker_compose: DockerComposeFile) -> Iterator[str]:
    with DockerCompose(
        docker_compose.path.parent,
        compose_file_name=docker_compose.path.name,
        services=["mssql"],
    ) as srv:
        SqlServerContainer._connect(srv.get_container("mssql"))
        yield docker_compose.content["services"]["app"]["environment"]["DB_CONN_URL"]

Unfortunately this does not work because ComposeContainer and DockerContainer have not the same interface.

AttributeError: 'ComposeContainer' object has no attribute 'exec'

Here my suggestions would be:

• Extract the _connect logic in some way that it can be used outside of the SqlServerContainer class or set the method as public
• Add an API allowing to map a "ComposeContainer" into another container class like in my case a SqlServerContainer. The main issue I foresee with that is, because the container is spinned through docker compose, arguments expected at construction can not be retrieved (for example, how could we find the username that we need to pass to SqlServerContainer for creating an instance?)

Does someone already encountered this use case? Does one already find a workaround or a pattern? Is there some improvements we could bring into the library?

[g0di]

Answering to myself about the second issue I mentioned. I found out that we can put healthcheck on services of our docker compose configuration, for example:

mssql:
    image: mcr.microsoft.com/mssql/server:2022-latest
    environment:
      SA_PASSWORD: "YourStrong@Passw0rd"
      ACCEPT_EULA: "Y"
    healthcheck:
      test: "/opt/mssql-tools/bin/sqlcmd -U 'SA' -P 'YourStrong@Passw0rd' -Q 'SELECT 1'"
      interval: 10s
      timeout: 5s
      retries: 5
    ports:
      - "1433:1433"
    volumes:
      # Mount local volumes to keep state of the database
      - mssql_data:/var/opt/mssql

Doing so, my pytest fixture actually yield only once the container is ready, i.e: when the service is healthy.

This is apparently expected as per the wait argument passed to DockerCompose which defaults to True

From now I'm wondering if this method will be enough to cover cases where we need to check the logs to know if a service is healthy, does this can be actually achieved using the docker compose healthcheck property? If someone has an example it could be interesting.

I'm also not a big fan of having to replace my connection URL mssql to localhost but I could not found a better way for the moment.

Maybe it could be interesting to add some guides/recipes to the documentation of the library to cover such cases? It might be easier for developers instead of having to figure out the technical API.