[Bug]: Docker 29.2.1 couldn't find valid docker environment

[PMolnes]

Module

PostgreSQL

Testcontainers version

2.0.3

Using the latest Testcontainers version?

Yes

Host OS

MacOS

Host Arch

ARM

Docker version

29.2.1

What happened?

I updated Docker Desktop to version 4.63.0 and afterwards when running ./gradlew test I get the error "couldn't find valid docker environment". Adding a docker-java.properties file in src/test/resources with api.version = 1.44 fixes the issue, but as seen in similar bug reports this is just a work around.

Same issue happens if I use version 2.0.2.

Relevant log output

./gradlew test

> Task :test

GroupCategoryTransactionTest > testInviteMemberToGroup() FAILED
    java.lang.ExceptionInInitializerError
        at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized0(Native Method)
        at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized(Unsafe.java:1160)
        at java.base/java.lang.reflect.Constructor.acquireConstructorAccessor(Constructor.java:549)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
        at java.base/java.util.Optional.orElseGet(Optional.java:364)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)

        Caused by:
        java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
            at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$getFirstValidStrategy$7(DockerClientProviderStrategy.java:274)
            at java.base/java.util.Optional.orElseThrow(Optional.java:403)
            at org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:265)
            at org.testcontainers.DockerClientFactory.getOrInitializeStrategy(DockerClientFactory.java:154)
            at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:196)
            at org.testcontainers.DockerClientFactory$1.getDockerClient(DockerClientFactory.java:108)
            at com.github.dockerjava.api.DockerClientDelegate.authConfig(DockerClientDelegate.java:109)
            at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:321)
            at dev.molnes.expense_tracker.integration.BaseApiTest.<clinit>(BaseApiTest.java:22)
            ... 8 more

Additional Information

package dev.molnes.expense_tracker.integration;

import dev.molnes.expense_tracker.helpers.DatabaseCleaner;
import org.junit.jupiter.api.AfterEach;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.ActiveProfiles;
import org.testcontainers.containers.PostgreSQLContainer;

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@ActiveProfiles("dev")
public abstract class BaseApiTest {

  @Autowired
  private DatabaseCleaner databaseCleaner;
  static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>("postgres:15")
      .withDatabaseName("test-db")
      .withUsername("test")
      .withPassword("test");

  static {
    postgres.start();
    System.setProperty("spring.datasource.url", postgres.getJdbcUrl());
    System.setProperty("spring.datasource.username", postgres.getUsername());
    System.setProperty("spring.datasource.password", postgres.getPassword());
  }

  @AfterEach
  void cleanUp() {
    databaseCleaner.cleanAll();
  }
}

[rnorth]

I can't quite get my head around why #11346 allows this to happen...

[rnorth]

@PMolnes I've been trying to reproduce on the same Docker Desktop version (4.63.0 (220185)).
I'm afraid I can't. Are you absolutely certain that 2.0.3 is affected by this bug?

So far the only thing I can think might be happening is the ping on this line fails for some unrelated reason, and Testcontainers then tries falling back to a Docker API version of 1.32.

Or perhaps is there another docker-java.properties file on your classpath somehow?

It might also help if you could step through DockerClientProviderStrategy's getClientForConfig method with a debugger, to give us an understanding of which code branch it's going down...