[Bug]: Docker 29.2.1 couldn't find valid docker environment

[PMolnes]

Module

PostgreSQL

Testcontainers version

2.0.3

Using the latest Testcontainers version?

Yes

Host OS

MacOS

Host Arch

ARM

Docker version

29.2.1

What happened?

I updated Docker Desktop to version 4.63.0 and afterwards when running ./gradlew test I get the error "couldn't find valid docker environment". Adding a docker-java.properties file in src/test/resources with api.version = 1.44 fixes the issue, but as seen in similar bug reports this is just a work around.

Same issue happens if I use version 2.0.2.

Relevant log output

./gradlew test

> Task :test

GroupCategoryTransactionTest > testInviteMemberToGroup() FAILED
    java.lang.ExceptionInInitializerError
        at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized0(Native Method)
        at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized(Unsafe.java:1160)
        at java.base/java.lang.reflect.Constructor.acquireConstructorAccessor(Constructor.java:549)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
        at java.base/java.util.Optional.orElseGet(Optional.java:364)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)

        Caused by:
        java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
            at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$getFirstValidStrategy$7(DockerClientProviderStrategy.java:274)
            at java.base/java.util.Optional.orElseThrow(Optional.java:403)
            at org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:265)
            at org.testcontainers.DockerClientFactory.getOrInitializeStrategy(DockerClientFactory.java:154)
            at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:196)
            at org.testcontainers.DockerClientFactory$1.getDockerClient(DockerClientFactory.java:108)
            at com.github.dockerjava.api.DockerClientDelegate.authConfig(DockerClientDelegate.java:109)
            at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:321)
            at dev.molnes.expense_tracker.integration.BaseApiTest.<clinit>(BaseApiTest.java:22)
            ... 8 more

Additional Information

package dev.molnes.expense_tracker.integration;

import dev.molnes.expense_tracker.helpers.DatabaseCleaner;
import org.junit.jupiter.api.AfterEach;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.ActiveProfiles;
import org.testcontainers.containers.PostgreSQLContainer;

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@ActiveProfiles("dev")
public abstract class BaseApiTest {

  @Autowired
  private DatabaseCleaner databaseCleaner;
  static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>("postgres:15")
      .withDatabaseName("test-db")
      .withUsername("test")
      .withPassword("test");

  static {
    postgres.start();
    System.setProperty("spring.datasource.url", postgres.getJdbcUrl());
    System.setProperty("spring.datasource.username", postgres.getUsername());
    System.setProperty("spring.datasource.password", postgres.getPassword());
  }

  @AfterEach
  void cleanUp() {
    databaseCleaner.cleanAll();
  }
}

[rnorth]

I can't quite get my head around why #11346 allows this to happen...

[rnorth]

@PMolnes I've been trying to reproduce on the same Docker Desktop version (4.63.0 (220185)).
I'm afraid I can't. Are you absolutely certain that 2.0.3 is affected by this bug?

So far the only thing I can think might be happening is the ping on this line fails for some unrelated reason, and Testcontainers then tries falling back to a Docker API version of 1.32.

Or perhaps is there another docker-java.properties file on your classpath somehow?

It might also help if you could step through DockerClientProviderStrategy's getClientForConfig method with a debugger, to give us an understanding of which code branch it's going down...

[PMolnes]

Sorry for the late reply. So after stepping through getClientForConfig, I can clearly see that it steps into this if check. But it looks like the code in my DockerClientProviderStrategy is not the same as the one you linked. Is this not the correct dependency testImplementation 'org.testcontainers:testcontainers-postgresql:2.0.3'?

if (configBuilder.build().getApiVersion() == RemoteApiVersion.UNKNOWN_VERSION) {
    configBuilder.withApiVersion(RemoteApiVersion.VERSION_1_32);
}

[PMolnes]

I think this issue was a dependency config problem on my side. I only had the postgresql dependency in my build.gradle file. Adding this one as well seems to fix the issue: testImplementation "org.testcontainers:testcontainers:2.0.2"

testImplementation 'org.testcontainers:testcontainers-postgresql:2.0.3'