Basic copy_to implementation by gossi · Pull Request #242 · testcontainers/testcontainers-elixir

gossi · 2026-02-24T09:57:59Z

Impl for #240

I did some research for the API in rust and node clients:

Rust
- 1 Method: with_copy_to
- Multiple target and source options: https://github.com/testcontainers/testcontainers-rs/blob/73ec33e1de5fc1776fc2310add9fd8f51ebd67ba/testcontainers/src/core/copy.rs#L17
Node
- Many methods:
- Plus respective options for each (under types)

For elixir, I choose the method from rust with_copy_to and then use pattern matching against the cases.

I personally only need one case here and did the impl for the most basic one 🙈.

gossi · 2026-02-24T12:16:57Z

container_test is reporting issues (socket closed). For once, the introduced copy_to_test is failing.

gossi · 2026-03-02T11:12:53Z

@jarlah this is the last missing feature for me to complete my library using testcontainers, I'd like to see this in 2.0 (it is a blocker for me).

Anything I can help out to get this through?

jarlah · 2026-03-13T19:26:34Z

so sorry.. been awfully busy the last weeks. Actually been a heavy user of testcontainers actually and its not easy to admit that its not as stable as it should be hah

jarlah · 2026-03-13T19:30:04Z

first off the hash test thats because you introduced new struct field in container. So thats easy to fix. Just use the new hash in the test.

jarlah · 2026-03-15T16:24:55Z

ill look into this and the other PR now, plus the AWFUL flakyness of the ryuk connection initialization! Im running two production projects and im not kidding when i say that ryuk fails so many times ai is skipping running tests all together from time to time ..

jarlah · 2026-03-16T07:18:17Z

haha sorry @gossi i destroyed this PR by syncing it. But anway. Master should be extremely more stable now. for ex just see here from a local run, only the periodically 1 in 10 flaky kafka test left

./run_tests.sh
========================================
Run 1/10
========================================
Cleaning up Docker containers...
Running tests...
PASSED

========================================
Run 2/10
========================================
Cleaning up Docker containers...
Running tests...
PASSED

========================================
Run 3/10
========================================
Cleaning up Docker containers...
Running tests...
PASSED

========================================
Run 4/10
========================================
Cleaning up Docker containers...
Running tests...
PASSED

========================================
Run 5/10
========================================
Cleaning up Docker containers...
Running tests...
PASSED

========================================
Run 6/10
========================================
Cleaning up Docker containers...
Running tests...
FAILED on run 6
Finished in 114.0 seconds (27.9s async, 86.1s sync)
  1) test kafka container provides a ready-to-use kafka container (Testcontainers.Container.KafkaContainerTest)
     test/container/kafka_container_test.exs:136
     ** (MatchError) no match of right hand side value: :leader_not_available
     code: {:ok, _} = KafkaEx.produce(topic_name, 0, "hey", worker_name: worker_name, required_acks: 1)
     stacktrace:
       test/container/kafka_container_test.exs:146: (test)

..............
Finished in 114.0 seconds (27.9s async, 86.1s sync)
120 tests, 1 failure

--- Full output saved to test_run_6.log ---

========================================
Run 7/10
========================================
Cleaning up Docker containers...
Running tests...
PASSED

========================================
Run 8/10
========================================
Cleaning up Docker containers...
Running tests...
PASSED

========================================
Run 9/10
========================================
Cleaning up Docker containers...
Running tests...
PASSED

========================================
Run 10/10
========================================
Cleaning up Docker containers...
Running tests...
PASSED

========================================
Summary: 10 runs completed
========================================
1 run(s) failed:
  - Run 6: exit code 2

its also possible to run tests completely offline. For ex when on a plane and you just want to run a couple of tests when you have all images locally.

jarlah · 2026-03-16T07:19:52Z

im making a new version from this. Actually, i think i will release version 2.0.0-rc2. Then, after using it for a couple of days, checking its more resilient, release 2.0.0 end of week

jarlah · 2026-03-16T07:35:02Z

v2.0.0-rc2 released. Will use it on my two prod apps this coming week

gossi

The hash fixes were the easy parts ;)

I run into couple others:

Creating, starting and waiting was (and is) inconsistent (see my comments on ryuk, that reveal that)
I tried to make the API a bit more explicit (one function each for creating, starting and waiting) - I got quite far I'd say
I wanted to solve my connreset problem, I fixed the inotify problem

I still have this issue:

1) test copy contents to target (CopyToTest)
Error:      test/copy_to_test.exs:5
     ** (MatchError) no match of right hand side value: {:error, :socket_closed_remotely}
     code: {:ok, %{body: body}} = Tesla.get("[http://127.0.0.1:#{mapped_port}/hello.txt](http://127.0.0.1/#{mapped_port}/hello.txt)")
     stacktrace:
       test/copy_to_test.exs:18: (test)

So, locally when running the tests, it's green and clear. I think this is a race condition when running this on GH actions, although I have no idea which socket is closed (docker, nginx 🤷).

I tried with and without http wait strategy, seems like something else decides to kill some process here.

Anything we can do to make that execution stable?

gossi · 2026-03-16T12:31:32Z

lib/testcontainers.ex

+    with {:ok, id} <- create_container(conn, ryuk_config, properties),
+         {:ok, container} <- start_container_with_id(conn, id),


I'm not sure if I made this better or worse here. In the end ryuk is started as any other container, with the same config, with the same expectation.

Yet it is handled differently.

The weird part, ryuk_config in that situation is used for both config and config_builder - which is a separate thing below o_O. That was (and is) super confusing when I tried to run ryuk and any other container through the same "handlers".

i would like any changes to this part of the code to be done in a separate PR unless explicitly required by the PRs main changes

gossi · 2026-03-16T12:32:18Z

lib/testcontainers.ex

+  defp create_and_start_and_wait_for_container(config, config_builder, state) do
+    with {:ok, id} <- create_container(state.conn, config, state.properties),
+         {:ok, container} <- start_container_with_id(state.conn, id),
+         :ok <- ContainerBuilder.after_start(config_builder, container, state.conn),


ie. ryuk doesn't get this after_start check.

again i think we should add minimal changes. are all these changes required just add copy to container?

gossi · 2026-03-16T14:09:25Z

ha! you made the http strategy stable and my PR is green now - WoW!

Please have a look :)

jarlah · 2026-03-16T15:18:50Z

.github/workflows/elixir.yml


    steps:
    - uses: actions/checkout@v4
+    - name: Install inotify-tools


is there a specific need for inotify tools ?

not necessarily. Inotify tools handle file nodes for i/o.

At times I had this as only error message. So, I fixed this at first. (I did the same at my other project using testcontainers). In the end it makes test output cleaner and does not clutter the console output with errors/warnings that detract from actual debugging, that's helpful.

I gonna extract this.

jarlah

Nice job, :) Sorry about the "thumbs down" but i feel the amount of changes in the core startup logic is too much just for a copy to container logic. Can you scope it down? to just copy to container logic? e.g. as little as changes required to get it working ?

jarlah · 2026-03-16T15:49:44Z

there is always a chance i didnt understand the reasoning behind the changes too, so 🐻 that in mind

gossi added 10 commits March 16, 2026 12:22

Basic copy_to implementation

6248427

fix hash

023c8f3

remove deprecation in running cli tests

01e8bcb

use inotify tools for phoenix

0c37cb4

inotify tools for all jobs

8ba9f49

refactor create and start container to guarantee correct runs

d2bef15

extract pulling image from create_and_start_container

ffa6de1

with ip?

5c5bea8

what about no wait strategy?

cebb862

address rebase

be4190c

gossi force-pushed the container-assets branch from e25eaac to be4190c Compare March 16, 2026 12:21

fix hash

57fd21a

gossi commented Mar 16, 2026

View reviewed changes

with waiting strategy

1b5ea3d

jarlah reviewed Mar 16, 2026

View reviewed changes

jarlah requested changes Mar 16, 2026

View reviewed changes

gossi added 2 commits March 16, 2026 17:12

revert inotify install

1231ebc

revert create container API

bf72aeb

		with {:ok, id} <- create_container(conn, ryuk_config, properties),
		{:ok, container} <- start_container_with_id(conn, id),

Conversation

gossi commented Feb 24, 2026

Uh oh!

gossi commented Feb 24, 2026

Uh oh!

gossi commented Mar 2, 2026

Uh oh!

jarlah commented Mar 13, 2026

Uh oh!

jarlah commented Mar 13, 2026

Uh oh!

jarlah commented Mar 15, 2026

Uh oh!

jarlah commented Mar 16, 2026

Uh oh!

jarlah commented Mar 16, 2026

Uh oh!

jarlah commented Mar 16, 2026

Uh oh!

gossi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gossi commented Mar 16, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jarlah left a comment

Choose a reason for hiding this comment

Uh oh!

jarlah commented Mar 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants