[Transfer Engine]Coro rpc communicator #795

JasonZhang517 · 2025-09-01T11:40:01Z

add a communicator including server and client pool
add pybind interface of the communicator
add test files

stmatengss · 2025-09-01T11:44:31Z

mooncake-integration/CMakeLists.txt

 pybind11_add_module(engine ${SOURCES} ${CACHE_ALLOCATOR_SOURCES}
    transfer_engine/transfer_engine_py.cpp 
+    ../mooncake-transfer-engine/src/transport/coro_rpc_connector/cororpc_interface.cpp
+    ../mooncake-transfer-engine/src/transport/coro_rpc_connector/cororpc_communicator.cpp


Why need to add these files here?

if we want to use coro_rpc_connector as a submodule of mooncake transfer engine, we have to compile it here

mooncake-transfer-engine/src/transport/coro_rpc_connector/cororpc_communicator.cpp

poor-circle · 2025-09-02T09:33:54Z

mooncake-transfer-engine/src/transport/coro_rpc_connector/cororpc_communicator.cpp

+    return false;
+}
+
+int CoroRPCCommunicator::sendData(const std::string& target_address,


sendData should be a wrapper of sendDataAsync, dont rewrite same code twice

use syncAwait(Lazy{}) as impl

mooncake-transfer-engine/src/transport/coro_rpc_connector/cororpc_communicator.cpp

mooncake-transfer-engine/src/transport/coro_rpc_connector/cororpc_interface.cpp

qicosmos · 2025-09-03T03:03:24Z

mooncake-transfer-engine/src/transport/coro_rpc_connector/cororpc_interface.cpp

+    auto future_ptr = std::make_shared<pybind11::object>(future_obj);
+    auto loop_ptr = std::make_shared<pybind11::object>(pybind11::reinterpret_borrow<pybind11::object>(loop));
+
+    auto task_func = std::make_shared<std::function<void()>>(


send_request or
auto lazy = [](int i) -> async_simple::coro::Lazy {
co_await c->send_data_async();
}

lazy(i).start({});

qicosmos · 2025-09-03T03:03:51Z

mooncake-transfer-engine/src/transport/coro_rpc_connector/cororpc_interface.cpp

+
+    auto task_func = std::make_shared<std::function<void()>>(
+        [communicator, target_addr, data_holder, future_ptr, loop_ptr]() {
+        int result = communicator->sendData(*target_addr, data_holder->data(), data_holder->size());


co_await communicator->sendDataAsync

qicosmos · 2025-09-03T03:08:14Z

mooncake-transfer-engine/src/transport/coro_rpc_connector/cororpc_interface.cpp

+    return future_obj;
+}
+
+int CoroRPCInterface::sendTensor(const std::string& target_address, pybind11::handle tensor) {


according async_send_tensor

mooncake-transfer-engine/src/transport/coro_rpc_connector/cororpc_interface.cpp

qicosmos · 2025-09-03T03:10:40Z

mooncake-transfer-engine/src/transport/coro_rpc_connector/cororpc_interface.cpp

+    return future_obj;
+}
+
+void CoroRPCInterface::setDataReceiveCallback(pybind11::function callback) {


use asyncio future

…ronous coroutine 3. change std::future to lazy return value

mooncake-transfer-engine/include/transport/coro_rpc_connector/cororpc_communicator.h

mooncake-transfer-engine/include/transport/coro_rpc_connector/cororpc_interface.h

mooncake-transfer-engine/src/transport/coro_rpc_connector/cororpc_communicator.cpp

mooncake-transfer-engine/src/transport/coro_rpc_connector/cororpc_interface.cpp

stmatengss · 2025-09-08T08:43:31Z

@JasonZhang517 Do you happen to have any updates for this PR?

mooncake-transfer-engine/tests/communicator_bandwidth_test.py

mooncake-integration/transfer_engine/transfer_engine_py.cpp

mooncake-integration/CMakeLists.txt

mooncake-transfer-engine/src/transport/CMakeLists.txt

mooncake-transfer-engine/tests/communicator_bandwidth_test.py

stmatengss · 2025-09-28T02:35:09Z

.github/workflows/ci.yml

      run: ${SCCACHE_PATH} --show-stats

    - name: Install dependencies
+      shell: bash


why move it here?

.gitignore

…_rpc_communicator

JasonZhang517 added 2 commits August 29, 2025 19:43

add basic implementation and python interfaces

e63fa88

add real communicator logic and interface

22e59a0

stmatengss reviewed Sep 1, 2025

View reviewed changes

JasonZhang517 added 8 commits September 1, 2025 19:47

add integration test

998a1c0

added tensor rebuild logic

f8cee4d

add tensor rebuild and tests

81cbfdd

removed useless lines

714ef16

removed blank lines

59c5105

turn single client creation to client pool

06a672f

merged conflicts

1c225b9

Merge remote-tracking branch 'upstream/main' into coro_rpc_communicator

1fc1ee0

poor-circle reviewed Sep 2, 2025

View reviewed changes

change std::future to async_simple coro lazy

f5deb6c

stmatengss requested a review from poor-circle September 2, 2025 12:16

qicosmos reviewed Sep 3, 2025

View reviewed changes

stmatengss requested a review from qicosmos September 4, 2025 03:45

JasonZhang517 added 6 commits September 4, 2025 11:50

1. removed some data copies 2. replace synchronous thread with asynch…

65ee0c8

…ronous coroutine 3. change std::future to lazy return value

1.added tests to ci 2. removed CLAUDE.md

8c061e3

add pybind installation in CI

aed64b1

fixed clang spelling

08146aa

fixed communicator spelling

584e386

fixed redundant functions in transfer engine py

e58e2c3

JasonZhang517 changed the title ~~Coro rpc communicator~~ [Transfer Engine]Coro rpc communicator Sep 4, 2025

fixed ci path

8953508

JasonZhang517 force-pushed the coro_rpc_communicator branch from 4162d03 to 8953508 Compare September 5, 2025 00:09

qicosmos reviewed Sep 5, 2025

View reviewed changes

JasonZhang517 added 3 commits September 8, 2025 15:23

fixed memory copy bugs, removed client pool creation

b2afec2

convert memorycopy to memoryview

fe165e9

set attachment for handleDataTransfer

a1d4c64

JasonZhang517 added 5 commits September 12, 2025 10:06

reformat headers

d4caef8

fixed CMakeLists.txt

e52aa02

add indication of protocol

fa10062

fixed rdma bugs

e72c7a0

fixed some minor bugs

732fe0e

stmatengss reviewed Sep 14, 2025

View reviewed changes

mooncake-transfer-engine/tests/communicator_bandwidth_test.py Outdated Show resolved Hide resolved

stmatengss reviewed Sep 14, 2025

View reviewed changes

mooncake-integration/transfer_engine/transfer_engine_py.cpp Outdated Show resolved Hide resolved

stmatengss reviewed Sep 14, 2025

View reviewed changes

mooncake-integration/CMakeLists.txt Outdated Show resolved Hide resolved

stmatengss reviewed Sep 14, 2025

View reviewed changes

mooncake-transfer-engine/src/transport/CMakeLists.txt Outdated Show resolved Hide resolved

JasonZhang517 added 6 commits September 15, 2025 10:06

removed Chinese comments

c40b13a

remove useless files

aa62943

removed unnecessary include in CMakeLists.txt

ff9525a

refactored client and server initialization methods

2b83a76

fixed server initialization pool size

ed76b8b

fixed interface pybind bugs

600db58

stmatengss reviewed Sep 16, 2025

View reviewed changes

mooncake-transfer-engine/tests/communicator_bandwidth_test.py Outdated Show resolved Hide resolved

fixed pybind

abd15a1

JasonZhang517 force-pushed the coro_rpc_communicator branch 2 times, most recently from 77b6863 to abd15a1 Compare September 23, 2025 07:50

JasonZhang517 added 5 commits September 23, 2025 16:05

update pybind11 parameters to -Dpybind11_DIR

20b1bc2

update cmakelists

fc7dde6

local py312 test complete

c790f21

adjused workflow

255a5a9

adjusted Cmakelists.txxt

61b03da

stmatengss reviewed Sep 28, 2025

View reviewed changes

restored some useless changes

2ac0105

stmatengss reviewed Sep 30, 2025

View reviewed changes

.gitignore Show resolved Hide resolved

JasonZhang517 added 3 commits October 10, 2025 20:37

remove Chinese comments

41e871f

remove blank lines

ba6bd7f

Merge remote-tracking branch 'origin/coro_rpc_communicator' into coro…

384e3b1

…_rpc_communicator

[Transfer Engine]Coro rpc communicator #795

Are you sure you want to change the base?

[Transfer Engine]Coro rpc communicator #795

Uh oh!

Conversation

JasonZhang517 commented Sep 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

poor-circle Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stmatengss commented Sep 8, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

poor-circle Sep 2, 2025 •

edited

Loading