[Feature] GPU Direct Storage(GDS) #301
Conversation
UESTC-AHao
requested review from
FangRun2,
Tarrei and
flesher0813
as code owners
| UC_INFO("GDS driver initialized successfully"); | ||
| } else { | ||
| UC_ERROR("GDS driver initialized unsuccessfully"); | ||
| } |
There was a problem hiding this comment.
Error handling is insufficient, failures need to be visible to the caller.
There was a problem hiding this comment.
Received,I'll fix it.
ucm/store/device/simu/simu_device.cc
Outdated
| } | ||
| Status S2DSync(int fd, void* address, const size_t length, const size_t fileOffset, const size_t devOffset) override | ||
| { | ||
| return Status::OK(); |
There was a problem hiding this comment.
return Status::Unsupported();
| bool DirectStorageQueue::Init(Device& device) | ||
| { | ||
| if (this->deviceId_ < 0) { return true; } | ||
| DeviceFactory::Setup(useDirect); |
There was a problem hiding this comment.
This interface should be invoked in Setup(), not in Init().
There was a problem hiding this comment.
U are right,I'll fix it.
| Status DirectStorageQueue::S2D(Task::Shard& shard, const Device& device) { | ||
| auto path = this->layout_->DataFilePath(shard.block, false); | ||
| int fd = -1; | ||
| auto status = HandlePool<std::string, int>::Instance().Get(path, fd, |
There was a problem hiding this comment.
Don't use a singleton, use a member variable instead for HandlePool
There was a problem hiding this comment.
Under the current circumstances, singletons in processes created by fork are hanging. I will fix this.
| int fd = -1; | ||
| auto status = HandlePool<std::string, int>::Instance().Get(path, fd, | ||
| [&path](int& newFd) -> Status { | ||
| newFd = open(path.c_str(), O_RDONLY | O_DIRECT, 0644); |
There was a problem hiding this comment.
Use file interface in file.h
| int fd = -1; | ||
| auto status = HandlePool<std::string, int>::Instance().Get(path, fd, | ||
| [&path](int& newFd) -> Status { | ||
| newFd = open(path.c_str(), O_WRONLY | O_CREAT | O_DIRECT, 0644); |
There was a problem hiding this comment.
When are the handles cached in HandlePool closed?
There was a problem hiding this comment.
I will use member variables and release resources in the destructor.
| for (size_t i = 0; i < streamNumber; i++) { | ||
| auto q = std::make_shared<DirectStorageQueue>(); | ||
| status = | ||
| q->Setup(deviceId, ioSize, bufferNumber, &this->failureSet_, layout, timeoutMs, useDirect); |
There was a problem hiding this comment.
DirectStorageQueue is always direct, and any other queue is never direct, the useDirect flag is redundant.
| q->Setup(deviceId, ioSize, bufferNumber, &this->failureSet_, layout, timeoutMs, useDirect); | ||
| if (status.Failure()) { break; } | ||
| this->queues_.emplace_back(std::move(q)); | ||
| } |
There was a problem hiding this comment.
Both the if and else branches contain duplicated code.
3 participants
Purpose
What this PR does / why we need it?
This feature introduces GPU Direct Storage (GDS) support to enable direct data transfer between GPU memory and storage, bypassing CPU memory as an intermediate buffer. This significantly reduces memory bandwidth bottlenecks and improves KV cache loading/offloading performance.
Modifications
Does this PR introduce any user-facing change?
A new parameter, useDirect, has been added to the service-startup command. When set to true, GDS transfer is enabled and the data path is Device Memory <-> Storage (direct). When set to false, the path becomes Device Memory <-> Host Memory <-> Storage, as shown below:
"kv_connector_extra_config": {
"ucm_connector_name": "UcmNfsStore",
"ucm_connector_config": {
"storage_backends": "/home/nfs",
"useDirect" : true
Test
How was this patch tested?
embed without GDS:
embed with GDS:
fetch without GDS:
fetch with GDS:
