Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .markdownlint.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
MD010: false
MD013:
line_length: 1024
MD024: false
MD046:
style: "fenced"
124 changes: 124 additions & 0 deletions releases/2.4.0/README-zh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# Dragonfly v2.4.0 发布了

Dragonfly v2.4.0 发布了!🎉🎉🎉 感谢为此版本做出贡献的[贡献者们](https://github.com/dragonflyoss/dragonfly/graphs/contributors),欢迎访问 [d7y.io](https://d7y.io) 网站。

![dragonfly](images/dragonfly-linear.png)

## 新增特性

### 负载感知调度算法

调度算法升级为一种两阶段调度算法,结合中央调度与节点级二次调度,基于实时负载感知来优化 P2P 下载性能。

![p1](images/p1.svg)

更多信息,请参阅 [Scheduling](https://d7y.io/docs/next/operations/deployment/applications/scheduler/#scheduling)。

### Vortex 协议支持 P2P 文件传输

Dragonfly 提供了基于 TLV 的新型 Vortex 传输协议,以提升 P2P 下载性能。使用 TLV(Tag-Length-Value)格式作为轻量级协议,替代 gRPC 进行节点间的数据传输。
与 gRPC 相比,基于 TCP 的 Vortex 可将大文件下载时间减少 50%,基于 QUIC 的 Vortex 可减少 40%,两者都能有效降低峰值内存使用。

更多信息,请参阅 [TCP Protocol Support for P2P File Transfer](https://github.com/dragonflyoss/design/blob/main/systems-analysis/tcp-protocol/tcp-protocol.md) 和 [QUIC Protocol Support for P2P File Transfer](https://github.com/dragonflyoss/design/blob/main/systems-analysis/quic-protocol/quic-protocol.md)。

### Request SDK

新增 SDK 用于将用户请求路由到 Seed Peer,使用一致性哈希算法,取代之前的 Kubernetes Service 负载均衡方式。

![p2](images/p2.svg)

更多信息,请参阅 [Request SDK](https://d7y.io/docs/next/advanced-guides/request-sdk/)。

### 指定集群 ID 实现多集群 Kubernetes 简化部署

Dragonfly 支持一种简化功能,用于部署和管理多个 Kubernetes 集群,方法是为每个集群显式分配一个 `schedulerClusterID`。
这种方式允许用户直接控制集群亲和性,而无需依赖基于位置的调度元数据,例如 IDC、主机名或 IP 地址。

使用此功能,每个 Peer、Seed Peer 和 Scheduler 都通过明确定义的 Scheduler Cluster ID 来确定其目标调度器集群。
这确保了集群之间的精确隔离以及可预测的跨集群行为。

![p3](images/p3.png)

更多信息,请参阅 [Create Dragonfly Cluster Simple](https://d7y.io/docs/next/getting-started/quick-start/multi-cluster-kubernetes/#create-dragonfly-cluster-simple)。

### Manager 和 Scheduler 组件的性能和资源优化

增强了 Manager 和 Scheduler 组件的性能和资源利用率,同时显著降低了 CPU 和内存开销,提升了系统效率和资源管理能力。

### 增强预热功能

- 支持预热到指定节点:采用基于优先级的选择逻辑,其中 IP 指定具有最高优先级,其次是基于数量的选择,最后是基于百分比的选择。

- 支持在单个预热任务中预热多个 URL。

- 支持通过 Scheduler gRPC 接口预热文件和镜像。

![p4](images/p4.png)

### 基于镜像 Blob SHA256 计算 ID 以避免重复下载

客户端现已支持直接根据镜像 Blob 的 SHA256 哈希值计算任务 ID,而不再使用下载 URL。此增强功能可以防止当同一 Blob 从不同镜像仓库
域名访问时产生的重复下载和数据冗余问题,新版本默认启用此功能。

### 缓存 HTTP 307 重定向 URL

新增对 HTTP 307(临时重定向)响应 URL 的缓存支持,以优化 Dragonfly 的多分片下载性能。当下载 URL 被拆分为多个分片时,
重定向目标地址现在会被缓存,从而消除冗余的重定向请求并降低延迟。

### Go 客户端已弃用,并由 Rust 客户端取代

Go 客户端已被弃用,并由 [Rust 客户端](https://github.com/dragonflyoss/client) 取代。未来所有的开发和维护工作将专注于 Rust 客户端,它提供了更好的性能、稳定性和可靠性。

更多信息,请参阅 [dragoflyoss/client](https://github.com/dragonflyoss/client)。

### 附加功能增强

- 在 Rust 客户端中启用 ARM64 架构的 64K 页面大小支持。
- 修复 dfget 版本输出中缺失的 git commit 元数据问题。
- 支持 containerd V3 配置中 `io.containerd.cri.v1.images` 插件的 `config_path` 配置。
- 在 `reqwest` 中使用 `hickory-dns` 替换 `glibc DNS` 解析器,以实现 DNS 缓存并防止分片下载过程中过多的 DNS 查询。
- 支持 `--include-files` 配置参数,可选择性地从目录中下载文件。
- 添加 `--no-progress` 配置参数,可禁用下载进度条输出。
- 支持 `backend` 操作中的自定义请求头,为 HTTP 请求提供灵活的请求头配置。
- 重构日志输出,减少冗余日志记录并提高日志可读性。

## 重要修复

- 将数据库字段类型从 `text` 修改为 `longtext`,以支持存储预热任务的信息。
- 修复了 Scheduler 关闭期间重复停止 Seed Peer 服务时导致的 panic 问题。
- 修复了在指定 Redis 密码但未设置用户名时,Broker 认证失败的问题。

## Nydus

### 新增特性

- Nydusd:为 RAFS V5 和 V6 格式添加运行时 CRC32 校验支持,增强数据完整性验证。
- Nydusd:支持在 Nydusd 恢复过程中重新发送 FUSE 请求,提高进程恢复的可靠性。
- Nydusd:增强 VFS 状态保存机制,用于进程热升级和故障转移。
- Nydusify:支持 Nydus 到 OCI 的反向转换能力(实验性功能),支持无缝迁移回 OCI 格式。
- Nydusify:实现镜像拷贝时零磁盘传输,显著减少复制操作期间对本地磁盘的使用。
- Snapshotter:将 `blob.meta` 打包到 Bootstrap 中,提高 RAFS V6 镜像的 Blob 下载时的可靠性

### 重要修复

- Nydusd:修复镜像仓库认证中 `access_token` 字段的认证令牌获取问题。
- Nydusd:为 umount API 添加了递归的 inode/dentry 失效处理。
- Nydus Image:修复 optimize 子命令中的多个问题,并添加后端配置支持。
- Snapshotter:为代理模式实现延迟父快照恢复,修复父快照缺失问题。

## 其他

您可以在 [CHANGELOG](https://github.com/dragonflyoss/dragonfly/blob/main/CHANGELOG.md) 中查看更多详细信息。

## 链接

- Dragonfly 网站:<https://d7y.io/>
- Dragonfly 仓库:<https://github.com/dragonflyoss/dragonfly>
- Dragonfly 客户端仓库:<https://github.com/dragonflyoss/client>
- Dragonfly 控制台仓库:<https://github.com/dragonflyoss/console>
- Dragonfly Charts 仓库:<https://github.com/dragonflyoss/helm-charts>
- Dragonfly 监控仓库:<https://github.com/dragonflyoss/monitoring>

## Dragonfly Github

![qrcode](images/qrcode.webp)
131 changes: 131 additions & 0 deletions releases/2.4.0/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# Dragonfly v2.4.0 is released

Dragonfly v2.4.0 is released!🎉🎉🎉 Thanks the [contributors](https://github.com/dragonflyoss/dragonfly/graphs/contributors) who made this release happend and welcome you to visit [d7y.io](https://d7y.io) website.

![dragonfly](images/dragonfly-linear.png)

## New features and enhancements

### load-aware scheduling algorithm

A two-stage scheduling algorithm combining central scheduling with node-level secondary scheduling to
optimize P2P download performance based on real-time load awareness.

![p1](images/p1.svg)

For more information, please refer to the [Scheduling](https://d7y.io/docs/next/operations/deployment/applications/scheduler/#scheduling).

### Vortex Protocol Support for P2P File Transfer

Dragonfly provide the new Vortex transfer protocol based on TLV to improve the download performance in internal network.
Use the TLV (Tag-Length-Value) format as a lightweight protocol to replace gRPC for data transfer between peers.
TCP-based Vortex reduces large file download time by `50%` and QUIC-based Vortex by `40%` compared to gRPC, both effectively reducing peak memory usage.

For more information, please refer to the [TCP Protocol Support for P2P File Transfer](https://github.com/dragonflyoss/design/blob/main/systems-analysis/tcp-protocol/tcp-protocol.md) and [QUIC Protocol Support for P2P File Transfer](https://github.com/dragonflyoss/design/blob/main/systems-analysis/quic-protocol/quic-protocol.md).

### Request SDK

A SDK for routing User requests to Seed Peers using consistent hashing, replacing the previous Kubernetes Service load balancing approach.

![p2](images/p2.svg)

For more details, please refer to [Request SDK](https://d7y.io/docs/next/advanced-guides/request-sdk/).

### Simple Multi‑Cluster Kubernetes Deployment with Scheduler Cluster ID

Dragonfly supports a simplified feature for deploying and managing multiple Kubernetes clusters by explicitly assigning a `schedulerClusterID` to each cluster.
This approach allows users to directly control cluster affinity without relying on location‑based scheduling metadata such as IDC, hostname, or IP.

Using this feature, each Peer, Seed Peer, and Scheduler determines its target scheduler cluster through a clearly defined scheduler cluster ID.
This ensures precise separation between clusters and predictable cross‑cluster behavior.

![p3](images/p3.png)

For more information, please refer to the [Create Dragonfly Cluster Simple](https://d7y.io/docs/next/getting-started/quick-start/multi-cluster-kubernetes/#create-dragonfly-cluster-simple).

### Performance and Resource Optimization for Manager and Scheduler Components

Enhanced service performance and resource utilization across Manager and Scheduler components while significantly reducing
CPU and memory overhead, delivering improved system efficiency and better resource management.

### Enhanced Preheating

- Support for IP-based peer selection in preheating jobs with priority-based selection logic where IP specification
takes highest priority, followed by count-based and percentage-based selection

- Support for preheating multiple URLs in a single request.

- Support for preheating file and image via Scheduler gRPC interface.

![p4](images/p4.png)

### Calculate task ID based on image blob SHA256 to avoid redundant downloads

The Client now supports calculating task IDs directly from the SHA256 hash of image blobs instead of using the download URL.
This enhancement prevents redundant downloads and data duplication when the same blob is accessed from
different registry domains.

### Cache HTTP 307 redirects for split downloads

Support for caching HTTP 307 (Temporary Redirect) responses to optimize Dragonfly's multi-piece download performance.
When a download URL is split into multiple pieces, the redirect target is now cached,
eliminating redundant redirect requests and reducing latency.

### Go Client Deprecated and Replaced by Rust Client

The Go client has been deprecated and replaced by the [Rust Client](https://github.com/dragonflyoss/client). All future development and maintenance will focus
exclusively on the Rust client, which offers improved performance, stability, and reliability.

For more information, please refer to the [dragoflyoss/client](https://github.com/dragonflyoss/client).

### Additional Enhancements

- Enable 64K page size support for ARM64 in the Dragonfly Rust client.
- Fix missing git commit metadata in dfget version output.
- Support for `config_path` of `io.containerd.cri.v1.images` plugin for containerd V3 configuration.
- Replaces glibc DNS resolver with hickory-dns in reqwest to implement DNS caching and prevent excessive DNS lookups during piece downloads.
- Support for the --include-files flag to selectively download files from a directory.
- Add the `--no-progress flag` to disable the download progress bar output.
- Support for custom request headers in backend operations, enabling flexible header configuration for HTTP requests.
- Refactored log output to reduce redundant logging and improve overall logging efficiency.

## Significant bug fixes

- Modified the database field type from `text` to `longtext` to support storing the information of preheating job.
- Fixed panic on repeated seed peer service stops during Scheduler shutdown.
- Fixed broker authentication failure when specifying the Redis password without setting a username.

## Nydus

### New features and enhancements

- Nydusd: Add CRC32 validation support for both RAFS V5 and V6 formats, enhancing data integrity verification.
- Nydusd: Support resending FUSE requests during nydusd restoration, improving daemon recovery reliability.
- Nydusd: Enhance VFS state saving mechanism for daemon hot upgrade and failover.
- Nydusify: Introduce Nydus-to-OCI reverse conversion capability, enabling seamless migration back to OCI format.
- Nydusify: Implement zero-disk transfer for image copy, significantly reducing local disk usage during copy operations.
- Snapshotter: Builtin blob.meta in bootstrap for blob fetch reliability for RAFS v6 image.

### Significant bug fixes

- Nydusd: Fix auth token fetching for `access_token` field in registry authentication.
- Nydusd: Add recursive inode/dentry invalidation for umount API.
- Nydus Image: Fix multiple issues in optimize subcommand and add backend configuration support.
- Snapshotter: Implement lazy parent recovery for proxy mode to handle missing parent snapshots.

## Others

You can see [CHANGELOG](https://github.com/dragonflyoss/dragonfly/blob/main/CHANGELOG.md) for more details.

## Links

- Dragonfly Website: <https://d7y.io/>
- Dragonfly Repository: <https://github.com/dragonflyoss/dragonfly>
- Dragonfly Client Repository: <https://github.com/dragonflyoss/client>
- Dragonfly Console Repository: <https://github.com/dragonflyoss/console>
- Dragonfly Charts Repository: <https://github.com/dragonflyoss/helm-charts>
- Dragonfly Monitor Repository: <https://github.com/dragonflyoss/monitoring>

## Dragonfly Github

![qrcode](images/qrcode.webp)
Binary file added releases/2.4.0/images/dragonfly-linear.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions releases/2.4.0/images/p1.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions releases/2.4.0/images/p2.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added releases/2.4.0/images/p3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added releases/2.4.0/images/p4.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added releases/2.4.0/images/qrcode.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.