From d1defad30bf1a0d3b4878441b7164611f56654d7 Mon Sep 17 00:00:00 2001 From: MizukiCry Date: Fri, 3 Mar 2023 21:22:38 +0800 Subject: [PATCH 01/14] fix some typos Signed-off-by: MizukiCry --- README.md | 4 ++-- README_cn.md | 2 +- docs/en/snapshotcloneserver_en.md | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 28994ed406..f5ccc69d36 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@
-**A could-native distributed storage system** +**A cloud-native distributed storage system** #### English | [简体中文](README_cn.md) ### 📄 [Documents](https://github.com/opencurve/curve/tree/master/docs) || 🌐 [Official Website](https://www.opencurve.io/Curve/HOME) || 🏠 [Forum](https://ask.opencurve.io/t/topic/7) @@ -46,7 +46,7 @@ ## About Curve -**Curve** is a mordern storage system developed by netease, currently supporting file storage(CurveFS) and block storage(CurveBS). Now it's hosted at [CNCF](https://www.cncf.io/) as a sandbox project. +**Curve** is a modern storage system developed by netease, currently supporting file storage(CurveFS) and block storage(CurveBS). Now it's hosted at [CNCF](https://www.cncf.io/) as a sandbox project. The core application scenarios of CurveBS mainly include: - the performance, mixed, capacity cloud disk or persistent volume of virtual machine/container, and remote disks of physical machines diff --git a/README_cn.md b/README_cn.md index 8e98b3e8cd..227b7752e6 100644 --- a/README_cn.md +++ b/README_cn.md @@ -4,7 +4,7 @@
-**A could-native distributed storage system** +**A cloud-native distributed storage system** #### [English](README.md) | 简体中文 ### 📄 [文档](https://github.com/opencurve/curve/tree/master/docs) || 🌐 [官网](https://www.opencurve.io/Curve/HOME) || 🏠 [论坛](https://ask.opencurve.io/t/topic/7) diff --git a/docs/en/snapshotcloneserver_en.md b/docs/en/snapshotcloneserver_en.md index c527450a92..137d3327e8 100644 --- a/docs/en/snapshotcloneserver_en.md +++ b/docs/en/snapshotcloneserver_en.md @@ -8,7 +8,7 @@ Snapshot is the read-only copy of the entire data of a cloud disk at a certain moment, and is an convenient and efficient way for data disaster tolerance, commonly used in data backup, creating images and application disaster tolerance. Snapshot system provides users an snapshot service interface, by which users can create, delete or cancel a snapshot, or recover data from snapshot, and even create their own image. -The snapshot service works as an independent component from core services of Curve, and it supports multi-level snapshot, which means a full backup for the first snapshot and incremental snapshot for the following ones. In the first snapshot, data will be stored entirely on S3, and only the modified data will be stored in the following tasks for saving spaces. The storing of snapshot data to S3 is asynchronous. Leader election and a high availability is implemanted by Etcd, and unfinished tasks will be resume automatically when the service restart. +The snapshot service works as an independent component from core services of Curve, and it supports multi-level snapshot, which means a full backup for the first snapshot and incremental snapshot for the following ones. In the first snapshot, data will be stored entirely on S3, and only the modified data will be stored in the following tasks for saving spaces. The storing of snapshot data to S3 is asynchronous. Leader election and a high availability is implemented by Etcd, and unfinished tasks will be resume automatically when the service restart. ### 1.2 Curve Snapshot Architecture @@ -76,7 +76,7 @@ Please refer to the [document](../cn/snapshotcloneserver_interface.md) of snapsh ### 2.1 Intro to Curve Clone Module -In section 1 we introduced the snapshot system, which is for creating snapshots of files of curvefs and incrementally dump the data to S3. But for a complete snapshot system, supporting only the snapshot creation is not enough, file recovery and cloning is the reason why take snapshot. In this section we introduce Curve cloning systerm(recovering can be considered as cloning to some extent). +In section 1 we introduced the snapshot system, which is for creating snapshots of files of curvefs and incrementally dump the data to S3. But for a complete snapshot system, supporting only the snapshot creation is not enough, file recovery and cloning is the reason why take snapshot. In this section we introduce Curve cloning system(recovering can be considered as cloning to some extent). According to the data source, there are two kinds of cloning, including cloning from snapshot and cloning from image, and if dividing by whether the data is entirely cloned before the service is provided, it can be divided into Lazy Clone and Not-Lazy Clone. From 4dfec3c6262b4b9ec9452b7b79ab2b4c5eb8e676 Mon Sep 17 00:00:00 2001 From: wanghai01 Date: Wed, 30 Nov 2022 15:52:18 +0800 Subject: [PATCH 02/14] expose chunkserver and snapshotcloneserver config to metric --- src/chunkserver/chunkserver.cpp | 1 + src/snapshotcloneserver/main.cpp | 1 + 2 files changed, 2 insertions(+) diff --git a/src/chunkserver/chunkserver.cpp b/src/chunkserver/chunkserver.cpp index 41ce5cc7ca..c50ac32300 100644 --- a/src/chunkserver/chunkserver.cpp +++ b/src/chunkserver/chunkserver.cpp @@ -103,6 +103,7 @@ int ChunkServer::Run(int argc, char** argv) { // 打印参数 conf.PrintConfig(); + conf.ExposeMetric("chunkserver_config"); curve::common::ExposeCurveVersion(); // ============================初始化各模块==========================// diff --git a/src/snapshotcloneserver/main.cpp b/src/snapshotcloneserver/main.cpp index c200f38b25..b44468b857 100644 --- a/src/snapshotcloneserver/main.cpp +++ b/src/snapshotcloneserver/main.cpp @@ -79,6 +79,7 @@ int main(int argc, char **argv) { } LoadConfigFromCmdline(conf.get()); conf->PrintConfig(); + conf->ExposeMetric("snapshot_clone_server_config"); google::InitGoogleLogging(argv[0]); snapshotcloneserver_main(conf); } From 3b5600092aa0dd8bf1769369dce5e02b7f4e575e Mon Sep 17 00:00:00 2001 From: wanghai01 Date: Fri, 13 Jan 2023 15:42:11 +0800 Subject: [PATCH 03/14] curvebs: add some topo metric Signed-off-by: wanghai01 --- src/mds/topology/topology_metric.cpp | 14 ++++++++--- src/mds/topology/topology_metric.h | 28 +++++++++++++++------- test/mds/topology/test_topology_metric.cpp | 5 ++++ 3 files changed, 36 insertions(+), 11 deletions(-) diff --git a/src/mds/topology/topology_metric.cpp b/src/mds/topology/topology_metric.cpp index 09a4b11a6a..4f50cfa673 100644 --- a/src/mds/topology/topology_metric.cpp +++ b/src/mds/topology/topology_metric.cpp @@ -120,6 +120,8 @@ void TopologyMetricService::UpdateTopologyMetrics() { pid, std::move(lptr)).first; } + auto servers = topo_->GetServerInLogicalPool(pid); + it->second->serverNum.set_value(servers.size()); it->second->chunkServerNum.set_value( chunkServerMetricInfo.size()); it->second->copysetNum.set_value( @@ -269,9 +271,11 @@ void TopologyMetricService::UpdateTopologyMetrics() { } } - // cluster io + // cluster metric uint64_t clusterReadIOPS = 0, clusterWriteIOPS = 0, clusterReadRate = 0, clusterWriteRate = 0, copysetNum = 0; + uint32_t serverNum = 0; + uint64_t logicalCapacity = 0, logicalAlloc = 0; for (auto metricIter = gLogicalPoolMetrics.begin(); metricIter != gLogicalPoolMetrics.end(); metricIter++) { clusterReadIOPS += metricIter->second->readIOPS.get_value(); @@ -279,10 +283,12 @@ void TopologyMetricService::UpdateTopologyMetrics() { clusterReadRate += metricIter->second->readRate.get_value(); clusterWriteRate += metricIter->second->writeRate.get_value(); copysetNum += metricIter->second->copysetNum.get_value(); + serverNum += metricIter->second->serverNum.get_value(); + logicalCapacity += metricIter->second->logicalCapacity.get_value(); + logicalAlloc += metricIter->second->logicalAlloc.get_value(); } - auto servers = topo_->GetServerInCluster(); - gClusterMetrics->serverNum.set_value(servers.size()); + gClusterMetrics->serverNum.set_value(serverNum); gClusterMetrics->chunkServerNum.set_value(chunkservers.size()); gClusterMetrics->copysetNum.set_value(copysetNum); gClusterMetrics->logicalPoolNum.set_value(lPools.size()); @@ -290,6 +296,8 @@ void TopologyMetricService::UpdateTopologyMetrics() { gClusterMetrics->writeIOPS.set_value(clusterWriteIOPS); gClusterMetrics->readRate.set_value(clusterReadRate); gClusterMetrics->writeRate.set_value(clusterWriteRate); + gClusterMetrics->logicalCapacity.set_value(logicalCapacity); + gClusterMetrics->logicalAlloc.set_value(logicalAlloc); diff --git a/src/mds/topology/topology_metric.h b/src/mds/topology/topology_metric.h index 4fe06a6b62..06694adfa2 100644 --- a/src/mds/topology/topology_metric.h +++ b/src/mds/topology/topology_metric.h @@ -114,6 +114,8 @@ using ChunkServerMetricPtr = std::unique_ptr; struct LogicalPoolMetric { const std::string kTopologyLogicalPoolMetricPrefix = "topology_metric_logicalPool_"; + // server number + bvar::Status serverNum; // chunkserver number bvar::Status chunkServerNum; // copyset number @@ -186,6 +188,8 @@ struct LogicalPoolMetric { bvar::Status writeIOPS; explicit LogicalPoolMetric(const std::string &logicalPoolName) : + serverNum(kTopologyLogicalPoolMetricPrefix, + logicalPoolName + "_server_num", 0), chunkServerNum(kTopologyLogicalPoolMetricPrefix, logicalPoolName + "_chunkserver_num", 0), copysetNum(kTopologyLogicalPoolMetricPrefix, @@ -276,24 +280,32 @@ struct ClusterMetric { bvar::Status readIOPS; // IOPS of writing bvar::Status writeIOPS; + // logical total capacity + bvar::Status logicalCapacity; + // bytes have alloced + bvar::Status logicalAlloc; ClusterMetric() : serverNum(kTopologyClusterMetricPrefix, - + "_server_num", 0), + + "server_num", 0), chunkServerNum(kTopologyClusterMetricPrefix, - + "_chunkserver_num", 0), + + "chunkserver_num", 0), copysetNum(kTopologyClusterMetricPrefix, - + "_copyset_num", 0), + + "copyset_num", 0), logicalPoolNum(kTopologyClusterMetricPrefix, - + "_logicalpool_num", 0), + + "logicalpool_num", 0), readRate(kTopologyClusterMetricPrefix, - + "_readRate", 0), + + "readRate", 0), writeRate(kTopologyClusterMetricPrefix, - + "_writeRate", 0), + + "writeRate", 0), readIOPS(kTopologyClusterMetricPrefix, - + "_readIOPS", 0), + + "readIOPS", 0), writeIOPS(kTopologyClusterMetricPrefix, - + "_writeIOPS", 0) {} + + "writeIOPS", 0), + logicalCapacity(kTopologyClusterMetricPrefix, + + "logicalCapacity", 0), + logicalAlloc(kTopologyClusterMetricPrefix, + + "logicalAlloc", 0) {} }; using ClusterMetricPtr = std::unique_ptr; diff --git a/test/mds/topology/test_topology_metric.cpp b/test/mds/topology/test_topology_metric.cpp index e77d5178b0..b5638b77e0 100644 --- a/test/mds/topology/test_topology_metric.cpp +++ b/test/mds/topology/test_topology_metric.cpp @@ -307,6 +307,7 @@ TEST_F(TestTopologyMetric, TestUpdateTopologyMetricsOneLogicalPool) { gChunkServerMetrics[0x43]->chunkSizeTotalBytes.get_value()); ASSERT_EQ(1, gLogicalPoolMetrics.size()); + ASSERT_EQ(3, gLogicalPoolMetrics[logicalPoolId]->serverNum.get_value()); //NOLINT ASSERT_EQ(3, gLogicalPoolMetrics[logicalPoolId]->chunkServerNum.get_value()); //NOLINT ASSERT_EQ(1, gLogicalPoolMetrics[logicalPoolId]->copysetNum.get_value()); //NOLINT ASSERT_EQ(2, gLogicalPoolMetrics[logicalPoolId]->scatterWidthAvg.get_value()); //NOLINT @@ -348,6 +349,10 @@ TEST_F(TestTopologyMetric, TestUpdateTopologyMetricsOneLogicalPool) { ASSERT_EQ(3, gClusterMetrics->writeIOPS.get_value()); ASSERT_EQ(3, gClusterMetrics->readRate.get_value()); ASSERT_EQ(3, gClusterMetrics->writeRate.get_value()); + ASSERT_EQ(1, gClusterMetrics->logicalPoolNum.get_value()); + ASSERT_EQ(3, gClusterMetrics->serverNum.get_value()); + ASSERT_EQ(3, gClusterMetrics->chunkServerNum.get_value()); + ASSERT_EQ(1, gClusterMetrics->copysetNum.get_value()); } TEST_F(TestTopologyMetric, TestUpdateTopologyMetricsCleanRetired) { From 45b436696b095237a7b36efa147c16422d92f356 Mon Sep 17 00:00:00 2001 From: Ziy1-Tan Date: Tue, 7 Mar 2023 18:38:33 +0800 Subject: [PATCH 04/14] curvefs: add memcached startup check Signed-off-by: Ziy1-Tan --- curvefs/test/client/client_memcache_test.cpp | 23 +++++++++++--------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/curvefs/test/client/client_memcache_test.cpp b/curvefs/test/client/client_memcache_test.cpp index d6023f0ca1..5113043b5c 100644 --- a/curvefs/test/client/client_memcache_test.cpp +++ b/curvefs/test/client/client_memcache_test.cpp @@ -50,7 +50,7 @@ class MemCachedTest : public ::testing::Test { } else if (0 == memcached_pid) { std::string memcached_port = "-p " + std::to_string(port); - ASSERT_EQ(0, execlp("memcached", "memcached", "-u root", + ASSERT_EQ(0, execlp("memcached", "memcached", "-uroot", memcached_port.c_str(), nullptr)); } @@ -65,13 +65,16 @@ class MemCachedTest : public ::testing::Test { // wait memcached server start std::string errorlog; int retry = 0; + bool ret = false; do { - if (client->Set("1", "2", 1, &errorlog) || retry > 100) { + if ((ret = client->Set("1", "2", 1, &errorlog)) || retry > 100) { break; } std::this_thread::sleep_for(std::chrono::milliseconds(100)); retry++; } while (1); + ASSERT_TRUE(ret) + << absl::StrCat("memcache start failed, errorlog: ", errorlog); LOG(INFO) << "=============== memcache start ok"; } @@ -97,35 +100,35 @@ TEST_F(MemCachedTest, MultiThreadTask) { }; // set - CountDownEvent taskEnvent(5); + CountDownEvent taskEvent(5); for (int i = 0; i < 5; i++) { workers.emplace_back([&, i]() { auto task = std::make_shared( kvstr[i].first, kvstr[i].second.c_str(), kvstr[i].second.length()); task->done = - [&taskEnvent](const std::shared_ptr &task) { - taskEnvent.Signal(); + [&taskEvent](const std::shared_ptr &task) { + taskEvent.Signal(); }; manager_.Set(task); }); } - taskEnvent.Wait(); + taskEvent.Wait(); ASSERT_EQ(5, g_kvClientMetric->kvClientSet.latency.count()); // get for (int i = 0; i < 5; i++) { workers.emplace_back([&, i]() { - CountDownEvent taskEnvent(1); + CountDownEvent taskEvent(1); char *result = new char[4]; auto task = std::make_shared(kvstr[i].first, result, 0, 4); task->done = - [&taskEnvent](const std::shared_ptr &task) { - taskEnvent.Signal(); + [&taskEvent](const std::shared_ptr &task) { + taskEvent.Signal(); }; manager_.Get(task); - taskEnvent.Wait(); + taskEvent.Wait(); ASSERT_EQ(0, memcmp(result, kvstr[i].second.c_str(), 4)); ASSERT_TRUE(task->res); }); From 87664bd392f3a74c6d126333ebd37088f87f0c58 Mon Sep 17 00:00:00 2001 From: zweix123 <1979803044@qq.com> Date: Thu, 2 Mar 2023 07:09:26 +0000 Subject: [PATCH 05/14] develop 'curve bs update reset' (for 'curve_ops_tool reset-peer') 1. Adjust peer parsing error output 2. Increase plain output 3. add json output 4. more tests Signed-off-by: zweix123 <1979803044@qq.com> --- tools-v2/README.md | 24 +- tools-v2/internal/error/error.go | 2 +- tools-v2/internal/utils/row.go | 2 + tools-v2/pkg/cli/command/curvebs/bs.go | 2 + .../cli/command/curvebs/update/peer/peer.go | 208 ++++++++++++++++++ .../pkg/cli/command/curvebs/update/update.go | 52 +++++ 6 files changed, 288 insertions(+), 2 deletions(-) create mode 100644 tools-v2/pkg/cli/command/curvebs/update/peer/peer.go create mode 100644 tools-v2/pkg/cli/command/curvebs/update/update.go diff --git a/tools-v2/README.md b/tools-v2/README.md index 055d53f71e..290c0a6c50 100644 --- a/tools-v2/README.md +++ b/tools-v2/README.md @@ -54,6 +54,8 @@ A tool for CurveFS & CurveBs. - [staus mds](#staus-mds) - [delete](#delete-1) - [delete peer](#delete-peer) + - [update](#update) + - [update peer](#update-peer) - [Comparison of old and new commands](#comparison-of-old-and-new-commands) - [curve fs](#curve-fs) - [curve bs](#curve-bs) @@ -981,6 +983,26 @@ Output: ``` +### update + +#### update peer + +reset peer + +Usage: +```bash +curve bs update peer 127.0.0.0:8200:0 --logicalpoolid=1 --copysetid=1 +``` + +Output: +``` ++----------------------+---------+---------+--------+ +| PEER | COPYSET | RESULT | REASON | ++----------------------+---------+---------+--------+ +| 127.0.0.0:8200:0 | (1:1) | success | null | ++----------------------+---------+---------+--------+ +``` + ## Comparison of old and new commands ### curve fs @@ -1036,7 +1058,7 @@ Output: | check-consistency | | | remove-peer | curve bs delete peer | | transfer-leader | | -| reset-peer | | +| reset-peer | curve bs update peer | | do-snapshot | | | do-snapshot-all | | | check-chunkserver | | diff --git a/tools-v2/internal/error/error.go b/tools-v2/internal/error/error.go index 03b702a6ba..4e846293a1 100644 --- a/tools-v2/internal/error/error.go +++ b/tools-v2/internal/error/error.go @@ -280,7 +280,7 @@ var ( return NewInternalCmdError(14, "invalid %s: %s") } ErrSplitPeer = func() *CmdError { - return NewInternalCmdError(15, "split peer[%s] failed") + return NewInternalCmdError(15, "split peer[%s] failed, peer should be like: 127.0.0.1:8200:0") } ErrMarshalJson = func() *CmdError { return NewInternalCmdError(16, "marshal %s to json error, the error is: %s") diff --git a/tools-v2/internal/utils/row.go b/tools-v2/internal/utils/row.go index 2fa6dafa62..09147bfd4a 100644 --- a/tools-v2/internal/utils/row.go +++ b/tools-v2/internal/utils/row.go @@ -115,6 +115,8 @@ const ( ROW_VALUE_OFFLINE = "offline" ROW_VALUE_UNKNOWN = "unknown" ROW_VALUE_SUCCESS = "success" + ROW_VALUE_FAILED = "failed" + ROW_VALUE_NULL = "null" ) // topology type diff --git a/tools-v2/pkg/cli/command/curvebs/bs.go b/tools-v2/pkg/cli/command/curvebs/bs.go index 9ed0856c80..deb52faa85 100644 --- a/tools-v2/pkg/cli/command/curvebs/bs.go +++ b/tools-v2/pkg/cli/command/curvebs/bs.go @@ -31,6 +31,7 @@ import ( "github.com/opencurve/curve/tools-v2/pkg/cli/command/curvebs/list" "github.com/opencurve/curve/tools-v2/pkg/cli/command/curvebs/query" "github.com/opencurve/curve/tools-v2/pkg/cli/command/curvebs/status" + "github.com/opencurve/curve/tools-v2/pkg/cli/command/curvebs/update" ) type CurveBsCommand struct { @@ -46,6 +47,7 @@ func (bsCmd *CurveBsCommand) AddSubCommands() { status.NewStatusCommand(), delete.NewDeleteCommand(), create.NewCreateCmd(), + update.NewUpdateCommand(), ) } diff --git a/tools-v2/pkg/cli/command/curvebs/update/peer/peer.go b/tools-v2/pkg/cli/command/curvebs/update/peer/peer.go new file mode 100644 index 0000000000..f5a0306a47 --- /dev/null +++ b/tools-v2/pkg/cli/command/curvebs/update/peer/peer.go @@ -0,0 +1,208 @@ +/* + * Copyright (c) 2023 NetEase Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* + * Project: CurveCli + * Created Date: 2023-02-27 + * Author: zweix + */ + +package peer + +import ( + "context" + "fmt" + "strconv" + "strings" + "time" + + "github.com/spf13/cobra" + "google.golang.org/grpc" + + cmderror "github.com/opencurve/curve/tools-v2/internal/error" + cobrautil "github.com/opencurve/curve/tools-v2/internal/utils" + basecmd "github.com/opencurve/curve/tools-v2/pkg/cli/command" + "github.com/opencurve/curve/tools-v2/pkg/config" + "github.com/opencurve/curve/tools-v2/pkg/output" + "github.com/opencurve/curve/tools-v2/proto/proto/cli2" + "github.com/opencurve/curve/tools-v2/proto/proto/common" +) + +const ( + updateExample = `$ curve bs update peer 127.0.0.0:8200:0 --logicalpoolid=1 --copysetid=1` +) + +type ResetRpc struct { + Info *basecmd.Rpc + Request *cli2.ResetPeerRequest2 + Client cli2.CliService2Client +} + +func (rRpc *ResetRpc) NewRpcClient(cc grpc.ClientConnInterface) { + rRpc.Client = cli2.NewCliService2Client(cc) +} + +func (rRpc *ResetRpc) Stub_Func(ctx context.Context) (interface{}, error) { + return rRpc.Client.ResetPeer(ctx, rRpc.Request) +} + +// Options rpc request options. +type Options struct { + Timeout time.Duration + RetryTimes int32 +} + +type PeerCommand struct { + basecmd.FinalCurveCmd + + // request parameters + opts Options + logicalPoolID uint32 + copysetID uint32 + + resetPeer *common.Peer +} + +var _ basecmd.FinalCurveCmdFunc = (*PeerCommand)(nil) // check interface + +// NewCommand ... +func NewPeerCommand() *cobra.Command { + peerCmd := &PeerCommand{ + FinalCurveCmd: basecmd.FinalCurveCmd{ + Use: "peer", + Short: "update(reset) the peer from the copyset", + Example: updateExample, + }, + } + basecmd.NewFinalCurveCli(&peerCmd.FinalCurveCmd, peerCmd) + return peerCmd.Cmd +} + +func (pCmd *PeerCommand) AddFlags() { + config.AddRpcRetryTimesFlag(pCmd.Cmd) + config.AddRpcTimeoutFlag(pCmd.Cmd) + + config.AddBSLogicalPoolIdFlag(pCmd.Cmd) + config.AddBSCopysetIdFlag(pCmd.Cmd) +} + +// ParsePeer parse the peer string +func ParsePeer(peer string) (*common.Peer, *cmderror.CmdError) { + cs := strings.Split(peer, ":") + if len(cs) != 3 { + pErr := cmderror.ErrSplitPeer() + pErr.Format(peer) + return nil, pErr + } + id, err := strconv.ParseUint(cs[2], 10, 64) + if err != nil { + pErr := cmderror.ErrSplitPeer() + pErr.Format(peer) + return nil, pErr + } + cs = cs[:2] + address := strings.Join(cs, ":") + return &common.Peer{ + Id: &id, + Address: &address, + }, nil +} + +func (pCmd *PeerCommand) Init(cmd *cobra.Command, args []string) error { + pCmd.opts = Options{} + + pCmd.SetHeader([]string{cobrautil.ROW_PEER, cobrautil.ROW_COPYSET, cobrautil.ROW_RESULT, cobrautil.ROW_REASON}) + pCmd.TableNew.SetAutoMergeCellsByColumnIndex(cobrautil.GetIndexSlice( + pCmd.Header, []string{}, + )) + + var err error + var e *cmderror.CmdError + + pCmd.opts.Timeout = config.GetFlagDuration(pCmd.Cmd, config.RPCTIMEOUT) + pCmd.opts.RetryTimes = config.GetFlagInt32(pCmd.Cmd, config.RPCRETRYTIMES) + + pCmd.copysetID, err = config.GetBsFlagUint32(pCmd.Cmd, config.CURVEBS_COPYSET_ID) + if err != nil { + return err + } + + pCmd.logicalPoolID, err = config.GetBsFlagUint32(pCmd.Cmd, config.CURVEBS_LOGIC_POOL_ID) + if err != nil { + return err + } + + // parse peer conf + if len(args) < 1 { + pErr := cmderror.ErrGetPeer() + pErr.Format("should specified the peer address") + return pErr.ToError() + } + pCmd.resetPeer, e = ParsePeer(args[0]) + if e != nil { + return e.ToError() + } + return nil +} + +func (pCmd *PeerCommand) Print(cmd *cobra.Command, args []string) error { + return output.FinalCmdOutput(&pCmd.FinalCurveCmd, pCmd) +} + +func (pCmd *PeerCommand) RunCommand(cmd *cobra.Command, args []string) error { + out := make(map[string]string) + out[cobrautil.ROW_PEER] = fmt.Sprintf("%s:%d", pCmd.resetPeer.GetAddress(), pCmd.resetPeer.GetId()) + out[cobrautil.ROW_COPYSET] = fmt.Sprintf("(%d:%d)", pCmd.logicalPoolID, pCmd.copysetID) + + var err *cmderror.CmdError + pCmd.Result, err = pCmd.execResetPeer() + if err != nil { + out[cobrautil.ROW_RESULT] = cobrautil.ROW_VALUE_FAILED + out[cobrautil.ROW_REASON] = err.ToError().Error() + pCmd.Error = err + return nil + } + out[cobrautil.ROW_RESULT] = cobrautil.ROW_VALUE_SUCCESS + out[cobrautil.ROW_REASON] = cobrautil.ROW_VALUE_NULL + list := cobrautil.Map2List(out, pCmd.Header) + pCmd.TableNew.Append(list) + return nil +} + +func (pCmd *PeerCommand) ResultPlainOutput() error { + return output.FinalCmdOutputPlain(&pCmd.FinalCurveCmd) +} + +func (pCmd *PeerCommand) execResetPeer() (interface{}, *cmderror.CmdError) { + rpcCli := &ResetRpc{ + Info: basecmd.NewRpc([]string{pCmd.resetPeer.GetAddress()}, pCmd.opts.Timeout, pCmd.opts.RetryTimes, "RsetPeer"), + Request: &cli2.ResetPeerRequest2{ + LogicPoolId: &pCmd.logicalPoolID, + CopysetId: &pCmd.copysetID, + RequestPeer: pCmd.resetPeer, + OldPeers: []*common.Peer{pCmd.resetPeer}, + NewPeers: []*common.Peer{pCmd.resetPeer}, + }, + } + + response, errCmd := basecmd.GetRpcResponse(rpcCli.Info, rpcCli) + + if errCmd.TypeCode() != cmderror.CODE_SUCCESS { + return response, errCmd + } + + return response, nil +} diff --git a/tools-v2/pkg/cli/command/curvebs/update/update.go b/tools-v2/pkg/cli/command/curvebs/update/update.go new file mode 100644 index 0000000000..1032cd31eb --- /dev/null +++ b/tools-v2/pkg/cli/command/curvebs/update/update.go @@ -0,0 +1,52 @@ +/* + * Copyright (c) 2022 NetEase Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* + * Project: CurveCli + * Created Date: 2023-02-27 + * Author: zweix + */ + +package update + +import ( + "github.com/spf13/cobra" + + basecmd "github.com/opencurve/curve/tools-v2/pkg/cli/command" + "github.com/opencurve/curve/tools-v2/pkg/cli/command/curvebs/update/peer" +) + +type UpdateCommand struct { + basecmd.MidCurveCmd +} + +var _ basecmd.MidCurveCmdFunc = (*UpdateCommand)(nil) + +func (updateCmd *UpdateCommand) AddSubCommands() { + updateCmd.Cmd.AddCommand( + peer.NewPeerCommand(), + ) +} + +func NewUpdateCommand() *cobra.Command { + updateCmd := &UpdateCommand{ + basecmd.MidCurveCmd{ + Use: "update", + Short: "update resources in the curvebs", + }, + } + return basecmd.NewMidCurveCli(&updateCmd.MidCurveCmd, updateCmd) +} From bc369887112c464f03b88380573e0a86e76f5ad4 Mon Sep 17 00:00:00 2001 From: Cyber-SiKu Date: Thu, 23 Feb 2023 17:19:28 +0800 Subject: [PATCH 06/14] [feat]curvefs/client:add memcache to warmup Signed-off-by: Cyber-SiKu --- curvefs/src/client/fuse_s3_client.cpp | 14 +++--- curvefs/src/client/fuse_s3_client.h | 3 +- .../src/client/kvclient/kvclient_manager.cpp | 16 +++---- .../src/client/kvclient/kvclient_manager.h | 14 +++--- curvefs/src/client/s3/client_s3_adaptor.cpp | 5 ++- curvefs/src/client/s3/client_s3_adaptor.h | 4 ++ .../src/client/s3/client_s3_cache_manager.cpp | 27 +++++++----- .../src/client/s3/client_s3_cache_manager.h | 31 +++++++++---- curvefs/src/client/warmup/warmup_manager.cpp | 35 +++++++++------ curvefs/src/client/warmup/warmup_manager.h | 43 ++++++++++++------- .../test/client/chunk_cache_manager_test.cpp | 24 +++++------ curvefs/test/client/client_memcache_test.cpp | 3 +- .../client/client_s3_adaptor_Integration.cpp | 15 ++++--- .../test/client/client_s3_adaptor_test.cpp | 7 +-- curvefs/test/client/data_cache_test.cpp | 13 +++--- .../test/client/file_cache_manager_test.cpp | 19 ++++---- curvefs/test/client/fs_cache_manager_test.cpp | 18 ++++---- curvefs/test/client/mock_client_s3_adaptor.h | 4 +- .../client/mock_client_s3_cache_manager.h | 9 ++-- curvefs/test/client/test_fuse_s3_client.cpp | 2 +- tools-v2/internal/error/error.go | 14 +++--- 21 files changed, 192 insertions(+), 128 deletions(-) diff --git a/curvefs/src/client/fuse_s3_client.cpp b/curvefs/src/client/fuse_s3_client.cpp index 93d5ca81b8..9d291aa69e 100644 --- a/curvefs/src/client/fuse_s3_client.cpp +++ b/curvefs/src/client/fuse_s3_client.cpp @@ -70,7 +70,7 @@ CURVEFS_ERROR FuseS3Client::Init(const FuseClientOption &option) { auto fsCacheManager = std::make_shared( dynamic_cast(s3Adaptor_.get()), opt.s3Opt.s3ClientAdaptorOpt.readCacheMaxByte, - opt.s3Opt.s3ClientAdaptorOpt.writeCacheMaxByte); + opt.s3Opt.s3ClientAdaptorOpt.writeCacheMaxByte, kvClientManager_); if (opt.s3Opt.s3ClientAdaptorOpt.diskCacheOpt.diskCacheType != DiskCacheType::Disable) { auto s3DiskCacheClient = std::make_shared(); @@ -84,11 +84,11 @@ CURVEFS_ERROR FuseS3Client::Init(const FuseClientOption &option) { diskCacheManager, s3DiskCacheClient); ret = s3Adaptor_->Init(opt.s3Opt.s3ClientAdaptorOpt, s3Client, inodeManager_, mdsClient_, fsCacheManager, - diskCacheManagerImpl, true); + diskCacheManagerImpl, kvClientManager_, true); } else { ret = s3Adaptor_->Init(opt.s3Opt.s3ClientAdaptorOpt, s3Client, inodeManager_, mdsClient_, fsCacheManager, - nullptr, true); + nullptr, kvClientManager_, true); } return ret; } @@ -112,13 +112,17 @@ bool FuseS3Client::InitKVCache(const KVClientManagerOpt &opt) { return false; } - g_kvClientManager = new KVClientManager(); - if (!g_kvClientManager->Init(opt, memcacheClient)) { + kvClientManager_ = std::make_shared(); + if (!kvClientManager_->Init(opt, memcacheClient)) { LOG(ERROR) << "FLAGS_supportKVcache = " << FLAGS_supportKVcache << ", but init kvClientManager fail"; return false; } + if (warmupManager_ != nullptr) { + warmupManager_->SetKVClientManager(kvClientManager_); + } + return true; } diff --git a/curvefs/src/client/fuse_s3_client.h b/curvefs/src/client/fuse_s3_client.h index 1d1ec6062c..5e6ab3ab97 100644 --- a/curvefs/src/client/fuse_s3_client.h +++ b/curvefs/src/client/fuse_s3_client.h @@ -57,7 +57,7 @@ class FuseS3Client : public FuseClient { }; warmupManager_ = std::make_shared( metaClient_, inodeManager_, dentryManager_, fsInfo_, readFunc, - s3Adaptor_); + s3Adaptor_, nullptr); } FuseS3Client(const std::shared_ptr &mdsClient, @@ -118,6 +118,7 @@ class FuseS3Client : public FuseClient { private: // s3 adaptor std::shared_ptr s3Adaptor_; + std::shared_ptr kvClientManager_; }; diff --git a/curvefs/src/client/kvclient/kvclient_manager.cpp b/curvefs/src/client/kvclient/kvclient_manager.cpp index cac4b0b43b..1691af6119 100644 --- a/curvefs/src/client/kvclient/kvclient_manager.cpp +++ b/curvefs/src/client/kvclient/kvclient_manager.cpp @@ -21,6 +21,7 @@ */ #include "curvefs/src/client/kvclient/kvclient_manager.h" +#include #include "src/client/client_metric.h" #include "src/common/concurrent/count_down_event.h" @@ -31,32 +32,27 @@ using curvefs::client::metric::KVClientMetric; namespace curvefs { namespace client { -KVClientManager *g_kvClientManager = nullptr; -KVClientMetric *g_kvClientMetric = nullptr; - #define ONRETURN(TYPE, RES) \ if (RES) { \ - g_kvClientMetric->kvClient##TYPE.qps.count << 1; \ + kvClientMetric_.kvClient##TYPE.qps.count << 1; \ } else { \ - g_kvClientMetric->kvClient##TYPE.eps.count << 1; \ - } + kvClientMetric_.kvClient##TYPE.eps.count << 1; \ + } \ bool KVClientManager::Init(const KVClientManagerOpt &config, const std::shared_ptr &kvclient) { client_ = kvclient; - g_kvClientMetric = new KVClientMetric(); return threadPool_.Start(config.setThreadPooln) == 0; } void KVClientManager::Uninit() { client_->UnInit(); threadPool_.Stop(); - delete g_kvClientMetric; } void KVClientManager::Set(std::shared_ptr task) { threadPool_.Enqueue([task, this]() { - LatencyGuard guard(&g_kvClientMetric->kvClientSet.latency); + LatencyGuard guard(&kvClientMetric_.kvClientSet.latency); std::string error_log; auto res = @@ -69,7 +65,7 @@ void KVClientManager::Set(std::shared_ptr task) { void KVClientManager::Get(std::shared_ptr task) { threadPool_.Enqueue([task, this]() { - LatencyGuard guard(&g_kvClientMetric->kvClientGet.latency); + LatencyGuard guard(&kvClientMetric_.kvClientGet.latency); std::string error_log; task->res = client_->Get(task->key, task->value, task->offset, diff --git a/curvefs/src/client/kvclient/kvclient_manager.h b/curvefs/src/client/kvclient/kvclient_manager.h index 19c08c3278..7214a64b6e 100644 --- a/curvefs/src/client/kvclient/kvclient_manager.h +++ b/curvefs/src/client/kvclient/kvclient_manager.h @@ -48,9 +48,6 @@ class GetKVCacheTask; using curve::common::TaskThreadPool; using curvefs::client::common::KVClientManagerOpt; -extern KVClientManager *g_kvClientManager; -extern KVClientMetric *g_kvClientMetric; - typedef std::function &)> SetKVCacheDone; typedef std::function &)> @@ -62,10 +59,10 @@ struct SetKVCacheTask { uint64_t length; SetKVCacheDone done; SetKVCacheTask() = default; - SetKVCacheTask(const std::string &k, const char *val, const uint64_t len) - : key(k), value(val), length(len) { - done = [](const std::shared_ptr &) {}; - } + SetKVCacheTask( + const std::string &k, const char *val, const uint64_t len, + SetKVCacheDone done = [](const std::shared_ptr &) {}) + : key(k), value(val), length(len), done(std::move(done)) {} }; struct GetKVCacheTask { @@ -98,12 +95,15 @@ class KVClientManager { void Get(std::shared_ptr task); + KVClientMetric *GetClientMetricForTesting() { return &kvClientMetric_; } + private: void Uninit(); private: TaskThreadPool threadPool_; std::shared_ptr client_; + KVClientMetric kvClientMetric_; }; } // namespace client diff --git a/curvefs/src/client/s3/client_s3_adaptor.cpp b/curvefs/src/client/s3/client_s3_adaptor.cpp index c39e79c5f5..0126ceac65 100644 --- a/curvefs/src/client/s3/client_s3_adaptor.cpp +++ b/curvefs/src/client/s3/client_s3_adaptor.cpp @@ -24,6 +24,7 @@ #include #include #include +#include #include "absl/memory/memory.h" #include "curvefs/src/client/s3/client_s3_adaptor.h" @@ -39,6 +40,7 @@ S3ClientAdaptorImpl::Init( std::shared_ptr mdsClient, std::shared_ptr fsCacheManager, std::shared_ptr diskCacheManagerImpl, + std::shared_ptr kvClientManager, bool startBackGround) { pendingReq_ = 0; blockSize_ = option.blockSize; @@ -66,6 +68,7 @@ S3ClientAdaptorImpl::Init( fsCacheManager_ = fsCacheManager; waitInterval_.Init(option.intervalSec * 1000); diskCacheManagerImpl_ = diskCacheManagerImpl; + kvClientManager_ = std::move(kvClientManager); if (HasDiskCache()) { diskCacheManagerImpl_ = diskCacheManagerImpl; if (diskCacheManagerImpl_->Init(option) < 0) { @@ -372,7 +375,7 @@ CURVEFS_ERROR S3ClientAdaptorImpl::FlushAllCache(uint64_t inodeId) { } // force flush data in diskcache to s3 - if (!g_kvClientManager && HasDiskCache()) { + if (!kvClientManager_ && HasDiskCache()) { VLOG(6) << "FlushAllCache, wait inodeId:" << inodeId << "related chunk upload to s3"; if (ClearDiskCache(inodeId) < 0) { diff --git a/curvefs/src/client/s3/client_s3_adaptor.h b/curvefs/src/client/s3/client_s3_adaptor.h index f6806324c5..80e57b2b4b 100644 --- a/curvefs/src/client/s3/client_s3_adaptor.h +++ b/curvefs/src/client/s3/client_s3_adaptor.h @@ -71,6 +71,7 @@ class S3ClientAdaptor { std::shared_ptr mdsClient, std::shared_ptr fsCacheManager, std::shared_ptr diskCacheManagerImpl, + std::shared_ptr kvClientManager, bool startBackGround = false) = 0; /** * @brief write data to s3 @@ -128,6 +129,7 @@ class S3ClientAdaptorImpl : public S3ClientAdaptor { std::shared_ptr mdsClient, std::shared_ptr fsCacheManager, std::shared_ptr diskCacheManagerImpl, + std::shared_ptr kvClientManager, bool startBackGround = false); /** * @brief write data to s3 @@ -274,6 +276,8 @@ class S3ClientAdaptorImpl : public S3ClientAdaptor { TaskThreadPool taskPool_; + + std::shared_ptr kvClientManager_ = nullptr; }; } // namespace client diff --git a/curvefs/src/client/s3/client_s3_cache_manager.cpp b/curvefs/src/client/s3/client_s3_cache_manager.cpp index b2b736357b..aa47d8ee67 100644 --- a/curvefs/src/client/s3/client_s3_cache_manager.cpp +++ b/curvefs/src/client/s3/client_s3_cache_manager.cpp @@ -96,7 +96,7 @@ FsCacheManager::FindOrCreateFileCacheManager(uint64_t fsId, uint64_t inodeId) { } FileCacheManagerPtr fileCacheManager = std::make_shared( - fsId, inodeId, s3ClientAdaptor_); + fsId, inodeId, s3ClientAdaptor_, kvClientManager_); auto ret = fileCacheManagerMap_.emplace(inodeId, fileCacheManager); g_s3MultiManagerMetric->fileManagerNum << 1; assert(ret.second); @@ -293,7 +293,8 @@ FileCacheManager::FindOrCreateChunkCacheManager(uint64_t index) { } ChunkCacheManagerPtr chunkCacheManager = - std::make_shared(index, s3ClientAdaptor_); + std::make_shared(index, s3ClientAdaptor_, + kvClientManager_); auto ret = chunkCacheMap_.emplace(index, chunkCacheManager); g_s3MultiManagerMetric->chunkManagerNum << 1; assert(ret.second); @@ -509,7 +510,7 @@ bool FileCacheManager::ReadKVRequestFromRemoteCache(const std::string &name, char *databuf, uint64_t offset, uint64_t length) { - if (!g_kvClientManager) { + if (!kvClientManager_) { return false; } @@ -519,7 +520,7 @@ bool FileCacheManager::ReadKVRequestFromRemoteCache(const std::string &name, event.Signal(); return; }; - g_kvClientManager->Get(task); + kvClientManager_->Get(task); event.Wait(); return task->res; @@ -616,7 +617,7 @@ int FileCacheManager::ReadKVRequest( WriteLockGuard writeLockGuard(chunkCacheManager->rwLockChunk_); DataCachePtr dataCache = std::make_shared( s3ClientAdaptor_, chunkCacheManager, chunkPos, req->len, - dataBuf + req->readOffset); + dataBuf + req->readOffset, kvClientManager_); chunkCacheManager->AddReadDataCache(dataCache); } } @@ -1477,8 +1478,9 @@ DataCachePtr ChunkCacheManager::FindWriteableDataCache( void ChunkCacheManager::WriteNewDataCache(S3ClientAdaptorImpl *s3ClientAdaptor, uint32_t chunkPos, uint32_t len, const char *data) { - DataCachePtr dataCache = std::make_shared( - s3ClientAdaptor, this->shared_from_this(), chunkPos, len, data); + DataCachePtr dataCache = + std::make_shared(s3ClientAdaptor, this->shared_from_this(), + chunkPos, len, data, kvClientManager_); VLOG(9) << "WriteNewDataCache chunkPos:" << chunkPos << ", len:" << len << ", new len:" << dataCache->GetLen() << ",chunkIndex:" << index_; WriteLockGuard writeLockGuard(rwLockWrite_); @@ -1733,7 +1735,8 @@ void ChunkCacheManager::AddWriteDataCacheForTest(DataCachePtr dataCache) { DataCache::DataCache(S3ClientAdaptorImpl *s3ClientAdaptor, ChunkCacheManagerPtr chunkCacheManager, uint64_t chunkPos, - uint64_t len, const char *data) + uint64_t len, const char *data, + std::shared_ptr kvClientManager) : s3ClientAdaptor_(std::move(s3ClientAdaptor)), chunkCacheManager_(chunkCacheManager), status_(DataCacheStatus::Dirty), inReadCache_(false) { @@ -1792,6 +1795,8 @@ DataCache::DataCache(S3ClientAdaptorImpl *s3ClientAdaptor, assert((actualLen_ % pageSize) == 0); assert((actualChunkPos_ % pageSize) == 0); createTime_ = ::curve::common::TimeUtility::GetTimeofDaySec(); + + kvClientManager_ = std::move(kvClientManager); } void DataCache::CopyBufToDataCache(uint64_t dataCachePos, uint64_t len, @@ -2312,7 +2317,7 @@ CURVEFS_ERROR DataCache::PrepareFlushTasks(uint64_t inodeId, s3Tasks->emplace_back(context); // generate flush to kvcache task - if (g_kvClientManager) { + if (kvClientManager_) { auto task = std::make_shared(); task->key = objectName; task->value = data + (*writeOffset); @@ -2398,11 +2403,11 @@ void DataCache::FlushTaskExecute( }); } // kvtask execute - if (g_kvClientManager && kvPendingTaskCal.load()) { + if (kvClientManager_ && kvPendingTaskCal.load()) { std::for_each(kvCacheTasks.begin(), kvCacheTasks.end(), [&](const std::shared_ptr &task) { task->done = kvdone; - g_kvClientManager->Set(task); + kvClientManager_->Set(task); }); kvTaskEnvent.Wait(); } diff --git a/curvefs/src/client/s3/client_s3_cache_manager.h b/curvefs/src/client/s3/client_s3_cache_manager.h index 7f17ad44a9..9a7733e0fc 100644 --- a/curvefs/src/client/s3/client_s3_cache_manager.h +++ b/curvefs/src/client/s3/client_s3_cache_manager.h @@ -29,9 +29,9 @@ #include #include #include +#include #include #include -#include #include "curvefs/proto/metaserver.pb.h" #include "curvefs/src/client/error_code.h" @@ -139,7 +139,8 @@ class DataCache : public std::enable_shared_from_this { public: DataCache(S3ClientAdaptorImpl *s3ClientAdaptor, ChunkCacheManagerPtr chunkCacheManager, uint64_t chunkPos, - uint64_t len, const char *data); + uint64_t len, const char *data, + std::shared_ptr kvClientManager); virtual ~DataCache() { auto iter = dataMap_.begin(); for (; iter != dataMap_.end(); iter++) { @@ -236,6 +237,8 @@ class DataCache : public std::enable_shared_from_this { std::atomic status_; std::atomic inReadCache_; std::map dataMap_; // first is block index + + std::shared_ptr kvClientManager_; }; class S3ReadResponse { @@ -255,9 +258,11 @@ class S3ReadResponse { class ChunkCacheManager : public std::enable_shared_from_this { public: - ChunkCacheManager(uint64_t index, S3ClientAdaptorImpl *s3ClientAdaptor) + ChunkCacheManager(uint64_t index, S3ClientAdaptorImpl *s3ClientAdaptor, + std::shared_ptr kvClientManager) : index_(index), s3ClientAdaptor_(s3ClientAdaptor), - flushingDataCache_(nullptr) {} + flushingDataCache_(nullptr), + kvClientManager_(std::move(kvClientManager)) {} virtual ~ChunkCacheManager() = default; void ReadChunk(uint64_t index, uint64_t chunkPos, uint64_t readLen, char *dataBuf, uint64_t dataBufOffset, @@ -322,13 +327,17 @@ class ChunkCacheManager curve::common::Mutex flushMtx_; DataCachePtr flushingDataCache_; curve::common::Mutex flushingDataCacheMtx_; + + std::shared_ptr kvClientManager_; }; class FileCacheManager { public: FileCacheManager(uint32_t fsid, uint64_t inode, - S3ClientAdaptorImpl *s3ClientAdaptor) - : fsId_(fsid), inode_(inode), s3ClientAdaptor_(s3ClientAdaptor) {} + S3ClientAdaptorImpl *s3ClientAdaptor, + std::shared_ptr kvClientManager) + : fsId_(fsid), inode_(inode), s3ClientAdaptor_(s3ClientAdaptor), + kvClientManager_(std::move(kvClientManager)) {} FileCacheManager() {} ChunkCacheManagerPtr FindOrCreateChunkCacheManager(uint64_t index); @@ -434,16 +443,20 @@ class FileCacheManager { S3ClientAdaptorImpl *s3ClientAdaptor_; curve::common::Mutex downloadMtx_; std::set downloadingObj_; + + std::shared_ptr kvClientManager_; }; class FsCacheManager { public: FsCacheManager(S3ClientAdaptorImpl *s3ClientAdaptor, - uint64_t readCacheMaxByte, uint64_t writeCacheMaxByte) + uint64_t readCacheMaxByte, uint64_t writeCacheMaxByte, + std::shared_ptr kvClientManager) : lruByte_(0), wDataCacheNum_(0), wDataCacheByte_(0), readCacheMaxByte_(readCacheMaxByte), writeCacheMaxByte_(writeCacheMaxByte), - s3ClientAdaptor_(s3ClientAdaptor), isWaiting_(false) {} + s3ClientAdaptor_(s3ClientAdaptor), isWaiting_(false), + kvClientManager_(std::move(kvClientManager)) {} FsCacheManager() {} virtual FileCacheManagerPtr FindFileCacheManager(uint64_t inodeId); virtual FileCacheManagerPtr FindOrCreateFileCacheManager(uint64_t fsId, @@ -551,6 +564,8 @@ class FsCacheManager { std::condition_variable cond_; ReadCacheReleaseExecutor releaseReadCache_; + + std::shared_ptr kvClientManager_; }; } // namespace client diff --git a/curvefs/src/client/warmup/warmup_manager.cpp b/curvefs/src/client/warmup/warmup_manager.cpp index dc9ccefdf5..ededad191e 100644 --- a/curvefs/src/client/warmup/warmup_manager.cpp +++ b/curvefs/src/client/warmup/warmup_manager.cpp @@ -33,6 +33,7 @@ #include #include "curvefs/src/client/inode_wrapper.h" +#include "curvefs/src/client/kvclient/kvclient_manager.h" #include "curvefs/src/client/s3/client_s3_cache_manager.h" #include "curvefs/src/common/s3util.h" #include "src/common/concurrent/concurrent.h" @@ -114,9 +115,7 @@ void WarmupManagerS3Impl::UnInit() { } void WarmupManagerS3Impl::Init(const FuseClientOption &option) { - listDentryLimit_ = option.listDentryLimit; - downloadMaxRetryTimes_ = option.downloadMaxRetryTimes; - warmupThreadsNum_ = option.warmupThreadsNum; + WarmupManager::Init(option); bgFetchStop_.store(false, std::memory_order_release); bgFetchThread_ = Thread(&WarmupManagerS3Impl::BackGroundFetch, this); initbgFetchThread_ = true; @@ -275,7 +274,7 @@ void WarmupManagerS3Impl::FetchDentry(fuse_ino_t key, fuse_ino_t ino, void WarmupManagerS3Impl::FetchChildDentry(fuse_ino_t key, fuse_ino_t ino) { VLOG(9) << "FetchChildDentry start: key:" << key << " inode: " << ino; std::list dentryList; - auto limit = listDentryLimit_; + auto limit = option_.listDentryLimit; CURVEFS_ERROR ret = dentryManager_->ListDentry(ino, &dentryList, limit); if (ret != CURVEFS_ERROR::OK) { LOG(ERROR) << "dentryManager_ ListDentry fail, ret = " << ret @@ -481,12 +480,7 @@ void WarmupManagerS3Impl::WarmUpAllObjs( } if (context->retCode == 0) { VLOG(9) << "Get Object success: " << context->key; - int ret = s3Adaptor_->GetDiskCacheManager()->WriteReadDirect( - context->key, context->buf, context->len); - if (ret < 0) { - LOG_EVERY_SECOND(INFO) - << "write read directly failed, key: " << context->key; - } + PutObjectToCache(context->key, context->buf, context->len); if (pendingReq.fetch_sub(1, std::memory_order_seq_cst) == 1) { VLOG(6) << "pendingReq is over"; cond.Signal(); @@ -494,7 +488,7 @@ void WarmupManagerS3Impl::WarmUpAllObjs( delete[] context->buf; return; } - if (++context->retry >= downloadMaxRetryTimes_) { + if (++context->retry >= option_.downloadMaxRetryTimes) { if (pendingReq.fetch_sub(1, std::memory_order_seq_cst) == 1) { VLOG(6) << "pendingReq is over"; cond.Signal(); @@ -648,7 +642,7 @@ void WarmupManagerS3Impl::AddFetchDentryTask(fuse_ino_t key, auto iter = inode2FetchDentryPool_.find(key); if (iter == inode2FetchDentryPool_.end()) { std::unique_ptr tp = absl::make_unique(); - tp->Start(warmupThreadsNum_); + tp->Start(option_.warmupThreadsNum); iter = inode2FetchDentryPool_.emplace(key, std::move(tp)).first; } if (!iter->second->Enqueue(task)) { @@ -667,7 +661,7 @@ void WarmupManagerS3Impl::AddFetchS3objectsTask(fuse_ino_t key, auto iter = inode2FetchS3ObjectsPool_.find(key); if (iter == inode2FetchS3ObjectsPool_.end()) { std::unique_ptr tp = absl::make_unique(); - tp->Start(warmupThreadsNum_); + tp->Start(option_.warmupThreadsNum); iter = inode2FetchS3ObjectsPool_.emplace(key, std::move(tp)).first; } if (!iter->second->Enqueue(task)) { @@ -678,6 +672,21 @@ void WarmupManagerS3Impl::AddFetchS3objectsTask(fuse_ino_t key, } } +void WarmupManagerS3Impl::PutObjectToCache(const std::string &filename, + const char *data, uint64_t len) { + int ret = + s3Adaptor_->GetDiskCacheManager()->WriteReadDirect(filename, data, len); + if (ret < 0) { + LOG_EVERY_SECOND(INFO) + << "write read directly failed, key: " << filename; + } + + if (kvClientManager_ != nullptr) { + kvClientManager_->Set( + std::make_shared(filename, data, len)); + } +} + } // namespace warmup } // namespace client } // namespace curvefs diff --git a/curvefs/src/client/warmup/warmup_manager.h b/curvefs/src/client/warmup/warmup_manager.h index f2d7073010..51a2758b99 100644 --- a/curvefs/src/client/warmup/warmup_manager.h +++ b/curvefs/src/client/warmup/warmup_manager.h @@ -42,6 +42,7 @@ #include "curvefs/src/client/dentry_cache_manager.h" #include "curvefs/src/client/fuse_common.h" #include "curvefs/src/client/inode_cache_manager.h" +#include "curvefs/src/client/kvclient/kvclient_manager.h" #include "curvefs/src/client/rpcclient/metaserver_client.h" #include "curvefs/src/client/s3/client_s3_adaptor.h" #include "curvefs/src/client/s3/client_s3_cache_manager.h" @@ -105,7 +106,7 @@ class WarmupProgress { explicit WarmupProgress(uint64_t total = 0, uint64_t finished = 0) : total_(total), finished_(finished) {} - WarmupProgress(const WarmupProgress& wp) + WarmupProgress(const WarmupProgress &wp) : total_(wp.total_), finished_(wp.finished_) {} void AddTotal(uint64_t add) { @@ -155,19 +156,25 @@ class WarmupManager { metaClient_(std::make_shared()), inodeManager_(std::make_shared(metaClient_)), dentryManager_( - std::make_shared(metaClient_)) {} + std::make_shared(metaClient_)) { + kvClientManager_ = nullptr; + } explicit WarmupManager(std::shared_ptr metaClient, std::shared_ptr inodeManager, std::shared_ptr dentryManager, std::shared_ptr fsInfo, - FuseOpReadFunctionType readFunc) + FuseOpReadFunctionType readFunc, + std::shared_ptr kvClientManager) : mounted_(false), metaClient_(std::move(metaClient)), inodeManager_(std::move(inodeManager)), dentryManager_(std::move(dentryManager)), fsInfo_(std::move(fsInfo)), - fuseOpRead_(std::move(readFunc)) {} + fuseOpRead_(std::move(readFunc)), + kvClientManager_(std::move(kvClientManager)) {} - virtual void Init(const FuseClientOption &option) = 0; + virtual void Init(const FuseClientOption &option) { + option_ = option; + } virtual void UnInit() { ClearWarmupProcess(); } virtual bool AddWarmupFilelist(fuse_ino_t key) = 0; @@ -183,6 +190,10 @@ class WarmupManager { fuseOpRead_ = read; } + void SetKVClientManager(std::shared_ptr kvClientManager) { + kvClientManager_ = std::move(kvClientManager); + } + /** * @brief * @@ -212,8 +223,7 @@ class WarmupManager { */ virtual bool AddWarmupProcess(fuse_ino_t key) { WriteLockGuard lock(inode2ProgressMutex_); - auto ret = - inode2Progress_.emplace(key, WarmupProgress()); + auto ret = inode2Progress_.emplace(key, WarmupProgress()); return ret.second; } @@ -252,9 +262,12 @@ class WarmupManager { FuseOpReadFunctionType fuseOpRead_; // warmup progress - std::unordered_map - inode2Progress_; + std::unordered_map inode2Progress_; BthreadRWLock inode2ProgressMutex_; + + std::shared_ptr kvClientManager_ = nullptr; + + FuseClientOption option_; }; class WarmupManagerS3Impl : public WarmupManager { @@ -264,10 +277,11 @@ class WarmupManagerS3Impl : public WarmupManager { std::shared_ptr inodeManager, std::shared_ptr dentryManager, std::shared_ptr fsInfo, FuseOpReadFunctionType readFunc, - std::shared_ptr s3Adaptor) + std::shared_ptr s3Adaptor, + std::shared_ptr kvClientManager) : WarmupManager(std::move(metaClient), std::move(inodeManager), std::move(dentryManager), std::move(fsInfo), - std::move(readFunc)), + std::move(readFunc), std::move(kvClientManager)), s3Adaptor_(std::move(s3Adaptor)) {} bool AddWarmupFilelist(fuse_ino_t key) override; @@ -384,6 +398,9 @@ class WarmupManagerS3Impl : public WarmupManager { void AddFetchS3objectsTask(fuse_ino_t key, std::function task); + void PutObjectToCache(const std::string &filename, const char *data, + uint64_t len); + protected: std::deque warmupFilelistDeque_; mutable RWLock warmupFilelistDequeMutex_; @@ -407,10 +424,6 @@ class WarmupManagerS3Impl : public WarmupManager { std::unordered_map> inode2FetchS3ObjectsPool_; mutable RWLock inode2FetchS3ObjectsPoolMutex_; - - uint32_t listDentryLimit_; - uint32_t downloadMaxRetryTimes_; - uint32_t warmupThreadsNum_; }; } // namespace warmup diff --git a/curvefs/test/client/chunk_cache_manager_test.cpp b/curvefs/test/client/chunk_cache_manager_test.cpp index 502c2ab056..5fe4b5e3bf 100644 --- a/curvefs/test/client/chunk_cache_manager_test.cpp +++ b/curvefs/test/client/chunk_cache_manager_test.cpp @@ -56,12 +56,12 @@ class ChunkCacheManagerTest : public testing::Test { option.diskCacheOpt.diskCacheType = (DiskCacheType)0; s3ClientAdaptor_ = new S3ClientAdaptorImpl(); auto fsCacheManager_ = std::make_shared( - s3ClientAdaptor_, option.readCacheMaxByte, - option.writeCacheMaxByte); + s3ClientAdaptor_, option.readCacheMaxByte, option.writeCacheMaxByte, + nullptr); s3ClientAdaptor_->Init(option, nullptr, nullptr, nullptr, - fsCacheManager_, nullptr); - chunkCacheManager_ = - std::make_shared(index, s3ClientAdaptor_); + fsCacheManager_, nullptr, nullptr); + chunkCacheManager_ = std::make_shared( + index, s3ClientAdaptor_, nullptr); } void TearDown() override { delete s3ClientAdaptor_; @@ -91,20 +91,20 @@ TEST_F(ChunkCacheManagerTest, test_add_read_data_cache) { uint64_t len = 1024 * 1024; char *buf = new char[len]; auto dataCache = std::make_shared( - s3ClientAdaptor_, chunkCacheManager_, offset, len, buf); + s3ClientAdaptor_, chunkCacheManager_, offset, len, buf, nullptr); VLOG(0) << "wghs001"; chunkCacheManager_->AddReadDataCache(dataCache); VLOG(0) << "wghs002"; ASSERT_EQ(len, s3ClientAdaptor_->GetFsCacheManager()->GetLruByte()); offset = 2 * 1024 * 1024; auto dataCache1 = std::make_shared( - s3ClientAdaptor_, chunkCacheManager_, offset, len, buf); + s3ClientAdaptor_, chunkCacheManager_, offset, len, buf, nullptr); chunkCacheManager_->AddReadDataCache(dataCache1); ASSERT_EQ(2 * len, s3ClientAdaptor_->GetFsCacheManager()->GetLruByte()); offset = 0; len = 512 * 1024; auto dataCache2 = std::make_shared( - s3ClientAdaptor_, chunkCacheManager_, offset, len, buf); + s3ClientAdaptor_, chunkCacheManager_, offset, len, buf, nullptr); chunkCacheManager_->AddReadDataCache(dataCache2); ASSERT_EQ(1.5 * 1024 * 1024, s3ClientAdaptor_->GetFsCacheManager()->GetLruByte()); @@ -236,7 +236,7 @@ TEST_F(ChunkCacheManagerTest, test_read_by_read_cache) { requests.clear(); auto dataCache = std::make_shared( - s3ClientAdaptor_, chunkCacheManager_, 1024, len, dataCacheBuf); + s3ClientAdaptor_, chunkCacheManager_, 1024, len, dataCacheBuf, nullptr); chunkCacheManager_->AddReadDataCache(dataCache); chunkCacheManager_->ReadByReadCache(offset, len, buf, 0, &requests); ASSERT_EQ(1, requests.size()); @@ -311,7 +311,7 @@ TEST_F(ChunkCacheManagerTest, test_flush) { uint64_t len = 1024 * 1024; char *buf = new char[len]; auto dataCache = std::make_shared( - s3ClientAdaptor_, chunkCacheManager_, offset, len, buf); + s3ClientAdaptor_, chunkCacheManager_, offset, len, buf, nullptr); EXPECT_CALL(*dataCache, Flush(_, _)) .WillOnce(Return(CURVEFS_ERROR::OK)) .WillOnce(Return(CURVEFS_ERROR::INTERNAL)) @@ -341,7 +341,7 @@ TEST_F(ChunkCacheManagerTest, test_release_read_dataCache) { uint64_t len = 1024 * 1024; char *buf = new char[len]; auto dataCache = std::make_shared( - s3ClientAdaptor_, chunkCacheManager_, offset, len, buf); + s3ClientAdaptor_, chunkCacheManager_, offset, len, buf, nullptr); chunkCacheManager_->AddReadDataCache(dataCache); chunkCacheManager_->ReleaseReadDataCache(offset); ASSERT_EQ(true, chunkCacheManager_->IsEmpty()); @@ -355,7 +355,7 @@ TEST_F(ChunkCacheManagerTest, test_truncate_cache) { uint64_t len = 1024 * 1024; char *buf = new char[len]; auto dataCache = std::make_shared( - s3ClientAdaptor_, chunkCacheManager_, offset, len, buf); + s3ClientAdaptor_, chunkCacheManager_, offset, len, buf, nullptr); /*EXPECT_CALL(*dataCache, Truncate(_)) .WillOnce(Return());*/ chunkCacheManager_->AddWriteDataCacheForTest(dataCache); diff --git a/curvefs/test/client/client_memcache_test.cpp b/curvefs/test/client/client_memcache_test.cpp index 5113043b5c..c0ce49e033 100644 --- a/curvefs/test/client/client_memcache_test.cpp +++ b/curvefs/test/client/client_memcache_test.cpp @@ -114,7 +114,8 @@ TEST_F(MemCachedTest, MultiThreadTask) { }); } taskEvent.Wait(); - ASSERT_EQ(5, g_kvClientMetric->kvClientSet.latency.count()); + ASSERT_EQ( + 5, manager_.GetClientMetricForTesting()->kvClientSet.latency.count()); // get for (int i = 0; i < 5; i++) { diff --git a/curvefs/test/client/client_s3_adaptor_Integration.cpp b/curvefs/test/client/client_s3_adaptor_Integration.cpp index f26157da9c..7ac01856ea 100644 --- a/curvefs/test/client/client_s3_adaptor_Integration.cpp +++ b/curvefs/test/client/client_s3_adaptor_Integration.cpp @@ -23,6 +23,7 @@ #include #include #include +#include #include "curvefs/src/client/inode_wrapper.h" #include "curvefs/src/client/s3/client_s3_adaptor.h" @@ -123,6 +124,7 @@ class ClientS3IntegrationTest : public testing::Test { protected: void SetUp() override { Aws::InitAPI(awsOptions_); + InitKVClientManager(); ASSERT_EQ(0, server_.AddService(&mockMetaServerService_, brpc::SERVER_DOESNT_OWN_SERVICE)); ASSERT_EQ(0, server_.Start(addr_.c_str(), nullptr)); @@ -143,9 +145,10 @@ class ClientS3IntegrationTest : public testing::Test { s3ClientAdaptor_ = new S3ClientAdaptorImpl(); auto fsCacheManager = std::make_shared( s3ClientAdaptor_, option.readCacheMaxByte, - option.writeCacheMaxByte); + option.writeCacheMaxByte, kvClientManager_); s3ClientAdaptor_->Init(option, mockS3Client, mockInodeManager, - mockMdsClient, fsCacheManager, nullptr); + mockMdsClient, fsCacheManager, nullptr, + kvClientManager_); s3ClientAdaptor_->SetFsId(2); curvefs::client::common::FLAGS_enableCto = false; } @@ -157,11 +160,11 @@ class ClientS3IntegrationTest : public testing::Test { } void InitKVClientManager() { - g_kvClientManager = new KVClientManager(); + kvClientManager_ = std::make_shared(); KVClientManagerOpt opt; std::shared_ptr mockKVClient(&mockKVClient_); - g_kvClientManager->Init(opt, mockKVClient); + kvClientManager_->Init(opt, mockKVClient); } protected: @@ -176,6 +179,8 @@ class ClientS3IntegrationTest : public testing::Test { Aws::SDKOptions awsOptions_; std::shared_ptr inode{InitInodeForIntegration()}; + + std::shared_ptr kvClientManager_; }; TEST_F(ClientS3IntegrationTest, test_first_write) { @@ -2834,8 +2839,6 @@ TEST_F(ClientS3IntegrationTest, test_fssync_overlap_write) { } TEST_F(ClientS3IntegrationTest, test_write_read_remotekvcache) { - InitKVClientManager(); - curvefs::client::common::FLAGS_enableCto = true; uint64_t offset_0 = 0, offset_4M = (4 << 20), chunkId = 10; diff --git a/curvefs/test/client/client_s3_adaptor_test.cpp b/curvefs/test/client/client_s3_adaptor_test.cpp index ec402f2eee..82a3cfffeb 100644 --- a/curvefs/test/client/client_s3_adaptor_test.cpp +++ b/curvefs/test/client/client_s3_adaptor_test.cpp @@ -47,7 +47,7 @@ using ::testing::WithArg; using rpcclient::MockMdsClient; -extern KVClientManager *g_kvClientManager; +// extern KVClientManager *g_kvClientManager; class ClientS3AdaptorTest : public testing::Test { protected: @@ -72,10 +72,10 @@ class ClientS3AdaptorTest : public testing::Test { option.fuseMaxSize = 131072; option.chunkFlushThreads = 5; option.diskCacheOpt.diskCacheType = (DiskCacheType)0; + kvClientManager_ = nullptr; s3ClientAdaptor_->Init(option, mockS3Client_, mockInodeManager_, mockMdsClient_, mockFsCacheManager_, - mockDiskcacheManagerImpl_); - g_kvClientManager = nullptr; + mockDiskcacheManagerImpl_, kvClientManager_); } void TearDown() override { @@ -88,6 +88,7 @@ class ClientS3AdaptorTest : public testing::Test { std::shared_ptr mockFsCacheManager_; std::shared_ptr mockS3Client_; std::shared_ptr mockMdsClient_; + std::shared_ptr kvClientManager_; }; uint64_t gInodeId = 1; diff --git a/curvefs/test/client/data_cache_test.cpp b/curvefs/test/client/data_cache_test.cpp index 615f357776..107335b28c 100644 --- a/curvefs/test/client/data_cache_test.cpp +++ b/curvefs/test/client/data_cache_test.cpp @@ -54,15 +54,16 @@ class DataCacheTest : public testing::Test { s3ClientAdaptor_ = new S3ClientAdaptorImpl(); auto fsCacheManager = std::make_shared( s3ClientAdaptor_, option.readCacheMaxByte, - option.writeCacheMaxByte); + option.writeCacheMaxByte, nullptr); s3ClientAdaptor_->Init(option, nullptr, nullptr, nullptr, - fsCacheManager, nullptr); + fsCacheManager, nullptr, nullptr); mockChunkCacheManager_ = std::make_shared(); uint64_t offset = 512 * 1024; uint64_t len = 1024 * 1024; char *buf = new char[len]; - dataCache_ = std::make_shared( - s3ClientAdaptor_, mockChunkCacheManager_, offset, len, buf); + dataCache_ = std::make_shared(s3ClientAdaptor_, + mockChunkCacheManager_, offset, + len, buf, nullptr); delete buf; } void TearDown() override {} @@ -102,7 +103,7 @@ TEST_F(DataCacheTest, test_write3) { std::vector mergeDataCacheVer; uint64_t offset1 = len; auto dataCache = std::make_shared( - s3ClientAdaptor_, mockChunkCacheManager_, offset1, len, buf); + s3ClientAdaptor_, mockChunkCacheManager_, offset1, len, buf, nullptr); mergeDataCacheVer.push_back(dataCache); dataCache_->Write(offset, len, buf, mergeDataCacheVer); ASSERT_EQ(0, dataCache_->GetChunkPos()); @@ -139,7 +140,7 @@ TEST_F(DataCacheTest, test_write6) { std::vector mergeDataCacheVer; uint64_t offset1 = offset + len; auto dataCache = std::make_shared( - s3ClientAdaptor_, mockChunkCacheManager_, offset1, len, buf); + s3ClientAdaptor_, mockChunkCacheManager_, offset1, len, buf, nullptr); mergeDataCacheVer.push_back(dataCache); dataCache_->Write(offset, len, buf, mergeDataCacheVer); ASSERT_EQ(512 * 1024, dataCache_->GetChunkPos()); diff --git a/curvefs/test/client/file_cache_manager_test.cpp b/curvefs/test/client/file_cache_manager_test.cpp index 8815ea6d2e..7d2e8be887 100644 --- a/curvefs/test/client/file_cache_manager_test.cpp +++ b/curvefs/test/client/file_cache_manager_test.cpp @@ -48,7 +48,7 @@ using ::testing::SetArgPointee; using ::testing::SetArgReferee; using ::testing::WithArg; -extern KVClientManager *g_kvClientManager; +// extern KVClientManager *g_kvClientManager; class FileCacheManagerTest : public testing::Test { protected: @@ -69,18 +69,18 @@ class FileCacheManagerTest : public testing::Test { option.chunkFlushThreads = 5; s3ClientAdaptor_ = new S3ClientAdaptorImpl(); auto fsCacheManager_ = std::make_shared( - s3ClientAdaptor_, option.readCacheMaxByte, - option.writeCacheMaxByte); + s3ClientAdaptor_, option.readCacheMaxByte, option.writeCacheMaxByte, + nullptr); mockInodeManager_ = std::make_shared(); mockS3Client_ = std::make_shared(); s3ClientAdaptor_->Init(option, mockS3Client_, mockInodeManager_, - nullptr, fsCacheManager_, nullptr); + nullptr, fsCacheManager_, nullptr, nullptr); s3ClientAdaptor_->SetFsId(fsId); - fileCacheManager_ = - std::make_shared(fsId, inodeId, s3ClientAdaptor_); + fileCacheManager_ = std::make_shared( + fsId, inodeId, s3ClientAdaptor_, nullptr); mockChunkCacheManager_ = std::make_shared(); curvefs::client::common::FLAGS_enableCto = false; - g_kvClientManager = nullptr; + kvClientManager_ = nullptr; } void TearDown() override { @@ -95,6 +95,7 @@ class FileCacheManagerTest : public testing::Test { std::shared_ptr mockChunkCacheManager_; std::shared_ptr mockInodeManager_; std::shared_ptr mockS3Client_; + std::shared_ptr kvClientManager_; }; TEST_F(FileCacheManagerTest, test_FindOrCreateChunkCacheManager) { @@ -159,8 +160,8 @@ TEST_F(FileCacheManagerTest, test_old_write) { uint64_t len = 1024; char buf[len] = {0}; - auto dataCache = std::make_shared(s3ClientAdaptor_, nullptr, - offset, 0, nullptr); + auto dataCache = std::make_shared( + s3ClientAdaptor_, nullptr, offset, 0, nullptr, nullptr); EXPECT_CALL(*dataCache, Write(_, _, _, _)).WillOnce(Return()); EXPECT_CALL(*mockChunkCacheManager_, FindWriteableDataCache(_, _, _, _)) .WillOnce(Return(dataCache)); diff --git a/curvefs/test/client/fs_cache_manager_test.cpp b/curvefs/test/client/fs_cache_manager_test.cpp index 80f2df3548..27c7341532 100644 --- a/curvefs/test/client/fs_cache_manager_test.cpp +++ b/curvefs/test/client/fs_cache_manager_test.cpp @@ -56,9 +56,9 @@ class FsCacheManagerTest : public testing::Test { option.chunkFlushThreads = 5; s3ClientAdaptor_ = new S3ClientAdaptorImpl(); fsCacheManager_ = std::make_shared( - s3ClientAdaptor_, maxReadCacheByte_, maxWriteCacheByte); + s3ClientAdaptor_, maxReadCacheByte_, maxWriteCacheByte, nullptr); s3ClientAdaptor_->Init(option, nullptr, nullptr, nullptr, - fsCacheManager_, nullptr); + fsCacheManager_, nullptr, nullptr); s3ClientAdaptor_->SetFsId(2); mockChunkCacheManager_ = std::make_shared(); @@ -118,7 +118,7 @@ TEST_F(FsCacheManagerTest, test_lru_set_and_delete) { for (size_t i = 0; i < maxReadCacheByte_ / smallDataCacheByte; ++i) { fsCacheManager_->Set(std::make_shared( s3ClientAdaptor_, mockChunkCacheManager_, - 0, smallDataCacheByte, buf), + 0, smallDataCacheByte, buf, nullptr), &outIter); } } @@ -130,9 +130,9 @@ TEST_F(FsCacheManagerTest, test_lru_set_and_delete) { EXPECT_CALL(*mockChunkCacheManager_, ReleaseReadDataCache(_)) .Times(expectCallTimes) .WillRepeatedly(Invoke([&counter](uint64_t) { counter.Signal(); })); - fsCacheManager_->Set(std::make_shared(s3ClientAdaptor_, - mockChunkCacheManager_, - 0, dataCacheByte, buf), + fsCacheManager_->Set(std::make_shared( + s3ClientAdaptor_, mockChunkCacheManager_, 0, + dataCacheByte, buf, nullptr), &outIter); counter.Wait(); @@ -146,9 +146,9 @@ TEST_F(FsCacheManagerTest, test_lru_set_and_delete) { .Times(expectCallTimes) .WillRepeatedly(Invoke([&counter](uint64_t) { counter.Signal(); })); - fsCacheManager_->Set(std::make_shared(s3ClientAdaptor_, - mockChunkCacheManager_, - 0, dataCacheByte, buf), + fsCacheManager_->Set(std::make_shared( + s3ClientAdaptor_, mockChunkCacheManager_, 0, + dataCacheByte, buf, nullptr), &outIter); counter.Wait(); } diff --git a/curvefs/test/client/mock_client_s3_adaptor.h b/curvefs/test/client/mock_client_s3_adaptor.h index 10a20668e4..0365573851 100644 --- a/curvefs/test/client/mock_client_s3_adaptor.h +++ b/curvefs/test/client/mock_client_s3_adaptor.h @@ -36,7 +36,7 @@ namespace client { class MockS3ClientAdaptor : public S3ClientAdaptor { public: - MOCK_METHOD7(Init, + MOCK_METHOD8(Init, CURVEFS_ERROR(const S3ClientAdaptorOption &option, std::shared_ptr client, std::shared_ptr inodeManager, @@ -44,6 +44,8 @@ class MockS3ClientAdaptor : public S3ClientAdaptor { std::shared_ptr fsCacheManager, std::shared_ptr diskCacheManagerImpl, + std::shared_ptr + kvClientManager, bool startBackGround)); MOCK_METHOD4(Write, int(uint64_t inodeId, uint64_t offset, uint64_t length, diff --git a/curvefs/test/client/mock_client_s3_cache_manager.h b/curvefs/test/client/mock_client_s3_cache_manager.h index 6990c90756..d6586eb346 100644 --- a/curvefs/test/client/mock_client_s3_cache_manager.h +++ b/curvefs/test/client/mock_client_s3_cache_manager.h @@ -24,6 +24,7 @@ #define CURVEFS_TEST_CLIENT_MOCK_CLIENT_S3_CACHE_MANAGER_H_ #include +#include #include #include "curvefs/src/client/s3/client_s3_cache_manager.h" @@ -56,7 +57,7 @@ class MockFileCacheManager : public FileCacheManager { class MockChunkCacheManager : public ChunkCacheManager { public: - MockChunkCacheManager() : ChunkCacheManager(0, nullptr) {} + MockChunkCacheManager() : ChunkCacheManager(0, nullptr, nullptr) {} ~MockChunkCacheManager() = default; MOCK_METHOD1(ReleaseReadDataCache, void(uint64_t)); @@ -82,8 +83,10 @@ class MockDataCache : public DataCache { public: MockDataCache(S3ClientAdaptorImpl *s3ClientAdaptor, ChunkCacheManagerPtr chunkCacheManager, uint64_t chunkPos, - uint64_t len, const char *data) - : DataCache(s3ClientAdaptor, chunkCacheManager, chunkPos, len, data) {} + uint64_t len, const char *data, + std::shared_ptr kvClientManager) + : DataCache(s3ClientAdaptor, chunkCacheManager, chunkPos, len, data, + std::move(kvClientManager)) {} ~MockDataCache() = default; MOCK_METHOD4(Write, diff --git a/curvefs/test/client/test_fuse_s3_client.cpp b/curvefs/test/client/test_fuse_s3_client.cpp index e6e9c6c367..eac9417f42 100644 --- a/curvefs/test/client/test_fuse_s3_client.cpp +++ b/curvefs/test/client/test_fuse_s3_client.cpp @@ -99,7 +99,7 @@ class TestFuseS3Client : public ::testing::Test { dentryManager_ = std::make_shared(); warmupManager_ = std::make_shared( metaClient_, inodeManager_, dentryManager_, nullptr, nullptr, - s3ClientAdaptor_); + s3ClientAdaptor_, nullptr); client_ = std::make_shared( mdsClient_, metaClient_, inodeManager_, dentryManager_, s3ClientAdaptor_, warmupManager_); diff --git a/tools-v2/internal/error/error.go b/tools-v2/internal/error/error.go index 4e846293a1..6800931cfd 100644 --- a/tools-v2/internal/error/error.go +++ b/tools-v2/internal/error/error.go @@ -196,13 +196,15 @@ func MergeCmdErrorExceptSuccess(err []*CmdError) CmdError { ret.Message = "" countSuccess := 0 for _, e := range err { - if e.Code == CODE_SUCCESS { - countSuccess++ - continue - } else if e.Code < ret.Code { - ret.Code = e.Code + if e != nil { + if e.Code == CODE_SUCCESS { + countSuccess++ + continue + } else if e.Code < ret.Code { + ret.Code = e.Code + } + ret.Message = e.Message + "\n" + ret.Message } - ret.Message = e.Message + "\n" + ret.Message } if countSuccess == len(err) { return *NewSucessCmdError() From 5df1ed3a60d047abac658f2f17dcc3c24f95b857 Mon Sep 17 00:00:00 2001 From: mOUNT41N <92198368+mOUNT41N@users.noreply.github.com> Date: Fri, 10 Mar 2023 13:40:49 +0800 Subject: [PATCH 07/14] [docs] fix README links Signed-off-by: mOUNT41N <92198368+mOUNT41N@users.noreply.github.com> Update README_cn.md Signed-off-by: mOUNT41N <92198368+mOUNT41N@users.noreply.github.com> --- README.md | 2 +- README_cn.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index f5ccc69d36..1fa73abdcf 100644 --- a/README.md +++ b/README.md @@ -226,7 +226,7 @@ Please refer to the [Test environment configuration](docs/cn/测试环境配置 See [Governance](https://github.com/opencurve/community/blob/master/GOVERNANCE.md). ## Contribute us -Participation in the Curve project is described in the [Curve Open Source Community Guidelines](Community_Guidelines.md) and is subject to a [contributor contract](https://github.com/opencurve/curve/blob/master/CODE_OF_CONDUCT.md). +Participation in the Curve project is described in the [Curve Developers Guidelines](developers_guide.md) and is subject to a [contributor contract](https://github.com/opencurve/curve/blob/master/CODE_OF_CONDUCT.md). We welcome your contribution! ## Code of Conduct diff --git a/README_cn.md b/README_cn.md index 227b7752e6..1165156b47 100644 --- a/README_cn.md +++ b/README_cn.md @@ -221,7 +221,7 @@ $ ./fio --thread --rw=randwrite --bs=4k --ioengine=nebd --nebd=cbd:pool//pfstest ## 贡献我们 -参与 Curve 项目开发详见[Curve 开源社区指南](Community_Guidelines_cn.md)并且请遵循[贡献者准则](https://github.com/opencurve/curve/blob/master/CODE_OF_CONDUCT.md), 我们期待您的贡献! +参与 Curve 项目开发详见[Curve 开发者指南](developers_guide_cn.md)并且请遵循[贡献者准则](https://github.com/opencurve/curve/blob/master/CODE_OF_CONDUCT.md), 我们期待您的贡献! ## 最佳实践 - [CurveBS+NFS搭建NFS存储](docs/practical/curvebs_nfs.md) From 2b4014167cc8eab21e8d46d57d587473ab44de52 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 7 Mar 2023 03:29:22 +0000 Subject: [PATCH 08/14] build(deps): bump golang.org/x/net in /tools-v2 Bumps [golang.org/x/net](https://github.com/golang/net) from 0.0.0-20220909164309-bea034e7d591 to 0.7.0. - [Release notes](https://github.com/golang/net/releases) - [Commits](https://github.com/golang/net/commits/v0.7.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] --- tools-v2/go.mod | 22 +++----------- tools-v2/go.sum | 80 +++---------------------------------------------- 2 files changed, 8 insertions(+), 94 deletions(-) diff --git a/tools-v2/go.mod b/tools-v2/go.mod index e24a1908a0..c6e6b1e204 100644 --- a/tools-v2/go.mod +++ b/tools-v2/go.mod @@ -12,6 +12,7 @@ require ( github.com/gookit/color v1.5.2 github.com/moby/term v0.0.0-20220808134915-39b0c02b01ae github.com/olekukonko/tablewriter v0.0.5 + github.com/schollz/progressbar/v3 v3.13.0 github.com/smartystreets/goconvey v1.7.2 github.com/spf13/cobra v1.5.0 github.com/spf13/pflag v1.0.5 @@ -22,10 +23,7 @@ require ( google.golang.org/protobuf v1.28.1 ) -require ( - github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect - github.com/schollz/progressbar/v3 v3.13.0 // indirect -) +require github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect require ( github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect @@ -33,9 +31,7 @@ require ( github.com/Microsoft/hcsshim v0.9.4 // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/cespare/xxhash/v2 v2.1.2 // indirect - github.com/containerd/cgroups v1.0.4 // indirect github.com/containerd/continuity v0.3.0 // indirect - github.com/coreos/go-systemd/v22 v22.3.2 // indirect github.com/docker/distribution v2.8.1+incompatible // indirect github.com/docker/docker v20.10.18+incompatible // indirect github.com/docker/docker-credential-helpers v0.6.4 // indirect @@ -45,9 +41,7 @@ require ( github.com/docker/go-units v0.5.0 // indirect github.com/fsnotify/fsnotify v1.5.4 // indirect github.com/fvbommel/sortorder v1.0.2 // indirect - github.com/godbus/dbus/v5 v5.0.4 // indirect github.com/gogo/protobuf v1.3.2 // indirect - github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect github.com/golang/protobuf v1.5.2 // indirect github.com/gopherjs/gopherjs v1.17.2 // indirect github.com/gorilla/mux v1.8.0 // indirect @@ -62,11 +56,8 @@ require ( github.com/moby/sys/mount v0.3.3 // indirect github.com/moby/sys/mountinfo v0.6.2 // indirect github.com/morikuni/aec v1.0.0 // indirect - github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86 // indirect - github.com/neelance/sourcemap v0.0.0-20200213170602-2833bce08e4c // indirect github.com/opencontainers/go-digest v1.0.0 // indirect github.com/opencontainers/image-spec v1.0.2 // indirect - github.com/opencontainers/runtime-spec v1.0.3-0.20210326190908-1c3f411f0417 // indirect github.com/pelletier/go-toml v1.9.5 // indirect github.com/pelletier/go-toml/v2 v2.0.5 // indirect github.com/pkg/errors v0.9.1 // indirect @@ -76,7 +67,6 @@ require ( github.com/prometheus/common v0.37.0 // indirect github.com/prometheus/procfs v0.8.0 // indirect github.com/rivo/uniseg v0.4.3 // indirect - github.com/shurcooL/httpfs v0.0.0-20190707220628-8d4bc4ba7749 // indirect github.com/sirupsen/logrus v1.9.0 // indirect github.com/smartystreets/assertions v1.13.0 // indirect github.com/spf13/afero v1.9.2 // indirect @@ -85,14 +75,10 @@ require ( github.com/subosito/gotenv v1.4.1 // indirect github.com/theupdateframework/notary v0.7.0 // indirect github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect - go.opencensus.io v0.23.0 // indirect golang.org/x/crypto v0.0.0-20220829220503-c86fa9a7ed90 // indirect - golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 // indirect - golang.org/x/net v0.0.0-20220909164309-bea034e7d591 // indirect - golang.org/x/sync v0.0.0-20220907140024-f12130a52804 // indirect + golang.org/x/net v0.7.0 // indirect golang.org/x/term v0.5.0 // indirect - golang.org/x/text v0.3.7 // indirect - golang.org/x/tools v0.1.12 // indirect + golang.org/x/text v0.7.0 // indirect google.golang.org/genproto v0.0.0-20220913154956-18f8339a66a5 // indirect gopkg.in/ini.v1 v1.67.0 // indirect gopkg.in/yaml.v2 v2.4.0 // indirect diff --git a/tools-v2/go.sum b/tools-v2/go.sum index 9b6e09ceb7..54f3a98e58 100644 --- a/tools-v2/go.sum +++ b/tools-v2/go.sum @@ -118,7 +118,6 @@ github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 h1:nvj0OLI3YqYXe github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0/go.mod h1:D/8v3kj0zr8ZAKg1AQ6crr+5VwKN5eIywRkfhyM/+dE= github.com/cenkalti/backoff/v4 v4.1.1/go.mod h1:scbssz8iZGpm3xbr14ovlUdkxfGXNInqkPWOWmG2CLw= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= -github.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko= github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc= github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cespare/xxhash/v2 v2.1.2 h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cbYE= @@ -141,11 +140,7 @@ github.com/cloudflare/cfssl v0.0.0-20180223231731-4e2dcbde5004/go.mod h1:yMWuSON github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc= github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= -github.com/cncf/udpa/go v0.0.0-20210930031921-04548b0d99d4/go.mod h1:6pvJx4me5XPnfI9Z40ddWsdw2W/uZgQLFXToKeRcDiI= github.com/cncf/xds/go v0.0.0-20210312221358-fbca930ec8ed/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= -github.com/cncf/xds/go v0.0.0-20210922020428-25de7278fc84/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= -github.com/cncf/xds/go v0.0.0-20211001041855-01bcc9b48dfe/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= -github.com/cncf/xds/go v0.0.0-20211011173535-cb28da3451f1/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa/go.mod h1:zn76sxSg3SzpJ0PPJaLDCu+Bu0Lg3sKTORVIj19EIF8= github.com/containerd/aufs v0.0.0-20200908144142-dab0cbea06f4/go.mod h1:nukgQABAEopAHvB6j7cnP5zJ+/3aVcE7hCYqvIwAHyE= github.com/containerd/aufs v0.0.0-20201003224125-76a6863f2989/go.mod h1:AkGGQs9NM2vtYHaUen+NljV0/baGCAPELGm2q9ZXpWU= @@ -161,8 +156,6 @@ github.com/containerd/cgroups v0.0.0-20200710171044-318312a37340/go.mod h1:s5q4S github.com/containerd/cgroups v0.0.0-20200824123100-0b889c03f102/go.mod h1:s5q4SojHctfxANBDvMeIaIovkq29IP48TKAxnhYRxvo= github.com/containerd/cgroups v0.0.0-20210114181951-8a68de567b68/go.mod h1:ZJeTFisyysqgcCdecO57Dj79RfL0LNeGiFUqLYQRYLE= github.com/containerd/cgroups v1.0.1/go.mod h1:0SJrPIenamHDcZhEcJMNBB85rHcUsw4f25ZfBiPYRkU= -github.com/containerd/cgroups v1.0.4 h1:jN/mbWBEaz+T1pi5OFtnkQ+8qnmEbAr1Oo1FRm5B0dA= -github.com/containerd/cgroups v1.0.4/go.mod h1:nLNQtsF7Sl2HxNebu77i1R0oDlhiTG+kO4JTrUzo6IA= github.com/containerd/console v0.0.0-20180822173158-c12b1e7919c1/go.mod h1:Tj/on1eG8kiEhd0+fhSDzsPAFESxzBBvdyEgyryXffw= github.com/containerd/console v0.0.0-20181022165439-0650fd9eeb50/go.mod h1:Tj/on1eG8kiEhd0+fhSDzsPAFESxzBBvdyEgyryXffw= github.com/containerd/console v0.0.0-20191206165004-02ecf6a7291e/go.mod h1:8Pf4gM6VEbTNRIT26AyyU7hxdQU3MvAvxVI0sc00XBE= @@ -244,11 +237,9 @@ github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3Ee github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk= github.com/coreos/go-systemd v0.0.0-20161114122254-48702e0da86b/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= -github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e h1:Wf6HqHfScWJN9/ZjdUKyjop4mf3Qdd+1TvvltAvM3m8= github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= github.com/coreos/go-systemd/v22 v22.0.0/go.mod h1:xO0FLkIi5MaZafQlIrOotqXZ90ih+1atmu1JpKERPPk= github.com/coreos/go-systemd/v22 v22.1.0/go.mod h1:xO0FLkIi5MaZafQlIrOotqXZ90ih+1atmu1JpKERPPk= -github.com/coreos/go-systemd/v22 v22.3.2 h1:D9/bQk5vlXQFZ6Kwuu6zaiXJ9oTPe68++AzAJc1DzSI= github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc= github.com/coreos/pkg v0.0.0-20160727233714-3ac0863d7acf/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= @@ -277,19 +268,14 @@ github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZm github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no= github.com/dnaeon/go-vcr v1.0.1/go.mod h1:aBB1+wY4s93YsC3HHjMBMrwTj2R9FHDzUr9KyGc8n1E= github.com/docker/cli v0.0.0-20191017083524-a8ff7f821017/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8= -github.com/docker/cli v20.10.17+incompatible h1:eO2KS7ZFeov5UJeaDmIs1NFEDRf32PaqRpvoEkKBy5M= -github.com/docker/cli v20.10.17+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8= github.com/docker/cli v20.10.18+incompatible h1:f/GQLsVpo10VvToRay2IraVA1wHz9KktZyjev3SIVDU= github.com/docker/cli v20.10.18+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8= github.com/docker/distribution v0.0.0-20190905152932-14b96e55d84c/go.mod h1:0+TTO4EOBfRPhZXAeF1Vu+W3hHZ8eLp8PgKVZlcvtFY= github.com/docker/distribution v2.7.1-0.20190205005809-0d3efadf0154+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= -github.com/docker/distribution v2.7.1+incompatible h1:a5mlkVzth6W5A4fOsS3D2EO5BUmsJpcB+cRlLU7cSug= github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= github.com/docker/distribution v2.8.1+incompatible h1:Q50tZOPR6T/hjNsyc9g8/syEs6bk8XXApsHjKukMl68= github.com/docker/distribution v2.8.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= github.com/docker/docker v1.4.2-0.20190924003213-a8608b5b67c7/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= -github.com/docker/docker v20.10.17+incompatible h1:JYCuMrWaVNophQTOrMMoSwudOVEfcegoZZrleKc1xwE= -github.com/docker/docker v20.10.17+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= github.com/docker/docker v20.10.18+incompatible h1:SN84VYXTBNGn92T/QwIRPlum9zfemfitN7pbsp26WSc= github.com/docker/docker v20.10.18+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= github.com/docker/docker-credential-helpers v0.6.3/go.mod h1:WRaJzqw3CTB9bk10avuGsjVBZsD05qeibJ1/TYlvc0Y= @@ -304,7 +290,6 @@ github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c/go.mod h1:Uw6Uezg github.com/docker/go-metrics v0.0.0-20180209012529-399ea8c73916/go.mod h1:/u0gXw0Gay3ceNrsHubL3BtdOL2fHf93USgMTe0W5dI= github.com/docker/go-metrics v0.0.1 h1:AgB/0SvBxihN0X8OR4SjsblXkbMvalQ8cjmtKQ2rQV8= github.com/docker/go-metrics v0.0.1/go.mod h1:cG1hvH2utMXtqgqqYE9plW6lDxS3/5ayHzueweSI3Vw= -github.com/docker/go-units v0.4.0 h1:3uh0PgVws3nIA0Q+MwDC8yjEPf9zjRfZZWXZYDct3Tw= github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= @@ -326,7 +311,6 @@ github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1m github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po= github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk= github.com/envoyproxy/go-control-plane v0.9.9-0.20210512163311-63b5d3c536b0/go.mod h1:hliV/p42l8fGbc6Y9bQ70uLwIvmJyVE5k4iMKlh8wCQ= -github.com/envoyproxy/go-control-plane v0.10.2-0.20220325020618-49ff273808a1/go.mod h1:KJwIaB5Mv44NWtYuAOFCVOjcI94vtpEz2JU/D2v6IjE= github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= github.com/erikstmartin/go-testdb v0.0.0-20160219214506-8d10e4a1bae5/go.mod h1:a2zkGnVExMxdzMo3M0Hi/3sEU+cWnZpSni0O6/Yb/P0= github.com/evanphx/json-patch v4.9.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= @@ -374,10 +358,8 @@ github.com/go-sql-driver/mysql v1.3.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= github.com/godbus/dbus v0.0.0-20151105175453-c7fdd8b5cd55/go.mod h1:/YcGZj5zSblfDWMMoOzV4fas9FZnQYTkDnsGvmh2Grw= github.com/godbus/dbus v0.0.0-20180201030542-885f9cc04c9c/go.mod h1:/YcGZj5zSblfDWMMoOzV4fas9FZnQYTkDnsGvmh2Grw= -github.com/godbus/dbus v0.0.0-20190422162347-ade71ed3457e h1:BWhy2j3IXJhjCbC68FptL43tDKIq8FladmaTs3Xs7Z8= github.com/godbus/dbus v0.0.0-20190422162347-ade71ed3457e/go.mod h1:bBOAhwG1umN6/6ZUMtDFBMQR8jRg9O75tm9K00oMsK4= github.com/godbus/dbus/v5 v5.0.3/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA= -github.com/godbus/dbus/v5 v5.0.4 h1:9349emZab16e7zQvpmsbtjc18ykshndd8y2PG3sgJbA= github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA= github.com/gogo/googleapis v1.2.0/go.mod h1:Njal3psf3qN6dwBtQfUmBZh2ybovJ0tlu3o/AC7HYjU= github.com/gogo/googleapis v1.4.0/go.mod h1:5YRNX2z1oM5gXdAkurHa942MDgEJyk02w4OecKY87+c= @@ -396,8 +378,6 @@ github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4er github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= -github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE= -github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y= @@ -435,7 +415,6 @@ github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/ github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= @@ -467,7 +446,6 @@ github.com/googleapis/gnostic v0.4.1/go.mod h1:LRhVm6pbyptWbWbuZ38d1eyptfvIytN3i github.com/googleapis/google-cloud-go-testing v0.0.0-20200911160855-bcd43fbb19e8/go.mod h1:dvDLG8qkwmyD9a/MJJN3XJcT3xFxOKAvTZGvuZmac9g= github.com/gookit/color v1.5.2 h1:uLnfXcaFjlrDnQDT+NCBcfhrXqYTx/rcCa6xn01Y8yI= github.com/gookit/color v1.5.2/go.mod h1:w8h4bGiHeeBpvQVePTutdbERIUf3oJE5lZ8HM0UgXyg= -github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGaHF6qqu48+N2wcFQ5qg5FXgOdqsJ5d8= github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= github.com/gopherjs/gopherjs v1.17.2 h1:fQnZVsXk8uxXIStYb0N4bGk7jeyTalG/wsZjQ25dO0g= github.com/gopherjs/gopherjs v1.17.2/go.mod h1:pRRIvn/QzFLrKfvEz3qUuEhtE/zLCWfreZ6J5gM2i+k= @@ -505,7 +483,6 @@ github.com/imdario/mergo v0.3.8/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJ github.com/imdario/mergo v0.3.10/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA= github.com/imdario/mergo v0.3.11/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA= github.com/imdario/mergo v0.3.12/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA= -github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM= github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8= github.com/inconshreveable/mousetrap v1.0.1 h1:U3uMjPSQEBMNp1lFxmllqCPM6P5u/Xq7Pgzkat/bFNc= github.com/inconshreveable/mousetrap v1.0.1/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= @@ -568,10 +545,7 @@ github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaO github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= github.com/mattn/go-isatty v0.0.17/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM= github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU= -github.com/mattn/go-runewidth v0.0.9 h1:Lm995f3rfxdpd6TSmuVCHVb/QhupuXlYr8sCI/QdE+0= github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI= -github.com/mattn/go-runewidth v0.0.13 h1:lTGmDsbAYt5DmK6OnoV7EuIF1wEIFAcxld6ypU4OSgU= -github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w= github.com/mattn/go-runewidth v0.0.14 h1:+xnbZSEeDbOIg5/mE6JF0w6n9duR1l3/WmbinWVwUuU= github.com/mattn/go-runewidth v0.0.14/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w= github.com/mattn/go-shellwords v1.0.3/go.mod h1:3xCvwCdWdlDJUrvuMn7Wuy9eWs4pE8vqg+NOMyg4B2o= @@ -582,7 +556,6 @@ github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182aff github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4= github.com/maxbrunsfeld/counterfeiter/v6 v6.2.2/go.mod h1:eD9eIE7cdwcMi9rYluz88Jz2VyhSmden33/aXg4oVIY= github.com/miekg/pkcs11 v1.0.2/go.mod h1:XsNlhZGX73bx86s2hdc/FuaLm2CPZJemRLMA+WTFxgs= -github.com/miekg/pkcs11 v1.0.3 h1:iMwmD7I5225wv84WxIG/bmxz9AXjWvTWIbM/TYHvWtw= github.com/miekg/pkcs11 v1.0.3/go.mod h1:XsNlhZGX73bx86s2hdc/FuaLm2CPZJemRLMA+WTFxgs= github.com/miekg/pkcs11 v1.1.1 h1:Ugu9pdy6vAYku5DEpVWVFPYnzV+bxB+iRdbuFSu7TvU= github.com/miekg/pkcs11 v1.1.1/go.mod h1:XsNlhZGX73bx86s2hdc/FuaLm2CPZJemRLMA+WTFxgs= @@ -620,10 +593,6 @@ github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRW github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw= github.com/ncw/swift v1.0.47/go.mod h1:23YIA4yWVnGwv2dQlN4bB7egfYX6YLn0Yo/S6zZO/ZM= -github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86 h1:D6paGObi5Wud7xg83MaEFyjxQB1W5bz5d0IFppr+ymk= -github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86/go.mod h1:kHJEU3ofeGjhHklVoIGuVj85JJwZ6kWPaJwCIxgnFmo= -github.com/neelance/sourcemap v0.0.0-20200213170602-2833bce08e4c h1:bY6ktFuJkt+ZXkX0RChQch2FtHpWQLVS8Qo1YasiIVk= -github.com/neelance/sourcemap v0.0.0-20200213170602-2833bce08e4c/go.mod h1:Qr6/a/Q4r9LP1IltGz7tA7iOK1WonHEYhu1HRBA7ZiM= github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno= github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A= github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U= @@ -653,7 +622,6 @@ github.com/opencontainers/go-digest v1.0.0-rc1.0.20180430190053-c9281466c8b2/go. github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= github.com/opencontainers/image-spec v1.0.0/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0= -github.com/opencontainers/image-spec v1.0.1 h1:JMemWkRwHx4Zj+fVxWoMCFm/8sYGGrUVojFA6h/TRcI= github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0= github.com/opencontainers/image-spec v1.0.2 h1:9yCKha/T5XdGtO0q9Q9a6T5NUCsTn/DrBg0D7ufOcFM= github.com/opencontainers/image-spec v1.0.2/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0= @@ -668,7 +636,6 @@ github.com/opencontainers/runtime-spec v1.0.1/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/ github.com/opencontainers/runtime-spec v1.0.2-0.20190207185410-29686dbc5559/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0= github.com/opencontainers/runtime-spec v1.0.2/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0= github.com/opencontainers/runtime-spec v1.0.3-0.20200929063507-e6143ca7d51d/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0= -github.com/opencontainers/runtime-spec v1.0.3-0.20210326190908-1c3f411f0417 h1:3snG66yBm59tKhhSPQrQ/0bCrv1LQbKt40LnUPiUxdc= github.com/opencontainers/runtime-spec v1.0.3-0.20210326190908-1c3f411f0417/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0= github.com/opencontainers/runtime-tools v0.0.0-20181011054405-1d69bd0f9c39/go.mod h1:r3f7wjNzSs2extwzU3Y+6pKfobzPh+kKFJ3ofN+3nfs= github.com/opencontainers/selinux v1.6.0/go.mod h1:VVGKuOLlE7v4PJyT6h7mNWvq1rzqiriPsEqVhc+svHE= @@ -703,8 +670,6 @@ github.com/prometheus/client_golang v1.1.0/go.mod h1:I1FGZT9+L76gKKOs5djB6ezCbFQ github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0= github.com/prometheus/client_golang v1.12.1/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY= -github.com/prometheus/client_golang v1.12.2 h1:51L9cDoUHVrXx4zWYlcLQIZ+d+VXHgqnYKkIuq4g/34= -github.com/prometheus/client_golang v1.12.2/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY= github.com/prometheus/client_golang v1.13.0 h1:b71QUfeo5M8gq2+evJdTPfZhYMAU0uKPkyPJ7TPsloU= github.com/prometheus/client_golang v1.13.0/go.mod h1:vTeo+zgvILHsnnj/39Ou/1fPN5nJFOEMgftOUOmlvYQ= github.com/prometheus/client_model v0.0.0-20171117100541-99fa1f4be8e5/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= @@ -721,7 +686,6 @@ github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y8 github.com/prometheus/common v0.6.0/go.mod h1:eBmuwkDJBwy6iBfxCBob6t6dR6ENT/y+J+Zk0j9GMYc= github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo= github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc= -github.com/prometheus/common v0.32.1 h1:hWIdL3N2HoUx3B8j3YN9mWor0qhY/NlEKZEaXxuIRh4= github.com/prometheus/common v0.32.1/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls= github.com/prometheus/common v0.37.0 h1:ccBbHCgIiT9uSoFY0vX8H3zsNR5eLt17/RQLUvn8pXE= github.com/prometheus/common v0.37.0/go.mod h1:phzohg0JFMnBEFGxTDbfu3QyL5GI8gTQJFhYO5B3mfA= @@ -736,14 +700,11 @@ github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+Gx github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= github.com/prometheus/procfs v0.2.0/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= -github.com/prometheus/procfs v0.7.3 h1:4jVXhlkAyzOScmCkXBTOLRLTz8EeU+eyjrwB/EPq0VU= github.com/prometheus/procfs v0.7.3/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= github.com/prometheus/procfs v0.8.0 h1:ODq8ZFEaYeCaZOJlZZdJA2AbQR98dSHSM1KW/You5mo= github.com/prometheus/procfs v0.8.0/go.mod h1:z7EfXMXOkbkqb9IINtpCn86r/to3BnA0uaxHdg830/4= github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU= github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc= -github.com/rivo/uniseg v0.4.2 h1:YwD0ulJSJytLpiaWua0sBDusfsCZohxjxzVTYjwxfV8= -github.com/rivo/uniseg v0.4.2/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88= github.com/rivo/uniseg v0.4.3 h1:utMvzDsuh3suAEnhH0RdHmoPbU648o6CvXxTx4SBMOw= github.com/rivo/uniseg v0.4.3/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88= github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg= @@ -758,8 +719,6 @@ github.com/schollz/progressbar/v3 v3.13.0 h1:9TeeWRcjW2qd05I8Kf9knPkW4vLM/hYoa6z github.com/schollz/progressbar/v3 v3.13.0/go.mod h1:ZBYnSuLAX2LU8P8UiKN/KgF2DY58AJC8yfVYLPC8Ly4= github.com/sclevine/spec v1.2.0/go.mod h1:W4J29eT/Kzv7/b9IWLB055Z+qvVC9vt0Arko24q7p+U= github.com/seccomp/libseccomp-golang v0.9.1/go.mod h1:GbW5+tmTXfcxTToHLXlScSlAvWlF4P2Ca7zGrPiEpWo= -github.com/shurcooL/httpfs v0.0.0-20190707220628-8d4bc4ba7749 h1:bUGsEnyNbVPw06Bs80sCeARAlK8lhwqGyi6UT8ymuGk= -github.com/shurcooL/httpfs v0.0.0-20190707220628-8d4bc4ba7749/go.mod h1:ZY1cvUeJuFPAdZ/B6v7RHavJWZn2YPVFQ1OSXhCGOkg= github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= github.com/sirupsen/logrus v1.0.4-0.20170822132746-89742aefa4b2/go.mod h1:pMByvHTf9Beacp5x1UXfOR9xyW/9antXMhjMPG0dEzc= github.com/sirupsen/logrus v1.0.6/go.mod h1:pMByvHTf9Beacp5x1UXfOR9xyW/9antXMhjMPG0dEzc= @@ -768,12 +727,10 @@ github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMB github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88= github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= -github.com/sirupsen/logrus v1.8.1 h1:dJKuHgqk1NNQlqoA6BTlM1Wf9DOH3NBjQyu0h9+AZZE= github.com/sirupsen/logrus v1.8.1/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= github.com/sirupsen/logrus v1.9.0 h1:trlNQbNUG3OdDrDil03MCb1H2o9nJ1x4/5LYw7byDE0= github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ= github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= -github.com/smartystreets/assertions v1.2.0 h1:42S6lae5dvLc7BrLu/0ugRtcFVjoJNMC/N3yZFZkDFs= github.com/smartystreets/assertions v1.2.0/go.mod h1:tcbTF8ujkAEcZ8TElKY+i30BzYlVhC/LOxJk7iOWnoo= github.com/smartystreets/assertions v1.13.0 h1:Dx1kYM01xsSqKPno3aqLnrwac2LetPvN23diwyr69Qs= github.com/smartystreets/assertions v1.13.0/go.mod h1:wDmR7qL282YbGsPy6H/yAsesrxfxaaSlJazyFLYVFx8= @@ -784,8 +741,6 @@ github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4k github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ= github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= -github.com/spf13/afero v1.8.2 h1:xehSyVa0YnHWsJ49JFljMpg1HX19V6NDZ1fkm1Xznbo= -github.com/spf13/afero v1.8.2/go.mod h1:CtAatgMJh6bJEIs48Ay/FOnkljP3WeGUG0MC1RfAqwo= github.com/spf13/afero v1.9.2 h1:j49Hj62F0n+DaZ1dDCvhABaPNSGNkt32oRFxI33IEMw= github.com/spf13/afero v1.9.2/go.mod h1:iUV7ddyEEZPO5gA3zD4fJt6iStLlL+Lg4m2cihcDf8Y= github.com/spf13/cast v0.0.0-20150508191742-4d07383ffe94/go.mod h1:r2rcYCSwa1IExKTDiTfzaxqT2FNHs8hODu4LnUfgKEg= @@ -856,7 +811,6 @@ github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2 github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ= github.com/xeipuuv/gojsonschema v0.0.0-20180618132009-1d523034197f/go.mod h1:5yf86TLmAcydyeJq5YvxkGPE2fm/u4myDekKRoLuqhs= github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU= -github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 h1:QldyIu/L63oPpyvQmHgvgickp1Yw510KJOqX7H24mg8= github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778/go.mod h1:2MuV+tbUrU1zIOPMxZ5EncGwgmMJsa+9ucAQZXxsObs= github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no= github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM= @@ -881,8 +835,6 @@ go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk= -go.opencensus.io v0.23.0 h1:gqCw0LfLxScz8irSi8exQc7fyQ0fKQU/qnC/X8+V/1M= -go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E= go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI= go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= @@ -906,8 +858,6 @@ golang.org/x/crypto v0.0.0-20201117144127-c1f2f97bffc9/go.mod h1:jdWPYTVW3xRLrWP golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4= golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4= golang.org/x/crypto v0.0.0-20211108221036-ceb1ce70b4fa/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= -golang.org/x/crypto v0.0.0-20220411220226-7b82a4e95df4 h1:kUhD7nTDoI3fVd9G4ORWrbV5NY0liEs/Jg2pv5f+bBA= -golang.org/x/crypto v0.0.0-20220411220226-7b82a4e95df4/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= golang.org/x/crypto v0.0.0-20220829220503-c86fa9a7ed90 h1:Y/gsMcFOcR+6S6f3YeMKl5g+dZMEWqcz5Czj/GWYbkM= golang.org/x/crypto v0.0.0-20220829220503-c86fa9a7ed90/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= @@ -920,8 +870,6 @@ golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u0 golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM= golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU= -golang.org/x/exp v0.0.0-20220907003533-145caa8ea1d0 h1:17k44ji3KFYG94XS5QEFC8pyuOlMh3IoR+vkmTZmJJs= -golang.org/x/exp v0.0.0-20220907003533-145caa8ea1d0/go.mod h1:cyybsKvd6eL0RnXn6p/Grxp8F5bW7iYuBgsNCOHpMYE= golang.org/x/exp v0.0.0-20220909182711-5c715a9e8561 h1:MDc5xs78ZrZr3HMQugiXOAkSZtfTpbJLDr/lwfgO53E= golang.org/x/exp v0.0.0-20220909182711-5c715a9e8561/go.mod h1:cyybsKvd6eL0RnXn6p/Grxp8F5bW7iYuBgsNCOHpMYE= golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js= @@ -948,8 +896,6 @@ golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s= -golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -998,10 +944,8 @@ golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qx golang.org/x/net v0.0.0-20210825183410-e898025ed96a/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= -golang.org/x/net v0.0.0-20220520000938-2e3eb7b945c2 h1:NWy5+hlRbC7HK+PmcXVUmW1IMyFce7to56IUvhUFm7Y= -golang.org/x/net v0.0.0-20220520000938-2e3eb7b945c2/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= -golang.org/x/net v0.0.0-20220909164309-bea034e7d591 h1:D0B/7al0LLrVC8aWF4+oxpv/m8bc7ViFfVS8/gXGdqI= -golang.org/x/net v0.0.0-20220909164309-bea034e7d591/go.mod h1:YDH+HFinaLZZlnHAfSS6ZXJJ9M9t4Dl22yv3iI2vPwk= +golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g= +golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= @@ -1024,9 +968,6 @@ golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20220907140024-f12130a52804 h1:0SH2R3f1b1VmIMG7BXbEZCBUu2dKmHschSmjqGUrW8A= -golang.org/x/sync v0.0.0-20220907140024-f12130a52804/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -1114,19 +1055,12 @@ golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBc golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220907062415-87db552b00fd h1:AZeIEzg+8RCELJYq8w+ODLVxFgLMMigSwO/ffKPEd9U= -golang.org/x/sys v0.0.0-20220907062415-87db552b00fd/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220913175220-63ea55921009 h1:PuvuRMeLWqsf/ZdT1UUZz0syhioyv1mzuFZsXs4fvhw= -golang.org/x/sys v0.0.0-20220913175220-63ea55921009/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.4.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.5.0 h1:MUK/U/4lj1t1oPg0HfuXDN/Z1wv31ZJ/YcPiGccS4DU= golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= -golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 h1:JGgROgKl9N8DuW20oFS5gxc+lE67/N3FcwmBPMe7ArY= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= -golang.org/x/term v0.0.0-20220722155259-a9ba230a4035 h1:Q5284mrmYTpACcm+eAKjKJH48BBwSyfJqmmGDTtT8Vc= -golang.org/x/term v0.0.0-20220722155259-a9ba230a4035/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.4.0/go.mod h1:9P2UbLfCdcvo3p/nzKvsmas4TnlujnuoV9hGgYzW1lQ= golang.org/x/term v0.5.0 h1:n2a8QNdAb0sZNpU9R1ALUXBbY+w51fCQDN+7EdxNBsY= golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k= @@ -1136,10 +1070,10 @@ golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3 golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk= golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= +golang.org/x/text v0.7.0 h1:4BRB4x83lYWy72KwLD/qYDuTu7q9PjSagHvijDw7cLo= +golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= @@ -1207,8 +1141,6 @@ golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4f golang.org/x/tools v0.0.0-20210108195828-e2f9c7f1fc8e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0= golang.org/x/tools v0.1.1/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= -golang.org/x/tools v0.1.12 h1:VveCTK38A2rkS8ZqFY25HIDFscX5X9OoEhJd3quQmXU= -golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= @@ -1282,8 +1214,6 @@ google.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6D google.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20210108203827-ffc7fda8c3d7/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20210226172003-ab064af71705/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20220519153652-3a47de7e79bd h1:e0TwkXOdbnH/1x5rc5MZ/VYyiZ4v+RdVfrGMqEwT68I= -google.golang.org/genproto v0.0.0-20220519153652-3a47de7e79bd/go.mod h1:RAyBrSAP7Fh3Nc84ghnVLDPuV51xc9agzmm4Ph6i0Q4= google.golang.org/genproto v0.0.0-20220913154956-18f8339a66a5 h1:ou3VRVAif8UJqz3l1r4Isoz7rrUWHWDHBonShMNYoQs= google.golang.org/genproto v0.0.0-20220913154956-18f8339a66a5/go.mod h1:0Nb8Qy+Sk5eDzHnzlStwW3itdNaWoZA5XeSG+R3JHSo= google.golang.org/grpc v0.0.0-20160317175043-d3ddb4469d5a/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw= @@ -1310,7 +1240,6 @@ google.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA5 google.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= google.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= google.golang.org/grpc v1.40.0/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34= -google.golang.org/grpc v1.46.0/go.mod h1:vN9eftEi1UMyUsIF80+uQXhHjbXYbm0uXoFCACuMGWk= google.golang.org/grpc v1.49.0 h1:WTLtQzmQori5FUH25Pq4WT22oCsv8USpQ+F6rqtsmxw= google.golang.org/grpc v1.49.0/go.mod h1:ZgQEeidpAuNRZ8iRrlBKXZQP1ghovWIVhdJRyCDK+GI= google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= @@ -1326,7 +1255,6 @@ google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlba google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= -google.golang.org/protobuf v1.28.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175w= google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= gopkg.in/airbrake/gobrake.v2 v2.0.9/go.mod h1:/h5ZAUhDkGaJfjzjKLSjv6zCL6O0LLBxU4K+aSYdM/U= From 09904f60eb4bfd4e88c1b43f343c98bee33baf2b Mon Sep 17 00:00:00 2001 From: Cyber-SiKu Date: Fri, 3 Mar 2023 17:30:12 +0800 Subject: [PATCH 09/14] [fix]curvefs/test/client/coredump Signed-off-by: Cyber-SiKu --- curvefs/src/client/common/config.h | 4 ++-- curvefs/test/client/test_disk_cache_manager_impl.cpp | 1 + 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/curvefs/src/client/common/config.h b/curvefs/src/client/common/config.h index 99b92a8003..baa8c69e28 100644 --- a/curvefs/src/client/common/config.h +++ b/curvefs/src/client/common/config.h @@ -104,7 +104,7 @@ struct DiskCacheOption { // the max time system command can run uint32_t cmdTimeoutSec; // threads for disk cache - uint32_t threads; + uint32_t threads = 10; // the write throttle bps of disk cache uint64_t avgFlushBytes; // the write burst bps of disk cache @@ -203,7 +203,7 @@ struct FuseClientOption { bool enableFuseSplice = false; bool disableXattr = false; uint32_t downloadMaxRetryTimes; - uint32_t warmupThreadsNum; + uint32_t warmupThreadsNum = 10; }; void InitFuseClientOption(Configuration *conf, FuseClientOption *clientOption); diff --git a/curvefs/test/client/test_disk_cache_manager_impl.cpp b/curvefs/test/client/test_disk_cache_manager_impl.cpp index b1c81faa9c..90e601873d 100644 --- a/curvefs/test/client/test_disk_cache_manager_impl.cpp +++ b/curvefs/test/client/test_disk_cache_manager_impl.cpp @@ -97,6 +97,7 @@ class TestDiskCacheManagerImpl : public ::testing::Test { TEST_F(TestDiskCacheManagerImpl, Init) { S3ClientAdaptorOption s3AdaptorOption; + s3AdaptorOption.diskCacheOpt.threads = 10; EXPECT_CALL(*diskCacheManager_, Init(_, _)).WillOnce(Return(-1)); int ret = diskCacheManagerImpl_->Init(s3AdaptorOption); ASSERT_EQ(-1, ret); From 3bca2313adc5c8b99a37f878309f2d57618f8152 Mon Sep 17 00:00:00 2001 From: Shivang Shandilya <101946115+ShivangShandilya@users.noreply.github.com> Date: Sun, 12 Mar 2023 19:11:06 +0530 Subject: [PATCH 10/14] Update README.md Signed-off-by: Shivang Shandilya <101946115+ShivangShandilya@users.noreply.github.com> --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 1fa73abdcf..334d895461 100644 --- a/README.md +++ b/README.md @@ -122,7 +122,7 @@ The core application scenarios of CurveFS mainly include:
- Docking Kubernates + Docking Kubernetes - Use [Curve CSI Driver](https://github.com/opencurve/curve-csi), The plugin implements the Container Storage Interface(CSI) between Container Orchestrator(CO) and Curve cluster. It allows dynamically provisioning curve volumes and attaching them to workloads. - For details of the documentation, see [CSI Curve Driver Doc](https://github.com/opencurve/curve-csi/blob/master/docs/README.md). From c1bd0f18ea7cdd2517c8d70590b416ee46618f2d Mon Sep 17 00:00:00 2001 From: Shivang Shandilya <101946115+ShivangShandilya@users.noreply.github.com> Date: Mon, 13 Mar 2023 22:10:44 +0530 Subject: [PATCH 11/14] Update README_cn.md Signed-off-by: Shivang Shandilya <101946115+ShivangShandilya@users.noreply.github.com> --- README_cn.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README_cn.md b/README_cn.md index 1165156b47..b9aa22b211 100644 --- a/README_cn.md +++ b/README_cn.md @@ -124,7 +124,7 @@ CurveFS的核心应用场景主要包括：
- 对接 Kubernates + 对接 Kubernetes - 使用 [Curve CSI Driver](https://github.com/opencurve/curve-csi) 插件在 Container Orchestrator (CO) 与 Curve 集群中实现了 Container Storage Interface(CSI)。 - 文档详见[CSI Curve Driver Doc](https://github.com/opencurve/curve-csi/blob/master/docs/README.md)。 From 7df086e765d801c7946db1af06f7adfb8b2af0b3 Mon Sep 17 00:00:00 2001 From: Zhizhen He Date: Wed, 8 Mar 2023 18:19:03 +0800 Subject: [PATCH 12/14] remove unused dependencies Signed-off-by: Zhizhen He --- tools-v2/go.mod | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/tools-v2/go.mod b/tools-v2/go.mod index c6e6b1e204..945fe889ad 100644 --- a/tools-v2/go.mod +++ b/tools-v2/go.mod @@ -12,6 +12,7 @@ require ( github.com/gookit/color v1.5.2 github.com/moby/term v0.0.0-20220808134915-39b0c02b01ae github.com/olekukonko/tablewriter v0.0.5 + github.com/pkg/xattr v0.4.9 github.com/schollz/progressbar/v3 v3.13.0 github.com/smartystreets/goconvey v1.7.2 github.com/spf13/cobra v1.5.0 @@ -23,8 +24,6 @@ require ( google.golang.org/protobuf v1.28.1 ) -require github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect - require ( github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect github.com/Microsoft/go-winio v0.5.2 // indirect @@ -52,6 +51,7 @@ require ( github.com/mattn/go-runewidth v0.0.14 // indirect github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect github.com/miekg/pkcs11 v1.1.1 // indirect + github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect github.com/mitchellh/mapstructure v1.5.0 // indirect github.com/moby/sys/mount v0.3.3 // indirect github.com/moby/sys/mountinfo v0.6.2 // indirect @@ -61,7 +61,6 @@ require ( github.com/pelletier/go-toml v1.9.5 // indirect github.com/pelletier/go-toml/v2 v2.0.5 // indirect github.com/pkg/errors v0.9.1 // indirect - github.com/pkg/xattr v0.4.9 github.com/prometheus/client_golang v1.13.0 // indirect github.com/prometheus/client_model v0.2.1-0.20210607210712-147c58e9608a // indirect github.com/prometheus/common v0.37.0 // indirect From d114c502c65c2a195e3e0cc3381c1c8f5c52eb78 Mon Sep 17 00:00:00 2001 From: Cyber-SiKu Date: Tue, 14 Mar 2023 17:32:39 +0800 Subject: [PATCH 13/14] [fix]curvebs/Dockerfile:cp curve-nbd Signed-off-by: Cyber-SiKu --- docker/debian9/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/debian9/Dockerfile b/docker/debian9/Dockerfile index 662cc89d5d..4158cf71d8 100644 --- a/docker/debian9/Dockerfile +++ b/docker/debian9/Dockerfile @@ -3,7 +3,7 @@ COPY entrypoint.sh / COPY curvebs /curvebs RUN mkdir -p /etc/curve /etc/nebd \ && chmod a+x /entrypoint.sh \ - && cp curvebs/tools/sbin/curve_ops_tool curvebs/nbd/sbin/curve-nbd \ + && cp curvebs/nbd/sbin/curve-nbd /usr/bin/ \ && cp curvebs/tools/sbin/curve_ops_tool /usr/bin/ \ && cp curvebs/tools-v2/sbin/curve /usr/bin/ ENTRYPOINT ["/entrypoint.sh"] From 9dd890bf2726e2a943b56391772f3878a0b2c97f Mon Sep 17 00:00:00 2001 From: mOUNT41N <92198368+mOUNT41N@users.noreply.github.com> Date: Tue, 14 Mar 2023 16:43:20 +0800 Subject: [PATCH 14/14] [docs] fix links Signed-off-by: mOUNT41N <92198368+mOUNT41N@users.noreply.github.com> --- developers_guide.md | 2 +- developers_guide_cn.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/developers_guide.md b/developers_guide.md index 66b69945ca..c5acbafaae 100644 --- a/developers_guide.md +++ b/developers_guide.md @@ -1,4 +1,4 @@ -[中文版](Community_Guidelines_cn.md) +[中文版](developers_guide_cn.md) ## Overview diff --git a/developers_guide_cn.md b/developers_guide_cn.md index f910a83b52..13f1f754f8 100644 --- a/developers_guide_cn.md +++ b/developers_guide_cn.md @@ -1,4 +1,4 @@ -[English version](Community_Guidelines.md) +[English version](developers_guide.md) ## 概述