From 222eb97b25a801663903af7fbd094e88318b5115 Mon Sep 17 00:00:00 2001 From: criyle Date: Tue, 6 Apr 2021 02:03:19 -0700 Subject: [PATCH] =?UTF-8?q?docs:=20=E5=8A=A0=E5=85=A5=E4=B8=AD=E6=96=87?= =?UTF-8?q?=E6=96=87=E6=A1=A3?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.cn.md | 472 +++++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 84 +++++++-- go.mod | 17 +- go.sum | 36 ++-- 4 files changed, 566 insertions(+), 43 deletions(-) create mode 100644 README.cn.md diff --git a/README.cn.md b/README.cn.md new file mode 100644 index 0000000..78b0cbb --- /dev/null +++ b/README.cn.md @@ -0,0 +1,472 @@ +# go-judge + +[![Go Reference](https://pkg.go.dev/badge/github.com/criyle/go-judge.svg)](https://pkg.go.dev/github.com/criyle/go-judge) [![Go Report Card](https://goreportcard.com/badge/github.com/criyle/go-judge)](https://goreportcard.com/report/github.com/criyle/go-judge) [![Release](https://img.shields.io/github/v/tag/criyle/go-judge)](https://github.com/criyle/go-judge/releases/latest) ![Build](https://github.com/criyle/go-judge/workflows/Build/badge.svg) + +[English](README.md) + +## 沙箱服务 + +快速，简单，安全 + +### 系统架构 + +```text ++------------------------------------------------------------------------+ +| 传输层 (HTTP / WebSocket / FFI / ...) | ++------------------------------------------------------------------------+ +| 工作协程 (环境池 和 环境生产者 ) | ++-----------------------------------------------------------+------------+ +| 运行环境 | 文件存储 | ++--------------------+----------------+---------------------+------+-----+ +| Linux (go-sandbox) | Windows (winc) | macOS (app sandbox) | 内存 | 磁盘 | ++--------------------+----------------+---------------------+------+-----+ +``` + +### REST API 接口 + +沙箱服务提供 REST API 接口来在受限制的环境中运行程序。本质是 `envexec` 的简单封装。 + +- /run POST 在受限制的环境中运行程序（下面有例子） +- /file GET 得到所有在文件存储中的文件列表 +- /file POST 上传一个文件到文件存储，返回一个文件 ID 用于提供给 /run 接口 +- /file/:fileId GET 下载文件 ID 指定的文件 +- /file/:fileId DELETE 删除文件 ID 指定的文件 +- /ws /run 接口的 WebSocket 版 +- /metrics 提供 prometheus 版监控 (使用 `ES_ENABLE_METRICS=1` 环境变量开启) +- /debug 提供 go 语音调试接口 (使用 `ES_ENABLE_DEBUG=1` 环境变量开启) +- /version 得到本程序编译版本和 go 语音运行时版本 + +### 命令行参数 + +服务相关: + +- 默认监听地址是 `:5050`，使用 `-http-addr` 指定 +- 默认 gRPC 接口处于关闭状态，使用 `-enable-grpc` 开启 +- 默认 gRPC 监听地址是 `:5051` ，使用 `-grpc-addr` 指定 +- 默认日志等级是 debug ，使用 `-silent` 关闭 或 使用 `-release` 开启 release 级别日志 +- 默认没有开启鉴权，使用 `-auth-token` 指定令牌鉴权 +- 默认没有开启 go 语音调试接口，使用 `-enable-debug` 开启 +- 默认没有开启监控接口，使用 `-enable-metrics` 开启 + +沙箱相关: + +- 默认同时运行任务数为 `4` ，使用 `-parallelism` 指定 +- 默认文件存储在内存里，使用 `-dir` 指定本地目录为文件存储 +- 默认 cgroup 的前缀为 `executor_server` ，使用 `-cgroup-prefix` 指定 +- 默认没有磁盘文件复制限制，使用 `-src-prefix` 限制 copyIn 操作文件目录前缀（需要绝对路径） +- 默认时间和内存使用检查周期为 100 毫秒(`100ms`)，使用 `-time-limit-checker-interval` 指定 +- 默认最大输出限制为 `256MiB`，使用 `-output-limit` 指定 +- 默认最大额外内存使用为 `16KiB` ，使用 `-extra-memory-limit` 指定 +- 默认最大 `copyOut` 文件大小为 `64MiB` ，使用 `-copy-out-limit` 指定 +- 使用 `-cpuset` 指定 `cpuset.cpus` （仅 Linux） +- 默认容器用户开始区间为 10000 使用 `-container-cred-start` 指定（仅 Linux） + - 举例，默认情况下第 0 个容器使用 10001 作为容器用户。第 1 个容器使用 10002 作为容器用户，以此类推 +- 使用 `-enable-cpu-rate` 开启 `cpu` cgroup 来启用 `cpuRate` 控制（仅 Linux） + - 使用 `-cpu-cfs-period` 指定 cfs_period if cpu rate is enabled (default 100ms) (valid value: \[1ms, 1s\]) +- 使用 `-seccomp-conf` 指定 `seecomp` 过滤器（需要编译标志 `seccomp`，默认不开启）（仅 Linux） +- 使用 `-pre-fork` 指定启动时创建的容器数量 +- 使用 `-tmp-fs-param` 指定容器内 `tmpfs` 的挂载参数（仅 Linux） + +### 环境变量 + +所以命令行参数都可以通过环境变量的形式来指定，（类似 `ES_HTTP_ADDR` 来指定 `-http-addr`）。使用 `executorserver --help` 查看所以环境变量 + +### 安装和运行 + +下载预编译二进制文件 [Release](https://github.com/criyle/go-judge/releases) 并在终端开启 + +或者使用 docker + +```bash +docker run -it --rm --privileged -p 5050:5050 criyle/executorserver +``` + +#### 编译 docker + +终端中运行 `docker build -t executorserver -f Dockerfile.exec .` + +沙箱服务需要特权级别 docker 来创建子容器和提供 cgroup 资源限制。 + +### 编译沙箱终端 + +编译 `go build ./cmd/executorshell` + +运行 `./executorshell`，需要打开 gRPC 接口来使用。提供一个沙箱内的终端环境。 + +### /run 接口返回状态 + +- Accepted: 程序在资源限制内正常退出 +- Memory Limit Exceeded: 超出内存限制 +- Time Limit Exceeded: + - 超出 `timeLimit` 时间限制 + - 或者超过 `clockLimit` 等待时间限制 +- Output Limit Exceeded: + - 超出 `pipeCollector` 限制 + - 或者超出 `-output-limit` 最大输出限制 +- File Error: + - `copyIn` 指定文件不存在 + - 或者 `copyIn` 指定文件大小超出沙箱文件系统限制 + - 或者 `copyOut` 指定文件不存在 +- Non Zero Exit Status: 程序用非 0 返回值退出 +- Signalled: 程序收到结束信号而退出（例如 `SIGSEGV`） +- Dangerous Syscall: 程序被 `seccomp` 过滤器结束 +- Internal Error: + - 指定程序路径不存在 + - 或者容器创建失败 + - 比如使用非特权 docker + - 或者在个人目录下以 root 权限运行 + - 或者其他错误 + +### 容器的文件系统 + +在 Linux 平台，默认只读挂载点包括主机的 `/lib`, `/lib64`, `/usr`, `/bin`, `/etc/alternatives`, `/etc/fpc.cfg`, `/dev/null`, `/dev/urandom` 和临时文件系统 `/w`, `/tmp` 以及 `/proc`。 + +使用 `mount.yaml` 定制容器文件系统。 + +### 包 + +- envexec: 核心逻辑包，在提供的环境中运行一个或多个程序 +- env: 环境的标准实现 + +### 注意 + +#### CentOS 7 + +需要开启 user 命名空间来使用 [stack overflow](https://superuser.com/questions/1294215/is-it-safe-to-enable-user-namespaces-in-centos-7-4-and-how-to-do-it/1294246#1294246) + +```bash +echo user.max_user_namespaces=10000 >> /etc/sysctl.d/98-userns.conf +sysctl -p +``` + +### 压力测试 + +使用 `wrk` 和 `t.lua`: `wrk -s t.lua -c 1 -t 1 -d 30s --latency http://localhost:5050/run`. + +注意，这些结果只是极限情况下的表现，实际情况和使用方式相关。通常沙箱服务相比于直接运行程序，通常有 1 毫秒左右额外延迟。 + +```lua +wrk.method = "POST" +wrk.body = '{"cmd":[{"args":["/bin/cat","a.hs"],"env":["PATH=/usr/bin:/bin"],"files":[{"content":""},{"name":"stdout","max":10240},{"name":"stderr","max":10240}],"cpuLimit":10000000000,"memoryLimit":104857600,"procLimit":50,"copyIn":{"a.hs":{"content":"main = putStrLn \\"Hello, World!\\""},"b":{"content":"TEST"}}}]}' +wrk.headers["Content-Type"] = "application/json;charset=UTF-8" +``` + +- 单线程 ~400-460 op/s Windows 10 WSL2 @ 5800X +- 多线程 ~1800-2000 op/s Windows 10 WSL2 @ 5800X + +单线程: + +```text +Running 30s test @ http://localhost:5050/run + 1 threads and 1 connections + Thread Stats Avg Stdev Max +/- Stdev + Latency 1.16ms 132.89us 6.20ms 90.15% + Req/Sec 0.87k 19.33 0.91k 85.33% + Latency Distribution + 50% 1.13ms + 75% 1.18ms + 90% 1.27ms + 99% 1.61ms + 25956 requests in 30.01s, 6.88MB read +Requests/sec: 864.88 +Transfer/sec: 234.68KB +``` + +### REST API 接口定义 + +```typescript +interface LocalFile { + src: string; // 文件绝对路径 +} + +interface MemoryFile { + content: string | Buffer; // 文件内容 +} + +interface PreparedFile { + fileId: string; // 文件 id +} + +interface Pipe { + name: string; // copyOut 文件名 + max: number; // 最大大小限制 +} + +interface Cmd { + args: string[]; // 程序命令行参数 + env?: string[]; // 程序环境变量 + + // 指定 标准输入、标准输出和标准错误的文件 + files?: (LocalFile | MemoryFile | PreparedFile | Pipe | null)[]; + tty?: boolean; // 开启 TTY （需要保证标准输出和标准错误为同一文件）同时需要指定 TERM 环境变量 （例如 TERM=xterm） + + // 资源限制 + cpuLimit?: number; // CPU时间限制，单位纳秒 + clockLimit?: number; // 等待时间限制，单位纳秒 （通常为 cpuLimit 两倍） + memoryLimit?: number; // 内存限制，单位 byte + stackLimit?: number; // 栈内存限制，单位 byte + procLimit?: number; // 线程数量限制 + strictMemoryLimit?: boolean; // 开启严格内存限制 （仅 Linux，设置 rlimit 内存限制） + + // 在执行程序之前复制进容器的文件列表 + copyIn?: {[dst:string]:LocalFile | MemoryFile | PreparedFile}; + + // 在执行程序后从容器文件系统中复制出来的文件列表 + copyOut?: string[]; + // 和 copyOut 相同，不过文件不返回内容，而是返回一个对应文件 ID ，内容可以通过 /file/:fileId 接口下载 + copyOutCached?: string[]; + // 指定 copyOut 复制文件大小限制，单位 byte + copyOutMax: number; +} + +enum Status { + Accepted, // normal + MemoryLimitExceeded, // mle + TimeLimitExceeded, // tle + OutputLimitExceeded, // ole + FileError, // fe + RuntimeError, // re + DangerousSyscall, // dgs + InternalError, // system error +} + +interface PipeIndex { + index: number; // cmd 的下标 + fd: number; // cmd 的 fd +} + +interface PipeMap { + in: PipeIndex; // 管道的输入端 + out: PipeIndex; // 管道的输出端 +} + +interface Request { + requestId?: string; // 给 WebSocket 使用 + cmd: Cmd[]; + pipeMapping: PipeMap[]; +} + +interface Result { + status: Status; + error?: string; // 详细错误信息 + exitStatus: number; // 程序返回值 + time: number; // 程序运行 CPU 时间，单位纳秒 + memory: number; // 程序运行内存，单位 byte + runTime: number; // 程序运行现实时间，单位纳秒 + // copyOut 和 pipeCollector 指定的文件内容 + files?: {[name:string]:string}; + // copyFileCached 指定的文件 id + fileIds?: {[name:string]:string}; +} + +// WebSocket 结果 +interface WSResult { + requestId: string; + results: []Result; + error?: string; +} +``` + +### 示例 + +单个文件编译运行（这个例子需要安装 `g++` 例如 `apt install g++` ): + +```json +{ + "cmd": [{ + "args": ["/usr/bin/g++", "a.cc", "-o", "a"], + "env": ["PATH=/usr/bin:/bin"], + "files": [{ + "content": "" + }, { + "name": "stdout", + "max": 10240 + }, { + "name": "stderr", + "max": 10240 + }], + "cpuLimit": 10000000000, + "memoryLimit": 104857600, + "procLimit": 50, + "copyIn": { + "a.cc": { + "content": "#include \nusing namespace std;\nint main() {\nint a, b;\ncin >> a >> b;\ncout << a + b << endl;\n}" + } + }, + "copyOut": ["stdout", "stderr"], + "copyOutCached": ["a.cc", "a"], + "copyOutDir": "1" + }] +} +``` + +```json +[ + { + "status": "Accepted", + "exitStatus": 0, + "time": 303225231, + "memory": 32243712, + "runTime": 524177700, + "files": { + "stderr": "", + "stdout": "" + }, + "fileIds": { + "a": "5LWIZAA45JHX4Y4Z", + "a.cc": "NOHPGGDTYQUFRSLJ" + } + } +] +``` + +```json +{ + "cmd": [{ + "args": ["a"], + "env": ["PATH=/usr/bin:/bin"], + "files": [{ + "content": "1 1" + }, { + "name": "stdout", + "max": 10240 + }, { + "name": "stderr", + "max": 10240 + }], + "cpuLimit": 10000000000, + "memoryLimit": 104857600, + "procLimit": 50, + "strictMemoryLimit": false, + "copyIn": { + "a": { + "fileId": "5LWIZAA45JHX4Y4Z" + } + } + }] +} +``` + +```json +[ + { + "status": "Accepted", + "exitStatus": 0, + "time": 1173000, + "memory": 10637312, + "runTime": 1100200, + "files": { + "stderr": "", + "stdout": "2\n" + } + } +] +``` + +多个程序（例如交互题）: + +```json +{ + "cmd": [{ + "args": ["/bin/cat", "1"], + "env": ["PATH=/usr/bin:/bin"], + "files": [{ + "content": "" + }, null, { + "name": "stderr", + "max": 10240 + }], + "cpuLimit": 1000000000, + "memoryLimit": 1048576, + "procLimit": 50, + "copyIn": { + "1": { "content": "TEST 1" } + }, + "copyOut": ["stderr"] + }, + { + "args": ["/bin/cat"], + "env": ["PATH=/usr/bin:/bin"], + "files": [null, { + "name": "stdout", + "max": 10240 + }, { + "name": "stderr", + "max": 10240 + }], + "cpuLimit": 1000000000, + "memoryLimit": 1048576, + "procLimit": 50, + "copyOut": ["stdout", "stderr"] + }], + "pipeMapping": [{ + "in" : {"index": 0, "fd": 1 }, + "out" : {"index": 1, "fd" : 0 } + }] +} +``` + +```json +[ + { + "status": "Accepted", + "exitStatus": 0, + "time": 1545123, + "memory": 253952, + "runTime": 4148800, + "files": { + "stderr": "" + }, + "fileIds": {} + }, + { + "status": "Accepted", + "exitStatus": 0, + "time": 1501463, + "memory": 253952, + "runTime": 5897700, + "files": { + "stderr": "", + "stdout": "TEST 1" + }, + "fileIds": {} + } +] +``` + +开启 CPURate 限制的死循环： + +```json +{ + "cmd": [{ + "args": ["/usr/bin/python3", "1.py"], + "env": ["PATH=/usr/bin:/bin"], + "files": [{"content": ""}, {"name": "stdout","max": 10240}, {"name": "stderr","max": 10240}], + "cpuLimit": 3000000000, + "clockLimit": 4000000000, + "memoryLimit": 104857600, + "procLimit": 50, + "cpuRate": 0.1, + "copyIn": { + "1.py": { + "content": "while True:\n pass" + } + }}] +} +``` + +```json +[ + { + "status": "Time Limit Exceeded", + "exitStatus": 9, + "time": 414803599, + "memory": 3657728, + "runTime": 4046054900, + "files": { + "stderr": "", + "stdout": "" + } + } +] +``` diff --git a/README.md b/README.md index f5e1815..f221922 100644 --- a/README.md +++ b/README.md @@ -2,8 +2,12 @@ [![Go Reference](https://pkg.go.dev/badge/github.com/criyle/go-judge.svg)](https://pkg.go.dev/github.com/criyle/go-judge) [![Go Report Card](https://goreportcard.com/badge/github.com/criyle/go-judge)](https://goreportcard.com/report/github.com/criyle/go-judge) [![Release](https://img.shields.io/github/v/tag/criyle/go-judge)](https://github.com/criyle/go-judge/releases/latest) ![Build](https://github.com/criyle/go-judge/workflows/Build/badge.svg) +[中文文档](README.cn.md) + ## Executor Service +Fast, Simple, Secure + ### Architecture ```text @@ -54,17 +58,17 @@ Sandbox: - `-output-limit` specifies size limit of POSIX rlimit of output (default 256MiB) - `-extra-memory-limit` specifies the additional memory limit to check memory limit exceeded (default 16KiB) - `-copy-out-limit` specifies the default file copy out max (default 64MiB) -- `-cpuset` specifies `cpuset.cpus` cgroup for each container -- `-container-cred-start` specifies container `setuid` / `setgid` credential start point (default: 10000) +- `-cpuset` specifies `cpuset.cpus` cgroup for each container (Linux only) +- `-container-cred-start` specifies container `setuid` / `setgid` credential start point (default: 10000) (Linux only) - for example, by default container 0 will run with 10001 uid & gid and container 1 will run with 10002 uid & gid... -- `-enable-cpu-rate` enabled `cpu` cgroup to control cpu rate using cfs_quota & cfs_period control -- `-cpu-cfs-period` specifies cfs_period if cpu rate is enabled (default 100ms) (valid value: \[1ms, 1s\]) -- `-seccomp-conf` specifies `seecomp` filter setting to load when running program (need build tag `seccomp`) +- `-enable-cpu-rate` enabled `cpu` cgroup to control cpu rate using cfs_quota & cfs_period control (Linux only) + - `-cpu-cfs-period` specifies cfs_period if cpu rate is enabled (default 100ms) (valid value: \[1ms, 1s\]) +- `-seccomp-conf` specifies `seecomp` filter setting to load when running program (need build tag `seccomp`) (Linux only) - for example, by `strace -c prog` to get all `syscall` needed and restrict to that sub set - however, the `syscall` count in one platform(e.g. x86_64) is not suitable for all platform, so this option is not recommended - the program killed by seccomp filter will have status `Dangerous Syscall` - `-pre-fork` specifies number of container to create when server starts -- `-tmp-fs-param` specifies the tmpfs parameter for `/w` and `/tmp` when using default mounting +- `-tmp-fs-param` specifies the tmpfs parameter for `/w` and `/tmp` when using default mounting (Linux only) ### Environment Variables @@ -77,7 +81,7 @@ Download compiled executable from [Release](https://github.com/criyle/go-judge/r Or, by docker ```bash -docker run -it --rm --privileged -p 5050:5050 criyle/executorserver:demo +docker run -it --rm --privileged -p 5050:5050 criyle/executorserver ``` #### Build Executor Server @@ -188,8 +192,8 @@ wrk.body = '{"cmd":[{"args":["/bin/cat","a.hs"],"env":["PATH=/usr/bin:/bin"]," wrk.headers["Content-Type"] = "application/json;charset=UTF-8" ``` -- Single thread ~400-460 op/s Windows 10 WSL2 -- Multi thread ~1100-1200 op/s Windows 10 WSL2 +- Single thread ~800-860 op/s Windows 10 WSL2 @ 5800X +- Multi thread ~1800-2000 op/s Windows 10 WSL2 @ 5800X Single thread: @@ -197,16 +201,16 @@ Single thread: Running 30s test @ http://localhost:5050/run 1 threads and 1 connections Thread Stats Avg Stdev Max +/- Stdev - Latency 2.23ms 287.79us 14.29ms 86.51% - Req/Sec 451.05 17.60 494.00 76.00% + Latency 1.16ms 132.89us 6.20ms 90.15% + Req/Sec 0.87k 19.33 0.91k 85.33% Latency Distribution - 50% 2.19ms - 75% 2.33ms - 90% 2.50ms - 99% 2.93ms - 13482 requests in 30.02s, 3.58MB read -Requests/sec: 449.15 -Transfer/sec: 121.98KB + 50% 1.13ms + 75% 1.18ms + 90% 1.27ms + 99% 1.61ms + 25956 requests in 30.01s, 6.88MB read +Requests/sec: 864.88 +Transfer/sec: 234.68KB ``` ### REST API Interface @@ -290,6 +294,7 @@ interface Request { interface Result { status: Status; error?: string; // potential system error message + exitStatus: number; time: number; // ns (cgroup recorded time) memory: number; // byte runTime: number; // ns (wall clock time) @@ -488,6 +493,49 @@ Single (this example require `apt install g++` inside the container): ] ``` +```json +{ + "cmd": [{ + "args": ["a"], + "env": ["PATH=/usr/bin:/bin"], + "files": [{ + "content": "1 1" + }, { + "name": "stdout", + "max": 10240 + }, { + "name": "stderr", + "max": 10240 + }], + "cpuLimit": 10000000000, + "memoryLimit": 104857600, + "procLimit": 50, + "strictMemoryLimit": false, + "copyIn": { + "a": { + "fileId": "5LWIZAA45JHX4Y4Z" + } + } + }] +} +``` + +```json +[ + { + "status": "Accepted", + "exitStatus": 0, + "time": 1173000, + "memory": 10637312, + "runTime": 1100200, + "files": { + "stderr": "", + "stdout": "2\n" + } + } +] +``` + Multiple (interaction problem): ```json diff --git a/go.mod b/go.mod index f4064c1..72f236a 100644 --- a/go.mod +++ b/go.mod @@ -3,7 +3,7 @@ module github.com/criyle/go-judge go 1.16 require ( - cloud.google.com/go v0.80.0 // indirect + cloud.google.com/go v0.81.0 // indirect github.com/creack/pty v1.1.11 github.com/criyle/go-sandbox v0.7.2 github.com/elastic/go-seccomp-bpf v1.1.0 @@ -13,7 +13,7 @@ require ( github.com/gin-contrib/pprof v1.3.0 github.com/gin-contrib/zap v0.0.1 github.com/gin-gonic/gin v1.6.3 - github.com/go-playground/validator/v10 v10.4.1 // indirect + github.com/go-playground/validator/v10 v10.4.2 // indirect github.com/gogo/protobuf v1.3.2 // indirect github.com/golang/protobuf v1.5.2 github.com/gorilla/websocket v1.4.2 @@ -30,14 +30,15 @@ require ( go.uber.org/multierr v1.6.0 // indirect go.uber.org/zap v1.16.0 golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2 - golang.org/x/net v0.0.0-20210330230544-e57232859fb2 - golang.org/x/oauth2 v0.0.0-20210323180902-22b0adad7558 // indirect + golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4 + golang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602 // indirect golang.org/x/sync v0.0.0-20210220032951-036812b2e83c - golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44 + golang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57 golang.org/x/term v0.0.0-20210317153231-de623e64d2a6 // indirect - google.golang.org/genproto v0.0.0-20210330181207-2295ebbda0c6 // indirect + golang.org/x/text v0.3.6 // indirect + google.golang.org/genproto v0.0.0-20210405174219-a39eb2f71cb9 // indirect google.golang.org/grpc v1.36.1 - google.golang.org/grpc/examples v0.0.0-20210326170912-4a19753e9dfd // indirect + google.golang.org/grpc/examples v0.0.0-20210405205600-8892a7b247c0 // indirect google.golang.org/protobuf v1.26.0 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect gopkg.in/yaml.v2 v2.4.0 @@ -46,7 +47,7 @@ require ( retract ( // File descripter leak when multiple container fork at the same time - [v0.9.5, v1.1.3] + [v0.9.5, v1.1.4] // Old version, don't use [v0.0.1, v0.9.4] ) diff --git a/go.sum b/go.sum index c0ea329..741853a 100644 --- a/go.sum +++ b/go.sum @@ -17,8 +17,8 @@ cloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKP cloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk= cloud.google.com/go v0.78.0/go.mod h1:QjdrLG0uq+YwhjoVOLsS1t7TW8fs36kLs4XO5R5ECHg= cloud.google.com/go v0.79.0/go.mod h1:3bzgcEeQlzbuEAYu4mrWhKqWjmpprinYgKJLgKHnbb8= -cloud.google.com/go v0.80.0 h1:kAdyAMrj9CjqOSGiluseVjIgAyQ3uxADYtUYR6MwYeY= -cloud.google.com/go v0.80.0/go.mod h1:fqpb6QRi1CFGAMXDoE72G+b+Ybv7dMB/T1tbExDHktI= +cloud.google.com/go v0.81.0 h1:at8Tk2zUz63cLPR0JPWm5vp77pEZmzxEQBEfRKn1VV8= +cloud.google.com/go v0.81.0/go.mod h1:mk/AM35KwGk/Nm2YSeZbxXdrNK3KZOYHmLkOqC2V6E0= cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc= @@ -147,8 +147,8 @@ github.com/go-playground/universal-translator v0.16.0/go.mod h1:1AnU7NaIRDWWzGEK github.com/go-playground/universal-translator v0.17.0 h1:icxd5fm+REJzpZx7ZfpaD876Lmtgy7VtROAbHHXk8no= github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+Scu5vgOQjsIJAF8j9muTVoKLVtA= github.com/go-playground/validator/v10 v10.2.0/go.mod h1:uOYAAleCW8F/7oMFd6aG0GOhaH6EGOAJShg8Id5JGkI= -github.com/go-playground/validator/v10 v10.4.1 h1:pH2c5ADXtd66mxoE0Zm9SUhxE20r7aM3F26W0hOn+GE= -github.com/go-playground/validator/v10 v10.4.1/go.mod h1:nlOn6nFhuKACm19sB/8EGNn9GlaMV7XkbRSipzJ0Ii4= +github.com/go-playground/validator/v10 v10.4.2 h1:RqFe5MzGf2UOFhxQYnjHabHOT6CLbYWkeXOfcXB7fsM= +github.com/go-playground/validator/v10 v10.4.2/go.mod h1:nlOn6nFhuKACm19sB/8EGNn9GlaMV7XkbRSipzJ0Ii4= github.com/go-sql-driver/mysql v1.4.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w= github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= github.com/gogo/googleapis v1.1.0/go.mod h1:gf4bu3Q80BeJ6H1S1vYPm8/ELATdvryBaNFGgqEef3s= @@ -549,8 +549,9 @@ golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc= -golang.org/x/net v0.0.0-20210330230544-e57232859fb2 h1:nGCZOty+lVDsc4H2qPFksI5Se296+V+GhMiL/TzmYNk= golang.org/x/net v0.0.0-20210330230544-e57232859fb2/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= +golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4 h1:4nGaVu0QrbjT/AK2PRLuQfQuh6DJve+pELhqTdAj3x0= +golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= @@ -562,8 +563,8 @@ golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= -golang.org/x/oauth2 v0.0.0-20210323180902-22b0adad7558 h1:D7nTwh4J0i+5mW4Zjzn5omvlr6YBcWywE6KOcatyNxY= -golang.org/x/oauth2 v0.0.0-20210323180902-22b0adad7558/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602 h1:0Ja1LBD+yisY6RWM/BH7TJVXWsSjs2VwBSmvSX4HdBc= +golang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= @@ -629,11 +630,11 @@ golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210309074719-68d13333faf2/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210314195730-07df6a141424/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44 h1:Bli41pIlzTzf3KEY06n+xnzK/BESIg2ze4Pgfh/aI8c= golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57 h1:F5Gozwx4I1xtr/sr/8CFbb57iKi3297KFs0QDbGN60A= +golang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20210317153231-de623e64d2a6 h1:EC6+IGYTjPpRfv9a2b/6Puw0W+hLtAhkV1tPsXhutqs= golang.org/x/term v0.0.0-20210317153231-de623e64d2a6/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= @@ -643,8 +644,9 @@ golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3 golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.5 h1:i6eZZ+zk0SOf0xgBpEpPD18qWcJda6q1sxt3S0kzyUQ= golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M= +golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= @@ -730,7 +732,7 @@ google.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ google.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE= google.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8= google.golang.org/api v0.41.0/go.mod h1:RkxM5lITDfTzmyKFPt+wGrCJbVfniCr2ool8kTBzRTU= -google.golang.org/api v0.42.0/go.mod h1:+Oj4s6ch2SEGtPjGqfUfZonBH0GjQH89gTeKKAEGZKI= +google.golang.org/api v0.43.0/go.mod h1:nQsDGjRXMo4lvh5hP0TKqF244gqhGcr/YSIykhUk/94= google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= google.golang.org/appengine v1.2.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= @@ -780,10 +782,10 @@ google.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6D google.golang.org/genproto v0.0.0-20210222152913-aa3ee6e6a81c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20210303154014-9728d6b83eeb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20210310155132-4ce2db91004e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20210312152112-fc591d9ea70f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20210323160006-e668133fea6a/go.mod h1:f2Bd7+2PlaVKmvKQ52aspJZXIDaRQBVdOOBfJ5i8OEs= -google.golang.org/genproto v0.0.0-20210330181207-2295ebbda0c6 h1:F+kz69Q2lpqBTskKa73+uBMwi7PBQRQEUBqHt0oc1K8= -google.golang.org/genproto v0.0.0-20210330181207-2295ebbda0c6/go.mod h1:9lPAdzaEmUacj36I+k7YKbEc5CXzPIeORRgDAUOu28A= +google.golang.org/genproto v0.0.0-20210319143718-93e7006c17a6/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20210402141018-6c239bbf2bb1/go.mod h1:9lPAdzaEmUacj36I+k7YKbEc5CXzPIeORRgDAUOu28A= +google.golang.org/genproto v0.0.0-20210405174219-a39eb2f71cb9 h1:Uga0hMhZnzB159cnlmhVPgEmZTwoZqwOb7e0pu+rlNI= +google.golang.org/genproto v0.0.0-20210405174219-a39eb2f71cb9/go.mod h1:P3QM42oQyzQSnHPnZ/vqoCdDmzH28fzWByN9asMeM8A= google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= google.golang.org/grpc v1.20.0/go.mod h1:chYK+tFQF0nDUGJgXMSgLCQk3phJEuONr2DCgLDdAQM= @@ -808,8 +810,8 @@ google.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAG google.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= google.golang.org/grpc v1.36.1 h1:cmUfbeGKnz9+2DD/UYsMQXeqbHZqZDs4eQwW0sFOpBY= google.golang.org/grpc v1.36.1/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= -google.golang.org/grpc/examples v0.0.0-20210326170912-4a19753e9dfd h1:UdBI3NZesDI4AuP02aKqpg73FiB0RO728xXQjlDfWk8= -google.golang.org/grpc/examples v0.0.0-20210326170912-4a19753e9dfd/go.mod h1:Ly7ZA/ARzg8fnPU9TyZIxoz33sEUuWX7txiqs8lPTgE= +google.golang.org/grpc/examples v0.0.0-20210405205600-8892a7b247c0 h1:T9WEcYHefnWQ9Uj+8SQ3tFS4RfH3u8ONRtRQM7BhDvo= +google.golang.org/grpc/examples v0.0.0-20210405205600-8892a7b247c0/go.mod h1:Ly7ZA/ARzg8fnPU9TyZIxoz33sEUuWX7txiqs8lPTgE= google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=