diff --git a/CHANGELOGS.md b/CHANGELOGS.md index 2690d6d..1b67d99 100644 --- a/CHANGELOGS.md +++ b/CHANGELOGS.md @@ -20,7 +20,7 @@ ### Added -- [CLI] 新增独立命令 `hpr-migration` 迁移命令 (目前为止仅支持 gitlab-mirror`) +- [CLI] 新增独立命令 `hpr-migration` 迁移命令 (目前为止仅支持 gitlab-mirrors`) - [CLI] 新增全局参数 `--file` 可以指定自定义的 hpr.json 配置文件. ### Fixed @@ -34,7 +34,7 @@ - [API] `/repositories/search` 搜索仓库 API 参数从 query 改为 uri path 方式，名称从 `q` 改为 `keyword` -## [0.6.2] (2018-06-20) +## [0.7.0] (2018-06-20) ### Fixed @@ -96,8 +96,8 @@ - 第一个测试版本 -[Unreleased]: https://github.com/icyleaf/hpr/compare/v0.6.2...HEAD -[0.6.2]: https://github.com/icyleaf/hpr/compare/v0.6.0...v0.6.2 +[Unreleased]: https://github.com/icyleaf/hpr/compare/v0.7.0...HEAD +[0.7.0]: https://github.com/icyleaf/hpr/compare/v0.6.0...v0.7.0 [0.6.0]: https://github.com/icyleaf/hpr/compare/v0.5.0...v0.6.0 [0.5.0]: https://github.com/icyleaf/hpr/compare/v0.3.0...v0.5.0 [0.3.0]: https://github.com/icyleaf/hpr/compare/v0.2.0...v0.3.0 diff --git a/Makefile b/Makefile index 66f6e5a..69dae21 100644 --- a/Makefile +++ b/Makefile @@ -1,5 +1,5 @@ hpr_image_name ?= icyleafcn/hpr -hpr_version ?= 0.6.2 +hpr_version ?= 0.7.0 all: build diff --git a/docker-compose.yml b/docker-compose.yml index 04431c3..86b0aa4 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -6,8 +6,7 @@ services: ports: - 8848:8848 volumes: - - ./config:/app/config - - ./repositories:/app/repositories + - /my/own/hprdir:/app environment: REDIS_URL: tcp://redis:6379 REDIS_PROVIDER: REDIS_URL diff --git a/docs/_coverpage.md b/docs/_coverpage.md index 3ba8802..11c9d13 100644 --- a/docs/_coverpage.md +++ b/docs/_coverpage.md @@ -1,6 +1,6 @@ ![logo](_media/icon.png) -# ḫpr 0.6.2 +# ḫpr 0.7.0 > 镜像任意 git 仓库到 gitlab 的同步工具，具有定时更新的功能。 diff --git a/docs/api.md b/docs/api.md index 0e25c7d..de84ac0 100644 --- a/docs/api.md +++ b/docs/api.md @@ -154,6 +154,8 @@ POST /repositores |---|---|---|---| | url | String | true | 仓库地址 | | name | String | false | 镜像名字，不填写默认从 url 自动获取 | +| create | String | false | 是否创建 gitlab 项目，默认是 "true" | +| clone | String | false | 是否克隆原仓库并推送到 gitlab，默认是 "true" | #### 返回样例 @@ -224,7 +226,7 @@ GET /info ```json { "hpr": { - "version": "0.6.2", + "version": "0.7.0", "repositroies": { "total": 2, "entry": [ diff --git a/docs/en/_coverpage.md b/docs/en/_coverpage.md index 095854f..7006bef 100644 --- a/docs/en/_coverpage.md +++ b/docs/en/_coverpage.md @@ -1,6 +1,6 @@ ![logo](../_media/icon.png) -# ḫpr 0.6.2 +# ḫpr 0.7.0 > Mirror git repositories to self-host gitlab services. diff --git a/docs/en/api.md b/docs/en/api.md index 09de724..cb794c6 100644 --- a/docs/en/api.md +++ b/docs/en/api.md @@ -22,8 +22,8 @@ GET /repositores | Name | Type | Required | Description | |---|---|---|---| -| page | Integer | false | | -| per_page | Integer | false | | +| page | Integer | false | The number of current page | +| per_page | Integer | false | The number of per page | #### Example Response @@ -67,7 +67,7 @@ GET /repositores/search/:name | Name | Type | Required | Description | |---|---|---|---| -| name | String | true | the name of repository | +| name | String | true | The name of mirrored repository | #### Example Response @@ -109,7 +109,7 @@ GET /repositores/:name | Name | Type | Required | Description | |---|---|---|---| -| name | String | false | Name of mirrored repository | +| name | String | false | The name of mirrored repository | #### Example Response @@ -149,8 +149,10 @@ POST /repositores | Name | Type | Required | Description | |---|---|---|---| -| url | String | true | | -| name | String | false | | +| url | String | true | The clone url of origin | +| name | String | false | The name of mirrored repository | +| create | String | false | Should to create gitlab project, by default is "true" | +| clone | String | false | Should to clone origin repository, by default is "true" | #### Example Response diff --git a/docs/en/install.md b/docs/en/install.md index 68013f9..3e2e02f 100644 --- a/docs/en/install.md +++ b/docs/en/install.md @@ -15,7 +15,7 @@ Check [Quick start](/en/quickstart.md)。 Pull specified version of hpr: ```bash -$ docker pull icyleafcn/hpr:0.6.2 +$ docker pull icyleafcn/hpr:0.7.0 ``` Or pull the latest version: @@ -99,6 +99,8 @@ Actions: -c, --create Create a mirror repository -u, --update Updated a mirrored repository -d, --delete Delete a mirrored repository + -v, --version Show version + -h, --help Show this help Option in server action: @@ -107,12 +109,12 @@ Option in server action: Option in create action: -U URL, --url URL The url of mirror repository - -M, --mirror-only Only mirror the repository without clone in create action + --no-create Do not create project in gitlab + --no-clone Do not clone mirror of git repository from url Global options: - -v, --version Show version - -h, --help Show this help + -f FILE, --file FILE the path of hpr.json config file Examples: @@ -120,13 +122,13 @@ Examples: $ hpr -s - o List all mirrored repositories: + o Start a API server with custom port and different config path: - $ hpr -l + $ hpr -s --port 3001 --file ~/.config/hpr/hpr.json - o Start a API server with custom port: + o List all mirrored repositories: - $ hpr -s --port 3001 + $ hpr -l o Search all repositories include icyleaf keywords: @@ -138,7 +140,7 @@ Examples: o Clone and push a new repository without create gitlab project: - $ hpr -c --mirror-only --url https://github.com/icyleaf/hpr.git icyleaf-hpr + $ hpr -c --no-create --url https://github.com/icyleaf/hpr.git icyleaf-hpr o Update a repository: @@ -150,5 +152,5 @@ Examples: More detail to check: https://icyleaf.github.io/hpr/ -hpr v0.6.2 in Crystal v0.24.2 +hpr v0.7.0 in Crystal v0.26.1 ``` diff --git a/docs/en/migrate.md b/docs/en/migrate.md index fa4a81d..a611002 100644 --- a/docs/en/migrate.md +++ b/docs/en/migrate.md @@ -23,17 +23,9 @@ Enable MR | merge_requests_enabled | gitlab.project_merge_request | Yes More about hpr's config to check [Configurateion](/en/configuration.md) page. -Then, migrate old repositories directory to hpr: +Next step, you need get the path of gitlab-mirrors's repositories directory: get $repo_dir value from config.sh，default is /home/gitmirror/repositories -```bash -# get $repo_dir value from config.sh，default is /home/gitmirror/repositories -$ cd /home/gitmirror/repositories - -# Copy the whole directory to hpr's directory -$ cp -r mirrors /path/to/hpr/repositories -``` - -Finally, edit docker-compose.yml file: +Then edit docker-compose.yml file, next move is run `docker-compose up -d` ```yaml version: '2' @@ -44,8 +36,8 @@ services: ports: - 8848:8848 volumes: - - ./config:/app/config - - /path/to/hpr/repositories:/app/repositories + - /my/own/hprdir:/app + - /home/gitmirror/repositories:/tmp/old-repositories environment: REDIS_URL: tcp://redis:6379 REDIS_PROVIDER: REDIS_URL @@ -58,22 +50,20 @@ services: image: redis:alpine ``` -Run `docker-compose up -d` +Hpr is running now, but the data is not migrate, hpr provides a migration command tool named "hpr-migration" to make this move easily: -Apply the update by schedule, you need do this: - -```ruby -# gem install http -require 'http' - -# Change to ip or address which hpr is -hpr_url = 'http://localhost:8848/repositories' - -repositories = HTTP.get(hpr_url).parse -repositories.each do |repo| - url = File.join(hpr_url, repo["name"]) - HTTP.put url -end +```bash +$ docker-compose exec hpr hpr-migration --endpoint "http://localhost:8848" /tmp/old-repositories +* project1 + - Configuring git remote ... + - Updating and pushing mirror +* project2 + - Create gitlab repository + - Configuring git remote ... + - Updating and pushing mirror +* project3 + - Existed, Skip ``` +You can get migrated data via [stats](/en/api.md#id=stats) api. By default, Update cycle is every hour in `schedule` (`config/hpr.json`). diff --git a/docs/en/quickstart.md b/docs/en/quickstart.md index ebf49ef..dcdea36 100644 --- a/docs/en/quickstart.md +++ b/docs/en/quickstart.md @@ -49,7 +49,8 @@ Here has 4 places to change your own. > About more params check [Configuration](configuration?id=basic_auth-接口认证) page. -Then set some optional `ENV` variables if you need in `docker-compose.yml`: +1. Create a data directory on a suitable volume on your host system. e.g. **/my/own/hprdir**. +2. Set some optional `ENV` variables if you need in `docker-compose.yml`: ```yaml version: '2' @@ -74,7 +75,7 @@ services: image: redis:alpine ``` -the `HPR_SSH_HOST` and `HPR_SSH_PORT` variables will update your gitlab ssh config, ignore if your gitlab server use 22 port in ssh protocol. +The `HPR_SSH_HOST` and `HPR_SSH_PORT` variables will update your gitlab ssh config, ignore if your gitlab server use 22 port in ssh protocol. Then run it: @@ -97,6 +98,11 @@ hpr_1 | | '_ \| '_ \| '__| hpr_1 | | | | | |_) | | hpr_1 | |_| |_| .__/|_| hpr_1 | |_| +hpr_1 | [12] Salt server starting ... +hpr_1 | [12] * Version 0.4.2 (Crystal 0.26.1) +hpr_1 | [12] * Environment: production +hpr_1 | [12] * Listening on http://0.0.0.0:8848/ +hpr_1 | [12] Use Ctrl-C to stop ``` Be attention to copy generated ssh public key in terminal output. diff --git a/docs/examples.md b/docs/examples.md index fb7bca8..bb691ed 100644 --- a/docs/examples.md +++ b/docs/examples.md @@ -3,5 +3,4 @@ 项目代码中提供了两个范例来帮助理解和本地测试使用。 - [自建 Gitlab 服务](https://github.com/icyleaf/hpr/tree/master/examples/selfhost) - - [采用 Gitlab.com 服务](https://github.com/icyleaf/hpr/tree/master/examples/saas) diff --git a/docs/install.md b/docs/install.md index c8d6c81..574e9c5 100644 --- a/docs/install.md +++ b/docs/install.md @@ -15,7 +15,7 @@ hpr 使用 Crystal 编写的工具可以被安装在 macOS、Linux、树莓派 获取指定版本的 hpr: ```bash -$ docker pull icyleafcn/hpr:0.6.2 +$ docker pull icyleafcn/hpr:0.7.0 ``` 或者获取最新版本的 hpr: @@ -99,6 +99,8 @@ Actions: -c, --create Create a mirror repository -u, --update Updated a mirrored repository -d, --delete Delete a mirrored repository + -v, --version Show version + -h, --help Show this help Option in server action: @@ -107,12 +109,12 @@ Option in server action: Option in create action: -U URL, --url URL The url of mirror repository - -M, --mirror-only Only mirror the repository without clone in create action + --no-create Do not create project in gitlab + --no-clone Do not clone mirror of git repository from url Global options: - -v, --version Show version - -h, --help Show this help + -f FILE, --file FILE the path of hpr.json config file Examples: @@ -120,13 +122,13 @@ Examples: $ hpr -s - o List all mirrored repositories: + o Start a API server with custom port and different config path: - $ hpr -l + $ hpr -s --port 3001 --file ~/.config/hpr/hpr.json - o Start a API server with custom port: + o List all mirrored repositories: - $ hpr -s --port 3001 + $ hpr -l o Search all repositories include icyleaf keywords: @@ -138,7 +140,7 @@ Examples: o Clone and push a new repository without create gitlab project: - $ hpr -c --mirror-only --url https://github.com/icyleaf/hpr.git icyleaf-hpr + $ hpr -c --no-create --url https://github.com/icyleaf/hpr.git icyleaf-hpr o Update a repository: @@ -150,5 +152,5 @@ Examples: More detail to check: https://icyleaf.github.io/hpr/ -hpr v0.6.2 in Crystal v0.24.2 +hpr v0.7.0 in Crystal v0.26.1 ``` diff --git a/docs/migrate.md b/docs/migrate.md index 9d1ccbf..1ddc71b 100644 --- a/docs/migrate.md +++ b/docs/migrate.md @@ -22,17 +22,9 @@ gitlab 分组名 | gitlab_namespace | gitlab.group_name | **必须** 其他参数不需要迁移，对于 hpr 额外的参数定义参加[配置文件](configuration.md)。 -配置文件迁移完毕好之后，需要把当前已经管理的仓库文件夹复制到 hpr 对于文件夹中。 +配置文件迁移完毕好之后，需要获取 gitlab-mirrors 的仓库路径。从 gitlab-mrrors 的 config.sh 拿到 $repo_dir 的路径，默认是 /home/gitmirror/repositories -```bash -# 从 gitlab-mrrors 的 config.sh 拿到 $repo_dir 的路径，默认是 /home/gitmirror/repositories -$ cd /home/gitmirror/repositories - -# 把 gitlab_namespace 的名称的文件夹制到新的项目，这里比方说是 mirrors -$ cp -r mirrors /path/to/hpr/repositories -``` - -编辑 docker-compose.yml +编辑 docker-compose.yml 并把刚才得到的仓库路径加到 volumes 里面。 ```yaml version: '2' @@ -43,8 +35,8 @@ services: ports: - 8848:8848 volumes: - - ./config:/app/config - - /path/to/hpr/repositories:/app/repositories + - /my/own/hprdir:/app + - /home/gitmirror/repositories:/tmp/old-repositories environment: REDIS_URL: tcp://redis:6379 REDIS_PROVIDER: REDIS_URL @@ -57,22 +49,23 @@ services: image: redis:alpine ``` -运行实例: `docker-compose up -d` +运行实例: `docker-compose up -d` -这里只是那老的数据迁移到 hpr，还需要设置定时任务更新，在配置好 ssh key 之后这里目前没有工具需要通过脚本完成： +目前只是把 hpr 运行起来了，但还没有真正迁移数据。Hpr 提供了一个迁移工具来帮助你轻松快速完全，确保还在刚才的目录下执行： -```ruby -# gem install http -require 'http' - -# 这里修改成你的 hpr 实例的地址或 IP -hpr_url = 'http://localhost:8848/repositories' - -repositories = HTTP.get(hpr_url).parse -repositories.each do |repo| - url = File.join(hpr_url, repo["name"]) - HTTP.put url -end +```bash +$ docker-compose exec hpr hpr-migration --endpoint "http://localhost:8848" /tmp/old-repositories +* project1 + - Configuring git remote ... + - Updating and pushing mirror +* project2 + - Create gitlab repository + - Configuring git remote ... + - Updating and pushing mirror +* project3 + - Existed, Skip ``` -如果你没有修改 `config/hpr.json` 的 `schedule` 的值默认是每小时进行更新。 \ No newline at end of file +> 温馨提示：工具会把三种情况都会考虑在内，分别包括本地已存在项目但 gitlab 没有/没有的项目/存在的项目。 + +命令执行完毕后可通过 [统计接口](api.md#id=统计信息) 获取同步状态，包括定时更新到时间。如果你没有修改 `config/hpr.json` 的 `schedule` 的值默认是每小时进行更新。 diff --git a/docs/quickstart.md b/docs/quickstart.md index 9cbed18..63124b2 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -60,8 +60,7 @@ services: ports: - 8848:8848 volumes: - - ./config:/app/config - - ./repositories:/app/repositories + - /my/own/hprdir:/app environment: REDIS_URL: tcp://redis:6379 REDIS_PROVIDER: REDIS_URL @@ -74,7 +73,8 @@ services: image: redis:alpine ``` -其中 `HPR_SSH_HOST` 和 `HPR_SSH_PORT` 变量用于设置 Docker 实例中的 SSH 配置。如果 SSH 端口是 22 的可忽略设置这俩参数。 +其中需要修改 **/my/own/hprdir** 为实际存放数据的地方而 `HPR_SSH_HOST` 和 `HPR_SSH_PORT` 变量用于设置 Docker 实例中的 SSH 配置。 +如果 SSH 端口是 22 的可忽略设置这俩参数。 编辑完成后运行下面命令快完成了！ @@ -83,7 +83,7 @@ $ docker-compose up ... hpr_1 | Generating public/private rsa key pair ... hpr_1 | -hpr_1 | GENERATED SSH PUBLIC KEY: +hpr_1 | SSH PUBLIC KEY: hpr_1 | ################################################################## hpr_1 | ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDq8O3HbLn9x8Uy8RUotlpOnxdakrmCyDpZrGBeLARmEbd6BOIBQ+UWm8NUKthQ7UOavmlsq4j8lY4kyFW2eFX2qWcbvI+s2gI+05MXax+mAukSszaNSnpAoTyJCRipilSkqiOV99V8JIJhrHPtTO0o/Ui 9WiyyWsUM4M9lEKHpZ486lDGk3IM2XQW+pxAoMKb0TYzqCsrduHUtjzy0M0BqgMPe9EtVQqCbnTMzDLXmRONoTYyTV51NQ12mMwEQcDaLQ28e5gqouQJKS81JaoRpQWa7pHsOCki6Fk9TB+EQFrGz5nOrmYYM+O1MKnFkzmVHv7Fh50Sz7d2nYzzOKAkR hpr@docker @@ -97,6 +97,12 @@ hpr_1 | | '_ \| '_ \| '__| hpr_1 | | | | | |_) | | hpr_1 | |_| |_| .__/|_| hpr_1 | |_| +hpr_1 | |_| +hpr_1 | [12] Salt server starting ... +hpr_1 | [12] * Version 0.4.2 (Crystal 0.26.1) +hpr_1 | [12] * Environment: production +hpr_1 | [12] * Listening on http://0.0.0.0:8848/ +hpr_1 | [12] Use Ctrl-C to stop ``` 最后从执行命令的输出找到生成的 SSH PUBLIC KEY（两个井号中间的部分，以 `ssh-rsa` 开头，`hpr@docker` 结尾）， diff --git a/shard.yml b/shard.yml index 604f266..74b82a8 100644 --- a/shard.yml +++ b/shard.yml @@ -1,5 +1,5 @@ name: hpr -version: 0.6.2 +version: 0.7.0 authors: - icyleaf diff --git a/src/hpr/cli/migration.cr b/src/hpr/cli/migration.cr index e441f6e..19ced56 100644 --- a/src/hpr/cli/migration.cr +++ b/src/hpr/cli/migration.cr @@ -79,7 +79,7 @@ Dir.glob("#{source_path}/*") do |repo_path| end if gitlab_project - puts " - Configuring git remote ..." + puts " - Configuring git remote" Hpr::Utils.write_mirror_to_git_config(repo_name, gitlab_project.as_h["path"].as_s) puts " - Updating and pushing mirror" diff --git a/src/hpr/version.cr b/src/hpr/version.cr index 3cd3660..92045cf 100644 --- a/src/hpr/version.cr +++ b/src/hpr/version.cr @@ -1,3 +1,3 @@ module Hpr - VERSION = "0.6.2" + VERSION = "0.7.0" end