Getting started with Longhorn Development

Setting up for Longhorn development

This page will go over setting up your environment for development work on the current version of longhorn. Since longhorn involves in system programming, it should be tested in Linux environment. If you want to use IDE GoLand in your MacOS, you may need to set up a remote linux server.

Installations

Config GoLand + remote server

Sync your code with remote server

1. Click Tools -> Deployment -> Configuration -> “+” button

   1. Choose SFTP

   2. In Connection

      1. Host: IP address of the remote server
      2. User Name: user name of the remote host
      3. Authentication: choose Key pair, find your private key path (usually ~/.ssh/id_rsa)
      4. Root path: click Autodetect

   3. In Mappings

      1. Local path: your repo path
      2. Deployment path: the corresponding repo path on your remote host

2. Click Tools -> Deployment -> Options Upload changed files automatically to the default server: choose Always or On explicit save action

Warning: Since the local repo and the remote repo are different local repos from git perspective, please keep git commit operations in the same local repo.

Run go fmt on explict save

Click GoLand -> Preference -> Tools -> File Wathcers -> "+" button -> choose "go fmt"

Install docker

curl -L https://releases.rancher.com/install-docker/18.09.sh | sh

You can choose other version here:

install-docker supported by rancher

Then add user in 'docker' group:

sudo usermod -aG docker $USER

Install Go

Install the latest version of go

sudo add-apt-repository ppa:longsleep/golang-backports

sudo apt-get update

sudo apt-get install golang-go

You should add the gopath bin directory to your path. Add the following to your .profile or .bashrc or other shell file

export GOPATH=$HOME/go

export PATH=$PATH:$GOPATH/bin

Install open-iscsi package

sudo apt install -y open-iscsi

Trash

Trash is a dependency management tool written by Rancher for use with rancher. You'll only need this to update your dependencies. See more info here: Trash - Go ./vendor manager

go get -u github.com/rancher/trash

Warning: Please run trash in case sensitive file system, otherwise you may encounter "Permission denied" issues. By default, the filesystem of Mac OS is case insensitive.

Dapper

Dapper - Docker Build Wrapper

go get github.com/rancher/dapper

Install pip and then python 2.7

sudo apt install python-pip

sudo apt install python2.7

Development

Components

For backend, Longhorn is consists of longhorn-manager, longhorn-engine and backupstore, along with longhorn-tests as integration tests, and longhorn charts for rancher deploying Longhorn.

How to update dependencies (libs in /vendor)

You will see vendor.conf in the project root directory. This file contains all libs with specific version the project needs. You can type trash command to update dependencies.

How to deploy modified Longhorn

Package your modified project

Because all things are containerized, you should package your project as docker image by running make.

Upload the image

Kubernetes always fetches images from DockerHub to create containers. Hence you need to push your images to DockerHub repo.

If you haven't get one, please register here and create private repos for the project.

Then you need to login on your dev machine.

docker login --username= --password=

Now you can push your image

docker push <USER_NAME>/<REPO_NAME>:<VERSION>

Deploy/Update your Longhorn

After you get a new docker image for this commit, you also need to update your deployment by modifying corresponding image value of those .yaml file in /deploy dir.

You can use update_image.sh to update them. Environment variable REPO needs to be set as your DockerHub username before using the script. And be careful that the script is valid only when the initial image value is unchanged and in Linux OS.

Then you can re-create or update your longhorn components with these new yaml files. Done

Test

For unit tests, They will be run when you make a new docker image. For integration tests, they are consist of pytest.

longhorn-manger & longhorn-tests

longhorn-manger unit tests are in directory /controller, /scheduler and so on. If you modified these module, you may need to add/modify the unit tests too.

Besides, longhorn-manger integration tests are in longhorn-tests. After modifying longhorn-manager, it's better to check whether adding/modifying integration tests is needed and run this test. You need to manually create new image and deploy them. The procedures are similar to Development. By the way, you can add pytest flags in test.yaml.

longhorn-engine

The integration tests are in directory /integration. This integration tests will also be run when you make new image.

If you just want to check specific test case, you can run dapper command.

e.g.dapper ./scripts/integration-test -- -s -x -k test_cli

backupstore

The integration tests are in directory /integration of repo longhorn-test.

Before running integration test for backupstore, you may need to modify the repo url of backupstore in vendor.conf and update /vendor.

e.g. github.com/rancher/backupstore 285193fc29528a1ee08272e52800eefd72e782e2 https://github.com/<YOUR_GITHUB_NAME>/backupstore.git

Charts

After you modified yaml files in /deploy, e.g. added a new flag, you may also need to update the corresponding fields in longhorn charts.