codecanvas

Previously named "runner"

Demo

NOTE: please do not rely on this API. This is a hobby project for learning and despite best effort to make it definitely can still crash resilient, it does not handle lots of malicious input. If you do find bugs, please do report them directly or submit a quick issue!

Intro

The runner project is to create an interface for users to run their code remotely without having to have any compiler on their machine. It is a toy project meant for learning how to build a backend in Go and experimenting how to build a somewhat multi-tenant system to run other people's code.

High Level Architecture Diagram

Project State

Sequence diagram with rough state of the project.

Mermaid Live Editor

sequenceDiagram
Client->>Server: submits code

loop Check
      Server->>Server: validate request format
end

Server-->>CodeRunner: call CodeRunner.Run()

loop TransformRequest
      CodeRunner->>CodeRunner: parse language
      CodeRunner->>CodeRunner: create language runtime props
end

CodeRunner-->>Controller: submit job to controller

loop FindRunner
    Controller->>Controller: find available runner
    Controller-->>Controller: lock runner


    loop RunnerExecutionFlow
        Controller->>Standard Shell: pre-run hook (compile)
        Standard Shell-->>Controller: 
        Controller->>Process: execute processor binary

        loop Process
            Process->>Process: set resource limits
            Process->>Process: set non-root uid and gid for user code
            Process->>Process: execute user code
        end

        Process-->>Controller: return exit code
        Controller->>Standard Shell: remove source code
        Standard Shell-->>Controller: return result
    end
    Controller-->>Controller: unlock runner
end




Controller-->>CodeRunner: return stdout, stderr, runtime errors

CodeRunner-->>Server: return CodeRunnerOutput
Server-->>Client: return server transformed response

Loading

Development

Repository Structure

These components live in the following paths:

• browser front-end: web-frontend (thank you to @arekouzounian for this!)
• command-line interface: cli/runner/ (another thank you to @arekouzounian for this!)
• API Server: api/ (thank you to @filipgraniczny for the help!)
• CodeRunner: engine/coderunner (thank you to @siwei-li for the help!)
• Runner Containers: engine/runtime

Dev Environment

Editors:

• Visual Studio Code
• GoLand from Jetbrains for free with an educational license

Extensions setup docs:

• Writing Go in VSCode: https://code.visualstudio.com/docs/languages/go
• Debugging Go in VSCode: https://github.com/golang/vscode-go/blob/master/docs/debugging.md

Recommended extensions (VSCode):

Search for these extension ids in VSCode and feel free to add your personal favs:

1. golang.go for running and debugging Go (see vscode-go debugging docs)
2. eamodio.gitlens git lens (pro tip, enable editor heat map in upper right corner)
3. ms-vscode-remote.remote-containers develop in containers with all dependencies pre-installed
4. ms-vscode-remote.remote-wsl for Windows WSL users
5. yzhang.markdown-all-in-one for writing docs

Docker:

We will likely end up using Docker and include instructions here. For now, you can install Docker Desktop if you like.

Using Dev Containers with VSCode (recommended)

To use a pre-built development container, you can use the VSCode and the dev container provided in .devcontainer/devcontainer.json. This approach will use a Docker container with Go, cobra, python3, and g++ pre-installed and ready to use.

Here is a waay to long video with ~5 mins showing setup and total 12 mins demoing using the container: runner devcontainer setup video

Steps:

1. Verify that you have Docker running
2. Open VSCode and install the Remote - Containers extension: ms-vscode-remote.remote-containers
3. Run the dev container
   1. Open the Command Palette (cmd + shift + P on macOS, F1 or ctrl + shift + p on Windows/Linux)
   2. Run Remote-Containers: Open Folder in Container
   3. Select the runner repository folder
4. Wait for the dev container to start up and open the VSCode Terminal as needed to run commands!

Also see Remote-Containers: open an existing folder in a container.

Development Workflow

This repository is primarily written in Go and the Makefile has a helper commands to make development easier and more consistent.

Note: before you start development, please run make install-hooks to install Git Hooks in your repository's .git/hooks directory. This will install a pre-commit hook that automatically formats your code with gofmt.

Using the Earthfile

Earthly is a Go CLI that works with Docker. It is build tool that lets you run continuous integration and deployment actions locally. The reason it is being used here is to make CI/CD for this repo easier.

For this repo in particular, we need some tests to run in a Dockerized Linux environment to be as close to our deployment image as possible. Earthly makes this really easy since it can can run tests as part of its build process.

• Earthly Website
• Learn the basics

Installing

For macOS users:

brew install earthly/earthly/earthly && earthly bootstrap

For other users: earthly.dev/get-earthly

Using

To use, just check the target you want to run in the Earthfile. It is effectively like a Makefile + Dockerfile and below are a few commands you may want to run during development.

# run all tests and lints, just like how they'll be run in CI when you open a PR
earthly +run-ci

# lint the sourcecode with the golangci lint tool
earthly +lint

# just test the go code with coverage
earthly +test-go

Using the Makefile

By now, you are probably familiar with Makefiles. If not, this wiki provides a great summary: cs104/wiki/makefile (written by Leif Wesche).

Here's a quick summary of what the targets will do:

# print out all the makefile targets
make

# create or create mocks for unit testing, helpful if you have
# modified any of the interfaces in a `types.go` file
make gen-mocks

# run the API server (blocking, you can't use the terminal anymore)
make run-api

# run all tests in the repository
make test

# run go fmt on the repository to format your code
make fmt

# install git-hooks to automatically format your code before you commit
make install-hooks

CLI Setup

CLI stands for command line interface.

Note: this step is not needed if you are using the dev container since cobra is pre-installed in the container.

Installing the cobra CLI to help with codegen

Install cobra dependencies: (required to generate new CLI commands)

go install github.com/spf13/cobra/cobra@v1.3.0

Adding New Commands

Add new cobra command

# change directories into the CLI sourcecode
cd cli/runner

# add new subcommand
cobra add <CHILD_COMMAND> -p <PARENT_COMMAND>

# example:
cobra add childCommand -p 'parentCommand'

Other Resources

• Where the CLI code lives in this repo: cli/runner
• Docs: Cobra Concepts
• Docs: Getting Started
• Examples:
  • adding a command line flag to CLI
  • reading what user set flag to

Running the Server

During CLI or even server development, you will likely want to run the server during testing.

In the root directory runner, you can run the API a couple ways:

# 1. use the Makefile
make run-api

# 2. just use the go command
go run api/main.go

Usually you'll want to run the server in the background to you can do other things with your terminal. However, you'd need to kill the process running on port 10100 once you're done. You can use the api/kill_server.sh script for this.

# 1. run the API in the background
go run api/main.go &

# 2. once you are done, use the script to shut down processes on port 10100
./api/kill_server.sh

You can also use the api/kill_server.sh script if you see this error:

error starting server: listen tcp :10100: bind: address already in use

Go Tips

Working with Go Modules

Go Module:

# you usually will not have to run this since we should already have a go.mod and go.sum file
go mod init github.com/<name>/<repo-name>

# add new library
go get <new dependency>

# organize modules and dependencies
go mod tidy

# remove dependency
go mod edit -dropreplace github.com/go-chi/chi

Testing

Unit Tests

What are unit tests and why do we use them?

Unit testing is used to help us make sure smaller "units" of the code work as expected and handle all expected error cases. This project will end up being pretty large and we want to use unit tests to verify individual components before piecing everything together.

In runner_test.go, we mock the response of the runtime to isolate what we are testing and produce consist results without actually calling our "real code" in the runtime module.

More about unit tests: Definition of a Unit Test.

How to generate mocks:

Install the Go CLI mockgen to create mocks from Go interfaces:

go install github.com/golang/mock/mockgen@v1.6.0

Using Mockgen to create new mocks for testing:

Basic command structure:

mockgen -source ./path/to/file/with/filename.go -destinaion ./path/to/write/mocks/filename.go InterfaceName

Example:

In engine/runtime/types.go there is the interface Runtime that we would like to mock for unit tests:

type Runtime interface {
  RunCmd(runprops *RunProps) (*RunOutput, error)
}

The command below will create a mock-able Runtime interface, helper functions to implement Runtime that you can call RunCmd on.

We can organize mocks in a submodule by making the engine/runtime/mocks directory and provide that and a filename to write the mocked classes.

mockgen -source ./engine/runtime/types.go -package=mocks -destination ./engine/runtime/mocks/Runtime.go Runtime

You can see an example here of how to actually use mocks in a unit test.

Note: The command above has been added to the Makefile. If you are creating mocks you want for a new file or interface, feel free to add those commands to the gen-mocks target so these are generated when you run make gen-mocks.

Integration Testing

Will add more about this later! Here's some reading from Martin Fowler for now!

For now the only sort of "end-to-end" integration test is here: runner/blob/main/engine/integ_test/integration_test.go

Documentation

When writing instructions for users and in the README, please follow syntax recommended by google developer docs