Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update documentation regarding containerlab migration. #83

Merged
merged 2 commits into from
Dec 3, 2021
Merged

Update documentation regarding containerlab migration. #83

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3be46e9
Update documentation regarding containerlab migration.
Gerrit91 Nov 15, 2021
c35dd75
Add info that containerlab requires root permissions.
Gerrit91 Nov 18, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 44 additions & 65 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,70 +1,61 @@
# mini-lab

The mini-lab is a small, virtual setup to locally run the metal-stack. It deploys the metal control plane and a partition with two simulated leaf switches. The lab can be used for trying out metal-stack, demonstration purposes or development.
The mini-lab is a small, virtual setup to locally run the metal-stack. It deploys the metal control plane and a metal-stack partition with two simulated leaf switches. The lab can be used for trying out metal-stack, demonstration purposes or development.

![overview components](docs/overview.png)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we redraw it completely, similar to how containerlab presents topologies in it's docs: https://containerlab.srlinux.dev/lab-examples/cfg-clos/#description, but with different(metal-stack like) palette?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That'd be great. But it's also quite a lot of work. 😓

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try to do some proto. But there's big chance it will look quirky)


This project can also be used as a template for writing your own metal-stack deployments.
ℹ This project can also be used as a template for writing your own metal-stack deployments.

<!-- TOC depthFrom:2 depthTo:6 withLinks:1 updateOnSave:1 orderedList:0 -->
<!-- TOC depthfrom:2 depthto:6 withlinks:true updateonsave:false orderedlist:false -->

- [mini-lab](#mini-lab)
- [Requirements](#requirements)
- [Known Limitations](#known-limitations)
- [Try it out](#try-it-out)
- [Requirements](#requirements)
- [Known Limitations](#known-limitations)
- [Try it out](#try-it-out)
- [Reinstall machine](#reinstall-machine)
- [Remove machine](#remove-machine)
- [Development of metal-api, metal-hammer and metal-core](#development-of-metal-api-metal-hammer-and-metal-core)
- [Free machine](#free-machine)
- [Flavors](#flavors)

<!-- /TOC -->

## Requirements

- Linux machine with hardware virtualization support
- [Vagrant](https://www.vagrantup.com/) == 2.2.14 with vagrant-libvirt plugin >= 0.3.0 (for running the switch and machine VMs)
- kvm as hypervisor for the VMs
- Ubuntu 20.04:

sudo apt update
sudo apt install -y qemu qemu-kvm libvirt-daemon bridge-utils virtinst libvirt-dev git curl build-essential wget

- kvm as hypervisor for the VMs (you can check through the `kvm-ok` command)
- [docker](https://www.docker.com/) >= 18.09 (for using kind and our deployment base image)
- [docker-compose](https://docs.docker.com/compose/) >= 1.25.4 (for ease of use and for parallelizing control plane and partition deployment)
- [kind](https://github.com/kubernetes-sigs/kind/releases) == v0.9.0 (for hosting the metal control plane on a kubernetes cluster v1.19.1)
- [ovmf](https://wiki.ubuntu.com/UEFI/OVMF) to have a uefi firmware for virtual machines
- the lab creates a virtual network 192.168.121.0/24 on your host machine, this hopefully does not overlap with other networks you have
- [containerlab](https://containerlab.srlinux.dev/install/)
- the lab creates a docker network on your host machine (`172.17.0.1`), this hopefully does not overlap with other networks you have
- (recommended) haveged to have enough random entropy (only needed if the PXE process does not work)

Here is some code that should help you setting up most of the requirements:

```bash
# Install kvm
sudo apt install -y git curl qemu qemu-kvm havaged

# Install Docker
curl -fsSL https://get.docker.com | sh
# if you want to be on the safe side, follow the original installation
# instructions at https://docs.docker.com/engine/install/ubuntu/

# Install vagrant and other stuff
wget https://releases.hashicorp.com/vagrant/2.2.14/vagrant_2.2.14_x86_64.deb
sudo apt-get install ./vagrant_2.2.14_x86_64.deb qemu-kvm virt-manager ovmf net-tools libvirt-dev haveged

# Ensure that your user is member of the group "libvirt" and "docker"
# Ensure that your user is member of the group "docker"
# you need to login again in order to make this change take effect
sudo usermod -G libvirt,docker -a ${USER}
sudo usermod -G docker -a ${USER}

# Install libvirt plugin for vagrant
vagrant plugin install vagrant-libvirt
# Install containerlab
bash -c "$(curl -sL https://get-clab.srlinux.dev)"

# Install kind (kubernetes in docker), for more details see https://kind.sigs.k8s.io/docs/user/quick-start/#installation
curl -Lo ./kind https://kind.sigs.k8s.io/dl/latest/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind
sudo curl -Lo /usr/local/bin/kind "https://kind.sigs.k8s.io/dl/latest/kind-linux-amd64"
sudo chmod +x /usr/local/bin/kind

# Install docker-compose, for more details see https://docs.docker.com/compose/install/
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo curl -Lo /usr/local/bin/docker-compose "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)"
sudo chmod +x /usr/local/bin/docker-compose
```

The following ports are getting used statically:
The following ports are used statically on your host machine:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still use all these ports?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes... but maybe we can get rid off it with the proposal I made here: #80 (comment)


| Port | Bind Address | Description |
|:----:|:------------ |:---------------------------------- |
Expand All @@ -82,15 +73,19 @@ The following ports are getting used statically:

## Try it out

Start the mini-lab with a kind cluster, a metal-api instance as well as some vagrant VMs with two leaf switches and two machine skeletons.

```bash
git clone https://github.com/metal-stack/mini-lab.git
cd mini-lab
```

Start the mini-lab with a kind cluster, a metal-api instance as well as two containers wrapping leaf switches and another container that hosts two user-allocatable machines:

```bash
make
# containerlab will ask you for root permissions (https://github.com/srl-labs/containerlab/issues/669)
```

Two machines in status `PXE booting` are visible with `metalctl machine ls`
After the deployment and waiting for a short amoung of time, two machines in status `PXE booting` become visible through `metalctl machine ls`:

```bash
docker-compose run metalctl machine ls
Expand All @@ -100,7 +95,7 @@ e0ab02d2-27cd-5a5e-8efc-080ba80cf258   PXE Booting 3s
2294c949-88f6-5390-8154-fa53d93a3313 PXE Booting 5s
```

Wait until the machines reach the waiting state
Wait until the machines reach the waiting state:

```bash
docker-compose run metalctl machine ls
Expand All @@ -110,42 +105,42 @@ e0ab02d2-27cd-5a5e-8efc-080ba80cf258   Waiting 8s
2294c949-88f6-5390-8154-fa53d93a3313   Waiting 8s v1-small-x86 vagrant
```

Create a machine/firewall with
Create a firewall and a machine with:

```bash
make firewall
make machine
```

__Alternatively__ you may want to issue the `metalctl` commands by your own:
__Alternatively__, you may want to issue the `metalctl` commands on your own:

```bash
docker-compose run metalctl network allocate \
--partition vagrant \
--partition mini-lab \
--project 00000000-0000-0000-0000-000000000000 \
--name vagrant
--name user-private-network

# Lookup the network ID and create a machine
# lookup the network ID and create a machine
docker-compose run metalctl machine create \
--description test \
--name machine \
--hostname machine \
--project 00000000-0000-0000-0000-000000000000 \
--partition vagrant \
--partition mini-lab \
--image ubuntu-20.04 \
--size v1-small-x86 \
--networks <network-ID>

# Create a firewall that is also connected to the virtual internet-vagrant-lab network
# create a firewall that is also connected to the virtual internet-mini-lab network
docker-compose run metalctl machine create \
--description fw \
--name fw \
--hostname fw \
--project 00000000-0000-0000-0000-000000000000 \
--partition vagrant \
--partition mini-lab \
--image firewall-ubuntu-2.0 \
--size v1-small-x86 \
--networks internet-vagrant-lab,$(privatenet)
--networks internet-mini-lab,$(privatenet)
```

See the installation process in action
Expand All @@ -170,12 +165,10 @@ e0ab02d2-27cd-5a5e-8efc-080ba80cf258   Phoned Home 2s 21s machin
Login with user name metal and the console password from

```bash
docker-compose run metalctl machine describe e0ab02d2-27cd-5a5e-8efc-080ba80cf258 | grep password

consolepassword: ...
docker-compose run metalctl machine consolepassword e0ab02d2-27cd-5a5e-8efc-080ba80cf258
```

If you want to access the firewall with SSH or have internet connectivity from the firewall and machine, you'll need to have a static route configured that points to the vagrant boxes of the leaf switches:
If you want to access the firewall with SSH or have internet connectivity from the firewall and machine, you'll need to have a static route configured that points to the leaf switches:

```bash
# Add the route to the network internet-vagrant-lab 100.255.254.0/24 via leaf01 and leaf02, whose IPs are dynamically allocated. Make sure there's no old route before execution.
Expand All @@ -185,7 +178,7 @@ make route
ssh metal@100.255.254.1
```

To remove the kind cluster and the vagrant boxes, run
To remove the kind cluster, the switches and machines, run:

```bash
make cleanup
Expand All @@ -201,9 +194,9 @@ docker-compose run metalctl machine reinstall \
e0ab02d2-27cd-5a5e-8efc-080ba80cf258
```

### Remove machine
### Free machine

Remove a machine with
Free a machine with

```bash
docker-compose run metalctl machine rm e0ab02d2-27cd-5a5e-8efc-080ba80cf258
Expand All @@ -222,17 +215,3 @@ In order to start specific flavor, you can define the flavor as follows:
export MINI_LAB_FLAVOR=cluster-api
make
```

## Development of metal-api, metal-hammer and metal-core

To simplify developing changes for the `metal-api`, `metal-hammer` and `metal-core`, it is possible to use development artifacts from within the mini-lab.
See the [dev instructions](DEV_INSTRUCTIONS.md) for more details.

FROM ubuntu:20.04

RUN apt update -y && \
DEBIAN_FRONTEND=noninteractive apt install -y virt-manager ovmf net-tools haveged \
qemu qemu-kvm bridge-utils virtinst libvirt-dev libvirt-daemon-system build-essential libvirt-clients

CMD ["/usr/sbin/init"]

2 changes: 1 addition & 1 deletion docs/overview.drawio
View file
Open in desktop

Large diffs are not rendered by default.

Binary file modified docs/overview.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.