Skip to content

Commit

Permalink
Implement Scrapli 'core' and platform migration [part 1] (#306)
Browse files Browse the repository at this point in the history
* Implement Scrapli 'core' and platform migration (#297)

* AOS-CX: Update CPU and MEM values for newer versions (#294)

Co-authored-by: Stefano Sasso <stefano.sasso@hpe.com>

* backdoor to reset VR or specific VMs (#285)

* backdoor to reset VR

* option to reset specific VMs

* give ocnos some time to boot in the login routine (#295)

* Add vrnetlab base image

* Add `cidfile` to gitignore

* Implement Scrapli

- Implement scrapli for telnet console and qemu monitor
- Add scrapli for core funcs (wait_write, read_until, expect)
- Add conditional use of scrapli via 'use_scrapli' var. Default is disabled
- Add colours to logging
- Log env vars
- Log if transparent mgmt intf is in use
- Log if scrapli is in use
- Log overlay image creation
- Log defined SMP and RAM

* cat8kv: Migrate to Scrapli

- Use Scrapli IOSXEDriver for config
- Update install VM var name to 'cat8kv' from 'csr'
- Fix installer class init so overlay image is only created once

* cat9kv: Migrate to Scrapli

- Remove license check
- Send bootstrap config via day0/CVAC config (mounted file to cdrom)
- Send startup config via Scrapli IOSXEDriver

* csr1kv: Migrate to Scrapli

- Use Scrapli IOSXEDriver for sending bootstrap and startup configs

* xrv: Migrate to Scrapli

- Use Scrapli IOSXRDriver to send bootstrap and startup configs

* xrv: Add convert-image target in Makefile

- Converts the qcow2 image into required vmdk format for vrnetlab via qemu-img.

* xrv9k: Migrate to Scrapli

- Use Scrapli IOSXRDriver for bootstrap and startup configs
- Change class names to 'XRv9k' instead of 'XRv'
- Explicitly wait for SDR baking to complete in install process
- Remove call home/LC check

* xrv: Remove env var printing

* n9kv: Migrate to Scrapli

- Use NXOSDriver for bootstrap and startup configs

* nxos: Migrate to Scrapli

- Use NXOSDriver for bootstrap and startup configs

* vios: Migrate to Scrapli

- Use IOSXEDriver for bootstrap and startup configs

* vrnetlab: Support  scrapli qemu monitor option for VM reset

* vrnetlab: Move logging colour config outside of class init method

* cat8kv: Fix logger warning (log.warning -> logger.warning)

* vrnetlab: Remove scrpali logging import

* Cisco devices: Add/tweak configuration saving:

- vios, csr, cat8kv, cat9kv -- add configuration saving
- XRv, XRv9k -- log configuration saving

* xrv, xrv9k: Return to root at end of bootstrap cfg

* vrnetlab: add bool formatter func

* sros: Migrate to Scrapli

- Use scrapli community 'nokia_sros' platform
- Remove wait_write clean_buffer override
- Check if tftpboot conifg exists *before* opening Scrapli connection
- Log command outputs with 'DEBUG_SCRAPLI' env var (defaults to false)

* Use kaelemc/scrapli_community in base image

* Disable eager mode for config saving on Cisco devices

* cat8kv: Migrate to CVAC configuration

* cat8kv: add log message and block while generating cfg ISO

* cat9kv: Migrate startup config to CVAC

* Remove erroneous Scrapli Community submodule

* sros: fix typos for classic CLI

* csr: Migrate to CVAC

* Switch back to scrapli/scrapli-community

* added uv lock/venv and env file for pylance resolve sequence (#303)

* added uv dep for scrapli

* use ruff formatting

* update base image with pinned scrapli community

* close sros driver connection to invoke on_exit commands (quit-config)

* added local deps

* use uv in the base image

* ruff formatting

* use single const for scrapli timeout

* Close the commandeered connection so the on close actions are run

* Connection error log type from info->error

* extracting image edit

* persist bof and config after bootstrap config is applied and close sros con regardless if config was provided or not

* Configure scrapli variant if startup config is classic

If the startup-configuration provided is classic then the default configuration engine will be set to classic mode.

In this case the scrapli device variant should also be set to classic so the scrapli magic can do it's thing with the correct prompt matching.

* Use a global var to determine when to send classic configs

As classic startup configurations are now supported for MD-CLI defaulting versions, the classic CLI will mean the default config engine is classic on node boot. In this commit all logic that determined when to send/not send config for classic versions is now replaced with a single 'classic_cfg' global variable.

Most of the logic across the code had repeated statements checking if the version was <= 22 or magc. 'classic_cfg' is set to True in this case. Else it is False.

* Use explicit `quit-config` and move persistBofAndConfig back to end of bootstrap.

* Don't enforce MD-CLI on versions older than 19.x

---------

Co-authored-by: Stefano Sasso <852093+ssasso@users.noreply.github.com>
Co-authored-by: Stefano Sasso <stefano.sasso@hpe.com>
Co-authored-by: João Machado <63718541+jcpvdm@users.noreply.github.com>
Co-authored-by: Roman Dodin <dodin.roman@gmail.com>

* Resolve merge conflicts for #306 (#311)

* AOS-CX: Update CPU and MEM values for newer versions (#294)

Co-authored-by: Stefano Sasso <stefano.sasso@hpe.com>

* backdoor to reset VR or specific VMs (#285)

* backdoor to reset VR

* option to reset specific VMs

* give ocnos some time to boot in the login routine (#295)

* Fix mgmt route VRF name on cat9kv (#296)

* fix vmx dockerfile

* vJunos-EVO: remove error in init file that prevents loading it (#301)

* Fix routeros  (#302)

* update ubuntu version for routeros

* change base image

---------

Co-authored-by: Félix Gaudin <felix.gaudin@inmanta.com>

* Fix for OcNOS 6.5.2-101 (#300)

* Add Cisco vIOS L2 support (#299)

* update veos for bookworm

* added uv lock/venv and env file for pylance resolve sequence (#303)

* Update Dockerfile to install qemu-utils (#304)

* AOS-CX: Update base image and cpu (#305)

* Update vqfx Makefile (#289)

* Update Makefile

In the last line is calling the variable $(VR_NAME) but it wasn't defined before so i update the variable NAME to VR_NAME

* try vqfx with the common workflow

* Revert "try vqfx with the common workflow"

This reverts commit c1f8866.

---------

Co-authored-by: Roman Dodin <dodin.roman@gmail.com>

* lowercase things

* update vqfx base image

* bump num nics (#308)

---------

Co-authored-by: Stefano Sasso <852093+ssasso@users.noreply.github.com>
Co-authored-by: Stefano Sasso <stefano.sasso@hpe.com>
Co-authored-by: João Machado <63718541+jcpvdm@users.noreply.github.com>
Co-authored-by: Roman Dodin <dodin.roman@gmail.com>
Co-authored-by: FelixGaudin <44848675+FelixGaudin@users.noreply.github.com>
Co-authored-by: Félix Gaudin <felix.gaudin@inmanta.com>
Co-authored-by: Xinyu Wei <xinyuwe@cisco.com>
Co-authored-by: Thomas Balzer <65245605+tjbalzer@users.noreply.github.com>
Co-authored-by: tdebruyn <5540603+tdebruyn@users.noreply.github.com>
Co-authored-by: Voyageur <167977253+Zegroj@users.noreply.github.com>

* Merge conflicts #2 (#313)

* AOS-CX: Update CPU and MEM values for newer versions (#294)

Co-authored-by: Stefano Sasso <stefano.sasso@hpe.com>

* backdoor to reset VR or specific VMs (#285)

* backdoor to reset VR

* option to reset specific VMs

* give ocnos some time to boot in the login routine (#295)

* Fix mgmt route VRF name on cat9kv (#296)

* fix vmx dockerfile

* vJunos-EVO: remove error in init file that prevents loading it (#301)

* Fix routeros  (#302)

* update ubuntu version for routeros

* change base image

---------

Co-authored-by: Félix Gaudin <felix.gaudin@inmanta.com>

* Fix for OcNOS 6.5.2-101 (#300)

* Add Cisco vIOS L2 support (#299)

* update veos for bookworm

* added uv lock/venv and env file for pylance resolve sequence (#303)

* Update Dockerfile to install qemu-utils (#304)

* AOS-CX: Update base image and cpu (#305)

* Update vqfx Makefile (#289)

* Update Makefile

In the last line is calling the variable $(VR_NAME) but it wasn't defined before so i update the variable NAME to VR_NAME

* try vqfx with the common workflow

* Revert "try vqfx with the common workflow"

This reverts commit c1f8866.

---------

Co-authored-by: Roman Dodin <dodin.roman@gmail.com>

* lowercase things

* update vqfx base image

* bump num nics (#308)

* DHCP and static MAC support for management interface  (#310)

* common/vrnetlab.py: added DHCP support for management interface

If CLAB_MGMT_DHCP environment variable is set, we assume that a DHCP client
inside of the VM will take care about setting the management IP and gateway.

From the implementation point of view the IPv4 and IPv6 addresses and gateways
are simply set to 'dhcp', and the platform specific `launch.py` script needs
to check it to decide if the interface needs to be configured manually or to
to enable dhcp (if not enabled by default).

The initial implementation has been tested with Broadcom Enterprise SONiC 4.4.0
using the "dell_sonic" kind (which actually is a rebranded Broadcom SONiC).
The primary focus is the ability to test the Zero Touch Provisioning feature of
SONiC but could also be used for testing PXE boot scenarios of other systems
using containerlab.

Note:
This feature is only working when CLAB_MGMT_PASSTHROUGH is true.

The feature needs to be defined explicitely within the node configuration
like this:
--------------------------------
env:
  CLAB_MGMT_PASSTHROUGH: "true"
  CLAB_MGMT_DHCP: "true"
--------------------------------

* dell_sonic: changed to check if dhcp is enabled for the management interface

Manually set IPv4/IPv6 addresses and gateways only if dhcp is not enabled.

* common/vrnetlab.py: added get_mgmt_mac method to VM class

get_mgmt_mac() checks for the environment variable 'CLAB_MGMT_MAC' which
can be used to provide a static MAC address for the management interface.
If there is no static MAC address set the former behavior is preserved
by calling gen_mac(0) generating a MAC address.

* vrnetlab: Refactor mgmt MAC generation, handle exceptions for CSR1000v/ROS/SROS

* compresss env var based var assignment

---------

Co-authored-by: vista <vista@birb.network>
Co-authored-by: Roman Dodin <dodin.roman@gmail.com>

---------

Co-authored-by: Stefano Sasso <852093+ssasso@users.noreply.github.com>
Co-authored-by: Stefano Sasso <stefano.sasso@hpe.com>
Co-authored-by: João Machado <63718541+jcpvdm@users.noreply.github.com>
Co-authored-by: Roman Dodin <dodin.roman@gmail.com>
Co-authored-by: FelixGaudin <44848675+FelixGaudin@users.noreply.github.com>
Co-authored-by: Félix Gaudin <felix.gaudin@inmanta.com>
Co-authored-by: Xinyu Wei <xinyuwe@cisco.com>
Co-authored-by: Thomas Balzer <65245605+tjbalzer@users.noreply.github.com>
Co-authored-by: tdebruyn <5540603+tdebruyn@users.noreply.github.com>
Co-authored-by: Voyageur <167977253+Zegroj@users.noreply.github.com>
Co-authored-by: Christian Wiese <morfoh@opensde.org>
Co-authored-by: vista <vista@birb.network>

* remove doubled func

---------

Co-authored-by: Kaelem Chandra <kc@kaelem.net>
Co-authored-by: Stefano Sasso <852093+ssasso@users.noreply.github.com>
Co-authored-by: Stefano Sasso <stefano.sasso@hpe.com>
Co-authored-by: João Machado <63718541+jcpvdm@users.noreply.github.com>
Co-authored-by: FelixGaudin <44848675+FelixGaudin@users.noreply.github.com>
Co-authored-by: Félix Gaudin <felix.gaudin@inmanta.com>
Co-authored-by: Xinyu Wei <xinyuwe@cisco.com>
Co-authored-by: Thomas Balzer <65245605+tjbalzer@users.noreply.github.com>
Co-authored-by: tdebruyn <5540603+tdebruyn@users.noreply.github.com>
Co-authored-by: Voyageur <167977253+Zegroj@users.noreply.github.com>
Co-authored-by: Christian Wiese <morfoh@opensde.org>
Co-authored-by: vista <vista@birb.network>
  • Loading branch information
13 people authored Jan 25, 2025
1 parent ad44c11 commit d41860b
Show file tree
Hide file tree
Showing 27 changed files with 1,273 additions and 1,147 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ cisco*.bin
*.xz
*.vmdk
*.iso
*cidfile

.DS_Store
*/.DS_Store
Expand Down
13 changes: 13 additions & 0 deletions build-base-image.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
# this script builds the vrnetlab base container image
# that is used in the dockerfiles of the NOS images

set -e

if [ -z "$1" ]; then
echo "Usage: $0 <version>"
exit 1
fi

sudo docker build -t ghcr.io/srl-labs/vrnetlab-base:$1 \
-f vrnetlab-base.dockerfile .
18 changes: 1 addition & 17 deletions c8000v/docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,20 +1,4 @@
FROM public.ecr.aws/docker/library/debian:bookworm-slim

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update -qy \
&& apt-get install -y \
bridge-utils \
iproute2 \
socat \
qemu-kvm \
tcpdump \
inetutils-ping \
ssh \
telnet \
procps \
genisoimage \
&& rm -rf /var/lib/apt/lists/*
FROM ghcr.io/srl-labs/vrnetlab-base:0.1.0

ARG VERSION
ENV VERSION=${VERSION}
Expand Down
256 changes: 124 additions & 132 deletions c8000v/docker/launch.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,38 +52,116 @@ def __init__(self, hostname, username, password, conn_mode, install_mode=False):
logger.info("License found")
self.license = True

super().__init__(username, password, disk_image=disk_image, ram=4096)
super().__init__(
username, password, disk_image=disk_image, ram=4096, use_scrapli=True
)
self.install_mode = install_mode
self.hostname = hostname
self.conn_mode = conn_mode
self.num_nics = 9
self.nic_type = "virtio-net-pci"
self.image_name = "config.iso"

if self.install_mode:
logger.trace("install mode")
self.image_name = "config.iso"
self.create_boot_image()

self.qemu_args.extend(["-cdrom", "/" + self.image_name])

def create_boot_image(self):
"""Creates a iso image with a bootstrap configuration"""

with open("/iosxe_config.txt", "w") as cfg_file:
if self.license:
cfg_file.write("do clock set 13:33:37 1 Jan 2010\r\n")
cfg_file.write("interface GigabitEthernet1\r\n")
cfg_file.write("ip address 10.0.0.15 255.255.255.0\r\n")
cfg_file.write("no shut\r\n")
cfg_file.write("exit\r\n")
cfg_file.write("license accept end user agreement\r\n")
cfg_file.write("yes\r\n")
cfg_file.write("do license install tftp://10.0.0.2/license.lic\r\n\r\n")
cfg_file.write("license boot level network-premier addon dna-premier\r\n")
cfg_file.write("platform console serial\r\n\r\n")
cfg_file.write("do clear platform software vnic-if nvtable\r\n")
cfg_file.write("do wr\r\n")
cfg_file.write("do reload\r\n")
self.logger.debug("Install mode")
self.create_config_image(self.gen_install_config())
else:
cfg = self.gen_bootstrap_config()
if os.path.exists(STARTUP_CONFIG_FILE):
self.logger.info("Startup configuration file found")
with open(STARTUP_CONFIG_FILE, "r") as startup_config:
cfg += startup_config.read()
else:
self.logger.warning("User provided startup configuration is not found.")
self.create_config_image(cfg)

self.qemu_args.extend(["-cdrom", "/" + self.image_name])

def gen_install_config(self) -> str:
"""
Returns the configuration to load in install mode
"""

config = ""

if self.license:
config += """do clock set 13:33:37 1 Jan 2010
interface GigabitEthernet1
ip address 10.0.0.15 255.255.255.0
no shut
exit
license accept end user agreement
yes
do license install tftp://10.0.0.2/license.lic
"""

config += """
license boot level network-premier addon dna-premier
platform console serial
do clear platform software vnic-if nvtable
do wr
do reload
"""

return config

def gen_bootstrap_config(self) -> str:
"""
Returns the system bootstrap configuration
"""

v4_mgmt_address = vrnetlab.cidr_to_ddn(self.mgmt_address_ipv4)

return f"""hostname {self.hostname}
username {self.username} privilege 15 password {self.password}
ip domain name example.com
!
crypto key generate rsa modulus 2048
!
line con 0
logging synchronous
!
line vty 0 4
logging synchronous
login local
transport input all
!
ipv6 unicast-routing
!
vrf definition clab-mgmt
description Containerlab management VRF (DO NOT DELETE)
address-family ipv4
exit
address-family ipv6
exit
exit
!
ip route vrf clab-mgmt 0.0.0.0 0.0.0.0 {self.mgmt_gw_ipv4}
ipv6 route vrf clab-mgmt ::/0 {self.mgmt_gw_ipv6}
!
interface GigabitEthernet 1
description Containerlab management interface
vrf forwarding clab-mgmt
ip address {v4_mgmt_address[0]} {v4_mgmt_address[1]}
ipv6 address {self.mgmt_address_ipv6}
no shut
exit
!
restconf
netconf-yang
netconf max-sessions 16
netconf detailed-error
!
ip ssh server algorithm mac hmac-sha2-512
ip ssh maxstartups 128
!
"""

def create_config_image(self, config):
"""Creates a iso image with a installation configuration"""

with open("/iosxe_config.txt", "w") as cfg:
cfg.write(config)

genisoimage_args = [
"genisoimage",
Expand All @@ -93,7 +171,8 @@ def create_boot_image(self):
"/iosxe_config.txt",
]

subprocess.Popen(genisoimage_args)
self.logger.debug("Generating boot ISO")
subprocess.Popen(genisoimage_args).wait()

def bootstrap_spin(self):
"""This function should be called periodically to do work."""
Expand All @@ -104,127 +183,40 @@ def bootstrap_spin(self):
self.start()
return

(ridx, match, res) = self.tn.expect(
[b"Press RETURN to get started!", b"IOSXEBOOT-4-FACTORY_RESET"], 1
(ridx, match, res) = self.con_expect(
[b"CVAC-4-CONFIG_DONE", b"IOSXEBOOT-4-FACTORY_RESET"]
)
if match: # got a match!
if ridx == 0: # login
self.logger.debug("matched, Press RETURN to get started.")
if self.install_mode:
self.logger.debug("Now we wait for the device to reload")
else:
self.wait_write("", wait=None)

# run main config!
self.bootstrap_config()
# add startup config if present
self.startup_config()
# close telnet connection
self.tn.close()
# startup time?
startup_time = datetime.datetime.now() - self.start_time
self.logger.info("Startup complete in: %s", startup_time)
# mark as running
self.running = True
return
if ridx == 0 and not self.install_mode: # configuration applied
self.logger.info("CVAC Configuration has been applied.")
# close telnet connection
self.scrapli_tn.close()
# startup time?
startup_time = datetime.datetime.now() - self.start_time
self.logger.info("Startup complete in: %s", startup_time)
# mark as running
self.running = True
return
elif ridx == 1: # IOSXEBOOT-4-FACTORY_RESET
if self.install_mode:
install_time = datetime.datetime.now() - self.start_time
self.logger.info("Install complete in: %s", install_time)
self.running = True
return
else:
self.log.warning("Unexpected reload while running")
self.logger.warning("Unexpected reload while running")

# no match, if we saw some output from the router it's probably
# booting, so let's give it some more time
if res != b"":
self.logger.trace("OUTPUT: %s", res.decode())
self.write_to_stdout(res)
# reset spins if we saw some output
self.spins = 0

self.spins += 1

return

def bootstrap_config(self):
"""Do the actual bootstrap config"""
self.logger.info("applying bootstrap configuration")

v4_mgmt_address = vrnetlab.cidr_to_ddn(self.mgmt_address_ipv4)

self.wait_write("", None)
self.wait_write("enable", wait=">")
self.wait_write("configure terminal", wait=">")

self.wait_write(f"hostname {self.hostname}")
self.wait_write(
"username %s privilege 15 password %s" % (self.username, self.password)
)
if int(self.version.split(".")[0]) >= 16:
self.wait_write("ip domain name example.com")
else:
self.wait_write("ip domain-name example.com")
self.wait_write("crypto key generate rsa modulus 2048")

self.wait_write("ipv6 unicast-routing")

self.wait_write("vrf definition clab-mgmt")
self.wait_write("description Containerlab management VRF (DO NOT DELETE)")
self.wait_write("address-family ipv4")
self.wait_write("exit")
self.wait_write("address-family ipv6")
self.wait_write("exit")
self.wait_write("exit")

self.wait_write(f"ip route vrf clab-mgmt 0.0.0.0 0.0.0.0 {self.mgmt_gw_ipv4}")
self.wait_write(f"ipv6 route vrf clab-mgmt ::/0 {self.mgmt_gw_ipv6}")

self.wait_write("interface GigabitEthernet1")
self.wait_write("vrf forwarding clab-mgmt")
self.wait_write(f"ip address {v4_mgmt_address[0]} {v4_mgmt_address[1]}")
self.wait_write(f"ipv6 address {self.mgmt_address_ipv6}")
self.wait_write("no shut")
self.wait_write("exit")
self.wait_write("restconf")
self.wait_write("netconf-yang")
self.wait_write("netconf max-sessions 16")
# I did not find any documentation about this, but is seems like a good idea!?
self.wait_write("netconf detailed-error")
self.wait_write("ip ssh server algorithm mac hmac-sha2-512")
self.wait_write("ip ssh maxstartups 128")

self.wait_write("line vty 0 4")
self.wait_write("login local")
self.wait_write("transport input all")
self.wait_write("end")
self.wait_write("copy running-config startup-config")
self.wait_write("\r", "Destination")

def startup_config(self):
"""Load additional config provided by user."""

if not os.path.exists(STARTUP_CONFIG_FILE):
self.logger.trace(f"Startup config file {STARTUP_CONFIG_FILE} is not found")
return

self.logger.trace(f"Startup config file {STARTUP_CONFIG_FILE} exists")
with open(STARTUP_CONFIG_FILE) as file:
config_lines = file.readlines()
config_lines = [line.rstrip() for line in config_lines]
self.logger.trace(f"Parsed startup config file {STARTUP_CONFIG_FILE}")

self.logger.info(f"Writing lines from {STARTUP_CONFIG_FILE}")

self.wait_write("configure terminal")
# Apply lines from file
for line in config_lines:
self.wait_write(line)
# End and Save
self.wait_write("end")
self.wait_write("copy running-config startup-config")
self.wait_write("\r", "Destination")


class C8000v(vrnetlab.VR):
def __init__(self, hostname, username, password, conn_mode):
Expand All @@ -240,17 +232,17 @@ class C8000v_installer(C8000v):
"""

def __init__(self, hostname, username, password, conn_mode):
super(C8000v_installer, self).__init__(hostname, username, password, conn_mode)
super(C8000v, self).__init__(username, password)
self.vms = [
C8000v_vm(hostname, username, password, conn_mode, install_mode=True)
]

def install(self):
self.logger.info("Installing C8000v")
csr = self.vms[0]
while not csr.running:
csr.work()
csr.stop()
cat8kv = self.vms[0]
while not cat8kv.running:
cat8kv.work()
cat8kv.stop()
self.logger.info("Installation complete")


Expand Down
20 changes: 1 addition & 19 deletions cat9kv/docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,22 +1,4 @@
FROM public.ecr.aws/docker/library/debian:bookworm-slim

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update -qy \
&& apt-get install -y --no-install-recommends \
bridge-utils \
iproute2 \
socat \
qemu-kvm \
qemu-utils \
python3 \
tcpdump \
inetutils-ping \
ssh \
telnet \
procps \
genisoimage \
&& rm -rf /var/lib/apt/lists/*
FROM ghcr.io/srl-labs/vrnetlab-base:0.1.0

ARG VERSION
ENV VERSION=${VERSION}
Expand Down
Loading

0 comments on commit d41860b

Please sign in to comment.