Clab Deploy --Restore-All Failing to start Continers

[rrw357719]

Attempting to start containers from snapshots using the clab deploy --restore-all command.
Receiving the following error message in the logs for a vmanage container:

└──> docker logs -f 745299fae240
2026-02-23 11:46:43,304: vrnetlab INFO Restoring from /snapshot.tar
2026-02-23 11:46:43,304: vrnetlab INFO Extracting snapshot: /snapshot.tar
2026-02-23 11:48:00,235: vrnetlab INFO Snapshot extracted to: /snapshot-data
2026-02-23 11:48:00,243: vrnetlab INFO Restoring from snapshot
2026-02-23 11:48:00,243: vrnetlab INFO Snapshot disk: /snapshot-data/vm0/disk.qcow2
2026-02-23 11:48:00,243: vrnetlab INFO Snapshot state: /snapshot-data/vm0/state.img
2026-02-23 11:48:00,243: vrnetlab INFO Loaded snapshot metadata: {'mac_addresses': ['0c:00:6f:3a:db:00', 'aa:c1:ab:b0:39:03'], 'num_nics': 5, 'provision_pci_bus': True, 'nics_per_pci_bus': 26, 'secondary_disks': ['disk_100G.qcow2']}
2026-02-23 11:48:00,243: vrnetlab INFO Restoring secondary disk: disk_100G.qcow2
2026-02-23 11:48:07,774: vrnetlab DEBUG Creating overlay disk image: /snapshot-data/vm0/disk-overlay.qcow2
2026-02-23 11:48:19,778: launch INFO Found zcloud.xml configuration at /config/zcloud.xml
wrote /cloud_init.iso with filesystem=iso9660 and diskformat=raw
2026-02-23 11:48:19,803: vrnetlab DEBUG Starting vrnetlab Sdwan_component
2026-02-23 11:48:19,803: vrnetlab DEBUG VMs: [<main.Sdwan_component_vm object at 0x73643bf10550>]
2026-02-23 11:48:19,803: vrnetlab DEBUG VM not started; starting!
2026-02-23 11:48:19,803: vrnetlab INFO ----------------START ENVIRONMENT VARIABLES-----------------
2026-02-23 11:48:19,803: vrnetlab INFO CLAB_INTFS: 1
2026-02-23 11:48:19,803: vrnetlab INFO CLAB_LABEL_CLAB_NODE_GROUP:
2026-02-23 11:48:19,803: vrnetlab INFO CLAB_LABEL_CLAB_NODE_KIND: cisco_sdwan
2026-02-23 11:48:19,803: vrnetlab INFO CLAB_LABEL_CLAB_NODE_LAB_DIR: /home/rdub/rlab/labs/cisco/sdwan/clab-sd1/man
2026-02-23 11:48:19,803: vrnetlab INFO CLAB_LABEL_CLAB_NODE_LONGNAME: man
2026-02-23 11:48:19,803: vrnetlab INFO CLAB_LABEL_CLAB_NODE_NAME: man
2026-02-23 11:48:19,803: vrnetlab INFO CLAB_LABEL_CLAB_NODE_TYPE: manager
2026-02-23 11:48:19,803: vrnetlab INFO CLAB_LABEL_CLAB_OWNER: rdub
2026-02-23 11:48:19,803: vrnetlab INFO CLAB_LABEL_CLAB_TOPO_FILE: /home/rdub/rlab/labs/cisco/sdwan/sd1.clab.yml
2026-02-23 11:48:19,803: vrnetlab INFO CLAB_LABEL_CONTAINERLAB: sd1
2026-02-23 11:48:19,803: vrnetlab INFO CLAB_MGMT_PASSTHROUGH: true
2026-02-23 11:48:19,803: vrnetlab INFO CONNECTION_MODE: tc
2026-02-23 11:48:19,803: vrnetlab INFO DOCKER_NET_V4_ADDR: 10.255.254.0/24
2026-02-23 11:48:19,803: vrnetlab INFO DOCKER_NET_V6_ADDR:
2026-02-23 11:48:19,803: vrnetlab INFO HOME: /root
2026-02-23 11:48:19,803: vrnetlab INFO HOSTNAME: man
2026-02-23 11:48:19,803: vrnetlab INFO LC_CTYPE: C.UTF-8
2026-02-23 11:48:19,803: vrnetlab INFO NO_PROXY: localhost,127.0.0.1,::1,.local,10.255.254.0/24,10.255.254.11,10.255.254.12,10.255.254.13,10.255.254.14,10.255.254.15,10.255.254.16,10.255.254.17,10.255.254.18,10.255.254.19,10.255.254.20,10.255.254.21,10.255.254.22,10.255.254.23,10.255.254.24,10.255.254.25,10.255.254.26,10.255.254.27,b1r1,b1r2,b2r1,b3r1,b4r1,br1-sw,c-sw1,ctrl,gw,inet,man,mpls,r1l1,r2l1,r3l1,r4l1,val
2026-02-23 11:48:19,803: vrnetlab INFO PASSWORD: admin
2026-02-23 11:48:19,804: vrnetlab INFO PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2026-02-23 11:48:19,804: vrnetlab INFO RESTORE_SNAPSHOT: 1
2026-02-23 11:48:19,804: vrnetlab INFO TERM: xterm
2026-02-23 11:48:19,804: vrnetlab INFO USERNAME: admin
2026-02-23 11:48:19,804: vrnetlab INFO no_proxy: localhost,127.0.0.1,::1,.local,10.255.254.0/24,10.255.254.11,10.255.254.12,10.255.254.13,10.255.254.14,10.255.254.15,10.255.254.16,10.255.254.17,10.255.254.18,10.255.254.19,10.255.254.20,10.255.254.21,10.255.254.22,10.255.254.23,10.255.254.24,10.255.254.25,10.255.254.26,10.255.254.27,b1r1,b1r2,b2r1,b3r1,b4r1,br1-sw,c-sw1,ctrl,gw,inet,man,mpls,r1l1,r2l1,r3l1,r4l1,val
2026-02-23 11:48:19,804: vrnetlab INFO -----------------END ENVIRONMENT VARIABLES------------------
2026-02-23 11:48:19,804: vrnetlab INFO Launching Sdwan_component_vm with 8 SMP/VCPU and 32768 M of RAM
2026-02-23 11:48:19,804: vrnetlab INFO Scrapli: Disabled
2026-02-23 11:48:19,804: vrnetlab INFO Transparent mgmt interface: Enabled
2026-02-23 11:48:19,804: vrnetlab INFO Using saved management MAC: 0c:00:6f:3a:db:00
2026-02-23 11:48:19,806: vrnetlab DEBUG number of provisioned data plane interfaces is 1
2026-02-23 11:48:19,806: vrnetlab DEBUG waiting for provisioned interfaces to appear...
2026-02-23 11:48:19,806: vrnetlab DEBUG highest allocated interface id determined to be: 1...
2026-02-23 11:48:19,806: vrnetlab DEBUG interfaces provisioned, continuing...
2026-02-23 11:48:19,806: vrnetlab INFO Using saved MAC from snapshot for eth1: aa:c1:ab:b0:39:03
2026-02-23 11:48:19,806: vrnetlab DEBUG qemu cmd: qemu-system-x86_64 -enable-kvm -display none -machine pc -chardev socket,id=monitor0,host=::,port=4000,server=on,wait=off -monitor chardev:monitor0 -chardev socket,id=serial0,host=::,port=5000,server=on,wait=off,telnet=on -serial chardev:serial0 -m 32768 -cpu host -smp 8 -drive if=ide,file=/snapshot-data/vm0/disk-overlay.qcow2 -incoming exec:cat\ <\ /snapshot-data/vm0/state.img -cdrom /cloud_init.iso -drive if=virtio,file=disk_100G.qcow2 -device pci-bridge,chassis_nr=1,id=pci.1 -device virtio-net-pci,netdev=p00,mac=0c:00:6f:3a:db:00 -netdev tap,id=p00,ifname=tap0,script=/etc/tc-tap-mgmt-ifup,downscript=no -device virtio-net-pci,netdev=p01,mac=aa:c1:ab:b0:39:03,bus=pci.1,addr=0x2 -netdev tap,id=p01,ifname=tap1,script=/etc/tc-tap-ifup,downscript=no
2026-02-23 11:48:21,813: vrnetlab INFO Resuming VM after snapshot restore
2026-02-23 11:48:21,813: vrnetlab INFO Closed telnet serial connection
Traceback (most recent call last):
File "/launch.py", line 329, in
vr.start()
File "/vrnetlab.py", line 1336, in start
vm.work()
File "/vrnetlab.py", line 1164, in work
self.bootstrap_spin()
File "/launch.py", line 235, in bootstrap_spin
(ridx, match, res) = self.tn.expect([b"System Ready", b"CDB boot error"], 1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/telnetlib.py", line 620, in expect
selector.register(self, selectors.EVENT_READ)
File "/usr/lib/python3.11/selectors.py", line 352, in register
key = super().register(fileobj, events, data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/selectors.py", line 238, in register
key = SelectorKey(fileobj, self._fileobj_lookup(fileobj), events, data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/selectors.py", line 225, in _fileobj_lookup
return _fileobj_to_fd(fileobj)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/selectors.py", line 39, in _fileobj_to_fd
raise ValueError("Invalid file object: "
ValueError: Invalid file object: <telnetlib.Telnet object at 0x73643bbee510>