HASS OS OVA on libvirt+kvm: Reboot after HASS OS update causes crash

[philpem]

Describe the issue you are experiencing

I installed the HASS OS 14.2 update this evening, and after the update concluded, I found that my virtual machine was crashed/hung with KVM reporting 100% CPU usage.

The last messages on the virtual console were:

[  OK  ] Finished System Reboot.
[  OK  ] Reached target System Reboot.
reboot: Restarting system

I found this Community post which reports a similar (possibly the same) bug going back to October 2023: https://community.home-assistant.io/t/ha-hangs-and-no-response-after-os-updates-on-kvm/632083/4

I can't put a number on how long it's been happening for me, except that it's been over a year.

Oddly if I go in to the Home Assistant web interface and reboot from there (System -> power menu at top right -> more -> Reboot) then the system doesn't crash. It only seems to happen after a HASS OS update.

What operating system image do you use?

ova (for Virtual Machines)

What version of Home Assistant Operating System is installed?

14.2

Did the problem occur after upgrading the Operating System?

Yes

Hardware details

QEMU/KVM on Libvirt.

VM config:

<domain type="kvm">
  <name>home-assistant</name>
  <uuid>REDACTED</uuid>
  <title>Home Assistant</title>
  <memory unit="KiB">2621440</memory>
  <currentMemory unit="KiB">2621440</currentMemory>
  <vcpu placement="static">2</vcpu>
  <os>
    <type arch="x86_64" machine="pc-i440fx-jammy">hvm</type>
    <loader readonly="yes" type="rom">/usr/share/OVMF/OVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/home-assistant_VARS.fd</nvram>
    <boot dev="hd"/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <vmport state="off"/>
  </features>
  <cpu mode="host-passthrough" check="none" migratable="on"/>
  <clock offset="utc">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/mnt/zfs/vmdisks/hassos_ova-5.12.qcow2"/>
      <target dev="hda" bus="ide"/>
      <address type="drive" controller="0" bus="0" target="0" unit="0"/>
    </disk>
    <controller type="usb" index="0" model="ich9-ehci1">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x7"/>
    </controller>
    <controller type="usb" index="0" model="ich9-uhci1">
      <master startport="0"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0" multifunction="on"/>
    </controller>
    <controller type="usb" index="0" model="ich9-uhci2">
      <master startport="2"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x1"/>
    </controller>
    <controller type="usb" index="0" model="ich9-uhci3">
      <master startport="4"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x2"/>
    </controller>
    <controller type="pci" index="0" model="pci-root"/>
    <controller type="ide" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x1"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x06" function="0x0"/>
    </controller>
    <interface type="bridge">
      <mac address="REDACTED"/>
      <source bridge="bridge-lan"/>
      <model type="e1000"/>
      <link state="up"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0"/>
    </interface>
    <interface type="bridge">
      <mac address="REDACTED"/>
      <source bridge="bridge-iot"/>
      <model type="e1000"/>
      <link state="up"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
    </interface>
    <serial type="pty">
      <target type="isa-serial" port="0">
        <model name="isa-serial"/>
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
    <channel type="spicevmc">
      <target type="virtio" name="com.redhat.spice.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="1"/>
    </channel>
    <input type="tablet" bus="usb">
      <address type="usb" bus="0" port="1"/>
    </input>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="ps2"/>
    <graphics type="spice" autoport="yes">
      <listen type="address"/>
    </graphics>
    <sound model="ich6">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x04" function="0x0"/>
    </sound>
    <audio id="1" type="spice"/>
    <video>
      <model type="bochs" vram="16384" heads="1" primary="yes"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0"/>
    </video>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="2"/>
    </redirdev>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="3"/>
    </redirdev>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
    </memballoon>
  </devices>
</domain>

Steps to reproduce the issue

1. Upgrade from one version of HASS OS to the succeeding version.
2. Monitor the virtual machine console while HASS OS restarts
3. Observe HASS OS crashing instead of rebooting the system

Anything in the Supervisor logs that might be useful for us?

Nothing that appears relevant

Anything in the Host logs that might be useful for us?

Nothing that appears relevant

System information

System Information

version	core-2025.1.4
installation_type	Home Assistant OS
dev	false
hassio	true
docker	true
user	root
virtualenv	false
python_version	3.13.1
os_name	Linux
os_version	6.6.73-haos
arch	x86_64
timezone	Europe/London
config_dir	/config

Home Assistant Community Store

GitHub API	ok
GitHub Content	ok
GitHub Web	ok
HACS Data	ok
GitHub API Calls Remaining	5000
Installed Version	2.0.5
Stage	running
Available Repositories	1500
Downloaded Repositories	6

Home Assistant Cloud

logged_in	false
can_reach_cert_server	ok
can_reach_cloud_auth	ok
can_reach_cloud	ok

Home Assistant Supervisor

host_os	Home Assistant OS 14.2
update_channel	stable
supervisor_version	supervisor-2024.12.3
agent_version	1.6.0
docker_version	27.2.0
disk_total	30.8 GB
disk_used	11.1 GB
healthy	true
supported	true
host_connectivity	true
supervisor_connectivity	true
ntp_synchronized	true
virtualization	kvm
board	ova
supervisor_api	ok
version_api	ok
installed_addons	Terminal & SSH (9.16.0), Mosquitto broker (6.5.0), File editor (5.8.0), Network UPS Tools (0.14.0), SQLite Web (4.3.0), ESPHome Device Builder (2024.12.4), eBUSd (24.1.1), Advanced SSH & Web Terminal (20.0.0)

Dashboards

dashboards	5
resources	3
views	21
mode	storage

Recorder

oldest_recorder_run	7 January 2025 at 19:20
current_recorder_run	29 January 2025 at 23:41
estimated_db_size	397.23 MiB
database_engine	sqlite
database_version	3.47.1

Spotify

api_endpoint_reachable	ok

Additional information

No response

[sairon]

Can you provide more details about the host OS, please? Can you try if performing systemctl reboot and ha os boot-slot other (you'll have to do it twice, or change back to the correct slot from GRUB menu on the next boot) from the command line triggers the behavior too?

[philpem]

Host OS is Linux, specifically Ubuntu 22.04 LTS with current updates.

Inside the VM, "os boot-slot other" seems to take a while to run, then pops up an error:

Post "http://supervisor/os/boot-slot": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Then returns to the "ha>" prompt briefly before reporting that the CLI has been terminated.
Shortly after, it shuts down and restarts the VM. When HA comes up, it prompts that there's a HASS OS update (to 14.2 ) available.

Doing "os boot-slot other" a second time does the same thing - except HA starts up with the 14.2 image.

"systemctl reboot" from the HA prompt didn't work ("unknown command" error), so I tried "host reboot" - which worked fine. Took a while to reboot (with a similar timeout error as above) but it didn't crash.

The trigger seems to be applying a HASS OS update, then letting it reboot by itself.