Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

need a way to telll qemu to use discard="unmap" so that FITRIM() will pass through, to reduce image sizes #301

Open
cmurf opened this issue Jul 22, 2022 · 0 comments

Comments

@cmurf
Copy link

cmurf commented Jul 22, 2022

Summary: The VM's need to have virtioblk set to use discard=unmap in order for guest issuance of fstrim to pass through to the host backing file. This has the effect of creating a sparse backing file, since the holes don't take up space when compressing, it reduces image compression time, storage and download resources.

Details:

  1. By mounting an oz created image on loop and using fstrim manually, I can demonstrate the large amount of garbage stuck in the raw file that should instead be holes.
# mount /dev/mapper/loop0p3 /mnt/
# ls -ls
5129956 -rw-r--r--. 1 root root 13958643712 Jul 22 15:03
Fedora-Workstation-Rawhide-20220721.n.0.aarch64.raw
# fstrim -v /mnt
/mnt: 8.5 GiB (9149927424 bytes) trimmed
# ls -ls
3293292 -rw-r--r--. 1 root root 13958643712 Jul 22 15:04
Fedora-Workstation-Rawhide-20220721.n.0.aarch64.raw

This is about 1.75GiB of garbage needlessly compressed, stored, downloaded, and decompressed - times every nightly and person who downloads and uses the image. It's a huge waste. Similar problem exists for Fedora Cloud Base image.

First mount
892856 -rw-rw-r--. 1 chris chris 4294967296 Jul 21 14:45 Fedora-Cloud-Base-36-20220721.0.x86_64.raw
fstrim /boot (partition 2)
856796 -rw-rw-r--. 1 chris chris 4294967296 Jul 21 14:45 Fedora-Cloud-Base-36-20220721.0.x86_64.raw
fstrim / (partition 5)
429316 -rw-rw-r--. 1 chris chris 4294967296 Jul 21 14:46 Fedora-Cloud-Base-36-20220721.0.x86_64.raw

I found this file in koji along with the logs and image files for the aaarch64 Workstation image above.
https://kojipkgs.fedoraproject.org//packages/Fedora-Workstation/Rawhide/20220721.n.0/images/libvirt-raw-xz-aarch64.xml

Also attaching that xml file.
libvirt-raw-xz-aarch64.xml.txt

It contains this section

<disk device="disk" type="file">
<target dev="vda" bus="virtio"/>
<source file="Fedora-Workstation-Rawhide-20220721.n.0.aarch64.raw-xz"/>
<driver name="qemu" type="raw-xz"/>
</disk>

Instead this needs:

<driver name="qemu" type="raw-xz" cache="unsafe" discard="unmap"/>  [1][2]

Alternative:
I'm told oz uses guestfs after installation to cutomize the image, so if it's easier to do the quiescing there, that's OK too.

[1]
The cache="unsafe" is consistent with other Fedora builder and compose systems, improving performance considerably.
[2]
I'm not certain driver type "raw-xz" supports discard=unmap, type "raw" definitely does.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant