Skip to content

Add k8s Pod volume metrics #2319

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ChrsMark
Copy link
Member

@ChrsMark ChrsMark commented Jun 3, 2025

Fixes #1485

Changes

This PR adds k8s.pod.volume.* metrics:

k8s.pod.volume.available
k8s.pod.volume.capacity
k8s.pod.volume.usage
k8s.pod.volume.inode.count
k8s.pod.volume.inode.used # This may not be equal to `inode.count - inode.free` because filesystem may share inodes with other filesystems.
k8s.pod.volume.inode.free

Collector ref: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.127.0/receiver/kubeletstatsreceiver/documentation.md#k8svolumeavailable
Collector implementation: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.127.0/receiver/kubeletstatsreceiver/internal/kubelet/volume.go#L16

These metrics are derived from the VolumeStats field of the PodStats of the Kubelet's stats API.

Note: The current implementation reports them as k8s.volume.* but I think they should be k8s.pod.volume.* to better indicate the scope since there is no concept a K8s Volume outside of the Pod context.

Attributes for volume types that are deprecated are not added here: open-telemetry/opentelemetry-collector-contrib#40477

Merge requirement checklist

@dashpole
Copy link
Contributor

dashpole commented Jun 3, 2025

Do you think it would make sense to consistently use used vs capacity everywhere, and omit free/available?

@ChrsMark
Copy link
Member Author

ChrsMark commented Jun 3, 2025

Do you think it would make sense to consistently use used vs capacity everywhere, and omit free/available?

If I understand the suggestion correctly, it seems that free/available would still be useful/needed?

From https://pkg.go.dev/k8s.io/kubelet@v0.33.0/pkg/apis/stats/v1alpha1#FsStats:

  • UsedBytes: This may differ from the total bytes used on the filesystem and may not equal CapacityBytes - AvailableBytes.
  • InodesUsed : This may not equal Inodes - InodesFree because this filesystem may share inodes with other "filesystems"

@dashpole
Copy link
Contributor

dashpole commented Jun 3, 2025

ah, right

@ChrsMark
Copy link
Member Author

ChrsMark commented Jun 4, 2025

However, based on this I think it makes sense to record also the UsedBytes as well. I will update the PR to add this.

@ChrsMark ChrsMark moved this to In Review in K8s SemConv SIG Jun 4, 2025
@ChrsMark ChrsMark moved this from Untriaged to Awaiting SIG approval in Semantic Conventions Triage Jun 4, 2025
@ChrsMark ChrsMark force-pushed the add_k8s_volume_metrics branch from 968b769 to ad1dfbd Compare June 4, 2025 08:45
@odubajDT
Copy link
Contributor

odubajDT commented Jun 6, 2025

Would it be possible to add a k8s.volume.usedmetric? Details here open-telemetry/opentelemetry-collector-contrib#40476

@ChrsMark
Copy link
Member Author

ChrsMark commented Jun 6, 2025

Would it be possible to add a k8s.volume.usedmetric? Details here open-telemetry/opentelemetry-collector-contrib#40476

It's done: 5459577

@ChrsMark ChrsMark force-pushed the add_k8s_volume_metrics branch 2 times, most recently from aac5ffb to 4e87084 Compare June 17, 2025 07:26
@ChrsMark ChrsMark force-pushed the add_k8s_volume_metrics branch from 4e87084 to fa23af3 Compare June 19, 2025 07:16
@ChrsMark ChrsMark requested a review from dashpole June 20, 2025 07:06
@ChrsMark ChrsMark requested a review from dashpole June 24, 2025 08:09
@ChrsMark ChrsMark force-pushed the add_k8s_volume_metrics branch from f1406f6 to bfe9ddb Compare June 26, 2025 08:22
@ChrsMark ChrsMark force-pushed the add_k8s_volume_metrics branch from bfe9ddb to 8e7c4f8 Compare June 30, 2025 14:18
@ChrsMark
Copy link
Member Author

ChrsMark commented Jun 30, 2025

@dashpole your comments should be addressed now, please take another look when you get the chance.

@ChrsMark ChrsMark force-pushed the add_k8s_volume_metrics branch 2 times, most recently from 93dfc7b to e34483e Compare July 7, 2025 07:37
@ChrsMark ChrsMark moved this from In Review to Approved by K8s SIG in K8s SemConv SIG Jul 8, 2025
@ChrsMark ChrsMark moved this from Awaiting SIG approval to Needs More Approval in Semantic Conventions Triage Jul 8, 2025
@ChrsMark
Copy link
Member Author

ChrsMark commented Jul 8, 2025

@open-telemetry/specs-semconv-approvers @open-telemetry/specs-semconv-maintainers please take a look

Signed-off-by: ChrsMark <chrismarkou92@gmail.com>
@ChrsMark ChrsMark force-pushed the add_k8s_volume_metrics branch from e34483e to f64c9fd Compare July 8, 2025 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:k8s enhancement New feature or request
Projects
Status: Approved by K8s SIG
Status: Needs More Approval
Development

Successfully merging this pull request may close these issues.

[k8s] Define semantic conventions for k8s volume metrics
5 participants