Remove meters from storage list when removed and handle NoRecordedValue flag when removing meters

[stonkie]

Fixes #5950
Design discussion issue #

Changes

This allows clean Dispose of Meter objects which will send NoRecordedValue data points to close the series (e.g. send a staleness marker to prometheus). It also fixes the internal storage leak which would prevent frequent rotation of the metrics.

NOTE : A significant refactoring of the protobuf serialization merged right after I created this PR, so I had to redo a significant part of the change on Dec 4th.

Merge requirement checklist

• CONTRIBUTING guidelines followed (license requirements, nullable enabled, static analysis, etc.)
• Unit tests added/updated
• Appropriate CHANGELOG.md files updated for non-trivial changes
• Changes in public API reviewed (if applicable)

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: stonkie / name: Kevin Coulombe (e4e377e, 2823545, f4fa00b, c9d5475, 60aa3db, cc18547, 30dca16, d12808d, 57bf5d0, 8ab84af, 04a3eee, c2dfaea, aeb1639)
• ✅ login: rajkumar-rangaraj / name: Rajkumar Rangaraj (32668c2, 12f4541)

[Kielek]

Hi @stonkie, thanks for the contribution.
Signing EasyCLA is hard requirement before we can accept contribution.

[codecov]

Codecov Report

Attention: Patch coverage is 37.50000% with 30 lines in your changes missing coverage. Please review.

Project coverage is 83.95%. Comparing base (e3665c9) to head (c9d5475).

Files with missing lines	Patch %	Lines
...tryProtocol/Implementation/MetricItemExtensions.cs	16.66%	30 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5997      +/-   ##
==========================================
- Coverage   84.14%   83.95%   -0.20%     
==========================================
  Files         275      275              
  Lines       12560    12603      +43     
==========================================
+ Hits        10569    10581      +12     
- Misses       1991     2022      +31     

Flag	Coverage Δ
unittests-Project-Experimental	83.82% <37.50%> (-0.28%)	⬇️
unittests-Project-Stable	83.94% <37.50%> (-0.09%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/OpenTelemetry/Metrics/Metric.cs	97.26% <100.00%> (+0.01%)	⬆️
...rc/OpenTelemetry/Metrics/Reader/MetricReaderExt.cs	92.24% <100.00%> (+0.42%)	⬆️
...tryProtocol/Implementation/MetricItemExtensions.cs	82.66% <16.66%> (-11.21%)	⬇️

... and 4 files with indirect coverage changes

---- 🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests

[stonkie]

@Kielek : I'm going through the motions to get this done at my company and there are more hoops than I expected. EasyCLA will probably wait until Monday.

[stonkie]

Discussions during SIG meeting led to asking to feature flag the NoRecordedValue DataPoint to avoid changing working behavior. I'll add that and adjust the tests asap.

[stonkie]

I'm not exactly sure how to format the changelog entry. Should I create an entry without version number or release date?

[Kielek]

I'm not exactly sure how to format the changelog entry. Should I create an entry without version number or release date?

Please put it under unreleased section.

## Unreleased

* Your description goes here.
  And here if it is long enough.
  ([#5997](https://github.com/open-telemetry/opentelemetry-dotnet/pull/5997))

```

[stonkie]

@Kielek Thanks, it's done.

[rajkumar-rangaraj]

There is a lot of refactoring in this file that is unrelated to the proposed changes. Could you please refactor these outside of this current PR? Doing so either before or after this PR merges would be fine.

[stonkie]

I'm off for the holidays. I'll look into that first week of January.

[rajkumar-rangaraj]

This should move to OTLP Exporter changelog.

[rajkumar-rangaraj]

To make the review process easier, could you please separate the SDK and OTLP Exporter changes into different PRs? I recommend finishing off the SDK part first and then moving to the OTLP Exporter changes. Additionally, it would be helpful to keep the refactoring outside the scope of these PRs.