Skip to content
This repository has been archived by the owner on Jun 8, 2024. It is now read-only.

Commit

Permalink
sketching out docs
Browse files Browse the repository at this point in the history
  • Loading branch information
KodrAus committed Mar 17, 2024
1 parent 0b23c37 commit a5791bb
Show file tree
Hide file tree
Showing 8 changed files with 74 additions and 20 deletions.
4 changes: 3 additions & 1 deletion batcher/src/internal_metrics.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,9 @@ impl Counter {
}

impl InternalMetrics {
pub fn sample(&self) -> impl Iterator<Item = emit::metrics::Metric<'static, emit::empty::Empty>> + 'static {
pub fn sample(
&self,
) -> impl Iterator<Item = emit::metrics::Metric<'static, emit::empty::Empty>> + 'static {
let InternalMetrics {
queue_overflow,
queue_batch_processed,
Expand Down
12 changes: 9 additions & 3 deletions batcher/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,9 @@ impl<T: Channel> Sender<T> {
}
}

pub fn sample_metrics(&self) -> impl Iterator<Item = emit::metrics::Metric<'static, emit::empty::Empty>> + 'static {
pub fn sample_metrics(
&self,
) -> impl Iterator<Item = emit::metrics::Metric<'static, emit::empty::Empty>> + 'static {
self.shared.sample_metrics()
}
}
Expand Down Expand Up @@ -350,7 +352,9 @@ impl<T: Channel> Receiver<T> {
}
}

pub fn sample_metrics(&self) -> impl Iterator<Item = emit::metrics::Metric<'static, emit::empty::Empty>> + 'static {
pub fn sample_metrics(
&self,
) -> impl Iterator<Item = emit::metrics::Metric<'static, emit::empty::Empty>> + 'static {
self.shared.sample_metrics()
}
}
Expand Down Expand Up @@ -434,7 +438,9 @@ struct Shared<T> {
}

impl<T: Channel> Shared<T> {
fn sample_metrics(&self) -> impl Iterator<Item = emit::metrics::Metric<'static, emit::empty::Empty>> + 'static {
fn sample_metrics(
&self,
) -> impl Iterator<Item = emit::metrics::Metric<'static, emit::empty::Empty>> + 'static {
let queue_length = { self.state.lock().unwrap().next_batch.channel.remaining() };

self.metrics.sample().chain(Some(emit::metrics::Metric::new(
Expand Down
7 changes: 6 additions & 1 deletion core/src/event.rs
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,12 @@ impl<'a, P> Event<'a, P> {
}

pub fn with_props<U>(self, props: U) -> Event<'a, U> {
Event { module: self.module, extent: self.extent, tpl: self.tpl, props }
Event {
module: self.module,
extent: self.extent,
tpl: self.tpl,
props,
}
}
}

Expand Down
6 changes: 2 additions & 4 deletions core/src/props.rs
Original file line number Diff line number Diff line change
Expand Up @@ -79,9 +79,7 @@ pub trait Props {
where
Self: Sized,
{
Dedup {
src: self,
}
Dedup { src: self }
}
}

Expand Down Expand Up @@ -408,7 +406,7 @@ mod internal {
) -> ControlFlow<()>;

fn dispatch_get(&self, key: Str) -> Option<Value>;

fn dispatch_count(&self) -> usize;

fn dispatch_is_unique(&self) -> bool;
Expand Down
49 changes: 41 additions & 8 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Emit is a structured logging framework for manually instrumenting Rust applicati
# Events
All diagnostics in Emit are represented as _events_. An event is a notable change in the state of a system that is made available to outside observers. Events carry both a human-readable description of what triggered them as well as a structured payload that can be used to process them. Events are temporal; they may be anchored to a point in time at which they occurred, or may cover a span of time for which they are active.
All diagnostics in Emit are represented as _events_. An event is a notable change in the state of a system that is broadcast to outside observers. Events carry both a human-readable description of what triggered them as well as a structured payload that can be used to process them. Events are temporal; they may be anchored to a point in time at which they occurred, or may cover a span of time for which they are active.
## Core data model
Expand All @@ -16,35 +16,68 @@ The core event model includes:
- **Template (`tpl` and `msg`):** A human-readable description of the event that its properties may be interpolated into.
- **Properties (`props`):** The payload of the event.
## Extensions
## Extents
The extent of an event is the time for which the event is relevant. This may be a single point in time if the event was triggered by something happening, or a span of time if the event was started at one point and completed at a later one.
## Templates
The primary source of information in an event is its _template_. A template is a human-readable description of an event with holes to interpolate values into. Templates are responsible for both capturing local state to include in an event, and to format that state into a human-readable description.
Templates are a useful low-cardinality identifier for events.
Emit's templates are inspired by [Message Templates](https://messagetemplates.org).
## Properties
Emit's properties are structured key-value pairs where keys are strings and values are anything from primitive types to complex records and sequences. Values can use the data model of either `serde` or `sval`.
## Observability signals
Emit doesn't hard-code common observability concepts into events. It instead relies on the presence well-known properties to carry that information.
### Logging
### Logs
Events with a point extent can represent log records. Well-known properties related to logs include:
- **Level (`lvl`):** A traditional log level that describes the relative severity of an event for coarse-grained filtering.
- **Debug:** A high-frequency point in the execution of an operation.
- **Info:** A significant point in the execution of an operation.
- **Warn:** An erroneous event that didn't cause its operation to fail.
- **Error:** An erroneous event that caused its operation to fail.
- **Error (`err`):** An error that caused the event.
### Tracing
### Traces
Events with a span extent can represent spans in a distributed trace. Events in a distributed trace also need to carry a _trace id_ and _span id_. Well-known properties related to traces include:
- **Trace id (`trace_id`):** An identifier that marks an event as belonging to a distributed trace.
- **Span id (`span_id`):** An identifier that marks an event as belonging to a span of execution in a distributed trace.
- **Parent span id (`span_parent`):** An identifier that links the span id of an event to the span id of its parent.
Emit doesn't define any direct APIs for trace propagation or sampling. That responsibility is left up to the caller.
### Metrics
Emit's model for metrics is based on _aggregations_. A metric captures the result of applying an aggregation over an underlying timeseries data source within the extent to produce a sample. Events with a point extent can represent cumulative metric samples. Events with a span extent can represent delta metric samples. Well-known properties related to metrics include:
- **Metric name (`metric_name`):** The name of a data source that marks an event as representing a metric sampled from that source.
- **Metric aggregation (`metric_agg`):** The aggregation over the data source the metric sample was computed with.
- **Last:** The latest value in the underlying source.
- **Sum:** The sum of all values in the underlying source.
- **Count:** The count of all values in the underlying source. A count is a monotonic sum of ones.
- **Metric value (`metric_value`):** The sampled value from the metric source.
- **Metric unit (`metric_unit`):** The unit the sampled value is in.
## Extents
Emit's metric support can represent common cases of counters and gauges, but can't express the full fidelity of other models.
The extent of an event is the time for which the event is relevant. This may be a single point in time if the event was triggered by something happening, or a span of time if the event was started at some point and completed at another.
# Context
## Templates
Emit doesn't require threading loggers through your program directly. You can store ambient state you want events to carry in the current _context_. Emit's context is a stack that can be managed either directly for synchronous operations, or through a future for asynchronous ones.
# Runtime
The primary source of information in an event is its _template_. A template is a human-readable description of an event with holes to interpolate values into. Templates are responsible for both capturing ambient state to package with an event, and to format that state into a human-readable representation. Templates are also useful as low-cardinality identifiers for a specific shaped event, independent of the properties captured by any particular instance of it.
The set of components needed to produce, receive, filter, and emit events is encapsulated in a _runtime_. A system will typically configure the built-in shared runtime and use it, but any or multiple runtimes can be used independantly.
*/

#![cfg_attr(not(feature = "std"), no_std)]
Expand Down
8 changes: 7 additions & 1 deletion src/metrics.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
use core::ops::ControlFlow;

use emit_core::{
empty::Empty, extent::{Extent, ToExtent}, path::Path, props::Props, str::{Str, ToStr}, value::{ToValue, Value}, well_known::{KEY_METRIC_AGG, KEY_METRIC_NAME, KEY_METRIC_VALUE}
empty::Empty,
extent::{Extent, ToExtent},
path::Path,
props::Props,
str::{Str, ToStr},
value::{ToValue, Value},
well_known::{KEY_METRIC_AGG, KEY_METRIC_NAME, KEY_METRIC_VALUE},
};

pub struct Metric<'a, P> {
Expand Down
4 changes: 3 additions & 1 deletion targets/file/src/internal_metrics.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,9 @@ impl Counter {
}

impl InternalMetrics {
pub fn sample(&self) -> impl Iterator<Item = emit::metrics::Metric<'static, emit::empty::Empty>> + 'static {
pub fn sample(
&self,
) -> impl Iterator<Item = emit::metrics::Metric<'static, emit::empty::Empty>> + 'static {
let InternalMetrics {
file_set_read_failed,
file_open_failed,
Expand Down
4 changes: 3 additions & 1 deletion targets/otlp/src/internal_metrics.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,9 @@ impl Counter {
}

impl InternalMetrics {
pub fn sample(&self) -> impl Iterator<Item = emit::metrics::Metric<'static, emit::empty::Empty>> + 'static {
pub fn sample(
&self,
) -> impl Iterator<Item = emit::metrics::Metric<'static, emit::empty::Empty>> + 'static {
let InternalMetrics {
otlp_event_discarded,
} = self;
Expand Down

0 comments on commit a5791bb

Please sign in to comment.