Skip to content

Commit

Permalink
add monotonic counter uint meter type (#105)
Browse files Browse the repository at this point in the history
This change is a companion to the fix for monotonic counter implementations in
spectatord.

Netflix-Skunkworks/spectatord#90

The original monotonic counter (`C`) was always intended to be used for the
case where a monotonic data source needs to be transformed into base units
for recording data. For example, transforming nanoseconds into seconds. This
requires division, which results in floats.

There is a valid use case for handling uints in monotonic counters, if the
data source is already in a base unit, such as bytes. Thus, a new meter type
`U` is added to spectatord which supports this use case.
  • Loading branch information
copperlight authored Jun 15, 2024
1 parent b3b93d6 commit 0b4efd6
Show file tree
Hide file tree
Showing 12 changed files with 350 additions and 213 deletions.
6 changes: 4 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ jobs:
CXX: "g++-11"
LANG: "en_US.UTF-8"
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Conan+Cmake Cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: |
~/.conan
Expand Down Expand Up @@ -57,6 +57,8 @@ jobs:
cd $BUILD_DIR
cmake -DCMAKE_BUILD_TYPE=$BUILD_TYPE ..
cmake --build .
echo "==== ldd ===="
ldd bin/spectatord_main || true
- name: Test spectator-cpp
run: |
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
.DS_Store
.idea/
cmake-build-debug/
cmake-build-debug
cmake-build/
venv/
27 changes: 16 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,8 @@

# Spectator-cpp

Simple library for instructing code to record dimensional time series. It sends all activity to
a [spectatord](https://github.com/Netflix-Skunkworks/spectatord) sidecar.

## Description

This implements a basic [Spectator](https://github.com/Netflix/spectator) library for instrumenting
C++ applications, sending all metric activity to a sidecar.
This implements a basic [Spectator](https://github.com/Netflix/spectator) library for instrumenting Go applications. It
consists of a thin client designed to send metrics through [spectatord](https://github.com/Netflix-Skunkworks/spectatord).

## Instrumenting Code

Expand Down Expand Up @@ -80,19 +75,29 @@ int main() {
}
}
```
## High-Volume Publishing
By default, the library sends every meter change to the spectatord sidecar immediately. This involves a blocking `send` call and underlying system calls, and may not be the most efficient way to publish metrics in high-volume use cases.
For this purpose a simple buffering functionality in `Publisher` is implemented, and it can be turned on by passing a buffer size to the `spectator::Config` constructor. It is important to note that, until this buffer fills up, the `Publisher` will not send nay meters to the sidecar. Therefore, if your application doesn't emit meters at a high rate, you should either keep the buffer very small, or do not configure a buffer size at all, which will fall back to the "publish immediately" mode of operation.
By default, the library sends every meter change to the spectatord sidecar immediately. This involves a blocking
`send` call and underlying system calls, and may not be the most efficient way to publish metrics in high-volume
use cases. For this purpose a simple buffering functionality in `Publisher` is implemented, and it can be turned
on by passing a buffer size to the `spectator::Config` constructor. It is important to note that, until this buffer
fills up, the `Publisher` will not send nay meters to the sidecar. Therefore, if your application doesn't emit
meters at a high rate, you should either keep the buffer very small, or do not configure a buffer size at all,
which will fall back to the "publish immediately" mode of operation.
## Local Development
```shell
# setup python venv and activate, to gain access to conan cli
./setup-venv.sh
source venv/bin/activate
./build.sh # [clean|skiptest]
# link clion default build directory to our build directory
ln -s cmake-build cmake-build-debug
./build.sh # [clean|clean --force|skiptest]
```

* CLion > Preferences > Plugins > Marketplace > Conan > Install
* CLion > Preferences > Build, Execution, Deploy > Conan > Conan Executable: $PROJECT_HOME/venv/bin/conan
* CLion > Bottom Bar: Conan > Left Button: Match Profile > CMake Profile: Debug, Conan Profile: default
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
conan==1.62.0
conan==1.64.1
20 changes: 12 additions & 8 deletions setup-venv.sh
Original file line number Diff line number Diff line change
@@ -1,17 +1,21 @@
#!/usr/bin/env bash

PYTHON3=$(which python3)
MAYBE_PYTHON=$(find /apps -maxdepth 1 -type l -name "python*")

if [[ -z $PYTHON3 ]]; then
echo "python3 is not available - please install"
exit 1
if [[ -n "$MAYBE_PYTHON" ]]; then
PYTHON3="$MAYBE_PYTHON/bin/python3"
echo "using $PYTHON3 ($($PYTHON3 -V))"
else
PYTHON3="python3"
echo "using $(which $PYTHON3) ($($PYTHON3 -V))"
fi

python3 -m venv venv

# create and activate virtualenv
$PYTHON3 -m venv venv
source venv/bin/activate

if [[ -f requirements.txt ]]; then
pip3 install --upgrade pip wheel
pip3 install --requirement requirements.txt
# use the virtualenv python
python -m pip install --upgrade pip wheel
python -m pip install --requirement requirements.txt
fi
25 changes: 14 additions & 11 deletions spectator/meter_type.h
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,17 @@

namespace spectator {
enum class MeterType {
AgeGauge,
Counter,
DistSummary,
Gauge,
MaxGauge,
AgeGauge,
MonotonicCounter,
Timer,
MonotonicCounterUint,
PercentileDistSummary,
PercentileTimer,
PercentileDistSummary
Timer
};

}

// Provide a formatter for MeterType
Expand All @@ -25,6 +25,9 @@ struct fmt::formatter<spectator::MeterType> : fmt::formatter<std::string_view> {
using namespace spectator;
std::string_view s = "unknown";
switch (meter_type) {
case MeterType::AgeGauge:
s = "age-gauge";
break;
case MeterType::Counter:
s = "counter";
break;
Expand All @@ -37,20 +40,20 @@ struct fmt::formatter<spectator::MeterType> : fmt::formatter<std::string_view> {
case MeterType::MaxGauge:
s = "max-gauge";
break;
case MeterType::AgeGauge:
s = "age-gauge";
break;
case MeterType::MonotonicCounter:
s = "monotonic-counter";
break;
case MeterType::Timer:
s = "timer";
case MeterType::MonotonicCounterUint:
s = "monotonic-counter-uint";
break;
case MeterType::PercentileDistSummary:
s = "percentile-distribution-summary";
break;
case MeterType::PercentileTimer:
s = "percentile-timer";
break;
case MeterType::PercentileDistSummary:
s = "percentile-distribution-summary";
case MeterType::Timer:
s = "timer";
break;
}
return fmt::formatter<std::string_view>::format(s, context);
Expand Down
5 changes: 2 additions & 3 deletions spectator/monotonic_counter_test.cc
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,10 @@ TEST(MonotonicCounter, Set) {
MonotonicCounter<TestPublisher> c{id, &publisher};
MonotonicCounter<TestPublisher> c2{id2, &publisher};

c.Set(42);
c.Set(42.1);
c2.Set(2);
c.Set(43);
std::vector<std::string> expected = {"C:ctr:42", "C:ctr2,key=val:2",
"C:ctr:43"};
std::vector<std::string> expected = {"C:ctr:42.1", "C:ctr2,key=val:2", "C:ctr:43"};
EXPECT_EQ(publisher.SentMessages(), expected);
}
} // namespace
25 changes: 25 additions & 0 deletions spectator/monotonic_counter_uint_test.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#include "stateless_meters.h"
#include "test_publisher.h"
#include <gtest/gtest.h>

namespace {

using spectator::Id;
using spectator::MonotonicCounterUint;
using spectator::Tags;
using spectator::TestPublisher;

TEST(MonotonicCounterUint, Set) {
TestPublisher publisher;
auto id = std::make_shared<Id>("ctr", Tags{});
auto id2 = std::make_shared<Id>("ctr2", Tags{{"key", "val"}});
MonotonicCounterUint<TestPublisher> c{id, &publisher};
MonotonicCounterUint<TestPublisher> c2{id2, &publisher};

c.Set(42);
c2.Set(2);
c.Set(43);
std::vector<std::string> expected = {"U:ctr:42", "U:ctr2,key=val:2", "U:ctr:43"};
EXPECT_EQ(publisher.SentMessages(), expected);
}
} // namespace
Loading

0 comments on commit 0b4efd6

Please sign in to comment.