Skip to content

Commit

Permalink
v2: Ditch Query helpers. (#17)
Browse files Browse the repository at this point in the history
This PR focuses around simplification of the tsplot package. Moving away
from something that was rather perscriptive in the type of Query that
you could make and towards a package that can accept a Time Series or
Time Series Itterator from the Google Metrics API and return a rendered
graph of those Time Series.
  • Loading branch information
jharshman authored Dec 14, 2023
1 parent b97b9fd commit 047e30c
Show file tree
Hide file tree
Showing 15 changed files with 218 additions and 641 deletions.
3 changes: 1 addition & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ jobs:
- uses: actions/checkout@v2
- uses: actions/setup-go@v2
with:
go-version: '^1.16.2'
go-version: '^1.20'
- run: make test
- run: make build
- run: make codegendiff
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
.idea/*
.DS_store
14 changes: 1 addition & 13 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,9 @@ test:
$(GOBIN) test -v ./...

.PHONY: build
build: $(BINDIR)/codegen
build:
make -C tscli

$(BINDIR)/codegen: $(SCRIPTDIR)/codegen.go
@mkdir -p $(BINDIR)
GOOS=linux GOARCH=amd64 $(GOBIN) build -o $(BINDIR)/codegen $<

.PHONY: install
install: build
cp $(BINDIR)/tscli /usr/local/bin/
Expand All @@ -26,11 +22,3 @@ install: build
clean:
rm -rf $(BINDIR)

CODEGENFILE="set_aggregation_opts.go"
.PHONY: codegen
codegen:
$(BINDIR)/codegen -output ./tsplot/$(CODEGENFILE)

.PHONY: codegendiff
codegendiff:
diff ./tsplot/$(CODEGENFILE) <($(BINDIR)/codegen -stdout)
82 changes: 21 additions & 61 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,84 +2,44 @@
[![Go Report Card](https://goreportcard.com/badge/github.com/bitly/tsplot)](https://goreportcard.com/report/github.com/bitly/tsplot)
[![Go Reference](https://pkg.go.dev/badge/github.com/bitly/tsplot.svg)](https://pkg.go.dev/github.com/bitly/tsplot)

This package provides a method of querying for raw time series data from the GCM APIs and additionally plotting that data for use in other applications.

This came to be due to what we consider a small limitation in the Google APIs which require us to re-draw graphs to include them in other applications such as
Slack bots. There is no facility in the Google API that provides a PNG of already graphed data.

## Authentication
This package makes no effort to assist in authentication to the Google APIs.
Instead, it will expect the caller to supply an authenticated client.

More information on authentication can be found in the official [Google Cloud documentation](https://cloud.google.com/docs/authentication).

## Query
tsplot helps to facilitate easy querying of the Google Cloud Monitoring API for time series matching the supplied criteria.
In addition it provides methods of overriding certain aspects of the query.

For example, the following code snippet will return a single time series for the following metric descriptor: `custom.googleapis.com/opencensus/fishnet/queuereader_fishnet/messages_total`.
```
func main() {
... snip ...
... snip ...
start := time.Now().Add(-1 * time.Hour)
end := time.Now()
mq := &tsplot.NewMetricQuery(
"bitly-gcp-prod", // GCP project
"custom.googleapis.com/opencensus/fishent/queuereader_fishnet/messages_total", // metric descriptor
&start, // start of time window
&end, // end of time window
)
// disable cross series reducer (MEAN reduction is default)
query.Set_REDUCE_NONE()
// set different alignment window. (Default is 1 minute)
query.SetAlignmentPeriod(time.Minute * 2)
tsi, err := mq.PerformWithClient(client) // client is provided by user
if err != nil {
fmt.Printf("error performing query: %v\n", err)
}
}
```

## Plotting
To plot the data, tsplot leverages the open source package [gonum/plot](github.com/gonum/plot) to create a graph and plot the data for a given time series.

The example below creates a new graph containing a singular time series, plots it, and saves the resulting plot to disk.
```
func main() {
... snip ...
ts := tsplot.TimeSeries{}
// optionally iterate over returned time series
timeSeries, _ := tsi.Next()
ts[metric] = ts.GetPoints()
// create the plot with some formatting options
p, err := ts.Plot([]tsplot.PlotOption{
tsplot.WithXAxisName("UTC"),
tsplot.WIthGrid(colornames.Darkgrey),
tsplot.WithTitle(metric)}...)
if err != nil {
return err
// create new request
request := monitoringpb.ListTimeSeriesRequest{
Name: fmt.Sprintf("projects/%s", project),
Filter: query,
Interval: &monitoringpb.TimeInterval{
EndTime: timestamppb.New(et),
StartTime: timestamppb.New(st),
},
Aggregation: nil,
SecondaryAggregation: nil,
View: monitoringpb.ListTimeSeriesRequest_FULL,
}
// optionally save the plot to disk
p.Save(8*vg.Inch, 4*vg.Inch, "./my-graph.png")
// execute the request and get the response from Google APIs
tsi := GoogleCloudMonitoringClient.ListTimeSeries(context.Background(), request)
// Create the plot from the GAPI TimeSeries
plot, _ := tsplot.NewPlotFromTimeSeriesIterator(tsi, "", nil)
// Save the new plot to disk.
plot.Save(8*vg.Inch, 4*vg.Inch, "my_plot.png")
}
```

### Example generated graphs:
Query across multiple time series with mean reducer:
![graph1](sample/1.png)

### Graph Color Scheme
I'm not a UX designer, but I have selected colors that I find higher contrast
and easier to see. I am basing this completely off my colorblindness which is
unique to me. Improvements to the color palette used are welcome.
24 changes: 23 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/bitly/tsplot

go 1.14
go 1.20

require (
cloud.google.com/go v0.80.0
Expand All @@ -11,3 +11,25 @@ require (
google.golang.org/genproto v0.0.0-20210401141331-865547bb08e2
google.golang.org/protobuf v1.26.0
)

require (
github.com/ajstarks/svgo v0.0.0-20180226025133-644b8db467af // indirect
github.com/fogleman/gg v1.3.0 // indirect
github.com/go-fonts/liberation v0.1.1 // indirect
github.com/go-latex/latex v0.0.0-20210118124228-b3d85cf34e07 // indirect
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 // indirect
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e // indirect
github.com/golang/protobuf v1.5.1 // indirect
github.com/google/go-cmp v0.5.5 // indirect
github.com/googleapis/gax-go/v2 v2.0.5 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/phpdave11/gofpdf v1.4.2 // indirect
github.com/spf13/pflag v1.0.5 // indirect
go.opencensus.io v0.23.0 // indirect
golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4 // indirect
golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84 // indirect
golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4 // indirect
golang.org/x/text v0.3.5 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/grpc v1.36.1 // indirect
)
1 change: 0 additions & 1 deletion go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -526,7 +526,6 @@ golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8T
gonum.org/v1/gonum v0.0.0-20180816165407-929014505bf4/go.mod h1:Y+Yx5eoAFn32cQvJDxZx5Dpnq+c3wtXuadVZAcxbbBo=
gonum.org/v1/gonum v0.8.2 h1:CCXrcPKiGGotvnN6jfUsKk4rRqm7q09/YbKb5xCEvtM=
gonum.org/v1/gonum v0.8.2/go.mod h1:oe/vMfY3deqTw+1EZJhuvEW2iwGF1bW9wwu7XCu0+v0=
gonum.org/v1/netlib v0.0.0-20190313105609-8cb42192e0e0 h1:OE9mWmgKkjJyEmDAAtGMPjXu+YNeGvK9VTSHY6+Qihc=
gonum.org/v1/netlib v0.0.0-20190313105609-8cb42192e0e0/go.mod h1:wa6Ws7BG/ESfp6dHfk7C6KdzKA7wR7u/rKwOGE66zvw=
gonum.org/v1/plot v0.0.0-20190515093506-e2840ee46a6b/go.mod h1:Wt8AAjI+ypCyYX3nZBvf6cAIx93T+c/OS2HFAYskSZc=
gonum.org/v1/plot v0.9.0 h1:3sEo36Uopv1/SA/dMFFaxXoL5XyikJ9Sf2Vll/k6+2E=
Expand Down
67 changes: 0 additions & 67 deletions scripts/codegen.go

This file was deleted.

2 changes: 1 addition & 1 deletion tscli/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ default: build
.PHONY: build
build:
@mkdir -p $(BINDIR)
GOOS=linux GOARCH=amd64 $(GOBIN) build -o $(BINDIR)/tscli
$(GOBIN) build -o $(BINDIR)/tscli
Loading

0 comments on commit 047e30c

Please sign in to comment.