Refactor, structural and performance optimizations (#33)

* pkg/api: providing a service cache

* pkg/prom: exposed metrics

* pkg/api: enhancements

* pkg/rt: interacting with rt.fastly.com

* pkg/prom: doc.go

* go mod tidy, go mod vendor

* pkg/api: Option -> CacheOption, Service -> Metadata

* pkg/rt: first draft of Manager, still needs tests

* pkg/rt: api.Cache.Service -> Metadata

* pkg/{api,rt}: clarifications and manager test

* cmd/fastly-exporter: port func main

* go mod tidy, go mod vendor

* pkg/rt: fix manager test flake

* rm old version

* Dockerfile: update

* Update README and improve -shard semantics

* pkg/rt: jsoniter ~10x faster unmarshaling rt.fastly.com responses

* README: use latest -h output

* README: fixes and readability

* go mod tidy, go mod vendor

* Add -api-timeout and -rt-timeout flags

* Add realtime_api_requests_total metric

With service_id, service_name, and result labels,
and result including "no data", this should help
low-traffic services not appear broken.

* Improve tests

* pkg/rt: try to clean up Subscriber.Run/query boundary

* -name-regex -> -name-{include,exclude}-regex

* Add pprof endpoints

* Update README.md

Co-Authored-By: Jordan Neufeld <neufeldtech@users.noreply.github.com>

* Update cmd/fastly-exporter/main.go

Co-Authored-By: Jordan Neufeld <neufeldtech@users.noreply.github.com>

* Grammar

Dockerfile

@@ -1,7 +1,7 @@
 # Based off https://medium.com/@chemidy/create-the-smallest-and-secured-golang-docker-image-based-on-scratch-4752223b7324
 
 # Accept the Go version for the image to be set as a build argument
-ARG GO_VERSION=1.11.2
+ARG GO_VERSION=1.12.7
 
 # First stage: build the executable
 FROM golang:${GO_VERSION}-alpine AS builder
@@ -19,14 +19,12 @@ RUN adduser -D -g '' appuser
 COPY . $GOPATH/src/github.com/peterbourgon/fastly-exporter/
 WORKDIR $GOPATH/src/github.com/peterbourgon/fastly-exporter/
 
-# Dependencies are maintained in the parent repository
-# RUN go get -d -v
-
 # Build the binary
 RUN CGO_ENABLED=0 go build \
 	-a \
-	-ldflags="-X main.version=$(git describe | sed -e 's/^v//')" \
-	-o /go/bin/fastly-exporter
+	-ldflags="-X main.programVersion=$(git describe | sed -e 's/^v//')" \
+	-o /go/bin/fastly-exporter \
+	./cmd/fastly-exporter
 
 # Second stage: build the container
 FROM scratch

README.md

@@ -1,16 +1,21 @@
 # fastly-exporter [![Latest Release](https://img.shields.io/github/release/peterbourgon/fastly-exporter.svg?style=flat-square)](https://github.com/peterbourgon/fastly-exporter/releases/latest) [![builds.sr.ht status](https://builds.sr.ht/~peterbourgon/fastly-exporter.svg)](https://builds.sr.ht/~peterbourgon/fastly-exporter?) [![Docker Status](https://img.shields.io/docker/build/mrnetops/fastly-exporter.svg)](https://hub.docker.com/r/mrnetops/fastly-exporter)
 
 This program consumes from the [Fastly Real-time Analytics API][rt] and makes
-the data available to [Prometheus][prom]. It can provide metrics for every
-service accessible to your API token, or an explicitly-specified set of
-services. And it reflects when new services are created, old services are
-deleted, or existing services have their names or versions updated.
+the data available to [Prometheus][prom]. It should behave like you expect:
+dynamically adding new services, removing old services, and reflecting changes
+to service metadata like name and version.
 
 [rt]: https://docs.fastly.com/api/analytics
 [prom]: https://prometheus.io
 
 ## Getting
 
+### Binary
+
+Go to the [releases page][releases].
+
+[releases]: https://github.com/peterbourgon/fastly-exporter/releases
+
 ### Docker
 
 Avaliable as [mrnetops/fastly-exporter][container] from [Docker Hub][hub].
@@ -22,18 +27,12 @@ Avaliable as [mrnetops/fastly-exporter][container] from [Docker Hub][hub].
 docker pull mrnetops/fastly-exporter
 ```
 
-### Binary
-
-Go to the [releases page][releases].
-
-[releases]: https://github.com/peterbourgon/fastly-exporter/releases
-
 ### Source
 
 If you have a working Go installation, you can install the latest revision from HEAD.
 
 ```
-go get github.com/peterbourgon/fastly-exporter
+go get github.com/peterbourgon/fastly-exporter/cmd/fastly-exporter
 ```
 
 ## Using
@@ -43,43 +42,73 @@ USAGE
   fastly-exporter [flags]
 
 FLAGS
+  -api-refresh 1m0s                        how often to poll api.fastly.com for updated service metadata
+  -api-timeout 15s                         HTTP client timeout for api.fastly.com requests (5–60s)
   -debug false                             Log debug information
   -endpoint http://127.0.0.1:8080/metrics  Prometheus /metrics endpoint
+  -name-exclude-regex ...                  if set, ignore any service whose name matches this regex
+  -name-include-regex ...                  if set, only include services whose names match this regex
   -namespace fastly                        Prometheus namespace
-  -service ...                             Specific Fastly service ID (optional, repeatable)
+  -rt-timeout 45s                          HTTP client timeout for rt.fastly.com requests (45–120s)
+  -service ...                             if set, only include this service ID (repeatable)
+  -shard ...                               if set, only include services whose hashed IDs modulo m equal n-1 (format 'n/m')
   -subsystem rt                            Prometheus subsystem
   -token ...                               Fastly API token (required; also via FASTLY_API_TOKEN)
   -version false                           print version information and exit
 ```
 
-A valid Fastly API token is mandatory. [See this link][token] for information
-on creating API tokens. The token can also be provided via the `-token` flag
-or the FASTLY_API_TOKEN environment variable.
-
-Optional `-service` IDs can be specified to limit monitoring to specific
-services. Service IDs are available at the top of your [Fastly dashboard][db].
+A valid Fastly API token is required. [See this link][token] for information on
+creating API tokens. The token can be provided via the `-token` flag or the
+FASTLY_API_TOKEN environment variable.
 
 [token]: https://docs.fastly.com/guides/account-management-and-security/using-api-tokens#creating-api-tokens
+
+By default, all services available to your token will be exported. You can
+specify an explicit set of service IDs by using the `-service xxx` flag.
+(Service IDs are available at the top of your [Fastly dashboard][db].) You can
+also include only those services whose name matches a regex by using the
+`-name-include-regex '^Production'` flag, or reject any service whose name
+matches a regex by using the `-name-exclude-regex '.*TEST.*'` flag.
+
 [db]: https://manage.fastly.com/services/all
 
-### Docker
+For tokens with access to a lot of services, it's possible to "shard" the
+services among different instances of the fastly-exporter by using the `-shard`
+flag. For example, to shard all services between 3 expoters, you would start
+each exporter as
 
 ```
-docker run -p 8080:8080 mrnetops/fastly-exporter -token $FASTLY_API_TOKEN
+fastly-exporter [common flags] -shard 1/3
+fastly-exporter [common flags] -shard 2/3
+fastly-exporter [common flags] -shard 3/3
 ```
 
-### Docker Compose
+Flags which restrict the services that are exported combine with AND semantics.
+That is, `-service A -service B -name-include-regex 'Foo'` would only export
+data for service A and/or B if their names also matched 'Foo'. Or, specifying
+`-name-include-regex 'Prod' -name-exclude-regex '^test-'` would only export data
+for services whose names contained "Prod" and did not start with "test-".
+
+### Docker
+
+This repo contains a Dockerfile if you want to build and package it yourself.
+You can also use a third-party Docker image.
+
+```
+docker run -p 8080:8080 mrnetops/fastly-exporter -token $MY_TOKEN
+```
 
-[Docker Compose][compose] for a full fastly-exporter + [Prometheus][prom] + 
-[Grafana][grafana] + Fastly dashboard stack
+This repo also contains a [Docker Compose][compose] file, which boots up a full
+fastly-exporter + [Prometheus][prom] + [Grafana][grafana] + Fastly dashboard
+stack.
 
 [compose]: https://github.com/docker/compose
 [grafana]: https://grafana.com
 
 ```
-FASTLY_API_TOKEN=${FASTLY_API_TOKEN} docker-compose up
+env FASTLY_API_TOKEN=$MY_TOKEN docker-compose up
 ```
 
-Access the [Grafana][grafana] dashboard via http://localhost:3000.
+Access the Grafana dashboard at http://localhost:3000.
 
 ![Fastly Dashboard in Grafana](https://raw.githubusercontent.com/peterbourgon/fastly-exporter/master/compose/Fastly-Dashboard.png)