Add time_buckets to EarningsTracker data

[ksedgwic]

Addresses #227

IMPORTANT - This PR makes a schema change and migrates data!

Abstract

This PR modifies the EarningsTracker schema to hold data in "buckets", one per day. This PR only modifies the collection and storage, the strategies and existing status continue to use earnings data from all time for now.

Motivation

As described in #227 there are several shortcomings w/ the current storage of EarningsTracker data:

1. Ancient data can dilute/overwhelm current data
2. Users would like income/expense reports for specific periods of time (especially recent periods)
3. The data doesn't "heal" as bugs are fixed and strategies are improved.

This PR enables time-based data earnings collection, future PR's should investigate modifying the strategies and reports to take advantage of the ability to filter by time range.

Specification

This PR updates the EarningsTracker table to have a "time bucket" column. The timestamps of incoming fee and balance events are quantized to a time bucket, currently one day.

The existing strategies and reports use the sql SUM to return the same answers that they would with the current schema.

The time and nodeid columns are indexed for efficient operations.

			CREATE TABLE IF NOT EXISTS EarningsTracker
			     ( node TEXT NOT NULL
			     , time_bucket REAL NOT NULL
			     , in_earnings INTEGER NOT NULL
			     , in_expenditures INTEGER NOT NULL
			     , out_earnings INTEGER NOT NULL
			     , out_expenditures INTEGER NOT NULL
			     , PRIMARY KEY (node, time_bucket)
			     );
			CREATE INDEX IF NOT EXISTS
			    idx_earnings_tracker_node_time ON EarningsTracker (node, time_bucket);
			CREATE INDEX IF NOT EXISTS
			    idx_earnings_tracker_time_node ON EarningsTracker (time_bucket, node);

Existing legacy data is migrated into the table with a time bucket value of 0. This allows it to be considered and used but eventually be aged out.

Two new CLI commands are added to allow inspecting the new data:

• clboss-recent-earnings:

  • Purpose: Returns a data structure equivalent to the
    offchain_earnings_tracker collection in clboss-status, but
    only includes recent earnings and expenditures.
  • Arguments:
    • days (optional): Specifies the number of days to include in
      the report. Defaults to a fortnight (14 days) if not provided.

• clboss-earnings-history:

  • Purpose: Provides a daily breakdown of earnings and expenditures.
  • Arguments:
    • nodeid (optional): Limits the history to a particular node if
      provided. Without this argument, the values are accumulated
      across all peers.
  • Output:
    • The history consists of an array of records showing the earnings
      and expenditures for each day.
    • The history includes an initial record with a time value of 0,
      which contains any legacy earnings and expenditures collected by
      CLBOSS before daily tracking was implemented.

[tsjk]

From the users perspective - is anything needed to be done when upgrading to a version that includes this? Or is it "just" that once the upgrade is carried out, downgrading requires some extra manual labor.

[ksedgwic]

Nothing needs to be done by the user for forward migration. Downgrading is possible (w/o loss of any data) but would need to be done manually or by future code.

[ksedgwic]

This is subsumed by #230