Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include share values in rewards manifest #827

Merged
merged 36 commits into from
Jul 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
3427ddf
output share values in rewards manifest for iot and mobile
andymck Jun 7, 2024
5986102
tweaks
andymck Jun 7, 2024
57609a1
fmt
andymck Jun 7, 2024
06d655e
Store coverage points received by boosting separately
michaeldjeffrey Jun 26, 2024
5bbdeda
Add struct for containing allocated Rewards for Data transfer and POC
michaeldjeffrey Jun 27, 2024
b9e360b
Move calucating reward shares for poc to coverage point calculator
michaeldjeffrey Jun 27, 2024
29a4351
Correct points for share calculation alway includes multipliers
michaeldjeffrey Jun 27, 2024
97d44eb
Use the AllocatedRewardShare struct for calculating unallocated data …
michaeldjeffrey Jun 27, 2024
64c9c0f
Add caller to timeout message for file sink during debugging
michaeldjeffrey Jun 27, 2024
78ca743
Fix first hex_boosting intregration test
michaeldjeffrey Jun 27, 2024
68ab48a
Fix second hex_boosting integration test
michaeldjeffrey Jun 27, 2024
eeebb65
Fix third hex_boosting integration test
michaeldjeffrey Jun 27, 2024
63eb9eb
Fix fourth hex_boosting integration test
michaeldjeffrey Jun 28, 2024
deeee63
Use new proto branch
maplant Jun 28, 2024
1a48209
Merge remote-tracking branch 'origin/main' into andymck/includes-shar…
maplant Jun 28, 2024
c96c9a1
Overview all changed tests
michaeldjeffrey Jun 28, 2024
870e653
Move reward share calculation where it belongs
michaeldjeffrey Jun 28, 2024
a329c82
Add doc for CoveragePoints::coverage_points
michaeldjeffrey Jun 28, 2024
192e2a5
remove whitespace
michaeldjeffrey Jun 28, 2024
4c92270
Unwrap calculating expected rewards in tests
michaeldjeffrey Jun 28, 2024
cc7a514
Update formatting of comment for Doc.rs readability
michaeldjeffrey Jul 1, 2024
758cb41
Fix broken doclink
michaeldjeffrey Jul 1, 2024
5a41baf
fully expose `HexPoints` members
michaeldjeffrey Jul 1, 2024
16f35b3
Add reward data to manifest struct
Jul 1, 2024
d9ae734
remove public from helper member
michaeldjeffrey Jul 3, 2024
2b021e0
rename points -> shares for poc rewards
michaeldjeffrey Jul 3, 2024
406cd4e
coverage_points() -> coverage_points_v1() to call out location differ…
michaeldjeffrey Jul 3, 2024
a8a3774
Merge remote-tracking branch 'origin/mj/hip-122-restrict-boosting-fro…
maplant Jul 8, 2024
3222652
Merge branch 'andymck/includes-share-values-in-manifest' of github.co…
maplant Jul 8, 2024
81b495e
Update to latest
maplant Jul 8, 2024
9bc4e1b
Merge remote-tracking branch 'origin/main' into andymck/includes-shar…
maplant Jul 10, 2024
4b39829
Update proto
maplant Jul 10, 2024
953088f
I guess this is more correct?
maplant Jul 10, 2024
6c0b5d8
Rename fields to be more accurate
maplant Jul 11, 2024
6a30efc
Update proto to master
maplant Jul 11, 2024
7e9e2fa
Annoying
maplant Jul 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions file_store/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,10 @@ pub enum DecodeError {
Uri(#[from] http::uri::InvalidUri),
#[error("integer conversion error")]
FromInt(#[from] std::num::TryFromIntError),
#[error("error parsing decimal")]
IntoDecimal(#[from] rust_decimal::Error),
#[error("empty field: {0}")]
EmptyField(&'static str),
#[error("unsupported region, type: {0}, value: {1}")]
UnsupportedRegion(String, i32),
#[error("unsupported datarate, type: {0}, value: {1}")]
Expand Down Expand Up @@ -166,6 +170,10 @@ impl DecodeError {
pub fn unsupported_invalidated_reason<E: ToString>(msg1: E, msg2: i32) -> Error {
Error::Decode(Self::UnsupportedInvalidReason(msg1.to_string(), msg2))
}

pub const fn empty_field(field: &'static str) -> Error {
Error::Decode(Self::EmptyField(field))
}
}

impl From<helium_crypto::Error> for Error {
Expand Down
62 changes: 62 additions & 0 deletions file_store/src/reward_manifest.rs
Original file line number Diff line number Diff line change
@@ -1,12 +1,27 @@
use crate::{error::DecodeError, traits::MsgDecode, Error};
use chrono::{DateTime, TimeZone, Utc};
use helium_proto as proto;
use rust_decimal::Decimal;

#[derive(Clone, Debug)]
pub struct RewardManifest {
pub written_files: Vec<String>,
pub start_timestamp: DateTime<Utc>,
pub end_timestamp: DateTime<Utc>,
pub reward_data: Option<RewardData>,
}

#[derive(Clone, Debug)]
pub enum RewardData {
MobileRewardData {
poc_bones_per_coverage_point: Decimal,
boosted_poc_bones_per_coverage_point: Decimal,
maplant marked this conversation as resolved.
Show resolved Hide resolved
},
IotRewardData {
poc_bones_per_beacon_reward_share: Decimal,
poc_bones_per_witness_reward_share: Decimal,
dc_bones_per_share: Decimal,
},
}

impl MsgDecode for RewardManifest {
Expand All @@ -31,6 +46,53 @@ impl TryFrom<proto::RewardManifest> for RewardManifest {
.ok_or(Error::Decode(DecodeError::InvalidTimestamp(
value.end_timestamp,
)))?,
reward_data: match value.reward_data {
Some(proto::reward_manifest::RewardData::MobileRewardData(reward_data)) => {
Some(RewardData::MobileRewardData {
poc_bones_per_coverage_point: reward_data
.poc_bones_per_reward_share
.ok_or(DecodeError::empty_field("poc_bones_per_coverage_point"))?
.value
.parse()
.map_err(DecodeError::from)?,
boosted_poc_bones_per_coverage_point: reward_data
.boosted_poc_bones_per_reward_share
.ok_or(DecodeError::empty_field(
"boosted_poc_bones_per_coverage_point",
))?
.value
.parse()
.map_err(DecodeError::from)?,
})
}
Some(proto::reward_manifest::RewardData::IotRewardData(reward_data)) => {
Some(RewardData::IotRewardData {
poc_bones_per_beacon_reward_share: reward_data
.poc_bones_per_beacon_reward_share
.ok_or(DecodeError::empty_field(
"poc_bones_per_beacon_reward_share",
))?
.value
.parse()
.map_err(DecodeError::from)?,
poc_bones_per_witness_reward_share: reward_data
.poc_bones_per_witness_reward_share
.ok_or(DecodeError::empty_field(
"poc_bones_per_witness_reward_share",
))?
.value
.parse()
.map_err(DecodeError::from)?,
dc_bones_per_share: reward_data
.dc_bones_per_share
.ok_or(DecodeError::empty_field("dc_bones_per_share"))?
.value
.parse()
.map_err(DecodeError::from)?,
})
}
None => None,
},
})
}
}
42 changes: 34 additions & 8 deletions iot_verifier/src/rewarder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,14 @@ use chrono::{DateTime, TimeZone, Utc};
use db_store::meta;
use file_store::{file_sink, traits::TimestampEncode};
use futures::future::LocalBoxFuture;
use helium_proto::services::poc_lora as proto;
use helium_proto::services::poc_lora::iot_reward_share::Reward as ProtoReward;
use helium_proto::services::poc_lora::{UnallocatedReward, UnallocatedRewardType};
use helium_proto::RewardManifest;
use helium_proto::{
reward_manifest::RewardData::IotRewardData,
services::poc_lora::{
self as proto, iot_reward_share::Reward as ProtoReward, UnallocatedReward,
UnallocatedRewardType,
},
IotRewardData as ManifestIotRewardData, RewardManifest,
};
use humantime_serde::re::humantime;
use price::PriceTracker;
use reward_scheduler::Scheduler;
Expand All @@ -31,6 +35,12 @@ pub struct Rewarder {
pub price_tracker: PriceTracker,
}

pub struct RewardPocDcDataPoints {
beacon_rewards_per_share: Decimal,
witness_rewards_per_share: Decimal,
dc_transfer_rewards_per_share: Decimal,
}

impl ManagedTask for Rewarder {
fn start_task(
self: Box<Self>,
Expand Down Expand Up @@ -116,7 +126,8 @@ impl Rewarder {
let reward_period = &scheduler.reward_period;

// process rewards for poc and dc
reward_poc_and_dc(&self.pool, &self.rewards_sink, reward_period, iot_price).await?;
let poc_dc_shares =
reward_poc_and_dc(&self.pool, &self.rewards_sink, reward_period, iot_price).await?;
// process rewards for the operational fund
reward_operational(&self.rewards_sink, reward_period).await?;
// process rewards for the oracle
Expand Down Expand Up @@ -145,12 +156,24 @@ impl Rewarder {
transaction.commit().await?;

// now that the db has been purged, safe to write out the manifest
let reward_data = ManifestIotRewardData {
poc_bones_per_beacon_reward_share: Some(helium_proto::Decimal {
value: poc_dc_shares.beacon_rewards_per_share.to_string(),
}),
poc_bones_per_witness_reward_share: Some(helium_proto::Decimal {
value: poc_dc_shares.witness_rewards_per_share.to_string(),
}),
dc_bones_per_share: Some(helium_proto::Decimal {
value: poc_dc_shares.dc_transfer_rewards_per_share.to_string(),
}),
};
self.reward_manifests_sink
.write(
RewardManifest {
start_timestamp: scheduler.reward_period.start.encode_timestamp(),
end_timestamp: scheduler.reward_period.end.encode_timestamp(),
written_files,
reward_data: Some(IotRewardData(reward_data)),
},
[],
)
Expand Down Expand Up @@ -205,13 +228,12 @@ impl Rewarder {
.ok_or(db_store::Error::DecodeError)
}
}

pub async fn reward_poc_and_dc(
pool: &Pool<Postgres>,
rewards_sink: &file_sink::FileSinkClient,
reward_period: &Range<DateTime<Utc>>,
iot_price: Decimal,
) -> anyhow::Result<()> {
) -> anyhow::Result<RewardPocDcDataPoints> {
let reward_shares = reward_share::aggregate_reward_shares(pool, reward_period).await?;
let gateway_shares = GatewayShares::new(reward_shares)?;
let (beacon_rewards_per_share, witness_rewards_per_share, dc_transfer_rewards_per_share) =
Expand Down Expand Up @@ -254,7 +276,11 @@ pub async fn reward_poc_and_dc(
reward_period,
)
.await?;
Ok(())
Ok(RewardPocDcDataPoints {
beacon_rewards_per_share,
witness_rewards_per_share,
dc_transfer_rewards_per_share,
})
}

pub async fn reward_operational(
Expand Down
3 changes: 2 additions & 1 deletion mobile_verifier/src/cli/reward_from_db.rs
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,8 @@ impl Cmd {
DataTransferAndPocAllocatedRewardBuckets::new(&epoch),
&epoch,
)
.ok_or(anyhow::anyhow!("no rewardable events"))?;
.ok_or(anyhow::anyhow!("no rewardable events"))?
.1;
for (_reward_amount, reward) in radio_rewards {
if let Some(proto::mobile_reward_share::Reward::RadioReward(proto::RadioReward {
hotspot_key,
Expand Down
25 changes: 18 additions & 7 deletions mobile_verifier/src/reward_shares.rs
Original file line number Diff line number Diff line change
Expand Up @@ -577,7 +577,10 @@ impl CoverageShares {
self,
reward_shares: DataTransferAndPocAllocatedRewardBuckets,
epoch: &'_ Range<DateTime<Utc>>,
) -> Option<impl Iterator<Item = (u64, proto::MobileRewardShare)> + '_> {
) -> Option<(
CalculatedPocRewardShares,
impl Iterator<Item = (u64, proto::MobileRewardShare)> + '_,
)> {
struct ProcessedRadio {
radio_id: RadioId,
points: coverage_point_calculator::CoveragePoints,
Expand Down Expand Up @@ -615,7 +618,8 @@ impl CoverageShares {
return None;
};

Some(
Some((
rewards_per_share,
processed_radios
.into_iter()
.map(move |radio| {
Expand All @@ -638,7 +642,7 @@ impl CoverageShares {
(poc_reward, mobile_reward_share)
})
.filter(|(poc_reward, _mobile_reward)| *poc_reward > 0),
)
))
}

/// Only used for testing
Expand Down Expand Up @@ -701,10 +705,10 @@ impl DataTransferAndPocAllocatedRewardBuckets {
/// Reference:
/// HIP 122: Amend Service Provider Hex Boosting
/// https://github.com/helium/HIP/blob/main/0122-amend-service-provider-hex-boosting.md
#[derive(Debug)]
struct CalculatedPocRewardShares {
normal: Decimal,
boost: Decimal,
#[derive(Copy, Clone, Debug, Default)]
pub struct CalculatedPocRewardShares {
pub(crate) normal: Decimal,
pub(crate) boost: Decimal,
}

impl CalculatedPocRewardShares {
Expand Down Expand Up @@ -1499,6 +1503,7 @@ mod test {
.unwrap()
.into_rewards(reward_shares, &epoch)
.unwrap()
.1
{
let radio_reward = match mobile_reward.reward {
Some(proto::mobile_reward_share::Reward::RadioReward(radio_reward)) => radio_reward,
Expand Down Expand Up @@ -1673,6 +1678,7 @@ mod test {
.unwrap()
.into_rewards(reward_shares, &epoch)
.unwrap()
.1
{
let radio_reward = match mobile_reward.reward {
Some(proto::mobile_reward_share::Reward::RadioReward(radio_reward)) => radio_reward,
Expand Down Expand Up @@ -1804,6 +1810,7 @@ mod test {
.unwrap()
.into_rewards(reward_shares, &epoch)
.unwrap()
.1
{
let radio_reward = match mobile_reward.reward {
Some(proto::mobile_reward_share::Reward::RadioReward(radio_reward)) => radio_reward,
Expand Down Expand Up @@ -1935,6 +1942,7 @@ mod test {
.unwrap()
.into_rewards(reward_shares, &epoch)
.unwrap()
.1
{
let radio_reward = match mobile_reward.reward {
Some(proto::mobile_reward_share::Reward::RadioReward(radio_reward)) => radio_reward,
Expand Down Expand Up @@ -1976,6 +1984,8 @@ mod test {
.expect("failed gw2 parse");

let now = Utc::now();
// We should never see any radio shares from owner2, since all of them are
// less than or equal to zero.
let epoch = now - Duration::hours(1)..now;

let uuid_1 = Uuid::new_v4();
Expand Down Expand Up @@ -2074,6 +2084,7 @@ mod test {
for (_reward_amount, mobile_reward) in coverage_shares
.into_rewards(reward_shares, &epoch)
.expect("rewards output")
.1
{
let radio_reward = match mobile_reward.reward {
Some(proto::mobile_reward_share::Reward::RadioReward(radio_reward)) => radio_reward,
Expand Down
Loading