Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add first batch of CredCon and Partner initiatives #27

Merged
merged 2 commits into from
Apr 10, 2020

Conversation

Beanow
Copy link

@Beanow Beanow commented Mar 30, 2020

Depends on #26

This is a first iteration

Edit: rather than letting this stall for too long, let's improve on this with follow ups.
Have a look at #29 for info on how to go from here.

This is a preview a first iteration, as it's in need of another weighting round.
Still missing from this, but identified at the team call:

  • CredCon: video interviews (need a user account for zak)
  • CredCon: CredRank prototype (have not decided fibonacci weight)
  • CredCon: Discord prototype (have not decided fibonacci weight)
  • Evan's explainer (have not decided fibonacci weight)

Also great improvements to make from here:

Good but not as critical:

  • Making the timestamps more accurate

Assumptions to validate

Fibonacci to absolute weights: 200x

Edit: see #27 (comment) we'll start out with 100x for this PR, then move to 200x when we're more confident about having solid coverage of connected contributions.

The major point to debate is the weights we should set. In the team call
we've decided on fibonacci scale weights, which I've included in these
files as _fibonacciWeight.

This PR includes a ballpark suggestion of a multiplier to use.
The intention is to "inflate cred" by weighing initiatives more heavily
than plugin-imported contributions so far. While keeping in mind that
more initiatives will follow and inflate cred further. And adding more
contribution edges will "dilute" the share currently flowing to champions.

A multiplier which I think is in this ballpark is: 200x
(e.g. a fibonacci weight of 2 = a node weight of 400)

Dependency structure

For the activities as part of CredCon (like hackathon, party, talks), I
found they had very similar dependencies on logistical initiatives and
CredCon itself. I've generalized this by depending on CredCon, rather than
directly depending on logistics.

Example: CredCon party, depends on CredCon, depends on car+food+lodging

This will make the life of historians easier too, as adding more logicstics
(maybe invites or scheduling) would only need to be set as a CredCon
dependency, rather than for all activities as well.

Example references / contributions

For the main CredCon initiative, I've added several clearly related forum
topics. These are both as an example, and to make sure this part of the
graph is not exclusively connected through identities.

Completion status

Other than both Partner initiatives, the CredCon initiatives are set as
completed.

Technical aspects to validate

The main point which needs careful manual validation, is to inspect the
graph. For example using the "legacy explorer" to confirm the initiative
nodes are added and have edges to all the URLs added.

That means:

  • Champions: Identities / GitHub users / Discourse users.
  • References / Contributions: Forum topics (in CredCon).
  • Dependencies between Initiatives.

Note: it's expected for links to an initiative to give a 404 not found,
until the PR is merged to master. However they should be listed in the
explorer.

@Beanow
Copy link
Author

Beanow commented Mar 30, 2020

Explorer previews

Score comparisons

For lack of a good compare feature, here is a summary of the resulting score differences >1 cred.
(Tip: try renaming the initiatives parameter in the project file to _initiatives to easily disable it).

Reminder: please be mindful not to add possibly hurtful comments, such as by comparing people to each other! These scores are included to check, whether Cred is flowing from initiatives to contributors in a way that seems reasonable.

Contributor Before After Change
discourse/LB 996 3234 2238.2
identity/decentralion 29971 32174 2202.6
identity/hammadj 62 1459 1397
identity/s_ben 3264 4429 1164.6
discourse/flyingzumwalt 100 589 488.9
identity/wchargin 22137 22427 290
identity/Beanow 10235 10429 194.1
identity/BrianLitwin 1603 1689 86
identity/vsoch 4243 4329 85.6
discourse/burrrata 3507 3586 78.9
discourse/1GLENCo 33 76 42.9
identity/amico 162 202 39.6
identity/mzargham 1010 1035 25
discourse/kronosapiens 90 113 22.1
discourse/themathematicianisin 22 29 7.6
discourse/Jessie 123 130 7.3
discourse/nothingismagick 80 87 6.3
discourse/protocol 316 320 4.5
discourse/dillchen 53 57 4.3
discourse/jwaup 46 50 4
identity/ericronne 257 260 3.1
discourse/jessicaschilling 77 79 1.7
discourse/insideNIMA 18 20 1.5
discourse/cslarson 37 39 1.4
github/wudolan 21 18 -2.1
identity/miyazono 504 473 -30.5

Some things which stand out to me.

  • We're clearly missing Cred for people who contributed to CredCon but aren't listed as Champions.
  • There are (expected) "knock-on" effects because Cred flows from the Champions onto other contributions, for example from likes.
  • Substantial Cred flow differences comparing Maker and MetaGame partnerships. In spite of equal weights being set, the reason seems to be how many out-edges the champions have, as removing the dependency on CredCon for MetaGame (which Maker doesn't have) only further increased the differences.
  • I'm confused by the -30.5 for miyazono. Of course by adding the "Evan's explainer" initiative this would become a big positive. However it's curious to see that adding to the graph and minting Cred, as much as 30 Cred managed to flow elsewhere.

Copy link
Contributor

@teamdandelion teamdandelion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be setting the weight in the initiative as the fibonacci weight. Then, we can set a type-level weight (a prefix weight) of 200 on the whole initiative plugin. This way we can tune and rebalance the overall weight of the initiatives without needing to individually modify every initiative.

@Beanow
Copy link
Author

Beanow commented Apr 7, 2020

We should be setting the weight in the initiative as the fibonacci weight. Then, we can set a type-level weight (a prefix weight) of 200 on the whole initiative plugin. This way we can tune and rebalance the overall weight of the initiatives without needing to individually modify every initiative.

I think it's better to use the absolute weights and lean towards 1x multiplier on the type level.
200x works because the initiatives listed here are fairly large ones. For smaller initiatives and contributions you might want to have 10, 50, 100, available as well.

@Beanow
Copy link
Author

Beanow commented Apr 7, 2020

Based on a discussion we had and https://discourse.sourcecred.io/t/rolling-out-initiatives/698 for context, the proposal is to start out with 100x for this PR. Because we know there will be updates and improvements needed to complete it. Then increase this to 200x later as we're more confident we've got good coverage.

As these following updates may cause overshooting, followed by dilution to happen for champions. Because more of the Initiatives Cred would flow to contributors who are currently not added yet.

image

Which is not very enjoyable to see I think. So it's better to do another increase of weights at the end and make it look more like this:

image

@Beanow
Copy link
Author

Beanow commented Apr 7, 2020

Based on the rationale explained in the last comment, added the commit which sets it to 100x.

@Beanow Beanow requested review from teamdandelion and s-ben April 7, 2020 20:43
@Beanow Beanow force-pushed the bean-empty-initiatives branch from cbbe5e5 to 6ad8e51 Compare April 8, 2020 16:14
Beanow added 2 commits April 8, 2020 18:16
## This is a preview

This is a preview, as it's in need of another weighting round.
Still missing from this, but identified at the team call:
- CredCon: video interviews (need a user account for zak)
- CredCon: CredRank prototype (have not decided fibonacci weight)
- CredCon: Discord prototype (have not decided fibonacci weight)
- Evan's explainer (have not decided fibonacci weight)

Also great improvements to make from here:
- Adding contributions and linking that to contributors

Good but not as critical:
- Making the timestamps more accurate


## Assumptions to validate


### Fibonacci to absolute weights: 200x

The major point to debate is the weights we should set. In the team call
we've decided on fibonacci scale weights, which I've included in these
files as `_fibonacciWeight`.

This PR includes a ballpark suggestion of a multiplier to use.
The intention is to "inflate cred" by weighing initiatives more heavily
than plugin-imported contributions so far. While keeping in mind that
more initiatives will follow and inflate cred further. And adding more
contribution edges will "dilute" the share currently flowing to champions.

A multiplier which I think is in this ballpark is: 200x
(e.g. a fibonacci weight of 2 = a node weight of 400)


### Dependency structure

For the activities as part of CredCon (like hackathon, party, talks), I
found they had very similar dependencies on logistical initiatives and
CredCon itself. I've generalized this by depending on CredCon, rather than
directly depending on logistics.

Example: CredCon party, depends on CredCon, depends on car+food+lodging

This will make the life of historians easier too, as adding more logicstics
(maybe invites or scheduling) would only need to be set as a CredCon
dependency, rather than for all activities as well.


### Example references / contributions

For the main CredCon initiative, I've added several clearly related forum
topics. These are both as an example, and to make sure this part of the
graph is not exclusively connected through identities.


### Completion status

Other than both Partner initiatives, the CredCon initiatives are set as
completed.


## Technical aspects to validate

The main point which needs careful manual validation, is to inspect the
graph. For example using the "legacy explorer" to confirm the initiative
nodes are added and have edges to *all* the URLs added.

That means:
- Champions: Identities / GitHub users / Discourse users.
- References / Contributions: Forum topics (in CredCon).
- Dependencies between Initiatives.

Note: it's expected for links to an initiative to give a 404 not found,
until the PR is merged to master. However they should be listed in the
explorer.
The 200x multiplier seems in the right ballpark, however we want to
avoid champions see Cred "dilute" because we'll add more contributions.

Start with 100x now, and move to 200x as we have a more complete coverage.
@Beanow Beanow force-pushed the bean-initiatives-batch1 branch from 70e7693 to 3065811 Compare April 8, 2020 16:17
@Beanow Beanow changed the base branch from bean-empty-initiatives to master April 8, 2020 16:18
Copy link
Contributor

@teamdandelion teamdandelion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good as a start! I'll note that @protocol's funding support is missing here, and will want to add that in. But we already know we need a system for allowing inlined contributions. Let's go with this for now!

Please update the title and description prior to merging.

@Beanow Beanow changed the title Preview: Add a first batch of initiatives to the project Add first batch of CredCon and Partner initiatives Apr 10, 2020
@Beanow
Copy link
Author

Beanow commented Apr 10, 2020

Updated the opening comment of this PR for clarity to mention later conclusions. Have kept it mostly faithful to the original comment though, to keep as reference what these question were.
You can view the changes through here
view comment edits

@Beanow
Copy link
Author

Beanow commented Apr 10, 2020

In the merged commit: 3452820

I've slightly reworded and distilled the conclusions of the discussion in this PR, rather than have the commit message be full of open questions.

@Beanow Beanow deleted the bean-initiatives-batch1 branch April 10, 2020 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants