Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Include additional classification metrics #88

Open
nrkarthikeyan opened this issue Jun 4, 2019 · 5 comments · Fixed by #477
Open

Include additional classification metrics #88

nrkarthikeyan opened this issue Jun 4, 2019 · 5 comments · Fixed by #477
Assignees
@divyagaddipati
Labels
good first issue Good for newcomers medium Intermediate skill level may be needed metrics Fairness metric

Comments

@nrkarthikeyan
Copy link
Collaborator

Include the following metrics:

  1. Equalized odds difference: max(|FPR_unpriv - FPR_priv|, |TPR_unpriv - TPR_priv|)
  2. Generalized equalized odds difference: max(|GFPR_unpriv - GFPR_priv|, |GTPR_unpriv - GTPR_priv|)
  3. Generalized selection rate: mean score possibly conditioned by the group E[\hat{S}]
@nrkarthikeyan nrkarthikeyan self-assigned this Jun 4, 2019
@nrkarthikeyan nrkarthikeyan removed their assignment Aug 24, 2019
@nrkarthikeyan nrkarthikeyan added the enhancement New feature or request label Apr 21, 2021
@nrkarthikeyan nrkarthikeyan added the good first issue Good for newcomers label Aug 25, 2022
@nrkarthikeyan
Copy link
Collaborator Author

This can be implemented for the "classic":
https://github.com/Trusted-AI/AIF360/blob/master/aif360/metrics/classification_metric.py

as well as the sklearn-compatible:
https://github.com/Trusted-AI/AIF360/blob/master/aif360/sklearn/metrics/metrics.py

versions

@nrkarthikeyan nrkarthikeyan added easy Beginner issues metrics Fairness metric medium Intermediate skill level may be needed and removed enhancement New feature or request easy Beginner issues labels Sep 15, 2022
@hoffmansc hoffmansc mentioned this issue Sep 20, 2022
Draft
2 tasks
@divyagaddipati
Copy link
Contributor

Hi! Working on this issue. Can I get assigned to this?

@divyagaddipati
Copy link
Contributor

what does "generalized selection rate" mean? I couldn't find much about it anywhere

@emilymacq emilymacq mentioned this issue Sep 22, 2023
Open
This was linked to pull requests Sep 22, 2023
Added methods for equalized odds difference #477
Merged
Include additional classification metrics #88 #489
Open
@sne3091 sne3091 removed a link to a pull request Sep 26, 2023
Include additional classification metrics #88 #489
Open
@hoffmansc hoffmansc reopened this Oct 11, 2023
@hoffmansc
Copy link
Collaborator

Status:

Classic:

  • Equalized odds difference
  • Generalized equalized odds difference
  • Generalized selection rate

sklearn:

  • Equalized odds difference
  • Generalized equalized odds difference
  • Generalized selection rate

@emilymacq
Copy link

Hi! This PR addresses the equalized odds difference and generalized equalized odds difference for sklearn.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@divyagaddipati divyagaddipati

Labels
good first issue Good for newcomers medium Intermediate skill level may be needed metrics Fairness metric
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Added methods for equalized odds difference divyagaddipati/AIF360
4 participants
@nrkarthikeyan @divyagaddipati @emilymacq @hoffmansc