Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Matthews Correlation Coefficient #561

Merged
merged 22 commits into from
Mar 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
1ebf921
Add Matthews Correlation Coefficient (MCC) metric
fcogidi Feb 2, 2024
f6987e1
Reorder arguments in _multilabel_mcc_reference function
fcogidi Feb 2, 2024
5614997
update linux version for code check
fcogidi Feb 7, 2024
81c7e7e
revert ubuntu version change & update action versions
fcogidi Feb 7, 2024
be63d4a
use torch instead of dlpack to convert tensors to numpy
fcogidi Feb 7, 2024
b6651f4
update numpy conversion methods
fcogidi Feb 7, 2024
2834deb
set dtype for multilabel test inputs
fcogidi Feb 7, 2024
6262b31
Merge branch 'main' into add_mcc
amrit110 Feb 11, 2024
624a540
Merge branch 'main' into add_mcc
fcogidi Feb 12, 2024
8c380c8
fix doctest error
fcogidi Feb 12, 2024
dfa89cf
update implementation of multilabel confusion matrix
fcogidi Feb 16, 2024
2612934
revert implementation update & add print statements for debugging
fcogidi Feb 16, 2024
4e17d12
fix tests
fcogidi Feb 16, 2024
1d664fa
Merge branch 'main' into add_mcc
fcogidi Feb 16, 2024
924514b
Merge branch 'main' into add_mcc
amrit110 Feb 20, 2024
668ac1f
use float64 for internal computations
fcogidi Feb 20, 2024
18b0686
Refactor test files and remove debug print statement
fcogidi Feb 20, 2024
7c33227
Fix data type in Matthews correlation coefficient calculation
fcogidi Feb 20, 2024
65b60ac
Update dtype in _mcc_reduce function
fcogidi Feb 21, 2024
00e99f1
Fix NaN handling in confusion matrix computation
fcogidi Feb 21, 2024
b39128c
use epsilon to check `denom` to avoid precision issues
fcogidi Mar 5, 2024
fe5c8c1
Merge branch 'main' into add_mcc
fcogidi Mar 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions cyclops/evaluate/metrics/experimental/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,11 @@
)
from cyclops.evaluate.metrics.experimental.mae import MeanAbsoluteError
from cyclops.evaluate.metrics.experimental.mape import MeanAbsolutePercentageError
from cyclops.evaluate.metrics.experimental.matthews_corr_coef import (
BinaryMCC,
MulticlassMCC,
MultilabelMCC,
)
from cyclops.evaluate.metrics.experimental.metric_dict import MetricDict
from cyclops.evaluate.metrics.experimental.mse import MeanSquaredError
from cyclops.evaluate.metrics.experimental.negative_predictive_value import (
Expand Down
1 change: 1 addition & 0 deletions cyclops/evaluate/metrics/experimental/confusion_matrix.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Confusion matrix."""

from types import ModuleType
from typing import Any, Optional, Tuple, Union

Expand Down
5 changes: 5 additions & 0 deletions cyclops/evaluate/metrics/experimental/functional/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,11 @@
from cyclops.evaluate.metrics.experimental.functional.mape import (
mean_absolute_percentage_error,
)
from cyclops.evaluate.metrics.experimental.functional.matthews_corr_coef import (
binary_mcc,
multiclass_mcc,
multilabel_mcc,
)
from cyclops.evaluate.metrics.experimental.functional.mse import mean_squared_error
from cyclops.evaluate.metrics.experimental.functional.negative_predictive_value import (
binary_npv,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
"""Functions for computing the confusion matrix for classification tasks."""

# mypy: disable-error-code="no-any-return"
import warnings
from types import ModuleType
from typing import Literal, Optional, Tuple, Union

Expand Down Expand Up @@ -40,6 +42,16 @@
if normalize == "all":
return safe_divide(confmat, xp.sum(confmat, axis=(-1, -2), keepdims=True))

nan_elements = int(0 or apc.size(confmat[xp.isnan(confmat)]))
if nan_elements:
confmat[xp.isnan(confmat)] = 0
warnings.warn(

Check warning on line 48 in cyclops/evaluate/metrics/experimental/functional/confusion_matrix.py

View check run for this annotation

Codecov / codecov/patch

cyclops/evaluate/metrics/experimental/functional/confusion_matrix.py#L47-L48

Added lines #L47 - L48 were not covered by tests
f"Encountered {nan_elements} NaN elements in the confusion matrix. "
"These elements were replaced with 0.",
category=RuntimeWarning,
stacklevel=1,
)

return confmat


Expand Down
Loading
Loading