You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, if votes is not provided to Lighthouse.evaluation_metrics_row, it outputs a row with per_class_IRA_kappas and multiclass_IRA_kappas fields defined as missing. If this is a multiclass result, then passing in such a row to evaluation_metrics_plot fails because it doesn't expect these fields to exist when not dealing with multi-rater metrics.
For example, for a 5 class EvaluationRow, it throws:
ERROR: DimensionMismatch("arrays could not be broadcast to a common size; got a dimension with lengths 12 and 8")
The text was updated successfully, but these errors were encountered:
alexmchan
changed the title
evaluation_metrics_plot fails when IRA fields are in the input EvaluationRowevaluation_metrics_plot fails when IRA fields are in the input EvaluationRow for multiclass result
May 19, 2022
yet another reason that it will be exciting to do #71---plots shouldn't ask for more than they truly need/expect to plot. none of this evaluation_metrics_plot behemoth nonsense.
Currently, if
votes
is not provided toLighthouse.evaluation_metrics_row
, it outputs a row withper_class_IRA_kappas
andmulticlass_IRA_kappas
fields defined asmissing
. If this is a multiclass result, then passing in such a row toevaluation_metrics_plot
fails because it doesn't expect these fields to exist when not dealing with multi-rater metrics.For example, for a 5 class
EvaluationRow
, it throws:The text was updated successfully, but these errors were encountered: