trainer.evaluate of metrics is inconsistent due to batch-wise aggregation

[jhnwu3]

See #conformal-prediction channel in discord for the reference.

[Dreamwavedata]

Hi! I'm interested in working on this as a contribution. Is it still available?

[jhnwu3]

Yeah, happy to take any help @Dreamwavedata

[Dreamwavedata]

@jhnwu3
Thank you! Is there more information on this bug?