Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Stratified evaluation of prediction models #8

Open
lhjohn opened this issue Oct 18, 2021 · 0 comments
Open

[Feature] Stratified evaluation of prediction models #8

lhjohn opened this issue Oct 18, 2021 · 0 comments
Labels
enhancement New feature or request

Comments

@lhjohn
Copy link
Collaborator

lhjohn commented Oct 18, 2021

During the coagulopathy and dementia studies at Erasmus MC, we found that it can be useful to not only evaluate models on the full test/validation set, but also in sub-populations of it.

Particularly, evaluation in certain age and sex strata showed significant limitations of a model that were initially masked by evaluating in the full test/validation set. Those limitations had impact on the final recommendations of how to effectively use the model.

I will attempt to implement a simple function to evaluate in custom age and sex strata of a given test/validation set.

@lhjohn lhjohn changed the title Stratified evaluation of prediction models [Feature] Stratified evaluation of prediction models Oct 19, 2021
@lhjohn lhjohn added the enhancement New feature or request label Nov 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant