-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add embedding evaluation model #201
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work Florian! Mostly some comments about adding documentation. And some potential future changes.
|
||
|
||
class DataGeneratorEmbeddingEvaluation: | ||
"""Generates data for training an embedding evaluation model. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SettingsMS2Deepscore is already stored in ms2ds_model, so we could just retrieve the settings from there, instead of passing it as a separate parameter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's true. But somehow also a weakness of the on-for-all setting object. For us that might be handy, but I could imagine that this hides a bit of what's happening to new users. What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, good point, I see how it can be confusing. I do actually not expect many people to dive that deep into the codebase. Some will train new models, but I would expect them to use the wrapper functions that we provide. So to me, the most important people to design for is ourselves in the future. So the design that is clearest to us in 1 or 2 years. Still I do not know which of the two that is... I would use the settings in the model, or I would make a separate evaluatorSettings class. We now save them separately, but still save all the settings of the "other" models in there as well, which is a bit unexpected.
EmbeddingEvaluatorModel
(Inception Time CNN)LinearModel
MS2DeepScoreEvaluated
matchms-style score --> gives "score" and "predicted_absolute_error"