Skip to content

v0.0.8: Independent evaluation and k_values supported; fixed bugs & use load_sbert in mnrl and evaluation

Compare
Choose a tag to compare
@kwang2049 kwang2049 released this 20 Dec 00:08
· 28 commits to main since this release

Independent evaluation and k_values supported

One can now run the gpl.toolkit.evaluation directly. Previously, it was only possible as part of the whole gpl.train workflow. Please check this example for more details.

And we have also added argument k_values in gpl.toolkit.evaluation.evaluate. This is for specifying the K values in "nDCG@K", "recall@K", etc.

Fixed bugs & use load_sbert in mnrl and evaluation

Now almost all methods that require a separation token has this argument called sep (previously it was fixed as a blank token " "). Two exceptions are mnrl (a loss function in SBERT repo, also the default training loss for the QGen method) and qgen, since they are from the BeIR repo (we will update the BeIR repo in the future if possible).