Evaluate Chai-1 Predictions Using Reported Metrics #82
-
|
Hi! I have some predictions generated using Chai-1, and I was wondering if there is an available script that can be used to evaluate these predictions according to the metrics mentioned in the technical report. Any guidance or references to existing resources would be greatly appreciated! Thank you in advance for your help. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
Hi Cai, I'm not aware about any off-the-shelf scripts for evaluation, but we heavily rely on posebusters, they provide CLI and pythonic API, see documentation and official dockq code. There is an initiative openstructure with a goal of making evaluation process simple and reproducible for everyone (which would be great). Unfortunately openstructure doesn't seem to be well-tested, as we observed major discrepancies between their predictions and official TMScore implementation, so we decided to not rely on it. With previous in mind, I think it's the best starting point. |
Beta Was this translation helpful? Give feedback.
Hi Cai,
I'm not aware about any off-the-shelf scripts for evaluation, but we heavily rely on posebusters, they provide CLI and pythonic API, see documentation and official dockq code.
There is an initiative openstructure with a goal of making evaluation process simple and reproducible for everyone (which would be great). Unfortunately openstructure doesn't seem to be well-tested, as we observed major discrepancies between their predictions and official TMScore implementation, so we decided to not rely on it. With previous in mind, I think it's the best starting point.