Port in plotting/analysis functionality from alchemical-analysis and yank

[davidlmobley]

In recent discussions with OpenEye about free energy calculations on their Orion cloud computing platform, it became clear they will benefit from an open analysis library (alchemlyb in particular) because users are running calculations with Yank, GROMACS, and probably soon AMBER on their platform. Thus they are likely prepared to invest some developer time in ensuring alchemlyb has the functionality needed. I was recently discussing this with Christopher Bayly ( @cbayly13 ) and Gaetano Calabro ( @nividic ) there and I said I'd summarize what I thought needed to be done on the issue tracker so that @nividic could begin work in the coming weeks if everyone here is on board with it.

I think the main things that I think we would want added to analysis in the short term is:

1. Overlap matrix analysis from alchemical-analysis, e.g. as in DOI 10.1007/s10822-015-9840-9
2. Mixing diagrams as seen in Yank's simulation health reports
3. Graphs of equilibration/number of effective samples as in Yank's simulation health reports
4. Numerical analysis of statistical inefficiency/number of effective samples in input datasets
5. Any other key features of Yank simulation health reports I'm missing? (I don't have a report in front of me at the moment.)

Possibly also there is a need for ensuring at least a threshold minimal number of samples are retained for analysis after decorrelation.

For Orion purposes I would also suggest running multiple analysis methods (BAR, MBAR, TI) whenever the data allows for it (often) and cross-checking results for consistency; inconsistencies are usually a warning of problems.

An additional list of possible changes is on the alchemical analysis features list where we were brainstorming about this.

Tagging also @harlor for thoughts as well as @hannahbaumann and @andrrizzi . And of course @dotsdl and @orbeckst .

@nividic the way to get started would be to basically pull the relevant code from alchemical-analysis and yank (allowable by the licenses) and adapt to (improve/generalize for) this.

[mrshirts]

Cool, I'll try to keep posted on this. Automated bootstrap error (better that the internal error estimator in most cases).

[harlor]

I already started point 4. in https://github.com/harlor/alchemlyb/tree/autocorrelation_analysis maybe I'll find time on the weekend to continue.

[luancarvalhomartins]

Dear all.

Is this still being implemented? I am working on something which would benefit from the plotting included in achemical-analysis. I think a can help with the porting.

[dotsdl]

Hi @luancarvalhomartins, welcome to alchemlyb!

I don't believe anyone is working on this at the moment; @harlor may want to share his thoughts. If you're interested in taking on the porting, I'd be happy to work with you in iterating on it.

There is interest beyond what is perhaps already listed here in the nascent FreeEnergyFramework that we will also want to build tooling to support, but I think starting with reproducing what is available from alchemical-analysis is the best place to begin. The patterns developed will certainly shape how we support plotting in other areas to come!

[luancarvalhomartins]

Ok, I have good and bad news about that. Good news is that I ported and expanded parts of alchemical-analysis, specifically plotdFvsTime, plotdFvsLambda1, plotCFM and I am now writing something somewhat similar to Replica Mixing Convergence from YANK. Bad news is that I have done in a temporary way to be able to use on the I am working on. I will write a patch to alchemlyb adding a analysis module, but I think I will do that fully after I finish the task at hand.

@dotsdl , I am writting a separate email so we can coordinate on the port.

[orbeckst]

In the spirit of rapid development, I would suggest just making a quick and dirty PR and then work from there with code reviews (instead of waiting for the "perfect" patch that's correct and 100% fits...). Experience shows that with new code coming into a reasonably established library such as alchemlyb there are many little things that need to be changed and adjusted anyway (and the only people who actually know these things are the developers/code reviewers), so no need to be even close to the final product on the first go round.

[davidlmobley]

Yes, I agree with the rest here. This is something we still want (and the lack of the functionality here means people occasionally ask me for help with alchemical-analysis and I have to decline) but it is not done yet.