Add new diagnostics!

[sweverett]

We should add a new suite of diagnostic tests & plots to help ensure we understand the current state of the pipeline. Here I will add a running list of things we want to add, per module:

Medsmaker:

• PSF e1/e2 across the focal plane (PIFF kind of does this but the plots are hard to read / interpret)
• Stack of PSF residuals across all stars (@mcclearyj)

Metacal:

• Binned shear calibration for g1/g2 (exists but needs to be updated)
• Binned shear calibration for gtan/gx (exists but needs to be turned into a script)
• Pujol and variant tests (@emhuff @andrevitorelli)

Shear Profile

• Causal graph of matches catalog containing truth + SE + mcal + annular cols

Analysis:

• Binned shear calibration (g1g2 & gtgx) across all reals & clusters (@mcclearyj @sweverett )

Anything suggested by others in the comments will be added above.

[sweverett]

Another thing - while this is not the current convention, I suggest that we name all files that run these diagnostics to follow the convention of diagnostic_{name}.py in the submodule directory it pertains to. We should edit existing diagnostics to follow this convention as well.

[mcclearyj]

The master_psf_diagnostic.py script does automatically make a residual star image. Example here:

The master_psf_diagnostic.py code also outputs the results of HSM fits to stars and PSF models to file here:

superbit-metacal/superbit_lensing/tests/psfmaker.py

Line 20 in 968e244

def make_output_table(makers=None,prefix=None,outfile='hsm_fit_result.fits'):

[sweverett]

Is this residual for one star or for the stack of all of them? That is what I intended for the test

[mcclearyj]

It is the mean residual:

superbit-metacal/superbit_lensing/tests/psfmaker.py

Line 125 in de44db4

def make_resid_plot(psf=None,stars=None,outname='star_psf_resid.png',vb=False):

If you are after a stacked residual, that should be easy to implement.