20230814 developer call notes

[hammer]

Pre-notes

• Notes from previous meeting

PRs

• sgkit#1080 Test on Python 3.11
• sgkit#1104 Add display_pedigree method

Issues

• sgkit#1107 Gene-drop estimation of genotype probabilities
• sgkit#1109 Windows build is failing
• sgkit#1110 Upstream build is failing
• Zarr integrity
  • zarr-python#486
  • zarr-python PR#489
  • zarr-python disc#1480
• Publication
  • https://github.com/pystatgen/sgkit-publication

Notes

Attendees

1. Jeff Hammerbacher
2. Tom White
3. Jerome Kelleher
4. Ben Jeffery
5. Rafal Wojdyla

Discussion

• sgkit#1109
  • TW: root cause is pandas 2
  • JH: let’s link to upstream issue
• sgkit#1080
  • TW: Numba has an edge case, no sign of moving upstream
• sgkit#1110
  • TW: next release of Numba/NumPy should fix
  • JK: What's the status of the validation build? It’s been broken for a while, should we just turn it off?
  • RW: validation is motivated by checking accuracy of methods
  • JH: let’s make validation “as of” vs continuous: sgkit#1112
  • JH: what’s the best way to track all of these workarounds?
  • BJ: tie issues to next release milestone
• sgkit#1104
  • Ready to merge once Windows fix is in
• sgkit#1107
  • JK had some good feedback for TM on it
• Zarr integrity
  • zarr-python#486: workaround broken by fsspec; fix it?
  • zarr-python PR#489: fixes issue in Zarr itself
  • zarr-python disc#1480: meeting in 2 days to discuss refactoring where this could be discussed. TW and JK to attend.
  • BJ: variable-length strings?
  • TW: variable-sized chunks: there’s a PR under review that implements it! Will be helpful for VCF parsing.
  • TW: filters/compressors being Python-centric is not great
• CLI
  • JK worked on it previously sgkit#53 → sgkit PR#57
  • TW: Dask progress bar is a different API for local and distributed
  • RW: https://typer.tiangolo.com/ is somewhat promising vs fire/click etc.
• Publication
  • JK writing on history of 3 fields
  • JK using detailed pedigree data from French Canadians to validate scalability
  • Fig 1: data model from EC needs some polish
  • Fig 2: performance at scale
  • JH: reminder: Liande Li’s work: thesis, repo
  • JK hit bug sgkit-publication PR#32 that TW will look into

[Mykkkosd]

write anr