VIP 1.0.0 onwards - summary of major changes

[VChristiaens]

Dear VIP community,

This announcement aims to summarise the major changes brought to VIP since v1.0.0 and as of v1.1.3 (a more detailed, but unsorted, list is available by inspection of each release description):

• Major restructuration of the VIP tree since v1.1.0:
  • exlib: removed, its only function has now been included in the var module;
  • fm: all functions related to forward modeling have been moved to this new module (including negfc, fake companion and fake disk injection routines);
  • invprob: ANDROMEDA was moved to this new module - awaiting more inverse-problem based approach algorithms;
  • psfsub: all PSF modeling+subtraction algorithms have been moved to this module, instead of having their own module;
  • specfit: exported out of VIP, expanded, renamed and converted into its own GitHub repository and package, for spectral characterization of directly imaged companions;
• More tutorials, going into more details, covering a larger fraction of the package, and now visualizable in the documentation:
  1. Quick start
  2. Pre-processing
  3. PSF modeling and subtraction
  4. Metrics (S/N ratio, contrast curves...)
  5. Forward modeling of planets
  6. Forward modeling of disks
  7. FFT-based vs. interpolation-based image operations
  8. Object-oriented framework for VIP
• Conventions update:
  • image operations (rotation, scaling, shifts) now use FFT-based methods by default, instead of interpolation-based opencv (open-cv is faster but much less accurate; see the tests in tutorial 7). Set imlib='opencv' in the call to any routine involving image rotation to experience the same behaviour as before.
  • for even-size images, the center is assumed to be on size//2 in 0-based index (motivated by default FFT-based method). The centering convention is identical as before for odd-size images (central pixel).
• New functions:
  • Significant additions to the fm/scattered_light_disk class (new subclasses and routines) to allow for the creation of better scattered-light models of debris disks;
  • fm/speckle_noise_uncertainty: function to estimate the residual speckle noise uncertainty affecting the parameters of a companion candidate (details in Wertz et al. 2017);
  • metrics/significance: function to convert a S/N ratio (Student statistics) into a Gaussian sigma (in terms of equivalent false positive fraction);
  • preproc/cube_fix_badpix_with_kernel: correction of bad pixels with a user-defined kernel;
  • preproc/find_scal_vector: finds optimal scaling factor between images at different wavelengths;
  • preproc/rotate_fft: FFT-based image rotation (Larkin et al. 1997);
  • preproc/scale_fft: FFT-based image scaling;
  • psfsub/nmf_annular: annular version of NMF with similar options as pca_annular;
• Functions with modified default behaviour:
  • fm/negfc_mcmc_sampling: the default likelihood expression was updated;
  • fm/firstguess: larger range of flux tested for grid search;
  • preproc/cube_collapse: can now handle NaN values;
  • psfsub/nmf: better handling of zero and negative values in input;
  • psfsub/pca_annular: default behaviour is to perform ADI+RDI when a reference cube is provided (set delta_rot to a large value for pure RDI);
  • var/frame_filter_lowpass and var/frame_filter_highpass: can now handle NaN values;
• A number of bugs fixed, among the major ones which may have affected your experience:
  • preproc/frame_center_radon: cube recentering via Radon transform should be fixed (as of v1.1.1);
  • Derotation of a cube of images done in multiprocessing: missing minus sign was leading to opposite direction of (de)rotation (prior to v1.0.3);
  • image rotation using skimage (affects all skimage versions >~ 0.19; now forced to be <~0.18.3 upon VIP installation).
  • vip_ds9: longer waiting time allowed when loading ds9 window before throwing an error (prior to v1.0.3)
• Functions with new features and options: all PCA routines, all bad pixel correction routines, and fits/write_fits,
• Major documentation update (now triggered at each push or pull request);
• More automated tests (now triggered at each push or pull request via GitHub Actions);
• Updated coverage (triggered at each push or pull request via GitHub Actions).

[VChristiaens]

This announcement aims to summarise the major changes brought to VIP between v0.7.0 and v1.0.0, as requested by our JOSS editor to further motivate the need for a new publication (a more detailed, but unsorted, list is available by inspection of each release description):

• New functions/modules:
  • Implementation of an automated testing scheme which peaked at 45% coverage of the whole package before v1.0.0;
  • Introduction of a new object-oriented framework (Introducing new object oriented API (classes HCICube, HCIFrame);
  • Addition of the ANDROMEDA algorithm (Cantalloube et al. 2015)
  • Addition of a least-square (LOCI) post-processing function (Lafrenière et al. 2007);
  • Addition of a ScatteredLightDisk class to generate synthetic circumstellar disks - useful for forward modeling;
  • New functions cube_inject_fakedisk and cube_inject_trace, to inject disk models or elongated traces in ADI cubes;
  • Addition of a receiver-operator class (ROC) in the metrics module;
  • Addition of a new create_synth_psf function, to create synthetic PSFs;
  • New 2D fitting function available: Airy disk (in addition to existing fitting functions to Gaussian and Moffat models);
  • New cube_shift function for the shifting of all images in a 3D cube.
• Functions with modified default behaviour or new features and options:
  • vip.pca.pca: all PCA-, LOCI- and median-based PSF subtraction routines were re-written for compatibility with 4D (ADI+IFS) data, instead of compability for 3D ADI data only. In particular, PCA-ASDI is available in both a single or double step (see e.g. Christiaens et al. 2019). The default behaviour now depends on the value of input parameters cube_ref, scaling_list and ncomp;
  • LLSG function was rewritten for better efficiency;
  • PSF normalization (normalize_psf) function now includes internal centering;
  • Improved automatic detection function.
• A number of bugs were fixed, among the major ones which may have affected your experience:
  • Fixed bug for even-sized frames in all recentering functions;
  • Fixed multiprocessing for annular PCA, LLSG and snrmap functions;
  • Fixed frame_diff function;
  • Fixed cube_correct_nan function.

[VChristiaens]

This announcement aims to summarise the major changes brought to VIP between v1.1.3 and v1.3.1 (a more detailed, but unsorted, list is available by inspection of each release description):

• New functions/modules:
  • Addition of the KLIP and LOCI Forward Model Match Filter algorithms algorithm by @chdahlqvist (Ruffio et al. 2017; Dahlqvist et al. 2021a);
  • Addition of the PACO Algorithm by @nenasedk (Flasseur et al. 2018);
  • Introduction of completeness curve and completeness map functions by @chdahlqvist (Dahlqvist et al. 2021b);
  • Addition of a new bad pixel identification algorithm specific to IFS data by @VChristiaens (leveraging the radial expansion of the PSF for better identification of bad pixels).
  • Addition of new tests to the test suite to bring the coverage to ~65% of all code lines in the package by @VChristiaens, @jmilou, @nenasedk and @chdahlqvist.
• Functions with modified default behaviour or new features and options:
  • Addition of an iterative spectral deconvolution algorithm for bad pixel correction (Aach & Metzler 2001) to preproc/cube_fix_badpix_interp (previously namedpreproc/cube_fix_badpix_with_kernel) by @VChristiaens - the Gaussian and PSF kernel correction can still be activated with the mode parameter;
  • All PCA functions can now run PCA-ADI channel per channel for input 4D IFS cube (i.e. without necessarily using SDI) by @VChristiaens;
  • Radon-transform based centering significantly improved with several alignment iteration and adaptive precision at each iteration by @VChristiaens;
  • Improved preproc/frame_shift algorithm with 'vip-fft' (default) method: sub-px and integer parts of the shift are done separately by @VChristiaens;
  • Updated fake planet injection routine (new options + improved internal shift procedure) by @VChristiaens.
• A number of bugs were fixed, among the major ones which may have affected your experience:
  • fm/cube_inject_companions: better treatment of radial transmission when provided + new optional argument radial_gradient by @VChristiaens