Absence of foreground triggers causing error in statmap jobs

[Prasia-Pankunni]

While doing MDC search, for some chunks, I am getting the error:

File "/home/prasia.p/o4_pycbcv2.3.7/bin/pycbc_coinc_statmap", line 325, in <module>
    ifar_foreground = ifar_exc
NameError: name 'ifar_exc' is not defined

This is in the directory /home/prasia.p/automation_test/MDC/MS220425h/chunk15 and in the osg2 node.
The executable is submitdir/work/O4a-main.dax_O4a-main/statmap/statmap_ID16_ID0000461.sh.

It seems like there are no enough clustered foreground triggers. However, as @spxiwh confirms, this code is supposed to work in this case. Could you please suggest what could be done for such cases ? This happens or a few number of chunks.

[spxiwh]

@GarethCabournDavies Can you take a look at this? coinc_statmap definitely seems to have a failure mode if no foreground triggers are present, which is not desired.

[GarethCabournDavies]

Okay, I see whats gone wrong - I'll add a patch, which can go onto both master and v23 branch

[GarethCabournDavies]

Tested patch works with coinc_statmap - I will need to test on later codes though

[GarethCabournDavies]

@Prasia-Pankunni could you check whether the patch in #4908 allows the full workflow to run, or does it hit a snag downstream?

[Prasia-Pankunni]

@Prasia-Pankunni could you check whether the patch in #4908 allows the full workflow to run, or does it hit a snag downstream?
@GarethCabournDavies
Yes, I can check..

Edit 1: The statmap job runs fine now after the change. I'll also wait till the worklow finish and see if this may cause any further problems

[Prasia-Pankunni]

@GarethCabournDavies I am not sure whether it is related to this issue directly or whether I need to address this separately but after the completion of statmap jobs(and a few other jobs) I am getting errors:

Traceback (most recent call last):
  File "/home/prasia.p/o4_pycbcv2.3.7/bin/pycbc_page_snrchi", line 60, in <module>
    r = numpy.logspace(numpy.log(chisq.min()), numpy.log(chisq.max()), 300)
  File "/home/prasia.p/o4_pycbcv2.3.7/lib/python3.9/site-packages/numpy/core/_methods.py", line 45, in _amin
    return umr_minimum(a, axis, None, out, keepdims, initial, where)
ValueError: zero-size array to reduction operation minimum which has no identity

in plot snrchi

AND another error in plot singles:

Traceback (most recent call last):
  File "/home/prasia.p/o4_pycbcv2.3.7/bin/pycbc_plot_singles_vs_params", line 158, in <module>
    max_z = z.max() if opts.max_z is None else opts.max_z
  File "/home/prasia.p/o4_pycbcv2.3.7/lib/python3.9/site-packages/numpy/core/_methods.py", line 41, in _amax
    return umr_maximum(a, axis, None, out, keepdims, initial, where)
ValueError: zero-size array to reduction operation maximum which has no identity

where there are '0' triggers after vetoing and hence throwing this error.

[GarethCabournDavies]

The page_snrchi error is unusual - so there are no un-vetoed triggers at all in the single-detector file? If so, I'm not sure that this is a use-case we really want to support.
I think you could also try reducing --min-snr to 5?

For plot_singles_vs_params, this works on main branch, but not on the v23_release_branch which you are working on. I would again suggest to reduce --min-z on your config to 5, as that should fix the problem in your case on the v23 release branch.

[GarethCabournDavies]

I'm happy that the error is gone now though, so merging #4908 will close this issue