Skip to content

[scfinder] Add clipping handling to PGA processing #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Nov 22, 2024

Conversation

FMassin
Copy link
Member

@FMassin FMassin commented Sep 30, 2024

Prevents the use of clipped data in further processing and adds additional logging and select the appropriate envelope for PGA processing based on instrument gain units.

Before, scfinder would process with FinDer the maximum PGA over all the channels (vertical or horizontal) of all the colocated instruments.

Ambiguituous processing of vertical and horizontal PGA remains.

Key Changes:

  1. Instrument Gain Unit Check and Prevailing (M/S**2 by default):
    • Integrated logic to check and log whether PGA data is from an accelerometer based on the instrument's gain unit (i.e., "M/S**2").
    • Adjusted the PGA processing to ensure data from different instruments (e.g., accelerometers vs. velocimeters) at the same location is handled correctly, i.e., M/S**2 prevails on others by default.

Updates in the updateMaximum Function (updates the PGA):

  • Prevailing of preferred gain units over the others .
  • Considering of clipped records while updating the maximum PGA.
  1. Discarding of Clipped data:
    • Clipping is now detected, logged, and skipped in the PGA buffer processing.
    • Added logic to skip channels recently clipped within the clipTimeout window.
    • Modified log statements to include clipping status where necessary.

New clipTimeout Parameter:

  • Added a new configuration parameter, clipTimeout, in scfinder.xml to specify the delay after which clipped envelope are excluded from processing.
  • Default value set to 30 seconds.
  • Similar to scvsmag's clipTimeout

Updates in the Amplitude Structure (used for envelopes and PGAs):

  • Tracking of whether the data is clipped via a clipped boolean flag, and when that last occured via the lastclipped timestamp.
  • Added gainUnit string to keep track of the channel gainUnit.
  1. Improved Debug Logging:
    • Logging clipped records and skipping logic.
    • Logging when amplitude is discarted to prefer another data gain units.
    • Logging gain unit information when available.

Unit Test

The two new features can be tested with the unit test data as follows:

scfinder --playback --offline --inventory-db file:///usr/local/src/seiscomp/src/base/sed-addons/test/vs/inventory.xml  --test -I file:///usr/local/src/seiscomp/src/base/sed-addons/test/vs/sorted.mseed --debug
  1. Instrument Gain Unit Check and Prevailing (M/S**2 by default):

    • Prevailing logic logging for each collocated instrument when PGA is updated, e.g.: HG [M/S**2] rules over HH [M/S]
    • Setting "" (empty) instead of M/**2 as finder.preferredGainUnits in scfinder.cfg switches the revailing of any unit off as can be seen in temp_data/*/data_* files.
    • Setting M/S instead of M/**2 as finder.preferredGainUnits in scfinder.cfg switches collocated instrument usage from accelerometers to velocimeters as can be seen in temp_data/*/data_* files (e.g., 20 HG less in data_2).
  2. Discarding of clipped data:

    • Log message for each clipped PGA that has been skipped: Instrument clipped and skipped
    • Log message after clipped PGA for 30s: Instrument clipped recently and skipped
    • Setting 2 as saturationThreshold in scfinder.cfg allows 4 station contributions less as can be seen in temp_data/*/data_* files.

@FMassin FMassin changed the title [scfinder] add support for clipping [scfinder] Add clipping handling to PGA processing Oct 7, 2024
@FMassin FMassin self-assigned this Oct 9, 2024
@FMassin FMassin requested a review from maboese October 9, 2024 22:23
@FMassin FMassin added the enhancement New feature or request label Oct 9, 2024
@FMassin FMassin added this to the dtgeo milestone Oct 9, 2024
@FMassin FMassin linked an issue Oct 9, 2024 that may be closed by this pull request
@@ -8,7 +8,7 @@
<configuration>
<parameter name="saturationThreshold" type="double" default="80" unit="percent">
<description>
Saturation threshold in percent of 2**31 of raw values (COUNTS).
Saturation threshold in percent of 2**23 of raw values (COUNTS).
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jenrandrews Saturation reference value is already fixed.

@FMassin FMassin merged commit 3fc4828 into SED-EEW:master Nov 22, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[scfinder] configurable management of clipped channels
1 participant