Missing data in generated measures

[GregEyd]

My study looks at patients within a care home setting who have been prescribed antipsychotics and antidepressants over a period of 2 years. I seem to have an issue with my measures when I group by region, specifically for London. I have data missing for the months of Aug 2020 onwards. In the measures files, this just appears as a blank numerator, the denominator is still correct. I know this issue is not due to small number suppression as the numerator should be in the hundreds or thousands. Strangely this does not seem to occur in my control data which looks at patients outside of the care home setting.

Here is a link to my repo where all the analysis scripts can be found

My population is defined in the following way

    population=patients.satisfying(
        """
        (age >= 65 AND age < 120) AND 
        is_registered_with_tpp AND 
        (sex = "M" OR sex = "F") AND 
        (care_home_type = "PC" OR care_home_type = "PN" OR care_home_type = "PS") 
        """,
        is_registered_with_tpp=patients.registered_as_of(
          "index_date"
        ),
    ),

For the control data the population is identical except care_home_type=NC

Care home status is defined using the 'variable care_home_status_as_of' using the logic seen in this paper

    care_home_type=patients.care_home_status_as_of(
        "index_date",
        categorised_as={
            "PC": """
              IsPotentialCareHome
              AND LocationDoesNotRequireNursing='Y'
              AND LocationRequiresNursing='N'
            """,
            "PN": """
              IsPotentialCareHome
              AND LocationDoesNotRequireNursing='N'
              AND LocationRequiresNursing='Y'
            """,
            "PS": "IsPotentialCareHome",
            "U": "DEFAULT",
        },
....

Again for the control data this contains "NC": "NOT IsPotentialCareHome"

See below for an example of a measure that is not working. In this example, antidepressent_ssri is just defined as a patient being prescribed an antidepressent from a codelist within 3 months prior to the index date. Region is just the nuts1 region from the registered practice. As mentioned previously, I get the full 2 years for every region for my control data (non care home) but missing August 2020 onwards for London for the test data

    Measure(
        id="ad_prescribing_rate_region",
        numerator="antidepressent_ssri",
        denominator="population",
        group_by = ["region"],
        small_number_suppression=True
    ),

[GregEyd]

Hi

After some extensive testing, I believe the issue is due to the "small_number_suppression=True" flag in the study definition. Based on my understanding, suppression is only supposed to apply to numerator and denominator values under 5 but I'm getting numbers suppressed when they are above 50 in some cases.

Interestingly, I've got cases where numerator values of 36, 39 and 40 are displayed for the first 3 months of my data but subsequent ones of 40+ are blanked out. I've also got denominators of 490 and 545 being blanked out when suppression is turned on.

Looking over the Github issues I believe my issue is the same as this one

[iaindillingham]

Thanks for your question, @GregEyd. Small number suppression is a two-step process:

1. Numerators and denominators that are greater than zero and less than or equal to five are replaced with nulls, and messages beginning Suppressed small numbers in column are written to the log.

2. If the sum of suppressed values is less than or equal to five, then all values equal to the next smallest value are replaced with nulls, and messages beginning Additional suppression in column are written to the log.

Two actions in your study generate measures and suppress small numbers: generate_measures_control and generate_measures. Could you search the logs for messages to verify that (additional) small number suppression has occurred? (I'm afraid I don't have L4 EMIS access, so can't verify this on your behalf.)

Could it be the case that your example of a numerator/denominator of greater than 50 being suppressed is because of additional small number suppression?

With respect to seemingly inconsistent small number suppression across months for the same measure, I believe that each month is handled separately. Consequently, the threshold for additional small number suppression would vary from month-to-month.

I will continue to investigate this and similar issues (see below). I'll start by applying for L4 EMIS access.

Iain

---

Some asides 🙂

• The Measure._suppress_column method handles small number suppression within the measures framework
• @acagreen17 and @robinyjpark have raised similar issues:
  • Small number suppression malfunctioning in measures opensafely-core/cohort-extractor#626
  • Duplicated rows in/large numbers are being suppressed from the measures table opensafely-core/cohort-extractor#685