Anomaly detection. Detection period value is included in the calculation of the AVG value for the training period

[winkel163]

Describe the bug
We had a case recently when a monitored value increased by around 25%, but anomaly detection test secceeded with anomaly_sensitivity parameter set to 2, anomaly_direction to both.

It was also unclear where Elementary takes the AVG value from. After some research, I found out that when calculating the AVG value, along with the traning period values, the detection period value is also taken into account.

As a result, the value to be examined affected the AVG value. I am not sure if this is the correct behavour.

In the picture, the value of TRAINING_AVG , highlighted in yellow, is calculated as the AVG for the 3 previous results PLUS the current one:

There may be an inaccuracy in the window function.
I also found similar topic in the Slack channel

Expected behavior
Elementary anomaly detection test failes.

Environment (please complete the following information):

• Elementary dbt package version: 0.15.2
• Data warehouse: Snowflake

[tcassou]

Hello there, and thanks for providing this great package!

We've ran into the same situation and conclusion about excluding the detection period from the training dataset. From a prediction perspective, training on the detection period can have a big impact on the anomaly score, and therefore affect the results (false positives / false negatives). I believe it's a common practice to guarantee that train / test sets are fully separated so that no test data "leaks" into the model and biases it.

This issue is probably the same as this one BTW: #1491

Are you planning to address this issue in the near future? Many thanks!