Skip to content

Commit

Permalink
Merge branch 'master' into release/v0.11.0
Browse files Browse the repository at this point in the history
  • Loading branch information
elongl authored Oct 3, 2023
2 parents ee46dd6 + 50d539e commit 7c78248
Show file tree
Hide file tree
Showing 6 changed files with 465 additions and 349 deletions.
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ Check out the [contributions guide](https://docs.elementary-data.com/general/con
<a href="https://github.com/ellakz"><img src="https://avatars.githubusercontent.com/u/18530437?v=4" width="50" height="50" alt=""/></a>
<a href="https://github.com/ofek1weiss"><img src="https://avatars.githubusercontent.com/u/55920061?v=4" width="50" height="50" alt=""/></a>
<a href="https://github.com/erikzaadi"><img src="https://avatars.githubusercontent.com/u/77775?v=4" width="50" height="50" alt=""/></a>
<a href="https://github.com/dapollak"><img src="https://avatars.githubusercontent.com/u/13542197?v=4" width="50" height="50" alt=""/></a>
<a href="https://github.com/RoiTabach"><img src="https://avatars.githubusercontent.com/u/25003091?v=4" width="50" height="50" alt=""/></a>
<a href="https://github.com/hahnbeelee"><img src="https://avatars.githubusercontent.com/u/55263191?v=4" width="50" height="50" alt=""/></a>
<a href="https://github.com/seanglynn-thrive"><img src="https://avatars.githubusercontent.com/u/93200565?v=4" width="50" height="50" alt=""/></a>
Expand Down Expand Up @@ -181,6 +182,9 @@ Check out the [contributions guide](https://docs.elementary-data.com/general/con
<a href="https://github.com/kovaacs"><img src="https://avatars.githubusercontent.com/u/20384210?v=4" width="50" height="50" alt=""/></a>
<a href="https://github.com/Arun-kc"><img src="https://avatars.githubusercontent.com/u/22231409?v=4" width="50" height="50" alt=""/></a>
<a href="https://github.com/pbadeer"><img src="https://avatars.githubusercontent.com/u/467756?v=4" width="50" height="50" alt=""/></a>
<a href="https://github.com/JavierLopezT"><img src="https://avatars.githubusercontent.com/u/11339132?v=4" width="50" height="50" alt=""/></a>
<a href="https://github.com/trevdoz"><img src="https://avatars.githubusercontent.com/u/34779611?v=4" width="50" height="50" alt=""/></a>
<a href="https://github.com/francoisforster"><img src="https://avatars.githubusercontent.com/u/1295590?v=4" width="50" height="50" alt=""/></a>

<!-- markdownlint-restore -->
<!-- prettier-ignore-end -->
Expand Down
37 changes: 37 additions & 0 deletions docs/guides/anomaly-detection-configuration/fail_on_zero.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
title: "fail_on_zero"
sidebarTitle: "fail_on_zero"
---

`fail_on_zero: true/false`

Elementary anomaly detection tests will fail if there is a zero metric value within the detection period.
If undefined, default is false.

- _Default: false_
- _Relevant tests: All anomaly detection tests_

<RequestExample>

```yml test
models:
- name: this_is_a_model
tests:
- elementary.volume_anomalies:
fail_on_zero: true
```
```yml model
models:
- name: this_is_a_model
config:
elementary:
fail_on_zero: true
```
```yml dbt_project.yml
vars:
fail_on_zero: true
```
</RequestExample>
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
title: "ignore_small_changes"
sidebarTitle: "ignore_small_changes"
---

```
ignore_small_changes:
spike_failure_percent_threshold: [int]
drop_failure_percent_threshold: [int]
```

If defined, an anomaly test will fail only if all the following conditions hold:

- The z-score of the metric within the detection period is anomoulous
- One of the following holds:
- The metric within the detection period is higher than `spike_failure_percent_threshold` percentages of the mean value in the training period, if defined.
- The metric within the detection period is lower than `drop_failure_percent_threshold` percentages of the mean value in the training period, if defined

Those settings can help to deal with situations where your metrics are stable and small changes causes to high z-scores, and therefore to anomaly.

If undefined, default is null for both spike and drop.

- _Default: none_
- _Relevant tests: All anomaly detection tests_

<RequestExample>

```yml test
models:
- name: this_is_a_model
tests:
ignore_small_changes:
spike_failure_percent_threshold: 2
drop_failure_percent_threshold: 50
```
```yml model
models:
- name: this_is_a_model
config:
elementary:
ignore_small_changes:
spike_failure_percent_threshold: 2
```
```yml source
sources:
- name: my_non_dbt_tables
schema: raw
tables:
- name: source_table
meta:
elementary:
ignore_small_changes:
drop_failure_percent_threshold: 50
```
```yml dbt_project.yml
vars:
ignore_small_changes:
spike_failure_percent_threshold: 10
```
</RequestExample>
4 changes: 4 additions & 0 deletions docs/guides/anomaly-detection-tests/volume-anomalies.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,10 @@ No mandatory configuration, however it is highly recommended to configure a `tim
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href="/guides/anomaly-detection-configuration/time-bucket"><font color="#CD7D55">period: [hour | day | week | month]</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href="/guides/anomaly-detection-configuration/time-bucket"><font color="#CD7D55">count: int</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/guides/anomaly-detection-configuration/seasonality"><font color="#CD7D55">seasonality: day_of_week</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/guides/anomaly-detection-configuration/fail_on_zero"><font color="#CD7D55">fail_on_zero: [true | false]</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/guides/anomaly-detection-configuration/ignore_small_changes"><font color="#CD7D55">ignore_small_changes:</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href="/guides/anomaly-detection-configuration/ignore_small_changes"><font color="#CD7D55">spike_failure_percent_threshold: int</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href="/guides/anomaly-detection-configuration/ignore_small_changes"><font color="#CD7D55">drop_failure_percent_threshold: int</font></a>
</code>
</pre>

Expand Down
4 changes: 3 additions & 1 deletion docs/mint.json
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,9 @@
"guides/anomaly-detection-configuration/exclude_regexp",
"guides/anomaly-detection-configuration/dimensions",
"guides/anomaly-detection-configuration/event_timestamp_column",
"guides/anomaly-detection-configuration/update_timestamp_column"
"guides/anomaly-detection-configuration/update_timestamp_column",
"guides/anomaly-detection-configuration/ignore_small_changes",
"guides/anomaly-detection-configuration/fail_on_zero"
]
}
]
Expand Down
Loading

0 comments on commit 7c78248

Please sign in to comment.