-
Notifications
You must be signed in to change notification settings - Fork 45
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Introduces the following new tools: 1. EWMA Anomaly Detection Server 2. HTTP/GRPC Shift Calculator Server 3. SQL Callout Server --------- Co-authored-by: Ethan <ethan@ethansstress.guest.corp.microsoft.com>
- Loading branch information
1 parent
0e5ce14
commit c020e0e
Showing
36 changed files
with
1,707 additions
and
90 deletions.
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
# syntax=docker/dockerfile:1 | ||
|
||
FROM mcr.microsoft.com/oss/go/microsoft/golang:1.21-cbl-mariner2.0 AS build | ||
|
||
COPY ./lib /workdir/lib | ||
|
||
COPY ./samples/anomaly-detection /workdir/samples/anomaly-detection | ||
|
||
WORKDIR /workdir/samples/anomaly-detection | ||
|
||
RUN go mod download | ||
|
||
RUN go install github.com/magefile/mage@latest | ||
|
||
RUN mage ci | ||
|
||
RUN go build -o ./bin/anomaly-detection ./cmd/anomaly-detection | ||
|
||
EXPOSE 2112 | ||
|
||
CMD [ "/bin/anomaly-detection" ] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,133 @@ | ||
# Anomaly Detection Server | ||
|
||
## Usage | ||
|
||
### Server as a Pod | ||
|
||
```sh | ||
# From the root of the anomaly-detection directory. | ||
docker build ../.. -f Dockerfile -t <yourcrhere>.azurecr.io/anomaly-detection:latest | ||
|
||
# Or if running from the root of the explore-iot-operations repository. | ||
# docker build . -f ./samples/anomaly-detection/Dockerfile -t <yourcrhere>.azurecr.io/anomaly-detection:latest | ||
|
||
# Push or load your newly built image into your cluster, depending on the k8s setup. | ||
# docker push <yourcrhere>.azurecr.io/anomaly-detection:latest # Using AKS + Connected ACR | ||
# minikube load <yourcrhere>.azurecr.io/anomaly-detection:latest # Using minikube | ||
# docker save <yourcrhere>.azurecr.io/anomaly-detection:latest | k3s ctr images import - # Using K3s | ||
|
||
kubectl run anomaly-detection --image=<yourcrhere>.azurecr.io/anomaly-detection:latest --stdin < config.yml | ||
``` | ||
|
||
### Configuration | ||
|
||
```yaml | ||
logger: # Logger settings | ||
level: 0 | ||
server: # Server settings | ||
route: /anomaly | ||
port: 3333 | ||
algorithm: # Algorithm settings | ||
temperature: # Temperature related algorithm settings | ||
lambda: 0.25 # Lambda value for temperature anomaly detection | ||
lFactor: 3 # L factor for temperature anomaly detection | ||
controlT: 90 # Control limit T constant | ||
controlS: 20 # Control limit S constant | ||
controlN: 10 # Control limit N constant | ||
vibration: # Vibration related algorithm settings | ||
lambda: 0.25 | ||
lFactor: 3 | ||
type: dynamic # Use dynamic algorithm, no requirement for control limit T, S, or N values | ||
humidity: # Humidity related algorithm settings | ||
lambda: 0.25 | ||
lFactor: 3 | ||
controlT: 80 | ||
controlS: 20 | ||
controlN: 10 | ||
``` | ||
## Algorithms | ||
Use the `type: dynamic` property in the configuration to choose an algorithm from the following. | ||
|
||
### Algorithm #1 - Control Limit Formula with Estimated Control Limits | ||
|
||
$$B = L\frac{S}{\sqrt{n}}\sqrt{\frac{\lambda}{2 - \lambda}[1 - (1 - \lambda)^{2i}]}$$ | ||
|
||
$$B_u = T + B$$ | ||
|
||
$$B_l = T - B$$ | ||
|
||
<center> | ||
|
||
| Symbol | Meaning | | ||
|---|---| | ||
| $B$ | Control Limit Value | | ||
| $B_u$ | Upper Control Limit | | ||
| $B_l$ | Lower Control Limit | | ||
| $L$ | L Factor | | ||
| $\lambda$ | Lambda Factor $(0 \leq \lambda \leq 1)$ | | ||
| $i$ | Iteration or Count | | ||
| $T$ | Constant Estimation of Sample Mean | | ||
| $S$ | Constant Estimation of Sample Std Dev | | ||
| $n$ | Constant Estimation of Rational Subgroup Size | | ||
|
||
</center> | ||
|
||
### Algorithm #2 - Control Limit Formula with Dynamically Evaluated Control Limits (`type: dynamic`) | ||
|
||
|
||
$$B = L\sigma_i\sqrt{\frac{\lambda}{2 - \lambda}[1 - (1 - \lambda)^{2i}]}$$ | ||
|
||
$$B_u = \mu_i + B$$ | ||
|
||
|
||
$$B_l = \mu_i - B$$ | ||
|
||
<center> | ||
|
||
| Symbol | Meaning | | ||
|---|---| | ||
| $B$ | Control Limit Value | | ||
| $B_u$ | Upper Control Limit | | ||
| $B_l$ | Lower Control Limit | | ||
| $L$ | L Factor | | ||
| $\lambda$ | Lambda Factor $(0 \leq \lambda \leq 1)$ | | ||
| $i$ | Iteration or Count | | ||
| $\mu_i$ | Sample Mean from Iteration $i$ | | ||
| $\sigma_i$ | Sample Std Dev from Iteration $i$ | | ||
|
||
</center> | ||
|
||
### EWMA Value and Anomaly Detection Conditions | ||
|
||
The following formulas are shared between the two algorithms. | ||
|
||
__EWMA Value Formula__ | ||
|
||
$$z_{i} = \lambda x_i + (1 - \lambda)z_{i-1}$$ | ||
|
||
__Anomaly Condition__ | ||
|
||
$$ | ||
a= | ||
\begin{cases} | ||
0, & \text{if } B_l \leq z_i \leq B_u \\ | ||
1, & \text{otherwise} | ||
\end{cases} | ||
$$ | ||
|
||
|
||
<center> | ||
|
||
| Symbol | Meaning | | ||
|---|---| | ||
| $B_u$ | Upper Control Limit | | ||
| $B_l$ | Lower Control Limit | | ||
| $\lambda$ | Lambda Factor $(0 \leq \lambda \leq 1)$ | | ||
| $i$ | Iteration or Count | | ||
| $x_i$ | Observed Value from Iteration $i$ | | ||
| $z_i$ | EWMA Value for Iteration $i$ | | ||
| $a$ | Anomaly Occurance, $1$ Implies an Anomaly | | ||
|
||
</center> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
logger: | ||
level: 0 | ||
server: | ||
route: /anomaly | ||
port: 3333 | ||
algorithm: | ||
temperature: | ||
lambda: 0.25 | ||
lFactor: 3 | ||
controlT: 90 | ||
controlS: 20 | ||
controlN: 10 | ||
vibration: | ||
lambda: 0.25 | ||
lFactor: 3 | ||
controlT: 50 | ||
controlS: 20 | ||
controlN: 10 | ||
humidity: | ||
lambda: 0.25 | ||
lFactor: 3 | ||
controlT: 80 | ||
controlS: 20 | ||
controlN: 10 |
Oops, something went wrong.