TSLies - Time Series Anomalies Detection Framework

What is it?

TSLies (Time Series anomaLIES) is an advanced anomaly detection framework using state-of-the-art Machine Learning techniques and the Poisson-FOCuS triggering algorithm for real-time anomaly detection in time series data. This framework provides a comprehensive suite of ML models, from deterministic to Bayesian approaches, for robust background modeling and anomaly detection in any time series dataset.

Installation and Dependencies

Prerequisites

Python 3.8+
CUDA-compatible GPU (optional, recommended for neural networks training)

Quick Installation

Install from PyPI:

pip install git+https://github.com/ICSC-SPoke3/TSLies

or clone and install locally.

git clone https://github.com/ICSC-SPoke3/TSLies.git
cd TSLies
pip install --upgrade setuptools pip
pip install -e .

Python Dependencies

Core packages include:

tensorflow
tensorflow-probability
tf_keras
keras
numpy
pandas
scikit-learn
scipy
matplotlib
seaborn
tqdm

Main Features

This repository provides a comprehensive framework for anomaly detection in time series data:

Core Capabilities

Real-time Background Modeling: Multiple ML architectures for background prediction
Bayesian Uncertainty Quantification: Probabilistic models for reliable anomaly detection
Spectral Domain Learning: Frequency-domain neural networks
FOCuS Change Point Detection: Optimal changepoint detection algorithm
Multi-dataset Validation: Cross-referencing with external event catalogs and metadata
Automated Visualization: Scientific plotting with LaTeX formatting

Key Innovations

Hybrid Model Ensemble: Combination of deterministic and Bayesian approaches
Modular Architecture: Individual files for each ML model type for maximum maintainability
Scalable Pipeline Architecture: Modular design for large-scale data processing
Advanced Uncertainty Handling: Critical for low false-positive anomaly detection

Architecture Overview

The framework follows a modular, three-stage pipeline architecture suitable for any time series anomaly detection task:

flowchart TD
    A[Raw Time Series Data] --> B[Data Preprocessing]
    C[External Features - if any] --> B
    D[Contextual Data - if any] --> B
    B --> E[Feature Engineering]
    E --> F[Background Modeling]
    F --> G[Anomaly Detection]
    G --> H[Catalog Validation]
    H --> I[Scientific Visualization]

Data Flow

Input Integration: Time series data, external features, contextual information
Background Prediction: ML models predict normal system behavior
Anomaly Detection: FOCuS algorithm identifies deviations from background
Validation: Cross-reference detections with known event catalogs or external metadata
Analysis: Generate scientific plots and performance metrics

Machine Learning Models

Deterministic Models

FFNNPredictor: Feed-Forward Neural Network for baseline background modeling
SpectralDomainFFNNPredictor: Frequency-domain Neural Network using FFT-based loss functions
RNNPredictor: Recurrent Neural Network for temporal dependencies

Bayesian Models (with Uncertainty Quantification)

BNNPredictor: Bayesian Neural Network with variational inference
PBNNPredictor: Probabilistic Bayesian Neural Network with enhanced uncertainty
ABNNPredictor: Advanced Bayesian Neural Network with tensorflow-probability
MCMCBNNPredictor: MCMC-based Bayesian Neural Network for full posterior sampling (to be finished)

Non-Parametric Models

MedianKNeighborsRegressor: K-Nearest Neighbors with median aggregation
MultiMeanKNeighborsRegressor: Multi-output K-NN for baseline comparisons

Model Selection Criteria

Uncertainty Quantification: Essential for reliable anomaly detection
Computational Efficiency: Real-time processing requirements
Interpretability: Understanding model decisions and feature importance

Modules

TSLies is organized into a modular architecture separating generic time series functionality from domain-specific applications:

Core Modules (`tslies/`)

tslies/config.py

Centralized configuration management:

File paths and directory structure
Configurable thresholds and parameters

tslies/background/

Complete ML model ecosystem with modular architecture:

mlobject.py: Base class with common ML functionality
losses.py: Custom loss functions (spectral, Bayesian NLL)
ffnnpredictor.py: Feed-Forward Neural Network predictor
rnnpredictor.py: Recurrent Neural Network for temporal dependencies
bnnpredictor.py: Bayesian Neural Network with uncertainty quantification
pbnnpredictor.py: Probabilistic Bayesian Neural Network
abnnpredictor.py: Approximate Bayesian Neural Network
mcmcbnnpredictor.py: MCMC-based Bayesian Neural Network
spectraldomainffnnpredictor.py: Frequency-domain neural network
knnpredictors.py: K-Nearest Neighbors regressors (median/mean variants)
Automated hyperparameter optimization and model persistence

tslies/trigger.py

Advanced anomaly detection:

FOCuS-Gaussian and FOCuS-Poisson algorithm implementation (Kester Ward, 2021)
Z-score detection
Multi-variate time series trigger merging
Temporal clustering and filtering

tslies/plotter.py

Scientific visualization suite:

Automated anomaly plotting
LaTeX-formatted scientific notation
Multi-panel time series with residuals
Export-ready figures for pubblications

tslies/utils.py

Essential utilities:

Data manipulation and masking
Time series processing
Logging and debugging
File I/O operations

tslies/dataset.py

Primary data processing:

Raw data file parsing and conversion
Multi-channel data handling
Temporal binning and aggregation

Domain-Specific Applications (`applications/`)

applications/acd/

ACD (Anti-Coincidence Detector) specific modules:

spacecraft.py: Spacecraft parametric data integration
solar.py: Solar environmental monitoring data integration
catalogs.py: Event catalog cross-referencing and validation
main_*.py: Complete analysis pipelines for ACD data

applications/intesa sanpaolo/

Intesa Sanpaolo application:

main_intesa.py: analysis pipelines for Intesa Sanpaolo data

Usage

Quick Start with TSLies

Import core TSLies modules:

# Import core TSLies components
from tslies.config import DIR, BACKGROUND_PREDICTION_FOLDER_NAME
from tslies.background import FFNNPredictor, BNNPredictor
from tslies.trigger import Trigger
from tslies.plotter import Plotter

Train a background model:

# Create and train a neural network background model
model = FFNNPredictor(df_data, y_cols, x_cols, y_pred_cols)
model.set_hyperparams(params)
model.create_model()
history = model.train()

Run anomaly detection:

# Apply FOCuS algorithm for changepoint detection
trigger = Trigger(tiles_df, y_cols, y_pred_cols, units, latex_y_cols)
anomalies, significance_df = trigger.run(thresholds, type='focus')

Visualize results:

# Generate scientific plots
plotter = Plotter(df=anomalies)
plotter.plot_anomalies_in_catalog(trigger_type, support_vars, thresholds, tiles_df, y_cols, y_pred_cols)

ACD-Specific Application

For ACD (Anti-Coincidence Detector) specific analysis:

# Import ACD-specific modules
from applications.acd.spacecraft import SpacecraftOpener
from applications.acd.solar import SunMonitor  
from applications.acd.catalogs import CatalogReader

# Or run complete ACD pipelines
from applications.acd import main_ml, main_trigger, main_acd

Advanced Usage

Model Comparison

# Compare multiple ML architectures
from tslies.background import (
    FFNNPredictor, 
    BNNPredictor, 
    SpectralDomainFFNNPredictor,
    RNNPredictor
)

models = [FFNNPredictor, BNNPredictor, SpectralDomainFFNNPredictor, RNNPredictor]
results = {}

for ModelClass in models:
    model = ModelClass(df_data, y_cols, x_cols, ...)
    model.create_model()
    history = model.train()
    results[ModelClass.__name__] = model.evaluate()

Data Structure

Directory Layout

This is the package

TSLies/
└── tslies/               # Core TSLies framework
    └── background/
    │   ├── mlobject
    │   ├── losses
    │   ├── ffnnpredictor
    │   ├── rnnpredictor
    │   ├── bnnpredictor
    │   ├── pbnnpredictor
    │   ├── abnnpredictor
    │   ├── mcmcbnnpredictor
    │   ├── spectraldomainffnnpredictor
    │   └── knnpredictors
    ├── config
    ├── dataset
    ├── explaiability
    ├── plotter
    ├── trigger
    └── utils

This is the folder containing the main.py script with the workflow, and the data folder.

MyUseCase/
├── data/pk/dataset.pk
└── main.py

After the TSLies has finished the analysis, you should expect these new directories.

MyUseCase/
├── data/pk/dataset.pk
├── logs/                   # System logs and debugging
├── results/                # Model outputs and anomalies detected
│   └── YYYY-MM-DD/
│       ├── background_prediction/
│       └── trigger_results/
└── main.py

Contributing

We welcome contributions to this Time Series Anomaly Detection Framework!

Contact

Lead Developer: Andrea Adelfio
Institution: INFN (Istituto Nazionale di Fisica Nucleare)
Email:

Project Status: Active development

Acknowledgments

FOCuS Algorithm: Ward (2021)
Research Community: Open-source machine learning libraries and frameworks
Computing Resources: High-performance computing support
ICSC:

Citation

If you use TSLies in your research, please cite:

@software{adelfio2025tslies,
  author = {Adelfio, Andrea},
  title = {TSLies: Time Series Anomaly Detection Framework},
  url = {https://github.com/andreaadelfio/TSLies},
  year = {2025},
  institution = {INFN}
}

To-do

review docs
check dependencies
check init files

Go to Top

Name		Name	Last commit message	Last commit date
Latest commit History 33 Commits
tslies		tslies
.gitignore		.gitignore
README.md		README.md
pyproject.toml		pyproject.toml
setup.py		setup.py

ICSC-Spoke3/TSLies

Folders and files

Latest commit

History

Repository files navigation