Outlier Detection and Filtering Serviceo

[Villarley]

Outlier Detection and Filtering Service

📝 Description

Implement an outlier detection service that identifies and filters anomalous or suspicious data before aggregation. This service should use multiple statistical techniques to detect prices that deviate significantly from consensus or are technically impossible (negative prices, >50% changes in seconds, etc.).

Context: Data from external APIs can have errors, glitches, or be manipulated. It's critical to filter this data before aggregating to avoid contaminating the consensus price that will be published on-chain.

Tech Stack:

• NestJS Service
• Statistical algorithms (IQR, Z-score, MAD)
• RxJS for stream processing
• TypeScript with strict types

✅ Requirements

• Implement an OutlierDetectionService with multiple detection strategies
• Detect prices outside reasonable ranges (e.g., negative price, >50% change)
• Implement IQR-based detection (Interquartile Range)
• Implement Z-score-based detection (standard deviation)
• Implement MAD-based detection (Median Absolute Deviation)
• Configure adjustable thresholds via environment variables
• Mark outliers instead of discarding them (for later analysis)
• Calculate data quality metrics per source
• Implement logging of detected outliers for debugging

🎯 Acceptance Criteria

• The service correctly detects negative or zero prices
• At least 3 outlier detection methods are implemented
• Thresholds are configurable via environment variables
• Outliers are marked with metadata explaining why they were detected
• Data quality metrics are generated per source (% of outliers)
• Unit tests exist with edge cases (>85% coverage)
• Each detection strategy and its parameters are documented
• The service doesn't crash with empty or insufficient data

📁 Expected files to change/structure

• apps/aggregator/src/services/outlier-detection.service.ts - Main service
• apps/aggregator/src/services/outlier-detection.service.spec.ts - Tests
• apps/aggregator/src/strategies/outlier-detectors/ - Strategies
  • iqr-detector.ts
  • zscore-detector.ts
  • mad-detector.ts
  • range-detector.ts
• apps/aggregator/src/interfaces/outlier-result.interface.ts - Detection result
• apps/aggregator/src/app.module.ts - Register service
• apps/aggregator/.env.example - Configuration variables
• apps/aggregator/README.md - Document strategies

[drips-wave]

This issue has been added to the Stellar Wave Program and is worth 200 Points.

🧑‍💻 Interested in contributing? Apply to work on this issue on Drips Wave, earn points, and receive rewards.

Warning

Only applications submitted via the apply link above will be considered. Please do not apply by commenting on the issue or opening a PR directly.

🤠 Repo maintainers: You can manage this issue's Points and Complexity on your Maintainer Dashboard here. Please keep an eye on applications and respond quickly 💙

ℹ️ Learn more about Drips Wave

[soma-enyi]

@soma-enyi has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Gm Gm Maintainer!!!
I would like to work on this issue!!!

ℹ️ Repo Maintainers: To accept this application, review their application or assign @soma-enyi to this issue.

[JerryIdoko]

@JerryIdoko has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hi team,
I would like to apply for this issue. Ensuring data quality before aggregation is crucial for the oracle's security, and I have experience implementing statistical filtering for time-series data.
Proposed Implementation Plan:
Architecture (Strategy Pattern):I will define an interface IOutlierStrategy that all detectors must implement. This allows the system to easily swap or chain multiple detectors (Range, IQR, Z-Score, MAD).The main OutlierDetectionService will act as the context, iterating through enabled strategies.
Algorithms:Range: Hard checks for $price \le 0$ or immediate volatility spikes ($>50%$).Z-Score: Will calculate mean and standard deviation: $Z = \frac{x - \mu}{\sigma}$. Effective for normal distributions.
IQR: Will calculate quartiles ($Q1, Q3$) to identify extreme values: $Low < Q1 - 1.5 \times IQR$ or $High > Q3 + 1.5 \times IQR$.MAD: A robust alternative to Z-Score for non-normal distributions, using the median of absolute deviations.
Data Handling: I will ensure the service handles BigNumber / high-precision decimals correctly to avoid floating-point errors common in financial data.
I will implement RxJS operators to process the data streams efficiently.As requested, outliers will be marked with metadata (e.g., { isOutlier: true, reason: 'z-score > 3' }) rather than deleted, allowing for audit trails.
Testing: Unit tests will cover edge cases: empty arrays, single data points, and perfectly flat data (division by zero protection).I can start immediately and adhere to the directory structure outlined in the description.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @JerryIdoko to this issue.

[soomtochukwu]

@soomtochukwu has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

GM @Villarley, please assign #18 to me.
I'm all in on building that robust OutlierDetectionService in NestJS with IQR, Z-score, MAD + range checks, configurable thresholds, outlier marking, quality metrics, logging, RxJS streams, strict typing, full unit tests (>85% coverage), and clean modular structure in the aggregator app. Let me grab this and deliver it solid

ℹ️ Repo Maintainers: To accept this application, review their application or assign @soomtochukwu to this issue.

[ritik4ever]

@ritik4ever has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hi, I’d like to take this issue .
I’ve worked on similar projects before and have relevant experience, so I should be able to handle this efficiently.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @ritik4ever to this issue.

[Mosas2000]

@Mosas2000 has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

I’ll like to take up this task, I’m a fullstack developer keen on impact and proven with a track record

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Mosas2000 to this issue.

[delfinacorr]

@delfinacorr has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hello! I'm a full-stack developer and I'm very interested in this topic. I'd like to help protect the protocol by implementing OutlierDetectionService.

I'm familiar with the Strategy Pattern and can implement the IQR, Z-Score, and MAD algorithms to ensure data integrity.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @delfinacorr to this issue.

[tali-creator]

@tali-creator has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

I’ve carefully gone through the issue description and taken time to understand the existing codebase and expected outcome. Based on my experience and the scope of this task, I’m confident I fully understand what needs to be done and how it fits into the project’s overall architecture.

I’m a full-stack Stark developer with 4+ years of hands-on experience, and I’m very confident in my ability to deliver a solid, production-ready solution. I’m 100% certain I can complete this task within 24 hours while maintaining code quality, clarity, and alignment with the project’s standards. I’d be glad to take this on and contribute effectively if assigned.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @tali-creator to this issue.

[Jopsan-gm]

@Jopsan-gm has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hello maintainers 👋

I’ve reviewed this issue and the relevant parts of the codebase. I’m a junior full-stack developer with hands-on experience contributing to real-world open source projects across both Web2 and Web3 contexts. My work includes JavaScript/TypeScript-based stacks, React on the frontend, Node.js on the backend, as well as contributing to blockchain-related repositories, including smart contract logic and protocol-level features.

I’m comfortable reading and understanding existing codebases, reasoning about on-chain and off-chain interactions, and delivering clean, focused implementations aligned with the project’s conventions. I’m used to working through GitHub workflows, iterating based on feedback, and prioritizing maintainability and clarity.

This issue aligns well with my experience, and I believe I can provide a solid and reliable solution. If it’s still available, I’d be happy to take it.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Jopsan-gm to this issue.

[dannyy2000]

@dannyy2000 has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

I’d like to take on this issue because ensuring the integrity of aggregator data is critical—any anomalous or manipulated price could contaminate the on-chain consensus.

I plan to implement a robust OutlierDetectionService that applies multiple statistical techniques (IQR, Z-score, MAD, and basic range checks) to identify suspicious data. Each outlier will be marked with metadata instead of being discarded, allowing for analysis and monitoring.

The service will:

Detect negative or zero prices and extreme changes (>50% in seconds)

Apply configurable thresholds via environment variables

Generate data quality metrics per source (% of outliers)

Be fully covered by unit tests with edge cases (>85% coverage)

Log all outliers for debugging and transparency

This work will improve the accuracy and reliability of the aggregation pipeline, ensuring the contract publishes trustworthy prices on-chain.

Thanks for considering my contribution!

ℹ️ Repo Maintainers: To accept this application, review their application or assign @dannyy2000 to this issue.

[Sam-Rytech]

@Sam-Rytech has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

I’d like to work on this. I’m a Full Stack SWE with experience building financial APIs and data pipelines where precision and outlier filtering are critical for system reliability. I am proficient with NestJS and RxJS.

I’m available to start immediately and will provide a robust, documented service with comprehensive unit test coverage.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Sam-Rytech to this issue.

[Ogstevyn]

@Ogstevyn has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hi team,
I’d be glad to implement the OutlierDetectionService to identify and mark anomalous or suspicious data before aggregation.

I’m a backend developer with hands-on experience building data validation and stream-processing services in TypeScript and NestJS. I’ve worked with statistical techniques, real-time data pipelines, and robust error handling to ensure data integrity. While I haven’t contributed to open-source projects yet, I’m eager to start and would be delighted to contribute to this project.

How I’ll resolve this

Implement OutlierDetectionService with multiple detection strategies:

IQR (Interquartile Range)

Z-score (Standard Deviation)

MAD (Median Absolute Deviation)

Range checks for negative or extreme values

Make thresholds configurable via environment variables

Mark detected outliers with metadata instead of discarding, for later analysis

Compute data quality metrics per source (% of outliers)

Add RxJS stream support for real-time data processing

Implement robust logging for debugging

Write unit tests with edge cases to ensure >85% coverage

Document each strategy, parameters, and usage in README

This will deliver a reliable, configurable, and well-tested service that ensures only clean data is used for aggregation and on-chain publishing.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Ogstevyn to this issue.

[Ogstevyn]

@Ogstevyn has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hi team,
I’d be glad to implement the OutlierDetectionService to identify and mark anomalous or suspicious data before aggregation.

I’m a backend developer with hands-on experience building data validation and stream-processing services in TypeScript and NestJS. I’ve worked with statistical techniques, real-time data pipelines, and robust error handling to ensure data integrity. While I haven’t contributed to open-source projects yet, I’m eager to start and would be delighted to contribute to this project.

How I’ll resolve this

Implement OutlierDetectionService with multiple detection strategies:

IQR (Interquartile Range)

Z-score (Standard Deviation)

MAD (Median Absolute Deviation)

Range checks for negative or extreme values

Make thresholds configurable via environment variables

Mark detected outliers with metadata instead of discarding, for later analysis

Compute data quality metrics per source (% of outliers)

Add RxJS stream support for real-time data processing

Implement robust logging for debugging

Write unit tests with edge cases to ensure >85% coverage

Document each strategy, parameters, and usage in README

This will deliver a reliable, configurable, and well-tested service that ensures only clean data is used for aggregation and on-chain publishing.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Ogstevyn to this issue.

[Amarjeet325]

@Amarjeet325 has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hi, I’d like to work on this issue.
I’m a full-stack developer and have previously contributed to multiple open-source projects.
Could you please assign this issue to me?

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Amarjeet325 to this issue.