Data Storage Layer

[Villarley]

Data Storage Layer - Cache and temporary persistence with Redis

📝 Description

Implement a storage layer using Redis to cache aggregated prices, maintain recent historical data, and facilitate fast access to prices by the API service. This layer should handle automatic expiration and provide efficient read/write methods.

Context: Aggregated prices need to be temporarily stored so the API service can query them quickly. We also need to maintain a recent history for trend calculations and analysis.

Tech Stack:

• NestJS with @nestjs/cache-manager or ioredis
• Redis as cache store
• RxJS for asynchronous operations
• TypeScript with strict types

✅ Requirements

• Implement a StorageService that uses Redis as backend
• Store aggregated prices with configurable TTL (e.g., 5 minutes)
• Implement methods to read/write prices by symbol
• Maintain price history (time-series) per symbol
• Implement batch operations for multiple symbols
• Handle automatic reconnection to Redis in case of failure
• Implement efficient data serialization/deserialization
• Add cache hits/misses metrics
• Configure Redis connection via environment variables

🎯 Acceptance Criteria

• The service successfully connects to Redis on startup
• Prices are stored with TTL and expire automatically
• Individual prices can be read/written by symbol
• Price history can be obtained (last N records)
• Batch operations work correctly for multiple symbols
• The service handles Redis reconnections automatically
• Cache hits/misses metrics are logged
• Unit tests exist with Redis mocks (>80% coverage)
• The data schema stored in Redis is documented

📁 Expected files to change/structure

• apps/aggregator/src/services/storage.service.ts - Main service
• apps/aggregator/src/services/storage.service.spec.ts - Tests
• apps/aggregator/src/config/redis.config.ts - Redis configuration
• apps/aggregator/src/interfaces/storage-options.interface.ts - Storage options
• apps/aggregator/src/app.module.ts - Register CacheModule and service
• apps/aggregator/package.json - Add dependencies (cache-manager, ioredis)
• apps/aggregator/.env.example - Redis variables
• apps/aggregator/README.md - Document Redis usage
• docker-compose.yml (root) - Add Redis service

[drips-wave]

This issue has been added to the Stellar Wave Program and is worth 200 Points.

🧑‍💻 Interested in contributing? Apply to work on this issue on Drips Wave, earn points, and receive rewards.

Warning

Only applications submitted via the apply link above will be considered. Please do not apply by commenting on the issue or opening a PR directly.

🤠 Repo maintainers: You can manage this issue's Points and Complexity on your Maintainer Dashboard here. Please keep an eye on applications and respond quickly 💙

ℹ️ Learn more about Drips Wave

[soma-enyi]

@soma-enyi has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Gm Gm Maintainer!!!
I would like to work on this issue!!!

ℹ️ Repo Maintainers: To accept this application, review their application or assign @soma-enyi to this issue.

[Manuel1234477]

@Manuel1234477 has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Can I work on this issue

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Manuel1234477 to this issue.

[emperorsixpacks]

@emperorsixpacks has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

I'm Andrew, a fullstack developer with extensive experience in building robust and efficient systems. I would love to contribute to this issue and provide a high-quality solution. I'm available to start immediately.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @emperorsixpacks to this issue.

[JerryIdoko]

@JerryIdoko has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hi there, I would like to work on this issue.

I have relevant experience building storage and caching layers with NestJS, Redis, and TypeScript, and I can deliver this robustly.

Relevant Experience:

• Recently implemented a Redis-backed caching system using @nestjs/cache-manager for the Stellar-Insured-Backend project.
• Experienced in handling data persistence, TTL configurations, and optimizing cache hit/miss strategies.
• Familiar with the Neko Protocol ecosystem and open-source contributions.

Proposed Implementation Plan:

1. Setup: Configure RedisModule using ioredis and @nestjs/cache-manager in apps/aggregator/src/config/redis.config.ts.
2. Service Layer: Implement StorageService to handle:
   Atomic Operations: Using Redis sets/lists for storing price history (Time Series) to ensure efficient reads/writes.
   TTL Management: configuring automatic expiration for aggregated prices.
   Batch Processing: Implementing mget/mset logic for handling multiple symbols efficiently.
3. Resilience: Ensure the service handles Redis connection failures gracefully with retry logic.
4. Testing: specific unit tests with Redis mocking to ensure >80% coverage.

I can start immediately. Thank you!

ℹ️ Repo Maintainers: To accept this application, review their application or assign @JerryIdoko to this issue.

[soomtochukwu]

@soomtochukwu has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hey @Villarley, please assign #20 to me.
I’m pumped to implement that Redis-powered Data Storage Layer with caching for aggregated prices, temp history persistence, fast API reads, auto-expiration, reconnections, batch ops, metrics, and solid unit tests using NestJS + ioredis. Let me take it and knock it out quick 🙏

ℹ️ Repo Maintainers: To accept this application, review their application or assign @soomtochukwu to this issue.

[kelly-musk]

@kelly-musk has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

I would love to work on this issue because i have good backend skills

ℹ️ Repo Maintainers: To accept this application, review their application or assign @kelly-musk to this issue.

[Mosas2000]

@Mosas2000 has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

I’ll like to take up this task, I’m a fullstack developer keen on impact and proven with a track record

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Mosas2000 to this issue.

[DevAyomi]

@DevAyomi has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hi maintainer, I will really love to work on this issue

Thanks.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @DevAyomi to this issue.

[gelluisaac]

@gelluisaac has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hi @maintainer, I’d love to work on this! I have experience with the tech stack involved (Backend/Blockchain) and I’ve reviewed the issue description.
Could you please assign this to me? I'm ready to start and will submit a PR/Draft as soon as I have the initial logic ready. Thanks!

ℹ️ Repo Maintainers: To accept this application, review their application or assign @gelluisaac to this issue.

[Amarjeet325]

@Amarjeet325 has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hi maintainers 👋

I’d like to work on this issue.
I’m a full-stack developer and have previously contributed to multiple open-source projects.
Could you please assign this issue to me?

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Amarjeet325 to this issue.

[Obiajulu-gif]

@Obiajulu-gif has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

I would love to implement the Redis-backed Data Storage Layer as a NestJS StorageService that caches aggregated prices, keeps short time-series history per symbol, supports batch ops, and exposes cache hit/miss metrics—while staying resilient to Redis outages.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Obiajulu-gif to this issue.

[Properprogress1]

@Properprogress1 has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

I’d like to implement this Redis-backed storage layer for the aggregator service.

I’ll design a StorageService optimized for a microservices environment, using Redis as a fast, ephemeral data store for aggregated prices and short-term historical time series data. The service will support TTL based caching, efficient batch read/write operations, and symbol-scoped price history retrieval for downstream analytics.

The implementation will be production-ready for Kubernetes: configuration via environment variables, resilient Redis reconnection handling, lightweight serialization for network efficiency, and observable cache hit/miss metrics for operational visibility. The API-facing service will benefit from low-latency reads while keeping Redis usage bounded through automatic expiration.

I’ll also include comprehensive unit tests with Redis mocks, clear schema documentation, and Docker Compose updates to support local and cluster-style deployments.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Properprogress1 to this issue.

[Jopsan-gm]

@Jopsan-gm has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hello maintainers 👋

I’ve reviewed this issue and the relevant parts of the codebase. I’m a junior full-stack developer with hands-on experience contributing to real-world open source projects across both Web2 and Web3 contexts. My work includes JavaScript/TypeScript-based stacks, React on the frontend, Node.js on the backend, as well as contributing to blockchain-related repositories, including smart contract logic and protocol-level features.

I’m comfortable reading and understanding existing codebases, reasoning about on-chain and off-chain interactions, and delivering clean, focused implementations aligned with the project’s conventions. I’m used to working through GitHub workflows, iterating based on feedback, and prioritizing maintainability and clarity.

This issue aligns well with my experience, and I believe I can provide a solid and reliable solution. If it’s still available, I’d be happy to take it.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Jopsan-gm to this issue.