Integration Tests and E2E Testing

[Villarley]

Integration Tests and E2E Testing

📝 Description

Create a complete suite of integration tests and end-to-end tests that validate the functioning of the aggregator as a whole. These tests should simulate real data from the ingestor, verify the complete pipeline, and validate the results stored in Redis.

Context: Unit tests validate individual components, but we need tests that validate that the complete system works correctly by integrating all the pieces.

Tech Stack:

• Jest for test runner
• @nestjs/testing for NestJS testing
• Testcontainers or Docker Compose for test Redis
• Mocks for the ingestor
• TypeScript with strict types

✅ Requirements

• Create integration test suite for the complete pipeline
• Implement ingestor mocks that emit test data
• Use Testcontainers or Docker Compose to start test Redis
• Create tests that validate the complete aggregation flow
• Validate that data is correctly stored in Redis
• Create tests for error scenarios (ingestor down, Redis down)
• Implement basic load tests (throughput)
• Create fixtures with realistic test data
• Configure CI to automatically run integration tests

🎯 Acceptance Criteria

• An integration test suite exists that can be run with npm run test:integration
• Tests automatically start Redis using Testcontainers or Docker
• The ingestor is mocked to provide controlled test data
• Tests validate the complete pipeline from reception to storage
• It's verified that aggregated prices are mathematically correct
• Tests exist for error handling (services down, invalid data)
• Tests run automatically in CI
• Total code coverage (unit + integration) is >80%
• How to run the tests is documented in the README

📁 Expected files to change/structure

• apps/aggregator/test/ - Integration test directory
  • integration/
    • aggregator-pipeline.integration.spec.ts
    • error-handling.integration.spec.ts
  • e2e/
    • aggregator.e2e.spec.ts
  • fixtures/
    • price-data.fixtures.ts
  • mocks/
    • ingestor.mock.ts
  • setup/
    • test-setup.ts
• apps/aggregator/jest.integration.config.js - Config for integration tests
• apps/aggregator/docker-compose.test.yml - Redis for testing
• apps/aggregator/package.json - Add test scripts and dependencies
• .github/workflows/ci.yml - Add integration tests to CI
• apps/aggregator/README.md - Document testing strategy

[drips-wave]

This issue has been added to the Stellar Wave Program and is worth 200 Points.

🧑‍💻 Interested in contributing? Apply to work on this issue on Drips Wave, earn points, and receive rewards.

Warning

Only applications submitted via the apply link above will be considered. Please do not apply by commenting on the issue or opening a PR directly.

🤠 Repo maintainers: You can manage this issue's Points and Complexity on your Maintainer Dashboard here. Please keep an eye on applications and respond quickly 💙

ℹ️ Learn more about Drips Wave

[Agbeleshe]

@Agbeleshe has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hello, can handle this and I would be glad to work on it. I understand this issue is about building a full integration and end-to-end testing layer that validates the aggregator as a complete system, not just isolated components. If assigned, I would design an integration test harness using Jest and @nestjs/testing, backed by Testcontainers or Docker Compose to spin up an isolated Redis instance automatically. I would implement realistic ingestor mocks and fixtures to simulate live data streams, then verify the entire pipeline from ingestion through aggregation to persistence, including mathematical accuracy of aggregated prices and correct Redis writes.

In addition, I would cover operational scenarios such as Redis unavailability, ingestor failures, and invalid payloads, alongside basic throughput tests to validate stability under load. I would structure the tests exactly as outlined (integration, e2e, fixtures, mocks, setup), add a dedicated Jest integration config, wire everything into CI, and document how to run and extend the suite in the README. I would really appreciate being assigned this issue, as comprehensive integration and E2E testing is critical for protocol reliability, and I would be glad to deliver a robust, maintainable testing framework for the project.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Agbeleshe to this issue.

[emperorsixpacks]

@emperorsixpacks has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

I'm Andrew, a fullstack developer with extensive experience in building robust and efficient systems. I would love to contribute to this issue and provide a high-quality solution. I'm available to start immediately.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @emperorsixpacks to this issue.

[JerryIdoko]

@JerryIdoko has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hi there,

I would like to apply to work on this issue. I understand the goal is to validate the complete aggregator pipeline, ensuring data integrity from the ingestor mocks through to Redis storage.

My plan to address the requirements:

Environment Setup: Implement Testcontainers (or Docker Compose) to spin up a transient Redis instance for isolated testing.

Mocking: Create robust mocks for the ingestor to simulate various data emission scenarios (valid data, invalid data, high throughput).

Test Suite: Develop the integration and E2E specs using @nestjs/testing and Jest, covering:

Happy path (complete flow validation).

Error handling (Redis downtime, service failures).

Mathematical verification of aggregated prices.

CI/Documentation: Integrate the new suite into the existing GitHub Actions workflow and document the testing strategy in the README.

I am available to start immediately. Please assign this to me.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @JerryIdoko to this issue.

[JoE11-y]

@JoE11-y has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hi there!

I'd love to pick up this issue, I understand the scope of the task and can submit a PR soon if assigned

ℹ️ Repo Maintainers: To accept this application, review their application or assign @JoE11-y to this issue.

[Mosas2000]

@Mosas2000 has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

I’ll like to take up this task, I’m a fullstack developer keen on impact and proven with a track record

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Mosas2000 to this issue.

[dxdleady]

@dxdleady has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hi, wanna grab this task - love to test, have a SDET background!
Will implement in flying colours.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @dxdleady to this issue.

[udofia2]

@udofia2 has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

I would like to work on this issue because i specialize in backend development using nestjs with typescript and i have written multiple integration tests as seen in my github repos.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @udofia2 to this issue.

[Themancalledpg]

@Themancalledpg has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hi, my name is Preciousgift, and I am a fullstack and smart contract developer skilled in React, Next.js, JavaScript, TypeScript, Tailwind CSS, and Soroban. For this task, I will:
Create an integration test suite using Jest and @nestjs/testing to simulate the full aggregator pipeline with realistic ingestor mock data.
Configure Testcontainers or Docker Compose to spin up a test Redis instance for validating data storage and retrieval.
Implement tests covering error scenarios, including ingestor or Redis downtime, and basic throughput/load tests to ensure reliability.
Document test execution in the README, ensure CI runs the suite automatically, and verify total coverage exceeds 80%.
ETA: 10 hours

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Themancalledpg to this issue.

[Manuel1234477]

@Manuel1234477 has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Can I work on this issue

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Manuel1234477 to this issue.

[Jopsan-gm]

@Jopsan-gm has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hello maintainers 👋

I’ve reviewed this issue and the relevant parts of the codebase. I’m a junior full-stack developer with hands-on experience contributing to real-world open source projects across both Web2 and Web3 contexts. My work includes JavaScript/TypeScript-based stacks, React on the frontend, Node.js on the backend, as well as contributing to blockchain-related repositories, including smart contract logic and protocol-level features.

I’m comfortable reading and understanding existing codebases, reasoning about on-chain and off-chain interactions, and delivering clean, focused implementations aligned with the project’s conventions. I’m used to working through GitHub workflows, iterating based on feedback, and prioritizing maintainability and clarity.

This issue aligns well with my experience, and I believe I can provide a solid and reliable solution. If it’s still available, I’d be happy to take it.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Jopsan-gm to this issue.

[dannyy2000]

@dannyy2000 has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

I’d like to take on this issue because having a full integration and end-to-end test suite is essential for ensuring the aggregator pipeline works reliably under realistic conditions. Unit tests alone can’t catch issues that emerge from interactions between the ingestor, aggregator, and Redis storage.

I will implement a dedicated integration test setup using Testcontainers to spin up Redis on demand, and mock the ingestor to emit controlled test data. Tests will cover the complete aggregation flow from ingestion to storage, including mathematical correctness of aggregated prices, error scenarios (Redis down, ingestor unavailable), and basic throughput/load validation.

Fixtures will provide realistic price data, and the test suite will be structured into integration/, e2e/, mocks/, and fixtures/ directories for clarity and maintainability. The CI configuration will be updated to automatically run these integration tests, ensuring regression safety.

This approach guarantees that the entire pipeline functions as expected in realistic scenarios, errors are handled gracefully, and future changes won’t break end-to-end functionality. Documentation will also be included so developers can easily run and extend the integration tests.

Thanks for considering my contribution!

ℹ️ Repo Maintainers: To accept this application, review their application or assign @dannyy2000 to this issue.

[Ogstevyn]

@Ogstevyn has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hi team,
I’d be happy to implement the integration and end-to-end testing suite for the aggregator pipeline.

I’m a backend developer with experience building comprehensive test suites for complex systems, including full pipelines with external dependencies, mocks, and containerized test environments. I focus on validating correctness, error handling, and performance. While I haven’t contributed to open-source projects yet, I’m eager to start and would be delighted to contribute to this project.

How I’ll resolve this

Create integration tests covering the full aggregation pipeline from ingestor to Redis storage

Implement mock ingestor to emit controlled test data and edge cases

Use Testcontainers or Docker Compose to spin up a test Redis instance

Validate that aggregated prices are correct and stored properly

Add tests for error handling scenarios (ingestor/Redis down, invalid data)

Include basic load tests to ensure throughput under realistic conditions

Organize fixtures, mocks, and setup scripts for maintainable tests

Configure CI to automatically run integration tests

Document test execution and coverage in README

This will deliver a reliable, maintainable testing suite ensuring that the aggregator works correctly under normal and failure scenarios, with automated CI validation.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Ogstevyn to this issue.

[Amarjeet325]

@Amarjeet325 has applied to work on this issue as part of the Stellar Wave Program's 1st wave.

Hi, I’d like to work on this issue.
I’m a full-stack developer and have previously contributed to multiple open-source projects.
Could you please assign this issue to me?

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Amarjeet325 to this issue.