diff --git a/apps/ingestor/README.md b/apps/ingestor/README.md index 25a5c50..fc76ee0 100644 --- a/apps/ingestor/README.md +++ b/apps/ingestor/README.md @@ -1,14 +1,17 @@ # Ingestor Service -Connects to external stock price APIs and fetches raw price data. +Fetches real-time stock price data from external providers and normalizes it into a unified format. ## Overview -The ingestor service is responsible for: -- Connecting to external stock price APIs (e.g., Alpha Vantage, Yahoo Finance, etc.) -- Fetching raw price data for specified stock symbols -- Normalizing initial data format -- Publishing raw data to the next stage (aggregator) +The Ingestor Service is responsible for: + +- Connecting to external stock price APIs (currently Finnhub) +- Automatically fetching prices on a configurable interval via the **Scheduler Service** +- Normalizing responses into a consistent data structure +- Exposing price data via REST endpoints for on-demand queries + +This service implements the **Adapter Pattern**, allowing new data providers to be added without modifying core logic. ## Getting Started @@ -16,6 +19,7 @@ The ingestor service is responsible for: - Node.js >= 18 - npm >= 9 +- Finnhub API Key ([register here](https://finnhub.io/register)) ### Installation @@ -33,7 +37,26 @@ Copy the example environment file and configure it: cp .env.example .env ``` -Edit `.env` with your API keys and configuration. +Edit `.env` with your configuration: + +```env +# Server Configuration +PORT=3000 + +# Finnhub API (Required) +FINNHUB_API_KEY=your_finnhub_api_key + +# Scheduler Configuration +FETCH_INTERVAL_MS=60000 +STOCK_SYMBOLS=AAPL,GOOGL,MSFT,TSLA +``` + +| Variable | Required | Default | Description | +|----------|----------|---------|-------------| +| `PORT` | No | `3000` | HTTP server port | +| `FINNHUB_API_KEY` | **Yes** | — | API key from Finnhub | +| `FETCH_INTERVAL_MS` | No | `60000` | Polling interval in milliseconds | +| `STOCK_SYMBOLS` | No | — | Comma-separated list of symbols to fetch | ### Running the Service @@ -59,48 +82,104 @@ Then start the service: npm start ``` -### Testing +### Linting -Run tests: +Check code style: ```bash -npm test +npm run lint ``` -Run tests in watch mode: +## API Endpoints + +### Get Stock Price ```bash -npm run test:watch +GET /prices/:symbol ``` -Run tests with coverage: +Fetches the current price for a given stock symbol. + +**Example Request:** ```bash -npm run test:cov +curl http://localhost:3000/prices/AAPL ``` -### Linting +**Example Response:** -Check code style: +```json +{ + "source": "Finnhub", + "symbol": "AAPL", + "price": 150.20, + "timestamp": 1706540400000 +} +``` + +**Response Fields:** + +| Field | Type | Description | +|-------------|----------|----------------------------------------| +| `source` | `string` | Data provider identifier | +| `symbol` | `string` | Stock ticker symbol (uppercase) | +| `price` | `number` | Current price in USD | +| `timestamp` | `number` | Unix timestamp in milliseconds | + +## Architecture -```bash -npm run lint ``` +┌─────────────────────────────────────────────────────────────┐ +│ Ingestor Service │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ Scheduler ──(interval)──► PriceFetcherService │ +│ │ │ +│ ▼ │ +│ HTTP Request ──► Controller ──► FinnhubAdapter ──► API │ +│ │ │ +│ ▼ │ +│ NormalizedStockPrice │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +**Flows:** +- **Scheduled:** The `SchedulerService` triggers `PriceFetcherService` at a configurable interval to fetch all configured symbols automatically. +- **On-Demand:** REST endpoints allow fetching a specific symbol's price via HTTP. + +Each provider adapter implements a common `PriceProvider` interface, enabling seamless addition of new data sources. ## Project Structure ``` apps/ingestor/ ├── src/ -│ ├── main.ts # Application entry point -│ └── app.module.ts # Root module -├── .env.example # Example environment variables -├── nest-cli.json # NestJS CLI configuration -├── package.json # Dependencies and scripts -├── tsconfig.json # TypeScript configuration -└── README.md # This file +│ ├── main.ts # Application entry point +│ ├── app.module.ts # Root module +│ ├── controllers/ # HTTP request handlers +│ ├── services/ # Business logic +│ ├── providers/ # External API adapters +│ └── modules/ # Feature modules +├── .env.example # Example environment variables +├── nest-cli.json # NestJS CLI configuration +├── package.json # Dependencies and scripts +├── tsconfig.json # TypeScript configuration +└── README.md # This file ``` +## Limitations + +This is a **Proof of Concept** implementation with the following constraints: + +- Single provider only (Finnhub) +- No retry logic on API failures +- No caching mechanism +- No persistent storage (pass-through only) +- No rate limiting protection +- No test coverage yet +- Basic error handling only + ## Status -🚧 Under construction - Business logic will be implemented in subsequent issues. +🚧 POC complete — Integration with Aggregator service planned for next phase.