From 3dd8bbcf49f780a5962a76436cc48c4f123ebdc9 Mon Sep 17 00:00:00 2001 From: Doga Tav Date: Thu, 8 Jan 2026 11:35:53 -0400 Subject: [PATCH 01/18] docs: add comprehensive project overview documentation --- docs/PROJECT_OVERVIEW.md | 257 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 257 insertions(+) create mode 100644 docs/PROJECT_OVERVIEW.md diff --git a/docs/PROJECT_OVERVIEW.md b/docs/PROJECT_OVERVIEW.md new file mode 100644 index 0000000..0b97cee --- /dev/null +++ b/docs/PROJECT_OVERVIEW.md @@ -0,0 +1,257 @@ +# OpenKommander Project Overview + +## Introduction + +OpenKommander is a comprehensive command-line utility and web-based admin UI for managing Apache Kafka clusters. It provides both CLI and REST API interfaces for Kafka administration tasks, along with a modern React-based frontend for visual cluster management. + +## Project Purpose + +OpenKommander simplifies Kafka cluster management by providing: + +- **Unified Interface**: Single tool for both command-line and web-based administration +- **Multi-Cluster Support**: Manage multiple Kafka clusters from one interface +- **Real-time Monitoring**: Live metrics and status updates for clusters, topics, and consumer groups +- **Developer-Friendly**: Easy setup with Docker/Podman support and comprehensive API +- **Production-Ready**: Built with IBM Sarama, a mature Kafka client library + +## Key Features + +### 1. Cluster Management +- Connect to multiple Kafka clusters +- View cluster metadata and broker information +- Monitor cluster health and status +- Support for KRaft mode (no Zookeeper required) + +### 2. Topic Management +- Create, delete, and update topics +- List all topics with detailed information +- Describe topic configurations and partitions +- Update partition counts dynamically +- View replication status and ISR (In-Sync Replicas) + +### 3. Message Production +- Produce messages to topics via CLI or API +- Support for message keys and partitioning +- Configurable acknowledgment settings +- Manual or automatic partition selection + +### 4. Broker Management +- List all brokers in the cluster +- View broker status and connectivity +- Monitor partition distribution +- Check leader and replica assignments + +### 5. Consumer Group Management +- List all consumer groups +- View consumer group status and lag +- Monitor topic assignments +- Track consumer group state (Stable, Rebalancing, Dead) + +### 6. Web UI +- Modern React-based interface using Carbon Design System +- Real-time metrics dashboard +- Interactive tables for managing resources +- Auto-refresh capabilities with configurable polling +- Responsive design for desktop and mobile + +### 7. REST API +- RESTful API for all operations +- OpenAPI 3.0 specification +- JSON request/response format +- Health check endpoints +- CORS support for web integration + +## Architecture Components + +### Backend (Go) +- **CLI Framework**: Cobra for command-line interface +- **Kafka Client**: IBM Sarama for Kafka operations +- **HTTP Server**: Built-in Go HTTP server for REST API +- **Session Management**: Persistent session handling +- **Logging**: Structured logging with configurable levels + +### Frontend (React) +- **UI Framework**: React 18 with Vite +- **Design System**: IBM Carbon Design System +- **Routing**: React Router v6 +- **State Management**: React Hooks +- **API Client**: Axios for HTTP requests +- **Styling**: SCSS with Carbon themes + +### Infrastructure +- **Containerization**: Docker and Podman support +- **Development Environment**: Docker Compose for local development +- **Multi-Cluster Setup**: Support for 3+ Kafka clusters +- **Build System**: Make for automation + +## Use Cases + +### 1. Development Environment +- Quickly spin up local Kafka clusters for testing +- Experiment with topics and message production +- Test consumer applications with real-time monitoring + +### 2. Operations & Monitoring +- Monitor cluster health and performance +- Track consumer lag and rebalancing +- Manage topic lifecycle (create, update, delete) +- View broker status and partition distribution + +### 3. Troubleshooting +- Inspect topic configurations +- Check consumer group states +- Verify message production +- Analyze partition assignments + +### 4. Automation & CI/CD +- Integrate with deployment pipelines +- Automate topic creation and configuration +- Script-based cluster management +- API-driven operations + +### 5. Multi-Cluster Management +- Manage development, staging, and production clusters +- Compare configurations across environments +- Centralized monitoring dashboard + +## Technology Stack + +### Backend +- **Language**: Go 1.24+ +- **Kafka Client**: IBM Sarama v1.46.3 +- **CLI Framework**: Cobra v1.10.1 +- **Table Rendering**: go-pretty v6.6.9 + +### Frontend +- **Framework**: React 18 +- **Build Tool**: Vite +- **UI Library**: IBM Carbon Design System +- **HTTP Client**: Axios +- **Router**: React Router v6 + +### Development Tools +- **Container Runtime**: Docker/Podman +- **Build System**: Make +- **Version Control**: Git +- **Package Management**: Go modules, npm + +## Project Structure + +``` +openkommander/ +├── cmd/ # CLI commands +├── internal/ # Internal packages +│ └── core/ +│ └── commands/ # Core business logic +├── pkg/ # Public packages +│ ├── cli/ # CLI implementation +│ ├── logger/ # Logging utilities +│ └── session/ # Session management +├── frontend/ # React web application +│ └── src/ +│ ├── components/ # React components +│ ├── pages/ # Page components +│ ├── services/ # API services +│ └── hooks/ # Custom React hooks +├── docs/ # Documentation +├── scripts/ # Setup and utility scripts +├── tests/ # Test files +└── docker/ # Docker configurations +``` + +## Getting Started + +### Quick Start +```bash +# Clone the repository +git clone https://github.com/IBM/openkommander.git +cd openkommander + +# Run interactive setup +./scripts/setup.sh +``` + +### Development Setup +```bash +# Start development environment +make setup +make container-start + +# Build and install CLI +make dev-run + +# Access the application +ok login +``` + +### Using the Web UI +```bash +# Start the REST server +ok server start -p 8081 + +# Access the UI at http://localhost:8081 +``` + +## Configuration + +### Environment Variables +- `KAFKA_BROKERS`: Comma-separated list of broker addresses +- `KAFKA_BROKER`: Primary broker address (backward compatibility) + +### Session Storage +- Configuration stored in `~/.ok/` +- Session persistence across CLI invocations +- Frontend assets in `~/.ok/frontend/` + +## Security Considerations + +- No authentication required for local development +- Session-based access control +- CORS configuration for API access +- Secure broker connections supported + +## Performance + +- Efficient Kafka client connection pooling +- Minimal memory footprint +- Fast topic and broker operations +- Optimized frontend with code splitting +- Real-time updates with configurable polling intervals + +## Limitations + +- Currently focused on cluster administration (not data streaming) +- Consumer group management is read-only +- No built-in authentication/authorization (relies on Kafka security) +- Single-user session model + +## Future Enhancements + +- Schema registry integration +- Kafka Connect management +- Advanced consumer offset management +- Multi-user authentication +- Metrics persistence and historical data +- Alert and notification system +- Topic data browsing and search + +## Contributing + +OpenKommander is an open-source project. Contributions are welcome! See [CONTRIBUTING.md](../CONTRIBUTING.md) for guidelines. + +## License + +This project is licensed under the Apache License 2.0. See [LICENSE](../LICENSE) for details. + +## Support + +- **Issues**: GitHub Issues +- **Documentation**: `/docs` directory +- **Code of Conduct**: [CODE_OF_CONDUCT.md](../CODE_OF_CONDUCT.md) +- **Security**: [SECURITY.md](../SECURITY.md) + +## Acknowledgments + +- Built with IBM Sarama Kafka client +- UI powered by IBM Carbon Design System +- Inspired by Kafka administration needs in enterprise environments \ No newline at end of file From bf5b7b188713cdad71108d4cda1e40d1752be51c Mon Sep 17 00:00:00 2001 From: Doga Tav Date: Thu, 8 Jan 2026 11:36:02 -0400 Subject: [PATCH 02/18] docs: add detailed system architecture documentation --- docs/ARCHITECTURE.md | 433 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 433 insertions(+) create mode 100644 docs/ARCHITECTURE.md diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..acb0ccd --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,433 @@ +# OpenKommander Architecture + +## System Architecture Overview + +OpenKommander follows a three-tier architecture with a Go backend, REST API layer, and React frontend, all designed to interact with Apache Kafka clusters. + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Users │ +└────────────┬────────────────────────────┬───────────────────┘ + │ │ + │ CLI │ Web Browser + │ │ +┌────────────▼────────────┐ ┌───────────▼──────────────────┐ +│ CLI Interface (ok) │ │ React Frontend (Web UI) │ +│ - Cobra Commands │ │ - Carbon Design System │ +│ - Interactive Prompts │ │ - Real-time Dashboard │ +│ - Formatted Output │ │ - Resource Management │ +└────────────┬────────────┘ └───────────┬──────────────────┘ + │ │ + │ │ HTTP/REST + │ │ +┌────────────▼────────────────────────────▼──────────────────┐ +│ Go Backend Application │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ REST API Server (HTTP) │ │ +│ │ - Health endpoints │ │ +│ │ - Topic management │ │ +│ │ - Broker information │ │ +│ │ - Consumer group data │ │ +│ │ - Message production │ │ +│ └─────────────────┬───────────────────────────────────┘ │ +│ │ │ +│ ┌─────────────────▼───────────────────────────────────┐ │ +│ │ Core Business Logic Layer │ │ +│ │ - commands/broker.go │ │ +│ │ - commands/cluster.go │ │ +│ │ - commands/topic.go │ │ +│ │ - commands/produce.go │ │ +│ │ - commands/status.go │ │ +│ └─────────────────┬───────────────────────────────────┘ │ +│ │ │ +│ ┌─────────────────▼───────────────────────────────────┐ │ +│ │ Session Management Layer │ │ +│ │ - Session persistence │ │ +│ │ - Client connection pooling │ │ +│ │ - Authentication state │ │ +│ └─────────────────┬───────────────────────────────────┘ │ +│ │ │ +│ ┌─────────────────▼───────────────────────────────────┐ │ +│ │ Kafka Client Layer (Sarama) │ │ +│ │ - ClusterAdmin interface │ │ +│ │ - Client interface │ │ +│ │ - Producer interface │ │ +│ └─────────────────┬───────────────────────────────────┘ │ +└────────────────────┼────────────────────────────────────────┘ + │ + │ Kafka Protocol + │ +┌────────────────────▼────────────────────────────────────────┐ +│ Apache Kafka Cluster(s) │ +│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ +│ │ Broker 1 │ │ Broker 2 │ │ Broker 3 │ │ +│ └──────────┘ └──────────┘ └──────────┘ │ +│ │ +│ Topics, Partitions, Consumer Groups, Messages │ +└─────────────────────────────────────────────────────────────┘ +``` + +## Component Details + +### 1. Frontend Layer (React) + +#### Technology Stack +- **React 18**: Modern React with hooks and concurrent features +- **Vite**: Fast build tool and dev server +- **Carbon Design System**: IBM's enterprise UI framework +- **React Router v6**: Client-side routing +- **Axios**: HTTP client for API calls + +#### Key Components + +##### Layout Components +- **MainLayout**: Main application shell with navigation +- **AppSideNav**: Sidebar navigation menu + +##### Page Components +- **OverviewPage**: Dashboard with cluster metrics +- **TopicsPage**: Topic management interface +- **BrokersPage**: Broker information display +- **ConsumerGroupsPage**: Consumer group monitoring + +##### Common Components +- **BaseTable**: Reusable data table component +- **BaseModal**: Modal dialog wrapper +- **ResourcePage**: Generic resource management page +- **StatusTag**: Status indicator component +- **ErrorNotification**: Error display component + +##### Custom Hooks +- **useApi**: API call management with loading/error states +- **useResourceManager**: CRUD operations for resources + +##### Services +- **api.js**: Centralized API client +- **localstorage.js**: Browser storage utilities + +#### Data Flow +``` +User Action → Component → Hook → API Service → Backend + ↓ + State Update + ↓ + Re-render +``` + +### 2. Backend Layer (Go) + +#### Package Structure + +##### Main Package (`main.go`) +- Application entry point +- Logger initialization +- CLI command execution + +##### CLI Package (`pkg/cli/`) +- Cobra command definitions +- Command routing +- User input handling +- Output formatting + +##### Core Commands (`internal/core/commands/`) + +**broker.go** +- `GetBrokerClient()`: Retrieve Kafka client for broker operations + +**cluster.go** +- `ListClusters()`: Get all cluster/broker information +- `GetClusterMetadata()`: Retrieve cluster metadata + +**topic.go** +- `CreateTopic()`: Create new topics +- `DeleteTopic()`: Remove topics +- `ListTopics()`: Get all topics +- `DescribeTopic()`: Get topic details +- `DescribeTopicConfig()`: Get topic configuration +- `UpdateTopic()`: Modify topic partitions + +**produce.go** +- `ProduceMessage()`: Send messages to topics + +**status.go** +- `GetAdminClient()`: Get Sarama admin client +- `GetClient()`: Get Sarama client +- `NewFailure()`: Error handling utility + +##### Session Package (`pkg/session/`) +- Session state management +- Broker connection persistence +- Client lifecycle management + +##### Logger Package (`pkg/logger/`) +- Structured logging +- Configurable log levels +- Pretty formatting for development + +#### Error Handling + +Custom `Failure` type for consistent error responses: +```go +type Failure struct { + Err error + HttpCode int +} +``` + +All command functions return `(result, *Failure)` for uniform error handling. + +### 3. REST API Layer + +#### API Design + +**Base URL**: `/api/v1` + +**Response Format**: +```json +{ + "status": "ok|error", + "data": { ... }, + "message": "..." +} +``` + +#### Endpoint Categories + +##### Health & Status +- `GET /health`: API health check +- `GET /status`: Broker status + +##### Cluster Management +- `GET /clusters`: List all clusters +- `GET /clusters/{id}/metadata`: Get cluster metadata + +##### Broker Management +- `GET /brokers`: List all brokers + +##### Topic Management +- `GET /topics`: List all topics +- `POST /topics`: Create topic +- `GET /topics/{name}`: Get topic details +- `DELETE /topics/{name}`: Delete topic + +##### Consumer Groups +- `GET /consumers`: List consumer groups +- `GET /consumers/{group}`: Get group details + +##### Message Production +- `POST /messages/{topic}`: Produce message + +### 4. Kafka Integration Layer + +#### Sarama Client Usage + +**ClusterAdmin Interface**: +- Topic CRUD operations +- Configuration management +- Partition management + +**Client Interface**: +- Broker discovery +- Metadata retrieval +- Connection management + +**Producer Interface**: +- Message production +- Partition selection +- Acknowledgment handling + +#### Connection Management + +```go +// Session maintains Kafka connections +type Session struct { + brokers []string + client sarama.Client + adminClient sarama.ClusterAdmin + authenticated bool +} +``` + +## Data Flow Patterns + +### 1. CLI Command Flow + +``` +User Input → Cobra Command → Core Command Function → Sarama Client → Kafka + ↓ +User Output ← Formatted Response ← Result/Error ← Response ← Kafka +``` + +### 2. Web UI Flow + +``` +User Action → React Component → API Call → REST Handler → Core Command → Sarama → Kafka + ↓ +UI Update ← State Update ← JSON Response ← HTTP Response ← Result ← Response ← Kafka +``` + +### 3. Real-time Metrics Flow + +``` +Timer → API Poll → Multiple Endpoints → Aggregate Data → Update State → Re-render + ↓ +3s interval +``` + +## Deployment Architecture + +### Development Environment + +``` +┌─────────────────────────────────────────────────────────┐ +│ Docker/Podman Compose Environment │ +│ │ +│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ +│ │ Kafka │ │ Kafka │ │ Kafka │ │ +│ │ Cluster 1 │ │ Cluster 2 │ │ Cluster 3 │ │ +│ │ :9092 │ │ :9095 │ │ :9098 │ │ +│ └──────────────┘ └──────────────┘ └──────────────┘ │ +│ │ +│ ┌──────────────────────────────────────────────────┐ │ +│ │ OpenKommander Container │ │ +│ │ - Go Backend │ │ +│ │ - React Frontend (built) │ │ +│ │ - Port 8081 │ │ +│ └──────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────┘ +``` + +### Production Deployment Options + +#### Option 1: Standalone Binary +``` +OpenKommander CLI → Kafka Cluster(s) +``` + +#### Option 2: Containerized Service +``` +Container (Go + Frontend) → Load Balancer → Kafka Cluster(s) +``` + +#### Option 3: Kubernetes Deployment +``` +K8s Pod (OpenKommander) → K8s Service → Kafka StatefulSet +``` + +## Security Architecture + +### Authentication Flow +``` +User → Login Command → Session Creation → Broker Connection → Authenticated Session +``` + +### Session Security +- Session data stored in `~/.ok/` +- File permissions restrict access +- No passwords stored (relies on Kafka security) + +### API Security +- CORS configuration for web access +- Session validation for all operations +- No built-in authentication (delegates to Kafka) + +## Scalability Considerations + +### Backend +- Stateless REST API (horizontal scaling possible) +- Connection pooling for Kafka clients +- Efficient session management + +### Frontend +- Code splitting for faster loads +- Lazy loading of components +- Optimized bundle size + +### Kafka Integration +- Multiple broker support +- Partition-aware operations +- Efficient metadata caching + +## Performance Optimizations + +### Backend +- Connection reuse via session management +- Minimal memory footprint +- Fast JSON serialization +- Efficient error handling + +### Frontend +- Virtual scrolling for large tables +- Debounced API calls +- Optimistic UI updates +- Configurable polling intervals + +### Kafka Operations +- Batch operations where possible +- Efficient metadata queries +- Smart partition selection + +## Monitoring & Observability + +### Logging +- Structured logging with levels (Debug, Info, Warn, Error) +- Configurable output format (pretty/JSON) +- Request/response logging in API + +### Metrics (Available via UI) +- Broker count +- Topic count +- Messages per minute +- Consumer lag +- Cluster health status + +### Error Tracking +- Consistent error format +- HTTP status codes +- User-friendly error messages +- Stack traces in debug mode + +## Technology Decisions + +### Why Go? +- Excellent Kafka client library (Sarama) +- Fast compilation and execution +- Strong concurrency support +- Easy deployment (single binary) + +### Why React? +- Component reusability +- Large ecosystem +- Excellent developer experience +- Strong community support + +### Why Carbon Design System? +- Enterprise-grade components +- Accessibility built-in +- Consistent design language +- IBM backing and support + +### Why Sarama? +- Mature and stable +- Full Kafka protocol support +- Active maintenance +- Production-proven + +## Future Architecture Enhancements + +### Planned Improvements +1. **Microservices Split**: Separate API and worker services +2. **Message Queue**: Add async job processing +3. **Caching Layer**: Redis for metadata caching +4. **Database**: Persist metrics and history +5. **Authentication**: OAuth2/OIDC integration +6. **Multi-tenancy**: Support multiple users +7. **WebSocket**: Real-time updates without polling +8. **Schema Registry**: Avro/Protobuf support + +### Scalability Roadmap +1. Horizontal scaling of API servers +2. Load balancing for high availability +3. Distributed session management +4. Metrics aggregation service +5. Historical data storage and analysis \ No newline at end of file From c0baf9dd76c05b07eb35a1b65a1ffb8c4d0934d3 Mon Sep 17 00:00:00 2001 From: Doga Tav Date: Thu, 8 Jan 2026 11:36:06 -0400 Subject: [PATCH 03/18] docs: add complete REST API reference with corrected endpoints --- docs/API_REFERENCE.md | 483 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 483 insertions(+) create mode 100644 docs/API_REFERENCE.md diff --git a/docs/API_REFERENCE.md b/docs/API_REFERENCE.md new file mode 100644 index 0000000..79aff42 --- /dev/null +++ b/docs/API_REFERENCE.md @@ -0,0 +1,483 @@ +# OpenKommander API Reference + +## Overview + +The OpenKommander REST API provides programmatic access to Kafka cluster management operations. All endpoints return JSON responses and follow RESTful conventions. + +**Base URL**: `http://localhost:8081/api/v1/{broker}` + +**Note**: Most endpoints require a `{broker}` path parameter specifying the broker address (e.g., `localhost:9092`). + +**Content-Type**: `application/json` + +## Authentication + +Currently, the API does not require authentication. Access control is managed through Kafka broker security settings. + +## Response Format + +### Success Response +```json +{ + "status": "ok", + "data": { ... } +} +``` + +### Error Response +```json +{ + "status_code": 404, + "message": "Resource not found", + "error": "detailed error message" +} +``` + +## Common HTTP Status Codes + +| Code | Meaning | Description | +|------|---------|-------------| +| 200 | OK | Request successful | +| 201 | Created | Resource created successfully | +| 202 | Accepted | Request accepted for processing | +| 400 | Bad Request | Invalid request parameters | +| 404 | Not Found | Resource not found | +| 409 | Conflict | Resource already exists | +| 500 | Internal Server Error | Server error occurred | + +--- + +## Health & Status Endpoints + +### Health Check + +Check if the API is running and healthy. + +**Endpoint**: `GET /{broker}/health` + +**Path Parameters**: +- `broker` (string): Broker address (e.g., `localhost:9092`) + +**Response**: +```json +{ + "status": "ok", + "message": "Health check successful" +} +``` + +**Example**: +```bash +curl http://localhost:8081/api/v1/localhost:9092/health +``` + +--- + +## Broker Endpoints + +### List Brokers + +Get information about all brokers in the cluster. + +**Endpoint**: `GET /{broker}/brokers` + +**Path Parameters**: +- `broker` (string): Broker address (e.g., `localhost:9092`) + +**Response**: +```json +{ + "status": "ok", + "data": [ + { + "id": 1, + "addr": "localhost:9092", + "connected": true, + "rack": "", + "state": null + } + ] +} +``` + +**Response Fields**: +- `id` (integer): Broker ID +- `addr` (string): Broker address +- `connected` (boolean): Connection status +- `rack` (string): Rack assignment +- `state` (object): TLS connection state (if applicable) + +**Example**: +```bash +curl http://localhost:8081/api/v1/localhost:9092/brokers +``` + +--- + +## Cluster Endpoints + +### List Clusters + +Get information about all available clusters/brokers. + +**Endpoint**: `GET /clusters` + +**Response**: +```json +{ + "status": "ok", + "data": [ + { + "id": 1, + "address": "localhost:9092", + "status": "Connected", + "rack": "rack-1", + "connected": true + } + ] +} +``` + +**Response Fields**: +- `id` (integer): Cluster/broker ID +- `address` (string): Network address (host:port) +- `status` (string): Connection status ("Connected" or "Disconnected") +- `rack` (string): Rack assignment (or "N/A") +- `connected` (boolean): Connection state + +**Example**: +```bash +curl http://localhost:8081/api/v1/clusters +``` + +### Get Cluster Metadata + +Get detailed metadata about a specific cluster. + +**Endpoint**: `GET /clusters/{clusterId}/metadata` + +**Path Parameters**: +- `clusterId` (string): The cluster identifier + +**Response**: +```json +{ + "status": "ok", + "data": { + "broker_count": 3, + "cluster_id": "openkommander-client", + "brokers": [ + { + "id": 1, + "address": "localhost:9092", + "connected": true, + "rack": "rack-1" + } + ] + } +} +``` + +**Example**: +```bash +curl http://localhost:8081/api/v1/clusters/1/metadata +``` + +--- + +## Topic Endpoints + +### List Topics + +Get all topics in the cluster. + +**Endpoint**: `GET /topics` + +**Response**: +```json +[ + { + "name": "test-topic", + "partitions": 3, + "replication_factor": 1, + "internal": false, + "replicas": 3, + "in_sync_replicas": 3, + "cleanup_policy": "delete" + } +] +``` + +**Response Fields**: +- `name` (string): Topic name +- `partitions` (integer): Number of partitions +- `replication_factor` (integer): Replication factor +- `internal` (boolean): Whether topic is internal +- `replicas` (integer): Total replicas +- `in_sync_replicas` (integer): In-sync replicas +- `cleanup_policy` (string): Cleanup policy ("delete" or "compact") + +**Example**: +```bash +curl http://localhost:8081/api/v1/topics +``` + +### Create Topic + +Create a new Kafka topic. + +**Endpoint**: `POST /topics` + +**Request Body**: +```json +{ + "name": "new-topic", + "partitions": 3, + "replication_factor": 1 +} +``` + +**Request Fields**: +- `name` (string, required): Topic name +- `partitions` (integer, required): Number of partitions (min: 1) +- `replication_factor` (integer, required): Replication factor (min: 1) + +**Response**: +```json +{ + "status": "created" +} +``` + +**Example**: +```bash +curl -X POST http://localhost:8081/api/v1/topics \ + -H "Content-Type: application/json" \ + -d '{ + "name": "my-topic", + "partitions": 3, + "replication_factor": 1 + }' +``` + +**Error Responses**: +- `400`: Invalid parameters (partitions < 1, replication_factor > broker count) +- `500`: Topic already exists or creation failed + +### Get Topic Details + +Get detailed information about a specific topic. + +**Endpoint**: `GET /topics/{name}` + +**Path Parameters**: +- `name` (string): Topic name + +**Response**: +```json +{ + "name": "test-topic", + "partitions": 3, + "replication_factor": 1, + "partition_ids": [0, 1, 2] +} +``` + +**Example**: +```bash +curl http://localhost:8081/api/v1/topics/test-topic +``` + +### Delete Topic + +Delete an existing topic. + +**Endpoint**: `DELETE /topics/{name}` + +**Path Parameters**: +- `name` (string): Topic name + +**Response**: +```json +{ + "status": "deleted" +} +``` + +**Example**: +```bash +curl -X DELETE http://localhost:8081/api/v1/topics/test-topic +``` + +**Error Responses**: +- `500`: Deletion failed + +--- + +## Error Handling + +### Error Response Format + +All errors follow this format: + +```json +{ + "status_code": 404, + "message": "User-friendly error message", + "error": "Detailed technical error" +} +``` + +### Common Errors + +#### 400 Bad Request +```json +{ + "status_code": 400, + "message": "Invalid request parameters", + "error": "Partitions must be greater than 0" +} +``` + +#### 404 Not Found +```json +{ + "status_code": 404, + "message": "Topic 'test-topic' not found", + "error": "resource not found" +} +``` + +#### 409 Conflict +```json +{ + "status_code": 409, + "message": "Consumer ID already exists", + "error": "duplicate resource" +} +``` + +#### 500 Internal Server Error +```json +{ + "status_code": 500, + "message": "Error connecting to cluster", + "error": "connection refused" +} +``` + +--- + +## Rate Limiting + +Currently, there are no rate limits enforced. For production use, consider implementing rate limiting at the reverse proxy level. + +--- + +## CORS + +The API supports CORS for web browser access. Configure CORS settings based on your deployment requirements. + +--- + +## Pagination + +Currently, the API does not support pagination. All list endpoints return complete results. For large datasets, consider implementing pagination in future versions. + +--- + +## Versioning + +The API is versioned through the URL path (`/api/v1`). Breaking changes will result in a new version (`/api/v2`). + +--- + +## SDK Examples + +### JavaScript/Node.js + +```javascript +const axios = require('axios'); + +const api = axios.create({ + baseURL: 'http://localhost:8081/api/v1', + headers: { 'Content-Type': 'application/json' } +}); + +// List topics +const topics = await api.get('/topics'); +console.log(topics.data); + +// Create topic +await api.post('/topics', { + name: 'my-topic', + partitions: 3, + replication_factor: 1 +}); + +// Produce message +await api.post('/messages/my-topic', { + message: 'Hello, Kafka!' +}); +``` + +### Python + +```python +import requests + +BASE_URL = 'http://localhost:8081/api/v1' + +# List topics +response = requests.get(f'{BASE_URL}/topics') +topics = response.json() + +# Create topic +requests.post(f'{BASE_URL}/topics', json={ + 'name': 'my-topic', + 'partitions': 3, + 'replication_factor': 1 +}) + +# Produce message +requests.post(f'{BASE_URL}/messages/my-topic', json={ + 'message': 'Hello, Kafka!' +}) +``` + +### cURL + +```bash +# List topics +curl http://localhost:8081/api/v1/topics + +# Create topic +curl -X POST http://localhost:8081/api/v1/topics \ + -H "Content-Type: application/json" \ + -d '{"name":"my-topic","partitions":3,"replication_factor":1}' + +# Produce message +curl -X POST http://localhost:8081/api/v1/messages/my-topic \ + -H "Content-Type: application/json" \ + -d '{"message":"Hello, Kafka!"}' +``` + +--- + +## WebSocket Support + +WebSocket support is planned for future releases to enable real-time updates without polling. + +--- + +## OpenAPI Specification + +The complete OpenAPI 3.0 specification is available at `docs/openapi.yaml`. You can use tools like Swagger UI or Postman to explore the API interactively. + +--- + +## Support + +For API issues or questions: +- GitHub Issues: https://github.com/IBM/openkommander/issues +- Documentation: `/docs` directory \ No newline at end of file From bca11e2c6da054166f0ff41df74da1aa4ba680ca Mon Sep 17 00:00:00 2001 From: Doga Tav Date: Thu, 8 Jan 2026 11:36:16 -0400 Subject: [PATCH 04/18] docs: add comprehensive user guide with CLI and Web UI instructions --- docs/USER_GUIDE.md | 729 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 729 insertions(+) create mode 100644 docs/USER_GUIDE.md diff --git a/docs/USER_GUIDE.md b/docs/USER_GUIDE.md new file mode 100644 index 0000000..417e69d --- /dev/null +++ b/docs/USER_GUIDE.md @@ -0,0 +1,729 @@ +# OpenKommander User Guide + +## Table of Contents + +1. [Introduction](#introduction) +2. [Installation](#installation) +3. [Getting Started](#getting-started) +4. [CLI Usage](#cli-usage) +5. [Web UI Usage](#web-ui-usage) +6. [Common Tasks](#common-tasks) +7. [Troubleshooting](#troubleshooting) +8. [Best Practices](#best-practices) + +--- + +## Introduction + +OpenKommander is a comprehensive tool for managing Apache Kafka clusters. It provides both a command-line interface (CLI) and a web-based user interface for performing common Kafka administration tasks. + +### What You Can Do + +- **Manage Topics**: Create, list, describe, update, and delete topics +- **Monitor Brokers**: View broker status and health +- **Produce Messages**: Send test messages to topics +- **Monitor Consumer Groups**: Track consumer lag and status +- **Manage Clusters**: Connect to and manage multiple Kafka clusters + +--- + +## Installation + +### Quick Installation (Recommended) + +Use the interactive setup script: + +```bash +git clone https://github.com/IBM/openkommander.git +cd openkommander +./scripts/setup.sh +``` + +The script will guide you through: +- Local installation (direct on your system) +- Docker installation +- Podman installation + +### Manual Installation + +#### Prerequisites +- Go 1.24 or higher +- Make +- Docker or Podman (for containerized setup) + +#### Steps + +1. **Clone the repository**: + ```bash + git clone https://github.com/IBM/openkommander.git + cd openkommander + ``` + +2. **Build the CLI**: + ```bash + make setup + make build + ``` + +3. **Install the CLI**: + ```bash + sudo make install + ``` + +4. **Verify installation**: + ```bash + ok --version + ``` + +--- + +## Getting Started + +### First-Time Setup + +1. **Start a Kafka cluster** (if you don't have one): + ```bash + make container-start + ``` + +2. **Connect to your cluster**: + ```bash + ok login + ``` + + You'll be prompted for broker addresses. For the development environment: + ``` + Enter broker addresses (comma-separated): localhost:9092 + ``` + +3. **Verify connection**: + ```bash + ok session + ``` + +4. **View cluster information**: + ```bash + ok metadata + ``` + +### Starting the Web UI + +1. **Start the REST server**: + ```bash + ok server start -p 8081 + ``` + +2. **Open your browser**: + Navigate to `http://localhost:8081` + +--- + +## CLI Usage + +### Basic Commands + +#### Session Management + +**Login to a cluster**: +```bash +ok login +``` + +**View current session**: +```bash +ok session +``` + +**Logout**: +```bash +ok logout +``` + +#### Cluster Information + +**View cluster metadata**: +```bash +ok metadata +``` + +**List clusters/brokers**: +```bash +ok cluster list +``` + +**View broker information**: +```bash +ok broker info +``` + +### Topic Management + +#### Create a Topic + +**Interactive mode**: +```bash +ok topic create my-topic +``` +You'll be prompted for partitions and replication factor. + +**With flags**: +```bash +ok topic create my-topic -p 3 -r 1 +``` + +**Flags**: +- `-p, --partitions`: Number of partitions +- `-r, --replication-factor`: Replication factor + +#### List Topics + +```bash +ok topic list +``` + +#### Describe a Topic + +```bash +ok topic describe my-topic +``` + +This shows: +- Topic metadata (name, replication factor, UUID) +- Partition details (leader, replicas, ISR) +- Configuration settings + +#### Update a Topic + +Increase partition count: +```bash +ok topic update my-topic -p 5 +``` + +**Note**: You can only increase partitions, not decrease them. + +#### Delete a Topic + +```bash +ok topic delete my-topic +``` + +You'll be prompted to confirm the deletion. + +### Message Production + +#### Basic Message + +```bash +ok produce my-topic -m "Hello, Kafka!" +``` + +#### Message with Key + +```bash +ok produce my-topic -k "user123" -m "User login event" +``` + +#### Specific Partition + +```bash +ok produce my-topic -m "Hello" -p 0 +``` + +#### Custom Acknowledgments + +```bash +ok produce my-topic -m "Important message" -a 1 +``` + +**Acknowledgment Options**: +- `-1`: Wait for all in-sync replicas (default, most reliable) +- `0`: No acknowledgment (fastest, least reliable) +- `1`: Wait for leader only (balanced) + +### Server Management + +**Start the REST server**: +```bash +ok server start -p 8081 +``` + +**With specific brokers**: +```bash +ok server start -p 8081 --brokers localhost:9092,localhost:9095 +``` + +--- + +## Web UI Usage + +### Overview Dashboard + +The Overview page displays real-time cluster metrics: + +- **Broker Count**: Number of active brokers +- **Topic Count**: Total topics in the cluster +- **Health Status**: Overall cluster health +- **Messages Per Minute**: Message throughput + +**Features**: +- Auto-refresh toggle (updates every 3 seconds) +- Last refresh timestamp +- Visual metric cards + +### Topics Page + +Manage all your Kafka topics from one interface. + +**View Topics**: +- Table shows: name, partitions, replication factor, status +- Status indicators: Active (green), Warning (yellow), Inactive (red) + +**Create Topic**: +1. Click "Add Topic" button +2. Fill in the form: + - Topic Name + - Number of Partitions + - Replication Factor +3. Click "Create" + +**Delete Topic**: +1. Click the delete icon next to the topic +2. Confirm deletion in the dialog + +**View Details**: +- Click on a topic row to see detailed information +- View partition distribution +- Check replication status + +### Brokers Page + +Monitor broker health and status. + +**Information Displayed**: +- Broker ID +- Host and Port +- Status (Selected/Inactive) +- Partition Count +- Leader Partitions +- In-Sync Partitions + +**Features**: +- Real-time status updates +- Connection status indicators +- Partition distribution view + +### Consumer Groups Page + +Monitor consumer group activity and lag. + +**Information Displayed**: +- Group ID +- Number of Members +- Subscribed Topics +- Total Lag +- Coordinator Broker +- State (Stable/Rebalancing/Dead) + +**Features**: +- View topic-level lag +- Monitor consumer group health +- Track rebalancing events + +**Status Indicators**: +- **Stable** (green): Group is healthy and consuming +- **Rebalancing** (yellow): Group is rebalancing partitions +- **Dead** (red): Group has no active members + +--- + +## Common Tasks + +### Task 1: Create and Test a New Topic + +```bash +# 1. Create the topic +ok topic create test-topic -p 3 -r 1 + +# 2. Verify creation +ok topic list + +# 3. View details +ok topic describe test-topic + +# 4. Send a test message +ok produce test-topic -m "Test message" + +# 5. Verify in web UI +# Open http://localhost:8081 and check Topics page +``` + +### Task 2: Monitor Consumer Lag + +**Via CLI**: +```bash +# View all consumer groups +ok cluster list + +# Check specific group (via API) +curl http://localhost:8081/api/v1/consumers/my-consumer-group +``` + +**Via Web UI**: +1. Navigate to Consumer Groups page +2. Look for groups with high lag values +3. Check if state is "Stable" +4. Review topic-level lag breakdown + +### Task 3: Increase Topic Partitions + +```bash +# 1. Check current partition count +ok topic describe my-topic + +# 2. Increase partitions +ok topic update my-topic -p 6 + +# 3. Verify the change +ok topic describe my-topic +``` + +### Task 4: Connect to Multiple Clusters + +```bash +# Connect to first cluster +ok login +# Enter: localhost:9092 + +# Perform operations... + +# Connect to second cluster +ok logout +ok login +# Enter: localhost:9095 + +# Now working with second cluster +``` + +### Task 5: Bulk Message Production + +Create a script to send multiple messages: + +```bash +#!/bin/bash +for i in {1..100}; do + ok produce my-topic -m "Message $i" +done +``` + +### Task 6: Export Topic List + +```bash +# Using CLI +ok topic list > topics.txt + +# Using API +curl http://localhost:8081/api/v1/topics > topics.json +``` + +--- + +## Troubleshooting + +### Connection Issues + +**Problem**: Cannot connect to Kafka cluster + +**Solutions**: +1. Verify broker addresses: + ```bash + # Test connectivity + telnet localhost 9092 + ``` + +2. Check if Kafka is running: + ```bash + make container-logs + ``` + +3. Verify network access: + ```bash + # Check firewall rules + # Ensure ports are not blocked + ``` + +### Topic Creation Fails + +**Problem**: "Replication factor cannot be greater than the number of brokers" + +**Solution**: Reduce replication factor to match or be less than broker count: +```bash +# If you have 1 broker, use replication factor 1 +ok topic create my-topic -p 3 -r 1 +``` + +### Session Not Found + +**Problem**: "Error: no session found" + +**Solution**: Login to establish a session: +```bash +ok login +``` + +### Web UI Not Loading + +**Problem**: Cannot access http://localhost:8081 + +**Solutions**: +1. Verify server is running: + ```bash + # Check if port is in use + lsof -i :8081 + ``` + +2. Start the server: + ```bash + ok server start -p 8081 + ``` + +3. Try a different port: + ```bash + ok server start -p 8082 + ``` + +### Permission Denied + +**Problem**: "Permission denied" when installing + +**Solution**: Use sudo for installation: +```bash +sudo make install +``` + +### Port Already in Use + +**Problem**: "Address already in use" + +**Solutions**: +1. Find what's using the port: + ```bash + lsof -i :8081 + ``` + +2. Kill the process or use a different port: + ```bash + ok server start -p 8082 + ``` + +--- + +## Best Practices + +### Topic Management + +1. **Plan Partition Count**: Consider your throughput needs + - More partitions = higher parallelism + - But more partitions = more overhead + - Start with 3-6 partitions for most use cases + +2. **Set Appropriate Replication Factor**: + - Production: Use replication factor of 3 + - Development: Replication factor of 1 is acceptable + - Never exceed broker count + +3. **Use Meaningful Names**: + ```bash + # Good + ok topic create user-events -p 3 -r 1 + + # Avoid + ok topic create topic1 -p 3 -r 1 + ``` + +4. **Test Before Production**: + - Create topics in dev environment first + - Test with sample data + - Verify consumer behavior + +### Message Production + +1. **Use Message Keys for Ordering**: + ```bash + # Messages with same key go to same partition + ok produce orders -k "user123" -m "Order placed" + ``` + +2. **Choose Appropriate Acknowledgments**: + - Critical data: Use `-a -1` (all ISR) + - Logs/metrics: Use `-a 1` (leader only) + - High throughput: Use `-a 0` (no ack) + +3. **Monitor Production Success**: + - Check for errors in output + - Verify messages in consumer groups + +### Monitoring + +1. **Regular Health Checks**: + ```bash + # Daily health check script + ok metadata + ok broker info + ok cluster list + ``` + +2. **Watch Consumer Lag**: + - Check lag regularly via Web UI + - Alert on high lag values + - Investigate rebalancing events + +3. **Enable Auto-Refresh in Web UI**: + - Keep dashboard open for monitoring + - Set appropriate refresh interval + - Watch for status changes + +### Security + +1. **Protect Session Files**: + ```bash + # Session data stored in ~/.ok/ + chmod 700 ~/.ok + ``` + +2. **Use Secure Connections**: + - Configure SSL/TLS in Kafka + - Use SASL authentication + - Restrict network access + +3. **Limit Access**: + - Use Kafka ACLs + - Restrict topic operations + - Monitor audit logs + +### Performance + +1. **Batch Operations**: + - Create multiple topics in scripts + - Use API for bulk operations + - Minimize individual CLI calls + +2. **Optimize Partition Count**: + - Balance between parallelism and overhead + - Consider consumer count + - Monitor broker load + +3. **Monitor Resource Usage**: + - Watch broker CPU/memory + - Check disk usage + - Monitor network throughput + +--- + +## Advanced Usage + +### Scripting with CLI + +Create automation scripts: + +```bash +#!/bin/bash +# setup-topics.sh + +TOPICS=("orders" "payments" "notifications") + +for topic in "${TOPICS[@]}"; do + ok topic create "$topic" -p 3 -r 1 + echo "Created topic: $topic" +done +``` + +### Using the API Programmatically + +```python +import requests + +API_BASE = "http://localhost:8081/api/v1" + +# Create multiple topics +topics = ["topic1", "topic2", "topic3"] +for topic in topics: + response = requests.post( + f"{API_BASE}/topics", + json={ + "name": topic, + "partitions": 3, + "replication_factor": 1 + } + ) + print(f"Created {topic}: {response.status_code}") +``` + +### Integration with CI/CD + +```yaml +# .github/workflows/kafka-setup.yml +name: Setup Kafka Topics + +on: + push: + branches: [main] + +jobs: + setup: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v2 + - name: Install OpenKommander + run: | + git clone https://github.com/IBM/openkommander.git + cd openkommander + make build + sudo make install + - name: Create Topics + run: | + ok login + ok topic create production-events -p 6 -r 3 +``` + +--- + +## Getting Help + +### Documentation +- Project Overview: `docs/PROJECT_OVERVIEW.md` +- Architecture: `docs/ARCHITECTURE.md` +- API Reference: `docs/API_REFERENCE.md` +- Development Guide: `docs/DEVELOPMENT_GUIDE.md` + +### Command Help +```bash +# General help +ok help + +# Command-specific help +ok topic --help +ok produce --help +``` + +### Community Support +- GitHub Issues: https://github.com/IBM/openkommander/issues +- Discussions: GitHub Discussions +- Contributing: See `CONTRIBUTING.md` + +### Reporting Issues +When reporting issues, include: +1. OpenKommander version +2. Operating system +3. Kafka version +4. Steps to reproduce +5. Error messages +6. Relevant logs + +--- + +## Next Steps + +- Explore the [API Reference](API_REFERENCE.md) for programmatic access +- Read the [Development Guide](DEVELOPMENT_GUIDE.md) to contribute +- Check out [Architecture](ARCHITECTURE.md) to understand internals +- Review [Best Practices](#best-practices) for production use \ No newline at end of file From 657e0d5275c444282c3e14e578afbe099ae6d01c Mon Sep 17 00:00:00 2001 From: Doga Tav Date: Thu, 8 Jan 2026 11:36:22 -0400 Subject: [PATCH 05/18] docs: add development guide with setup, build, and contribution instructions --- docs/DEVELOPMENT_GUIDE.md | 788 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 788 insertions(+) create mode 100644 docs/DEVELOPMENT_GUIDE.md diff --git a/docs/DEVELOPMENT_GUIDE.md b/docs/DEVELOPMENT_GUIDE.md new file mode 100644 index 0000000..44efcd3 --- /dev/null +++ b/docs/DEVELOPMENT_GUIDE.md @@ -0,0 +1,788 @@ +# OpenKommander Development Guide + +## Table of Contents + +1. [Development Environment Setup](#development-environment-setup) +2. [Project Structure](#project-structure) +3. [Building the Project](#building-the-project) +4. [Running Tests](#running-tests) +5. [Development Workflow](#development-workflow) +6. [Code Style Guidelines](#code-style-guidelines) +7. [Contributing](#contributing) +8. [Debugging](#debugging) +9. [Release Process](#release-process) + +--- + +## Development Environment Setup + +### Prerequisites + +- **Go**: Version 1.24 or higher +- **Node.js**: Version 18 or higher +- **npm**: Version 9 or higher +- **Make**: GNU Make +- **Podman** or **Docker**: For containerized development +- **Git**: Version control + +### Initial Setup + +1. **Clone the repository**: + ```bash + git clone https://github.com/IBM/openkommander.git + cd openkommander + ``` + +2. **Install Go dependencies**: + ```bash + make setup + ``` + +3. **Install frontend dependencies**: + ```bash + cd frontend + npm install + cd .. + ``` + +4. **Start development environment**: + ```bash + make container-start + ``` + +5. **Verify setup**: + ```bash + make container-logs + ``` + +### IDE Setup + +#### VS Code + +Recommended extensions: +- Go (golang.go) +- ESLint (dbaeumer.vscode-eslint) +- Prettier (esbenp.prettier-vscode) +- Docker (ms-azuretools.vscode-docker) + +**Settings** (`.vscode/settings.json`): +```json +{ + "go.useLanguageServer": true, + "go.lintTool": "golangci-lint", + "editor.formatOnSave": true, + "go.formatTool": "goimports", + "[javascript]": { + "editor.defaultFormatter": "esbenp.prettier-vscode" + }, + "[javascriptreact]": { + "editor.defaultFormatter": "esbenp.prettier-vscode" + } +} +``` + +#### GoLand / IntelliJ IDEA + +1. Open project directory +2. Enable Go modules support +3. Configure Go SDK (1.24+) +4. Enable ESLint for frontend code + +--- + +## Project Structure + +``` +openkommander/ +├── internal/ # Internal packages (not importable) +│ └── core/ +│ └── commands/ # Core business logic +│ ├── broker.go # Broker operations +│ ├── cluster.go # Cluster management +│ ├── topic.go # Topic operations +│ ├── produce.go # Message production +│ └── status.go # Status and error handling +├── pkg/ # Public packages (importable) +│ ├── cli/ # CLI implementation +│ ├── cluster/ # Cluster utilities +│ ├── constants/ # Application constants +│ ├── logger/ # Logging utilities +│ ├── rest/ # REST API server +│ └── session/ # Session management +├── frontend/ # React web application +│ ├── src/ +│ │ ├── components/ # React components +│ │ │ ├── common/ # Reusable components +│ │ │ ├── layout/ # Layout components +│ │ │ └── overview/ # Overview-specific components +│ │ ├── pages/ # Page components +│ │ ├── services/ # API services +│ │ ├── hooks/ # Custom React hooks +│ │ └── config/ # Configuration +│ ├── public/ # Static assets +│ └── package.json # Frontend dependencies +├── docs/ # Documentation +├── scripts/ # Utility scripts +├── tests/ # Test files +├── docker/ # Docker configurations +├── main.go # Application entry point +├── go.mod # Go module definition +├── go.sum # Go dependency checksums +├── Makefile # Build automation +└── README.md # Project README +``` + +### Key Directories + +- **`internal/core/commands/`**: Core Kafka operations +- **`pkg/cli/`**: CLI command definitions using Cobra +- **`pkg/session/`**: Session state management +- **`frontend/src/components/`**: React UI components +- **`frontend/src/services/`**: API client code + +--- + +## Building the Project + +### Backend (Go) + +**Build CLI binary**: +```bash +make build +``` + +Output: `./openkommander` (or `./openkommander.exe` on Windows) + +**Install CLI globally**: +```bash +sudo make install +``` + +Installs to: `/usr/local/bin/ok` + +**Development build and install**: +```bash +make dev-run +``` + +This runs: `setup` → `build` → `install` + +### Frontend (React) + +**Development server**: +```bash +cd frontend +npm run dev +``` + +Access at: `http://localhost:5173` + +**Production build**: +```bash +cd frontend +npm run build +``` + +Output: `frontend/dist/` + +**Build and copy to config directory**: +```bash +make frontend-build +``` + +Copies to: `~/.ok/frontend/` + +### Full Build + +Build both backend and frontend: +```bash +make build +make frontend-build +``` + +--- + +## Running Tests + +### Backend Tests + +**Run all tests**: +```bash +go test ./... +``` + +**Run tests with coverage**: +```bash +go test -cover ./... +``` + +**Run tests with verbose output**: +```bash +go test -v ./... +``` + +**Run specific package tests**: +```bash +go test ./internal/core/commands/ +``` + +**Generate coverage report**: +```bash +go test -coverprofile=coverage.out ./... +go tool cover -html=coverage.out +``` + +### Frontend Tests + +**Run tests**: +```bash +cd frontend +npm test +``` + +**Run tests with coverage**: +```bash +cd frontend +npm run test:coverage +``` + +**Run tests in watch mode**: +```bash +cd frontend +npm run test:watch +``` + +### Integration Tests + +See `tests/TESTING_GUIDELINES.md` for integration testing procedures. + +--- + +## Development Workflow + +### 1. Feature Development + +**Create a feature branch**: +```bash +git checkout -b feature/my-new-feature +``` + +**Make changes and test**: +```bash +# Backend changes +make build +./openkommander [command] + +# Frontend changes +cd frontend +npm run dev +``` + +**Commit changes**: +```bash +git add . +git commit -m "feat: add new feature" +``` + +### 2. Code Review Process + +1. Push branch to GitHub +2. Create Pull Request +3. Address review comments +4. Ensure CI passes +5. Merge when approved + +### 3. Local Testing + +**Start development environment**: +```bash +make container-start +``` + +**Execute into container**: +```bash +make container-exec +``` + +**View logs**: +```bash +make container-logs +``` + +**Stop environment**: +```bash +make container-stop +``` + +### 4. Hot Reload Development + +**Backend** (requires manual restart): +```bash +# Terminal 1: Watch for changes +make build && ./openkommander server start -p 8081 + +# Terminal 2: Make changes and rebuild +make build +``` + +**Frontend** (automatic hot reload): +```bash +cd frontend +npm run dev +# Changes auto-reload in browser +``` + +--- + +## Code Style Guidelines + +### Go Code Style + +**Follow standard Go conventions**: +- Use `gofmt` for formatting +- Use `golint` for linting +- Follow [Effective Go](https://golang.org/doc/effective_go.html) + +**Naming conventions**: +```go +// Good +func CreateTopic(name string) error { } +var topicName string + +// Avoid +func create_topic(name string) error { } +var topic_name string +``` + +**Error handling**: +```go +// Always check errors +result, err := someFunction() +if err != nil { + return nil, NewFailure(fmt.Sprintf("operation failed: %v", err), http.StatusInternalServerError) +} +``` + +**Comments**: +```go +// CreateTopic creates a new Kafka topic with the specified configuration. +// Returns success message and nil on success, or empty string and Failure on error. +func CreateTopic(topicName string, numPartitions, replicationFactor int) (string, *Failure) { + // Implementation +} +``` + +**Package organization**: +- Keep packages focused and cohesive +- Avoid circular dependencies +- Use internal packages for non-exported code + +### JavaScript/React Code Style + +**Use Prettier for formatting**: +```bash +cd frontend +npm run format +``` + +**ESLint for linting**: +```bash +cd frontend +npm run lint +``` + +**Component structure**: +```jsx +// Good: Functional component with hooks +import React, { useState, useEffect } from 'react'; + +const MyComponent = ({ prop1, prop2 }) => { + const [state, setState] = useState(null); + + useEffect(() => { + // Effect logic + }, []); + + return ( +
+ {/* JSX */} +
+ ); +}; + +export default MyComponent; +``` + +**Naming conventions**: +- Components: PascalCase (`MyComponent.jsx`) +- Hooks: camelCase with 'use' prefix (`useMyHook.jsx`) +- Utilities: camelCase (`apiClient.js`) +- Constants: UPPER_SNAKE_CASE (`API_BASE_URL`) + +**File organization**: +``` +components/ + MyComponent/ + MyComponent.jsx + MyComponent.scss + MyComponent.test.jsx +``` + +--- + +## Contributing + +### Before Contributing + +1. Read [CONTRIBUTING.md](../CONTRIBUTING.md) +2. Check [CODE_OF_CONDUCT.md](../CODE_OF_CONDUCT.md) +3. Review existing issues and PRs +4. Discuss major changes in an issue first + +### Contribution Workflow + +1. **Fork the repository** +2. **Create a feature branch**: + ```bash + git checkout -b feature/my-feature + ``` +3. **Make your changes** +4. **Add tests** for new functionality +5. **Run tests**: + ```bash + go test ./... + cd frontend && npm test + ``` +6. **Commit with conventional commits**: + ```bash + git commit -m "feat: add new feature" + ``` +7. **Push to your fork**: + ```bash + git push origin feature/my-feature + ``` +8. **Create Pull Request** + +### Commit Message Format + +Follow [Conventional Commits](https://www.conventionalcommits.org/): + +``` +(): + + + +