Skip to content

Commit

Permalink
Export release-8966@e5e05f23
Browse files Browse the repository at this point in the history
  • Loading branch information
Hasso Plattner Institut committed Jan 14, 2025
1 parent b34f5e2 commit 1e80475
Show file tree
Hide file tree
Showing 4 changed files with 175 additions and 0 deletions.
8 changes: 8 additions & 0 deletions docs/app/.pages
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# vim: ft=yaml

nav:
- index.md
- development
- features
- deployment
- ...
11 changes: 11 additions & 0 deletions docs/app/deployment/index.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Deployment

This section provides guidance on deploying and operating our platform software, either [the open source version](opensource) or managing the internal setup.

Our deployment process is designed to streamline the configuration and management of the application and its peripheral services. You'll find detailed information on:

- **Software packages**: Insights into how the application and its services are packaged for deployment (Debian packages).
- **Configuration**: Guidelines on setting up application and service-specific configurations, such as databases, caching, and messaging.
- **CI pipelines**: An overview of the continuous integration pipelines that support deployment automation and ensure robust delivery practices.

Whether you're setting up for the first time or fine-tuning an existing deployment, this guide is your go-to resource for achieving a smooth and efficient operational environment.
147 changes: 147 additions & 0 deletions docs/app/deployment/opensource/index.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
# Operating a platform (Open Source)

This documentation outlines the fundamental requirements and steps to operate the Xikolo platform.
The Xikolo platform is modular and based on the following open-source repositories:

- [xikolo-core](https://github.com/openHPI/xikolo-core): core application with (micro)service-based domain applications.
- [xikolo-lanalytics](https://github.com/openHPI/xikolo-lanalytics): reporting and learning analytics capabilities.

The following information provides an overview to help evaluate whether the platform is suitable for your operational needs.

## Services

The platform consists of several services working together to provide its functionality.
These are divided into necessary and optional services:

### Necessary services

| Service | Function |
|------------------------------------|------------------------------------------------------------------------------------|
| **account-api** | Enables user login and management. |
| **account-sidekiq** | Background job processing with Sidekiq for the account service. |
| **collabspace-api** | Provides a shared space within courses. |
| **collabspace-sidekiq** | Background job processing with Sidekiq for the collabspace service. |
| **course-api** | Lists all courses and allows managing them. |
| **course-msgr** | Asynchronous messaging service for the course service. |
| **course-sidekiq** | Background job processing with Sidekiq for the course service. |
| **grouping-api** | Manages user groups (e.g., for A/B tests). |
| **grouping-sidekiq** | Background job processing with Sidekiq for the grouping service. |
| **lanalytics-api** | Provides detailed platform reports and statistics. |
| **lanalytics-msgr** | Asynchronous messaging service for the lanalytics service. |
| **lanalytics-sidekiq** | Background job processing with Sidekiq for the lanalytics service. |
| **news-api** | Provides course and platform announcements. |
| **news-msgr** | Asynchronous messaging service for the news service. |
| **news-sidekiq** | Background job processing with Sidekiq for the news service. |
| **notification-api** | Sends all kind of emails to users. |
| **notification-msgr** | Asynchronous messaging service for the notification service. |
| **notification-sidekiq** | Background job processing with Sidekiq for the notification service. |
| **web-api** | User interface and main domain application. |
| **web-msgr** | Asynchronous messaging service for the web service. |
| **web-sidekiq** | Background job processing with Sidekiq for the web service. |
| **web-delayed** | Background job processing with Delayed for the web service. |
| ... (more application services) | Service functionality provided for peerassessment, pinboard, quiz, and timeeffort. |
| **elasticsearch-master** | Search engine and event store for lanalytics. |
| **postgres-lanalytics** | Database for the lanalytics service. |
| **postgres-web** | Database for the core services. |
| **rabbitmq** | Asynchronous messaging. |
| **redis-cache** | Application caching. |
| **redis-sidekiq** | Background job processing with Sidekiq. |
| **S3** | Storage backend for files. |
| ... (more infrastructure services) | ... |

### Optional services

| Service | Function |
|--------------------------|-----------------------------------------------------|
| **etherpad-service** | Supports collaborative writing with Etherpad. |
| **h5p-lti-1p0-provider** | Provides interactive exercises using H5P (via LTI). |
| **mongo** | Database for H5P. |
| **zammad-websocket** | Offers helpdesk functionality with Zammad. |
| ... | ... |

## Infrastructure

The platform can be operated using the provided `Dockerfile`s with common container technologies (e.g., using Nomad for orchestration).
An example setup for a platform, which is currently in production, uses the following resources.

!!! note
Based on the actual platform traffic and size, fewer resources may be sufficient.

### Application resources

| Component | Count | Resources |
|-------------------------------------| ----- | ----------------- |
| **Background jobs** | 2 | 4 vCPUs, 8 GB RAM |
| **Services (account, course, ...)** | 3 | 4 vCPUs, 8 GB RAM |
| **Web** | 3 | 4 vCPUs, 8 GB RAM |
| **Tasks (optional management VM)** | 1 | 2 vCPUs, 4 GB RAM |

### Infrastructure resources

| Component | Count | Resources |
| ----------------------- | ----- | ----------------------- |
| **Consul** | 3 | 2 vCPUs, 2 GB RAM |
| **eLB (external LB)** | 2 | 2 vCPUs, 2 GB RAM |
| **iLB (internal LB)** | 2 | 2 vCPUs, 2 GB RAM |
| **RabbitMQ** | 3 | 2 vCPUs, 2 GB RAM |
| **Redis** | 1 | 4 vCPUs, 8 GB RAM, Disk |
| **Postgres** | 3 | 4 vCPUs, 8 GB RAM, Disk |
| **Lanalytics Postgres** | 1 | 4 vCPUs, 8 GB RAM, Disk |
| **Elasticsearch** | 1 | 4 vCPUs, 8 GB RAM, Disk |

### Redundancy and distribution

To ensure stable operation, it is advisable to run services redundantly and distribute them across multiple availability zones.
This approach offers the following benefits:

- **Fault tolerance**: Services remain available in other zones if one zone fails.
- **Load distribution**: Redundancy ensures an even distribution of requests, optimizing performance.
- **Higher availability**: Redundant setups guarantee continuous access to the platform, even during maintenance or unexpected outages.

Infrastructure components (e.g., databases, RabbitMQ, Consul) should also be configured redundantly to ensure critical data and functionalities remain accessible.
A distribution across at least two to three availability zones is recommended.

## Prerequisites for operation

To successfully operate the platform, the following points should be considered:

1. **Hardware resources**: The instance types and specifications outlined above should be provisioned.
2. **Services**: Installation and configuration of Nomad, Consul, and other supporting services.
3. **Configuration**: A valid application configuration is required (i.e., through the `xikolo.yml` file for the application).
This also applies to supporting services like S3. Example configurations are available in the `/docker` directory.

!!! tip
Some application features and functionality as well as user permissions are managed via the database, e.g. via feature flippers.
Knowledge of Ruby / Ruby on Rails is required.

4. **Backup & maintenance**: Regular backups of databases and stored files, as well as updates for the infrastructure services.

!!! important
Additionally, regular updates of the source code are required, including rebasing on the latest application code made available in the repositories mentioned above.

5. **Monitoring**: Effective monitoring is critical to ensure the platform's reliability and performance.

!!! info
The following (optional) services may be integrated for comprehensive monitoring.

- **[Mnemosyne](https://github.com/mnemosyne-mon)**: Provides distributed tracing and performance analysis for applications.
- **Grafana**: Delivers visualizations and dashboards for monitoring system and application metrics.
- **InfluxDB**: Serves as a time-series database for storing performance metrics.
- **Telegraf**: Acts as a metrics collection agent for system and application monitoring.
- **Sentry**: Tracks and reports application errors, offering insights for debugging and stability.

Ensure that these tools are configured properly and routinely monitored to detect and address potential issues proactively.

## Advantages of the platform

- **Modular design**: Services can be enabled or disabled as needed, allowing for tailored configurations to suit specific requirements.
- **Open source**: The platform's source code is openly available, enabling customizations and enhancements while adhering to the AGPLv3 license terms.
- **Scalable architecture**: The platform is designed to handle varying workloads efficiently, making it suitable for small-scale deployments as well as large-scale, high-traffic applications.
Resources can be adjusted dynamically to match the demands of your operations.

## Further information

Detailed installation and configuration instructions can be also found in the `README` files of the repositories:

- [xikolo-core](https://github.com/openHPI/xikolo-core)
- [xikolo-lanalytics](https://github.com/openHPI/xikolo-lanalytics)
9 changes: 9 additions & 0 deletions docs/app/index.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Home

Welcome to the developer documentation for the Xikolo Platform.

It comprises the following areas:

- :material-keyboard: [Development](development/index.md) for everything you need to get started and contribute to the codebase - setup tutorials, tips for troubleshooting and coding guidelines.
- :material-star-box: [Features](features/index.md) for an introduction into product domain terminology and concepts.
- :material-package-variant: [Deployment](deployment/index.md) for all configuration and deployment details to use Xikolo in a production environment.

0 comments on commit 1e80475

Please sign in to comment.