Kolibri Studio

Kolibri Studio is a web application designed to deliver educational materials to Kolibri. It supports:

• Organizing and publishing content channels in the format suitable for import from Kolibri
• Curating content and remixing of existing channels into custom channels aligned to various educational standards, country curricula, and special needs
• Creating learning pathways and assessments
• Uploading new content through the web interface or programatically using ricecooker-powered content import scripts

Kolibri Studio uses Django for the backend and is transitioning from Backbone.js to Vue.js for the frontend.

If you are looking for help setting up custom content channels, uploading and organizing resources using Kolibri Studio, please refer to the User Guide.

Getting started

Get the code

• Install and set up Git on your computer. Try this tutorial if you need more practice
• Sign up and configure your GitHub account if you don't have one already.
• Fork the studio repo to create a copy of the studio repository under your own github username. This will make it easier to submit pull requests. Read more details about forking from GitHub
• Clone your repo locally

Tip: Register your SSH keys on GitHub to avoid having to repeatedly enter your password.

Install and run services

Studio requires some background services to be running:

• Minio
• Postgres
• Redis

The instructions below show how to set up the services using Docker. This works for many people, but not everyone. If docker is giving you issues, you can also manually install the services either on your host machine or in a virtual machine (for example, using Vagrant with Virtualbox or VMWare).

First, install Docker and docker-compose.

After installing the above, you're ready to start the services by running:

make dcservicesup

The above command may take longer the first time it's run.

To confirm that the services are running, you should see three containers when executing docker ps. For example:

> docker ps
CONTAINER ID        IMAGE                             COMMAND                  CREATED             STATUS              PORTS                    NAMES
e09c5c203b93        redis:6.0.9                       "docker-entrypoint.s…"   51 seconds ago      Up 49 seconds       0.0.0.0:6379->6379/tcp   studio_vue-refactor_redis_1
6164371efb6b        minio/minio                       "minio server /data"     51 seconds ago      Up 49 seconds       0.0.0.0:9000->9000/tcp   studio_vue-refactor_minio_1
c86bbfa3a59e        postgres:12.10                      "docker-entrypoint.s…"   51 seconds ago      Up 49 seconds       0.0.0.0:5432->5432/tcp   studio_vue-refactor_postgres_1

To stop the services, press Ctrl + C in the terminal where you ran make dcservicesup. Once you've done that, you may run the following command to remove the docker containers (they will be recreated when you run dcservicesup again):

make dcservicesdown

Python dependencies

To develop on Kolibri Studio, you'll need:

• Python 3.6+

Managing Python installations can be quite tricky. We highly recommend using something like pyenv or package managers like Homebrew <http://brew.sh/>__ on Mac or apt on Debian to manage your Python installations. Never modify your system's built-in version of Python.

Once you've set up a Python virtual environment for the project, you can install the dependencies:

# Install all dependencies
pip install -r requirements.txt
pip install -r requirements-dev.txt

# Set up pre-commit hooks
pre-commit install

Adding or updating dependencies

We use pip-tools to ensure all our dependencies use the same versions on all deployments.

To add a dependency, add it to either requirements.in or requirements-dev.in, then run pip-compile requirements[-dev|-docs].in to generate the .txt file. Please make sure that both the .in and .txt file changes are part of the commit when updating dependencies.

To update a dependency, use pip-compile --upgrade-package [package-name] requirements[-dev|-docs].in

For more details, please see the pip-tools docs on Github.

Yarn and Javascript dependencies

As described above, Kolibri Studio has dependencies that rely on Node.js version 16.x. nodeenv is a useful tool for using specific versions of Node.js tools in Python environments. You'll also need yarn installed.

All the javascript dependencies are listed in package.json. To install them run the following yarn command:

# Set up Node 16.x environment
nodeenv -p --node=16.16.0
# Install yarn 'globally' to the project if you haven't already installed it globally
npm install -g yarn
# Install javascript dependencies
yarn install --network-timeout 1000000

The network-timeout helps avoid a timeout issue with the Material Icons dependency.

Initial setup

To set up the database, run:

yarn run devsetup

Running the development server

In one terminal tab, start the celery workers for asynchronous task processing:

yarn run celery

In another terminal tab, start the Django webserver and the webpack build using:

yarn run devserver:hot  # with Vue hot module reloading
# or
yarn run devserver  # without hot module reloading

This will take a few minutes to build the frontend. When it's done, you can sign in with a@a.com password a at http://localhost:8080/accounts/login/

Additional tools

Running tests

You can run tests using the following command:

yarn run test

View more testing tips

Profiling and local production testing

If you want to test the performance of your changes, you can start up a local server with settings closer to a production environment like so:

# build frontend dependencies
yarn run build
# run the server (no webpack)
yarn run localprodserver

Once the local production server is running, you can also use Locust to test your changes under scenarios of high demand like so:

cd deploy/chaos/loadtest
make timed_run
make stop_slaves  # mac: killall python

Profiling

In case you need to profile the application to know which part of the code are more time consuming, there are two different profilers available to work in two different modes. Both will store the profiling output in a directory that's determined by the PROFILE_DIR env variable. If this variable is not set, the output files will be store in a folder called profiler inside the OS temp folder (/tmp/profile usually) Note that both profiling modes are incompatible: you can either use one or the other, but not both at the same time. In case the env variables are set for both modes, All request profiling mode will be used.

All requests profiling mode

This mode will create interactive html files with all the profiling information for every request the Studio server receives. The name of the files will contain the total execution time, the endpoint name and a timestamp.

To activate it an env variable called PROFILE_STUDIO_FULL must be set.

Example of use:

PROFILE_STUDIO_FULL=y yarn runserver

Afterwards no further treatment of the generated files is needed. You can open directly the html files in your browser.

Endpoint profiling mode

When using the all requests mode it's usual that the profile folder is soon full of information for requests that are not interesting for the developer and it's hard to find the files for some specific endpoints.

If an env variable called PROFILE_STUDIO_FILTER is used, the profiler will be executed only on the http requests containing the text stated by the variable.

Example of use:

PROFILE_STUDIO_FILTER=edit yarn localprodserver

For this case, only html requests having the text edit in their request path will be profiled. The profile folder will not have html files, but binary dump files (with the timestamp as filename) of the profiler information that can be later seen by different profiling tools (snakeviz that can be installed using pip is recommended). Also while the server is running, the ten most time consuming lines of code of the filtered request will be shown in the console where Studio has been launched.

Example of snakeviz use:

snakeviz /tmp/profile/studio\:20200909161405011678.prof

will open the browser with an interactive diagram with all the profiling information

Linting

Front-end linting is run using:

yarn run lint-frontend

Some linting errors can be fixed automatically by running:

yarn run lint-frontend:format

Make sure you've set up pre-commit hooks as described above. This will ensure that linting is automatically run on staged changes before every commit.

Storybook

Storybook is a development environment for UI components. If this is your first encounter with this tool, you can check this presentation or its website. You are encouraged to use it any time you need to develop a new UI component. It is especially suitable for smaller to middle size components that represent basic UI building blocks.

An example is worth a thousand words so please have a look at these simple stories of an example component to see how to write yours. For detailed information on writing stories you can go through this tutorial.

You can also check official addons.

Run development server

yarn run storybook

With detailed webpack information (useful when debugging loaders, addons and similar):

yarn run storybook:debug

Bundle

yarn run storybook:build

The output is saved to storybook-static/.

Current usage notes

We've decided not to push our stories to the codebase and keep them locally in the near future. Although this limits the number of advantages Storybook provides, it allows us to start using it as soon as possible without the need to agree on all conventions and it also gives the whole team enough time to test the development workflow so we can decide later if we want to adopt this tool in a larger scale.

Taking into account the above-mentioned, all stories except of example DetailsRow.stories.js will be ignored by git as long as you use a naming convention for Storybook source files: *.stories.js.

Although we don't share stories at this point, Storybook is installed and configured in the codebase to prevent the need for everyone to configure everything locally. If you update Storybook Webpack settings, install a new plugin and similar, you are welcome to share such updates with other members of the team.