Skip to content

Server Installation

Jump to bottom
Jason Miller edited this page Apr 29, 2021 · 13 revisions

These are the instructions to get a basic Django web server for CIVET up and running.

Getting the repo

To get the CIVET source code:

git clone https://github.com/idaholab/civet.git

This creates a directory in $PWD/civet. Further instructions will be based out of this directory.

Environment

It is recommended to setup a python virtual environment for CIVET to ensure that exact versions of python dependencies are installed. This can be either a conda environment, a virtualenv, or whatever you prefer. Once inside the environment, and in the civet directory run:

pip install -r requirements.txt

This installs the bare minimum dependencies for CIVET to work.

Basic database setup

By default Django uses MySQL for its database. This should really only be used for development or for a server with very minimal database activity. If there are going to be lots of jobs then switching to a more robust database server like PostgreSQL would be a good idea. To use another database please consult the Django documentation on how to configure civet/settings.py to use it. Regardless of the database, these steps need to be performed:

./manage.py makemigrations
./manage.py migrate
./manage.py createsuperuser

The createsuperuser step creates a user with admin privileges. The password should be strong and kept secret. If any of these steps fails then make sure you have a suitable python environment.

If you are using an internal version of GitHub or GitLab then additional steps would be required and will not be covered here.

Running the server

Note: If you are attempting to run the server and wish to use OAuth features (logging in), it will first be necessary to populate the database with recipes: ./manage.py load_recipes see 'Recipes' below)

At this point you should be able to run:

./manage.py runserver

which starts a local webserver at http://localhost:8000. Going to this URL in a browser should show a basic CIVET homepage. It will be basically empty since there is no data in the database. If you want others to see the website you can specify a host name or ip address and a port on the command.

./manage.py runserver 127.0.0.1:9999

Which would start up the server on port 9999. Run ./manage.py runserver --help for further options.

Additionally, if you have DEBUG = False in civet/settings.py then you will likely need to run with the --insecure option to be able to serve static files correctly.

./manage.py runserver --insecure 127.0.0.1:9999

An https server can also be started.

./manage.py runsslserver

Which starts a server at https://localhost:8000. For a specific address and port:

./manage.py runsslserver 127.0.0.1:9999
Warning

Note that Django does not recommend using runserver or runsslserver for a production web server as it not gone through security and performance tests. CIVET has only used this configuration for testing and has not been in a production environment. It is definitely recommended to use a production level web server and database for a dedicated CIVET server. Internally we use Nginx for a the web server and PostgreSQL for the database.

Settings

See civet/settings.py for various settings. There is a configuration python dictionary for each server you want to configure (ie GitHub, GitLab, BitBucket). Most of the options are similar for each and the default dictionary for each one includes all the available options. There is also help text describing each option in the file.

The defaults are setup for testing. For production there are several settings that should be changed.

  • SECRET_KEY need to be set to a randomly generated string. Be careful to not commit this to your repo if it is publicly accesible.
  • DEBUG should be set to False.
  • ALLOWED_HOSTS needs to be set to the servers that your server will accept connections for. For example

ALLOWED_HOSTS = ['localhost', 'civet.inl.gov']

  • ABSOLUTE_BASE_URL should be set to the absolute URL for the server. For example https://civet.inl.gov.
  • In the python dictionary for your configuration, remote_update should be set to True
  • In the python dictionary for your configuration, install_webhook should be set to True

Additionally, set the client_id/secret generated by the git server in the python dictionary. For example, set client_id and secret_id to the values generated by GitHub under <user icon> -> Settings -> OAuth applications.

Recipes

The recipes that CIVET will use live in a repository. By default they are in ../civet_recipes from the top civet directory. This can be changed in civet/settings.py. You can view some sample recipes at civet_example_recipes. When the recipes change a command needs to be run so that the database is updated. This can be done manually, by cron job, or some custom program. The command that needs to be run is:

./manage.py load_recipes

This will load all the recipes in ../civet_recipes. If there is a failure then nothing is changed in the database.

Clone this wiki locally