Installation

The SIM-CITY infrastructure is structured as follows. Geographic data is served by a Geoserver, which gets its data from a PostGIS database. The user interface, sim-city-cs, talks to a web service that runs new simulations, using sim-city-client. It uses a CouchDB database to keep track of the simulations and an existing WebDAV storage to store the files.

Below, find the instructions to install all these components.

HTTP server

All services will be served over HTTP. To allow multiple services on the same host and port, we use nginx as a proxy server.

Install it with

sudo apt-get install nginx

and add proxies to /etc/nginx/sites-enabled/0-default, e.g.

location /couchdb {
    rewrite /couchdb/(.*) /$1 break;
    proxy_pass http://localhost:5984;
    proxy_redirect off;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location ~ /\.ht {
    deny all;
}

After every change to the configuration, reload the nginx service.

CouchDB service

CouchDB is used to store the input and output of simulations, and keep track of where they are running. It has its own authentication system.

Unless you already have access to a CouchDB installation, install CouchDB, either as a system package:

sudo apt-get install couchdb

or with a dedicated docker image.

It can be administered at http://localhost:5984/_utils/index.html. Set up an admin user here. Once it is configured, add a proxy to it with nginx. The database will be initialised with the SIM-CITY client.

Geoserver

Geographic data will be stored in a PostGIS database and served with a Geoserver. There is a howto for installing these: https://nlesc.gitbooks.io/geospatial-handbook/content/geoserver-howto.html.

SIM-CITY client

The SIM-CITY client manages starting jobs, and assigning tasks. See the README therein to see its usage.

It needs to be installed at two locations: your laptop and any cluster you want to run on. For example, assume you're using the <lisa.surfsara.nl> cluster. First, add lisa to your ssh path, by adding the following lines to ~/.ssh/config:

Host lisa
HostName lisa.surfsara.nl
User myuser

and copy your public SSH key to Lisa:

$ scp ~/.ssh/id_rsa.pub lisa:.
$ ssh lisa
lisa$ mkdir -p .ssh
lisa$ cat id_rsa.pub >> .ssh/authorized_keys
lisa$ rm id_rsa.pub

On your laptop and on Lisa, make the following ~/.simcity_client file:

[task-db]
# CouchDB URL
url = https://myhost/couchdb/
# CouchDB username
username = myuser
# CouchDB password
password = mypassword
# CouchDB database
database = simcity
    
[Execution]
# Base execution directory for local working files
tmp_dir = $TMPDIR
# Base directory for output files that need to be stored
output_dir = $HOME/input
# Base directory for input files
input_dir = $HOME/output

[lisa-host]
# where the run script is located
path = sim-city-client/scripts
script = lisaRun.sh
host = torque://lisa
method = xenon
max_time = 1440

See sim-city-client/config.ini.dist for additional configuration examples.

On lisa, run:

mkdir $HOME/input $HOME/output $HOME/tmp
git clone https://github.com/indodutch/sim-city-client.git
module load python/2.7
pip install --user virtualenv
~/.local/bin/virtualenv simcity

And add the following to ~/.bashrc:

export TMPDIR=${TMPDIR:-$HOME/tmp}
module load python/2.7
. simcity/bin/activate

Then run

source ~/.profile
make install

On your own machine, run the same commands except module load python/2.7.

Initialise the CouchDB database for the SIM-CITY client by running:

simcity init -u myadmin

SIM-CITY web service

To access these functionalities from a web service, install sim-city-webservice:

git clone https://github.com/indodutch/sim-city-webservice.git
cd sim-city-webservice
pip install -U .

To the ~/.simcity_client file, add a few new lines:

[Simulations]
# the maximum number of concurrent jobs that the service has
max_jobs = 4
# default host to submit to
default_host = lisa

And then serve the service with

 make serve

To start the web service automatically, create a new daemon that does these steps using your local daemon provider (e.g., upstart or systemd).

SIM-CITY CS

Install the website according to the instructions in sim-city-cs.