Usage

First, install SIM-CITY client on your own machine and on the target cluster.

Now you can install any command or model that you want on lisa, and run it with

simcity create ~/command.sh
simcity submit lisa

To run it locally as well, install exactly the same command ~/command.sh on your own machine. You can then process jobs locally with

simcity run --local -eP

If a command had an error and you have fixed the reason for the error, you can run

simcity scrub error

If you don't want to process the commands with errors anymore, run

simcity delete error

To check if all jobs are still running or pending, and restart jobs on lisa run

simcity check lisa

The command shell script gets a number of environment variables, based on the settings of the Execution section in the SIM-CITY configuration.

Variable	Usage
SIMCITY_IN	Input directory, contains any files uploaded with the task and set in the files property of the task input property. Is a subdirectory of the input_dir setting in the configuration file.
SIMCITY_TMP	Working directory. On clusters with a networked file system, it may be efficient to copy any input files to a local file system, and then do the task. Is a subdirectory of the tmp_dir setting in the configuration file.
SIMCITY_OUT	Output directory, all files put here will be uploaded back to SIM-CITY. Directories are not supported. Is a subdirectory of the output_dir setting in the configuration file.
SIMCITY_PARAMS	Path to the JSON file containing all settings from the task input property.
SIMCITY_JOB_ID	Internal SIM-CITY job ID.

The command will get any arguments given with the simcity create command or in a SIM-CITY webservice parameters property in a simulation configuration.

Parameter exploration

SIM-CITY client can also be used to make a scenario exploration. For example, in a fire response simulation, a fire is started in every ward, and for each the fire response time is measured. First the job is submitted in python:

import csv
import simcity
import sys
import os
import geojson

if len(sys.argv) != 2:
    sys.exit("Usage: {} ensemble_name".format(sys.argv[0]))

host = "lisa"
command = "~/bangalore-matsim-0.4/matsim.sh"
version = "0.4"
simulation = 'matsim'
ensemble = sys.argv[1]
max_jobs = 16

# starting a fire in the center of each ward
with open('input/blr/gis/wards.csv') as f:
    for row in csv.DictReader(f):
        fire = {'id': row['id'], 'x': row['lat'], 'y': row['lon']}

        simcity.add_task({
            'command': command,
            'name': simulation,
            'version': version,
            'ensemble': ensemble,
            'input': {
                'numberOfThreads': 16,
                'fires': [fire],
                'populationSampleFactor': 0.01,
            },
        })

for i in range(max_jobs):
    simcity.submit_if_needed(hostname=host, max_jobs=max_jobs)

If some jobs have gotten an error, evaluate why, and scrub them with

simcity scrub error
simcity check lisa

After the simulations have finished, analyse the output in python:

def add_responsetime(wards):
    """
    Add response times to a dict with geojson wards.

    In the ward properties, a first_responder and second_responder
    property are added, with response time in seconds. This data is
    retrieved from the ensemble, which is assumed to start a single
    fire in each ward.
    """
    db = simcity.get_task_database()
		
    design_doc = simcity.ensemble_view(
        db, simulation, version,
        '/couchdb/' + simcity.get_config().section('task-db')['database'], ensemble)
		
    tasks = db.get_from_view('all_docs', design_doc=design_doc)
		
    for task in tasks:
        if task.done == 0:
            print("skipping ward {0}, its simulation is not done"
                  .format(task.input['fires'][0]['id']))
        elif task.done == -1:
            print("skipping ward {0}, its simulation has errors: {1}"
                  .format(task.input['fires'][0]['id'], task.error))
        else:
            try:
                # Get the fire engine paths data from the database
                firepaths_str = simcity.download_attachment(task, task.id, 'GeoFirePaths.json')
                # Parse it
                with open(os.path.join(task.id, 'GeoFirePaths.json')) as f:
                    firepaths = geojson.load(f)
                collection = geojson.FeatureCollection.to_instance(firepaths)
                # Collect response times and put it in the relevant wards.
                responsetime = sorted([path.properties['responsetime'] for path in collection.features])
                ward = wards[task['input']['fires'][0]['id']]
                ward['properties']['first_responder'] = responsetime[0]
                ward['properties']['second_responder'] = responsetime[1]
            except (KeyError, IndexError) as ex:
                print(ex)