04_qambo_noisy_ddqn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Noisy Double Deep Q Learning - A simple ambulance dispatch point allocation model\n",
    "\n",
    "## Reinforcement learning introduction\n",
    "\n",
    "### RL involves:\n",
    "* Trial and error search\n",
    "* Receiving and maximising reward (often delayed)\n",
    "* Linking state -> action -> reward\n",
    "* Must be able to sense something of their environment\n",
    "* Involves uncertainty in sensing and linking action to reward\n",
    "* Learning -> improved choice of actions over time\n",
    "* All models find a way to balance best predicted action vs. exploration\n",
    "\n",
    "### Elements of RL\n",
    "* *Environment*: all observable and unobservable information relevant to us\n",
    "* *Observation*: sensing the environment\n",
    "* *State*: the perceived (or perceivable) environment \n",
    "* *Agent*: senses environment, decides on action, receives and monitors rewards\n",
    "* *Action*: may be discrete (e.g. turn left) or continuous (accelerator pedal)\n",
    "* *Policy* (how to link state to action; often based on probabilities)\n",
    "* *Reward signal*: aim is to accumulate maximum reward over time\n",
    "* *Value function* of a state: prediction of likely/possible long-term reward\n",
    "* *Q*: prediction of likely/possible long-term reward of an *action*\n",
    "* *Advantage*: The difference in Q between actions in a given state (sums to zero for all actions)\n",
    "* *Model* (optional): a simulation of the environment\n",
    "\n",
    "### Types of model\n",
    "\n",
    "* *Model-based*: have model of environment (e.g. a board game)\n",
    "* *Model-free*: used when environment not fully known\n",
    "* *Policy-based*: identify best policy directly\n",
    "* *Value-based*: estimate value of a decision\n",
    "* *Off-policy*: can learn from historic data from other agent\n",
    "* *On-policy*: requires active learning from current decisions\n",
    "\n",
    "## Key DQN components\n",
    "\n",
    "<img src=\"./images/dqn_components.png\" width=\"700\"/>\n",
    "\n",
    "\n",
    "## General method for Q learning:\n",
    "\n",
    "Overall aim is to create a neural network that predicts Q. Improvement comes from improved accuracy in predicting 'current' understood Q, and in revealing more about Q as knowledge is gained (some rewards only discovered after time).\n",
    "\n",
    "<img src=\"./images/dqn_process.png\" width=\"600|\"/>\n",
    "    \n",
    "Target networks are used to stabilise models, and are only updated at intervals. Changes to Q values may lead to changes in closely related states (i.e. states close to the one we are in at the time) and as the network tries to correct for errors it can become unstable and suddenly lose signficiant performance. Target networks (e.g. to assess Q) are updated only infrequently (or gradually), so do not have this instability problem.\n",
    "\n",
    "## Training networks\n",
    "\n",
    "Double DQN contains two networks. This ammendment, from simple DQN, is to decouple training of Q for current state and target Q derived from next state which are closely correlated when comparing input features.\n",
    "\n",
    "The *policy network* is used to select action (action with best predicted Q) when playing the game.\n",
    "\n",
    "When training, the predicted best *action* (best predicted Q) is taken from the *policy network*, but the *policy network* is updated using the predicted Q value of the next state from the *target network* (which is updated from the policy network less frequently). So, when training, the action is selected using Q values from the *policy network*, but the the *policy network* is updated to better predict the Q value of that action from the *target network*. The *policy network* is copied across to the *target network* every *n* steps (e.g. 1000).\n",
    "\n",
    "<img src=\"./images/dqn_training.png\" width=\"700|\"/>\n",
    "\n",
    "## Noisy layers\n",
    "Noisy layers are an alternative to epsilon-greedy exploration (here, we leave the epsilon-greedy code in the model, but set it to reduce to zero immediately after the period of fully random action choice).\n",
    "\n",
    "For every weight in the layer we have a random value that we draw from the normal distribution. This random value is used to add noise to the output. The parameters for the extent of noise for each weight, sigma, are stored within the layer and get trained as part of the standard back-propogation.\n",
    "\n",
    "A modification to normal nosiy layers is to use layers with ‘factorized gaussian noise’. This reduces the number of random numbers to be sampled (so is less computationally expensive). There are two random vectors, one with the size of the input, and the other with the size of the output. A random matrix is created by calculating the outer product of the two vectors.\n",
    "\n",
    "## References\n",
    "\n",
    "Double DQN:\n",
    "\n",
    "Mnih, V., Kavukcuoglu, K., Silver, D. et al. Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015). https://doi.org/10.1038/nature14236\n",
    "\n",
    "van Hasselt H, Guez A, Silver D. (2015) Deep Reinforcement Learning with Double Q-learning. arXiv:150906461 http://arxiv.org/abs/1509.06461\n",
    "\n",
    "Noisy networks:\n",
    "Fortunato M, Azar MG, Piot B, et al. (2019) Noisy Networks for Exploration. arXiv:170610295 http://arxiv.org/abs/1706.10295\n",
    "\n",
    "Code for the nosiy layers comes from:\n",
    "\n",
    "Lapan, M. (2020). Deep Reinforcement Learning Hands-On: Apply modern RL methods to practical problems of chatbots, robotics, discrete optimization, web automation, and more, 2nd Edition. Packt Publishing.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Code structure\n",
    "\n",
    "<img src=\"./images/dqn_program_structure.png\" width=\"700|\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "################################################################################\n",
    "#                           1 Import packages                                  #\n",
    "################################################################################\n",
    "\n",
    "from amboworld.environment import Env\n",
    "\n",
    "import math\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import random\n",
    "\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "from torch.nn import functional as F\n",
    "\n",
    "# Use a double ended queue (deque) for memory\n",
    "# When memory is full, this will replace the oldest value with the new one\n",
    "from collections import deque\n",
    "\n",
    "# Supress all warnings (e.g. deprecation warnings) for regular use\n",
    "import warnings\n",
    "\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "################################################################################\n",
    "#                           2 Define model parameters                          #\n",
    "################################################################################\n",
    "\n",
    "# Set whether to display on screen (slows model)\n",
    "DISPLAY_ON_SCREEN = False\n",
    "# Discount rate of future rewards\n",
    "GAMMA = 0.99\n",
    "# Learing rate for neural network\n",
    "LEARNING_RATE = 0.003\n",
    "# Maximum number of game steps (state, action, reward, next state) to keep\n",
    "MEMORY_SIZE = 10000000\n",
    "# Sample batch size for policy network update\n",
    "BATCH_SIZE = 5\n",
    "# Number of game steps to play before starting training (all random actions)\n",
    "REPLAY_START_SIZE = 50000\n",
    "# Number of steps between policy -> target network update\n",
    "SYNC_TARGET_STEPS = 1000\n",
    "# Exploration rate (epsilon) is probability of choosing a random action\n",
    "EXPLORATION_MAX = 1.0\n",
    "EXPLORATION_MIN = 0.0\n",
    "# Reduction in epsilon with each game step\n",
    "EXPLORATION_DECAY = 0.0\n",
    "# Training episodes\n",
    "TRAINING_EPISODES = 50\n",
    "# Save results\n",
    "RESULTS_NAME = 'noisy_ddqn'\n",
    "\n",
    "# SIM PARAMETERS\n",
    "RANDOM_SEED = 42\n",
    "SIM_DURATION = 5000\n",
    "NUMBER_AMBULANCES = 3\n",
    "NUMBER_INCIDENT_POINTS = 1\n",
    "INCIDENT_RADIUS = 2\n",
    "NUMBER_DISPTACH_POINTS = 25\n",
    "AMBOWORLD_SIZE = 50\n",
    "INCIDENT_INTERVAL = 60\n",
    "EPOCHS = 2\n",
    "AMBO_SPEED = 60\n",
    "AMBO_FREE_FROM_HOSPITAL = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "################################################################################\n",
    "#                      3 Define DQN (Deep Q Network) class                     #\n",
    "#                    (Used for both policy and target nets)                    #\n",
    "################################################################################\n",
    "\n",
    "class DQN(nn.Module):\n",
    "    \"\"\"Deep Q Network. Udes for both policy (action) and target (Q) networks.\"\"\"\n",
    "\n",
    "    def __init__(self, observation_space, action_space):\n",
    "        \"\"\"Constructor method. Set up neural nets.\"\"\"\n",
    "\n",
    "        # nerurones per hidden layer = 2 * max of observations or actions\n",
    "        neurons_per_layer = 2 * max(observation_space, action_space)\n",
    "\n",
    "        # Set starting exploration rate\n",
    "        self.exploration_rate = EXPLORATION_MAX\n",
    "\n",
    "        # Set up action space (choice of possible actions)\n",
    "        self.action_space = action_space\n",
    "\n",
    "        super(DQN, self).__init__()\n",
    "        self.net = nn.Sequential(\n",
    "            nn.Linear(observation_space, neurons_per_layer),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(neurons_per_layer, neurons_per_layer),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(neurons_per_layer, neurons_per_layer),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(neurons_per_layer, action_space)\n",
    "        )\n",
    "\n",
    "    def act(self, state):\n",
    "        \"\"\"Act either randomly or by redicting action that gives max Q\"\"\"\n",
    "\n",
    "        # Act randomly if random number < exploration rate\n",
    "        if np.random.rand() < self.exploration_rate:\n",
    "            action = random.randrange(self.action_space)\n",
    "        else:\n",
    "            # Otherwise get predicted Q values of actions\n",
    "            q_values = self.net(torch.FloatTensor(state))\n",
    "            # Get index of action with best Q\n",
    "            action = np.argmax(q_values.detach().numpy()[0])\n",
    "\n",
    "        return action\n",
    "\n",
    "    def forward(self, x):\n",
    "        \"\"\"Forward pass through network\"\"\"\n",
    "        return self.net(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "################################################################################\n",
    "#                    4 Define policy net training function                     #\n",
    "################################################################################\n",
    "\n",
    "def optimize(policy_net, target_net, memory):\n",
    "    \"\"\"\n",
    "    Update  model by sampling from memory.\n",
    "    Uses policy network to predict best action (best Q).\n",
    "    Uses target network to provide target of Q for the selected next action.\n",
    "    \"\"\"\n",
    "      \n",
    "    # Do not try to train model if memory is less than reqired batch size\n",
    "    if len(memory) < BATCH_SIZE:\n",
    "        return    \n",
    " \n",
    "    # Reduce exploration rate (exploration rate is stored in policy net)\n",
    "    policy_net.exploration_rate *= EXPLORATION_DECAY\n",
    "    policy_net.exploration_rate = max(EXPLORATION_MIN, \n",
    "                                      policy_net.exploration_rate)\n",
    "    # Sample a random batch from memory\n",
    "    batch = random.sample(memory, BATCH_SIZE)\n",
    "    for state, action, reward, state_next, terminal in batch:\n",
    "        \n",
    "        state_action_values = policy_net(torch.FloatTensor(state))\n",
    "        \n",
    "        # Get target Q for policy net update\n",
    "       \n",
    "        if not terminal:\n",
    "            # For non-terminal actions get Q from policy net\n",
    "            expected_state_action_values = policy_net(torch.FloatTensor(state))\n",
    "            # Detach next state values from gradients to prevent updates\n",
    "            expected_state_action_values = expected_state_action_values.detach()\n",
    "            # Get next state action with best Q from the policy net (double DQN)\n",
    "            policy_next_state_values = policy_net(torch.FloatTensor(state_next))\n",
    "            policy_next_state_values = policy_next_state_values.detach()\n",
    "            best_action = np.argmax(policy_next_state_values[0].numpy())\n",
    "            # Get target net next state\n",
    "            next_state_action_values = target_net(torch.FloatTensor(state_next))\n",
    "            # Use detach again to prevent target net gradients being updated\n",
    "            next_state_action_values = next_state_action_values.detach()\n",
    "            best_next_q = next_state_action_values[0][best_action].numpy()\n",
    "            updated_q = reward + (GAMMA * best_next_q)      \n",
    "            expected_state_action_values[0][action] = updated_q\n",
    "        else:\n",
    "            # For termal actions Q = reward (-1)\n",
    "            expected_state_action_values = policy_net(torch.FloatTensor(state))\n",
    "            # Detach values from gradients to prevent gradient update\n",
    "            expected_state_action_values = expected_state_action_values.detach()\n",
    "            # Set Q for all actions to reward (-1)\n",
    "            expected_state_action_values[0] = reward\n",
    " \n",
    "        # Set net to training mode\n",
    "        policy_net.train()\n",
    "        # Reset net gradients\n",
    "        policy_net.optimizer.zero_grad()  \n",
    "        # calculate loss\n",
    "        loss_v = nn.MSELoss()(state_action_values, expected_state_action_values)\n",
    "        # Backpropogate loss\n",
    "        loss_v.backward()\n",
    "        # Update network gradients\n",
    "        policy_net.optimizer.step()  \n",
    "\n",
    "    return"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "################################################################################\n",
    "#                            5 Define memory class                             #\n",
    "################################################################################\n",
    "\n",
    "class Memory():\n",
    "    \"\"\"\n",
    "    Replay memory used to train model.\n",
    "    Limited length memory (using deque, double ended queue from collections).\n",
    "      - When memory full deque replaces oldest data with newest.\n",
    "    Holds, state, action, reward, next state, and episode done.\n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(self):\n",
    "        \"\"\"Constructor method to initialise replay memory\"\"\"\n",
    "        self.memory = deque(maxlen=MEMORY_SIZE)\n",
    "\n",
    "    def remember(self, state, action, reward, next_state, done):\n",
    "        \"\"\"state/action/reward/next_state/done\"\"\"\n",
    "        self.memory.append((state, action, reward, next_state, done))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "################################################################################\n",
    "#                       6  Define results plotting function                    #\n",
    "################################################################################\n",
    "\n",
    "def plot_results(run, exploration, score, mean_call_to_arrival, \n",
    "                 mean_assignment_to_arrival):\n",
    "    \"\"\"Plot and report results at end of run\"\"\"\n",
    "\n",
    "    # Set up chart (ax1 and ax2 share x-axis to combine two plots on one graph)\n",
    "    fig = plt.figure(figsize=(6,6))\n",
    "    ax1 = fig.add_subplot(111)\n",
    "    ax2 = ax1.twinx()\n",
    "\n",
    "    # Plot results\n",
    "    lns1 = ax1.plot(\n",
    "        run, exploration, label='exploration', color='g', linestyle=':')\n",
    "\n",
    "    lns2 = ax2.plot(run, mean_call_to_arrival,\n",
    "             label='call to arrival', color='r')\n",
    "    lns3 = ax2.plot(run, mean_assignment_to_arrival,\n",
    "             label='assignment to arrival', color='b', linestyle='--')\n",
    "\n",
    "    # Get combined legend\n",
    "    lns = lns1 + lns2 + lns3\n",
    "    labs = [l.get_label() for l in lns]\n",
    "    ax1.legend(lns, labs, loc='upper center', bbox_to_anchor=(0.5, -0.1), ncol=3)\n",
    "        \n",
    "    # Set axes\n",
    "    ax1.set_xlabel('run')\n",
    "    ax1.set_ylabel('exploration')\n",
    "    ax2.set_ylabel('Response time')\n",
    "    filename = 'output/' + RESULTS_NAME +'.png'\n",
    "    plt.savefig(filename, dpi=300)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "################################################################################\n",
    "#                                 7 Main program                               #\n",
    "################################################################################\n",
    "\n",
    "def qambo():\n",
    "    \"\"\"Main program loop\"\"\"\n",
    "\n",
    "    ############################################################################\n",
    "    #                          8 Set up environment                            #\n",
    "    ############################################################################\n",
    "\n",
    "    # Set up game environemnt\n",
    "    sim = Env(\n",
    "        random_seed = RANDOM_SEED,\n",
    "        duration_incidents = SIM_DURATION,\n",
    "        number_ambulances = NUMBER_AMBULANCES,\n",
    "        number_incident_points = NUMBER_INCIDENT_POINTS,\n",
    "        incident_interval = INCIDENT_INTERVAL,\n",
    "        number_epochs = EPOCHS,\n",
    "        number_dispatch_points = NUMBER_DISPTACH_POINTS,\n",
    "        incident_range = INCIDENT_RADIUS,\n",
    "        max_size = AMBOWORLD_SIZE,\n",
    "        ambo_kph = AMBO_SPEED,\n",
    "        ambo_free_from_hospital = AMBO_FREE_FROM_HOSPITAL\n",
    "    )\n",
    "\n",
    "    # Get number of observations returned for state\n",
    "    observation_space = sim.observation_size\n",
    "\n",
    "    # Get number of actions possible\n",
    "    action_space = sim.action_number\n",
    "\n",
    "    ############################################################################\n",
    "    #                    9 Set up policy and target nets                       #\n",
    "    ############################################################################\n",
    "\n",
    "    # Set up policy and target neural nets (and keep best net performance)\n",
    "    policy_net = DQN(observation_space, action_space)\n",
    "    target_net = DQN(observation_space, action_space)\n",
    "    best_net = DQN(observation_space, action_space)\n",
    "    \n",
    "    # Set loss function and optimizer\n",
    "    policy_net.optimizer = optim.Adam(\n",
    "            params=policy_net.parameters(), lr=LEARNING_RATE)\n",
    "    \n",
    "    # Copy weights from policy_net to target\n",
    "    target_net.load_state_dict(policy_net.state_dict())\n",
    "    \n",
    "    # Set target net to eval rather than training mode\n",
    "    # We do not train target net - ot is copied from policy net at intervals\n",
    "    target_net.eval()\n",
    "\n",
    "    ############################################################################\n",
    "    #                            10 Set up memory                              #\n",
    "    ############################################################################\n",
    "\n",
    "    # Set up memomry\n",
    "    memory = Memory()\n",
    "\n",
    "    ############################################################################\n",
    "    #                     11 Set up + start training loop                      #\n",
    "    ############################################################################\n",
    "\n",
    "    # Set up run counter and learning loop\n",
    "    run = 0\n",
    "    all_steps = 0\n",
    "    continue_learning = True\n",
    "    best_reward = -np.inf\n",
    "\n",
    "    # Set up list for results\n",
    "    results_run = []\n",
    "    results_exploration = []\n",
    "    results_score = []\n",
    "    results_mean_call_to_arrival = []\n",
    "    results_mean_assignment_to_arrival = []\n",
    "\n",
    "    # Continue repeating games (episodes) until target complete\n",
    "    while continue_learning:\n",
    "\n",
    "        ########################################################################\n",
    "        #                           12 Play episode                            #\n",
    "        ########################################################################\n",
    "\n",
    "        # Increment run (episode) counter\n",
    "        run += 1\n",
    "\n",
    "        ########################################################################\n",
    "        #                             13 Reset game                            #\n",
    "        ########################################################################\n",
    "\n",
    "        # Reset game environment and get first state observations\n",
    "        state = sim.reset()\n",
    "\n",
    "        # Reset total reward and rewards list\n",
    "        total_reward = 0\n",
    "        rewards = []\n",
    "\n",
    "        # Reshape state into 2D array with state obsverations as first 'row'\n",
    "        state = np.reshape(state, [1, observation_space])\n",
    "\n",
    "        # Continue loop until episode complete\n",
    "        while True:\n",
    "\n",
    "            ####################################################################\n",
    "            #                       14 Game episode loop                       #\n",
    "            ####################################################################\n",
    "\n",
    "            ####################################################################\n",
    "            #                       15 Get action                              #\n",
    "            ####################################################################\n",
    "\n",
    "            # Get action to take (se eval mode to avoid dropout layers)\n",
    "            policy_net.eval()\n",
    "            action = policy_net.act(state)\n",
    "\n",
    "            ####################################################################\n",
    "            #                 16 Play action (get S', R, T)                    #\n",
    "            ####################################################################\n",
    "\n",
    "            # Act\n",
    "            state_next, reward, terminal, info = sim.step(action)\n",
    "            total_reward += reward\n",
    "\n",
    "            # Update trackers\n",
    "            rewards.append(reward)\n",
    "\n",
    "            # Reshape state into 2D array with state observations as first 'row'\n",
    "            state_next = np.reshape(state_next, [1, observation_space])\n",
    "\n",
    "            # Update display if needed\n",
    "            if DISPLAY_ON_SCREEN:\n",
    "                sim.render()\n",
    "\n",
    "            ####################################################################\n",
    "            #                  17 Add S/A/R/S/T to memory                      #\n",
    "            ####################################################################\n",
    "\n",
    "            # Record state, action, reward, new state & terminal\n",
    "            memory.remember(state, action, reward, state_next, terminal)\n",
    "\n",
    "            # Update state\n",
    "            state = state_next\n",
    "\n",
    "            ####################################################################\n",
    "            #                  18 Check for end of episode                     #\n",
    "            ####################################################################\n",
    "\n",
    "            # Actions to take if end of game episode\n",
    "            if terminal:\n",
    "                # Get exploration rate\n",
    "                exploration = policy_net.exploration_rate\n",
    "                # Clear print row content\n",
    "                clear_row = '\\r' + ' ' * 79 + '\\r'\n",
    "                print(clear_row, end='')\n",
    "                print(f'Run: {run}, ', end='')\n",
    "                print(f'Exploration: {exploration: .3f}, ', end='')\n",
    "                average_reward = np.mean(rewards)\n",
    "                print(f'Average reward: {average_reward:4.1f}, ', end='')\n",
    "                mean_assignment_to_arrival = np.mean(info['assignment_to_arrival'])\n",
    "                print(f'Mean assignment to arrival: {mean_assignment_to_arrival:4.1f}, ', end='')\n",
    "                mean_call_to_arrival = np.mean(info['call_to_arrival'])\n",
    "                print(f'Mean call to arrival: {mean_call_to_arrival:4.1f}, ', end='')\n",
    "                demand_met = info['fraction_demand_met']\n",
    "                print(f'Demand met {demand_met:0.3f}')\n",
    "\n",
    "                # Add to results lists\n",
    "                results_run.append(run)\n",
    "                results_exploration.append(exploration)\n",
    "                results_score.append(total_reward)\n",
    "                results_mean_call_to_arrival.append(mean_call_to_arrival)\n",
    "                results_mean_assignment_to_arrival.append(mean_assignment_to_arrival)\n",
    "\n",
    "                # Save model if best reward\n",
    "                total_reward = np.sum(rewards)\n",
    "                if total_reward > best_reward:\n",
    "                    best_reward = total_reward\n",
    "                    # Copy weights to best net\n",
    "                    best_net.load_state_dict(policy_net.state_dict())\n",
    "\n",
    "                ################################################################\n",
    "                #             18b Check for end of learning                    #\n",
    "                ################################################################\n",
    "\n",
    "                if run == TRAINING_EPISODES:\n",
    "                    continue_learning = False\n",
    "\n",
    "                # End episode loop\n",
    "                break\n",
    "\n",
    "            ####################################################################\n",
    "            #                        19 Update policy net                      #\n",
    "            ####################################################################\n",
    "            \n",
    "            # Avoid training model if memory is not of sufficient length\n",
    "            if len(memory.memory) > REPLAY_START_SIZE:\n",
    "        \n",
    "                # Update policy net\n",
    "                optimize(policy_net, target_net, memory.memory)\n",
    "\n",
    "                ################################################################\n",
    "                #             20 Update target net periodically                #\n",
    "                ################################################################\n",
    "                \n",
    "                # Use load_state_dict method to copy weights from policy net\n",
    "                if all_steps % SYNC_TARGET_STEPS == 0:\n",
    "                    target_net.load_state_dict(policy_net.state_dict())\n",
    "\n",
    "    ############################################################################\n",
    "    #             21 Learning complete - plot and save results                 #\n",
    "    ############################################################################\n",
    "\n",
    "    # Target reached. Plot results\n",
    "    plot_results(results_run, results_exploration, results_score,\n",
    "                 results_mean_call_to_arrival, results_mean_assignment_to_arrival)\n",
    "    \n",
    "    # SAVE RESULTS \n",
    "    run_details = pd.DataFrame()\n",
    "    run_details['run'] = results_run\n",
    "    run_details['exploration '] = results_exploration\n",
    "    run_details['mean_call_to_arrival'] = results_mean_call_to_arrival\n",
    "    run_details['mean_assignment_to_arrival'] = results_mean_assignment_to_arrival\n",
    "    filename = 'output/' + RESULTS_NAME + '.csv'\n",
    "    run_details.to_csv(filename, index=False)\n",
    "\n",
    "    ############################################################################\n",
    "    #                             Test best model                              #\n",
    "    ############################################################################\n",
    "    \n",
    "    print()\n",
    "    print('Test Model')\n",
    "    print('----------')\n",
    "    \n",
    "    best_net.exploration_rate = 0\n",
    "    best_net.eval()\n",
    "\n",
    "    # Set up results dictionary\n",
    "    results = dict()\n",
    "    results['call_to_arrival'] = []\n",
    "    results['assign_to_arrival'] = []\n",
    "    results['demand_met'] = []\n",
    "\n",
    "    # Replicate model runs\n",
    "    for run in range(30):\n",
    "\n",
    "        # Reset game environment and get first state observations\n",
    "        state = sim.reset()\n",
    "        state = np.reshape(state, [1, observation_space])\n",
    "\n",
    "        # Continue loop until episode complete\n",
    "        while True:\n",
    "            # Get action to take (se eval mode to avoid dropout layers)\n",
    "            best_net.eval()\n",
    "            action = best_net.act(state)\n",
    "            # Act\n",
    "            state_next, reward, terminal, info = sim.step(action)\n",
    "            # Reshape state into 2D array with state observations as first 'row'\n",
    "            state_next = np.reshape(state_next, [1, observation_space])\n",
    "            # Update state\n",
    "            state = state_next\n",
    "            if terminal:\n",
    "                print(f'Run: {run}, ', end='')\n",
    "                mean_assignment_to_arrival = np.mean(info['assignment_to_arrival'])\n",
    "                print(f'Mean assignment to arrival: {mean_assignment_to_arrival:4.1f}, ', end='')\n",
    "                mean_call_to_arrival = np.mean(info['call_to_arrival'])\n",
    "                print(f'Mean call to arrival: {mean_call_to_arrival:4.1f}, ', end='')\n",
    "                demand_met = info['fraction_demand_met']\n",
    "                print(f'Demand met: {demand_met:0.3f}')\n",
    "\n",
    "                # Add to results\n",
    "                results['call_to_arrival'].append(mean_call_to_arrival)\n",
    "                results['assign_to_arrival'].append(mean_assignment_to_arrival)\n",
    "                results['demand_met'].append(demand_met)\n",
    "                \n",
    "                # End episode loop\n",
    "                break\n",
    "\n",
    "    results = pd.DataFrame(results)\n",
    "    filename = './output/results_' + RESULTS_NAME +'.csv'\n",
    "    results.to_csv(filename, index=False)\n",
    "    print()\n",
    "    print(results.describe())    \n",
    "\n",
    "    return run_details"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Run: 1, Exploration:  1.000, Average reward: -717.6, Mean assignment to arrival: 24.7, Mean call to arrival: 30.5, Demand met 1.000\n",
      "Run: 2, Exploration:  1.000, Average reward: -717.7, Mean assignment to arrival: 24.7, Mean call to arrival: 30.5, Demand met 1.000\n",
      "Run: 3, Exploration:  1.000, Average reward: -722.4, Mean assignment to arrival: 24.8, Mean call to arrival: 30.7, Demand met 1.000\n",
      "Run: 4, Exploration:  1.000, Average reward: -719.0, Mean assignment to arrival: 24.8, Mean call to arrival: 30.0, Demand met 1.000\n",
      "Run: 5, Exploration:  1.000, Average reward: -712.2, Mean assignment to arrival: 24.6, Mean call to arrival: 29.7, Demand met 1.000\n",
      "Run: 6, Exploration:  1.000, Average reward: -708.9, Mean assignment to arrival: 24.6, Mean call to arrival: 29.7, Demand met 1.000\n",
      "Run: 7, Exploration:  1.000, Average reward: -712.1, Mean assignment to arrival: 24.7, Mean call to arrival: 29.8, Demand met 1.000\n",
      "Run: 8, Exploration:  1.000, Average reward: -719.0, Mean assignment to arrival: 24.8, Mean call to arrival: 30.1, Demand met 1.000\n",
      "Run: 9, Exploration:  1.000, Average reward: -718.0, Mean assignment to arrival: 24.8, Mean call to arrival: 30.0, Demand met 1.000\n",
      "Run: 10, Exploration:  1.000, Average reward: -732.8, Mean assignment to arrival: 25.0, Mean call to arrival: 30.5, Demand met 1.000\n",
      "Run: 11, Exploration:  0.000, Average reward: -311.5, Mean assignment to arrival: 16.0, Mean call to arrival: 19.3, Demand met 1.000\n",
      "Run: 12, Exploration:  0.000, Average reward: -285.8, Mean assignment to arrival: 15.0, Mean call to arrival: 18.3, Demand met 1.000\n",
      "Run: 13, Exploration:  0.000, Average reward: -283.7, Mean assignment to arrival: 15.0, Mean call to arrival: 18.2, Demand met 1.000\n",
      "Run: 14, Exploration:  0.000, Average reward: -281.7, Mean assignment to arrival: 14.8, Mean call to arrival: 18.4, Demand met 1.000\n",
      "Run: 15, Exploration:  0.000, Average reward: -279.3, Mean assignment to arrival: 14.8, Mean call to arrival: 18.1, Demand met 0.999\n",
      "Run: 16, Exploration:  0.000, Average reward: -287.0, Mean assignment to arrival: 15.0, Mean call to arrival: 18.1, Demand met 1.000\n",
      "Run: 17, Exploration:  0.000, Average reward: -264.6, Mean assignment to arrival: 14.2, Mean call to arrival: 17.4, Demand met 1.000\n",
      "Run: 18, Exploration:  0.000, Average reward: -253.4, Mean assignment to arrival: 13.6, Mean call to arrival: 16.6, Demand met 1.000\n",
      "Run: 19, Exploration:  0.000, Average reward: -257.3, Mean assignment to arrival: 13.7, Mean call to arrival: 16.8, Demand met 1.000\n",
      "Run: 20, Exploration:  0.000, Average reward: -246.4, Mean assignment to arrival: 13.7, Mean call to arrival: 16.4, Demand met 1.000\n",
      "Run: 21, Exploration:  0.000, Average reward: -278.4, Mean assignment to arrival: 14.6, Mean call to arrival: 17.4, Demand met 1.000\n",
      "Run: 22, Exploration:  0.000, Average reward: -256.2, Mean assignment to arrival: 14.0, Mean call to arrival: 16.8, Demand met 1.000\n",
      "Run: 23, Exploration:  0.000, Average reward: -247.8, Mean assignment to arrival: 13.4, Mean call to arrival: 16.5, Demand met 1.000\n",
      "Run: 24, Exploration:  0.000, Average reward: -258.4, Mean assignment to arrival: 13.8, Mean call to arrival: 17.0, Demand met 1.000\n",
      "Run: 25, Exploration:  0.000, Average reward: -255.1, Mean assignment to arrival: 13.7, Mean call to arrival: 17.0, Demand met 1.000\n",
      "Run: 26, Exploration:  0.000, Average reward: -289.0, Mean assignment to arrival: 14.9, Mean call to arrival: 18.3, Demand met 1.000\n",
      "Run: 27, Exploration:  0.000, Average reward: -263.1, Mean assignment to arrival: 14.0, Mean call to arrival: 16.8, Demand met 1.000\n",
      "Run: 28, Exploration:  0.000, Average reward: -273.3, Mean assignment to arrival: 14.3, Mean call to arrival: 17.1, Demand met 1.000\n",
      "Run: 29, Exploration:  0.000, Average reward: -248.7, Mean assignment to arrival: 13.6, Mean call to arrival: 16.6, Demand met 1.000\n",
      "Run: 30, Exploration:  0.000, Average reward: -279.1, Mean assignment to arrival: 14.8, Mean call to arrival: 18.1, Demand met 1.000\n",
      "Run: 31, Exploration:  0.000, Average reward: -240.9, Mean assignment to arrival: 13.5, Mean call to arrival: 16.1, Demand met 1.000\n",
      "Run: 32, Exploration:  0.000, Average reward: -263.4, Mean assignment to arrival: 14.0, Mean call to arrival: 17.3, Demand met 1.000\n",
      "Run: 33, Exploration:  0.000, Average reward: -239.1, Mean assignment to arrival: 13.1, Mean call to arrival: 15.9, Demand met 1.000\n",
      "Run: 34, Exploration:  0.000, Average reward: -244.4, Mean assignment to arrival: 13.3, Mean call to arrival: 16.2, Demand met 1.000\n",
      "Run: 35, Exploration:  0.000, Average reward: -238.8, Mean assignment to arrival: 13.1, Mean call to arrival: 16.0, Demand met 1.000\n",
      "Run: 36, Exploration:  0.000, Average reward: -244.4, Mean assignment to arrival: 13.3, Mean call to arrival: 16.4, Demand met 1.000\n",
      "Run: 37, Exploration:  0.000, Average reward: -285.1, Mean assignment to arrival: 14.6, Mean call to arrival: 18.0, Demand met 1.000\n",
      "Run: 38, Exploration:  0.000, Average reward: -232.6, Mean assignment to arrival: 12.9, Mean call to arrival: 16.5, Demand met 1.000\n",
      "Run: 39, Exploration:  0.000, Average reward: -257.5, Mean assignment to arrival: 13.7, Mean call to arrival: 16.7, Demand met 1.000\n",
      "Run: 40, Exploration:  0.000, Average reward: -271.9, Mean assignment to arrival: 14.2, Mean call to arrival: 17.3, Demand met 1.000\n",
      "Run: 41, Exploration:  0.000, Average reward: -280.2, Mean assignment to arrival: 14.4, Mean call to arrival: 17.2, Demand met 1.000\n",
      "Run: 42, Exploration:  0.000, Average reward: -266.3, Mean assignment to arrival: 14.1, Mean call to arrival: 17.2, Demand met 1.000\n",
      "Run: 43, Exploration:  0.000, Average reward: -281.1, Mean assignment to arrival: 14.4, Mean call to arrival: 17.4, Demand met 1.000\n",
      "Run: 44, Exploration:  0.000, Average reward: -291.6, Mean assignment to arrival: 14.5, Mean call to arrival: 17.7, Demand met 1.000\n",
      "Run: 45, Exploration:  0.000, Average reward: -281.4, Mean assignment to arrival: 14.4, Mean call to arrival: 17.7, Demand met 1.000\n",
      "Run: 46, Exploration:  0.000, Average reward: -277.3, Mean assignment to arrival: 14.2, Mean call to arrival: 17.5, Demand met 1.000\n",
      "Run: 47, Exploration:  0.000, Average reward: -258.6, Mean assignment to arrival: 13.8, Mean call to arrival: 16.9, Demand met 1.000\n",
      "Run: 48, Exploration:  0.000, Average reward: -245.4, Mean assignment to arrival: 13.4, Mean call to arrival: 16.3, Demand met 1.000\n",
      "Run: 49, Exploration:  0.000, Average reward: -295.5, Mean assignment to arrival: 14.7, Mean call to arrival: 18.4, Demand met 1.000\n",
      "Run: 50, Exploration:  0.000, Average reward: -296.8, Mean assignment to arrival: 14.9, Mean call to arrival: 18.3, Demand met 1.000\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa0AAAGMCAYAAABppv2kAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3iUVfbA8e9JgyR06YiCgChIFUHBQlMBCyg2VFaQn6Bi11XsZXcVe8GKCoqiyC6IICigYkFEmtJEOkgPRVooIcn5/XFnSAKTZJLMZCYv5/M888y8/cwQ5swt772iqhhjjDElQUykAzDGGGOCZUnLGGNMiWFJyxhjTIlhScsYY0yJYUnLGGNMiWFJyxhjTIkRF+kACiomJkYTExMjHYYxxpQo+/btU1Ut8QWVEpe0EhMTSU1NjXQYxhhToojI/kjHEAolPusaY4w5dljSMsYYU2JY0jLGGFNiWNIyxhhTYljSMsYYU2JY0jLGGFNiWNIyxhhTYljSMsYYU2JY0jLGGFNihC1picgwEUkRkUW5bBcReU1EVojIAhFpGa5YjDHGeEM4S1ofAF3y2N4VaOB79AfeCmMsxhhjPCBsSUtVfwR25LFLd2CEOjOBCiJSI1zxGGOMKfki2aZVC1iXbXm9b13YPPjNgzR+s/Hh5fum3EeLd1ocXr590u20ea8NbNsGL7/MmgZVea3XSYe33/jFjXQa0enw8vVjr6fLx1mFyav/dzWXfnrp4eUeo3pwxegrDi9f9MlF9BrT6/Dy+R+dT59xfQ4vn/fBedw0/qbDy23fb8vAiQMPL7ca2oq7v7778HLTt5py/9T7AcjIzOCCjy7gUMah4D8QY4wpYSI5yrsEWKcBdxTpj6tCJCEhodAXbFKtCWkZaYeXm1VrRqzEuoWMDC5enUD/0bvh1ppw6BDVyydzy9i/Ye1aOPFEzqh5BnUr1D18fJtabdiTtufw8lnHn5Xj/GefcDYxkvW74JwTziEpPunw8nknnkelxEqHlzvU6UCNMlmFzU51O1GnQp3Dy51P6syplU89vHxhvQtpVq0ZACPmj2DqqqnsPLCTKslVCvPxGGNM1BPVgHkiNCcXqQN8qaqnBdj2DvC9qn7qW14KtFfVTXmdMzk5WQs1NcnixfDZZ/6LZz2LwJ498N//wl9/QaVK0Ls39OsHFSpAw4Zw0UVuexTbm7aX1LRUqiRXyZEojTEGQET2qWpypOMoqkiWtMYDt4nIKKANsCu/hFUkf/wB//43BErSItC5Mzz/PHTvDqVKZW178EF47DGYNg06dAhbeEVVJi6JMlt3QRlLWMYY7wpbSUtEPgXaA5WBLcDjQDyAqr4tIgK8juthuA/oq6pz8jtvoUtagWR/7xKothLYvx8aNYJy5WDuXIiLznkzd/a7jjIfj2b3+pVUqnJCpMMxxkQZr5S0wlo9GA4hTVrBGjMGrrgC3nwTbrmleK8dDH98wNL539GwafSWCI0xkWFJK0IikrRUoVMnmD8fli937V7RYs0aaN4c3b8fSUsjY9lSYhucHOmojDFRxitJyxpAgiECr7wCO3fC449HOposhw5Br16givzrXwDEplmXd2OMd1nSClbTpnDzzfDWW7Ao4MhUxe+RR2DmTHjvPbaeUBmAzVtXRzgoY4wJH0taBfHUU65Dxp13Bu6FWJy+/hqeew4GDIArr2Rj+t8AbExZFdm4jDElkoiUFpFZIjJfRBaLyJO+9ZVEZKqILPc9V8zl+C4istQ3nuygcMVpSasgjjsO/vUv+O47+Pzz8F1n82Z3n1jv3jB2LOzbl3P7xo3wj39Akybw8ssAND6xFQDNKzQMX1zGGC87CHRU1WZAc6CLiJwJDAK+VdUGwLe+5RxEJBZ4AzembCOgl4g0CkeQlrQKasAAlyxuugl++im051aFTz6Bxo1h5EiYNAl69oTKld3zJ5/A33/D9ddDaqq7WToxEYC4pDIAxBw4GNqYjDHHBN84sHt9i/G+h+LGif3Qt/5DoEeAw1sDK1R1laqmAaN8x4WcJa2CiouDceNcIuncGT79NDTn3bwZLr8crrsOTj4Zfv8dtmyBb76Bvn3hl1/ctsqV3Y3Or78Op2YN6bRdXWlsfcqK0MRjjPGaOBGZk+3R/8gdRCRWRH4HUoCpqvorUM0/8IPvuWqAcxfbWLLReadstDvpJJdEevSAa6913c4HDcr9BuW8+EtXt9/uqgGffx7uvhtifWMidurkHkOGuE4XY8dC+fLQp0+O02zPTOU4YEPKSo4v6vszxnhRuqq2ymsHVc0AmotIBeBzETlqCL5cBD2WbFFZ0iqsSpVg6lS48UZ46CFYtcrdfBwfH9zxmZnw/ffw0kswcSKceSYMHw6nnBJ4/5gYaNvWPQKof3wTAFpXalKIN2OMMVlUdaeIfI8bsWiLiNRQ1U2+6aNSAhyyHqidbfl4YGM4YrPqwaIoVQo+/th1PX/vPbj4Yti9O+9jNmyA//wHGjRwJajp013pavr03BNWEGIS3ejxcuBAoc9hjDl2iUgVXwkLEUkEOgN/4saJvcG32w3AFwEOnw00EJG6IpIAXOM7LuSspFVUIq5HYd26rpNGo0auI0X16jkfqq5zxddfu1JWhw6uC/3llx/uTFEUe2LSKQus27I8x88dY4wJUg3gQ19PwBhgtKp+KSK/AKNFpB/wF3AlgIjUBN5T1W6qmi4itwGTgVhgmKouDkeQNoxTKE2bBq+9Bps2uY4VmzZBWtb8WtSs6dqibrwR6tUL6aVT9m6hatnqzPm/brR6d2JIz22MKfm8MoyTlbRCqUOHnNOXqLqhnzZvhr17oUWLsI0SXyW5Klq6NK0qNs5/Z2OMKaEsaYWTCFSs6B5hv5S4akZr0zLGeJh1xPCITM1kt6Sxfovdp2WM8S5LWh4hCNt0H3//HZZepsYYExWsetAjRISTapwK5epHOhRjjAkbK2l5ibVpGWM8zkpaHrL24BYStx0MODCYMcZ4gSUtD1mXto1amZmRDsMYY8LGkpaHnN3wfFi/PtJhGGNM2FiblpeULg3790c6CmOMCRtLWh4yf/cy9uzeGukwjDEmbCxpeciaA5vJ3Lcv0mEYY0zYWNLykO7Nr6G8JkQ6DGOMCRtLWl5ibVrGGI+zpOUh07fOdVOhWLd3Y4xHWdLykNUHt7gXNiqGMcajLGl5SO/W/+deWNIyxniUJS0vSUx0z9auZYzxKEtaHjJlw4/uhSUtY4xHWdLykL/SfDcWW/WgMcajLGl5yP+1vc29sJKWMcajLGl5ib9Ny0paxhiPsqTlIePXTHEvrKRljPEoS1oesjlzl3thJS1jjEdZ0vKQ/mff6V5YScsY41GWtLykdGn3bEnLGONRlrQ85PM1X7sXVj1ojPEoS1oeskPdXFpqc2oZYzzKkpaH9Gvn7tOSgwcjHIkxxoSHJS0vSUgAEWvTMsZ4liUtDxm/bAIH4oWDqbsiHYoxxoSFJS0PSctI42B8jLVpGWM8Ky7SAZjQuaLRFVD+TkiPdCTGmJJGRGoDI4DqQCYwVFVfFZHPgIa+3SoAO1W1eYDj1wB7gAwgXVVbhSNOS1pek5hobVrGmMJIB+5V1XkiUhaYKyJTVfVq/w4i8iKQV/tDB1XdFs4grXrQQ6atnsbqA5tI3b090qEYY0oYVd2kqvN8r/cAS4Ba/u0iIsBVwKeRidCxpOUhinIwPgaxm4uNMUUgInWAFsCv2VafA2xR1eW5HKbAFBGZKyL9wxWbVQ96SMe6HeH45pAukQ7FGBN94kRkTrbloao69MidRKQMMAa4S1V3Z9vUi7xLWe1UdaOIVAWmisifqvpjSCLPxpKW1yQmwt69kY7CGBN98u0cISLxuIQ1UlXHZlsfB1wOnJ7bsaq60fecIiKfA62BkCetsFYPikgXEVkqIitEZFCA7eVFZIKIzBeRxSLSN5zxeN3cjXOZnjKHfXt2RDoUY0wJ42uzeh9YoqovHbG5M/Cnqq7P5dhkX+cNRCQZuABYFI44w5a0RCQWeAPoCjQCeolIoyN2Gwj8oarNgPbAiyKSEK6YvC42Jpb0UvHEHEiLdCjGmJKnHdAb6Cgiv/se3XzbruGIqkERqSkik3yL1YDpIjIfmAVMVNWvwxFkOKsHWwMrVHUVgIiMAroDf2TbR4GyvgxfBtiB3WVUaM2rN4dTu8L330c6FGNMCaOq04GADeKq2ifAuo1AN9/rVUCzcMbnF87qwVrAumzL68nWfdLndeBUYCOwELhTVTPDGJP3lS5tU5MYYzwrnEkrUMbWI5YvBH4HagLNgddFpNxRJxLpLyJzRGROeroVxHKzYscKPlnxOempeyIdijHGhEU4k9Z6oHa25eNxJars+gJj1VkBrAZOOfJEqjpUVVupaqu4OOvwmJuE2ATiksoQc9DatIwx3hTOpDUbaCAidX2dK64Bxh+xz19AJwARqYYb32pVGGPytBPKn8BVrW4g5lA6ZGREOhxjjAm5sBVbVDVdRG4DJgOxwDBVXSwiN/u2vw38C/hARBbiqhMfCPe4VZ5XurR7PnAAkpMjG4sxxoSYqB7ZzBTdkpOTNTU1NdJhRKVt+7bxXt9mDBq9EbZtg+OOi3RIxpgoISL7VLXE/5K1sQc9JCE2gbIVqroFG+ndGONBlrQ8pFypcgw85163YN3ejTEeZEnLaxIT3bOVtIwxHmRJy0MyMjMYMPVOt2BJyxjjQZa0PCRGYqhVrb5bsOpBY4wHWdLyEBHhsQufdgtW0jLGeJAlLa/xt2lZScsY40GWtDym5xfXuRdW0jLGeJAlLY9pfKJvYlIraRljPMiSlsc81e1598JKWsYYD7Kk5TXZxx40xhiPsaTlMZ3/e4l7YSUtY4wHWdLymLYnnYeKWNIyxniSJS2Pearjv5DERKseNMZ4kiUtLypd2kpaxhhPsqTlMZd8egnb2GclLWOMJ1nS8phOdTsRl1TGSlrGGE+ypOUxd515FxUqVLeSljHGkyxpeVFiopW0jDGeZEnLY3p/3pt5O5dY0jLGeJIlLY/pXLczFSvWsOpBY4wnWdLymBua30Dd6qdaScsY40mWtLzIbi42xniUJS2PuX3S7YxeOcFKWsYYT7Kk5TGdT+pMvZqnWUnLGONJlrQ8pvsp3Tm9XjsraRljPMmSlgdpqVJW0jLGeJIlLY95bNpjPP7rYDh0CDIyIh2OMaaEEJHaIjJNRJaIyGIRudO3/gkR2SAiv/se3XI5vouILBWRFSIyKFxxxoXrxCYyOtTpQPrJM2HaVFdFWKZMpEMyxpQM6cC9qjpPRMoCc0Vkqm/by6r6Qm4Hikgs8AZwPrAemC0i41X1j1AHaSUtj/j6a3j6aWhfpwPnn3apW2lVhMaYIKnqJlWd53u9B1gC1Ary8NbAClVdpappwCigezjitKRVwh04AFddBV27wsMPw5aUTA7Fx7qN1hnDGFMIIlIHaAH86lt1m4gsEJFhIlIxwCG1gHXZltcTfMIrEEtaJVzp0iACHTq45bdnfESfKbe6BStpGWOyxInInGyP/oF2EpEywBjgLlXdDbwF1AOaA5uAFwMdFmCdhijuHCxpBbBjB3z6KfTpA0OGZK3fvTtiIeUweza0awcrVrjlUaPgllvc61PLt+LyFte6BStpGWOypKtqq2yPoUfuICLxuIQ1UlXHAqjqFlXNUNVM4F1cVeCR1gO1sy0fD2wM/Vuwjhg5vPIKjB0LP/8MmZlQsSI0bOi27d0LFSpArVrQtKlLGtdeC3XqhDaGzEyYNw8mTIB69eAf/3DrLrjA5aB9+2D+fKhRAzZsgPr1XUmrc2f47Tdo2LAxiQeuAz6xkpYxJmgiIsD7wBJVfSnb+hqqusm3eBmwKMDhs4EGIlIX2ABcA1wbjjiPqaS1Z48rnaxcCatWuecDB+DDD932SZPcPg89BBddBGecAbG+5qGMDBg8GBYscI+HH3aP4cNdiayopkyBzz+H8eNh40aIiYE773RJKybG9WBPTIRKlVz71aBBUK5c1vEVK7pHemY6qbGZJIOVtIwxBdEO6A0sFJHffeseAnqJSHNcdd8aYACAiNQE3lPVbqqaLiK3AZOBWGCYqi4OR5CiGpZqx7BJTk7W1NTUQh17440uyfhVrgynngo//OBKKwcPQqlSwZ1r7VoYORKuvx5OOMElnI8+gosvhoQE8H+s11/vzv3zzzB3rkuK/ocqvPmm2699e5gzB7p0ge7doVs3OO644N/btm3w3/9CWp0JjBpzKb+8j+tSeOGFwZ/EGONZIrJPVZMjHUdRHVNJ65dfXJVavXrukb2kUlTDhsGDD0JKSs71mZkuaQ0YAEN9Ncjx8e7alSvDn3+6dWvXQrVqrmNFYSxeDKedBi+/u5Hj4l+kd5+XXCbt0aPwb8oY4xmWtCKkKEkr3NLTYfVq91rEPU46yT3v2OESWNmywZfmCmLNGqhb11ddeeafrgj5ySfQq1foL2aMKXG8krSOqTatcIuLgwYNAm+rVCm8105Kcs+792SwXfdxHFhHDGOM51iXd49I9v1+WrR+FY2Gne4WrCOGMcZjLGl5RGKiey4TU42nuj7nFqykZYzxGKse9IiYGFi2DKpUKUeF5LuA+62kZYzxHCtpeUiDBpBc9hAb96egMTFW0jLGeI6VtDzk3Xdhf8IG7lxTl0OlShNnJS1jjMdYSctDXn4Zvhtfk3cufgdJTLTqQWOM51jS8pDkZDh0IIH+p/cnNinZqgeNMZ5jSctDkpJgb6qy+u/VZJRKsJKWMcZzLGl5SFIS7N57iJNeO4mdcsBKWsYYz7Gk5SHJyXBwfxzDuw8nqVxlK2kZYzzHxh70kJ073TiH5csD553nbt6aNi3SYRljokC0jD0oItWAp4GaqtpVRBoBZ6nq+8EcbyUtD6lQAcqVU5ZuW0paQqyVtIwx0egD3LxbNX3Ly4C7gj04rElLRLqIyFIRWSEig3LZp72I/C4ii0Xkh3DG43VTp8LDjyinvHEKq/ZvsjYtY0w0qqyqo4FMAFVNBzKCPThsSUtEYoE3gK5AI9zsl42O2KcC8CZwqao2Bq4MVzzHgunTYfAzMYy87BOqV6lrJS1jTDRKFZHjcDMhIyJnAruCPTicJa3WwApVXaWqacAooPsR+1wLjFXVvwBU9YgpFE1BJCW52ZB71O9FhQrVLGkZY6LRPcB4oJ6I/AyMAG4P9uBwDuNUC1iXbXk90OaIfU4G4kXke6As8KqqjjjyRCLSH+gPkJCQEJZgvcA/Pclvfy2lWWwGZax60BgTZVR1noicBzQEBFiqqoeCPT6cSUsCrDuyq2IccDrQCUgEfhGRmaq6LMdBqkOBoeB6D4YhVk/wTwR50YdXMXa70NFKWsaYKONrOuoG1MHlgAtEBFV9KZjjw5m01gO1sy0fD2wMsM82VU3F1XP+CDTD9SYxBeRPWv85+xUa7xsJBxZHNiBjjDnaBOAAsBBfZ4yCCGfSmg00EJG6wAbgGlwbVnZfAK+LSByQgKs+fDmMMXnaFVdAz54QH98BfvsZ0tPdI84G8zfGRI3jVbVpYQ8O6ttMREoBPckqzgGgqk/ldoyqpovIbbj++LHAMFVdLCI3+7a/rapLRORrYAEu476nqosK+2aOdf7ctChlEdUy9lIFXLf3MmUiGZYxxmT3lYhcoKpTCnNwUCNi+BLLLmAu2frTq+qLhbloUdiIGLn76y949lmYVL47j2xZR79hv0FKClSpEunQjDERFkUjYlwGfIzrvX4I1/9BVbVcMMcHW290vKp2KVyIprjs2gVvvglPvPE0nY8bDfxmNxgbY6LNi8BZwEItxDiCwd6nNUNEmhT05KZ4+bu81y3TmBNrNHQL1oPQGBNdlgOLCpOwIPiS1tlAHxFZDRwkqzhX6MY0E3r+3oPLNm1gTZXN1AFLWsaYaLMJ+F5EvsLlE4CQd3nvWojATDHzl7Q+mfcFh47/gGfBqgeNMdFmte+R4HsUSFBJS1XXikgz4Bzfqp9UdX5BL2bCKykJ4uOhW71LuLFlLHCzlbSMMVFFVZ8syvHBdnm/E7gJGOtb9bGIDFXVIUW5uAmt2FhISwOoDTObuZVW0jLGRAEReUVV7xKRCRw9OhKqemkw5wm2erAf0MY3cgUi8izwC2BJKwotSllE5u7lNAUraRljosVHvucXinKSYJOWkHO+kwwCjy1oIuzee+G7XTOoVet5vgQraRljooKqzvW9bK6qr2bf5qvNC2o+xWC7vA8HfhWRJ0TkCWAmENTUyKZ4TZwI1bddzdMXv+JWWEnLGBMEEaktItNEZIlvUt47feufF5E/RWSBiHzumwcx0PFrRGShb1LfOXlc6oYA6/oEG2ewHTFe8k0fcjauhNVXVX8L9iKm+CQlQVxGeZqeeIZbYUnLGBOcdOBe39QhZYG5IjIVmAo86Bua71ngQeCBXM7RQVW3BdogIr1w48/WFZHx2TaVBbYHG2SeSUtEyqnqbhGpBKzxPfzbKqnqjmAvZIpHcjKk/L2XaZtn0gGsetAYExRV3YS7hwpV3SMiS4BaR4wROBO4opCXmOE7f2XcqBh+e3DjzwYlv5LWJ8DFuDEHs/f2EN/yScFeyBSPpCRYtnYr103q5+aBsZKWMaaARKQO0AL49YhNNwKf5XKYAlNERIF3fPMgZm1UXQusxQ3hVGh5Ji1Vvdj3XLcoFzHFp1o1qH+wOkNumAyPtLaSljHGL+6ItqahRyYWABEpA4wB7lLV3dnWP4yrQhyZy/nbqepGEakKTBWRP1X1xxDGDwR/n9a3qtopv3Um8kaMADcJdEtITLSSljHGL11VW+W1g4jE4xLWSFUdm239Dbhat065jRmoqht9zyki8jnQGgh50sqz96CIlPa1Z1UWkYoiUsn3qAPUDHUwJjT+3PYn4/4cB6VLW0nLGBMUERFcr/Al2ccBFJEuuI4Xl6rqvlyOTfZ13kBEkoELgFznRhSRRBFpWJg48+vyPgDXnnWK79n/+AJ4ozAXNOH14YdwWc8MLvvsMtRKWsaY4LUDegMdfd3WfxeRbsDruB5+U33r3gYQkZoiMsl3bDVguojMB2YBE1X160AXEZFLgN+Br33LzY/oTZin/Nq0XgVeFZHbbcimkmH5clg2vRHzRv0GI6+ykpYxJiiqOp3Ag0ZMCrDOXx3Yzfd6FdAsyEs9gas6/N537O++2rugBHuf1hAROQ1oBJTOtn5EsBcyxSM5GTIzhUaVmiNW0jLGRJ90Vd3laiMLLqgRMUTkcdw4g0OADsBzQFCDG5ri5Z9T66M5Y8ksVcqSljEm2iwSkWuBWBFpICJDcPdwBSXYYZyuADoBm1W1L64YWKrAoZqw88+pddOYO8koHW/Vg8aYaHM70Bg3AeSnwG7grmAPDjZp7VfVTCBdRMoBKdiNxVGpShWo1yCDqddNIzaprJW0jDFRRVX3qerDqnoG0AZ4VlWD/nUdbNKa4xsk8V1c78F5uB4iJsp07w4rlsXS+fT6xCQmWknLGBNVROQTESnn6xq/GFgqIv8M9vh8k5av7/4zqrpTVd8Gzgdu8FUTmii0ZucaRswfQVpCnJW0jDHRppFvpI0euJ6JJ+C62gcl36Tlu/t5XLblNaoa9OCGpngtWQLdzk/ihiFvkhqTbiUtY0y0ifeNvNED+EJVDxFgJuPcBFs9OFNEzihMdKZ4paXBkjlVefO8cZSrWM1KWsaYaPMObsaQZOBHETkR1xkjKMHOXNwBGCAia4FUfKO8q2rTgsVqws3f5b2sVCc2MdlKWsaYqKKqrwGvZVu1VkQ6BHt8sEmra4GiMhHj7/I+ecl0usdkUNZKWsaYKCIipYCeQB1y5qCngjk+qOpB3zwoFYBLfI8KvnUmyvhLWh/PHcO2zD2QkQGHDkU2KGOMyfIF0B03zUlqtkdQgp2a5E7gJsA/VP3HIjLUxiOMPsnJ0KJFJtd3f4Tae99zKw8cgPj4yAZmjDHO8arapbAHB1s92A9oo6qpACLyLPALblgnE0Xi42HevBjgOHizrFu5fz+ULRvRuIwxxmeGiDRR1YWFOTjYpCVARrblDAKPBmyiwPZ92/ls8WdckbGbqmCdMYwx0eRsoI+IrMYN5VSgjn3BJq3hwK++2SjB9a9/v6CRmuJxWfcEfkrbRsuLt7ukZZ0xjDHRo0gd+4LtiPES0BfYAfwN9FXVV4pyYRM+m9aW4bJq93N63XZuhZW0jDFRoqgd+/IsaYlIpWyLa3yPw9tUdUdBgjXFIzlJyDhYmvgy5dwKK2kZY6JEUTv25Vc9OBc3vEag9ivFRnqPSolJGfy5aT0r922gHljSMsZEkyJ17Mszaalq3SKHZ4pdQukMli3fwG+7/nRJy6oHjTHRo0gd+4LtiIGIXI7r9aHAT6o6Lp9DTIS0bRNP/fot6d68FPCClbSMMdEke8c+wd1oHHTHvmBvLn4TqI+bZRLgZhE5X1UHFjBYUwyeeUaA0rDMd2+WlbSMMVFCVV8Ske9xhSBwHft+C/b4YEta5wGn+aYpQUQ+BAp1Y5gJv0zN5Pmfn6dDXH1ag5W0jDHRSIBMCnjPb7BTkyzFTdTlVxuwObWi1OBnhEE9u/H95pluhZW0jDFRQkQeAz4EKgKVgeEi8kiwxwdb0joOWCIis3zLZ+Dm2BoPoKqXBh+yCbe9e4W4HadxT8eHsTYtY0yU6QW0UNUDACIyGJgH/DuYg4NNWo8VLjYTCcnJkJ4uaHwZt8JKWsaY6LEGKA34v5hKASuDPTjYpLVVVf/IvkJE2qvq98FeyBQf//Qk4/6YxpVxcVbSMsZEk4PAYhGZiuuNfj4wXUReA1DVO/I6ONikNVpERgDP4zLkc0Ar4KzCRm3Cxz8R5JeLpnFl6dKWtIwx0eRz38Pv+4IcHGzSagM8C8wAygIjgXYFuZApPvXrw2WXZTL4gv9A4ntWPWiMiRqq+qH/tYhUBGqratAd+4JNWoeA/UAirqS1WlUzCxKoKT4dO0LHjr6OoYmJVtIyxkQN3z1al+Lyz+/AVhH5QVXvCeb4YLu8z8YlrVa4G8J6icj/Ch6uKS6vznyVUYtGQenSVtIyxkST8qq6G7gcGK6qpwOdgz042KR1E7AceEhVNwO34zKkiUKzZ8O9nbD+tfMAACAASURBVG9g6P9WWEnLGBNt4kSkBnAV8GVBDw42afUFzsT1rwfYgxsvykShuDjISK3AnS0esZKWMSbaPAVMBlaq6mwROQlXKApK0B0xVLWliPwGoKp/i0h8wWM1xcHf5T01FStpGWOiiqr+F/hvtuVVQM9gjw+2pHVIRGJxfeoRkSr+13kRkS4islREVojIoDz2O0NEMkTkiiDjMXnwd3mftPgHK2kZY4IiIrVFZJqILBGRxb7JGhGRSiIyVUSW+54r5nJ8sN/3J4vItyKyyLfctCDDOAWbtF7D9auvKiL/AaYDT+d1gC/JvQF0BRrhOm80ymW/Z3HFRRMC/pLW3L+WWEnLGBOsdOBeVT0V1xw00PedPQj4VlUbAN/6lnMI9vve513gQVyvdHzd3a8JNsigkpaqjgTuB54BNgE9fEW8vLQGVqjqKlVNA0YRuB3sdmAMkBJs0CZvZcpA797w0rU3W9IyxgRFVTep6jzf6z3AEqAW7nvbf2/Vh0CPAIcH+30PkKSqs45Ylx5snEFPAqmqfwJ/Brs/7s2uy7a8HneT8mEiUgu4DOiIG4TXhEBCAowY4Vv4n1UPGmMKRkTqAC2AX4FqqroJXGITkaoBDsn3+z6bbSJSj6zmpitwhaGgBJ20CiHQHClHtoO9AjygqhkiuU+pIiL9gf4ACQkJIQvQy4b9NozNe7bwkJW0jDFOnIjMybY8VFWHHrmTiJTB1X7dpaq78/puzn5YgHW59XsYCAwFThGRDcBq4LpgLgLhTVrrcfNu+R0PbDxin1bAKN+HUhnoJiLpqjou+06+D3YoQHJycr4dQAzc2eUSkk/9mYdabbeSljEGIF1VW+W1g69X+BhgpKqO9a3eIiI1fKWsGgRuygnm+x443Fuws4gk45qo9gNXA2uDeRPBdsQojNlAAxGpKyIJuIa28dl3UNW6qlpHVesA/wNuPTJhmcKpWqYKnY/vkdWmpZbrjTG5E1d6eB9YoqovZds0HrjB9/oG4IsAh+f7fS8i5UTkQRF5XUTOB/b5zrcCd6NxUMJW0lLVdBG5DdcrMBYYpqqLReRm3/a3w3Vt47q9p6biurxnZkJ6OsTbrXXGmFy1A3oDC0XEP+LRQ8Bg3Ewf/YC/gCsBRKQm8J6qdsvt+/6I838E/A38ghtl6X4gAdexL+gRlkRL2C/w5ORkTU1NjXQYUa9+021kJGxn9bUT4d57YdcuKFcu0mEZYyJERPapanIEr79QVZv4XscC24ATfD0VgxbO6kETQYdid5KyM9WVtMA6YxhjIu2Q/4WqZuBmCylQwoLwdsQwEfTATfU5eBBInO9WWGcMY0xkNROR3b7XAiT6lgVQVQ2qKsiSlkfdeqvvxahE92wlLWNMBKlqbCjOY9WDHvW/RZ9z/aibs6oHraRljPEAS1oe9eqTdfi032DU2rSMMR5iScuj2tVrQXxGBSTRVz1oJS1jjAdY0vKopCQ4eBAyqlR3K/74I7IBGWNMCFjS8qj1+93Yxluq1oJmzeCDDyIbkDHGhIAlLY9Ki9kFwM7dadCnD8yZA4sWRTYoY4wpIktaHnXbZW149FGoWakSXHcdxMVZacsYU+LZME7HissvhxkzYN06G4PQmGNQpIdxChUraXnU9yt/5qKh/ViZssGt6NMHtmyBr7+OaFzGGFMUlrQ8avavcUwa8D6/zPDNzda1K1StCsOHRzYwY4wpAktaHtWxoZvpunxsTbciPh6uvx4mTICtWyMYmTHGFJ4lLY9KSnLPOZr/+vZ182p98klEYjLGmKKypOVRa1PdzcTLN6/PWnnaadCqlfUiNMaUWJa0PCqudBoAu/dm5tzQpw/8/rt7GGNMCWNJy6POPbk5zz4L13Q9IeeGXr0gIcE6ZBhjSiS7T+tYdNVV8N13sHGjS2DGGM+z+7RMVFv992rOeekffD575tEb+/aF7dvhyy+LPzBjjCkCS1oepSgzn3iF4a8cf/TGCy6AmjUL3iEjI8M9jDHhk5ICmZn573eMsqTlUSdVPIlalSpRITZA0oqNhd69YdIk2Lw5/5OtWAEPPwwnngiVK8P770MJq1Yudjt22BePKbjly6FGDahdG+66C375xf6vHcGSloclJ8O+fbls7NPHlZpat4arr4YXXoDvv4fdu932vXtdSezcc6FBAxg8GJo3h6ZN4f/+Dzp1csnMHG3HDpfg33or0pGYkmbmTPdj55RT4O23oW1bqFMH/vlPN1ODJTBLWl61fd921u1fwuqULYF3OOUUGDkS2rSBX391/yk6dIAKFdy2GjVc29eWLS5hrVvn2sCmTYOhQ2HuXGjSBJ5/3t2wbLJMmeKS/qRJkY7ElDQLFkCpUjB5svu/9+GH7v7KV16BM86Ae+6JdIQRZ70HPerv/X9Tt8UaapWpzeI5lfM/YOtWl4jmzHGPKlVcaaxtWxA5ev+NG2HgQBg3Dlq2hHffdc8GbrgBRoyAsmVdqSsuLtIRmZLiwgth2zb3fzG7HTvg88+hcWM488xCndorvQctaXnYhAnu+ZJLwnQBVRg71iWvLVtc545WrXI+qlQJ08WjVGYmVK/uql537HCl2Natw3OtbdtcG6Pxjho1oEuXsNxH6ZWkZdWDHnbJJWFMWOBKYD17wpIl8Oqr0LEjLFsGjz8O3bq5UeXr14fffgtjEFFm7lxXan34Ybf8/ffhuc7ChVCtmquuNd6QkuI6RjVtGulIopolLY/KyMzg9Od78NjIceG/WMWKcMcd8NFHLoHt3Om+rF94AfbsgTvvPHYakL/6yiXzf/wDTj01fEnrq69cqe7HH8Nz/lB77z3XLmNyt2CBe7aklSdLWh4VIzGkfN2Pl2/rUvwXL1cOzjsP7r0XnnwSfvoJJk4s/jgiYdIkVx1YuTK0b+/eezg6qviTYUkpxQ4eDE89ZbcB5MWSVlAsaXmUiHBNy0vIOFg6soH06+e6zA8a5P0bk7duhVmz3ISb4JLW3r0wb15or3PokEuGEPpzh8OWLbByJfz9N8yfH+looteCBa5N61hrBy4gS1oelpwM+/dH+MdtfDw8/TQsXuyqD71syhRXDdqtm1s+7zz3HOoqwrlzXTI84wx3K8L27aE9f6jNmJH1OlzVpV6wYIGVsoJgScvDPlryDpDHDcbFpWdPV2X26KMui3rVpEnuV/Lpp7vlatXC067lP9/dd7vnaK8i/Plnd+9RnTolo+PInj3w2mvu5vAxY1ypdulSV1IMV9tserr7YWdJK192A4mHVSzrRnDftw/KlIlgICLw7LPu5uU33oD77otgMGGSkeFuCO3WDWKy/RZs396VMNPTQ3e/1rRp7n6dCy5wy/PmQefOoTl3OMyY4W5/aNwYPvvMfVaxsZGOKrBVq+DSS10CCSQhwf0Ie+op1zM2VJYuhbS0iCYtERkGXAykqOppvnWfAQ19u1QAdqpq8wDHrgH2ABlAuqq2ClecVtLysFGD+jJ6dIQTll/79q6t5+mn3S9Wr5k921XT+asG/Tp0CG27VloaTJ/uznvccXDCCdFd0jpwwFVntm3r/gZ27YreeH/4wdUIbNzofoBs3OgmS50yBT7+GF5+2Q1h9sUXbtSYAQNgw4bQXNvfCaNZs9Ccr3A+AHL03FLVq1W1uS9RjQHG5nF8B9++YUtY/qBK1CMpKUlNCfX776oiqg88EOlIQu/RR1VjYlS3b8+5fssWVVB99tnQXOfnn935xoxxyz16qJ58cmjOHQ7Tp7t4x41T3bjRvX7uuUhHdbR33lGNi1M95RTVZcvy3nfTJtXbblONj1ctXVr1vvtUt20r2vUHDXLnO3iwaOfJA5Cq+Xy/AnWARQHWC7AOaJDLcWuAyvmdPxQPK2l52IXvXsOVg9+JnoJNs2Zw/fXuRuT16yMdTWh99ZUbXqdSpZzrq1aFRo1C15bjP8+557rnFi3cyOB794bm/KH288/u+ayzXM+4U06Jrnat9HR3j+GAAa6KdeZM19s1L9Wrw5AhrkrvqqvgxRehbt2idTRasMC1f0bvpKznAFtUdXku2xWYIiJzRaR/OAOxpOVhydvb8b8HB0RXL2P/vTpPPBH+a6WkFM9NzVu2uPEaj6wa9Gvf3lXpHTpU9GtNm+baPfzDN7Vs6d5jVP0jZzNjhksCVau65Q4dwnfvWkGlpLgq6yFD3D2FX34J5csHf3zdum5A24UL3aC2t9xS+OrC4uk5GCcic7I9CpJcegGf5rG9naq2BLoCA0Xk3CJFmgdLWh42qOPtAETVUI116rixCocPhz/+CN911qxxcxK9+GL4ruE3ebJ7zitphaJd6+BBlwQ6dMha16KFe47G+7VUXbxt22at87fxHTkgbHHavdv9aKpXz40oMny4G72lsJ1DGjd2bV7p6e5+xILascPVPIQ/aaWraqtsj6HBHCQiccDlwGe57aOqG33PKcDnQJgG3LSk5WlJSe45qpIWwEMPuZvIwlnaGjnSdVp46in3izqcJk1yVUbNj+pU5YTqfq1Zs9wtA+3bZ62rWdN1s4/Gzg0rVrgbrtu1y1rn/ywiUUV44IAbSqpePTdSS5curoTap0/Rz33SSa609vHHbuLGgoiOThh56Qz8qaoB6/RFJFlEyvpfAxcAi8IVjCUtD3v4p7uAKLhP60iVK7vS1pgxrj0m1FRd0jr5ZPfmn3wy9NfwS093Ja2uXQNP4QJZ7VpFTVrTprlr+L/4wS23bBlcSevFF4t3Ykp/e1b2klbVqq5kUtiklZrq2kXvv999nsFUuaanu9LUySe7e9tatHC9Pf/7X9fGFioPPuja7e64o2B39EfJ8E0i8inwC9BQRNaLSD/fpms4ompQRGqKiH/CuGrAdBGZD8wCJqrq12ELtDh6e4TyYb0Hg/fPsS8oqL7+eqQjCWDTJtVSpVRvuin05/7tN9dL7a23VG+9VTU2VvWPP0J/HdWs3nGjR+e93623qpYpo5qWVvhrdeig2qLF0esHDXI93w4cyP3Y1FTVpCTVGjVUMzMLH0NB3HSTaoUKqhkZOdffdpuLpTA95QYOdJ93fLx7LldOtWdP1fffV92wQXXrVtVvv1V95RXVG29UbdVKNTHR7XvGGarffBOa95abESPctYYPD/6Yfv1Uq1QJ+78LQfQeLAmPiAdQ0IclreClpal++aXq2rWRjiQXt9yimpDgvmxC6b773Jf4tm2qKSnui+3ii0N7Db+HHnJJ8e+/897vv/91/91mzizcdfbvd0n+7ruP3jZ6tDv3nDn5Xx9UFywoXAwF1aiRateuR68fM8bF8fPPBTvf1KnuuLvvVt29W/Xzz11irFUr671lf1Spotq5s9t/woTiSdYZGapnnqlarZrqrl3BHdOqlWqnTuGNSy1pWdIyRbdypbu36b77QnfO9HT3JXbJJVnrBg92f+rffhu66/g1b6567rn57+e/X2vw4KO3paervvyy6oMP5v7FOm2aO378+KO3LV/utr37bu7Xv+oql7xB9fnn84+3qHbscNf697+P3rZ1a+7bcrNzp2rt2qoNG6ru25dzW2am6vz57v6vF15QnTJFdfPmosVfFL/+6t7f/ffnv296urvX6557wh6WJS1LWlFvwIQB2vyeh/S33yIdSR6uvdZVm+3YEZrzffed+7MeNSpr3f79qiee6BJMenrBz7lihfu1ftttOR+33OKu9cwzwZ2ncWPVLl1yrlu7VvW88/Rw6eCllwIf+9hjLsEHKtFlZLiEdOutgY9NTVVNTlYdMMCVfs4/P7h4i2LiRPd+pk0LvL1p04KVLvr2de//119DEl7Y9enjqjDzu1H5zz/d5/TBB2EPyZKWJa2o9/qvr2tiub25fpdFhfnz3Z/hv/4VmvP16+eSYGpqzvWffKIFbmtQdb/qGzVyX0CVKh39qF3bJbVgDByYs11r5EjV8uXduuHDVbt3d9WagarNzj1X9fTTcz/3uee6aqlA/NVxU6e65Fuq1NGfT6g9/LCrNt27N/D2O+90JYy82uH8Jkxw8T/8cGhjDKeNG92/a/YSfyD+qt1588IekiUtS1olQu3a7kdfVLvoItXKlYv+Rbp/v0sCvXsfvS0zU7V1a9WaNXP/Ig3kttvcf5MpU4oWm2pWu9JXX6n26uVet23rqklVXSmqbl3V4493VWh++/a5tr+8qlHvust1OAhUkrzmGvf5Hjqk+vXXWTGEU/v2rq0mN+PGuTh++CHv82zbplq9uiuZhXGIo7B49ln3Hr/+Ovd9HnnEJff9+8MejiUtS1olwimnqF55ZaSjyMdPP7k/xddey32f1NT8e5T4SxS5fUn4r/PUU8HFNWmS2/+uu4LbPz/+dq3YWFei+ve/XSLJbu5cVxK68MKsXnfffOOOmzgx93N/+KHbZ/HinOv37XO/+P29NFNTc+/QESppaa534B135L7Pjh1uHMonnsj7XNdc40q5v/8e2hiLw4EDqvXquXa43JLSJZe4knwxsKRlSSvq/XPKP7V07UXarVukIwlCu3aqJ5wQuEv4d9+p1qnjShu//JL7OXr2VK1a9ehEkN3ll7v2nY0b845nyxbXA6xJk9D+Cm7b1g1wO2tW7vu8/XbO5Or/NZ5Xb7SFC90xH3+cc/3nn7v1kydnrTv//PB+Uc6e7a752Wd579eihSuR5eazz7TAHTaizeTJ7j3kVko+8URX6i4GlrQsaUW9kQtG6vGnrczzeyFqfPml+3P88MOsdXv2ZHV2aNDAVZ1Vr666bt3Rx+/c6UoQef26V3U97eLjXfuPv1ruSJmZ7hdwqVKh7x5+4MDR9y0Fuv5117mSyDffuITeunXexxw6FLgX2rXXura37D8GXnjBfaZ//VW495CfV1915w/075TdPfe4zzjQj4K5c1WPO87dW5XXj5CSoH9/9295ZFvlzp1aoI48RWRJy5JWiTB//tE1RlEpM9OVaho1cl/q337rSlci7sstNVV10SJX1XX66Ue3f73/vvtzDqZ32Wefud52ycluSooju5n7Szovvxy691dQe/aonnqqu9coPj646Vxat3Y3IPvt369atqzrnJLdggXu/b3/fmhj9rvqKldqzo+/g8V332WtW7kyq72vShXVJUvCE2Nx2r3b/S03aJDz79ZfXZ1XtW8IWdIK5uRuQrGlwApgUIDt1wELfI8ZQLP8zmlJy8NGjnR/kp07u+eTTz761+n48S6RXXNNzmTTsaNq/frB30D6119Z1+naNesG5yVLXIeGCy7Iv0QUbosXu7ahYDtODBjgRqDwfwZffKEB2/gyM12HlKuuCn3MmZnuPrlrrsl/3507XTf2Rx91N4HfcYdL0ImJrqfgzp2hjy9S/LdiZK8JeOMNDapEGiKWtPJPWLHASuAkIAGYDzQ6Yp+2QEXf667Ar/md15JW8J7+8Wkte+sF+umnkY4kSIcOuSpAEdV77z36JlK/p592f7pPP+2W1693xzz+eMGul5HhxrhKTFStWFH1o49UW7Z01VL5tXkVl9GjXS+8YHo8+kuIq1a55euvd+8rUDthnz5uW2HuW8vL2rUuhiFDgtu/VSuXQMuWdQnspptCP0JKtPD3RPXfu9a/v/s3KKZhtSxp5Z+0zgImZ1t+EHgwj/0rAhvyO68lreBNXDZRT+s6XatWLaax5kJh2TJXp5mXzEz3S17ElSb8bTRLlxbumkuXujYu/w2+n39euPNE2qxZLv4xY1zbWbly7qbcQPz3rYX6Zl3/eYO97+iRR9z+PXqEb3zIaLF3r6sNqFPHVf+eeWbeHVFCzCtJKy4cg/D61MJNz+y3HmiTx/79gK/CGM8xp1uDbnx7KrzzY6QjKYD8Zo0FN7L5+++7EeKvu86NHN6qlRvFuzBOPtlNTPjqqy5t9ehRuPNEWpMmbk6oefMgPt7NG3XllYH3Pf989zlOngytQzj10YwZbtqZJk2C2//hh+GGG6B+/dDFEK2Sk+GDD+Ccc9w0JgsXQr9++R5mcgpn0go0T4MG3FGkAy5pnZ3L9v5Af4CE6J2OOiolJbnZOVRznzmjREpKgnHj4IwzYNUquP32op0vLs59kZRkpUu7KVB++w3WrYMKFaBTp8D7Vq7spjSZPBkefTQ018/IcNOFnHmm+zyDjflYSFh+7drBPfdkTU4a4elISqJwzqe1Hqidbfl4YOORO4lIU+A9oLuqbg90IlUdqr7ZNuOC/c9geHvO2wz+9VFU3fx3nnP88fDFF3D55dC7d6SjiQ4tWsCcOe5z6d4d8vqRd+GFMHMm7NpV9Ovu3+9KdYsWwVVXFf18Xvavf2XN42VJq8DCmbRmAw1EpK6IJOAmEhuffQcROQEYC/RW1WVhjOWY1KRqE85rcDoQhbMXh0rr1m4yyeOOi3Qk0aFlSzdT865duVcN+l14oSsdffdd4O2//QY33+wSUV62b3clunHjXBVr//6Fi/1YkZgIo0bB//1f7rNdm1yJa58L08lFugGv4HoSDlPV/4jIzQCq+raIvAf0BNb6DklX1VZ5nTM5OVlTPfsNHHrbtrnvsJNPDr7GxpRgP/7oZjYuXx62bIFSpXLfNy3NJfvrrz96RuNJk1yJKTUVYmJc8nrySVetmN3q1W7a+rVr3WzRPXuG/j2ZkBCRfaqaHOk4iiqsSSscLGkZk4fdu11b1vXXw4gR+e/fvbvrELByZVaj59tvw8CB0KyZO8c777ikVrYsPPEE3Hqr6+gxdy5cdJFLfuPHw9kBm6RNlPBK0gpn9aCJsNGLRxN7z0k89J8tbNoU6WhMsShXzrVnPfNMcPtfeKErLa1YAZmZ8MADcMstrvT0449w2mkwZAjMn++qYu+6y7XDPP+8K9GVLg0//2wJyxQbK2l52MItC3n2ozmM/GdffvgBzj030hGZqLNihbvN4IUXYNYsGD3aVQUOGXJ0fbIqTJzoer8tX+46fUycCDVqRCZ2UyBeKWlZ0vK4efPg9NNdLY91sDMB1asHa9a4ktazz8I//5n3/RFpafDVV9Cxo6syNCWCV5KWVQ96XJOmGZxwgvLxx5GOxEStHj1cG9Vnn8H99+d/Q19CgmsLs4RlIsBKWh72zapvOP+j8+nz9xo+fO1E1qyBE06IdFQm6hw86DpwVKkS6UhMGFlJy0S9ehXr8WT7J+nXN47kZHfbjTFHKVXKEpYpMaykdYzYv9/d02iMOTZZSctEvUzNZP+h/aRnppOY6Dp/7d8f6aiMMabwLGl52KKURSQ9ncQXf36BKnToAAMGRDoqY4wpPEtaHlazbE0GdxpM46qNEXFjdP7vf6EZH9UYYyLB2rSOIbNmQZs2MHQo3HRTpKMxxhQnr7RpWdLysEzNZM/BPZSKK0XpuNKourn5ypaFX36JdHTGmOLklaRl1YMetmXvFio8W4EPfv8AcPeM9u3rplBasiSysRljTGHYZBUeVr50eV684EXOOv6sw+t693a35Jx4YgQDM8aYQrLqQWOMOQZY9aCJeqpKSmoKe9P25li/fz+89BJ8/31k4jLGRB8RGSYiKSKyKNu6J0Rkg4j87nt0y+XYLiKyVERWiMigcMZpScvDMjSDai9U4+VfXs6xPi4OnnvOzYxujDE+HwBdAqx/WVWb+x6TjtwoIrHAG0BXoBHQS0QahStIS1oeFiuxvN71dbo26JpjfXy8a9v68ktISYlQcMaYqKKqPwI7CnFoa2CFqq5S1TRgFNA9pMFlY0nLw0SEga0H0qpmq6O29e0L6elw0knQtm3W+qeecttefRU2by78tZcvh2uugWHDCn8OY0xUuE1EFviqDysG2F4LWJdteb1vXVhY0vK4jXs2smP/0T+eGjWC9993Nxl36pS1fulSmDLFzapeqxZccAGMHVuA6210E9+eeipMmAB79oTgTZRAI0fCpk2RjsKYHOJEZE62R/8gjnkLqAc0BzYBLwbYJ9AEbGHr4Wdd3j3utDdP47om1zGk25Cjtt1449H7jxzpnv/8070eORKmT4fLL4eMDDdPYLVqULWqe1SuDLGx7pg33nCT3qanwy23wCOPuO7148dD9erQunUY32gU+eMPuP56aNkS5s6NdDTGC0I0S0O6qh5d7ZIHVd3ify0i7wJfBthtPVA72/LxwMZCRRgE6/LucR8v+Jh6FetxVu2z8t85AP/I8ElJMG2am2E9OxGX2Hr1ciWr0aPhySddtSO4+QVPPBFatHAztB8LUlJcYgfYutUldmMK6sAB939m5Ej4+mtYtcr9UCysYLq8i0gd4EtVPc23XENVN/le3w20UdVrjjgmDlgGdAI2ALOBa1V1ceGjzSNGS1omWIcOwcqV7ks5JQW2bHHPV14Jp52W+3H/+Y8rdS1cmPd+XrJ4sXuvzz7rZrAPtT/+gMaN3W0L550X+vObyFm9Gv79bxgzxg1uXbWqax9+4AGoWbPw580vaYnIp0B7oDKwBXjct9wcV923BhigqptEpCbwnqp28x3bDXgFiAWGqep/Ch9pPu/Dkpa3rdu1jriYOGqUrRGxGLZvhxNOgKuuguHDIxZGsdiyxZVIL74YLrkE1qxxiT4mxK3HDz0EzzwD77wD/YNpmTAlxtq10LQpXHYZXHedm1IoLgQNOV65udiSlsc1fL0hLaq3YNQVoyIax+23uy/YNWuK9msx2r39tmvPW7wYUlPdl02LFqG/TqtW7taFkjLwcXq6ew7Fl++xIC0NEhJCe06vJC3rPehxz3V+joFnDIx0GNx1lyttrV5dPNfbs8f1fPzxx+K5nt+4cdCgges9ecYZ4UlY69e7Dh7du7svt7178z8m0tq2hebNIx1F9Bs8GL79NvQJy0ssaXlc91O6c86J50Q6DOrVg2XLoF274rneiBEwdWrx/rLftQu++w569HAdVAD++gtuuMHdtxYqs2e7Hpvnnus6ebz9dujOHQ6pqS7mxYtdko1Gqjl/UA0cCPfcAwsWFF8Mu3a5tt9vvy2+a5ZElrQ8bs3ONaz+u5iKlAZ+ywAAIABJREFUN/mIiXE9ov74I7zXUXXd71u1ynnjdLh99ZXrrNKjR9a6+Hj45BN4663QXeeyy1yvxLPOgjp1YOLE0J07HLJXYYYrCezfD08/DV98Efwxu3e7f5err4YaNdwPq5073bZGjeD116FZM3frwmuvwbZt4Ynd77vv3G0lF14Y3uuUeKpaoh5JSUlqgnf2sLO144cdIx3GYZdeqlq3ruqhQ+G7xrffqoLqBx+oLlumev75qqtXh+96frffrlqtmmp6es71V1+tWqGCampq6K85aJBqXJzqzp2hP3co/fST6nXXqS5cGJ7zjxrl/s1B9bnnVDMz895/xQrVhg3d/rVqudiGDlXdvTtrn23bVIcMUT39dLdfz57hid1vwADVsmVVDx4Mz/mBVI2C7/CiPiIeQEEflrQK5ttV3+q01dMiHcZhY8e6v7rRo8N3jcsvVz3uONX9+1X/+ks1KUm1e/fwXc8vM1N18+aj1//4o3vP776b+7Fz5qhOnpz/NUaPVj3vvKzr/PRT+D/PaLVzp+p337nXmZmqM2aoXnWV+zwGDjz6x0N269erNm2q+s03+Sc4VdX581UXLHCvV61S7dEjazkUMjNV69QJ79+pJS1LWqYQ0tNV69dXbd06uC+Lwnj9ddVXXslaHjzY/aV/+WV4rpefzEz3BdmsWeD3vHSpKy3Fx7skm5err1atWjXrC/nQIdVKlVRvuCHkYYfEggWuhLJsmXvvoSrxjhunWrOmavnyOUtHGRmq//yn+/ceOfLo4yZMyPrsMjIKd+0JE9x1RVR793ZJrKi2bVM98UTVN98s+rlyY0nLklaJsPrv1fpHyh+RDiOHN95wf3k//lg81zt4UPWUU1RPOsmVvsLh/vtVb7459+0ff6x6992qBw5krcuewF5/3SWtW27J/RwHD6qWK6far1/O9aNGqc6aVbi4w+3f/3b/1ps2qb7wgnu9fXvhz5eWptqrlztP06aqs2cH3m/q1KzPNzPTHTdwoDvuvfcKf32/7dvdv3np0u7f7d57i/4jLDMzvNXmlrQsaZUIl392uTZ+o3Gkw8ghNVW1ShX3Hz2UDhxQHTYscNvRN9+4v/YXXgjtNVXdL/bq1VWvuCL4Y5YscW0l8+ZlrRswQDUhQXXdusDHTJ7s3sP48UWLtziddZZqq1bu9Xffufi/+qrw53viCXeOxx93iSg/y5aptmmj2qGDO+7ee/OuNiyo9etVb7pJ9bbbinaecNU6ZGdJy5JWiTBz3Uz9dtW3kQ7jKKtWhf4/6kcfub/o3NqGRo0KT2eIX35x1/3447z3y8x0JYB33lEtU0a1cmXV77/P2r5mjfvV/vTTgY+/9VbXPrdv39HbvvlGdeLEwr+HcNi61VWhPf64W969O+dyYTz2mOo//hH8/rNmuc85Pt79oAkX/9/ysmUF/xs7cMB1Tvrgg9DHlZ0lLUtaJgTWrlVduTI05zrzTNWTT86/rSJ7FV0oPPCAa5PasSPv/ZYv18M93Nq0Cdx+tWBB7sl82DDVRx8NvK1dO9WWLfO+/rvvuvaY4jJihHuv2avwmjRR7dKlaOct6I+djRtV//yzaNcMxt9/uwTZo0fBSnP+3q7hLkF7JWnZfVoet2bnGuZsnBPpMAJKT3eDvfbu7e5PKYq5c2HmTLj11rzH+Vuxwo1WMX580a6X3bhx0L49VAw0PV429evDffe5xw8/QO3aR+/TpIm7MfngwaO39e3rJukM5KKLYN683OfwmjIFHn4YypbNO8ZQio934+a1bJm1rk0bmDXLpe6CePHFrJtuJdDsTXmoUQMaNizYMYVRoQI89pj7e7jjjuDf4+TJWZ+VCUKks2ZBH1bSKpgBEwZo1eerRjqMXPmr9F56qWjn6dtXNTnZ/drNS1qaauPGqrVr5+x1VlgHD6rec0/gnmqFNXas+8W+YUPWuvnz8y7JzZ+vuXYy2LpVtUYN1UaNAlctFqdff1X93/8K1uFgxgzVmBjXdlQS+HsvPvNMcPs3a6bavn14Y1L1Tkkr4gEU9GFJq2AWblkYlW1afpmZqpdc4nphLVtWuHNkZKiec47ryBCMGTNc28rttxfueuG2cqWrbrzjjqx1p532/+3de1yUVf4H8M8ZEJSrinhDCS/cBgxLwzU1LbOwsrbUfiqrmVqWWbldtpu7P5Zstyx3f6G5qZOh5l3XG6u2lkpipSKCchmQm4wCKgjDcGdmvr8/HhjHkZvGMMzwfb9evODM83Cec2Ce5zvnPOc5RxpM0BS9nmjAAKlryvT1Z5+V7umcOyc9LDt7tnnKbaykpG0ekq2oIPL1lYaDt8WHjPag0xHNmCFdXVu6z1hQcGcB7rfgoMVBi7WRK1ekGSPGjr37Z2f0+jsbzv7661Lg+vnnlvc1LtPly7e2eOLjzTNMed48KZDn50tBrDWt0YULpQdUjcurUEi/+/nnUjoyUkq31X3EpvzpT9LzY43dPzx9WrqP0xpvvCGVt+EhYmtRXU308cctvydVKqmOycnmLxMHLQ5aViGnJIeO5xy3dDFatGGD9PxRY91X+fk3A0NGBtHOndJouYQE6eKbn3/nxysrk7oITZ95MpWdfevzQNOnS2dN795S685cn5Kzsojs7IiWLJGCVWsCTWnp7QMA9u2TZoloCGR5eVJX20cftX2ZjcnlRBMnNr7tkUek4f4tOXNGqndHbRG3VnGx9OHG0jhocdCyCkt/XEoiQli6GK2m10sjvRQKorlziYYMkd6lDQ/PfvklGUbgGX819ZBpc3Jzm2/ZFRRIx+/e/eaUPbGxUqtl/nypZejvb75Wy4svEnXrJt3zCA5uu3yfeEKab89cD7JmZ0v/k3/+s/HtH3wgdX+2dH9Np5NGTJaXt30Z29PUqdJD4abvUZ1OelyiLZ8baw4HLQ5aViHrRhYdzT5K+vZ4erENxMdLQQK4OXz4iy+kbjki6V5JUpL0fNOePUTffCNd2O62W5HoZhecsZISKVg4ObWuC9EcLl2SusXupGW0Zo3Uwlm2jOizzxofHt4w/6O5prVauVLKv6l7lHv3Stvj4prOw0rerq2Slyd12/boQZSYePP1+Hhq1fN9bYWDFgctZgZxcdLzRGlp7XPhapgLcfTom4GvokJ67qlLl9ZNYmtu6ek3g3ZLGqbIEqLph3Bra6VWnLm6rMLCpMETTWkYfLBiRePbKyqIAgOJtm83T/ksITtb6o7u1evm/atPPpH+Do1NsmwOthK0hFQX6+Hs7EwVFRWWLobVUKlVSCtKw8M+D6OLXRdLF6dD2rQJmDMHWLkSWLxYWptp1iwgPByYNs3Spbszly5Ja2wNGgQkJgJubu1fhl9/BYqLpWfHmuLjA4SGAjt23L4tKgp4800gLq79Fg1tDxcvSs8lDhok1W3CBGlNr3Pn2uf4QohKInJun6OZDwctG7fy1Eq8cfgNFL1bBA8nD0sXp0MiAiZPBk6ckFbYlcul1+70IdaOYtUq6eI4bFjz+6WnSw8jT5jQLsW6hVIpPVztbHIJra2VFmMcPFh6ANvWKJWAi4v0YcLDA3j7beDTT9vn2By0LISD1p3J1+QjpyQHD3g9AAc7B0sXp8PKzZU+AQNAefntF1Nb9NBDQGGhFLzaKkBv3w706XP3gXD9emD+fGkV6LCwtilTR5SYCNx3n7RacXvNhGErQYuncbJx/V37Y4z3GA5YLfDxAQ4ckJZf7wwBC5CCw8WLUguzMZcvtz4vvR7YuBFYuFBq6bWktBT48EPg5Mlb8/jsM+libutLzru7S1M+jRtn6ZJYH25p2bh8TT4SChIwwWcCXBxcLF0c1oFUVAD9+wPPPCMFnAZXr0r3lA4dAlJTpX0++ACYMQMYPvz2fH75BViyRJpTMDRUukfo59f8sauqpC6yd94B/v73m6/HxUlds3wxb3vc0mJW4adLP2HK1ilQqVWWLgrrYJydpQEnO3dKLR8iqXsuMBDYs0cKKJ6eQF4esHat1AKaOVNqnTXIyAAefFBqlW3cKAWwlgIWAHTrJgXAU6dufX3sWA5YrHkctGzcpMGTcOalM/Dp7mPporAOaMECKVjFxQETJ0pdhsHBQFIS8Oc/Aw4OwD33ANnZ0izx+/dLQe2tt6Tf9/MDtm6V7ovNnt38DPumRo2SBr7odMCRI9IM/aWl5qknsx3cPchYJ0YkDbt2d5e6BIOCpEDWVPApLAQ++UTqArxwofHlVVqr4VGD8+eB114DcnKArCwpULK2Zyvdg2YNWkKIMABfArADoCCiT022i/rtTwCoBDCXiBKay5OD1p25VnENJy6dwHif8ejl1MvSxWHMICNDuge2aJF0X+vLL6V1qJh52ErQMlv3oBDCDsBXACYDkAOYKYSQm+w2GYBv/dfLAP5lrvJ0VheuXsC0ndOQej3V0kVh7Ba+vsCNG1JXZK9eUguPsZaY855WKIBMIsomoloA2wA8Y7LPMwA21s8y8iuA7kKIfmYsU6cT6hWKpFeSMLjHYCgSFMguyQYAFFUWQZGgwKXSSwCkFpkiQWEYsFGgKYAiQYF8TT4A4ErZFSgSFCgsLwQA5KnzoEhQ4HrFdQBATkkOFAkK3Ki6AQDIupEFRYICpdXSTYqM4gwoEhTQ1GgAAMoiJRQJClTUSq3m1OupUCQoUFVXBUAKtooEBWp1tQCApMIkKBIU0Oq1AICEggQoEhRo6CmIz4/HNwnfGOp96vIpRCdGG9I/q37GpqRNhnRcXhw2n99sSMfmxmJb8jZD+ljOMexIuTldw4/ZP2JX6i5D+r9Z/8WetD2G9OHMw9in3GdI/yfjP4jJiDGkD6QfwMGLBw3pvcq9+D7ze0N6d+pu/JD9gyG9M2UnjuYcNaS3JW9DbO7Np223XNiCE5dujlX/7vx3OJl3c/z4hsQN+PXyr4b0t+e+xekrpw1pRYICZ/PPAgD0pIciQYFzBdLUDHW6OigSFDh/9TwAoFpbDUWCAinXUgAAlXWVUCQokHY9DQCgqdFAkaBARnEGAEBdrYYiQYHMG5kAgJKqkkbfe3nqSzh/Hjh4EBj7fDyK6/i9B7T83uv0zDU/FIBpkLoEG9KzAawy2ScGwFij9I8ARjaS18sA4gHEOzg4NDm3Fmta8tVkQgRoR/IOIiI6m3+WEAHap9xHRES/qH4hRIAOXTxERETHc44TIkBHs6WFjP6b+V9CBCjukjTLaUx6DCECdPqyNP367tTdhAhQUmESERFtvbCVEAFKu55GRETR56IJEaDsG9lERLQ2fi0hAnRZLU2qt+rUKkIE6Fr5NSIiWvHzCkIESF2tJiKiv5/4OyECVFUnLVAUcSyCEAHDRMAf/vAh2UfaG+r7zvfvkNMnN+epfP3g69Tj0x6G9MIDC6nP530M6bl755L3P70N6Vm7Z9HQqKGG9LQd00j+ldyQnrJlCt339X2G9OObHqdR60YZ0hOiJ9BD3z5kSD/4zYP06MZHDemRa0fSE5ufMKSHrR5Gz2571pD2X+lP/7Pzfwxpn//zoTl7bk4m2H9Ff1qwb4Eh3Wt5L1oUs8iQdv2bK/3x8B8NacePHem9I+8Z0ogA/eXoX4iIqFZbS4gALYtdRkRE5TXlhAjQ8rjlRERUXFlMiAB9+euXRERUoCkgRID+deZfRESUW5JLiACtT1hPREQZRRmECNB3SdJMsM29944cIervXUV4z53fe/Vaeu/dLdjI3IPmDFrTGwlaK032+U8jQWtEc/nyhLl3p1ZbSyq1iipqK4iIqEZbQyq1iiprpfUhquuqSaVWGU5M03RVXRWp1CqqrpNW9ausrSSVWkU12ppG0xW1FaRSq6hWW0tE0oVQpVZRnU5aD0NToyGVWkVanbbRdFl1GanUKtLppVls1dVqUqlVhguFabq0qpRUapWhviVVJYaLUmPpG5U36ErZzfXsiyuLKb/s5sJcRRVFVKApaDJ9veI6FWpuznR6rfwaXS2/2mT6avlVw0WRiKhQU0jXK643mS7QFFBRRZEhnV+WT8WVxYb0lbIrdKPyRpPpy+rLVFJVYkir1CoqrSptNK3X60mlVhku0jq9jlRqFZVVlzWa1uq0pFKrSFOjISKiOl0dqdQqKq+R1hAxfa81997T64nU5fzeu5P33t2ylaBltoEYQojRACKI6PH69Af1Lbu/G+2zBsBxItpan04HMIGICprKlwdiMMbYneOBGC07A8BXCDFICOEAYAaA/Sb77AcwR0h+B0DdXMBijDFmHkKI9UKIa0KIZKPXPhdCKIUQ54UQe4QQ3Zv43VwhxAUhRKIQIt6c5TRb0CIiLYDFAL4HkAZgBxGlCCFeEUK8Ur/bQQDZADIBrAOwyFzlYYwx1qxoAKbTFB8BEExE9wLIAPBBM7//MBENJ6KRZiofAH64mDHGOoXWdA8KIXwAxBBRcCPbngUwjYjCG9mWC2kQXVHblLZpPI0TY4x1DvZCiHijr5fv8PfnATjUxDYC8F8hxNm7yPeO2Jszc8YYYx2G9m677oQQHwHQAtjcxC5jiChfCNEbwBEhhJKIfrrbgjaHW1qMMcaaJIR4AcBTAMKpiftJRJRf//0agD2QJpcwCw5ajDHGGlU/f+x7AJ4mosom9nEWQrg2/AzgMQDJje3bFjhoMcYYgxBiK4BfAPgLIS4LIeYDWAXAFVKXX6IQ4uv6ffsLIRrmJesDIE4IkQTgNID/ENFhs5WTRw8yxpjt44eLGWOMsXbGQYsxxpjV4KDFGGPMaljdPS0hhB5AVQu72UN6pqCz4Xp3Pp217lzvO9eNiKy+oWJ1Qas1hBDx5p7/qiPienc+nbXuXO/Oy+qjLmOMsc6DgxZjjDGrYatBa62lC2AhXO/Op7PWnevdSdnkPS3GGGO2yVZbWowxxmyQzQUtIUSYECJdCJEphHjf0uUxlyaWxu4phDgihLhY/72HJctoDkKIgUKIY0KINCFEihDizfrXbbruQoiuQojTQoik+nr/tf51m653AyGEnRDinBAipj5t8/VubAn7zlDvlthU0BJC2AH4CsBkAHIAM4UQcsuWymyicfvS2O8D+JGIfAH8WJ+2NVoAbxNRIIDfAXit/n9s63WvAfAIEYUAGA4gTAjxO9h+vRu8CSDNKN1Z6m26hH1nqXeTbCpoQVrDJZOIsomoFsA2AM9YuExmUb/A2g2Tl58BsKH+5w0Aft+uhWoHRFRARAn1P2sgXci8YON1J0l5fbJL/RfBxusNAEKIAQCeBKAwetnm692EzlpvA1sLWl4AVEbpy/WvdRZ9iKgAkC7uAHpbuDxmJYTwAXAfgFPoBHWv7yJLBHANwBEi6hT1BvB/AP4EQG/0Wmeod2NL2HeGejfL3tIFaGOikdd4eKQNEkK4ANgNYAkRlQnR2L/ethCRDsBwIUR3AHuEEMGWLpO5CSGeAnCNiM4KISZYujzt7LYl7C1doI7A1lpalwEMNEoPAJBvobJYwlUhRD8AqP9+zcLlMQshRBdIAWszEf27/uVOUXcAIKJSAMch3dO09XqPAfC0ECIXUnf/I0KI72D79W5qCXubr3dLbC1onQHgK4QYJIRwADADwH4Ll6k97QfwQv3PLwDYZ8GymIWQmlTfAEgjon8YbbLpugshPOtbWBBCdAPwKAAlbLzeRPQBEQ0gIh9I5/NRIvoDbLzezSxhb9P1bg2be7hYCPEEpD5wOwDriegTCxfJLOqXxp4AoBeAqwD+F8BeADsAeAPIAzCdiEwHa1g1IcRYACcAXMDNexwfQrqvZbN1F0LcC+nGux2kD5s7iChSCOEBG663sfruwXeI6Clbr7cQYjCk1hUg3cbZQkSf2Hq9W8PmghZjjDHbZWvdg4wxxmwYBy3GGGNWg4MWY4wxq8FBizHGmNXgoMUYY8xqcNBijDFmNThoMXaXhITPIcbaEZ9wjN0BIYRP/VpeqwEkANAZbZsmhIiu/zlaCBElhPhZCJEthJhmoSIzZlM4aDF25/wBbCSi+wBUNLNfPwBjATwF4NP2KBhjto6DFmN37hIR/dqK/fYSkZ6IUgH0MXehGOsMOGgxdueMW1fG86B1Ndmvxuhn2187hbF2wEGLsd/mqhAisH5AxrOWLgxjts7WFoFkrL29DyAG0orZyQBcLFscxmwbz/LOGGPManD3IGOMMavBQYsxxpjV4KDFGGPManDQYowxZjU4aDHGGLMaHLQYY4xZDQ5ajDHGrAYHLcYYY1bDojNinD17tre9vb0CQDA4gDLGWEehB5Cs1WoXjBgx4pqlC2PMokHL3t5e0bdv30BPT88SmUzGU3MwxlgHoNfrxfXr1+WFhYUKAE9bujzGLN26Cfb09CzjgMUYYx2HTCYjT09PNaResA7F0kFLxgGLMcY6nvprs6VjxG06XIGslZeX17CCgoI26W6NjIzsrdFoDP+b8ePHDy0qKrJri7xtXVRUlMecOXO8AeCtt97q/5e//OW2xRc3bdrU/ezZs6ZrX5nFTz/95DR37tyBd/O7xnXp7DZv3uz+4Ycf9rV0OUxFRUV55ObmdrnTbW3tt/x9mjpPOipemsQC9Ho9iAh2do3HoTVr1vR56aWXbri6uuoBIDY2NrNdC2jj9u7d212r1apHjBhR3Zb51tXVoUuXLrekH3roocqHHnqosi2P0xmFh4erAagtXQ5T3333Xa/hw4dX+fj41N3Jtt+isfdZR/37mAO3tACsXr2657BhwwIDAgLks2bNuufo0aPOfn5+8srKSlFWViYbOnRo0JkzZ7rGxMS4jhw50n/SpElDhgwZEjRr1ixvnU53W34RERF9fH19g3x9fYMiIyN7A0B6errD4MGDg/7whz94BwUFybOyshzCw8O9g4ODA4cOHRr0xz/+sT8ALFu2rPe1a9e6jB8/3m/UqFF+wK2tuObynjFjxj1Dhw4NGjNmjG95ebnNrJS7atUqDz8/P7m/v7/897///SAA2LJli/u9994bEBgYKH/wwQf9VCpVqz6AHTlyxPmHH37ovnTp0gEBAQHylJQUx59//rlbSEhIgJ+fn3zSpElDrl+/ftuniaaO99Zbb/WfOXPmPWPGjPF97rnnBpmmY2JiXB9++OGhOp0OXl5ew4xbzN7e3sEqlcr+butiTR599NEhQUFBgUOHDg364osvegGAVqvF1KlTfXx9fYP8/Pzkf/3rX3sD0jkwZMiQID8/P/lTTz01GLi11ZmSkuIYEhISEBwcHLhkyZL+Tk5O9wFATEyMa2hoqH9YWNjgQYMGBT399NOD9Ho9AOkcWrx4sdfw4cMDgoODA+Pi4pzGjh3rO3DgwODly5d7NpTzz3/+c5/g4OBAPz8/ecM52dT59e233/ZITk52mjNnzuCAgAC58TnX2LZ9+/a5BgYGyv38/OTTp0/3qaqquu0cXbFiRa/g4OBAf39/+eOPPz6kocdl6tSpPgsWLBgwatQov0WLFg0wTTf8fYqLi+28vLyGNVyXNBqNrG/fvvfW1NSIpvK2Nh2q0KHrQv2jTkV5AECNtkaErgv1X31mdU8A0NRoZKHrQv3XJazrAQDFlcV2oetC/TckbugOAAWaAvvQdaH+Wy5scQeAPHVeq078hISErrt27eoZHx+vVCqVqTKZjFJTU7uGhYWVLlmyxOu1114bMH369OIHHnigGgAuXLjg/OWXX6rS09NTcnNzHTdu3NjDOL8TJ044bdmyxePs2bNp8fHxaRs3bvQ8efJkNwDIzc3t+uKLLxanpaWl+vn51f7jH/+4kpycnKZUKlNOnjzpeurUqW5Lly691rt377rY2NiMU6dOZbQ277y8vK5vvPHGtczMzBR3d3edabnaxLx5AxEa6t+mX/PmNdt1Fh8f3/WLL77oFxsbm5Genp66Zs2aPACYNGlSeWJiojItLS112rRpNyIjI1vVNTJp0qSKRx99tHTZsmWXlUplalBQUM3cuXMH/e1vf7uckZGRGhQUVPXee+/1b+T3mjze+fPnnb7//vvMAwcO5DSWBgA7Ozs89thjpZs3b+4OAEePHnUeMGBA7cCBA7V3W5e7FRoKf9OvTz+FJwBoNJA1tj0qCh4AUFAAe9NtrTnm5s2bc1NSUtISExNT16xZ06ewsNDul19+cSooKOhy8eLFlIyMjNTXXnutGACioqL6Jicnp2ZkZKRGR0dfMs1r8eLFAxctWnQtOTk5rX///re0YtLS0rp99dVXqszMzJS8vDzHI0eOGBblHDhwYG1iYqJy1KhR5fPmzfM5cOBA1qlTp5SffvppfwD497//7ZaZmdn1/PnzaWlpaamJiYlOhw4dcgEaP79efPHFkuDg4MqNGzdmK5XKVBcXF8P9edNtMpkMCxcuHLR9+/asjIyMVK1Wi88//9wTJsLDw0uSk5PT0tPTU/39/auioqJ6NWzLysrqevLkyYx169ZdbiwNAB4eHrqAgIDKgwcPugLAtm3b3MePH692dHSk5vK2Jh0qaFnC4cOHXZOTk51CQkICAwIC5HFxcW7Z2dmOy5cvL4iNjXVLSkpy+vjjjwsb9h82bFiFXC6vtbe3x/PPP3/jxIkTt6xUe/z4cZcnnnii1M3NTe/u7q5/8sknS44dO+YKAP369audOHFiRcO+GzZs6CmXywPlcrn84sWLXZOSkpq9z9Jc3l5eXjUPPvhgFQDcd999lbm5uY5t+XeylO+//95typQpJf369dMCQJ8+fXQAkJOT4zBu3DhfPz8/eVRUVF+lUtntbvIvLi6202g0dk8++WQ5ALz00kvFv/76622rDzd3vLCwsFLjC5ZpusGsWbNu7Nq1qycAbN68uefUqVNvtGVdOrLPPvusj7+/v3zEiBGBhYWFXVJSUroGBATUqFQqxxdeeGHgrl273Hr06KEDAH9//6pnn3120OrVq3t26dLltr/juXPnXObNm3cDABYsWFBsvG3YsGEVQ4YMqbOzs0NQUFBlVlaWQ8O2559/vrR+n8r777+/okePHvoRbynHAAAGeUlEQVT+/ftrHR0d9UVFRXaHDx92++mnn9zkcrm8vjekq1Kp7Ar89vMrKSmp64ABA2ruvffeGgCYO3ducVxcnKvpfmfPnu02YsQIfz8/P/nu3bs9UlJSDNeE5557rsTe/uZncdN0g+nTp5ds3bq1BwDs2LGj54wZM0paytuadKhuiNMvnU5v+NnR3pGM066OrnrjtIeTh8443c+1n9Y47e3urW3NMYlITJ8+vfirr766Yvx6Xl6efWVlpUyr1YrKykqZm5ubHgCEuLVFb5pubiVoJycnfcPPSqXSYdWqVX3Onj2b5unpqZs6dapPdXV1sx8imsvbwcHBsNHOzo6qqqra/gPJ+vWqNs+zBUQEIcRtFV+8eLH3m2++WRgeHq6OiYlxjYyMvK111JaaO56zs7PeeF/TdIOJEydWzJ8/3zE/P9/+8OHD3T/55JN8S9Tl9GmkN7XN1RX65rb36wdtc9sbExMT4xobG+saHx+vdHV11YeGhvpXVVXJPD09dcnJyal79uxxW716de/t27f33LlzZ+6xY8cuHjp0yHXv3r3dly9f3v/ixYvJrT2Wo6Oj8XkArVZrOEG7du1KACCTyW45X2QyGerq6gQRYcmSJQXvvvtukXGe6enpDr/1/GrtCvEvv/zyoF27dmWOHj26KioqyiM2NtYQ2FxcXG55X5mmG8ycObM0MjLS6+rVq3bJyclOU6ZMKWspb2vS6VtaYWFhZTExMT2uXLliDwBXr161y8jIcJg7d67PRx99lD9t2rTixYsXD2jY/8KFC85KpdJBp9Nh165dPceNG6cxzu+RRx4pP3jwYHeNRiMrKyuTHTx4sMfDDz+sMT1uSUmJXbdu3fQ9e/bUqVQq++PHj7s3bHN2dtap1erb/jetzduWhIWFle3fv79nYWGhHSD9fwBAo9HYeXt71wFAdHS0x53k6eLioisrK5MBUneKm5ub7vDhwy4A8M0333iMHj263PR3fsvxGshkMkyePLl00aJFA4cOHVrVt29fXVvl3ZGVlpbaubu761xdXfXnzp3rmpSU5AwABQUF9jqdDnPnzi1dtmzZlQsXLjjpdDpkZWU5TJkyRbN69erLGo3GTq1W33KPcfjw4eXR0dE9AGD9+vU926qckydPLtu0aVOvhnMvJyenS8N1oSkuLi460/I1tm348OHVV65ccUhOTnYEgI0bN3qYXjsAoLKyUubt7V1XU1Mjtm3bdld1c3d314eEhFQsXLjQe+LEieqG1lhb5N0RdKiWliWMGDGieunSpVcmTpzop9fr0aVLF5o8eXKpvb09vfLKKze0Wi3uv//+gP3797vKZDIMHz68/O233x6gVCq7jRo1SjN79uxS4/zGjh1bOWvWrOL7778/EABmz559fcyYMVXp6ekOxvuNHj26Kjg4uNLX1zfI29u7ZsSIEYYL5QsvvFA0efJk3969e9cZ39dqbd62ZOTIkdVvv/12wbhx4wJkMhkFBwdX7t69O/ejjz7Knzlz5pA+ffrUjhw5siIvL6/V3TXh4eE3Xn31VZ+vv/66z65du7K+/fbbnFdfffWeN954Q+bt7V2zdevWXNPf+S3HMz32+PHjA6OiogzHaKu8O6qpU6eq165d6+nn5ycfMmRIdUhISAUA5Obmdpk/f76PXq8XABAZGXlZq9WKWbNmDdJoNHZEJBYuXHi1V69et4x2WrlypSo8PHxQVFRU38cee6zUxcXl9tFQd+G5554rS0lJ6frAAw8EAFLPyObNm3Ps7e2bbCbNmTOn6PXXX7/n3Xff1cfHx6cZdwubbvv6669zp0+fPkSn0yEkJKTynXfeuW6a3/vvv58fGhoa6OXlVRsYGFhZXl5+V4+6PP/88yXz5s0bHBMTY2gVt1XeliZa22w1h6SkpNyQkJCilvfsGGJiYlxXrFjR59ixYzwEnTEL0Wg0MmdnZ71MJsPatWt7bN++veePP/6YZely2aKkpKReISEhPpYuh7FO39JijFmXkydPOr355pveRAQ3NzdddHR0rqXLxNoPt7QYY4w1qiO2tDr9QAzGGGPWw9JBS99wE5YxxljHUX9tbnRYvSVZOmglX79+3Z0DF2OMdRz162m5A2j1M3LtxaIDMbRa7YLCwkJFYWEhr1zMGGMdh2HlYksXxJRFB2Iwxhhjd4JbN4wxxqwGBy3GGGNWg4MWY4wxq8FBizHGmNXgoMUYY8xq/D8yXDV9VrIr2QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x432 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Test Model\n",
      "----------\n",
      "Run: 0, Mean assignment to arrival: 15.6, Mean call to arrival: 19.6, Demand met: 1.000\n",
      "Run: 1, Mean assignment to arrival: 15.6, Mean call to arrival: 18.6, Demand met: 1.000\n",
      "Run: 2, Mean assignment to arrival: 15.4, Mean call to arrival: 18.3, Demand met: 1.000\n",
      "Run: 3, Mean assignment to arrival: 15.3, Mean call to arrival: 18.4, Demand met: 1.000\n",
      "Run: 4, Mean assignment to arrival: 15.6, Mean call to arrival: 18.7, Demand met: 1.000\n",
      "Run: 5, Mean assignment to arrival: 15.4, Mean call to arrival: 18.3, Demand met: 1.000\n",
      "Run: 6, Mean assignment to arrival: 15.4, Mean call to arrival: 18.4, Demand met: 1.000\n",
      "Run: 7, Mean assignment to arrival: 15.6, Mean call to arrival: 18.7, Demand met: 1.000\n",
      "Run: 8, Mean assignment to arrival: 15.6, Mean call to arrival: 19.1, Demand met: 1.000\n",
      "Run: 9, Mean assignment to arrival: 15.5, Mean call to arrival: 18.3, Demand met: 1.000\n",
      "Run: 10, Mean assignment to arrival: 15.5, Mean call to arrival: 18.6, Demand met: 1.000\n",
      "Run: 11, Mean assignment to arrival: 15.5, Mean call to arrival: 18.4, Demand met: 1.000\n",
      "Run: 12, Mean assignment to arrival: 15.4, Mean call to arrival: 18.6, Demand met: 1.000\n",
      "Run: 13, Mean assignment to arrival: 15.6, Mean call to arrival: 18.9, Demand met: 1.000\n",
      "Run: 14, Mean assignment to arrival: 15.4, Mean call to arrival: 18.4, Demand met: 1.000\n",
      "Run: 15, Mean assignment to arrival: 15.5, Mean call to arrival: 18.6, Demand met: 1.000\n",
      "Run: 16, Mean assignment to arrival: 15.6, Mean call to arrival: 19.2, Demand met: 1.000\n",
      "Run: 17, Mean assignment to arrival: 15.4, Mean call to arrival: 18.4, Demand met: 1.000\n",
      "Run: 18, Mean assignment to arrival: 15.7, Mean call to arrival: 19.4, Demand met: 1.000\n",
      "Run: 19, Mean assignment to arrival: 15.4, Mean call to arrival: 18.4, Demand met: 1.000\n",
      "Run: 20, Mean assignment to arrival: 15.6, Mean call to arrival: 18.6, Demand met: 1.000\n",
      "Run: 21, Mean assignment to arrival: 15.6, Mean call to arrival: 18.5, Demand met: 1.000\n",
      "Run: 22, Mean assignment to arrival: 15.5, Mean call to arrival: 18.7, Demand met: 1.000\n",
      "Run: 23, Mean assignment to arrival: 15.5, Mean call to arrival: 18.3, Demand met: 1.000\n",
      "Run: 24, Mean assignment to arrival: 15.6, Mean call to arrival: 19.0, Demand met: 1.000\n",
      "Run: 25, Mean assignment to arrival: 15.6, Mean call to arrival: 18.9, Demand met: 1.000\n",
      "Run: 26, Mean assignment to arrival: 15.5, Mean call to arrival: 18.3, Demand met: 1.000\n",
      "Run: 27, Mean assignment to arrival: 15.6, Mean call to arrival: 18.8, Demand met: 1.000\n",
      "Run: 28, Mean assignment to arrival: 15.4, Mean call to arrival: 18.3, Demand met: 1.000\n",
      "Run: 29, Mean assignment to arrival: 15.4, Mean call to arrival: 18.5, Demand met: 1.000\n",
      "\n",
      "       call_to_arrival  assign_to_arrival  demand_met\n",
      "count        30.000000          30.000000        30.0\n",
      "mean         18.635021          15.515405         1.0\n",
      "std           0.335577           0.090776         0.0\n",
      "min          18.256320          15.323324         1.0\n",
      "25%          18.363393          15.446185         1.0\n",
      "50%          18.564526          15.511076         1.0\n",
      "75%          18.743197          15.581536         1.0\n",
      "max          19.556426          15.699036         1.0\n"
     ]
    }
   ],
   "source": [
    "######################## MODEL ENTRY POINT #####################################\n",
    "\n",
    "# Run model and return last run results\n",
    "last_run = qambo()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}