Lesson13.5Xavier方法和Kaiming方法.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lesson 13.5 Xavier方法与kaiming方法（HE初始化）\n",
    "\n",
    "我们介绍**参数初始化优化方法**，也就是针对tanh和Sigmoid激活函数的Xavier方法，以及针对ReLU激活函数的Kaiming方法（HE初始化）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一、Xavier方法\n",
    "\n",
    "### 1.Xavier初始化参数方法基础理论\n",
    "\n",
    "&emsp;&emsp;回顾Glorot条件，我们要求正向传播时候数据流经每一层前后的方差一致，并且反向传播时候数据流经每一层，该层梯度前后方差一致。我们将前者称为向前传播条件，后者称为反向传播条件。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "我们先看当向前传播条件成立时候，有如下计算过程。首先，数据流至某一层时，该层神经元接收到的数据可以通过如下方法算得：\n",
    "\n",
    "$$z_j = \\sum^n_{i=1}w_ix_i$$\n",
    "\n",
    "- 其中$z_j$表示当前某层神经元接收到的数据，\n",
    "- $x_i$表示上一层某神经元传出的数据，\n",
    "- $w_i$代表连接对应两个神经元之间的权重\n",
    "\n",
    "\n",
    "如果我们假设w是服从正态分布的，则w服从$(0, \\sqrt{\\frac{2}{n_{in}+n_{out}}})$的随机变量。据此，我们就能设置每一层的初始参数了。当然，在PyTorch中，可以使用init方法进行初始化。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.Xavier初始化参数执行方法\n",
    "\n",
    "#### 2.1 PyTorch实现Xavier均匀分布的参数创建\n",
    "\n",
    "- `torch.nn.init.xavier_uniform_(参数)`进行初始化参数设置。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;31mSignature:\u001b[0m \u001b[0mnn\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0minit\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mxavier_uniform_\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtensor\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mTensor\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mgain\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mfloat\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;36m1.0\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m->\u001b[0m \u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mTensor\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mDocstring:\u001b[0m\n",
      "Fills the input `Tensor` with values according to the method\n",
      "described in `Understanding the difficulty of training deep feedforward\n",
      "neural networks` - Glorot, X. & Bengio, Y. (2010), using a uniform\n",
      "distribution. The resulting tensor will have values sampled from\n",
      ":math:`\\mathcal{U}(-a, a)` where\n",
      "\n",
      ".. math::\n",
      "    a = \\text{gain} \\times \\sqrt{\\frac{6}{\\text{fan\\_in} + \\text{fan\\_out}}}\n",
      "\n",
      "Also known as Glorot initialization.\n",
      "\n",
      "Args:\n",
      "    tensor: an n-dimensional `torch.Tensor`\n",
      "    gain: an optional scaling factor\n",
      "\n",
      "Examples:\n",
      "    >>> w = torch.empty(3, 5)\n",
      "    >>> nn.init.xavier_uniform_(w, gain=nn.init.calculate_gain('relu'))\n",
      "\u001b[1;31mFile:\u001b[0m      c:\\users\\hp\\python\\python310\\lib\\site-packages\\torch\\nn\\init.py\n",
      "\u001b[1;31mType:\u001b[0m      function\n"
     ]
    }
   ],
   "source": [
    "nn.init.xavier_uniform_?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$fan_{in}$由张量的列决定，$fan_{out}$由张量的行数决定\n",
    "\n",
    "随机数的范围在$[-a,a]$之间，其中a的表达式为：\n",
    "$\\text{gain} \\times \\sqrt{\\frac{6}{\\text{fan\\_in} + \\text{fan\\_out}}}$\n",
    "\n",
    "其中gain为增益系数，用于手动调整均匀分布的边界，暂时不建议调整。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 1., 2., 3.],\n",
       "        [4., 5., 6., 7.]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 创建张量\n",
    "t = torch.arange(8).reshape(2, 4).float()\n",
    "t\n",
    "\n",
    "# fan_in 张量的列，有存在转置关系，此处为4列\n",
    "# fan_out 2行"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.5770,  0.3678,  0.2310,  0.3201],\n",
       "        [-0.7702, -0.1159, -0.2467, -0.5198]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 实现均匀分布的数值转换\n",
    "nn.init.xavier_uniform_(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.5770,  0.3678,  0.2310,  0.3201],\n",
       "        [-0.7702, -0.1159, -0.2467, -0.5198]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t  # 该函数会在原对象基础上直接进行修改"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Sigmoid激活函数\n",
    "\n",
    "&emsp;&emsp;接下来，在建模过程中使用Xavier初始化方法测试效果。首先是数据准备与核心参数定义。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x2d53b60efd0>"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#————————————————————————————用Xavier初始化方法测试效果————————————————————————————\n",
    "# 设置随机数种子\n",
    "torch.manual_seed(420)  \n",
    "\n",
    "# 创建最高项为2的多项式回归数据集\n",
    "features, labels = tensorGenReg(w=[2, -1], bias=False, deg=2)\n",
    "\n",
    "# 进行数据集切分与加载\n",
    "train_loader, test_loader = split_loader(features, labels)\n",
    "\n",
    "# 初始核心参数\n",
    "lr = 0.03\n",
    "num_epochs = 20\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x2d53b60efd0>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[ 0.7995, -0.2561],\n",
       "        [-0.1097, -0.0529],\n",
       "        [-0.8137,  0.8860],\n",
       "        [-0.0984,  0.9770]], requires_grad=True)"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[ 0.7130,  0.3764, -0.0196, -0.0096],\n",
       "        [ 0.0199, -0.8417, -0.2067, -0.4007],\n",
       "        [-0.0515,  0.1059, -0.6787,  0.6582],\n",
       "        [ 0.7072,  0.5386,  0.3889,  0.1370]], requires_grad=True)"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[-0.6241,  0.3488,  0.3069,  0.8371],\n",
       "        [-0.8167,  0.7516,  0.6801,  0.1606],\n",
       "        [ 0.2885,  0.1036, -0.0692,  0.1744],\n",
       "        [ 0.5015,  0.0987, -0.0787, -0.0286]], requires_grad=True)"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[-0.8437, -0.5244,  0.8917,  0.0646]], requires_grad=True)"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设置随机数种子\n",
    "torch.manual_seed(420)  \n",
    "\n",
    "# ——————————实例化模型——————————\n",
    "# =====保留原参数=====\n",
    "sigmoid_model3 = Sigmoid_class3()       \n",
    "\n",
    "# =====使用Xavier初始化参数=====\n",
    "sigmoid_model3_init = Sigmoid_class3()              \n",
    "# 修改init模型初始参数：修改所有线性层的参数\n",
    "for m in sigmoid_model3_init.modules():\n",
    "    if isinstance(m, nn.Linear):\n",
    "        nn.init.xavier_uniform_(m.weight)   # 对每一个权重初始化，截距也可以初始化，但是没必要\n",
    "        \n",
    "# 创建模型容器\n",
    "model_l = [sigmoid_model3, sigmoid_model3_init]           \n",
    "name_l = ['sigmoid_model3', 'sigmoid_model3_init']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "#————————————————————————————模型训练————————————————————————————\n",
    "train_l, test_l = model_comparison(model_l = model_l, \n",
    "                                   name_l = name_l, \n",
    "                                   train_data = train_loader,\n",
    "                                   test_data = test_loader,\n",
    "                                   num_epochs = 2, \n",
    "                                   criterion = nn.MSELoss(), \n",
    "                                   optimizer = optim.SGD, \n",
    "                                   lr = lr, \n",
    "                                   cla = False, \n",
    "                                   eva = mse_cal)\n",
    "                                   \n",
    "# num_epochs = 2 实际只遍历了1次数据，等于1时没有遍历"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEXCAYAAACu1P9TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAXH0lEQVR4nO3de5CddZ3n8fcnnSshY7goIAGCBnHQFcQedFYUlUuhqHiJtehOGbxUdlgxa+16m2F3FZ0pnZq1dicloxOdEWfwTi0jQxADo8ICogYlq4g6IeJKcCQoUUkCSbq/+0efbg6hO53kdHL6nOf9qjp1nsvvPM+3n3Q+59e/5znPSVUhSep/M7pdgCTpwDDwJakhDHxJaggDX5IawsCXpIYw8CWpIQx8aT9L8v4kV3S7DsnAl6SGMPClvZBkZrdrkPaVgS8BSU5N8r0kv0vypSRfSPJnSV6U5N4k70nyr8CnkhyS5Jokm5I82Jpe1Lat45Pc2NrW9cDh3fvJpEcZ+Gq8JLOBq4DLgUOBzwGvbmtyZGv5ccByRv7ffKo1fyywDfhoW/vPArczEvQfBJbt1x9A2kPxXjpquiQvZCTkF1XrP0SSm4FvADcAa4Dfq6qHJ3j9KcDXq+qQJMcCG4AnVNWW1vrPAsNV9Uf7+2eRdscevgRPBjbWY3s/P2+b3tQe9kkOSvI3SX6W5LfATcDCJAOtbT04GvYtP9ufxUt7ysCX4BfA0UnStuyYtuld/wz+L8CJwHOr6veAF7aWp7WtQ5LMb2t/7BTXK+0TA1+CbwJDwMVJZiY5HzhtN+0XMDJuvznJocD7RldU1c+AtcClSWYnOR14xf4rXdpzBr4ar6q2A68B3gJsBv4IuAZ4ZIKX/C9gHvAAcBtw3S7r3wA8F/g1I28Gfz/VNUv7wpO20jiSfAv4eFV9qtu1SFPFHr4EJDkjyZGtIZ1lwLN4fM9d6ml+alAacSLwRWA+I5dVLq2qX3S3JGlqOaQjSQ3hkI4kNcS0HtI5/PDDa/Hixd0uQ5J6xu233/5AVT1xvHXTOvAXL17M2rVru12GJPWMJBN+stshHUlqCANfkhrCwJekhjDwJakhDHxJaggDX5IawsCXpIYw8CWpIQx8qU/dcsstXHjhhWzdurXbpWiaMPClPrVq1So2bNjAAw880O1SNE0Y+FKf2r59OwDDw8NdrkTThYEv9anR72QfDX7JwJf63COPTPTVvGoaA1/qU6NfbmQPX6MMfKnPeZWORhn4Up/btm1bt0vQNGHgS31uy5Yt3S5B04SBL/Wp4Rq5HNPA1ygDX+pTO3bsAAx8PcrAl/rU6MlaA1+jDHypD1UVD297GPCkrR5l4Et9aPv27WO3VPCyTI0y8DVt3HfffbzjHf+JW2+9tdul9Lz2T9f6SVuNmpLAT3Jukh8nWZ/kveOsn5PkC63130qyeCr2q/7xy1/+kkvf/z6++93v8cEPfIB169Z1u6Se9rGPfWxs+s4772TlypVdrEbTxcxON5BkALgMOBu4F/hOkqur6odtzd4CPFhVS5JcAPwF8O863Xc3rVy5knXr1vHQQw9x8MEHc/LJJ7NixYpul9VzHn74Ya644go+c8UVDA0PM29gmO0Pb+Xtb387L3nJS7jooos44ogjul1mz/npT386Nr1lyxbWr1/fxWo0XXQc+MBpwPqq2gCQ5PPA+UB74J8PvL81fSXw0SSp0Zt9TIGqYnh4mOHhYYaGhsaeh4aGqKqxZbu2GW9+19cPDQ2xc+fOsceOHTu48cYbeeihhzjvvPNYvXo1Dz74IMcddxyzZ89m9uzZDAwMMHPmzLHHjBkzGBgYGHsenW5fPt5j13bt25gxo7dH5B555BH+40V/zPq7N7Bg1jC/G57BtqGRn+mwuUPcfOPX+fa3buOvP/ZxFi9e3N1ipT4wFYF/NPDztvl7gedO1Kaqdib5DXAY8LhvZkiyHFgOcOyxx0640/vuu48LLrigo8I7tXTp0rFe/ZVXXslHPvKRrtazZs0a5s6d29Ua9sbdd9/N+rs38NqnbGPt/bPYOXP+2BvowQO/46Jn/JY/u7249dZbDXxpCkxF4E+pqloFrAIYHByc8C+ABQsW8JSnPIUNGzYcsNp2tXr16sc8d9MLXvACZs6cdv+cu/W0pz2N4xcfxw0bf8YshjjvFeeNvYHeeM0X+Kd75jJ3zmzOOOOMLlfae0bvhS+1m4qE2Agc0za/qLVsvDb3JpkJPAH4VSc7XbBgAZdffnknm3iczZs3s379enbs2MH27dvZsWPH2PTOnTvZvn372OO6665j06ZNXHnllQAcdthhnHnmmcydO3dsWGfWrFljj9H5OXPmMGvWLJ7+9Kdz0EEHTWn9vWbmzJm8813v5m1vexuHzXnsG+ihM4p1v5rFW9/6Ro4++uguV9p7DHyNZyoC/zvACUmOZyTYLwDesEubq4FlwDeBpcDXpnL8fqosXLiQwcHBPWq7bdu2x5wIW7JkCRdffPH+Kq1vnXTSSRw8/yC2b3+IbY9sG3sDHZo3EljPe97zullezzLwNZ6OA781Jn8x8FVgAPi7qrozyQeAtVV1NfC3wD8kWQ/8mpE3hZ7mFTlTY2BggFe9+jVcccUVj1l+/8MDPOOk3+eEE07oUmW9rddP6Gv/mJJB36q6Frh2l2X/vW36YeB1U7Ev9Z9ly5ZxzTXXsHnzZuYOFAF2ZCaX/Nf/Zk91H3ncNB67Aeq6OXPmcNlll5GEArYNhXe9690sWrSo26X1rC1btjBv3jyWLl3KvHnzvIGaAANf08QxxxzD2WefzSND4dBDFnLOOed0u6SeNvoZkRUrVnDeeefx0EMPdbskTQO9dR2f+toZZ5zBmjVreOqSExgYGOh2OT3t4IMPfsxVT/61JDDwNY2ceuqpPP/5z2fZsmXdLqXnzZ8/n23bHr3qaf78+V2uSNOBga9pY/78+XzoQx/qdhlS33IMX5IawsCXpIYw8CWpIQx8SWoIA1+SGsLAl6SGMPAlqSEMfElqCANfkhrCwJekhjDwJakhDHypDy1ZsmRs+tBDD33MvJrLwJf60IoVK8a+9er000/3KzkFGPhS3xoN/JkzvSmuRhj4Up/KjJHAnz17dpcr0XRh4Et9akZG/nvPmjWry5VoujDwpT41OqQzZ86cLlei6cLAl/qcQzoaZeBLfW7u3LndLkHThIEv9Tl7+BrVUeAnOTTJ9Un+pfV8yATthpLc0Xpc3ck+Je0dx/A1qtMe/nuBf66qE4B/bs2PZ1tVndJ6vLLDfUraA6NX59jD16hOA/984NOt6U8Dr+pwe5KmyMKFCwEDX4/qNPCPqKpftKb/FThignZzk6xNcluSV+1ug0mWt9qu3bRpU4flSfKTtho16W9CkhuAI8dZdUn7TFVVkppgM8dV1cYkTwG+luT7VXX3eA2rahWwCmBwcHCi7UmaxOh1+PbwNWrSwK+qsyZal+SXSY6qql8kOQq4f4JtbGw9b0jyDeDZwLiBL2lqVI30l+zha1SnQzpXA8ta08uAL+/aIMkhSea0pg8Hng/8sMP9SpqEN0/TrjoN/A8DZyf5F+Cs1jxJBpN8stXm94G1SdYBXwc+XFUGvnSADAwMdLsETRMdvfVX1a+AM8dZvhZ4a2v6VuDfdLIfSfvOwNcoP2kr9bnRoR3JwJf6nIGvUQa+1OdGr9aRDHypT40GvT18jTLwJakhDHxJaggDX+pTT33qUwEvy9Sj/Aie1KdWrFjBs5/9bI466qhul6JpwsCX+tSTnvQkXvva13a7DE0jDulIUkMY+JLUEAa+JDWEgS9JDWHgS1JDGPiS1BAGviQ1hIEvSQ1h4EtSQxj4ktQQBr4kNYSBL0kNYeBLUkMY+JLUEAa+JDWEgS9JDdFR4Cd5XZI7kwwnGdxNu3OT/DjJ+iTv7WSfkqR902kP/wfAa4CbJmqQZAC4DHgpcBLw+iQndbhfSdJe6ugrDqvqLoAku2t2GrC+qja02n4eOB/4YSf7liTtnQMxhn808PO2+Xtby8aVZHmStUnWbtq0ab8XJ0lNMWkPP8kNwJHjrLqkqr481QVV1SpgFcDg4GBN9fYlqakmDfyqOqvDfWwEjmmbX9RaJkk6gA7EkM53gBOSHJ9kNnABcPUB2K8kqU2nl2W+Osm9wB8Cq5N8tbX8yUmuBaiqncDFwFeBu4AvVtWdnZUtSdpbnV6lcxVw1TjL7wNe1jZ/LXBtJ/uSJHXGT9pKUkMY+JK0B6qKqt6+cNDAl6Q98MEPfpD3vuc93S6jIx2N4UtSU9xwww3dLqFj9vAlqSEMfElqCANfkhrCwJekhjDwJakhDHxJaggDX5IawsCXpIYw8CWpIQx8SWoIA1+SGsLAl6SGMPAlqSEMfElqCANfkhrCwJekhjDwJakhDHxJaggDX5IawsCXpIboKPCTvC7JnUmGkwzupt09Sb6f5I4kazvZpyRp38zs8PU/AF4D/M0etH1xVT3Q4f4kSfuoo8CvqrsAkkxNNZKk/eZAjeEXsCbJ7UmWH6B9SpLaTNrDT3IDcOQ4qy6pqi/v4X5Or6qNSZ4EXJ/kR1V10wT7Ww4sBzj22GP3cPOSpMlMGvhVdVanO6mqja3n+5NcBZwGjBv4VbUKWAUwODhYne5bkjRivw/pJJmfZMHoNHAOIyd7JannVPVuP7TTyzJfneRe4A+B1Um+2lr+5CTXtpodAdycZB3wbWB1VV3XyX4lqVuGhoa6XcI+6/QqnauAq8ZZfh/wstb0BuDkTvYjSdNFY3v4ktQ0w8PD3S5hnxn4krQXDHxJ6mPtwzgO6UhSH2sPeXv4ktTH2kPeHr4k9bH2wLeHL0l9zDF8SWqI9l59L3/wysCXpEnYw5ekhjDwJakhPGkrSQ1hD1+SGsIeviQ1xCc+8Ymx6UsvvZSVK1d2sZp919HtkSWpCTZs2DA2fddddzFnzpwuVrPv7OFLUkMY+JI0iV4+UdvOwJekhjDwJakhDHxJmoRDOpKknmLgS1JDGPiS1BAGviRNYuvWrcybN4+lS5cyb948tmzZ0u2S9omftJWkSWzdupXzzjuPFStWAHDLLbd0uaJ901HgJ/lL4BXAduBu4E1VtXmcducCfwUMAJ+sqg93sl9JOpAOOuggVq9eDcDq1atZtGhRlyvaN50O6VwPPLOqngX8BPiTXRskGQAuA14KnAS8PslJHe5Xkg6Ygw46iG3btnHllVeybds25s+f3+2S9klHgV9Va6pqZ2v2NmC8t73TgPVVtaGqtgOfB87vZL+SpL03lSdt3wx8ZZzlRwM/b5u/t7VsXEmWJ1mbZO2mTZumsDxJarZJx/CT3AAcOc6qS6rqy602lwA7gc90WlBVrQJWAQwODvbHx9skaRqYNPCr6qzdrU9yIfBy4Mwa//PHG4Fj2uYXtZZJkg6gjoZ0WlffvBt4ZVVtnaDZd4ATkhyfZDZwAXB1J/uVJO29TsfwPwosAK5PckeSjwMkeXKSawFaJ3UvBr4K3AV8saru7HC/kqS91NF1+FW1ZILl9wEva5u/Fri2k31JkjrjrRUkaRKLFy8emz7xxBNZsmTcvu60560VJGkSb37zm/nKV0auOv/TP/1Tjj/++C5XtG/s4UvSJJKMO91rDHxJmsSMGTPGne41vVu5JB0g7b16A1+S+pg9fElqCANfkhqiPeQ9aStJfaw98AcGBrpYSWcMfEmahEM6ktQQBr4kNUT7MI5DOpLUxxzDl6SG8CodSWogx/AlqSEc0pGkhjDwJakhHMOXJE17Br4kNYSBL0kNYeBLUkMY+JLUEAa+JDWEgS9JDTGzkxcn+UvgFcB24G7gTVW1eZx29wC/A4aAnVU12Ml+JUl7r9Me/vXAM6vqWcBPgD/ZTdsXV9Uphr0kdUdHgV9Va6pqZ2v2NmBR5yVJkvaHqRzDfzPwlQnWFbAmye1Jlu9uI0mWJ1mbZO2mTZumsDxJarZJx/CT3AAcOc6qS6rqy602lwA7gc9MsJnTq2pjkicB1yf5UVXdNF7DqloFrAIYHBysPfgZJEl7YNLAr6qzdrc+yYXAy4Ezq2rcgK6qja3n+5NcBZwGjBv4kqT9o6MhnSTnAu8GXllVWydoMz/JgtFp4BzgB53sV5K09zodw/8osICRYZo7knwcIMmTk1zbanMEcHOSdcC3gdVVdV2H+5Uk7aWOrsOvqiUTLL8PeFlregNwcif7kSR1zk/aSlJDGPiS1BAGviQ1hIEvSQ1h4EtSQ3R0lY4kNcXixYvZumVLt8voiIEvSXtg5cqV7Ny5c/KG05iBL0l7YOHChd0uoWOO4UtSQxj4ktQQBr4kNYSBL0kNYeBLUkMY+JLUEAa+JDVEJvhWwmkhySbgZ92uYxKHAw90u4g+4vGcWh7PqdULx/O4qnrieCumdeD3giRrq2qw23X0C4/n1PJ4Tq1eP54O6UhSQxj4ktQQBn7nVnW7gD7j8ZxaHs+p1dPH0zF8SWoIe/iS1BAGviQ1hIG/h5L8XZL7k/xggvVJsjLJ+iT/N8mpB7rGXpFkbpJvJ1mX5M4kl47TZk6SL7SO57eSLO5CqT0jyT1Jvp/kjiRrx1nv7+ceSHJi6xiOPn6b5B27tOnZY2ng77nLgXN3s/6lwAmtx3LgYwegpl71CPCSqjoZOAU4N8nzdmnzFuDBqloC/E/gLw5siT3pxVV1ygTXifv7uQeq6setY3gK8BxgK3DVLs169lga+Huoqm4Cfr2bJucDf18jbgMWJjnqwFTXW1rH6KHW7KzWY9erB84HPt2avhI4M0kOUIn9yN/PvXcmcHdV7fpp/549lgb+1Dka+Hnb/L2tZRpHkoEkdwD3A9dX1bd2aTJ2PKtqJ/Ab4LADWmRvKWBNktuTLB9nvb+fe+8C4HPjLO/ZY2ngqyuqaqj1Z/Mi4LQkz+xySb3u9Ko6lZHhhrcleWG3C+plSWYDrwS+1O1appKBP3U2Ase0zS9qLdNuVNVm4Os8/vzI2PFMMhN4AvCrA1pcD6mqja3n+xkZcz5tlyb+fu6dlwLfrapfjrOuZ4+lgT91rgbe2DqD/zzgN1X1i24XNR0leWKSha3pecDZwI92aXY1sKw1vRT4WvkpwXElmZ9kweg0cA6w69Vk/n7undcz/nAO9PCxnNntAnpFks8BLwIOT3Iv8D5GTjZSVR8HrgVeBqxn5Mz+m7pTaU84Cvh0kgFGOh1frKprknwAWFtVVwN/C/xDkvWMnCy/oHvlTntHAFe1zmnPBD5bVdcl+WPw93Nvtd40zwb+Q9uyvjiW3lpBkhrCIR1JaggDX5IawsCXpIYw8CWpIQx8SWoIA1+SGsLAl/ZQkm8kedydKJMMJlk5wWvuSXL4OMvfn+Sd+6NOaSJ+8ErqUFWtBR53D3ppurGHr56WZHGSu5J8ovVlKmuSzGvvjSc5PMk9rekLk/xjkutbve+Lk/znJN9LcluSQyfZ5etaX97ykyQvaG3zRUmuaU0f1qrhziSfBMZu6ZzkktbrbgZObFv+1CTXte50+X+SPL21/PLWF23cmmRDkqVTeezUPAa++sEJwGVV9QxgM/DaSdo/E3gN8AfAnwNbq+rZwDeBN07y2plVdRrwDkZur7Gr9wE3t2q5CjgWIMlzGLk9xCmMfCz/D9peswp4e1U9B3gn8Ndt644CTgdeDnx4ktqk3XJIR/3gp1V1R2v6dmDxJO2/XlW/A36X5DfAP7WWfx941iSv/d+T7OeFjLyZUFWrkzzYWv4C4Kqq2gqQ5OrW88HAvwW+1Pb9LnPatvePVTUM/DDJEZPUJu2Wga9+8Ejb9BAwD9jJo3/Bzt1N++G2+WEm/z8x2nZoD9ruiRnA5tZ3A+xuf9A2PCTtC4d01K/uYeQ7SWHk9soHyk3AGwCSvBQ4pG35q1rnFxYArwCoqt8CP03yutZrkuTkA1ivGsTAV7/6H8BFSb4HPO6yyP3oUuCFSe5kZGjn/wFU1XeBLwDrgK8A32l7zb8H3pJkHXAnI9+ZKk05b48sSQ1hD1+SGsKTttIuklwGPH+XxX9VVZ/qRj3SVHFIR5IawiEdSWoIA1+SGsLAl6SGMPAlqSH+P3cgtpQEbrH1AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "weights_vp(sigmoid_model3, att=\"grad\")\n",
    "\n",
    "# 第一层出现了明显的梯度消失"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEXCAYAAACgUUN5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAcCUlEQVR4nO3dfXRddZ3v8fcnj00LhUID1KYlxXJ1rNKKmeIdFXEULNMZiiOORbzWWbhyr1J7EXQNLNYCxXGJXtQxIyP0OuVBLw/KAqZDixQfuCCPDUiBMlMIyEN7S1uB8tC0TZN87x9nn3h6moc2ZycnJ/vzWuus7P3be5/z7V7p+eT320+KCMzMLLuqyl2AmZmVl4PAzCzjHARmZhnnIDAzyzgHgZlZxjkIzMwyzkFgViaSvi7pZ+Wuw8xBYGaWcQ4CsxRIqil3DWbD5SAwG4Sk4yX9XtKbkn4h6SZJ/yjpJEkbJf2DpJeBqyVNkXS7pG2SXkummwrea5ak/5u8113A1PL9y8z+xEFgNgBJdcCtwDXAYcANwCcKVjkqaT8aaCX3/+nqZH4msBP4UcH61wOPkAuAbwJLRvQfYLaf5HsNmfVP0onkvvybIvmPIul3wN3Ar4A1wOSI2DXA9vOA30bEFEkzgeeAQyJiR7L8eqA3Ij470v8Ws8G4R2A2sLcBm2Lvv5ZeKpjeVhgCkiZKukrSC5LeAO4BDpVUnbzXa/kQSLwwksWb7S8HgdnANgPTJamgbUbBdHF3+nzgHcAJETEZODFpV/JeUyRNKlh/Zsr1mg2Lg8BsYA8APcBSSTWSFgHzB1n/YHLHBbZLOgy4JL8gIl4A2oFvSKqT9EHgb0audLP95yAwG0BEdAF/C5wNbAc+C9wO7B5gk38CGoA/Ag8Cvyxa/hngBOBVciFxXdo1mw2HDxabHQBJDwFXRsTV5a7FLC3uEZgNQtKHJR2VDA0tAY5j37/0zSqar4Y0G9w7gJ8Dk8id/nlGRGwub0lm6fLQkJlZxqUyNCRphaStkp4cYPlJkl6X9Fjyurhg2QJJGyR1SLogjXrMzGz/pdIjSK7AfAu4LiLe3c/yk4CvRsRfF7VXA08DJwMbgbXAmRHx1GCfN3Xq1Ghubi65bjOzLHnkkUf+GBGNxe2pHCOIiHskNQ9j0/lAR0Q8ByDpRmARMGgQNDc3097ePoyPMzPLLkn9Xs0+mmcN/VdJ6yTdIWlO0jadvS/Z35i07UNSq6R2Se3btm0b6VrNzDJjtILgUeDoiJgL/DNw24G+QUQsj4iWiGhpbNynZ2NmZsM0KkEQEW9ExFvJ9GqgVtJUYBN737ulKWkzM7NRMipBkFyQo2R6fvK5r5A7OHxs8sCOOmAxsHI0ajIzs5xUDhZLugE4CZgqaSO5+6jUAkTElcAZwBcldZO7Kdfi5Na+3ZKWAncC1cCKiFifRk1mZrZ/KvKCspaWlvBZQ2ZmB0bSIxHRUtzuew2ZmWWcg8DMLOMcBGZmJbj77rtpbW2lu7u73KUMm+8+amZWgu9///ts376dnTt3cvDBB5e7nGFxj8DMrAS7dw/0wLrK4SAwMytB/szLSjwDM89BYGZWguRaWXp7e8tcyfA5CMzMUuAegZlZxrlHYGaWce4RmJllnHsEZmYZ5x6BmVnGuUdgZpZx7hGYmWVcT09PuUsYNgeBmVkK3CMwM8s4HyMwM8u4zA8NSVohaaukJwdYfpakxyU9Iel+SXMLlj2ftD8myc+fNLOK5KEhuAZYMMjyPwAfjoj3AN8Elhct/0hEzOvvWZpmZpWgknsEqTyYJiLukdQ8yPL7C2YfBJrS+Fwzs7HCxwgOzNnAHQXzAayR9Iik1oE2ktQqqV1S+7Zt20a8SDOzA1HJQTCqj6qU9BFyQfDBguYPRsQmSUcAd0n6z4i4p3jbiFhOMqTU0tJSuYNxZjYuVXIQjFqPQNJxwE+ARRHxSr49IjYlP7cCtwLzR6smM7NS5Q8SOwiGIGkmcAvw3yLi6YL2SZIOzk8DpwD9nnlkZjYW5Z9QVslnDaUyNCTpBuAkYKqkjcAlQC1ARFwJXAwcDvxLstO6kzOEjgRuTdpqgOsj4pdp1GRmNpp81lDEmUMs/wLwhX7anwPm7ruFmVllqeQega8sNjNLgY8RmJllnIPAzCzjPDRkZpZRXV1dgHsEZmaZFUkAuEdgZpZxDgIzs4zKf/17aMjMLOPcIzAzyzgHgZlZxnloyMzMKpaDwMwsBe4RmJllnI8RmJllnHsEZmYZ5x6BmVnGuUdgZpZxDgIzs4zLfBBIWiFpq6R+HzyvnDZJHZIel3R8wbIlkp5JXkvSqMfMbLRlPgiAa4AFgyw/FTg2ebUCPwaQdBi5B92fAMwHLpE0JaWazMxGTSU/vD6VIIiIe4BXB1llEXBd5DwIHCppGvBx4K6IeDUiXgPuYvBAMTMbk9wjGNp04KWC+Y1J20DtZmYVJfM9gtEgqVVSu6T2bdu2lbscMzPa2tr6rh+47bbbaGtrK3NFwzNaQbAJmFEw35S0DdS+j4hYHhEtEdHS2Ng4YoWame2vjo6OvunNmzfvNV9JRisIVgKfS84eej/wekRsBu4ETpE0JTlIfErSZmZmo6QmjTeRdANwEjBV0kZyZwLVAkTElcBq4K+ADqAT+Ptk2auSvgmsTd7q0ogY7KCzmZmlLJUgiIgzh1gewDkDLFsBrEijDjMzO3AVc7DYzMxGhoPAzGyYKvmOo4UcBGZmw+QgMDPLOAeBmZmNCw4CM7Nhco/AzCzjHARmZhnX2dlJQ0MDZ5xxBg0NDezYsaPcJQ1LKheUmZll0Y4dO1i4cCHLli0D4L777itzRcPjIDAzG6aJEyeyatUqAFatWkVTU1OZKxoeB4GZ2TBNnDiRnTt3cvPNNwMwadKkMlc0PD5GYGaWcQ4CM7OMcxCYmWWcg8DMbJgklbuEVDgIzMwyzkFgZjZMs2bN6pueMmUKs2fPLmM1w+cgMDMbptbW1r7pE044oe/CskqTShBIWiBpg6QOSRf0s/wHkh5LXk9L2l6wrKdg2co06jEzGw3V1dV901VVlft3dckXlEmqBq4ATgY2AmslrYyIp/LrRMRXCtb/MvDegrfYGRHzSq3DzGy0FQZB4XSlSSPC5gMdEfFcRHQBNwKLBln/TOCGFD7XzKysCr/8a2oq90YNaQTBdOClgvmNSds+JB0NzAJ+U9A8QVK7pAclnZ5CPWZmo6Lwy7+Sg2C0K18M3BwRPQVtR0fEJknHAL+R9EREPFu8oaRWoBVg5syZo1OtmdkgqqurkUREVHQQpNEj2ATMKJhvStr6s5iiYaGI2JT8fA64m72PHxSutzwiWiKipbGxsdSazcxSUZMMD2U9CNYCx0qaJamO3Jf9Pmf/SHonMAV4oKBtiqT6ZHoq8AHgqeJtzczGqupxEAQlVx4R3ZKWAncC1cCKiFgv6VKgPSLyobAYuDH2frbbnwFXSeolF0qXFZ5tZGZWKWpra8tdwrClEmERsRpYXdR2cdH81/vZ7n7gPWnUYGZWDvm/bCs5CCr3CggzszGkkoeGHARmZiXI34HUPQIzs4xzEJiZZZyDwMws43yMwMwso3yMwMws4/KXRjkIzMwyzkNDZmYZlR8achCYmWWcg8DMLOOy/oQyM7PMc4/AzCzj3CMwM8s4B4GZWUblryNwEJiZZZyDwMwqRkTw6KOPsmfPnnKXMq5UVVXu12nlVm5mw7Ju3TrOPfdcbrnllnKXMi7kLyjL/6xEDgKzjNm6dSsAGzZsKHMl40vmewSSFkjaIKlD0gX9LP+8pG2SHkteXyhYtkTSM8lrSRr1mJmNtkoOgpKvgJBUDVwBnAxsBNZKWhkRTxWtelNELC3a9jDgEqCF3DOgH0m2fa3UuszMRkNjYyMvvvhi5oeG5gMdEfFcRHQBNwKL9nPbjwN3RcSryZf/XcCCFGoyswHkv7Dypz1aaS688EJOP/10Jk+eXO5Shi2Na6KnAy8VzG8ETuhnvU9KOhF4GvhKRLw0wLbT+/sQSa1AK8DMmTNTKNssm/IBUMl/wY4lc+bMYc6cOeUuoySjNaj170BzRBxH7q/+aw/0DSJieUS0RERLY2Nj6gWaZYV7BFYsjSDYBMwomG9K2vpExCsRsTuZ/Qnwvv3d1szS5R6BFUsjCNYCx0qaJakOWAysLFxB0rSC2dOA/0im7wROkTRF0hTglKTNzMxGScnHCCKiW9JScl/g1cCKiFgv6VKgPSJWAssknQZ0A68Cn0+2fVXSN8mFCcClEfFqqTWZ2cA8NGTFUrmBdkSsBlYXtV1cMH0hcOEA264AVqRRh5kNzUNDVqxyr4AwM7NUOAjMzDLOQWBmlnEOAjOzjHMQmJllnIPAzCzjHARmZhnnIDAzyzgHgVlG+cpiy3MQmGXMmjVryl2CjTEOArOMefnll8tdgo0xDgKzjPG9hqyYg8DMLOMcBGYZ44PEVsxBYJZRDgTLcxCYZYyPEVgxB4FZRrlHYHkOAjOzjEslCCQtkLRBUoekC/pZfp6kpyQ9LunXko4uWNYj6bHktbJ4WzNLl4eGrFjJzyyWVA1cAZwMbATWSloZEU8VrPZ7oCUiOiV9Efgu8Olk2c6ImFdqHWZ2YDw0ZHlp9AjmAx0R8VxEdAE3AosKV4iI30ZEZzL7INCUwueaWQncI7C8NIJgOvBSwfzGpG0gZwN3FMxPkNQu6UFJpw+0kaTWZL32bdu2lVSwmblHYH9S8tDQgZD0WaAF+HBB89ERsUnSMcBvJD0REc8WbxsRy4HlAC0tLf4NNhsmHyOwYmn0CDYBMwrmm5K2vUj6GHARcFpE7M63R8Sm5OdzwN3Ae1OoyczM9lMaQbAWOFbSLEl1wGJgr7N/JL0XuIpcCGwtaJ8iqT6Zngp8ACg8yGxmI8RDQ5ZX8tBQRHRLWgrcCVQDKyJivaRLgfaIWAn8L+Ag4BdJd/TFiDgN+DPgKkm95ELpsqKzjcwsZQ4AK5bKMYKIWA2sLmq7uGD6YwNsdz/wnjRqMLMD42MElucri80yJt8jcM/A8hwEZhnV29tb7hJsjHAQmGVMvifgILA8B4FZRjkILM9BYJYx+R5BT09PmSuxscJBYJYxDgIr5iAwyxgHgRVzEJhllIPA8hwEVhHuvfdenn766XKXMS7kewTd3d1lrsTGCgeBjXm9vb1cdNFFnHfeeeUupeK1tbXxxhtvAPDss8/S1tZW5opsLHAQ2Ji3ZcsWgL4vMBu+jo6Ovp5AZ2cnHR0dZa7IxgIHgY15jz/+eN90PhTMLD0OAhvz7r///r7pBx54oIyVmI1PDgIb03p7e1n78EN8+G27OWIiPPzww+UuyWzcGdVHVZodqG3btvHWjk6OmdHN7h7xbMcz5S7JbNxxj8DGtD179gBQVwV1VcGePV1lrshs/HEQ2Jh2+OGHA/DKripe2V1FY+MRZa7IbPxxENiY1tDQwBGNU9ncWcXmnXXMPLq53CWZjTupBIGkBZI2SOqQdEE/y+sl3ZQsf0hSc8GyC5P2DZI+nkY9Nn703Ts/RK8fqGU2Iko+WCypGrgCOBnYCKyVtLLoIfRnA69FxGxJi4HvAJ+W9C5gMTAHeBvwK0n/JSIq+iYobW1te12oM3v2bJYtW1bGiirX+eefz9Ztf6R6YjW9PT386ld3cdZZZ9Hc3Fzu0szGjTTOGpoPdETEcwCSbgQWAYVBsAj4ejJ9M/Aj5Z6cvQi4MSJ2A3+Q1JG834idLB4R9PT09Pvq7e0dcFnhq7u7u+/nnj17+n52dXWxa9cu7r333r0ufNqyZQvTp0+nvr6euro6amtrqamp6XtVV1fvNV1VVUV1dXXfq3h+oLbxZs2aNbS3twOwuTP376tWL+f+z2Vc/r3vM3v27HKWZzZupBEE04GXCuY3AicMtE5EdEt6HTg8aX+waNvp/X2IpFagFWDmzJmDFvTVr32Nhx96aP//BSOgoaGBhQsXsmrVKjZv3swPf/jDstVy7bXXMmvWrLJ9/nDccccdfPvb36a+Oqiqm9i3L6foTXbt2s6yLy+l7Z9/5DA4QDt27Njrd3PHjh3lLsnGgIo5WBwRyyOiJSJaGhsbB1132lFHjVJVBWrq6K0/iJ5JjYSqWbhwIcuWLWPhwoWEquidcAhROwE0urv8mGOOob6+flQ/s1Tbt2/ne5dfzpzDujm4tnevfbmnV1x8/OvU9nTy3e9cVu5SK85bb7211/586623yl2SjQFp9Ag2ATMK5puStv7W2SipBjgEeGU/tz1g559/Pueff/6Q6/X29tLV1cXu3bvp7OzkzTffZPv27ezYsYNdu3axe/duurq66Orq2mcYKP/q7u7uW6erq4vOzp0888zrrFq1CoBVq1ZRX1vDu945iwkTJlBXV7fX8FBtbW3fq3i+vr6eCRMmMHHiRCZPnswhhxzCpEmT+t6npmZ8Xg+4ZcsWuvbs4S+O3M2al+r32pdH1ARTG3qZc+huHn3xxTJXWnkOOuigvfZnU1NTmSuysSCNb5K1wLGSZpH7El8MfKZonZXAEnJj/2cAv4mIkLQSuF7S98kdLD4WGLV7CFRVVTFhwgQmTJjAIYccwrRp01J537a2NtatW8d9991HU1MTc+fO9cHiAzB79mzefswsftbxPI31e9i5cyc333wzAA2HBne+WM/vXq7nE584tcyVVp5JkybttT8nTZpU5opsLCg5CJIx/6XAnUA1sCIi1ku6FGiPiJXAvwI/TQ4Gv0ouLEjW+zm5A8vdwDmVfsYQ4C/9ElVXV3PZd77LeV85l5c2buJtk3qYXNsLwM4e8dOnJ/KhD32Ic845p8yVmo0PqYwtRMRqYHVR28UF07uATw2w7beAb6VRh40fRx55JD++8iouvPAC1j/5JGcet5MN22u4/YUJfPKTn2Tp0qXj8kwps3KomIPFlj2TJ0/m8su/x9vf/na+t+4gbn9hAqeddhrLli1zCJilyEFgY1pDQwNLv/ynobYvfelL5C5BMbO0jM/TTmxcOe644zj99NNpbm5m4sSJ5S7HbNxxENiYV11d7QfXm40gDw2ZmWWcg8DMLOMcBGYZMnv2bKqqcv/tVSXfq8kAB4FZpixbtqzv3lMNDQ2++NEAB4FZ5uRPv/W1GJbnIDDLGAeBFXMQmGVMPgjG691r7cA5CMwypi8Iah0EluMgMMuY/FlDtTW1Za7ExgoHgVnG5HsEdXV1Za7ExgoHgVnGOAismIPALGPyQ0P1dZX1LGsbOQ4Cs4w59NBDAait9TECy3EQmGVMS0sL4CCwPykpCCQdJukuSc8kP6f0s848SQ9IWi/pcUmfLlh2jaQ/SHosec0rpR4zG1r+2ED+VhNmpfYILgB+HRHHAr9O5ot1Ap+LiDnAAuCfJB1asPxrETEveT1WYj1mNoR8EPjKYssrNQgWAdcm09cCpxevEBFPR8QzyfT/A7YCjSV+rpkNU35IKCLKXImNFaUGwZERsTmZfhk4crCVJc0H6oBnC5q/lQwZ/UCS+6pmI8zHBqzYkNeYS/oVcFQ/iy4qnImIkDTgnxiSpgE/BZZERG/SfCG5AKkDlgP/AFw6wPatQCvAzJkzhyrbzAbg6wes2JBBEBEfG2iZpC2SpkXE5uSLfusA600GVgEXRcSDBe+d703slnQ18NVB6lhOLixoaWlxn9ZsmHyzOStW6tDQSmBJMr0E+LfiFSTVAbcC10XEzUXLpiU/Re74wpMl1mNmQ/DQkBUrNQguA06W9AzwsWQeSS2SfpKs83fAicDn+zlN9P9IegJ4ApgK/GOJ9ZiZ2QEqqY8YEa8AH+2nvR34QjL9M+BnA2z/l6V8vpmZlc5XFptlVP7mc2YOArOM8nUElucgMMsY9wSsmIPALGPyPQEHguU5CMzMMs5BYGaWcQ4Cs4zJDwn5YLHlOQjMzDLOQWBmlnEOArOM8VlDVsxBYJYxTU1NAMydO7fMldhY4fvRmmXMu971Lq677jpmzJhR7lJsjHAQmGVQc3NzuUuwMcRDQ2ZmGecgMDPLOAeBmVnGOQjMzDLOQWBmlnEOAjOzjHMQmJllnCrxDoSStgEvlLuO/TAV+GO5ixgnvC/T5f2ZrkrZn0dHRGNxY0UGQaWQ1B4RLeWuYzzwvkyX92e6Kn1/emjIzCzjHARmZhnnIBhZy8tdwDjifZku7890VfT+9DECM7OMc4/AzCzjHARmZhnnICiRpBWStkp6coDlktQmqUPS45KOH+0aK4WkCZIelrRO0npJ3+hnnXpJNyX78yFJzWUotWJIel7SE5Iek9Tez3L/fu4nSe9I9mP+9Yakc4vWqcj96SAo3TXAgkGWnwocm7xagR+PQk2VajfwlxExF5gHLJD0/qJ1zgZei4jZwA+A74xuiRXpIxExb4Dz3P37uZ8iYkOyH+cB7wM6gVuLVqvI/ekgKFFE3AO8Osgqi4DrIudB4FBJ00anusqS7KO3ktna5FV8NsMi4Npk+mbgo/JT2Evh38/h+SjwbEQU3+GgIveng2DkTQdeKpjfmLRZPyRVS3oM2ArcFREPFa3Stz8joht4HTh8VIusLAGskfSIpNZ+lvv3c3gWAzf0016R+9NBYGNKRPQkXe8mYL6kd5e5pEr3wYg4ntyQxTmSTix3QZVOUh1wGvCLcteSFgfByNsEzCiYb0rabBARsR34Lfsef+nbn5JqgEOAV0a1uAoSEZuSn1vJjWfPL1rFv58H7lTg0YjY0s+yityfDoKRtxL4XHI2wfuB1yNic7mLGoskNUo6NJluAE4G/rNotZXAkmT6DOA34asi+yVpkqSD89PAKUDx2W3+/TxwZ9L/sBBU6P6sKXcBlU7SDcBJwFRJG4FLyB3kJCKuBFYDfwV0kDvL4O/LU2lFmAZcK6ma3B8pP4+I2yVdCrRHxErgX4GfSuogd5B+cfnKHfOOBG5NjqXXANdHxC8l/Q/w7+dwJIF6MvDfC9oqfn/6FhNmZhnnoSEzs4xzEJiZZZyDwMws4xwEZmYZ5yAwM8s4B4GZWcY5CMxKJOluSfvc2VNSi6S2AbZ5XtLUftq/LumrI1Gn2UB8QZnZCImIdmCfZwCYjTXuEdi4JKlZ0n9I+t/JQ27WSGoo/Otd0lRJzyfTn5d0m6S7kr/Wl0o6T9LvJT0o6bAhPvJTyUN1npb0oeQ9T5J0ezJ9eFLDekk/AfpunS3pomS73wHvKGh/u6RfJncOvVfSO5P2a5KHn9wv6TlJZ6S57yx7HAQ2nh0LXBERc4DtwCeHWP/dwN8Cfw58C+iMiPcCDwCfG2LbmoiYD5xL7jYjxS4BfpfUciswE0DS+8jdJmMeuVsT/HnBNsuBL0fE+4CvAv9SsGwa8EHgr4HLhqjNbFAeGrLx7A8R8Vgy/QjQPMT6v42IN4E3Jb0O/HvS/gRw3BDb3jLE55xILmSIiFWSXkvaPwTcGhGdAJJWJj8PAv4C+EXBc3fqC97vtojoBZ6SdOQQtZkNykFg49nugukeoAHo5k894QmDrN9bMN/L0P9X8uv27Me6+6MK2J48m2Gwz4OCYSaz4fDQkGXN8+SeNwu521iPlnuAzwBIOhWYUtB+enL84mDgbwAi4g3gD5I+lWwjSXNHsV7LEAeBZc3lwBcl/R7Y5/TNEfQN4ERJ68kNEb0IEBGPAjcB64A7gLUF25wFnC1pHbCe3PNwzVLn21CbmWWcewRmZhnng8Vm+0nSFcAHipp/GBFXl6Mes7R4aMjMLOM8NGRmlnEOAjOzjHMQmJllnIPAzCzj/j+tXimXgeJ65AAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "weights_vp(sigmoid_model3_init, att=\"grad\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们发现，在num_epochs取值为2的时候（只迭代了一轮），经过Xavier初始化的模型梯度整体更加稳定，并且没有出现梯度消失的情况，反观原始模型sigmoid_model2，第一层的梯度已经非常小了，已经出现了梯度消失的倾向。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "#————————————————————————————增加迭代轮数————————————————————————————\n",
    "train_l, test_l = model_comparison(model_l = model_l, \n",
    "                                   name_l = name_l, \n",
    "                                   train_data = train_loader,\n",
    "                                   test_data = test_loader,\n",
    "                                   num_epochs = num_epochs, \n",
    "                                   criterion = nn.MSELoss(), \n",
    "                                   optimizer = optim.SGD, \n",
    "                                   lr = lr, \n",
    "                                   cla = False, \n",
    "                                   eva = mse_cal)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d54e31a530>]"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d54e31a950>]"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x2d54e327820>"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'mse_train')"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAzO0lEQVR4nO3deXxU1f3/8ddnliSTPSEJJBkgEBRkX8JSV0CligribtXWlWqxta210lpbtFq1P792U7GKiijiAorYWsVdQQTZERARZEnYwhZCyDYz5/fHnUBYEpLMHj7Px2MeM5m5956TyeSdm3PPIsYYlFJKxR5bpCuglFKqZTTAlVIqRmmAK6VUjNIAV0qpGKUBrpRSMUoDXCmlYpQGuFJKxSgNcKWCSEQ6iMh+EbFHui6q9dMAV8pPRK4XkTmBHMMYs8kYk2yM8QarXko1RANcqWbQM2sVTTTAVdQRkQ0icpeILBeRChF5VkTaisj/RKRcRD4QkQwRSRCRl0Rkl4jsFZGvRKSt/xhp/v22ikiJiDzQWPiKyCnAU8AP/E0ge/3PTxaRiSLyjohUAMNE5AIRWSIi+0Rks4hMqHecAhExIuLwf/2JiPxZROb66z5bRLJC+PapE4gGuIpWlwLnAicDFwH/A34PZGN9bn8B/ARIA9oDbYBbgUr//pMBD9AF6AeMAG5uqDBjzGr//vP8TSDp9V7+EfAgkALMASqAHwPpwAXAbSJycSPfy4+AG4AcIA74zfG+eaWaQgNcRat/GWO2G2NKgM+B+caYJcaYKuBNrFCuxQruLsYYrzFmkTFmn/8sfCTwS2NMhTFmB/A34KoW1uUtY8xcY4zPGFNljPnEGLPC//VyYBpwViP7P2+M+dYYUwm8BvRtYT2UOowj0hVQqgHb6z2uPMbXycCLWGffr4hIOvAScA/QEXACW0Wkbh8bsLmFdTlsPxEZDDwM9MQ6o44HXm9k/231Hh/w112pgOkZuIpZxphaY8x9xpjuwKnAhVhNG5uBaiDLGJPuv6UaY3oc75BNfP5lYBbQ3hiThtV2LkftpVSIaYCrmCUiw0Skl//i5D6sJhWfMWYrMBv4PxFJFRGbiBSKSGPNHGCd5btFJO4426UAu40xVSIyCKuNW6mw0wBXsawdMB0rvFcDn2I1q4B1Jh4HrAL2+LfLPc7xPgJWAttEZGcj2/0MuF9EyoE/YrVrKxV2oivyKKVUbNIzcKWUilEa4OqEIiJP+QfqHHl7KtJ1U6q5tAlFKaViVFj7gWdlZZmCgoJwFqmUUjFv0aJFO40x2Uc+H9YALygoYOHCheEsUimlYp6IbDzW89oGrpRSMeq4AS4iz4nIDhH5ut5zmSLyvois9d9nhLaaSimljtSUM/DJwHlHPDce+NAYcxLwof9rpZRSYXTcNnBjzGciUnDE06OBof7HLwCfAHcHs2JKtSa1tbUUFxdTVVUV6aqoKJaQkIDb7cbpdDZp+5ZexGzrn28CrJnW2ja0oYiMBcYCdOjQoYXFKRXbiouLSUlJoaCggHozJCp1kDGGXbt2UVxcTKdOnZq0T8AXMY3VkbzBzuTGmKeNMUXGmKLs7KN6wSh1QqiqqqJNmzYa3qpBIkKbNm2a9V9aSwN8u4jk+gvNBXa08DhKnTA0vNXxNPcz0tImlFlYy1k97L9/q4XHaZplr8L+bZDWHtI7WPfJOaC/EEqpE9hxA1xEpmFdsMwSkWLgT1jB/ZqI3ARsBK4IZSX5egasfe/w5xwJkOb2h3p7SOtg3dcFfEou2COw4JDPBxWlUFYMZZut+30l4K3117denZPbgk0XOVdKtUxTeqFc3cBLZwe5Lg1ac/azVA/ZS0btNlKqtpB0YCuO8mKkbDPs3QRrvrZCsz6xQ2r+oVBPzoG4FIhLgvhkiPPfDj5OgvgU67HT1fDZfXU5lJUcHdB1X+/bAt6aw/dxJoLNCdVlhz1tbA5IzcOkujFp7fGm5ONLzceb7KY2JZ/a5DyMMwmvMdhEcMXZSXTacdh1/JUKjptvvplf//rXdO/ePWRljBw5kpdffpn09PTDnp8wYQLJycn85jehX+O5KWXV3+bee+/lrbfewmazkZOTw+TJk8nLywt5PZsrJtbEfOh/q/lkTV1AO4D2OO0dSElwkpLgICXBQWaqj4723bhtO8ljBzneHWR6dpC+bxvJOz4hrnoXdl9NY8UcZMSG15GI15mM15GIz5EEvlriK7bgrN132LY+bOxzZrPHmcMuW2d2JA1iK1lsMVkU+zLZ6Mlke00CNV5DgqmgrW8X7dhJLqXkyU7ydu0ib/cu8uVb2rGbOPEddvw9JplS04ZSk04F8RwwCVSJixq7C4/dhceReLCOvrgkcCYh8YnY4pOxxadgT0jGmZBMnNNBHLUkmErifFXE+aqJ81XiNP57XxVObxUOXyVObyUObxUObyV2/626TU/29L4Jh92O3SbYbYLDf1//a1v958W6b0q7Xt2kasZYV8SNMf57/0/df2wVXJMmTQp5Ge+8807Iywi2u+66iz//+c8A/POf/+T+++/nqaeib8LKmAjwu37YlWsHd6S8upbyKk+9W+1h9wsOtOHDqlTKqzpQXu056jgOPCRSRTJVJEkVSVSRJJUkU0Wi/7lkKkmUKpJrq0iqrNuuEg+JbDGD2WKy2GraUGLasMVksdeeSZzEkWiz47JbZ8iJTod1thxnp3ucnaI4O/EOK/hsItgE7DahRoTNNqFEYJFNcBgvKZ5dJFdvI7VqGynV20iu3kp65TbaVe/C5tmH3VuJ03vAH7K14MVa/TEEDph4DhCPBxt5373BY3O2M917vFXJjmZ934eHM3BYQB9P56wkXvnpEHJSEppdfrS57+2VrNqy7/gbNkP3vFT+dFHjS35WVFRwxRVXUFxcjNfr5d5772XixIk8+uijFBUV8eyzz/LII4+Qnp5Onz59iI+P5/HHH+f666/H5XKxZMkSduzYwXPPPceUKVOYN28egwcPZvLkyQBMmzaNv/zlLxhjuOCCC3jkkUeAQ3MgZWVl8eCDD/LCCy+Qk5ND+/btGTBgQIP1HTp0KP369ePzzz+noqKCKVOm8NBDD7FixQquvPJKHnjgAQAee+wxnnvuOcD6j+KXv/wlQINlrVu3jnHjxlFaWkpiYiLPPPMM3bp1O6zs1NTUw963aL0AHRMB3iMvjR55ac3ax+cz7K85FPTVtb6DfR3rzu6sx3V7mIOP6wdLXdjYbUJenJ3EOAeJcfboaM7w1EBtBdRUQM0BqNnvf1xhPa49gK96P57KcjyeWv+Zuguv3bp57Al4HdZ9rc06o6+1JeCxu6iROHxG8PgMXq8H50c/4eHdL3LR2RdTllSA1+fD6wOvz4fHZ/D5jLWt/+apd+/zGbzGIFgtU+Jf/9d6zMHmqvqv1/2+CODxGf792Tpue2kxL98ymHiHXjdoiXfffZe8vDz++9//AlBWVsbEiRMB2LJlC3/+859ZvHgxKSkpDB8+nD59+hzcd8+ePcybN49Zs2YxatQo5s6dy6RJkxg4cCBLly4lJyeHu+++m0WLFpGRkcGIESOYOXMmF1988cFjLFq0iFdeeYWlS5fi8Xjo379/owEOEBcXx8KFC/nHP/7B6NGjWbRoEZmZmRQWFvKrX/2KDRs28PzzzzN//nyMMQwePJizzjoLn8/XYFljx47lqaee4qSTTmL+/Pn87Gc/46OPPjqq7HvuuYcpU6aQlpbGxx9/HOjbHxIxEeAtYbMJqQlOUhOcgCvS1QkNR5x1czU8FY0Na2HI463Se1x5z8NTp3HWivFw0/vgiA/0iM1yUttkbn95CX+cuZKHL+0VtWdETXG8M+VQ6dWrF3feeSd33303F154IWecccbB1xYsWMBZZ51FZmYmAJdffjnffvvtwdcvuugiRIRevXrRtm1bevXqBUCPHj3YsGEDGzduZOjQodSN9bjmmmv47LPPDgvwzz//nDFjxpCYmAjAqFGjjlvnum169epFjx49yM21ljXt3LkzmzdvZs6cOYwZM4akpCQALrnkEj7//HN8Pt8xy9q/fz9ffPEFl19++cEyqquP/S/sgw8+yIMPPshDDz3E448/zn333Xfc+oabXg1TTZOWD6OfgK3L4MP7w178hb3zGDeskFcXbmbKvGPOrKmO4+STT2bx4sX06tWLP/zhD9x/f9N/jvHx1h9sm8128HHd1x7P0c2VwRLscn0+H+np6SxduvTgbfXq1Y3uc8011zBjxoxmlxUOGuCq6bpdAANvhnmPw9oPwl78ned25ZxTcrj/P6v4Yl1ji8arY9myZQuJiYlce+213HXXXSxevPjgawMHDuTTTz9lz549eDyeZgfWoEGD+PTTT9m5cyder5dp06Zx1lmHXy8588wzmTlzJpWVlZSXl/P2228H/D2dccYZzJw5kwMHDlBRUcGbb77JGWec0WBZqampdOrUiddffx2wmkiXLVt21HHXrl178PFbb711VBt5tGi1TSgqREY8ABu/gJm3wm1fWN0zw8RmE/52ZV/GPPkF46YuZtbtp9M+MzFs5ce6FStWcNddd2Gz2XA6nUycOPFgt7r8/Hx+//vfM2jQIDIzM+nWrRtpaU2/7pSbm8vDDz/MsGHDDl7EHD169GHb9O/fnyuvvJI+ffqQk5PDwIEDA/6e+vfvz/XXX8+gQYMA6yJmv379ABosa+rUqdx222088MAD1NbWctVVVx3W3g8wfvx41qxZg81mo2PHjlHZAwXCvCZmUVGR0RV5WoHtq+CZYdDxNLhmOtjC+4/c9zsrGP34HPLSXcy47VSS4qP/PGT16tWccsopka5Go/bv309ycjIej4cxY8Zw4403MmbMmEhX64RzrM+KiCwyxhQdua02oajma9sdfvggrPsQvnwy7MV3ykri8R/159vt5dz52jJ8Pl2YOxgmTJhA37596dmzJ506dTrsAqSKTtF/6qKiU9FNsO5j+GACFJwGef3CWvyZJ2fz+5Gn8MB/V/PPj9byy3NODmv5rdGjjz4akXLHjRvH3LlzD3vujjvu4IYbbohIfWKJBrhqGREY9S+YeBpMvwl++pk1LUEY3XR6J1Zt3cffP1hLt3apnNezXVjLV8HxxBNPRLoKMUubUFTLJWbCpc/A7vXwv9+GvXgR4S9jetGnfTq/fm0p32wL7uhGpaKdBrgKTMHpcOZvYOlUWDE97MUnOO08fd0AkuMd3DJlIXsqmjbfjVKtgQa4CtxZ48E9CP7zK9izIezFt01N4KnrBrC9rJpxLy+m1us7/k5KtQIa4CpwdgdcOgkQmHGzNfd5mPXvkMFfLunFF+t28eB/Gx9Zp1RroQGugiOjI1z0Nyj+Cj55KCJVuGyAmxtP68TkLzbw2lebI1KHWHPzzTezatWqkJYxcuRI9u7de9TzEyZMCFvPl6aUVX+be++9l969e9O3b19GjBjBli1bGtxv1qxZPPzww40ee8uWLVx22WUALF26NGhT7GqAq+DpeSn0uxY+fwy+/ywiVfj9yG6c3iWLe2auYNHG3RGpQyyZNGlSSBdzAGs+8CMXc4h2d911F8uXL2fp0qVceOGFjc4bM2rUKMaPH9/o8fLy8pg+3bpGFMwA126EKrjO/ytsmg9vjIVb50JSm7AW77DbePxH/Rj9xFx++uJi3v75aeSmRdlslP8bD9tWBPeY7XrB+Y2fBep84KGZD3zy5MksXLjw4HuVmprKwoUL2bZtG3/961+57LLL2LBhAxdeeCGLFy/mj3/8I5WVlcyZM4ff/e53XHnllY3+3BqjAa6CKy4JLnsWJp0Ds26Hq14O++LT6YlxPPPjIsY8MZexUxbx+q0/IMGpc4jrfODhmQ9869atzJkzh2+++YZRo0YdbDqp+37uv//+g4EfKA1wFXy5feCc++C938FXk2DQLWGvwsltU/j7Vf24ZcpCxs9Yzt+u7Bs9c4gf50w5VHQ+8PDMB37xxRdjs9no3r0727dvb9I+LaVt4Co0htwGXc6F9+6B7SsjUoVzu7flznNPZubSLTz92fqI1CGa6Hzg4ZkPvH49Qz1ZoAa4Cg0RuHgiJKTB9ButJd8i4PbhXTjnlBwee/9bvCf4pFc6H3h0zAeekpJCeXl5UI6lAa5CJzkbxjwFpd/A7D9EpAoiwtCuOVR7fOwor4pIHaLFihUrGDRoEH379uW+++7jD3849DOpPx/4aaedRkFBQYvnA+/Tpw8DBgxodD7w888/P+jzgQ8ePPjgfOCNlTV16lSeffZZ+vTpQ48ePXjrrbeOOu748ePp2bMnvXv3Zvbs2fzjH/8IuK51hg0bxqpVq+jbty+vvvpqQMfS+cBV6L19Byx5CX5XDM7w9wj5eM0Obnj+K6bf+gOKCjLDXj7ofOCq6XQ+cBVdThoBPo+1nmYEtM+w/mgU76mMSPmxQucDjz3aC0WFXr7/xKH4K+gwJOzF56VbAV6yVwO8MTofeOOef/75o5pSTjvttIhOh6sBrkIvpS2kd4DNCyJSfGKcgzZJcRTvicyF1DrGmOjpyhhFYmU+8BtuuCHkf1Sa26StTSgqPNyDoDhy1z/yM1wRbUJJSEhg165dIe9WpmKXMYZdu3aRkJDQ5H30DFyFh3sgfD0dyoohzR3+4jNcfLM1OF23WlS+201xcTGlpaURq4OKfgkJCbjdTf/90ABX4dHe342r+KuIBHh+uosPV++IWDOG0+mkU6dOYS9XtW7ahKLCo20vcCRErBnFnZFItcdH6f5jD5tWKhZpgKvwcMRBbt+IXcjMr+uJol0JVSsSUICLyK9EZKWIfC0i00Sk6a3v6sTjLrL6gnvCfxbsztS+4Kr1aXGAi0g+8AugyBjTE7ADVwWrYqoVcg8Eb3Xw58JugrozcA1w1ZoE2oTiAFwi4gASgYbXHVKq/SDrvvirsBedkuAkzeWkZG9k+4IrFUwtDnBjTAnwKLAJ2AqUGWNmH7mdiIwVkYUislC7UJ3gUvMgNT8iAQ5WV0I9A1etSSBNKBnAaKATkAckici1R25njHnaGFNkjCmqm+xdncDcA2FzZAI8P92lFzFVqxJIE8o5wPfGmFJjTC3wBnBqcKqlWi33QCjbBOXbwl90RiLFeyp1NKRqNQIJ8E3AEBFJFGtkxNlA40tbKOWuN6AnzPIzXFTWetlzoDbsZSsVCoG0gc8HpgOLgRX+Yz0dpHqp1iq3D9icEQlw98FpZfVCpmodAuqFYoz5kzGmmzGmpzHmOmOMDnNTjXMmQG7viIzI1ME8qrXRkZgq/NyDoGQxeMPblNE+w1qhXHuiqNZCA1yFn7sIPJVhX60+1eUgOd6hTSiq1dAAV+EXoQE9IoI7w6Ur86hWQwNchV9ae0huG7ELmdqEoloLDXAVfiJWd8JIdCX0D+bRvuCqNdAAV5HhHgi710PFzvAWm5FIebWHfZWesJarVChogKvIODigJ7zdCfPr+oLrpFaqFdAAV5GR1w/EDsXhXeDh0GAebQdXsU8DXEVGXCK06xn2dnAdzKNaEw1wFTnugdaAHp83bEVmJsXhctr1DFy1ChrgKnLcg6BmP+wI3xxoIkJ+hksXdlCtgga4ihx3kXUf5mYU7QuuWgsNcBU5mZ0hsY0GuFItpAGuIidCA3ry0xMpq6ylvErnBVexTQNcRZZ7IOz8Fg7sDl+R/q6EOieKinUa4Cqy6gb0lCwOW5F1g3m0K6GKdRrgKrLy+4PYwtqMooN5VGuhAa4iKz4FcrqHdURmVlI8cQ6bNqGomKcBriLPXQTFi8DnC0txNpvgTnfpwg4q5mmAq8hzD4LqMutiZpjkZ7i0DVzFPA1wFXkHZyYMbzu4toGrWKcBriKvTRdISAtrgOenu9hVUUNlTfjmYVEq2DTAVeTZbGEf0OP2r1Cvc6KoWKYBrqKDe6A1qVXVvvAU5+9KuFmbUVQM0wBX0cE9EDBQsigsxelgHtUaaICr6JA/wLoP0xJrOSkJOO2iFzJVTNMAV9HBlQ7Z3cLWDm63CblpLh3Mo2KaBriKHu4iK8CNCU9xGTqYR8U2DXAVPdwDoXI37F4fluLy03Uwj4ptGuAqergHWfebwzMvijsjkR3l1VTVal9wFZs0wFX0yO4KcSlhawev64mytawqLOUpFWwBBbiIpIvIdBH5RkRWi8gPglUxdQKy2cE9IGwzEx6aVlbbwVVsCvQM/B/Au8aYbkAfIHzLi6vWyT0Qtq+EmoqQF5Wfrn3BVWxrcYCLSBpwJvAsgDGmxhizN0j1Uicq90AwPtiyJORF5aYlYLdpX3AVuwI5A+8ElALPi8gSEZkkIklBqpc6UdXNTBiGC5kOu412qQnahKJiViAB7gD6AxONMf2ACmD8kRuJyFgRWSgiC0tLSwMoTp0QEjMhszBsIzLzM3Qwj4pdgQR4MVBsjJnv/3o6VqAfxhjztDGmyBhTlJ2dHUBx6oTRfpB1ITMMA3p0XnAVy1oc4MaYbcBmEenqf+psYFVQaqVObO4iqCiFvRtDX1S6i+37qqjxhGc5N6WCKdBeKD8HporIcqAv8JeAa6RU3YCeMDSjuDMS8RnYpn3BVQwKKMCNMUv9zSO9jTEXG2P2BKti6gSW0x2ciWG5kFk3mKdYF3ZQMUhHYqroY3dAXv+wjMg8NJhH28FV7NEAV9Gp/UDYthxqQxusuWkuRHQwj4pNGuAqOrkHgs8DW5eFtJg4h422KQl6Bq5ikga4ik51A3rC0IySr/OCqxilAa6iU3IOpHcMy4VMtw7mUTFKA1xFL/fAMHUldLG1rAqPV/uCq9iiAa6iV/tBUL4FyopDWkx+eiJen2F7eXVIy1Eq2DTAVfRyF1n3IW4HP9iVcLe2g6vYogGuolfbXuBICHkzSt1gHm0HV7FGA1xFL0cc5PYN+YXMuoUdtCuhijUa4Cq6tR9o9QX3hK59OsFpJys5XgfzqJijAa6im3sgeKth24rQFpPh0vlQVMzRAFfRLUwDevIzXHoGrmKOBriKbql5kOoOeTt43WAeny/0i0goFSwa4Cr6uYtC35Uw3UWt17BD+4KrGKIBrqJfwelQthl2rw9ZEe6MRABKtB1cxRANcBX9Codb9+s+ClkROi+4ikUa4Cr6ZXaG9A6w7uOQFZGvAa5ikAa4in4i1ln495+BtzYkRSTGOchMitMAVzFFA1zFhsLhUL0vpMPq89N1WlkVWzTAVWzodCaILeTt4Lqwg4olGuAqNrgyIL8opAGen24N5jFG+4Kr2KABrmJH4XDYshgO7A7J4d0ZLqo9PnburwnJ8ZUKNg1wFTsKh4PxWRczQyD/YF9wbQdXsUEDXMWO/AEQnwrrPgzJ4Q/1Bdd2cBUbNMBV7LA7rIuZ6z6GELRTa19wFWs0wFVs6XK2Nax+13dBP3RqgpPUBIfOSqhihga4ii0hHlbvzkjUJhQVMzTAVWzJKLCG1ocowPMzdDCPih0a4Cr2FA6H7z8HT/C7+1mDebQvuIoNGuAq9hQOh9oK2Dw/6IfOT3dxoMbL3gOhmXNFqWDSAFexp+AMsDlC0oxSNy+49kRRsSDgABcRu4gsEZH/BKNCSh1XQiq4B4UowK2uhLqwg4oFwTgDvwNYHYTjKNV0hcNh6zKo2BnUw+rCDiqWBBTgIuIGLgAmBac6SjVR4XDAwPpPgnrYNJeTpDi7BriKCYGegf8d+C3ga2gDERkrIgtFZGFpaWmAxSnll9cXEtKD3owiIv6+4BrgKvq1OMBF5EJghzFmUWPbGWOeNsYUGWOKsrOzW1qcUoez2aHzUCvAg9zlT+cFV7EikDPw04BRIrIBeAUYLiIvBaVWSjVFl7OhfCuUfhPUw+pgHhUrWhzgxpjfGWPcxpgC4CrgI2PMtUGrmVLH03mYdR/kZhR3hovyKg9lldoXXEU37QeuYld6e8g6OegBnp/unxdc28FVlAtKgBtjPjHGXBiMYynVLIXDYcNcqK0K2iF1XnAVK/QMXMW2wuHgqYRN84J2yPyDg3n0DFxFNw1wFdsKTgebM6jNKG2S4khw2rQroYp6GuAqtsUlQYch1io9QSIiB1eoVyqaaYCr2Fc4HLavgPLtQTukOyORYp0PRUU5DXAV++pW6QnisPp8/7zgSkUzDXAV+9r1hsSsoK5W785wsfdALfurPUE7plLBpgGuYp/NBoXDrHZwX4PT8jRL3bzg2g6uopkGuGodCodDxQ7YsTIoh8tP13nBVfTTAFetQ5CH1bfXecFVDNAAV61Dai7kdA9agGclxxNnt2kTiopqGuCq9SgcDhvnQU3gzR42m2hPFBX1NMBV61E4DLzVsPGLoBwuP91FsQ6nV1FMA1y1Hh1PA3t80JpR3BkuSnRCKxXFNMBV6+F0QcdTgxbg+ekudu6voarWG5TjKRVsGuCqdSkcDqWrYd+WgA/lztSeKCq6aYCr1qVuWH0QzsLrFnbQecFVtNIAV61L2x6Q3DYoAe7WecFVlNMAV62LiHUWHoRh9W1TE3DYRJtQVNTSAFetT+FwqNwN25YFdBi7TchNT9DBPCpqaYCr1qfzUOs+GM0o6YnaBq6ilga4an2Sc6Bdr6Cs0pOf4dI2cBW1NMBV61Q4HDZ9CdX7AzqMO8PF9n3VVHu0L7iKPhrgqnUqPBt8tbBhTkCHqZtWduveqmDUSqmg0gBXrVOHIeBwBdwOXrewg/ZEUdFIA1y1To54KDg9CAGuCzuo6KUBrlqvwuGway3s3dTiQ7RLS8AmegauopMGuGq9gjCs3mm30S41QQNcRSUNcNV6ZXeF1PygtIPrYB4VjTTAVeslYi3ysP4T8LW8G6A7w6WDeVRU0gBXrVvhcKgqgy1LWn6InGS27qtiw86KIFZMqcBpgKvWrdNQQAJqRrl8gBun3ca/P1sfrFopFRQa4Kp1S2oDeX3huw9bfIic1ASuKHIzY1Ex28p0QI+KHi0OcBFpLyIfi8gqEVkpIncEs2JKBU3XkbD5y4DOwn96ZiFeY5j0uZ6Fq+gRyBm4B7jTGNMdGAKME5HuwamWUkH0g9shqyu8eRtU7GrRIdpnJjK6Tx5T529id0VNkCuoVMu0OMCNMVuNMYv9j8uB1UB+sCqmVNDEJcJlz1pzhM+6HYxp0WFuG1pIZa2XyXO/D3IFlWqZoLSBi0gB0A+Yf4zXxorIQhFZWFpaGozilGq+dr3gnAmw5h1Y+FyLDnFS2xR+2KMtk7/YQHlVbXDrp1QLBBzgIpIMzAB+aYzZd+TrxpinjTFFxpii7OzsQItTquUG32Z1K3zvHihd06JDjBvWhX1VHqbOb/nwfKWCJaAAFxEnVnhPNca8EZwqKRUiNhtcPNFqUpl+E3iqm32I3u50zjgpi0mff09Vrc4RriIrkF4oAjwLrDbGPBa8KikVQintYPSTsH0FfHBfiw4xblgXdu6v5vWFm4NcOaWaJ5Az8NOA64DhIrLUfxsZpHopFTpdz4OBt8CXT7Sof/jgTpkM6JjBU5+up9brC0EFlWqaQHqhzDHGiDGmtzGmr//2TjArp1TIjPgzZJ8CM2+Dip3N2lVEGDeskJK9lcxauiVEFVTq+HQkpjoxOV1w6SSo3AtvjWt218JhXXPo1i6FJz/5Dp+vZd0SlQqUBrg6cbXrCefeB9++C19Natau1ll4F9aVVvDeym0hqqBSjdMAVye2wbdCl3Nh9h9gx+pm7TqyVy4FbRJ54pPvMC0cHKRUIDTA1YlNBC5+EuJTrK6FtU2frMpuE24bWsjXJfv4bG3z2tGVCgYNcKWSc6yuhTtWwgcTmrXrmH5uctMSeOLj70JTN6UaoQGuFMDJI2DQT2H+RFj7QZN3i3PYGHtmZxZ8v5uvNuwOYQWVOpoGuFJ1zr0fcrpbXQv3N33enqsGdiAzKY4n9SxchZkGuFJ1nAlw6bPWEmxv/azJXQtdcXZuOr0TH68pZeWWshBXUqlDNMCVqq9tdxjxAKydDQueafJu1w7pSEq8gyc/XhfCyil1OA1wpY406BY46YdW18LtK5u0S5rLyXU/6Mg7X29lXen+EFdQKYsGuFJHEoHRT0BCGsy4GWorm7Tbjad3Is5u46lPoussfPLc7xn9+By279P1PFsbDXCljiU525p6dscqeP9PTdolKzmeqwd14M0lJZTsbVroh9rzc79nwturWFZcxq0vLaLao1PgtiYa4Eo15KRzrEUgFvwbvp3dpF1uObMzAM98FvnFj1/8ciP3vb2KH/Zoy7+u7seSTXu5d+bXOmq0FdEAV6ox50yAtj2tppQFz4Cv8TPY/HQXl/TPZ9qCTezc3/wFI4Jl2oJN3Dvza845JYd/Xd2fi/rk8fPhXXhtYTFT5m2MWL1UcGmAK9UYZwJcNRVye8M7v4F/nwUb5ja6y61nFVLj9fHcnMgsfvz6ws38/s0VDO2azRPX9CfOYf2a/+qckznnlBzu/88qvlinQ/9bAw1wpY4nowB+8jZcPhkq98Dkkda8KWUlx9y8c3YyI3vl8uK8jZRVhnfx4zeXFPPbGcs5vUsWT107gHiH/eBrNpvwtyv70ikriXFTF7N594Gw1k0Fnwa4Uk0hAj3GwO1fwVl3w+q34fGB8Pn/HXNtzZ8NLaS82sOL8zaErYqzlm3hzteWMaRTG56+rogEp/2obVISnDx93QA8PsPYFxdxoMYT+ortXg9fPgXe8P4xOxFogCvVHHGJMOz3cPsCKBwGH94PTwyGNe8etlmPvDSGdc3mubkbwhKS76zYyq9eXUpRQSbPXl+EK+7o8K7TOTuZf13dj2+27eOu6ctDe1GzZDFMOhfevRtev75FC0mrhmmAK9USGQVW2/i1b4DdCdOuhKmXw85D86GMG9aF3RU1vLIgtIsfz165jV9MW0Lf9uk8d/1AEuMcx91naNcc7j6vG/9dvpUnQ9Vvfd1H8MJF1h+9M++Cb/4Dr17brCl7VeM0wJUKRJez4bYvYMSDsHEePDnE6jdeXU5RQSaDOmXy9GfrqfGEZvHjD1dvZ9zLi+mZn8bkGwaSHH/88K7z0zM7M6pPHo/OXsNH32wPbsVWTIepV1h/6G6cDcP/ABf+Hda+D9Oughptfw8GDXClAmV3wqm3w88XQe8rYO7frfbx5a8xbmgh2/ZV8eaS4qAX+8maHdz20mJOyU3lhRsHkZLgbNb+IsIjl/ame24qd0xbGrwpAL6cCDNugvaD4Pr/Qmqu9XzRDdYI1/WfwMtXQLVOORAoDXClgiWlrbW6z00fQEo7eOMWzpxzHaPb7uSx979l8tzv2VYWnOaDOWt3MvbFRXTJSWbKjYNIczUvvOu44uz8+7oBOB02bpmykH1VAVxoNAY+uA/eHQ/dLrSal1zph2/T7xq45BnY+AW8dClU7Wt5eQoJ56isoqIis3DhwrCVp1TE+Hyw9CX44D7MgV286zybifvPYrnpTL8OGZzfsx3n98ylfWZisw89b90ubpi8gII2SUy7ZQgZSXHH3tAYKP4Klrxk9QQ59edw0girR80Rvly/i2snzefMk7N55sdF2G1Hb9Morwf+c4dV1oDr4YLHwNbwhVRWzrTO0nP7wrUzjg56dRgRWWSMKTrqeQ1wpUKoci988jAseh48Vex2FTCLM3l6TxFbyKJHXioje+VyXs92FGYnH/dwC77fzU+eW4A7w8W0sUPISo4/eqPy7bD8FStMd34LziRIzISyzVBwhjVdbl7fo3Z7cd4G7n1rJeOGFXLXD7s1/XusOQDTb4Bv37W6WA793TH/SBzlm//Caz+xpvC9bqZVR3VMGuBKRVJVGax6C5a9AhvnYhC2ZBQxw3M6/y7tSQUuTm6bzHk9czm/Zzu6tUtBjgjBRRt38+NnF9A2LYFXxg4hJyXh0IveWvj2PSu0184G44X2Q6DftdDjYnAkwMLn4dOH4cAu6HUFnH0vpHc4eAhjDL97YwWvfLWZJ37Unwt65x7/+zqwG6ZdDZvnw8j/Z03F2xxr34dXroGsk6wQT85u3v4nCA1wpaLFng2w/DVYNg12r8c4XKzPGsYrNafx/NaOeIyNgjaJB8O8tzuNZcVlXDtpPtkp8bwydghtU/3hvWO1FdrLX4WKUkhuB32vhr7+UDxSVRnM+Zt1odEYGHIrnP7rg00Y1R4vVz/9Jau3ljPjtlPpnpfa8PdRVgIvXWI1z1zyjPWHoiXWfWz9EcjoCD9+y7p+oA6jAa5UtKlro142Db6eAVVleJPbsSb7PF488ANe25yK12fIT3exr6qWjMQ4Xv3pEHLja6ztl7wEJYvA5oSu50O/66BwONib0JWwrBg+esD6j8CVAWf9FopuAkccO/ZVMerxudhtwts/P53MY7Wxl66BFy+x/iBc/TJ0OjOw92LDHKvbYWquNW1Bal5gx2tlNMCVima1VbD2PStQ184GnwdPTi++zhrJC+VFrD+QwKSzqshe+zqsngWeKsjpYTWR9L4CkrJaVu7W5fD+vVbXvoxOcM6foPvFLCsu4/J/z2NAhwym3DQIp71eh7XNX8HLl1t/OK6dDrl9gvIWsOlLeOkySGpjhXi95p0TnQa4UrGiYqd1hr1sGmxZAmKHpGzYv81aJajX5VZw5/Zt2sXC4zEGvvvQCvIdq8A9CEY8wIzSfO58fRnXn1rAhFE9rG2/nQ2v/dhq5rjuDcjsHHj59RUvgpfGQHyqFeKZnYJ7/BilAa5ULNrxjdWjZNd30P1i6HYBOF2hKcvnhaVT4aMHrT8Wp1zE47breHSRh/tH9+A8z8dkf3Qn1W1OoeSCF/G4svEZg88YjLH+DhgMPkO95wwGsNuEvDQXOSnx2I7XRXHLUnjxYnC4rBDP6hKa7/d4Kvda1yv2fG9d+M3sDFldreadYPzhbAYNcKVU09RUwLwnYM7fMd5qZrtG8vXeOO50TmeOtwe31v6K/TS//zqA0y7kprnIT3eRn3Ho3p3hwp2eSLu0BGv+8u0r4YVRVl/yH8+CnGZ0a2wqrwf2lRwK6T0bDt12fw9Ve4+9X1wKZHetd+sGWSdDekewhWZsZEgCXETOA/4B2IFJxpiHG9teA1ypGFK+HT55CLN4CmK8FOedx5IBD2Mc8dgEbCII1pB8mxy6t4lAvddtItR4vZTsraJkTyUleysp2XOAkr2V7Civpn4EiUDblATyM1wMcG3nFyV34hAfS4a+QGVmNzAGMR7EV4vNZ91bj2sRn8e6N/77es/bPRUkHdhCUsUmEvZvJmH/ZuL2FyO+QzNFGpsDX1p7yChAMjtjyyiw5nLJKLD6qO/+Hkq/gdI1mNI1ULoGqTg0h4zPkUBtRheq07tQmXYSB9K6sD+lkP1JbmqMg4EFGU2aaOxYgh7gImIHvgXOBYqBr4CrjTGrGtpHA1ypGFS6BooXQp+rGh9d2QI1Hh9byyop2VNJ8d7KegFv3ceXrWeK4wFy2IMXG3ES2KLMe0wym0wOm00OG00Om0xbNpkcNvly2EYmXuotgCHgtNuIs9uw2wWP11Dj9eHx+vD5YzOV/XSRLZxkK6GLlHCSlNDFVoJbDq14VGPsbDDtiL/mZTp27duiejcU4C37c2AZBHxnjFnvL+AVYDTQYIArpWJQXVNBCMQ5bHRsk0THNknHfN3rG8ru4jPYuWgyYJ0lG5uz3s2BscVZ93bnEa8d2sbrcFHpyqPSkUyNx0e819DJ68Pt9VHk8VHrNdR6fdR6fdR4fdR6Dv/a4zU47TacdvHf23A6BKfN/5zDhtNmY49DWGq38Y2pJK3ie1LL15Ncvo6csu+Ib+cO+vsXSIDnA/UnOi4GBh+5kYiMBcYCdOig3YKUUk1ntwnZHU6GDn+JdFVaoDDkJYR8NkJjzNPGmCJjTFF2tg6TVUqpYAkkwEuA9vW+dvufU0opFQaBBPhXwEki0klE4oCrgFnBqZZSSqnjaXEbuDHGIyK3A+9hdSN8zhizMmg1U0op1ahALmJijHkHeCdIdVFKKdUMuqSaUkrFKA1wpZSKURrgSikVo8I6mZWIlAIbW7h7FrDzuFtFjtYvMFq/wGj9AhPt9etojDlqIE1YAzwQIrLwWHMBRAutX2C0foHR+gUm2uvXEG1CUUqpGKUBrpRSMSqWAvzpSFfgOLR+gdH6BUbrF5hor98xxUwbuFJKqcPF0hm4UkqpejTAlVIqRkVdgIvIeSKyRkS+E5Hxx3g9XkRe9b8+X0QKwli39iLysYisEpGVInLHMbYZKiJlIrLUf/tjuOrnL3+DiKzwl33U+nVi+af//VsuIv3DWLeu9d6XpSKyT0R+ecQ2YX3/ROQ5EdkhIl/Xey5TRN4XkbX++4wG9v2Jf5u1IvKTMNbv/4nIN/6f35sikt7Avo1+FkJYvwkiUlLvZziygX0b/V0PYf1erVe3DSKytIF9Q/7+BcwYEzU3rFkN1wGdgThgGdD9iG1+Bjzlf3wV8GoY65cL9Pc/TsFaE/TI+g0F/hPB93ADkNXI6yOB/wECDAHmR/BnvQ1rgELE3j/gTKA/8HW95/4KjPc/Hg88coz9MoH1/vsM/+OMMNVvBODwP37kWPVrymchhPWbAPymCT//Rn/XQ1W/I17/P+CPkXr/Ar1F2xn4wXU2jTE1QN06m/WNBl7wP54OnC0iEo7KGWO2GmMW+x+XA6uxlpaLJaOBKcbyJZAuIrkRqMfZwDpjTEtH5gaFMeYzYPcRT9f/jL0AXHyMXX8IvG+M2W2M2QO8D5wXjvoZY2YbY+qWU/8SazGViGjg/WuKpvyuB6yx+vlz4wpgWrDLDZdoC/BjrbN5ZEAe3Mb/IS4D2oSldvX4m276AfOP8fIPRGSZiPxPRHqEt2YYYLaILPKvR3qkprzH4XAVDf/iRPL9A2hrjNnqf7wNaHuMbaLlfbwR6z+qYzneZyGUbvc38TzXQBNUNLx/ZwDbjTFrG3g9ku9fk0RbgMcEEUkGZgC/NMbsO+LlxVjNAn2AfwEzw1y9040x/YHzgXEicmaYyz8u/wpOo4DXj/FypN+/wxjrf+mo7GsrIvcAHmBqA5tE6rMwEWtF377AVqxmimh0NY2ffUf971K0BXhT1tk8uI2IOIA0YFdYameV6cQK76nGmDeOfN0Ys88Ys9//+B3AKSJZ4aqfMabEf78DeBPrX9X6omEt0/OBxcaY7Ue+EOn3z297XbOS/37HMbaJ6PsoItcDFwLX+P/IHKUJn4WQMMZsN8Z4jTE+4JkGyo30++cALgFebWibSL1/zRFtAd6UdTZnAXVX/C8DPmroAxxs/jazZ4HVxpjHGtimXV2bvIgMwnqPw/IHRkSSRCSl7jHWxa6vj9hsFvBjf2+UIUBZveaCcGnwzCeS71899T9jPwHeOsY27wEjRCTD30Qwwv9cyInIecBvgVHGmAMNbNOUz0Ko6lf/msqYBsqN9Jq65wDfGGOKj/ViJN+/Zon0VdQjb1i9JL7FukJ9j/+5+7E+rAAJWP96fwcsADqHsW6nY/07vRxY6r+NBG4FbvVvczuwEuuq+pfAqWGsX2d/ucv8dah7/+rXT4An/O/vCqAozD/fJKxATqv3XMTeP6w/JFuBWqx22Juwrql8CKwFPgAy/dsWAZPq7Xuj/3P4HXBDGOv3HVb7cd1nsK5XVh7wTmOfhTDV70X/Z2s5VijnHlk//9dH/a6Ho37+5yfXfebqbRv29y/Qmw6lV0qpGBVtTShKKaWaSANcKaVilAa4UkrFKA1wpZSKURrgSikVozTAlVIqRmmAK6VUjPr/gFZEGmnSHU0AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 训练误差\n",
    "for i, name in enumerate(name_l):\n",
    "    plt.plot(list(range(num_epochs)), train_l[i], label=name)\n",
    "plt.legend(loc = 1)\n",
    "plt.title('mse_train')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d5494af430>]"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d5494af850>]"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x2d54e32a560>"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'mse_test')"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAA2uElEQVR4nO3deXxU1fn48c8zySSTkJ0kLAkkYSm7LAZQEEStihZx36pVtC11be2vWrEuVat1b2trq1VExf1bV9x3EVDAsO8gECAESIAEQvaZOb8/7iQGCJBk7mzheb9e85rl3nvOk8nkyZ1zzyLGGJRSSkUeR6gDUEop1TaawJVSKkJpAldKqQilCVwppSKUJnCllIpQmsCVUipCaQJXSqkIpQlcKaUilCZwpZohIpNEZLZNZRWKyE/tKEuppjSBK6VUhNIErsKa7+z1FhFZKiKVIvKsiHQSkY9EpEJEPheRVBFxichLIrJLRMpF5HsR6eQrI9l33DYR2Soi94lI1GHq7Ac8BRwvIvtEpNz3eqyIPCoim0Vkh4g8JSJxvm3pIvK+r+7dIjJLRBwi8iLQHXjPV9YfA/6mqaNGdKgDUKoFzgdOxfq8LgKGAr8EVgEfAr8FtgPJQDegFhgCVPuOfx4oAXoBHYD3gS3Af5urzBizSkSuAX5ljDmhyaYHgZ6+suuBV4C7gNuAPwBFQIZv3+OsoswvRGSMr6zP2/4WKHUwPQNXkeBfxpgdxpitwCxgnjFmkTGmBngbK6HXAx2BXsYYjzFmgTFmr+8s/EzgJmNMpTGmBPg7cElrAhARASYDvzfG7DbGVAB/bVJOPdAFyDHG1BtjZhmdKU4FmJ6Bq0iwo8nj6maeJwAvYp19vyYiKcBLwO1ADuAEtlk5GLBOXLa0MoYMIB5Y0KQcARqaYh4B7gY+9W1/2hjzYCvrUKpVNIGrdsEYUw/cA9wjIrlYTStrfPe1QLoxxt2aIg94vhPrn8UA3zeBA+uvwGpG+YOIDAS+FJHvjTFfNFOWUrbQJhTVLojISSIyyHdxci9Wk4bXGLMN+BR4TESSfBcWe4rIiUcocgeQLSIxAMYYL/AM8HcRyfTVmSUip/seTxCRXr6mlj2AB/A2KauHvT+xUprAVfvRGXgDK3mvAmZiNasAXAHEACuBMt9+XY5Q3pfACmC7iOz0vXYr8AMwV0T2Ap8DfXzbevue7wO+A/5jjPnKt+0B4A5fD5Wb/fkhlWpK9DqLUkpFJj0DV0qpCKUJXB21fANx9jVzeyrUsSnVEtqEopRSESqo3QjT09NNbm5uMKtUSqmIt2DBgp3GmIwDXw9qAs/NzaWgoCCYVSqlVMQTkU3Nva5t4EopFaE0gSulVIQ6YgIXkWkiUiIiy5u89oiIrPZN8fm2b+4JpZRSQdSSNvDngSeA6U1e+wy4zRjjFpGHsKbTvNX+8JRqH+rr6ykqKqKmpibUoagw5nK5yM7Oxul0tmj/IyZwY8w3vsmBmr72aZOnc4ELWhOkUkeboqIiEhMTyc3Npclshko1Msawa9cuioqKyMvLa9ExdrSBXw18dKiNIjJZRApEpKC0tNSG6pSKPDU1NXTs2FGTtzokEaFjx46t+pbmVwIXkdsBN/DyofYxxjxtjMk3xuRnZBzUjVGpo4Ymb3Ukrf2MtDmBi8gkYAJwWcBXHtn4Dcx9Ejytmc5ZKaXatzYlcBEZD/wRmGiMqbI3pGasnAEfT4GnT4RN3wW8Ott5PVCvF6+UUvZqSTfCV7HmN+4jIkUi8kusXimJwGcisjjgk/+c+Qhc/BLU7IHnxsPb18C+koBW6ZeavfDDF/DVAzD9bHiwOzyUC5/cDvv0OoAKH7/61a9YuXJlQOs488wzKS8vP+j1u+++m0cffTSgdbemrqb73HnnnRxzzDEMGTKE0047jeLi4mCE2Wot6YVyaTMvPxuAWA5NBPqdBT1PgVmPwbf/hNUfwMl3QP4vISqEK8MZA+WbYPM82OK77VgBGBAHZA6AwZdAbQXM/Q8UTIMRv4ZRv4MOHUMXt1LA1KlTA17Hhx9+GPA67HbLLbfwl7/8BYB//vOf3HvvvTz1VPhNUhlZa2LGxMMpd8LgS+GjW+CjP8LC6fCzx6D7ccGJwV0H25fC5rk/Jux9vjV2YxIhOx/GTYFuIyArH1xJPx479haY+TDM+SfMnwojfwOjboT4tODE3hrGQMU22LESSlZYr436rfXPVPnlnvdWsLJ4r61l9u+axJ/PGnDYfSorK7nooosoKirC4/Fw55138uSTT/Loo4+Sn5/Ps88+y0MPPURKSgqDBw8mNjaWJ554gkmTJhEXF8eiRYsoKSlh2rRpTJ8+ne+++46RI0fy/PPPA/Dqq6/y17/+FWMMP/vZz3jooYeAH+dASk9P5/777+eFF14gMzOTbt26ceyxxx4y3nHjxjF06FBmzZpFZWUl06dP54EHHmDZsmVcfPHF3HfffQD87W9/Y9q0aYD1jeKmm24COGRd69ev5/rrr6e0tJT4+HieeeYZ+vbtu1/dSUk//t1WVlaG7QXoyErgDdJ7weVvwar34OPbYNrpMPjncOo9kJBpb131NbBpjnUhdct8KF4Ibl97dkoO5J0I3UdCt5GQ2R8cUYcuK703nP8MjL0ZZj4Es/8O85+xEvnx14cukddWQMkq65tDyUorae9YDjXl+++XnA0Dzw9JiMp/H3/8MV27duWDDz4AYM+ePTz55JMAFBcX85e//IWFCxeSmJjIySefzODBgxuPLSsr47vvvmPGjBlMnDiROXPmMHXqVIYPH87ixYvJzMzk1ltvZcGCBaSmpnLaaafxzjvvcM455zSWsWDBAl577TUWL16M2+1m2LBhh03gADExMRQUFPD4449z9tlns2DBAtLS0ujZsye///3vKSws5LnnnmPevHkYYxg5ciQnnngiXq/3kHVNnjyZp556it69ezNv3jyuu+46vvzyy4Pqvv3225k+fTrJycl89dVXB20PB5GZwME6E+w/EXqdAt88Ct/+q0mzytX+Navs3gDrPocfPofCWVBfBQ4ndBlsNdk0JOzEzm0rP6MPXDDNOiP/+kGY9SjMfxqOuxaOuw7iUtoe+wHKq+pYWrSHLWVVxEcZ0uuK6Fj5AykVa0nYs5a4sjU4927+8YCYBMjsB/3Phk4DoVN/yOgLL54Ln94JPxkPMR1si69BVZ2b4vIaisurG29bG57vqSbOGcUzV+TTLS3e9rqD7UhnyoEyaNAg/vCHP3DrrbcyYcIExowZ07ht/vz5nHjiiaSlWScRF154IWvXrm3cftZZZyEiDBo0iE6dOjFo0CAABgwYQGFhIZs2bWLcuHE0dBW+7LLL+Oabb/ZL4LNmzeLcc88lPt76HU6cOPGIMTfsM2jQIAYMGECXLtZSpj169GDLli3Mnj2bc889lw4drM/keeedx6xZs/B6vc3WtW/fPr799lsuvPDCxjpqa2ubrfv+++/n/vvv54EHHuCJJ57gnnvuOWK8wRa5CbxBTAf46Z9hyM/hw1usppVF0+HMx6xE2xL11VA4B374DLPuM2T3egDqknIo63kBxRknsCt9BFmZHemeFk+HWJvetsx+cNELsH05zHzQOiuf+5R1Nn7cNeBKblVx1XUeVhTvYcmWcjZvXEvN1mWk7FvPTxxbGCxF9JatxEo9AG7jYKPpwhrTjdXekaw23VlturHTnUlcsZP4ndHE/xBFfKwQ7/yBY7iK2/b+ns+euY1Z2ZOJc0bhckYRFxNFXNN732NXk8fxMVEYA9v2VLO1MUHXNHlcTVlV/X4/i0OgU5KLrilxHJOdwsw1JUx+cQFvXns88TGR/7ENhZ/85CcsXLiQDz/8kDvuuINTTjmlxcfGxsYC4HA4Gh83PHe73S0e+t1aR6q3tbxeLykpKSxevLjFx1x22WWceeaZmsADKr03/OJtWDXD16xyGpX9L2bzsFsp9SZRVlXH3ho3e6vr2VtTj7N8A913f0efirn0rVlCLHXUEMNcb3++9lzB197BFJZ0gcbOLj9eqU9PiKFbWjzdfbemjzsnuXA4Wtle1nmg1ctm21LrjPzrv1oXPEfdACOvgdjEgw5xe7ys2VHBmvUb2LVhMe7tK0nd9wO9ZQsXSRGJUm3t6ITa+M6YjH7UpI1nZ3IfyhN/wu74XCo90dTVuelc5yG5zsPAOjfVdR6qGm9uquo8VNd5mF3Xmy+ixzK29BX+sXM4693p1NR72/a7AhJio8lKiaNriosh3VLomhLne2691inJhTPqx05SM9eWctVz87nlf0t54udDw7ZNMpwVFxeTlpbG5ZdfTkpKyn4XMIcPH85NN91EWVkZiYmJvPnmm41n2S0xYsQIfvvb37Jz505SU1N59dVXufHGG/fbZ+zYsUyaNInbbrsNt9vNe++9x29+8xu/fqYxY8YwadIkpkyZgjGGt99+mxdffBFjTLN1JSUlkZeXx//+9z8uvPBCjDEsXbp0v+YigHXr1tG7d28A3n333YPayMNFRCTwmnoPZVV17K6so6yynt1VdZRV1lHmu99dVW/dV9ZRVpVAdeX9TOYtfrXiDbqueI9X3BfxlmcMwx2rGedYwoSoJeSIdeGxOKorMxPPZH3y8exIy6dDhwS6uJz8Js5JcpyTJJeTpLhojIEtZVVs3l3Flt3W/cLNZby/dBse74/jmGKiHGSnxTUm9IYEn5kYS7TDgQg4RIhyCA6xRl45BOt5bC9k/FRihy0lce6juL68D++3/6ZmxPXs7HEehetXsXfTUqR0FR2r1tOLLQyQHy+GVccmU53al6isMZA10GqTz+xLbFwqAC4gGchq6y9iTw94Ip8Pen0CF7+E12uodXuprreSfU29h+o663l1vZX4q+vdVNd58RpD1xSXL0HHkeRq3RnbiT/J4NbxfXngo9X0/zqJ60/q1daf4qi1bNkybrnlFhwOB06nkyeffJKbb74ZgKysLP70pz8xYsQI0tLS6Nu3L8nJLf8G2KVLFx588EFOOumkxouYZ5999n77DBs2jIsvvpjBgweTmZnJ8OHD/f6Zhg0bxqRJkxgxYgRgXcQcOnQowCHrevnll7n22mu57777qK+v55JLLjkogU+ZMoU1a9bgcDjIyckJyx4oEOQ1MfPz801bVuS57a2lvDp/S7PbkuOcpHWIITXeuk+Jj/E9jyHHFDFy1YN0LPkWgyAYTHQckjcWev0Uev8U0nr49TPVe7xsK69h8+4qNu2u3C/Bb9pVRUVN20ePDpYfuCn6TU6KWrLf69USx674nrjT+5LY/RjScgcjnfpDh4zA9xL55hH48j64Ygb0ODGwdR3AGMNNry9mxpJipl6Rzyn9OgW1fn+sWrWKfv36hTqMw9q3bx8JCQm43W7OPfdcrr76as4999xQh3XUae6zIiILjDH5B+4bEQn8u/W7KNxVSWpjcnaS2iGGlDgn0VFHGItkjNVbpXgh5I6BnNHgdLXxJ2i9PVX1bN5dRem+Grxe8BiDMQavAW/Dvdf8+LjJdo/XepxWtpSsiqWk5Qyga++hRKflhK47X30N/HuEde3hN7OC3ge/pt7DBU99S+HOKt65fhS9Mg9uXgpHkZDAb775Zj7//HNqamo47bTTePzxx7WpKgTaXQJXYWbVe/D65XDGIzByctCrLy6vZuITs0l0OXnn+tEkxwXmApqdIiGBh8r111/PnDlz9nvtd7/7HVdddVWIIgqt1iTwiGgDV2Gm7wToMQ6+us/qFx7kEaVdU+J48vJj+fkzc/nda4t49srhRLX2wrEKG//+979DHULE0jUxVeuJwPiHoHaflcRDYHhuGvdMHMjXa0p55JM1IYlBqVDTBK7aJrOvNafLgudh+7KQhPDzkd25bGR3npq5nncXbw1JDEqFkiZw1XbjpoArBT661bpYHAJ/PmsAI3LT+OMbS1m+dU9IYlAqVDSBq7aLS4VT7rLmilnxVkhCiIl28J/Lh9GxQwyTpxewc1/zw6KVao80gSv/DLsCOh8Dn94FdYFf26M56QmxPH1FPrur6rjupYXUuds+QvRoo/OBN79Pa+YDnzFjBg8++OBhyy4uLuaCC6y13xcvXmzbFLuawJV/HFFwxkOwtwjm/CNkYQzMSuah849hfuFu7nlvRcjiiDRTp06lf//+Aa3jww8/JCUlJaB12O2WW25h6dKlLF68mAkTJnDvvfcect+JEycyZcqUw5bXtWtX3njjDcDeBK7dCJX/ckZZ3QnnPA5DLoPUnJCEcfaQLFZtq+Cpmevp3zWJy0aGJo4j+miK/Rd+Ow+CMw5/FqjzgQdmPvDnn3+egoKCxvcqKSmJgoICtm/fzsMPP8wFF1xAYWEhEyZMYOHChdx1111UV1cze/ZsbrvtNi6++OLD/t4ORxO4ssep98Kaj+DTO+DiF0MWxi2n92H19r38+d0V9M5MZEReGC6WESI6H3hw5gPftm0bs2fPZvXq1UycOLGx6aTh57n33nsbE76/NIEreyRnwwn/z+oXvvEbyBsbkjCiHMLjlwzl3H/P4dqXFjDjxhPISokLSSyHdIQz5UDR+cCDMx/4Oeecg8PhoH///uzYsaNFx7SVtoEr+4y6AVK6W90KPW2fxMtfyXFOnr4inzq3l9+8WEB1nSdksYSThvnABw0axB133HHYdt0D2T0vd6jqbTofeMNt1apVhz3msssu480332x1zGBNwBZImsCVfZxxcNr91rJsBdNCGkqvzAT+cckQVhTvZcpbSwP+hxQJiouLiY+P5/LLL+eWW25h4cKFjduGDx/OzJkzKSsrw+12typhgTUf+MyZM9m5cycej4dXX32VE0/cf7bKsWPH8s4771BdXU1FRQXvvfee3z/TmDFjeOedd6iqqqKyspK3336bMWPGHLKupvOBg5VglyxZclC569ata3xs93zgiYmJVFRU2FKWJnBlr35nWeuEfnU/VO0OaSin9OvEzaf14d3FxTz9zYaQxhIOli1bxogRIxgyZAj33HMPd9xxR+O2pvOBjx49mtzc3DbPBz548GCOPfbYw84HfsYZZ9g+H/jIkSMb5wM/XF0vv/wyzz77LIMHD2bAgAG8++67B5U7ZcoUBg4cyDHHHMOnn37K448/7nesDU466SRWrlzJkCFDeP311/0qS2cjVPYrWQVPjoZjJ8GEv4U0FGMM17+ykI+Xb2fBHaeS2iEmJHFEwmyEOh94eGjNbIR6Bq7sl9kPhv8KFjwXsnlSGogI5w7NxmugcFdlSGMJd3fffTdDhgxh4MCB5OXl7XcBUoUn7YWiAuOk22DZ/6wLmpM+CN0CFEB2qtULZWt5NUO7p4YsjnAXrFGRB4qU+cCfe+65g5pSRo8eHdLpcI+YwEVkGjABKDHGDPS9lga8DuQChcBFxpiywIWpIk5cKpxyJ7z/e1jxNgw8L2ShZDUk8LLqkMUAVnOOrnBzsEiZD/yqq64K+D+V1jZpt6QJ5Xlg/AGvTQG+MMb0Br7wPVdqf8OutEYIfnpnyOZJAUhyOUl0RbO1PHQJ3OVysWvXLu0Now7JGMOuXbtwuVq+5OMRz8CNMd+ISO4BL58NjPM9fgH4Gri1xbWqo4MjCs54GJ47w5on5aQ/hSyUrJQ4ikJ4Bp6dnU1RURGlpaUhi0GFP5fLRXZ2dov3b2sbeCdjzDbf4+3AIZcHF5HJwGSA7t27t7E6FbFyRsGA86x5Uo6/AVxJRz4mALJT49myO3TfApxOJ3l5eSGrX7VPfvdCMdZ3wkN+LzTGPG2MyTfG5DcMs1VHmWFXgLsGtswLWQjZqXFsLa/WJgzVrrQ1ge8QkS4AvvsS+0JS7U63EeCIthZ+CJGslDj21brZWx26If5K2a2tCXwGcKXv8ZXAwUOZlGoQ0wG6DoPCECZwX0+UovLQNaMoZbcjJnAReRX4DugjIkUi8kvgQeBUEVkH/NT3XKlDyxkFxQuhLjSDaRr6gofyQqZSdmtJL5RLD7HpFJtjUe1Z7glWT5Si76HHuKBX3zClbKj7gitlJx1Kr4Kj20gQR8iaUdI6xOByOkLaF1wpu2kCV8HhSrIWPw7RhUwRISslTs/AVbuiCVwFT+4JUFQA9TUhqT4rNV4vYqp2RRO4Cp6c0eCpha0LQlJ9dqqegav2RRO4Cp7uxwESsmaUrJQ4yqrqqarTvuCqfdAEroInPg06DQhZAs8Ok1kJlbKLJnAVXDmjYct88NQHveqGroTaF1y1F5rAVXDljIL6KiheFPSqs1PjASjSroSqndAEroIrZ7R1H4JmlMzEWJxRok0oqt3QBK6CKyED0vuEZECPwyF0SY7TwTyq3dAEroIvZxRsngue4PcGsQbzaF9w1T5oAlfBl3sC1FXA9qVBrzo7NbQr8yhlJ03gKvhyRln3m74NetVZqXGUVNRS6/YEvW6l7KYJXAVfUldIzQvJhcyGroTbykMznF8pO2kCV6GRO9o6A/d6g1ptw8IOeiFTtQeawFVo5JwANeVQsjKo1Wan+PqC64VM1Q5oAleh0dgOHtxmlM7JLhyiw+lV+6AJXIVGag4kd4PC2UGtNibaQackl47GVO2CJnAVOjm+dnBjglqtLuyg2gtN4Cp0ckZB1U7YuTao1WZpX3DVTmgCV6GTe4J1H+RmlOzUOLbvrcHtCW4PGKXspglchU5aD0joHPQBPVkp8Xi8hh0VtUGtVym7aQJXoSNiNaNsmhPUdvAsXdhBtROawFVo5Y6Gim2we0PQqmwYjblVFzhWEc6vBC4ivxeRFSKyXEReFRGXXYGpo0SOrx08iM0oDUurFe3WM3AV2dqcwEUkC/gtkG+MGQhEAZfYFZg6SmT0gfiOQR3Q43JGkZ4Qo8PpVcTztwklGogTkWggHij2PyR1VGloBw/yAg9ZKbqwg4p8bU7gxpitwKPAZmAbsMcY86ldgamjSM4JsGczlG8OWpVZqTqYR0U+f5pQUoGzgTygK9BBRC5vZr/JIlIgIgWlpaVtj1S1XyGYHzwrJY6i8mq83uCOAlXKTv40ofwU2GiMKTXG1ANvAaMO3MkY87QxJt8Yk5+RkeFHdard6jQAXMlBHdCTnRpPndvLzkrtC64ilz8JfDNwnIjEi4gApwCr7AlLHVUcUdB9VNDPwEH7gqvI5k8b+DzgDWAhsMxX1tM2xaWONjmjYPd6qNgelOp0YQfVHvjVC8UY82djTF9jzEBjzC+MMfp9VLVN7mjrPkjNKA0JXCe1UpFMR2Kq8NB5MMQkBq0ZJcnlJMkVrU0oKqJpAlfhISoauo8M6oCerNR4bUJREU0TuAofOaOgdDVU7gxKdbqwg4p0msBV+AjyvCjZqdZoTBPkFYGUsosmcBU+ug6F6LigNaNkp8axr9bNnur6oNSnlN00gavwER0D3YYHbV6Uhr7g2hNFRSpN4Cq85JwAO5ZDdVnAq9K+4CrSaQJX4SV3NGBg89yAV6WjMVWk0wSuwkvWsRAVE5QBPWkdYnA5HdqEoiKWJnAVXpxxkJUflJ4oIkJ2arwuraYiliZwFX5yR8O2JVBbEfCqdGEHFck0gavwkzMKjAc2zwt4Vbqwg4pkmsBV+Ok2EhzRQekPnpUSR1lVPZW17oDXpZTdNIGr8BPTwRrUE4QEnq1dCVUE0wSuwlPOKNi6EOoCe4GxMYFrM4qKQJrAVXjKOQG89VD0fUCryUqJB6BIz8BVBNIErsJT95EgjoA3o2QmxuKMEj0DVxFJE7gKT65k6Dwo4POiOBxCl+Q4isq0L7iKPJrAVfjKOcFqQnEHdqW+hmlllYo0msBV+ModDZ5a2LogoNXowg4qUmkCV+Gr+/HWfYCbUbJS4yipqKXW7QloPUrZTRO4Cl/xaZA5IOAXMhtmJSwurwloPUrZTRO4Cm+5o2HLfPAEbtWc7FSrK6E2o6hIowlchbecUVBfCcWLA1bFj6MxtSeKiiyawFV4yxlt3W8K3PzgnZNdOETPwFXk8SuBi0iKiLwhIqtFZJWIHG9XYEoBkJAJ6T8J6PzgzigHnZJcurCDijj+noE/DnxsjOkLDAZW+R+SUgfIGWUtseYNXC+R7NQ4HU6vIk6bE7iIJANjgWcBjDF1xphym+JS6ke5Y6B2r7XIQ4BoX3AVifw5A88DSoHnRGSRiEwVkQ42xaXUj3JPsO43fhOwKrJS49i+twa3xxuwOpSymz8JPBoYBjxpjBkKVAJTDtxJRCaLSIGIFJSWlvpRnTpqJXaG9D5QOCtgVWSlxOPxGnZUBHbYvlJ28ieBFwFFxpiGda/ewEro+zHGPG2MyTfG5GdkZPhRnTqq5Y2BTd8FrD94lq8rYdFu7UqoIkebE7gxZjuwRUT6+F46BVhpS1RKHSh3jNUffOvCgBSvK/OoSORvL5QbgZdFZCkwBPir3xEp1ZzcMdZ9YWDawRuG0+uFTBVJ/ErgxpjFvuaRY4wx5xhjyuwKTKn9dOgInQbCxsC0g7ucUaQnxOgZuIooOhJTRY68sbBlXsDmB89KidPBPCqiaAJXkSN3DLhrArZOZnZqvJ6Bq4iiCVxFjpxR1jqZAeoPnuVbmcfrNQEpXym7aQJXkSMuBboMDlg7eFZKHHVuLzsrtS+4igyawFVkyR1jNaHU2d9fu6EniraDq0ihCVxFlryx4K2HLXNtLzo7TbsSqsiiCVxFlu7HgSM6IM0ojX3B9UKmihCawFVkiU2ErsMCciEz0eUkyRWtZ+AqYmgCV5EnbywUL4LaCtuLztKuhCqCaAJXkSdvDBiPNbmVzazBPDqhlYoMmsBV5Ok2EqJiYONM24vOTrUWdjBG+4Kr8KcJXEUeZxxkjwjI/ODZqXFU1nnYUx2YaWuVspMmcBWZ8sbAtqVQbe/8adoXXEUSTeAqMuWNBQwUzrG12MaFHTSBqwigCVxFpqxjITrO9maU7NR4QPuCq8igCVxFpuhY6D7S9v7gqfFO4pxR2hdcRQRN4Cpy5Y2FkpVQudO2IkXENyuhdiVU4U8TuIpcuWOte5ubUXRhBxUpNIGryNV1KMQk2t6Mku2bF1ypcKcJXEWuqGjIOd72ia2yUuMor6qnstZta7lK2U0TuIpsuWNg1zrYu822InVWQhUpNIGryJZnfzt4dqrOC64igyZwFdk6DwJXsq3t4FkpVl9wndRKhTtN4CqyOaKsZhQbE3hmYizOKKFIm1BUmNMEriJf7hgo3wRlm2wpzuEQuqbEaROKCnt+J3ARiRKRRSLyvh0BKdVqeWOsexvbwbNStCuhCn92nIH/DlhlQzlKtU1GP4hPt7U7oQ7mUZHArwQuItnAz4Cp9oSjVBs4HJB7gnUGbtNCDNmp8ZRW1FJT77GlPKUCwd8z8H8AfwS8h9pBRCaLSIGIFJSWlvpZnVKHkDcG9m6F3RtsKa5hWtlte2psKU+pQGhzAheRCUCJMWbB4fYzxjxtjMk3xuRnZGS0tTqlDi/vROvept4ojYN5tBlFhTF/zsBHAxNFpBB4DThZRF6yJSqlWqtjL0jobNuFzOzGhR20L7gKX21O4MaY24wx2caYXOAS4EtjzOW2RaZUa4hYozI32tMO3jnZhUN0OL0Kb9oPXLUfeWOgsgRK1/hdlDPKQecklzahqLBmSwI3xnxtjJlgR1lKtVmurz+4Xe3gqXE6GlOFNT0DV+1Hai4kd4dC+y5k6hm4CmeawFX7IWI1oxTOBu8he7a2WFZqHNv31uD2+F+WUoGgCVy1L3ljoboMdiz3u6js1Hg8XsP2vdoXXIUnTeCqfcm1b14U7Quuwp0mcNW+JGdBWg9bLmQ2jMbUroQqXGkCV+1P3ljY9C14/FvTsuEMXCe1UuFKE7hqf3LHQO1e2L7Er2JczijSE2K1CUWFLU3gqv1pWCfTpmYUbUJR4UoTuGp/EjIho68t84Nn68IOKoxpAlftU+4Y2DwX3HV+FZOVag3m8XrtmWdcKTtpAlftU95YqK+E4oV+FZOVEkedx8vOfbU2BaaUfTSBq/Yp9wRA/G5GaZxWVptRVBjSBK7ap/g06DwQNs70q5jGvuDaE0WFIU3gqv3KHQtb5kN924fCN47G1DNwFYY0gav2K28MeGqh6Ps2F5HocpLkitaVeVRY0gSu2q+cUSAOv/uDZ6fGaxOKCkuawFX75UqGLkP8nthKB/OocKUJXLVveWOgqADqKttcRMPCDsaGtTaVspMmcNW+5Y0Fb701qKeNslPjqKzz6LzgKuxoAlftW7fjwBHtVzPKyX0ziXYIj3++zsbAlPKfJnDVvsUmQFa+Xxcye2QkcNXoXF4v2MKyoj02BqeUfzSBq/YvbwwUL4KyTW0u4sZTetOxQwx/nrFc28JV2NAErtq/ob+AmAR469dtXuQhyeXkj6f3ZeHmct5ZvNXmAJVqG03gqv1LzYEJf4ct82DmQ20u5oJjszkmO5kHPlzNvlr/VvtRyg6awNXRYdAFMORy+OaRNk9w5XAId08cQElFLf/+6gebA1Sq9dqcwEWkm4h8JSIrRWSFiPzOzsCUst0ZD0HHnvDWZKja3aYihnVP5bxhWTw7ayOFO9vet1wpO/hzBu4G/mCM6Q8cB1wvIv3tCUupAIhNgAumQdVOePcGaOPFyCnj++KMEu77YKXNASrVOm1O4MaYbcaYhb7HFcAqIMuuwJQKiC6D4af3wJoP4PupbSoiM8nFjaf05vNVJXy9psTmAJVqOVvawEUkFxgKzGtm22QRKRCRgtLSUjuqU8o/x10LvU+DT26H7cvbVMRVo3PJ7RjPve+vpM7ttTlApVrG7wQuIgnAm8BNxpi9B243xjxtjMk3xuRnZGT4W51S/hOBs/8DcSnwxtVQ1/qpYmOjo7jrrP5sKK1k+neFtoeoVEv4lcBFxImVvF82xrxlT0hKBUFCBpz7X9i5Fj65rU1FnNy3E+P6ZPD45+sordA1M1Xw+dMLRYBngVXGmL/ZF5JSQdLzJBj9O1jwPKx4p01F3DmhP9X1Hh75ZLWtoSnVEv6cgY8GfgGcLCKLfbczbYpLqeA4+Q7IOhbe+y2Ub2714T0zErj6hDz+r6CIJVvK7Y9PqcPwpxfKbGOMGGOOMcYM8d0+tDM4pQIuygnnPwteL7zZtqH2N57ci/SEWO5+bwVer86TooJHR2IqlZYHZ/0DtsyFbx5u9eGJLie3ju/Dos3lvL1I50lRwaMJXCnwDbW/zBpqXzi71YefPyybwd1SePBjnSdFBY8mcKUanPEwpOa1aai9wyHcfVZ/Sitq+deX4bXww4riPdzxzjLKKutCHYqymSZwpRo0DLXfVwIzbmz1UPuh3VM5f1g202ZvZGOYzJMy54edXPzfubw0dzPXvLRABx21M5rAlWqq6xA49R5Y/T4UPNvqw28d34fY6Cjuez/086S8u3grk56bT1ZKHHf8rB/zNu7WBSnaGU3gSh1o5LXQ61T4+E+wY0WrDs1McnHjyb34YnUJX4VwnpSpszbwu9cWM7R7Kv93zfH8akwPrj+pJ6/O38K0OYUhi0vZSxO4UgdyOOCcJ8GVDG/8stVD7a8anUdeegf+8l7w50nxeg33vb+S+z5YxZmDOjP96hEkxzkB+MOpfTh9QCfu/2BlSP+5KPtoAleqOQkZcN5/oXQVfHp7qw6NiXZw14T+bNhZyfPfbgxQgAerc3u56fXFTJ29kSuPz+Fflw7D5Yxq3O5wCH+/eAh9Oydx4yuLWLujImixqcDQBK7UofQ82RpqXzANVr7bqkNP6pvJSX0y+OcXP1BSUROgAH9UUVPPVc/PZ8aSYv44vg93TxxAlEMO2i8+JpqpV+YTFxPFL1/4nt3aMyWiaQJX6nBOugO6DoO3r4U3fwWLXoI9RS069M4J/al1e3j44zUBDbGkooaL/zuXeRt289iFg7luXC+sqYqa1zUljmeuyKdkby3XvKg9UyKZJnClDic6Bi6aDv3Ogo3fwLvXw98HwL+OhQ9uhlXvQ3V5s4f2yEjg6tF5vLGgiMUBmidlQ+k+zvvPtxTuqmTqlfmcf2x2i44b0i2FRy4czPzC3dz+9rLA9kwxBvZuC1z5RzEJZpei/Px8U1BQELT6lLKVMVCyCjZ8bd0KZ0N9JYjDOkvveRL0GAfZwyE6FrCaNk5+bCZdkl08fslQ8tI72BbOos1lXP389zhEmDZpOIO7pbS6jL99tpZ/frGO28/sx6/H9rAttka1+6yJwpa/CSf8Hk6+y7pIrFpFRBYYY/IPel0TuFJt5K6DrQVWMl//FWxdAMYDznjIGW0l8x7jeG97Kr99fTHGQJ9OiZw+sDPjB3SmX5fEwzZ1HM6Xq3dw/cuLyEiMZfrVI8ht4z8Gr9dw46uL+HD5NqZekc8p/Tq1qZxmla6F1y+HXeus96NwFgw831pMw+myr56jgCZwpQKtZg8UzvGdoX9lLRYB0CGDym5jmdnhDJ7fmsX3m8owBrqnxTN+YGdOH9CZod1ScDRz0bE5//f9Fm57exn9uyQxbdJwMhJj/Qq7us7DRf/9jg2l+3jzulH07ZzkV3kALH/LGs0a7bJGt+aNhTn/gM/vhu7HwyWvQHya//UcJTSBKxVse7bCxplWQl/7CdSUQ0Y/KgZfxceOE3l/9V6+Xb+Teo8hMzGW0wZ0YvyALozskYYz6uBmBmMMT3z5A499tpYxvdN56vJj6RAbbUuo2/fUMPGJ2TijHLx7w2jSE9r4T8FTD5/dBXP/A9kj4KIXIKnrj9uXvwlvXwMp3eGy/0FaAJpt2iFN4EqFUl2Vlby+fwa2LYHYJBh8KfsGT+KL0mQ+Xr6dr9eUUl3vITnOyU/7dWL8wM6M6Z2OyxmFx2v484zlvDR3M+cNzeLB848hJtretuSlReVc+NR3DMxK5pVfjyQ2OurIBzW1txj+d5U1Le/Ia+HUe62LwAfa9B28dilIFFz6GnQbbs8P0I5pAlcqHBgDRQUw/2lY8TZ466228hGTqc49lW/W7+aT5dv5fNUO9ta4iY+J4qQ+mVTVuflqTSnXnNiTW8f3aXnbuTFQttHq+tjtuOYTahPvLy3mhlcWcd6wLB67cHDL69n4zY8LRJ/9L6ut+3B2roOXL4CK7XDeM9B/YsvqOUppAlcq3OwrgYUvQMFzsHcrJHeD/Kth2BXUu9KYu2EXHy/fzicrdrCrspa7JvTnqtF5RyizFIoXWhdUG27VZda2Dhkw9HI4dhKk5h6yiMc/X8ffP1/LreP7cu24noevzxirbfuLe6FjL7j4Jcjo07Kfv3InvHqJ9Q/t9PvhuOugjRd12ztN4EqFK48b1nxoNa9s/AaiYmDAeTBiMmQfi8drqKpzk+hy7n9cXaXVHNM0WTes6ykOyOwPWcOsNT/j0mDJa7D2Iyvp9jrF+mfR+3SI2r8d3RjDb19bzPtLi3nq8mM5fUDn5uOuLod3roM1H1jxTvwnxCa27mevr4a3fg2r3oMRv4HxD4CjlU03RwFN4EpFgpLV8P1UWPIq1O2DrkOtRN5vIpQV7p+sS1aC8Y2iTO5uJevsfCthdxkMMc10LdyzFRZOt878K7ZBUhYMuxKG/WK/i4019R4u/u93rCvZx/+uOZ4BXZP3L2f7Mnj9F7BnC5x2P4z8TdvPnr1e+OxO+O4J6HMmnD+1+diPYprAlYokNXth6esw/xnYecBQfFeKlaQbb8MgIbN15XvcsPZja56X9V9YFxT7nGGdlfc4CRwOSvbWMPGJOTgE3rlhNJmJvr7bi1+B938Pcalw4QvQfWSzVdR7vNTUe6iu9+D1QkZibLPzszSa9zR8fKv1z+fS1yHRxj7pEU4TuFKRyBirWWXjTMjoZyXrtB72thXv3gALXoBFL0LVLqt9/NirYOjlLC93cuFT35Gb3oHROR04dfPfGbl7BqtdQ/hXxz9R6kmixu2hus7ju7eSdk29B7d3/9wSE+Ugp2M8uekd6JHegbyGW0YHMhJirQumaz6yLobGp8Plb7S8Pb2d0wSulDo8d63VFl0wDTbNsdri+01kfvo53D1zD4+YxxjABl5yns8r8b/AGRNDnNOByxlFnDMKV+PN0fg8zhmFK8Zq0y7aXcXGnZVs3FnJpl1V1Hl+nESrQ0wUeRkdyEtP4LjYTZy/5v8RbeqpOe8FEvqeHNz3weu1LirvXg+71lv/4Cq2QUoOdBpg3Tr2gijnkcuyiSZwpVTLlayGBc/B4lehdo/VxBKTAOc+BX3P9Lt4j9dQXF7dmNA37qxkw85KCndWUlRWRVdKec75MDmynXsd17M8fTwJsdG4nA5io6OIbbiPtv6BxEY7iHU6cPm2uZrs43I6iIly4HAIDd9bBC/OqhJcFRuJ3VOIa691i9lbSGzFJhye2sZYvVGxuOMzce7bhhi39ZojhurknlSm9qEquQ8VyX2oSOlDTWw6HiN4jcHrNXiMwWusKQuO79mRTkltm0JAE7hSqvXqqmDFW9ZF0+NvgI5H6FZog1q3hy27q9i8tZi+M6+ja3kB7yReyhLHADweD26PB7fHjdvjxe3x4PF48Hi8OPDiwCC+mwNjvSaGGNx0lxJyZTu5sp0c2UGc/DgXeq1xsslkssl0ZqPpTGHDvbcz20nF4CCGenpKMX1kC30dm+nru+8sZY3l7DYJrPZ2Z43pxirTnTXebqw12VTj4vmrhjOuTyuvVfgEJIGLyHjgcSAKmGqMefBw+2sCV0q1irvOmlNl6Wt+F+V1OKlJ6E51Yg5ViblUJ3SnOjGXysRcal2dMb5ZEo3x3XzHGWMwQJQIDgc4RIhyCA6xbrF15XQoX0P8njXE715NXNkaXGWrcbirreMR3Mk5eCY8jqv3uDbFfqgE3uaJFEQkCvg3cCpQBHwvIjOMMaFfjlsp1T5Ex1jNNsdfB/U1Vv92cVgXcUWaPG9yQ5rs43vNEY0jsTPxjijigY62BtkR6Ak0aVryeqG8EHasQHasxLljOc7kQ/Sn94M/M+GMAH4wxmwAEJHXgLMBTeBKKfuIWF0LI4nDYfUWSuthLQYSqGr8ODYL2NLkeZHvtf2IyGQRKRCRgtLSUj+qU0op1VTAl8YwxjxtjMk3xuRnZGQEujqllDpq+JPAtwLdmjzP9r2mlFIqCPxJ4N8DvUUkT0RigEuAGfaEpZRS6kjafBHTGOMWkRuAT7C6EU4zxqywLTKllFKH5dd6TMaYD4EPbYpFKaVUKwT8IqZSSqnA0ASulFIRKqhzoYhIKbCpjYenAzttDMduGp9/ND7/aHz+C+cYc4wxB/XDDmoC94eIFDQ3F0C40Pj8o/H5R+PzXyTEeCBtQlFKqQilCVwppSJUJCXwp0MdwBFofP7R+Pyj8fkvEmLcT8S0gSullNpfJJ2BK6WUakITuFJKRaiwS+AiMl5E1ojIDyIypZntsSLyum/7PBHJDWJs3UTkKxFZKSIrROR3zewzTkT2iMhi3+2uYMXnq79QRJb56j5o/Tqx/NP3/i0VkWFBjK1Pk/dlsYjsFZGbDtgnqO+fiEwTkRIRWd7ktTQR+UxE1vnuUw9x7JW+fdaJyJVBjO8REVnt+/29LSIphzj2sJ+FAMZ3t4hsbfI7bHYV5CP9rQcwvtebxFYoIosPcWzA3z+/GWPC5oY1KdZ6oAcQAywB+h+wz3XAU77HlwCvBzG+LsAw3+NEYG0z8Y0D3g/he1gIpB9m+5nAR4AAxwHzQvi73o41QCFk7x8wFhgGLG/y2sPAFN/jKcBDzRyXBmzw3af6HqcGKb7TgGjf44eai68ln4UAxnc3cHMLfv+H/VsPVHwHbH8MuCtU75+/t3A7A29cps0YUwc0LNPW1NnAC77HbwCniIgEIzhjzDZjzELf4wpgFc2sQhTmzgamG8tcIEVEuoQgjlOA9caYto7MtYUx5htg9wEvN/2MvQCc08yhpwOfGWN2G2PKgM+A8cGIzxjzqTHG7Xs6F2su/pA4xPvXEi35W/fb4eLz5Y2LgFftrjdYwi2Bt2SZtsZ9fB/iPdi9RmkL+JpuhgLzmtl8vIgsEZGPRGRAcCPDAJ+KyAIRmdzM9hYthRcEl3DoP5xQvn8AnYwx23yPtwOdmtknXN7Hq7G+UTXnSJ+FQLrB18Qz7RBNUOHw/o0Bdhhj1h1ieyjfvxYJtwQeEUQkAXgTuMkYs/eAzQuxmgUGA/8C3glyeCcYY4YBZwDXi8jYINd/RL4FQCYC/2tmc6jfv/0Y67t0WPa1FZHbATfw8iF2CdVn4UmsZdqHANuwminC0aUc/uw77P+Wwi2Bt2SZtsZ9RCQaSAZ2BSU6q04nVvJ+2Rjz1oHbjTF7jTH7fI8/BJwikh6s+IwxW333JcDbWF9VmwqHpfDOABYaY3YcuCHU75/PjoZmJd99STP7hPR9FJFJwATgMt8/mYO04LMQEMaYHcYYjzHGCzxziHpD/f5FA+cBrx9qn1C9f60Rbgm8Jcu0zQAarvhfAHx5qA+w3XxtZs8Cq4wxfzvEPp0b2uRFZATWexyUfzAi0kFEEhseY13sWn7AbjOAK3y9UY4D9jRpLgiWQ575hPL9a6LpZ+xK4N1m9vkEOE1EUn1NBKf5Xgs4ERkP/BGYaIypOsQ+LfksBCq+ptdUzj1EvaFekvGnwGpjTFFzG0P5/rVKqK+iHnjD6iWxFusK9e2+1+7F+rACuLC+ev8AzAd6BDG2E7C+Ti8FFvtuZwLXANf49rkBWIF1VX0uMCqI8fXw1bvEF0PD+9c0PgH+7Xt/lwH5Qf79dsBKyMlNXgvZ+4f1j2QbUI/VDvtLrGsqXwDrgM+BNN+++cDUJsde7fsc/gBcFcT4fsBqP274DDb0yuoKfHi4z0KQ4nvR99laipWUuxwYn+/5QX/rwYjP9/rzDZ+5JvsG/f3z96ZD6ZVSKkKFWxOKUkqpFtIErpRSEUoTuFJKRShN4EopFaE0gSulVITSBK6UUhFKE7hSSkWo/w8AIKF8RVIYQAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 测试误差\n",
    "for i, name in enumerate(name_l):\n",
    "    plt.plot(list(range(num_epochs)), test_l[i], label=name)\n",
    "plt.legend(loc = 1)\n",
    "plt.title('mse_test')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&emsp;&emsp;由此我们可知，Xavier初始化的作用核心在于保证各层梯度取值的平稳分布，从而确保各层模型学习的有效性，最终在模型结果的表现上。\n",
    "\n",
    "- Xavier初始化参数的模型学习效率更高、收敛速度更快"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&emsp;&emsp;当然，在一些极端情况下，Xavier初始化效果会更加明显。我们以四层sigmoid隐藏层的神经网络为例，观察Xavier初始化在规避梯度消失问题时的效果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x2d53b60efd0>"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[ 0.8858,  0.6076],\n",
       "        [ 0.7526,  0.3184],\n",
       "        [-0.2945,  0.9237],\n",
       "        [ 0.7596, -0.6387]], requires_grad=True)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[-0.0020, -0.2599, -0.8290,  0.0234],\n",
       "        [-0.5126, -0.1948, -0.2230,  0.7317],\n",
       "        [ 0.3355, -0.7767,  0.4933,  0.5650],\n",
       "        [ 0.3391,  0.3513,  0.3385,  0.0232]], requires_grad=True)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[-0.7911, -0.4265,  0.5541,  0.2841],\n",
       "        [ 0.6846,  0.2079,  0.8334, -0.3973],\n",
       "        [-0.4116,  0.3072, -0.4335,  0.3472],\n",
       "        [ 0.0940, -0.3414,  0.2058,  0.8155]], requires_grad=True)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[ 0.4660, -0.8182,  0.3571,  0.5147],\n",
       "        [ 0.6018, -0.2091,  0.7026,  0.6874],\n",
       "        [-0.0111, -0.1543,  0.6806,  0.1008],\n",
       "        [ 0.3697,  0.0858,  0.4200, -0.2422]], requires_grad=True)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[-0.6746,  0.9178,  0.5031,  0.2799]], requires_grad=True)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#————————————————————————————4个隐藏层的sigmoid的情况————————————————————————————\n",
    "\n",
    "# 设置随机数种子\n",
    "torch.manual_seed(24)  \n",
    "\n",
    "# 实例化模型\n",
    "sigmoid_model4 = Sigmoid_class4()                   # 保留原参数\n",
    "sigmoid_model4_init = Sigmoid_class4()              # 使用Xavier初始化参数\n",
    "\n",
    "# 修改init模型初始参数\n",
    "for m in sigmoid_model4_init.modules():\n",
    "    if isinstance(m, nn.Linear):\n",
    "        nn.init.xavier_uniform_(m.weight)\n",
    "        \n",
    "# 创建模型容器\n",
    "model_l = [sigmoid_model4, sigmoid_model4_init]           \n",
    "name_l = ['sigmoid_model4', 'sigmoid_model4_init']\n",
    "\n",
    "# 核心参数\n",
    "lr = 0.03\n",
    "num_epochs = 40\n",
    "\n",
    "# 模型训练\n",
    "train_l, test_l = model_comparison(model_l = model_l, \n",
    "                                   name_l = name_l, \n",
    "                                   train_data = train_loader,\n",
    "                                   test_data = test_loader,\n",
    "                                   num_epochs = num_epochs, \n",
    "                                   criterion = nn.MSELoss(), \n",
    "                                   optimizer = optim.SGD, \n",
    "                                   lr = lr, \n",
    "                                   cla = False, \n",
    "                                   eva = mse_cal)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d54a1b6d70>]"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d54a1b7040>]"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x2d5494898d0>"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'mse_train')"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAuWklEQVR4nO3deZxcVZ338c+vqmvppbo6vXenExJIJGQhCyFB2VEZZQnEB4RndEaYcXCQmQFGHBVRARHBQZRndGCQVUVgANkUERUEghDMSiCBhCWQrdNL0vtWy3n+OLc6naa70+mu7lu36/d+vep1q6pr+fVN+lunzj3nXDHGoJRSynt8bheglFJqZDTAlVLKozTAlVLKozTAlVLKozTAlVLKozTAlVLKozTAlVLKozTAlUojEZkqIm0i4ne7FjXxaYAr5RCRC0RkxWhewxjzgTGmwBiTSFddSg1GA1ypg6Ata5VJNMBVxhGRrSLyVRF5TUTaReROEakQkd+JSKuI/FFEJolIWER+KSKNItIkIn8VkQrnNaLO83aJyA4RuW6o8BWRI4DbgI86XSBNzv33iMitIvKUiLQDJ4vI6SKyVkRaRGSbiFzd53WmiYgRkRzn9p9F5Lsi8pJT+zMiUjqGu09lEQ1wlan+D/BJ4CPAmcDvgCuBMuz/238DvgBEgSlACfDPQKfz/HuAODADWAicCnxxsDczxmxynv+y0wVS1OfHfwt8D4gAK4B24O+BIuB04GIROXuI3+VvgQuBciAIXHGgX16p4dAAV5nqv4wxu40xO4AXgZXGmLXGmC7gUWwox7DBPcMYkzDGrDbGtDit8NOAy4wx7caYOuBHwPkjrOVxY8xLxpikMabLGPNnY8wG5/ZrwP3AiUM8/25jzGZjTCfwv8CCEdah1H5y3C5AqUHs7nO9c4DbBcAvsK3vB0SkCPgl8E3gECAA7BKR1HN8wLYR1rLf80RkKXADMBfbog4BDw3x/No+1zuc2pUaNW2BK88yxsSMMdcYY2YDHwPOwHZtbAO6gVJjTJFzKTTGzDnQSw7z/l8BTwBTjDFRbN+5fOhZSo0xDXDlWSJysojMcw5OtmC7VJLGmF3AM8APRaRQRHwicpiIDNXNAbaVXyMiwQM8LgLsMcZ0icgSbB+3UuNOA1x5WSXwMDa8NwHPY7tVwLbEg8BGYK/zuKoDvN6zwBtArYg0DPG4LwPXikgr8G1sv7ZS4070jDxKKeVN2gJXSimP0gBXWUVEbnMm6vS/3OZ2bUodLO1CUUopjxrXceClpaVm2rRp4/mWSinleatXr24wxpT1v39cA3zatGmsWrVqPN9SKaU8T0TeH+h+7QNXSimP0gBXSimP0gBXSimP0sWslBoHsViM7du309XV5XYpKoOFw2FqamoIBALDerwGuFLjYPv27UQiEaZNm0afFRKV6mWMobGxke3btzN9+vRhPUe7UJQaB11dXZSUlGh4q0GJCCUlJQf1LU0DXKlxouGtDuRg/494IsD/uHE3D7z6gdtlKKVURvFEH/iDq7bxwuZ6jp5ezGFlejITpZQCj7TAv7d8LrlBP1c8tJ5EUtduUSpdvvjFL7Jx48YxfY/TTjuNpqamD91/9dVXc9NNN43pex/Mew30mB/+8IeICA0NQy0P7x5PBHh5JMw1y+aw9oMm7njxXbfLUWrCuOOOO5g9e/aYvsdTTz1FUVHRmL7HWNi2bRvPPPMMU6dOdbuUQXmiCwVg2fxqfrehlh/+YTOnzCpnZkXE7ZKUGpFrnnyDjTtb0vqas6sL+c6ZQ5/ys729nc9+9rNs376dRCLBt771LW699VZuuukmFi9ezJ133smNN95IUVER8+fPJxQK8ZOf/IQLLriA3Nxc1q5dS11dHXfddRc///nPefnll1m6dCn33HMPAPfffz/XX389xhhOP/10brzxRmDfGkilpaV873vf495776W8vJwpU6Zw1FFHDVrvSSedxMKFC3nxxRdpb2/n5z//Od///vfZsGED5513Htdddx0AN998M3fddRdgv1FcdtllAIO+1zvvvMMll1xCfX09eXl5/OxnP2PWrFkfev/LL7+cH/zgB5x11lkH9W8xnjwT4CLCdcvncuqPXuCKh9bzyMUfI8fviS8QSmWEp59+murqan77298C0NzczK233grAzp07+e53v8uaNWuIRCKccsopzJ8/v/e5e/fu5eWXX+aJJ55g2bJlvPTSS9xxxx0cffTRrFu3jvLycr72ta+xevVqJk2axKmnnspjjz3G2Wef3fsaq1ev5oEHHmDdunXE43EWLVo0ZIADBINBVq1axS233MJZZ53F6tWrKS4u5rDDDuPyyy9n69at3H333axcuRJjDEuXLuXEE08kmUwO+l4XXXQRt912GzNnzmTlypV8+ctf5tlnn93vfR9//HEmT5683z7IRAcMcBG5C3u27zpjzFznvmLgQWAasBX4rDFm79iVaZUWhPjuWXO55Fdr+J8X3uWSk2eM9VsqlXYHaimPlXnz5vGVr3yFr33ta5xxxhkcf/zxvT979dVXOfHEEykuLgbg3HPPZfPmzb0/P/PMMxER5s2bR0VFBfPmzQNgzpw5bN26lffff5+TTjqJsjK74unnPvc5Xnjhhf0C/MUXX2T58uXk5eUBsGzZsgPWnHrMvHnzmDNnDlVV9rSmhx56KNu2bWPFihUsX76c/Px8AD7zmc/w4osvkkwmB3yvtrY2/vKXv3Duuef2vkd3d/d+79nR0cH111/PM888c8D63DacJuw9wKf63fd14E/GmJnAn5zb4+L0I6s4/cgqfvzHzbxZm96voUpNZB/5yEdYs2YN8+bN46qrruLaa68d9nNDoRAAPp+v93rqdjweT3utY/W+yWSSoqIi1q1b13vZtGnTfo955513eO+995g/fz7Tpk1j+/btLFq0iNra2tH9MmPggAFujHkB2NPv7rOAe53r9wJnp7esoV27bA6F4QBXPLSeWCI5nm+tlGft3LmTvLw8Pv/5z/PVr36VNWvW9P7s6KOP5vnnn2fv3r3E43EeeeSRg3rtJUuW8Pzzz9PQ0EAikeD+++/nxBNP3O8xJ5xwAo899hidnZ20trby5JNPjvp3Ov7443nsscfo6Oigvb2dRx99lOOPP37Q9yosLGT69Ok89NBDgJ2+vn79+v1ec968edTV1bF161a2bt1KTU0Na9asobKyctT1pttI+8ArjDG7nOu1QMVgDxSRi4CLgLQdzS0pCHHd2XO5+L413Pbnd/jXj89My+sqNZFt2LCBr371q/h8PgKBALfeeitXXHEFAJMnT+bKK69kyZIlFBcXM2vWLKLR6LBfu6qqihtuuIGTTz659yBm/4N/ixYt4rzzzmP+/PmUl5dz9NFHj/p3WrRoERdccAFLliwB7EHMhQsXAgz6Xvfddx8XX3wx1113HbFYjPPPPz/j+7oHM6xzYorINOA3ffrAm4wxRX1+vtcYM+lAr7N48WKTzjPy/Nv9a/nd67t4/JLjmF1dmLbXVSrdNm3axBFHHOF2GUNqa2ujoKCAeDzO8uXL+Yd/+AeWL1/udllZZ6D/KyKy2hizuP9jRzqMY7eIVDkvXAXUjfB1RuWaZXOI5ga54qH19MQ/3JWSTBq64wk6exIuVDcBdLdBUruossXVV1/NggULmDt3LtOnT9/vAKTKTCPtQnkC+AJwg7N9PG0VHYRJ+UGuXz6Xi36xmmO+/ycAYomkczH7zdo8sibK2Qsmc+b8asoiocFecn/GwM610LAFSmdA2SwI5o/Fr5IZulrg/b/Aey/Yy+4NEIpC9QKYfBRMXmS3hdVuV6rGwHjNiuzvkksu4aWXXtrvvksvvZQLL7zQlXq85IBdKCJyP3ASUArsBr4DPAb8LzAVeB87jLD/gc4PSXcXSsovX3mf13c0k+MXAn4fQb+v93pIEoQ7dvHb9+Ks2hnD7xOOm1HK8oWTOXVOBXnBfp9hySRsfxU2PgGbnoTmvotoCUw6BMpn2zAvnw3lR0DJDAiE0/57kUxATzuEIjAWK9n1tMP2v8K7z9vA3rkWTAL8IZi6FKZ+DNrrYMdq2P0GJJ2j/pEqqF5kA73maBvqocxYo6azJ0Fdaxd72nsoygtSFglREHJ/uoMXulBUZjiYLpRh9YGny1gFOACxTttSbtgM9W9C/Vv2sued3uCJRWrY5p/CK63lrO2qYJt/KtNmLeKUeYdQ3byGkg9+T8m2Zwh21pH0BdlbdSwNUz5FR+k8ijq3Udj6NnlNmwnueQvfnreRVKAhtlU6aZpzmU6y6BBi0an0RKZi/CECiU78iU4CiQ6kpwNi7dDTYUO0owHadkNbvQ3MNufS0QAmCeEiTNnhUDKTZMlMkqUfIVk8k2T0EIzPh98nBHw+fL4+IW8MJGLQ0Qh7t/a5vLfvettup3w/1CyG6SfYS80SCIQxxuxb3jLWBbUbbJjvXGO3jW87z/dB+RyYcrR9bs3RUHJYWj504okkLV1x9nb00NQRo7mzh73tMfZ29FDf1k19Sze7W7vY3dLN7pYuWrs+PLQsL+inPBKiLBKiPBKmzLlenB9kUl6Q4vwgxfkBJuUFKcoL4vftX7ftikvSGUvQGUvQFUsgQG7QT27ATzjgJ5Tj228pUGMMrd1xdjd3UdvSRW57LVMOnUk8kSSeNPhE8PuEHJ/d9r3u8wkY6PuXaf9MzX73Sd9rsu+2CPhEnIsuY+s1Ey/AH/w8vPMc9n+pz/5PFZ9zW+y2o5He//Lig+JDofRwKDvcXm+rdUL9TUz9ZiSxb/B+hwmRJ910mBB/Ts7n6cQSnk0uoI28QUsKSpzZwTrm5WznEKmlMlFLtallstlNhRz8nKYuE6CRIhqJ0kCUehOlwURpM2EmSx0zZCeHyU7KpLn3Od0mh1pTjF+SBIkTIE6QmN3Kh/v9kwi1poTtlPMBFWw35bxhprHKzKLNhDEGksaGROq/hQjk+IQcn89u/YLf5yPgF6K0c0TyLeYk3mKu2cwcs4UIHQDsNRE2Mp1GimiSCM0SpVkKafYV0ipRmn1R6n3ldEsAY3Au+97bYOjoSQwYyCkBv1AeCVNRGKKiMEx5JER5YZiKwjCT8gI0d8aob+2mzrnUt3Y52+5BX1cEinIDhHL8dDnHT7oHOL4y0PNyA/sCvamjh/Y+x15+tqyKiqmHOiHtwxhDPGlIjsPfXyrIfSKICIL9SzHOp4RxbmMAgWLnm0v/DzI1PiZegK++17asTdL5605i/+c5tzFQUGHDuvRw2/rLGaKfO5mwLdD6t4jv3kTjrq3srTiG5uoTMYFc+/kggjitmkTS0N4Tp607QVtXnPbuOK3d8d7rsWSSUI6vt/smT3ooiddSEttJcfdOfCZOt+TS7QvTJX0uhOmUMB05RcT8BYhv/z80n/P55O+9LeTGWyjuep/izq0Ud7xHfk89cfw2viWHuASIESBGDnHJocNXQFOomr2hGlpCFSR9od7fS/r+UQv2/bDvS+oP3RhiSXs8IZZIOltDItmnJSm21ZgjCSq7P2Bqx+tMaX+d8s53yYvvJS/eTDDZ+aF/hvdy5/D/pv23bTmmanHqEoTcoJ+ivABFuQGK8oL2el7QuR0gmhsYceuyK5Zgb0cPe9pti35PRw97253bHT109iTIC9owDgf8fVrbPsIBP2C7a3pb5X2ud/YkieYGqIzaD5bKwjB5HbXMmT17/29J2A/MhLN/48l911P7IUWcndT32abvtT4t9tQHsb0415P7rju7u/ffOLX/BYglDC1dMXJ8PioK7bcUbcGPr4kX4Mr7Yp32W1J7g+0a+utd8M6z8M1dY9O/n2G81Afe0RNnV1MX7T1xQjl+qqJhIuEcDfJxMh7DCJU6OIFciNbYES0zPgHTj4d4pw105ZqB1gPPC+ZwaFk+h5TYEVdbG9t5r6Gdzp6RTZmf6OuBP/HEE9xwww1DvvbOnTs555xzAFi3bh1PPfXUMCsfmvuH51V2ik6x2+ZtUFDmbi1Z7I477hjwfhEhmhsgEs5hT3sPdS1dbKlrozg/yOSi3INqjacrrMbbcNcDX7Zs2QEX5qqurubhhx8GbICvWrWK0047bdQ1aoArdxT1CfDJi9ytZbz97ut2RE86Vc6DTw/dChzpeuBX/MuXCIfDvLpqDXX1ddx155088Kv7dD1wxz333MOqVat6104vLCxk1apV1NbW8oMf/IBzzjmHrVu3csYZZ7BmzRq+/e1v09nZyYoVK/jGN77BeeedN+TrD0UDXLkj1QJv2uZuHVlkNOuBNzU18fyKl7jrvoc45zPLdT3wIezatYsVK1bw5ptvsmzZst6uk9Tvc+211/YG/mhpgCt35E6CYIFtgWebA7SUx8po1wMP+n3MnDWbsvJyXQ98CGeffTY+n4/Zs2eze/fuEb3GcGmAK3eI2IOazdvdriRrpNYDf+qpp7jqqqv4+Mc/PuznhkIhAn4f4vMRCH54Xe5AIDAWJY/peuCD6bseONC7Hvirr746rCVl+9Y51qP8dBSKck90CjR9cODHqbQY7XrgPp+Q42PfLK8+dD3w4YtEIrS2tqbltTTAlXuKpmRnF4pLNmzYwJIlS1iwYAHXXHMNV111Ve/P+q4HfuyxxzJt2rQB1wP3+3wM1Kbsux74/PnzOeqoo4ZcD/zTn/502tcDX7p0ae964EO913333cedd97J/PnzmTNnDo8/Pr5r8Z188sls3LiRBQsW8OCDD47qtXQij3LPizfDn66Bb+zImMWwxooXJvIMZz3w9xraiSeSzKyIuFTlxKcTeZQ3FDnja7UVnhGGsx54wC/EEuPX6FND04OYyj3RGrtt3m6X5VWuGs6syIDfRzyZJGnsOjjp4JX1wO+++25uueWW/e479thj+elPf+pSRRrgyk29Y8Gz40DmfsvzelTAb7+0xxNJgjn+tLymmwF4MC688MIx/1A52C5t7UJR7olUgi8nK7pQwuEwjY2NYz6sbKwF/PYDSLtR0s8YQ2NjI+Hw8E8Ooy1w5R6fHwonZ8VszJqaGrZv3059fb3bpYxKLJFkd0s3scYgecH0tMDVPuFwmJqammE/XgNcuSuaHUMJA4EA06dPd7uMUWvpirHs6me48rRZXHTCYW6Xk/W0C0W5q2iKzsb0kEgoh/ygn9rm7gM/WI05DXDlrugUaN1lz9+pMp6IUBENU9vy4TMsqfGnAa7cVTTFnhqvZYfblahhqoqG2dXc5XYZCg1w5TZdVtZzKgtz2a0BnhE0wJW7es/Mo/3gXlEZDbG7tZtEUocSuk0DXLmrdzamtsC9ojKaSyJpaGjTA5lu0wBX7gqEIb88a2ZjTgRVhXaiSa12o7hOA1y5T5eV9ZTKqA1wPZDpPg1w5b5ojR7E9JBUgNc261BCt2mAK/dFnck8Hl8nJFsU5wUJ+n3UtmgfuNs0wJX7iqZCohvavb1OSLbw+YTywpC2wDPAqAJcRC4XkTdE5HURuV9Ehr+MllIpOhbcc3QyT2YYcYCLyGTg34DFxpi5gB84P12FqSxSlBoLriNRvKIymsvuFg1wt422CyUHyBWRHCAP2Dn6klTWSY0F1xa4Z1QWhtjV3OX59c29bsQBbozZAdwEfADsApqNMc/0f5yIXCQiq0RkldfXQlZjJFwEwYjOxvSQymgu3fEkTR26CJmbRtOFMgk4C5gOVAP5IvL5/o8zxtxujFlsjFlcVlY28krVxCWiY8E9pio1lFC7UVw1mi6UTwDvGWPqjTEx4NfAx9JTlso60SnaheIhFTobMyOMJsA/AI4RkTyxZ2r9OLApPWWprFM0RQ9iekiVzsbMCKPpA18JPAysATY4r3V7mupS2SZaA13N0NXidiVqGMoiIXyiXShuG9U5MY0x3wG+k6ZaVDbru6xseLa7tagDCvh9lBboZB636UxMlRmKptqtHsj0DJ3M4z4NcJUZemdjaj+4V1RGwzqZx2Ua4CozFFSAL6AtcA+pLNQWuNs0wFVm8PkgOlkn83hIZTSX1q447d1xt0vJWhrgKnPoWHBP0ck87tMAV5mjaKp2oXiITuZxnwa4yhzRKdBaC/EetytRw6CTedynAa4yR7QGMNCi/eBekDq1mo5EcY8GuMocRX0m86iMFw74KcoLsEsn87hGA1xlDj0zj+dUFoa1D9xFGuAqc6RO7KAHMj2jKhrWUSgu0gBXmSMnZCf0aAvcMyqj2gJ3kwa4yixRXVbWSyoLc2lo66E7nnC7lKykAa4yS9EUPYjpIamhhHUt3S5Xkp00wFVmiToBnky6XYkahgqdjekqDXCVWYqmQqIH2uvcrkQNg07mcZcGuMosqZEoeiDTE3on82iAu0IDXGWW3jPzaIB7QSSUQ17Qry1wl2iAq8xSpAHuJSJihxK26GxMN2iAq8wSjkIoql0oHlKlY8FdowGuMk/RFG2Be0iFTqd3jQa4yjzRGm2Be0hVNMzu1m4SSeN2KVlHA1xlnqhO5vGSymguiaShsU0n84w3DXCVeYqmQHczdDW7XYkahspCHQvuFg1wlXl0WVlP0ck87tEAV5lHx4J7ip6Zxz0a4CrzFGkL3EuK84IE/KItcBdogKvMk18O/qC2wD3C5xNnKKFO5hlvGuAq8/h8MGk6vPucnqHeI6qiYW2Bu2BUAS4iRSLysIi8KSKbROSj6SpMZblTvgm1G+BP17hdiRqGisKw9oG7YLQt8FuAp40xs4D5wKbRl6QUMPssOPqf4OWfwFu/c7sadQCpFrgxOplnPI04wEUkCpwA3AlgjOkxxjSlqS6l4NTroHIePHaxTuzJcJXRXLrjSZo6Ym6XklVG0wKfDtQDd4vIWhG5Q0Ty+z9IRC4SkVUisqq+vn4Ub6eyTiAM594LiRg8/I+QiLtdkRpEajKPnplnfI0mwHOARcCtxpiFQDvw9f4PMsbcboxZbIxZXFZWNoq3U1mp5DA48xbY9go89z23q1GDSI0F10WtxtdoAnw7sN0Ys9K5/TA20JVKr3nnwKK/hxU3w9t/dLsaNQCdjemOEQe4MaYW2CYihzt3fRzYmJaqlOrvUzdC+Wz49ZegtdbtalQ/ZZEQItqFMt5GOwrlX4H7ROQ1YAFw/agrUmogwTw49x6IdcAjX4Rkwu2KVB8Bv4+ygpBO5hlnowpwY8w6p3/7SGPM2caYvekqTKkPKTscTv8hbH0RXvhPt6tR/ehknvGnMzGVtyz4W5j/f+HPN8AHKw/8eDVuKgrDbN/bSVdMvx2NFw1w5T2n3QTigy3PuF2J6uPwygjvNbSz+Lo/8u8PruO5t+qIJZJulzWh5bhdgFIHLVQAkUpo3eV2JaqPyz7xEZZOL+GJ9Tt4+vVafr12B5PyAnx6XhXL5lezZFoxPp+4XeaEogGuvClSBS073K5C9eH3CcfNLOW4maV89+y5vLC5gSfX7+TRNTv41coPqCgMsfiQYoryAvaSGySaF6AoN0BRXpBobgC/TzDGkDCGRNJgDCSS9rYxkB/yEwkHiIRzKAjmpO0DwTiv77UPGA1w5U2FVVC/2e0q1CBCOX4+ObuCT86uoKMnzp821fHk+p1sqm2huSNGU2ds1CdBFoGCYA6RcA6RcICCcA65AT/hgI9QwE84x14PO/cBNHfGaO6MO9sYzR09NHfGaOmKk0ga/D4h6PcR8AvBHD+hnNR1H7nBHApCfvKDORSEcsh3LgUhP7nBHJJJQ08iSXc8SU/qkkj0Xr/ytCMod2asposGuPKmSDW8+7zbVahhyAvmcOb8as6cX917nzGGtu44TR02SJs6YjR19vSGqF8EEbHXfSAiCNDRk6C1K0ZrV5wWJ3hbu+K0dsVo647T0RNnT3uSrniC7liSrljCXuK2Lz6aGyCaG6Aw17b8DynO670v4PfRk0gQSxgnfG3wxhJJumNJOmIJ2rvjNLZ10NYdp707Tlt3nFjiwx9EAb/9IAjm7Lt09KT/4K4GuPKmwiroboHuNtsnrjxFRJyukABTxuk9jTGIpL+LpDueoKM7gT8V2n7fuHXFaIArb4o4rbnWXRCa6W4tyhPGIrzBdheFcvxj8toHosMIlTcVVtlty05361DKRRrgypv6tsCVylIa4MqbtAWulAa48qhgPoSj2gJXWU0DXHlXpFpb4CqraYAr7yqs0gBXWU0DXHlXpFq7UFRW0wBX3lVYBW279WTHKmtpgCvvilSBSUJ7nduVKOUKDXDlXYXOWPAW7UZR2UkDXHlXxBkL3qoHMlV20gBX3qUtcJXlNMCVd+WVgi+gLXCVtTTAlXf5fM6ZeTTAVXbSAFfeppN5VBbTAFfeFqnSyTwqa2mAK28rrLYHMc3ozq+olBdpgCtvi1RBrN2eXk2pLKMBrrxNhxKqLKYBrrxNJ/OoLKYBrryt98w82gJX2WfUAS4ifhFZKyK/SUdBSh2U1LkxdSihykLpaIFfCmxKw+sodfACYcgt1i4UlZVGFeAiUgOcDtyRnnKUGoHUUEKlssxoW+A/Bv4DSA72ABG5SERWiciq+vr6Ub6dUgOIVGkLXGWlEQe4iJwB1BljVg/1OGPM7caYxcaYxWVlZSN9O6UGV1ilLXCVlUbTAj8WWCYiW4EHgFNE5JdpqUqpgxGphvZ6SMTcrkSpcTXiADfGfMMYU2OMmQacDzxrjPl82ipTargKqwADrbVuV6LUuNJx4Mr7UkMJdVErlWVy0vEixpg/A39Ox2spddB6J/PogUyVXbQFrrxPJ/OoLKUBrrwvrxj8IR1KqLKOBrjyPhEdSqiykga4mhgi1XoQU2UdDXA1Mei5MVUW0gBXE0Pq3Jh6ajWVRTTA1cRQWA3xLujc63YlSo0bDXA1MfSemUf7wVX20ABXE4OeG1NlIQ1wNTGkWuAtO9ytQ6lxpAGuJgbtQlFZSANcTQw5Qcgv06GEKqtogKuJIzWUUKksoQGuJg49N6bKMhrgauLQc2OqLKMBriaOwmroaIR4t9uVKDUuNMDVxKEjUVSW0QBXE0fvmXk0wFU/dW/ChofdriLtNMDVxNF7Zh6dzKP6+ct/waNfgkTM7UrSSgNcTRyF2oWiBtGwGZJx2Pu+25WklQa4mjjCRRDI0y4U9WGNW/bfThAa4GriENGhhOrD2hv3LTPcoAGuVObSyTyqv76tbm2BK5XBtAWu+ku1uiPV0PC2u7WkmQa4mlgKq6C1Vk+tpvZp3AL+IBx2irbAlcpokWpI9NgZmUqBbXUXHwplh0N7PXQ2uV1R2miAq4mldzKPdqMoR+MWKJkBpTOd2xOnG0UDXE0svZN5NMAVkIjDnvdseJc4AT6BRqJogKuJpXcyjwa4Apreh2TMhvekQ8CXM6H6wUcc4CIyRUSeE5GNIvKGiFyazsKUGpGCChCfDiVUVqq1XToT/AGYNH1CtcBzRvHcOPAVY8waEYkAq0XkD8aYjWmqTamD5w9Afrm2wJWVam2XzLDb0pnaBw5gjNlljFnjXG8FNgGT01WYUiNWWKUtcGU1bIG8EsgrtrdLZkDjO5BMuFtXmqSlD1xEpgELgZUD/OwiEVklIqvq6+vT8XZKDS1SrQtaKavx7X0HL8G2wBPd0PSBezWl0agDXEQKgEeAy4wxLf1/boy53Riz2BizuKysbLRvp9SBFVbpKBRlNWyG0hn7bpdMrKGEowpwEQlgw/s+Y8yv01OSUqMUqYKuJoh1ul2JclNnk524078FDhPmQOZoRqEIcCewyRhzc/pKUmqUCnUsuGJfK7u0T4DnldhlhyfIUMLRjEI5Fvg7YIOIrHPuu9IY89Soq1JqNAqdY+l3/Q1MmgbRGucy1W6Lptj7QxE3q1RjLdXK7tsCF7GBPkFa4CMOcGPMCkDSWItS6TH1o/CJq20LrGkb7HoN3nzKHrxKCUbg0nWQX+pWlWqsNW4B8dsP675KZsK7z7lSUrqNpgWuVGbKCcJxl+9/nzG2P7R5G9RugCcvhdd/DUsvcqdGNfYattjwzgnuf3/pDFj/K+hu9fy3MJ1Kr7KDCBSUw+Sj4KgLoGIevPaA21WpsdT49v793ykTaCSKBrjKTvPPgx2rJ0xfqOonmbATdkpmfPhnvSNRNMCV8qZ559o1U1570O1K1Fho3maPeQzUAi8+1P7bT4CRKBrgKjtFKuHQk2yAJ5NuV6PSLdW6LhkgwHNCUDR1Qnz70gBX2evI8+2U6m2vuF2JSrfGPqsQDqRkprbAlfK0I86AQD6s14OZE07DFghFIX+Q5TtKZzqLWnn725cGuMpewXw44kx44zGIdbldjUqnxi12uKAMMlWlZAbEOjy/7LAGuMpu88+D7mbY/LTblah0anh74P7vlAmyJooGuMpu00+0i1/paJSJo7vNtqxLBxhCmDJBxoJrgKvs5vPDvHNgyzPQ3uh2NSodGocYgZISqYRggbbAlfK8I8+HZBze0BWRJ4SBViHsT8Q5O48GuFLeVjkXKubqaJSJomELIHbCzlBKZ3p+NqYGuFIAR54HO1Z5/g9aYVvVRVMgkDv040pm2hmbHj7xhwa4UqBT6yeShi1D93+nlM4AjB0P7lEa4EqBPY/m9BNtgBvjdjVqpIwTyEP1f6f0jkTxbj+4BrhSKfPPh6b34QOdWu9ZLTsh1j7wKoT9lRxmtx7uNtMAVypl1hkQyNN1wr3sQGug9BXMh8IabYErNSGECpyp9Y/q1HqvGug8mEMpneHpseAa4Er1deR50NUMW37vdiVqJBrftguUFVYP7/ElM+1zPHrcQwNcqb4OPQkKKmH1PZ5fqS4rNWyxfduDLWLVX+lM6G6BtrqxrWuMaIAr1ZfPD8f8M7zzLDz4OehqcbsidTAatwyv/zsldbDTo/3gGuBK9XfsZfDp/4TNv4c7Pwl73nW7IjUcsU5o2jb8/m/w/KqEGuBK9ScCSy+Cv3sU2nbD7SfDO8+5XZU6kD3vAubgWuCFNZCTqwGu1IRz6InwT8/Z5WZ/+Rl45dbhHeyKdXr2oJin9Y5AGcYY8BSfz/aZe7QLJcftApTKaMXT4Yt/gEf/GZ7+OtS+DmfcbE+Mm9K8Az54Gd7/i93WbYTqRXDKVXDYKcM/oKZGp3EEAZ56/K716a9nHGiAK3UgoQh89hfw/A3w/I3Q8BYs+BxsWwnvv2RPjAx2fekpS2HmqfD6r22r/ZDj4OPfgqnHuPs7ZIOGt6Fwsh3PfzBKPwKbnoB49/4fzB6gAa7UcPh8cPKVUD4bHrsYfnMZ5JXCIR+FpRfbbcU88Dt/UidfCavvhRf+E+76Gxvqp1wFVfNd/TUmtMYtB9/6BttnbpKw5z0on5X+usaQBrhSB2PO2bY13dVi//AH6x7JCdkDoQs/B6/eDit+DP9zAsw+C078mm3Vt9XZg6Rtu6HV2bbVQbwLqhfC1I/ClKMhHB3P39CbjLF94Ed+9uCf23coYTYFuIh8CrgF8AN3GGNuSEtVSmWySKW9DEcwH467HI66EF7+Kbzy37Dx8YEfm1cCBRUgfljxIzA3AWJb/VOXwpRj7DY6BTr22MBvr4O2eme7214HG/q5RRAucrbRfdfzSiG/1I55nyja6uyEnIMZQpiSCvD+I1EScWjZDnu3wt73Ib8MJi8a/r89QOde2LkOdq6BYy6BQPjg6xvCiANcRPzAT4FPAtuBv4rIE8aYjekqTqkJI7cITvkmLP0SvP6IPdlAQQUUlNttfhn4A/se391mTzDxwUrY9gq89hCsusv5oQADjHLxh+zriUBnM3Q3D16P+GyQF5TbS76zzSux3wC6Wmwgdrfsu97VYn/mD9r6c8J9tmE7HM+XA4mefpeY3ab6mHOL7fvk9d+W2G8mwXx7PCGYb1+7/7ccY6CnDTqbbEB2NcGO1fZnQ53IeDDhQjv7dtOT9gQPe7fa7pTmbfZUe/1FquxB6uqF+y75JfbfbNd62LnWBvbOtfvPIZjxibR3oY2mBb4EeNsY8y6AiDwAnAVogCs1mPxSG+IHEiqw0/oPPcneTibs6JYPXoHW2g8Hb0E5hAr3D7tkwgZvZ5MNudS2vWFf9017vd02bLH3JbrtcwP5NthChXYbjtqWfyDPPibWBfFOu+1o2Hc7mbAB7w/aD6Sc0L7AD0dtiDdvt0HX0bjv/QYjPltLMN9+SHS32d9hoGD1Beyp8UZi8lHw1m9teE+aZkN57mfs9UnT7Rl+WmudcF4LO9bYx6fkl9t9mfpgLayB6gWw8PP2taoW2A+qNBtNgE8GtvW5vR1Y2v9BInIRcBHA1KlTR/F2SmUxnx8q59nLwTwnd5K9DIcx0NNuW73+cTg8ZgzEOmyQd+yx2+5We19Pu21l97Tvu8Q6bZDnTrLfaHIn7esWyp0EkWrbEh6Jz/7criM+1PGGSdP2H03U1Qy7XrOt7bo3oWiq7WKpXmg/UMfBmP8rGWNuB24HWLx4sc5uUCpTiRz8ELzRvl/QaV0Xudy48+eA/yAPFoejMP14e3HJaGZi7gCm9Lld49ynlFJqHIwmwP8KzBSR6SISBM4HnkhPWUoppQ5kxF0oxpi4iPwL8HvsMMK7jDFvpK0ypZRSQxpVH7gx5ingqTTVopRS6iDoaoRKKeVRGuBKKeVRGuBKKeVRGuBKKeVRYsbxzCEiUg+8P8KnlwINaSwnnbS2kdHaRkZrGxkv13aIMaas/53jGuCjISKrjDGL3a5jIFrbyGhtI6O1jcxErE27UJRSyqM0wJVSyqO8FOC3u13AELS2kdHaRkZrG5kJV5tn+sCVUkrtz0stcKWUUn1ogCullEd5IsBF5FMi8paIvC0iX3e7nr5EZKuIbBCRdSKyyuVa7hKROhF5vc99xSLyBxHZ4myHeXqWcantahHZ4ey7dSJymku1TRGR50Rko4i8ISKXOve7vu+GqM31fSciYRF5VUTWO7Vd49w/XURWOn+vDzrLTWdKbfeIyHt99tuC8a7NqcMvImtF5DfO7ZHtM2NMRl+wS9W+AxwKBIH1wGy36+pT31ag1O06nFpOABYBr/e57wfA153rXwduzKDargauyID9VgUscq5HgM3A7EzYd0PU5vq+w55ducC5HgBWAscA/wuc79x/G3BxBtV2D3BOBvyf+3fgV8BvnNsj2mdeaIH3njzZGNMDpE6erPoxxrwA7Ol391nAvc71e4Gzx7OmlEFqywjGmF3GmDXO9VZgE/acr67vuyFqc52x2pybAedigFOAh5373dpvg9XmOhGpAU4H7nBuCyPcZ14I8IFOnpwR/4EdBnhGRFY7J3DONBXGmF3O9Vqgws1iBvAvIvKa08XiSvdOXyIyDViIbbFl1L7rVxtkwL5zugLWAXXAH7DflpuMManTxrv299q/NmNMar99z9lvPxKRkAul/Rj4DyDp3C5hhPvMCwGe6Y4zxiwCPg1cIiInuF3QYIz9fpYRrRDHrcBhwAJgF/BDN4sRkQLgEeAyY0xL35+5ve8GqC0j9p0xJmGMWYA9J+4SYJYbdQykf20iMhf4BrbGo4Fi4GvjWZOInAHUGWNWp+P1vBDgGX3yZGPMDmdbBzyK/U+cSXaLSBWAs61zuZ5expjdzh9ZEvgZLu47EQlgA/I+Y8yvnbszYt8NVFsm7TunnibgOeCjQJGIpM725frfa5/aPuV0SRljTDdwN+O/344FlonIVmx38CnALYxwn3khwDP25Mkiki8ikdR14FTg9aGfNe6eAL7gXP8C8LiLtewnFY6O5bi075w+yDuBTcaYm/v8yPV9N1htmbDvRKRMRIqc67nAJ7F99M8B5zgPc2u/DVTbm30+kAXbzzyu+80Y8w1jTI0xZho2y541xnyOke4zt4/GDvOI7WnYo+/vAN90u54+dR2KHRWzHnjD7dqA+7Ffp2PYfrR/xPav/QnYAvwRKM6g2n4BbABew4ZllUu1HYftHnkNWOdcTsuEfTdEba7vO+BIYK1Tw+vAt537DwVeBd4GHgJCGVTbs85+ex34Jc5IFZf+353EvlEoI9pnOpVeKaU8ygtdKEoppQagAa6UUh6lAa6UUh6lAa6UUh6lAa6UUh6lAa6UUh6lAa6UUh71/wEgLxRVILogrAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 训练误差\n",
    "for i, name in enumerate(name_l):\n",
    "    plt.plot(list(range(num_epochs)), train_l[i], label=name)\n",
    "plt.legend(loc = 1)\n",
    "plt.title('mse_train')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d54e2cce20>]"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d54a1e6320>]"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x2d54a1b4070>"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'mse_test')"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAyBklEQVR4nO3dd3wc1bn/8c+zq5VWvXcXueGOCy4EMDUJYHoCFxJSDMnPJCEBEkiA3CSXJIQAlxTSIPQSYm5ChxhCB4OJjRs2bthg2ZYlq7hIsuqW8/tjRrJsJKutNDva5/16rWd3tTvzaCx99+jMmTNijEEppZT7eJwuQCmlVN9ogCullEtpgCullEtpgCullEtpgCullEtpgCullEtpgCullEtpgCullEtpgCvVCRFZICLvRGhdpSLy2UisS6mONMCVUsqlNMBVVLNbrz8UkbUi0iAi94tIvoi8KCL1IvKqiGSKiF9E/iYie0Rkv4i8LyL59jrS7fdViMguEblZRLxH2OZE4G7gMyJyQET2288niMgdIrJDRCpF5G4RSbS/liMiL9jb3isiS0TEIyKPAiOA5+11/WjAd5qKGXFOF6BUD3wR+BzWz+tqYAbwDWAjsBi4CtgNpAPDgRZgOtBkv/8hoAoYCyQDLwA7gb92tjFjzEYR+RbwTWPMCR2+dCswxl53APg78DPgRuBaoAzItV97rLUq81URmWev69W+7wKlPk1b4MoN/miMqTTG7AKWAMuMMauNMc3A01iBHgCygbHGmJAxZqUxps5uhc8HrjHGNBhjqoDfAZf0pgAREWAh8H1jzF5jTD1wS4f1BIBCYKQxJmCMWWJ0pjg1wLQFrtygssP9pk4epwCPYrW+HxeRDOBvwH8DIwEfUGFlMGA1XHb2soZcIAlY2WE9ArR1xfwvcBPwsv31e4wxt/ZyG0r1iga4GhKMMQHg58DPRaQEq2tls71sAXKMMcHerPKwxzVYHxaT7b8EDt9+PVY3yrUiMgV4XUTeN8a81sm6lIoI7UJRQ4KInCIiU+2Dk3VYXRphY0wF8DLwGxFJsw8sjhGRk7pZZSUwTETiAYwxYeBe4Hcikmdvs1hETrfvny0iY+2ullogBIQ7rGt0ZL9jpTTA1dBRADyBFd4bgbewulUAvgbEAxuAffbrCrtZ3+vAemC3iNTYz10PbAX+IyJ1wKvAePtr4+zHB4D3gL8YY96wv/Zr4Cf2CJXr+vNNKtWR6HEWpZRyJ22BK6WUS2mAq5hln4hzoJPb3U7XplRPaBeKUkq51KAOI8zJyTElJSWDuUmllHK9lStX1hhjcg9/flADvKSkhBUrVgzmJpVSyvVEZHtnz2sfuFJKuZQGuFJKuZQGuFJKuZTOhaLUIAgEApSVldHc3Ox0KSqK+f1+hg0bhs/n69HrNcCVGgRlZWWkpqZSUlJCh9kMlWpnjGHPnj2UlZUxatSoHr1Hu1CUGgTNzc1kZ2dreKsuiQjZ2dm9+itNA1ypQaLhrbrT258RVwT4Wx9V85c3tzpdhlJKRRVXBPjSrTX87pWPqG0MOF2KUkpFDVcE+PyphQRChlc2Vnb/YqVUj33zm99kw4YNA7qN+fPns3///k89f9NNN3HHHXcM6LZ7s63OXvOb3/wGEaGmpqaLdznLFaNQjh6WTnFGIovXVXDhMcOcLkepIeO+++4b8G0sXrx4wLcxEHbu3MnLL7/MiBEjnC6lS64IcBFh/tQCHlpaSl1zgDR/z8ZIKhWNfv78ejaU10V0nZOK0vifcyYf8TUNDQ3813/9F2VlZYRCIX76059y1113cccddzBr1izuv/9+brvtNjIyMpg2bRoJCQn86U9/YsGCBSQmJrJ69Wqqqqp44IEHeOSRR3jvvfeYO3cuDz30EACLFi3illtuwRjDWWedxW233QYcnAMpJyeHX/3qVzz88MPk5eUxfPhwjjnmmC7rPfnkk5kxYwZLliyhoaGBRx55hF//+tesW7eOiy++mJtvvhmA3/72tzzwwAOA9RfFNddcA9Dltj7++GOuvPJKqqurSUpK4t5772XChAmf2v73v/99br/9ds4777xe/V8MJlcEOFjdKPcu2carGyr5wkxthSvVWy+99BJFRUX861//AqC2tpa77roLgPLycn75y1+yatUqUlNTOfXUU5k2bVr7e/ft28d7773Hc889x7nnnsu7777Lfffdx+zZs1mzZg15eXlcf/31rFy5kszMTD7/+c/zzDPPcP7557evY+XKlTz++OOsWbOGYDDIzJkzjxjgAPHx8axYsYI777yT8847j5UrV5KVlcWYMWP4/ve/T2lpKQ8++CDLli3DGMPcuXM56aSTCIfDXW5r4cKF3H333YwbN45ly5bxne98h9dff/2Q7T777LMUFxcfsg+ikWsCfPrwDIrS/Sxet1sDXLlady3lgTJ16lSuvfZarr/+es4++2zmzZvX/rXly5dz0kknkZWVBcBFF13ERx991P71c845BxFh6tSp5OfnM3XqVAAmT55MaWkp27dv5+STTyY315rx9NJLL+Xtt98+JMCXLFnCBRdcQFJSEgDnnntutzW3vWbq1KlMnjyZwkLrUqajR49m586dvPPOO1xwwQUkJycD8IUvfIElS5YQDoc73daBAwdYunQpF110Ufs2WlpaDtlmY2Mjt9xyCy+//HK39TnNNQEuIpw5tZBH/7Od+uYAqdqNolSvHHXUUaxatYrFixfzk5/8hNNOO63H701ISADA4/G03297HAwGe3zqd291t93eCofDZGRksGbNmi5f8/HHH7Nt27b21ndZWRkzZ85k+fLlFBQU9HqbA8kVo1DazJ9aQGswzGsbq5wuRSnXKS8vJykpia985Sv88Ic/ZNWqVe1fmz17Nm+99Rb79u0jGAzy5JNP9mrdc+bM4a233qKmpoZQKMSiRYs46aSTDnnNiSeeyDPPPENTUxP19fU8//zz/f6e5s2bxzPPPENjYyMNDQ08/fTTzJs3r8ttpaWlMWrUKP75z38C1unrH3zwwSHrnDp1KlVVVZSWllJaWsqwYcNYtWpV1IU3uKgFDjBjeCYFaX4Wr6vg/BnFTpejlKusW7eOH/7wh3g8Hnw+H3fddRfXXXcdAMXFxfz4xz9mzpw5ZGVlMWHCBNLT03u87sLCQm699VZOOeWU9oOYhx/8mzlzJhdffDHTpk0jLy+P2bNn9/t7mjlzJgsWLGDOnDmAdRBzxowZAF1u67HHHuPb3/42N998M4FAgEsuuSTq+7q70u01MUXkAeBsoMoYM8V+7n+Bc4BW4GPgMmPM/u42NmvWLNPfK/Lc9Nx6/r58B6t++jlSElz1+aNi2MaNG5k4caLTZRzRgQMHSElJIRgMcsEFF3D55ZdzwQUXOF1WzOnsZ0VEVhpjZh3+2p50oTwEnHHYc68AU4wxRwMfATf2rdTeO+voQlqDYV7fpN0oSkXSTTfdxPTp05kyZQqjRo065ACkik7dNmGNMW+LSMlhz3U8PPsf4MII19WlY0ZkkpeawOK1FZw7rWiwNqvUkDdYZ0Ue7sorr+Tdd9895Lmrr76ayy67zJF63CQSfRCXA//X1RdFZCGwEIjIGU0ej3DmlAIef38nDS1BkrUbRSlX+/Of/+x0Ca7Vr1EoIvLfQBB4rKvXGGPuMcbMMsbMahsj2l/zpxbSEgzzxmbtRlFKxa4+B7iILMA6uHmp6e5IaITNKskiJyWBxesqBnOzSikVVfoU4CJyBvAj4FxjTGNkS+qe1+5GeWNTNY2tvR/Mr5RSQ0G3AS4ii4D3gPEiUiYi3wD+BKQCr4jIGhG5e4Dr/JQzpxbQFAjx5ubqwd60AgiHYfOL0BCd02wqFQu6DXBjzJeMMYXGGJ8xZpgx5n5jzFhjzHBjzHT79q3BKLajuaOyyU6Od74b5ZM34c3bYNcqGNyeJOe0HIB/fBUWXQJ3ToM3b4WWeqeravfOlhrO+sMS7n7rY8LhGPk/6SOdD/zIr+nJfODPPfcct9566xHXXV5ezoUXWoP11qxZE7Epdl07hMPrEU6fUsAzq3fRHAjh93kHt4BgK7z+C1j6R+vxm7dAxgiYdD5MPh+KZkJPrm8XbAWvr2evjQb7d8KiL0HVejj5x1D5Ibz5a1h+D8y7DmZdDj6/I6W1BEPc8e/N3LtkG+mJPm59cRNLtlTzm4umU5DuTE3RTucD71pP5wM/99xzu52Yq6ioiCeeeAKwAnzFihXMnz+/3zW6NsABzppayN+X7eDNzVWcMaWw1+83xrBs214WLd9BfXOQsDGEwubgMgxhY/B5PUwfkcHcUVkcMzKT1MYyeOJyKF8Fs74BJ/4QPn4N1j8D//kLLP2DHebn0TjuHJqTh5HevAvv/u2wbxvs3WYt95VCfQUUzYBTfgJjT4vuIN+5HB6/FILN8OV/wrjPWs/vWgmv/QL+fSO892c45UY4+hLwDt6P19aqeq5atIYNFXV85dgR/Pf8STy7Zhc/f34DZ9z5Nrd98WhOnxwlc1m8eAPsXhfZdRZMhTOP3ArU+cAHZj7whx56iBUrVrTvq7S0NFasWMHu3bu5/fbbufDCCyktLeXss89m1apV/OxnP6OpqYl33nmHG2+8kYsvvviI6z8SVwf43FFZZCXHs3jd7l4FuDGGt7fU8KfXt/B+6T6ykuMpTPfj9QgeEbwewSuCxwM+j4cDzUHuffsT7nrzY87xvset8ffj8Xj56Lg/MvKES0hP9FEz9iK2pp3JjhG7SPj4RUoqX2HK0r+QtPSPJB22/T2ebGrii6n1zyBQ+Hkm17xMxmNfZFvS0TyfcznrfVNpDoRpDoQIhg0TClKZOzqbY0dlkZfWv5ZkXXOATRX1bKyoY0N5HdUHWmgNhq1b6OAyEAoTCIbJSU3gqPxU5pu3OXnTLwinFuH9+vNIXocf+OJjCF76NHvWvULi2zeT9uyVVP/7dl5MvwRfUhq58QGy4lrJjGshzdNCijQTH2pAskbDST/q1/djjOGxZTu4+V8bSIqP476vzeKzk/IBuGTOCGaPyuLqx1dzxaMr+fLcEfz0rEkkxh/5r7XWYJi65gDZyfF9upJ8IBS2f36i68P4pZdeorCwkKeffY7WYJiavfu5849/ZndtM0vXbuFnN/2C/1v8JrnZGVx20TnMmK7zgfdljpSKigreeecdNm3axLnnntveddL2/fziF79oD/z+ckeAN9dCcx2EAxAKQKgVQgHiQgG+NbKc5RtX07IjTELBBIhP7nI1xhhe3VjFn17fwgdltYxLC3H3CQFOy6vClzMKCqdBYman721sqKPu6Wsp2PoPNsdN5Iqm71D6ejbyxiukxMdR33JwNExy/DTG5h3PlJGGU2UlqeF6KryF7DD5fBLMoapZ2NsQYF9DK3trWonnc1wS9yYLG5/iqh3XsDJuOouSv8aOpEl4PcIzq3fx2LIdAIzJ9nN+4V5O8m9lXMuHJNZ+AqkFkDGSUMZImlOG05hUTH1iMfWkUlnfwsaKejZU1LKxop4dew8OGspM8lGUkUhCnAef10OqP46EOA/xcR7ivR68Hg+VtQ1M3vh7Tg0/xXuhSXy78mpCf97OuPw9jMpJYV9jK6U1Dezc10ggZIDrOd3zPj8y/+Rrzbd9aj8GjYcG/LQAadLIF9dMJyEpjfREX/stzV7mpiYwIiuJ4VlJnc57s+dAC9c/uY5XN1Zy4lG53HHR0eSlHvoBNyY3hae+fTy/eXkzf337E5Z9soc/fGkGk4sOTtRU3xxg1Y79rCjdy/Jte1mzcz8twTA5KfFMKkpnclGafUtnZFZSezAbY6iub2FDRR2bdlsfihsr6vi4uoE0fxyzS7KYMyqLY0dn4+nYFd9NSzlSgqFwe0OgKRAitXAUL/77Za646gecdNrpzJx7HIFQmOZAkLWrV/CZ409g9PACapsCzDv9HMpKP2bPgRaM0fnAe+P888/H4/EwadIkKisH9jq+7gjwV2+CFQ90+qWFwEIP8MCvrCcyRkDuRMgdD7kTIG8CoaxxvLVmI+8ueYO0us1cl1DGzMwykpvK4fC5tTJLoHA6FE23loXToL6CpH9eRlLNR3DCDxh/yo95KexhbVktyz7ZQ1V9C6Nzkxmbl8LYvBQK0vwdWm7z6JlzIHAzvH8/x7zzW46p/QEUzId51xFqTqB6/VsES5eSs/8D/FubASgzOXziGUl21XaKzPtkSj3JQDKQC9SZROJNLl6TzeiEPM5OKyJp2giyC0ZSPHIsOUUliP8IM861HICnr4BdL9By9NfwTv0x19a08FHlATZX1rNkSzVZyfGML0jl9CkFjMpOZmR2EqNyPktu8k9g91rw+GjxJlHdEk9Fcxy7GwyV9S1kbXuBL3zyUyb597EhmMKWqgPUNgWobQrQGgx/qpSs5HiGZyUxIiuJEVmJZCbF89e3P6G2McDPzp7EguNKumzxxsd5uHH+ROaNy+UH/1jDBX9eysITR1PfHOD90n1s2l1H2FjHVSYXpXHp3JEUZfjZvLue9eV13LfkE/vDCVIS4phUmEZ8nIeNFXXsaWht305Rup+JhWmcNjGfmvoWlpfu5eUN1i/w/ecVkljTQHKCl+R469cuFDaE7O66jrewfTBckPYeNREQ+NRfBKbDP22fEcGQoTkQojV0cD/GeTyMHXcUryx5j7dfe4X7fn8rp552KskJcZTkpBDXnEhyQhxFGYkUpPvJSvKxS2DX/iZqm1ppCgmBUFjnA+/BlLId6xzoU2TcEeBHX2z1E3vjwRNnLb0+8PoIEsf/e+wD0sO1TIrbRUldGaNrNzFiy2vEY/0He4FT7ZuJ80DGWKTwM5A/xeo7zB5j9UeXr4GKNVC+GjY8c3D74oWkbPjq0zDmFAD8XpgzymphRYwvEY77LhzzdVh2N7z7R9i8GC9QgFj1zvoq4eHHssU/mXer/GzaXUdCnJekBC+Znmbyw5XkBCrIbK0gvXkXRS0VHNVajbd+Jez9N+wFNnfYpnis78/jtfateMHjse4HW6G1Hs64jYS5VzBHhDnjevH9FFt/siYAw+xbu1HHwyfwy5NSYcJxh7ytORCitilAZV0zO/Y2snNvk71sZG3Zfl5cV0EwbDgqP4VHLp/DxMK0HpVzwrgcXrrmRK5/ci1/emMrSfFeZo7I5KrTxjG7JIvpwzM6nZqhJRhiS+UB1pfXsr68jvXlddQ1BzhtYh4TC9OYWJjGhIJUMpLiP/Xe3bXNLNu2h6RQDYFgmN3NgS7r69h91/ZrbzDtg5uMsWPaYKW5TTr8C9YHUVJ8HFnxHhJ9Xvw+Lz6vh/LycooKspn0jQUMy8855ADm7Nmzueaaa9i3bx+pqaksfv5ZpkyZwpjcFOK8HmqbAmzaXc+B/U2d1j5nzhyuuuoqampqyMzMZNGiRXzve9875DUnnngiCxYs4MYbbyQYDPL8889zxRVXdLk/emLevHksWLCAG264AWMMTz/9NI8++ijGmE631XE+8IsuughjDGvXrj2kq6RtPvA2HfvwIyE1NZX6+siM2nJHgI841rp1Ig644PyjWPJRNVuArfbPsdeEyA6Uk99cSm7rDooKi5g88wS8+ZMg/vBeaSBrNIw59eDjxr1Q8YEV6I174bjvQUpepL+zziWkWgdGZ3/TOjCaPhyGzwa7tewBxgPjj+rleoMt1kHTunL7tsvqmjIhCIfAhCEctO/bz035Iow+qft191ZmibXcV/qpL/nt0MlP83P0sIxPfxuhMFX1LeSmJuDz9u5ctKzkeO756jHs2t9Efpq/R+9PiPMypTidKcU9nx+7TUG6n/OmF7NxYx1HFaQSDIVpbA2BgLctsO2bZ4APYPd2PvCMDOsDLcVumWck+ihrCrT/hdCRzgfec6eccgq33nor06dP7/dBzG7nA4+kSMwHroYIY+DXw2H6l2H+7U5XM+CGwnzgrcEwm3bXUZyRSHZKwhHWpPoj0vOBKxV5IlYrvJMWuHJGd/OB+7yCIO3HA5Tz3NGFooamzJFQs8XpKpStuzMVRQSf1zqYGUlumQ/8wQcf5M477zzkueOPP97R6XA1wJVzMktg66vWvCqeof/HoDGmT+PKo4nP6zlkdEskuGU+8Msuu2zAP1R626U99H9rVPTKLLHO6jwwsGNlo4Hf72fPnj0DPqxsoPniPBFvgSuLMYY9e/bg9/f8ZD1tgSvnZI6ylvtKIa33UyG4ybBhwygrK6O62t2zZ9Y2BTjQEiS8NzGqZ31wK7/fz7Bhw7p/oU0DXDmn41DCkZ9xspIB5/P5GDVqlNNl9Nuj75Xy02fXs/zHp/V7WgfVf9qFopyTMRwQHYniIkUZiYB1hqZynga4ck5cAqQVw/7tTleieqgtwMv3NztciQINcOU0HQvuKgcDXFvg0UADXDkrc6QGuIuk+a1T67ULJTpogCtnZZZY87MENBDcQEQoyvBrCzxKaIArZ7WNRNm/w9EyVM8VZSRSXqsBHg00wJWzjjAroYpORRmJehAzSmiAK2dpgLtOcUYiextaaWoNOV1KzOs2wEXkARGpEpEPOzyXJSKviMgWe9n5dciU6k5yLviSNMBdpCjDOoFHu1Gc15MW+EPAGYc9dwPwmjFmHPCa/Vip3tNpZV2nKF2HEkaLbgPcGPM21oW4OjoPeNi+/zBwfmTLUjFFA9xVdCx49OhrH3i+MabCvr8byO/qhSKyUERWiMgKt0/kowZIZgns2w4un6kvVhSk+xGBXXog03H9PohprPkxu/zNM8bcY4yZZYyZlZub29/NqaEoswQCDdBQ43Qlqgd8Xg/5qToWPBr0NcArRaQQwF5WdfN6pbqmI1FcR0/miQ59DfDngK/b978OPBuZclRM0gB3HWssuAa403oyjHAR8B4wXkTKROQbwK3A50RkC/BZ+7FSfZMxwlpqgLtGcUYi5bXNhMN63MJJ3V7QwRjzpS6+dFqEa1GxypcIKQUa4C5SlJFIazDMnoZWclMTnC4nZumZmCo66FBCV9GhhNFBA1xFBw1wV2k/G1MD3FEa4Co6ZJZA3S4ItjhdieqBYr20WlTQAFfRIbMEMLB/p9OVqB5IT/SRFO/VWQkdpgGuokP7vOClTlahesi6sIMOJXSaBriKDjoW3HX0wg7O0wBX0SElH+L8GuAuUqxnYzpOA1xFB48HMvQCx25SlJ5IzYFWmgN6YQenaICr6KFDCV2lbSx4Ra0eyHSKBriKHjqtrKvoyTzO0wBX0SNzJLTUQdM+pytRPaBjwZ2nAa6iR/tIlG2OlqF6Jj89ARFtgTtJA1xFDx1K6CoJcV5yUxI0wB2kAa6iR8ZIa6kB7hrWyTx6ENMpGuAqeiSkQHKuBriLFOvZmI7SAFfRpW0kinKFogw/u/Y3YXTkkCM0wFV00bHgrlKUkUhLMMzehlanS4lJGuAqumSWQG0ZhAJOV6J64OBYcO0Hd4IGuIoumSVgQlaIq6jXNhZcJ7Vyhga4ii46lNBV9GxMZ2mAq+iiAe4qmUk+/D6PBrhDNMBVdEktBG+8BrhLHLywg/aBO6FfAS4i3xeR9SLyoYgsEhF/pApTMcrjhfThGuAuUpyRqPOhOKTPAS4ixcBVwCxjzBTAC1wSqcJUDNOhhK5SlK4n8zilv10ocUCiiMQBSUB5/0tSMU8D3FWKMhKpqm+hJagXdhhsfQ5wY8wu4A5gB1AB1BpjXj78dSKyUERWiMiK6urqvleqYkdmCTTvh6b9DheieqIow+o5raxtcbiS2NOfLpRM4DxgFFAEJIvIVw5/nTHmHmPMLGPMrNzc3L5XqmJH+xXq9ZR6N9B5wZ3Tny6UzwLbjDHVxpgA8BRwXGTKUjFNhxK6io4Fd05/AnwHcKyIJImIAKcBGyNTloppmTqtrJsUpFtdKBrgg68/feDLgCeAVcA6e133RKguFcv86ZCYpQHuEn6fl5yUBD2d3gFx/XmzMeZ/gP+JUC1KHZRZAlWbrAscizhdjeqGNa2snswz2PRMTBWdxs+HHUth2d1OV6J6QMeCO0MDXEWnedfChLPhpRth84tOV6O6UWRfmUcv7DC4NMBVdPJ44Av3QOE0eOIbULHW6YrUERRl+GlsDVHbpPO4DyYNcBW94pPhS49DYgb8/WKoq3C6ItUFHQvuDA1wFd3SCuHL/wctdbDoEmhtcLoi1Qm9Mo8zNMBV9CuYChc+ALvXwlMLIRx2uiJ1GD2Zxxka4ModjjodTr8FNr0Ar+rI1WiTnRxPfJxe2GGw9WscuFKDau63YM9WWPoHyB4DxyxwuiJl83iEonS/9oEPMm2BK/cQgTNug7GfhX9dC6XvOF2R6qBtKKEaPBrgyl28cXDhg5CQCmv+7nQ1qgO9tNrg0wBX7uNPg4yRcKDK6UpUB0UZiVTWN/OjJz7gmdW7qKzTMB9o2geu3CklH+p1XHg0uWBGMZt31/Hv9ZX8Y0UZAGNykzluTA7Hjcnm2NHZZCbHH/KeUNjQGgzTGgzTEgqR5vfh93mdKL9dIBSmfH8TO/Y2sn1PIzv3NrJjbyOJPi/jC1LbbwVpfsTheXo0wJU7peRawwpV1BiVk8xfvzqLUNiwsaKOpR/XsPTjPTy5qoxH/2NdnCMnJYFAyArs1lCYUPjTp94XpPkZmZ1k35KtZVYyI7KSiPMKLXbgtwbDtARD1uNQmEAwjMcjeD1CXPvS0/7YALVNAfY3ttpL+9bUSm1jgMr6ZnbsbaR8f/MhdcV7PQzLTKSxNcRTq3e1P5+e6GN8fipHFaQwPj8VEaG+OUh9c4ADLcH2+3XN1v2bz5/CMSMzI7rPNcCVO6XkW10o4bB12r2KGl6PMKU4nSnF6Sw8cQyBUJi1ZbUs3VpDeW0T8V4P8XH2zes9eD/Ow/6GVkr3NLJ9TwNvbK6mur5swOtNSYgjPdFHTko804dnct60JEZkJTEi21oWpPnxeKyW9v7GVjbvrmdzZb213F3Ps6vLqW8JHvL9p/rjrFuCjxR/HMUZfnzeyLfWNcCVOyXngQlB0z5Izna6GnUEPq+HY0Zm9qn12dAStLsyGti5twmDsT8ADgZ/gr30eTwYDMGwIRSyl2FDMBwmbE+ylZ7oIz0xnowkHxmJPtISffi8PW8AZCTFM3d0NnNHH/yZM8ZQWdeCRyDV78Pv8wxa14oGuHKnlDxreaBSA3wIS06IY2JhGhML05wupUsi0n5VosGmf3sqd2oL8AYdiaJilwa4cqeUfGupQwlVDNMAV+6UnGstNcBVDNMAV+7kTwdvgtUHrlSM0gBX7iRi9YM3VDtdiVKO0QBX7pWSpy1wFdP6FeAikiEiT4jIJhHZKCKfiVRhSnUrOQ8OaAtcxa7+tsDvBF4yxkwApgEb+1+SUj2kLXAV4/p8Io+IpAMnAgsAjDGtQGtkylKqB1LyoLEGwiHwODsBklJO6E8LfBRQDTwoIqtF5D4RST78RSKyUERWiMiK6mr9c1dFUEo+mDA07nG6EqUc0Z8AjwNmAncZY2YADcANh7/IGHOPMWaWMWZWbm5uPzan1GF0LLiKcf0J8DKgzBizzH78BFagKzU42s/G1H5wFZv6HODGmN3AThEZbz91GrAhIlUp1RPt86Fo15yKTf2djfB7wGMiEg98AlzW/5KU6qGOMxIqFYP6FeDGmDXArMiUolQvxadAXKL2gauYpWdiKvdqO51eA1zFKA1w5W56Mo+KYRrgyt1S8vUgpopZGuDK3ZJztQWuYpYGuHK3lHxo3AuhgNOVKDXoNMCVu6XkAgYaapyuRKlBpwGu3K3tbEy9uLGKQRrgyt2S207m0QBXsUcDXLlbiga4il0a4Mrd9HR6FcM0wJW7xSdbp9TrWHAVgzTAlfvpWHAVozTAlful5GsfuIpJGuDK/VJyNcBVTNIAV+6Xkq/jwFVM0gBX7pecB037INjqdCVKDSoNcOV+emk1FaM0wJX76VhwFaM0wJX7tc+Hoi1wFVs0wJX7JedaS22BqxijAa7cT+dDUTFKA1y5ny8REtI0wFXM6XeAi4hXRFaLyAuRKEipPknJ07HgKuZEogV+NbAxAutRqu/0dHoVg/oV4CIyDDgLuC8y5SjVR8l6Or2KPf1tgf8e+BEQ7n8pSvWDtsBVDOpzgIvI2UCVMWZlN69bKCIrRGRFdbWO01UDJCUXWmoh0Ox0JUoNmv60wI8HzhWRUuBx4FQR+dvhLzLG3GOMmWWMmZWbm9uPzSl1BHpxYxWD+hzgxpgbjTHDjDElwCXA68aYr0SsMqV6o/3ixvpXnoodOg5cDQ06H4qKQXGRWIkx5k3gzUisS6k+aZ+RULtQVOzQFrgaGtrnQ9EAV7FDA1wNDXEJ4M/QAFcxRQNcDR0p+doHrmKKBrgaOlLydE5wFVM0wNXQkZKnLXAVUzTA1dCRnKfjwFVM0QBXQ0dKHrTWQ2uj05UoNSg0wNXQoWPBVYzRAFdDR9t8KDqUUMUIDXA1dOjJPKorxkAo6HQVEacBroaO9ha4jkRRh/nwSbhjLLTUO11JRGmAq6EjOcda6lhwdbjSJdC0D6qG1tUfNcDV0OH1QVK2tsDVp1VusJfrna0jwjTA1dCSnKd94OpQxhxseWsLXKkolqIBrg5Tu9M6PwCgaoOztUSYBrgaWlLydBy4OlRb90nuBKsLxRhn64kgDXA1tOjV6dXhqux+7ykXQtPeIfXzoQGuhpbkXAg0QssBpytR0aJyA6SPgOFzrMdVQ+dApga4Glp0LLg6XNUGyJsI+ZPtx0PnQKYGuBpaUuyzMXUsuAIItkLNR5A/yTpPIDn3YJ/4EKABroYWbYGrjvZshXAQ8uzWd94k7UJRKmol2zMSDqEDVaof2oYN5k+ylnmToGoThMPO1RRBGuBqaEnOAfFogCtL5XrwxEH2OOtx/iQINsG+bc7WFSF9DnARGS4ib4jIBhFZLyJXR7IwpfrE47VOp9ex4AqsFnj2OIiLtx7nDa0Dmf1pgQeBa40xk4BjgStFZFJkylKqH3QsuGpTteFg9wlA7viDzw8BfQ5wY0yFMWaVfb8e2AgUR6owpfosOVcDXFlTx+7fYfV7t0lIgcwSDfCORKQEmAEs6+RrC0VkhYisqK7WoV1qEGgLXMHBbpK28d9t8iYNmaGE/Q5wEUkBngSuMcbUHf51Y8w9xphZxphZubm5/d2cUt1LybX6wIfQnBeqD9qmjs07rGc3b5I1vDDYMvg1RVi/AlxEfFjh/Zgx5qnIlKRUP6XkQ7AZWj7VnlCxpGoDxKdAxohDn8+bCCZkneDjcv0ZhSLA/cBGY8xvI1eSUv3UPhZcu+xiWqV9Cr3Ioc+3dakMgW6U/rTAjwe+CpwqImvs2/wI1aVU36W0BbiejRmzjLHnQOlkYFz2WPD4hsSBzLi+vtEY8w4g3b5QqcHWFuDrn7LGhedPhoRUZ2tSg+tApTV17OEHMMG69F7OUbEd4EpFrcwSq9/z/fusG0DmKCiYAgVHQ8FUKJoBqQWOlqkGUFcHMNvkT4Lt7w1ePQNEA1wNPfHJcPVaqCuH3eugcp213P0hbHwBMNaf0F99GkbNc7paNRDaWtddBXjeRFj3T2iuBX/64NUVYRrgamgSgfRi6zb+jIPPtxywWmdPXwHPXwXfXgq+ROfqVAOjcoM1Gik5u/OvdzylfsSxg1dXhOlkViq2JKTAiLlwzu9h7yfw1u1OV6QGQtX6rlvfcPD0epf3g2uAq9g0+mSYfiks/YPVtaKGjnAIqjd3fgCzTfpwiE91/VBCDXAVuz5/M/gz4LnvWb/0amjYu806ketILXARqx/c5bMSaoCr2JWUBWfeBuWrYPk9TlejIqXtijv53UyOmjfReq2Lp1zQAFexbcoXYezn4LVfWjPXKfer3AAI5Iw/8uvyJ0PTPqjfPShlDQQNcBXbROBseyaIf13r6taYslWth6zREJ905NflTbRf795+cA1wpTJGwKk/gS0vw4dPOl2N6q/KDd13n0CHoYQa4Eq529wroGgmvHQDNO51uhrVV4Ema3ho3hFGoLRJzrbGirv4QKYGuFJgzZly7h+s8H75p05Xo/qqehNgetYCB6sbpe20exfSAFeqTcFUOP4qWPM3+OQtp6tRfVHZzSn0h8ubbI0Zd+kwUg1wpTo66XrrANiz37XmT1HuUrUB4vzW/2FP5E2EYBPsKx3QsgaKBrhSHfkS4Qv3WieC3HMKvPM717bOYlLleuvK8x5vz17f1tXi0m4UDXClDjdsFnznPzD+THj1JnjoLOvsPhX9qjb07ABmm9wJgLj2QKYGuFKdSc6G/3oELvir1Tq7+wRY+bCOE49mDXusCzn09AAmWFMPZ5YcPHvTZXQ6WaW6IgLTLoGRx8Mz37amn938ojVape2qP8ZYZ/JVrrfnHf/QGgmROx5O+IF1EQk1OLqbA7wr+ZNd2wLXAFeqOxnD4WvPwbK7rS6VvxxrnYJfvRkqP4TGPQdfmz7culzXR/ZJQUedCSdeZ3XLqIHV1wDPm2h9MAeaweePfF0DSANcqZ7weOAz34Exp1qt8VWPWL/44+dbww/zJ1u3xEzr9U37YNk9sOwuuO80a/raeddByQmfvkq6iozK9db+7+2l8vImgQlBzWYonDYwtQ0QDXCleiNvAix8A8JhK9S7kpgJJ19vhf6KB2HpH+Hhs2H4XKtrJWu0NdKl7RbocD8csvplc8dbMyaqnmk7gNnbD8i2FnvVRg1wpWLCkcK7o4RU6+SgOf8PVv8N3r0TFl3c8+0kZVtdMm233PGQWmidMt56wLq1HDj0fjgI3niIi7eW3gTrSuzeeIhLsA7c+TOsa0EmZlj345M7D75g66HrDjRZQ/S88fY67fV6fAcfi6eT2wD/1REOWwE8/cu9f2/2GOt76DiUMBS0WuTla6BiDVSsta7mNOpE61ZwdPdDFVsbYOdy2P4ubF8K8+/o3QHWHuhXgIvIGcCdgBe4zxhza0SqUmqo8SVaIT7z67D1FSsI4/zWzec/eD/O7oPdt83qY6/5CGq2wKYXDu1r75KAJw7Cgd7V54mzAt2fDqFAhw+DXq7niKV5IC7R+qsiKQsSs6wPqKS2ZTbEp9j7I9H6sPEl2vso0QrZUCu01Fu3thpb6qGhynrc2/5vsD50co6Cra9Z/y8Va6yD0cEm6+u+ZKubbP9OeOVn1nP+dBh5wsFAz5sILXWwYxlsf8cK7PLV1oepeK2WfUtdxHZlmz4HuIh4gT8DnwPKgPdF5DljjHun9lJqoMXFw4Szun9dzlgY97lDn2vYA3u2WKNe4pOtsItPtlqGbfd9SVZr1xgriEOth96CrdBab12NvWk/NO8/7H6dFZTt623bTof1m5C9vqC1DAfsbdnbw4AJ27eO98PQ2ghNe605Zxr3wP7t1rK5tv/7Nj4FRh7Xt/cWHwOrHrbqKZwGsy63lkXTIXvswdZ2/W4ofQe2vQXb3obN/7Ke92dYAW3C1l8jxcfAcVdByfFWt1lCav+/v06I6eO4VhH5DHCTMeZ0+/GNAMaYX3f1nlmzZpkVK1b0aXtKqSEsFLQO/LbWH3o8INDUYdlitcoTUq2wTkix76da9+P8fe+qaW2E+grIHNXz7jGwLgKybQns/A+kFlmBXTyr+7nIe0lEVhpjPjWUqT9dKMXAzg6Py4C5nWx4IbAQYMSIEf3YnFJqyPLGQUoukOvM9uOTrL7w3soYATMutW4OGPAzMY0x9xhjZhljZuXmOvSfo5RSQ1B/AnwXMLzD42H2c0oppQZBfwL8fWCciIwSkXjgEuC5yJSllFKqO33uAzfGBEXku8C/sYYRPmCMceeMMEop5UL9GgdujFkMLI5QLUoppXpBp5NVSimX0gBXSimX0gBXSimX6vOZmH3amEg1sL2Pb88BaiJYTiRpbX2jtfWN1tY3bq5tpDHmUyfSDGqA94eIrOjsVNJooLX1jdbWN1pb3wzF2rQLRSmlXEoDXCmlXMpNAX6P0wUcgdbWN1pb32htfTPkanNNH7hSSqlDuakFrpRSqgMNcKWUcilXBLiInCEim0Vkq4jc4HQ9HYlIqYisE5E1IuLo5YZE5AERqRKRDzs8lyUir4jIFnuZGUW13SQiu+x9t0ZE5jtU23AReUNENojIehG52n7e8X13hNoc33ci4heR5SLygV3bz+3nR4nIMvv39f/s2UqjpbaHRGRbh/02fbBr61CjV0RWi8gL9uPe7zdjTFTfsGY6/BgYDcQDHwCTnK6rQ32lQI7Tddi1nAjMBD7s8NztwA32/RuA26KotpuA66JgvxUCM+37qcBHwKRo2HdHqM3xfQcIkGLf9wHLgGOBfwCX2M/fDXw7imp7CLjQ6Z85u64fAH8HXrAf93q/uaEFPgfYaoz5xBjTCjwOnOdwTVHJGPM2sPewp88DHrbvPwycP5g1temitqhgjKkwxqyy79cDG7EuGej4vjtCbY4zlgP2Q599M8CpwBP2807tt65qiwoiMgw4C7jPfiz0Yb+5IcA7u/ZmVPwA2wzwsoistK//GW3yjTEV9v3dQL6TxXTiuyKy1u5icaR7pyMRKQFmYLXYomrfHVYbRMG+s7sB1gBVwCtYfy3vN8YE7Zc49vt6eG3GmLb99it7v/1ORBKcqA34PfAjIGw/zqYP+80NAR7tTjDGzATOBK4UkROdLqgrxvrbLGpaIcBdwBhgOlAB/MbJYkQkBXgSuMYYU9fxa07vu05qi4p9Z4wJGWOmY11ScQ4wwYk6OnN4bSIyBbgRq8bZQBZw/WDXJSJnA1XGmJX9XZcbAjyqr71pjNllL6uAp7F+iKNJpYgUAtjLKofraWeMqbR/ycLAvTi470TEhxWQjxljnrKfjop911lt0bTv7Hr2A28AnwEyRKTtYjGO/752qO0Mu0vKGGNagAdxZr8dD5wrIqVYXcKnAnfSh/3mhgCP2mtvikiyiKS23Qc+D3x45HcNuueAr9v3vw4862Ath2gLR9sFOLTv7P7H+4GNxpjfdviS4/uuq9qiYd+JSK6IZNj3E4HPYfXRvwFcaL/Mqf3WWW2bOnwgC1Yf86DvN2PMjcaYYcaYEqw8e90Ycyl92W9OH4nt4dHa+VhH3z8G/tvpejrUNRprVMwHwHqnawMWYf05HcDqQ/sGVt/aa8AW4FUgK4pqexRYB6zFCstCh2o7Aat7ZC2wxr7Nj4Z9d4TaHN93wNHAaruGD4Gf2c+PBpYDW4F/AglRVNvr9n77EPgb9kgVp27AyRwchdLr/aan0iullEu5oQtFKaVUJzTAlVLKpTTAlVLKpTTAlVLKpTTAlVLKpTTAlVLKpTTAlVLKpf4//VnTk5RD5fwAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 测试误差\n",
    "for i, name in enumerate(name_l):\n",
    "    plt.plot(list(range(num_epochs)), test_l[i], label=name)\n",
    "plt.legend(loc = 1)\n",
    "plt.title('mse_test')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "sigmoid_model4是Lesson 13.2中出现严重梯度消失的模型，由于前几层基本丧失学习能力，sigmoid_model4本身效果并不好。但加入Xavier初始化之后，我们发现，init模型能够极大程度规避梯度消失问题，从而获得更好的效果。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- tanh激活函数\n",
    "\n",
    "不过正如此前所说，相比于sigmoid激活函数，Xavier初始化方法更适用于tanh激活函数，核心原因在于tanh激活函数本身能够生成Zero-centered Data，配合Xavier初始化生成的参数，能够更好的确保各层梯度平稳、确保各层平稳学习。\n",
    "\n",
    "我们以三层tanh激活函数隐藏层的神经网络为例，测试Xavier初始化效果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x2d53b60efd0>"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设置随机数种子\n",
    "torch.manual_seed(420)  \n",
    "\n",
    "# 创建最高项为2的多项式回归数据集\n",
    "features, labels = tensorGenReg(w=[2, -1], bias=False, deg=2)\n",
    "\n",
    "# 进行数据集切分与加载\n",
    "train_loader, test_loader = split_loader(features, labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x2d53b60efd0>"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设置随机数种子\n",
    "torch.manual_seed(420)  \n",
    "\n",
    "# 实例化模型\n",
    "tanh_model3 = tanh_class3()                   # 保留原参数\n",
    "tanh_model3_init = tanh_class3()              # 使用Xavier初始化参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x2d53b60efd0>"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[ 0.6107, -0.6019],\n",
       "        [ 0.9517, -0.7944],\n",
       "        [-0.3051, -0.6891],\n",
       "        [ 0.7712,  0.3751]], requires_grad=True)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[-0.4319, -0.6698, -0.5014, -0.1671],\n",
       "        [-0.4420,  0.6311, -0.3644, -0.5666],\n",
       "        [-0.2672, -0.8457, -0.4206, -0.4725],\n",
       "        [ 0.1864,  0.7043,  0.0935, -0.5047]], requires_grad=True)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[ 0.3565, -0.3687, -0.4099, -0.1660],\n",
       "        [-0.4519,  0.0952,  0.7030,  0.1181],\n",
       "        [ 0.5231, -0.7523, -0.6813, -0.1152],\n",
       "        [ 0.0008,  0.5406, -0.7617,  0.3613]], requires_grad=True)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[-0.9402,  0.1767,  0.7240,  0.1511]], requires_grad=True)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设置随机数种子\n",
    "torch.manual_seed(420)  \n",
    "\n",
    "# 修改init模型初始参数\n",
    "for m in tanh_model3_init.modules():\n",
    "    if isinstance(m, nn.Linear):\n",
    "        nn.init.xavier_uniform_(m.weight)\n",
    "        \n",
    "# 创建模型容器\n",
    "model_l = [tanh_model3, tanh_model3_init]           \n",
    "name_l = ['tanh_model3', 'tanh_model3_init']\n",
    "\n",
    "# 核心参数\n",
    "lr = 0.03\n",
    "num_epochs = 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 模型训练\n",
    "train_l, test_l = model_comparison(model_l = model_l, \n",
    "                                   name_l = name_l, \n",
    "                                   train_data = train_loader,\n",
    "                                   test_data = test_loader,\n",
    "                                   num_epochs = 2, \n",
    "                                   criterion = nn.MSELoss(), \n",
    "                                   optimizer = optim.SGD, \n",
    "                                   lr = lr, \n",
    "                                   cla = False, \n",
    "                                   eva = mse_cal)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEXCAYAAACu1P9TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABAU0lEQVR4nO3deZhcVbXw/++qocd0ekiPmbozk5AQMjKEkITpMggRBWUQEAcQr6/3qjg994rI8Py86k8FxasoQsTrAGi8CEgAgRfFgEkgc8g8dpLuTs9zddXZ7x9VFTqdrp7qdJ06XevzPP2k+tQZFkX1ql377L22GGNQSik18nmcDkAppVRiaMJXSqkUoQlfKaVShCZ8pZRKEZrwlVIqRWjCV0qpFKEJX6lhJiL3isivnY5DKU34SimVIjThKzUIIuJzOgalhkoTvlKAiMwXkXdFpFlEnhaR34vIAyKyXESOiMhXReQ48LiI5IvIcyJSIyL1kcfju51rkoj838i5XgYKnfsvU+p9mvBVyhORNGA18ARQAPwWuLbbLqWR7eXAHYT/bh6P/D4RaAd+3G3/3wAbCCf6+4HbhvU/QKkBEq2lo1KdiFxIOMmPN5E/CBH5O/A68ArwEjDaGNMR4/izgdeMMfkiMhHYB+QaY1ojz/8GsIwxHxvu/xal+qItfKVgLFBpTm39HO72uKZ7sheRLBH5mYgcFJEm4A0gT0S8kXPVR5N9xMHhDF6pgdKErxQcA8aJiHTbNqHb455fg78EzADOMcaMBi6MbJfIufJFJLvb/hNtjlepIdGErxSsBULA50TEJyIrgcV97J9DuN++QUQKgG9GnzDGHATWA98SkTQRuQC4evhCV2rgNOGrlGeMCQAfAj4JNAAfA54DOmMc8kMgEzgBvAW82OP5m4BzgDrCHwa/sjtmpYZCb9oq1QsReRv4qTHmcadjUcou2sJXChCRZSJSGunSuQ04i9Nb7kq5ms4aVCpsBvAUkE14WOV1xphjzoaklL20S0cppVKEdukopVSKSOouncLCQlNRUeF0GEop5RobNmw4YYwp6u25pE74FRUVrF+/3ukwlFLKNUQk5sxu7dJRSqkUoQlfKaVShCZ8pZRKEZrwlVIqRWjCV0qpFKEJXymlUoQmfKWUShGa8JVSKkVowldKqX5s376dD197Ldeu/CDvvvuu0+EMmSZ8pZTqx+7du6mpraW2vo733nvP6XCGTBO+Ukr1o62t7eTj9vZ2ByOJT1LX0lFKqWTQ3t6OAH6RU5K/22jCV0qpfrS0tJDm8ZAOtLa2Oh3OkGnCV0qpfrS2tpIBpKEJXymlRrSWlpZwwjeGlpYWp8MZMk34SinVj5aWFjIsizSgpbnZ6XCGTEfpKKVUP5qbmsgAMoCmxkanwxkyTfhKKdWP5qYmMoFMcHWXjiZ8pZTqR1P3hN/aimVZToc0JJrwlVKqD4FAgPbOTrKAbMBy8Y1bTfhKKdWHxkiffVbkB6ChocGpcOKiCV8ppfoQTe7ZkZ/u29xGh2UqpVQfuif8tMi2+vp6p8KJiy0tfBH5pYhUi8jWGM8vF5FGEdkY+bnHjusqpdRwiyb3UZEf0Bb+E8CPgV/1sc/fjDEfsOl6SimVELW1tUA42ft7bHMbW1r4xpg3gDo7zqWUUsmktrYWvwjpgBch2+Ohrs6d6S6RN23PE5FNIvIXETkz1k4icoeIrBeR9TU1NQkMTymlTldXV0eOCIIAkAOcOHHC2aCGKFEJ/x2g3BgzF/gR8KdYOxpjHjXGLDTGLCwqKkpQeEop1bsTNTXkdJtolWNZnHBpYzQhCd8Y02SMaYk8fgHwi0hhIq6tlFLxqK6uJqfb76MBt/Y+JCThi0ipiEjk8eLIdd1510MplTKMMZw4cYLcbttGAw2NjXR1dTkV1pDZMkpHRH4LLAcKReQI8E0iN7SNMT8FrgPuEpEg0A7cYIwxdlxbKaWGS2NjI4GuLkZ32zaa9z8IysrKnAptSGxJ+MaYG/t5/seEh20qpZRrVFdXA5zSws/r9pzbEr6WVlBKqRiqqqqAUxN+bo/n3EQTvlJKxXD8+HEA8rtty+3xnJtowldKqRiqqqpIEzlZJRMgDWGUx6MtfKWUGkmOHTtGXrdJV1F5xnDs2DGHoho6TfhKKRXD0cpK8npZ3SrPGI4eOeJARPHRhK+UUr0wxnD06FEKenmuAKiqriYYDCY6rLhowldKqV40NjbS3tFxyg3bqHwgZFknh226hSZ8pZTqxdGjRwFitvAB1/Xja8JXSqleHIn00feW8Mf02MctNOErpVQvKisrEei1SycH8IlQWVmZ4KjiowlfKaV6UVlZSa7Hg7/HkEwAD0KBiLbwU019fT1NTU1Oh6GUstmhQ4co6GVIZlSBZXHk0KEERhQ/TfhxWLNmDStXruQDH/gA27dvdzocpZSNjhw+fLKvvjdjgKPHjmH18aGQbDThx2Hfvn29PlZKuVtjYyMtra39JvxAV5erFkPRhB+HmpoaJD375GOl1MgQ7ZvvL+EDHD58eNjjsYsm/DgcO3aMYFoOkp7tykJKSqneDSThR9doddONW1sWQElVRyqPYmUUEcK4bniWUiq2w4cP46H3IZlROUCaCIdcdONWW/hD1NbWRmNDPSZ9NKH00Rw85J6vdUqpvh0+fJh8jwdfL0MyowRhDO5q4WvCH6Lop7qVmYvJzKOhvo7W1laHo1JK2eFwP0MyowqM4dDBgwmIyB6a8Ido//79AFiZ+ViZeadsU0q5lzGGI0eOnOyj70shcLyqyjVVM21J+CLySxGpFpGtMZ4XEXlYRPaIyGYRmW/HdZ20d+9exOPDZIzGyio4uU0p5W61tbV0dHb2ecM2agxgWZZriqjZ1cJ/Ari8j+evAKZFfu4A/tum6zpm565dhLLyQTyYtFGIL509e/Y4HZZSKk4DGaET5bYiarYkfGPMG0BdH7usBH5lwt4C8kSkzI5rO8GyLHbt2k0oK/K/W4RgVgE7drznbGBKqbgNJeG7ZSx+ovrwxwHdX5EjkW2nEZE7RGS9iKxP1slMR44cob2tFSu76OS2UHYRe/ftJRAIOBiZUipeR44cwStC7gD2zQIyPB7XDMtOupu2xphHjTELjTELi4qK+j/AAdG6OaFR78dnjSoiFAxqt04cNm7cyPe+9z2ee+45p0NRKayyspJ8ETx9DMmMEoSCyE1eN0hUwq8EJnT7fXxkmytt374d8fkxkdE5AFZ28cnn1ND89re/5dlnn+Xhhx5yOhSVwo4cPjygIZlRBcZQqV06p3gWuDUyWudcoNEY447b2r3Yum0bwawikPdfPpOejaRna8KPQ011uDxFR2cnLS0tDkejUlFfC5fH4qYFze0alvlbYC0wQ0SOiMgnReQzIvKZyC4vAPuAPcDPgc/acV0ndHR0sG/vXkKjik97riuriM2btzgQ1chQdfw4o/zhlpXWJorfyy+/zI9//GMOHDjgdCiuEV24fDAJP7qgebLec+zOllo6xpgb+3neAP9qx7Wc9t5772FZFlYvCT+UU0z1oX9SV1dHQcFg3jKqubmZ5tY2zikJ8nZVGkePHmXKlClOh+Vq3/3ud+no6CAQCPDFL37R6XBcIbpwed4gjon+pR89epSysuQefJh0N22TXW83bKOiHwLarTN40VIV8wq7TvldDU1zczMdHR3A+0lM9e/48eNA30XTeoru64bJV5rwB2nHjh1IxmjwZ572nJVdCOJhx44dDkTmbtGyFNNygxRkapmKeEXHhRuP4eAh99R6cdpQEv5oQLodm8w04Q/S9h076MqKMSXD48Nk5bPjPZ2ANVi7d+8m0ycUZVpMzA6wa6e+hvGIrsBmxhuqq6ppb293OCJ3OH78OJkeD+kDGJIZ5UXI9Xhccd9JE/4gNDY2UlNdHW7JxxDMGsPOnTsJ37ZQA7V921YqcrrwCEweHeLgocO0tbU5HZZr7dmzB/EJZrzBGKN1ngaoqqqKvCH86Y62LKq0hT+yRCdVhbJjT7q2ssbQ3NTEiRMnEhWW67W1tbF7z16m54b776flBjHGsHVrr7X41ABs27YNk29O3lHU+0oDU3X8OKPN4Bclz40cm+w04Q9C9GtytDpmb6LP6aLmA7dp0yYsy+KM/PA45mm5Qbwe2LBhg8ORuVN7ezu79+zGGmNBJki2sGWLDhceiJrq6gGVVOgpF6iprU36b/aa8Adh//79SFom+LNi7mNlhW/3aMIfuLfffhu/F2bkhRN+hi+c9N9+a63DkbnT1q1bsUIWpiicfEKFId7d+G7SJyOndXZ20tzaSk6M51/A8AK9v4ajgWAwSFNT07DFZwdN+INw4MABQun9fP77MpC0LA66aBUcJxljePNvbzA7P0Ca9/3t8wsD7Nt/QIcUDsGGDRvCf9nRW03F0NTYpI2QftTW1gLETPjHIj+9iR6T7F25mvAH4eChw4Qy+v/CF0ofrePIB2jHjh1U1ZxgUXHXKdsXRn5//fXXHYjK3d7+59vhur2RaZWmONwq/ec//+lcUC4QTfijh3BsNOFHz5GsNOEPUHNzM81NjZiM/t8OoYzRHHZJ9TynvfTSS/g9sKDo1IRfnGkxJTfEmhf/ol0Rg1BXV8fePXuxSrrdeMwCyRX+uU4Tfl/q6sJLemQP4dhRkX814Y8Q0a4FawAJ36SPprGhQYcV9qOzs5OXX1rDgqIA2f7Tk/rSsg72HzjIzp07HYjOndavXw+AKTn19QwVh9i0adPJ2bfqdPX19cD7yXswoh8SDQ0NdoUzLDThD1B0gQOT3n/Cj34ouGGqtZNee+01mltaWTGus9fnzy8NkO4VVq9eneDI3GvdunV40j2nTRU1pYZgV5DNmzc7E5gLRJN17CEZsaUDPhFN+CPFoFr4kX3csuyZE4wxPP3U7xk3yjArv/eyslk+uKC0g1deeflk60vFZoxh3fp1BIuCnDZRtBDEIzrUtQ+NjY1keDz4BjHLNkoQskVobGwchsjsowl/gI4cOYKkZ4HX3+++VuRbgFtWwXHCO++8w+49e7l8QhvSx9/Xv0zsINgV5A9/+EPignOpI0eOUFdbB6cXcgUfmDGGDe9owo+lsbFxSK37qCy0S2fEOHDgIMG0AU7J8KUh6Vnawu/DqlVPkJcBS0r7XgN4bLbFgqIAf3jmaV0UpR/RyVXR8fc9WWMs9uzeo3V1YmhubiYzjgECGZZFc3OzjRHZTxP+ABhj2L9/P1a3JQ37E0zPY+9eHffcm3feeYeNGzfxgYltp4y9j+WDkzpobWvn6aefHv7gXGzbtm1ImsQcSG4KDZZl6U3wGJoaG8mIJ+EDzdql435Hjx6lvb0Nq48aOj1ZWQXs27/PFcueJZIxhp8/+ij5GXBRjJu1PVWMDrGoKMDvf/fbpP/K7KTtO7Zj5Vun999HRSqCaPnu3rW2tJARx/EZkPTfQjXhD8C2bdsAsLJPX/QkltCoIoJdXScLrqmwv/3tb2zbvp0PTWodUOs+6rop7XR0dLBq1arhC87Furq6OLD/AKavUo/p4MnysHv37sQF5iKtra1xJfx0oDXJh2Jrwh+AjRs3Ir70k3VyBsLKKQXg3XffHa6wXKerq4v//skjjBtluLCs7777nsaNslg2tpM//Wm13hvpxaFDhwiFQv2uzRfKDbF7jyb83rS1t5MWx/HphOeWJPNEQbsWMb9cRHaKyB4R+Vovz39cRGpEZGPk51N2XDcRLMviH2vX0pVTBjLwl8ukZUNWPmvXvjWM0bnLM888Q+XRY9w0tQXvEN55101uJ00MP/7xj+wPzuWi9e5Nbt/JxuQaDh8+TCAwuA/ckc4YQ0dnZ1wJPw2wjKGzc2BdlU6IO+GLiBd4BLgCmAXcKCKzetn198aYsyM/v4j3uomybds26mprCeWXD/rYQF45mzdvOjllO5XV1tay6onHmVvYxdzCod3XyE03rKxoZe3at3jrLf0g7W7v3r2IJ/YN25PywApZWuuph0AggDEm7oQPJPVsZjta+IuBPcaYfcaYAPA7YKUN500KL774IuL1ESwYfMIPFk7BsizWrFkzDJG5y09/+lMCnR3cMj12H+eTOzN5cufpawV3d/nETsqyDQ8/9ENtpXaza9eucOu+n7/oaB//rl27EhCVe0STdKxZNi9gTlbLfCxGmeTosSM94Y8DuneqHols6+nDIrJZRJ4RkQmxTiYid4jIehFZX1NTY0N4Q9fY2MiLa9YQKJgM3sF/9pvMfKycUv74x9UpPVpn8+bNrFmzhismtlOaFXs1oYPNXg42930n1+eBW6a3cKTyKL/73e/sDtWVLMt6f4ROf0aBpImO1Okh2niIlfCPAZ2RnwP0XiY5emwyN0QSddP2z0CFMeYs4GUg5lALY8yjxpiFxpiFRUUDHxUzHJ5++mm6AgG6SucM+RyB0tlUVR3n1VdftTEy9wgGg3z///8ehZmwcpI9LZ+zxgRZVBzgyV+t4rgLlpUbbnv37qW9rf39+vd9EbAKLN55951hj8tNokl6EAPHThM9dkT34QOVQPcW+/jItpOMMbXGmOir8AtggQ3XHVYnTpzg979/imDBJMwgRuf0FMovx2SP4dGf/zyp3wjDZfXq1ezbf4Cbp7WQEc9fUw83T2+DUJAf/Uhv4EbvZ0Tr3vfHlBgOHzqsxf266eoKl+f2xXGO6LHRcyUjOxL+OmCaiEwSkTTgBuDZ7juISFm3X68Bkv775E9+8hMCXV0EJiyK70QidExYTHVVVcp1QdTW1vLYL37BnDFBFhbZ+0dQmGFYOamVv/3tb7z99tu2nttNjDG8/MrL4QVP+r798f4xY8MfDH/961+HLzCXiSbpeNokKZHwjTFB4HPAGsKJ/CljzDYRuU9Erons9nkR2SYim4DPAx+P97rDae3atbzyyit0lp01oAVP+mPljiNYMJknVq1i//79NkToDj/72c/o7Gzn1umtfRZIG6orJnZSmm14+Ic/SOo/suG0detWDuw/gFU+gP77qFFAETz752fDY/fVyXts8ST8aDJN5tfUlj58Y8wLxpjpxpgpxpgHI9vuMcY8G3n8dWPMmcaYucaYFcaY9+y47nCor6/n//v2tyG7gK6xZ9t23s6K87DEz33335/UN3XssmPHDl588UWumNBBWfYgktEg+D1wy7QWDlce5Y9//OOwXCPZPfHEE0i6YMoHN9knNDXE8WPHefnll4cpMnexM+En8wANnWnbTSgU4oEHHqCxsYm2ycvAY2Onsz+T9kkXsHfPHh555BH7zpuEjAlPjhqdDtdMGt7KjHMLg8wd08UTj/8y6WuR2+3NN99k3bp1hGaEBt/5PA4ogJ/+7Ke0trYOR3iuEm2Vx/NFNGVa+CPFE088wbp16+iYeC4ma+CF0gYqlF9OV+lsVq9ezUsvvWT7+ZPFm2++yZYtW/nwpFay4rkLNkA3Tmujvb2dJ598cvgvliQaGhr4zne/g+QKZtoQpvILhOaFqKur46GHHkrqcgCJYFnhb6HxJMTosdFzJSNN+BGvvfYaq1atoqtoOsHiM4btOoEJi7FGl/Ht//ovtm/fPmzXcYplWfz80Z9Rlm1YPjYxXVfjR1lcUNrJ6j/+gerq6oRc00ldXV3c8817aGhsILg4GPOvWDYKsrGPNmsBWDMtXnzxRf70pz8NS6xuEU3S8bTwo8dqwk9yO3bs4IEHHsTKKSFQsYRhucMY5fHQPvUigt4Mvvb1r1NVVTV813LAG2+8wf4DB7l2UuuQ6uUM1bWTO7BCIX7zm98k7qIOCAaDPPjgg2x8dyOhBX0XS5MGQRr6fi+bWQZTZvjhD3/I66+/bmusbmJHwtcuHRc4duwYX/nqV+nyptM+7RJ7++1j8WfSNu0yGptb+fJXvjJi+lCNMfzmf35NSbbh3JLEjpopyrRYUtrJ88/9ecTWzA8EAtx///28+uqrWGdZg75R2ysB61wLM8Zw77338sorr8R/TheKdmkNY1MvKaR0wm9ububLX/4KTa3ttE2/DPwDHMhsA5OVT9vUizhw4CDf+MY9SX1nf6C2bdvGezt3cfn4djwO/OVcWd5BZ6CL5557LvEXH2aNjY3c/eW7ee2118LJfoaNfe4+CF0Qwhpjcd999/HrX/865fr07Uj42qWTxILBIN/4xjc4dOQwbVMuxmQOfTbtUFm54+msWML69ev4/ve/7/o/smeffZYMn3BBmTMzisePspiZH+S5Z//X9a9ld7t27eJTn/4UmzZtwlpsc7KP8kNoaQhrgsWjjz7Kvd+6l7YkX8zDTnYm/GSWkgnfGMMPfvAD3nnnHTorLsDKHetYLMHiGQTGzuW5557jqaeeciyOeHV2dvLG/32dxUUdZCZgZE4sF5Z1cvR4FVu3bnUuCJtYlsUf/vAH7vzMnVQ3VRNcHrSnGycWL5hzDNYci9dee41PfPITKbP+rZ0NhGRubKRkwl+9ejV//vOfCZTNJVg03elw6Bq/kGB+BT/5yU9cWyZgw4YNtLV3cG6ps5PKFhQH8HrCSym6WVVVFXfffTcPPfQQwaIgwUuC4fIJw03AnGEILQtxrP4Yd37mTlatWjUiuhwHwg2t9HikXMLfunUrD//oR4TyJtA1YaHT4YSJ0DllGVZWAfd+6z5XVoBct24daV5hZv7QEsOTOzNPlkd+YP2ofuvix5Llgxl5Qdb9050fnMYYnnvuOW697VY2bNyANd/CWmKF189LpCIIXhokODbIY489xh133jGi12e2o1Ue/bDQFn6SaGlp4Zv3fgvLn03HlOVxD79MO7iWtINr7QnO66d96kW0dXTyrfvuc12LavOmjUzN7cI/xHfUwWYv7SEP7SEP7zX4+62L35eZeV3s3beflpaWIZ/DCYcPH+bz//Z5vvOd79A+qp3gpUHMFONcszMNzLmG0Hkh9h7ey6c+/SkeffTRlKz6OlKkVMJ/5JFHqKmppn3ycvDF32TytNbiaa2NP7AIk5FLe/l5bNu6laefftq28w63YDDI/v0HmJyTHB9Sk0eH43BLi7Srq4tVq1Zx28dvY/P2zVgLLEIXhsJFzpLBeAj+S5DghCC//vWvueXWW1i/fr3TUdlK+/BHmC1btvD8888TKJ2DlVPsdDgxhcZMJZg/kV889phrJmVVVVURDIUoy0qOCSdlkVW1Kisr+9nTeVu2bOHjt3+cxx57jEBJgOBlQcxkB1v1saSBWRTu269qqeKLX/wi999//4iZ82DnKB1N+Engpz/9GZKWRde4eU6H0jcRAuXnEQyGeOKJJ5yOZkBqa8PfcgoykmP8cV56OI5oXMmora2N73//+/zrv/4rR2qPELoghDnPDLimvWOKw3371kyLV/76CjfdfBMvvfRSUie5REm2z+jepETC3759O1u2bKaj7Czwxlq1cnDSDq7F01aLp62WjO3P2deXD5j0HAKF03lxzZqkTlpR0ZnCmb7k+KNP84ZLJyfrDOZ169bxsVs+xp/+9CesaRbBy4JQ1v9xScMLZrYheEmQlrQWHnjgAb76ta9y4sQJpyMbMjtn2ibzh19KJPy//OUviNdv6xBMT2stEupCQl14m4/b2pcP0FV6JqFg0BVr4UZrh3iTqInj8UjS1TQJBAI8/PDDfOlLX6K2s5bQRSHM2Sa+dfVikI0CDUADeF739F1EbahyIbQihDXX4u1/vs2tt93q2uGwOkpnBHnzH2vpGj0OvGlOhzJgJjMPsvJZu9a+bw7DJS0t/Lp2JUePDsZAMGROxpUMjh07xl2fvYtnnnkGa6o17OPqpUGQrshPTf9F1IZ+ITDTw639Vn8r//Ef/8GPf/xj140y0z78EaKhoYETNdWEckrsPXEoQGZmJtdddx2ZmZkQsn/CUVd2MTt2JO3iYCfl5OQA0NKVHG+n9hCEzPtxOW3btm18+o5Ps+fAHkLnhzDzTHxLKyWj0RBcEcSaavHUU0/x5a982VXDYrWWzggRrY9u0u3945dggKuuuorPf/7zXHXVVUjQ/oRvMkbT2tpCe/vwrhoVr6KiIgDqOpOjT6e+I/y2LiwsdDgS2LhxI//+hX+n2WomeFEwvNLUSOUBM89gLbTY8M4GvvDFL7gm6dtZDz+ZW/gOVj1JjJNfLW0ue2x8aTz//PMAPP/88xif/cMrjIQTV7J/PS4oKCDN7+d4W3I0W4+3h+MYO9a5GkkABw8e5Ktf+yqBtADBZUHIcDSchDGTDKH0ELvW7uIb93yD737nu/h8yZ1q7FjiMGVa+CJyuYjsFJE9IvK1Xp5PF5HfR55/W0Qq7LjuQGRnZ4cfBDvsPbE3jfb2dp555plwC3wY7g9IsBMRCXcZJTGPx0NFRTlHWpIj4UfjqKiocCyGYDDIvd+6lw6rg+AFqZPsTxoLofkhNqzfwO9+9zuno+mXHUscpkTCFxEv8AhwBTALuFFEZvXY7ZNAvTFmKvAD4L/ive5AlZaW4vF68bQ3JOqStvG0N1BcUpr0rSOAadNncLAljaF+m20Pyin3RNqDQ29r7W/yMm5s2fsf9g549dVX2btnL8Gzg+BcGI4ykwxmrGHVr1YlfdeOnWvaJtvosO7saOEvBvYYY/YZYwLA74CVPfZZCayKPH4GuFhkONcRfF96ejpTpkzF1+yygmTG4G+tZs7sM52OZEBmzZpFc8BwvG1ob6m2oJxyT6RtiAnfGNjTnM6sM2cP6Xi7rFmzBskRGO9QAF2cOqggsQuQnWTNtOjs6OTvf/+7MwEMkCb8gRsHHO72+xFOvzV1ch9jTBBoJMagNBG5Q0TWi8j6mpoaG8KDJeefh6elGgkk50Sc3niaqzCBNs477zynQxmQOXPmALCrYWjfRrJ8hueff56HH36Y559/nqwhTuKqbvfQ0GFOxuOUvfv3EioIOTf9sotTPkCdSvjkg3iE/fv3OxTAwESTtCb8BDPGPGqMWWiMWRgd/RGvSy+9FIzBV7PblvMlgq9mJ+npGSxZssTpUAakvLycvNE57KgfWsLP9JlT7okMddZu9Ppnn332kI63i8/nAyf/7v2c8gGKPRPMB88Kj1pJ9m7J6MAITfj9qwQmdPt9fGRbr/uIiA/IBRJWM2DChAnMnz+f9JodYCXv/4woCbSRVrePK664nKysLKfDGRARYd6ChWxrSB9yP74dttX5yc/Lpby83LkggNmzZuM94XUu6fs5dVCBUwm/CjAwc+ZMhwIYmJOzxeM4RzSZJvOoOjsS/jpgmohMEpE04Abg2R77PAvcFnl8HfCqSfBg1ZtuugnT2YqvZlciLzsk/mObEQwf/ehHnQ5lUBYuXEh9BxwdYj9+vIyB7Q1pLFy0mATdIorpmmuuwXQYZFdyzE1whAW+7T4Kiwo555xznI6mT3a18IUR3sKP9Ml/DlgD7ACeMsZsE5H7ROSayG6PAWNEZA/wReC0oZvDbdGiRcyaNYuMYxshlLyfwNLZQlr1Di677DLGjXPXLJ0FCxYAsK3Wmebk4RYvjZ3vx+GkefPmsWzZMrzbvVDtdDQOMCCbBFNv+LfP/xt+v1NfMQamqyt8kyNWC7+DU2+C9zbIWxA8IgQCzi7z2RdbmmLGmBeMMdONMVOMMQ9Gtt1jjHk28rjDGHO9MWaqMWaxMWafHdcdDBHhM5/5DKazFf/x5F3g2n9kA16Ph0984hNOhzJoY8eOZWxpCduG2I8fr2114esmQ8IXEb7yla8wfvx4fGt94N5CkoNnQLYKnj0ePvrRj7Js2TKnI+pXMBhEiJ0QOzj1JnisWT0+RngL303OPvtslixZQvrxTUigLa5zWdljMF4/xusnlFOKlR1/JSxPSw3+E7u5/vrrKC0tjft8TliwaDE7GtKxHOjH317vY8K4sZSU2Fw3aYhycnL4wfd/wNjisfj+5kMOp0D3TghkneB5z8PVV1/NXXfd5XREA9LV1YVHBIkxrCqDU2+Cx5pH502FFr6bfPazn8VjLPxH4luiLVB+HlbWGKysMXTM+gCB8jiHTxpD+qG3GJ2bx6233hrfuRw0b9482roMB+JYk3YoQha815jOvAVJsjB9RHFxMT955CfMOmMWnrciZYqTdyJmfFrA+7oXz0EPt99+O3fffTcejztSTDAYxNfHfZ8MTr0JHivhaws/yUyYMIGPfOQj+Gt242mxZ5y/Hbx1+/A0V/GZO+9wdIZovKLDId9LcLfOoRYv7V3G8eGYvcnPz+ehHz7Etddei2e3B++rXmhyOiobGZADgu8VH1mdWTz44IPcfvvtjt84H4xAIGBLAVNv5FzJKuUSPsCtt97K6Nxc0g+9haNjCKNCQTIOr2PK1KlcccUVTkcTl8LCQspKitk9xAlYQxWd8OX0hKtY0tLS+MIXvsCDDz5ITigH3ys+5L3ha+2bPIPxR36KDCZvmN7n7eB504NnnYc5M+fw+C8fZ+nSpcNzrWHU1dVlSyVJLyN/WKbrZGdnc8enP4WnuQpv3QGnw8F/fAt0tvBvn/88Xm9yFCCLx5lzzmJ3c2IXH9nT6KNwTEHS9N/HsnTpUp781ZMsXbIUzxYPvld9UG//dczZBvKAPLCWW+Hfbb0AyF7B95KPtNo0Pve5z/Hwww9TVuamtRrfZ1fC96Et/KR05ZVXUl5RQUblemcnY3W1k35sMxdccEFSdkcMxaxZs2jogLqOxH2l39eSxsxZ7qg7VFBQwAMPPMB9991HLrl4/+pFNgkkb8PwVE2Rvvp3PJx95tmsemIVH/nIR1zTX9+brq4ue7p0jDk5xDMZuff/UJx8Ph+fvesuaG90dDJW2tGNiAlx5513OhaD3aZPD68dfKA5Md06bUGoaoUZM2Yk5Hp2Wb58Of/z6//h6g9cjWeXB99LPjjmdFR9CIWHW/pe9jGqcxRf//rX+eEPf8j48U5ViLOPJvwUcO6553Lm7NnhyVhW4ptX0tmKv/o9Lr/8csdLAdhpypQpQHgiVCJErzNt2rSEXM9OOTk5fPnLX+ZHP/oR4wrG4f27F3lboNPpyHo4Ab5XfHh2eLjk4kv4zf/8hiuuuMJVN2b70tXVhdeG+3le0ISfrESET3/qU+GSC9U7E359/7FNeARuu+22/nd2kezsbEqLixK2IEr0OpMmTUrI9YbD3LlzeeLxJ/j4xz+Or9IXbu33rEjlhFB4xqz3NS+FGYV873vf4xvf+AZ5eXlOR2arYDBoW8LXPvwkNn/+fObMOYv045sT2pcvgTbSanZyxeWXu/ZGV1/KJ03maHtiunSOtnrJSE+nuLg4IdcbLmlpaXziE5/gsV88xuTxk/H+w4usc7BvvwF8f/Xh2eVh5cqVPPmrJ1m8eLFDwQwv27p0gKC28JPbbbfdCp2t+Gr3JuyavuNbwVjcfPPNCbtmIk2cOJHjrd6EjHo91uZl/Phxrr5p2N2UKVN49GePcsstt+A96MX3V194BYkEkn2C71UfuZ5cvve97/GlL33JNZVbh6LLxnH4mvCT3KJFi5g8eQrpx7ckZlx+KEB6zU6WLVs2Im549Wb8+PF0hgwNgeHv463q8DNh4si5BwLg9/v59Kc/zQ9+8ANGe0aHh28moovHAtkgeDZ4WDBvAaueWDViW/XdBYNBW5KhBx2Hn/REhBtvvAHa6vE2Hhn26/mqd2GCndxwww3Dfi2nRCt9VrUNbz9+yIITbeHCbSPR/PnzefyXjzN96vRwF8+eYfwADYL37148+zzcfPPNfPe73yU/P3/4rpdEQqGQLclQJ165xEUXXUR+fsGgKmla2WMGXzTNWKRXb2fWmWcya1bPtd5Hjuh9ier24X2L1XV6CJmRm/AhPHv5Rw//iCVLluB514PsHIakH0n2Uh2u8nnnnXeOiEmAAxWysYUfXR83GWnCj/D7/XzoQ9fibaxE2gY29TFQft6gi6Z56w9BRxMf/chHhhKma5SWliIi1Axzwo+efyQnfICMjAzuv/9+VqxYgWezB9nff9I3eQMsqWCBd60XOSHcc889fOADH7AhYnexLMu2hK/F01xi5cqV+Px+/FXbhu0aaVXbKCwqdmW9kcHw+/0UFuRT05GYhD8SRzr15PP5+M///E8WLFyA9x1vvzX2zdlmQCUVZIvAcbj77ru5+OKLbYrWXSzLsmW9eSG8hm+y0oTfTV5eHv9y2WWk1e6BYKwlDobO01qLp+kY1334Q0m/qLMdxo4bT0378HYLVHd48Hg8rh+SOVB+v5/7vnUfJSUl+P7pg3gHhBwHzy4P1157LVdffbUtMbqRMcaWhB89V7LShN/DddddhwkF8Q/DRCzf8a2kpaenzB/W2HHjqOkc3qXtato9FBeOSYkP0KicnBzu+cY90BYudTBkQfC942Ni+UQ++9nP2hdgCkv2ecea8HuYMmUK8xcsIL16u60TsSTQhr9uL1ddeSU5OTm2nTeZjR07lrp2Q+cwdmkeb/MxbsLE4btAkpo9ezZXX301nn0eaBnaOWS3YFoNd3/pbtLT0+0NMEUlb9s+LK6ELyIFIvKyiOyO/NvrGC4RCYnIxsjPs/FcMxFuvOGGcLmFWvuW3vVVbUOM4frrr7ftnMkuOsdgIEMzy3NCZHotMr0WZ+R1UZ7T/6eEMVDV7huxcxn6c/vtt+P3+pEdQ2hXBsG728u55547Yqq0xkNEbEnWJnKuZBVvC/9rwF+NMdOAv0Z+7027MebsyM81cV5z2C1evJjyigr7JmKFAqTXvMfSpUtTKjlNnBhueR9t6/9tdsuMdspzQpTnhPjPhS3cMqO932OaAkJrlzl5nVQzZswYrrrqKryHvYMutiaHBNNpRuxM78HyeDy2JfxknvEdb2QrgVWRx6uAD8Z5vqQgInzs5puhrQ5vw+G4z+erfg/T1clNN91kQ3TuMXHiRDwiVA5TEbXK1vB5KyoqhuX8bvDBD34QEzLIwcG1Kj37PVRMquCss84apsjcxev12rL4WChyrmQVb8IvMcZEK3gfB2ItN5QhIutF5C0R+WCc10yIiy++mKLiYtKObYqvlW+FyKjaxrx580b0RKvepKenM7asdNjKJB8eAVUy4zV58mSmTpsabuUPVDNQB1ddeVVSdz8kks/vx45bTRYuT/gi8oqIbO3lZ2X3/Ux4LFKszFhujFkI3AT8UESm9HG9OyIfDutrapxbZNzn83HzTTfhaa7C0zz0VSl8Nbswna3ccsstNkbnHlOnz+BQ6/Asd3iw2Uve6BzGjBnkbOcR5tJLLsXUmQHfvJUj4SS/YsWKYYzKXdLS0mwpShqEpL4B3m/CN8ZcYoyZ3cvP/wJVIlIGEPm3OsY5KiP/7gNeB+b1cb1HjTELjTELi4qKhvCfZJ+rrrqKvPx80o5uGtoJLIv045uZccYZLFiwwN7gXGL69OlUt0Frl/0tyQMtfqafcUbKt1KXLVsGgFQO7HXwVHqYOWtmysxdGIi09HRbEn6I8IdHsoq3S+dZILp6x23A//bcQUTyRSQ98rgQWAJsj/O6CZGens5NN96It7EST3PVoI/31e6BjmY+ftttKZuUZs6cCcDeJnu/5naE4HCLh5kzU6ubrDdjx45l8pTJeI4O4M+5DaiHZRcuG/a43CQ9PZ2gDX+jXUBGEpeRjjfhfxu4VER2A5dEfkdEForILyL7zATWi8gm4DXg28YYVyR8gGuuuYZROaNJO7pxcAcai/Rjm5k8ZSrnn3/+sMTmBmeccQYeEXY32Dsxal+jD2NIufsisVy49MJwqYV+JohHvwWM9NIeg5WZmUnAhoQfEHF3l05fjDG1xpiLjTHTIl0/dZHt640xn4o8/ocxZo4xZm7k38fsCDxRsrKyuOGjH8HbcBhPa+2Aj/PW7Yf2Bj5+260p27qH8HKHU6ZMZmeDvTNudzb4EBFmz55t63ndKprA5Wjf7zXPUQ8TJk5gwoQJiQjLNbKysrBjYcKACNnZ2TacaXgk74DRJHLttdeSmZmFf6CtfGNIP7aZceMnaEsKmDd/Abub/ARsnHG7vd7PtKlTUmbWcn+mTp1KcUlx3/34nUANLF+2PFFhuUZmZiYdNgzE74SkXhlME/4A5OTkcO21H8RXfwDpaOp3f29jJdJayy0fuzmph2glyoIFC+gKGXbZ1K3TEYJdjT4WLFxky/lGAhFh+bLleKo9MQuqyVEBAxdeeGFig3OBnJwcOo2FFWOgYRlQ0e0nVm3WDmMYNWrUMERoD034A3Tdddfh9XoHtECK//gW8vMLuPTSSxMQWfKbO3cufp+XzXX2dOvsqPcRssJLU6r3LV++HGOZmN06ckQoKS1h+vTpCY4s+UW/Kca6BXIlwie7/VzZS5m0IIaAJvyRobCwkEsvuYS0E7shGLu3T9rq8TZWcv311+H3D2+lSLfIysrirLlz2VRrz82sjSf8ZKSn6yzRHmbNmsWYwjEnx9mfIgCeag8rlq9I6XtKseTm5gLQf0GP2KLH5uXlxRvOsNGEPwgf/vCHMaEufCd2xdzHX7Udn9+fkqsG9eX885dQ2SJUDaCuTl+MgXdrM1i0eHFSj3d2gscTTuieqtO7deSoYCzD8uXLHYkt2UUTfmsc54geGz1XMtKEPwgzZszgjDNmkl6zs/dyC6Eu0ur2ctGKFUn9Ke+ECy64AID1NfF969nf7KWu/f3zqVNdeOGF4do6x09txUulUFhUeHJehDpV9O91iJWmgfcTfjL/7WvCH6Rrrrka2urxtJxe9sFXdwATDHDNNUlfEDThysrKmDZ1Cuuq4+vWWVftx+v1sGTJEpsiG1nmzJnD6NzRUNltYxA8VR6WXbhMu3NiKCgoAOJL+NFjo+dKRprwB2nFihX4/WnhWbQ9+Gr3UFJaxpw5cxyILPmtuOhi9jR6OdExtKRjDLxdncn8+fMZPXq0zdGNDF6vlyXnL8Fb5eVk+cdqMCGT0hMA+5Ofn4+I0BzHOaLHFhYW2hHSsNCEP0jZ2dmcf/55pNUfANOtoGpXB96mo1x26SXaioohWqzr7aqh9b3va/JS3QYXXZSaC20P1LnnnosJGKgL/y7HhbT0NObOnetsYEnM5/ORl5sbV8JvAjLS03Uc/kizbNkyTKDtlG4dX8MhMEbHOPdh3LhxnDFjOmurMoZ0/FtVafh9Xn2N+zF//nxEBKkONzy8NV7mnT1Pb3L3o7i4mMY4jm8Eih0u+NgfTfhDcM455yAipyyO4m04TH7BGB3j3I9LLr2MA00ejrYO7q1nGXirOoNzzj1PZ9f2Izc3l4pJFUiNQCeYJqPLGA5AcUkJTXGsVtUkQnFpqY0R2U8T/hDk5ORwxhkz8TUfDW8wBn/zMc5ZvEi7c/px8cUX4xHhzeOntzajSxz2Znu9j/oOuOyyy4Y7xBHhrDln4WnwQKT8k9Yc6l9ZWRkNxmCGuNhhgwhlZbHm4CYHTfhDNH/+PDwtJyAURNrrMV0dzJsXs8y/ihgzZgwLFizgH1WZp41svWVGe8y1bN88lkZWZibnnXdeAqJ0v+nTp2MCBqkURIRp06Y5HVLSKy0tJWAMbUM4NoChxbIoKYm16F9y0IQ/RLNnzwZj4Wk7gbel+v1tql+X/cu/UNMGuxoHVmeoMwTrajJYvmJFUpeeTSaTJ08GQA4LxSXFSX0jMVlEW+d1Qzi2PvLv2LFjbYtnOGjCH6IzzjgDAE/rCTyttWRmZjFu3DiHo3KHpUuXkpGexpvHBpa836nx0xE02p0zCBMnTgRAQkL5xHKHo3GHaLIeSsKPHqMJf4QaM2YMOaNz8dUdxNt8jClTJuOJ44ZPKsnKymLJBUt5uyaDoNX//m8eT6eocIzeeByEnJwcsrLDrXptiAyMHQl//PjxtsUzHDRDxWHO7Nl4m4/haW/QyVaDdNlll9EaMGyq7bvUQnNA2FLr55JLL9MP1EG6/777ueuuu7jpppucDsUV0tPTKRozhhNDOLYWGJWdnfQTAu1ddy7FPPDA/TQ0NADhFr8auEWLFjE6ZxRvV3WyoChGAXfCpRRCBi655JIERjcyLFq0SEtID9LE8nKq6uoY7ECdE+CKVcS0yRQHn89HYWEhhYWFOhxzkHw+H8tXXMSGExl09rES1ltV6UwYP46pU6cmLjiVsiZMnMgJkUEPzazzeJhYnvz3SjThK8esWLGCzqBh04neu3UaA8J7DT4uuljLVajEKC8vp8OyBlVErRNDo2VRPtITvohcLyLbRMQSkYV97He5iOwUkT0i8rV4rqlGjrlz5zI6Z1TMksnv1PixDFrDXSVMdHTT6bVwY4vuGz02mcXbwt8KfAh4I9YOIuIFHgGuAGYBN4rIrDivq0YAn8/HBUsvZFNt76N1NtSkUVZacnJMuVLDraKiAhhawo8em8ziSvjGmB3GmJ397LYY2GOM2WeMCQC/A1bGc101cpx//vm0dp2+wHlnCLbVp7HkgqXanaMSprCwkKzMzEEnfJ/Xm/Rj8CExffjjgMPdfj8S2dYrEblDRNaLyPqamsG87MqNFi5ciM/rZXOP4Znv1fvoChnOPfdchyJTqUhEKC8vp3oQx1QTHn/v8yX/oMd+E76IvCIiW3v5GZZWujHmUWPMQmPMwqIkLzWq4peVlcWZZ57J1vpTi6ltqfPj9/u0hrtKuEmTJ1M9iDkfNR4Pk1zS7djvR5IxJt4B0JVA9wGq4zl1ATaV4hYsXMjjmzfT2iVk+8PD4XbUpzFn9hytnaMSbtKkSbxgWbQC2fTdnRjAUG8ZJk2alJjg4pSILp11wDQRmSQiacANwLMJuK5yiblz52KA3ZFiam1BONziYa6WUlAOiCbvgXTr1BCeo+WGG7YQ/7DMa0XkCHAe8LyIrIlsHysiLwAYY4LA54A1wA7gKWPMtvjCViPJzJkz8Xg87I7cuN3X6MMyWn1UOWMwCT+6j1tGksV1l8EYsxpY3cv2o8CV3X5/AXghnmupkSsjI4NJFRXsawwP+NrXFH5bRiuSKpVIhYWFjMrKoqqt/8r41YSHF7thhA7oTFuVJKbPmMGhlvCN24MtXspKinUpQ+UIEaFi0qQBtfCrgPKJE10xQgc04askMWXKFBo7DY0B4UirnynTdG1g5ZzoSJ3+auq4aYQOaMJXSSJ60+twi5fjreKam2BqZJo0aRLt/dTU6cDQYFmuGaEDmvBVkoguHLG1NlwO2Q2lZtXINZAbt24qqRClCV8lheLiYrxeD9vrw32h0fVFlXLCQGrqRJ/TFr5Sg+Tz+SgcU3ByhE5JSYnDEalUVlBQwKjs7D5b+NWA3+dzVeNEE75KGkVF7yf5wsJCByNRqS46Uqe/Fv7EiRPxer2JCitumvBV0iiILBOZMyobv7/vtW6VGm4VFRXU9FFT54THQ4WLunNAE75KIsXFxaf8q5STysvLabUs2noZmtmFod4lq1x1547ZAiol3H777cybN89VN8HUyNV99aueab2WcA0dt40m04SvkkZOTg5Lly51OgylgPcT/glOT/jRvn23tfC1S0cppXpRUlKCz+ultpfnotvGjYu5llNS0oSvlFK98Pl8jC0ri5nwCwvGkJmZmeiw4qIJXymlYhg/cSJ1vYzUqUUYP9Fd/fegCV8ppWIaN24cdcacVkSt3iMny4G4iSZ8pZSKYezYsQSMOaWIWieGFstyTQ387jThK6VUDNGkXt9tW32P59xEE75SSsUQrZPTW8IvLS1NeDzx0oSvlFIxRIv4NXTbFn3spqJpUZrwlVIqhszMTEbn5JyW8NP9fvLy8hyJKR5xJXwRuV5EtomIJSIL+9jvgIhsEZGNIrI+nmsqpVQilZSU0Njt90agqKgIEXEqpCGLt4W/FfgQ8MYA9l1hjDnbGBPzg0EppZJNSWkpTd3G4jeJUOLC/nuIM+EbY3YYY3baFYxSSiWboqIimrr93iRCkUsruiaqD98AL4nIBhG5o68dReQOEVkvIutravpafkAppYZfUVER7ZZFAIOFodmyKCoqcjqsIem3WqaIvAL09v3lP4wx/zvA61xgjKkUkWLgZRF5zxjTazeQMeZR4FGAhQsXnl6IWimlEmhMZGGeFiANsLptc5t+E74x5pJ4L2KMqYz8Wy0iq4HFDKzfXymlHBVN7s2EEz6E17x1o2Hv0hGRbBHJiT4GLiN8s1cppZJefn4+EG7hR0ssuLWFH++wzGtF5AhwHvC8iKyJbB8rIi9EdisB/i4im4B/As8bY16M57pKKZUovSX86Da3iWvFK2PMamB1L9uPAldGHu8D5sZzHaWUckp0glUrEOyxzW10pq1SSvXB5/MxKjubNsJJ3+f1kp2d7XRYQ6IJXyml+jF69GhOEF7LNicnx5WzbEETvlJK9aukpIS9wHtAsUvH4EOcffhKKZUKvnnvvezfvx+AiRMnOhzN0GnCV0qpfhQUFLh27H132qWjlFIpQhO+UkqlCE34SimVIjThK6VUitCEr5RSKUITvlJKpQhN+EoplSLEmORdY0REaoCDTsfRj0LghNNBjCD6etpLX097ueH1LDfG9DodOKkTvhuIyHpdmN0++nraS19Pe7n99dQuHaWUShGa8JVSKkVowo/fo04HMMLo62kvfT3t5erXU/vwlVIqRWgLXymlUoQmfKWUShGa8AdIRH4pItUisjXG8yIiD4vIHhHZLCLzEx2jW4hIhoj8U0Q2icg2EflWL/uki8jvI6/n2yJS4UCoriEiB0Rki4hsFJH1vTyv788BEJEZkdcw+tMkIv/eYx/Xvpaa8AfuCeDyPp6/ApgW+bkD+O8ExORWncBFxpi5wNnA5SJybo99PgnUG2OmAj8A/iuxIbrSCmPM2THGiev7cwCMMTsjr+HZwAKgDVjdYzfXvpaa8AfIGPMGUNfHLiuBX5mwt4A8ESlLTHTuEnmNWiK/+iM/PUcPrARWRR4/A1wsbl05Ojno+3PwLgb2GmN6zvZ37WupCd8+44DD3X4/EtmmeiEiXhHZCFQDLxtj3u6xy8nX0xgTBBqBMQkN0l0M8JKIbBCRO3p5Xt+fg3cD8Ntetrv2tdSErxxhjAlFvjaPBxaLyGyHQ3K7C4wx8wl3N/yriFzodEBuJiJpwDXA007HYidN+PapBCZ0+318ZJvqgzGmAXiN0++PnHw9RcQH5AK1CQ3ORYwxlZF/qwn3OS/usYu+PwfnCuAdY0xVL8+59rXUhG+fZ4FbI3fwzwUajTHHnA4qGYlIkYjkRR5nApcC7/XY7Vngtsjj64BXjc4S7JWIZItITvQxcBnQczSZvj8H50Z6784BF7+WPqcDcAsR+S2wHCgUkSPANwnfbMQY81PgBeBKYA/hO/u3OxOpK5QBq0TES7jR8ZQx5jkRuQ9Yb4x5FngMeFJE9hC+WX6Dc+EmvRJgdeSetg/4jTHmRRH5DOj7c7AiH5qXAnd22zYiXkstraCUUilCu3SUUipFaMJXSqkUoQlfKaVShCZ8pZRKEZrwlVIqRWjCV0qpFKEJX6kBEpHXReS0SpQislBEHo5xzAERKexl+70icvdwxKlULDrxSqk4GWPWA6fVoFcq2WgLX7maiFSIyA4R+XlkMZWXRCSze2tcRApF5EDk8cdF5E8i8nKk9f05EfmiiLwrIm+JSEE/l7w+snjLLhFZGjnnchF5LvJ4TCSGbSLyC+BkSWcR+Y/IcX8HZnTbPkVEXoxUuvybiJwR2f5EZKGNf4jIPhG5zs7XTqUeTfhqJJgGPGKMORNoAD7cz/6zgQ8Bi4AHgTZjzDxgLXBrP8f6jDGLgX8nXF6jp28Cf4/EshqYCCAiCwiXhzib8LT8Rd2OeRT4P8aYBcDdwE+6PVcGXAB8APh2P7Ep1Sft0lEjwX5jzMbI4w1ART/7v2aMaQaaRaQR+HNk+xbgrH6O/WM/17mQ8IcJxpjnRaQ+sn0psNoY0wYgIs9G/h0FnA883W19l/Ru5/uTMcYCtotIST+xKdUnTfhqJOjs9jgEZAJB3v8Gm9HH/la33y36/5uI7hsawL4D4QEaImsD9HU96NY9pNRQaJeOGqkOEF6TFMLllRPlDeAmABG5Asjvtv2DkfsLOcDVAMaYJmC/iFwfOUZEZG4C41UpRBO+Gqm+B9wlIu8Cpw2LHEbfAi4UkW2Eu3YOARhj3gF+D2wC/gKs63bMzcAnRWQTsI3wmqlK2U7LIyulVIrQFr5SSqUIvWmrVA8i8giwpMfmh4wxjzsRj1J20S4dpZRKEdqlo5RSKUITvlJKpQhN+EoplSI04SulVIr4f5y9/KdrgA7wAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "weights_vp(tanh_model3, att=\"grad\")\n",
    "\n",
    "# 层间梯度不均衡"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEXCAYAAACu1P9TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABUnUlEQVR4nO2deZhU5ZX/P+dWdVfvC/TG3myyNJuAICJG3EZcwbjhvkSjxjgzyfxmnywzk4zmmRlnMjEmxiSaGOOOoLihooiCskMD3SwNvdH0Bt30Wl1V9/39UVXYQDe9Vd1bVf1+nqcfqu69dc+pS9W33nve854jSik0Go1GE/sYdjug0Wg0GmvQgq/RaDSDBC34Go1GM0jQgq/RaDSDBC34Go1GM0jQgq/RaDSDBC34Gk2YEZEficgLdvuh0WjB12g0mkGCFnyNpg+IiNNuHzSa/qIFX6MBRGS2iGwTkSYReVVEXhaRfxeRi0WkQkT+TkSOAr8XkUwReVtEakXkeODxyE7nGisinwbOtQbIsu+daTRfowVfM+gRkXhgBfAcMAT4M7Cs0yF5ge1jgAfxf29+H3g+GmgDftHp+BeBLfiF/t+Au8P6BjSaXiK6lo5msCMiF+EX+ZEq8IUQkfXAJ8CHwAdAmlKqvZvXzwLWKqUyRWQ0UAKkK6VaAvtfBEyl1B3hfi8azdnQI3yNBoYDlerU0U95p8e1ncVeRJJE5NciUioiJ4B1QIaIOALnOh4U+wCl4XReo+ktWvA1GqgCRoiIdNo2qtPj02+Dvw9MAuYrpdKAiwLbJXCuTBFJ7nT86BD7q9H0Cy34Gg1sAHzAoyLiFJHrgXlnOT4Vf9y+QUSGAD8M7lBKlQKbgR+LSLyIXAhcGz7XNZreowVfM+hRSnUANwD3Aw3AHcDbgLubl/wPkAjUARuB907bfxswHziG/8fgD6H2WaPpD3rSVqPpAhH5EviVUur3dvui0YQKPcLXaAAR+YaI5AVCOncDMzhz5K7RRDV61aBG42cS8AqQjD+t8kalVJW9Lmk0oUWHdDQajWaQoEM6Go1GM0iI6JBOVlaWys/Pt9sNjUajiRq2bNlSp5TK7mpfRAt+fn4+mzdvttsNjUajiRpEpNuV3Tqko9FoNIMELfgajUYzSNCCr9FoNIMELfgajUYzSNCCr9FoNIMELfgajUYzSNCCr9FoNIMELfgajUYzSIjohVeRzr/9+0/YuHEjADfd+E3uueceex3SaDSas6BH+APg88/X0+B10NjuY8OGjXa7o9FoNGdFj/D7SVtbG60tLfhGTkHaGzlaXW23SxqNRnNW9Ai/n1QHBN50paJcKTQcP4bH47HZK41Go+keLfj9pKrK3xtDuVJQrlSUUtTU1NjslUaj0XSPFvx+UllZCYCZkIbpSjtlm0ajiS18Ph9utxu3243X67XbnX4TEsEXkd+JSI2IFHaz/2IRaRSR7YG/H4TCrp1UVFQgznhwJqAS/IJfXl5us1cajSbUeL1ebr7pJi6//HIuv/xybli2DLfbbbdb/SJUI/zngCt7OOYzpdSswN+/hsiubRw+fBhfQjqIoOISEWc8paXdlqHWaDRRSkNDA7V1dUwFpgMNjY1RG74NieArpdYBx0Jxrmih5NBhfAkZ/ici+BIyOHz4sJ0uaTSaMFBfXw/ATGDOaduiDStj+AtEZIeIvCsiBd0dJCIPishmEdlcW1troXu9p6GhgYbjxzCTMk9u8yVmcuDgQXRTeI0mtgjqUBqQetq2aMMqwd8KjFFKzQT+D3izuwOVUs8opeYqpeZmZ3fZltF2Dh48CICZNOTkNjNpKM1NTVH7QdBoNF3TWfDTT9sWbVgi+EqpE0qp5sDjd4A4EcmywnY4KC4uBsBM+votmMlDT9mn0Whig5qaGhwipAAuhATDGNwx/J4QkTwRkcDjeQG70RkEA/bu3YskpkFcwsltZtJQEKGoqMhGzzQaTaiprq4mXQQDASADOHr0qK0+9ZeQlFYQkT8DFwNZIlIB/BCIA1BK/Qq4EXhYRLxAG3CriuJgd2HhbjxJp4WbHE5U0lAKd++2xymNRhMWjlRWkmGaEBR806QqStfchETwlVLLe9j/C+AXobBlN9XV1dTX1+EbM+GMfd7kbPbs2YPX68Xp1GWKNJpYoOrIEcZ1ep4BbD96FKUUgcBF1KBX2vaRHTt2AGCm5p2xz5eWh7u9nQMHDljtlkajCQOtra0cb2wks9O2oUC7283x48ftcqvfaMHvI9u3b0ecrlMydIIEfwS2bdtmtVsajSYMBMulDO20LfjNr6iosNyfgaIFv49s3rIFT2oeyJmXTsUnQ2IGW7dutcEzjUYTaoLlUjoLfjA3Twt+jHPkyBGOVlXhSxve7TGetOFs276djo4OCz2LDcrKyli3bh27du2y2xWNBuha8NMBhwhlZWW2+DQQ9MxiH/jqq68A8KWP6PYYX/oIOqr3UFhYyOzZs61yLSb4h7/7W8orjwDwxhtvkJUVtUs1NDFCaWkpmYZBvPl1UqEDIUskKmtn6RF+H9i4cSMkpKES0rs9xpc2DAzjZK9bTe9QSnG0uoZhST7g6wYzGo2dHC4pYahpnrE9yzQ5VFJig0cDQwt+L3G73f74fdoIOFsqliMeX0oen3/xhXXOxQCNjY14vF4mpvtrjUfr0nVN7ODz+SgtKyO3i305QNXRo1FXJlkLfi/ZunUrHW43vszRPR7rzRxNeVlZVE7q2EVwRD8p0y/40bqSMVLwer0888wz/Od//ieffPKJ3e5EJZWVlXi8XnK62JeL/6402irkasHvJevXr0cccf6QTQ/4MsYA8Pnnn4fbrZjhyBF/7D4/1UdSnJxsIanpHwcPHuSFF15g1Vur+NWvf2W3O1FJSSBkc+aKG06O+kuiLKyjBb8XmKbJus8+w5M+Eoye57lVQiokD2Xdus8s8C42CN4N5Sb6yE306rujARIs7qWGKurq6nTZ7n5w4MABDKCrmr1DgHgR9u/fb7FXA0MLfi/YvXs3jQ0NeDPze/2ajozRFBbu4tixQdUXpt+UlpYyNBESnDA8yUvpoegaOUUawTskla3ocHfQ2Nhos0fRx/79+8kyDOI4c87OQMgBDmjBjz0+/fRTMBz4Mkb1+jXeIfkopfjsMz3K7w0lBw8wPMkDwIgUHzV19TQ1NdnsVfRy5MgRJE5QQ/wje33H1Hf2FxeT10WGTpBhSrF/376ounvSgt8DSik+XvuJf7GVM773r0scAonprNUTZj3i8Xg4fLiUMSn+lMzgv8FGM5q+U1ZWhkpVJ1s0ReMiITs5duwYdceO0f2KGxgOtLS1RdV8kxb8Hti7dy91tTV4h4zr+eDOiNCRmc/2bdv07XQPlJSU4PX5yE/zZ+jkp/oFX/cW6D8HSg5gppqQDOIQDh06ZLdLUUWwkdHZUjSC+6Kp6ZEW/B745JNPwDDw9iId83R8Q8ZimqYO6/RAYWEhABMCOfjpLkV2kn/uRNN3GhoaaDjW4K8BYIBKU7qCax/Zt28fwtkFPxd/iYVoGphowT8L/nDOWnxpI8Dp6vPrzaShkJDGxx+vDYN3scPOnTsZkgBDXV/HQiemudm5Y3tUxUcjhb179wKcjN+bGSZFxUX6WvaBoqIisgyDhC4mbIM4EfKAYi34sUFxcTE11dV4h4zt3wkCYZ2tW7fosE43KKXYtnULUzLcpyxgnpLp5XhDY1TWK7GbPXv2+JszZQQ2DIWW5hYdx+8lSin27t7N8LNM2AYZrhRFRUWYvTg2EtCCfxbWrl0L0r9wTpBgWGf9+vUh9Cx22L9/Pw2NJygY4j1le/B5sGCdpvfs2LEDyZRAk1FQWf6Rva5C2jtqa2s51tDAyF4cOxJobWuLmiwoLfjd8HV2zjBwJvT8gm4wk7MgIZVPPv00hN7FDsEiczOGek7ZnpNoMixZsXHjBjvcilra29spLCzEl+X7emMKGIkGmzdvts+xKCIYEjtbhk6Q4DF79uwJmz+hRAt+Nxw4cIDqo1X9D+cEEcGTkc/mTZtobm4OjXMxxGfrPmV8uo8M15nx5TlZbrZt26bz8fvA9u3b8Xq9qNxO11PAm+Plq01f4fP5un+xBvALvkOky5IKp5MNuERO/khEOlrwu2HdunUggjdzzIDP5R2Sj8/nY8MGPVrtTFVVFcX79jM3u+uKg3NzOvD5dDisL3zxxReIU86sBzAMmpuao2Ykaid79+whD7pcYXs6BsJwBXuiJKMsJIIvIr8TkRoRKexmv4jIz0XkgIjsFJGI7wzy6bp1/h61cYkDPpeZkoPEJ/l/RDQn+eijjwA4P9fT5f7xaT6yk+CjDz+00q2oxefz8cmnn2DmmuA4dZ/KU4gh/lXjmm7x+XwU7d3LyD5kNI1AcfDgwagolRyqEf5zwJVn2b8EmBj4exB4OkR2w8KRI0c4fOgQnoz+T9aegggdGaP58quvdOvDAEop3nv3Hc7J8JGd2HWGgwhckNPG5s2bdX38XrBz504ajjegRnYhVnFg5pp89PFHUZNRYgdlZWW0ud29it8HGQl4fb6oWOsQEsFXSq0DzlYl7HrgD8rPRiBDRHquM2wTXwSal/Sm9n1v8WWMpr2tje3bt4fsnNFMYWEhZeUVfGNY+1mPWzS8A1Mp3nvvPYs8i17ef/99f/2c4V2PTtUYRX1dPdu2bbPYs+ghGPLqTYZOkGCFrWiI41sVwx8BlHd6XkE3k+Ai8qCIbBaRzXaN6jZs2ACJGWdtZdhXfGnDEcOpWx8GePPNN0l0CvNzz37Hk5dkMjXTy6qVb+oJx7PQ0tLCRx9/hG+Er9tO1Wq4QuKFt99+21rnooi9e/eSYBinNC3viTSENMPQgt8flFLPKKXmKqXmZmd3VYk6vLS3t7Nt+3Z/7ftQ4nDiTRvGF1/oidvjx4/zydqPWTSsjYSe2wtw2ch2qmtqT955ac5kzZo1uNvdqHFniT07wDfaH+dvaGiwzLdoYu+ePQw3TYxeTNh2ZrhpsjcKJsStEvxKvr7zAf8dU6VFtvvEjh078Ho8+EIt+IA3fQRHjlSe7O40WFmxYgUer4/LRvZukmtOtoesRHj1lZfD7Fl0Ypomr7z6CmTi78xxFtR4hc/rY+XKlZb4Fk243W5KSkr6FL8PMgKoqKyM+BRiqwR/FXBXIFvnfKBRKRWRNUU3b97sr32f2pss3L7hS/d/lLZs2RLyc0cL7e3trHjjdc7N8jA8uXeThw4DrhjZyvYdO6PittlqvvzySyrKKzAnmvQ4ME3zZ+y89vprUZFVYiUHDx7EZ5r9Evzg8HDfvn2hdCnkhCot88/ABmCSiFSIyP0i8pCIPBQ45B2gBDgA/AZ4JBR2w8GWrVsxU3LA0XOsIb50A/GlvQ/RqIQMxJU0qCfN3n77bRpPNHFtflufXnfxCDfJcfCnP70QJs+ilz++8EckWVCjepdKaE4yaWxo1BPhpxEsc9wfwR9+2jkilV5EUHtGKbW8h/0K+E4obIWT5uZmDh44gHf4rF4db7TU982ACJ6UPLZs3YZSCpG+xQmjnY6ODv784p+YlOHlnIy+TcAmOeHykW2s/OwzDh06xNixA1wBHSNs3bqVwl2FmLPM3g/fsoGh8Pwfn+eqq64iLi4unC5GDfv27SPZMEjvR9pqEkKmIYNjhB8r7N69G6VUWMI5QXypeRw/Vs/Ro0fDZiNSeeedd6itq2fZ2L6N7oP8xSg3Lofw/PPPhdaxKEUpxbO/fRZJkrNP1p6OgG+qj7qaOlavXh0+B6OM4qIi8kwT6eOEbZBhpklxhIccteB3orCwEET8IZ0wYabkfm1rEOF2u/nD888xMcN3RmXM3pIar7h8ZCtr167V7Q/xF54r3FWIb7LvjJW1PZILZMHvfv872tr69wMcS/jbbB4+GZrpD8OAyqoqWltbQ+VWyNGC34nde/ZA0hBwhO8W10zKRBxxg66myapVq6irP8aN41oZSCTr6jFuEpzC7373u9A5F4V4vV6e+uVTSKqgxvajsYmAb7qPhuMNvPLKK6F3MMooKyvD6/P1qmBadwRfG8mDES34AZRS7N1bhCepL0su+oEY+JKGsjeKuuQMlNbWVl74w/NMzfT2e3QfJCVOsWRUK5999tmgzthZvXo1ZaVleKd5+/8tzgI1QvHCn16grq4upP5FG0GRDoXgl5SUDNifcKEFP0B1dTUtzU3++vVhxpc8lAMHDgyalaOvv/46xxtPcNP40NzqXjm6ndR4ePY3vwnJ+aKNEydO8Otnfu2ffO1PSkknzBkmHR0dPP10RJe3CjsHDx7EIdKnFbankw4kiOgRfjQQLHxkhnuEH7DR4XYPigVYJ06c4MU/vcC5WR4m9jEzpzuSnHDtmFY2bd48KFNcn332WZqbm/HN8vWcd98TKeA7x8eaNWvYsWNHSPyLRg4dOkS2CI4BXFBByFb+c0UqWvADBH+VzaTMsNsyk/zLIaOhut5A+fOf/0xraxs3jQ/txOBlI90MSYBnfv3rQdWce+/evaxcuRJzgvl1z9oBoqYoJFn4z//6T7zegYXcopXDJSVkh6CKaA6KwzqkE/kcPnwYSUgDR3zYbZmJGSdtxjL19fW8/tqrnJ/bwejU0Iav4h1wfX4Lu/fsGTQF6bxeL0/87AlIAFUQwh85J3hneSk9XMpLL70UuvNGCW63m+ra2jN6xvSHbKCxqYkTJ06E4GyhRwt+gJKSQ3hcadYYM5xIYnrMC/6LL75IR0cHN4wLT9rfN4Z3kJMEz/7mmUExyn/11VcpOViCd5b3ZIPysyHbBdneyxDFcP8E7u9///uoacgdKioqKlBKDSh+HyQ4A1hWVhaCs4UeLfj4i09VVFSgAiNvK/C40jh8uNQye1ZTV1fHyjffZGGem2G9rJnzx+JE/ljc+w5jTgOW5rew/8DBmG+DWFFRwbO/fdZf676XE7XSIEhD72PS5rkmXvHfRQymJinBH7hQCH6wdl2k/mhqwQdqamrweDpOhlqsQCWkU1FREbNfrJdeegmv18PSsWdvcNKZ0iYHpU19W0G0MK+D3CTF888/F7OjfNM0efyJx/HixZzdiwJp/SXRn5u/Y/sO3nrrrTAZiTyCyROhEPxM/P89kZqQoQUfKC/392YxEywK6QBmQjoeT0dMtu5rbGxk1co3WZDrJjcpvD9oDgOuGdPKvn372bRpU1ht2cXKlSvZuWMnvhk+GHiL5bOixirIhad++RTV1dXhNRYhVFVVkWQYJITgl9SJkG4YWvAjmeDtVyg7XPWEGbAVqbd+A+HNN9+k3d3Btfm9H90PhEXDOshMgD//+UVL7FnJkSNH+OXTv4Q8UPkW3MEI+Ob4cHvcPP7E4zF719SZqqoqMkL4PjNMk6NVEVn9XQs++EVXHHGouCTLbKrA3USsCb7H42HFG68zY6iHkSnWhKucBlw+spUtW7ZG9KKXvmKaJj/9j5/SYXbgmxOCnPvekgy+GT62bN7CqlWrLDJqH9VVVaSHUPDTQQt+JFNRUeEXYAvLFav4ZMRwngwnxQrr16/n2PEGrhhlzeg+yOLhHcQZxJRAvfHGG1+HcqwbiwD4q2/mwi+e+kXEhidCRU1tLaG8t08H6o8di8j5OS34QGlZOd74VGuNiqAS02JuhP/WW6vISoQZQ61dwJMar5if4+b999+LiU5O5eXlPP2rp1HDlDWhnNMR8M310eHr4D8e/4+IFK9Q0NraSmtbG6GcvUsDfKbJsWPHQnjW0DDoBd/r9XL0aNXJmLqltuPTKI3QfN3+UFdXx9YtW1mY14ZhQ2+XC4d10NraFvXNzn0+Hz/5yU/wihdzThizcnoiCXwz/Vk7b7zxhk1OhJf6en8To1AO94Ln0oIfgVRVVWH6fKhE6wXfTEynqqoqZpazr1u3DlMpLsjrsMX+1CFeMhJg7dq1ttgPFa+88gp79uzBNzP8WTk9ofIVapji6V89HXPhR/ha8FNCeM6g4AfPHUkMesEPrna1Mgc/iErIwPT5Yias88Xnn5OXrBjRy4VWocYQmDXEzVdfbsTj8djiw0A5fPgwv3n2N6jhCjU6AjJkBMw5Jl68/PQ/fhpzFV4bGhqA0Ap+8FzBc0cSg17wg5Xt7BD8YKG2WCix4PF42L59OzOG2Bs/n5nlobWtnaIo7Dfg9Xr5yU9/gs/w2RvKOZ1E8M3ysbtwN6+//rrd3oSU48ePA5AcwnMGzxWzIR0RuVJEikXkgIj8fRf77xGRWhHZHvj7VijshoKSkhIkIdWSommnYyZkgEhEN0zoLcXFxXR4PEzJtDc8NSnDbz8aS/2++uqrFBcV+8seJ9jtzamo0Qo1XPHrZ34dM3ek8PUoPJRJUPEIcSI0NjaG8KyhYcCCLyIO4ClgCTAVWC4iU7s49GWl1KzA37MDtRsq9u3fjydxSM8HhgOHExLTY6JM8v79+wEYl2av4KfFK3KSvvYnWigvL+fZZ/21ctSoCAjlnI6AOdsf2omlrJ0TJ06QYBgDqoPfFUmxKvjAPOCAUqpEKdUBvARcH4Lzhp3W1lYqKyosaXrSHd7EoezdG33hh9M5dOgQyXHCEJf9YjUquYOSg9HzI2qaJk/87Al/Vk44a+UMlET/gqxdO3fFTK2dxsbGkIZzgiRBRJZIDoXgjwA6T99X0HU9v2+KyE4ReU1ERoXA7oApLi5GKYWZEopK2P3Dl5xNfX1d1PcUPXr0KNmJPivXrnVLdoJJdXVN1JQFeOedd/wLrKbbn5XTEypfQQ788ulfRv1nFgIj/DB8ThJMM2YFvze8BeQrpWYAa4DnuztQRB4Ukc0isjnchcV27doFgM9GwTdTcwAoLCy0zYdQUFdbQ0Z8ZKSXZiaYtLvdtLaGpoduODl+/DhP/fIpyA4ULot0BHyzfbS72/m///s/u70ZMCdOnCAxDIKfCDTFaEinEug8Yh8Z2HYSpVS9UiqYvvEsMKe7kymlnlFKzVVKzc3ODq8Q79ixA5IywWnfDJmZNBRxOKNykrEzra0tJDkjQ7ASHX4/okHwn376aVpaW/DNtrBWzkBJBd9kH2vXro36CqVNjY1hmR9PBBpjdIS/CZgoImNFJB64FTiloImIDOv09DpgbwjsDoiOjg527NyJJ3VYzweHE8OBNyWXTZu32OvHAPF0eIiLkCTfoB8dHfYsAOsthYWFvPfee5gTTUK6tt8C1CSFpApPPvlk1K55AGhqagpLmaJEoLm5OQxnHhgD/ooqpbzAo8D7+IX8FaXUbhH5VxG5LnDYYyKyW0R2AI8B9wzU7kApLCykw+3Gl9bL9kFhxJc2nLLSw1FdG18MA3MAA/w/FieebIDy75tT+tT56nSC+SOGESG/QF1gmiZP/s+TSJKgpkbGnVGfcIB3ppeKioqozc1XStHc2hq2Eb7H66W93doigj0Rkm+EUuodpdQ5SqnxSqmfBLb9QCm1KvD4H5RSBUqpmUqpxUop29NSNmzYgBgOfOk2j/ABb4Y/IhbNzbhdLhduX/9jEqVNDtp8Bm0+g6KGuD53vupMR8APl8vV73OEmw8++ID9+/bjm+YDp93e9JNhoPIUv3/u9xG5qrQnWlpaME0zbCN88N9BRBKROwQKI0opPlu/Hm9qni0Lrs7wJzETElL57LPP7Hal36SmpdHqjYwgdIvH70dKSigXzIcOt9vNr5/5NQwhMsonDABzhklbWxvPP99tHkbEEsyiCUdiVPCckZapMygF/+DBgxyprMQ7ZKzdrvgRwZMxhk2bN0dk3K83DB2aRYMnMoaqDR1CanIS8fH2/5h3xYoVK6ivq/enYUbGb2T/SQcz3+TNN9+kKkKbfnRHcGFUOEb4wXNG2uKrQSn4H3/8MYjgzRxjtysn8Q4Zh8/rjdpRfnZ2NvXtBpGQ+l7fbpCdk2O3G13S2trKH/74B8gFItPFPqOmKkxM/vCHP9jtSp+wQvAjLdQ16ATfNE3ee/8DfOkjIS5yVrmYKdmQmMb7779vtyv9Yvjw4bR5FE0e+4esNe1xjBgZEWv7zmDFihU0NzXjK4ihqpNJ4Bvr4913342q7ljBwmnhCPxFasXMQSf4W7dupa62Bk/WBLtdORUROoZMYNu2bVF3awwwapRfYKta7f1I+UyobhVGjhxpqx9d4Xa7eenll/yje/uqeYQFNVmhRPHSSy/Z7UqvCYpxOEorJOKP1gV/VCKFQSf4b731FhKXgC+CwjlBvNnnoJTinXfesduVPjNmjP96Hmnpf3ZNKKhpM/CakJ+fb6sfXfHee+/R2NCIb3IMje6DJIJvtI/Vq1dH3Ki2O+rq6nCJ4ArDRIoDIcUwIq4JyqAS/Lq6Oj5dt46OoRPAiIwJxs4oVwq+jFGsXLUq6haz5OXlkeByUd5sr+BXBH5wIk3wTdPk5VdehkzAgkoesl2gAWgA4xPD/zzMqHMUHo+HlStXht1WKKirqyM1jMWfUpSKuLU1g0rw33rrLUyfD0/ulAGfK750A0ZrPUZrPQl73ia+dEMIPARP7lQajh9n3bp1ITmfVRiGwdix+VQ02/tDWtHsQEQYOzZCMrACbN26lYryCswJ1lTDlAZBPIG/WkEaLDCa5s/Lf+PNN6KibWf10aOkhbHMc7pS1Bw9Grbz94dBI/gdHR28seJNfBmjUCFoWG601CM+D+Lz4Gg6itESmls3X/pISEzn5ZdfCcn5rGTsuPFUtsXZ6kNFi4NhebkkJERWB5GVK1ciLonMWvchxBxvcrz+eFQ0kj9aVUVGGM+fARytro6oqq2DRvA//PBDGhuO05E3zW5Xzo4I7pwCior2nqzmGS2MHTuWxnZFU4d9mTqVrXHkjx1nm/2uaGhoYP369fhG+8DeiFf4yQNJlIifh2pvb+dYQwOZYbQxBGhrb4+oXPxBIfimafKnF19EJQ/FTBtutzs94s2eiMQl8OKLL9rtSp8YPXo0AEdtytQxFVS3yMkJ5Ejh448/xufz+WvJxzqGf/J248aNET15W1npL+gbzl53wXOXl5ef9TgrGRSC/8UXX1BeVoY7bzoR0aGjJxxxuHOm8Pnnn0dVg/MRI/yF6I622jOMPeYWPObXfkQKaz5cg6QLYY0fRBBqjMI0TdauXWu3K90S/F6Fc/48eO7S0tIwWukbMS/4Sin++McXICEV35DIutU/G57cqYjDGVWj/JzA6tZjbns+Vsfa/XZzc3Ntsd8VdXV17C7cjW9kDKZidkc6SJrwySef2O1Jtxw6dAgBssJoIwOIE+HQoUNhtNI3Yl7wd+zYwd69e3DnTocILpd7BnGJdGSdwwdr1lBdXW23N70iISGBlOQkGtz23EU1BH5osrLC+TXuG59//jkAasQgCOd0wjfcx44dOyKueFiQgwcPkm0YxIUxZcpAyAEO7N8fNht9JYoUsH/84Q9/ROKT8OacY7crfcYzbAamGV2rF1OSk2mzqWpmsFpnJFXJ3LBhA5IiUdfgZKCoYf6wTqR2xCreu5e8MKZkBhmmFPv37cO0wFZviGnBLy4uZvPmTbhzCyJyoVVPKFcKnqwJrHrrrYhbot0dLpeLDtMewfcE7EZKlUyv18uWrVvw5cRAVcy+MhQkXti8ebPdnpxBXV0ddceOYUXxjRFAc2vryUliu4lpwX/hhReQOBeenIEvtLILz7AZeDo6eO211+x2pVcoBlfo4mzs27cPd7sblTMIr4mAmWWyeUvkCX5hYSGAJYIfLOEXtGk3MSv45eXlrFu3Dnf2ZHBGxoivP6jEDLyZ+bz2+htR0ZS7rbWNBIc9AucK2G1ra7PF/umcXEdhQSmFSERlKaqPVkdcPZkdO3YQL4IVCdrZQJJhsH37dgus9UzMCv7LL78MYuDJK7DblQHjGT6DttYW3n77bbtdOStKKRpPNJEab4/gp8b746SREv7au3cvRrJBWJqmRgFqqP9zsHfvXps9OZXNmzYxWikcFsTZDIR802TLpk0RseI2JgW/oaGBd999z18kLS4c7Q2sxUzJwUwbxksvvxLRNUpqamro8HjISbQnBTEn0S/4kRIvLd5XjC99EKVjnk6G/58DBw7Y6kZnamtrKS0rY7yFNscDNXV1VFRUWGi1a2JS8N966y08ng48kV5GoQ905BZQV1vD+vXr7XalW0pKSgAYkWxPRkJuoonD8Kfc2Y3H4+FI5RFUuv2jOttwgpFqRNTiwY0bNwIw0UKbQVsbNoSmwOJACIngi8iVIlIsIgdE5O+72O8SkZcD+78UkfxQ2O0Kn8/HihVv4ksfgUoKZ6UMa/FljoaEVN54Y4XdrnTLzp07cRgwNs2euxCnAePSfOzYsd0W+52pqqry38Kn2u2JvfiSfZSVl9ntxknWr19PpmFY2l0yEyFHJCLalw5Y8EXEATwFLAGmAstFZOpph90PHFdKTQCeBJ4YqN3u+PLLL6mrq8WTMzlcJuxBDDqyJrF9+7aIqs3RmY0bNjA+zYfLxgJhUzM6KCoqtr2OS01NDQAqaRCP8PG//+qayFg42NLSwqZNm5hsmojFebJTlGLXzp22zy+FYoQ/DziglCpRSnUALwHXn3bM9cDzgcevAZeKhKeozTvvvIvEJ+HLiKwCWqHAm30OiPDee+/Z7coZlJeXc7CkhHk5blv9OC/Hg2matoe+jh075n8wSCdsT5IATY1N+Hz2z2V8/vnneL1e7Aj0FgCmUrb3uQiF4I8AOg85KwLbujxGKeUFGglDV8+mpiY+/+JzOoaMja4yCr1ExSfhSxvBu++9HzEr94K8/fbbGALzcjps9WNMqo/hKYq3Vtnbdampqcn/wN72APYTeP8tLS32+gGs+eADMgzDkvz708kDssVgzQcf2GD9ayJOFUXkQRHZLCKb+9oebMOGDfi8XrxDrZyDtxbv0PHU1dZQXFxstysnaWtr4+23VjEnu4MhCfaGMETgsuFt7C0qZs+ePbb5cbJFZazXv++JwPvv6LB3IFBfX8+mzZuZbpoYNix7FoQZymTnrl0cOXLEcvtBQiH4lXy9oAz8C9hOz4s7eYyIOIF0oMvVGEqpZ5RSc5VSc7Oz+7Zi5bPPPkNcyZjJsbvSxZs5CkRsD1l05s0336SpuYUlo9vtdgWARcPdpMQLzz/3nG0+nMy5HmwlFU4n8P7tzkH/4IMPME2Tc230YRb+y2FnSDYUgr8JmCgiY0UkHrgVWHXaMauAuwOPbwQ+ViH+BPh8Pr7atJmOtBHRUfO+vzgTMFNy+fKrr+z2BPCHLl780wtMG+LlnAz747QAiU5YMqqVDRs32tY17GQ9n8i4JPYReP8ul8s2F5RSvL1qFaNFyLbxFzgDYTyw+q23bFtPM2DBD8TkHwXeB/YCryildovIv4rIdYHDfgsMFZEDwPeAM1I3B8qBAwdoa23BFwUdrQaKN20Y+/fto7m52W5XeP755zlxoolbJ0RW2Ye/GNVOZgL8/H//x5b5jpMVO+2NZNhP4P0nJdm3AHLr1q2UV1ZyXgSsdJ0L1NbXn1wPYDUhieErpd5RSp2jlBqvlPpJYNsPlFKrAo/blVI3KaUmKKXmKaVKQmG3M0VFRQCYqZHT/CJcmCm5KKXYb3Od7QMHDvD6669x0XA3+WmRNZRNcMIt41so3reft956y3L7Q4YEGtzZFeXyQGJiIjfeeCOJiYngscmPdkhNS8XptK9a7ZsrVpBkGERCkZXJQJph8OYKe9bTRNykbX/Zv38/EpeAio+cWujhwpfsT3CyU/C9Xi8/e+Jxkp0mt06IjGJlp7Mwr4OpQ7w8/cunTubFW0WwzaI02xRC8MDVV1/NY489xtVXX22b4EuTMHKkHXkxfqqrq/ls/XrmmGZYm530FgfCeabJV5s2UVZm/YK0mBH8iopKfK406+L3vo5TR1A+C+/d4xKROJettTlefPFFior3cdfEZtuKpfWECNw/uQVfh5snHn/c0onD3Nxc4uLj/AnIdhAHq1ev5uc//zmrV6+2Jz1UgaPJwdj8sTYY9/PGG2+gTJN5tnlwJnMBh4gtJc9jRvCPVFVhWji6F2/HKSMo8VobrDXjU2xrfVhcXMzvf/87zs/t4Pw8u2IFvSM3yWT5xGY2bd7MG2+8YZldh8PBpEmTMI7Z9BWL86fLvvbaa/5y0XYIfiuY7SaTJ9uz6r21tZVVK1dSgH/CNFJIQZipFO+uXk1jo7UjgpgR/ObmJpTTukwA5Yw/ZQSlLK65bzpcnDjRZKlN8H+JfvTDH5Aeb3LP5MiaqO2OS0d0MDPLwy9/+ZSllRtnzZwFx7Evfm4zUuMX2enTp9tif/Xq1bS0tnKBLdbPzgWA2+Nh5UprFwjGjOC3t7WhHBYOYxzxp46gHNYKvnLEWb56USnFf/3Xf1FVVcUjU5tIiYvMUM7piMCDU1tINrz86Ic/sKyRzHnnnQcmEBmlZCxHqoTMIZmMGzfOctter5dXXnqJMSKMiqDRfZBchInA66++itttXTmSmBF8kZh5K71GLC4f8c4777BmzRqWjW1jcmbk1uXvivR4xcMFTZSXV/Df//3flsTzp0+fTkpqClIReYITdrxgVBt846JvEKayWWdl7dq1VNfWsjACUjG740LgeGMj77//vmU2Y0YlnXFOxIwuERoQpo/4OOtS3Q4ePMj/PPnfFAzxcv3YyFhR21cKhnhZNq6NDz74gHfeeSfs9pxOJ4svXoyjyjHowjpSKSiv4pJLLrHctlKKF//0J3IMg0mWW+89Y4HhIvz5T3+yrLhczAh+eno64rW3UqOVOHztZGZaU++/tbWVH/zLP5MoHh4paMaI4gHr0rHtFAzx8uR//7cljVKuvvpqlFchZVF80fqBccggb1geM2fOtNz2pk2bOFhSwkKb6ub0FkFYpBSVVVWW1cqPGcHPzs7G6LB/5alVGB0tDB0a8oKjXfLkk09SWVnJIwVNpLsi9xa5NxgCjxQ0k+Tw8KMf/iDsDc+nTJnC+AnjcRx0QHRfut7TANTCsqXLbAnn/OmFF0gzDGZYbrnvTAWGGAYvvvCCJWHGmBH8MaNH43CfsNsNa/C2ozraGD16dNhNvf/++7z//vssHdvG1CGxETJLdykemtpEWVk5P//5z8NqS0S45eZbUI0KjobVVMQgxYIrwcU111xjue2ioiK2bd/OAtPEGcLR/Tsoftvp750Q/XobCAtNk6J9+9ixY0dIznl2ezHC2LFjUR1tiNv+utvhxmjxN9cYPz68ZaCPHDnCk//9X0zK9LE0SuP23TFtiJdr89tYvXo1n376aVhtXXbZZWRlZ+HYOwhG+c1glBssvX4pqanW93d86aWXSBBhbojPWwUc7vRXFcJznwskGwZ/fvHFEJ61a2JG8KdN8/exMZpjPwfO0VyNiDBlypSw2TBNk5/+5Ccor5uHp0Z33L47bhjXzrh0k5898fjXHarCgNPp5M477vQXBI/xj6fsEeKccdxyyy2W266qquKTtWuZqxQJERy7P504hPmmyYaNG8Pe8D1mBH/ixIkkJCbiaDy9FH94MJOHohxxKEccvtQ8zGRr4ukAzhOVjB03PqwjqBUrVrBz1y7unNhMVmJkddcKFU4DHpraRHtbC08++WRYbV199dVk52Tj3O20ZJSvMhQqLvCXrVAZFhhtBKPM4IYbbiArKyv89k7j9ddfB6U433LLA2ce4BTh1VdfDaudmBF8p9PJ/HnziD9RARZMfnSMWYCZNBQzaSjtU6+hY8yCsNsEwNuO0VTNwgvCZ6+uro5nfv0rZgz1smhYbNf3HZ5ssiy/jU8//ZQvvvgibHbi4+N54FsPoI4ppDz8o081S0EGkAHmxab/eZgxdhkkJiZy++23h93W6bS2trL67bcpANKjaHQfJDlQbuGD997jxInwzUXGjOADLFq0COVuwWiK3ftm57HDoBSLFi0Km41f/epXeDrc3D2pxbJadG1eOaUYXZvXui/tVWPaGZ6i+L+f/29YW/FdfvnljBs/DsduR+w1Rqnxr6y98447ycjIsNz8mjVraGltjcrRfZD5+MstrF69Omw2YkrwL7zwQuJdLpz11tVLsZq4+oMMHzGSSZPCs6TkwIEDfPDBBywZ3UZuknWhnFavnFKMrtVCwXcacMfEZiqPVIW1dr7D4eDR7zyKalbI/ugbhXaLAudOJ1k5Wdx0003Wm1eKN1esYJjIKb1Wo41hCGNEWPXmm2FL0YwpwU9KSmLxxRcTf+ygteWKLULaGjBOVHHN1VeFLb/5ueeeIzFOuGaMtYvYkpzqlGJ0SU5r01mmD/EyKdPHC3/8Q1hrm8ydO5fzF5yPo8hhX3OUECOHBXVc8fC3H7allWFRUREHS0qYqxQSheGczswNLMTatm1bWM4fU4IPsHTpUpTXg7M29kb5cdV7cDgc/oYWYaAqsOLv8hGtJFtcGC3RqU4pRpdoseCLwNL8VuqPHQ97muaj33kUMQUpjG5xAsADjt0Opk6dymWXXWaLC++++y5xIlGx0KonCoAEEd59992wnD/mBH/q1KlMmjwZV00hqBjKLvG2E1+3n8suuyxsJRVWr16NoLhk5OApUdGZgiFecpMVq8Jcsnb06NHc+M0bMQ4Z/lWpUYwUCapN8dhjj9myqtbr9fLxhx8yKcpSMbsjDqFAKT795BPa20N/Cxhzgi8i3H7bbdB2AsexQ3a7EzLiju5B+TzceuutYbPx6SdrmZzpJSsh1lcHdY0hcGFuO7sKC8Oalw9w9913k5KagmNHFC/GagHHPgdXXHEFU6dOtcWFrVu3cqK5OSZG90FmAO1ud1gancec4IM/W2fU6NEkHNluSYpm2PF24KrZzQULF4Ztde3Ro0cpLStndlbszX30hXOzPCil+Oqrr8JqJzU1lfvvux9qCO2yTQuRXYLT6eTBBx+0zYdPP/0UlwgTbPMg9IwBkgwjLKHFAQm+iAwRkTUisj/wb5exBhHxicj2wN+qgdjsDQ6Hg/vuvRdaj+OoLwm3ubATd3QXyuPm3nvuCZuN4uJiACamx0a9nP4yOtWHyyknr0c4uf766xkxcgTOXU5/o5Root5fQuG25beRk5NjiwtKKTZ8/jnjlYqIBuWhwoEwyTT5cuPGkJdNHugI/++Bj5RSE4GPAs+7ok0pNSvwd90AbfaKxYsXM3bcOBKObAEzipOePW24qgu56KJvhC0VE6C0tBSAkSlRfK1CgCEwIskb9iXu4F8s+MjDj6BOKORwFAmWAscuB+kZ6Sxfvtw2N0pLS6k7doyJtnkQPiYAzS0tIR94DFTwrweeDzx+Hlg6wPOFDMMwePihh6DtBM6avXa702/iK7Yipo8HH3wgrHYaGxtJcAouR1jNRAXp8T5ONDZYYuvCCy9kasFUHHuiaDHWUaAW7r3nXpKSkmxzI1hd0voGiuEn+J5CXUFzoIKfq5QKRiCPArndHJcgIptFZKOILB2gzV4zf/58Zs+Z44/lR2FzFGk9TlxtEddff33YSyG73W4t9gHiDMKSIdEVIsJD334I1aaQg1Ewylfg3OMkNy+Xa6+91lZXCgsLSTUMrGkDZC0pCEMNg8LCwpCet0fBF5EPRaSwi7/rOx+n/EvDupshHaOUmgvcBvyPiHQ78ygiDwZ+HDbX1tb25b10dS6+++ij4HUTX7F1QOeyHKVwlW0kMTGJe++9N+zmkpKSaBvc4fuTtPuE5ORky+zNmjWLOXPm4Ch2QKT/H1SBOqa49557iYuLs9WV/cXFDDPNqF9s1R3DTJP9Vod0lFKXKaWmdfG3EqgWkWEAgX9rujlHZeDfEuAT/CWgu7P3jFJqrlJqbnZ2dj/e0qmMHz+ea6+9lriaPUjr8QGfzyocDWU4Giv51v33WVKbZMiQIXT4FE0dsfnl6QvHOpwMGWpttcf77rsP1a6Qkgi+/gqce53kDcvjiiuusNUVr9dLWXk59kwXW0MucLSmhtbW1pCdc6AhnVXA3YHHdwNnrFgRkUwRcQUeZwELgT0DtNsnvvWtb5GcnIyr9IvoSNM0vSSUfcmo0aNZtmyZJSbHjfNHDcuaB3dcp8MHVc1y8npYxfTp05l17iwc+yM4ll/tH93fdeddOJ1OW12pr6/H6/NhXVFy6xkS+LempstxdL8YqOA/DlwuIvuBywLPEZG5IvJs4JgpwGYR2QGsBR5XSlkq+BkZGTz4wAM4TlThOBb5aZpxR3ZC+wm+99d/bdkXa8qUKRiGwZ5j9n6R7aa4wYlPfd1Qx0ruuP0OVGvkNjx3FDvIHJpp++gevhbBdIvstcMp1VytmOEJvreIEXylVL1S6lKl1MRA6OdYYPtmpdS3Ao+/UEpNV0rNDPz721A43leuu+46xk+YQGL5VxFdWE3am3BV7WTx4sXMmTPHMrupqalMKyhgS50rKm6CwsXW2jji4pzMnj3bctvnnXeev3zy/ghcfdsA1MAtN91CfHy83d7Q1NQEQKJF9trhlGquVgh+8L0F32soiMmVtl3hcDj4m+9/H+VuIb4iPJXoQoGrdAPxcU6+853vWG778iuuoKLZoOSE9WGdMak+Eh0miQ6TyRkexqRaH9fo8MEXNYksWnQRCQkJltsXEZbfutzf8Dx0g7qQIPv8jcntzswJEoxrW/XTkwCnVHO14tMRrDsaSTH8qKKgoICrr76auOrdETmB6zheiqOhjPvuu9eW1YuXXXYZCS4X75dbX+L2zkltjEn1MSbVxz/PbebOSW2W+/D50XhaOpStorZ48WLSM9Ix9kfQV7MdHOUOrlpylS2Nyc+GVcGvBDilmqsVgh+O9xZBnypr+Pa3v01ycjIJkTaB6/OSULaRMWPybWkiAZCcnMzSZcvYWO2iqmVwfTS8JqwqTWbypHNsCecEiY+PZ9nSZUiVQIttbpyCHBKUqbjhhhvsduUkhuH/fEZbRYq+EHxvwfcaCgbXtxr/BO7DD30b40QVjvqDdrtzkriqHdDexPe//z1bMyCWL19OfLyLlw9YFR2NDD6scFHbCvfed78tZX47c+2112IYRmQsxFLgOORg9uzZjBkzxm5vTpKSkgJY10NmGP4QiwvIDzwPN8F73OB7DQWDTvABrrnmGiZNnuyfwPXaP4Er7Y3EV+3k8ssvZ9asWbb6kpmZyV13383m2nh21Q+OjJ3GDuGNQ8nMO+88zj/f/q6o2dnZLFy4EMfhCEjRrALVoli6dKnNjpxKero/h8Wqm6CrEIbhF/r7Ea6yIJgUjNwH32soGJSCbxgG3//e91CeNuIr7V+B6yrdSIIrnocffthuVwC46aabGDViOL8rTqU90ld+hoDni5LwKIPv2tTEoyuWLl2Kciuk0l5/jBKDzCGZXHjhhbb6cTp5eXlA1PePOSvBWcbgew0Fg1LwASZPnsw1ETCB6zhehqOhnPvvu4+sLGtXd3aHy+Xi7/7hH6lrgz/HeGhnY3UcX9XEc+9990dUyGLOnDnkDcvDOGjjV7QFpEq49pprbV9odTppaWmkJCVRZ7cjYaQOf0XVUFQcCDJoBR/gwQcfJCkpCVfZRnsmcE0fCeX+FbXf/OY3rbd/FmbMmMHNN9/CRxUJbKm1t2ZKuKhrM/hdUSpTpkwOayex/mAYBsuWLvN/6xvt8UFKBBGJmFTMzogIE845h6MRckcWDqqA8ePG4XCELk16UAt+RkYG37r/fhyNlTiOl1luP+7obmhr5C8feyziRlAADzzwABMnjOc3e1Opa4utj4rXhF8UpoDTxQ9/+KOIvP5LlizB4XTYU1/HBEepgwULFpCb210RXHuZPHkyVYDHolVqwRi+FfhQHBGDyVOmhPS8sfUt7gdLly5l5KhRJFR8ZW2jFE8brqrtnL9gAfPmzbPObh+Ij4/nx//6b5iOBH6+KxVPDOXAvbg/kQONDv7u7/+B4cOH2+1Ol2RkZHDJ4ktwlFlfRVMq/c3JI22ytjOzZs3CqxTlFtm7yqLJWvCP7tuVGfIkjkEv+E6nk8e++11oa+xzoxQzeShmcv/KN8VXbEWUj0dtWFHbF0aOHMk//fM/U3LC4LmipIhautBfPjsSzwflCdx8880sXrzYbnfOytKlS1EdCim1dpRvHDTIG5YXsYMRgJkzZ+J0ONhntyNhYD/+sFWo14QMesGH/jdK6RizgI4xC/psT9oa/I1Nrrsu7I1NQsGiRYu46667+PSIizU2rMINJQcbHfyuKJlzz53FQw89ZLc7PTJt2jTGTxiP46CF9XUagFpYtnRZSBf9hJrk5GTOPfdcigwDFXHFhwbGXhEKpkwhMzO07V0i93/TQkSE7zzyCMrT7q9UGWbiyzeRkJBgSWOTUHHfffexcOEFvLA/KWrz8+vbhSd3pjE0O4cf//hfIzJufzoiws033WxpfR3ZL8S74rnmmmusMTgAvnHxxdSbJlU9Hxo11KKoUoqLLr445OfWgh9g4sSJXHbZZbhqdiMd4VvOYTTX4Dxeyu233WZJY5NQYRgG//IvPyA/P5//K0zjSJSVXmj3wZM70+gwEnj8iZ9F1bW/5JJLSEtPs6a+TqBuzpIrl0Rc3ZyuWLx4MU6nk8gth9h3dgCGCJdddlnIzx1d39owc//99yMo4o6EtnFwZ+IrtpCWlm5bvZyBkJSUxONP/AxXUir/uSM9arpjmQqeLkymrMnBD3/0Y8ubmwwUl8vFDctu8NfXCV2l3C6Rg4Lyqaj5fKamprJo0SJ2GIZl2TrhxIdiq2Fw3nnnhWVdjhb8TowYMYKrliwhrrYIcTeH/PzGiSocjZXceecdJCUlhfz8VpCXl8dPH3+CBk8cT+6Mjsydlw8ksqU2nu8+9hgLFvR9ziUSWLZsGc44J7IvjD+yXnCU+FMxo2FuKcjSpUtpM0122e1ICNgLNJkmy8JUqE4L/mncfffdOETCMsqPP7KN9IzMiE516w0FBQX84z/9E/saHDy7J7Izd9ZWxrO6NIFly5ZF3OK2vpCZmclVS67CUerodcUwlaFQGb3/z5HDgmpXLF++vJ9e2sOsWbMYl5/PFyJRPXmrUHwuwrC8PObPnx8WG1rwTyM3N5errrqKuLp9IY3lG03VOBqPcMftt+FyRXemC/jjyvfffz+fH3Wx8pD1zUJ6w+5jTp4rSmbevPP47ne/a7c7A+aWW24Bk16P8tUshZrVSwE0wbHfwZSpU5g5c+YAvLQeEeHW226jWin22+3MADgMVCjFrcuXh3R1bWe04HfB7bffjoHCebQwZOeMq9pBSmoa1113XcjOaTd33XUXV1xxBa+VJPJldWSVX6hqMfj5rjRGjR7Nj37046jIyOmJUaNGcfHFF+MocUCIi7xKuaCaFXfcfkfEFJDrC5deeik5Wdl8EsWj/E8QMtLTWbJkSdhsaMHvguHDh7N48WJctUV9ysvvDmk9jvN4GTfd+E0SE2OnGJmI8Ld/+7dMn1bAr/akcMiG1ohd0eIR/mtnGs7EFB5/4mchrSduN3fccQfKo5ADIRRl5W9QPiZ/DAsXLgzdeS0kLi6Ou+65m/IoHeUfRlGC4vY77ghre00t+N2wfPlylNeDs7Z4wOeKO1pIXHw8y5YtC4FnkUV8fDz//pOfMmRIFk/uTOO4297RoS9QI6eu3clPfvofEVs2ob9MnDiR888/H8eBEJZbOAKqUXHXnXdF9EKrnliyZAl5ubl8KIIZRaN8hWKNCEMyM7n++uvDamtA/7sicpOI7BYRU0TmnuW4K0WkWEQOiMjfD8SmVZxzzjnMnDULV/UeUANIRfG0EV9/gKuWLImq3O++kJmZyX888TNaVTz/tysVr42ZOy8fTGRXvZPvff/7zJgxwz5Hwsjdd9/tr5Ufio5YChx7HeQNy4v4MhM9ERcXx7ceeIAqpQhdMDb8FANlSnHvffeFdXQPAx/hFwI3AOu6O0BEHMBTwBJgKrBcRKYO0K4l3HzTTeBuHlAlzbjafSjTF9UZIr1hwoQJ/P0//CP7Ghz8cZ89YauNR+N4pzSBpUuXRsUq0f5SUFDA7NmzcewPQUeso8BxuPuuu2NinuPSSy9l/LhxrImSvHwfig/EYOSIEVx99dVhtzcgwVdK7VVK9RTzmAccUEqVKKU6gJeA8N63hIgFCxaQlZ1DXM2e/p1AmcTXFnHuueeSn58fUt8ikUsuuYTly5fzUUUC66viLbVd2Wzwm6JUphVMjYmMnJ64++67UW0KOTSAUb4CR5GDrJwsrrjiitA5ZyMOh4NHv/tdGkyTDXY70ws2AbXK5JHvfMeSH1wrAnYj4JQKphWBbV0iIg+KyGYR2VxbWxt2586G0+nk+uuuxdF4BGk/0efXOxorob0p6vPu+8IDDzzAzBkz+F1RChXN1sSD233wv4VpJCWn8uN//Tfi4iIrYygczJo1i6kFU3Hsc0B/Q2i1QB3ccdsdMXXN5syZw8ILLmCdCE0RPMpvRfGxYTD73HMtmyzv8RspIh+KSGEXf2EZpSulnlFKzVVKzQ1la6/+cvXVVyMi/Zq8ddYUk5aWHnH9QMOJ0+nkhz/6EckpqfyiMI2OPoQcxqT6GJPa9xjFH4uTqGoRfvCjH4e0HVwkIyLcfdfdqBaFlPVvlO8ocpCekW5JKMFqvvPoo/gMg/ftduQsfAS4lbK0l3KPgq+UukwpNa2Lv5W9tFEJjOr0fGRgW1SQlZXFvPnzcdUf7NvkracdZ2MZV175FzE1euoNWVlZ/NO//ICKZuHF/b2P5985qY07J7X1ydbG6jg+PeLijjvuZM6cOX11Nao5//zzGTd+nH+U39eB7HGgGm695daYWAh4OiNHjuTW5cvZAZRG4Cj/CIpNwNJlyxg/frxldq24594ETBSRsSISD9wKrLLAbshYcuWVKHczxoneF2F1HisB0+TKK68Mo2eRy7x587j55pv5sCKBHXXhiU0eaxd+X5zKlMmToqrUdKgQEW6/7XZ/6eQ+1geWYiExKTHsaYB2cuedd5I9dChvi+CLINE3UawWIS0tjfvvv99S2wNNy1wmIhXAAmC1iLwf2D5cRN4BUEp5gUeB9/HXBnpFKbV7YG5by8KFC3ElJOCsP9jr18TVH2RMfj4TJkwIo2eRzQMPPMDY/DE8W5RKiye0t6xKwW+LkvHi5J//5QcxkWHSHxYvXkxWdpY/Y6e3tIJRYXD9ddfH1KK000lMTOTRxx7jqPKPpiOF7fjTMB9+5BHLS1APNEtnhVJqpFLKpZTKVUr9RWD7EaXUVZ2Oe0cpdY5SarxS6icDddpqXC4XFy1aRHxDKZg9h3XE3YLRVM3lYahnHU24XC7+4R//icYOo0+hnd7w+dF4dtTF8eC3H2LUqFE9vyBGcTqd3HTjTf7mKA29e40cEAwxYj5VGODiiy9m7pw5fCQGzREwym9F8YFhUDB1qi13/9G7rM5iFi9ejPK4cZw40uOxjuOHTr5msDN58mRuueUWPj3iouh4aEbhzR7hT/uTKZg6dVCIVk9cc801xLvie1duwQeOww4uXHQhubm54XfOZkSEv/7e9/Aawnt2O4N/orZNKb7/N39jy6pmLfi95LzzzsPlSsBx/HCPxzqPlzJmTP6gHnl25p577iE3J5vnilPwhWAV7ssHEmnxGvzN//t/UV0KIFSkpqZyxeVX4Ch3gOfsx0q5oNyKG5aFp956JDJq1CiW33YbO/DXrLGLysBE7Q3f/KZtoV79beklLpeL+fPnEd9YzlkLwHvdOJqOctFFi6xzLsJJTEzksb/8KyqahY8qB5YRUtrk4JNKF9/85jctzW6IdK677jqUt+cUTeOQwYiRIzj33HMt8iwyuPPOO8nNzuYtMWyZwDVRvCVCZkYG9913n+X2g2jB7wMXXHAByt2CtB3r9hhHYyUoxfnnn2+hZ5HPhRdeyJw5s3n9UFK/J3CVgj/tSyItLZV77rkntA5GOZMmTSJ/bD7G4bN8pZuBOrjm6muisgTyQEhISOCxv/orapTJRhvsbwUqleKRRx+1daJcC34fmDdvHgDOhopuj3E0VpCcnMKUKVOscisqEBEeeeQ7tHTA6tL+jfILjznZc9zJPffeFxUNtq1ERFhy5RI4Rrd9b6VUEBEuv/xyS32LFC688ELmz5vHWotX4LaiWGMYzJwxw/ZrrwW/D2RlZZE/dmz3E7dKEd90hLlz5wzaNMGzMXHiRC655BLeL0/iRB8boCsFr5UkkZuTzbXXXhsmD6ObSy+9FPDH6bvCUeFg+vTp5OTkWOlWxCAiPPaXf4nPMFhjod2PgHal+Ku//mvb76y04PeRuXPm4GyuBvPMEgDiPoFqbx50Kz77wj333EOHqXi3rG+j/F3HnBxsdHDnXXcTH29tYbZoIScnh6lTp2Ic6eJrfQLUCTXoM8dGjRrFzbfcwjag3IJR/tHARO31S5dGxJyTFvw+MmvWLJTPi9FSd8Y+R1P1yWM0XZOfn89FF32DjyqTaOtDA4+3SxPJGjpk0K5c7i2LFi3yl004rUKFVPlHloOprlN33HXXXQzJzOTdMLdDVCjeRUhJSbF8RW13aMHvI9OnTwfA0XT0jH1G01FSUlMZM2aM1W5FFcuXL6fVo/j0SO9G+aVNDvYcc3LjTTfr0X0PLFiwAAA5emrowDhqkD82f1Dk3vdEUlISD37725Qrxa4w2ikGSlDcd//9pKWlhdFS79GC30cyMzMZNmw4RnPNGfviWmqZVjDN9jhdpDN16lSmTpnCR5WJZ81wDbKm3IUrPk7H7nvB2LFjyRySCdWdNvpA6oT58+bb5lekceWVVzJh/PiwNUoJNjYZNXJkRNUr0oLfD6ZNKyCu9bSQjrcDWo9TUBAVzbxsZ+myZVS1CHt6WH3b6oUN1QlcetnlOjOnF4gIc2bPwVnn/LqCZj0oUzF79mxbfYskDMPgke98hwbT5KswnH8r/sYmDz38cEQlcGjB7weTJk3y5+N3tJ7cZrTWn9yn6ZnFixeTlJTIZz10xtpUHY/bp2K6ZWGomT59OmabCYGPp9T77zinTZtmo1eRx9y5czlv7lzWGQbtIRzle1CsNQymFRRE3JyJFvx+MHHiROBrke/8OLhPc3ZcLheXXHIpm2oTcJ+l58nnR12MHDGcgoIC65yLcqZO9d9lyjE5+e+IkSP0HVIXPPjtb9NqmnwRwnN+CTSZJg9++9sRF97Vgt8PgulVRuvxk9uM1mOkpqUzdOhQu9yKOi699FLcXsWOuq4bxDS4haIGJ5ddfkXEfXEimbFjx+JwOPzZOoCj0cGUyXohYFdMmjSJRYsWsUEM2kIwyu9Asd4wmDtnTkRm62nB7wdpaWlkDhmKtH0t+I62BsaPH2ejV9HHzJkzyUhPY1NN12GdrbVxmAq+8Y1vWOxZdBMfH8+o0aOQEwIeMFvMiMgBj1Tuvfde2lVomp5vAlpMk3ttrJdzNrTg95Ox+WNwtDf4nyiFo72RfJ2O2SecTifnL7iAncddXVbR3FYXR15uDuPG6R/SvjJu7DgcTQ444X+en59vqz+RzIQJE7hw4UI2GgbuAYzyvSg+Nwxmz559Mn070tCC309Gjx6No73Rv+bf24byuhk9erTdbkUdCxYsoKVDcfDEqR2bPCbsOe5iwQULdTinH4waNQqzxfSP8gPPNd1z+x130GaabBnAObbjj93feeedIfIq9GjB7ycjR45EeTvA68Zo9w+jRowYYbNX0cfs2bMREXYfOzWOf6DRidunmDt3rk2eRTfDhw8HBXLEL/h5eXk2exTZFBQUMGP6dDYY/SufrFB8IQYTJ0yI6PRXLfj9ZPjw4QAY7hMY7U2nbNP0nvT0dMaPG0tRw6mCX3TciYhE5MRXNDBy5EjAL/jZOdl6hXIvuPmWW2gwTYr68doD+PPub77lloi+I42cFQFRRnDEJO5mpKP5lG2avjFj5ixWrzyEzwRHYAiyr9FJ/pjROpWwn0ybNo0nn3yStrY2HWrsJQsXLiQ3O5uvauvoaxLwV0BGenrEF6fTI/x+EqxJIh3NiLuZtPR0XK6BdXMarEydOhW3T3Gk1f9xVApKmuIpmBaZE1/RgIgwZ84cLrzwQi34vcThcHDd0qWUoKjtQ1inEUUxcM2110b8ndSABF9EbhKR3SJiiki3wVYROSwiu0Rku4hsHojNSCElJYV4lwujoxXxtJKdnW23S1HL5MmTASg54b/hrG0zaOlQJ7drNFZx1VVXYRgG2/rwmm34q1hEw2rwgY7wC4EbgHW9OHaxUmqWUiomZuFEhCFDhiKeVhyeVnK04PebESNGkOCKp6zJn6lT2uz/165Gz5rBy9ChQ5k/fz7bDQOzF6N8hWK7YTBr5syomMMbkOArpfYqpYpD5Uy0kZU1FOexw0hLvV5hOwAcDgdjx46lPCD0Fc0ORETn32ts4corr6TJNDnci2MrgXrT5MolS8LsVWiwatJWAR+IiAJ+rZR6prsDReRB4EEg4mOP9917L+vXr0dEuPrqq+12J6oZkz+WLw/vA+BIi4Pc7CwSEhJs9kozGFmwYAGJLhc73W56GnLswr+AcNGiRVa4NmB6FHwR+RDoKv3kn5RSK3tp50KlVKWI5ABrRKRIKdVlGCjwY/AMwNy5c63rNNwP5s6dq/PEQ8SoUaN4r13R7oPqNgejJuXb7ZJmkJKQkMCChQv58pNPME0Tg67TLBWKPYbBeeedFzXZZD0KvlLqsoEaUUpVBv6tEZEVwDx6F/fXDBKC8c/aNoPqdicz9CI2jY1cdNFFfPzxx5QB+d0cUwU0mCYXXXSRdY4NkLCnZYpIsoikBh8DV+Cf7NVoTjJs2DAAypsdtHQovaZBYyvz58/HYRjsO8sxwX0XXHCBFS6FhIGmZS4TkQpgAbBaRN4PbB8uIu8EDssF1ovIDvzrE1Yrpd4biF1N7JGTkwPA/gbnKc81GjtITk5mxowZ7JfuJXK/CJPOOYfMzEwLPRsYA5q0VUqtAFZ0sf0IcFXgcQkwcyB2NLFPZmYmhsjJXPysrCybPdIMds6bN49ntm+nGUg5LY7fjqJCwe3zo6tPsF5pq4kIHA4HmRnplAVSM3Waq8ZugkXQDnWxrxQwOx0TLWjB10QMGZkZeEz/SCojI8NeZzSDnnPOOYcEl4vSLvaVAk6HI+pab2rB10QM5807n/S0FKYVTCUlJcVudzSDHKfTSUFBAWVdVL8sw9+/OtrWiuhqmZqI4ZFHHuGRRx6x2w2N5iRTCwrYtnUrJZw6Oj4iwvXTptnlVr/Rgq/RaDTdMH36dEzg96fvUCpi2xieDS34Go1G0w3z58/nqaeeoqOj45TtcXFxURe/By34Go1G0y0iEpUj+e7Qk7YajUYzSNCCr9FoNIMELfgajUYzSNCCr9FoNIMELfgajUYzSNCCr9FoNIMELfgajUYzSBClIreLoIjUQpe1iyKJLKDObidiCH09Q4u+nqElGq7nGKVUdlc7IlrwowER2ayU0o1tQ4S+nqFFX8/QEu3XU4d0NBqNZpCgBV+j0WgGCVrwB84zdjsQY+jrGVr09QwtUX09dQxfo9FoBgl6hK/RaDSDBC34Go1GM0jQgt9LROR3IlIjIoXd7BcR+bmIHBCRnSISXe3sLUREEkTkKxHZISK7ReTHXRzjEpGXA9fzSxHJt8HVqEFEDovILhHZLiKbu9ivP5+9QEQmBa5h8O+EiPzVacdE7bXUgt97ngOuPMv+JcDEwN+DwNMW+BStuIFLlFIzgVnAlSJy/mnH3A8cV0pNAJ4EnrDWxahksVJqVjd54vrz2QuUUsWBazgLmAO0AitOOyxqr6UW/F6ilFoHHDvLIdcDf1B+NgIZIjLMGu+ii8A1ag48jQv8nZ49cD3wfODxa8ClIiIWuRiL6M9n37kUOKiUOn21f9ReSy34oWMEUN7peUVgm6YLRMQhItuBGmCNUurL0w45eT2VUl6gERhqqZPRhQI+EJEtIvJgF/v157Pv3Ar8uYvtUXstteBrbEEp5QvcNo8E5onINJtdinYuVErNxh9u+I6IXGS3Q9GMiMQD1wGv2u1LKNGCHzoqgVGdno8MbNOcBaVUA7CWM+dHTl5PEXEC6UC9pc5FEUqpysC/NfhjzvNOO0R/PvvGEmCrUqq6i31Rey214IeOVcBdgRn884FGpVSV3U5FIiKSLSIZgceJwOVA0WmHrQLuDjy+EfhY6VWCXSIiySKSGnwMXAGcnk2mP599Yzldh3Mgiq+l024HogUR+TNwMZAlIhXAD/FPNqKU+hXwDnAVcAD/zP699ngaFQwDnhcRB/5BxytKqbdF5F+BzUqpVcBvgT+KyAH8k+W32uduxJMLrAjMaTuBF5VS74nIQ6A/n30l8KN5OfDtTtti4lrq0goajUYzSNAhHY1GoxkkaMHXaDSaQYIWfI1GoxkkaMHXaDSaQYIWfI1GoxkkaMHXaDSaQYIWfI2ml4jIJyJyRiVKEZkrIj/v5jWHRSSri+0/EpG/CYefGk136IVXGs0AUUptBs6oQa/RRBp6hK+JakQkX0T2ishvAs1UPhCRxM6jcRHJEpHDgcf3iMibIrImMPp+VES+JyLbRGSjiAzpweRNgeYt+0RkUeCcF4vI24HHQwM+7BaRZ4GTJZ1F5J8Cr1sPTOq0fbyIvBeodPmZiEwObH8u0GjjCxEpEZEbQ3ntNIMPLfiaWGAi8JRSqgBoAL7Zw/HTgBuA84CfAK1KqXOBDcBdPbzWqZSaB/wV/vIap/NDYH3AlxXAaAARmYO/PMQs/Mvyz+v0mmeA7yql5gB/A/yy075hwIXANcDjPfim0ZwVHdLRxAKHlFLbA4+3APk9HL9WKdUENIlII/BWYPsuYEYPr32jBzsX4f8xQSm1WkSOB7YvAlYopVoBRGRV4N8U4ALg1U79XVydzvemUsoE9ohIbg++aTRnRQu+JhZwd3rsAxIBL1/fwSac5Xiz03OTnr8TwWN9vTi2NxhAQ6A3wNnsQafwkEbTH3RIRxOrHMbfkxT85ZWtYh1wG4CILAEyO21fGphfSAWuBVBKnQAOichNgdeIiMy00F/NIEILviZW+U/gYRHZBpyRFhlGfgxcJCK78Yd2ygCUUluBl4EdwLvApk6vuR24X0R2ALvx90zVaEKOLo+s0Wg0gwQ9wtdoNJpBgp601WhOQ0SeAhaetvl/lVK/t8MfjSZU6JCORqPRDBJ0SEej0WgGCVrwNRqNZpCgBV+j0WgGCVrwNRqNZpDw/wHR1tIgaWG7/gAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "weights_vp(tanh_model3_init, att=\"grad\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x2d53b60efd0>"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[ 0.7995, -0.2561],\n",
       "        [-0.1097, -0.0529],\n",
       "        [-0.8137,  0.8860],\n",
       "        [-0.0984,  0.9770]], requires_grad=True)"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[ 0.7130,  0.3764, -0.0196, -0.0096],\n",
       "        [ 0.0199, -0.8417, -0.2067, -0.4007],\n",
       "        [-0.0515,  0.1059, -0.6787,  0.6582],\n",
       "        [ 0.7072,  0.5386,  0.3889,  0.1370]], requires_grad=True)"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[-0.6241,  0.3488,  0.3069,  0.8371],\n",
       "        [-0.8167,  0.7516,  0.6801,  0.1606],\n",
       "        [ 0.2885,  0.1036, -0.0692,  0.1744],\n",
       "        [ 0.5015,  0.0987, -0.0787, -0.0286]], requires_grad=True)"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[-0.8437, -0.5244,  0.8917,  0.0646]], requires_grad=True)"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设置随机数种子\n",
    "torch.manual_seed(420)  \n",
    "\n",
    "# 实例化模型\n",
    "tanh_model3 = tanh_class3()                   # 保留原参数\n",
    "tanh_model3_init = tanh_class3()              # 使用Xavier初始化参数\n",
    "\n",
    "# 修改init模型初始参数\n",
    "for m in tanh_model3_init.modules():\n",
    "    if isinstance(m, nn.Linear):\n",
    "        nn.init.xavier_uniform_(m.weight)\n",
    "        \n",
    "# 创建模型容器\n",
    "model_l = [tanh_model3, tanh_model3_init]           \n",
    "name_l = ['tanh_model3', 'tanh_model3_init']\n",
    "\n",
    "# 核心参数\n",
    "lr = 0.03\n",
    "num_epochs = 40\n",
    "\n",
    "# 模型训练\n",
    "train_l, test_l = model_comparison(model_l = model_l, \n",
    "                                   name_l = name_l, \n",
    "                                   train_data = train_loader,\n",
    "                                   test_data = test_loader,\n",
    "                                   num_epochs = num_epochs, \n",
    "                                   criterion = nn.MSELoss(), \n",
    "                                   optimizer = optim.SGD, \n",
    "                                   lr = lr, \n",
    "                                   cla = False, \n",
    "                                   eva = mse_cal)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d54f88e650>]"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d54f839930>]"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x2d54f88df60>"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'mse_train')"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABGP0lEQVR4nO3dd3zV1f348de52XsDGYwAsmTLEpHpFrTOunfRalu1Wld/rdra1tZR+9W6996jorgQUPYMISSsAIGEkAXZOzm/P85NyLg3ubm5yc1N3s/Hg8dNPnedfEje93ze55z3UVprhBBCeB6LuxsghBDCORLAhRDCQ0kAF0IIDyUBXAghPJQEcCGE8FASwIUQwkNJABdCCA8lAVwIF1NKDVJKlSqlvNzdFtG7SQAXogml1HVKqVWdeQ2t9UGtdbDWus5V7RLCFgngQnSQ9KxFTyEBXPRISqkDSqk/KKWSlVJlSqlXlFL9lVJLlVIlSqkflFIRSil/pdTbSqkCpVShUmqjUqq/9TXCrM/LVkplKaUeaSv4KqVGA88DJ1tTIIXW468rpZ5TSn2tlCoD5imlzlVKbVVKFSulDimlHmryOkOUUlop5W39foVS6q9KqdXWtn+nlIruwtMn+ggJ4KInuwg4HRgBLAKWAg8AMZjf3d8B1wJhwEAgCrgFqLA+/3WgFhgOTALOAG6y92Za6zTr89daUyDhTe6+AvgbEAKsAsqAa4Bw4Fzg10qpX7Txs1wBXA/0A3yBu9v74YVojwRw0ZM9rbXO0VpnAT8D67XWW7XWlcBnmKBcgwncw7XWdVrrzVrrYmsv/BzgDq11mdY6F/g3cJmTbflCa71aa12vta7UWq/QWm+3fp8MvAfMaeP5r2mtd2utK4APgYlOtkOIRt7uboAQbchp8nWFje+Dgbcwve/3lVLhwNvAH4HBgA+QrZRqeI4FOORkW5o9Tyk1HXgUGIvpUfsBH7Xx/CNNvi63tl2ITpEeuPBoWusarfXDWusxwExgISa1cQioAqK11uHWf6Fa6xPbe0kHj78L/A8YqLUOw+TOVatnCdGFJIALj6aUmqeUGmcdnCzGpFTqtdbZwHfAE0qpUKWURSk1TCnVVpoDTC8/QSnl287jQoCjWutKpdQ0TI5biG4lAVx4ugHAx5jgnQasxKRVwPTEfYFU4Jj1cbHtvN6PwA7giFIqv43H3Qr8RSlVAvwZk9cWolsp2ZFHCCE8k/TAhRDCQ0kAF32OUup560Kdlv+ed3fbhOgISaEIIYSH6tZ54NHR0XrIkCHd+ZZCCOHxNm/enK+1jml5vFsD+JAhQ9i0aVN3vqUQQng8pVSGreOSAxdCCA8lAVwIITyUBHAhhPBQUsxKCA9RU1NDZmYmlZWV7m6K6CL+/v4kJCTg4+Pj0OMlgAvhITIzMwkJCWHIkCE0qbAoegmtNQUFBWRmZpKYmOjQcySFIoSHqKysJCoqSoJ3L6WUIioqqkNXWBLAhfAgErx7t47+/3pEAP9xZw7/Xb7X3c0QQogexSMC+Ko9Bfx3+V5k2b8QQhznEQE8Ltyf8uo6iipq3N0UIfqswsJCnn32WaefP3fuXLesxD5w4ABjx451+DEbNmxg4sSJTJw4kQkTJvDZZ591RzOd0m4AV0q9qpTKVUqlNDn2mFJqp1IqWSn1mXUvwi4TFx4AQFZhRTuPFEJ0lc4GcE8xduxYNm3aRFJSEt988w0333wztbW17m6WTY5MI3wdeAZ4s8mx74H7tda1Sql/AvcD97q+eUZDAD9cWMmJcWFd9TZCeIyHv9xB6uFil77mmLhQHlxkf8vQ++67j/T0dCZOnMi8efNITk7m2LFj1NTU8Mgjj3D++edz4MABzj77bGbNmsWaNWuIj4/niy++ICDA/A1/9NFH3HrrrRQWFvLKK69w6qmn2nyv119/nc8//5yysjL27NnD3XffTXV1NW+99RZ+fn58/fXXREZGkpSUxC233EJ5eTnDhg3j1VdfJSIigs2bN3PDDTcAcMYZZzS+bl1dHffddx8rVqygqqqK2267jZtvvrnZewcGBjZ+XVlZ2aMHjtvtgWutfwKOtjj2nda64SNpHZDQBW1rFBfuD0B2kfTAhXCXRx99lGHDhpGUlMRjjz3GZ599xpYtW1i+fDl33XVX4xjVnj17uO2229ixYwfh4eF88sknja9RW1vLhg0beOqpp3j44YfbfL+UlBQ+/fRTNm7cyB//+EcCAwPZunUrJ598Mm++afqT11xzDf/85z9JTk5m3Lhxja95/fXX8/TTT7Nt27Zmr/nKK68QFhbGxo0b2bhxIy+99BL79+9v9d7r16/nxBNPZNy4cTz//PN4e/fMJTOuaNUNwAf27lRKLQYWAwwaNMipN4gO8sPXyyIpFCGs2uopdwetNQ888AA//fQTFouFrKwscnJyAEhMTGTixIkAnHTSSRw4cKDxeRdeeKHN47bMmzePkJAQQkJCCAsLY9GiRQCMGzeO5ORkioqKKCwsZM4cs0/1tddeyyWXXEJhYSGFhYXMnj0bgKuvvpqlS5cC8N1335GcnMzHH38MQFFREXv27GHEiBHN3nv69Ons2LGDtLQ0rr32Ws4++2z8/f2dP2FdpFMBXCn1R6AWeMfeY7TWLwIvAkyZMsWpaSQWiyI23J/DhbKEWIie4J133iEvL4/Nmzfj4+PDkCFDGheg+Pn5NT7Oy8uLiorjHa+G+7y8vNrNKzd9HYvF0vi9xWJxOiettebpp5/mzDPPbHbc3ofJ6NGjCQ4OJiUlhSlTpjj1nl3J6VkoSqnrgIXAlbob5vfFhQVwWHrgQrhNSEgIJSUlgOm59uvXDx8fH5YvX05Ghs1y1V0qLCyMiIgIfv75ZwDeeust5syZQ3h4OOHh4axatQowHzYNzjzzTJ577jlqasyMtt27d1NWVtbsdffv39/4AZGRkcHOnTvpqRvRONUDV0qdBdwDzNFal7u2SbbFhvuzNr2gO95KCGFDVFQUp5xyCmPHjmXq1Kns3LmTcePGMWXKFEaNGuWWNr3xxhuNg5hDhw7ltddeA+C1117jhhtuQCnVbBDzpptu4sCBA0yePBmtNTExMXz++efNXnPVqlU8+uij+Pj4YLFYePbZZ4mOju7OH8th7e6JqZR6D5gLRAM5wIOYWSd+QENEXae1vqW9N5syZYp2dh7oE9/t4r/L97L7kbPx9vKI6etCuFRaWhqjR492dzNEF7P1/6yU2qy1bpXDabcHrrW+3MbhV5xvnnPiwgOo15BTUkW8dVqhEEL0ZT1zbowNx+eCV0gAF6KX+Pbbb7n33uZLSBITE3v06seexGMCeLx1LrgMZArRe5x55pmtZoQIx3lMMjk2TJbTCyFEUx4TwIP8vAkP9JEeuBBCWHlMAAfTC5fFPEIIYXhUAI8P95ceuBBCWHlUAI8Ll9WYQriL1AO3bebMme224aabbiI1NRWAv//97w62vH0eF8CLK2spqZSNHYToblIP3LY1a9a0+5ovv/wyY8aMAVwbwD1mGiEcnwueXVRJiL+Pm1sjhBstvQ+ObHftaw4YB2c/avduqQduW3BwMKWlpaxYsYKHHnqI6OhoUlJSOOmkk3j77bdRSjF37lwef/xxPv74YyoqKpg4cSInnnhiszotzvCoHnjDXHCZSihE95N64O3bunUrTz31FKmpqezbt4/Vq1e3OocBAQEkJSV1OniDh/XAG+aCSx5c9Hlt9JS7g9QDt23atGkkJJj9bSZOnMiBAweYNWtWu89zlkcF8H4hfnhZlARwIdxM6oG332ZHfsbO8qgUireXhQGh/mTLXHAhup3UA3cNHx+fxvfvLI/qgYPZH1Ny4EJ0P6kH7hqLFy9m/PjxTJ48udN58HbrgbtSZ+qBN7j9/a1sOXiMn++Z76JWCeEZpB5439CReuAelUIBM5XwSFEldfXd98EjhBA9keelUML8qanT5JdW0T+05+0SLYRwnKfUAy8oKGDBggWtji9btoyoqCg3tMjwjABeeBBK8yDhpMbFPFmFFRLARZ+jtW53YYkn8ZR64FFRUSQlJXX5+3Q0pe0ZKZRV/4Z3LwWarMaUmSiij/H396egoKDDf+TCM2itKSgocGi+eQPP6IEHREDFMdC62dZqQvQlCQkJZGZmkpeX5+6miC7i7+/fuBDIEZ4RwP3DQddBdSmh/sEE+3nLVELR5/j4+JCYmOjuZogexDNSKAER5rbiGEop4qQuuBBCeEoADze3FYWAtS54kQRwIUTf5iEB/HgPHGRrNSGEAAcCuFLqVaVUrlIqpcmxSKXU90qpPdbbiC5tpX+4ua0sBExZ2aNl1VTW1HXp2wohRE/mSA/8deCsFsfuA5ZprU8Allm/7zoteuAyE0UIIRwI4Frrn4CjLQ6fD7xh/foN4BeubVYLjTnwlgFc0ihCiL7L2Rx4f611tvXrI0B/ew9USi1WSm1SSm1yev6qTyB4+TYOYsZLD1wIITo/iKnNsjC7S8O01i9qradorafExMQ49yZKmTy4tQfeP9QfpWRrNSFE3+ZsAM9RSsUCWG9zXdckOwIiGgcxfb0txAT7SQ9cCNGnORvA/wdca/36WuAL1zSnDQHhjT1wkLngQgjhyDTC94C1wEilVKZS6kbgUeB0pdQe4DTr910rIKIxBw4mDy4FrYQQfVm7tVC01pfbuat1cdyu5B8OOamN38aF+/NDWk6vK68phBCO8oyVmNAsBw4mhVJVW8/Rsmr3tUkIIdzIswJ4VTHUmd2iZS64EKKv86AAHm5uK4sAiAs7vjOPEEL0RR4UwFsupze7VshUQiFEX+U5AbxFQavIIF/8vC1ky1RCIUQf5TkBvEUPXClFfLiUlRVC9F0eFMDDzW2LxTySAxdC9FUeFMAbeuCFjYdkazUhRF/mOQHcP8zctuiB55ZUUVUrGzsIIfoezwngXj7gG9J8MY91KmFOUZWbGiWEEO7jOQEcbBa0AqSolRCiT/LAAF7Y+K3MBRdC9GWeFcCbbOoAsjemEKJv86wA3qKglb+PF1FBvmTJXHAhRB/keQG8SQ8crBs7SA9cCNEHeVgADzc5cH18C87YMJkLLoTomzwsgEdAXRXUHA/YDT1wre3uqyyEEL2SZwXwhoJWTdIo8eEBlFXXUVxZ6542CSGEm3hWAG9YTt9iZx6QmShCiL7HwwJ4uLltNpVQ5oILIfomDwvgrQtaxUsPXAjRR3lWALeRA48O9sPHS8lccCFEn+NZAdxGDtxiUQyQqYRCiD7IswK4Xwgor9aLecICZGs1IUSf06kArpS6Uym1QymVopR6Tynl76qG2XnDVgWtwCzmOVIsKRQhRN/idABXSsUDvwOmaK3HAl7AZa5qmF0tCloBDAgLIKeoivp6WcwjhOg7OptC8QYClFLeQCBwuPNNakeLglZgeuDVdfUcLa/u8rcXQoiewukArrXOAh4HDgLZQJHW+ruWj1NKLVZKbVJKbcrLy3O+pQ1sFLQaEGYyN0eKJI0ihOg7OpNCiQDOBxKBOCBIKXVVy8dprV/UWk/RWk+JiYlxvqUN7OTAAbIlgAsh+pDOpFBOA/ZrrfO01jXAp8BM1zSrDbZ64KENPXCZiSKE6Ds6E8APAjOUUoFKKQUsANJc06w2+IdDZRHU1zceigr2w9uipAcuhOhTOpMDXw98DGwBtltf60UXtcu+gAhAQ1VR4yEvi6J/qL/kwIUQfYp3Z56stX4QeNBFbXFM04JWDSszMQOZ0gMXQvQlnrUSE2wWtAITwGUxjxCiL/G8AG6joBVAbKg/2UWyM48Qou/wvABuo6AVmB54ZU09RRU13d8mIYRwAw8M4OHmtmUPPMzUBZc8uBCir/C8AN6YQilsdlhWYwoh+hrPC+A+/uAdYKMHLqsxhRB9i+cFcLBZ0ComxA+LktWYQoi+w0MDeHirFIqPl4WYED+ZSiiE6DM8NIBHtArgYOqCSwpFCNFXeHAAP9bqcKwspxdC9CGeGcBt7MoD1tWYEsCFEH2EZwbwgPBWg5hgZqKUVNVSUimLeYQQvZ/nBvCacqitana4YS54jgxkCiH6AA8N4LYLWslqTCFEX+KZAdxeQStZzCOE6EM8M4DbKWjVL9QPkOX0Qoi+wUMDeLi5bdED9/P2IirIV3rgQog+wUMDuO0cODRMJZTl9EKI3s8zA7idHDiYPLj0wIUQfYGHBvAwQNmcCy5bqwkh+grPDOAWL/APtdMDD6CwvIaK6jo3NEwIIbqPZwZwsF/QKtS6sYP0woUQvZyHB3DbOXCAbBnIFEL0cp4bwNsoaAWynF4I0ft1KoArpcKVUh8rpXYqpdKUUie7qmHtsrErDxwP4DITRQjR23l38vn/Ab7RWl+slPIFAl3QJscEhNvsgQf6ehMW4COrMYUQvZ7TAVwpFQbMBq4D0FpXA9WuaZYDGgYxtQalmt0lc8GFEH1BZ1IoiUAe8JpSaqtS6mWlVFDLBymlFiulNimlNuXl5XXi7VrwDwddB1Ulre6SjR2EEH1BZwK4NzAZeE5rPQkoA+5r+SCt9Yta6yla6ykxMTGdeLsW7BS0AumBCyH6hs4E8EwgU2u93vr9x5iA3j3sFLQCGBAaQH5pFdW19d3WHCGE6G5OB3Ct9RHgkFJqpPXQAiDVJa1yRBsFrWJlKqEQog/o7CyU3wLvWGeg7AOu73yTHNRGQauGqYRHiisZGNl9E2OEEKI7dSqAa62TgCmuaUoHtZMDB5kLLoTo3Tx3JWZbOfCGHrgspxdC9GKeG8B9AsHL12YOPMTfh2A/b+mBCyF6Nc8N4ErZLWgF0D/UT+aCCyF6Nc8N4GC3oBWYuuDSAxdC9GaeHcDtFLQCWY0phOj9PDyAh7fRA/cnt6SS2jpZzCOE6J08PIBHQEWRzbsGhPlTryG/tPvqawkhRHfy7ADeZg5cduYRQvRunh3AAyKgugTqalrdNSA0AEDy4EKIXsvDA3i4ua1snUaR1ZhCiN7OwwO4/YJW4YE++HlbZHd6IUSv5dkBvI2CVkopqQsuhOjVPDuAt1HQChrmgssgphCid/LwAB5ubmU1phCiD/LwAN6QA7cdwAeE+ZNTXEl9ve7GRgkhRPfw7ADuH2ZubQxigpmJUlOnKSiTxTxCiN7HswO4lw/4htjvgYc21AWXNIoQovfx7AAObRa0ig0zi3lkNaYQojfqBQE8rM0cOCBzwYUQvVIvCOARdnPgUUG++HgpmYkihOiVPD+At1HQymJR9A+VuuBCiN7J8wN4GzlwwLoaU3LgQojepxcE8HDTA9e253r3D/Unp7iqe9skhBDdoBcE8Aioq4Ya273shh64thPghRDCU3U6gCulvJRSW5VSS1zRoA5ro6AVwICwACpr6imqaF0zXAghPJkreuC3A2kueB3ntFPQSuqCCyF6q04FcKVUAnAu8LJrmuOEdgpaNc4FlwAuhOhlOtsDfwq4B7C79btSarFSapNSalNeXl4n386GdgpaSQ9cCNFbOR3AlVILgVyt9ea2Hqe1flFrPUVrPSUmJsbZt7OvMQdeaPPumGA/vCyKzGPlrn9v4TbfpGSz60iJu5shhFt1pgd+CnCeUuoA8D4wXyn1tkta1RHt9MC9vSyMjg1hc4bt+4Xnqa2r5/b3k/jrklR3N0X0EYeOlpNX0vOmIzsdwLXW92utE7TWQ4DLgB+11le5rGWO8gsB5dXmYp4ZiVFsPVRIZU1d97VLdJl9+WVU1dazJj2/R/5Rid7nutc28MBn293djFY8fx64UscX89gxfWgU1bX1bDtU2G3NEl0nLbsYgHptUilCdKWiihrS88rYknGsx60ncUkA11qv0FovdMVrOaWNglYA04ZEohSs33+0+9okukxqdjG+XhaGxQTx5TYJ4KJr7cgqAqCgrJqswp5VlsPze+AAwQMgN83ucvqwQB9GDQhl/f6Cbm6Y6Aqph4sZ3i+YCybFs+HAUal1I7rUdmsAB9h2qKiNR3a/3hHAJ1wGeWmwb4Xdh0xPjGRzxjGqa+3OeBQeIi27hDFxoSwcHwfAV8nSCxddZ3tWEf1D/fD1spCcWeju5jTTOwL4+EshqB+sfcbuQ2YMjaSypr7H/QeIjsktqSS/tIrRsaEMiQ5ifEIYX2477O5miV4sJauIiQPDGR0XSlIPG0frHQHc2w+mL4a9P0CO7all0xKjAMmDe7q0bDP3e0xsKACLxsexLbOIjIIydzZL9FLFlTUcKChnXHwYExLCSMkqoq6+5wxk9o4ADjDlRvAOgLX/tXl3ZJAvI/oHs26f5ME9WcMMlIYAfu74WACWSBpFdIEdWeb3bWx8GBMSwimrriM9r9TNrTqu9wTwwEiYdBUkfwAlR2w+ZMbQKDZnHKOmTvLgnir1cDFxYf6EBfoAEBcewNQhEZJGEV0ixTqAOTY+jAkDwwB61HTk3hPAAWb8GuprYcOLNu+enhhFeXVd43+K8Dxp2cWMiQttdmzRhDh2Hilhd44srReutT2riNgwf6KD/RgaHUywnzfbetA4Wu8K4FHDYPRC2PgKVLfOiU5LjAQkD+6pKmvM5evo2OYB/OyxsVgULJFeuHCxlKwixsabnrfFohgXH0ZyZs/pAPauAA5w8m/Nsvqt77S6KybEj2ExQayXPLhH2p1TQr0+nv9uEBPix8xh0XyZnN3jVsoJz1VSWcO+/DLGWQM4wISB4aRlF1NV2zPKcvS+AD5oOiRMhXX/hfrWJ3n60Cg2HjhGreTBPU7qYTOg1LIHDrBoQiz788vYYX2MEJ3V8PvWLIAnhFFTpxtnQ7lb7wvgADN/C8cOwM7Wu7xNT4yktKqW1Gz5Q/c0adnFBPl6MSgysNV9Z544AG+LksFM4TLbmwxgNhg/MBzoOQOZvTOAj1oIEUNgTeuFPTOGWueD75M8uKdJyy5hVGwoFotqdV94oC+zR8SwJDmb+h40T1d4rpSsIgaE+hMT4td4LM46oNlTBjJ7ZwC3eMGM2yBzAxxc3+yu/qH+DIkKlLooHkZrTVp2MaNjQ+w+ZtGEWLIKK9h6qGtrv+/OKZH68n3A9qwixsY3T9cppZiQ0HMGMntnAAeYdKXZrWft063ump4YxYb9R3vUiirRtsxjFZRU1TImNszuY04b3R8/b0uXVyi875Nkrnx5HXtze86CDuFapVW17Msva5Y+aTBhYDjpeaWUVNa4oWXN9d4A7hsEU2+EtCVwdF+zu2YMi6S4spadRyQP3lXKq2t5d/1Bl31I7mgcwLTfAw/x92H+qH4sSc7usg/nksoatmUWUVlTzx0fbJXiaL1UWnYxWjcfwGwwPiEMrZtXKXSX3hvAAaYtBi8fWPtss8PTEyUP3tX+l3SYBz7bzk97XLORdVp2MRYFowa0noHS1KIJceSXVnXZVNH1+8yV2w2nJJKSVcyT3+/ukvcR7rXdmiKxFcAnJIQDPaO0bO8O4CEDYNwlkPQOlB8P1nHhAQyMDJA8eBdq6J0s35nrktdLyy5mSHQQAb5ebT5u3sh+BPl68WUX1UZZtTcffx8L9549ksunDeSFn9JZmy6/R71NSlYR/UL86Bfq3+q+iCBfBkUG9ojKpr07gAOc/BuoKTerM5uYnhjF+v1H252xUFlTJ3+gTkixpjyWpeW6ZHFNanaxzfnfLQX4enH6mP4sTcnukpo3q/fmM3VIJH7eXvxp4RgSo4K468Mkisrdnw8VrrO9yQpMWyYMDO8RUwl7fwDvPwZOOAPWPw/V5Y2HpydGUlhew+5c+xPy6+o1v3l3C5e/tE7y5R1QU1dPWnYx0cF+ZBVWsDunc4N9RRU1ZB6raLUC055FE+IoLK9h1Z78Tr1vS7nFlezJLWXW8GgAAn29eeqyieSWVPHHz7d3yyrQmrp6Dvewbb16m/LqWtLzStsO4AlhHC6qJLekshtb1lrvD+AAs+6E8nyTSrFqbz641pqH/reDH9JMCmDNXid74UVZcGgDVPWdGQt7c0uprq1n8exEAJbtzOnU6+1sUUK2PaeeEEOovzdfJrt2Uc/qdPOBcIo1gAOMTwjnztNHsCQ5m8+2Zrn0/Wx54rvdzHt8BbnF7g0cvVladjH1dgYwG0ywLuhJdnMevG8E8EEnQ8I0WP1/UGcudRMiAogL87ebB3/p5328tS6Dm2cPZWBkgPN1xD+4Cl45Hf6RAE+fBB9dBz8/CXt+gFLX5Ie7TMYaKO54Lrmh2uP8Uf0ZGx/a6Tx4Yw3wOMcCuK+3hbPGDuC7HTlU1riuZsWqPQVEBPq0+iC5Zc4wpg2J5M9f7ODQ0XI7z+68oooa3l6XQVVtPe9tONRl79PXtTWA2eDEuFAsCrfnwftGAFfK9MKLDsKOz6yHFNOHmvngLS99v0rO5u9f7+Tc8bHce9YoZiRGseFA+/nyVkpy4PAWmHAFzL0fYkZB1hZY9jC8cxE8fgI8PgK2f+yqn9R1jqTA6wvh+z93+Kk7Dpsl70Ojg5g/qj+bM45xrKza6aakZhcTGeRLvyYr4tqzcHwcpVW1rNztmlkwWmvWpOczc1h0q5WgXhbFk7+cgALu/CCpy+rsvL0ug9KqWob3C+bdDRlS176LbM8y6b/+ofZ/3wJ9vRnRP4QkNy/o6RsBHGDEWSaArvp34+710xMjyS+tbrbDxqYDR7nzwySmDI7giUsmYLEoZgyNorC8hl0drTe99wdzO+PXMPdeuOwduCMZ7s2A676Csx6F4P7w5R1O9XS7TH09fPV70HWQ/qP5vgO2ZxUxJs4seV8wqh/1mk4F0rTsEkbHhqBU6yX09swcFkVkkK/LaqPsyy8ju6iSmcOjbN6fEBHIIxeMZVPGMZ5bke6S92yqsqaO11YfYPaIGO4/exQ5xVV8t6NzqSlhW4p1BWZ7v28TEsJJzix0awVMpwO4UmqgUmq5UipVKbVDKXW7KxvmchYLnHIH5KbCnu+A43nwddY8+L68Um56cxPx4QG8dM0U/H3MlLXpQyOtj+tgGmXPdxA8AAaMa348IByGzDKB/dI3oL4GvrnP6R/N5ba9C4fWw7D5ZuzgSLLDT62r16QeLubEOHP5OS4+jOhgP350Mo1SW1fPrpwSh/PfDby9LJw9dgDL0nIpr6516r2bWrPX5L9nNcl/t3T+xHjOnxjHU8v2uHzm0idbMskvreKWOUOZO7IfCREBvLn2gEvfQ0BFdR17ckvaTJ80GD8wjMLyGg52YdqsPZ3pgdcCd2mtxwAzgNuUUmNc06wuMu5iCE0wvXBgcFQg/UP9WLevgILSKq57bSNeSvH69VOJCPJtfFpCRGDH8+B1tbBvOZxwmknh2BM5FGb/AVI/h93fOvmDuVD5UfjuTzBwBvziOXMsfZnDT9+fX0pFTV3jH4DFopg3MoYVu3KbpxY2vQovzbdZ8repffllVNfWOzSFsKVFE+KoqKljWVrnxxpW7c0nISLAZiXEpv5y/ljiwwO44uV1/P6DJLJcMGOkrl7z0k/7mJAQxslDo/CyKK6aMZj1+4+y60jPKGvaW6RaBzDbmoHSoHFBT2YRFB6EWufThM5yOoBrrbO11lusX5cAaUC8qxrWJbx8TKnZg2shY63JgydGsW7fUW56cxM5xZW8dO0UBkcFtXqqo/PGG2VuhMoiGH56+4+d+TuT3vnqbps7CXWrZQ+bdp/7hFkI1X8c7P3R4afbKsG5YHQ/iitrjxeAqiqFZX+FrM1wcF2br9fRAcympg6JpF+IH0s6ORulrl6zNr2AU4ZFt3tZHRbgw5e/ncXNs4exZHs28x5fwT+WplFU4fw88W9SjnCgoJxb5gxrfP9fThmIn7eFt9YdcPp1Xe2H1BxufmsTFdU9Y7MDZ+w43P4AZoORA0Lw87awe38GPDMV1vynq5vXikty4EqpIcAkYH07D3W/yVdDQCSsfgow6ZH80iqSDhXyn8smMXlQhM2nNeTB25o33sze70F5wdC57T/W2xcW/tsMsq78p2Ov3xUObYTNb5jUzoCx5tjw+SadUuXYz52SVYy/j4VhMcc/BGedEIOPlzqeRtn0ClQcNecn7cs2Xy/1cDG+XhaGxQR3+MfxsijOHR/L8l15nSo8lJJVRHFlLaecYD990lRYgA/3nT2K5XfPZeG4WF78aR9zH1vOq6v2d7h2itaa51emkxgdxBknDmg8HhHky6IJcXy6JYviHlBUaV9eKbe/v5Vvd+Tw3ErXjwF0htaab3cc4W9fpbY7K2l7ZhFRQb7EhrVegdmSj5eFE+NC8d2/DGorYdc3rmqywzodwJVSwcAnwB1a61arXZRSi5VSm5RSm/LyXDMjoFN8g2D6LbD7G8jZwewTYvDztvDnhWM4a+wAu0+bbt1Pc52juc0938GgGSbf7YjBM2HyNaaG+ZEUx57jSnW1ZuAyZADMbZKPH7bA5OgPrHLoZVKyihgdG4q31/FfrWA/b6YnRpkAXl1mpnMOm28WWO1c0jiobEtqdjEn9A/Gx8u5X9VFE+Korq3n+1TnB/xWWfPfM4fZHsC0Jz48gCd/OZEvfzOLMXGh/GVJKqc9uZIlyYcdHvhak17A9qwiFs8eileL2S/XnDyY8uo6Pt2c2aF2uVplTR23vbsVX28Lc0bE8MLKdDKPuS8v3NTe3FKueXUDN7+1mZd+3s+fPk9p89w3rMB0dMB8fEI4w4/9bL7J2tysZEd36FQAV0r5YIL3O1rrT209Rmv9otZ6itZ6SkxMTGfeznWm/Qp8gmD1fxgYGcj2h87k+lMS23zKwMhAEiICGgc821RyBI5sh+Gndaxdpz0MARGw5I4Oz/zotE2vmMHKs/4Bfk0q/g2aAT6BsLf9PHh9vWbH4WLGxrW+/Jw/qh97cks5tvIFMzA65z6zAXXRIchOsvuaZgZKx9MnDSYNDCc+PKBTs1FW781n1IAQooMdn8bY1Nj4MN6+cTqvXz+VAB8vfvPuVu7/dLtD6bjnV6YTE+LHBZNaZyfHJ4QzYWA4b63LcOtMiIe/TCUtu5gnL53IPy4ch1Lwj693uq09YKpG/v3rNM566ieSDhXy54Vj+M284Xy0OZM31hyw+ZzKmjr25Ja2qgHelknxgZyqtlEZNQbQZtyrG3VmFooCXgHStNZPuq5J3SAwEk66zsy/PpaBr7djp2HG0CjW7y9o/w+vYfrgCQ7kv1u268y/m/z55tc69tzOKDkCPz5iesVjftH8Pm8/M2Mmvf08eMbRckqram3+ASwY3Q9/qvDf+IxJKw2aDiPObjONkltSSX5pVacCuFKKhRNi+XlPvlNz0Str6tiUcazN2SeOtmPuyH58ffup3DZvGO9vPMQ9nyS3WfY2JauIn/fkc8MpiY0zolq69uTBpOeVscZN9Xq+SMrivQ0HuWXOMOaN6kdceAC3zh3OV9uznV/81glaaz7bmsn8J1by4k/7uHByPMvvnssNsxL5/ekjOH1Mf/76VRpr0luXWUjLLqauXjuU/24wTaURoirYOHix2X+gA+NFrtCZHvgpwNXAfKVUkvXfOS5qV9c7+TZQFlj7X4efMmNoFMccyYPv+R5CYqH/2I63a/ylkDgHfnjYBNbu8N3/Mzm8cx63PWNm2AI4mm72GW1Dio0BzAaDo4L4bdgqAqoLYM695mBQlEkdpbXeuxRo3Di2o1MIW1o0Po7aepMH7ahNB45RXVvfbPl8Z3hZFH84cxR3nHYCH2/O5A8fbbMbxJ9fmU6InzdXzhhk9/XOGRdLZJCv3V5lV0rPK+WBT7czZXAEd58xovH44tlDiQ8P4OEvU7t105SUrCIueX4td36wjbgwfz6/7RT+dfGExisni0Xx5KUTSIwO4rZ3trRaNdtQgM2RGSgNBhxZTgW+fF89FobNMzO2uvFqqDOzUFZprZXWerzWeqL139eubFyXCouH8b+ELW9CmWNFjxry4G3WEa+rhfTlMHxB29MH7VEKzn3SBNRv7u/48ztq30rY/pFZqRo1zPZjhi8wt+2kUVKyivD1snBCPxubLtRUcG3956ytP5GyAdOOHx99HuTvgrzWdbUbdgXvbAA/MS6UxOggp2qjrE7Px9uimGb9v3eVO04bwe9PH8GnW7O468PWqzczCsr4ens2V8wYRKi/j93X8ffx4pdTB/JDWo5Lpiw6qqK6jtve2YKfjxdPXzGp2ZiHv48XD5wzmrTsYt7feLBb2vPu+oOc98wq9ueX8a+LxvPZracw0VqvpKkQfx9eumYKdfWaxW9tbrZGICWziIhAH+LDAxx7U61Ru74hzX8ym7IqTUenJNusNekmfWclpi2n/M4EyvUvOPTwgZGBxIe3Mx88cwNUFZkBOmdFD4fZd8OOT03NlK5SWw1f3WU2gJ51p/3HRQ2HsIHtplFSDhcxckCI7ZTU5jcIringqZoLGwcFARh1rrnd2TqNkpZdTHx4AGGB9gOYI5RSLBwfy9r0AvJKqjr03NV785k8KIIgP+9OtcGW3y04gT+cOZLPkw5z54fbmgXxl37eh7fFwg3tjM0AXDnd9NDfXZ/h8jba8/CXO9h5pIQnL51AbFjrgHfOuAFMT4zk8W93dXmp3edWpPPAZ9uZO7IfP949l0unDrS58XWDxOgg/u/ySew6Usw9Hyc3jh90dACT3FQoOkhu3Hx25ZRQOXiuOe7AeJGr9O0AHjPSBJANL0ClY+ViTR68jfnge74Hi7dj0wfbcsrtEHWCmRlS3UUj+mufhoI9JnXi00avQymTH9//U2MxsJa01qRkFdu+/KyphNVPUT9oJql+4/ix6cKasHiIP8lmHjy1nU2MO2LRhDjqNSxNcbxkQWF5Nduziuwun3eF2+YN596zRvHltsPc/kESNXX15JdW8dGmTC6YFE9/GxsKtJQQEcj8Uf15f8Mhqmqdn4N96Gg5j3+7i7fWZZBRYH89wmdbM3l/4yFunTuMuSP72XyMUooHF51IUUUNTy3rml2LtNb8Y2ka//xmJ+dPjOOFq08iLMCxD/u5I/txz1mjWJKczXMr06msqWN3TkmH0ifsMgkHvzFnU1ev2VEaBP3GHB8D6wZ9O4ADnPp7s3DljYVwdH+7D58xNJKjZdXssbeh7d7vYeB08O/AL4It3n6w6CkozIAvbnX9rJTibPjpcRi10LHB1uELoKoYMjfZvDvzWAVFFTW2R/C3vgUl2Vjm3sfsETEs35Xb/ANw9CI4vBUKj1fYq6ypY19eaacGMJsa0T+EEf2DWdKBDY/X7StA67aXz7vCr+cO44/njOar5Gx+995WXv55P9V19SyeM9Th17h25mAKyqr5envHa+ocOlrO/Z8mM+/xFfx3xV7+9HkKcx5bwex/LeePn23nm5QjjXPN9+aW8MCnKUxLjOT3p49o83XHxIVy+bRBvLk2gz0drSPUjrp6zQOfpfDCyn1cNWMQ/750Yoenmt48eyjnTYjjsW938cLKfdR2cACTXd9A/EmMPsGch9+9l8QP1WOpPbCG75PS2ZdX2uVjAK6/LvQ08SfBZe/B57fAC3PggueOX9bbcLx+SgEjB7ToHTZMH1zwoGvaNmQWnPGIGWQMiTUzVJzJq9vy41+hvhbO+Ktjj0+cYwZ905fB4JNb3d04gNlyCmFtlSldMOhkSJzNgsIsvkrOZsfhYsYlWB87ahH88BDs/Apm3ALA7pwS6nXn899NLRofxxPf7ya7qMLmZX9Lq/bmE+Tr1Vj7uSv9avZQlIJHvkoD4MwT+3do8dIpw6IZGh3Em2szuGBSgkPPOXS0nGdX7OWjTZlYlOLK6YO4Ze4wKmvq+XlPHj/tzufzrVm8s/4gXhbFpIHh5JdWEejrxdOXN89723PXGSP5ctth/rIklTdvmNahgmT2VNfWc+eHSXyVnM1t84Zx9xkjnXpdpRT/vGg86Xml/PsHc5XgcAAvyYGsTTDv/zEgzJ8HF41hbXoBSw+fyGn6Q9758D1W1E/Cz9vC8H7BjOwfwo2nJjbWCHIV6YEDjDoHbv4JIhPh/StMwLSTKmjIg9usI944fbAT+e+WTv4NzLgV1j0La59xzWse3mo2t5jxa1OLxREB4RA/xW5+L+VwEd4W1fpDbevbUJwFc+4BpZgzIgalWmzyED0cYkabRT1WqY270LsugC+cEAeYcsGOWL23gOlDo5xeRNRRN506lIcWjSHE35vfzDuhQ8+1WOujbD1Y2Phhak/THvcnm7O4cvogVt4zl4fPH0tsWACJ0UFcc/IQXr52Clv/fAYfLJ7Br+cMo6aunuyiSv79y4kOpXYAIoN8ufP0Efy8J79xc5TOqKiu41dvbuKr5GweOGcUfzhzVKc+FAJ8vXjh6pOIDPIlPNCHhAgHBzD3WOsWjTwbgOtPSeTFa6bwxF23oL0DeHxiHo9fMoFrZw4hOtiPtfsKuqTEgPTAG0QMgRu+hW8fgDVPm2Xll7wGoXGtHjp9aCQrduWhtW7+y7PnOwiJg/4nuq5dSsEZfzOj29/9P1PdcPwlzr+e1mZ2S2A0nHp3x547fAGseNSsNgtsPitje1YxJ/QPaT5fubba9L4TpsHQeQBEBfsxaWA4P+7M5Y7TmlyCj14IPz8BZfnowCi2HDxGkK9Xu8WjOiIxOoix8aF8mZzNTae2/cGVVVjB/vwyrpox2GXv74jrTknk6pOHtFp16YiLTkrgsW93cf+n2xkdG0Jtvaa+XlNbr6mz/quw7vHatMfd1tWIr7eF6UOjmD40irvPHNn6d94BV80YzDvrD/LIV6nMHhGNn3fbG1PbU1RRw42vb2TLwWM8euE4Lptmf3plRyREBPLur6aTV1Ll+M+2a6kZ2G/5t+7jjxoyi+icVVx8adfXRpEA3pSPPyx80lzuf3k7PD8LLnrZDOA1MWNoFJ9uyWJPbikj+lt7nHW1kL4CxpznujRHA4sFLnjBTHf8/NcQHOP8IGnqF6aY18KnwL+DvdthC2DFP8xqs7EXNR7WWrMjq4gFo1sMaCW9Y1ZaLnqq2TlZMLo/j327i9ySSvqFWHtyoxfBT4+xZunbPHRoErtzSjl9TP82ZxM4Y+H4OB5dupNDR8sZ2MaHw+q9Ddundd0Apj3OBG8wNVhunjOUd9YfJL+0CotSeHspvCwKL2Vuvb1MT/3mOUMdSiO15Exv18fLlKq45tUNPPxlKrOGRxPo60WQnzcBPuY20NeLAF8vKqvryC2pIq+kitySSnKLq8grrSK3uIqUw0XkFFfyzBWTOWdcbIfb0ZZRA0IZZb+SRnM1FWaq8OSrbf+tDz8NvrnXrJuIGOLCVrYmAdyW8ZdA7Hj48Fp460JTG2T2PSaQAic3yYM3BvDG6YMdXH3pKG8/+OXb8NrZ8P5VcP3Xpo0dUVMJ3/8J+p1o6q50VPxkMzi798dmATy7qJKCsurmI/i11WbruPiTTOBvYv6ofjz27S5W7Mzj0qkDSc8r5a0NXvxKx1Cx7XN8+53Evy4ez3kTWl/9dNa542J5dOlOvkw+zK1zh9t93Oq9+UQH+zKyv41ZMGlLoDQHTrq+8Xeip7jjtBHNr2x6iNkjYlg4PpZ31x/k3fUdmxseFuBDvxA/hkQF8eiF45nlYFGxLrNvJdRWmE1ibGm6bmLqjV3aFAng9sSMhF8tgyW/N73OokxY9B+weJEQEdA4H/yak4eYx7tq+mBbAsLhyo/NHpvvXAw3fg8RHbjEX/esqVt8zRdgceIy1mKtrtiw2sza+2jIuTYboFnxd1Nd8dwnWvVSRg0IITbMn3fWZ/C/bYdZtTcfHy/F3H5zmVf0BfN/NQHV0asDBw2MDGTSoHCWbMu2G8C11qzeW8BMW+VjN79udlBCm8voC14wK0pFu/7vsknce9YoyqprKa+uo7yqzvq1+b6sqpYAHy9iQvzpF+pHTLAfMSF+dssIuM3upeAbYiYZ2BI1HMIGmXUTEsDdyDcILnjeXAatfBSqS+GCF1HevkwfGsnKpnnwPd+bTRA6O32wPWHxcNUn8OqZ8PZFcON3rfLRNpXkmB7xyHM69yEzbIFJw+TthH6jAbME2aKazBjZ/IbJfZ90nc0rEqUUp43uz1vrMogN8+fuM0bwy6mDiDkaDq99ZKZiNunhu9qi8XH8ZUkqcx5bzuCoIIZEBTa7raypI7+0qvX0wQ0vwdd3mxrvJ5xuxiSenwUXv2pzZk6XafLh6UksFtVm2soj1Neb6YPD55urYluUMr3w7R+byRBenVuI1hYJ4O1RCubdD37B5g+2qhR++RYzEk0efG9uKScElELOdjjtoe5pU7/RcPn78OYv4N1LTWolpJ0E3vJHzGXf6Q5OG7SnYTxg77LjATyriOH9ggnw9TLHl9xp8oDntO59N7j7jJGcNdas1mucjhY0HYJiTIqirQBeVWo2nhg43eyy1EGXTh1IUUUN6XmlZBSUszXjGCVVrbdda1b/e+2z8O395gPwktfNH++gGfDRdfD6uTD//5kt+7o6pfLzE+Yq4Novuzy/KmzIToLSI+b3oC3DF5iCdIc2wJBTuqw5EsAdNfO34BtsgtPbF3PyWa8CJg9+gp+1HrAju++4yuCZZoD1kxvh6SnmQ2baYtuf9tnJsOUtMx0x2n7e1yHhAyF6hEmjzPwNYAL4rOHRkLPDjBv0Gw0XvwZe9n+9wgJ9WheIsniZP4yUT0y+3sfGVLWj++C9KyAvzfSIocNBPNjPmzubLELRWnOsvIYDBWVkFJRxIL+cEH/v4zUxVv8Hvv+zqdty0StmAw6A2AmweKUZ8F72MGSstqZUuihHu+cHs5MRGj642lx9tbWCVjguN82kF0ec2fbjdi016yHamyqcONtU2kxf1qUBvGeNwPR0U643QfPQOhK+vIxRoTWmPvje710/fdARY86DW9eZnuC3D8ALs+HA6uaP0drcFxABc/7gmvcdtgAy1kBNBbnFleSWVDE1ugreucRcqVzxYcdnuDQYfZ5JVe1f2fq+vT/Ai3NND+jyD0wO8tPFZgFQJyiliAzyZfKgCC6YlMCdp484Ps3wp8dM8B57kflQagjeDfxDTQrl3Cdh/88mpdLy/8AVCg/Bp78yS7Uved3Ubl/y+45VvtMakt4zJRHcoSDdfMj3NMWH4Y1F5mp2+d/bPqe7l5pUaXtpS/8wGDity5fVSwDvqHEXwy/fRuXs4DX1EPvTd6HTHdi8uKtEDYMrP4LL3jWphdfPgU9+dbwU7c6v4MDPMM8axF1h+AJTBCxjDSmHiwikkvN23GlKElzxocnTOytxNviFQtr/jh/TGlY9ZT4gwgbC4hUw8iy4/D0zM+aj61xfQEhr88f84yMw/jK44EX7VxRKmcGqm34wm1+8sRB+/JvrNrmtrTY/Y10NXPomnHiB2RBj27tmIw5Hf54fHjIrjt88H9Y9131lT4/uMx+0T58Ez800HzxVdkpRdLe6GvjoelNvaMz5ZkvDL28304JbKjxkVlqPtDP7pKXhCyB7G5R23U5kEsCdMfJsuPIjomuP8E7d3aiq4lbpk4MF5Tz53S5m/fNHLnx2NfvyuvAXVimz/P+29Wa6Y+rnJq2y5hmTt48ZZaa8ucrgU8DLF9J/ZEfmUZ72eZrAY2mmh9rRqY0tefuay9idX5s/ouoy+PgG+OFB8wd243fHc79+IebDK2YkvH+luSpwBa1NSmTlP2HSVfCLZ9tMBzWKHQ83r4Rxl8BP/zJXRJmbO9+e7/9klm2f/8zxFNice81l/NL7TJ61LfX1sPQesw/sSdeZNNU395l0oJ0Vxy5RlGWC4TNTIfV/Jg0541bY9Co8d7KZjuduPzwEh9bBef8Hl7xhFrdteQM+vLp1Ebnd1j0v28t/N2iYPtuFu/So7tyKacqUKXrTJtvFkDzRkR0/E/DhpYRYqrHcu59ySyBLtx/hw02HWL//KEqZQkgpWUVU1dbz1/PHctFJjtWp6JSCdFh6r0ntAFz5iblCcKU3zoPSXJZXjWBe8RcmheCqKVOpX8CH18D5z5qeYk4KnPagGSS0dZVTlg+vnWMuha/9wsw9d9aRFFMnZvc3MOUGMxDrzMDkrm9MgCw9YoLWvD+CrxMzMFI+hY+vN69x1j+a31dxzKSUaqtMLj6kf+vn19fB/34HSW+bsgxnPGI+oH78i5kplDgHLn3DdVdnAKW55rU3vgK63qQeT73r+EB7xlpToO3oPph6k9lK0K/jm1Z3WsPv2bTFcM5jx49veAm+/oNJgVz+/vF0yVsXmsU5v9vi2OvX18Pjw82A/oUvdqqpSqnNWusprY5LAHee1ppL//4244JLKIufxVfbsymtqmVIVCCXTBnIhZPjiQ0L4EhRJbe/v5X1+49ywaR4/vqLsQR3QX3pFo0zAy6FBxsLRLlUw8AesCzilyy4vXO/oM1Ul8G/hpo0jX8YXPRq+x9AxYfNIqeKQrjuKxjQwd2QCtJNyiTlE5PCmX0XzPxd59JilUWmh7fpVXPVcN7TJkXkqPw9JkD3G2N+ppb5dzCX9C+fblJJ13zRfBC7rsakLnZ8alIuc+9r/vMkvWuCe8Rgk/qyt6EHmNWHe3+w5s+VGTz1CQBvf5M28vEH7wDI3WHq69dWwcTLzZVCuI0l79Xl5oNy3XNmYPz8/3bs3HRW/l5zbmNGwvVLW5/bHZ+bMYeIRDNtNyDc/E5OWwxn/s3x9/nkJti3Au7a3akZShLAu8jvP0ji061ZBPp6cc64WC6dMpCpQyJaLQCpq9c88+Ne/rNsN4MiA3n68snHq/F5otw09LMns7RuKpmnPcviOR0rvtSu//0WDieZAbu2AktTxzJMEK+rhuu/cWzGTeEhkypJetdMDZzxa3Op78oe6YFV5uc5ug8mX2sqQLa3XqC6DF5aAGW5ptBaWBtXbskfmmDTtJdeU2ny5ruXwul/MfXlbclYCx9caXrql74JQ+ccv6+mwqxvSP0cdn9rBpd9g81soZpKqLOzOcbYi2DuA46d/5a98QV/7vq1FNXl8PJppr7QzT+ZDxBbDqwyM558g0zqacXfzQepvQU8tiRZK53e/JOZteQkCeBd5HBhBZszjjFvVD+HetXr9xVwxwdJ5JdWce9Zo7hxVqJLSmy6w6YNq7ns03ze/NUpzBzm4qlzzi5Wyd9jgriXr8n7B0aaaX2B0RAYZb4OiIDyAjOnepOZDsqUG01t+GDbGxR0WnW5WdG79hkI7g8TLjdTQQdOax2wtIbPboHkD0zvr2FpdluW3gvrnzfTHEeebcYE9i03m3VM+1Xbzz26H967DAr2wtn/hKB+sOMzE7Rrysx5G73IbHg95NTj4wH1deYqqaYSasrN1z6BHR/Ebtobt3iZFFjiHPNhkjDV/oIZZ2ht6glte9+xc5uzwyyYK8k2mxb/Id2x8ZAGJTnwxAjzwXTqXU43WwJ4D3KsrJp7Pknm+9Qc5o2M4f5zRnNCv2CPC+T/Xb6Xx77dxbYHz3B4J5RucSQF3r/cpI9sUiZQaA2TrjQDv/Z6Ya6WtcUMLB9ab+qxK4vZ/HrwTFNEbfBMs9PLl7fD3PtN2sMRdTVmKlz2NjNonZ1k0hITr3Ds+ZVFZrC4YdpbQ9A+8QIYPKtjQctZh7eanZn2rYTDW0z+3DvATJMdOscE9dgJzpWBaLDpNVhyR8fObeFB+OAq8+HVkfRJg+dngV8YXO/8dFcJ4D2M1pq31mXwyFdpVNfWExnky7QhkUxLNP9Gx4Y6XZXOGeXVtRw6WkHmsXIOHS3n0LEKso5VMCDMn6lDIpmaGHG8cqDVre9sJiWrmJ/umddt7eyQ2mrT0y4vgPJ8M9jZ8H1dNUy62vH0jKtVl0HmRpNCOLjGlC+ubdiU2LqF3ZUfdyxvWnLEzHwpL4ALX4KxF3asTXW1ZgwgpH/3BW17KovMfPr9K01AzzMbXRAQYUpBDJtvShR35IP38FZ45QwTiDt6bsH5q8LvHzRXXvceMDOnnCABvIc6XFjBqr35rN93lA0HCjh01PwRh/h7M3VIJCcNjiA80AcvpbA0KQva8LVSpsh9eU0dldV1pkhQTS0V1XVUVNdRVVtPndZoramv5/jX2uTlCytqyDxaTkFZ8znL/j4W4sLNAGy5tRD90Oggpjb5kLni5XWMjw/nv1dO7vbz1uvU1Zjec8Yakw+e/yfnimQd3W+CX9xElzfRrUpzTSDft9wUiSqxbsoRPcIE82HzzfRWi5f5cKwuNamZxq9LzYK2+nqTj+7OAmT7fzJXR5e92+ZuX22RAO4hDhdWsGH/UdbvP8qG/QWk59nfXNYeX2+Lqa/s44Wft6Ux2FusAd/LYr62KAgN8CEhIpCEiAAGRgYy0HobFeSLUoqaunp2HC5mw/4CNuw/xsYDRymqOD53+J6zRrZZllUIl9PaFFNL/9H8O7C6ydVLG7z9zSBkQqs42LVqq61L6k91erqkBHAPVVRRQ2VNXeOOKvW66S1oNAE+phh+gI/558hehc6qr9fszi1h4/6jpGaX8Jv5w4/XDBHCHWoqzSYlmZtM2scnyMwc8Q0ys2Z8g8wc/NAEjy392yUBXCl1FvAfwAt4WWv9aFuPlwAuhBAdZy+AO91VU0p5Af8FzgbGAJcrpcY430QhhBAd0Zlr7WnAXq31Pq11NfA+cL5rmiWEEKI9nQng8cChJt9nWo81o5RarJTapJTalJfXdVW5hBCir+nyaoRa6xe11lO01lNiYmK6+u2EEKLP6EwAzwKazqJPsB4TQgjRDToTwDcCJyilEpVSvsBlwP/aeY4QQggXcXqtrNa6Vin1G+BbzDTCV7XWPXC/JCGE6J06VexAa/018LWL2iKEEKIDunUlplIqD8hw8unRQL4Lm+NK0jbnSNucI21zjie3bbDWutUskG4N4J2hlNpkayVSTyBtc460zTnSNuf0xrbJpsZCCOGhJIALIYSH8qQA7sJdc11O2uYcaZtzpG3O6XVt85gcuBBCiOY8qQcuhBCiCQngQgjhoTwigCulzlJK7VJK7VVKObiVdPdQSh1QSm1XSiUppdy6W4VS6lWlVK5SKqXJsUil1PdKqT3W24ge1LaHlFJZ1nOXpJQ6x01tG6iUWq6USlVK7VBK3W497vZz10bb3H7ulFL+SqkNSqlt1rY9bD2eqJRab/17/cBaaqOntO11pdT+JudtYne3rUkbvZRSW5VSS6zfd/y8aesmtz31H2aZfjowFPAFtgFj3N2uJu07AES7ux3WtswGJgMpTY79C7jP+vV9wD97UNseAu7uAectFphs/ToE2I3ZpMTt566Ntrn93AEKCLZ+7QOsB2YAHwKXWY8/D/y6B7XtdeBid//OWdv1e+BdYIn1+w6fN0/ogcvGEQ7SWv8EHG1x+HzgDevXbwC/6M42NbDTth5Ba52ttd5i/boESMPUtnf7uWujbW6njVLrtz7WfxqYD3xsPe6u82avbT2CUioBOBd42fq9wonz5gkB3KGNI9xIA98ppTYrpRa7uzE29NdaZ1u/PgL0d2djbPiNUirZmmJxS3qnKaXUEGASpsfWo85di7ZBDzh31jRAEpALfI+5Wi7UWtdaH+K2v9eWbdNaN5y3v1nP27+VUn7uaBvwFHAPUG/9PgonzpsnBPCebpbWejJmb9DblFKz3d0ge7S5NusxvRDgOWAYMBHIBp5wZ2OUUsHAJ8AdWuvipve5+9zZaFuPOHda6zqt9UTMfgDTgFHuaIctLdumlBoL3I9p41QgEri3u9ullFoI5GqtN3f2tTwhgPfojSO01lnW21zgM8wvcU+So5SKBbDe5rq5PY201jnWP7J64CXceO6UUj6YAPmO1vpT6+Eece5sta0nnTtrewqB5cDJQLhSqqHSqdv/Xpu07SxrSkprrauA13DPeTsFOE8pdQCTEp4P/AcnzpsnBPAeu3GEUipIKRXS8DVwBpDS9rO63f+Aa61fXwt84ca2NNMQHK0uwE3nzpp/fAVI01o/2eQut587e23rCedOKRWjlAq3fh0AnI7J0S8HLrY+zF3nzVbbdjb5QFaYHHO3nzet9f1a6wSt9RBMPPtRa30lzpw3d4/EOjhaew5m9D0d+KO729OkXUMxs2K2ATvc3TbgPczldA0mh3YjJre2DNgD/ABE9qC2vQVsB5IxwTLWTW2bhUmPJANJ1n/n9IRz10bb3H7ugPHAVmsbUoA/W48PBTYAe4GPAL8e1LYfrectBXgb60wVd/0D5nJ8FkqHz5sspRdCCA/lCSkUIYQQNkgAF0IIDyUBXAghPJQEcCGE8FASwIUQwkNJABdCCA8lAVwIITzU/wdWb7/HlKv8QAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 训练误差\n",
    "for i, name in enumerate(name_l):\n",
    "    plt.plot(list(range(num_epochs)), train_l[i], label=name)\n",
    "plt.legend(loc = 1)\n",
    "plt.title('mse_train')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d549ea6230>]"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d549ea4d00>]"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x2d54f85e920>"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'mse_test')"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABHcklEQVR4nO3dd1ib1/XA8e8VSyyzjfEE7z3xyp62kzijadNm7zptM9ukbUbbJP2ladokbdLMpnGWMxtnNtPZiWPHNt4LGw8M2GaYvRHo/v64AjMECEkgBOfzPH6AV9L7XmR0dHXuufcqrTVCCCH8j8XXDRBCCOEeCeBCCOGnJIALIYSfkgAuhBB+SgK4EEL4KQngQgjhpySACyGEn5IALoQQfkoCuBBOKKWuVEqt9NK5MpVSp3njXEI0JwFcCCH8lARw0as5eq+/VUptUUpVKqWWKqUSlVIfK6XKlVKfK6VilFJWpdTLSqlCpVSJUmqdUirRcY4ox+MOK6UOKqXuU0oFdHDNCcDTwHylVIVSqsRxPEQp9ZBSKksplaeUelopFeq4LV4p9YHj2kVKqe+UUhal1DJgOPA/x7l+1+1Pmug3An3dACFc8GPgdMzf60ZgBnANsBP4CLgJyAWigGFALTAdqHY8/gUgHxgNhAMfANnAv51dTGu9Uyn1C+BarfVxzW56ABjlOLcNeBX4E3AHcCuQAyQ47jvPnEpfppQ63nGuz91/CoRoS3rgwh88prXO01ofBL4D1mitN2qta4B3MAHdBsQBo7XWDVrr9VrrMkcv/EzgFq11pdY6H/gncGFXGqCUUsAS4Nda6yKtdTlwf7Pz2IAkYITW2qa1/k7LSnGim0kPXPiDvGbfVzv5OQJYhul9v66UigZeBu4CRgBBwGETgwHTccnuYhsSgDBgfbPzKKAxFfMgcA+wwnH7M1rrB7p4DSG6RAK46BO01jbgXuBepVQyJrWyy/G1FojXWtd35ZStfj6CebOY5Pgk0Pr65Zg0yq1KqcnAl0qpdVrrL5ycSwivkBSK6BOUUicrpaY4BifLMCkNu9b6MLACeFgpNcAxsDhKKXViJ6fMA4YqpYIBtNZ24D/AP5VSAx3XHKKUWuj4frFSarQj1VIKNAD2Zuca6d3fWAgJ4KLvGAQsxwTvncA3mLQKwOVAMLADKHbcL6mT830JbAdylVJHHMd+D+wBflBKlQGfA+Mct41x/FwBrAae1Fp/5bjtr8AfHBUqt3nySwrRnJJxFiGE8E/SAxdCCD8lAVz0W46JOBVO/j3t67YJ4QpJoQghhJ/q0TLC+Ph4nZyc3JOXFEIIv7d+/fojWuuE1sd7NIAnJyeTlpbWk5cUQgi/p5Q64Oy45MCFEMJPSQAXQgg/JQFcCCH8lKyFIoSfsNls5OTkUFNT4+umiG5itVoZOnQoQUFBLt1fArgQfiInJ4fIyEiSk5NptiKi6CO01hQWFpKTk0NKSopLj5EUihB+oqamhri4OAnefZRSiri4uC59wpIALoQfkeDdt3X1/9cvAvgXO/N48us9vm6GEEL0Kn4RwFfuOcLjX+5Bpv0LIcRRfhHAB0eFUlXXQFl1VzZUEUJ4U0lJCU8++aTbjz/ppJN8MhM7MzOTyZMnu3yftWvXMn36dKZPn860adN45513eqKZbvGPAB4dCsCh0upO7imE6C6eBnB/MXnyZNLS0ti0aROffPIJ1113HfX1vbPz6BdlhEnRVgAOlVQzIWmAj1sjhO/d+7/t7DhU5tVzThw8gLvPntTu7bfffjt79+5l+vTpnHzyyWzZsoXi4mJsNhv33Xcf5557LpmZmZxxxhkcd9xxrFq1iiFDhvDee+8RGmo6YW+++Sa/+tWvKCkpYenSpRx//PFOr/XCCy/w7rvvUllZSUZGBrfddht1dXUsW7aMkJAQPvroI2JjY9m0aRO/+MUvqKqqYtSoUTz33HPExMSwfv16rr76agAWLFjQdN6GhgZuv/12vv76a2pra7n++uu57rrrWlw7LCys6fuamppePXDsHz3wqMYeuExgEMJXHnjgAUaNGsWmTZt48MEHeeedd9iwYQNfffUVt956a9MYVUZGBtdffz3bt28nOjqat956q+kc9fX1rF27lkceeYR77723w+tt27aNt99+m3Xr1nHXXXcRFhbGxo0bmT9/Pi+99BIAl19+OX/729/YsmULU6ZMaTrnVVddxWOPPcbmzZtbnHPp0qVERUWxbt061q1bx3/+8x/279/f5tpr1qxh0qRJTJkyhaeffprAwN7Z1+2drWolITKEQIvicImkUIQAOuwp9wStNXfeeSfffvstFouFgwcPkpeXB0BKSgrTp08HYNasWWRmZjY97vzzz3d63JmTTz6ZyMhIIiMjiYqK4uyzzwZgypQpbNmyhdLSUkpKSjjxRLM/9RVXXMEFF1xASUkJJSUlnHDCCQBcdtllfPzxxwCsWLGCLVu2sHz5cgBKS0vJyMhg7NixLa49d+5ctm/fzs6dO7niiis444wzsFqt7j9h3cQvAniARZE4wMohCeBC9AqvvPIKBQUFrF+/nqCgIJKTk5smoISEhDTdLyAggOrqo6/bxtsCAgI6zSs3P4/FYmn62WKxuJ2T1lrz2GOPsXDhwhbH23szmTBhAhEREWzbto3U1FS3rtmd/CKFAjA42iopFCF8KDIykvLycsD0XAcOHEhQUBBfffUVBw44Xa66W0VFRRETE8N3330HwLJlyzjxxBOJjo4mOjqalStXAubNptHChQt56qmnsNlsAOzevZvKysoW592/f3/TG8SBAwdIT0+nt25E4xc9cDCVKBuyin3dDCH6rbi4OI499lgmT57M7NmzSU9PZ8qUKaSmpjJ+/HiftOnFF19sGsQcOXIkzz//PADPP/88V199NUqpFoOY1157LZmZmcycOROtNQkJCbz77rstzrly5UoeeOABgoKCsFgsPPnkk8THx/fkr+WyHt0TMzU1VbtbB/rAx+ksXbmPXf93BhZL7x0VFqK77Ny5kwkTJvi6GaKbOft/Vkqt11q3yeH4VQrF1qA5UlHr66YIIUSv4D8plGalhAMH9L7RYCFE13366af8/ve/b3EsJSWlV89+7E06DeBKqeeAxUC+1npyq9tuBR4CErTWR7qniUbzyTzTh0V356WEED1k4cKFbSpChOtcSaG8ACxqfVApNQxYAGR5uU1ONfXApZRQCCEAFwK41vpboMjJTf8Efgf0yChodFgQoUEBHJZSQiGEANwcxFRKnQsc1FpvduG+S5RSaUqptIKCAncu13gekqJlMo8QQjTqcgBXSoUBdwJ/cuX+WutntNapWuvUhISErl6uhcFRoTKZRwghHNzpgY8CUoDNSqlMYCiwQSk1yJsNc2ZwtFXWQxHCR2Q9cOeOOeaYTttw7bXXsmPHDgDuv/9+F1veuS4HcK31Vq31QK11stY6GcgBZmqtc73WqnYkRYVSUFFLXb29uy8lhGhF1gN3btWqVZ2e89lnn2XixImAdwO4K2WErwEnAfFKqRzgbq31Uq+1oAsGR1vRGvLKahgWG9b5A4Toqz6+HXK3evecg6bAGQ+0e7OsB+5cREQEFRUVfP3119xzzz3Ex8ezbds2Zs2axcsvv4xSipNOOomHHnqI5cuXU11dzfTp05k0aVKLdVrc4UoVykVa6yStdZDWemjr4O3oiXdrDXijpp15JI0iRI+T9cA7t3HjRh555BF27NjBvn37+P7779s8h6GhoWzatMnj4A1+NBMTTAoFZGs1ITrqKfcEWQ/cuTlz5jB06FAApk+fTmZmJscdd1ynj3OXXwXwwU2zMaUSRQhfkvXAO2+zK7+jp/xmMSuAsOBAosOCOCw9cCF6nKwH7h1BQUFN1/eUX/XAwaRRpAcuRM+T9cC9Y8mSJUydOpWZM2d6nAf3m/XAG1374jpyiqv55JYTvNQqIfyDrAfeP/TJ9cAbJUWFynooQgiBP6ZQoq2UVtuorK0nPMTvmi+EaMZf1gMvLCzk1FNPbXP8iy++IC4uzgctMvwuAg5x1IIfLq1m9MBIH7dGiJ6lte50Yok/8Zf1wOPi4ti0aVO3X6erKW3/SKEcWAUblgHNasFlIFP0M1arlcLCwi6/yIV/0FpTWFjoUr15I//ogW9/B7a8ATMvIynq6M48QvQnQ4cOJScnB0+WZRa9m9VqbZoI5Ar/COBhcVBTCg02BkVZUQpZVlb0O0FBQaSkpPi6GaIX8Y8USphjkKCqiKAACwMjQ2RZWSFEv+dnAbwQcEzmkdmYQoh+zj8CeLhjFlSVWfRwSHQoh2UQUwjRz/lHAG/TA7dyqLRaRuOFEP2anwRwRw+80vTAk6JDqbHZKa7yzoIwQgjhj/wkgMear1VFAAyJllJCIYTwjwAeEATWqKYceONkHlkTRQjRn3UawJVSzyml8pVS25ode1Apla6U2qKUekcpFd2trQSTB2/MgUsPXAghXOqBvwAsanXsM2Cy1noqsBu4w8vtaissvikHHh8eQnCARUoJhRD9miubGn8LFLU6tkJr3bhX0A+A63M/3RUW15QDt1gUg6KsUkoohOjXvJEDvxr4uL0blVJLlFJpSqk0j9ZwCI9ryoGDo5RQUihCiH7MowCulLoLqAfa3RdIa/2M1jpVa52akJDg/sUac+CO2u8h0bKxgxCif3M7gCulrgQWA5fonphRExYPDXVQazZVTYq2kltWQ4NdJvMIIfontwK4UmoR8DvgHK11lXeb1A4n66E02DX55dILF0L0T66UEb4GrAbGKaVylFLXAI8DkcBnSqlNSqmnu7mdzdZDMQG8cWce2dhBCNFfdboeuNb6IieHl3ZDWzrWugferBZ81oiYHm+OEEL4mn/MxISjAbyy9WxMqUQRQvRP/hfAHT3wAdZAIkICJYUihOi3/CeAh0RCQHBTLbhSSmrBhRD9mv8EcKVarIcCZllZqQUXQvRX/hPAwbEeytEAPiTaKjlwIUS/5WcBPLZlDzwqlCMVddTYGnzYKCGE8A3/CuDh8W3WQwHIlTSKEKIf8q8A3ioH3jSZR9IoQoh+yM8CeDzUlEKD2QszSWZjCiH6MT8L4C33xmxMoRyWUkIhRD/kXwG8aT0Ukwe3BgUQFx7MIcmBCyH6If8K4K1mY4JZE0Um8wgh+iM/C+COHnhl80qUUKkFF0L0S34WwNv2wIdEh8remEKIfsnPAnjjIGbzyTxWymvrKaux+ahRQgjhG/4VwAOCwBrVZj0UQHrhQoh+x78CODjWQzmaAx/SuLGD5MGFEP2MHwbwuDbroQBSiSKE6Hdc2RPzOaVUvlJqW7NjsUqpz5RSGY6vPbenWXh8iwA+MDIEi5IUihCi/3GlB/4CsKjVsduBL7TWY4AvHD/3jFYrEgYGWBg0wCopFCFEv9NpANdafwsUtTp8LvCi4/sXgfO826wONObAtW46lBQdKikUIUS/424OPFFrfdjxfS6Q2N4dlVJLlFJpSqm0goICNy/XTFgc2G1QW950KCnKKjvzCCH6HY8HMbXWGtAd3P6M1jpVa52akJDg6eXarIcCjsk8pTVo3W4zhBCiz3E3gOcppZIAHF/zvdekTjTNxjya1UmKslJXb6ewsq7HmiGEEL7mbgB/H7jC8f0VwHveaY4LnK2HEi2lhEKI/seVMsLXgNXAOKVUjlLqGuAB4HSlVAZwmuPnnuFkOv3gKNnYQQjR/wR2dget9UXt3HSql9viGic58MGO2ZiyKqEQoj/xv5mYwREQENyiBx4bHkxIoEVSKEKIfsX/ArhSjlrwwmaHFElRVtmZRwjRr/hfAIc266EADI4Olb0xhRD9in8G8PC4FjlwMItaySCmEKI/8c8A7rQHbiW/vIb6BruPGiWEED3LTwN4yxw4mBSKXUNeea2PGiWEED3LTwN4HNSWQsPRbdSSohylhJIHF0L0E/4ZwMPbbm482DEb86AEcCFEP+GfAdzJ7vRNPXApJRRC9BN+GsDbrocSaQ0i0hooKRQhRL/hpwG8bQ8czJooB6WUUAjRT/hnAG9aD6VlAE+Ktsp6KEKIfsM/A3ioYw9lZ7MxJQcuhOgn/DOABwSBNbpFDhxgcJSVoso6qusafNMuIYToQf4ZwMHpbMwkx7rgkkYRQvQH/hvAw+PbrIfSWAsuaRQhRH/gvwE8LK7FvphwdGMHWRdcCNEf+HcAb5UDHxTVGMClBy6E6Ps8CuBKqV8rpbYrpbYppV5TSlm91bBONebAtW46FBIYQHxEiOTAhRD9gtsBXCk1BLgJSNVaTwYCgAu91bBOhceD3Qa1ZS0OD46WnXmEEP2DpymUQCBUKRUIhAGHPG+Si9qZjZkUZZUcuBCiX3A7gGutDwIPAVnAYaBUa72i9f2UUkuUUmlKqbSCggL3W9pa03oobUsJD5dUo5ulVoQQoi/yJIUSA5wLpACDgXCl1KWt76e1fkZrnaq1Tk1ISHC/pa210wMfEh1KZV0DZTX13ruWEEL0Qp6kUE4D9mutC7TWNuBt4BjvNMsFTWuCt9obM7pxWVlJowgh+jZPAngWME8pFaaUUsCpwE7vNMsF7ebAzWQeyYMLIfo6T3Lga4DlwAZgq+Ncz3ipXZ0LjoCAkDa14EOiGwO4VKIIIfq2QE8erLW+G7jbS23pGqWczsZMiAwh0KIkhSKE6PP8dyYmmDx4qxx4gEWROMAqPXAhRJ/n3wHcyYqE4JjMIzlwIUQf5+cBPL5NDhwcteAyG1MI0cf5eQBvmwOHo1ur2e0ymUcI0Xf5dwAPj4faUqiva3F4SHQotgbNkcpaHzVMCCG6n38H8LBY87W6ZS+8aWceGcgUQvRhfh7AG9dDaTUbM0pmYwoh+j4/D+DOZ2M2bq12UHrgQog+zL8DeLijB96qFjwmLAhrkIXDUkoohOjD/DuAN/XAW+bAlVIMllJCIUQf598BPNQxiOmsFjzaykHpgQsh+jD/DuABgWCNdj4bMypUBjGFEH2afwdwMHnwKmc98FDyy2uxNdh90CghhOh+/h/A21sPJcqK1pAreXAhRB/VBwJ4fJt9MeFoKaEMZAoh+qo+EMBj212REGQyjxCi7/L/AB4ebwJ4q13oj26tJj1wIUTf5P8BPCwO7DaoLWtxODwkkAHWQFkXXAjRZ3kUwJVS0Uqp5UqpdKXUTqXUfG81zGXtrIcCJg8uKRQhRF/laQ/8UeATrfV4YBo9uSt9o3ZmY4IJ4JJCEUL0VW4HcKVUFHACsBRAa12ntS7xUrtcF94YwJ3tzGPlkPTAhRAeqq5roLa+wdfNaMOTHngKUAA8r5TaqJR6VikV3vpOSqklSqk0pVRaQUGBB5drRzsrEoLpgZdU2aiu631PvBDCf1zx3Frufm+7r5vRhicBPBCYCTyltZ4BVAK3t76T1voZrXWq1jo1ISHBg8u1o8McuCkllF64EMJdWmu2HixlS06pr5vShicBPAfI0Vqvcfy8HBPQe1ZwOASEOO2By848QghPFVbWUW1rIKuoCq171z67bgdwrXUukK2UGuc4dCqwwyut6gqljtaCtzIkurEWXHrgQgj3ZBVVAVBRW09hZV0n9+5ZgR4+/kbgFaVUMLAPuMrzJrkhLBYq8tocThxgRSlJoQgh3JftCOAABwqriI8I8WFrWvIogGutNwGp3mmKBwbPgG1vg60GgqxNh4MDLcRHhEgKRQjhtpYBvJJZI2J82JqW/H8mJsCEc6CuAvZ/0+amwVJKKITwQHZRNdFhQShleuC9Sd8I4CknQHAk7Pxfm5vMZB4J4EII92QVVTEyPpzBUaFN+fDeom8E8MAQGLsAdn0M9pY130mOvTF72+ix8MyX6XnsLajwdTNEP5BdXMWw2DCGx4aRWVjp6+a00DcCOMD4xWY2ZtYPLQ4PjrZSVddAabXNRw0T3ma3a65/ZSP3fdDzRU+if7E12DlUUs2wmDCS48PIkhRKNxlzuqkHT/+gxeHB0bKsbF+TXVxFta2B7/cUUlFb7+vmiD7scEkNdg3DY8MYHhtOYWUd5TW9pzPYdwJ4SCSMPAl2ftBibfCkKNnYoa/ZnWdSJ3UNdr7ele/j1oi+rDHnPTQ2lOS4MKB3DWT2nQAOMGExlGZB7pamQ009cNlarc/YnVcOQFRoEJ9ub1v/L4S3ZBebYD08NozhjgDemwYy+1YAH3cmKIvphTskRIQQHGjhwJHeNfgg3JeRV87gKCtnTB7EV+n5vXKVONE3ZBdVEWhRJEWFMiLOrNXXmwYy+1YAD4+H4fNb5MEtFsX0YdGsy2y7XrjwT7vzKhiTGMmCSYlU1Nazem/bZRSE8IasoiqGxIQSYFFEhAQSHxHcqwYy+1YAB1ONkr8DCvc2HZo3Mo6tB0sp60WDD8I9DXbN3oIKxiZGcMyoeMKDA1ixQ9IoontkF5sKlEa9rZSwDwbws8zXZr3weSNjsWtIk16438sqqqK23s6YxEisQQGcNG4gn+3Iw26XOn/hfdlFpga8UXJcuPTAu1XMCBg0tUUefObwGIIDLPywTwK4v2scwBybGAnAgkmJFJTXsjG72JfNEn1QRW09RZV1DIsNbTo2PC6Mw2U11Nh6x7hL3wvgABPOhpy1UJ4LgDUogBnDoyVX2gdkOAL4mIERAJw8fiBBAYoVUo0ivKxxEavhzXrgI+LC0BpyintHL7xvBvDxi83X9A+bDs0bGcf2Q6UyI9PP7c6rYEh0KOEhZiHNAdYg5o+K59PtubJcgvCqxgDePAfeWInSW2rB+2YAHzgBYke1yoPHSR68D9idV864QZEtji2YmEhmYRUZ+bI2ivCeLGc9cMf3mRLAu5FSZlLP/m+hugSAGcOjCQ60SBrFj9U32NlXUMmYxIgWxxdMTEQp+HRbro9aJvqinOJqIkICiQ4LajoWGx5MZEggWb2kEqVvBnCA8WeDvR4yVgAmDz5zeDQ/7JcA7q8yC6uoa7AzdmDLHvjAAVZmDIuWckLhVVmOChSlVNMxpRTD48I40EtmY/bdAD5kFkQMarFGuMmDl0ke3E9ltKpAaW7BpEFsPVjKQVn7XXhJdlEVw2JC2xwfERfWd3LgSqkApdRGpdQHnd+7B1kspiZ8z+dgMy/qeSPj0BrW7Zc8uD/anVeBUjB6YESb2xZOGgTAiu2SRhGe01qTXVzVIv/daERcODnFVTT0grkH3uiB3wzs9MJ5vG/CYrBVwd6vAJg+LJqQQAur90kaxR/tzi9nWEwYocEBbW5LiQ9nzMAIKScUXlFQUUuNzd5iEk+jEbFh2Bp0r9jpy6MArpQaCpwFPOud5nhZ8vFgjWqqRjF58Bh+kADulzLyyhmb2Lb33WjhpEGszSyiuLKuB1sl+qKmEsJYZymU3lNK6GkP/BHgd4C9vTsopZYopdKUUmkFBQUeXq6LAoJg7CLY9RE0mIX/542MY8fhMkqrJA/uT2wNdvYfqWSMk/x3owWTEmmwa75I7941wvfkV7A5u6RbryF8K7vI9K6dp1Ac64IX+b4Sxe0ArpRaDORrrdd3dD+t9TNa61StdWpCQoK7l3Pf+MVQXQwHvgdg/iiTB18j1Sh+JfNIJbYG3WEPfMqQKAZHWfm0m/Pgt7+1hUueXUN+mawx31c1beQQ0zaADxpgNUtU+3kP/FjgHKVUJvA6cIpS6mWvtMqbRp8KQWGw6RUApg2LIiRQ1kXpbrYGO1/tyvfa7MjGXXicVaA0UkqxYNIgvt1dQFVd92y1Vllbz6bsEipq63ng4/RuuYbwveyiKgZGhmANajveYrEohseGcaAX1IK7HcC11ndorYdqrZOBC4EvtdaXeq1l3hIcDqlXw9Y34cgeQgIDmDVC8uDd7aOth7nq+XVeGzDelVeORcGohPZ74GAm9dTW2/l29xGvXLe1dZlF1Ns1s5NjeHvjQVlnvo/KarUKYWsjYntHKWHfrQNv7tibzYbH3/4dMHnwnblllFTJYFd32XGoDIDPvDS5JiOvnBFx4U57RM3NSYklKjSo28oJV+8rJChA8fSlsxgcZeWP726jvqHdISDhp3KKq53mvxuNiAsnq6jK5+vveCWAa62/1lov9sa5ukXEQJhzraMXntEsDy69p+6Snmsm3Xy+M88rf+S788qbViDsSGCAhVMnDOSL9Hxs3RBYV+8tZMawGOIiQvjj4omk55bz8g8HvH4d4Tt19XYOl1Y7ncTTaERcGFV1DRRU1PZgy9rqHz1wgGNuhkArfPN3pg6NwhpkkTRKN9qVW05oUADZRdVN+Wt31dY3kFlY1WH+u7mFkwZRWm1jrZffoEurbWw7WMq8UXEALJo8iOPHxPPwZ7spKPftC1l4z6GSauyaDlMow3vJDvX9J4BHJMDsa2HbckKK9zJrRIwsbNVNSqts5JbVcNGc4YDphXti/5FKGuy6zSJW7TlhTAJhwQF8uPWwR9dtbe3+IuwajnEEcKUU95wziRpbA3/7RAY0+4rGneg7CuDJvaQWvP8EcDC58EArfPt35qXEkZ5b3umkj4y8ch5esatbPo73Vem5Jv99/Nh4pg2L9jgP7koFSnOhwQGcOiGRT7blejU/vXpvISGBFmYMj246NiohgquPS2H5+hzWH5BdgfoCZ8vItjYkOhSLwuerEvavAB4eD3N+DluXc3KcebF1lAfPLqrikmfX8NiXe2T9lC7Y5Vh0avygSE6fMJBN2SXkl7tfM52RV06ARTEyIdzlxyyemkRRZR2rvPgpa9XeI8waEUNIYMuB1JtOGcOgAVb+9N62XrE+hvBMdlE1QQGKxAHWdu8THGhhSEyoz9cF718BHOCYmyAojAkZT3eYBy8or+WypWuorbdjUbBWysVctiu3nEhrIIMGWDltYiIAX+50f3bk7rxykuPC2gTOjpw4NoHIkEA+2HLI7es2V1RZR3pueVP6pLnwkEDuOmsC2w+V8eraLK9cT/hOdlEVQ2PCCLCoDu83Ijbc58vK9r8A7uiFB2x/m3MGlzsN4OU1Nq58fi25ZTU8d+VsJiQN8PqAWF+2K7ec8YMiUUoxLjGSoTGhHuXBM/IqXE6fNLIGBXD6RJNGqav3PI3S+Hcy30kAB9PjP2ZUHA99uosiWYvFr2UXVzG0gwqURsPjfD+Zp/8FcGjqhV9rX056bnmLF1yNrYElL61nV245T106i1kjYpidHMuGrGKvBIK+TmvNrmbbnimlOG1CIt9lHHFrdmSNrYHMwo7XQGnP4mlJlNXUs3KP52vwrN5bSFhwAFOHRju9XSnFvedMorK2nr/LgKZfyypyvoxsa8lxYZRU2Xy6v0D/DODhcTB3CWMKVjBG5bDG0btqsGtueX0Tq/cV8tAF0zh53EAA5qbEUmOzs/VgqS9b7RcOldZQXlPPuEEDmo6d7pgduTKj67Mj9xZUYNd0uAZKe44bnUBUaBAfbPa8GmXV3iPMTo4lKKD9l8yYxEiuPCaZN9Ky2SSLXfmlshobJVW2DitQGg2PNWMyWT7Mg/fPAA6mFx4czq+D3uGHfYVorfnDu1v5ZHsuf1o8kfNmDGm66+yUWACZNu2CXY4KlPHNNh6ekxJLpDXQrTRKRhcrUJoLDrSwcFIiK3bkUWNr6PLjG+WX1bC3oNJp/ru1m08bw8DIEG58bQNHfDzJQ3RdtgsVKI2S4xs3OPZdGqX/BvCwWNTc61hk+YHDGRt5eMVuXlubzfUnj+Lq41Ja3DU+IoSRCeH9Kw9eUwZPHgPf/6tLD2ucgdl838qgAAsnjxvIFzvzu1ylsTuvnECLaqq77arFUwdTUVvPN7vdT6Os7iT/3VykNYh/X5ZKQXkt17yYRnWd+28couc1LiM7zMkqhK01BvksHw5k9t8ADjD/BuoDwjindBmPf7WHi+YM47YF45zedW5KLOsyi/pPmdgnt0P+dtjwInRhKvzu3HKSoqxENdvJG+C0iYkUVtZ1ObWwO6+ClPhwggPd+1OdPyqOmLAgPtjifhpl9d5CIq2BTBoc5dL9pw+L5tELZ7Alp4Rb3tjYf/5m+oCu9MDDggMZGBlC5hHpgftGWCzFk69iccAarh5dzX3nTWmxA3Vzc1JiKa+pZ5ejh+myFX+EpQvgm7/DoY1g94OB0B3vm+V348dB4R4o2OXyQ9Nzjw5gNnfi2AQCLarLaZSM/HK30ieNggIsLJqcxBc789zuDa/aW8jclLhOy8qaWzhpEH88ayKfbs/jLx/2zh0HRVvZxVVEWgPbdEDaM8LHO9T37wAOJC68FXtgKHfFft7hC3ROivn4vLYrG0FUFcEPT0HRPvjqfnjmJHh4HLz7K9j+DlSXeNb47lCeB/+7GZKmwyVvmmPp/3PpobYGO3sLKpwG8KjQIOaOjOXzLszKrK5rIKuoyuUp9O05e2oSVXUNfOnGTj05xVVkFVW5lP9u7erjUrjq2GSe+34/z3+/v8uPFz0vq6jKpfRJo+Gx4TKI6VNhsVhmXk7AtuVQ1v6kjyHRoQyJDu3ahJ7t74DdBpe+BbdlwI/+DSnHQ/qH8OaV8PeR8PyZcLDDTY16jtbw/g1mI+jz/wMxI2DILNj5gUsP3+/YNWe8kwAOcNqERDLyK1z+yLm3oAKtYZwHPXCAuSPjiI8IcWtST+N6Oa7kv535w1kTWTAxkT9/sKPblrgV3pPtYglho+S4MHLLajwaJPeEBHCAeb8E3QBr/t3h3eakxLJ2f7Hry6NueQMSJsCgqWYxrWkXwk+eg9/uhatXwHG/NimK927sHamV9c9Dxgo4/f8gYaw5Nn4xHN4EJdmdPrxxAHNc4gCnt582wczKdDWNstsxJd+dGvDmAiyKM6cM4sv0fCpqu1aLvnpfIbHhwW6/iQRYFI9eOIOpQ6K46fWNfXovzSMVtV5b/90X7HZNdnG1042M29O4KqGvBjIlgAPEpsCEc0wAq20/xz0nJZYjFbXsd6UHWbQPstfAtJ9B67x6QCAMnwun/hEW/MUMFrqYpug2hXvh07tg1Clm1cZGE842X9M/7PQUu3LLCLAoRg10XjEyLDaM8YMizYu8NAd2fdLh+XbnVRAcYCE5zvUeUXsWTx1Mbb2dL7qQg9das3pvIfNGxmLpQv67tdDgAJ69YjbxESFc8+K6poGyvqSu3s41L6zj5y+l8dWu7t1U2h1HKmpZtafjeQgFFbXU1du72AM3f+u+GsiUAN7omBuhphQ2tr+t5xxHPbhL5YSb3wAUTPlpx/ebfD7EjYGv/+a7XnhDPby9BAKC4dwnwNLszyJ+DCSMh/TO0yi7ck3FSEdrlpw+MZFNBwqof+VCeO1CKG5/M4SMvHJGJoQT2MHkGVeljohh0ABrl6pRDhRWcbi0hvmj4j2+fkJkCC9cNZu6ejtXPr+W0irfzd7rDg98nM7mnFJiwoL4y4c7e83qnXa75tU1WZzy0Ndc/OyaDtNoTRsZdyGAj4gLQ2FnyOo/QU6ax+3tKgngjYamwvD5sPpJE9CcGBkfTnxEcOcBXGuTPkk5AaKGdHxfSwCc+Hvf9sJX/gMOpsHif8KAwW1vH78YDnwPlR0P4O7KK3M6gNncaRMSuU69R2D+VkA3bTbt/HzlHqdPGlksijOnJPHNrgLKalwLnk313yPdy3+3NnpgJM9cnkp2UTVXvbDWraUFtNY+nbrtzCfbcnnu+/1cdWwyf/vxVPbkV/DqGt8v6rXzcBk/eXoVd76zlYmDBzB1aBR3vL2VgyXVTu/flRLCRtFhwSyy7mBSzhvw3T+80u6ucDuAK6WGKaW+UkrtUEptV0rd7M2G+cQxN0JpFux8z+nNSilmJ8d2PpCZvRaK95uctyt82Qs/uB6+fsB8Uph8vvP7TFgM2g67P273NBW19WQXVTO+k4A7xZLJTUHvsC7iFJOu2fgK2NsOAFXW1pNTXM1YF7ZRc9XiaUnUNdj5bLtraZRVewsZGBnCqC4sY9uZeSPjePTC6WzKLuG6ZeuprXd98Kuqrp6rX1jH3Ps/7zW59OyiKn67fDPThkZxxxkTOH1iIseOjuOfn+/22Z6zVXX13P/RThY/tpLMwioevmAar/18Ho9dNAO7XfPr1zc5rc1v7IEPiXY9Bw5wZfAX5ps9n/V4ZZknPfB64Fat9URgHnC9Umqid5rlI2PPgNhRsOqxdievzEmJJae4ut13cQC2vA6BoUfzx53xVS+8rgrevg4iB8GZD7Z/v6TpEDWsw2qUxgHHDnvg9bVY3vsVVYHR3Fx+KbZpl0BZDuz7qs1d9+SbKfTe6oEDzBgWzZDoUJeqURrz3/NHxbU7N8BdZ0xJ4oHzp/JdxhF+88Zmlyb6lFTVcemza/hmdwHhwYEsWZZGfpn7a6x7Q129nRte3QDA4xfPJDjQglKKP5w1kbJqG49+kdHjbfpsRx6n/+Nbnvl2Hz9NHcqXt57Ij2cNRSnFiLhw/nzuZNZmFvHU13vaPDa7qJpBA6ydbpzdQmkOs+vWsc4yDRrqXEo1epPbAVxrfVhrvcHxfTmwE+gkX9DLWSxwzA1mws2BVU7vMjvZsS5Ke2mU+lrY9rYJ3iFdCD6+6IV/eR8UZsB5T0FodPv3UwrGnwV7v4Ra5/tbNk5w6jCAf/M3yN/O/vn3c6jWyg9B8yA0FjYsa3PXxjcEdxaxao9SisVTk/gu40invcM9+RUcqah1q/7bFT+dPYw/nDWBD7ce5s63t3ZY2ZRbWsNP/72abQfLeOLimbx87VzKa+pZsmy918vXCsprXZ45+tePd7I5p5QHfzKtxeJPE5IGcOGc4SxbfaDpjbi7Hamo5ecvpfHzl9KICAlk+S/m89fzpxIdFtzifufPHMLZ0wbzz88z2JjVcgel7OKqLlWgALD+RRSaW2uuQcckw9blHv4mXeOVHLhSKhmYAaxxctsSpVSaUiqtoMDzZT273bSLICzO9MKdmJA0gMiQwPbTKLs/hZoSU33SFZYAOPF3PdcLLzsM6/4DMy+HkSd2fv/xi6GhFvZ87vTmXbnlhAUHtD8JImc9rPwnTL+EcSdcQGhQAJ/tLoGpPzMVLpWFNNg15TU28str2JRdQnCghRFuroHSnsVTB1Nv13zaSU320fy35wOY7bn2+JHccPJo3kjL5q8fpzsN4vsKKvjxU6s4WFzNC1fN5owpSUxIGsA/fjqNTdkl3PXONtfLWjtQV2/nwU/TmXv/5yx65Fs+2Zbb4Xk/2XaY57/P5OpjU1g0eVCb239z+lhCgwL4y4c7PG5bZzZll3D2Yyv5dncBd5wxng9uOo5UR0erNaUU9503mUEDrNz8+qYWZaXZRVUurULYpMEGG17i8MDjybLHUzbqHNj/DVT0XJzzOIArpSKAt4BbtNZlrW/XWj+jtU7VWqcmJCR4ernuFxQKc5aYfG/B7jY3B1gUqckx7Q9kbnkDIhIh5aSuX3vyjyFutJl239298NWPm9zzcb9x7f7D55vecjsfEXflmgFHp+V2tmp49xcQmQSL/oo1KIATxsbz+rpsLt0wFuw2/vLAPYy68yOm3LOCOX/5glfWZDEuMbJL09ddMXnIAEbEhXVajbJqTyFDokO73iProlsXjOXy+SN45tt9PPn13ha3bc0p5YKnV1Nta+C1JfM4ZvTRN5NFk5O45bQxvLUhh6UrPZvluSu3nPOe+J4nvtrLGVOSaNCaX7y8nnOf+J7vMgraBPKswip+u3wL04ZGcfsZ452eMz4ihBtPHc1Xuwr4uhvLCt9Yl8VPn15NgEXx1i+P4boTR3W45C+YWcGPXDidnOIq7n5vOwC19Q3kltV0aRYmuz6CilzKJ18GQMbAhWasaMe77v46XRboyYOVUkGY4P2K1vpt7zSpF5h9rektrn4czmm7Gt+clDi+2pXOkYpa4iNCjt5QVWR64HOvM7XeXdWYC3/75yZQTjzHg1+iA1VFkPY8TPmJqYF3RUAgjDsTdr4P9XUQePSjaeMmDqc7Juq08eV9cGQ3XPo2WM2CUL88aTQhgQGEBA4mZ/8Efq6/J2LmjYSFBBIaHEBoUECLzYO9pTGN8vQ3+3jw03RSR8Qyc0QMUaFH176w2zU/7C/ktAmJXs9/O2vPPWdPorTaxoOf7iIqNIhL541g1d4jLHlpPVGhQSy7Zg4jE9qmkm46ZQy7csu5/6OdjEmM5MSxXesgNdg1S1fu46FPdzMgNJD/XJ7K6RMTqW+w8/bGgzz6eQaXLV3L/JFx/HbROGYOj6G2voEbXtuA4mjeuz1XHpPCq2uyuO/DnRw3Ot4r5aCNausbuOf9Hby2Novjx8TzrwtnEBMe3PkDHWYnx3LDyaP515d7OGlcApMGD0DrrlWgsG4pRA0jeupZ8NHX/HktPBOSgu27ZXxUfQox4cHEhgWbr+HBJEV1Mb/uArcDuDJ/2UuBnVrrnq+f6U7h8SaVsulVOOUPEDGwxc1zUmIASMssYtHkpKM3bHvLTJ13tfrEmck/Nrnib/5m0haWbqj0XPNvsFWamaBdMWExbHoZ9n8LY05rOlxQUUtRZZ3z/HfWD7D6CZh1FYw+tenw9GHR/OuiGeaHtCXwwa+5eUI5DJ3lzm/UJZfOG8Ha/UX8+5t9PGHfi1Jm+dvU5BhSk2MYYA2ipMrWbfnv1iwWxUMXTKOipp4/vretqQwvOT6Ml66ey6Ao55vrNj5u/5FKbnh1A+9df6zTQO9MdlEVt/53M2szi1g4KZH7fzSFOEdnJDDAwk9Th3Hu9MG8uiaLJ77aw/lPruK0CYlEWgPZklPKvy+b1Wm6ITjQwp1nTmDJsvW8ujaLy+cnd+l5ac/h0mp++fIGNmWX8KuTRnHrgnFufVK76dQxfLfnCHe+s5U7zpgA4HoK5cgeky455Q8MjArjJ7OGsv1QGe/Vz+O62td48eOVHKJl+u25K1M5ZXw7nRw3KXfzZ0qp44DvgK1A4+f9O7XWH7X3mNTUVJ2W1vPF7m45kgGPzzZ56ZPvbHFTXb2dKfd8yiVzR/Cns5sV3jx7mqns+JXzAVCXbfmv6YX/dJn3e+G15fDPyTDiWLjo1a491lZj1m+ZegGc/WjT4e8yCrhs6VpevXZui4/51FXC08eBvR5+uar9Qd2aUnhonBk3aHbe7lZd18Cm7BLSMotIO1DMhgPFlDfLia6+4xSSolqlUOx2+OAWUyZ65sNHlxzwghpbA1c8t5Y1+4uYOTya566c3WYQzpnsoirOfeJ7osOCePf6YxlgbX8lPa01b6zL5v8+2IFFKe45ZxLnzxzS4SeNytp6XliVydPf7KW8pp6rj01p+XffAa01lzy7hh2Hy/jmtpNdXuWvPT/sK+SGVzdQXdfAwz+d1rID5YaswirO/Nd31DXYqau388Mdp7b7htnCp3fBmqfh1zsgsllQLtwLj82k9pR7yZ+8hOKqOooq6yiuquPYUfEM7GCn+44opdZrrVNbH3e7B661Xgl07+dLX4ofY1IGa/8Dx94CwUffmYMDLcwcHsPazGYTWwr3Qs46OP3Pnl+7O3vhac+bQdbjXcx9NxdkNT3v9I/grH+YlA9HK1DGtu6Bf36vWVLgiv91XJFjjYJJ58HWt2Dh/RDs3YHL9oQGBzB/VFzTQlUNds3uvHLSMotAqbbBG+CzP5o10gNDzZvTKX+A+dc3PReesAYF8OwVqXy45TDnTB9MWLBrL89hsWE8dclMLnl2DTe9tpGlV8wmwKKoq7eTVVTF/iOV7D9Swb6CSrYfKmPrwVKOGRXHgxdMc6nmOTwkkOtPHs0lc4ezcs8RFkxsO2jZHqUUf1w8kbP+9R2PfLGbu8+e5PJjG2mtyS6q5n9bDvGPz3YzIi6M15fMY/RAz0tMh8eF8edzJ/Gb/24mONDCwMiQzh9kqzYztscvbhm8AeJGweAZhOx8m2En3NK1QVE3eJQD7/OOuRF2fQibX225Pghmm7XHv8ygvMZGpDXIDF6iYMoFnl/XEgAn/A7eWeLdXLitxqQzUk40M0/dMeEc2PGeebMaPg8wi1jFRwS3HA/Y/Dqs/TfMuc7MSO3MzMth82vm3NMvdq9tHgqwKCYkDWBCkvPFuPjhKTMuMmcJHH8rfPAbE9B3vg/nPumV3nikNYgL5wzv8uPmjozj3nMncdc72zjvie8pr7GRXVzdoiQwLjyYlPhw/nzuJC6dO6LL67tEhwWzeKqTmbqdmJA0gJ/NNmWFl8wdwehOJmfZGuzsOFRG2oFi1h8oYl1mMQXlZnu6BRMTefin08xrzkt+NGMIa/YVcbCk2rXnZPu7phM0+xrnt0/+Cay4y6RZ4kd7rZ3OSADvyPB5ZjnVVY/DpPMh7Ghp0tyUWP6lYf2BYk4am2AC1siTnE9Fd8fkH8O3f/duL3zzq1CRC+d3vOpih8acDpYg2Pm/pgC+u9ku9Njt8NV98N3DMOI4OO1u1847fL6pwNmwrPMAXl8LX/zZvAlN+pH7v0tXbH8XPrnD/F8sesC8yV74ihn3+Og20xs/+U7zpu+F3ni7crfC7k/g2F+3GSi/ZO4ICsrNioCThkRx9rTBpMSHMzIhgpS4cI/TF564dcFYPth8iEufXcPQmFCCAy2EBFoIDrQQHBhAcID5PvNIJZuyS6h21LcPjQnl2FFxpCbHkpocw7jESK8PLCuleODHU1x/QNpSM2cj+Xjnt0/6Eaz4g/nbOOn33mlkOySAd0QpUxXy6s/g0Wnmo/K8X4F1ADOGRxNoUazdX8RJ1r1QcqBNrtwjAYFHe+HLzoUZl5tBxCA3y9oa6mHlI+YNKcWFuu/2WKNM3Xj6B7DgPhq0CeAXzxlhct7vXGeC+8zLTY440MXKAKVgxqXw+T0d91zqquC/l5l6dGWBgBAYf6b7v48rDqwyi30Nmws/fvZogFbKVPIkHw8f/gY+v/tob3yg8/I6j5Qdhpd/Yt6Eyw7DWQ+3WenyltPGcstp3svLe0t8RAj//Nl0XllzgLoGO7U2OxW19dTa7I6fG6hrsJM4wMrPZg8zA8ojYl3LR3uBy28Kh7eYT58L/9p2ldFGUUNgxDGwbbkZQ+vGSiYJ4J0ZuxB++b3ZUefrv5qBi2NvJmzOEqYMjTL14LY3ICjM9M68acpPoDQb1r8Ib18LIQPMjM3pl5reZ1f+MLa/bd5kFnXwh+eq8YvNQF7edrICU6ix2ZkRVQnPLYK8bSaPPe9XXb/OtIvhi/+Djcvg9Hvb3l5bDq9eaBbWOuPvJm21/CpTnph8rGe/U3vy082qiTEj4KLXnL+BRibCz142z/GHt8G/j3f0xm/yXm/cVgNvXAJ1FTD1QkcvcJTpVPiJ0yYmctpE71Zh9Li05yDQCtMv6vh+k8+HD2+FvO0waHK3NUdWI3RF4iTzcXnJ1zB0tuklPjqN660r2J9zCL39HcfUee9N+wYcufDb4ObNZiBw3Jlmmdqlp5kKme/+0eEuQk3sdnPfhAlmvRdPjT8LUJD+Abtyy5iq9rLoh4uhaD9c9LoJKu68SUQmmjfMza+1XRGyuhheOheyVpte8Nzr4OI3zRotr11kUgveVnYIXv6xecFesrxFCq0NpUza6/q1MHaR+Rt5bpH5NOEprc02dwfXm12dznvKjEV8epdL67T7TEM9lGRB5vcmxfjtg7Dptd6xeUlrdZXmU01HaspMhdjkH0NoTMf3nXgeqADTC+9GbpcRusOvygg7krXG5Hn3f0u5DiVSVcNl75jV9bpbTZkZ6Nv0iglmlkCzo9CJv2+/0iP9Q3j9YrNN2tRO1id31dKFUFfJh9EXcWr63QRHJ2G5+A1I9HA9s/SP4PWL4MLXjqZGKgpg2Y/gyC644AXHG4hDSbbZNFo3wNWfuj4xqTM1pWa7u+JMuOpjSJrq+mO1NvnPD281+frT7jEDn+6OY6x63AyKnXTn0ZxqXRW8uBjyd8JVH8HgGZ2fx95gPt0U7DYzBhv/oZt9r8ygeVf/lrU2aa0t/zVBuzQHyg85ztlKyolw3pMQNbRr1+guZYdN56AwAyaeaz45DZnZ9n7rnjX/p9d+6dp8hWXnm3PevMXjT73tlRFKAPdAxc4v2fna7SRHahJu/QEsAVTU1rM5u4SNWcVszCphU3YJAwdY+d3CcZw0LsG7AzCFe81a3htfhsjBsPA+M9ja/Bpaw7OnQuURuHGDezNEnVn1mBmoAbZYJjD11g/MBChPNdTDPyeaXP1Fr0HpQfPiKs0xdevOAkt+Ojy30PSKrlnRZuJVl9XXwSs/MamaS950/4257LDpOWd8avLk5z5hUjFdsecL05bxi+GCF1u+CVTkw39ONWvUXPsFRA9r/zy52+D9G+HQBggKN5/ulDLjCM3/2aqhtswMyJ92Lwye3nkbc9ab/H/mdxCeAPHjTFuihppPSNHDzNcBQ2Drf80nBxUAZ/7drIXTzbNdO1SSBS+eA5UFpnOzdbn5/VNOgGNvhlGnmvZpDU8da5636751rc2bXoV3fwnXfA7DZnvUTAng3WTRI99i15pZI2LYmFXCrrzyppVoRw+MYNrQaNYfKCKzsIr5I+O448zxTB0a7d1GZK8zg2i5W8wf3pkPQcI4c9u+b+Clc0zddntlT+4oyYIn5vGJnsv7w37Lk1cc471zf3a3eYO46mOT+68qhkv+awaG2pO91gT6uFFw5YdNU/ZdZm8wg5U73oUd70NlPpz3dOe5zs5obd5gP7kD0GZ8YOblrgWAwr3wn5NN8Lv6U+cpuvyd5hNI1DC4+hOwtiqBtNWY1MX3j4A12gTN1m/yzdXXmjzvN3+H6iJTEnfKH5x/sinca6qBdrwLYfFw0u0w84rOB66L9pvAlrXapB4XP+KdN/+uKtxrgnddOVzylgmyNaVmzOmHJ6H8MCRONj3yAYPNJ56zH4VZV7p2/ppSeHAMpF4NZzzgUVMlgHeT+z/ayTPf7mOANZDpw2OYOTyaGcNjmD40uqlsq67ezmtrs3j0iwyKKus4Z9pgfrtwnHeL/O0N5oX35f+ZfN78600Vy+sXQ0G6+RgX5N0R/ZrqSib++WtuOHk0v1kwznsnPrIHHp9leoTWKDNI6ewjbWsZn8NrPzMliZcs7/z3tTeYqf7b3zFpqcp8M0Fn7EJTETPmdO/8PmDe8N673ixDMPp0OOG3ZjylvbRKTZmZ2VtZAEu+gpjk9s+990tTnTLqZLjojaOfsg6sNr3uwgyzNMTC+zvO47e4fil8/y8zb8Beb978T/itCbQV+WYTkA0vmiqgY240yzB3Zflke4Opqf/yPvN/fPa/ur+aqLm8HeYNXzfAZe+2TZHV18HWN2HVv8zrRwWYQoVb07s21vX6JaZq5Tc7PRrQlgDeTWrrG8gtNauYdTYJoLzGxr+/2cezK/fRYNdcNi+ZG08Z3aVFeDpVUWAG0Da9DOEDTVA6/f/g2Ju8dw2HbQdLWfzYSp64eCZnTfVsSnMbL51rXmSXv2sGkV215U3Tax+/2AQWW5XJF9uqzfovdVXmWNkhMzZQkWuC9pjTTf3u2IXdNxPUbjfVI5/9ybQhPMEMeI4/y6QsGitc7Hbzxpuxwvz+rkyEWv+CSdekXmNy7l/ca3K20cNND7fZOjRdUnbYzEXY8JIJYBMWm08oDbWmJ3rC79rORuyKvO1mU5G8reZN89S7PU+BdebQRpOfDgiGK94/+mnVGbvd7LSz5mkYeXLXX0fb3jaVUlf8z7X/x3ZIAO9FcktreOTz3fw3LZvwkEB+ljqMs6YmMX1YtPdy5Flr4KNbTUC/Ma1rvSMXLV+fw21vbubz35zglWnNLTRuHOFOZc8PT8MnnUygCAw1ywJM+hGMWej9CqKO1JRCxmdmOdKMz0zONSjM5NrHnWHSIqsfhzMehLlLXD/vZ3+C7x81qZKaUjO4ffJd3vndCnabN4X0D0yFxal/Mukqb6ivg28eMCuAajtEDDKld4mTYdAU8zVutHfGb7LWmDEFazRc8R7EjvT8nB2pq4IHR5uSYCcrm7pKAngvtDuvnH+s2M0X6XnYGjRDokM5a2oSZ01JYurQKM+Dud1utnnycuqk0V8+3MGLqw+w496FXl0q1Cty1pvpzkFhZh2boMZ/oaaHHRDs28GzRvV1cGClqb7Z9RGUHTTHZ15u0gpdaaPdbnLLBelmko+7yyV0pKHeewPhreVug31fm7kEudvM72F3bOAcaIWE8WYyVcrxZjE2V9NBjfZ9Y0pOIweZnndPVcG8da2p0Ll1t+sT21qRAN6LlVbb+GxHHh9uOcR3GUeot2uGxphgvnDSIIIDLJRW2yiuqqOkykaJ42txlY1qWz2x4cEMjLSSEBnCwMgQBkZaGTgghLjwYAIsisq6BgrKazlSUdvma32DJsIaSGRIIJHWICKsgUSEBBJpNf+GxoQxMDLE6ZvJZUvXUFhRx0c3tzOlWHSN1nB4sxmMnvozCHRhYaW+rL7OrCOft83U+eduMQP29dWAMr305BNMQB8+/+i2gLYaMwGu+ICZvFZywIxBpH9kPjVc9q5naZ+u2vWJGZu5+L8mRecGCeB+orTKxqc7cvlwy2G+32OCuTNhwQHEhAVjDbI4lqu0tbmPRZmVE2tsbWtxLQpiw0MIClBU1NRTUVff3j7ORIcFMS4xkvGDIhk3aADjBkUyblAkpz78NceOiucfP5vuya8shOvq68yEpszvzIBw9lqTj1cWiB9rdoWvaLVdXkCwqdJJmmqqsbrac/dGm7e8bipuOpsA1A4J4H6opKqO7/cUEhigiA4NIiY8mOjQIKLCgggJbDmiXVvfwJGKOvLLasgvryW/vJaCshqqbQ3ER4QQHxFCQuTRr7GO3nkju11TZWugvMZGRU095bX1lFXbyDxSya68ctJzy9mdW05lXctNdO84YzzXneilXKgQXWWrgYNpsP87OLzJlDPGjDCDt9EjzPcRg7pnY5QeJAFceMxu1xwsqSY9t5xduWXkFFdzwymjGdqVfQSFEF3m9Q0dRP9jsSiGxYYxLDaM0/19USIh+gD//lwhhBD9mEcBXCm1SCm1Sym1Ryl1u7caJYQQonNuB3ClVADwBHAGMBG4SCnl4VJ0QgghXOVJD3wOsEdrvU9rXQe8DpzrnWYJIYTojCcBfAiQ3eznHMexFpRSS5RSaUqptIKCAg8uJ4QQorluH8TUWj+jtU7VWqcmJCR09+WEEKLf8CSAHwSaryA/1HFMCCFED/AkgK8DxiilUpRSwcCFwPveaZYQQojOeDQTUyl1JvAIEAA8p7X+Syf3LwAOuHm5eOCIm4/tbtI290jb3CNtc48/t22E1rpNDrpHp9J7QimV5mwqaW8gbXOPtM090jb39MW2yUxMIYTwUxLAhRDCT/lTAH/G1w3ogLTNPdI290jb3NPn2uY3OXAhhBAt+VMPXAghRDMSwIUQwk/5RQDvzcvWKqUylVJblVKblFI+3W5IKfWcUipfKbWt2bFYpdRnSqkMx1f3NuXrnrbdo5Q66HjuNjnmFfiibcOUUl8ppXYopbYrpW52HPf5c9dB23z+3CmlrEqptUqpzY623es4nqKUWuN4vb7hmOjXW9r2glJqf7PnbXpPt61ZGwOUUhuVUh84fu7686a17tX/MJOE9gIjgWBgMzDR1+1q1r5MIN7X7XC05QRgJrCt2bG/A7c7vr8d+Fsvats9wG294HlLAmY6vo8EdmOWSPb5c9dB23z+3AEKiHB8HwSsAeYB/wUudBx/GvhlL2rbC8BPfP0352jXb4BXgQ8cP3f5efOHHrgsW+sirfW3QFGrw+cCLzq+fxE4ryfb1KidtvUKWuvDWusNju/LgZ2YlTV9/tx10Daf00aF48cgxz8NnAIsdxz31fPWXtt6BaXUUOAs4FnHzwo3njd/COAuLVvrQxpYoZRar5Ra4uvGOJGotT7s+D4X6G2bWd6glNriSLH4JL3TnFIqGZiB6bH1queuVdugFzx3jjTAJiAf+AzzablEa13vuIvPXq+t26a1bnze/uJ43v6plArxRdswS5D8DrA7fo7DjefNHwJ4b3ec1nomZmei65VSJ/i6Qe3R5rNZr+mFAE8Bo4DpwGHgYV82RikVAbwF3KK1Lmt+m6+fOydt6xXPnda6QWs9HbMa6RxgvC/a4UzrtimlJgN3YNo4G4gFft/T7VJKLQbytdbrPT2XPwTwXr1srdb6oONrPvAO5o+4N8lTSiUBOL7m+7g9TbTWeY4XmR34Dz587pRSQZgA+YrW+m3H4V7x3DlrW2967hztKQG+AuYD0UqpQMdNPn+9NmvbIkdKSmuta4Hn8c3zdixwjlIqE5MSPgV4FDeeN38I4L122VqlVLhSKrLxe2ABsK3jR/W494ErHN9fAbznw7a00BgcHX6Ej547R/5xKbBTa/2PZjf5/Llrr2294blTSiUopaId34cCp2Ny9F8BP3HczVfPm7O2pTd7Q1aYHHOPP29a6zu01kO11smYePal1voS3HnefD0S6+Jo7ZmY0fe9wF2+bk+zdo3EVMVsBrb7um3Aa5iP0zZMDu0aTG7tCyAD+ByI7UVtWwZsBbZggmWSj9p2HCY9sgXY5Ph3Zm947jpom8+fO2AqsNHRhm3AnxzHRwJrgT3Am0BIL2rbl47nbRvwMo5KFV/9A07iaBVKl583mUovhBB+yh9SKEIIIZyQAC6EEH5KArgQQvgpCeBCCOGnJIALIYSfkgAuhBB+SgK4EEL4qf8HGHk8qYxI0G8AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 测试误差\n",
    "for i, name in enumerate(name_l):\n",
    "    plt.plot(list(range(num_epochs)), test_l[i], label=name)\n",
    "plt.legend(loc = 1)\n",
    "plt.title('mse_test')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同样，能够看出经过Xavier参数初始化后的模型梯度更加平稳，进而我们判断，经过初始化之后的模型初始迭代时收敛速度更快"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.2 PyTorch实现Xavier高斯分布的参数创建\n",
    "\n",
    "用`torch.nn.init.xavier_normal_`进行初始化参数设置。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;31mSignature:\u001b[0m \u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mnn\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0minit\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mxavier_normal_\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtensor\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mTensor\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mgain\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mfloat\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;36m1.0\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m->\u001b[0m \u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mTensor\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mDocstring:\u001b[0m\n",
      "Fills the input `Tensor` with values according to the method\n",
      "described in `Understanding the difficulty of training deep feedforward\n",
      "neural networks` - Glorot, X. & Bengio, Y. (2010), using a normal\n",
      "distribution. The resulting tensor will have values sampled from\n",
      ":math:`\\mathcal{N}(0, \\text{std}^2)` where\n",
      "\n",
      ".. math::\n",
      "    \\text{std} = \\text{gain} \\times \\sqrt{\\frac{2}{\\text{fan\\_in} + \\text{fan\\_out}}}\n",
      "\n",
      "Also known as Glorot initialization.\n",
      "\n",
      "Args:\n",
      "    tensor: an n-dimensional `torch.Tensor`\n",
      "    gain: an optional scaling factor\n",
      "\n",
      "Examples:\n",
      "    >>> w = torch.empty(3, 5)\n",
      "    >>> nn.init.xavier_normal_(w)\n",
      "\u001b[1;31mFile:\u001b[0m      c:\\users\\hp\\python\\python310\\lib\\site-packages\\torch\\nn\\init.py\n",
      "\u001b[1;31mType:\u001b[0m      function\n"
     ]
    }
   ],
   "source": [
    "torch.nn.init.xavier_normal_?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\\text{std} = \\text{gain} \\times \\sqrt{\\frac{2}{\\text{fan\\_in} + \\text{fan\\_out}}}$$\n",
    "\n",
    "当然，修改参数的方法也是相同的，例如:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[-0.0735, -0.0941],\n",
       "        [-0.1332, -0.9259],\n",
       "        [ 0.2803, -0.3529],\n",
       "        [-0.1686,  0.8779]], requires_grad=True)"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[-0.8206,  0.2159, -0.9876, -0.0095],\n",
       "        [-0.1434,  0.5362, -0.3858,  0.0785],\n",
       "        [-0.0542,  0.5039,  0.9254,  0.6225],\n",
       "        [-0.6864,  0.1003,  0.4815, -0.8140]], requires_grad=True)"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[-0.7034,  0.4468, -0.4812, -0.4097]], requires_grad=True)"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#————————————————————————————torch.nn.init.xavier_normal_进行初始化参数设置。————————————————————————————\n",
    "\n",
    "# 实例化模型\n",
    "sigmoid_model2 = Sigmoid_class2()                  \n",
    "\n",
    "# 修改init模型初始参数\n",
    "for m in sigmoid_model2.modules():\n",
    "    if isinstance(m, nn.Linear):\n",
    "        nn.init.xavier_normal_(m.weight)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其他测试初始化效果的相关实验和此前操作流程一致。\n",
    "\n",
    "从理论上来说，均匀分布和高斯分布并没有根本性区别，二者任选其一使用即可。不过也有一些不是很严谨的实验证明了均匀分布比高斯分布能够产生相对较大的梯度，因而模型学习效果会更好。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 二、Kaiming方法（HE初始化）\n",
    "\n",
    "### 1.HE初始化基础理论\n",
    "\n",
    "&emsp;&emsp;尽管Xavier初始化能够在Sigmoid和tanh激活函数叠加的神经网络中起到一定的效果，但由于ReLU激活函数属于非饱和类激活函数，并不会出现类似Sigmoid和tanh激活函数使用过程中可能存在的梯度消失或梯度爆炸问题，反而因为ReLU激活函数的不饱和特性，ReLU激活函数的叠加极有可能出现神经元活性消失的问题，很明显，该类问题无法通过Xavier初始化解决。\n",
    "\n",
    "&emsp;&emsp;当然，He初始化也遵循Glorot条件，即参数初始化结果要求正向传播时各层接收到的数据方差保持一致、反向传播时各层参数梯度的方差保持一致。\n",
    "\n",
    "当然，根据参数方差，我们就能确定参数满足均匀分布时的边界，以及满足高斯分布时的基本形态。均匀分布时$bound = \\sqrt{3var}$，因此参数分布区间为:$$(-\\sqrt{\\frac{6}{fan_{in}}}, \\sqrt{\\frac{6}{fan_{in}}})$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.HE初始化在PyTorch中实现\n",
    "\n",
    "#### 2.1 PyTorch实现HE初始化的均匀分布参数创建\n",
    "\n",
    "我们可以使用`torch.nn.init.kaiming_uniform_`进行初始化参数设置。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;31mSignature:\u001b[0m\n",
      "\u001b[0mnn\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0minit\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mkaiming_uniform_\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0mtensor\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mTensor\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0ma\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mfloat\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;36m0\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0mmode\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mstr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m'fan_in'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0mnonlinearity\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mstr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m'leaky_relu'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mDocstring:\u001b[0m\n",
      "Fills the input `Tensor` with values according to the method\n",
      "described in `Delving deep into rectifiers: Surpassing human-level\n",
      "performance on ImageNet classification` - He, K. et al. (2015), using a\n",
      "uniform distribution. The resulting tensor will have values sampled from\n",
      ":math:`\\mathcal{U}(-\\text{bound}, \\text{bound})` where\n",
      "\n",
      ".. math::\n",
      "    \\text{bound} = \\text{gain} \\times \\sqrt{\\frac{3}{\\text{fan\\_mode}}}\n",
      "\n",
      "Also known as He initialization.\n",
      "\n",
      "Args:\n",
      "    tensor: an n-dimensional `torch.Tensor`\n",
      "    a: the negative slope of the rectifier used after this layer (only\n",
      "        used with ``'leaky_relu'``)\n",
      "    mode: either ``'fan_in'`` (default) or ``'fan_out'``. Choosing ``'fan_in'``\n",
      "        preserves the magnitude of the variance of the weights in the\n",
      "        forward pass. Choosing ``'fan_out'`` preserves the magnitudes in the\n",
      "        backwards pass.\n",
      "    nonlinearity: the non-linear function (`nn.functional` name),\n",
      "        recommended to use only with ``'relu'`` or ``'leaky_relu'`` (default).\n",
      "\n",
      "Examples:\n",
      "    >>> w = torch.empty(3, 5)\n",
      "    >>> nn.init.kaiming_uniform_(w, mode='fan_in', nonlinearity='relu')\n",
      "\u001b[1;31mFile:\u001b[0m      c:\\users\\hp\\python\\python310\\lib\\site-packages\\torch\\nn\\init.py\n",
      "\u001b[1;31mType:\u001b[0m      function\n"
     ]
    }
   ],
   "source": [
    "nn.init.kaiming_uniform_?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "相关参数解释:\n",
    "- mode：参数表示选择带入扇入还是扇出的神经元个数进行计算，正如前文所说，理论上二者对建模没有明显影响，可任选其一，但实际由于模型个体差异，在实际使用过程中还是略有差异，我们可以根据实际效果进行选择；\n",
    "- a：为使用ReLU变种激活函数时的修正系数；\n",
    "- nonlinearity：表示所选用的变种ReLU激活函数类型，需要配合a参数使用，相关使用方法我们将在后续介绍ReLU变种激活函数的使用时一并介绍。\n",
    "\n",
    "$$bound = \\sqrt{\\frac{6}{(1+a^2)fan_{in}}}$$\n",
    "\n",
    "和Xavier方法一样，kaiming方法直接对张量的值进行修改，并且张量列数表示$fan_{in}$的个数，行数表示$fan_{out}$的个数，据此，我们简单测试下公式正确性。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.,  1.,  2.,  3.,  4.,  5.],\n",
       "        [ 6.,  7.,  8.,  9., 10., 11.]])"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = torch.arange(12).reshape(2, 6).float()\n",
    "t\n",
    "\n",
    "# 2是fan_out\n",
    "# 6是fan_in"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x2d53b60efd0>"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.6107, -0.6019,  0.9517, -0.7944, -0.3051, -0.6891],\n",
       "        [ 0.7712,  0.3751, -0.4988, -0.7734, -0.5790, -0.1929]])"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#————————————————————————————kaiming_uniform_()————————————————————————————\n",
    "# 设置随机数种子\n",
    "torch.manual_seed(420)  \n",
    "torch.nn.init.kaiming_uniform_(t)         # 此时扇入个数是6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x2d53b60efd0>"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.4318, -0.4256,  0.6730, -0.5617, -0.2157, -0.4873],\n",
       "        [ 0.5453,  0.2653, -0.3527, -0.5469, -0.4094, -0.1364]])"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设置随机数种子\n",
    "torch.manual_seed(420)  \n",
    "torch.nn.init.kaiming_uniform_(t,a=1)         # 此时分母是"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7071067811865476"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "math.sqrt(0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.,  1.,  2.],\n",
       "        [ 3.,  4.,  5.],\n",
       "        [ 6.,  7.,  8.],\n",
       "        [ 9., 10., 11.],\n",
       "        [12., 13., 14.],\n",
       "        [15., 16., 17.]])"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = torch.arange(18).reshape(6, 3).float()\n",
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 设置随机数种子\n",
    "torch.manual_seed(420)  \n",
    "torch.nn.init.kaiming_uniform_(t)        # 此时扇入个数是3\n",
    "\n",
    "# 6 是fan_out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，尝试在建模过程中使用kaiming方法，验证是否能够加快迭代效率，并且规避神经元活性失效问题。\n",
    "\n",
    "\n",
    "> HE初始化如何帮助模型规避Dead ReLU Problem？  \n",
    "> 核心在于模型初始化时如果参数完全随机选择，就有可能出现初始参数全部输出0的结果，而通过HE初始化的参数不会出现上述情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x2d53b60efd0>"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设置随机数种子\n",
    "torch.manual_seed(420)  \n",
    "\n",
    "# 创建最高项为2的多项式回归数据集\n",
    "features, labels = tensorGenReg(w=[2, 1], bias=False, deg=2)\n",
    "\n",
    "# 进行数据集切分与加载\n",
    "train_loader, test_loader = split_loader(features, labels)\n",
    "\n",
    "# 初始核心参数\n",
    "lr = 0.001  # 学习率设置的较小\n",
    "num_epochs = 20\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x2d53b60efd0>"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[ 1.3847, -0.4435],\n",
       "        [-0.1899, -0.0916],\n",
       "        [-1.4094,  1.5347],\n",
       "        [-0.1705,  1.6922]], requires_grad=True)"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[ 1.0084,  0.5323, -0.0277, -0.0135],\n",
       "        [ 0.0282, -1.1904, -0.2923, -0.5667],\n",
       "        [-0.0729,  0.1498, -0.9598,  0.9308],\n",
       "        [ 1.0001,  0.7617,  0.5500,  0.1938]], requires_grad=True)"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[-0.8826,  0.4933,  0.4340,  1.1838],\n",
       "        [-1.1550,  1.0630,  0.9618,  0.2271],\n",
       "        [ 0.4080,  0.1464, -0.0979,  0.2467],\n",
       "        [ 0.7092,  0.1396, -0.1113, -0.0405]], requires_grad=True)"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[-0.9433, -0.5863,  0.9970,  0.0722]], requires_grad=True)"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设置随机数种子\n",
    "torch.manual_seed(420)  \n",
    "\n",
    "# 实例化模型 3个隐藏层的ReLU模型\n",
    "relu_model3 = ReLU_class3()                   # 保留原参数\n",
    "relu_model3_init = ReLU_class3()              # 使用HE初始化参数\n",
    "\n",
    "# 修改init模型初始参数\n",
    "for m in relu_model3_init.modules():\n",
    "    if isinstance(m, nn.Linear):\n",
    "        nn.init.kaiming_uniform_(m.weight)\n",
    "        \n",
    "# 创建模型容器\n",
    "model_l = [relu_model3, relu_model3_init]           \n",
    "name_l = ['relu_model3', 'relu_model3_init']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 训练模型 \n",
    "train_l, test_l = model_comparison(model_l = model_l, \n",
    "                                   name_l = name_l, \n",
    "                                   train_data = train_loader,\n",
    "                                   test_data = test_loader,\n",
    "                                   num_epochs = num_epochs, \n",
    "                                   criterion = nn.MSELoss(), \n",
    "                                   optimizer = optim.SGD, \n",
    "                                   lr = lr, \n",
    "                                   cla = False, \n",
    "                                   eva = mse_cal)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d549e71a20>]"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d549e707f0>]"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x2d54a1c7040>"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'mse_train')"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAArpElEQVR4nO3deXxcdb3/8ddnJpOtSdo0TZuUbrSg0BZK29AFLW6XRVS0imyiUBSsSBEVfaC9vyvee7lXuSpX5QpWwIIsVioIIsgmUJCugbaUltJSCrSkO92z5/v745yk0zTLJLOe6fv5eMxjzpxz5pxPJpP3nHzne87XnHOIiEjwhNJdgIiI9I4CXEQkoBTgIiIBpQAXEQkoBbiISEApwEVEAkoBLiISUApwkQQzs2Fmtt/MwumuRbKbAlwkipldZmYvxrMN59w7zrki51xzouoS6YgCXKSHdGQtmUIBLhnJzDaa2ffMbKWZHTCzO8xskJk9bmb7zOxpMys1s3wzu8fMdprZbjNbamaD/G309Z9XY2abzew/uwpfMzsRuA2Y6jeB7PbnzzWzW83sMTM7AHzMzD5lZq+Y2V4ze9fMbojazggzc2aW4z9+zsz+w8z+6df+pJkNSOLLJ0cJBbhksi8AZwAfAD4DPA78ECjHe+9eA1wK9AWGAmXATKDWf/5coAk4DhgPnAl8rbOdOefW+M9f6DeB9ItafDFwI1AMvAgcAL4C9AM+BXzDzD7Xxc9yMTADGAjkAtd198OLdEcBLpns1865rc65zcALwGLn3CvOuTrgIbxQbsQL7uOcc83OuWrn3F7/KPwc4Frn3AHn3DbgZuDCXtbysHPun865FudcnXPuOefcq/7jlcD9wEe6eP7vnXNvOOdqgT8Bp/SyDpE2OekuQKQLW6Omazt4XAT8Ae/o+49m1g+4B5gNDAciQI2ZtT4nBLzby1oOe56ZTQZ+AozFO6LOAx7o4vlboqYP+rWLxEVH4BJozrlG59yPnXOjgdOAT+M1bbwL1AMDnHP9/FuJc25Md5uMcf59wCPAUOdcX7y2czviWSJJpACXQDOzj5nZSf6Xk3vxmlRanHM1wJPAz82sxMxCZjbKzLpq5gDvKH+ImeV2s14xsMs5V2dmk/DauEVSSgEuQVcBzMcL7zXA83jNKuAdiecCq4H3/fUqu9neP4DXgC1mtqOL9a4C/t3M9gH/hteuLZJSphF5RESCSUfgIiIBpQCXo46Z3eafqNP+dlu6axPpCTWhiIgEVEr7gQ8YMMCNGDEilbsUEQm86urqHc658vbzUxrgI0aMYNmyZancpYhI4JnZ2x3NVxu4iEhAKcBFRAJKAS4iElDdtoGb2VDgbmAQ3vUg5jjnfulf//gKYLu/6g+dc48lq1AROaSxsZFNmzZRV1eX7lIkgfLz8xkyZAiRSCSm9WP5ErMJ+K5z7mUzKwaqzewpf9nNzrmf9bJWEemlTZs2UVxczIgRI4i62qIEmHOOnTt3smnTJo499tiYntNtE4pzrsY597I/vQ/vehPHxFWpiMSlrq6OsrIyhXcWMTPKysp69F9Vj9rAzWwE3kX0F/uzrvaHvLrTzEp7si0RiY/CO/v09Hcac4CbWRHwZ7wRTvYCtwKj8EYWqQF+3snzrjSzZWa2bPv27R2t0r21f4cXftG754qIZKmYAtzMInjhfa9z7kEAf6irZudcC/A7YFJHz3XOzXHOVTnnqsrLjziRKDYbnoMXOvx8EBE5anUb4OYd098BrHHO/SJqfvR1lacDqxJfnq+kEhr2Q93epO1CRJKjqCj9o8dddtllzJ8/P+Z1vvrVrzJu3DhOPvlkzjvvPPbv35+KMnssliPwDwFfBj5uZsv92znATWb2qpmtBD4GfDtpVRb7nxX7tnS9noikhXOOlpaWdJeRMDfffDMrVqxg5cqVDBs2jFtuuSXdJXWo226EzrkX6Xisv9T1+W4L8Peg/AMp261IEPz4r6+x+r3E/nc6enAJP/pM18OHbty4kbPOOovJkydTXV3N+eefz6OPPkp9fT3Tp0/nxz/+8WHrP/fcc/zsZz/j0UcfBeDqq6+mqqqKyy67rMPtjxgxgosuuojHH3+cnJwc5syZww9+8APWr1/P9773PWbOnIlzju9///s8/vjjmBn/+q//ygUXXIBzjlmzZvHUU08xdOhQcnMPjZBXXV3Nd77zHfbv38+AAQOYO3culZWHD9RUUlICeB9MtbW1GfuFcTDOxCwZ7N3vrUlvHSJymHXr1nHVVVdx8803s3nzZpYsWcLy5cuprq5mwYIFcW9/2LBhLF++nGnTprU1cSxatIgf/ehHADz44IMsX76cFStW8PTTT/O9732PmpoaHnroIdauXcvq1au5++67eemllwDvBKhZs2Yxf/58qqurufzyy5k9e3aH+54xYwYVFRW8/vrrzJo1K+6fJRlSejXCXiuu8O73KcBF2uvuSDmZhg8fzpQpU7juuut48sknGT9+PAD79+9n3bp1nH766XFt/9xzzwXgpJNOYv/+/RQXF1NcXExeXh67d+/mxRdf5KKLLiIcDjNo0CA+8pGPsHTpUhYsWNA2f/DgwXz84x8HYO3ataxatYozzjgDgObm5iOOvlv9/ve/p7m5mVmzZjFv3jxmzJgR18+SDMEI8Nw+kNdXAS6SYfr06QN4TQ0/+MEP+PrXv97pujk5OYe1k8dywkpeXh4AoVCobbr1cVNTU4/rdc4xZswYFi5cGNP64XCYCy+8kJtuuikjAzwYTSjg9UTZ+166qxCRDpx11lnceeedbb01Nm/ezLZt2w5bZ/jw4axevZr6+np2797NM888E/d+p02bxrx582hubmb79u0sWLCASZMmcfrpp7fNr6mp4dlnnwXggx/8INu3b28L8MbGRl577bXDtumcY/369W3TjzzyCCeccELctSZDMI7AwfsiU0fgIhnpzDPPZM2aNUydOhXwug7ec889DBw4sG2doUOHcv755zN27FiOPfbYtuaWeEyfPp2FCxcybtw4zIybbrqJiooKpk+fzj/+8Q9Gjx7NsGHD2urKzc1l/vz5XHPNNezZs4empiauvfZaxow51AzlnOPSSy9l7969OOcYN24ct956a9y1JkNKx8SsqqpyvR6R56FvwFvPw3dWJ7YokQBas2YNJ554YrrLkCTo6HdrZtXOuar26warCWXfFmhpTnclIiIZIVhNKK4ZDmw/1CtFRAJv+vTpvPXWW4fN++lPf8pZZ52VpoqCIzgB3toXfF+NAlwkizz00EPpLiGwgtOE0hraOplHRAQIVIC3HoGrK6GICAQpwIsGgoV0BC4i4gtOgIfCUDRIVyQUEfEFJ8DBP5lHTSgiQZLt1wO/7bbbuPvuu7vc9rJly7jmmmsA76qMrRfXildweqGA1xNl55vprkJE2nHO4ZwjFArWMWFnbr755rZLyn7nO9/hlltu4frrr+9w3ZkzZ3a7vaqqKqqqvPNwnnvuOYqKijjttNPirjNYAV5cCRtfSHcVIpnl8ethy6uJ3WbFSfDJn3S5iq4H7rnhhhsoKiriuuuu46Mf/SiTJ0/m2WefZffu3dxxxx1Mmzat7We/5ZZbuO222wiHw9xzzz38+te/Ztq0aV2+zl0JWIBXQN0eaDgIuYXprkbkqLdu3Truuusu9u7dy/z581myZAnOOc4991wWLFgQ9+VkW68H/u1vf5vLLruMf/7zn9TV1TF27Fhmzpx52PXAd+zYwamnnsrpp5/OwoUL264HvnXrVkaPHs3ll1/edj3whx9+mPLycubNm8fs2bO58847j9j3jBkzeOyxxxg9ejQ//3nsY/I2NTWxZMkSHnvsMX784x/z9NNPty0bMWIEM2fObAv8eAUrwKNP5ikbld5aRDJFN0fKyaTrgR/p85//PAATJ05k48aNcf383QlEgG/ccYA1NXv5ZNvQagpwkUyg64F3XnM4HO5VjT0RiG8cbnv+Tb7zpxUcyPMvTamuhCIZRdcDj11xcTH79u1LyLYCcQR+walD+ePSd3n8bTgPNLCDSIbR9cBj95nPfIbzzjuPhx9+OO4vMQNxPXDnHGf97wIKI2H+sud8mHgpnP3fSahQJBh0PfDslXXXAzczzq8ayvJNe2goHKQjcBERAhLgAJ+fMIRI2HivuZ/awEWyyPTp0znllFMOuz3xxBPpLusIN9544xF13njjjWmtKRBt4AD9++RyxuhBvLauD8PDb9N5t3qRo4NzrssTTIIiKNcDnz17NrNnz07qPnrapB2YI3CAC04dxrtN/WjZVwMpbLsXyTT5+fns3Lmzx3/wkrmcc+zcuZP8/PyYnxOYI3CADx83gJfzBhJuboSDO6HPgHSXJJIWQ4YMYdOmTWzfvj3dpUgC5efnM2TIkJjXD1SAh0PGcccdD2th63sbGXS8AlyOTpFIhGOPPTbdZUiaBaoJBWDSyV5/zYWvJPjiPSIiARO4AB90jHfU8foba2lpUfufiBy9AhfgrYMb59Vu459v7khzMSIi6dNtgJvZUDN71sxWm9lrZvYtf35/M3vKzNb596XJLxcIR3B9yhkW2c28pe+mZJciIpkoliPwJuC7zrnRwBTgm2Y2GrgeeMY5dzzwjP84Jay4kpNKDvLka1t5/0BDqnYrIpJRug1w51yNc+5lf3ofsAY4BvgscJe/2l3A55JU45FKBjMssoeG5hYeemVzynYrIpJJetQGbmYjgPHAYmCQc67GX7QFGJTY0rpQXEF+7TbGDenLn5a9q5MZROSoFHOAm1kR8GfgWufc3uhlzkvQDlPUzK40s2VmtixhJx0UD4aDO7hwwiBe37KPlZv2JGa7IiIBElOAm1kEL7zvdc496M/eamaV/vJKYFtHz3XOzXHOVTnnqsrLyxNRM5R4I/N8emSI/EiIP+rLTBE5CsXSC8WAO4A1zrlfRC16BLjUn74UeDjx5XWi2Bsbs7hhO586aTB/XfEeBxuSO3SRiEimieUI/EPAl4GPm9ly/3YO8BPgDDNbB/yL/zg1/L7g7KvhglOHsr++icde1SVmReTo0u21UJxzL0KnV2/9RGLLiVHr6PR7azh1dCkjB/ThT0vf5byJsV8ERkQk6IJ3JiZAQSmE82Dfe5gZX6waypKNu3hz+/50VyYikjLBDHAz74tMf2SeL0w8hnDI+NMyfZkpIkePYAY4QHEl7PW6oQ8szufjJwzkz9WbaWxuSXNhIiKpEewA33docOMLqoayY389z77eYW9GEZGsE9wALxnsHYH7Z2F+9IPlDCzO0wWuROSoEdwAL66Aplqo887CzAmH+MLEITy7dhtb99aluTgRkeQLcIB7Z2Oyr6Zt1vlVQ2lxML96U5qKEhFJneAGeFtf8EPt4McO6MPkY/vzgC5wJSJHgeAGeAdH4AAXnDqUjTsPsvitXWkoSkQkdQIc4IdOp4/2ybGVFOfl6MtMEcl6wQ3wSIF3RubewwO8IDfMZ8cP5rFXa9hT25im4kREki+4AQ7eVQnbHYEDXFA1jPqmFh5Z8V4HTxIRyQ4BD/CKDgN87DElnFhZwp/UjCIiWSzYAV5SeUQTCoCZcUHVEF7dvIfX3tNoPSKSnYId4MWD4cA2aD5yMIfPjT+G3JyQjsJFJGsFO8BLKsG1wP6tRyzqV5jL2WMq+Mvy96hrbE5DcSIiyRXsAG/rC97xaDwXnDqUPbWNPPGaRusRkeyTJQHecW+TqSPLGNq/QH3CRSQrBTvAo4ZW60goZJw/cSgvvbmTd3YeTGFhIiLJF+wALxwAoZxOj8ABzqsaQsjggWodhYtIdgl2gIdCUFTRaRs4QGXfAk7/QDl/rt6kC1yJSFYJdoCD3xe86zMuzxpTwXt76tiw40CKihIRSb7gB3hxZYdnY0abOrIMgIVv7kxFRSIiKZElAd51N8HhZYVUlOSzcIMCXESyR/ADvKQS6vdC/f5OVzEzpo4qY/GGnWoHF5GsEfwAL/a7EsbQjLJjfwPrt3Ue9CIiQRL8AC/xT+bp5ovMKa3t4GpGEZEsEfwA7+Z0+lZD+xdwTL8CfZEpIlkjiwK86yNwM2PKyDIWv7WLlha1g4tI8AU/wPOKIK+k09Ppo00dVcauAw28sW1fCgoTEUmu4Ac4xNQXHGDKyP6A+oOLSHboNsDN7E4z22Zmq6Lm3WBmm81suX87J7lldqOTodXaG1JayND+agcXkewQyxH4XODsDubf7Jw7xb89ltiyeqhkcExNKOB1J1Q7uIhkg24D3Dm3ANiVglp6r7gS9m+BlpZuV506qow9tY2s2bI3BYWJiCRPPG3gV5vZSr+JpbSzlczsSjNbZmbLtm/fHsfuulBcCS1NcHBHt6tO0XVRRCRL9DbAbwVGAacANcDPO1vROTfHOVflnKsqLy/v5e66EePJPOBdXnZEWSGLNmT2PxUiIt3pVYA757Y655qdcy3A74BJiS2rh2I8nb7V1FFlLH5rJ81qBxeRAOtVgJtZZdTD6cCqztZNiR4cgYPXjLKvronV76kdXESCK6e7FczsfuCjwAAz2wT8CPiomZ0COGAj8PXklRiDPgPBQt2eTt+q7frgG3Zw0pC+yaxMRCRpug1w59xFHcy+Iwm19F44xwvxbk6nbzWwJJ+R5X1YtGEXV54+KsnFiYgkR3aciQn+0GqxtYGDdxS+5K1dNDV33/VQRCQTZU+AFw+OuQkFvHbw/fVNrFI7uIgEVBYFeEXMTSig/uAiEnzZE+AllVD7PjTWxrR6eXEexw8sYpEGeBCRgMqeAO9hX3Dw+oMv3biLRrWDi0gAZVGAV3j3PWwHP9jQzMpNe5JUlIhI8mRPgJf4R+AxnswDh9rB1YwiIkGUPQHeNrRa7E0o/fvkckJFsQJcRAIpewI8vy9ECnvUFxy8o/BlG9+noUnt4CISLNkT4GYxj8wTbcrIMmobm1mxaXdy6hIRSZLsCXDwT+bpaYD3x0z9wUUkeLIrwEsqe/QlJkC/wlxOrChRO7iIBE52BXhxhdeN0PXsOt9TR5VR/fb71Dc1J6kwEZHEy7IAHwzN9d4ZmT0wZWQZ9U0tvPLO7uTUJSKSBNkV4D0c2KHVpGP7E1I7uIgETHYFeC9OpwfoWxBhzOC+agcXkUDJsgBvPZ2+ZwEOXm+UV97ZTV2j2sFFJBiyLMBbm1B6HuBTR5XR0NzCy2/3rP1cRCRdsivAc3KhcECPrgve6tQR/QmHjIVqRhGRgMiuAIceD63Wqjg/wthj1A4uIsGRfQFeXNmrNnDw2sGXv7ub2ga1g4tI5lOAR5k6sozGZseyt3cluCgRkcTLvgAvGQwHtkNTQ4+feuqI/uSETP3BRSQQsi/AW7sS7t/a46f2ycvh5CFqBxeRYMjCAO/dyTytpowsY+WmPRyob0pgUSIiiZd9Ad7L0+lbTR1VRlOLY+lGtYOLSGbLvgCP8wi8anh/ImH1BxeRzJd9AV7YH8K5vQ7wgtwwpwztx6INOgIXkcyWfQHeOrRaL07maTVlZBmrNu9hX11jAgsTEUms7Atw6NXQatGmjiyjWe3gIpLhsjPAezG0WrQJw0vJDYfUH1xEMlq3AW5md5rZNjNbFTWvv5k9ZWbr/PvS5JbZQ8WVvRparVV+JMz4YWoHF5HMFssR+Fzg7Hbzrgeecc4dDzzjP84cxZXQeADq9/Z6E1NGlvHae3vYU6t2cBHJTN0GuHNuAdD+UPSzwF3+9F3A5xJbVpxK/K6EcXyROXVUGS0Olrylo3ARyUy9bQMf5JxrTcctwKDOVjSzK81smZkt2759ey9310NxjMzT6pSh/cjLUTu4iGSuuL/EdM45oNPGZufcHOdclXOuqry8PN7dxaZ1ZJ44Ajw/EmbCsFJdF0VEMlZvA3yrmVUC+PfbEldSArQ1ofS+Jwp4zShrtuxl98GeX9lQRCTZehvgjwCX+tOXAg8nppwEiRRAfr+4jsDBC3DnUG8UEclIsXQjvB9YCHzQzDaZ2VeBnwBnmNk64F/8x5mltSthHMYN6Ud+JKRmFBHJSDndreCcu6iTRZ9IcC2JFefJPAC5OSGqhvdXgItIRsrOMzEh7tPpW00dVcbrW/axc399AooSEUmc7A3wkkpvVJ7m+AZmmDKyDIDF6g8uIhkmewO8uAJcizc+ZhxOHtKXwtywmlFEJONkcYC3DuwQXzt4JByiakR/Xli3g/qm5gQUJiKSGNkb4G1Dq8XfDn7exCG8teMAl89dqmuEi0jGyN4AT8DZmK3OHTeYn39xHIs27OLCOYvYtq8u7m2KiMQrewO8TzlYOCEBDvCFiUO4/dIqNmw/wHm3LmTjjgMJ2a6ISG9lb4CHwnEPrdbexz44kPuumMy+uka+cOtLvLppT8K2LSLSU9kb4OCfjRnfl5jtjR9WyvxvnEZ+JMyFcxbywroUXWFRRKSdLA/wirhPp+/IqPIiHrzqNIb2L+TyuUt5ePnmhO9DRKQ72R3gJYMT2oQSbVBJPvO+PpXxw0r51h+Xc8eLbyVlPyIincnuAC+uhPo90JCcLxz7FkS4+/JJnD2mgv94dDX//fgaXC/H4RQR6ansDvDW64InoRmlVX4kzP99aQJfmjyM3z6/ge8+sILG5pak7U9EpFW3VyMMtNah1fa+B2WjkrabcMj4z8+NZVBJPr946g12HWjgN1+aQGFudr+8IpJe2X0E3nY6fXLawaOZGdd84nj+a/pJLHhjOxf9bjG7DmgkHxFJnuwO8LbT6RPblbArF08exq2XTOT1mr2cd9tLbHr/YMr2LSJHl+wO8LxiyC1Kaht4R84aU8E9X5vMjn31fP43L7GmZm9K9y8iR4fsDnBIysk8sTh1RH8emHkaITPO/+1CXY5WRBIu+wO8pDJpfcG788GKYv581WkMLM7jy3cs5s4X31I3QxFJmOwP8AQNrdZbx/Qr4MFvfIiPfGAg//7oar7+h2r2HNQlaUUkfkdBgPun07ekr29238IIv/vKRP71Uyfyj9e3cc6vXuCVd95PWz0ikh2yP8BLBkNLIxxMbxu0mfG1aSN5YOZUAL5420Juf2GDmlREpNeyP8DbBnZI/ReZHRk/rJTHrpnGx08YyH/+bQ1X3L2M3QfVX1xEeu4oCvDUdiXsSt/CCL/98kR+9JnRPP/Gds755QtUv60mFRHpmewP8DSczBMLM2PGh45l/szTCIeNC367kN8+/yYtLWpSEZHYZH+AFw0CLK09Uboybmg/Hp01jTNGD+K/H3+dr929TKfgi0hMsj/AwxEoGphxR+DR+hZE+M2XJvDvnx3Di+t28KlfvcCyjbvSXZaIZLjsD3BI2sg8iWRmfGXqCB686jRyc0JcMGcRv3luvZpURKRTR0mAp/dknp4Ye0xfHp31Yc4eW8FNf1/LjLlL2bm/Pt1liUgGOjoCvO8xsGsDbF+b7kpiUpwf4ZaLxvOfnxvLwg07OedXL7DkLTWpiMjhjo4AnzzTuyrh3E/DttfTXU1MzIxLpgznoatOozA3h0tuX8zKTbvTXZaIZJC4AtzMNprZq2a23MyWJaqohBtwPFz2N7AQzP0UbF2d7opiNmZwXx78xmmUF+dx1b0v6zoqItImEUfgH3POneKcq0rAtpKn/ANeiIcjcNenYcuqdFcUs9I+udxy8Xi27q3juw8s1xebIgIcLU0orQYc54V4Tr4X4jUr011RzMYPK2X2OSfy9JptzHlhQ7rLEZEMEG+AO+BJM6s2sys7WsHMrjSzZWa2bPv27XHuLgHKRsFlj0KkD9z1GXhveboritmlp43gUydX8j9PrNUAESISd4B/2Dk3Afgk8E0zO739Cs65Oc65KudcVXl5eZy7S5D+I2HG3yCvBO4+FzZXp7uimJgZP/3CyQzvX8is+19h2766dJckImkUV4A75zb799uAh4BJiSgqJUpHeCGe3w/ung6bMvc72GhFeTn85pIJ7Ktr5Fv3L6epOX3XOReR9Op1gJtZHzMrbp0GzgSC880gQL9hXpt4YX/4w3R4d0m6K4rJCRUl3Pi5k1i4YSc3P/1GussRkTSJ5wh8EPCima0AlgB/c879PTFlpVC/oV6I9yn3QvydRemuKCZfmDiEC08dyv89+yb/eH1russRkTTodYA75zY458b5tzHOuRsTWVhK9T3GC/HiCvjD52HjP9NdUUxuOHcMoytL+Pa8FWx6/2C6yxGRFDu6uhF2paTSC/G+x8C958FbL6S7om7lR8LceskEWpzjm/e+TH1Tc7pLEpEUUoBHK67wQrzfMLj3i7Dh+XRX1K3hZX34n/PGsWLTHm7825p0lyMiKaQAb69oIFz6KPQ/Fu47H978R7or6tbZYyu4Ytqx3L3wbR5ZkbnXPReRxFKAd6So3AvxsuPgvgth/dPprqhb3z/7BKqGl3L9n1eyftu+dJcjIimgAO9MnzK49K/eNVTuvwjeeDLdFXUpEg5xy8UTKIiE+cY9L3OwoSndJYlIkinAu1LYH77yCAwcDfO+BG88ke6KulTRN59fXjie9dv3M/uhVTini16JZDMFeHcK+8NX/gKDxsC8SzI+xD98/ACu/cQHeOiVzdy/5N10lyMiSaQAj0VBKXz5ocCE+KyPH8fpHyjnhr++xqrNe9JdjogkiQI8Vu1DfG3mnnQaChn/e8EplPXJ5Rv3VmsQCJEspQDviegQ/9OXMzrE+/fJ5ZaLJ1Czu47r5q9Qe7hIFlKA91RBKXz5L4EI8YnDS/nhOSfy1OqtzFmgQSBEso0CvDcK+h0K8QxvTpnxoRGcc1IFP/376/xVJ/mIZBUFeG+1hnjF2IwOcTPjZ18cR9Xw/lw7bzl/X7Ul3SWJSIIowOMRkBAvzM3hzhmncvKQvsy6/2WeWaPLz4pkAwV4vNpC/CQ/xB9Pd0UdKsrLYe6MSZxQUcI37nmZBW9kwPikIhIXBXgiFPTzeqdUnATzvpyxId63IMIfvjqJUQOLuOLuZbz05o50lyQicVCAJ0pAQrxfYS73fHUSw8sK+ercZSzduCvdJYlILynAE6l9iL/+WLor6lBZUR73fG0ylX3zmfH7pbzyzvvpLklEekEBnmitIV55MvzpKxkb4gOL87nviin075PLV+5colPuRQJIAZ4MBf3gkgczPsQr+uZz3xWTKcmPcMkdi1lTszfdJYlIDyjAk6V9iK95NN0VdWhIaSH3XzGF/Jwwl9y+mHVbNRiESFAowJMpOsTnfQnuvxi2rEp3VUcYVlbIfVdMJhQyLr59MW/tOJDukkQkBgrwZCvo5w0K8bHZsPFFuO1D8MBlsH1tuis7zMjyIu772mRaWhwX/24R7+46mO6SRKQbCvBUyCuCj3wfrl0B066DdU/Bb6bAg1+HnW+mu7o2xw8q5p6vTaa2sZkL5yxi8+7adJckIl1QgKdSQSl84v/Bt1bC1Kth9cNwy6nw8NWw+510VwfAiZUl/OHyyeyta+Ti3y1iy566dJckIp1QgKdDnzI48z/gWytg0hWwch78agL87buwN/1XDDxpSF/uunwSO/bVc/Hti9i2TyEukokU4OlUPAg++VO45hUYfwlUz4VfngJ//yHsT++1SiYMK2Xu5ZOo2V3HJbcvZuf++rTWIyJHslSO1FJVVeWWLVuWsv0Fzvsb4fn/gRX3QU4+TP46nHaNN7Bymry0fgcz5i6lsm8+nzypkonDSpkwvJT+fXLTVpPI0cbMqp1zVUfMV4BnoB3r4fmfwKvzIbcIpl4FU67yerSkwT/X7+CmJ9by2uY9NLV475eR5X2YOKyUicNLqRpRysgBRYRClpb6RLKdAjyItq2BZ/8L1jziPc7vB30GQOEAKCzz2tILBxya16fMm986L1KQ0HLqGptZ8e5uqt95n5fffp/qt9/nfX/A5L4FESYM68fE4aVMHN6fcUP7Upibk9D9ixytFOBBVrPCu7rhgR1wcId/v/PQraWp4+dF+hwK9YJSyCmAnDyveabD+7yoxx2tkw+RQojkQ04BLiePDXtaqH5nT1ugr9u2H4BwyBhdWeIHeiljj+lLv4IIRfk5RML66kWkJ5IS4GZ2NvBLIAzc7pz7SVfrK8CTwDmo2w0HWgN9x6GgP7jr0HTtbmiqh6Y67745aropzl4m4Twv1COFNIfzqHW57GuO8H5DmB31IQ62RKgjQoOL0EAOLaEI5OQRCudikTxCkTzCOXnk5OYTzs0nNzePSF4BeXl55OYVkF9QQF5uPqFIPqHcPMKRfMI5eYQi+YRzcwnn5hGJFJATiRAJh8gJGeGQYaYmHckOnQV4r//HNbMw8H/AGcAmYKmZPeKcW937MqXHzLyj64JS4LjebcM5aG44PNCPuK+DxjporIWmWu++sdaff9Bb1lRLuLGWosZaiprqqGysxTXWUl+7h6b6g9DcSKilgVBLI+GWBsKNjYQaE/cfYLMzGohwgBwaiNBIDo1EaLBD000WabtvshzvnhyaLNefF6E55N23WISmkH9vuRAK4SxMCyGc5eDMe+wsRIuFwZ92FsaFwt49IQj58wl728BwFgJCWCjkT4dxIQMLec+xkPe79bdnZhjeGKdmDsN/jPNWa1vu2qa9PZm3HHChMGBYKOxvN4RZyK8phIW8rYbM37X/XIjePm11EPWYw5ZHPS9qHh18nnb2EdvRh6+1LTt0b9hh86Kff8T6Ua+FN88O+3naao2qt/2y1u1wxLaPLLR9bSdUFNOvMLFf/sfTSDkJWO+c2wBgZn8EPgsowIPG7FDzSaI3DeR3tUJzk/ffQHMDNDVAcwMtjfUcrDtIbW0tBw/WUld3kPq6WlxTAy1N9dBUj/Nv1tyAa27AmhuwpgZo8eaFWuf5HxihlgYKWxoIO//Dwx0g3NJIjmsg7JrIiZqOuEZCtCT8tQiCZmc04wV6C+bf/A8d8O+9RIp+fOhj2Lpcl6h7AOcOLafdOh3N60r75d2t3xPRP2FvvXXGzxg/7VMJqOaQeAL8GODdqMebgMntVzKzK4ErAYYNGxbH7iQrhXO8G33aZoWAIv+WNi3Nh5qamhu9adfszXct3n1LU7t5Tf60Py962vnrtN3coe0cNr/9OlHLowPpsEPOIw4/O1jmwDmca8a1OFxLE7gWnL9t19KCtd7TDC0thF0LobYanb++a7c9L5odYK2P/XngvEla53HoPnoeHSxrm0cHy2jb56FZhwLWtXvcOvOwD462Gg7favvNucOWtv+AaD/RwbKoqYHHVBy5YpyS3k3AOTcHmANeG3iy9yeSEKEw5BYChemuJKGMzpstJHji6Q6wGRga9XiIP09ERFIgngBfChxvZseaWS5wIfBIYsoSEZHu9LoJxTnXZGZXA0/gdSO80zn3WsIqExGRLsXVBu6cewzIzAEfRUSynE6JExEJKAW4iEhAKcBFRAJKAS4iElApvRqhmW0H3u7l0wcAOxJYTqKpvviovviovvhlco3DnXPl7WemNMDjYWbLOroaV6ZQffFRffFRffELQo3tqQlFRCSgFOAiIgEVpACfk+4CuqH64qP64qP64heEGg8TmDZwERE5XJCOwEVEJIoCXEQkoDIuwM3sbDNba2brzez6Dpbnmdk8f/liMxuRwtqGmtmzZrbazF4zs291sM5HzWyPmS33b/+Wqvr8/W80s1f9fR8xgrR5fuW/fivNbEIKa/tg1Ouy3Mz2mtm17dZJ6etnZnea2TYzWxU1r7+ZPWVm6/z70k6ee6m/zjozuzSF9f2Pmb3u//4eMrN+nTy3y/dCEuu7wcw2R/0Oz+nkuV3+rSexvnlRtW00s+WdPDfpr1/cnD8MUibc8C5L+yYwEsgFVgCj261zFXCbP30hMC+F9VUCE/zpYuCNDur7KPBoGl/DjcCALpafAzyONzDLFGBxGn/XW/BOUEjb6wecDkwAVkXNuwm43p++HvhpB8/rD2zw70v96dIU1XcmkONP/7Sj+mJ5LySxvhuA62L4/Xf5t56s+tot/znwb+l6/eK9ZdoReNtAyc65BqB1oORonwXu8qfnA5+wjoawTgLnXI1z7mV/eh+wBm9s0CD5LHC38ywC+plZZRrq+ATwpnOut2fmJoRzbgGwq93s6PfYXcDnOnjqWcBTzrldzrn3gaeAs1NRn3PuSedck/9wEd5oWGnRyesXi1j+1uPWVX1+bpwP3J/o/aZKpgV4RwMltw/ItnX8N/EeoCwl1UXxm27GA4s7WDzVzFaY2eNmNia1leGAJ82s2h9Qur1YXuNUuJDO/3DS+foBDHLO1fjTW4BBHayTKa/j5Xj/UXWku/dCMl3tN/Hc2UkTVCa8ftOArc65dZ0sT+frF5NMC/BAMLMi4M/Atc65ve0Wv4zXLDAO+DXwlxSX92Hn3ATgk8A3zez0FO+/W/4QfOcCD3SwON2v32Gc9790Rva1NbPZQBNwbyerpOu9cCswCjgFqMFrpshEF9H10XfG/y1lWoDHMlBy2zpmlgP0BXampDpvnxG88L7XOfdg++XOub3Ouf3+9GNAxMwGpKo+59xm/34b8BDev6rRMmEw6k8CLzvntrZfkO7Xz7e1tVnJv9/WwTppfR3N7DLg08CX/A+ZI8TwXkgK59xW51yzc64F+F0n+03365cDfB6Y19k66Xr9eiLTAjyWgZIfAVq/8T8P+Ednb+BE89vM7gDWOOd+0ck6Fa1t8mY2Ce81TskHjJn1MbPi1mm8L7tWtVvtEeArfm+UKcCeqOaCVOn0yCedr1+U6PfYpcDDHazzBHCmmZX6TQRn+vOSzszOBr4PnOucO9jJOrG8F5JVX/R3KtM72W+6B0X/F+B159ymjham8/XrkXR/i9r+htdL4g28b6hn+/P+He/NCpCP96/3emAJMDKFtX0Y79/plcBy/3YOMBOY6a9zNfAa3rfqi4DTUljfSH+/K/waWl+/6PoM+D//9X0VqErx77cPXiD3jZqXttcP74OkBmjEa4f9Kt53Ks8A64Cngf7+ulXA7VHPvdx/H64HZqSwvvV47cet78HWXlmDgce6ei+kqL4/+O+tlXihXNm+Pv/xEX/rqajPnz+39T0XtW7KX794bzqVXkQkoDKtCUVERGKkABcRCSgFuIhIQCnARUQCSgEuIhJQCnARkYBSgIuIBNT/By6jtTfHIlAHAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 训练误差\n",
    "for i, name in enumerate(name_l):\n",
    "    plt.plot(list(range(num_epochs)), train_l[i], label=name)\n",
    "plt.legend(loc = 1)\n",
    "plt.title('mse_train')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d549c46ec0>]"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d549c45a50>]"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x2d54f4b60b0>"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'mse_test')"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAsxklEQVR4nO3deXxU5dnw8d81k30FkrDviwsioEYBBbR171O1KC5UcW2Vt1WrrbZauujztk/Van1a7SvFahVXKkqlFqq4FRUQQdkRg4gKhl1IAtnnev84J2EICZlk1jO5vp/PfObMOWfOueYwXLnnPvciqooxxhjv8cU7AGOMMe1jCdwYYzzKErgxxniUJXBjjPEoS+DGGONRlsCNMcajLIEbY4xHWQI3xhiPsgRuTDNE5GoReTdCx9okImdE4ljGBLMEbowxHmUJ3CQ0t/R6u4isFJF9IvKYiHQTkXkiUi4ir4tIZxHJEJGnRWSXiOwRkQ9EpJt7jHz3faUiskVEfiMi/sOc82hgGjBGRCpEZI+7Pl1E7heRL0Rkm4hME5FMd1uhiLzinnu3iLwjIj4ReQroC/zTPdZPo37RTIeREu8AjAnBRcCZON/Xj4DjgOuAdcBc4GZgK5AP9AGqgZFApfv+J4DtwGAgG3gF+BL4S3MnU9V1IjIF+J6qjg3adA8wyD12LfAs8CvgTuAnwGagyN13tHMonSwi49xjvd7+S2DMoawEbrzgIVXdpqpbgHeA91X1I1WtAmbjJPRaoAAYrKr1qrpMVcvcUvi3gFtUdZ+qbgceBC5rSwAiIsD1wK2qultVy4H/CTpOLdAD6Keqtar6jtpIcSbKrARuvGBb0HJlM69zgKdwSt/Pi0gn4GlgKtAPSAVKnRwMOAWXL9sYQxGQBSwLOo4ADVUxvwfuAl5zt09X1XvaeA5j2sQSuEkKqloL3A3cLSL9capW1rvP1UChqta15ZBNXu/E+WNxjPtLoOn5y3GqUX4iIsOAN0XkA1V9o5ljGRMRVoVikoKIfENEjnVvTpbhVGkEVLUUeA14QETy3BuLg0Tk1FYOuQ3oLSJpAKoaAB4FHhSRru45e4nI2e7yt0VksFvVsheoBwJBxxoY2U9sjCVwkzy6A7Nwkvc64D841SoAVwJpwFrga3e/Hq0c701gDbBVRHa6634GbAAWi0gZ8DpwpLttiPu6AlgE/D9Vfcvd9jvgF24LldvC+ZDGBBO7z2KMMd5kJXBjjPEoS+Cmw3I74lQ085gW79iMCYVVoRhjjEfFtBlhYWGh9u/fP5anNMYYz1u2bNlOVS1quj6mCbx///4sXbo0lqc0xhjPE5HPm1tvdeDGGONRlsCNMcajLIEbY4xH2VgoxnhQbW0tmzdvpqqqKt6hmAjKyMigd+/epKamhrS/JXBjPGjz5s3k5ubSv39/gkZHNB6mquzatYvNmzczYMCAkN5jVSjGeFBVVRUFBQWWvJOIiFBQUNCmX1WWwI3xKEveyaet/6beSODr/w3v/CHeURhjTELxRgLf+DYsuB+s278xxjTyRgLP6wm1+6C6LN6RGGPaKCcnJ94hcPXVVzNr1qyQ97nuuusYMWIEw4cPZ+LEiVRUVMQizDbzTgIHKPsqvnEYY5qlqgQCgdZ39IgHH3yQFStWsHLlSvr27cvDDz8c75Ca5Y1mhPm9neeyLdD16PjGYkyCufufa1j7VWR/nQ7tmcevzzvmsPts2rSJs88+m1GjRrFs2TIuueQSXnnlFaqrq5kwYQJ33333Qfu//fbb3H///bzyyisA3HjjjRQXF3P11Vc3e/z+/fszadIk5s2bR0pKCtOnT+fOO+9kw4YN3H777UyZMgVV5ac//Snz5s1DRPjFL37BpZdeiqpy0003MX/+fPr06UNaWlrjcZctW8aPf/xjKioqKCws5IknnqBHj4MnaMrLywOcP0yVlZUJe8PYSuDGmHYrKSnhBz/4AQ8++CBbtmxhyZIlLF++nGXLlrFgwYKwj9+3b1+WL1/OuHHjGqs4Fi9ezK9//WsAXnrpJZYvX86KFSt4/fXXuf322yktLWX27NmsX7+etWvXMmPGDBYuXAg4HaBuuukmZs2axbJly7j22muZOnVqs+e+5ppr6N69Ox9//DE33XRT2J8lGrxRAs/pDoglcGOa0VpJOZr69evH6NGjue2223jttdc47rjjAKioqKCkpITx48eHdfzzzz8fgGOPPZaKigpyc3PJzc0lPT2dPXv28O677zJp0iT8fj/dunXj1FNP5YMPPmDBggWN63v27Mk3v/lNANavX8/q1as588wzAaivrz+k9N3gb3/7G/X19dx0003MnDmTa665JqzPEg3eSOApaZDT1alCMcYkjOzsbMCparjzzju54YYbWtw3JSXloHryUDqspKenA+Dz+RqXG17X1dW1OV5V5ZhjjmHRokUh7e/3+7nsssu47777EjKBe6MKBZxqFCuBG5OQzj77bB5//PHG1hpbtmxh+/btB+3Tr18/1q5dS3V1NXv27OGNN94I+7zjxo1j5syZ1NfXs2PHDhYsWMBJJ53E+PHjG9eXlpby1ltvAXDkkUeyY8eOxgReW1vLmjVrDjqmqrJhw4bG5Tlz5nDUUUeFHWs0eKMEDpDXC3ZvjHcUxphmnHXWWaxbt44xY8YATtPBp59+mq5duzbu06dPHy655BKGDRvGgAEDGqtbwjFhwgQWLVrEiBEjEBHuu+8+unfvzoQJE3jzzTcZOnQoffv2bYwrLS2NWbNmcfPNN7N3717q6uq45ZZbOOaYA9VQqspVV11FWVkZqsqIESN45JFHwo41GmI6J2ZxcbG2e0aeubfDyplwxxeRDcoYD1q3bh1HH20tspJRc/+2IrJMVYub7uutKpSqvVCdmA3qjTEm1rxVhQJOPXjREfGNxRgTMRMmTOCzzz47aN29997L2WefHaeIvMNDCbyhLfgWS+DGJJHZs2fHOwTP8lYVClhLFGOMcXkngedaAjfGmGDeSeCpGZBVYJ15jDHG5Z0EDtaZxxhjgngsgfeyBG6MxyT7eODTpk1jxowZhz320qVLufnmmwFnVMaGwbXC5bEE3tOqUIxJQB15PPApU6Zw5ZVXHvZ4xcXF/OlPfwIim8BbbUYoIn2AGUA3QIHpqvpHEbkL+D6ww93156o6NyJRNUNVkbxeULkbaishNTNapzLGW+bdAVtXRfaY3Y+Fc+857C42HrjjrrvuIicnh9tuu43TTjuNUaNG8dZbb7Fnzx4ee+wxxo0b1/jZH374YaZNm4bf7+fpp5/moYceYty4cYe9zocTSjvwOuAnqvqhiOQCy0RkvrvtQVW9v91nD9Ejb3/KOyU7ePbEoM48BYOifVpjTCtKSkp48sknKSsrY9asWSxZsgRV5fzzz2fBggVhDyfbMB74rbfeytVXX817771HVVUVw4YNY8qUKQeNB75z505OPPFExo8fz6JFixrHA9+2bRtDhw7l2muvbRwP/OWXX6aoqIiZM2cydepUHn/88UPOfc011zB37lyGDh3KAw88EHLMdXV1LFmyhLlz53L33Xfz+uuvN27r378/U6ZMaUz44Wo1gatqKVDqLpeLyDqgV9hnboPsdD8LP93FhuF5DAZL4MYEa6WkHE02HvihLrzwQgBOOOEENm3aFNbnb02b6sBFpD9wHPC+u+pGEVkpIo+LSOcW3nO9iCwVkaU7duxobpdWTTiuF9lpfl74xK1jsxuZxiSEpuOBL1++nOXLl7Nhwwauu+66g/ZNpPHAG+JctWoVr732Wov7N4wH/uKLL4Z8joY4/X5/u2Jsi5ATuIjkAC8Ct6hqGfAIMAgYiVNCb/Y3hqpOV9ViVS0uKipqV5C5GalceHxvnl/vXgy7kWlMQrHxwEOXm5tLeXl5RI4V0lgoIpKKk7yfUdWXAFR1W9D2R4FXIhJRCyaP6cdTiz+nKiOPDCuBG5NQbDzw0J133nlMnDiRl19+OeybmK2OBy7O7dcngd2qekvQ+h5u/TgiciswSlUvO9yxwhoPHLhs+iJ+U3oDg444Bpn0XLuPY4zX2Xjgyast44GHUgI/BZgMrBKR5e66nwOTRGQkTtPCTUDLk+FFyJVj+vPl3zvRdfsX5EX7ZMYYk+BCaYXyLtBcI8iotfluyZlDu/Gv1CJ0b/tL8caYxOKV8cB/+9vf8sILLxy07uKLL2bq1KlxishL44EDqX4f3XsPIv+L1/ls224GdOsS75CMiRtVPWwHE6/wynjgU6dOjXqybusUl97qSg8Mde8Gv/Luh3GOxJj4ycjIYNeuXW3+D28Sl6qya9cuMjIyQn6Pp0rgAHld+wGwdNVq9p93GllpnvsIxoStd+/ebN68mfb2rTCJKSMjg969e4e8v/eynzs3Zn7NDl5e/hWTTuob54CMib3U1FQGDBgQ7zBMnHmuCqVharXhefuYsehz+wlpjOmwvJfAM/IgLZdx3WpYV1rGss+/jndExhgTF95L4AD5vRiUsZfcjBRmLPo83tEYY0xceDOB5/UkpaKUiSf0Zt7qUnaUV8c7ImOMiTnPJnDKvmLy6H7U1ivPL/ki3hEZY0zMeTSB94LyrQzsks64IYU88/4X1NUnz3ROxhgTCo8m8J6AQsU2rhzTn61lVcxfu63VtxljTDLxaAI/MLXaN4/qSq9OmXYz0xjT4Xg0gTttwSnbgt8nXD66L4s27qJkW2QGSTfGGC/weAJ3Jna4tLgPaX4fTy22UrgxpuPwZgLP6ASpWY0JvCAnnW8P78FLH26hojq6c9AZY0yi8GYCF3FK4Xs3N66aPKYfFdV1zP5w82HeaIwxycObCRwa24I3GNmnE8f2yrfxUYwxHYaHE3ivgxK4iHDlmH6UbK9g8cbdcQzMGGNiw8MJvCeUl0KgvnHVeSN60ikrlacWb4pfXMYYEyPeTuBaDxXbG1dlpPq5tLgPr67ZRuneyjgGZ4wx0efhBH6gM0+wK0b3I6DKc+/b+CjGmOTm4QR+oDNPsD5dsvjGkV15dsmX1NTZ+CjGmOTl4QTuzhvXpAQOTpPCnRXV/HvN1hgHZYwxsePdBJ7VBfzph5TAAU4dUkS/giyeWrQp9nEZY0yMeDeBN3TmaaYE7vMJV4zqxwebvmZdaVkcgjPGmOjzbgKHQ9qCB7u4uDfpKT4bpdAYk7RaTeAi0kdE3hKRtSKyRkR+5K7vIiLzRaTEfe4c/XCbyOvZbBUKQKesNL4zshf/+GgLeytrYxyYMcZEXygl8DrgJ6o6FBgN/FBEhgJ3AG+o6hDgDfd1bDV25mm+tcnkMf2orK3nxWU2PooxJvm0msBVtVRVP3SXy4F1QC/gAuBJd7cnge9EKcaW5fWC+hrYv6vZzcN65XN83048tfhzAgEbH8UYk1zaVAcuIv2B44D3gW6qWupu2gp0a+E914vIUhFZumPHjnBiPVQLbcGDXTmmP5/t3Me7G3ZG9tzGGBNnISdwEckBXgRuUdWDmnaoM/xfs0VcVZ2uqsWqWlxUVBRWsIdoMrFDc849tjsF2Wl2M9MYk3RCSuAikoqTvJ9R1Zfc1dtEpIe7vQewvaX3R01jd/qWS+DpKX4mntCbt9Zvp7zKbmYaY5JHKK1QBHgMWKeqfwjaNAe4yl2+Cng58uG1IrsIfCmHTeAApx5RRH1AWfKZDTNrjEkeoZTATwEmA98UkeXu41vAPcCZIlICnOG+ji2fD3Kb78wT7Ph+nUlP8Vk9uDEmqaS0toOqvgtIC5tPj2w47dBCb8xgGal+ThrQhYUbmm+tYowxXuTtnphw2M48wU4eVMj6beVsL6+KQVDGGBN9SZLAv4JW5sEcO7gQwErhxpikkQQJvBfUVUHl14fdbWjPPDplpfKe1YMbY5KE9xN4futNCQH8PmHMwALe27DTZq03xiQF7yfwFqZWa84pgwv5am8Vn+3cF+WgjDEm+pIggbfenb5BQz34e59aPbgxxvu8n8BzuoH4QyqB9yvIolenTN4rsXpwY4z3eT+B+/yQ2z2kBC4inDK4gIWf7qTeRic0xnic9xM4hNwWHJx68LKqOtZ8tTfKQRljTHQlUQJvvQQOTocewLrVG2M8L0kSeC/Yu6XVzjwARbnpHNU919qDG2M8L0kSeE+o3QfVoc1Af8rgQj7Y9DVVtfVRDswYY6IneRI4hFyNcsrgAmrqAiz7/PC9N40xJpElSQJ3O/PsDe1G5kkDCkjxidWDG2M8LUkSeOideQBy0lM4rm8nFloCN8Z4WHIk8JzugIRchQJOa5SVW/ayd79Ns2aM8abkSOApaZDTNeQSOMDYIYWowqKNVgo3xnhTciRwaFNbcICRfTqRnebnPRsf3BjjUUmUwHu1KYGn+n2McoeXNcYYL+qwCRzg5EEFbNy5j6/2VEYpKGOMiZ4kSuA9oXovVJeH/JaxQ9zhZa0UbozxoCRK4A0TO5SG/JYju+VSmJNmCdwY40lJlMDb1hYcnOFlTx5UyHuf7rJp1owxnpOECbxt9eBjBxeyo7yaku0VUQjKGGOiJ3kSeG4P57mNCfwUtx78XZulxxjjMcmTwFMzIKuwTVUoAL06ZdK/IMvqwY0xnpM8CRza3JmnwSmDC1m8cRe19YEoBGWMMdHRagIXkcdFZLuIrA5ad5eIbBGR5e7jW9ENM0TtaAsOTj34vpp6Vm7eE/mYjDEmSkIpgT8BnNPM+gdVdaT7mBvZsNqpDXNjBhszqAAReLfEutUbY7yj1QSuqguA3TGIJXx5PaFyN9Tsb9PbOmWlMaxnvtWDG2M8JZw68BtFZKVbxdK5pZ1E5HoRWSoiS3fs2BHG6ULQ0JmnPPTOPA1OGVzIR19+zb7quggHZYwx0dHeBP4IMAgYCZQCD7S0o6pOV9ViVS0uKipq5+lC1I7OPA1OGVxAbb2yZJM3fmwYY0y7EriqblPVelUNAI8CJ0U2rHZq7E7f9huZJ/bvQlqKj/esPbgxxiPalcBFpEfQywnA6pb2jakwSuAZqX6K+3XmvU/tRqYxxhtCaUb4HLAIOFJENovIdcB9IrJKRFYC3wBujXKcoUnLgszO7SqBg1MPvq60jJ0V1REOzBhjIi+UViiTVLWHqqaqam9VfUxVJ6vqsao6XFXPV9W23zWMlna2BQcngQMstFK4McYDkqsnJrS7LTjAsb3yyc1IsdnqjTGekKQJvH0lcL9PGDOwgHdKdtrwssaYhJeECbwX7NsBde2rxx47pJAteyr5YnfbOgMZY0ysJWECd1uitKMzDxyoB7fZ6o0xiS55E3g7q1EGFmbTPS/DutUbYxJeEibw9nfmAWeatVMGF7Lw050EAlYPboxJXEmYwNvfmafB2CEFfL2/lrWlZREKyhhjIi/5Enh6LqTntbsEDnDyoIZ6cKtGMcYkruRL4BBWW3CAbnkZDOmaw7uWwI0xCSyJE3j7S+DgtEb5YNNuquvqIxSUMcZEVvIm8L3tL4GDk8CragN8+PmeyMRkjDERlqQJvBdUbIP62nYfYtTALvh9YvXgxpiElaQJvCegUL61/YfISGVE73ze+9QSuDEmMSVpAu/tPEegHnzFl3soq2p/Sd4YY6IlSRN4+G3BwUngAYXFNrysMSYBJXkCD68EflzfTmSm+m18cGNMQkrOBJ6RD6nZYSfw9BQ/Jw7oYu3BjTEJKTkTuEjYnXkajB1cwIbtFWzdWxWBwIwxJnKSM4FDRDrzQPA0a1YKN8YkliRO4O2fGzPY0d3z6JKdZtUoxpiEk8QJvKczqUMgvK7wPp8wZlABb6/fwZY9lREKzhhjwpfcCVzroWJ72Ie6YfxAausDTHxkIRu2l0cgOGOMCV8SJ/DwJnYINrx3J2ZeP4a6gDJx2iI++uLrsI9pjDHhSuIEHpnOPA2G9szjxSknk5eRyncffZ//fLIjIsc1xpj2SuIEHrkSeIO+BVnM+j9j6F+Yzfee/IA5KyJ3bGOMaavkTeBZXcCfHrESeIOuuRnMvGE0x/XtzI+e/4gZizZF9PjGGBOqVhO4iDwuIttFZHXQui4iMl9EStznztENsx0i2JmnqbyMVGZcexKnH9WNX728hj/M/wRVmwDZGBNboZTAnwDOabLuDuANVR0CvOG+TjwRagvenIxUP9OuOJ6LT+jNn94o4Zcvr6beZrE3xsRQqwlcVRcAu5usvgB40l1+EvhOZMOKkCiVwBuk+H3cN3E4N5w6kKcXf8HNz39kU7AZY2ImpZ3v66aqpe7yVqBbSzuKyPXA9QB9+/Zt5+naKb8XrCmFQAB80anuFxHuPPdoCrLT+J+5H7N3fy3TJp9ATnp7L60xxoQm7KymTuVvi3UHqjpdVYtVtbioqCjc07VNXi8I1ML+6HeDv378IH4/cTiLNu7i8kcXs3tfTdTPaYzp2NqbwLeJSA8A9zn87o7REOG24K25uLgP0644gY+3ljNx2kLrem+Miar2JvA5wFXu8lXAy5EJJ8IiNLFDW5w5tBtPXTeKHeXV1vXeGBNVoTQjfA5YBBwpIptF5DrgHuBMESkBznBfJ54odOYJxUkDuljXe2NM1IXSCmWSqvZQ1VRV7a2qj6nqLlU9XVWHqOoZqtq0lUpiyCoEX2rMqlCCNXS9z8+0rvfGmOhI3p6Y4LQ8yesR8xJ4g74FWbwwxel6f83flvDQGyXWVtwYEzHJncAhqp15QtE1N4O/3zCa80b05IH5n3D5Xxfb9GzGmIjoAAk8up15QpGbkcr/XjqS308czoov93LuHxfw+tptcY3JGON9HSSBfwVxHqtERLi4uA+v3DyWHvmZfG/GUu6as8Z6bhpj2q0DJPBeUFcFlYnREmRQUQ6zf3gy15zSnycWbmLCnxfy6Y6KeIdljPGgDpDAY9uZJxTpKX5+fd4xPHZVMaV7KznvoXd5YemXNqKhMaZNOkACj09b8FCcfnQ35v1oPMN753P7rJXcMnM55VW18Q7LGOMRHSCBuyXwvZvjG0cLuudn8Mz3RvOTM4/gnyu+4tsPvcuKL/fEOyxjjAckfwLP6QbiT8gSeAO/T7jp9CH8/YYx1NUrFz2ykOkLPiVgbcaNMYeR/Anc54fc7gmdwBsU9+/C3JvHccbR3fifuR9z9RMfsKO8Ot5hGWMSVPIncHA78yTOTczDyc9K5ZErjuc33xnG+xt3ce4f3+GdEuuGb4w5VAdJ4D09UQJvICJcMbofc24cS+esVCY/toT7X10f77CMMQmmYyTwoiNh1wZYPy/ekbTJkd1zmXPjWCae0JuH39rAvFWlrb/JGNNhdIwEfsqPoOdImHUtbPkw3tG0SWaan3suPJZhvfL45ctr2LvfmhkaYxwdI4GnZcN3/w7ZhfDspbDni3hH1CYpfh/3XjScr/fX8Jt/rY13OMaYBNExEjhATle4fBbUV8MzF0PlnnhH1CbH9MznhvEDeWHZZt4tif4cn8aYxNdxEjg4deGXPgO7PoWZV0CdtyYevvn0IQwszOaOl1ayv6Yu3uEYY+KsYyVwgAHj4II/w6Z3YM5NcR+lsC0yUv387sJj2fx1JQ+89km8wzHGxFnHS+AAIy6Fb0yFlc/D24k5nWdLRg0s4IrRffnbe5/ZXJvGdHAdM4EDjL8dRl4B/7kHPnom3tG0yc/OOYpueRnc8eIqauoC8Q7HGBMnHTeBi8B5/wsDT4N/3gwb345zQKHLzUjltxOGsX5bOY+8/Wm8wzHGxEnHTeAA/lS4ZAYUHgEzJ8M27zTR++ZR3Th/RE8efquET7aVxzscY0wcdOwEDpCRD5e/AKlZTvPCMu/0dvz1eUPJSU/hZy+utNnujemALIED5PeGy//uTLv23KVQ7Y0pzgpy0vn1ecfw0Rd7eHLhpniHY4yJMUvgDXqMgIufgK2rnS739d5oZ33ByJ6cdmQRv391PV/u3h/vcIwxMWQJPNgRZ8F/3Q8lr8K8n3qijbiI8NsJx+IT+PnsVTavpjEdSFgJXEQ2icgqEVkuIksjFVRcFV/rDH619DFY+FC8owlJr06Z/Ozco3inZCcvfuiNcc+NMeGLRAn8G6o6UlWLI3CsxHD6XXDMBJj/S1gzO97RhOSKUf0o7teZ//vKWpvFx5gOwqpQmuPzwXemQZ/R8NIN8MX78Y6oVT6fcM9Fw6msqeeuOWviHY4xJgbCTeAKvCYiy0Tk+uZ2EJHrRWSpiCzdscNDU4OlZsBlzzotVJ67zBkAK8EN7prDzacP5l+rSnl1zdZ4h2OMibJwE/hYVT0eOBf4oYiMb7qDqk5X1WJVLS4qKgrzdDGWXeC0EReBpy+C8m3xjqhVN5w6iKO65/LLf6xmb6VN/mBMMgsrgavqFvd5OzAbOCkSQSWUgkEwaSZUbIOnL3TaiiewVL+P+yYOZ2dFNffMWxfvcIwxUdTuBC4i2SKS27AMnAWsjlRgCaXPiXDZM7DzE3jmkoTv6DO8dye+P24gzy35koWf2uQPxiSrcErg3YB3RWQFsAT4l6r+OzJhJaBB34SJj8OWpTDzcqhL7JYet5xxBP0KsrjzpVVU1tTHOxxjTBS0O4Gr6kZVHeE+jlHV30YysIR09HnOZBAb30743pqZac7kD5/v2s//vm6TPxiTjKwZYVuN/C6ccy98/Iozo08gccfjPnlQIZNO6sOj72xk5eY98Q7HGBNhlsDbY/QUZ0afFc/Cv+9I6C73d5x7NIU56fx01kqqaq0qxZhkYgm8vcbfDmNuhCV/gbd/F+9oWpSfmcq9Fw1n/bZyfvbiShsrxZgkYgm8vUTgrN/AcZPhP/fCwofjHVGLvnFUV24760heXv4Vj/wn8TskGWNCkxLvADxNBM77I1SXw2tTISMPjr8y3lE16wenDeLjreX8/tX1HNE1lzOGdot3SMaYMFkJPFw+P1z4KAw+A/75o4Qd/EpEuO+i4Qzrmc+Pnv/IpmEzJglYAo+ElDS45CnoMwpe/D6UvB7viJqVmeZn+pUnkJWewveeXMrX+2riHZIxJgyWwCMlLQu+OxO6Hg0zr4DPF8U7omb1yM/kL5NPYGtZFT945kNq6xO3GaQx5vAsgUdSRj5c8RLk94JnL4Gvlsc7omYd37czv5twLIs27uK//7k23uEYY9rJEnik5RTBlS87yfzpC2FHYvaCvOiE3lw/fiBPLf6cpxd/Hu9wjDHtYAk8GvJ7w+R/gPjgqe/Ani/iHVGzfnbOUXzjyCLumrOGRZ/uinc4xpg2sgQeLYWDYfJsZ+TCGRdAxfZ4R3QIv0/446Tj6FeQxQ+eWWaz2hvjMZbAo6n7sc6EEOVb4Ylvw5YP4x3RIfIyUvnrVScSUPjek0upqE7cAbqMMQezBB5tfUc5rVOq9sBfT4d5P4OqsnhHdZABhdn8+bvHs2FHBbfOXE4gYN3tjfECS+CxMGA83PgBFF8L7/8F/nwSrJ2TUINgjR1SyC/+62jmr93GH+Yn5o1XY8zBLIHHSkY+/NcD8L3XIasQ/j7ZmSw5gW5wXn1yfy47sQ8Pv7WBOSu+inc4xphWWAKPtd7FcP3bzkBYny2AP4+C9/4E9fGfgFhE+O8LhnFi/87c/sIKVm3eG++QjDGHYQk8HvwpcPJN8MMlMOBUmP9LmH4afPlBvCMjLcXHI1ecQGFOOt+fsZTtZVXxDskY0wJL4PHUqQ9Meg4ufRr274bHzoRXfgyVe+IaVmFOOo9eWczeylquf2qZTQRhTIKyBB5vIs5cmzcugVFTYNnfnJucq1+M603OoT3zePDSESz/cg8/n73KJoIwJgFZAk8U6blw7j3w/Tcht4czafIzE2H3Z3EL6ZxhPbj1jCN46cMtPPzmBiuJG5NgJJYlq+LiYl26dGnMzudZgXr44K/wxv+FQC2c+jOnztyfGvNQVJUbn/2If60qJc3vY0SffEYNKOCkAV04vl9nctJtThBjok1Elqlq8SHrLYEnsLKvnI4/6+ZATncoGAS53Z3l3KBHw+v0XKdKJsLq6gP855MdLPlsN4s/283qLXupDyh+nzCsZx4nDejCqAEFnNi/C/lZsf8jY0yyswTuZZ+8Ciued7rkV2x1nmubGbckNavlBJ9dCGnZkJrp7Bf8nJLRpsS/r7qOD7/4miWf7eb9jbtZ/uUeauoDiMCR3XIZNaALowY6Cb0oNz2CF8KYjskSeDJRdebhDE7oDY+mr2v3hXbMpkk9NRNSMg+sS8tySvhpOZCe5yyn50B6LjX+bEr2wEfb6lnyVS2LttSwqzaVAD4GFmUzakAXRvbpRFFuOp2y0uiSlUbnrDRyM1Lw+SL/i8GYZGMJvKNqSPT7dkJdJdQ2PPa38NzcPpVQU+E8qiuc44Sgxp/FfjL5uj6dskAGVaRRpWlUkk4VqVSTTiAlE0nNRNKySEnLxJ+RTVpGNumZ2aRn5pCVnUNWdi7ZObmkZuaSlplDWlYu6Zm5pKZadY3pGFpK4HYHKtml5zqPwiGRO2Z9rfOHoabCea4udxJ7ddlB69PcR151OdUVe6ir2U+gphJq9kJdJb66KlIClaTUVJNa0/aeqFWayn4yqJRMqiSdasmk2pdJjfuoTcmi3p9BXUo2AX8m9SmZ1PkzCPgzCaRkEkjNRN1lTXX+kGhqlvMHJSWDlBQfqX4hxec8+30+UnyC3yekuOv9Pmlcl+o/+HXDPik+weeu84vg84FPGpbtF4hpv7ASuIicA/wR8AN/VdV7IhKVSWz+VMjq4jxC4AMyW9spUN9Y2tfa/ezfV0FZeRkVFRXs21dG9b4KtKYCrdkPtfuQmn346vbjq92Pv24/KXX7SamvJLe+ktTAdtJqq0ivqSQjUEUGVfho2y/NgAqVNPxacH451OKnlhRqSaEOP9XqPDvr3Gc9sFzn7luLn1pNoQ4fdfipx08tfupw1gUkhQB+6iWFgPid1w3PPmebSgqKoor7SdR5NLxWddZow7bGzQfa8IuPgPhQ8YP7rOID8YP4CYgP8fkbtyN+8Dn7HNhXCOBz/lVFkIZ7JyIIB26lOMvOOnczIoJPnD9e0vjsrBOavHa3N+zb3H7OPiA4fxTh0PfCgfMFx4IbX0OswQsNUQd/lsbPQNCxgt7fdFtDzAS999vH9qRvQVbI38FQtDuBi4gf+DNwJrAZ+EBE5qiqTbJo2s7nd+vUcxAguzNkR+rYqs4fh7oqqNl3UPWQ1u4nULOf+mrnOVCzj0C1sz1Qs5+02krSaveTW1uJ1Nei9TUQqEXqa50mnvW1SGAfUl+LBOqQQA0SqMMXqEXUefZpHT5tRxt6BcJpei9NnoOPG6Ga0wA+AgiKDxVBEQL43XWCyoHtTUOQoOrb4HCE5tcH/V0KWiUHrTv0NaAHjtiwdOCSaLPrpcn6hmOr+9z4+YKXtfntDY9dKffTd+y3ml7CsIRTAj8J2KCqGwFE5HngAsASuEksIs5N2LSsQ341CM7PR3+0YwgEIFDnJP1AHdQHL9c6v0DcPwjOfnVB29ztTTPxIS2Hmm4PWlZAA6D1zrEanwNNXgetP2Sbu0618Vg+DeBrXN+wvwbt655TA4fG1+xnaOGzHPRBGhb14HVNXx+y7sCvhYPOEerrhs+FHrSs7rI2fu4D6w7ad2CvFj5T+4WTwHsBXwa93gyMarqTiFwPXA/Qt2/fME5njIf5fOBLA9LiHYmJsJZ+6MRC1LvSq+p0VS1W1eKioqJon84YYzqMcBL4FqBP0Ove7jpjjDExEE4C/wAYIiIDRCQNuAyYE5mwjDHGtKbddeCqWiciNwKv4twDelxV10QsMmOMMYcVVjtwVZ0LzI1QLMYYY9rAxgM3xhiPsgRujDEeZQncGGM8KqajEYrIDuDzdr69ENgZwXAizeILj8UXHosvfIkcYz9VPaQjTUwTeDhEZGlzwykmCosvPBZfeCy+8HkhxqasCsUYYzzKErgxxniUlxL49HgH0AqLLzwWX3gsvvB5IcaDeKYO3BhjzMG8VAI3xhgTxBK4McZ4VMIlcBE5R0TWi8gGEbmjme3pIjLT3f6+iPSPYWx9ROQtEVkrImtE5EfN7HOaiOwVkeXu41exis89/yYRWeWee2kz20VE/uRev5UicnwMYzsy6LosF5EyEbmlyT4xvX4i8riIbBeR1UHruojIfBEpcZ87t/Deq9x9SkTkqhjG93sR+dj995stIp1aeO9hvwtRjO8uEdkS9G/Y7Dxirf1fj2J8M4Ni2yQiy1t4b9SvX9jUnf4nER44oxp+CgzEmbpkBTC0yT4/AKa5y5cBM2MYXw/geHc5F/ikmfhOA16J4zXcBBQeZvu3gHk4E4iMBt6P47/1VpwOCnG7fsB44HhgddC6+4A73OU7gHubeV8XYKP73Nld7hyj+M4CUtzle5uLL5TvQhTjuwu4LYR//8P+X49WfE22PwD8Kl7XL9xHopXAG+fZVNUaoGGezWAXAE+6y7OA00VanFgvolS1VFU/dJfLgXU4U8t5yQXADHUsBjqJSI84xHE68KmqtrdnbkSo6gJgd5PVwd+xJ4HvNPPWs4H5qrpbVb8G5gPnxCI+VX1NVevcl4txJlOJixauXyhC+b8etsPF5+aNS4DnIn3eWEm0BN7cPJtNE2TjPu6XeC9QEJPogrhVN8cB7zezeYyIrBCReSJyTGwjQ4HXRGSZOx9pU6Fc41i4jJb/48Tz+gF0U9VSd3kr0K2ZfRLlOl6L84uqOa19F6LpRreK5/EWqqAS4fqNA7apakkL2+N5/UKSaAncE0QkB3gRuEVVy5ps/hCnWmAE8BDwjxiHN1ZVjwfOBX4oIuNjfP5WuTM4nQ+80MzmeF+/g6jzWzoh29qKyFSgDnimhV3i9V14BBgEjARKcaopEtEkDl/6Tvj/S4mWwEOZZ7NxHxFJAfKBXTGJzjlnKk7yfkZVX2q6XVXLVLXCXZ4LpIpIYaziU9Ut7vN2YDbOT9VgiTCX6bnAh6q6remGeF8/17aGaiX3eXsz+8T1OorI1cC3gcvdPzKHCOG7EBWquk1V61U1ADzawnnjff1SgAuBmS3tE6/r1xaJlsBDmWdzDtBwx38i8GZLX+BIc+vMHgPWqeofWtine0OdvIichHONY/IHRkSyRSS3YRnnZtfqJrvNAa50W6OMBvYGVRfESosln3hevyDB37GrgJeb2edV4CwR6exWEZzlros6ETkH+Clwvqrub2GfUL4L0Yov+J7KhBbOG+85dc8APlbVzc1tjOf1a5N430Vt+sBpJfEJzh3qqe66/8b5sgJk4Pz03gAsAQbGMLaxOD+nVwLL3ce3gCnAFHefG4E1OHfVFwMnxzC+ge55V7gxNFy/4PgE+LN7fVcBxTH+983GScj5Qevidv1w/pCUArU49bDX4dxTeQMoAV4Hurj7FgN/DXrvte73cANwTQzj24BTf9zwHWxoldUTmHu470KM4nvK/W6txEnKPZrG574+5P96LOJz1z/R8J0L2jfm1y/ch3WlN8YYj0q0KhRjjDEhsgRujDEeZQncGGM8yhK4McZ4lCVwY4zxKEvgxhjjUZbAjTHGo/4/ByaALBN6J1oAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 测试误差\n",
    "for i, name in enumerate(name_l):\n",
    "    plt.plot(list(range(num_epochs)), test_l[i], label=name)\n",
    "plt.legend(loc = 1)\n",
    "plt.title('mse_test')\n",
    "\n",
    "\n",
    "# 我们发现，使用HE初始化之后模型收敛速度明显提升。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.2 PyTorch实现HE初始化的正态分布参数创建\n",
    "\n",
    "使用`torch.nn.init.kaiming_normal_`来进行满足正态分布的初始化参数设置\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;31mSignature:\u001b[0m\n",
      "\u001b[0mnn\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0minit\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mkaiming_normal_\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0mtensor\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mTensor\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0ma\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mfloat\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;36m0\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0mmode\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mstr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m'fan_in'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0mnonlinearity\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mstr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m'leaky_relu'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mDocstring:\u001b[0m\n",
      "Fills the input `Tensor` with values according to the method\n",
      "described in `Delving deep into rectifiers: Surpassing human-level\n",
      "performance on ImageNet classification` - He, K. et al. (2015), using a\n",
      "normal distribution. The resulting tensor will have values sampled from\n",
      ":math:`\\mathcal{N}(0, \\text{std}^2)` where\n",
      "\n",
      ".. math::\n",
      "    \\text{std} = \\frac{\\text{gain}}{\\sqrt{\\text{fan\\_mode}}}\n",
      "\n",
      "Also known as He initialization.\n",
      "\n",
      "Args:\n",
      "    tensor: an n-dimensional `torch.Tensor`\n",
      "    a: the negative slope of the rectifier used after this layer (only\n",
      "        used with ``'leaky_relu'``)\n",
      "    mode: either ``'fan_in'`` (default) or ``'fan_out'``. Choosing ``'fan_in'``\n",
      "        preserves the magnitude of the variance of the weights in the\n",
      "        forward pass. Choosing ``'fan_out'`` preserves the magnitudes in the\n",
      "        backwards pass.\n",
      "    nonlinearity: the non-linear function (`nn.functional` name),\n",
      "        recommended to use only with ``'relu'`` or ``'leaky_relu'`` (default).\n",
      "\n",
      "Examples:\n",
      "    >>> w = torch.empty(3, 5)\n",
      "    >>> nn.init.kaiming_normal_(w, mode='fan_out', nonlinearity='relu')\n",
      "\u001b[1;31mFile:\u001b[0m      c:\\users\\hp\\python\\python310\\lib\\site-packages\\torch\\nn\\init.py\n",
      "\u001b[1;31mType:\u001b[0m      function\n"
     ]
    }
   ],
   "source": [
    "nn.init.kaiming_normal_?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "实际上，PyTorch中所使用的HE初始化时，初始参数的方差为：\n",
    "\n",
    "$$Var(w) = \\frac{2}{(1+a^2)fan_{in}}$$\n",
    "\n",
    "其中a为ReLU变种激活函数的修正系数，需要配合ReLU变种激活函数共同使用，扇入神经元个数和扇出神经元个数等效，并且$$std(w) = \\sqrt{\\frac{2}{(1+a^2)fan_{in}}}$$\n",
    "\n",
    "\n",
    "- 简单实践"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x2d53b60efd0>"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[ 0.4132,  1.2154],\n",
       "        [-0.0425, -1.3647],\n",
       "        [-0.5896,  2.1108],\n",
       "        [ 0.2548,  0.1661]], requires_grad=True)"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[ 0.5093, -0.2730, -0.1495,  1.4481],\n",
       "        [ 1.5209,  1.1812,  0.8868, -0.7768],\n",
       "        [ 0.6132, -0.8655, -0.3035, -0.1526],\n",
       "        [ 0.2752, -0.5203, -0.7091, -0.5119]], requires_grad=True)"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[ 0.7865, -0.0605,  0.6548, -0.9418],\n",
       "        [-1.1811,  0.8337,  0.5161, -0.8076],\n",
       "        [ 0.6926,  0.9038, -0.4330, -0.1764],\n",
       "        [-0.4042,  0.3451, -0.5829, -0.0914]], requires_grad=True)"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([[-0.7091,  0.1566, -0.7193, -0.9234]], requires_grad=True)"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设置随机数种子\n",
    "torch.manual_seed(420)  \n",
    "\n",
    "# 实例化模型\n",
    "relu_model3 = ReLU_class3()                   # 保留原参数\n",
    "\n",
    "relu_model3_init = ReLU_class3()              # 使用HE初始化参数\n",
    "# 修改init模型初始参数\n",
    "for m in relu_model3_init.modules():\n",
    "    if isinstance(m, nn.Linear):\n",
    "        nn.init.kaiming_normal_(m.weight)\n",
    "        \n",
    "# 创建模型容器\n",
    "model_l = [relu_model3, relu_model3_init]           \n",
    "name_l = ['relu_model3', 'relu_model3_init']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_l, test_l = model_comparison(model_l = model_l, \n",
    "                                   name_l = name_l, \n",
    "                                   train_data = train_loader,\n",
    "                                   test_data = test_loader,\n",
    "                                   num_epochs = num_epochs, \n",
    "                                   criterion = nn.MSELoss(), \n",
    "                                   optimizer = optim.SGD, \n",
    "                                   lr = lr, \n",
    "                                   cla = False, \n",
    "                                   eva = mse_cal)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d549b48670>]"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d549b48a30>]"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x2d549c55ed0>"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'mse_train')"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAArFklEQVR4nO3deXxU5d3//9cnO0vCGvawqBR3QCOuWGtVrF0UpSht3e9abyuttdq7fu2vau/b76/1trWtVi1Wq61WUSo3aPWuG4oLLqEssqgs0goGCCBLgOyf7x/nJExCliEzyWRm3s/HYx5z5pzrnPOZk8knV665znWZuyMiIsknI9EBiIhI+yiBi4gkKSVwEZEkpQQuIpKklMBFRJKUEriISJJSAhcRSVJK4CJxZmbDzazczDITHYukNiVwkQhmdpmZvRHLMdz9X+7e091r4xWXSHOUwEUOkGrW0lUogUuXZGbrzOxGM1tqZrvN7EEzG2hmz5vZLjN7ycz6mFmemT1qZlvNbLuZvWdmA8Nj9Ar3KzWzDWb2X60lXzM7DLgfODFsAtkern/YzO4zs+fMbDfwBTP7spktMrOdZvaJmd0acZyRZuZmlhW+ftXM/tPM3gxjf8HM+nfg5ZM0oQQuXdkFwJnA54CvAs8D/wcoJPjsfg+4FOgFFAH9gKuBveH+DwM1wCHAeOAs4N9aOpm7rwz3XxA2gfSO2PwN4HYgH3gD2A1cAvQGvgz8u5md18p7+QZwOTAAyAFuaOvNi7RFCVy6srvdfZO7bwBeB95x90XuXgHMJkjK1QSJ+xB3r3X3he6+M6yFnwNc5+673X0zcBdwUTtjmePub7p7nbtXuPur7v5++Hop8Djw+Vb2/6O7f+Tue4EngXHtjEOkQVaiAxBpxaaI5b3NvO4J/Jmg9v2EmfUGHgVuBkYA2UCpmdXvkwF80s5YGu1nZscDPweOJKhR5wJPtbL/xojlPWHsIjFRDVySmrtXu/tt7n44cBLwFYKmjU+ASqC/u/cOHwXufkRbh4xy/V+AuUCRu/ciaDu3/fYS6UBK4JLUzOwLZnZU+OXkToImlTp3LwVeAH5pZgVmlmFmB5tZa80cENTyh5lZThvl8oFt7l5hZhMI2rhFOpUSuCS7QcAsguS9EniNoFkFgpp4DrAC+CwsN7iN470CLAc2mtmWVspdA/zMzHYBPyVo1xbpVKYZeUREkpNq4CIiSUoJXNKOmd0f3qjT9HF/omMTORBRN6GEXxKVABvc/StmNgp4gqAP7kLgYnev6rBIRUSkkQNJ4NcDxUBBmMCfBJ529yfCmssSd7+vtWP079/fR44cGWvMIiJpZeHChVvcvbDp+qhu5DGzYQS3C98OXG/BnRGns6/r1CPArUCrCXzkyJGUlJQcQNgiImJm/2xufbRt4L8GfgTUha/7AdvdvSZ8vR4Y2sKJrzKzEjMrKSsriz5iERFpVZsJ3My+Amx294XtOYG7z3D3YncvLizc7z8AERFpp2iaUE4GvmZm5wB5QAHwG6C3mWWFtfBhwIaOC1NERJpqM4G7+03ATQBmdhpwg7t/08yeAqYQ9ES5FJjTcWGKSKTq6mrWr19PRUVFokOROMrLy2PYsGFkZ2dHVT6W0Qj/g2AEuP8CFgEPxnAsETkA69evJz8/n5EjRxIx2qIkMXdn69atrF+/nlGjRkW1zwElcHd/FXg1XF4LTDjAGEUkDioqKpS8U4yZ0a9fPw6ks4fuxBRJUkreqedAf6bJkcA//F94/VeJjkJEpEtJjgS+9lWYfydo5EQRkQbJkcALhkD1bqjcmehIROQA9eyZ+NnjLrvsMmbNmhV1mSuvvJKxY8dy9NFHM2XKFMrLyzsjzAOWPAkcYGdpYuMQkWa5O3V1dW0XTBJ33XUXS5YsYenSpQwfPpx77rkn0SE1KzkmNS4I79LfuQEGHJrYWES6mNueWc6KT+P73+nhQwq45autTx+6bt06Jk2axPHHH8/ChQuZOnUqzz77LJWVlUyePJnbbrutUflXX32VO++8k2effRaAa6+9luLiYi677LJmjz9y5EimTZvG888/T1ZWFjNmzOCmm25i9erV3HjjjVx99dW4Oz/60Y94/vnnMTN+8pOfcOGFF+LuTJ8+nRdffJGioiJycvbNkLdw4UKuv/56ysvL6d+/Pw8//DCDBzeeqKmgoAAI/jDt3bu3y35hnGQ18E8TG4eINLJq1SquueYa7rrrLjZs2MC7777L4sWLWbhwIfPnz4/5+MOHD2fx4sVMnDixoYnj7bff5pZbbgHg6aefZvHixSxZsoSXXnqJG2+8kdLSUmbPns2HH37IihUr+NOf/sRbb70FBDdATZ8+nVmzZrFw4UKuuOIKbr755mbPffnllzNo0CA++OADpk+fHvN76QjJUQPPD/86KoGL7KetmnJHGjFiBCeccAI33HADL7zwAuPHjwegvLycVatWceqpp8Z0/K997WsAHHXUUZSXl5Ofn09+fj65ubls376dN954g2nTppGZmcnAgQP5/Oc/z3vvvcf8+fMb1g8ZMoTTTz8dgA8//JBly5Zx5plnAlBbW7tf7bveH//4R2pra5k+fTozZ87k8ssvj+m9dITkSOBZOdCjMGhCEZEuo0ePHkDQ1HDTTTfxne98p8WyWVlZjdrJoxkGIDc3F4CMjIyG5frXNTU1Le3WInfniCOOYMGCBVGVz8zM5KKLLuKOO+7okgk8OZpQIGhGUQ1cpEuaNGkSDz30UENvjQ0bNrB58+ZGZUaMGMGKFSuorKxk+/btvPzyyzGfd+LEicycOZPa2lrKysqYP38+EyZM4NRTT21YX1payrx58wAYM2YMZWVlDQm8urqa5cuXNzqmu7N69eqG5blz53LooV3zu7fkqIFD8EXmZ82OaS4iCXbWWWexcuVKTjzxRCDoOvjoo48yYMCAhjJFRUVMnTqVI488klGjRjU0t8Ri8uTJLFiwgLFjx2Jm3HHHHQwaNIjJkyfzyiuvcPjhhzN8+PCGuHJycpg1axbf+9732LFjBzU1NVx33XUcccS+Zih359JLL2Xnzp24O2PHjuW++1qdqyZhop5SLR6Ki4u93TPy/O2HsOyv8B/r4hqTSDJauXIlhx12WKLDkA7Q3M/WzBa6e3HTssnVhLL3M6jak+hIRES6hORpQskPuxLuKoV+Byc2FhGJm8mTJ/Pxxx83WveLX/yCSZMmJSii5JE8CbyhL/gGJXCRFDJ79uxEh5C0kqgJpf5uTPVEERGB6CY1zjOzd81siZktN7PbwvUPm9nHZrY4fIzr0EgL6m/mUV9wERGIrgmlEjjd3cvNLBt4w8yeD7fd6O6tD/EVLzk9IK+3BrQSEQlFM6mxA/VjKWaHj8QMzF0wVE0oIiKhqNrAzSzTzBYDm4EX3f2dcNPtZrbUzO4ys9yWjxAnBUPUhCKSZFJ9PPD777+fP/3pT60eu6SkhO9973tAMCpj/eBasYoqgbt7rbuPA4YBE8zsSOAm4FDgOKAvwSz1+zGzq8ysxMxKDmSyzmYVDFYNXKQLSufxwK+++mouueSSVo9XXFzMb3/7WyC+CfxAZ6XfbmbzgLPd/c5wdaWZ/RG4oYV9ZgAzILgTM5ZgKRgKuzdDTVUwwJWIwPM/ho3vx/eYg46CL/281SIaDzxw66230rNnT2644QZOO+00jj/+eObNm8f27dt58MEHmThxYsN7v+eee7j//vvJzMzk0Ucf5e6772bixImtXufWRNMLpdDMeofL3YAzgQ/MbHC4zoDzgGXtjiJaBRE384hIwmk88P3V1NTw7rvv8utf/3q/P2IjR47k6quv5gc/+EHD+4pFNDXwwcAjZpZJkPCfdPdnzewVMysEDFgMXB1TJNGInNihz4gOP51IUmijptyRNB74/s4//3wAjj32WNatWxfT+29LNL1QlgL7DRvm7qd3SEStqb+ZZ5fawUW6Ao0H3nLMmZmZ7YrxQCTPnZigqdVEuiiNBx69/Px8du3aFZdjJc9YKAC5BZDTUwlcpIvReODR++pXv8qUKVOYM2dOzF9iJs944PXuLoaBh8PU1vtdiqQyjQeeulJzPPB6mlpNRARItiYUCL7I/Pi1REchInGSLOOB33777Tz11FON1n39619vsRtiZ0jCBD4Edm2EulrIyEx0NCIJ4+6t3mCSLJJlPPCbb765w5P1gTZpJ2cTitdC+ea2y4qkqLy8PLZu3XrAv/DSdbk7W7duJS8vL+p9krAGHjGxQ0HzHfBFUt2wYcNYv349MY8vJF1KXl4ew4YNi7p8EibwiKnVODahoYgkSnZ2NqNGjUp0GJJgydmEAuqJIiJpL/kSePd+kJmjccFFJO0lXwI3U19wERGSMYFD8EWmhpQVkTSXpAlcU6uJiCRxAv8U1AdWRNJYkibwoVBbBXu2JjoSEZGESc4Enh/ewKNmFBFJY9HMiZlnZu+a2RIzW25mt4XrR5nZO2a22sxmmlnnzTIceTemiEiaiqYGXgmc7u5jgXHA2WZ2AvAL4C53PwT4DLiyw6JsqtHdmCIi6anNBO6B8vBldvhw4HRgVrj+EYKZ6TtHzwFgmbBTXQlFJH1F1QZuZplmthjYDLwIrAG2u3v9jJ3rgaEt7HuVmZWYWUncBt7JyAzawdWEIiJpLKoE7u617j4OGAZMAKKe4dPdZ7h7sbsXFxYWti/K5qgvuIikuQPqheLu24F5wIlAbzOrH81wGNC52bRANXARSW/R9EIpNLPe4XI34ExgJUEinxIWuxSY00ExNq9gqG7mEZG0Fs144IOBR8wskyDhP+nuz5rZCuAJM/svYBHwYAfGub+CIVC9Gyp2QLfenXpqEZGuoM0E7u5LgfHNrF9L0B6eGJHjgiuBi0gaSs47MWHfzTy71A4uIukpiRO4ZuYRkfSWvAm85yDAlMBFJG0lbwLPyoEeheoLLiJpK3kTOGhqNRFJa0mewIcqgYtI2kryBK7b6UUkfSV/Aq/YAVW7Ex2JiEinS/IEXj+xg4aVFZH0k+QJXBM7iEj6SpEEri8yRST9JHcC1+TGIpLGkjuB53SHbn1UAxeRtJTcCRyCLzJ36UtMEUk/KZDA1RdcRNJTiiRwNaGISPpJgQQ+FHaXQU1loiMREelU0cyJWWRm88xshZktN7Pvh+tvNbMNZrY4fJzT8eE2o74nitrBRSTNRDMnZg3wQ3f/h5nlAwvN7MVw213ufmfHhReFyL7gfUYmNBQRkc4UzZyYpUBpuLzLzFYCQzs6sKg13E6vdnARSS8H1AZuZiMJJjh+J1x1rZktNbOHzKxPC/tcZWYlZlZSVlYWW7TN0d2YIpKmok7gZtYT+CtwnbvvBO4DDgbGEdTQf9ncfu4+w92L3b24sLAw9oibyiuAnHwlcBFJO1ElcDPLJkjej7n70wDuvsnda929DngAmNBxYbZBfcFFJA1F0wvFgAeBle7+q4j1gyOKTQaWxT+8KKkvuIikoWh6oZwMXAy8b2aLw3X/B5hmZuMAB9YB3+mA+KJTMATWzEvY6UVEEiGaXihvANbMpufiH047FQyB8o1QWwOZ0fxNEhFJfsl/JyYECdzroHxToiMREek0KZLAw77guhtTRNJIiiRwTa0mIuknRRK47sYUkfSTGgm8Wx/IzFUNXETSSmokcDP1BReRtJMaCRyCZhQlcBFJIymUwHU7vYikl9RK4Ls2Ql1doiMREekUSZHA13+2h2eXttE8UjAUaqtgz9bOCUpEJMGSIoHf9eIqrn9yCeu27G65kPqCi0iaSYoE/qOzx5CTmcGtzyzH3ZsvVBAOjqgvMkUkTSRFAh9YkMd1Z4zm1Q/LeHFFC+OdNNzMoxq4iKSHpEjgAJeeNJIxA/O57ZkV7K2q3b9Aj0LIyFINXETSRtIk8OzMDH527hFs2L6Xe19dvX+BjEzIH6wBrUQkbSRNAgc4/qB+TB4/lN+/tpaPm/tCU33BRSSNJFUCB7jpnEPJzcrgp3OW7f+Fpm6nF5E0Es2cmEVmNs/MVpjZcjP7fri+r5m9aGarwuc+HR8uDMjP4/qzPsfrq7bw9+UbG2+sv52+pZ4qIiIpJJoaeA3wQ3c/HDgB+K6ZHQ78GHjZ3UcDL4evO8XFJ4zg0EH5/OyZFeypqtm3IX8wVO+Biu2dFYqISMK0mcDdvdTd/xEu7wJWAkOBc4FHwmKPAOd1UIz7ycrM4D/PO5JPd1RwzysRX2g23MyjZhQRSX0H1AZuZiOB8cA7wEB3r+/ysREY2MI+V5lZiZmVlJWVxRJrI8eN7MsFxwzjgdfXsqasPFipiR1EJI1EncDNrCfwV+A6d98Zuc2DbxObbXh29xnuXuzuxYWFhTEF29SPv3QoedmZ3Do3vENTNXARSSNRJXAzyyZI3o+5+9Ph6k1mNjjcPhjY3DEhtqwwP5cbzhrD66u28PyyjZA/CDAlcBFJC9H0QjHgQWClu/8qYtNc4NJw+VJgTvzDa9u3ThjBEUMK+M9nV7C7xqDnQPUFF5G0EE0N/GTgYuB0M1scPs4Bfg6caWargDPC150uM8P42blHUrqjgrtfWa2+4CKSNrLaKuDubwDWwuYvxjec9jl2RB+mFg/jD6+v5doxhfTc+a9EhyQi0uGS7k7MlvzH2YfSPSeTNzfn4mpCEZE0kDIJvF/PXG48+1AWbe+OVe6EyvJEhyQi0qFSJoEDfGPCcLL6BH3Bd2/9JMHRiIh0rJRK4JkZxrkTjwNgzmvvJTgaEZGOlVIJHGD06DEALFq+nI827UpwNCIiHSflEjj5wd2YI7K3Nz/krIhIiki9BJ6dB936MmlYHW+v3cbcJeoTLiKpKfUSOEDBUA7J28HYYb24/W8r2VVRneiIRETiLkUT+BBsVyk/O/dIysor+c1LqxIdkYhI3KVsAmfnp4wt6s20CcP541vr+HCjvtAUkdSSogl8KOzZAtUV3HjWGArysrhl7rJERyUiElcpmsDDccF3ldKnRw7Xnj6at9duY/mnOxIbl4hIHKVoAh8cPIejEl5wzFBysjJ48j3dnSkiqSNFE3jjqdV6d89h0hGD+J/Fn1JRXZvAwERE4idFE3j91Gr7RiW86Lgiduyt5u/LNyYoKBGR+ErNBJ6bD7kFsKu0YdWJB/WjqG83nixRM4qIpIbUTOAQdiXcVwPPyDC+fmwRb67eyr+27klgYCIi8RHNnJgPmdlmM1sWse5WM9vQZIq1rqWZqdWmHDuMDIOnFqoWLiLJL5oa+MPA2c2sv8vdx4WP5+IbVhw0k8CH9O7GqZ8r5KmS9dTWaZArEUlubSZwd58PbOuEWOIrfwjs2gi1jcdBubC4iI07K5j/UVmCAhMRiY9Y2sCvNbOlYRNLn5YKmdlVZlZiZiVlZZ2YNAuGAA7lmxqt/uJhA+nXI4eZ6hMuIkmuvQn8PuBgYBxQCvyypYLuPsPdi929uLCwsJ2na4cmfcHr5WRlcP4xQ3lp5Sa2lFd2XjwiInHWrgTu7pvcvdbd64AHgAnxDSsOGvqC7z8e+IXHFVFT58z+h2avF5Hk1a4EbmaDI15OBrreSFGtJPBDBuRzzPDePPHevzRjj4gkrWi6ET4OLADGmNl6M7sSuMPM3jezpcAXgB90cJwHrlsfyOrWqC94pIuOG86ast3841+fdXJgIiLxkdVWAXef1szqBzsglvgya7YrYb0vHz2Y255Zzsz3PuHYEX07OTgRkdil7p2Y0GoC75GbxVeOHsKzS0spr6zp5MBERGKXtgkc4MIJReypquVZTXwsIkko9RP4rk+hrq7ZzeOLejN6QE9maoArEUlCKZ7Ah0JdTTC9WjPMjAuPK2LRv7bz0SbNmSkiySXFE/j+44I3NXn8ULIzTXdmikjSSZME3nIbd7+euZx5+EBmL9pAVU3zTS0iIl1Riifw5m+nb2pqcRHbdlfx0spNrZYTEelKUjuBd+8PGdmtNqEATBxdyJBeeTyhZhQRSSKpncAzMiB/cJs18MwMY0pxEa+vKmPD9r2dFJyISGxSO4FDm33B63392GEAzCpZ39ERiYjEhRJ4qKhvd04+uD9PlnxCnWbrEZEkkD4JPIpRBy88rogN2/fy5prm+42LiHQlaZDAh0LNXtjb9qiDZx0xkN7ds9UnXESSQhok8HDo8iiaUXKzMjlv3FBeWL6Jz3ZXdXBgIiKxSYMEHl1f8HoXHldEVW0dsxdpth4R6drSIIG3fTt9pMMGFzB2WC+eLPlEs/WISJeW+gm850CwDNhVGvUuU48r4oONu1i6fkcHBiYiEptoplR7yMw2m9myiHV9zexFM1sVPvfp2DBjkJkdJPEoa+AAXx07hLzsDN2ZKSJdWjQ18IeBs5us+zHwsruPBl4OX3ddUfYFbyiel82XjxrCM0s+ZU+VZusRka6pzQTu7vOBbU1Wnws8Ei4/ApwX37Di7AATOARfZpZX1vDc+xs7KCgRkdi0tw18oLvXNypvBAa2VNDMrjKzEjMrKSsra+fpYtT3INi6Gta9GfUux43sw0H9e/CkmlFEpIuK+UtMD7pqtNhdw91nuHuxuxcXFhbGerr2Ofm6IIk/Pg02LmuzOASz9Uw9roh3121jTVl5x8YnItIO7U3gm8xsMED4vDl+IXWA7n3hW09Dbk949AL47J9R7Xb+MUPJzDCe1JyZItIFtTeBzwUuDZcvBebEJ5wO1LsIvvVXqKmAR8+H3W2PdzIgP4/TDx3AXxduoLpWs/WISNcSTTfCx4EFwBgzW29mVwI/B840s1XAGeHrrm/AYfCNJ2HHBnjs61DZdtPIRccVsaW8klc+6Nr/ZIhI+ommF8o0dx/s7tnuPszdH3T3re7+RXcf7e5nuHvTXipd1/Dj4et/hNIl8OTFUNP6mCef/1whA/JzefTtf1KrYWZFpAtJ/TsxmzPmS/C138KaV2DONVDXcvNIVmYGl540ktdXbeErd7/BgjVbOzFQEZGWpWcCBxj/LfjiLfD+U/DCza2OF37NaQdzzzfGs3NvNdMeeJt/f3Qhn2zb04nBiojsLyvRASTUKT+A8s3w9r3B7fanXNdsMTPjK0cP4YzDBvLA/LXc++oaXv5gM9+eOIprTjuEHrnpfRlFJDHStwYOYAaT/i8cOQVeugUWPdZq8bzsTKZ/cTTzbjiNLx81mN/NW8MX7nyVWQvXaxo2Eel06Z3AIZi5/rz74KDTYO50+PB/29xlUK887rpwHE9fcxJDenfjhqeWMPneN1n4z7Zn/RERiRclcICsHLjwURh8NDx1Gfzrnah2O2Z4H57+95P41dSxbNxZwQX3vcV1TyyidMfejo1XRAQl8H1y8+EbTwUDX/1lKmz+IKrdMjKM848Zxis/PI1rv3AIzy3byOl3vsZvXlrF3qraDg5aRNKZEniknoVw8dOQlRvcrbljfdS79sjN4oZJY3j5+s9z+qEDuOuljzjjV6/xzJJPNbOPiHQIJfCm+owMbrmv3AV/Ph/2HNg9SkV9u/O7bx7DzKtOoFe3bKY/voipv1/A0vXbOyRcEUlfSuDNGXQUTHscPlsHf7kQqg68z/fxB/Xjmemn8PPzj+LjLbv52j1vcvkf36VkXfLctCoiXZt15r/3xcXFXlJS0mnni9mKufDkJTD6LLjosWB6tnbYWVHNnxf8kwff+Jhtu6uYMKov3/3CIZw6uj9mFuegRSTVmNlCdy/eb70SeBveexD+dj0cNRXOvSdoH2+nvVW1PPHev5gxfy2lOyo4YkgB3/3CIUw6YhCZGUrkItI8JfBYzP9veOW/gqaVCx6Cws/FdLiqmjr+Z9EG7n9tDWu37Oag/j24+rSDOW/cUHKy1KolIo0pgcfqw+fhf64JxhP/0i9g/MXBnZwxqK1z/nfZRn43bzUrSncypFce3z71IC46bjjdcjLjFLiIJDsl8HjYWQqzr4KP58MRk+Erv4ZuvWM+rLvz2kdl3DtvDe+u20a/HjlcccoovnXCCHp1a1+7u4ikDiXweKmrhTd/EzSpFAyFC/4QjDEeJ+9+vI17X13Nqx+WkZ+bxbdOHMEVJ4+iML/9be8iktyUwONtfQnMuiK42ee0m2Di9ZARv2aPZRt2cN9ra3ju/VJyMjO4+IQR3Hj2GHKz1LQikm46JIGb2TpgF1AL1DR3gkgplcABKnbAs9fDslkw4hQ4fwb0GhrXU6wpK+e+V9cwa+F6jhvZh99fXEzfHjlxPYeIdG0tJfB4dHn4gruPayt5p6S8XkETynn3waeL4P6TYeWzcT3FwYU9ufPrY7l72niWrt/Bub97g1WbdsX1HCKSnNRnLVZmMO4b8J350Hs4zPwm/O2HUB3fEQm/OnYIT1x1Anur6jj/3reY/1FZXI8vIskn1gTuwAtmttDMrmqugJldZWYlZlZSVpbCSaf/IXDlS3DitfDeH+CB02HzyrieYvzwPsy59mSG9unG5Q+/x58XrIvr8UUkucTaBj7U3TeY2QDgRWC6u89vqXzKtYG3ZPVLMPvqYECsSf8Xiq+Iuc94pPLKGr7/+CJe/mAzl500kp98+TCyMvXPlEiq6pA2cHffED5vBmYDE2I5Xso45Az497dgxMnBbfgzv3XAoxq2pmduFjMuKebbE0fx8FvruPKREnZWVMft+CKSHNqdwM2sh5nl1y8DZwHL4hVY0us5AL45C866HT76O9x3Mqx6KW6Hz8wwbv7y4fz/5x/Fm6u3MOW+t/hk24GPmigiySuWGvhA4A0zWwK8C/zN3dueUDKdZGTASdfCv70IeQXw2AUw57tB98M4mTZhOH+6YgIbd1Rw3u/eZOE/NVytSLrQjTydpaYSXv05vPlryB8MX/0tjD4jbodfW1bOlY+UsOGzvdwx5WjOGx/f/ugikjgd2Q9copGVC2fcAv/2UjD/Zpxr4wcV9mT2NSdx7Ig+XDdzMb984UPq6jSVm0gqUwLvbEOPhateg1N+AIv/AveeGPRaiYPe3XN45IoJXFhcxN2vrGb644s0sbJIClMCT4TsPDjj1qDfeE5PePQCmHNtXGrjOVkZ/PyCo7j5nMN4blkpF81YwOadFbHHLCJdjhJ4Ig07NriD85QfwOLH4lYbNzO+fepBzLi4mFWbyzn3d2+y/NP4fXEqIl2DEniidWBt/MzDBzLr6pMAuOC+t3jwjY+pVbu4SMpQAu8q6mvjJ18X19r44UMKmHPtyZx0cH/+89kVXPj7BawtK489XhFJOCXwriQ7D868Da58EXJ6BLXxudNjro0PyM/jwUuL+dXUsXy0aRdf+s3rPDB/rWrjIklOCbwrGlYM33kdTv4+LHo0qI1/9HeIbdwazj9mGC9d/3kmji7k9udWMuX+t1i9WbVxkWSlBN5VZefBmT/bVxv/y1T4/amw5AmoqWr3YQcU5PHAJcfym4vG8fGW3Zzz29f5/WtrVBsXSUK6EzMZVFfA0idgwb2w5UPoOQgmfDsY5bB733YfdvOuCn4yexkvrNjE2KLe3DnlaEYPzI9j4CISD5oTMxXU1cGaV2DBPbB2HmR1g3HT4IRroP/odh3S3XlmaSm3zFnG7sparjtzNFdNPEjD04p0IUrgqWbTcnj7Xlj6JNRWwehJcOJ3YdSp7Rp7vGxXJT+ds4znl23k6GG9+O8pYxkzSLVxka5ACTxVlW+G9x4MZgHaswUGHgUnXgNHXhCMv3KA/ra0lP9vzjLKK2r43hcP4TufP5hs1cZFEkoJPNVVV8D7Twbt5GUroedAOC5sJ+/R74AOtbW8kp/OXc7flpZy5NAC/nvKWA4bXNBBgYtIW5TA04V70E7+9r3BjUBZeTD2IjjmEuhVBN36QmZWVId6/v2gNr5jbzWXnzyK8UW9GdGvByP6dadHbnTHEJHYKYGno80fBIl8yRNQWxmuNOjWG7r3hx79oXu/8Ll/4+dweRv53PbcKuYs/rTRofv3zGVkv+4M79edkWFSH9GvByP7dad395xOf6siqaxDEriZnQ38BsgE/uDuP2+tvBJ4guzeAuteD553bwnayndvgT1b973esxW8rvn9c3tRl9ebKsumyjOpqMtkT20Gu2sy2FWdQXmNUU1Ww4PMHPLy8ujerRs9unWjZ4/u9OzejZzsHLKzs8nJySEnO5vsnFwyMrMgI/KR2fJrywgfmcEXtRmZTdaFyxnNrWu6nNl4OSMzrhNPi8RTSwm83f8Hm1km8DvgTGA98J6ZzXX3Fe0PUzpEj/5wxOTWy9TVQcX2Jgl+C+zeCnu2kLF3O3m1VeTVVlFQWx30fAmf62qqqK7aRU11JXXVlXhtFVRVk1FRTda2anKoIcOS40Yht0zcMoM/AmTgjf5IZOAWrA9eW5PniAfNbbM2yjS/3TLqly14hmaW2bfc8Do8XsMmaziONTw3d6wmyw2Hb+3czbxucVvEuv3Wt7atmT+wsf7R3a8C661sizxnC9e+tes47pvQ/5DY4m0ilobMCcBqd18LYGZPAOcCSuDJKCMjuCmoe1/gcwe2K5AbPpqqqqnjn9v3UrqtnD2VVeytqGRvZQV7KyupqKiiorKSiqpqKqsqqaysoqqqisrq4Lm6qpqqmuA5kxoycDJwjDoyqWt4ndGwXNewnGl1WLg9s2F9XcTyvvUNx7JgOZNg38yI8vXrgl/FiHNbEI9BRByR6/bFFezrZFBLBtXh8SKPE/E6PBYNr4NtQMNxAs6+NLl/mX3liNjWZB/b/7hN9298jv2P11DG6mPaf7+mcURub3vb/snUmlnX3DHry3rEFm+ytfGZWirX9Bo1LtnWz2dt3tGMOaXrJPChwCcRr9cDx8cWjqSanKwMRvXvwaj+Pdp9DHensqaOOndq65o83Kmpdercqalz6uqC58jttXVBGcfBoc7Bceoc6rx+XfDaw2citnt9Ofa9Dndr2N+pL7dv2cPYPTxucNR9FTuPeH+N329wDJopG7kuWL//vi1dw30xNY6R+vfO/mWIeB/7n7vxOZvG3PT8Le3X9H209Z73j6DpuVpe19p5mivDfmW8hX0ab28UXbhwzUEHNxtvLDq8K4GZXQVcBTB8+PCOPp2kIDMjLzsz0WGIdDmx3KGxASiKeD0sXNeIu89w92J3Ly4sLIzhdCIiEimWBP4eMNrMRplZDnARMDc+YYmISFva3YTi7jVmdi3wd4JuhA+5+/K4RSYiIq2KqQ3c3Z8DnotTLCIicgA0SpGISJJSAhcRSVJK4CIiSUoJXEQkSXXqaIRmVgb8s5279we2xDGceFN8sVF8sVF8sevKMY5w9/1upOnUBB4LMytpbjSurkLxxUbxxUbxxS4ZYmxKTSgiIklKCVxEJEklUwKfkegA2qD4YqP4YqP4YpcMMTaSNG3gIiLSWDLVwEVEJIISuIhIkupyCdzMzjazD81stZn9uJntuWY2M9z+jpmN7MTYisxsnpmtMLPlZvb9ZsqcZmY7zGxx+PhpZ8UXnn+dmb0fnnu/GaQt8Nvw+i01s2M6MbYxEddlsZntNLPrmpTp1OtnZg+Z2WYzWxaxrq+ZvWhmq8LnPi3se2lYZpWZXdqJ8f23mX0Q/vxmm1nvFvZt9bPQgfHdamYbIn6G57Swb6u/6x0Y38yI2NaZ2eIW9u3w6xezYKqlrvEgGJZ2DXAQkAMsAQ5vUuYa4P5w+SJgZifGNxg4JlzOBz5qJr7TgGcTeA3XAf1b2X4O8DzB1H0nAO8k8Ge9keAGhYRdP+BU4BhgWcS6O4Afh8s/Bn7RzH59gbXhc59wuU8nxXcWkBUu/6K5+KL5LHRgfLcCN0Tx82/1d72j4muy/ZfATxN1/WJ9dLUaeMNEye5eBdRPlBzpXOCRcHkW8EWzWKemjo67l7r7P8LlXcBKgrlBk8m5wJ888DbQ28wGJyCOLwJr3L29d+bGhbvPB7Y1WR35GXsEOK+ZXScBL7r7Nnf/DHgROLsz4nP3F9y9Jnz5NsFsWAnRwvWLRjS/6zFrLb4wb0wFHo/3eTtLV0vgzU2U3DRBNpQJP8Q7gH6dEl2EsOlmPPBOM5tPNLMlZva8mR3RuZHhwAtmtjCcj7SpaK5xZ7iIln9xEnn9AAa6e2m4vBEY2EyZrnIdryD4j6o5bX0WOtK1YRPPQy00QXWF6zcR2OTuq1rYnsjrF5WulsCTgpn1BP4KXOfuO5ts/gdBs8BY4G7gfzo5vFPc/RjgS8B3zezUTj5/m8Ip+L4GPNXM5kRfv0Y8+F+6S/a1NbObgRrgsRaKJOqzcB9wMDAOKCVopuiKptF67bvL/y51tQQezUTJDWXMLAvoBWztlOiCc2YTJO/H3P3pptvdfae7l4fLzwHZZta/s+Jz9w3h82ZgNsG/qpGimoy6g30J+Ie7b2q6IdHXL7SpvlkpfN7cTJmEXkczuwz4CvDN8I/MfqL4LHQId9/k7rXuXgc80MJ5E339soDzgZktlUnU9TsQXS2BRzNR8lyg/hv/KcArLX2A4y1sM3sQWOnuv2qhzKD6Nnkzm0BwjTvlD4yZ9TCz/Pplgi+7ljUpNhe4JOyNcgKwI6K5oLO0WPNJ5PWLEPkZuxSY00yZvwNnmVmfsIngrHBdhzOzs4EfAV9z9z0tlInms9BR8UV+pzK5hfMmelL0M4AP3H19cxsTef0OSKK/RW36IOgl8RHBN9Q3h+t+RvBhBcgj+Nd7NfAucFAnxnYKwb/TS4HF4eMc4Grg6rDMtcBygm/V3wZO6sT4DgrPuySMof76RcZnwO/C6/s+UNzJP98eBAm5V8S6hF0/gj8kpUA1QTvslQTfqbwMrAJeAvqGZYuBP0Tse0X4OVwNXN6J8a0maD+u/wzW98oaAjzX2mehk+L7c/jZWkqQlAc3jS98vd/vemfEF65/uP4zF1G2069frA/dSi8ikqS6WhOKiIhESQlcRCRJKYGLiCQpJXARkSSlBC4ikqSUwEVEkpQSuIhIkvp/hw6rz0kWaOYAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 训练误差\n",
    "for i, name in enumerate(name_l):\n",
    "    plt.plot(list(range(num_epochs)), train_l[i], label=name)\n",
    "plt.legend(loc = 1)\n",
    "plt.title('mse_train')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d549d898d0>]"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2d549d8aef0>]"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x2d54a1c41c0>"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'mse_test')"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAtYklEQVR4nO3deXxU1fn48c8zkw1ISISENWFREAUU1KC4gIoiiopikWqtBaWl/ty1avVLF+23ttWvltZa9Yui4o6iiKVYRZYvLmxB2RGDihpkCWAIYQlJ5vn9cW/CJEzIkExmcjPP+/Wa19w598y9z9xMntyce885oqoYY4zxHl+sAzDGGFM/lsCNMcajLIEbY4xHWQI3xhiPsgRujDEeZQncGGM8yhK4McZ4lCVwY4zxKEvgxoQgImNF5KMIbWujiJwfiW0ZE8wSuDHGeJQlcNOkuWevd4vIShHZIyKTRaS9iLwrIrtF5AMROUpEUkTkJRHZISJFIrJURNq720h337dZRDaJyB9FxH+YfR4PPAWcLiIlIlLklieLyCMi8q2IbBWRp0SkhbsuU0RmuvveKSIfiohPRF4EugD/crd1T6MfNBM3EmIdgDFh+BEwFOf7+hlwEjAOWAfMAm4FtgDpQA5QCvQH9rnvfx7YBvQAWgEzge+A/w21M1VdJyI3AD9X1bOCVv0FOMbddhnwCvA74D7gV0ABkOXWHehsSq8VkUHutj6o/yEw5lB2Bm684B+qulVVNwEfAotV9TNV3Q9Mx0noZUBboIeqVqjqMlUtds/ChwO3q+oeVd0GTASuOpIARESA8cAdqrpTVXcDfwraThnQEeiqqmWq+qHaSHGmkdkZuPGCrUHL+0K8TgVexDn7fk1EMoCXgAlAVyAR2OzkYMA5cfnuCGPIAloCy4K2I0BlU8z/APcD77vrJ6nqX45wH8YcEUvgpllQ1TLgAeABEemG07Sy3n0uBTJVtfxINlnj9XacPxZ93P8Eau5/N04zyq9EpC8wV0SWquqcENsyJiKsCcU0CyJyroic4F6cLMZp0gio6mbgfeBREWntXlg8RkTOrmOTW4FsEUkCUNUA8DQwUUTaufvsLCLD3OVLRKSH29SyC6gAAkHbOjqyn9gYS+Cm+egATMNJ3uuA/8NpVgH4GZAErAV+cOt1rGN7c4E1wBYR2e6W/RrYACwSkWLgA6CXu66n+7oEWAg8oarz3HV/Bn7j3qFyV0M+pDHBxK6zGGOMN9kZuDHGeJQlcBO33I44JSEeT8U6NmPCYU0oxhjjUVG9jTAzM1O7desWzV0aY4znLVu2bLuqZtUsj2oC79atG3l5edHcpTHGeJ6IfBOq3NrAjTHGoyyBG2OMR1kCN8YYj7KxUIzxoLKyMgoKCti/f3+sQzERlJKSQnZ2NomJiWHVtwRujAcVFBSQlpZGt27dCBod0XiYqrJjxw4KCgro3r17WO+xJhRjPGj//v20bdvWknczIiK0bdv2iP6rsgRujEdZ8m5+jvRn6o0E/sV78OFfYx2FMcY0Kd5I4F/NhwWPgHX7N8aYKmEncBHxi8hnIjLTfd1dRBaLyAYRmVo58H2jSM+Bsj2w74dG24UxpnGkpqbGOgTGjh3LtGnTwq4zbtw4+vXrx4knnsioUaMoKSmJRphH7EjOwG/DGSi/0kPARFXtgTNI/rhIBlZNerbzvOtIpzE0xkSDqhIIBOqu6BETJ05kxYoVrFy5ki5duvD444/HOqSQwrqNUESygYuBB4E73WmjhgA/catMwZnQ9clGiDEogRdAx36NsgtjvOqBf61h7ffFEd1m706t+f2lfQ5bZ+PGjQwbNozTTjuNZcuWMXr0aGbOnElpaSkjR47kgQceqFZ//vz5PPLII8ycOROAm2++mdzcXMaOHRty+926dePqq6/m3XffJSEhgUmTJnHfffexYcMG7r77bm644QZUlXvuuYd3330XEeE3v/kNP/7xj1FVbrnlFmbPnk1OTg5JSQcbCJYtW8add95JSUkJmZmZPP/883TsWH2CptatWwPOH6Z9+/Y12QvG4Z6B/w24h4Nz/LUFioImiS0AOkc2tCAZXZznIjsDN6Ypyc/P58Ybb2TixIls2rSJJUuWsHz5cpYtW8aCBQsavP0uXbqwfPlyBg0aVNXEsWjRIn7/+98D8NZbb7F8+XJWrFjBBx98wN13383mzZuZPn0669evZ+3atbzwwgt88skngNMB6pZbbmHatGksW7aM66+/ngkTJoTc93XXXUeHDh34/PPPueWWWxr8WRpDnWfgInIJsE1Vl4nIOUe6AxEZD4wH54dRLy3bQkILa0IxJoS6zpQbU9euXRk4cCB33XUX77//PieddBIAJSUl5OfnM3jw4AZtf8SIEQCccMIJlJSUkJaWRlpaGsnJyRQVFfHRRx9x9dVX4/f7ad++PWeffTZLly5lwYIFVeWdOnViyJAhAKxfv57Vq1czdOhQACoqKg45+6703HPPUVFRwS233MLUqVO57rrrGvRZGkM4TShnAiNEZDiQArQG/g5kiEiCexaeDWwK9WZVnQRMAsjNza3fbSQiTjOKJXBjmpRWrVoBTlPDfffdxy9/+cta6yYkJFRrJw+nw0pycjIAPp+varnydXl5eW1vq5Wq0qdPHxYuXBhWfb/fz1VXXcXDDz/cJBN4nU0oqnqfqmarajfgKmCuql4DzANGudXGADMaLUpwE3hBo+7CGFM/w4YN49lnn626W2PTpk1s27atWp2uXbuydu1aSktLKSoqYs6cOQ3e76BBg5g6dSoVFRUUFhayYMECTj31VAYPHlxVvnnzZubNmwdAr169KCwsrErgZWVlrFmzpto2VZUNGzZULb/zzjscd9xxDY61MTRkLJRfA6+JyB+Bz4DJkQmpFhk5ToceY0yTc8EFF7Bu3TpOP/10wLl18KWXXqJdu3ZVdXJychg9ejR9+/ale/fuVc0tDTFy5EgWLlxIv379EBEefvhhOnTowMiRI5k7dy69e/emS5cuVXElJSUxbdo0br31Vnbt2kV5eTm33347ffocbIZSVcaMGUNxcTGqSr9+/Xjyyca5P6OhojonZm5urtZ7Rp7/exjmPQgTtkJiSmQDM8Zj1q1bx/HHHx/rMEwjCPWzFZFlqppbs643emLCwVsJi0M2tRtjTNzxznCy6TnO867voO0xsY3FGBMxI0eO5Ouvv65W9tBDDzFs2LAYReQdHkrgQZ15jDHNxvTp02Mdgmd5pwmldWdArDOPMca4vJPAE5IgrYOdgRtjjMs7CRycdvBd38Y6CmOMaRI8lsCtM48xxlTyVgLPyIFdm6AZDVtpTHPX3McDf+qpp3jhhRcOu+28vDxuvfVWwBmVsXJwrYbyzl0o4DShVJTCnkJIax/raIwxLlVFVfH5vHVOWJuJEydWDSl755138vjjj3PvvfeGrHvDDTfUub3c3Fxyc51+OPPnzyc1NZUzzjijwXF6LIEH3UpoCdwYx7v3wpZVkd1mhxPgor8ctoqNB+64//77SU1N5a677uKcc87htNNOY968eRQVFTF58mQGDRpU9dkff/xxnnrqKfx+Py+99BL/+Mc/GDRo0GGP8+F4LIFXdub5FrJPiW0sxhjy8/OZMmUKxcXFTJs2jSVLlqCqjBgxggULFjR4ONnK8cDvuOMOxo4dy8cff8z+/fvp27cvN9xwQ7XxwLdv386AAQMYPHgwCxcurBoPfOvWrfTu3Zvrr7++ajzwGTNmkJWVxdSpU5kwYQLPPvvsIfu+7rrrmDVrFr179+bRRx8NO+by8nKWLFnCrFmzeOCBB/jggw+q1nXr1o0bbrihKuE3lMcSuHXmMeYQdZwpNyYbD/xQV1xxBQCnnHIKGzdubNDnr4u3EniLDEhubZ15jGkibDzw2mP2+/31ivFIeO+Kg91KaEyTY+OBhy8tLY3du3dHZFveOgMH68xjTBNk44GH79JLL2XUqFHMmDGjwRcxvTMeeKWZd8Kat+DXGyMSkzFeZOOBN18RHQ9cRFJEZImIrBCRNSLygFv+vIh8LSLL3Uf/SH2Aw0rPhn0/QGntN9YbY0w8CKcJpRQYoqolIpIIfCQi77rr7lbVw3dvirQMd2b7XQXQrmnOU2eMCZ9XxgN/8MEHeeONN6qVXXnllUyYMCFGEYWRwNVpY6k83U10H9Frd6kp+FZCS+AmjqnqYTuYeIVXxgOfMGFCoyfrI23SDusuFBHxi8hyYBswW1UXu6seFJGVIjJRRJJree94EckTkbzCwsIjCi6k4M48xsSplJQUduzYccS/8KbpUlV27NhBSkr4c/6GdReKqlYA/UUkA5guIn2B+4AtQBIwCWeW+j+EeO8kdz25ubkN/7aldQBfgt1KaOJadnY2BQUFROSkyDQZKSkpZGdnh13/iG4jVNUiEZkHXKiqj7jFpSLyHNDwfqHh8PmhdSfrzGPiWmJiIt27d491GCbGwrkLJcs980ZEWgBDgc9FpKNbJsDlwOrGC7OG9Bw7AzfGxL1wzsA7AlNExI+T8F9X1ZkiMldEsgABlgN1j6kYKenZ8E1kxtM1xhivCuculJXAIV2mVHVIo0QUjvQcKP4eKsrB773OpMYYEwneGwsFnDNwrYCSLbGOxBhjYsabCTzDvZXQLmQaY+KYNxN41b3gdiHTGBO/PJrAK3tjWmceY0z88mYCT2oFLdrYGbgxJq55M4GD0w5ubeDGmDjm3QRunXmMMXHOwwk8G3Z9BzaYjzEmTnk4gefAgRLYXxTrSIwxJiY8nMCDxgU3xpg45N0Ebp15jDFxzrsJ3DrzGGPinHcTeKss8CdbZx5jTNzybgIXce9EsTNwY0x88m4CByeBWxu4MSZOhTMjT4qILBGRFSKyRkQecMu7i8hiEdkgIlNFJKnxw60hwzrzGGPiVzhn4KXAEFXtB/QHLhSRgcBDwERV7QH8AIxrtChrk57jjAleXhr1XRtjTKzVmcDVUeK+THQfCgwBprnlU3DmxYyuyjtRijdFfdfGGBNrYbWBi4hfRJYD24DZwJdAkaqWu1UKgM61vHe8iOSJSF5hYWEEQg5inXmMMXEsrASuqhWq2h/IBk4Fjgt3B6o6SVVzVTU3KyurflHWxjrzGGPi2BHdhaKqRcA84HQgQ0QqZxTOBqLfjtHaPem3M3BjTBwK5y6ULBHJcJdbAEOBdTiJfJRbbQwwo5FirF1CMqR2sM48xpi4lFB3FToCU0TEj5PwX1fVmSKyFnhNRP4IfAZMbsQ4a2edeYwxcarOBK6qK4GTQpR/hdMeHlvp2bBlVayjMMaYqPN2T0w42JnHJnYwxsQZ7yfw9ByoKIU922MdiTHGRFXzSOBgFzKNMXGnGSRw68xjjIlP3k/g1pnHGBOnvJ/AUzIgKdXOwI0xccf7CbxqYgc7AzfGxBfvJ3BwLmRaAjfGxJlmksBtZh5jTPxpHgk8Iwf27YQDe2IdiTHGRE3zSOBV94LbxA7GmPjRzBK4deYxxsSPZpLArTOPMSb+NI8EntYRxG8XMo0xcaV5JHB/ArTuZGfgxpi4Es6MPDkiMk9E1orIGhG5zS2/X0Q2ichy9zG88cM9DOvMY4yJM+HMyFMO/EpVPxWRNGCZiMx2101U1UcaLzyHqrKleD8d01vUXik9B75b1NihGGNMk1HnGbiqblbVT93l3TjzYXZu7MCC/df0VYx6ciF7D5TXXik9G4q/h0BF9AIzxpgYOqI2cBHphjO92mK36GYRWSkiz4rIUbW8Z7yI5IlIXmFhYb2CHHlSNpuK9vH43A21V8rIgUA57N5Sr30YY4zXhJ3ARSQVeBO4XVWLgSeBY4D+wGbg0VDvU9VJqpqrqrlZWVn1CvLU7m340cnZPP3hV2zYtjt0pap7we1CpjEmPoSVwEUkESd5v6yqbwGo6lZVrVDVAPA0jTzB8X3Dj6NlUgK/eXs1Gmr+y6oEbhcyjTHxIZy7UASYDKxT1b8GlXcMqjYSWB358A7KTE3mngt7seirnby9PESX+XS3Wd4SuDEmToRzBn4mcC0wpMYtgw+LyCoRWQmcC9zRmIECXD2gC/1yMnjw3+vYta+s+srkNGdyB+vMY4yJE+HchfKRqoqqnqiq/d3HLFW9VlVPcMtHqOrmRg/WJzx4eV927jnAo++vP7RCRo61gRtj4obnemL27ZzOz07vxouLvmFlQVH1lTaxgzEmjngugQPcecGxZKYm85u3V1MRCLqgmW5n4MaY+OHJBN46JZHfXHw8Kwt28cribw6uSM+G0mLYVxSz2IwxJlo8mcABRvTrxJk92vLwe+sp3F3qFGbYveDGmPjh2QQuIvzhsr7sL6vgT7PWOYXWmccYE0c8m8ABjslK5ZeDj2H6Z5tY+OWOoIkd7EKmMab583QCB7h5SA9y2rTgtzNWcyAlE/xJlsCNMXHB8wk8JdHPAyP6sGFbCc98vBFad7bOPMaYuOD5BA4w5Lj2DOvTnsfm5LO/VWdrAzfGxIVmkcABfndpHwRhWVEra0IxxsSFZpPAO2e04Pbze7K0qBW6ewuUH4h1SMYY06iaTQIHuP6s7pSndkZQ9u20s3BjTPPWrBJ4ot/HxYOcYcnfnmfzYxpjmrdmlcABjj+uDwCfrlpV++w9xhjTDDS7BE5rZ2KHbgk7a5+9xxhjmoHml8ATU6BVO4Zll7Hoq53MWP59rCMyxphGEc6UajkiMk9E1orIGhG5zS1vIyKzRSTffQ45K31MpGdzTNIP9MvJ4I//Xnvo7D3GGNMMhHMGXg78SlV7AwOBm0SkN3AvMEdVewJz3NdNQ0YOsqvg8LP3GGOMx4UzpdpmVf3UXd4NrAM6A5cBU9xqU4DLGynGI+dO7NC3U+vaZ+8xxhiPO6I2cBHpBpwELAbaB82DuQVoX8t7xotInojkFRYWNiTW8KXnQPk+2Lujavae385YYxc0jTHNStgJXERSgTeB21W1OHidOpkxZHZU1UmqmququVlZWQ0KNmxBw8q2Tknk1vN6suK7IlZt2hWd/RtjTBSElcBFJBEneb+sqm+5xVtFpKO7viOwrXFCrIfKBO6OSnhZ/04kJ/h4Pc96Zxpjmo9w7kIRYDKwTlX/GrTqHWCMuzwGmBH58Oopo4vz7I5K2DolkYv6duCd5d+zv6wihoEZY0zkhHMGfiZwLTBERJa7j+HAX4ChIpIPnO++bhpaHAWJLauNSjg6N4fi/eW8t2ZLDAMzxpjISairgqp+BEgtq8+LbDgRIuLeiXIwgQ88ui3ZR7Vg2rICLuvfOYbBGWNMZDS/npiV0rOrzczj8wmjTsnmow3bKfhhbwwDM8aYyGi+CTwj55CZeUad4lzcfHPZplhEZIwxEdV8E3h6NuzdDgcOnm1nH9WSM45py7RPvyMQsHvCjTHe1owTeI7zXFz9bHt0bg7f7dzHoq93xCAoY4yJnOafwGvMjzmsTwfSUhKYlmcTHxtjvK0ZJ/DqnXkqpST6GdGvE7NWb6Z4v41SaIzxruabwFt3AvEdciETnGaU/WUBZq7YHOKNxhjjDc03gfsTIa3jIU0oACdmp3Ns+1TeWGZd640x3tV8EzhUDStbk4gwOjeHz74tsnkzjTGe1cwTeDYUfRty1eUndSbBJ7xhFzONMR7VvBN4Rg4Ufw+BQwewykxNZshx7Xjz002UVQRiEJwxxjRM807g6dkQKIOSrSFXj87NYXtJKfPXR2miCWOMiaBmnsAr7wUP3UxyTq8sMlOTecPGCTfGeFCcJPDQCTrB7+NHJ3dm7ufb2F5SGsXAjDGm4Zp5Ag/dmSfYlbnZlAeUtz+zAa6MMd4Szow8z4rINhFZHVR2v4hsqjHBQ9OT0hpS0mttQgHo0S6Nk7pkMHXpdzbpsTHGU8I5A38euDBE+URV7e8+ZkU2rAiqMbFDKFeekkP+thJWFNikx8YY76gzgavqAmBnFGJpHLV05gl2Sb+OpCT67GKmMcZTGtIGfrOIrHSbWI6qrZKIjBeRPBHJKyyMwe16NWbmCaV1SiLD+3bknRU26bExxjvqm8CfBI4B+gObgUdrq6iqk1Q1V1Vzs7Ky6rm7BkjPhtJdsP/wzSOjcrPZbZMeG2M8pF4JXFW3qmqFqgaAp4FTIxtWBGUc/l7wSgO7tyWnTQtet2YUY4xH1CuBi0jHoJcjgdW11Y25OjrzVPL5hFEn5/Dxhh18t9MmPTbGNH3h3Eb4KrAQ6CUiBSIyDnhYRFaJyErgXOCORo6z/urozBPsR6d0RgTe/NQGuDLGNH0JdVVQ1atDFE9uhFgaR2p78CXWeSETnEmPz+qRyRt5Bdw6pCc+n0QhQGOMqZ/m3RMTwOeD9M51NqFUGnVKNpuK9rHoK5v02BjTtDX/BA5hdeapNKxPB1qnJNjFTGNMkxc/CXznV1Be94BVKYl+RvTvxLurt7Brn016bIxpuuIjgR9/KewphBk3QRjjnYzOzaG0PMDMld9HIThjjKmf+Ejgxw2HIb+FVW/AvAfrrH5C53R6tU/jdZtuzRjThMVHAgcY9Cs46VpY8D/w6YuHrSoiXJmbzYrvivhiq016bIxpmuIngYvAJRPh6HNh5u3w5dzDVh9ZNemxXcw0xjRN8ZPAAfyJMHoKZPaC18fA1jW1Vm2bmsx5x7dj+mc26bExpmmKrwQOzgQP17wOiS3h5dFQvLnWqs6kxweY9/m2KAZojDHhib8EDs4Ihde8Dvt+gFdGQ2lJyGpnH5tFVlqyXcw0xjRJ8ZnAATr2gyufh62rYdr1UFF+SJUEv48rTu7MvPXb2LZ7f/RjNMaYw4jfBA5w7AUw/BHIfw/+8+uQ94hfeUoOFTbpsTGmCYrvBA4wYByccSssfQYW/vOQ1T3apXJylwxeWPgNG7aFbmoxxphYsAQOcP4D0PsyeP83sHbGIavvuqAXxfvKGP73D/nr7C9s2jVjTJNgCRycEQtH/i9kD4C3xsN3S6utPqNHJnN+dQ7DT+jAY3PyuejvH/LJhu0xCtYYYxzhTOjwrIhsE5HVQWVtRGS2iOS7z7VOauwZiS3g6lchrSO8epUz+FWQrLRk/nbVSbw47lQCqvzkmcXcOXU5O0rqHiDLGGMaQzhn4M8DF9YouxeYo6o9gTnua+9rlQnXTAOtgJevhL07D6kyqGcW790+mFuG9OBfK7/nvL/+H1OXfksgUPcgWcYYE0l1JnBVXQDUzGSXAVPc5SnA5ZENK4Yye8BVr0LRt/DaNSGHoE1J9POrC3rx7m2DOLZdGr9+cxVXTVpEvo2bYoyJovq2gbdX1coujFuA9rVVFJHxIpInInmFhYX13F2UdT0dLn8Svv0E3r4RAqG70vdol8Zr4wfy8I9O5Ittuxn+2Ic88t56u8hpjImKBl/EVFUFam0/UNVJqpqrqrlZWVkN3V30nDAKzvs9rJ522CFofT5h9IAc5tx5Npf268Tj8zYw7G8L+DDfI3+sjDGeVd8EvlVEOgK4z81zsJCz7oCTx8CHj8CyKYet2jY1mb+O7s8rPz8NnwjXTl7Cba99RuFuu8hpjGkc9U3g7wBj3OUxwKE3TzcHInDxo3DMefCv22DxpDrfckaPTN69bRC3nteTd1dt4bxH5/PqErvIaYyJvHBuI3wVWAj0EpECERkH/AUYKiL5wPnu6+bJnwg/fgl6DYd374bZv6u1TbxSSqKfO4cey6zbBnF8x9bc99YqfvTUJ3yYX4iGMaWbMcaEQ6KZUHJzczUvLy9q+4uoQAXMuhvyJkPfUXD5E5CQXOfbVJVpywp49P0v2FK8n/45Gdx2Xk/O6ZWFiEQhcGOM14nIMlXNPaTcEvgRUIWP/wYf3A/dBjln5i0ywnpraXkFb+QV8OT8L9lUtI8TOqdzy5AeDO3d3hK5MeawLIFH0oqpzgz3mT3hmjec8cXDdKA8wPTPCvjnvC/5dudeju/YmluG9ODCPh3w+SyRG2MOZQk80r6aD1OvhaRU+Ok0aN/niN5eXhFgxvLv+ee8DXy1fQ/Htk/lpnN7cMmJnfBbIjfGBLEE3hi2rHa63B8ocZpTjj77iDdREVBmrvyex+duIH9bCUdnteLmc3swol8nEvw21pgxxhJ449lVAC+Ngh0bnN6bJ15Zr80EAsp/1mzhsTn5fL5lN13btuSmc3ow8uTOJFoiNyauWQJvTPuKYOpPYeOHcP79cObtzj3k9RAIKB+s28pjc/NZvamYzhktuPHcYxh1SjbJCf5IRm2M8QhL4I2tvBTe/n+w+k0Y8Au46CHw1T/hqirz1xfy9zn5LP+uiK5tW/LKLwbSOaNFBIM2xnhBbQnc/jePlIRkuOIZd3q2p50LnAf21ntzIsK5x7Vj+o1n8Nx1A9i55wA/fWaxdc03xlSxBB5JPh9c8N9w0cOwfha8MAL27GjQJkWEc3u147mxA9iyaz/XTl5M0d4DEQrYGONllsAbw2m/hNEvwJZVMHko7Py6wZvM7daGp3+Wy1eFexj73FJKSssjEKgxxsssgTeW3iPgZzNg304niW9a1uBNntUzk8d/chKrNu3iF1PybNxxY+KcJfDG1GUgXP++M9/m85fAF+81eJMX9OnAo1f2Y9HXO7jx5U8pqzj8wFrGmObLEnhjyzoWxn3gdLt/9eo6xxUPx+UndeaPl/dl7ufbuGPqcipsqFpj4pIl8GhIaw9jZ8Ex58K/boV5f3IGxmqAa07ryn8NP46ZKzfzX2+tsmFqjYlDCbEOIG4kp8LVr8HM2+H/HnJ6cF76d2e88XoaP/gYSvaX89jcDbRKTuC3lxxvIxsaE0calMBFZCOwG6gAykPdaG6C+BNhxOOQngPz/wy7t8DoKZCcVu9N3jH0WHaXlvPsx1+TlpLAHUOPjWDAxpimLBJn4Oeq6vYIbCc+iMA590LrTvCv2+G54c6QtGkd6rk54bcX92ZPaTl/n5NPanICvxh8dGRjNsY0SdYGHisn/wx+MhV2fAnPDIXC9fXelM8n/PmKE7n4hI48OGsdryz+NoKBGmOaqoYmcAXeF5FlIjI+EgHFlZ5D4bp/Q/l+mHwBfLOw3pvy+4SJP+7PkOPaMeHtVcxYvimCgRpjmqKGJvCzVPVk4CLgJhEZXLOCiIwXkTwRySssLGzg7pqhTifBz2dDq0x44TJYO6Pem0pK8PHENSdzWvc23Pn6Ct5fsyWCgRpjmpoGJXBV3eQ+bwOmA6eGqDNJVXNVNTcrK6shu2u+juoG42ZDp/7w+hhY9GS9N5WS6OeZMQPo2zmdm1/5jI/y7fKEMc1VvRO4iLQSkbTKZeACYHWkAos7Lds4Xe+Puxj+cy+8NwEC9etlmZqcwJTrBnB0Vit+8UIey77ZGeFgjTFNQUPOwNsDH4nICmAJ8G9V/U9kwopTiS2cQbBOHQ8LH4c3r4ey/fXaVEbLJF4Ydyod0lMY+9xS1ny/K8LBGmNird4JXFW/UtV+7qOPqj4YycDils/vDEc79L9hzXR46QrY90O9NtUuLYWXfn4aackJ/PSZxSzdaGfixjQndhthUyQCZ94KP5oM3y2BycOg6Lt6bapzRgteHT+Qo1om8ZOnF/H60vptxxjT9FgCb8pOGAXXvuX02Hz6XFj9Vr3GUOnathXTbzyTgUe35Z43V/LHmWttACxjmgFL4E1d98Ew7n1o3RmmXQevjIYfvjnizaS3TOS5sQMYe0Y3nvnoa8ZNWUrx/rJGCNgYEy2WwL2g3XHw8zkw7M+w8WN4YiB8/BhUHNmsPAl+H/eP6MOfRp7AR/nbueKJT9i4fU8jBW2MaWyWwL3CnwCn3wg3LYbuZ8Ps38Kkc+o1089PTuvCi+NOY3tJKZc/8TGffGn3ihvjRZbAvSYjB65+FUa/CHu3w9Pnwax7YH/xEW3m9GPaMuOmM8lMTeZnk5fw8uIjb5YxxsSWJXAvEnHm3LxpMQz4OSyZBP88DdbNPKLNdG3birduPINBPTOZMH01v5+xmnKbos0Yz7AE7mUp6XDxI043/JZtYOo18OpPnMkiwtQ6JZFnxgzgF4O6M2XhN4x9bim79trFTWO8wBJ4c5AzAMbPh/MfgC/nOmfji56EQHiz1vt9woSLe/PwqBNZ/PUOLn/iY74sLGncmI0xDWYJvLnwJ8JZt8ONCyHnNGc8lWfOg80rwt7E6NwcXvnFQIr3lXH5Pz9mwRc2eqQxTZkl8OamTXf46ZtOL85dBTDpXGdgrAPh3S44oFsb3r7pTDpntOC655fy/Mdf24TJxjRREs1fztzcXM3Ly4va/uLevh9g9u/h0ymQkgHt+0DbHpB5LGT2dJYzujq3KNZQUlrO7a8t54N1W7n61C784bI+JPrt770xsSAiy0LNOWwJPB58sxCWvwTb853HvqBBrfxJ0OZoN7H3hLY93QTfg0ByBo+8v54n5n/JCZ3TGdQzk14d0ujVIY2jM1NJSrCEbkw0WAI3B+3ZATvcZL4jH7ZvcJ53fgWBoN6dLTMhsycbpTPztrTg2z1+SjSJPZpCqaSQnp5B+8w2dMrKJKdjO47ulEV2Via+xKTYfTZjmqHaEngkZqU3XtOqrfPoMrB6eUWZM85KteSeT7fC+VxXuv3Qb8te4Fv3EaSMBA74WxJIbIk/qRWJLdJISGmFJKRAQrJz1p+Q7C4nB5WlQEKSW+a+rlz2J4H4QXzOffC+yuXKR/DrEOt9Cc6F3qr9ucv+RKe+MR5kCdwc5E+EzB7Oo9dF1deV7YMDe+FACZTtdS6Kuo/9e3ezbcdOdu7cyQ+7iijZvYu9JcX4SvbQQkpp9cN+Un07aeErJ0XKSZZykikjiTISKSdRD+DXA/g1vNseI/+5ayT14D8i/qBlX0LQww++xBqva6z3B60XX4g/Or7qf3RClgf90Qql1j8+IcpFnPLDPlPLuhrlobZX6z4q31v5mXCeq5UFLVeVy8HyatuqJ1XQAKAHl6uVhXodcKZuR4Ni9Ls/S/+hP9fDlSe0CHm9qSEatDURuRD4O+AHnlHVv0QkKtP0JLZwHq3aHrIqBejiPoLtKCnli60lrN9SzDc791Kyv5yS0qDH/nL2lJazu9R5RgMkUk4yB0imnCTKSJYykignkXJ8BPATQFD8oiT5IDkBkn2Q5IdEHyT5lCR/9deJPkj2qbvtMhKlgqSqPyDu9rWMBPePSYKWk6AHSCgrw3/AWfbrHnxa4T7KEXdZKl8Hgl+XIYGD64wB4Jo3oef5Ed1kvRO4iPiBfwJDgQJgqYi8o6prIxWc8ba2qcmcnprM6cccmvRrUlX2lVUEJfYKdpeWsae0ghL3+UB5gAMVAee5PEBZRYDSoLLS8gAllevddWWVr0sDVASU8oC6zwHKK0K/jizFh5JABYLiI4DPLZOg5Zqv/RKosT78IQ6EQz+DuOXOo/oy7vLh6nHIMgfryMH1HPJ+8AnO55Pgz+o8/FX1Ak5d9w90ZV0Jqu+Tg5/FJ+L+YyBI5T8IgMjB1wfLBZ+oezRxn4WAUnVkaz4HVCojcSNztuUngF+cEwk/ip8APnGXxdmyn4B7suFszSkPMHBfFv3D/imGpyFn4KcCG1T1KwAReQ24DLAEbo6YiNAyKYGWSQm0S4tdHKpKQAmZ4NX9zzug6iSCgDplOO8JqPu6cr0qgYC7PuC+rnpARUCr6lRUlgcOrlNVt9zZV1WMbqIMvv+g2nLQZ6lWpgff78R98L2VZQfravX1VZ/JXQ4+DsGfWzl4LKhe95BjXevPIEQZTsCBGvug8jjXss+AHoy/qpWnMiFXS/wH1wmQUK3FRg5G4W6vAqXcPYbBx+fgM9VeozAos1stn7j+GpLAOwPB83MVAKfVrCQi44HxAF261Pwn25imRUTwC/h9fpLtCpFp4hr9Rl5VnaSquaqam5WV1di7M8aYuNGQBL4JyAl6ne2WGWOMiYKGJPClQE8R6S4iScBVwDuRCcsYY0xd6t3Kp6rlInIz8B7ObYTPquqaiEVmjDHmsBp0mUZVZwGzIhSLMcaYI2CjERljjEdZAjfGGI+yBG6MMR4V1eFkRaQQ+Kaeb88EtkcwnEiz+BrG4msYi6/hmnKMXVX1kI40UU3gDSEieaHGw20qLL6GsfgaxuJrOC/EWJM1oRhjjEdZAjfGGI/yUgKfFOsA6mDxNYzF1zAWX8N5IcZqPNMGbowxpjovnYEbY4wJYgncGGM8qsklcBG5UETWi8gGEbk3xPpkEZnqrl8sIt2iGFuOiMwTkbUiskZEbgtR5xwR2SUiy93H76IVn7v/jSKyyt13Xoj1IiKPucdvpYicHMXYegUdl+UiUiwit9eoE9XjJyLPisg2EVkdVNZGRGaLSL77fFQt7x3j1skXkTFRjO9/RORz9+c3XUQyannvYb8LjRjf/SKyKehnOLyW9x72d70R45saFNtGEVley3sb/fg1mDP9UNN44Ixq+CVwNJAErAB616hzI/CUu3wVMDWK8XUETnaX04AvQsR3DjAzhsdwI5B5mPXDgXdx5okaCCyO4c96C04HhZgdP2AwcDKwOqjsYeBed/le4KEQ72sDfOU+H+UuHxWl+C4AEtzlh0LFF853oRHjux+4K4yf/2F/1xsrvhrrHwV+F6vj19BHUzsDr5pnU1UPAJXzbAa7DJjiLk8DzhM5OHtdY1LVzar6qbu8G1iHM7Wcl1wGvKCORUCGiHSMQRznAV+qan175kaEqi4AdtYoDv6OTQEuD/HWYcBsVd2pqj8As4ELoxGfqr6vWjXd/SKcyVRiopbjF45wftcb7HDxuXljNPBqpPcbLU0tgYeaZ7Nmgqyq436JdwF1T3seYW7TzUnA4hCrTxeRFSLyroj0iW5kKPC+iCxz5yOtKZxjHA1XUfsvTiyPH0B7Vd3sLm8B2oeo01SO4/U4/1GFUtd3oTHd7DbxPFtLE1RTOH6DgK2qml/L+lgev7A0tQTuCSKSCrwJ3K6qxTVWf4rTLNAP+AfwdpTDO0tVTwYuAm4SkcFR3n+d3BmcRgBvhFgd6+NXjTr/SzfJe21FZAJQDrxcS5VYfReeBI4B+gObcZopmqKrOfzZd5P/XWpqCTyceTar6ohIApAO7IhKdM4+E3GS98uq+lbN9aparKol7vIsIFFEMqMVn6pucp+3AdNx/lUN1hTmMr0I+FRVt9ZcEevj59pa2azkPm8LUSemx1FExgKXANe4f2QOEcZ3oVGo6lZVrVDVAPB0LfuN9fFLAK4AptZWJ1bH70g0tQQezjyb7wCVV/xHAXNr+wJHmttmNhlYp6p/raVOh8o2eRE5FecYR+UPjIi0EpG0ymWci12ra1R7B/iZezfKQGBXUHNBtNR65hPL4xck+Ds2BpgRos57wAUicpTbRHCBW9boRORC4B5ghKruraVOON+Fxoov+JrKyFr2G+s5dc8HPlfVglArY3n8jkisr6LWfODcJfEFzhXqCW7ZH3C+rAApOP96bwCWAEdHMbazcP6dXgksdx/DgRuAG9w6NwNrcK6qLwLOiGJ8R7v7XeHGUHn8guMT4J/u8V0F5Eb559sKJyGnB5XF7Pjh/CHZDJThtMOOw7mmMgfIBz4A2rh1c4Fngt57vfs93ABcF8X4NuC0H1d+ByvvyuoEzDrcdyFK8b3ofrdW4iTljjXjc18f8rsejfjc8ucrv3NBdaN+/Br6sK70xhjjUU2tCcUYY0yYLIEbY4xHWQI3xhiPsgRujDEeZQncGGM8yhK4McZ4lCVwY4zxqP8PImsw6dqQt4UAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 测试误差\n",
    "for i, name in enumerate(name_l):\n",
    "    plt.plot(list(range(num_epochs)), test_l[i], label=name)\n",
    "plt.legend(loc = 1)\n",
    "plt.title('mse_test')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同样，经过HE初始化后的参数，模型收敛速度更快，也证明了HE初始化的有效性。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 三、参数初始化的作用局限\n",
    "\n",
    "&emsp;&emsp;截止目前，我们讨论了围绕Glorot条件进行的参数初始化方法，当然，合理的设置初始参数值能够一定程度上使得模型各层都得到有效的学习，模型训练过程更加平稳、收敛速度也更快。但由于我们设置的是初始条件，伴随着模型不断训练，由于受到激活函数导函数本身特性影响，仍然有可能在迭代过程中出现梯度不均衡的现象。\n",
    "\n",
    "&emsp;&emsp;然而模型一旦开始训练，我们是不能手动参与修改模型参数的。那此时应该如何处理梯度不均衡的问题呢？我们知道，影响梯度计算的三个核心因素，分别是**参数状态值、激活值和输入的数据**，参数状态值由模型迭代的数学过程决定，激活值很大程度上由我们所选取的激活函数决定，如果从Glorot条件入手，我们就只剩下一个可以人工修改的选项：每一个线性层接收到的数据。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10.0 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.0"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "3296471c9107718e6db2104fe506aa09d2e74f3fa2347eed57e489ba84541490"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}