循环神经网络.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 使用numpy实现简单的RNN前向传递"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### RNN伪代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "state_t = 0  # t时刻的状态\n",
    "for input_t in input_sequence:  # 对序列元素遍历\n",
    "    output_t = f(input_t, state_t)\n",
    "    state_t = output_t  # 前一次的输出变成下一次迭代的状态"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 更详细的RNN伪代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "state_t = 0  # t时刻的状态\n",
    "for input_t in input_sequence:  # 对序列元素遍历\n",
    "    output_t = activation(dot(W, input_t) + dot(U, state_t) + b)\n",
    "    state_t = output_t  # 前一次的输出变成下一次迭代的状态"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 简单RNN的numpy实现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(100, 64)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "timesteps = 100  # 时间步\n",
    "input_features = 32  # 输入特征空间的纬度\n",
    "output_features = 64  # 输出特征空间纬度\n",
    "\n",
    "inputs = np.random.random((timesteps, input_features))  # 输入数据：随机噪声，仅作为示例\n",
    "state_t = np.zeros((output_features,))  # 输出数据，全零向量\n",
    "\n",
    "# 创建随机的权重矩阵\n",
    "W = np.random.random((output_features, input_features))\n",
    "U = np.random.random((output_features, output_features))\n",
    "b = np.random.random((output_features,))\n",
    "\n",
    "successive_outputs = []\n",
    "for input_t in inputs:\n",
    "    output_t = np.tanh(np.dot(W, input_t) + np.dot(U, state_t) + b)\n",
    "    successive_outputs.append(output_t)\n",
    "    state_t = output_t\n",
    "\n",
    "final_output_sequence = np.stack(successive_outputs, axis=0)\n",
    "print(final_output_sequence.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### keras中的循环层\n",
    "以上的numpy对应一个实际的keras层，即SimpleRNN层"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.layers import SimpleRNN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "与numpy的RNN实现略微不同，SimpleRNN可以处理序列批量，而不是只能处理单个序列。因此，它接收的形状为(batch_size, timesteps, input_features)的输入，而不是(timesteps, input_features)。\n",
    "\n",
    "与Keras中的所有循环层一样，SimpleRNN可以在两种不同的模式下运行：一种是返回每个时间步连续输出的完整序列，即形状为(batch_size,timesteps,output_features)的三维张量；另一种是只返回每个输入序列的最终输出，即形状为(batch_size,output_features)的二维张量。这两种模式由return_sequences这个构造函数参数来控制。我们来看一个使用SimpleRNN的例子，它只返回最后一个时间步的输出。 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_2 (Embedding)      (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "simple_rnn_2 (SimpleRNN)     (None, 32)                2080      \n",
      "=================================================================\n",
      "Total params: 322,080\n",
      "Trainable params: 322,080\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Embedding, SimpleRNN\n",
    "model = Sequential()\n",
    "model.add(Embedding(10000, 32))\n",
    "model.add(SimpleRNN(32))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面返回完整的状态序列"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_3 (Embedding)      (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "simple_rnn_3 (SimpleRNN)     (None, None, 32)          2080      \n",
      "=================================================================\n",
      "Total params: 322,080\n",
      "Trainable params: 322,080\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "model.add(Embedding(10000, 32))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 搞一波IMDB的电影评论分类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 准备IMDB数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading data...\n",
      "25000 train_sequences\n",
      "25000 test_sequences\n",
      "Pad sequence (sample x time)\n",
      "input_train shape: (25000, 500)\n",
      "input_test shape: (25000, 500)\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import imdb\n",
    "from keras.preprocessing import sequence\n",
    "\n",
    "max_features = 10000  # 作为特征的单词个数\n",
    "maxlen = 500  # 在这么多单词之后截断文本\n",
    "batch_size = 32\n",
    "\n",
    "print('Loading data...')\n",
    "(input_train, y_train), (input_test, y_test) = imdb.load_data(num_words=max_features)\n",
    "\n",
    "print(len(input_train), 'train_sequences')\n",
    "print(len(input_test), 'test_sequences')\n",
    "print('Pad sequence (sample x time)')\n",
    "\n",
    "input_train = sequence.pad_sequences(input_train, maxlen=maxlen)\n",
    "input_test = sequence.pad_sequences(input_test, maxlen=maxlen)\n",
    "print('input_train shape:', input_train.shape)\n",
    "print('input_test shape:', input_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 用Embedding和SimpleRNN层来训练模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "W0905 20:58:26.705943 140736197415872 deprecation_wrapper.py:119] From /Users/schp/miniconda2/envs/keras2.0.8/lib/python3.6/site-packages/keras/optimizers.py:711: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.\n",
      "\n",
      "W0905 20:58:26.730252 140736197415872 deprecation_wrapper.py:119] From /Users/schp/miniconda2/envs/keras2.0.8/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:2819: The name tf.log is deprecated. Please use tf.math.log instead.\n",
      "\n",
      "W0905 20:58:26.738861 140736197415872 deprecation.py:323] From /Users/schp/miniconda2/envs/keras2.0.8/lib/python3.6/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
      "W0905 20:58:27.074650 140736197415872 deprecation.py:506] From /Users/schp/miniconda2/envs/keras2.0.8/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:625: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Call initializer instance with the dtype argument instead of passing it to the constructor\n",
      "W0905 20:58:27.104167 140736197415872 deprecation_wrapper.py:119] From /Users/schp/miniconda2/envs/keras2.0.8/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:899: The name tf.assign_add is deprecated. Please use tf.compat.v1.assign_add instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 18s - loss: 0.6617 - acc: 0.6002 - val_loss: 0.5787 - val_acc: 0.7582\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 17s - loss: 0.4235 - acc: 0.8253 - val_loss: 0.4043 - val_acc: 0.8212\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 17s - loss: 0.2841 - acc: 0.8880 - val_loss: 0.4072 - val_acc: 0.8356\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 18s - loss: 0.1992 - acc: 0.9263 - val_loss: 0.4020 - val_acc: 0.8232\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 19s - loss: 0.1389 - acc: 0.9497 - val_loss: 0.4552 - val_acc: 0.8456\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 18s - loss: 0.0913 - acc: 0.9683 - val_loss: 0.4790 - val_acc: 0.8336\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 18s - loss: 0.0621 - acc: 0.9799 - val_loss: 0.6167 - val_acc: 0.8026\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 17s - loss: 0.0421 - acc: 0.9874 - val_loss: 0.6611 - val_acc: 0.7892\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 17s - loss: 0.0338 - acc: 0.9897 - val_loss: 0.6187 - val_acc: 0.8200\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 18s - loss: 0.0209 - acc: 0.9936 - val_loss: 0.6617 - val_acc: 0.8148\n"
     ]
    }
   ],
   "source": [
    "from keras.layers import Dense\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(max_features, 32))\n",
    "model.add(SimpleRNN(32))\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\n",
    "history = model.fit(input_train, y_train, epochs=10, batch_size=128, validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 绘制结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxU5b3H8c+XNQLKbkWQJCIKSAiEFPS6F0VqVaqlFMQWV6pXbEvtgtVbfOHVequ1ttarUkurBeFSrRZr1bpgUetCUAKCsoioQdSwuoBC4Hf/eM7AZMgygUlmMvm9X695ZeYsM7+ZJN/zzHPOeY7MDOecc9mrWboLcM45V7886J1zLst50DvnXJbzoHfOuSznQe+cc1nOg94557KcB30TJKm5pE8l9Uzlsukk6QhJKT9WWNKpktbEPV4u6YRklt2H17pH0s/2dX3nqtMi3QW42kn6NO5hG+ALYGf0+LtmNrMuz2dmO4F2qV62KTCzo1LxPJIuAc43s5PjnvuSVDy3c4k86BsBM9sdtFGL8RIze6q65SW1MLOKhqjNudr432P6eddNFpD035L+T9IsSZ8A50s6VtJLkjZLWifpt5JaRsu3kGSS8qLHM6L5j0n6RNKLkvLrumw0/6uSVkjaIul2SS9IuqCaupOp8buSVknaJOm3ces2l/RrSRskrQZG1PD5XCNpdsK0OyTdGt2/RNIb0ft5K2ptV/dcZZJOju63kfTnqLalwOCEZa+VtDp63qWSzo6mFwC/A06IusXWx32218Wtf1n03jdIelhSt2Q+m7p8zrF6JD0laaOkDyT9JO51/iv6TD6WVCLp0Kq6ySQ9H/s9R5/n/Oh1NgLXSuotaV70Guujz6193Pq50Xssj+b/RlJOVHPfuOW6SdoqqXN179dVwcz81ohuwBrg1IRp/w1sB84ibLwPAL4MDCV8azscWAFMjJZvARiQFz2eAawHioGWwP8BM/Zh2YOBT4CR0bwfAjuAC6p5L8nU+DegPZAHbIy9d2AisBToAXQG5oc/5ypf53DgU6Bt3HN/BBRHj8+KlhHwFWAbMCCadyqwJu65yoCTo/u3AM8CHYFcYFnCsqOBbtHv5Lyohi9F8y4Bnk2ocwZwXXR/eFTjQCAH+F/gmWQ+mzp+zu2BD4HvA62Bg4Ah0byrgVKgd/QeBgKdgCMSP2vg+djvOXpvFcDlQHPC3+ORwDCgVfR38gJwS9z7eT36PNtGyx8XzZsG3BD3OlcBD6X7/7Cx3dJegN/q+AurPuifqWW9HwF/ie5XFd53xS17NvD6Pix7EfBc3DwB66gm6JOs8Zi4+X8FfhTdn0/oworNOyMxfBKe+yXgvOj+V4HlNSz7d+CK6H5NQf9u/O8C+M/4Zat43teBr0X3awv6e4Eb4+YdRNgv06O2z6aOn/O3gQXVLPdWrN6E6ckE/epaahgVe13gBOADoHkVyx0HvA0oerwIODfV/1fZfvOum+zxXvwDSX0kPRp9Ff8YmAp0qWH9D+Lub6XmHbDVLXtofB0W/jPLqnuSJGtM6rWAd2qoF+B+YGx0/7zocayOMyW9HHUrbCa0pmv6rGK61VSDpAsklUbdD5uBPkk+L4T3t/v5zOxjYBPQPW6ZpH5ntXzOhxECvSo1zatN4t/jIZLmSFob1fCnhBrWWNjxX4mZvUD4dnC8pP5AT+DRfaypyfKgzx6JhxbeTWhBHmFmBwE/J7Sw69M6QosTAEmicjAl2p8a1xECIqa2wz/nAKdK6k7oWro/qvEA4AHgF4RulQ7AP5Os44PqapB0OHAnofuic/S8b8Y9b22Hgr5P6A6KPd+BhC6itUnUlaimz/k9oFc161U377OopjZx0w5JWCbx/f0P4WixgqiGCxJqyJXUvJo67gPOJ3z7mGNmX1SznKuGB332OhDYAnwW7cz6bgO85t+BIklnSWpB6PftWk81zgF+IKl7tGPupzUtbGYfELoX/kTotlkZzWpN6DcuB3ZKOpPQl5xsDT+T1EHhPIOJcfPaEcKunLDNu5TQoo/5EOgRv1M0wSzgYkkDJLUmbIieM7NqvyHVoKbPeS7QU9JESa0lHSRpSDTvHuC/JfVSMFBSJ8IG7gPCTv/mkiYQt1GqoYbPgC2SDiN0H8W8CGwAblTYwX2ApOPi5v+Z0NVzHiH0XR150Gevq4DxhJ2jdxN2mtYrM/sQ+BZwK+EftxfwGqEll+oa7wSeBpYACwit8trcT+hz391tY2abgUnAQ4QdmqMIG6xkTCF8s1gDPEZcCJnZYuB24JVomaOAl+PWfRJYCXwoKb4LJrb+44Quloei9XsC45KsK1G1n7OZbQFOA75B2PisAE6KZt8MPEz4nD8m7BjNibrkLgV+Rtgxf0TCe6vKFGAIYYMzF3gwroYK4EygL6F1/y7h9xCbv4bwe/7CzP5dx/fu2LODw7mUi76Kvw+MMrPn0l2Pa7wk3UfYwXtdumtpjPyEKZdSkkYQjnDZRjg8bwehVevcPon2d4wECtJdS2PlXTcu1Y4HVhP6pk8HzvGdZ25fSfoF4Vj+G83s3XTX01h5141zzmU5b9E751yWy7g++i5dulheXl66y3DOuUZl4cKF682sysOZMy7o8/LyKCkpSXcZzjnXqEiq9uxw77pxzrks50HvnHNZrtaglzRd0keSXq9mvqJxp1dJWiypKG7eeEkro9v4VBbunHMuOcn00f+JcJGE6saY+CphvOrehDGv7wSGRmNiTCGMW27AQklzzWxTXYvcsWMHZWVlfP7553Vd1TWgnJwcevToQcuW1Q3f4pxLh1qD3szmK7q6UDVGAvdF41+8FA3w1A04GXjSzDYCSHqScBWgWXUtsqysjAMPPJC8vDzCgIgu05gZGzZsoKysjPz8/NpXcM41mFT00Xen8tjTZdG06qbvRdKE6DJlJeXl5XvN//zzz+ncubOHfAaTROfOnf1bl3P7YOZMyMuDZs3Cz5kzU/v8GbEz1symmVmxmRV37Vr1qLYe8pnPf0fO1d3MmTBhArzzDpiFnxMmpDbsUxH0a6l88YUe0bTqpjvnXEao75Z0Mq65BrZurTxt69YwPVVSEfRzge9ER98cA2wxs3XAE8BwSR0ldSRcnu2JFLxeg9uwYQMDBw5k4MCBHHLIIXTv3n334+3btyf1HBdeeCHLly+vcZk77riDmen4S3OugWVCwDZESzoZ71YzVFt10/dJbReVJew8XUcYbrYMuBi4DLgsmi/gDsK1JZcAxXHrXgSsim4XJnMR28GDB1uiZcuW7TWtJjNmmOXmmknh54wZdVq9RlOmTLGbb755r+m7du2ynTt3pu6FGqm6/q5c0zNjhlmbNmYhXsOtTZvU/p8mIze3cg2xW25u46wDKLF9vTi4mY01s25m1tLMepjZH8zsLjO7K5pvZnaFmfUyswIzK4lbd7qZHRHd/piqjVNNGnIrvWrVKvr168e4ceM4+uijWbduHRMmTKC4uJijjz6aqVOn7l72+OOPZ9GiRVRUVNChQwcmT55MYWEhxx57LB999BEA1157Lbfddtvu5SdPnsyQIUM46qij+Pe/w4V1PvvsM77xjW/Qr18/Ro0aRXFxMYsWLdqrtilTpvDlL3+Z/v37c9lll8U2vKxYsYKvfOUrFBYWUlRUxJo1awC48cYbKSgooLCwkGtS+Z3RuQQN0VWRjAZpSSfhhhugTZvK09q0CdNTJSN2xqZSQ/8Rvfnmm0yaNIlly5bRvXt3brrpJkpKSigtLeXJJ59k2bJle62zZcsWTjrpJEpLSzn22GOZPn16lc9tZrzyyivcfPPNuzcat99+O4cccgjLli3jv/7rv3jttdeqXPf73/8+CxYsYMmSJWzZsoXHH38cgLFjxzJp0iRKS0v597//zcEHH8wjjzzCY489xiuvvEJpaSlXXXVVij4d5/aWKQHbs5rLyVc3vb6MGwfTpkFuLkjh57RpYXqqZF3QN/QfUa9evSguLt79eNasWRQVFVFUVMQbb7xRZdAfcMABfPWrXwVg8ODBu1vVic4999y9lnn++ecZM2YMAIWFhRx99NFVrvv0008zZMgQCgsL+de//sXSpUvZtGkT69ev56yzzgLCCU5t2rThqaee4qKLLuKAAw4AoFOnTnX/IFyjke7+8UwJ2IZoSSdr3DhYswZ27Qo/UxnykIVB39B/RG3btt19f+XKlfzmN7/hmWeeYfHixYwYMaLK48pbtWq1+37z5s2pqKio8rlbt25d6zJV2bp1KxMnTuShhx5i8eLFXHTRRX58uwMyYwdkpgRsQ7SkM0XWBX06/4g+/vhjDjzwQA466CDWrVvHE0+k/iCj4447jjlz5gCwZMmSKr8xbNu2jWbNmtGlSxc++eQTHnzwQQA6duxI165deeSRR4BwItrWrVs57bTTmD59Otu2bQNg48aNKa/bZYZM6B/PpICt75Z0psi48ej3V+wXdc01obumZ88Q8g3xCywqKqJfv3706dOH3NxcjjvuuJS/xpVXXsl3vvMd+vXrt/vWvn37Sst07tyZ8ePH069fP7p168bQoUN3z5s5cybf/e53ueaaa2jVqhUPPvggZ555JqWlpRQXF9OyZUvOOussrr/++pTX7tIvU/rHx43L3lDNRBl3zdji4mJLvPDIG2+8Qd++fdNUUWapqKigoqKCnJwcVq5cyfDhw1m5ciUtWmTGNtt/V5ktLy901yTKzQ0tWtd4SVpoZsVVzcuMdHBJ+/TTTxk2bBgVFRWYGXfffXfGhLzLfDfcEPrk47tv0rUD0jWcrOujz3YdOnRg4cKFlJaWsnjxYoYPH57uklyS0n20C2RW/7hrON4UdK4BxI52ibWkY0e7QMOHrPePNz3eoneuAWTC0S6u6fKgd64BZMrRLq5p8qB3rgFkytmgrmnyoE/CKaecstfJT7fddhuXX355jeu1a9cOgPfff59Ro0ZVuczJJ59M4uGkiW677Ta2xn3vP+OMM9i8eXMypbsMkSlng7qmyYM+CWPHjmX27NmVps2ePZuxY8cmtf6hhx7KAw88sM+vnxj0//jHP+jQocM+P59reH60i0snD/okjBo1ikcffXT3RUbWrFnD+++/zwknnLD7uPaioiIKCgr429/+ttf6a9asoX///kAYnmDMmDH07duXc845Z/ewAwCXX3757iGOp0yZAsBvf/tb3n//fU455RROOeUUAPLy8li/fj0At956K/3796d///67hzhes2YNffv25dJLL+Xoo49m+PDhlV4n5pFHHmHo0KEMGjSIU089lQ8//BAIx+pfeOGFFBQUMGDAgN1DKDz++OMUFRVRWFjIsGHDUvLZNiVN5XR7l3ka3eGVP/gBVDH8+n4ZOBCijKxSp06dGDJkCI899hgjR45k9uzZjB49Gknk5OTw0EMPcdBBB7F+/XqOOeYYzj777Gqvn3rnnXfSpk0b3njjDRYvXkxRUdHueTfccAOdOnVi586dDBs2jMWLF/O9732PW2+9lXnz5tGlS5dKz7Vw4UL++Mc/8vLLL2NmDB06lJNOOomOHTuycuVKZs2axe9//3tGjx7Ngw8+yPnnn19p/eOPP56XXnoJSdxzzz388pe/5Fe/+hXXX3897du3Z8mSJQBs2rSJ8vJyLr30UubPn09+fr6Ph+NcI5JUi17SCEnLJa2SNLmK+bmSnpa0WNKzknrEzdspaVF0m5vK4htSfPdNfLeNmfGzn/2MAQMGcOqpp7J27drdLeOqzJ8/f3fgDhgwgAEDBuyeN2fOHIqKihg0aBBLly6tcsCyeM8//zznnHMObdu2pV27dpx77rk899xzAOTn5zNw4ECg+qGQy8rKOP300ykoKODmm29m6dKlADz11FNcccUVu5fr2LEjL730EieeeCL5+flA4xrKOBNOVHIunWpt0UtqTrhU4GmESwkukDTXzOJT6BbgPjO7V9JXgF8A347mbTOzgakquKaWd30aOXIkkyZN4tVXX2Xr1q0MHjwYCIOElZeXs3DhQlq2bEleXt4+DQn89ttvc8stt7BgwQI6duzIBRdcsF9DC8eGOIYwzHFVXTdXXnklP/zhDzn77LN59tlnue666/b59TJVJp2o5Fy6JNOiHwKsMrPVZrYdmA2MTFimH/BMdH9eFfMbvXbt2nHKKadw0UUXVdoJu2XLFg4++GBatmzJvHnzeKeqEaPinHjiidx///0AvP766yxevBgIQxy3bduW9u3b8+GHH/LYY4/tXufAAw/kk08+2eu5TjjhBB5++GG2bt3KZ599xkMPPcQJJ5yQ9HvasmUL3bt3B+Dee+/dPf20007jjjvu2P1406ZNHHPMMcyfP5+3334baDxDGfuJSs4lF/TdgffiHpdF0+KVAudG988BDpTUOXqcI6lE0kuSvl7VC0iaEC1TUl5eXofyG9bYsWMpLS2tFPTjxo2jpKSEgoIC7rvvPvr06VPjc1x++eV8+umn9O3bl5///Oe7vxkUFhYyaNAg+vTpw3nnnVdpiOMJEyYwYsSI3TtjY4qKirjgggsYMmQIQ4cO5ZJLLmHQoEFJv5/rrruOb37zmwwePLhS//+1117Lpk2b6N+/P4WFhcybN4+uXbsybdo0zj33XAoLC/nWt76V9Oukk5+o5FwSwxRLGgWMMLNLosffBoaa2cS4ZQ4FfgfkA/OBbwD9zWyzpO5mtlbS4YRW/zAze6u61/Nhihu3TPtd+bC8rqmoaZjiZFr0a4HD4h73iKbtZmbvm9m5ZjYIuCaatjn6uTb6uRp4Fki+yencfvITlZxLLugXAL0l5UtqBYwBKh09I6mLpNhzXQ1Mj6Z3lNQ6tgxwHFDzoSTOpZCfqORcEkfdmFmFpInAE0BzYLqZLZU0FSgxs7nAycAvJBmh6yZ2bF5f4G5JuwgblZsSjtZJmplVe2y6ywyZdrWyGB+W1zV1SZ0wZWb/AP6RMO3ncfcfAPY6x9/M/g0U7GeN5OTksGHDBjp37uxhn6HMjA0bNpCTk5PuUpxzCRrFmbE9evSgrKyMTD4ix4UNco8ePWpf0DnXoBpF0Lds2XL3GZnOOefqxgc1c865LOdB7+qNjzHjXGZoFF03rvHxMWacyxzeonf1wseYcS5zeNC7euFjzDiXOTzoXb3wi2E7lzk86F298DFmnMscHvSuXvgYM85lDj/qxtUbH2PGuczgLXrnnMtyHvTOOZflPOidcy7LedA751yW86B3zrksl1TQSxohabmkVZImVzE/V9LTkhZLelZSj7h54yWtjG7jU1m8c8652tUa9JKaA3cAXwX6AWMl9UtY7BbgPjMbAEwFfhGt2wmYAgwFhgBTJHVMXfnOOedqk0yLfgiwysxWm9l2YDYwMmGZfsAz0f15cfNPB540s41mtgl4Ehix/2U755xLVjJB3x14L+5xWTQtXilwbnT/HOBASZ2TXBdJEySVSCrxywU651xqpWpn7I+AkyS9BpwErAV2JruymU0zs2IzK+7atWuKSnLOOQfJDYGwFjgs7nGPaNpuZvY+UYteUjvgG2a2WdJa4OSEdZ/dj3qdc87VUTIt+gVAb0n5kloBY4C58QtI6iIp9lxXA9Oj+08AwyV1jHbCDo+muXrkl/BzzsWrNejNrAKYSAjoN4A5ZrZU0lRJZ0eLnQwsl7QC+BJwQ7TuRuB6wsZiATA1mubqSewSfu+8A2Z7LuHnYe9c0yUzS3cNlRQXF1tJSUm6y2i08vJCuCfKzYU1axq6GudcQ5G00MyKq5rnZ8ZmGb+En3MukQd9lvFL+DnnEnnQZxm/hJ9zLpEHfZbxS/g55xL5pQSzkF/CzzkXz1v0zjmX5TzonXMuy3nQO+dclvOgd865LOdB75xzWc6D3jnnspwHvXPOZTkPeuecy3Ie9M45l+U86J1zLst50DvnXJZLKugljZC0XNIqSZOrmN9T0jxJr0laLOmMaHqepG2SFkW3u1L9BpxzztWs1kHNJDUH7gBOA8qABZLmmtmyuMWuJVxi8E5J/YB/AHnRvLfMbGBqy3bOOZesZFr0Q4BVZrbazLYDs4GRCcsYcFB0vz3wfupKdM45tz+SCfruwHtxj8uiafGuA86XVEZozV8ZNy8/6tL5l6QTqnoBSRMklUgqKS8vT75655xztUrVztixwJ/MrAdwBvBnSc2AdUBPMxsE/BC4X9JBiSub2TQzKzaz4q5du6aoJOf22LwZNm5MdxXOpUcyFx5ZCxwW97hHNC3excAIADN7UVIO0MXMPgK+iKYvlPQWcCRQsr+FO1ebDRvg4Ydhzhx45hmoqIDu3aGgAAYMCLeCAujTB1q1Sne1ztWfZIJ+AdBbUj4h4McA5yUs8y4wDPiTpL5ADlAuqSuw0cx2Sjoc6A2sTln1ziXYsAEeegj+8hd4+mnYuRMOPxyuugo6d4YlS2Dx4jBvx46wTosWIezjw3/AgLBRkNL7fpxLhVqD3swqJE0EngCaA9PNbKmkqUCJmc0FrgJ+L2kSYcfsBWZmkk4EpkraAewCLjMz/wLtUqq6cP/Rj2D0aBg0aO/A3rEDVqwIob94cdgAPPcc3H//nmU6dty79d+/P7Rr17Dvz7n9JTNLdw2VFBcXW0mJ9+y4mq1fX7lbZudO6NULvvnNcKsq3JOxaRO8/nrlDcCSJfDpp3uW6dVr7w1Ar17QvHnq3p9zdSVpoZkVVzXPLw6epXbsCKHVtWv2dD+sX7+n5R4f7j/+cWi5Dxy4/++1Y0c44YRwi9m1C9as2dPtE9sAzJ0b5gEccEBo7SduALp02b96nEsFb9FnATN49114+eU9t1dfhW3boFMnKCwMt4EDw8++faF163RXnZzqwn306NByT0W476utW2HZsj0bgCVLoLQ01BzTrVvlfv8BA8IGwVv/LtVqatF70DdCmzdDScmeUH/lFfjwwzAvJweKimDIEMjNDUFUWhpCaNu2sEyLFiHsYxuA2O3gg9P3nuLFwn3OHJg3L4T7EUfs6ZZJZ7jXxiz8LhJb/0uXwvbtYZkjjgj7D77znfBNwLlU8KBvxHbsCGHxyit7gv3NN/fM79MHhg4NwT50aGgxtmy59/Ps3AmrVsGiRSH4Y7e1cQfKdutWOfgHDoTevcOGob6Vl+9puSeG++jRoZ5MDfdk7NgBK1eG3+PvfgcLF4YN6/e+B5dfHr55Obc/POgbyMyZcM01oRulZ0+44QYYNy759c1CX3B8qL/6Knz+eZh/8MGVQ/3LX4YOHfav5vXrKwd/aWn4FhA79DAnJ3Q1JLb+27ffv9eF6sM91i3T2MO9Ombw7LPwy1/C449D27Zw6aUwaVL4u3FuX3jQN4CZM2HChNBvG9OmDUybVn3Yb94cQj0+2GMjQOTkwODBe0J96NDQFdMQwbd9O7zxxt4bgPi+57y8vcM/Px+a1XKudSzc58wJYbdzZ/jWEOuWydZwr87ixXDzzTB7dtgAjBkTdi4XFqa7MtfYeNA3gLw8eOedvafn5oZW+vbt4Z861qf+8suwfPme5fr2rRzqBQVVd8GkixmsW7d318+KFXuOPDnwwNB1FB/+BQXw2Wfw17+Glnt8uMda7gMGNK1wr8q778Jtt4WGwWefwemnw09+Aqec4p+NS44HfQNo1iyEYVWOPTZ0wXzxRXgc64KJ3b785dR0haTD1q3huPPE1v8nn4T5Urjt2gVHHrmn5e7hXrVNm+DOO+E3v4GPPgrf6n7yEzj33IbZV+IaLw/6BlBdi16C//iPysHes2d2h1zsuPNY6AOcc46He118/jncdx/cckvYiRsbxuGCC0KXoEud9evDN9VOncJ5D507h8+4sf2tetA3gJkz4aKL9hxCB6Gf/a67YPz49NXlGredO+Fvf4P/+Z/Q5delC1x5JVxxRQgkt+9WrYJbb4U//nHPAQ8xOTnhs44Ff+x+TdPSfaisnxnbAL74Ihyp0rp1uJ+bW/ejbpxL1Lx56LY555wwFs8vfwlTpoTgv/hi+OEPw7dJl7xXXgk7wB98MOwH+/a3w07wzz4Lrfv424YN4eerr4afmzZV/7wHHFD7BiHxcU5Ow7xnb9GnwK23hq/Vw4eHnY5t26a7IpfNli4NXTozZ4ZustGjw5E6gwalu7LMtWsX/OMfIeDnzw+HJV9+efh21K1b8s9TURHCvqqNQXWPN2+u/vnatq0c/oMGwU037dt79K6bemIWWlfXXw+jRsGMGY1naAHX+JWVhZ22d98ddn6femrYcXvqqY2vf7m+fPFF2CDecks4ZPiww8L5CpdcEo4SawgVFeGiN9V9W4h/3KtXqHdf1BT0mFlG3QYPHmyNwc6dZldeaQZmF11kVlGR7opcU7Vpk9lNN5kdckj4exw40Oz++8127Eh3ZekT+0y6dQufSWGh2YwZZtu3p7uy+kMYNr7KXE3VpQSblIqKcPTD7beHPtJ77vFBqlz6dOgAP/1pONLpnnvCmEbnnRfOVbj99tD33FS8917oRj3sMJg8OZzV/c9/wmuvhf1lmXRuSkPyoK+jzz8Px4H/+c+hy+aWW/xrsssMrVuHHbTLloWx+g89NIyl07Nn6GKMnXWdjRYvDjtVDz88dGedfXbYgfrPf8Jpp/n/aFJBL2mEpOWSVkmaXMX8npLmSXpN0mJJZ8TNuzpab7mk01NZfEP79FP42tfCP9FvfwvXXut/QC7zNGsGI0fCCy/A88+HsfWnTg2Bf8UV8NZb6a4wNczCFcVOPz2chf3QQzBxYnh/M2f6zulKquvTid0Ilw98CzgcaAWUAv0SlpkGXB7d7wesibtfCrQG8qPnaV7T62VqH/2GDWZDh5o1b252773prsa5ulm2zOzii81atTJr1sxs1CizP/zBbNGixtdvvWNH2AcxaFDof//Sl8xuvNFs48Z0V5Ze1NBHn8xx9EOAVWa2GkDSbGAksCx+ewEcFN1vD7wf3R8JzDazL4C3Ja2Knu/Fum2O0mvdunDo5IoV8MAD8PWvp7si5+qmb9/Qfz91avg2Om1a+FuG0OVTWBiGW4jdjj468/qzP/0U/vAH+PWvw1noRx0Fv/89nH9+wx2P3lglE/TdgffiHpcBQxOWuQ74p6QrgbbAqXHrvpSwbvd9qjRN1qwJh6t98EE4DnfYsHRX5Ny+Oy1Fgj0AAA3nSURBVPTQcJz2jTeGoRUWLtxzmzEjjLMDIfwHDNg7/Fu1aviaP/gg7FS+885wDPvxx4eN1Zln1j5aqgtSdWbsWOBPZvYrSccCf5bUP9mVJU0AJgD0zKABuZctCztytm2Dp56CY45Jd0XOpUazZqFFfNRR4QgdCCcVrVq1J/hffRVmzQrDeEAI+cTw79+//sJ/+XL41a/CmD/bt4ezg3/8Y/8/3BfJBP1a4LC4xz2iafEuBkYAmNmLknKALkmui5lNI/TzU1xcnBFncJWUwIgR4evrv/4Vhtt1Lps1axZGGD3ySBg7NkzbtQtWr67c8p89O5ykBSHkCwpC6BcVhZ8FBft34uALL4QzWOfODc9/4YXhMObevff/PTZVtZ4ZK6kFsAIYRgjpBcB5ZrY0bpnHgP8zsz9J6gs8Teii6QfcT+iXPzSa3tvMdlb3eplwZuy//gVnnRVOTX7yyXDVI+dcYLZ3+C9cuOdU/5YtQ0s/vuU/YEDN4b9rVxi87eab4cUXw0iSV1wRjqLJlGsZZ7r9GtTMzCokTQSeIByBM93MlkqaStjLOxe4Cvi9pEmEHbMXRHuBl0qaQ9hxWwFcUVPIZ4JHHw3DGeTnh5Dv3qj2KDhX/6Rwqn6vXmGcHQjh//bblYP/wQfDDmAIY+lXFf4QumZ+9atwsEN+fuiPv/BCHzMqlXysmzizZsF3vhOOQHj88TDIkHNu38SugZzY8t+4Mcxv0SKM+/7xx36BlVTwYYqTcNdd8J//CSeeGPoGDzqo9nWcc9WTQgs9Pz98S4YQ/u+8syf0P/ggnNF68sl+8mF98qAnjO09eXI46/Uvf0n/BQScy1ZSGD8/Lw++8Y10V9N0NOmjUM3g6qtDyI8dG06h9pB3zmWbJtui37Ur7NW/6y647DL43e98BErnXHZqki36HTvCadN33RVa8//7vx7yzrns1eRa9Nu2hWGGH300nAr+05+muyLnnKtfTSroP/44jFM9f35ozX/3u+muyDnn6l+TCfr168OQBqWlYazq2CnezjmX7ZpE0K9dGwYne/vtcNGQr30t3RU551zDyfqgf+utMMzwhg3hbNeTTkp3Rc4517CyOuiXLAkXDNmxA+bNC6dZO+dcU5O1h1e+/HJovTdvHna+esg755qqrAz6p58OV4Lq1ClcHLlfv3RX5Jxz6ZN1Qf/ww3DGGWEgpeeeC2NqOOdcU5ZVQf/nP4dR8gYNChcP6dYt3RU551z6ZU3Qv/kmjB8fhjt96qnQbeOccy7JoJc0QtJySaskTa5i/q8lLYpuKyRtjpu3M27e3FQWH69Pn9Bt8/e/Q7t29fUqzjnX+NR6eKWk5sAdwGlAGbBA0lwzWxZbxswmxS1/JTAo7im2mdnA1JVcvbPPbohXcc65xiWZFv0QYJWZrTaz7cBsYGQNy48FZqWiOOecc/svmaDvDrwX97gsmrYXSblAPvBM3OQcSSWSXpL09X2u1Dnn3D5J9ZmxY4AHzGxn3LRcM1sr6XDgGUlLzOyt+JUkTQAmAPTs2TPFJTnnXNOWTIt+LXBY3OMe0bSqjCGh28bM1kY/VwPPUrn/PrbMNDMrNrPirl27JlGSc865ZCUT9AuA3pLyJbUihPleR89I6gN0BF6Mm9ZRUuvofhfgOGBZ4rrOOefqT61dN2ZWIWki8ATQHJhuZkslTQVKzCwW+mOA2WZmcav3Be6WtIuwUbkp/mgd55xz9U+Vczn9iouLraSkJN1lOOdcoyJpoZkVVzUva86Mdc45VzUPeuecy3Ie9M45l+U86J1zLst50DvnXJbzoHfOuSznQe+cc1nOg94557KcB71zzmU5D3rnnMtyHvTOOZflPOidcy7LedA751yW86B3zrks50HvnHNZzoPeOeeynAe9c85luaSCXtIIScslrZI0uYr5v5a0KLqtkLQ5bt54SSuj2/hUFu+cc652tV4zVlJz4A7gNKAMWCBpbvy1X81sUtzyVwKDovudgClAMWDAwmjdTSl9F84556qVTIt+CLDKzFab2XZgNjCyhuXHArOi+6cDT5rZxijcnwRG7E/Bzjnn6iaZoO8OvBf3uCyathdJuUA+8Exd1pU0QVKJpJLy8vJk6nbOOZekVO+MHQM8YGY767KSmU0zs2IzK+7atWuKS3LOuaYtmaBfCxwW97hHNK0qY9jTbVPXdZ1zztWDZIJ+AdBbUr6kVoQwn5u4kKQ+QEfgxbjJTwDDJXWU1BEYHk1zzjnXQGo96sbMKiRNJAR0c2C6mS2VNBUoMbNY6I8BZpuZxa27UdL1hI0FwFQz25jat+Ccc64misvljFBcXGwlJSXpLsM55xoVSQvNrLiqeX5mrHPOZTkPeuecy3Ie9M45l+U86J1zLst50DvnXJbzoHfOuSznQe+cc1nOg94557KcB71zzmU5D3rnnMtyHvTOOZflPOidcy7LedA751yW86B3zrks50HvnHNZzoPeOeeyXFJBL2mEpOWSVkmaXM0yoyUtk7RU0v1x03dKWhTd9roEoXPOufpV66UEJTUH7gBOA8qABZLmmtmyuGV6A1cDx5nZJkkHxz3FNjMbmOK6nXPOJSmZFv0QYJWZrTaz7cBsYGTCMpcCd5jZJgAz+yi1ZTrnnNtXyQR9d+C9uMdl0bR4RwJHSnpB0kuSRsTNy5FUEk3/elUvIGlCtExJeXl5nd6Ac865mtXadVOH5+kNnAz0AOZLKjCzzUCuma2VdDjwjKQlZvZW/MpmNg2YBuHi4CmqyTnnHMm16NcCh8U97hFNi1cGzDWzHWb2NrCCEPyY2dro52rgWWDQftbsnHOuDpIJ+gVAb0n5kloBY4DEo2ceJrTmkdSF0JWzWlJHSa3jph8HLMM551yDqbXrxswqJE0EngCaA9PNbKmkqUCJmc2N5g2XtAzYCfzYzDZI+g/gbkm7CBuVm+KP1nHOOVf/ZJZZXeLFxcVWUlKS7jKcc65RkbTQzIqrmudnxjrnXJbzoHfOuSznQe+cc1nOg94557KcB71zzmU5D3rnnMtyHvTOOZflPOidcy7LedA751yW86B3zrks50HvnHNZzoPeOeeynAe9c85lOQ9655zLch70zjmX5TzonXMuyyUV9JJGSFouaZWkydUsM1rSMklLJd0fN328pJXRbXyqCk80cybk5UGzZuHnzJn19UrOOde41HopQUnNgTuA0wgXAV8gaW78JQEl9QauBo4zs02SDo6mdwKmAMWAAQujdTel8k3MnAkTJsDWreHxO++ExwDjxqXylZxzrvFJpkU/BFhlZqvNbDswGxiZsMylwB2xADezj6LppwNPmtnGaN6TwIjUlL7HNdfsCfmYrVvDdOeca+qSCfruwHtxj8uiafGOBI6U9IKklySNqMO6SJogqURSSXl5efLVR959t27TnXOuKUnVztgWQG/gZGAs8HtJHZJd2cymmVmxmRV37dq1zi/es2fdpjvnXFOSTNCvBQ6Le9wjmhavDJhrZjvM7G1gBSH4k1l3v91wA7RpU3lamzZhunPONXXJBP0CoLekfEmtgDHA3IRlHia05pHUhdCVsxp4AhguqaOkjsDwaFpKjRsH06ZBbi5I4ee0ab4j1jnnIImjbsysQtJEQkA3B6ab2VJJU4ESM5vLnkBfBuwEfmxmGwAkXU/YWABMNbON9fFGxo3zYHfOuarIzNJdQyXFxcVWUlKS7jKcc65RkbTQzIqrmudnxjrnXJbzoHfOuSznQe+cc1nOg94557Jcxu2MlVQOvJPuOvZTF2B9uovIIP55VOafxx7+WVS2P59HrplVecZpxgV9NpBUUt3e76bIP4/K/PPYwz+Lyurr8/CuG+ecy3Ie9M45l+U86OvHtHQXkGH886jMP489/LOorF4+D++jd865LOcteuecy3Ie9M45l+U86FNI0mGS5sVdJP376a4p3SQ1l/SapL+nu5Z0k9RB0gOS3pT0hqRj011TOkmaFP2fvC5plqScdNfUkCRNl/SRpNfjpnWS9KSkldHPjql4LQ/61KoArjKzfsAxwBWS+qW5pnT7PvBGuovIEL8BHjezPkAhTfhzkdQd+B5QbGb9CUOgj0lvVQ3uT+x9De3JwNNm1ht4Onq83zzoU8jM1pnZq9H9Twj/yHtdI7epkNQD+BpwT7prSTdJ7YETgT8AmNl2M9uc3qrSrgVwgKQWQBvg/TTX06DMbD6QeH2OkcC90f17ga+n4rU86OuJpDxgEPByeitJq9uAnwC70l1IBsgHyoE/Rl1Z90hqm+6i0sXM1gK3AO8C64AtZvbP9FaVEb5kZuui+x8AX0rFk3rQ1wNJ7YAHgR+Y2cfpricdJJ0JfGRmC9NdS4ZoARQBd5rZIOAzUvS1vDGK+p5HEjaAhwJtJZ2f3qoyi4Vj31Ny/LsHfYpJakkI+Zlm9td015NGxwFnS1oDzAa+ImlGektKqzKgzMxi3/AeIAR/U3Uq8LaZlZvZDuCvwH+kuaZM8KGkbgDRz49S8aQe9CkkSYQ+2DfM7NZ015NOZna1mfUwszzCTrZnzKzJttjM7APgPUlHRZOGAcvSWFK6vQscI6lN9H8zjCa8czrOXGB8dH888LdUPKkHfWodB3yb0HpdFN3OSHdRLmNcCcyUtBgYCNyY5nrSJvpm8wDwKrCEkEVNajgESbOAF4GjJJVJuhi4CThN0krCt56bUvJaPgSCc85lN2/RO+dclvOgd865LOdB75xzWc6D3jnnspwHvXPOZTkPeuecy3Ie9M45l+X+H8yWeInJ6BL9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxU5dn/8c/FJiIICriBEFwqBEHECLigQtXiBopoQdyloD8VW5dHqtRHUev6qMVSlSLUJZpSqJZWLO2j1KVWJCBGFnlABAmgIiqKoBC4fn/cE5jELBOY5ExOvu/Xa16Zc+aec66ZJNfcc2/H3B0REan96kUdgIiIpIcSuohITCihi4jEhBK6iEhMKKGLiMSEErqISEwooUuZzKy+mW0ws3bpLBslMzvEzNI+TtfMTjaz5Unbi82sdypld+JcE8zslp19fgXHvcvM/pDu40rNahB1AJIeZrYhabMJ8D2wNbE9wt1zq3I8d98KNE132brA3Q9Lx3HMbBhwobuflHTsYek4tsSTEnpMuPv2hJqoAQ5z9/8tr7yZNXD3opqITURqhppc6ojEV+o/mtnzZvYNcKGZHWNmb5vZV2a2xszGmlnDRPkGZuZmlpXYfjbx+Mtm9o2Z/cfMOlS1bOLx08zs/8xsvZk9amb/NrNLy4k7lRhHmNlSM/vSzMYmPbe+mT1sZuvMbBnQr4L351Yzyyu1b5yZPZS4P8zMFiVez4eJ2nN5xyo0s5MS95uY2TOJ2BYAR5UqO9rMliWOu8DM+if2dwF+C/RONGd9nvTe3p70/CsTr32dmb1oZvun8t5UxszOScTzlZm9amaHJT12i5mtNrOvzeyDpNfay8zmJvZ/amYPpHo+SRN31y1mN2A5cHKpfXcBm4GzCB/kuwNHAz0J39QOAv4PuCZRvgHgQFZi+1ngcyAHaAj8EXh2J8ruA3wDDEg8dj2wBbi0nNeSSox/AZoDWcAXxa8duAZYALQFWgKvhz/5Ms9zELAB2CPp2J8BOYntsxJlDOgLbAK6Jh47GViedKxC4KTE/QeBfwF7Ae2BhaXKng/sn/idXJCIYd/EY8OAf5WK81ng9sT9UxMxdgMaA78DXk3lvSnj9d8F/CFxv1Mijr6J39EtwOLE/c7ACmC/RNkOwEGJ+7OBIYn7zYCeUf8v1LWbauh1y5vu/ld33+bum9x9trvPcvcid18GjAdOrOD5U9w93923ALmERFLVsmcC89z9L4nHHiYk/zKlGOM97r7e3ZcTkmfxuc4HHnb3QndfB9xbwXmWAfMJHzQApwBfunt+4vG/uvsyD14FXgHK7Pgs5XzgLnf/0t1XEGrdyeed7O5rEr+T5wgfxjkpHBdgKDDB3ee5+3fAKOBEM2ubVKa896Yig4Fp7v5q4nd0L+FDoSdQRPjw6Jxotvso8d5B+GA+1Mxauvs37j4rxdchaaKEXresTN4ws45m9pKZfWJmXwNjgFYVPP+TpPsbqbgjtLyyByTH4e5OqNGWKcUYUzoXoWZZkeeAIYn7FyS2i+M408xmmdkXZvYVoXZc0XtVbP+KYjCzS83svUTTxldAxxSPC+H1bT+eu38NfAm0SSpTld9ZecfdRvgdtXH3xcANhN/DZ4kmvP0SRS8DsoHFZvaOmZ2e4uuQNFFCr1tKD9l7glArPcTd9wRuIzQpVKc1hCYQAMzMKJmAStuVGNcAByZtVzascjJwspm1IdTUn0vEuDswBbiH0BzSAvhHinF8Ul4MZnYQ8BhwFdAycdwPko5b2RDL1YRmnOLjNSM07axKIa6qHLce4Xe2CsDdn3X34wjNLfUJ7wvuvtjdBxOa1f4HmGpmjXcxFqkCJfS6rRmwHvjWzDoBI2rgnH8DupvZWWbWALgOaF1NMU4Gfm5mbcysJXBzRYXd/RPgTeAPwGJ3X5J4aDegEbAW2GpmZwI/rkIMt5hZCwvj9K9JeqwpIWmvJXy2/YxQQy/2KdC2uBO4DM8DV5hZVzPbjZBY33D3cr/xVCHm/mZ2UuLcNxH6PWaZWScz65M436bEbRvhBVxkZq0SNfr1ide2bRdjkSpQQq/bbgAuIfyzPkHovKxW7v4p8FPgIWAdcDDwLmHcfLpjfIzQ1v0+ocNuSgrPeY7Qybm9ucXdvwJ+AbxA6FgcRPhgSsV/E74pLAdeBp5OOm4B8CjwTqLMYUByu/M/gSXAp2aW3HRS/Py/E5o+Xkg8vx2hXX2XuPsCwnv+GOHDph/QP9GevhtwP6Hf4xPCN4JbE089HVhkYRTVg8BP3X3zrsYjqbPQhCkSDTOrT/iKP8jd34g6HpHaTDV0qXFm1i/RBLEb8CvC6Ih3Ig5LpNZTQpcoHA8sI3yd/wlwjruX1+QiIilSk4uISEyohi4iEhORLc7VqlUrz8rKiur0IiK10pw5cz539zKH+kaW0LOyssjPz4/q9CIitZKZlTvjWU0uIiIxoYQuIhITSugiIjGhhC4iEhNK6CIiMVGrEnpuLmRlQb164WdulS57LCISb7XmItG5uTB8OGzcGLZXrAjbAEN3eX05EZHar9bU0G+9dUcyL7ZxY9gvIplr0SKYPz/qKDLDihVw++2wcGH1HL/WJPSPP67afhGJzqZN8MwzcPzxkJ0NXbpA9+4wbhx8+WXU0dWszZth6lTo1w86dIAxY+C116rnXLUmobcr5+Jh5e0XkZq3eDFcfz20bQsXXwyffQYPPgiPPgrucM01cMABoZn01VdhW4yvZ7R4Mdx0U3gvBg0KtfLbboOPPoKrrqqmk7p7JLejjjrKq+LZZ92bNHEPfxbh1qRJ2C8i0fn+e/e8PPc+fcL/ZYMG7ued5/7KK+5bt5YsO2eO+//7f+7Nm4eyBx3kfued7itXRhN7un37rftTT7n37r3jvRg40H36dPeiovScA8j3cvJqrUno7iF5t2/vbhZ+KpmLROfDD91HjXLfZ5+QSbKy3H/9a/c1ayp/7saN4f+3+EOgXj33005znzIlfEDUNnPnlvygOvRQ9/vuS+29qKqKEnpk66Hn5OS4FucSqV2KiuBvf4PHH4d//APM4KyzYMQIOPVUqF+/6sf88EOYNAn+8AdYtQpat4aLLoIrrgjt75nq66/huedgwgSYMwcaNw5NK8OGwQknhPemOpjZHHfPKfMxJXQRqczKlSFxTZgAq1dDmzYhcQ0bFtqI02Hr1vAh8eSTMG0abNkCvXqFxP7Tn0KzZuk5z65wh7feCu/D5MlhpF3XrvCzn4V+gb32qv4YlNBFpMq2boUZM+CJJ0Kt3B1+8hO48ko44wxoUI2zWNauDaNknnwydCbusQecf35I7sceW32138rimTAhDMNs2hQuuCB8oOXk1Gw8SugikrJPPoGJE2H8+DBuep99QiL92c/CsLua5A6zZoXEnpcHGzbAYYfB5ZeHUTT77Vd95962DV55JSTxF14I3xiOOSYk8fPPD0k9CkroIlKhbdtg5szQNv7ii6GtvG/fUBsfMAAaNYo6wpDM//SnkNz//e/QXn/mmeHD5rTT0veNobAwtOc/+SQsXw577x0+PK64Ag4/PD3n2BVK6CJSps8/D8nriSdg6dKQvC67LCyr8aMfRR1d+T74IHyLeOqpMNZ9//3hkktCzf3QQ6t+vC1bYPr0UBufPj18wP34x6E2fvbZocMzUyihi8h27vDmm6E2PmVKmMl4/PFhpMqgQZmVvCqzZQu89FKoTRcn4hNOCLXpQYOgSZOKn//hh+G5kyaFpqb99w8faJdfDgcfXDOvoaqU0EWEr76Cp58OtfGFC2HPPUNTwogRmdGUsKtWrw419okTw7eNPfeEIUNCck/uuPzuu9Am/vvfh2amevVCJ++wYXD66dXb2ZsOSugidZQ7vPNOSOJ5eWGNlaOPDm3jP/1pGD0SN+7w+uuh5j1lSnjNXbrApZeGtZ+eeQa++CJ08F5xRdjfpk3UUadOCV2kDnrzTbj2Wpg3LyTuoUNDbbx796gjqznr18Pzz4fknp8fOnfPOSfUxvv2DbXz2kYJXaSOWbYsNDM0bw433xzGTO+5Z9RRRWvJkjDxp1WrqCPZNRUl9AxvLRKRqtq4EQYODE0P//u/mdu5V9N2ZvRLbZPSFw4z62dmi81sqZmNKqfM+Wa20MwWmNlz6Q1TRFLhHiYAFRSEpgYl87ql0hq6mdUHxgGnAIXAbDOb5u4Lk8ocCvwSOM7dvzSzfaorYBEp3yOPhAWjfv3rcEEFqVtSqaH3AJa6+zJ33wzkAQNKlfkZMM7dvwRw98/SG6aIVObVV8MFFQYOhFFlfo+WuEslobcBViZtFyb2JfsR8CMz+7eZvW1mZdYNzGy4meWbWf7atWt3LmIR+YEVK8IwxB/9KMz8rOnFqyQzpGvQTgPgUOAkYAjwezNrUbqQu4939xx3z2ndunWaTi1St23aFGrlmzeHdVgyYZlZiUYqCX0VcGDSdtvEvmSFwDR33+LuHwH/R0jwIlKN3MMkoblzITc3s9dfkeqXSkKfDRxqZh3MrBEwGJhWqsyLhNo5ZtaK0ASzLI1xikgZxo0L0/nvuCOsPCh1W6UJ3d2LgGuAGcAiYLK7LzCzMWbWP1FsBrDOzBYCM4Gb3H1ddQUtImF6+y9+Af37w+jRUUcjmaBWzhT99tt4rkEhkqrCQjjqKGjRIqzV0rx51BFJTalopmitW8lg3Djo2DGsHCdSF333HZx7bpgR+uKLSuayQ61L6L16hWUyb7kl6khEap47XH11qJU//TR06hR1RJJJal1CP+ooGDkyLM7/n/9EHY1IzXriibDe9+jRYdVAkWS1sg19wwbIzg5fNefOhYYN0xycSAZ66y046SQ45RSYNi1cU1Pqnli1oUO42va4cTB/Pjz4YNTRiFS/1atDu3m7dvDss0rmUrZamdABzjor/IGPGROuCygSV5s3h+tjfvNN6ATda6+oI5JMVWsTOsDYseEKJFdeGTqLROLouutCf9GkSfG49qdUn1qd0A84AO65Jyzin5sbdTQi6TdhQhgAcPPNcN55UUcjma5Wdoom27YNjj02NLt88AG0bJmG4EQywKxZcMIJoSN0+nS1m0sQu07RZPXqwfjxYaLRTTdFHY1Ienz6aegjOuCAcMEKJXNJRa1P6ABdu8INN4Q2xn/9K+poRHbNli2heeWLL+CFF/StU1IXi4QOcNtt0KFD6CD9/vuooxHZeTfcAG+8EdrPu3WLOhqpTWKT0Js0CZ1HixeHjlKR2uipp+DRR+H66+GCC6KORmqb2CR0gFNPDf8E99wTOkhFapM5c2DECOjbF+67L+popDaKVUIHePjhsLTuiBFhBIxIbbB2bVibZd99IS8PGjSIOiKpjWKX0PfZBx54ICz+P2lS1NGIVK6oKFzg+bPP4M9/Bl1uV3ZW7BI6wOWXh/G7N90U/klEMtnNN8PMmWH47VFHRR2N1GaxTOhmoYN0w4ZwiS6RTPXcc/DQQ3DttXDxxVFHI7VdLBM6hIX/f/nL8A8zY0bU0Yj80Lx5MGwY9O4N//M/UUcjcVDrp/5X5Lvv4IgjwkSN+fPD0EaRTLBuHeTkhL/NOXNCZ6hIKmI99b8ijRuHK7x89BHceWfU0YgEW7fCkCFhjfOpU5XMJX1indAhLGx02WXhQhjvvx91NCJw663wz3/C734HPXtGHY3ESUoJ3cz6mdliM1tqZqPKePxSM1trZvMSt2HpD3XnPfAAtGgBw4drbLpEa/LkMGnoyivhiiuijkbiptKEbmb1gXHAaUA2MMTMssso+kd375a4TUhznLukZcsw4ejtt8PoF5EovP9++LZ47LHwm99EHY3EUSo19B7AUndf5u6bgTxgQPWGlX5Dh4aL6/7yl6HtUqQmffllmAnavDlMmRKutCWSbqkk9DbAyqTtwsS+0s41swIzm2JmB5Z1IDMbbmb5Zpa/du3anQh355nBY4+F6zOOHFmjp5Y6buvWUKH4+OOQzPffP+qIJK7S1Sn6VyDL3bsC/wSeKquQu4939xx3z2kdwfzmgw+GX/0qjCz4619r/PRSR91+O7z8crgG7rHHRh2NxFkqCX0VkFzjbpvYt527r3P34lXIJwAZO4H5xhuhc2e4+uowk1SkOr3wAtx1V+gAHTEi6mgk7lJJ6LOBQ82sg5k1AgYD05ILmFnyl8j+wKL0hZhejRqFNTNWrgy1dZHqsmhRmM7fowf89reh2U+kOlWa0N29CLgGmEFI1JPdfYGZjTGz/oliI81sgZm9B4wELq2ugNPh2GPDsLGxY8MsPZF0W78ezj47zE6eOjVMchOpbrGe+l+Rr74K670ccEC4urrWn5Z02bIlXOD55ZfhlVfCyp8i6VJnp/5XpEWLUEOfOzdc8ktkZ61aFWrhN90Uknfz5qHT/aGHlMylZtXZGjqAO5x1FvzrX7BwIbRrF2k4Ugts3BgqAW+/HW6zZkFhYXisUSPo3j1M5+/bN/xtqd1c0q2iGnqdbmgwg3HjIDs7jHqZNk3/gLKDOyxZsiNxv/02vPdeGFcO0KFDWPq2V6+QxLt1g912izZmqdvqdEIHaN8+rMR4ww3ha/OgQVFHJFH58kt4550dCXzWLPjii/BYs2ZhtMrNN+9I4PvsE228IqXV6SaXYkVF4Z/1k0/CULPmzaOOSKpbUVFYW6W45v3227B4cXjMLMxV6NVrR/Lu1Anq1482ZhFQk0ulGjQIY9N79gxrvfzud1FHJOm2enXJppP8/NAeDqGm3bNnGDPeq1e48MSee0Ybr8jOUEJPyMkJ13UcOxYuugiOOSbqiGRnbdr0w47LlYnViBo1giOPhJ/9LCTxXr0gK0t9JxIPanJJ8s03oYO0RYuQEBo2jDoiSVVREeTlhRmZc+aEbQgdl8XNJr16qeNSaj81uaSoWbMw6mXAgHDR3lE/uJSHZJrNm+Hpp+Gee2DZMjj8cPiv/woJvGdPXd5N6hYl9FL694eBA+GOO+C888IKjZJ5Nm2CJ5+E++8PzSk5OeEiJmeeCfXq7HQ5qev0p1+GsWNDc8tVV4WxyJI5NmwI14ft0CH0ebRvDzNmhOGG/fsrmUvdpj//MrRpE77C//Of8NxzUUcjENbeueuukMBvugm6dAkzfN94A049VZ2aIqCEXq4rrwxtsL/4xY7JJVLzPv88LHPcvn34eeyx8J//hA/bE0+MOjqRzKKEXo769cPY9C++CDVCqVmffBLe96wsuPvuUAt/992w6FWvXlFHJ5KZlNAr0LVrWBJg4kR47bWoo6kbPv44tI1nZYXVCs85B+bPhz/9KQw5FJHyKaFX4r//O3TAjRgB3ycuspebGxJOvXrhZ25ulBHGw4cfhsk+hxwCTzwRJnctXgzPPBPmBohI5TRssRJNmsBjj0G/fnDvvSHhDB++Y9r4ihVhG8KV3aVqFi2CX/86dD43bBg+OG+6SUsZi+wMzRRN0QUXhNUYW7UK64KU1r49LF9e42HVWvPmhbbxqVPDh+ZVV8H118P++1f+XJG6TFcsSoOHHw6Jp6xkDqHtVyo3a1a48MORR8I//gG33ho+CB94QMlcZFcpoado331D0imPmggq9tprcMopYYTKW2+FNehXrAg/W7WKOjqReFBCr4LLL4fDDvvh/iZNQvOBlOQeauEnnAAnnRTWH3/ggZDIR48Oi6CJSPoooVdBvXrwwgthjPoee4TZie3bh/Hq6hDdYdu2cDm/nj3hJz+Bjz4KF+L+6CO48UZo2jTqCEXiKaVRLmbWD/gNUB+Y4O73llPuXGAKcLS7154ezyro1AluuSU0FUydGsZGb94crjX5/fc/vG3eXPb+dD7WoEFIksW3Zs1Kblf11qxZWDe8qtPpt24N78ndd0NBARx0EPz+9+HCEY0aVc/vQ0R2qDShm1l9YBxwClAIzDazae6+sFS5ZsB1wKzqCDST3HIL/PGPcO65u3acRo3C2tzFt9Lbu+0WvgnstdcP9yeX37o1rOW+YUPJ29q1JbeLh1qmovSHRGU3szABa/Fi6NgxjB8fPDgcR0RqRir/bj2Ape6+DMDM8oABwMJS5e4E7gNiP1G+cWN49VV46aWyk3B5yTl5/87UgHfV1q0hqZdO/OXdyvqQWLXqh/uKR74ecUSY0XnOObr+pkgUUknobYCVSduFQM/kAmbWHTjQ3V8ys3ITupkNB4YDtKvlw0LatNkxoai2qF8/NKc0a5a+Y7qHtck3boSWLbXqoUiUdrlT1MzqAQ8BN1RW1t3Hu3uOu+e0bt16V08tGcAsjPJp1UrJXCRqqST0VcCBSdttE/uKNQMOB/5lZsuBXsA0MytzJpOIiFSPVBL6bOBQM+tgZo2AwcC04gfdfb27t3L3LHfPAt4G+sd1lIuISKaqNKG7exFwDTADWARMdvcFZjbGzPpXd4AiIpKalAaVuft0YHqpfbeVU/akXQ9LRESqSjNFRURiQgldRCQmlNBFRGJCCV1EJCaU0EVEYkIJXUQkJpTQRURiQgldRCQmlNBFRGJCCV1EJCaU0EVEYkIJXUQkJpTQRURiQgldRCQmlNBFRGJCCV1EJCaU0EVEYkIJXUQkJpTQRURiQgldRCQmlNBFRGJCCV1EJCZSSuhm1s/MFpvZUjMbVcbjV5rZ+2Y2z8zeNLPs9IcqIiIVqTShm1l9YBxwGpANDCkjYT/n7l3cvRtwP/BQ2iMVEZEKpVJD7wEsdfdl7r4ZyAMGJBdw96+TNvcAPH0hiohIKhqkUKYNsDJpuxDoWbqQmV0NXA80AvqWdSAzGw4MB2jXrl1VYxURkQqkrVPU3ce5+8HAzcDocsqMd/ccd89p3bp1uk4tIiKkltBXAQcmbbdN7CtPHnD2rgQlqcnNhawsqFcv/MzNjToiEYlSKgl9NnComXUws0bAYGBacgEzOzRp8wxgSfpClLLk5sLw4bBiBbiHn8OHK6mL1GWVJnR3LwKuAWYAi4DJ7r7AzMaYWf9EsWvMbIGZzSO0o19SbRELALfeChs3lty3cWPYLyJ1k7lHMyAlJyfH8/PzIzl3HNSrF2rmpZnBtm01H4+I1Awzm+PuOWU9ppmitVR5g4Q0eEik7lJCr6XuvhuaNCm5r0mTsF9E6iYl9Fpq6FAYPx7atw/NLO3bh+2hQ6OOTESiksrEIslQQ4cqgYvIDqqhi4jEhBK6iEhMKKGLiMSEErqISEwooYuIxIQSuohITCihi4jEhBK6iEhMKKGLiMSEErqISEwooYuIxIQSuohITCihi4jEhBK6iEhMKKGLiMSEErqISEwooYuIxIQSuohITKSU0M2sn5ktNrOlZjaqjMevN7OFZlZgZq+YWfv0hyqZKjcXsrKgXr3wMzc36ohE6qZKE7qZ1QfGAacB2cAQM8suVexdIMfduwJTgPvTHahkptxcGD4cVqwA9/Bz+HAldZEopFJD7wEsdfdl7r4ZyAMGJBdw95nuvjGx+TbQNr1hSqa69VbYuLHkvo0bw34RqVmpJPQ2wMqk7cLEvvJcAbxc1gNmNtzM8s0sf+3atalHKRnr44+rtl9Eqk9aO0XN7EIgB3igrMfdfby757h7TuvWrdN5aolIu3ZV2y8i1SeVhL4KODBpu21iXwlmdjJwK9Df3b9PT3iS6e6+G5o0KbmvSZOwX0RqVioJfTZwqJl1MLNGwGBgWnIBMzsSeIKQzD9Lf5iSqYYOhfHjoX17MAs/x48P+0WkZjWorIC7F5nZNcAMoD4w0d0XmNkYIN/dpxGaWJoCfzIzgI/dvX81xi0ZZOhQJXCRTFBpQgdw9+nA9FL7bku6f3Ka4xIRkSrSTFERkZhQQhcRiQkldBGRmFBCFxGJCSV0EZGYUEIXEYkJJXQRkZhQQhcRiQkldBGRmFBCFxGJCSV0iQ1dCk/qupTWchHJdMWXwiu+elLxpfBAC4dJ3aEausSCLoUnooQuMaFL4YkooUtM6FJ4IkroEhO6FJ6IErrEhC6FJ6JRLhIjuhSe1HWqoYuIxIQSuohITCihi4jEREpt6GbWD/gNUB+Y4O73lnr8BOARoCsw2N2n7EwwW7ZsobCwkO+++25nni41rHHjxrRt25aGDRtGHYqIkEJCN7P6wDjgFKAQmG1m09x9YVKxj4FLgRt3JZjCwkKaNWtGVlYWZrYrh5Jq5u6sW7eOwsJCOnToEHU4IkJqTS49gKXuvszdNwN5wIDkAu6+3N0LgG27Esx3331Hy5YtlcxrATOjZcuW+jYlkkFSSehtgJVJ24WJfdVCybz20O/qh7Tio0SpRsehm9lwYDhAO83JlpjRio8StVRq6KuAA5O22yb2VZm7j3f3HHfPad269c4cooR014bWrVtHt27d6NatG/vttx9t2rTZvr158+aUjnHZZZexePHiCsuMGzeO3DRV3Y4//njmzZuXlmPJrtGKjxK1VGros4FDzawDIZEPBi6o1qhSUB21oZYtW25PjrfffjtNmzblxhtL9vO6O+5OvXplfxZOmjSp0vNcffXVOxegZDSt+ChRq7SG7u5FwDXADGARMNndF5jZGDPrD2BmR5tZIXAe8ISZLajOoKFma0NLly4lOzuboUOH0rlzZ9asWcPw4cPJycmhc+fOjBkzZnvZ4hpzUVERLVq0YNSoURxxxBEcc8wxfPbZZwCMHj2aRx55ZHv5UaNG0aNHDw477DDeeustAL799lvOPfdcsrOzGTRoEDk5OZXWxJ999lm6dOnC4Ycfzi233AJAUVERF1100fb9Y8eOBeDhhx8mOzubrl27cuGFF6b9PauLtOKjRC2lNnR3nw5ML7XvtqT7swlNMTWmpmtDH3zwAU8//TQ5OTkA3Hvvvey9994UFRXRp08fBg0aRHZ2donnrF+/nhNPPJF7772X66+/nokTJzJq1KgfHNvdeeedd5g2bRpjxozh73//O48++ij77bcfU6dO5b333qN79+4VxldYWMjo0aPJz8+nefPmnHzyyfztb3+jdevWfP7557z//vsAfPXVVwDcf//9rFixgkaNGm3fJ7vm7rtLfmsErfgoNavWzhSt6drQwQcfvD2ZAzz//PN0796d7t27s2jRIhYuXPiD5+y+++6cdtppABx11FEsX768zGMPHDjwB2XefPNNBg8eDMARRxxB586dK4xv1qxZ9DIAw9UAAAueSURBVO3bl1atWtGwYUMuuOACXn/9dQ455BAWL17MyJEjmTFjBs2bNwegc+fOXHjhheTm5mpiUJpoxUeJWq1N6DW9/vUee+yx/f6SJUv4zW9+w6uvvkpBQQH9+vUrczx2o0aNtt+vX78+RUVFZR57t912q7TMzmrZsiUFBQX07t2bcePGMWLECABmzJjBlVdeyezZs+nRowdbt25N63nrqqFDYfly2LYt/FQyl5pUaxN6lLWhr7/+mmbNmrHnnnuyZs0aZsyYkfZzHHfccUyePBmA999/v8xvAMl69uzJzJkzWbduHUVFReTl5XHiiSeydu1a3J3zzjuPMWPGMHfuXLZu3UphYSF9+/bl/vvv5/PPP2dj6Q4JEal1avV66FGtf929e3eys7Pp2LEj7du357jjjkv7Oa699louvvhisrOzt9+Km0vK0rZtW+68805OOukk3J2zzjqLM844g7lz53LFFVfg7pgZ9913H0VFRVxwwQV88803bNu2jRtvvJFmzZql/TVIdHJzwwCBjz8OzZB3361vC3WBuXskJ87JyfH8/PwS+xYtWkSnTp0iiSfTFBUVUVRUROPGjVmyZAmnnnoqS5YsoUGDzPoM1u8s85Qe0guhOVLt+fFgZnPcPaesxzIrO8h2GzZs4Mc//jFFRUW4O0888UTGJXPJTBUN6VVCjzdliAzVokUL5syZE3UYUgtpglPdVWs7RUWkbJrgVHcpoYvETE0P6ZXMoYQuEjOZNMFJywnXLLWhi8RQVEN6k2k54ZqnGnqSPn36/GCS0COPPMJVV11V4fOaNm0KwOrVqxk0aFCZZU466SRKD9Ms7ZFHHikxwef0009Pyzort99+Ow8++OAuH0ekKrSccM1TQk8yZMgQ8vLySuzLy8tjyJAhKT3/gAMOYMqUnbo+NvDDhD59+nRatGix08cTiVImjbapK00/Gdvk8vOfQ7qv29CtGyRWrS3ToEGDGD16NJs3b6ZRo0YsX76c1atX07t3bzZs2MCAAQP48ssv2bJlC3fddRcDBpS4tCrLly/nzDPPZP78+WzatInLLruM9957j44dO7Jp06bt5a666ipmz57Npk2bGDRoEHfccQdjx45l9erV9OnTh1atWjFz5kyysrLIz8+nVatWPPTQQ0ycOBGAYcOG8fOf/5zly5dz2mmncfzxx/PWW2/Rpk0b/vKXv7D77ruX+xrnzZvHlVdeycaNGzn44IOZOHEie+21F2PHjuXxxx+nQYMGZGdnk5eXx2uvvcZ1110HhMvNvf7665pRKilr1y40s5S1vybVpaYf1dCT7L333vTo0YOXX34ZCLXz888/HzOjcePGvPDCC8ydO5eZM2dyww03UNEs28cee4wmTZqwaNEi7rjjjhJjyu+++27y8/MpKCjgtddeo6CggJEjR3LAAQcwc+ZMZs6cWeJYc+bMYdKkScyaNYu3336b3//+97z77rtAWCjs6quvZsGCBbRo0YKpU6dW+Bovvvhi7rvvPgoKCujSpQt33HEHEJYDfvfddykoKODxxx8H4MEHH2TcuHHMmzePN954o8IPCpHSMmW0TV1q+snYGnpFNenqVNzsMmDAAPLy8njyySeBsGb5Lbfcwuuvv069evVYtWoVn376Kfvtt1+Zx3n99dcZOXIkAF27dqVr167bH5s8eTLjx4+nqKiINWvWsHDhwhKPl/bmm29yzjnnbF/xceDAgbzxxhv079+fDh060K1bN6DiJXohrM/+1VdfceKJJwJwySWXcN55522PcejQoZx99tmcffbZQFgg7Prrr2fo0KEMHDiQtm1rdMl7qeWKa79RrymTSU0/1U019FIGDBjAK6+8wty5c9m4cSNHHXUUALm5uaxdu5Y5c+Ywb9489t133zKXzK3MRx99xIMPPsgrr7xCQUEBZ5xxxk4dp1jx0ruwa8vvvvTSS1x99dXMnTuXo48+mqKiIkaNGsWECRPYtGkTxx13HB988MFOxyl1UyYsJ5xJE62quy1fCb2Upk2b0qdPHy6//PISnaHr169nn332oWHDhsycOZMVZTUOJjnhhBN47rnnAJg/fz4FBQVAWHp3jz32oHnz5nz66afbm3cAmjVrxjfffPODY/Xu3ZsXX3yRjRs38u233/LCCy/Qu3fvKr+25s2bs9dee/HGG28A8Mwzz3DiiSeybds2Vq5cSZ8+fbjvvvtYv349GzZs4MMPP6RLly7cfPPNHH300UroUitlStNPcVv+ihXgvqMtP51JPWObXKI0ZMgQzjnnnBIjXoYOHcpZZ51Fly5dyMnJoWPHjhUe46qrruKyyy6jU6dOdOrUaXtN/4gjjuDII4+kY8eOHHjggSWW3h0+fDj9+vXb3pZerHv37lx66aX06NEDCJ2iRx55ZIXNK+V56qmntneKHnTQQUyaNImtW7dy4YUXsn79etydkSNH0qJFC371q18xc+ZM6tWrR+fOnbdffUmkNsmUpp+aWDRNy+fKLtHvTCQ19eqFmnlpZqFJKlUVLZ+rJhcRkRpQE235SugiIjWgJtryU0roZtbPzBab2VIzG1XG47uZ2R8Tj88ys6ydDSiqJiCpOv2uRFJXE4umVZrQzaw+MA44DcgGhphZdqliVwBfuvshwMPAfTsTTOPGjVm3bp0SRS3g7qxbt47GjRtHHYpIrVHdwzhTGeXSA1jq7ssAzCwPGAAkX4Z+AHB74v4U4LdmZl7FzNy2bVsKCwtZu3ZtVZ4mEWncuLEmG4lkkFQSehtgZdJ2IdCzvDLuXmRm64GWwOfJhcxsODAcoF0ZPQENGzakQ4cOqcYuIiJJarRT1N3Hu3uOu+e0bt26Jk8tIhJ7qST0VcCBSdttE/vKLGNmDYDmwLp0BCgiIqlJJaHPBg41sw5m1ggYDEwrVWYacEni/iDg1aq2n4uIyK5JaaaomZ0OPALUBya6+91mNgbId/dpZtYYeAY4EvgCGFzciVrBMdcCFS+IkvlaUaqfoI7T+7GD3ouS9H6UtCvvR3t3L7PNOrKp/3FgZvnlTcGti/R+7KD3oiS9HyVV1/uhmaIiIjGhhC4iEhNK6LtmfNQBZBi9HzvovShJ70dJ1fJ+qA1dRCQmVEMXEYkJJXQRkZhQQt8JZnagmc00s4VmtsDMros6pqiZWX0ze9fM/hZ1LFEzsxZmNsXMPjCzRWZ2TNQxRcnMfpH4P5lvZs8n5q3UCWY20cw+M7P5Sfv2NrN/mtmSxM+90nU+JfSdUwTc4O7ZQC/g6jKWFK5rrgMWRR1EhvgN8Hd37wgcQR1+X8ysDTASyHH3wwmTEwdHG1WN+gPQr9S+UcAr7n4o8EpiOy2U0HeCu69x97mJ+98Q/mHbRBtVdMysLXAGMCHqWKJmZs2BE4AnAdx9s7t/FW1UkWsA7J5Y56kJsDrieGqMu79OmD2fbADwVOL+U8DZ6TqfEvouSlyd6UhgVrSRROoR4L+AKlzqNrY6AGuBSYkmqAlmtkfUQUXF3VcBDwIfA2uA9e7+j2ijity+7r4mcf8TYN90HVgJfReYWVNgKvBzd/866niiYGZnAp+5+5yoY8kQDYDuwGPufiTwLWn8Sl3bJNqHBxA+6A4A9jCzC6ONKnMkFjFM29hxJfSdZGYNCck8193/HHU8EToO6G9my4E8oK+ZPRttSJEqBArdvfgb2xRCgq+rTgY+cve17r4F+DNwbMQxRe1TM9sfIPHzs3QdWAl9J5iZEdpIF7n7Q1HHEyV3/6W7t3X3LEJn16vuXmdrYO7+CbDSzA5L7PoxJS/XWNd8DPQysyaJ/5sfU4c7iROSlxu/BPhLug6shL5zjgMuItRG5yVup0cdlGSMa4FcMysAugG/jjieyCS+qUwB5gLvE3JOnVkGwMyeB/4DHGZmhWZ2BXAvcIqZLSF8g7k3befT1H8RkXhQDV1EJCaU0EVEYkIJXUQkJpTQRURiQgldRCQmlNBFRGJCCV1EJCb+Pw+4VD9hnfnxAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(acc) + 1)\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}