deep-learning-with-python-notebooks/3.5-classifying-movie-reviews.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "原文代码作者：François Chollet\n",
    "\n",
    "github：https://github.com/fchollet/deep-learning-with-python-notebooks\n",
    "\n",
    "中文注释制作：黄海广\n",
    "\n",
    "github：https://github.com/fengdu78\n",
    "\n",
    "代码全部测试通过。\n",
    "\n",
    "配置环境：keras 2.2.1（原文是2.0.8，运行结果一致），tensorflow 1.8，python 3.6，\n",
    "\n",
    "主机：显卡：一块1080ti；内存：32g（注：绝大部分代码不需要GPU）\n",
    "![公众号](data/gongzhong.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'2.2.1'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import keras\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Classifying movie reviews: a binary classification example\n",
    "# 电影评论分类：二分类问题\n",
    "\n",
    "This notebook contains the code samples found in Chapter 3, Section 5 of [Deep Learning with Python](https://www.manning.com/books/deep-learning-with-python?a_aid=keras&a_bid=76564dff). Note that the original text features far more content, in particular further explanations and figures: in this notebook, you will only find source code and related comments.\n",
    "\n",
    "----\n",
    "\n",
    "\n",
    "Two-class classification, or binary classification, may be the most widely applied kind of machine learning problem. In this example, we \n",
    "will learn to classify movie reviews into \"positive\" reviews and \"negative\" reviews, just based on the text content of the reviews.\n",
    "\n",
    "二分类问题可能是应用最广泛的机器学习问题。在这个例子中，你将学习根据电影评论的文字内容将其划分为正面或负面。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The IMDB dataset\n",
    "## IMDB 数据集\n",
    "\n",
    "We'll be working with \"IMDB dataset\", a set of 50,000 highly-polarized reviews from the Internet Movie Database. They are split into 25,000 \n",
    "reviews for training and 25,000 reviews for testing, each set consisting in 50% negative and 50% positive reviews.\n",
    "\n",
    "Why do we have these two separate training and test sets? You should never test a machine learning model on the same data that you used to \n",
    "train it! Just because a model performs well on its training data doesn't mean that it will perform well on data it has never seen, and \n",
    "what you actually care about is your model's performance on new data (since you already know the labels of your training data -- obviously \n",
    "you don't need your model to predict those). For instance, it is possible that your model could end up merely _memorizing_ a mapping between \n",
    "your training samples and their targets -- which would be completely useless for the task of predicting targets for data never seen before. \n",
    "We will go over this point in much more detail in the next chapter.\n",
    "\n",
    "Just like the MNIST dataset, the IMDB dataset comes packaged with Keras. It has already been preprocessed: the reviews (sequences of words) \n",
    "have been turned into sequences of integers, where each integer stands for a specific word in a dictionary.\n",
    "\n",
    "The following code will load the dataset (when you run it for the first time, about 80MB of data will be downloaded to your machine):\n",
    "\n",
    "本节使用 IMDB 数据集，它包含来自互联网电影数据库（IMDB）的 50 000 条严重两极分化的评论。数据集被分为用于训练的 25 000 条评论与用于测试的 25 000 条评论，训练集和测试集都包含 50% 的正面评论和 50% 的负面评论。\n",
    "\n",
    "为什么要将训练集和测试集分开？因为你不应该将训练机器学习模型的同一批数据再用于测试模型！模型在训练数据上的表现很好，并不意味着它在前所未见的数据上也会表现得很好，而且你真正关心的是模型在新数据上的性能（因为你已经知道了训练数据对应的标签，显然不再需要模型来进行预测）。例如，你的模型最终可能只是记住了训练样本和目标值之间的映射关系，但这对在前所未见的数据上进行预测毫无用处。下一章将会更详细地讨论这一点。\n",
    "\n",
    "与 MNIST 数据集一样，IMDB 数据集也内置于 Keras 库。它已经过预处理：评论（单词序列） 已经被转换为整数序列，其中每个整数代表字典中的某个单词。\n",
    "下列代码将会加载 IMDB 数据集（第一次运行时会下载大约 80MB 的数据，可以不翻墙，反复试几次）。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.datasets import imdb\n",
    "\n",
    "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The argument `num_words=10000` means that we will only keep the top 10,000 most frequently occurring words in the training data. Rare words \n",
    "will be discarded. This allows us to work with vector data of manageable size.\n",
    "\n",
    "The variables `train_data` and `test_data` are lists of reviews, each review being a list of word indices (encoding a sequence of words). \n",
    "`train_labels` and `test_labels` are lists of 0s and 1s, where 0 stands for \"negative\" and 1 stands for \"positive\":\n",
    "\n",
    "参数 num_words=10000 的意思是仅保留训练数据中前 10 000 个最常出现的单词。低频单词将被舍弃。这样得到的向量数据不会太大，便于处理。\n",
    "\n",
    "train_data 和 test_data 这两个变量都是评论组成的列表，每条评论又是单词索引组成 的列表（表示一系列单词）。train_labels 和 test_labels 都是 0 和 1 组成的列表，其中 0 代表负面（negative），1 代表正面（positive）。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1,\n",
       " 14,\n",
       " 22,\n",
       " 16,\n",
       " 43,\n",
       " 530,\n",
       " 973,\n",
       " 1622,\n",
       " 1385,\n",
       " 65,\n",
       " 458,\n",
       " 4468,\n",
       " 66,\n",
       " 3941,\n",
       " 4,\n",
       " 173,\n",
       " 36,\n",
       " 256,\n",
       " 5,\n",
       " 25,\n",
       " 100,\n",
       " 43,\n",
       " 838,\n",
       " 112,\n",
       " 50,\n",
       " 670,\n",
       " 2,\n",
       " 9,\n",
       " 35,\n",
       " 480,\n",
       " 284,\n",
       " 5,\n",
       " 150,\n",
       " 4,\n",
       " 172,\n",
       " 112,\n",
       " 167,\n",
       " 2,\n",
       " 336,\n",
       " 385,\n",
       " 39,\n",
       " 4,\n",
       " 172,\n",
       " 4536,\n",
       " 1111,\n",
       " 17,\n",
       " 546,\n",
       " 38,\n",
       " 13,\n",
       " 447,\n",
       " 4,\n",
       " 192,\n",
       " 50,\n",
       " 16,\n",
       " 6,\n",
       " 147,\n",
       " 2025,\n",
       " 19,\n",
       " 14,\n",
       " 22,\n",
       " 4,\n",
       " 1920,\n",
       " 4613,\n",
       " 469,\n",
       " 4,\n",
       " 22,\n",
       " 71,\n",
       " 87,\n",
       " 12,\n",
       " 16,\n",
       " 43,\n",
       " 530,\n",
       " 38,\n",
       " 76,\n",
       " 15,\n",
       " 13,\n",
       " 1247,\n",
       " 4,\n",
       " 22,\n",
       " 17,\n",
       " 515,\n",
       " 17,\n",
       " 12,\n",
       " 16,\n",
       " 626,\n",
       " 18,\n",
       " 2,\n",
       " 5,\n",
       " 62,\n",
       " 386,\n",
       " 12,\n",
       " 8,\n",
       " 316,\n",
       " 8,\n",
       " 106,\n",
       " 5,\n",
       " 4,\n",
       " 2223,\n",
       " 5244,\n",
       " 16,\n",
       " 480,\n",
       " 66,\n",
       " 3785,\n",
       " 33,\n",
       " 4,\n",
       " 130,\n",
       " 12,\n",
       " 16,\n",
       " 38,\n",
       " 619,\n",
       " 5,\n",
       " 25,\n",
       " 124,\n",
       " 51,\n",
       " 36,\n",
       " 135,\n",
       " 48,\n",
       " 25,\n",
       " 1415,\n",
       " 33,\n",
       " 6,\n",
       " 22,\n",
       " 12,\n",
       " 215,\n",
       " 28,\n",
       " 77,\n",
       " 52,\n",
       " 5,\n",
       " 14,\n",
       " 407,\n",
       " 16,\n",
       " 82,\n",
       " 2,\n",
       " 8,\n",
       " 4,\n",
       " 107,\n",
       " 117,\n",
       " 5952,\n",
       " 15,\n",
       " 256,\n",
       " 4,\n",
       " 2,\n",
       " 7,\n",
       " 3766,\n",
       " 5,\n",
       " 723,\n",
       " 36,\n",
       " 71,\n",
       " 43,\n",
       " 530,\n",
       " 476,\n",
       " 26,\n",
       " 400,\n",
       " 317,\n",
       " 46,\n",
       " 7,\n",
       " 4,\n",
       " 2,\n",
       " 1029,\n",
       " 13,\n",
       " 104,\n",
       " 88,\n",
       " 4,\n",
       " 381,\n",
       " 15,\n",
       " 297,\n",
       " 98,\n",
       " 32,\n",
       " 2071,\n",
       " 56,\n",
       " 26,\n",
       " 141,\n",
       " 6,\n",
       " 194,\n",
       " 7486,\n",
       " 18,\n",
       " 4,\n",
       " 226,\n",
       " 22,\n",
       " 21,\n",
       " 134,\n",
       " 476,\n",
       " 26,\n",
       " 480,\n",
       " 5,\n",
       " 144,\n",
       " 30,\n",
       " 5535,\n",
       " 18,\n",
       " 51,\n",
       " 36,\n",
       " 28,\n",
       " 224,\n",
       " 92,\n",
       " 25,\n",
       " 104,\n",
       " 4,\n",
       " 226,\n",
       " 65,\n",
       " 16,\n",
       " 38,\n",
       " 1334,\n",
       " 88,\n",
       " 12,\n",
       " 16,\n",
       " 283,\n",
       " 5,\n",
       " 16,\n",
       " 4472,\n",
       " 113,\n",
       " 103,\n",
       " 32,\n",
       " 15,\n",
       " 16,\n",
       " 5345,\n",
       " 19,\n",
       " 178,\n",
       " 32]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_labels[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we restricted ourselves to the top 10,000 most frequent words, no word index will exceed 10,000:\n",
    "\n",
    "由于限定为前 10 000 个最常见的单词，单词索引都不会超过 10 000。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9999"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "max([max(sequence) for sequence in train_data])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For kicks, here's how you can quickly decode one of these reviews back to English words:\n",
    "\n",
    "如果好奇的话，你可以用下列代码将索引解码为单词。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# word_index is a dictionary mapping words to an integer index（word_index 是一个将单词映射为整数索引的字典）\n",
    "word_index = imdb.get_word_index()\n",
    "# We reverse it, mapping integer indices to words（键值颠倒，将整数索引映射为单词）\n",
    "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n",
    "# We decode the review; note that our indices were offset by 3\n",
    "# because 0, 1 and 2 are reserved indices for \"padding\", \"start of sequence\", and \"unknown\".\n",
    "#将评论解码。注意，索引减去了 3，\n",
    "# 因为 0、1、2 是为“padding”（填充）、“start of sequence”（序列开始）、“unknown”（未知词）分别保留的索引\n",
    "decoded_review = ' '.join([reverse_word_index.get(i - 3, '?') for i in train_data[0]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"? this film was just brilliant casting location scenery story direction everyone's really suited the part they played and you could just imagine being there robert ? is an amazing actor and now the same being director ? father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it was released for ? and would recommend it to everyone to watch and the fly fishing was amazing really cried at the end it was so sad and you know what they say if you cry at a film it must have been good and this definitely was also ? to the two little boy's that played the ? of norman and paul they were just brilliant children are often left out of the ? list i think because the stars that play them all grown up are such a big profile for the whole film but these children are amazing and should be praised for what they have done don't you think the whole story was so lovely because it was true and was someone's life after all that was shared with us all\""
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "decoded_review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparing the data\n",
    "\n",
    "## 准备数据\n",
    "We cannot feed lists of integers into a neural network. We have to turn our lists into tensors. There are two ways we could do that:\n",
    "\n",
    "* We could pad our lists so that they all have the same length, and turn them into an integer tensor of shape `(samples, word_indices)`, \n",
    "then use as first layer in our network a layer capable of handling such integer tensors (the `Embedding` layer, which we will cover in \n",
    "detail later in the book).\n",
    "* We could one-hot-encode our lists to turn them into vectors of 0s and 1s. Concretely, this would mean for instance turning the sequence \n",
    "`[3, 5]` into a 10,000-dimensional vector that would be all-zeros except for indices 3 and 5, which would be ones. Then we could use as \n",
    "first layer in our network a `Dense` layer, capable of handling floating point vector data.\n",
    "\n",
    "We will go with the latter solution. Let's vectorize our data, which we will do manually for maximum clarity:\n",
    "\n",
    "你不能将整数序列直接输入神经网络。你需要将列表转换为张量。转换方法有以下两种。\n",
    "\n",
    "* 填充列表，使其具有相同的长度，再将列表转换成形状为 (samples, word_indices) 的整数张量，然后网络第一层使用能处理这种整数张量的层（即 Embedding 层，本书后面会详细介绍）。\n",
    "\n",
    "* 对列表进行 one-hot 编码，将其转换为 0 和 1 组成的向量。举个例子，序列 [3, 5] 将会 被转换为 10 000 维向量，只有索引为 3 和 5 的元素是 1，其余元素都是 0。然后网络第一层可以用 Dense 层，它能够处理浮点数向量数据。\n",
    "\n",
    "下面我们采用后一种方法将数据向量化。为了加深理解，你可以手动实现这一方法，如下所示。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "def vectorize_sequences(sequences, dimension=10000):\n",
    "    # Create an all-zero matrix of shape (len(sequences), dimension)（创建一个形状为 (len(sequences), dimension) 的零矩阵）\n",
    "    results = np.zeros((len(sequences), dimension))\n",
    "    for i, sequence in enumerate(sequences):\n",
    "        results[i, sequence] = 1.  # set specific indices of results[i] to 1s（将 results[i] 的指定索引设为 1）\n",
    "    return results\n",
    "\n",
    "# Our vectorized training data（将训练数据向量化）\n",
    "x_train = vectorize_sequences(train_data)\n",
    "# Our vectorized test data（将测试数据向量化）\n",
    "x_test = vectorize_sequences(test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's what our samples look like now:\n",
    "\n",
    "样本现在变成了这样："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.,  1.,  1., ...,  0.,  0.,  0.])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We should also vectorize our labels, which is straightforward:\n",
    "\n",
    "你还应该将标签向量化，这很简单。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Our vectorized labels（标签向量化）\n",
    "y_train = np.asarray(train_labels).astype('float32')\n",
    "y_test = np.asarray(test_labels).astype('float32')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now our data is ready to be fed into a neural network.\n",
    "\n",
    "现在可以将数据输入到神经网络中。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building our network\n",
    "\n",
    "\n",
    "Our input data is simply vectors, and our labels are scalars (1s and 0s): this is the easiest setup you will ever encounter. A type of \n",
    "network that performs well on such a problem would be a simple stack of fully-connected (`Dense`) layers with `relu` activations: `Dense(16, \n",
    "activation='relu')`\n",
    "\n",
    "The argument being passed to each `Dense` layer (16) is the number of \"hidden units\" of the layer. What's a hidden unit? It's a dimension \n",
    "in the representation space of the layer. You may remember from the previous chapter that each such `Dense` layer with a `relu` activation implements \n",
    "the following chain of tensor operations:\n",
    "\n",
    "`output = relu(dot(W, input) + b)`\n",
    "\n",
    "Having 16 hidden units means that the weight matrix `W` will have shape `(input_dimension, 16)`, i.e. the dot product with `W` will project the \n",
    "input data onto a 16-dimensional representation space (and then we would add the bias vector `b` and apply the `relu` operation). You can \n",
    "intuitively understand the dimensionality of your representation space as \"how much freedom you are allowing the network to have when \n",
    "learning internal representations\". Having more hidden units (a higher-dimensional representation space) allows your network to learn more \n",
    "complex representations, but it makes your network more computationally expensive and may lead to learning unwanted patterns (patterns that \n",
    "will improve performance on the training data but not on the test data).\n",
    "\n",
    "There are two key architecture decisions to be made about such stack of dense layers:\n",
    "\n",
    "* How many layers to use.\n",
    "* How many \"hidden units\" to chose for each layer.\n",
    "\n",
    "In the next chapter, you will learn formal principles to guide you in making these choices. \n",
    "For the time being, you will have to trust us with the following architecture choice: \n",
    "two intermediate layers with 16 hidden units each, \n",
    "and a third layer which will output the scalar prediction regarding the sentiment of the current review. \n",
    "The intermediate layers will use `relu` as their \"activation function\", \n",
    "and the final layer will use a sigmoid activation so as to output a probability \n",
    "(a score between 0 and 1, indicating how likely the sample is to have the target \"1\", i.e. how likely the review is to be positive). \n",
    "A `relu` (rectified linear unit) is a function meant to zero-out negative values, \n",
    "while a sigmoid \"squashes\" arbitrary values into the `[0, 1]` interval, thus outputting something that can be interpreted as a probability.\n",
    "\n",
    "## 构建网络\n",
    "输入数据是向量，而标签是标量（1 和 0），这是你会遇到的最简单的情况。有一类网络在这种问题上表现很好， 就是带有 relu 激 活的全连接层（Dense）的简单堆叠，比如Dense(16, activation='relu')。\n",
    "\n",
    "传入 Dense 层的参数（16）是该层隐藏单元的个数。一个隐藏单元（hidden unit）是该层 表示空间的一个维度。我们在第 2 章讲过，每个带有 relu 激活的 Dense 层都实现了下列张量运算：\n",
    "\n",
    "output = relu(dot(W, input) + b)\n",
    "\n",
    "16 个隐藏单元对应的权重矩阵 W 的形状为 (input_dimension, 16)，与 W 做点积相当于将输入数据投影到 16 维表示空间中（然后再加上偏置向量 b 并应用 relu 运算）。你可以将表示空间的维度直观地理解为“网络学习内部表示时所拥有的自由度”。隐藏单元越多（即更高维的表示空间），网络越能够学到更加复杂的表示，但网络的计算代价也变得更大，而且可能会导致学到不好的模式（这种模式会提高训练数据上的性能，但不会提高测试数据上的性能）。对于这种 Dense 层的堆叠，你需要确定以下两个关键架构：\n",
    "\n",
    "* 网络有多少层；\n",
    "\n",
    "* 每层有多少个隐藏单元。\n",
    "\n",
    "下一章中的原则将会指导你对上述问题做出选择。现在你只需要相信我选择的下列架构：\n",
    "\n",
    "* 两个中间层，每层都有 16 个隐藏单元；\n",
    "\n",
    "* 第三层输出一个标量，预测当前评论的情感。\n",
    "\n",
    "中间层使用 relu 作为激活函数，最后一层使用 sigmoid 激活以输出一个 0~1 范围内的概率值（表示样本的目标值等于 1 的可能性，即评论为正面的可能性）。relu（rectified linear unit，整流线性单元）函数将所有负值归零，而 sigmoid 函数则将任意值“压缩”到 [0,1] 区间内，其输出值可以看作概率值。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's what our network looks like:\n",
    "\n",
    "网络架构如下：\n",
    "\n",
    "![3-layer network](data/3_layer_network.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And here's the Keras implementation, very similar to the MNIST example you saw previously:\n",
    "\n",
    "代码清单是其 Keras 实现，与前面见过的 MNIST 例子类似。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import models\n",
    "from keras import layers\n",
    "\n",
    "model = models.Sequential()\n",
    "model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n",
    "model.add(layers.Dense(16, activation='relu'))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Lastly, we need to pick a loss function and an optimizer. Since we are facing a binary classification problem and the output of our network \n",
    "is a probability (we end our network with a single-unit layer with a sigmoid activation), is it best to use the `binary_crossentropy` loss. \n",
    "It isn't the only viable choice: you could use, for instance, `mean_squared_error`. But crossentropy is usually the best choice when you \n",
    "are dealing with models that output probabilities. Crossentropy is a quantity from the field of Information Theory, that measures the \"distance\" \n",
    "between probability distributions, or in our case, between the ground-truth distribution and our predictions.\n",
    "\n",
    "Here's the step where we configure our model with the `rmsprop` optimizer and the `binary_crossentropy` loss function. Note that we will \n",
    "also monitor accuracy during training.\n",
    "\n",
    "最后，你需要选择损失函数和优化器。由于你面对的是一个二分类问题，网络输出是一个概率值（网络最后一层使用 sigmoid 激活函数，仅包含一个单元），那么最好使用 `binary_crossentropy` （二元交叉熵）损失。这并不是唯一可行的选择，比如你还可以使用 `mean_squared_error`（均方误差）。但对于输出概率值的模型，交叉熵（crossentropy）往往是最好的选择。交叉熵是来自于信息论领域的概念，用于衡量概率分布之间的距离，在这个例子中就是真实分布与预测值之间的距离。\n",
    "\n",
    "下面的步骤是用 rmsprop 优化器和 `binary_crossentropy`  损失函数来配置模型。注意，我们还在训练过程中监控精度。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='rmsprop',\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are passing our optimizer, loss function and metrics as strings, which is possible because `rmsprop`, `binary_crossentropy` and \n",
    "`accuracy` are packaged as part of Keras. Sometimes you may want to configure the parameters of your optimizer, or pass a custom loss \n",
    "function or metric function. This former can be done by passing an optimizer class instance as the `optimizer` argument:\n",
    "\n",
    "上述代码将优化器、损失函数和指标作为字符串传入，这是因为 rmsprop、binary_ crossentropy 和 accuracy 都是 Keras 内置的一部分。有时你可能希望配置自定义优化器的 参数，或者传入自定义的损失函数或指标函数。前者可通过向 optimizer 参数传入一个优化器类实例来实现，如代码所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import optimizers\n",
    "\n",
    "model.compile(optimizer=optimizers.RMSprop(lr=0.001),\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The latter can be done by passing function objects as the `loss` or `metrics` arguments:\n",
    "\n",
    "后者可通过向 loss 和 metrics 参数传入函数对象来实现， 如代码所示。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import losses\n",
    "from keras import metrics\n",
    "\n",
    "model.compile(optimizer=optimizers.RMSprop(lr=0.001),\n",
    "              loss=losses.binary_crossentropy,\n",
    "              metrics=[metrics.binary_accuracy])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Validating our approach\n",
    "\n",
    "In order to monitor during training the accuracy of the model on data that it has never seen before, we will create a \"validation set\" by \n",
    "setting apart 10,000 samples from the original training data:\n",
    "\n",
    "## 验证你的方法\n",
    "为了在训练过程中监控模型在前所未见的数据上的精度，你需要将原始训练数据留出 10 000个样本作为验证集。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_val = x_train[:10000]\n",
    "partial_x_train = x_train[10000:]\n",
    "\n",
    "y_val = y_train[:10000]\n",
    "partial_y_train = y_train[10000:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "We will now train our model for 20 epochs (20 iterations over all samples in the `x_train` and `y_train` tensors), in mini-batches of 512 \n",
    "samples. At this same time we will monitor loss and accuracy on the 10,000 samples that we set apart. This is done by passing the \n",
    "validation data as the `validation_data` argument:\n",
    "\n",
    "现在使用 512 个样本组成的小批量，将模型训练 20 个轮次（即对 x_train 和 y_train 两 个张量中的所有样本进行 20 次迭代）。与此同时，你还要监控在留出的 10 000 个样本上的损失和精度。你可以通过将验证数据传入 validation_data 参数来完成。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 15000 samples, validate on 10000 samples\n",
      "Epoch 1/20\n",
      "15000/15000 [==============================] - 5s 343us/step - loss: 0.5084 - binary_accuracy: 0.7813 - val_loss: 0.3797 - val_binary_accuracy: 0.8684\n",
      "Epoch 2/20\n",
      "15000/15000 [==============================] - 3s 171us/step - loss: 0.3004 - binary_accuracy: 0.9047 - val_loss: 0.3004 - val_binary_accuracy: 0.8897\n",
      "Epoch 3/20\n",
      "15000/15000 [==============================] - 3s 179us/step - loss: 0.2179 - binary_accuracy: 0.9285 - val_loss: 0.3085 - val_binary_accuracy: 0.8711\n",
      "Epoch 4/20\n",
      "15000/15000 [==============================] - 3s 167us/step - loss: 0.1750 - binary_accuracy: 0.9437 - val_loss: 0.2840 - val_binary_accuracy: 0.8832\n",
      "Epoch 5/20\n",
      "15000/15000 [==============================] - 3s 176us/step - loss: 0.1427 - binary_accuracy: 0.9543 - val_loss: 0.2841 - val_binary_accuracy: 0.8872\n",
      "Epoch 6/20\n",
      "15000/15000 [==============================] - 2s 165us/step - loss: 0.1150 - binary_accuracy: 0.9650 - val_loss: 0.3166 - val_binary_accuracy: 0.8772\n",
      "Epoch 7/20\n",
      "15000/15000 [==============================] - 3s 171us/step - loss: 0.0980 - binary_accuracy: 0.9705 - val_loss: 0.3127 - val_binary_accuracy: 0.8846\n",
      "Epoch 8/20\n",
      "15000/15000 [==============================] - 3s 168us/step - loss: 0.0807 - binary_accuracy: 0.9763 - val_loss: 0.3859 - val_binary_accuracy: 0.8649\n",
      "Epoch 9/20\n",
      "15000/15000 [==============================] - 3s 171us/step - loss: 0.0661 - binary_accuracy: 0.9821 - val_loss: 0.3635 - val_binary_accuracy: 0.8782\n",
      "Epoch 10/20\n",
      "15000/15000 [==============================] - 2s 166us/step - loss: 0.0561 - binary_accuracy: 0.9853 - val_loss: 0.3843 - val_binary_accuracy: 0.8792\n",
      "Epoch 11/20\n",
      "15000/15000 [==============================] - 3s 176us/step - loss: 0.0439 - binary_accuracy: 0.9893 - val_loss: 0.4153 - val_binary_accuracy: 0.8779\n",
      "Epoch 12/20\n",
      "15000/15000 [==============================] - 3s 177us/step - loss: 0.0381 - binary_accuracy: 0.9921 - val_loss: 0.4525 - val_binary_accuracy: 0.8690\n",
      "Epoch 13/20\n",
      "15000/15000 [==============================] - 2s 163us/step - loss: 0.0300 - binary_accuracy: 0.9928 - val_loss: 0.4698 - val_binary_accuracy: 0.8729\n",
      "Epoch 14/20\n",
      "15000/15000 [==============================] - 3s 171us/step - loss: 0.0247 - binary_accuracy: 0.9945 - val_loss: 0.5023 - val_binary_accuracy: 0.8726\n",
      "Epoch 15/20\n",
      "15000/15000 [==============================] - 2s 165us/step - loss: 0.0175 - binary_accuracy: 0.9979 - val_loss: 0.5342 - val_binary_accuracy: 0.8693\n",
      "Epoch 16/20\n",
      "15000/15000 [==============================] - 3s 172us/step - loss: 0.0149 - binary_accuracy: 0.9983 - val_loss: 0.5710 - val_binary_accuracy: 0.8697\n",
      "Epoch 17/20\n",
      "15000/15000 [==============================] - 3s 175us/step - loss: 0.0151 - binary_accuracy: 0.9971 - val_loss: 0.6024 - val_binary_accuracy: 0.8697\n",
      "Epoch 18/20\n",
      "15000/15000 [==============================] - 3s 169us/step - loss: 0.0075 - binary_accuracy: 0.9996 - val_loss: 0.6789 - val_binary_accuracy: 0.8632\n",
      "Epoch 19/20\n",
      "15000/15000 [==============================] - 3s 175us/step - loss: 0.0118 - binary_accuracy: 0.9975 - val_loss: 0.6698 - val_binary_accuracy: 0.8680\n",
      "Epoch 20/20\n",
      "15000/15000 [==============================] - 3s 172us/step - loss: 0.0041 - binary_accuracy: 0.9999 - val_loss: 0.6931 - val_binary_accuracy: 0.8654\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(partial_x_train,\n",
    "                    partial_y_train,\n",
    "                    epochs=20,\n",
    "                    batch_size=512,\n",
    "                    validation_data=(x_val, y_val))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On CPU, this will take less than two seconds per epoch -- training is over in 20 seconds. At the end of every epoch, there is a slight pause \n",
    "as the model computes its loss and accuracy on the 10,000 samples of the validation data.\n",
    "\n",
    "Note that the call to `model.fit()` returns a `History` object. This object has a member `history`, which is a dictionary containing data \n",
    "about everything that happened during training. Let's take a look at it:\n",
    "\n",
    "在 CPU 上运行，每轮的时间不到 2 秒，训练过程将在 20 秒内结束。每轮结束时会有短暂的停顿，因为模型要计算在验证集的 10 000 个样本上的损失和精度。\n",
    "注意，调用 model.fit() 返回了一个 History 对象。这个对象有一个成员 history，它是一个字典，包含训练过程中的所有数据。我们来看一下。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['val_loss', 'val_binary_accuracy', 'loss', 'binary_accuracy'])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "history_dict = history.history\n",
    "history_dict.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It contains 4 entries: one per metric that was being monitored, during training and during validation. Let's use Matplotlib to plot the \n",
    "training and validation loss side by side, as well as the training and validation accuracy:\n",
    "\n",
    "字典中包含 4 个条目，对应训练过程和验证过程中监控的指标。在下面两个代码清单中， 我们将使用 Matplotlib 在同一张图上绘制训练损失和验证损失，以及训练精度和验证精度）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3X2c1XP+//HHSyJdKKrdKDVFLrouI/mKCmuLlUVLKWTRhly1uz99sbGtdrG+arMtchGrKMtSiHaXXK80uYhKCsUoSrpUaOr1++N9GqdxZubMnPM558zM8367nduc8znv8/m85syZz+u83+/P+/02d0dERARgt2wHICIiuUNJQUREiikpiIhIMSUFEREppqQgIiLFlBRERKSYkoKklZnVMrPNZtYynWWzycwOMrO0X7ttZieY2fK4x0vM7JhkylbiWPeY2TWVfX0Z+73RzO5P934le3bPdgCSXWa2Oe5hXeBbYHvs8a/cfWpF9ufu24H66S5bE7j7IenYj5ldCAxx995x+74wHfuW6k9JoYZz9+KTcuyb6IXu/p/SypvZ7u5elInYRCTz1HwkZYo1D0w3s4fNbBMwxMyOMrPXzWy9ma0yswlmVjtWfnczczPLiz2eEnv+GTPbZGb/NbPWFS0be76fmX1gZhvM7HYze9XMhpYSdzIx/srMlpnZOjObEPfaWmY2zszWmtmHQN8y3p/rzGxaiW0Tzey22P0LzWxx7Pf5MPYtvrR9FZpZ79j9umb2YCy2hcDhCY77UWy/C82sf2x7R+CvwDGxprkv497bG+JePzz2u681syfMbL9k3pvymNnPY/GsN7PnzeyQuOeuMbOVZrbRzN6P+117mNmbse1fmNmfkz2eRMDdddMNdwdYDpxQYtuNwHfAKYQvEXsBRwBHEmqabYAPgBGx8rsDDuTFHk8BvgTygdrAdGBKJcr+CNgEnBp7biSwDRhayu+STIwzgIZAHvDVzt8dGAEsBFoAjYGXwr9KwuO0ATYD9eL2vRrIjz0+JVbGgOOArUCn2HMnAMvj9lUI9I7dvxV4AdgHaAUsKlH2TGC/2N/k7FgMP449dyHwQok4pwA3xO6fGIuxC1AH+BvwfDLvTYLf/0bg/tj9w2JxHBf7G10Te99rA+2BFUCzWNnWQJvY/XnAoNj9BsCR2f5fqMk31RQkGa+4+5PuvsPdt7r7PHef6+5F7v4RMAnoVcbrH3X3AnffBkwlnIwqWvZnwNvuPiP23DhCAkkoyRj/5O4b3H054QS881hnAuPcvdDd1wI3lXGcj4D3CMkK4CfAencviD3/pLt/5MHzwHNAws7kEs4EbnT3de6+gvDtP/64j7j7qtjf5CFCQs9PYr8Ag4F73P1td/8GGAX0MrMWcWVKe2/KMhCY6e7Px/5GNwF7E5JzESEBtY81QX4ce+8gJPe2ZtbY3Te5+9wkfw+JgJKCJOPT+AdmdqiZPW1mn5vZRmAM0KSM138ed38LZXcul1Z2//g43N0J36wTSjLGpI5F+IZbloeAQbH7ZxOS2c44fmZmc83sKzNbT/iWXtZ7tdN+ZcVgZkPN7J1YM8164NAk9wvh9yven7tvBNYBzePKVORvVtp+dxD+Rs3dfQnwa8LfYXWsObJZrOj5QDtgiZm9YWYnJfl7SASUFCQZJS/HvIvw7fggd98bGE1oHonSKkJzDgBmZux6EisplRhXAQfEPS7vktnpwAmxb9qnEpIEZrYX8CjwJ0LTTiPgX0nG8XlpMZhZG+AO4GKgcWy/78ftt7zLZ1cSmqR27q8BoZnqsyTiqsh+dyP8zT4DcPcp7n40oemoFuF9wd2XuPtAQhPh/wGPmVmdFGORSlJSkMpoAGwAvjazw4BfZeCYTwHdzOwUM9sduAJoGlGMjwBXmllzM2sMXF1WYXf/AngFmAwscfelsaf2BPYA1gDbzexnwPEViOEaM2tkYRzHiLjn6hNO/GsI+fFCQk1hpy+AFjs71hN4GLjAzDqZ2Z6Ek/PL7l5qzasCMfc3s96xY/+W0A8018wOM7M+seNtjd22E36Bc8ysSaxmsSH2u+1IMRapJCUFqYxfA+cR/uHvInxTjlTsxHsWcBuwFjgQeIswriLdMd5BaPt/l9AJ+mgSr3mI0HH8UFzM64GrgMcJnbUDCMktGdcTaizLgWeAv8ftdwEwAXgjVuZQIL4d/t/AUuALM4tvBtr5+mcJzTiPx17fktDPkBJ3X0h4z+8gJKy+QP9Y/8KewC2EfqDPCTWT62IvPQlYbOHqtluBs9z9u1Tjkcqx0DQrUrWYWS1Cc8UAd3852/GIVBeqKUiVYWZ9zaxhrAnid4QrWt7Iclgi1YqSglQlPYGPCE0QfYGfu3tpzUciUglqPhIRkWKqKYiISLEqNyFekyZNPC8vL9thiIhUKfPnz//S3cu6jBuogkkhLy+PgoKCbIchIlKlmFl5I/MBNR+JiEgcJQURESmmpCAiIsUi7VMws77AXwiTX93j7jeVeH4c0Cf2sC7wo9jkXhWybds2CgsL+eabb1INWTKgTp06tGjRgtq1S5uaR0SyJbKkEJuGYCJhfvlCYJ6ZzXT3RTvLuPtVceUvA7pW5liFhYU0aNCAvLw8wuSZkqvcnbVr11JYWEjr1q3Lf4GIZFSUzUfdgWWxBUa+A6bx/UIkiQwizN5YYd988w2NGzdWQqgCzIzGjRurVieSo6JMCs3ZdZGQQkqZ/97MWhHmWH++lOeHmVmBmRWsWbMm4cGUEKoO/a1EcleUSSHRf35pc2oMJCzDuD3Rk+4+yd3z3T2/adNyx16IiFQrhYUwejS8/370x4oyKRSy68pRLQhTHScykEo2HeWCtWvX0qVLF7p06UKzZs1o3rx58ePvvktuWvjzzz+fJUuWlFlm4sSJTJ06tcwyyerZsydvv/12WvYlIunnDs8/D2ecAXl5cOONMGdO9MeN8uqjeYTFuFsTluMbSFi/dhdmdghhwY3/RhjLLqZOhWuvhU8+gZYtYexYGJzCEiONGzcuPsHecMMN1K9fn9/85je7lHF33J3ddkuchydPnlzucS699NLKBykiVcLGjfD3v8Pf/gaLF8O++8LIkTB8OLRpE/3xI6spuHsRYQnB2cBi4BF3X2hmY8ysf1zRQcA0z9B0rVOnwrBhsGJFyMQrVoTHafoCvotly5bRoUMHhg8fTrdu3Vi1ahXDhg0jPz+f9u3bM2bMmOKyO7+5FxUV0ahRI0aNGkXnzp056qijWL16NQDXXXcd48ePLy4/atQounfvziGHHMJrr70GwNdff80ZZ5xB586dGTRoEPn5+eXWCKZMmULHjh3p0KED11xzDQBFRUWcc845xdsnTJgAwLhx42jXrh2dO3dmyJAhaX/PRGqqhQvhkkugeXO47DKoVw8mTw5NR7fckpmEABGPU3D3WcCsEttGl3h8Q5QxlHTttbBly67btmwJ21OpLZRm0aJFTJ48mTvvvBOAm266iX333ZeioiL69OnDgAEDaNeu3S6v2bBhA7169eKmm25i5MiR3HfffYwaNeoH+3Z33njjDWbOnMmYMWN49tlnuf3222nWrBmPPfYY77zzDt26dSszvsLCQq677joKCgpo2LAhJ5xwAk899RRNmzblyy+/5N133wVg/fr1ANxyyy2sWLGCPfbYo3ibSHW3YwdMnw4NGkC7dqE5p5RKf4Vs2wZPPAETJ8KLL8Kee8LAgSE5dO+e+v4ro8aNaP7kk4ptT9WBBx7IEUccUfz44Ycfplu3bnTr1o3FixezaNGiH7xmr732ol+/fgAcfvjhLF++POG+Tz/99B+UeeWVVxg4cCAAnTt3pn379mXGN3fuXI477jiaNGlC7dq1Ofvss3nppZc46KCDWLJkCVdccQWzZ8+mYcOGALRv354hQ4YwdepUDT6TGmP8eDj7bDjlFDjwQKhfH7p2DV8kx46Fxx+HJUugqCi5/a1aBb//fUguZ54ZWixuvjnUCu6/P3sJAargLKmpatky/AESbY9CvXr1iu8vXbqUv/zlL7zxxhs0atSIIUOGJLxef4899ii+X6tWLYpK+aTtueeePyhT0Va40so3btyYBQsW8MwzzzBhwgQee+wxJk2axOzZs3nxxReZMWMGN954I++99x61atWq0DFFqpL582HUKDj1VLj6ali06PvbK6/AQw99X7Z2bTj44FCbaNcODjss/Dz4YNhjD3j55VAr+Oc/QwLp2xfuugv69YNc+TeqcUlh7NjQhxDfhFS3btgetY0bN9KgQQP23ntvVq1axezZs+nbt29aj9GzZ08eeeQRjjnmGN59992ENZF4PXr04Le//S1r166lYcOGTJs2jd/85jesWbOGOnXq8Itf/ILWrVszfPhwtm/fTmFhIccddxw9e/Zk6tSpbNmyhQYNGqT1dxDJFZs2heacH/8Y7rsvdPoeddSuZTZvDpeK7kwUixfDW2/Bo4+GfksITU1NmsDq1dCoEVx+OVx8MRx0UOZ/p/LUuKSws98gnVcfJatbt260a9eODh060KZNG44++ui0H+Oyyy7j3HPPpVOnTnTr1o0OHToUN/0k0qJFC8aMGUPv3r1xd0455RROPvlk3nzzTS644ALcHTPj5ptvpqioiLPPPptNmzaxY8cOrr76aiUEqdYuuQQ++gheeCEkhETq14f8/HCLt3UrfPBBSBKLFoX99OkDgwaFL6K5qsqt0Zyfn+8lF9lZvHgxhx12WJYiyi1FRUUUFRVRp04dli5dyoknnsjSpUvZfffcyv/6m0mu+/vf4bzz4IYb4Prrsx1N6sxsvrvnl1cut84UkrLNmzdz/PHHU1RUhLtz11135VxCEMl1H3wQagnHHgvXXZftaDJLZ4tqplGjRsyfPz/bYYhUWd9+G/oR9twzjF/KlQ7gTFFSEBGJM2pU6Ch+4glo0SLb0WRejRunICJSmqefDmMSRowIl6DWREoKIiLAypUwdCh07gx//nO2o8keJQURqfG2b4dzzgnjl6ZNgzp1sh1R9igppEHv3r2ZPXv2LtvGjx/PJZdcUubr6tevD8DKlSsZMGBAqfsueQluSePHj2dL3Gi8k046KS3zEt1www3ceuutKe9HJNfdfHOYpvr22+HQQ7MdTXYpKaTBoEGDmDZt2i7bpk2bxqBBg5J6/f7778+jjz5a6eOXTAqzZs2iUaNGld6fSE3y2mthAZuBA+H887MdTfYpKaTBgAEDeOqpp/j2228BWL58OStXrqRnz57F4wa6detGx44dmTFjxg9ev3z5cjp06ADA1q1bGThwIJ06deKss85i69atxeUuvvji4mm3r4+NppkwYQIrV66kT58+9OnTB4C8vDy+/PJLAG677TY6dOhAhw4diqfdXr58OYcddhgXXXQR7du358QTT9zlOIm8/fbb9OjRg06dOnHaaaexbt264uO3a9eOTp06FU/E9+KLLxYvMtS1a1c2bdpU6fdWJErr14eJ7lq2hDvvBK0UWw0vSb3ySkj3gmJduoQrEkrTuHFjunfvzrPPPsupp57KtGnTOOusszAz6tSpw+OPP87ee+/Nl19+SY8ePejfv3+p6xTfcccd1K1blwULFrBgwYJdpr4eO3Ys++67L9u3b+f4449nwYIFXH755dx2223MmTOHJk2a7LKv+fPnM3nyZObOnYu7c+SRR9KrVy/22Wcfli5dysMPP8zdd9/NmWeeyWOPPVbm+gjnnnsut99+O7169WL06NH8/ve/Z/z48dx00018/PHH7LnnnsVNVrfeeisTJ07k6KOPZvPmzdSpyQ20krPc4aKL4LPP4NVXoYzZYGoU1RTSJL4JKb7pyN255ppr6NSpEyeccAKfffYZX3zxRan7eemll4pPzp06daJTp07Fzz3yyCN069aNrl27snDhwnInu3vllVc47bTTqFevHvXr1+f000/n5ZdfBqB169Z06dIFKHt6bgjrO6xfv55evXoBcN555/HSSy8Vxzh48GCmTJlSPHL66KOPZuTIkUyYMIH169drRLXkpLvvDpPWjR2b3amqc021+28t6xt9lH7+858zcuRI3nzzTbZu3Vr8DX/q1KmsWbOG+fPnU7t2bfLy8hJOlx0vUS3i448/5tZbb2XevHnss88+DB06tNz9lDWv1c5ptyFMvV1e81Fpnn76aV566SVmzpzJH/7wBxYuXMioUaM4+eSTmTVrFj169OA///kPh9b03jvJKQsXwhVXwIknQomVc2s81RTSpH79+vTu3Ztf/vKXu3Qwb9iwgR/96EfUrl2bOXPmsCLRYg5xjj32WKbG1gZ97733WLBgARCm3a5Xrx4NGzbkiy++4Jlnnil+TYMGDRK22x977LE88cQTbNmyha+//prHH3+cY445psK/W8OGDdlnn32KaxkPPvggvXr1YseOHXz66af06dOHW265hfXr17N582Y+/PBDOnbsyNVXX01+fj7vv/9+hY8pEpWtW+Gss2DvveGBB9Kzglp1Uu1qCtk0aNAgTj/99F2uRBo8eDCnnHIK+fn5dOnSpdxvzBdffDHnn38+nTp1okuXLnSP1Ws7d+5M165dad++/Q+m3R42bBj9+vVjv/32Y86cOcXbu3XrxtChQ4v3ceGFF9K1a9cym4pK88ADDzB8+HC2bNlCmzZtmDx5Mtu3b2fIkCFs2LABd+eqq66iUaNG/O53v2POnDnUqlWLdu3aFa8iJ5ILRo4MNYXZs6FZs2xHk3s0dbZkhf5mkg2PPQYDBsBvfwu33JLtaDIr2amzI604mVlfM1tiZsvM7Icrz4cyZ5rZIjNbaGYPJSojIpKqFSvgwgvhiCPgxhuzHU3uiqz5yMxqAROBnwCFwDwzm+nui+LKtAX+Fzja3deZ2Y+iikdEaq5168J4hO3bwzQWccugSwlR1hS6A8vc/SN3/w6YBpScd/AiYKK7rwNw99WVPVhVawaryfS3kkzZti1MXXHQQfDf/4bLUNu0yXZUuS3KpNAc+DTucWFsW7yDgYPN7FUze93MEq5ib2bDzKzAzArWrFnzg+fr1KnD2rVrdbKpAtydtWvXakCbRModnnoKOnaEyy8PA1DfeitcdSRli/Lqo0RDdkuetXcH2gK9gRbAy2bWwd13mc3N3ScBkyB0NJfcaYsWLSgsLCRRwpDcU6dOHVrUxNVLJCPeeQd+/Wt47jk45BB48kk4+WRNYZGsKJNCIXBA3OMWwMoEZV53923Ax2a2hJAk5lXkQLVr16Z169apxCoiVdznn8Pvfgf33gv77AMTJsDw4VC7drYjq1qibD6aB7Q1s9ZmtgcwEJhZoswTQB8AM2tCaE76KMKYRKSa2bo1TFXRtm0YjHbllbBsGVx2mRJCZURWU3D3IjMbAcwGagH3uftCMxsDFLj7zNhzJ5rZImA78Ft3XxtVTCJSfezYEa4kGjUKPv0UTjstrIvQtm22I6vaqsXgNRGpWV59NYxMfuMN6NoVbrsNevfOdlS5LScGr4mIpNPHH4criHr2hMJCuP9+KChQQkgnzX0kIjnv669hzJgwC3KtWnD99WGqinr1sh1Z9aOkICI5bc4cuOCCUEs491z44x+heckRT5I2aj4SkZy0aRNcfDEcd1yY3vrFF8PVRUoI0VJSEJGc869/QYcOcNddcNVVsGABHHtstqOqGZQURCRnrF8fmop++lOoWzdcZXTbbeG+ZIaSgojkhKefDrWD++8PYw/eeguOOirbUdU8SgoiklVffQXnnAM/+1mYnmLuXPjTn0BzJmaHkoKIZM3jj0O7dmFk8ujRYcxBfrnDqyRKuiRVRDJuzZowN9H06WFa62efDT8l+1RTEJGMcQ+JoF07+Oc/w7KYb7yhhJBLVFMQkYz4/HO45JLQZHTEETB5MrRvn+2opCTVFEQkUkVFMHFiqB3MmhVmMn3tNSWEXKWagohEZvbsMJvpokXQpw/87W9w6KHZjkrKopqCiKTd+++HJTD79oVvv4UnngjLYyoh5D4lBRFJm6++giuugI4d4ZVX4NZbYeFCOPVUrZFcVaj5SERStm1bmKfo+uvDVBUXXRSmuv7Rj7IdmVRUjagpTJ0KeXlhpsW8vPBYRNLj2Wehc+cw7qBr1zA9xZ13KiFUVdU+KUydCsOGwYoV4RrpFSvCYyUGkdQsXgwnnQT9+oWawowZ8O9/Q6dO2Y5MUhFpUjCzvma2xMyWmdmoBM8PNbM1ZvZ27HZhumO49lrYsmXXbVu2hO0iUnFffQWXXx76DV599ft+g/791W9QHUTWp2BmtYCJwE+AQmCemc1090Ulik539xFRxfHJJxXbLiKJbdsWmoWuvx42bAg17jFjoGnTbEcm6RRlTaE7sMzdP3L374BpwKkRHi+hli0rtl1EfujZZ0Oz0OWXQ7du8PbbcMcdSgjVUZRJoTnwadzjwti2ks4wswVm9qiZHZDuIMaO/eECHXXrhu0iUralS+GUU0K/QVERzJwZ+g06dsx2ZBKVKJNCotZFL/H4SSDP3TsB/wEeSLgjs2FmVmBmBWvWrKlQEIMHw6RJ0KpVaO9s1So8Hjy4QrsRqVE2bYKrrw5TUbzwQpia4r33QoJQv0H1Zu4lz9Np2rHZUcAN7v7T2OP/BXD3P5VSvhbwlbs3LGu/+fn5XlBQkO5wRQTYsQMefDCsfPb55zB0aFjwplmzbEcmqTKz+e5e7moVUdYU5gFtzay1me0BDARmxhcws/3iHvYHFkcYj4iUYe7csPzl0KGhRj13bpjJVAmhZoksKbh7ETACmE042T/i7gvNbIyZ9Y8Vu9zMFprZO8DlwNCo4hGRxFatComgR49wVd4DD4RZTLt3z3Zkkg2RNR9FRc1HIunx7bfwl7/AH/4A330HV10Vxu80aJDtyCQKyTYfae4jkRrGHZ56KkxpvWxZGHT2f/8HBx2U7cgkF1T7aS5E5Hvvvx8uL+3fH3bfPYw/mDFDCUG+p5qCSDXnHha5uece+OtfoV49GDcOLr0UatfOdnSSa5QURKqh7dvDvEQzZoTbhx+G8QUXXgg33qgZTKV0Sgoi1cSWLfCvf4Uk8NRT8OWXoSZw3HHw61+HJqPmieYUEImjpCBSha1eHRLAE0+E6Se++QYaNgxLYZ56algOc++9sx2lVCVKCiJVzAcffN8s9Nproc/ggAPCamenngrHHqu+Aqk8JQWRKmDtWrjtNnj88bC4DUCXLjB6dEgEXbpoTiJJDyUFkRw3Ywb86lehj6BXL7j44tA/0KpVtiOT6khJQSRH7VzhbOrUsAbys8+GGoFIlDR4TSQHzZgB7drB9Olwww3wxhtKCJIZqimI5BDVDiTbVFMQyREzZ4ZFbaZPD+sgq3Yg2aCagkiWffUVXHEFTJkSagfPPKNkINmjmoJIFu2sHUybptqB5AbVFESyIL520KmTageSO1RTEMmwJ5/8vnYwejTMm6eEILlDNQWRDFm3LtQOHnww1A5mzYKuXbMdlciuVFMQidiOHSERtG8PDz/8fe1ACUFykWoKIhGaMydMW/3WW5CfD08/rWQguS3SmoKZ9TWzJWa2zMxGlVFugJm5mZW7qLRIVbB4MZxySljLYO3aMBht7lwlBMl9kSUFM6sFTAT6Ae2AQWbWLkG5BsDlwNyoYhHJlNWr4ZJLoGNHeOkluPlmWLIEzj4bdlNjrVQBUX5MuwPL3P0jd/8OmAacmqDcH4BbgG8ijEUkUlu3wh//CAcdBJMmhZlMly2D//f/oE6dbEcnkrwok0Jz4NO4x4WxbcXMrCtwgLs/VdaOzGyYmRWYWcGaNWvSH6lIJe3sRD74YLj22tBctHAh3H47NG2a7ehEKi7KpJBoyQ8vftJsN2Ac8OvyduTuk9w9393zm+o/TXLEnDlwxBFw7rnQrBm88EJYFvOQQ7IdmUjlRZkUCoED4h63AFbGPW4AdABeMLPlQA9gpjqbJde9/35Y5Oa448LCNzs7kXv1ynZkIqmLMinMA9qaWWsz2wMYCMzc+aS7b3D3Ju6e5+55wOtAf3cviDAmqaK+/TaMBN64MXsxrF4Nl14KHTrAiy/CTTeFBKFOZKlOIvsou3sRMAKYDSwGHnH3hWY2xsz6R3VcqZ4uuSR8O99/fxg2DAoy9NXBHf77Xxg+PHQi33XX953IV18Ne+2VmThEMsXcvfxSOSQ/P98LKnFGmDs3XBVy9936VlfV3H8/nH9+WKf4u+/CnEFbt0K3bmHboEHQoEF6j/npp6ED+YEH4IMPwsn/jDPguuvUZyBVk5nNd/dym+drzOnxvffgvvtg3LhsRyIV8e67oZZw3HEwcWL4G65cCX/9K2zbFpLC/vuHb/Jvvpnasb7+OiSCE06AVq3C1UTNmsG998Lnn4fnlBCkukuqpmBmBwKF7v6tmfUGOgF/d/f1Ecf3A5WtKbjD6aeHScjeeCMsZiK5bePGMDXE5s1hmogf/3jX593h9ddDk8706fDNN6H8r34FAwdC/frlH2PHDnj55VAj+Mc/wrHatAlXFJ1zTrgvUh0kW1NINim8DeQDeYQ+gpnAIe5+UopxVlhlkwKEK0U6doTGjUObtAYV5S53OOss+Oc/4fnn4dhjyy6/bl1Ym+Cuu8I4gQYNYMiQ0P+QaFrqDz+Ev/893JYvD+XPPBPOOw969gRLdEG1SBWWbFLA3cu9AW/Gfv4WuCx2/61kXpvu2+GHH+6pePZZd3C/8sqUdiMRmzAh/J1uvrlir9uxw/2VV9zPPde9Tp2wj+7d3e+9133VKvd77nHv2TNsN3P/yU/cp0xx//rraH4PkVwBFHgS59hkawpzgfHAtcAp7v6xmb3n7h0qnbYqKZWawk6XXx5GnM6eDSeemKbAJG3mzoVjjoG+fcNgsMpeGPDVV6Ef4K67wgR1Ox16aKgRDBkCLVqkJ2aRXJfu5qN2wHDgv+7+sJm1Bs5y95tSD7Vi0pEUtm6Fww+H9etDR2bjxmkKTlK2dm24qmi33ULH8T77pL5Pd3j11TDi+MQTwyhkNQ9JTZPWpFBix/sQ5itaUNngUpGOpACh4/LII8P0xo8+qpNELtixI/w9/vOfcBLP19h2kbRJ6yWpZvaCme1tZvsC7wCTzey2VIPMpq5d4cYbQ0fmAw9kOxqBMM30rFnJausmAAATTklEQVQwfrwSgki2JNta29DdNwKnA5Pd/XDghOjCyoxf/xp694bLLgtXo0j2zJkTBoYNGhTGHIhIdiSbFHY3s/2AM4Eyp7muSmrVCrWEWrXCNelFRdmOqGZatSokg4MPDqPO1ZQnkj3JJoUxhPEJH7r7PDNrAyyNLqzMadkS7rgjzG/zpz9lO5qap6goJIRNm0LfTjIDzkQkOkklBXf/h7t3cveLY48/cvczog0tcwYNCjNd/v73YbSzZM7o0WHG0TvvhPbtsx2NiCTb0dzCzB43s9Vm9oWZPWZm1eoK74kToXlzGDw4THUg0Xv66VA7u+ii0HwnItmXbPPRZMLUFvsTltR8Mrat2mjUKEx58OGHMHJktqOp/pYvD4mga1eYMCHb0YjITskmhabuPtndi2K3+4Fqty5mr15hofW774YZM7IdTfX17bdhnqEdO8IkdJqDSiR3JJsUvjSzIWZWK3YbAqyNMrBsGTMmTKB24YVhumRJv9/8BubNg8mT4cADsx2NiMRLNin8knA56ufAKmAAcH5UQWXTHnvAQw+FfoVf/jJMkSDpM316WAth5Eg47bRsRyMiJVV65TUzu9Ldx6c5nnKla5qL8kycCCNGhBPYpZdGfric9/HH8NhjULcu7LvvD297713+xHVLloSRyp06hXmIatfOSOgiQoRzH8Ud4BN3b1mpF6cgU0nBHU46KZy83nwTDjss8kPmpK+/DgvU//nPoS+gNLvtFiavS5Qw9t03PHfPPaFJ7q23NDupSKYlmxR2T+UYKbw255mFNu+OHcNlqq+/HpqWKmvdunCCrSonQ3d4+OHQ8f7ZZ2Ecx403hprCV1+Vf1uzJqxt/NVXYTZa9/D+Pflk1XkPRGqiVJJCuVUMM+sL/AWoBdxTcqptMxsOXApsBzYDw9x9UQoxpVWzZuFKpNNOg6ZNw6jbli1h7NiQKEravBmWLQsnw6VLv/+5dGlY9Q3g+OPhqqugX7/KrxMQtTffDGtOvPpqmMZ6+nQ4+ujvny+5LGZ5tm+HDRvC79uoUXpjFZH0KjMpmNkmEp/8DdirnNfWAiYCPwEKgXlmNrPESf8hd78zVr4/cBvQN/nwo/f112FupI0bw+MVK8KVSQUFYcH4+ASwatWur23eHNq2DWtDt20b1hC+80742c/CPD9XXBEWe6lXL/O/VyKrV4fF6u+9F5o0Cc09Q4eG3z8VtWqFJiQRyX2V7lMod8dmRwE3uPtPY4//F8DdE84wZGaDgHPdvV9Z+81Un8JOeXkhEZSmadNwgm/bNtx23j/ooMQn+23bwhw/48aFyzIbNQrrCI8YAQccENmvUabvvgsd6r//PWzZEmoJo0dDw4bZiUdE0i/yjuYkAhgA9HX3C2OPzwGOdPcRJcpdCowE9gCOc/cfTLRnZsOAYQAtW7Y8fEVZZ+k022230i9LXbeu8s0h7mESvnHjwpoOZjBgQGhaOvLIysdbUc88E465ZElo0ho3Dg45JHPHF5HMSOsiO5WNIcG2H5xe3X2iux8IXA1cl2hH7j7J3fPdPb9p08wOpG5ZyvVVrVql1j5uBv/zP2FE70cfwZVXhhN0jx5w1FHwyCPRTuW9dGloxjrppDCy+KmnwgI3SggiNVuUSaEQiG8QaQGsLKP8NODnEcZTKWPHhitu4tWtG7anS6tWcOutUFgY5gFaswbOOgvatAmXgq5bl75jbdwYrihq3x5eeins/7334OST03cMEam6omw+2h34ADge+AyYB5zt7gvjyrTd2VxkZqcA15dXvcl0nwLA1KmhA/aTT8q++ihdtm8PM4iOGxfGSdSrFzp8L7gAGjQIZdy/v5V8nGgbhM7xa66BL76A88+HP/4xXGElItVf1vsUYkGcBIwnXJJ6n7uPNbMxQIG7zzSzvxCW9dwGrANGxCeNRLKRFLLp7bfDmsUPPxw6hFPVo0eojRxxROr7EpGqIyeSQhRqWlLY6fPP4bnnQvu/2fdLVu68H39LtB1CH0jv3rk7PkJEopOJEc2SQc2aRdtkJSIC0XY0i4hIFaOkICIixZQURESkmJKCiIgUU1IQEZFiSgoiIlJMSUFERIopKYiISDElBRERKaakICIixZQURESkmJKCiIgUU1IQEZFiSgoiIlJMSSEDpk6FvLywjkFeXngsIpKLtJ5CxKZOhWHDYMuW8HjFivAYtD6CiOQe1RQidu213yeEnbZsCdtFRHKNkkLEPvmkYttFRLJJSSFiLVtWbLuISDZFmhTMrK+ZLTGzZWY2KsHzI81skZktMLPnzKxVlPFkw9ixULfurtvq1g3bRURyTWRJwcxqAROBfkA7YJCZtStR7C0g3907AY8Ct0QVT7YMHgyTJkGrVmAWfk6apE5mEclNUV591B1Y5u4fAZjZNOBUYNHOAu4+J67868CQCOPJmsGDlQREpGqIsvmoOfBp3OPC2LbSXAA8k+gJMxtmZgVmVrBmzZo0higiIvGiTAqWYJsnLGg2BMgH/pzoeXef5O757p7ftGnTNIYoIiLxomw+KgQOiHvcAlhZspCZnQBcC/Ry928jjEdERMoRZU1hHtDWzFqb2R7AQGBmfAEz6wrcBfR399URxiIiIkmILCm4exEwApgNLAYecfeFZjbGzPrHiv0ZqA/8w8zeNrOZpeyuRtPcSSKSKZHOfeTus4BZJbaNjrt/QpTHrw40d5KIZJJGNOc4zZ0kIpmkpJDjNHeSiGSSkkKO09xJIpJJSgo5TnMniUgmKSnkOM2dJCKZpJXXqgDNnSQimaKagoiIFFNSqAE0+E1EkqXmo2pOg99EpCJUU6jmNPhNRCpCSaGa0+A3EakIJYVqToPfRKQilBSqOQ1+E5GKUFKo5jT4TUQqQlcf1QAa/CYiyVJNQcqlcQ4iNYdqClImjXMQqVlUU5AyaZyDSM2ipCBl0jgHkZol0qRgZn3NbImZLTOzUQmeP9bM3jSzIjMbEGUsUjka5yBSs0SWFMysFjAR6Ae0AwaZWbsSxT4BhgIPRRWHpCYd4xzUUS1SdURZU+gOLHP3j9z9O2AacGp8AXdf7u4LgB0RxiEpSHWcw86O6hUrwP37jmolBpHcFGVSaA58Gve4MLatwsxsmJkVmFnBmjVr0hKcJG/wYFi+HHbsCD8rctWROqpFqpYok4Il2OaV2ZG7T3L3fHfPb9q0aYphSSapo1qkaokyKRQCB8Q9bgGsjPB4koPUUS1StUSZFOYBbc2stZntAQwEZkZ4PMlB6qgWqVoiSwruXgSMAGYDi4FH3H2hmY0xs/4AZnaEmRUCvwDuMrOFUcUj2aGOapGqxdwr1cyfNfn5+V5QUJDtMCRD8vJCIiipVavQ6S0iyTGz+e6eX145jWiWnKaOapHMUlKQnJaOjmr1SYgkT0lBclqqHdXqkxCpGCUFyWmpdlRr8JxIxSgpSM5LZUR1Ovok1PwkNYmSglRrqfZJqPlJaholBanWUu2TUPOT1DRKClKtpdonoeYnqWm0RrNUe4MHV3496ZYtEw+eq2jzk9a4lqpCNQWRMuRK85NqG5IpSgoiZciV5id1dkumKCmIlCOVS2LTMSI7HbUN1TQkWUoKIhFKx9ThqdY2VNOQilBSEIlQqs1PkHptQzUNqQglBZGIpdL8BKnXNlTTkIpQUhDJcanWNlTTkIpQUhCpAlKpbVSHmkaqSUVJqQLcvUrdDj/8cBeRipkyxb1VK3ez8HPKlORf26qVezid73pr1Sozr58yxb1u3V1fW7du8r9Dqq9Ph1Te/3QBCjyJc2zWT/IVvSkpiGRWqidVs8RJwSy512c7KbmndlLPhaTknnxSUPORiJQp230aqTZfZbv5q8r1ySSTOSp7A/oCS4BlwKgEz+8JTI89PxfIK2+fqimIVC2pflPOdk0h1denWlNKV02DbNcUzKwWMBHoB7QDBplZuxLFLgDWuftBwDjg5qjiEZHsSLWmkWpHebY72nPh6q8KSSZzVOYGHAXMjnv8v8D/ligzGzgqdn934EvAytqvagoiNU+qHbXZ7GjPdp/MTmS7pgA0Bz6Ne1wY25awjLsXARuAxiV3ZGbDzKzAzArWrFkTUbgikqtSHQCYzUt6s90nU1FRJgVLsM0rUQZ3n+Tu+e6e37Rp07QEJyKSjHRMVZLNpFRRUS6yUwgcEPe4BbCylDKFZrY70BD4KsKYREQqLJWFmtJxbAh9CJ98EmoIY8dGF0+USWEe0NbMWgOfAQOBs0uUmQmcB/wXGAA8H2v7EhGRmEwmpciSgrsXmdkIQmdyLeA+d19oZmMIHR4zgXuBB81sGaGGMDCqeEREpHyRrtHs7rOAWSW2jY67/w3wiyhjEBGR5GlEs4iIFFNSEBGRYkoKIiJSzKraxT5mtgZYke04StGEMCo7Vym+1OR6fJD7MSq+1KQSXyt3L3egV5VLCrnMzArcPT/bcZRG8aUm1+OD3I9R8aUmE/Gp+UhERIopKYiISDElhfSalO0AyqH4UpPr8UHux6j4UhN5fOpTEBGRYqopiIhIMSUFEREppqRQQWZ2gJnNMbPFZrbQzK5IUKa3mW0ws7djt9GJ9hVhjMvN7N3YsQsSPG9mNsHMlpnZAjPrlsHYDol7X942s41mdmWJMhl//8zsPjNbbWbvxW3b18z+bWZLYz/3KeW158XKLDWz8zIU25/N7P3Y3+9xM2tUymvL/CxEHOMNZvZZ3N/xpFJe29fMlsQ+j6MyGN/0uNiWm9nbpbw20vewtHNK1j5/ySzPptsuS4juB3SL3W8AfAC0K1GmN/BUFmNcDjQp4/mTgGcIixz1AOZmKc5awOeEQTVZff+AY4FuwHtx224BRsXujwJuTvC6fYGPYj/3id3fJwOxnQjsHrt/c6LYkvksRBzjDcBvkvgMfAi0AfYA3in5/xRVfCWe/z9gdDbew9LOKdn6/KmmUEHuvsrd34zd3wQs5ofLjOa6U4G/e/A60MjM9stCHMcDH7p71keou/tL/HCBp1OBB2L3HwB+nuClPwX+7e5fufs64N9A36hjc/d/eVjCFuB1wiJWWVPK+5eM7sAyd//I3b8DphHe97QqKz4zM+BM4OF0HzcZZZxTsvL5U1JIgZnlAV2BuQmePsrM3jGzZ8ysfUYDC0ua/svM5pvZsATPJ7N+diYMpPR/xGy+fzv92N1XQfjHBX6UoEwuvJe/JNT8EinvsxC1EbEmrvtKaf7IhffvGOALd19ayvMZew9LnFOy8vlTUqgkM6sPPAZc6e4bSzz9JqFJpDNwO/BEhsM72t27Af2AS83s2BLPJ7U2dpTMbA+gP/CPBE9n+/2riKy+l2Z2LVAETC2lSHmfhSjdARwIdAFWEZpoSsr6ZxEYRNm1hIy8h+WcU0p9WYJtKb1/SgqVYGa1CX+8qe7+z5LPu/tGd98cuz8LqG1mTTIVn7uvjP1cDTxOqKLHS2b97Kj1A9509y9KPpHt9y/OFzub1WI/Vycok7X3Mtap+DNgsMcamEtK4rMQGXf/wt23u/sO4O5Sjp3Vz6KFteFPB6aXViYT72Ep55SsfP6UFCoo1v54L7DY3W8rpUyzWDnMrDvhfV6bofjqmVmDnfcJHZLvlSg2Ezg3dhVSD2DDzmpqBpX67Syb718JO9cQJ/ZzRoIys4ETzWyfWPPIibFtkTKzvsDVQH9331JKmWQ+C1HGGN9PdVopxy5eyz1WexxIeN8z5QTgfXcvTPRkJt7DMs4p2fn8RdWjXl1vQE9C9WwB8HbsdhIwHBgeKzMCWEi4kuJ14H8yGF+b2HHficVwbWx7fHwGTCRc9fEukJ/h97Au4STfMG5bVt8/QoJaBWwjfPu6AGgMPAcsjf3cN1Y2H7gn7rW/BJbFbudnKLZlhLbknZ/BO2Nl9wdmlfVZyOD792Ds87WAcILbr2SMsccnEa64+TCqGBPFF9t+/87PXVzZjL6HZZxTsvL50zQXIiJSTM1HIiJSTElBRESKKSmIiEgxJQURESmmpCAiIsWUFERizGy77TqDa9pm7DSzvPgZOkVy1e7ZDkAkh2x19y7ZDkIkm1RTEClHbD79m83sjdjtoNj2Vmb2XGzCt+fMrGVs+48trHHwTuz2P7Fd1TKzu2Nz5v/LzPaKlb/czBbF9jMtS7+mCKCkIBJvrxLNR2fFPbfR3bsDfwXGx7b9lTAFeSfChHQTYtsnAC96mNCvG2EkLEBbYKK7twfWA2fEto8Cusb2MzyqX04kGRrRLBJjZpvdvX6C7cuB49z9o9jEZZ+7e2Mz+5IwdcO22PZV7t7EzNYALdz927h95BHmvW8be3w1UNvdbzSzZ4HNhNlgn/DYZIAi2aCagkhyvJT7pZVJ5Nu4+9v5vk/vZMJcVIcD82Mzd4pkhZKCSHLOivv539j91wizegIMBl6J3X8OuBjAzGqZ2d6l7dTMdgMOcPc5wP8DGgE/qK2IZIq+kYh8by/bdfH2Z91952Wpe5rZXMIXqUGxbZcD95nZb4E1wPmx7VcAk8zsAkKN4GLCDJ2J1AKmmFlDwuy149x9fdp+I5EKUp+CSDlifQr57v5ltmMRiZqaj0REpJhqCiIiUkw1BRERKaakICIixZQURESkmJKCiIgUU1IQEZFi/x8h4ax5TbQIAAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "acc = history.history['binary_accuracy']\n",
    "val_acc = history.history['val_binary_accuracy']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(acc) + 1)\n",
    "\n",
    "# \"bo\" is for \"blue dot\"（'bo' 表示蓝色圆点）\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "# b is for \"solid blue line\"（'b' 表示蓝色实线）\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8FeW9x/HPj0URQZaAG8im1A0BMYJWENQWQa0o2ipiXdBSrWi19fbicovlurVVa622Fau2apTaWizeuiGiaBElVBIEFVABI4gBWQWBkN/945nAIeTknORsCfm+X695nVmemfmdycn5nXmemWfM3REREalOo1wHICIidZ+ShYiIJKRkISIiCSlZiIhIQkoWIiKSkJKFiIgkpGQhSTOzxma2wcw6pbNsLpnZIWaW9uvHzexbZrY4ZvpDMxuQTNla7OtPZnZjbdcXSUaTXAcgmWNmG2ImmwObgW3R9A/dvaAm23P3bUCLdJdtCNz90HRsx8wuBy5090Ex2748HdsWqY6SxW7M3bd/WUe/XC9391filTezJu5elo3YRBLR57FuUTVUA2Zmt5rZX83sKTNbD1xoZseb2UwzW2Nmy83sPjNrGpVvYmZuZl2i6Sei5S+Y2Xoze8vMuta0bLR8qJktMLO1ZvY7M/u3mV0SJ+5kYvyhmS0ys9Vmdl/Muo3N7DdmtsrMPgKGVHN8bjaziZXmPWBm90Tjl5vZ+9H7+Sj61R9vWyVmNigab25mj0exzQOOqWK/H0fbnWdmZ0bzjwLuBwZEVXwrY47tLTHrXxG991Vm9qyZHZDMsanJca6Ix8xeMbMvzexzM/tZzH7+Jzom68ys0MwOrKrKz8zerPg7R8dzerSfL4Gbzay7mU2L3svK6Li1ilm/c/QeS6PlvzWzZlHMh8eUO8DMNppZXrz3Kwm4u4YGMACLgW9VmncrsAX4DuGHw17AsUA/wllnN2ABMCYq3wRwoEs0/QSwEsgHmgJ/BZ6oRdl9gfXAsGjZT4CtwCVx3ksyMf4TaAV0Ab6seO/AGGAe0BHIA6aHf4Mq99MN2ADsHbPtL4D8aPo7URkDTgY2AT2jZd8CFsdsqwQYFI3fBbwGtAE6A/Mrlf0ecED0N7kgimG/aNnlwGuV4nwCuCUaHxzF2BtoBvweeDWZY1PD49wKWAH8GNgT2AfoGy27ASgCukfvoTfQFjik8rEG3qz4O0fvrQy4EmhM+Dx+AzgF2CP6nPwbuCvm/bwXHc+9o/InRMsmALfF7OenwKRc/x/W5yHnAWjI0h86frJ4NcF61wN/i8arSgB/jCl7JvBeLcqOAt6IWWbAcuIkiyRjPC5m+T+A66Px6YTquIplp1X+Aqu07ZnABdH4UGBBNWX/D7gqGq8uWSyN/VsAP4otW8V23wNOj8YTJYu/ALfHLNuH0E7VMdGxqeFx/j5QGKfcRxXxVpqfTLL4OEEM5wKzovEBwOdA4yrKnQB8Alg0PQcYnu7/q4Y0qBpKPo2dMLPDzOxfUbXCOmA80K6a9T+PGd9I9Y3a8coeGBuHh//ukngbSTLGpPYFLKkmXoAngRHR+AXA9osCzOwMM3s7qoZZQ/hVX92xqnBAdTGY2SVmVhRVpawBDktyuxDe3/btufs6YDXQIaZMUn+zBMf5IGBRnBgOIiSM2qj8edzfzJ42s8+iGP5cKYbFHi6m2Im7/5twltLfzHoAnYB/1TImQW0WEn5pxnqQ8Ev2EHffB/g54Zd+Ji0n/PIFwMyMnb/cKkslxuWEL5kKiS7t/SvwLTPrSKgmezKKcS/g78AdhCqi1sDLScbxebwYzKwb8AdCVUxetN0PYrab6DLfZYSqrYrttSRUd32WRFyVVXecPwUOjrNevGVfRTE1j5m3f6Uyld/fLwlX8R0VxXBJpRg6m1njOHE8BlxIOAt62t03xyknSVCykMpaAmuBr6IGwh9mYZ//B/Qxs++YWRNCPXj7DMX4NHCtmXWIGjv/u7rC7r6CUFXyKPChuy+MFu1JqEcvBbaZ2RmEuvVkY7jRzFpbuA9lTMyyFoQvzFJC3ryccGZRYQXQMbahuZKngMvMrKeZ7UlIZm+4e9wztWpUd5wnA53MbIyZ7WFm+5hZ32jZn4BbzexgC3qbWVtCkvyccCFFYzMbTUxiqyaGr4C1ZnYQoSqswlvAKuB2CxcN7GVmJ8Qsf5xQbXUBIXFICpQspLKfAhcTGpwfJPyyzqjoC/k84B7CP//BwLuEX5TpjvEPwFRgLjCLcHaQyJOENognY2JeA1wHTCI0Ep9LSHrJGEc4w1kMvEDMF5m7FwP3Ae9EZQ4D3o5ZdwqwEFhhZrHVSRXrv0ioLpoUrd8JGJlkXJXFPc7uvhb4NnAOoUF9ATAwWvxr4FnCcV5HaGxuFlUv/gC4kXCxwyGV3ltVxgF9CUlrMvBMTAxlwBnA4YSzjKWEv0PF8sWEv/MWd59Rw/culVQ0/ojUGVG1wjLgXHd/I9fxSP1lZo8RGs1vyXUs9Z1uypM6wcyGEKoVviZcellG+HUtUitR+88w4Khcx7I7UDWU1BX9gY8J1RNDgLPUICm1ZWZ3EO71uN3dl+Y6nt2BqqFERCQhnVmIiEhCu02bRbt27bxLly65DkNEpF6ZPXv2Snev7lJ1YDdKFl26dKGwsDDXYYiI1CtmlqgXA0DVUCIikgQlCxERSUjJQkREElKyEBGRhJQsREQkoYwlCzN7xMy+MLP34iy36PGJi8ys2Mz6xCy72MwWRsPFmYpRRCQVBQXQpQs0ahReCwoSrVF/95/JS2f/THhecLyugYcSHrvYnfDoxj8A/aKujMcRHr/pwGwzm+zuqzMYq4hIjRQUwOjRsHFjmF6yJEwDjKxtP791eP8ZO7Nw9+mErpvjGQY85sFMoLWFB8ufCkxx9y+jBDGF0FeQiOxmUv1lnMtf9jfdtOOLusLGjWF+slKJPx37r4lctll0YOdHKJZE8+LN34WZjTazQjMrLC0tzVigIpJ+Fb+MlywB9x2/jJP9wkx1/Ypt1PbLemmc7gnjza9q36nEn+r+ayqXyaKqx096NfN3nek+wd3z3T2/ffuEd6uLSB2S6i/jVNdP9cu6U5wH8sabX1mq8ae6/5rKZbIoYefnEHckPPAm3nwRqWNy+cs81fVT/bK+7TZo3nznec2bh/nJSDX+VPdfU7lMFpOBi6Kroo4D1rr7cuAlYLCZtTGzNsDgaJ6I1CG5/mWe6vqpflmPHAkTJkDnzmAWXidMSL5xOdX4U91/jbl7RgbCg+OXA1sJZwuXAVcAV0TLDXgA+IjwnNz8mHVHAYui4dJk9nfMMce4SEPzxBPunTu7m4XXJ57I3vqdO7uHNLHz0Llz8vtu3nzndZs3Tz6GVNdPNf5UpRp/ugCFnsx3ejKF6sOgZCENTa6/bM2q/rI1q9l7yFWyqwtf1qm+/3RINlnsNk/Ky8/Pd3VRLg1Jly6h6qeyzp1h8eK6v35dUFAQ2iiWLg3VP7fdlp17JOoSM5vt7vmJyqm7D5Ecqs8NxNluYM2EkSNDYisvD68NLVHUhJKFSI7U9wbirDewSk4pWYjkSK4v3UzHmYF+mTccShYiKchlNVKqv+x1ZiA1oQZukVqq3JEbhF/myX7h7g4NxFL/qYFbJMNyXY0kkk1KFtKg1edqJJFsyuTzLETqtFSfB9CpU9XVSDXpyG3kSCUHqR90ZiENlqqRRJKnZCENlqqRRJKnaihpsFSNJJI8nVlIg6VqJJHkKVlIvZbK1UyqRhJJnqqhpN5K9WqminJKDiKJ6cxC6q1Ur2YSkeQpWUi9lerVTCKSPCULqbdS7WJbRJKnZCE5lUoDta5mEskeJQvJmVQf/qOrmUSyR12US86oi26R3FMX5VLnqYFapP5QspCcUQO1SP2hZCEpUQO1SMOgZCG1pgZqkYZDDdxSa2qgFqn/1MAtGacGapGGQ8lCak0N1CINh5KF1JoaqEUaDiULqTU1UIs0HHqehaREz4MQaRh0ZiEiIgkpWTRwqdxUJyINh6qhGrB0PJZURBoGnVk0YHosqYgkS8miAdNNdSKSrIwmCzMbYmYfmtkiMxtbxfLOZjbVzIrN7DUz6xizbJuZzYmGyZmMs6HSTXUikqyMJQszaww8AAwFjgBGmNkRlYrdBTzm7j2B8cAdMcs2uXvvaDgzU3E2ZLqpTkSSlckzi77AInf/2N23ABOBYZXKHAFMjcanVbFcMkg31YlIsjKZLDoAn8ZMl0TzYhUB50TjZwMtzSwvmm5mZoVmNtPMzqpqB2Y2OipTWFpams7YG4yRI0MPseXl4VWJQkSqkslkYVXMq9wf+vXAQDN7FxgIfAaURcs6Rd3mXgDca2YH77Ix9wnunu/u+e3bt09j6CIiEiuT91mUAAfFTHcElsUWcPdlwHAAM2sBnOPua2OW4e4fm9lrwNHARxmMV0RE4sjkmcUsoLuZdTWzPYDzgZ2uajKzdmZWEcMNwCPR/DZmtmdFGeAEYH4GY623dAe2iGRDxs4s3L3MzMYALwGNgUfcfZ6ZjQcK3X0yMAi4w8wcmA5cFa1+OPCgmZUTEtqd7q5kUYnuwBaRbNFjVesxPdZURFKlx6o2ALoDW0SyRcmiHtMd2CKSLUoW9ZjuwBaRbFGyqMd0B7aIZIueZ1HP6bGmIpINOrMQEZGElCxERCQhJQsREUlIyUJERBJSshARkYSULEREJCElixxTr7EiUh/oPoscUq+xIlJf6Mwih266aUeiqLBxY5gvIlKXKFnkkHqNFZH6Qskih9RrrIjUF0oWOaReY0WkvlCyyCH1Gisi9YWuhsox9RorIvWBzixERCQhJQsREUlIyUJERBJSshARkYSULEREJCElCxERSUjJQkREElKyEBGRhJQsREQkISULERFJSMlCREQSUrIQEZGElCxERCQhJYsUFRRAly7QqFF4LSjIdUQiIumnLspTUFAAo0fveI72kiVhGtTtuIjsXnRmkYKbbtqRKCps3Bjmi4jsTpQsUrB0ac3mi4jUVxlNFmY2xMw+NLNFZja2iuWdzWyqmRWb2Wtm1jFm2cVmtjAaLs5knLXVqVPN5ouI1FcZSxZm1hh4ABgKHAGMMLMjKhW7C3jM3XsC44E7onXbAuOAfkBfYJyZtclUrLV1223QvPnO85o3D/NFRHYnmTyz6AsscveP3X0LMBEYVqnMEcDUaHxazPJTgSnu/qW7rwamAEMyGGutjBwJEyZA585gFl4nTFDjtojsfjKZLDoAn8ZMl0TzYhUB50TjZwMtzSwvyXXrhJEjYfFiKC8Pr0oUIrI7SipZmNnBZrZnND7IzK4xs9aJVqtinleavh4YaGbvAgOBz4CyJNfFzEabWaGZFZaWliZ8HyIiUjvJnlk8A2wzs0OAh4GuwJMJ1ikBDoqZ7ggsiy3g7svcfbi7Hw3cFM1bm8y6UdkJ7p7v7vnt27dP8q2IiEhNJZssyt29jFBVdK+7XwcckGCdWUB3M+tqZnsA5wOTYwuYWTszq4jhBuCRaPwlYLCZtYkatgdH80REJAeSTRZbzWwEcDHwf9G8ptWtECWXMYQv+feBp919npmNN7Mzo2KDgA/NbAGwH3BbtO6XwP8SEs4sYHw0T0REcsDcd2kK2LVQuOT1CuAtd3/KzLoC57n7nZkOMFn5+fleWFiY6zBEROoVM5vt7vmJyiXVN5S7zweuiTbcBmhZlxKFiIhkVrJXQ71mZvtEN8sVAY+a2T2ZDU1EROqKZNssWrn7OmA48Ki7HwN8K3NhiYhIXZJssmhiZgcA32NHA7cAK1bAqafCxIm5jkREJHOSfZ7FeMJVTf9291lm1g1YmLmw6ofNm2H4cJgxA6ZOhT32CNMiIrubpM4s3P1v7t7T3a+Mpj9293MSrbc7cw8POpoxAx55BPr2hfPPhxdfzHVkySspgb/9DW64AV7SXSwiUo1kG7g7mtkkM/vCzFaY2TOx3Yk3RL/+NTz2GIwbB5deCs8/D0ceCWefDa+/nuvodrV1KxQWwn33haTWqRMcdBB873vwy1/CkCEwdCjMn5/rSEWkLkq2zeJRwt3XBxI69HsumtcgPfccjB0L3/0u/PznYV7r1vDyy9C1K5xxBrzzTm5jXLkyxHnDDTBwILRqBcceCz/+Mbz1FpxwAvz2tzBrFqxfD/fcAzNnQs+e8KMfQTa72lq4EDZtyt7+RKTmkr0pb4679040L5eydVPe3LnwzW/CoYfC9Om7Ps/is8/gxBNh9Wp47bXw5Ztp5eXhjGDGjB3DwqhFqWlTOProEPM3vwnHHw8d45wTrloFv/gF/P73sPfecPPNcM01sOee6Y9582b4+9/hgQdC8howICTbZs3Svy8RiS/Zm/Jw94QD8ApwIdA4Gi4EpiazbraGY445xjPtiy/cu3RxP+AA95KS+OU++cS9Y0f3ffd1/+CDzMY0ebL7gQe6h1YU9/bt3YcNc//lL93feMN948aab/P9991PPz1sr2tX97/9zb28PD3xLlnifuONIU5wP+QQ9zFjwvjw4e5lZenZj4gkByj0ZPJAUoWgE6EaqhT4AngW6JTMutkaMp0svv7avX9/92bN3N95J3H5Dz4IyaJDB/ePP05/PGvWuF9ySfgL9url/pe/uC9cmL4vdXf3l192P+qosI/+/ZN731UpL3efMsX9rLPcGzUKw5lnur/4ovu2baHMb34T9vOjH6X3PYhI9dKaLKpcEa6t7bqZGDKZLMrL3S+9NBytp55Kfr2iIvc2bdy7dav+TKSmpk5179QpfOnedJP75s3p23ZlZWXuDz4YEh+4f//77p9+mty6a9a4//a37oceGtZt18597Nhw5lWV//qvUO7WW9MWvogkkI1ksbS262ZiyGSyuOuucKT+539qvu7bb7u3aOF+2GGhGisVX33lfvXVIZZvfMN95szUtlcTa9e633CD+557uu+1l/vPf+6+fn3VZYuK3EePdm/ePMTar5/7Y4+5b9pU/T62bQvJCNwffjj97yFVZWUhAS5Z4j53rvubb7o//7z7xInu//iH++uvu7/3nvvy5ZlN4CLplGyySKqBO06jyKfuflDiktmRqQbu558PVzcNHw5PPw2NavEg2unTw6Wphx4K06aFK6dqauZMuPhiWLAgNDrfcceujevZsHhxuBLsr3+FAw6A22+Hiy6CsjL4xz9Cg/Wbb4aG6hEj4Kqr4Jhjkt/+1q3wne/AK6/As8+GY59J7jBpUrhwYe1aWLcuDBXjsa8bNtRs2y1aQF7ezkPbtrvOy8uDHj1y8/cUSbaBO5VksdTdO9Vq5QzIRLKYNy9cPXTIIfDGG+EKodp66aXwJXjMMTBlSvgiScaWLeEKpTvvDFcxPfoonHxy7eNIl7feguuug7ffhqOOCpfafv45dOsGV14Z7j3Jy6vdtjdsgJNOCsf/1VfhuOPSG3uFdevghz/c0VVLixawzz5haNWq6teq5rVsGf5Oq1btGL78cufp2HmrV4ckFWvffeFnPwvHTklDsiktV0MB64F1VQzrgbJkTl2yNaS7Gqq0NFwJtN9+7kuXpmeb//iHe+PG7iedlNxVSkVFofEaQpvJmjXpiSNdtm1zf/JJ9x493E87zf1f/9rRYJ2qFSvcDz7YvW3bcHVWur37rnv37qHd5/bb3bduTf8+4ikrc1+50n3BAve33nJ/5hn3b387/J333df97rtDlaNINpDpNou6NqQzWWze7H7iiaF+Pt3tAo8/7m4WLk2NV6+9dav7HXe4N20aktXkyemNob5YtCh8eXbq5P7ZZ+nZZnm5+x//GP62Bx7oPn16erabDm+84X7KKeG/cv/9wxVitbn0WaQmlCxqqbzc/fLLw5EpKEjLJnfx4INh+9/97q6/aBcscD/uuLD83HPDGU5DNnt2uEDgqKPcV69ObVvr1rmPGBGO7amnpn7BQaZMn+5+8sk7ksa99yppSOYkmyyS7XW2wbjvPvjTn+DGG+GCCzKzj9Gj4auv4Cc/CfXTjzwS5v/+96HeulkzePLJ0IeTWWZiqC/69AkN56efDmedFTpqrM1d3sXFoXuWRYvgtttCI31tLlbIhgEDQi/Gr78e2quuvTb03zV2bPjspOMud/dwscTMmaHdafVqaNIk3PHfpEn88XjLO3SAXr3Ca0P/zO62ksko9WFIx5nFCy+EOuyzzkpf3Xt1xo8Pvx5HjdpR/TBkSPqqXHYnBQU7zrZqcpd3ebn7hAnhZsoDDgiXt9Y306a5DxwY3v+BB7rfd1/iy5ArW7Mm3GT5i1+4Dx0a7v+puOu/VatwKXa3bu4HHRSOU/v27q1bh7O6Zs1CW1tF+eqGvLzwWf7JT8Ll0sXF7lu2ZOKoSLqgaqiamT/ffZ99QoNyvPsH0q28fMeNaHvvHaqndPdyfHffHY7VmDHJHad169wvuCCsM3hwaDSvz1591X3AgPB+OnRwv//+qpPGtm3hPpCHHnK/7DL3I44I7WQQXnv0CFWtDz/sPm9e8j+MystDtenGjeHYrloVjmlJSbjn5P77w3aPPTYkmIoEssce7kcfHS7SuPde99deS71KUdIn2WRR60tn65pULp1dtQr69Qu9r86aFbrvzhb3cM9Cv36hx1qp3vXXw913h/tMxo6NXy622mn8+ND7bl2tdqoJ93Cvzrhx4X6WDh1ClWmXLuFy5pkzQ4/H69aF8m3bhkuPjz8+vPbtGy73zbSyslDNNWdOGIqK4N13d+7NuHNn6N07DO3ahXtstmzZMSSajp1XXp5avN27w/e/D4MG7R6fk5rI+H0WdU1tk8XWrTB4cOip9bXXwj+V1F3l5eGf+sknwz0nl1yy83J3ePhhuPpqaNMGnnoqdNG+u3EP96CMGwf//neY16hR6OW4IjFU3CNUV9oQ3MO9OBXJoyKRLFiw630nEHo7bto0PIEydqg8r2lTaNy49nGVl4dnvaxfH57x8v3vh+Gww2q/zZpYujTcW9StW256XVaySNJHH4Uuxe+8M3xApO7bsiXc2f3qqzB5Mpx2Wpi/YUO4qe2JJ+Bb3wqv++2X21gzzT380Nm6FfLzk7/Zsy7ZuDFc8BGbBBo3zm6S27QJ/vnP8ECzl14KCeTYY0PvBOefH8580mXdunB2OGVKGBYsCPPNwtlW9+7wjW/seP3GN8L8Jhm6HEnJogbWrw934Ur9sX59qDL44IOQNJo3D9VOCxeGK4huuCG1X5vScH3+eThzfeyxcAbUpEm4Gu+ii8JrTZ/vUlYWqrdffjkkh5kzYdu28JkdOBC+/e1wB//ChSFxVLxWVCVCSKTduu2cQCrGDzwwtaozJQvZ7a1YER7otHp1+GXYunWodho0KNeRye6iqAgefxwKCkISadMmnGlcdFFoZ4x39vPRRzuSw6uvhr7FzEJ3P9/+dhi++c34icc9tO/EJo+K8YUL4euvd5Tda69Qlf7ss7V7j0oW0iAsWhSqEXv0CP/Uu3u1k+RGWVno3PKxx0LHk19/HX7ZX3QRXHhh6Cfs1Vd3JIhPPgnrdeoUEsPgwaFPt3RUZ5WXhydyxiaS1q13POK5ppQspMEoK8tcfa5IZevWhUcCP/ZYuHESQjVQeXloMzr55B0Jonv3unOBQTxKFiIiGbZ4caj63Lw5XFTRr19oX6hPkk0W+j0mIlJLXbqEiykaggZ2+4mIiNSGkoWIiCSkZCEiIgkpWYiISEJKFiIikpCShYiIJKRkISIiCWU0WZjZEDP70MwWmdkuTx8ws05mNs3M3jWzYjM7LZrfxcw2mdmcaPhjJuMUEZHqZeymPDNrDDwAfBsoAWaZ2WR3nx9T7GbgaXf/g5kdATwPdImWfeTuvTMVn4iIJC+TZxZ9gUXu/rG7bwEmAsMqlXGg4rldrYBlGYxHRERqKZPJogPwacx0STQv1i3AhWZWQjiruDpmWdeoeup1MxtQ1Q7MbLSZFZpZYWns8xpFRCStMpksquprsXKvhSOAP7t7R+A04HEzawQsBzq5+9HAT4AnzWyXJwe7+wR3z3f3/Pbt26c5fBERqZDJZFECHBQz3ZFdq5kuA54GcPe3gGZAO3ff7O6rovmzgY+Ab2QwVhERqUYmk8UsoLuZdTWzPYDzgcmVyiwFTgEws8MJyaLUzNpHDeSYWTegO/BxBmMVEZFqZOxqKHcvM7MxwEtAY+ARd59nZuOBQnefDPwUeMjMriNUUV3i7m5mJwLjzawM2AZc4e5fZipWERGpnh5+JCLSgCX78CPdwS0iIgkpWYiISEJKFiIikpCShYiIJKRkISIiCSlZiIhIQkoWIiKSkJKFiIgkpGQhIiIJKVmIiEhCShYiIpKQkoWIiCSkZCEiIgkpWYiISEJKFiIikpCShYiIJKRkISIiCSlZiIhIQkoWIiKSkJKFiIgkpGQhIiIJKVmIiEhCShYiIpKQkoWIiCSkZCEiIgkpWYiISEJKFiIikpCShYiIJNQk1wGISP23detWSkpK+Prrr3MdisTRrFkzOnbsSNOmTWu1vpKFiKSspKSEli1b0qVLF8ws1+FIJe7OqlWrKCkpoWvXrrXahqqhRCRlX3/9NXl5eUoUdZSZkZeXl9KZn5KFiKSFEkXdlurfR8lCREQSUrIQkawrKIAuXaBRo/BaUJDa9latWkXv3r3p3bs3+++/Px06dNg+vWXLlqS2cemll/Lhhx9WW+aBBx6gINVg6yk1cItIVhUUwOjRsHFjmF6yJEwDjBxZu23m5eUxZ84cAG655RZatGjB9ddfv1MZd8fdadSo6t/Ijz76aML9XHXVVbULcDegMwsRyaqbbtqRKCps3Bjmp9uiRYvo0aMHV1xxBX369GH58uWMHj2a/Px8jjzySMaPH7+9bP/+/ZkzZw5lZWW0bt2asWPH0qtXL44//ni++OILAG6++Wbuvffe7eXHjh1L3759OfTQQ5kxYwYAX331Feeccw69evVixIgR5Ofnb09kscaNG8exxx67PT53B2DBggWcfPLJ9OrViz59+rB48WIAbr/9do466ih69erFTZk4WAlkNFmY2RAz+9DMFpnZ2CqWdzKzaWb2rpkVm9lpMctuiNb70MxOzWScIpI9S5fWbH6q5s+fz2WXXca7775Lhw4duPPOOyksLKSoqIhQtncoAAAP80lEQVQpU6Ywf/78XdZZu3YtAwcOpKioiOOPP55HHnmkym27O++88w6//vWvtyee3/3ud+y///4UFRUxduxY3n333SrX/fGPf8ysWbOYO3cua9eu5cUXXwRgxIgRXHfddRQVFTFjxgz23XdfnnvuOV544QXeeecdioqK+OlPf5qmo5O8jCULM2sMPAAMBY4ARpjZEZWK3Qw87e5HA+cDv4/WPSKaPhIYAvw+2p6I1HOdOtVsfqoOPvhgjj322O3TTz31FH369KFPnz68//77VSaLvfbai6FDhwJwzDHHbP91X9nw4cN3KfPmm29y/vnnA9CrVy+OPPLIKtedOnUqffv2pVevXrz++uvMmzeP1atXs3LlSr7zne8A4Ua65s2b88orrzBq1Cj22msvANq2bVvzA5GiTJ5Z9AUWufvH7r4FmAgMq1TGgX2i8VbAsmh8GDDR3Te7+yfAomh7IlLP3XYbNG++87zmzcP8TNh77723jy9cuJDf/va3vPrqqxQXFzNkyJAq7z3YY489to83btyYsrKyKre955577lKmojqpOhs3bmTMmDFMmjSJ4uJiRo0atT2Oqi5xdfecX5qcyWTRAfg0ZrokmhfrFuBCMysBngeursG6mNloMys0s8LS0tJ0xS0iGTRyJEyYAJ07g1l4nTCh9o3bNbFu3TpatmzJPvvsw/Lly3nppZfSvo/+/fvz9NNPAzB37twqz1w2bdpEo0aNaNeuHevXr+eZZ54BoE2bNrRr147nnnsOCDc7bty4kcGDB/Pwww+zadMmAL788su0x51IJpNFVWmwcsodAfzZ3TsCpwGPm1mjJNfF3Se4e76757dv3z7lgEUkO0aOhMWLobw8vGYjUQD06dOHI444gh49evCDH/yAE044Ie37uPrqq/nss8/o2bMnd999Nz169KBVq1Y7lcnLy+Piiy+mR48enH322fTr12/7soKCAu6++2569uxJ//79KS0t5YwzzmDIkCHk5+fTu3dvfvOb36Q97kQsmVOmWm3Y7HjgFnc/NZq+AcDd74gpMw8Y4u6fRtMfA8cBl8WWNbOXom29FW9/+fn5XlhYmJH3IiLVe//99zn88MNzHUadUFZWRllZGc2aNWPhwoUMHjyYhQsX0qRJ7u9UqOrvZGaz3T0/0bqZjH4W0N3MugKfERqsL6hUZilwCvBnMzscaAaUApOBJ83sHuBAoDvwTgZjFRFJiw0bNnDKKadQVlaGu/Pggw/WiUSRqoy9A3cvM7MxwEtAY+ARd59nZuOBQnefDPwUeMjMriNUM13i4VRnnpk9DcwHyoCr3H1bpmIVEUmX1q1bM3v27FyHkXYZTXfu/jyh4Tp23s9jxucDVVYauvttQIaujxARkZrQHdwiIpKQkoWIiCSkZCEiIgkpWYhIvTdo0KBdbrC79957+dGPflTtei1atABg2bJlnHvuuXG3neiy/HvvvZeNMb0jnnbaaaxZsyaZ0OsNJQsRqfdGjBjBxIkTd5o3ceJERowYkdT6Bx54IH//+99rvf/KyeL555+ndevWtd5eXVT/L/4VkTrl2muhih65U9K7N0Q9g1fp3HPP5eabb2bz5s3sueeeLF68mGXLltG/f382bNjAsGHDWL16NVu3buXWW29l2LCdu6lbvHgxZ5xxBu+99x6bNm3i0ksvZf78+Rx++OHbu9gAuPLKK5k1axabNm3i3HPP5Re/+AX33Xcfy5Yt46STTqJdu3ZMmzaNLl26UFhYSLt27bjnnnu291p7+eWXc+2117J48WKGDh1K//79mTFjBh06dOCf//zn9o4CKzz33HPceuutbNmyhby8PAoKCthvv/3YsGEDV199NYWFhZgZ48aN45xzzuHFF1/kxhtvZNu2bbRr146pU6em7W+gZCEi9V5eXh59+/blxRdfZNiwYUycOJHzzjsPM6NZs2ZMmjSJffbZh5UrV3Lcccdx5plnxu2Y7w9/+APNmzenuLiY4uJi+vTps33ZbbfdRtu2bdm2bRunnHIKxcXFXHPNNdxzzz1MmzaNdu3a7bSt2bNn8+ijj/L222/j7vTr14+BAwfSpk0bFi5cyFNPPcVDDz3E9773PZ555hkuvPDCndbv378/M2fOxMz405/+xK9+9Svuvvtu/vd//5dWrVoxd+5cAFavXk1paSk/+MEPmD59Ol27dk17/1FKFiKSVtWdAWRSRVVURbKo+DXv7tx4441Mnz6dRo0a8dlnn7FixQr233//Krczffp0rrnmGgB69uxJz549ty97+umnmTBhAmVlZSxfvpz58+fvtLyyN998k7PPPnt7z7fDhw/njTfe4Mwzz6Rr16707t0biN8NeklJCeeddx7Lly9ny5YtdO3aFYBXXnllp2q3Nm3a8Nxzz3HiiSduL5PubswbfJtFup8FLCK5cdZZZzF16lT+85//sGnTpu1nBAUFBZSWljJ79mzmzJnDfvvtV2W35LGqOuv45JNPuOuuu5g6dSrFxcWcfvrpCbdTXd97Fd2bQ/xu0K+++mrGjBnD3LlzefDBB7fvr6ouyzPdjXmDThYVzwJesgTcdzwLWAlDpP5p0aIFgwYNYtSoUTs1bK9du5Z9992Xpk2bMm3aNJYsWVLtdk488UQKoi+B9957j+LiYiB0b7733nvTqlUrVqxYwQsvvLB9nZYtW7J+/foqt/Xss8+yceNGvvrqKyZNmsSAAQOSfk9r166lQ4fwdIa//OUv2+cPHjyY+++/f/v06tWrOf7443n99df55JNPgPR3Y96gk0U2nwUsIpk3YsQIioqKtj+pDmDkyJEUFhaSn59PQUEBhx12WLXbuPLKK9mwYQM9e/bkV7/6FX37hueu9erVi6OPPpojjzySUaNG7dS9+ejRoxk6dCgnnXTSTtvq06cPl1xyCX379qVfv35cfvnlHH300Um/n1tuuYXvfve7DBgwYKf2kJtvvpnVq1fTo0cPevXqxbRp02jfvj0TJkxg+PDh9OrVi/POOy/p/SQjY12UZ1ttuihv1CicUVRmFvrZF5HkqIvy+iGVLsob9JlFtp8FLCJSXzXoZJHtZwGLiNRXDTpZ5PJZwCK7m92lSnt3lerfp8HfZzFypJKDSKqaNWvGqlWryMvLy+jlm1I77s6qVato1qxZrbfR4JOFiKSuY8eOlJSUUFpamutQJI5mzZrRsWPHWq+vZCEiKWvatOn2O4dl99Sg2yxERCQ5ShYiIpKQkoWIiCS029zBbWalQPWdvuRWO2BlroOohuJLjeJLjeJLTSrxdXb39okK7TbJoq4zs8JkbqnPFcWXGsWXGsWXmmzEp2ooERFJSMlCREQSUrLIngm5DiABxZcaxZcaxZeajMenNgsREUlIZxYiIpKQkoWIiCSkZJEmZnaQmU0zs/fNbJ6Z/biKMoPMbK2ZzYmGn+cgzsVmNjfa/y6PFrTgPjNbZGbFZtYni7EdGnNs5pjZOjO7tlKZrB5DM3vEzL4ws/di5rU1sylmtjB6bRNn3YujMgvN7OIsxvdrM/sg+vtNMrPWcdat9rOQwfhuMbPPYv6Gp8VZd4iZfRh9FsdmMb6/xsS22MzmxFk3G8evyu+VnHwG3V1DGgbgAKBPNN4SWAAcUanMIOD/chznYqBdNctPA14ADDgOeDtHcTYGPifcMJSzYwicCPQB3ouZ9ytgbDQ+FvhlFeu1BT6OXttE422yFN9goEk0/suq4kvms5DB+G4Brk/i7/8R0A3YAyiq/P+UqfgqLb8b+HkOj1+V3yu5+AzqzCJN3H25u/8nGl8PvA90yG1UtTIMeMyDmUBrMzsgB3GcAnzk7jm9K9/dpwNfVpo9DPhLNP4X4KwqVj0VmOLuX7r7amAKMCQb8bn7y+5eFk3OBGrfL3WK4hy/ZPQFFrn7x+6+BZhIOO5pVV18Fh7M8T3gqXTvN1nVfK9k/TOoZJEBZtYFOBp4u4rFx5tZkZm9YGZHZjWwwIGXzWy2mY2uYnkH4NOY6RJyk/TOJ/4/aa6P4X7uvhzCPzOwbxVl6spxHEU4U6xKos9CJo2JqskeiVOFUheO3wBghbsvjLM8q8ev0vdK1j+DShZpZmYtgGeAa919XaXF/yFUq/QCfgc8m+34gBPcvQ8wFLjKzE6stLyqx5xl9fpqM9sDOBP4WxWL68IxTEZdOI43AWVAQZwiiT4LmfIH4GCgN7CcUNVTWc6PHzCC6s8qsnb8EnyvxF2tinm1PoZKFmlkZk0Jf9ACd/9H5eXuvs7dN0TjzwNNzaxdNmN092XR6xfAJMLpfqwS4KCY6Y7AsuxEt91Q4D/uvqLygrpwDIEVFVVz0esXVZTJ6XGMGjPPAEZ6VIFdWRKfhYxw9xXuvs3dy4GH4uw318evCTAc+Gu8Mtk6fnG+V7L+GVSySJOofvNh4H13vydOmf2jcphZX8LxX5XFGPc2s5YV44SG0PcqFZsMXBRdFXUcsLbidDeL4v6iy/UxjEwGKq4suRj4ZxVlXgIGm1mbqJplcDQv48xsCPDfwJnuvjFOmWQ+C5mKL7YN7Ow4+50FdDezrtGZ5vmE454t3wI+cPeSqhZm6/hV872S/c9gJlvyG9IA9Cec4hUDc6LhNOAK4IqozBhgHuHKjpnAN7McY7do30VRHDdF82NjNOABwpUoc4H8LMfYnPDl3ypmXs6OISFpLQe2En6pXQbkAVOBhdFr26hsPvCnmHVHAYui4dIsxreIUFdd8Tn8Y1T2QOD56j4LWYrv8eizVUz40jugcnzR9GmEq38+ymZ80fw/V3zmYsrm4vjF+17J+mdQ3X2IiEhCqoYSEZGElCxERCQhJQsREUlIyUJERBJSshARkYSULEQSMLNttnNvuGnrAdXMusT2eCpSVzXJdQAi9cAmd++d6yBEcklnFiK1FD3P4Jdm9k40HBLN72xmU6OO8qaaWado/n4Wni9RFA3fjDbV2Mweip5X8LKZ7RWVv8bM5kfbmZijtykCKFmIJGOvStVQ58UsW+fufYH7gXujefcTunnvSejE775o/n3A6x46QexDuPMXoDvwgLsfCawBzonmjwWOjrZzRabenEgydAe3SAJmtsHdW1QxfzFwsrt/HHX29rm755nZSkIXFluj+cvdvZ2ZlQId3X1zzDa6EJ450D2a/m+gqbvfamYvAhsIPes+61EHiiK5oDMLkdR4nPF4ZaqyOWZ8GzvaEk8n9NN1DDA76glVJCeULERSc17M61vR+AxCL6kAI4E3o/GpwJUAZtbYzPaJt1EzawQc5O7TgJ8BrYFdzm5EskW/VEQS28vM5sRMv+juFZfP7mlmbxN+eI2I5l0DPGJm/wWUApdG838MTDCzywhnEFcSejytSmPgCTNrRegJ+DfuviZt70ikhtRmIVJLUZtFvruvzHUsIpmmaigREUlIZxYiIpKQzixERCQhJQsREUlIyUJERBJSshARkYSULEREJKH/B9CWJFzkZR5vAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.clf()   # clear figure（清空图像）\n",
    "acc_values = history_dict['binary_accuracy']\n",
    "val_acc_values = history_dict['val_binary_accuracy']\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The dots are the training loss and accuracy, while the solid lines are the validation loss and accuracy. Note that your own results may vary \n",
    "slightly due to a different random initialization of your network.\n",
    "\n",
    "As you can see, the training loss decreases with every epoch and the training accuracy increases with every epoch. That's what you would \n",
    "expect when running gradient descent optimization -- the quantity you are trying to minimize should get lower with every iteration. But that \n",
    "isn't the case for the validation loss and accuracy: they seem to peak at the fourth epoch. This is an example of what we were warning \n",
    "against earlier: a model that performs better on the training data isn't necessarily a model that will do better on data it has never seen \n",
    "before. In precise terms, what you are seeing is \"overfitting\": after the second epoch, we are over-optimizing on the training data, and we \n",
    "ended up learning representations that are specific to the training data and do not generalize to data outside of the training set.\n",
    "\n",
    "In this case, to prevent overfitting, we could simply stop training after three epochs. In general, there is a range of techniques you can \n",
    "leverage to mitigate overfitting, which we will cover in the next chapter.\n",
    "\n",
    "Let's train a new network from scratch for four epochs, then evaluate it on our test data:\n",
    "\n",
    "点是训练损失和准确率，而实线是验证损失和准确性。 请注意，由于网络的随机初始化不同，您自己的结果可能略有不同。\n",
    "\n",
    "如你所见，训练损失每轮都在降低，训练精度每轮都在提升。这就是梯度下降优化的预期 结果——你想要最小化的量随着每次迭代越来越小。但验证损失和验证精度并非如此：它们似 乎在第四轮达到最佳值。这就是我们之前警告过的一种情况：模型在训练数据上的表现越来越好， 但在前所未见的数据上不一定表现得越来越好。准确地说，你看到的是过拟合（overfit）：在第二轮之后，你对训练数据过度优化，最终学到的表示仅针对于训练数据，无法泛化到训练集之外的数据。\n",
    "\n",
    "在这种情况下，为了防止过拟合，你可以在 3 轮之后停止训练。通常来说，你可以使用许 多方法来降低过拟合，我们将在第 4 章中详细介绍。\n",
    "\n",
    "我们从头开始训练一个新的网络，训练 4 轮，然后在测试数据上评估模型。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/4\n",
      "25000/25000 [==============================] - 3s 120us/step - loss: 0.4749 - acc: 0.8217\n",
      "Epoch 2/4\n",
      "25000/25000 [==============================] - ETA: 0s - loss: 0.2655 - acc: 0.910 - 3s 109us/step - loss: 0.2658 - acc: 0.9097\n",
      "Epoch 3/4\n",
      "25000/25000 [==============================] - 3s 105us/step - loss: 0.1982 - acc: 0.9299\n",
      "Epoch 4/4\n",
      "25000/25000 [==============================] - 3s 108us/step - loss: 0.1679 - acc: 0.9404\n",
      "25000/25000 [==============================] - 4s 171us/step\n"
     ]
    }
   ],
   "source": [
    "model = models.Sequential()\n",
    "model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n",
    "model.add(layers.Dense(16, activation='relu'))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))\n",
    "model.compile(optimizer='rmsprop',\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "model.fit(x_train, y_train, epochs=4, batch_size=512)\n",
    "results = model.evaluate(x_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.32315461338043211, 0.87348000000000003]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our fairly naive approach achieves an accuracy of 88%. With state-of-the-art approaches, one should be able to get close to 95%.\n",
    "\n",
    "这种相当简单的方法得到了 88% 的精度。利用最先进的方法，你应该能够得到接近 95% 的 精度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using a trained network to generate predictions on new data\n",
    "\n",
    "After having trained a network, you will want to use it in a practical setting. You can generate the likelihood of reviews being positive \n",
    "by using the `predict` method:\n",
    "\n",
    "## 使用训练好的网络在新数据上生成预测结果\n",
    "\n",
    "训练好网络之后，你希望将其用于实践。你可以用 predict 方法来得到评论为正面的可能性大小。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.14026152],\n",
       "       [ 0.99970287],\n",
       "       [ 0.29552525],\n",
       "       ..., \n",
       "       [ 0.07234977],\n",
       "       [ 0.04342838],\n",
       "       [ 0.48153383]], dtype=float32)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the network is very confident for some samples (0.99 or more, or 0.01 or less) but less confident for others (0.6, 0.4). \n",
    "\n",
    "如你所见，网络对某些样本的结果非常确信（大于等于 0.99，或小于等于 0.01），但对其他结果却不那么确信（0.6 或 0.4）。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Further experiments\n",
    "\n",
    "\n",
    "* We were using 2 hidden layers. Try to use 1 or 3 hidden layers and see how it affects validation and test accuracy.\n",
    "* Try to use layers with more hidden units or less hidden units: 32 units, 64 units...\n",
    "* Try to use the `mse` loss function instead of `binary_crossentropy`.\n",
    "* Try to use the `tanh` activation (an activation that was popular in the early days of neural networks) instead of `relu`.\n",
    "\n",
    "These experiments will help convince you that the architecture choices we have made are all fairly reasonable, although they can still be \n",
    "improved!\n",
    "\n",
    "## 进一步的实验\n",
    "通过以下实验，你可以确信前面选择的网络架构是非常合理的，虽然仍有改进的空间。\n",
    "\n",
    "* 前面使用了两个隐藏层。你可以尝试使用一个或三个隐藏层，然后观察对验证精度和测试精度的影响。\n",
    "\n",
    "* 尝试使用更多或更少的隐藏单元，比如 32 个、64 个等。\n",
    "\n",
    "* 尝试使用 mse 损失函数代替 binary_crossentropy。\n",
    "\n",
    "* 尝试使用 tanh 激活（这种激活在神经网络早期非常流行）代替 relu。\n",
    "\n",
    "这些实验将有助于说服您，我们所做的架构选择都是相当合理的，尽管它们仍然可以改进！\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusions\n",
    "\n",
    "\n",
    "Here's what you should take away from this example:\n",
    "\n",
    "* There's usually quite a bit of preprocessing you need to do on your raw data in order to be able to feed it -- as tensors -- into a neural \n",
    "network. In the case of sequences of words, they can be encoded as binary vectors -- but there are other encoding options too.\n",
    "* Stacks of `Dense` layers with `relu` activations can solve a wide range of problems (including sentiment classification), and you will \n",
    "likely use them frequently.\n",
    "* In a binary classification problem (two output classes), your network should end with a `Dense` layer with 1 unit and a `sigmoid` activation, \n",
    "i.e. the output of your network should be a scalar between 0 and 1, encoding a probability.\n",
    "* With such a scalar sigmoid output, on a binary classification problem, the loss function you should use is `binary_crossentropy`.\n",
    "* The `rmsprop` optimizer is generally a good enough choice of optimizer, whatever your problem. That's one less thing for you to worry \n",
    "about.\n",
    "* As they get better on their training data, neural networks eventually start _overfitting_ and end up obtaining increasingly worse results on data \n",
    "never-seen-before. Make sure to always monitor performance on data that is outside of the training set.\n",
    "\n",
    "## 小结\n",
    "下面是你应该从这个例子中学到的要点。\n",
    "\n",
    "* 通常需要对原始数据进行大量预处理，以便将其转换为张量输入到神经网络中。单词序列可以编码为二进制向量，但也有其他编码方式。\n",
    "\n",
    "* 带有 relu 激活的 Dense 层堆叠，可以解决很多种问题（包括情感分类），你可能会经常用到这种模型。\n",
    "\n",
    "* 对于二分类问题（两个输出类别），网络的最后一层应该是只有一个单元并使用 sigmoid激活的 Dense 层，网络输出应该是 0~1 范围内的标量，表示概率值。\n",
    "\n",
    "* 对于二分类问题的 sigmoid 标量输出，你应该使用 binary_crossentropy 损失函数。\n",
    "\n",
    "* 无论你的问题是什么，rmsprop 优化器通常都是足够好的选择。这一点你无须担心。\n",
    "\n",
    "* 随着神经网络在训练数据上的表现越来越好，模型最终会过拟合，并在前所未见的数据上得到越来越差的结果。一定要一直监控模型在训练集之外的数据上的性能。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}