5.9. 含并行连结的网络(GoogLeNet).ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "5.9. 含并行连结的网络(GoogLeNet).ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/chongzicbo/Dive-into-Deep-Learning-tf.keras/blob/master/5.9.%20%E5%90%AB%E5%B9%B6%E8%A1%8C%E8%BF%9E%E7%BB%93%E7%9A%84%E7%BD%91%E7%BB%9C(GoogLeNet).ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cDUHRIktx3pa",
        "colab_type": "text"
      },
      "source": [
        "##5.9. 含并行连结的网络（GoogLeNet）\n",
        "在2014年的ImageNet图像识别挑战赛中，一个名叫GoogLeNet的网络结构大放异彩 [1]。它虽然在名字上向LeNet致敬，但在网络结构上已经很难看到LeNet的影子。GoogLeNet吸收了NiN中网络串联网络的思想，并在此基础上做了很大改进。在随后的几年里，研究人员对GoogLeNet进行了数次改进，本节将介绍这个模型系列的第一个版本。\n",
        "\n",
        "###5.9.1. Inception 块\n",
        "GoogLeNet中的基础卷积块叫作Inception块，得名于同名电影《盗梦空间》（Inception）。与上一节介绍的NiN块相比，这个基础块在结构上更加复杂，如图5.8所示。\n",
        "<img src=\"https://zh.gluon.ai/_images/inception.svg\" width=\"500\"/>\n",
        "\n",
        "<center>图 5.8 Inception块的结构</center>\n",
        "由图5.8可以看出，Inception块里有4条并行的线路。前3条线路使用窗口大小分别是 $1 \\times 1$ 、 $3\\times 3$ 和 $5\\times5$ 的卷积层来抽取不同空间尺寸下的信息，其中中间2个线路会对输入先做 $1 \\times 1$ 卷积来减少输入通道数，以降低模型复杂度。第四条线路则使用 $3\\times3$ 最大池化层，后接 $1 \\times 1$ 卷积层来改变通道数。4条线路都使用了合适的填充来使输入与输出的高和宽一致。最后我们将每条线路的输出在通道维上连结，并输入接下来的层中去。\n",
        "\n",
        "Inception块中可以自定义的超参数是每个层的输出通道数，我们以此来控制模型复杂度。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FtAxhse4ajDl",
        "colab_type": "code",
        "outputId": "d6fc0753-e994-40e5-e119-685873e27fef",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 62
        }
      },
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers,Sequential\n",
        "from tensorflow.data import Dataset\n",
        "from tensorflow.keras import losses,optimizers\n",
        "import time\n",
        "import numpy as np\n",
        "from tensorflow import image\n",
        "import os\n",
        "import sys"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<p style=\"color: red;\">\n",
              "The default version of TensorFlow in Colab will soon switch to TensorFlow 2.x.<br>\n",
              "We recommend you <a href=\"https://www.tensorflow.org/guide/migrate\" target=\"_blank\">upgrade</a> now \n",
              "or ensure your notebook will continue to use TensorFlow 1.x via the <code>%tensorflow_version 1.x</code> magic:\n",
              "<a href=\"https://colab.research.google.com/notebooks/tensorflow_version.ipynb\" target=\"_blank\">more info</a>.</p>\n"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "o-Zgs5p1bXCN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "tf.enable_eager_execution()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HTIITCd0beSE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class Inception(layers.Layer):\n",
        "  #c1-c4为每条线路里的层的输出通道数\n",
        "  def __init__(self,c1,c2,c3,c4):\n",
        "    super(Inception,self).__init__()\n",
        "    #线路1，单1 x 1卷积层\n",
        "    self.p1_1=layers.Convolution2D(c1,kernel_size=1,activation='relu')\n",
        "    #线路2,1 x 1卷积层后接3 x 3卷积层\n",
        "    self.p2_1=layers.Convolution2D(c2[0],kernel_size=1,activation='relu')\n",
        "    self.p2_2=layers.Convolution2D(c2[1],kernel_size=3,padding='same',activation='relu')\n",
        "\n",
        "    #线路3，1 x 1卷积层后接5 x 5 卷积层\n",
        "    self.p3_1=layers.Convolution2D(c3[0],kernel_size=1,activation='relu')\n",
        "    self.p3_2=layers.Convolution2D(c3[1],kernel_size=5,padding='same',activation='relu')\n",
        "\n",
        "    #线路4，3 x 3最大池化层后接1 x 1卷积层\n",
        "    self.p4_1=layers.MaxPool2D(pool_size=3,strides=1,padding='same')\n",
        "    self.p4_2=layers.Convolution2D(c4,kernel_size=1,activation='relu')\n",
        "\n",
        "  def call(self,x):\n",
        "    p1=self.p1_1(x)\n",
        "    p2=self.p2_2(self.p2_1(x))\n",
        "    p3=self.p3_2(self.p3_1(x))\n",
        "    p4=self.p4_2(self.p4_1(x))\n",
        "    return tf.concat((p1,p2,p3,p4),axis=-1) #在通道维上连结输出"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zdBnK5YLymys",
        "colab_type": "text"
      },
      "source": [
        "###5.9.2. GoogLeNet模型\n",
        "GoogLeNet跟VGG一样，在主体卷积部分中使用5个模块（block），每个模块之间使用步幅为2的 $3\\times3$ 最大池化层来减小输出高宽。第一模块使用一个64通道的 $7\\times7$ 卷积层。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "03ajyi2Cd2Wk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "b1=Sequential([\n",
        "  layers.Convolution2D(64,kernel_size=7,strides=2,padding='same',activation='relu'),\n",
        "  layers.MaxPool2D(pool_size=3,strides=2,padding='same')               \n",
        "])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BLAtFzm0yvNK",
        "colab_type": "text"
      },
      "source": [
        "第二模块使用2个卷积层：首先是64通道的 $1\\times1$ 卷积层，然后是将通道增大3倍的 $3\\times3$ 卷积层。它对应Inception块中的第二条线路。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jJnhpcDVePuQ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "b2=Sequential(\n",
        "    [\n",
        "     layers.Convolution2D(64,kernel_size=1,activation='relu'),\n",
        "    layers.Convolution2D(192,kernel_size=3,padding='same',activation='relu'),\n",
        "    layers.MaxPool2D(pool_size=3,strides=2,padding='same')\n",
        "    ]\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZcW6P2b5y3Ek",
        "colab_type": "text"
      },
      "source": [
        "第三模块串联2个完整的Inception块。第一个Inception块的输出通道数为 $64+128+32+32=256$ ，其中4条线路的输出通道数比例为 $64:128:32:32=2:4:1:1$ 。其中第二、第三条线路先分别将输入通道数减小至 $96/192=1/2$ 和 $16/192=1/12$ 后，再接上第二层卷积层。第二个Inception块输出通道数增至 $128+192+96+64=480$ ，每条线路的输出通道数之比为$ 128:192:96:64=4:6:3:2$ 。其中第二、第三条线路先分别将输入通道数减小至 $128/256=1/2$ 和 $32/256=1/8 $。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5XwGGoeSet66",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "b3=Sequential([\n",
        "  Inception(64,(96,128),(16,32),32),\n",
        "  Inception(128,(128,192),(32,96),64),\n",
        "  layers.MaxPool2D(pool_size=3,strides=2,padding='same')               \n",
        "])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hlkMxmmQzG4-",
        "colab_type": "text"
      },
      "source": [
        "第四模块更加复杂。它串联了5个Inception块，其输出通道数分别是 $192+208+48+64=512$ 、 $160+224+64+64=512$ 、 $128+256+64+64=512$ 、 $112+288+64+64=528$ 和 $256+320+128+128=832$ 。这些线路的通道数分配和第三模块中的类似，首先含 $3\\times3$ 卷积层的第二条线路输出最多通道，其次是仅含 1×1 卷积层的第一条线路，之后是含 $5\\times5$ 卷积层的第三条线路和含$ 3\\times3$ 最大池化层的第四条线路。其中第二、第三条线路都会先按比例减小通道数。这些比例在各个Inception块中都略有不同。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Fhry2Hj1fw-t",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "b4=Sequential([\n",
        "  Inception(192,(96,208),(16,48),64),\n",
        "  Inception(160, (112, 224), (24, 64), 64),\n",
        "  Inception(128, (128, 256), (24, 64), 64),\n",
        "  Inception(112, (144, 288), (32, 64), 64),\n",
        "  Inception(256, (160, 320), (32, 128), 128), \n",
        "  layers.MaxPool2D(pool_size=3,strides=2,padding='same')              \n",
        "])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bMPzA7mjzWS4",
        "colab_type": "text"
      },
      "source": [
        "第五模块有输出通道数为 $256+320+128+128=832$ 和 $384+384+128+128=1024$ 的两个Inception块。其中每条线路的通道数的分配思路和第三、第四模块中的一致，只是在具体数值上有所不同。需要注意的是，第五模块的后面紧跟输出层，该模块同NiN一样使用全局平均池化层来将每个通道的高和宽变成1。最后我们将输出变成二维数组后接上一个输出个数为标签类别数的全连接层。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8lfgT8O9g6kd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "b5=Sequential([\n",
        "  Inception(256, (160, 320), (32, 128), 128),\n",
        "  Inception(384, (192, 384), (48, 128), 128), \n",
        "  layers.GlobalAveragePooling2D()            \n",
        "])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gf0JiMNchIIr",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "net=Sequential([b1,b2,b3,b4,b5,layers.Dense(10)])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3tnLR30HzdG4",
        "colab_type": "text"
      },
      "source": [
        "GoogLeNet模型的计算复杂，而且不如VGG那样便于修改通道数。本节里我们将输入的高和宽从224降到96来简化计算。下面演示各个模块之间的输出的形状变化。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "r8inwisshPgD",
        "colab_type": "code",
        "outputId": "d520d741-e6c9-4227-fdbb-0e1ab7b01622",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 118
        }
      },
      "source": [
        "X=tf.random.uniform(shape=(1,96,96,1))\n",
        "for layer in net.layers:\n",
        "  X=layer(X)\n",
        "  print(layer.name,'output shape:\\t',X.shape)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "sequential output shape:\t (1, 24, 24, 64)\n",
            "sequential_1 output shape:\t (1, 12, 12, 192)\n",
            "sequential_6 output shape:\t (1, 6, 6, 480)\n",
            "sequential_7 output shape:\t (1, 3, 3, 832)\n",
            "sequential_8 output shape:\t (1, 1024)\n",
            "dense_1 output shape:\t (1, 10)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SOQHIlztzgR7",
        "colab_type": "text"
      },
      "source": [
        "###5.9.3. 获取数据和训练模型\n",
        "我们使用高和宽均为96像素的图像来训练GoogLeNet模型。训练使用的图像依然来自Fashion-MNIST数据集。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VvGaVyc4iIsE",
        "colab_type": "code",
        "outputId": "aa6c3c54-cd57-4ab0-86a9-7849b1b3e51b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 440
        }
      },
      "source": [
        "lr,num_epochs,batch_size=0.05,5,128\n",
        "optimizer=optimizers.SGD(learning_rate=lr)\n",
        "loss=losses.SparseCategoricalCrossentropy(from_logits=True)\n",
        "buffer_size=1000\n",
        "def load_data_fashion_mnist(batch_size,buffer_size):\n",
        "  (x_train,y_train),(x_test,y_test)=keras.datasets.fashion_mnist.load_data()\n",
        "  x_train=x_train[:,:,:,np.newaxis]#将三维张量增加一个channel维\n",
        "  x_test=x_test[:,:,:,np.newaxis]\n",
        "  train_iter=Dataset.from_tensor_slices((x_train,y_train)).map(lambda x,y:(x/255,y)).shuffle(buffer_size).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
        "  test_iter=Dataset.from_tensor_slices((x_test,y_test)).map(lambda x,y:(x/255,y)).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
        "  return train_iter,test_iter\n",
        "train_iter,test_iter=load_data_fashion_mnist(batch_size=batch_size,buffer_size=buffer_size)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz\n",
            "32768/29515 [=================================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz\n",
            "26427392/26421880 [==============================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz\n",
            "8192/5148 [===============================================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz\n",
            "4423680/4422102 [==============================] - 0s 0us/step\n",
            "WARNING:tensorflow:Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c722598> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c722598>. It was defined on this line: train_iter=Dataset.from_tensor_slices((x_train,y_train)).map(lambda x,y:(x/255,y)).shuffle(buffer_size).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n",
            "WARNING: Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c722598> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c722598>. It was defined on this line: train_iter=Dataset.from_tensor_slices((x_train,y_train)).map(lambda x,y:(x/255,y)).shuffle(buffer_size).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n",
            "WARNING:tensorflow:Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c722378> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c722378>. It was defined on this line: train_iter=Dataset.from_tensor_slices((x_train,y_train)).map(lambda x,y:(x/255,y)).shuffle(buffer_size).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n",
            "WARNING: Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c722378> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c722378>. It was defined on this line: train_iter=Dataset.from_tensor_slices((x_train,y_train)).map(lambda x,y:(x/255,y)).shuffle(buffer_size).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n",
            "WARNING:tensorflow:Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c722ae8> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c722ae8>. It was defined on this line: test_iter=Dataset.from_tensor_slices((x_test,y_test)).map(lambda x,y:(x/255,y)).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n",
            "WARNING: Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c722ae8> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c722ae8>. It was defined on this line: test_iter=Dataset.from_tensor_slices((x_test,y_test)).map(lambda x,y:(x/255,y)).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n",
            "WARNING:tensorflow:Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c25a0d0> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c25a0d0>. It was defined on this line: test_iter=Dataset.from_tensor_slices((x_test,y_test)).map(lambda x,y:(x/255,y)).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n",
            "WARNING: Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c25a0d0> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7f791c25a0d0>. It was defined on this line: test_iter=Dataset.from_tensor_slices((x_test,y_test)).map(lambda x,y:(x/255,y)).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZoCRttnuktdG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "net.compile(optimizer=optimizer,loss=loss,metrics=['accuracy'])\n",
        "history=net.fit_generator(train_iter,validation_data=test_iter,epochs=num_epochs)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cKNI_qk-kvw8",
        "colab_type": "code",
        "outputId": "d0f438e3-422e-49f2-d5e5-1a598b880830",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 353
        }
      },
      "source": [
        "net.summary()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_9\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "sequential (Sequential)      multiple                  3200      \n",
            "_________________________________________________________________\n",
            "sequential_1 (Sequential)    multiple                  114944    \n",
            "_________________________________________________________________\n",
            "sequential_6 (Sequential)    multiple                  552432    \n",
            "_________________________________________________________________\n",
            "sequential_7 (Sequential)    multiple                  2809168   \n",
            "_________________________________________________________________\n",
            "sequential_8 (Sequential)    multiple                  2487536   \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              multiple                  10250     \n",
            "=================================================================\n",
            "Total params: 5,977,530\n",
            "Trainable params: 5,977,530\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5RH91Az9lAVI",
        "colab_type": "text"
      },
      "source": [
        "###5.9.4. 小结\n",
        "* Inception块相当于一个有4条线路的子网络。它通过不同窗口形状的卷积层和最大池化层来并行抽取信息，并使用 1×1 卷积层减少通道数从而降低模型复杂度。\n",
        "* GoogLeNet将多个设计精细的Inception块和其他层串联起来。其中Inception块的通道数分配之比是在ImageNet数据集上通过大量的实验得来的。\n",
        "* GoogLeNet和它的后继者们一度是ImageNet上最高效的模型之一：在类似的测试精度下，它们的计算复杂度往往更低。"
      ]
    }
  ]
}