7.1. 优化与深度学习.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "7.1. 优化与深度学习.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/chongzicbo/Dive-into-Deep-Learning-tf.keras/blob/master/7.1.%20%E4%BC%98%E5%8C%96%E4%B8%8E%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y21nwiBZUkZm",
        "colab_type": "text"
      },
      "source": [
        "##7.1. 优化与深度学习\n",
        "本节将讨论优化与深度学习的关系，以及优化在深度学习中的挑战。在一个深度学习问题中，我们通常会预先定义一个损失函数。有了损失函数以后，我们就可以使用优化算法试图将其最小化。在优化中，这样的损失函数通常被称作优化问题的目标函数（objective function）。依据惯例，优化算法通常只考虑最小化目标函数。其实，任何最大化问题都可以很容易地转化为最小化问题，只需令目标函数的相反数为新的目标函数即可。\n",
        "\n",
        "###7.1.1. 优化与深度学习的关系\n",
        "虽然优化为深度学习提供了最小化损失函数的方法，但本质上，优化与深度学习的目标是有区别的。 在“模型选择、欠拟合和过拟合”一节中，我们区分了训练误差和泛化误差。 由于优化算法的目标函数通常是一个基于训练数据集的损失函数，优化的目标在于降低训练误差。 而深度学习的目标在于降低泛化误差。为了降低泛化误差，除了使用优化算法降低训练误差以外，还需要注意应对过拟合。\n",
        "\n",
        "本章中，我们只关注优化算法在最小化目标函数上的表现，而不关注模型的泛化误差。\n",
        "\n",
        "###7.1.2. 优化在深度学习中的挑战\n",
        "我们在“线性回归”一节中对优化问题的解析解和数值解做了区分。深度学习中绝大多数目标函数都很复杂。因此，很多优化问题并不存在解析解，而需要使用基于数值方法的优化算法找到近似解，即数值解。本书中讨论的优化算法都是这类基于数值方法的算法。为了求得最小化目标函数的数值解，我们将通过优化算法有限次迭代模型参数来尽可能降低损失函数的值。\n",
        "\n",
        "优化在深度学习中有很多挑战。下面描述了其中的两个挑战，即局部最小值和鞍点。为了更好地描述问题，我们先导入本节中实验需要的包或模块。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FtTULgqip-H2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%matplotlib inline\n",
        "from mpl_toolkits import mplot3d\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "from IPython import display"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BhvbGUbfUwFt",
        "colab_type": "text"
      },
      "source": [
        "####7.1.2.1. 局部最小值\n",
        "对于目标函数 $f(x)$ ，如果 $f(x)$ 在 $x$ 上的值比在 $x$ 邻近的其他点的值更小，那么 $f(x)$ 可能是一个局部最小值（local minimum）。如果 $f(x)$ 在 $x$ 上的值是目标函数在整个定义域上的最小值，那么 $f(x)$ 是全局最小值（global minimum）。\n",
        "\n",
        "举个例子，给定函数\n",
        "$$\n",
        "f(x) = x \\cdot \\text{cos}(\\pi x), \\qquad -1.0 \\leq x \\leq 2.0,\n",
        "$$\n",
        "\n",
        "我们可以大致找出该函数的局部最小值和全局最小值的位置。需要注意的是，图中箭头所指示的只是大致位置。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d8r0VkSC9FJ1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def use_svg_display():\n",
        "  #用矢量图显示\n",
        "  display.set_matplotlib_formats('svg')\n",
        "def set_figsize(figsize=(3.5,2.5)):\n",
        "  use_svg_display()\n",
        "  #设置图的尺寸\n",
        "  plt.rcParams['figure.figsize']=figsize"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6NBJeod7srXL",
        "colab_type": "code",
        "outputId": "3ed13a87-6b99-44e3-b75c-7ece203b7394",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 261
        }
      },
      "source": [
        "def f(x):\n",
        "  return x*np.cos(np.pi*x)\n",
        "set_figsize((4.5,2.5))\n",
        "x=np.arange(-1.0,2.0,0.1)\n",
        "fig,=plt.plot(x,f(x))\n",
        "fig.axes.annotate('local minimum',xy=(-0.3,-0.25),xytext=(-0.77,-1.0),arrowprops=dict(arrowstyle='->'))\n",
        "fig.axes.annotate('global minimum',xy=(1.1,-0.95),xytext=(0.6,0.8),arrowprops=dict(arrowstyle='->'))\n",
        "plt.xlabel('x')\n",
        "plt.ylabel('f(x)');\n",
        "# plt.show() "
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 324x180 with 1 Axes>"
            ],
            "image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<!-- Created with matplotlib (https://matplotlib.org/) -->\n<svg height=\"180.65625pt\" version=\"1.1\" viewBox=\"0 0 305.329712 180.65625\" width=\"305.329712pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n <defs>\n  <style type=\"text/css\">\n*{stroke-linecap:butt;stroke-linejoin:round;}\n  </style>\n </defs>\n <g id=\"figure_1\">\n  <g id=\"patch_1\">\n   <path d=\"M 0 180.65625 \nL 305.329712 180.65625 \nL 305.329712 0 \nL 0 0 \nz\n\" style=\"fill:none;\"/>\n  </g>\n  <g id=\"axes_1\">\n   <g id=\"patch_2\">\n    <path d=\"M 42.620313 143.1 \nL 293.720313 143.1 \nL 293.720313 7.2 \nL 42.620313 7.2 \nz\n\" style=\"fill:#ffffff;\"/>\n   </g>\n   <g id=\"matplotlib.axis_1\">\n    <g id=\"xtick_1\">\n     <g id=\"line2d_1\">\n      <defs>\n       <path d=\"M 0 0 \nL 0 3.5 \n\" id=\"mbd0f00d667\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n      </defs>\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54.033949\" xlink:href=\"#mbd0f00d667\" y=\"143.1\"/>\n      </g>\n     </g>\n     <g id=\"text_1\">\n      <!-- −1.0 -->\n      <defs>\n       <path d=\"M 10.59375 35.5 \nL 73.1875 35.5 \nL 73.1875 27.203125 \nL 10.59375 27.203125 \nz\n\" id=\"DejaVuSans-8722\"/>\n       <path d=\"M 12.40625 8.296875 \nL 28.515625 8.296875 \nL 28.515625 63.921875 \nL 10.984375 60.40625 \nL 10.984375 69.390625 \nL 28.421875 72.90625 \nL 38.28125 72.90625 \nL 38.28125 8.296875 \nL 54.390625 8.296875 \nL 54.390625 0 \nL 12.40625 0 \nz\n\" id=\"DejaVuSans-49\"/>\n       <path d=\"M 10.6875 12.40625 \nL 21 12.40625 \nL 21 0 \nL 10.6875 0 \nz\n\" id=\"DejaVuSans-46\"/>\n       <path d=\"M 31.78125 66.40625 \nQ 24.171875 66.40625 20.328125 58.90625 \nQ 16.5 51.421875 16.5 36.375 \nQ 16.5 21.390625 20.328125 13.890625 \nQ 24.171875 6.390625 31.78125 6.390625 \nQ 39.453125 6.390625 43.28125 13.890625 \nQ 47.125 21.390625 47.125 36.375 \nQ 47.125 51.421875 43.28125 58.90625 \nQ 39.453125 66.40625 31.78125 66.40625 \nz\nM 31.78125 74.21875 \nQ 44.046875 74.21875 50.515625 64.515625 \nQ 56.984375 54.828125 56.984375 36.375 \nQ 56.984375 17.96875 50.515625 8.265625 \nQ 44.046875 -1.421875 31.78125 -1.421875 \nQ 19.53125 -1.421875 13.0625 8.265625 \nQ 6.59375 17.96875 6.59375 36.375 \nQ 6.59375 54.828125 13.0625 64.515625 \nQ 19.53125 74.21875 31.78125 74.21875 \nz\n\" id=\"DejaVuSans-48\"/>\n      </defs>\n      <g transform=\"translate(41.892543 157.698438)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-8722\"/>\n       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\n       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\n       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-48\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_2\">\n     <g id=\"line2d_2\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"93.391316\" xlink:href=\"#mbd0f00d667\" y=\"143.1\"/>\n      </g>\n     </g>\n     <g id=\"text_2\">\n      <!-- −0.5 -->\n      <defs>\n       <path d=\"M 10.796875 72.90625 \nL 49.515625 72.90625 \nL 49.515625 64.59375 \nL 19.828125 64.59375 \nL 19.828125 46.734375 \nQ 21.96875 47.46875 24.109375 47.828125 \nQ 26.265625 48.1875 28.421875 48.1875 \nQ 40.625 48.1875 47.75 41.5 \nQ 54.890625 34.8125 54.890625 23.390625 \nQ 54.890625 11.625 47.5625 5.09375 \nQ 40.234375 -1.421875 26.90625 -1.421875 \nQ 22.3125 -1.421875 17.546875 -0.640625 \nQ 12.796875 0.140625 7.71875 1.703125 \nL 7.71875 11.625 \nQ 12.109375 9.234375 16.796875 8.0625 \nQ 21.484375 6.890625 26.703125 6.890625 \nQ 35.15625 6.890625 40.078125 11.328125 \nQ 45.015625 15.765625 45.015625 23.390625 \nQ 45.015625 31 40.078125 35.4375 \nQ 35.15625 39.890625 26.703125 39.890625 \nQ 22.75 39.890625 18.8125 39.015625 \nQ 14.890625 38.140625 10.796875 36.28125 \nz\n\" id=\"DejaVuSans-53\"/>\n      </defs>\n      <g transform=\"translate(81.249909 157.698438)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-8722\"/>\n       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-48\"/>\n       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\n       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-53\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_3\">\n     <g id=\"line2d_3\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"132.748682\" xlink:href=\"#mbd0f00d667\" y=\"143.1\"/>\n      </g>\n     </g>\n     <g id=\"text_3\">\n      <!-- 0.0 -->\n      <g transform=\"translate(124.79712 157.698438)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-48\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_4\">\n     <g id=\"line2d_4\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"172.106049\" xlink:href=\"#mbd0f00d667\" y=\"143.1\"/>\n      </g>\n     </g>\n     <g id=\"text_4\">\n      <!-- 0.5 -->\n      <g transform=\"translate(164.154487 157.698438)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-48\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_5\">\n     <g id=\"line2d_5\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"211.463416\" xlink:href=\"#mbd0f00d667\" y=\"143.1\"/>\n      </g>\n     </g>\n     <g id=\"text_5\">\n      <!-- 1.0 -->\n      <g transform=\"translate(203.511853 157.698438)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-49\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_6\">\n     <g id=\"line2d_6\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"250.820783\" xlink:href=\"#mbd0f00d667\" y=\"143.1\"/>\n      </g>\n     </g>\n     <g id=\"text_6\">\n      <!-- 1.5 -->\n      <g transform=\"translate(242.86922 157.698438)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-49\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_7\">\n     <g id=\"line2d_7\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"290.178149\" xlink:href=\"#mbd0f00d667\" y=\"143.1\"/>\n      </g>\n     </g>\n     <g id=\"text_7\">\n      <!-- 2.0 -->\n      <defs>\n       <path d=\"M 19.1875 8.296875 \nL 53.609375 8.296875 \nL 53.609375 0 \nL 7.328125 0 \nL 7.328125 8.296875 \nQ 12.9375 14.109375 22.625 23.890625 \nQ 32.328125 33.6875 34.8125 36.53125 \nQ 39.546875 41.84375 41.421875 45.53125 \nQ 43.3125 49.21875 43.3125 52.78125 \nQ 43.3125 58.59375 39.234375 62.25 \nQ 35.15625 65.921875 28.609375 65.921875 \nQ 23.96875 65.921875 18.8125 64.3125 \nQ 13.671875 62.703125 7.8125 59.421875 \nL 7.8125 69.390625 \nQ 13.765625 71.78125 18.9375 73 \nQ 24.125 74.21875 28.421875 74.21875 \nQ 39.75 74.21875 46.484375 68.546875 \nQ 53.21875 62.890625 53.21875 53.421875 \nQ 53.21875 48.921875 51.53125 44.890625 \nQ 49.859375 40.875 45.40625 35.40625 \nQ 44.1875 33.984375 37.640625 27.21875 \nQ 31.109375 20.453125 19.1875 8.296875 \nz\n\" id=\"DejaVuSans-50\"/>\n      </defs>\n      <g transform=\"translate(282.226587 157.698438)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-50\"/>\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"text_8\">\n     <!-- x -->\n     <defs>\n      <path d=\"M 54.890625 54.6875 \nL 35.109375 28.078125 \nL 55.90625 0 \nL 45.3125 0 \nL 29.390625 21.484375 \nL 13.484375 0 \nL 2.875 0 \nL 24.125 28.609375 \nL 4.6875 54.6875 \nL 15.28125 54.6875 \nL 29.78125 35.203125 \nL 44.28125 54.6875 \nz\n\" id=\"DejaVuSans-120\"/>\n     </defs>\n     <g transform=\"translate(165.210938 171.376563)scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-120\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_2\">\n    <g id=\"ytick_1\">\n     <g id=\"line2d_8\">\n      <defs>\n       <path d=\"M 0 0 \nL -3.5 0 \n\" id=\"m4c1f71dccc\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n      </defs>\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#m4c1f71dccc\" y=\"134.923853\"/>\n      </g>\n     </g>\n     <g id=\"text_9\">\n      <!-- −1 -->\n      <g transform=\"translate(20.878125 138.723072)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-8722\"/>\n       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_2\">\n     <g id=\"line2d_9\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#m4c1f71dccc\" y=\"91.622727\"/>\n      </g>\n     </g>\n     <g id=\"text_10\">\n      <!-- 0 -->\n      <g transform=\"translate(29.257813 95.421946)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-48\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_3\">\n     <g id=\"line2d_10\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#m4c1f71dccc\" y=\"48.321601\"/>\n      </g>\n     </g>\n     <g id=\"text_11\">\n      <!-- 1 -->\n      <g transform=\"translate(29.257813 52.12082)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-49\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"text_12\">\n     <!-- f(x) -->\n     <defs>\n      <path d=\"M 37.109375 75.984375 \nL 37.109375 68.5 \nL 28.515625 68.5 \nQ 23.6875 68.5 21.796875 66.546875 \nQ 19.921875 64.59375 19.921875 59.515625 \nL 19.921875 54.6875 \nL 34.71875 54.6875 \nL 34.71875 47.703125 \nL 19.921875 47.703125 \nL 19.921875 0 \nL 10.890625 0 \nL 10.890625 47.703125 \nL 2.296875 47.703125 \nL 2.296875 54.6875 \nL 10.890625 54.6875 \nL 10.890625 58.5 \nQ 10.890625 67.625 15.140625 71.796875 \nQ 19.390625 75.984375 28.609375 75.984375 \nz\n\" id=\"DejaVuSans-102\"/>\n      <path d=\"M 31 75.875 \nQ 24.46875 64.65625 21.28125 53.65625 \nQ 18.109375 42.671875 18.109375 31.390625 \nQ 18.109375 20.125 21.3125 9.0625 \nQ 24.515625 -2 31 -13.1875 \nL 23.1875 -13.1875 \nQ 15.875 -1.703125 12.234375 9.375 \nQ 8.59375 20.453125 8.59375 31.390625 \nQ 8.59375 42.28125 12.203125 53.3125 \nQ 15.828125 64.359375 23.1875 75.875 \nz\n\" id=\"DejaVuSans-40\"/>\n      <path d=\"M 8.015625 75.875 \nL 15.828125 75.875 \nQ 23.140625 64.359375 26.78125 53.3125 \nQ 30.421875 42.28125 30.421875 31.390625 \nQ 30.421875 20.453125 26.78125 9.375 \nQ 23.140625 -1.703125 15.828125 -13.1875 \nL 8.015625 -13.1875 \nQ 14.5 -2 17.703125 9.0625 \nQ 20.90625 20.125 20.90625 31.390625 \nQ 20.90625 42.671875 17.703125 53.65625 \nQ 14.5 64.65625 8.015625 75.875 \nz\n\" id=\"DejaVuSans-41\"/>\n     </defs>\n     <g transform=\"translate(14.798438 83.771094)rotate(-90)scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-102\"/>\n      <use x=\"35.205078\" xlink:href=\"#DejaVuSans-40\"/>\n      <use x=\"74.21875\" xlink:href=\"#DejaVuSans-120\"/>\n      <use x=\"133.398438\" xlink:href=\"#DejaVuSans-41\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"line2d_11\">\n    <path clip-path=\"url(#p29eeede395)\" d=\"M 54.033949 48.321601 \nL 61.905422 54.559091 \nL 69.776896 63.59765 \nL 77.648369 73.806493 \nL 85.519842 83.594257 \nL 93.391316 91.622727 \nL 101.262789 96.975041 \nL 109.134262 99.258256 \nL 117.005736 98.628997 \nL 124.877209 95.740909 \nL 132.748682 91.622727 \nL 140.620156 87.504545 \nL 148.491629 84.616458 \nL 156.363102 83.987198 \nL 164.234576 86.270414 \nL 172.106049 91.622727 \nL 179.977523 99.651198 \nL 187.848996 109.438962 \nL 195.720469 119.647805 \nL 203.591943 128.686364 \nL 211.463416 134.923853 \nL 219.334889 136.922727 \nL 227.206363 133.660344 \nL 235.077836 124.71002 \nL 242.949309 110.355825 \nL 250.820783 91.622727 \nL 258.692256 70.213473 \nL 266.563729 48.35473 \nL 274.435203 28.566303 \nL 282.306676 13.377273 \n\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n   </g>\n   <g id=\"patch_3\">\n    <path d=\"M 42.620313 143.1 \nL 42.620313 7.2 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"patch_4\">\n    <path d=\"M 293.720313 143.1 \nL 293.720313 7.2 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"patch_5\">\n    <path d=\"M 42.620313 143.1 \nL 293.720313 143.1 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"patch_6\">\n    <path d=\"M 42.620313 7.2 \nL 293.720313 7.2 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"patch_7\">\n    <path d=\"M 109.113525 123.324204 \nQ 109.1229 113.886339 109.131165 105.566508 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\n    <path d=\"M 107.127192 109.564519 \nL 109.131165 105.566508 \nL 111.12719 109.568492 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\n   </g>\n   <g id=\"text_13\">\n    <!-- local minimum -->\n    <defs>\n     <path d=\"M 9.421875 75.984375 \nL 18.40625 75.984375 \nL 18.40625 0 \nL 9.421875 0 \nz\n\" id=\"DejaVuSans-108\"/>\n     <path d=\"M 30.609375 48.390625 \nQ 23.390625 48.390625 19.1875 42.75 \nQ 14.984375 37.109375 14.984375 27.296875 \nQ 14.984375 17.484375 19.15625 11.84375 \nQ 23.34375 6.203125 30.609375 6.203125 \nQ 37.796875 6.203125 41.984375 11.859375 \nQ 46.1875 17.53125 46.1875 27.296875 \nQ 46.1875 37.015625 41.984375 42.703125 \nQ 37.796875 48.390625 30.609375 48.390625 \nz\nM 30.609375 56 \nQ 42.328125 56 49.015625 48.375 \nQ 55.71875 40.765625 55.71875 27.296875 \nQ 55.71875 13.875 49.015625 6.21875 \nQ 42.328125 -1.421875 30.609375 -1.421875 \nQ 18.84375 -1.421875 12.171875 6.21875 \nQ 5.515625 13.875 5.515625 27.296875 \nQ 5.515625 40.765625 12.171875 48.375 \nQ 18.84375 56 30.609375 56 \nz\n\" id=\"DejaVuSans-111\"/>\n     <path d=\"M 48.78125 52.59375 \nL 48.78125 44.1875 \nQ 44.96875 46.296875 41.140625 47.34375 \nQ 37.3125 48.390625 33.40625 48.390625 \nQ 24.65625 48.390625 19.8125 42.84375 \nQ 14.984375 37.3125 14.984375 27.296875 \nQ 14.984375 17.28125 19.8125 11.734375 \nQ 24.65625 6.203125 33.40625 6.203125 \nQ 37.3125 6.203125 41.140625 7.25 \nQ 44.96875 8.296875 48.78125 10.40625 \nL 48.78125 2.09375 \nQ 45.015625 0.34375 40.984375 -0.53125 \nQ 36.96875 -1.421875 32.421875 -1.421875 \nQ 20.0625 -1.421875 12.78125 6.34375 \nQ 5.515625 14.109375 5.515625 27.296875 \nQ 5.515625 40.671875 12.859375 48.328125 \nQ 20.21875 56 33.015625 56 \nQ 37.15625 56 41.109375 55.140625 \nQ 45.0625 54.296875 48.78125 52.59375 \nz\n\" id=\"DejaVuSans-99\"/>\n     <path d=\"M 34.28125 27.484375 \nQ 23.390625 27.484375 19.1875 25 \nQ 14.984375 22.515625 14.984375 16.5 \nQ 14.984375 11.71875 18.140625 8.90625 \nQ 21.296875 6.109375 26.703125 6.109375 \nQ 34.1875 6.109375 38.703125 11.40625 \nQ 43.21875 16.703125 43.21875 25.484375 \nL 43.21875 27.484375 \nz\nM 52.203125 31.203125 \nL 52.203125 0 \nL 43.21875 0 \nL 43.21875 8.296875 \nQ 40.140625 3.328125 35.546875 0.953125 \nQ 30.953125 -1.421875 24.3125 -1.421875 \nQ 15.921875 -1.421875 10.953125 3.296875 \nQ 6 8.015625 6 15.921875 \nQ 6 25.140625 12.171875 29.828125 \nQ 18.359375 34.515625 30.609375 34.515625 \nL 43.21875 34.515625 \nL 43.21875 35.40625 \nQ 43.21875 41.609375 39.140625 45 \nQ 35.0625 48.390625 27.6875 48.390625 \nQ 23 48.390625 18.546875 47.265625 \nQ 14.109375 46.140625 10.015625 43.890625 \nL 10.015625 52.203125 \nQ 14.9375 54.109375 19.578125 55.046875 \nQ 24.21875 56 28.609375 56 \nQ 40.484375 56 46.34375 49.84375 \nQ 52.203125 43.703125 52.203125 31.203125 \nz\n\" id=\"DejaVuSans-97\"/>\n     <path id=\"DejaVuSans-32\"/>\n     <path d=\"M 52 44.1875 \nQ 55.375 50.25 60.0625 53.125 \nQ 64.75 56 71.09375 56 \nQ 79.640625 56 84.28125 50.015625 \nQ 88.921875 44.046875 88.921875 33.015625 \nL 88.921875 0 \nL 79.890625 0 \nL 79.890625 32.71875 \nQ 79.890625 40.578125 77.09375 44.375 \nQ 74.3125 48.1875 68.609375 48.1875 \nQ 61.625 48.1875 57.5625 43.546875 \nQ 53.515625 38.921875 53.515625 30.90625 \nL 53.515625 0 \nL 44.484375 0 \nL 44.484375 32.71875 \nQ 44.484375 40.625 41.703125 44.40625 \nQ 38.921875 48.1875 33.109375 48.1875 \nQ 26.21875 48.1875 22.15625 43.53125 \nQ 18.109375 38.875 18.109375 30.90625 \nL 18.109375 0 \nL 9.078125 0 \nL 9.078125 54.6875 \nL 18.109375 54.6875 \nL 18.109375 46.1875 \nQ 21.1875 51.21875 25.484375 53.609375 \nQ 29.78125 56 35.6875 56 \nQ 41.65625 56 45.828125 52.96875 \nQ 50 49.953125 52 44.1875 \nz\n\" id=\"DejaVuSans-109\"/>\n     <path d=\"M 9.421875 54.6875 \nL 18.40625 54.6875 \nL 18.40625 0 \nL 9.421875 0 \nz\nM 9.421875 75.984375 \nL 18.40625 75.984375 \nL 18.40625 64.59375 \nL 9.421875 64.59375 \nz\n\" id=\"DejaVuSans-105\"/>\n     <path d=\"M 54.890625 33.015625 \nL 54.890625 0 \nL 45.90625 0 \nL 45.90625 32.71875 \nQ 45.90625 40.484375 42.875 44.328125 \nQ 39.84375 48.1875 33.796875 48.1875 \nQ 26.515625 48.1875 22.3125 43.546875 \nQ 18.109375 38.921875 18.109375 30.90625 \nL 18.109375 0 \nL 9.078125 0 \nL 9.078125 54.6875 \nL 18.109375 54.6875 \nL 18.109375 46.1875 \nQ 21.34375 51.125 25.703125 53.5625 \nQ 30.078125 56 35.796875 56 \nQ 45.21875 56 50.046875 50.171875 \nQ 54.890625 44.34375 54.890625 33.015625 \nz\n\" id=\"DejaVuSans-110\"/>\n     <path d=\"M 8.5 21.578125 \nL 8.5 54.6875 \nL 17.484375 54.6875 \nL 17.484375 21.921875 \nQ 17.484375 14.15625 20.5 10.265625 \nQ 23.53125 6.390625 29.59375 6.390625 \nQ 36.859375 6.390625 41.078125 11.03125 \nQ 45.3125 15.671875 45.3125 23.6875 \nL 45.3125 54.6875 \nL 54.296875 54.6875 \nL 54.296875 0 \nL 45.3125 0 \nL 45.3125 8.40625 \nQ 42.046875 3.421875 37.71875 1 \nQ 33.40625 -1.421875 27.6875 -1.421875 \nQ 18.265625 -1.421875 13.375 4.4375 \nQ 8.5 10.296875 8.5 21.578125 \nz\nM 31.109375 56 \nz\n\" id=\"DejaVuSans-117\"/>\n    </defs>\n    <g transform=\"translate(72.138338 134.923853)scale(0.1 -0.1)\">\n     <use xlink:href=\"#DejaVuSans-108\"/>\n     <use x=\"27.783203\" xlink:href=\"#DejaVuSans-111\"/>\n     <use x=\"88.964844\" xlink:href=\"#DejaVuSans-99\"/>\n     <use x=\"143.945312\" xlink:href=\"#DejaVuSans-97\"/>\n     <use x=\"205.224609\" xlink:href=\"#DejaVuSans-108\"/>\n     <use x=\"233.007812\" xlink:href=\"#DejaVuSans-32\"/>\n     <use x=\"264.794922\" xlink:href=\"#DejaVuSans-109\"/>\n     <use x=\"362.207031\" xlink:href=\"#DejaVuSans-105\"/>\n     <use x=\"389.990234\" xlink:href=\"#DejaVuSans-110\"/>\n     <use x=\"453.369141\" xlink:href=\"#DejaVuSans-105\"/>\n     <use x=\"481.152344\" xlink:href=\"#DejaVuSans-109\"/>\n     <use x=\"578.564453\" xlink:href=\"#DejaVuSans-117\"/>\n     <use x=\"641.943359\" xlink:href=\"#DejaVuSans-109\"/>\n    </g>\n   </g>\n   <g id=\"patch_8\">\n    <path d=\"M 220.407145 63.062567 \nQ 219.88643 96.908884 219.382913 129.637299 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\n    <path d=\"M 221.444207 125.668538 \nL 219.382913 129.637299 \nL 217.444681 125.607006 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\n   </g>\n   <g id=\"text_14\">\n    <!-- global minimum -->\n    <defs>\n     <path d=\"M 45.40625 27.984375 \nQ 45.40625 37.75 41.375 43.109375 \nQ 37.359375 48.484375 30.078125 48.484375 \nQ 22.859375 48.484375 18.828125 43.109375 \nQ 14.796875 37.75 14.796875 27.984375 \nQ 14.796875 18.265625 18.828125 12.890625 \nQ 22.859375 7.515625 30.078125 7.515625 \nQ 37.359375 7.515625 41.375 12.890625 \nQ 45.40625 18.265625 45.40625 27.984375 \nz\nM 54.390625 6.78125 \nQ 54.390625 -7.171875 48.1875 -13.984375 \nQ 42 -20.796875 29.203125 -20.796875 \nQ 24.46875 -20.796875 20.265625 -20.09375 \nQ 16.0625 -19.390625 12.109375 -17.921875 \nL 12.109375 -9.1875 \nQ 16.0625 -11.328125 19.921875 -12.34375 \nQ 23.78125 -13.375 27.78125 -13.375 \nQ 36.625 -13.375 41.015625 -8.765625 \nQ 45.40625 -4.15625 45.40625 5.171875 \nL 45.40625 9.625 \nQ 42.625 4.78125 38.28125 2.390625 \nQ 33.9375 0 27.875 0 \nQ 17.828125 0 11.671875 7.65625 \nQ 5.515625 15.328125 5.515625 27.984375 \nQ 5.515625 40.671875 11.671875 48.328125 \nQ 17.828125 56 27.875 56 \nQ 33.9375 56 38.28125 53.609375 \nQ 42.625 51.21875 45.40625 46.390625 \nL 45.40625 54.6875 \nL 54.390625 54.6875 \nz\n\" id=\"DejaVuSans-103\"/>\n     <path d=\"M 48.6875 27.296875 \nQ 48.6875 37.203125 44.609375 42.84375 \nQ 40.53125 48.484375 33.40625 48.484375 \nQ 26.265625 48.484375 22.1875 42.84375 \nQ 18.109375 37.203125 18.109375 27.296875 \nQ 18.109375 17.390625 22.1875 11.75 \nQ 26.265625 6.109375 33.40625 6.109375 \nQ 40.53125 6.109375 44.609375 11.75 \nQ 48.6875 17.390625 48.6875 27.296875 \nz\nM 18.109375 46.390625 \nQ 20.953125 51.265625 25.265625 53.625 \nQ 29.59375 56 35.59375 56 \nQ 45.5625 56 51.78125 48.09375 \nQ 58.015625 40.1875 58.015625 27.296875 \nQ 58.015625 14.40625 51.78125 6.484375 \nQ 45.5625 -1.421875 35.59375 -1.421875 \nQ 29.59375 -1.421875 25.265625 0.953125 \nQ 20.953125 3.328125 18.109375 8.203125 \nL 18.109375 0 \nL 9.078125 0 \nL 9.078125 75.984375 \nL 18.109375 75.984375 \nz\n\" id=\"DejaVuSans-98\"/>\n    </defs>\n    <g transform=\"translate(179.977523 56.981826)scale(0.1 -0.1)\">\n     <use xlink:href=\"#DejaVuSans-103\"/>\n     <use x=\"63.476562\" xlink:href=\"#DejaVuSans-108\"/>\n     <use x=\"91.259766\" xlink:href=\"#DejaVuSans-111\"/>\n     <use x=\"152.441406\" xlink:href=\"#DejaVuSans-98\"/>\n     <use x=\"215.917969\" xlink:href=\"#DejaVuSans-97\"/>\n     <use x=\"277.197266\" xlink:href=\"#DejaVuSans-108\"/>\n     <use x=\"304.980469\" xlink:href=\"#DejaVuSans-32\"/>\n     <use x=\"336.767578\" xlink:href=\"#DejaVuSans-109\"/>\n     <use x=\"434.179688\" xlink:href=\"#DejaVuSans-105\"/>\n     <use x=\"461.962891\" xlink:href=\"#DejaVuSans-110\"/>\n     <use x=\"525.341797\" xlink:href=\"#DejaVuSans-105\"/>\n     <use x=\"553.125\" xlink:href=\"#DejaVuSans-109\"/>\n     <use x=\"650.537109\" xlink:href=\"#DejaVuSans-117\"/>\n     <use x=\"713.916016\" xlink:href=\"#DejaVuSans-109\"/>\n    </g>\n   </g>\n  </g>\n </g>\n <defs>\n  <clipPath id=\"p29eeede395\">\n   <rect height=\"135.9\" width=\"251.1\" x=\"42.620313\" y=\"7.2\"/>\n  </clipPath>\n </defs>\n</svg>\n"
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "baZ1DovgVDgE",
        "colab_type": "text"
      },
      "source": [
        "深度学习模型的目标函数可能有若干局部最优值。当一个优化问题的数值解在局部最优解附近时，由于目标函数有关解的梯度接近或变成零，最终迭代求得的数值解可能只令目标函数局部最小化而非全局最小化。\n",
        "\n",
        "####7.1.2.2. 鞍点\n",
        "刚刚我们提到，梯度接近或变成零可能是由于当前解在局部最优解附近造成的。事实上，另一种可能性是当前解在鞍点（saddle point）附近。\n",
        "\n",
        "举个例子，给定函数\n",
        "$$\n",
        "f(x) = x^3,\n",
        "$$\n",
        "\n",
        "我们可以找出该函数的鞍点位置。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KlaThZhj-XDe",
        "colab_type": "code",
        "outputId": "b5d141c9-cf46-4a57-d229-f8789fa1dc38",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 261
        }
      },
      "source": [
        "x=np.arange(-2.0,2.0,0.1)\n",
        "fig,=plt.plot(x,x**3)\n",
        "fig.axes.annotate('saddle point',xy=(0,-0.2),xytext=(-0.52,-5.0),arrowprops=dict(arrowstyle='->'))\n",
        "plt.xlabel('x')\n",
        "plt.ylabel('f(x)')\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 324x180 with 1 Axes>"
            ],
            "image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<!-- Created with matplotlib (https://matplotlib.org/) -->\n<svg height=\"180.65625pt\" version=\"1.1\" viewBox=\"0 0 300.920313 180.65625\" width=\"300.920313pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n <defs>\n  <style type=\"text/css\">\n*{stroke-linecap:butt;stroke-linejoin:round;}\n  </style>\n </defs>\n <g id=\"figure_1\">\n  <g id=\"patch_1\">\n   <path d=\"M 0 180.65625 \nL 300.920313 180.65625 \nL 300.920313 0 \nL 0 0 \nz\n\" style=\"fill:none;\"/>\n  </g>\n  <g id=\"axes_1\">\n   <g id=\"patch_2\">\n    <path d=\"M 42.620313 143.1 \nL 293.720313 143.1 \nL 293.720313 7.2 \nL 42.620313 7.2 \nz\n\" style=\"fill:#ffffff;\"/>\n   </g>\n   <g id=\"matplotlib.axis_1\">\n    <g id=\"xtick_1\">\n     <g id=\"line2d_1\">\n      <defs>\n       <path d=\"M 0 0 \nL 0 3.5 \n\" id=\"m75b22b907c\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n      </defs>\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54.033949\" xlink:href=\"#m75b22b907c\" y=\"143.1\"/>\n      </g>\n     </g>\n     <g id=\"text_1\">\n      <!-- −2 -->\n      <defs>\n       <path d=\"M 10.59375 35.5 \nL 73.1875 35.5 \nL 73.1875 27.203125 \nL 10.59375 27.203125 \nz\n\" id=\"DejaVuSans-8722\"/>\n       <path d=\"M 19.1875 8.296875 \nL 53.609375 8.296875 \nL 53.609375 0 \nL 7.328125 0 \nL 7.328125 8.296875 \nQ 12.9375 14.109375 22.625 23.890625 \nQ 32.328125 33.6875 34.8125 36.53125 \nQ 39.546875 41.84375 41.421875 45.53125 \nQ 43.3125 49.21875 43.3125 52.78125 \nQ 43.3125 58.59375 39.234375 62.25 \nQ 35.15625 65.921875 28.609375 65.921875 \nQ 23.96875 65.921875 18.8125 64.3125 \nQ 13.671875 62.703125 7.8125 59.421875 \nL 7.8125 69.390625 \nQ 13.765625 71.78125 18.9375 73 \nQ 24.125 74.21875 28.421875 74.21875 \nQ 39.75 74.21875 46.484375 68.546875 \nQ 53.21875 62.890625 53.21875 53.421875 \nQ 53.21875 48.921875 51.53125 44.890625 \nQ 49.859375 40.875 45.40625 35.40625 \nQ 44.1875 33.984375 37.640625 27.21875 \nQ 31.109375 20.453125 19.1875 8.296875 \nz\n\" id=\"DejaVuSans-50\"/>\n      </defs>\n      <g transform=\"translate(46.662855 157.698438)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-8722\"/>\n       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-50\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_2\">\n     <g id=\"line2d_2\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"112.565417\" xlink:href=\"#m75b22b907c\" y=\"143.1\"/>\n      </g>\n     </g>\n     <g id=\"text_2\">\n      <!-- −1 -->\n      <defs>\n       <path d=\"M 12.40625 8.296875 \nL 28.515625 8.296875 \nL 28.515625 63.921875 \nL 10.984375 60.40625 \nL 10.984375 69.390625 \nL 28.421875 72.90625 \nL 38.28125 72.90625 \nL 38.28125 8.296875 \nL 54.390625 8.296875 \nL 54.390625 0 \nL 12.40625 0 \nz\n\" id=\"DejaVuSans-49\"/>\n      </defs>\n      <g transform=\"translate(105.194324 157.698438)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-8722\"/>\n       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_3\">\n     <g id=\"line2d_3\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"171.096886\" xlink:href=\"#m75b22b907c\" y=\"143.1\"/>\n      </g>\n     </g>\n     <g id=\"text_3\">\n      <!-- 0 -->\n      <defs>\n       <path d=\"M 31.78125 66.40625 \nQ 24.171875 66.40625 20.328125 58.90625 \nQ 16.5 51.421875 16.5 36.375 \nQ 16.5 21.390625 20.328125 13.890625 \nQ 24.171875 6.390625 31.78125 6.390625 \nQ 39.453125 6.390625 43.28125 13.890625 \nQ 47.125 21.390625 47.125 36.375 \nQ 47.125 51.421875 43.28125 58.90625 \nQ 39.453125 66.40625 31.78125 66.40625 \nz\nM 31.78125 74.21875 \nQ 44.046875 74.21875 50.515625 64.515625 \nQ 56.984375 54.828125 56.984375 36.375 \nQ 56.984375 17.96875 50.515625 8.265625 \nQ 44.046875 -1.421875 31.78125 -1.421875 \nQ 19.53125 -1.421875 13.0625 8.265625 \nQ 6.59375 17.96875 6.59375 36.375 \nQ 6.59375 54.828125 13.0625 64.515625 \nQ 19.53125 74.21875 31.78125 74.21875 \nz\n\" id=\"DejaVuSans-48\"/>\n      </defs>\n      <g transform=\"translate(167.915636 157.698438)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-48\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_4\">\n     <g id=\"line2d_4\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"229.628354\" xlink:href=\"#m75b22b907c\" y=\"143.1\"/>\n      </g>\n     </g>\n     <g id=\"text_4\">\n      <!-- 1 -->\n      <g transform=\"translate(226.447104 157.698438)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-49\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_5\">\n     <g id=\"line2d_5\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"288.159823\" xlink:href=\"#m75b22b907c\" y=\"143.1\"/>\n      </g>\n     </g>\n     <g id=\"text_5\">\n      <!-- 2 -->\n      <g transform=\"translate(284.978573 157.698438)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-50\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"text_6\">\n     <!-- x -->\n     <defs>\n      <path d=\"M 54.890625 54.6875 \nL 35.109375 28.078125 \nL 55.90625 0 \nL 45.3125 0 \nL 29.390625 21.484375 \nL 13.484375 0 \nL 2.875 0 \nL 24.125 28.609375 \nL 4.6875 54.6875 \nL 15.28125 54.6875 \nL 29.78125 35.203125 \nL 44.28125 54.6875 \nz\n\" id=\"DejaVuSans-120\"/>\n     </defs>\n     <g transform=\"translate(165.210938 171.376563)scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-120\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_2\">\n    <g id=\"ytick_1\">\n     <g id=\"line2d_6\">\n      <defs>\n       <path d=\"M 0 0 \nL -3.5 0 \n\" id=\"m3cb81aa0c0\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n      </defs>\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#m3cb81aa0c0\" y=\"111.979167\"/>\n      </g>\n     </g>\n     <g id=\"text_7\">\n      <!-- −5 -->\n      <defs>\n       <path d=\"M 10.796875 72.90625 \nL 49.515625 72.90625 \nL 49.515625 64.59375 \nL 19.828125 64.59375 \nL 19.828125 46.734375 \nQ 21.96875 47.46875 24.109375 47.828125 \nQ 26.265625 48.1875 28.421875 48.1875 \nQ 40.625 48.1875 47.75 41.5 \nQ 54.890625 34.8125 54.890625 23.390625 \nQ 54.890625 11.625 47.5625 5.09375 \nQ 40.234375 -1.421875 26.90625 -1.421875 \nQ 22.3125 -1.421875 17.546875 -0.640625 \nQ 12.796875 0.140625 7.71875 1.703125 \nL 7.71875 11.625 \nQ 12.109375 9.234375 16.796875 8.0625 \nQ 21.484375 6.890625 26.703125 6.890625 \nQ 35.15625 6.890625 40.078125 11.328125 \nQ 45.015625 15.765625 45.015625 23.390625 \nQ 45.015625 31 40.078125 35.4375 \nQ 35.15625 39.890625 26.703125 39.890625 \nQ 22.75 39.890625 18.8125 39.015625 \nQ 14.890625 38.140625 10.796875 36.28125 \nz\n\" id=\"DejaVuSans-53\"/>\n      </defs>\n      <g transform=\"translate(20.878125 115.778386)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-8722\"/>\n       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-53\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_2\">\n     <g id=\"line2d_7\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#m3cb81aa0c0\" y=\"70.406566\"/>\n      </g>\n     </g>\n     <g id=\"text_8\">\n      <!-- 0 -->\n      <g transform=\"translate(29.257813 74.205785)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-48\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_3\">\n     <g id=\"line2d_8\">\n      <g>\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#m3cb81aa0c0\" y=\"28.833966\"/>\n      </g>\n     </g>\n     <g id=\"text_9\">\n      <!-- 5 -->\n      <g transform=\"translate(29.257813 32.633184)scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-53\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"text_10\">\n     <!-- f(x) -->\n     <defs>\n      <path d=\"M 37.109375 75.984375 \nL 37.109375 68.5 \nL 28.515625 68.5 \nQ 23.6875 68.5 21.796875 66.546875 \nQ 19.921875 64.59375 19.921875 59.515625 \nL 19.921875 54.6875 \nL 34.71875 54.6875 \nL 34.71875 47.703125 \nL 19.921875 47.703125 \nL 19.921875 0 \nL 10.890625 0 \nL 10.890625 47.703125 \nL 2.296875 47.703125 \nL 2.296875 54.6875 \nL 10.890625 54.6875 \nL 10.890625 58.5 \nQ 10.890625 67.625 15.140625 71.796875 \nQ 19.390625 75.984375 28.609375 75.984375 \nz\n\" id=\"DejaVuSans-102\"/>\n      <path d=\"M 31 75.875 \nQ 24.46875 64.65625 21.28125 53.65625 \nQ 18.109375 42.671875 18.109375 31.390625 \nQ 18.109375 20.125 21.3125 9.0625 \nQ 24.515625 -2 31 -13.1875 \nL 23.1875 -13.1875 \nQ 15.875 -1.703125 12.234375 9.375 \nQ 8.59375 20.453125 8.59375 31.390625 \nQ 8.59375 42.28125 12.203125 53.3125 \nQ 15.828125 64.359375 23.1875 75.875 \nz\n\" id=\"DejaVuSans-40\"/>\n      <path d=\"M 8.015625 75.875 \nL 15.828125 75.875 \nQ 23.140625 64.359375 26.78125 53.3125 \nQ 30.421875 42.28125 30.421875 31.390625 \nQ 30.421875 20.453125 26.78125 9.375 \nQ 23.140625 -1.703125 15.828125 -13.1875 \nL 8.015625 -13.1875 \nQ 14.5 -2 17.703125 9.0625 \nQ 20.90625 20.125 20.90625 31.390625 \nQ 20.90625 42.671875 17.703125 53.65625 \nQ 14.5 64.65625 8.015625 75.875 \nz\n\" id=\"DejaVuSans-41\"/>\n     </defs>\n     <g transform=\"translate(14.798438 83.771094)rotate(-90)scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-102\"/>\n      <use x=\"35.205078\" xlink:href=\"#DejaVuSans-40\"/>\n      <use x=\"74.21875\" xlink:href=\"#DejaVuSans-120\"/>\n      <use x=\"133.398438\" xlink:href=\"#DejaVuSans-41\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"line2d_9\">\n    <path clip-path=\"url(#pcc0cef18bd)\" d=\"M 54.033949 136.922727 \nL 59.887096 127.43586 \nL 65.740243 118.896848 \nL 71.593389 111.255804 \nL 77.446536 104.462841 \nL 83.299683 98.468072 \nL 89.15283 93.221609 \nL 95.005977 88.673567 \nL 100.859124 84.774057 \nL 106.712271 81.473193 \nL 112.565417 78.721086 \nL 118.418564 76.467851 \nL 124.271711 74.663601 \nL 130.124858 73.258447 \nL 135.978005 72.202503 \nL 141.831152 71.445881 \nL 147.684299 70.938696 \nL 153.537445 70.631058 \nL 159.390592 70.473082 \nL 165.243739 70.414881 \nL 171.096886 70.406566 \nL 176.950033 70.398252 \nL 182.80318 70.34005 \nL 188.656326 70.182074 \nL 194.509473 69.874437 \nL 200.36262 69.367251 \nL 206.215767 68.61063 \nL 212.068914 67.554686 \nL 217.922061 66.149532 \nL 223.775208 64.345281 \nL 229.628354 62.092046 \nL 235.481501 59.33994 \nL 241.334648 56.039075 \nL 247.187795 52.139566 \nL 253.040942 47.591523 \nL 258.894089 42.345061 \nL 264.747236 36.350292 \nL 270.600382 29.557329 \nL 276.453529 21.916285 \nL 282.306676 13.377273 \n\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n   </g>\n   <g id=\"patch_3\">\n    <path d=\"M 42.620313 143.1 \nL 42.620313 7.2 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"patch_4\">\n    <path d=\"M 293.720313 143.1 \nL 293.720313 7.2 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"patch_5\">\n    <path d=\"M 42.620313 143.1 \nL 293.720313 143.1 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"patch_6\">\n    <path d=\"M 42.620313 7.2 \nL 293.720313 7.2 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"patch_7\">\n    <path d=\"M 171.391806 100.37507 \nQ 171.254768 87.222572 171.129379 75.188047 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\n    <path d=\"M 169.171162 79.208667 \nL 171.129379 75.188047 \nL 173.170944 79.166993 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\n   </g>\n   <g id=\"text_11\">\n    <!-- saddle point -->\n    <defs>\n     <path d=\"M 44.28125 53.078125 \nL 44.28125 44.578125 \nQ 40.484375 46.53125 36.375 47.5 \nQ 32.28125 48.484375 27.875 48.484375 \nQ 21.1875 48.484375 17.84375 46.4375 \nQ 14.5 44.390625 14.5 40.28125 \nQ 14.5 37.15625 16.890625 35.375 \nQ 19.28125 33.59375 26.515625 31.984375 \nL 29.59375 31.296875 \nQ 39.15625 29.25 43.1875 25.515625 \nQ 47.21875 21.78125 47.21875 15.09375 \nQ 47.21875 7.46875 41.1875 3.015625 \nQ 35.15625 -1.421875 24.609375 -1.421875 \nQ 20.21875 -1.421875 15.453125 -0.5625 \nQ 10.6875 0.296875 5.421875 2 \nL 5.421875 11.28125 \nQ 10.40625 8.6875 15.234375 7.390625 \nQ 20.0625 6.109375 24.8125 6.109375 \nQ 31.15625 6.109375 34.5625 8.28125 \nQ 37.984375 10.453125 37.984375 14.40625 \nQ 37.984375 18.0625 35.515625 20.015625 \nQ 33.0625 21.96875 24.703125 23.78125 \nL 21.578125 24.515625 \nQ 13.234375 26.265625 9.515625 29.90625 \nQ 5.8125 33.546875 5.8125 39.890625 \nQ 5.8125 47.609375 11.28125 51.796875 \nQ 16.75 56 26.8125 56 \nQ 31.78125 56 36.171875 55.265625 \nQ 40.578125 54.546875 44.28125 53.078125 \nz\n\" id=\"DejaVuSans-115\"/>\n     <path d=\"M 34.28125 27.484375 \nQ 23.390625 27.484375 19.1875 25 \nQ 14.984375 22.515625 14.984375 16.5 \nQ 14.984375 11.71875 18.140625 8.90625 \nQ 21.296875 6.109375 26.703125 6.109375 \nQ 34.1875 6.109375 38.703125 11.40625 \nQ 43.21875 16.703125 43.21875 25.484375 \nL 43.21875 27.484375 \nz\nM 52.203125 31.203125 \nL 52.203125 0 \nL 43.21875 0 \nL 43.21875 8.296875 \nQ 40.140625 3.328125 35.546875 0.953125 \nQ 30.953125 -1.421875 24.3125 -1.421875 \nQ 15.921875 -1.421875 10.953125 3.296875 \nQ 6 8.015625 6 15.921875 \nQ 6 25.140625 12.171875 29.828125 \nQ 18.359375 34.515625 30.609375 34.515625 \nL 43.21875 34.515625 \nL 43.21875 35.40625 \nQ 43.21875 41.609375 39.140625 45 \nQ 35.0625 48.390625 27.6875 48.390625 \nQ 23 48.390625 18.546875 47.265625 \nQ 14.109375 46.140625 10.015625 43.890625 \nL 10.015625 52.203125 \nQ 14.9375 54.109375 19.578125 55.046875 \nQ 24.21875 56 28.609375 56 \nQ 40.484375 56 46.34375 49.84375 \nQ 52.203125 43.703125 52.203125 31.203125 \nz\n\" id=\"DejaVuSans-97\"/>\n     <path d=\"M 45.40625 46.390625 \nL 45.40625 75.984375 \nL 54.390625 75.984375 \nL 54.390625 0 \nL 45.40625 0 \nL 45.40625 8.203125 \nQ 42.578125 3.328125 38.25 0.953125 \nQ 33.9375 -1.421875 27.875 -1.421875 \nQ 17.96875 -1.421875 11.734375 6.484375 \nQ 5.515625 14.40625 5.515625 27.296875 \nQ 5.515625 40.1875 11.734375 48.09375 \nQ 17.96875 56 27.875 56 \nQ 33.9375 56 38.25 53.625 \nQ 42.578125 51.265625 45.40625 46.390625 \nz\nM 14.796875 27.296875 \nQ 14.796875 17.390625 18.875 11.75 \nQ 22.953125 6.109375 30.078125 6.109375 \nQ 37.203125 6.109375 41.296875 11.75 \nQ 45.40625 17.390625 45.40625 27.296875 \nQ 45.40625 37.203125 41.296875 42.84375 \nQ 37.203125 48.484375 30.078125 48.484375 \nQ 22.953125 48.484375 18.875 42.84375 \nQ 14.796875 37.203125 14.796875 27.296875 \nz\n\" id=\"DejaVuSans-100\"/>\n     <path d=\"M 9.421875 75.984375 \nL 18.40625 75.984375 \nL 18.40625 0 \nL 9.421875 0 \nz\n\" id=\"DejaVuSans-108\"/>\n     <path d=\"M 56.203125 29.59375 \nL 56.203125 25.203125 \nL 14.890625 25.203125 \nQ 15.484375 15.921875 20.484375 11.0625 \nQ 25.484375 6.203125 34.421875 6.203125 \nQ 39.59375 6.203125 44.453125 7.46875 \nQ 49.3125 8.734375 54.109375 11.28125 \nL 54.109375 2.78125 \nQ 49.265625 0.734375 44.1875 -0.34375 \nQ 39.109375 -1.421875 33.890625 -1.421875 \nQ 20.796875 -1.421875 13.15625 6.1875 \nQ 5.515625 13.8125 5.515625 26.8125 \nQ 5.515625 40.234375 12.765625 48.109375 \nQ 20.015625 56 32.328125 56 \nQ 43.359375 56 49.78125 48.890625 \nQ 56.203125 41.796875 56.203125 29.59375 \nz\nM 47.21875 32.234375 \nQ 47.125 39.59375 43.09375 43.984375 \nQ 39.0625 48.390625 32.421875 48.390625 \nQ 24.90625 48.390625 20.390625 44.140625 \nQ 15.875 39.890625 15.1875 32.171875 \nz\n\" id=\"DejaVuSans-101\"/>\n     <path id=\"DejaVuSans-32\"/>\n     <path d=\"M 18.109375 8.203125 \nL 18.109375 -20.796875 \nL 9.078125 -20.796875 \nL 9.078125 54.6875 \nL 18.109375 54.6875 \nL 18.109375 46.390625 \nQ 20.953125 51.265625 25.265625 53.625 \nQ 29.59375 56 35.59375 56 \nQ 45.5625 56 51.78125 48.09375 \nQ 58.015625 40.1875 58.015625 27.296875 \nQ 58.015625 14.40625 51.78125 6.484375 \nQ 45.5625 -1.421875 35.59375 -1.421875 \nQ 29.59375 -1.421875 25.265625 0.953125 \nQ 20.953125 3.328125 18.109375 8.203125 \nz\nM 48.6875 27.296875 \nQ 48.6875 37.203125 44.609375 42.84375 \nQ 40.53125 48.484375 33.40625 48.484375 \nQ 26.265625 48.484375 22.1875 42.84375 \nQ 18.109375 37.203125 18.109375 27.296875 \nQ 18.109375 17.390625 22.1875 11.75 \nQ 26.265625 6.109375 33.40625 6.109375 \nQ 40.53125 6.109375 44.609375 11.75 \nQ 48.6875 17.390625 48.6875 27.296875 \nz\n\" id=\"DejaVuSans-112\"/>\n     <path d=\"M 30.609375 48.390625 \nQ 23.390625 48.390625 19.1875 42.75 \nQ 14.984375 37.109375 14.984375 27.296875 \nQ 14.984375 17.484375 19.15625 11.84375 \nQ 23.34375 6.203125 30.609375 6.203125 \nQ 37.796875 6.203125 41.984375 11.859375 \nQ 46.1875 17.53125 46.1875 27.296875 \nQ 46.1875 37.015625 41.984375 42.703125 \nQ 37.796875 48.390625 30.609375 48.390625 \nz\nM 30.609375 56 \nQ 42.328125 56 49.015625 48.375 \nQ 55.71875 40.765625 55.71875 27.296875 \nQ 55.71875 13.875 49.015625 6.21875 \nQ 42.328125 -1.421875 30.609375 -1.421875 \nQ 18.84375 -1.421875 12.171875 6.21875 \nQ 5.515625 13.875 5.515625 27.296875 \nQ 5.515625 40.765625 12.171875 48.375 \nQ 18.84375 56 30.609375 56 \nz\n\" id=\"DejaVuSans-111\"/>\n     <path d=\"M 9.421875 54.6875 \nL 18.40625 54.6875 \nL 18.40625 0 \nL 9.421875 0 \nz\nM 9.421875 75.984375 \nL 18.40625 75.984375 \nL 18.40625 64.59375 \nL 9.421875 64.59375 \nz\n\" id=\"DejaVuSans-105\"/>\n     <path d=\"M 54.890625 33.015625 \nL 54.890625 0 \nL 45.90625 0 \nL 45.90625 32.71875 \nQ 45.90625 40.484375 42.875 44.328125 \nQ 39.84375 48.1875 33.796875 48.1875 \nQ 26.515625 48.1875 22.3125 43.546875 \nQ 18.109375 38.921875 18.109375 30.90625 \nL 18.109375 0 \nL 9.078125 0 \nL 9.078125 54.6875 \nL 18.109375 54.6875 \nL 18.109375 46.1875 \nQ 21.34375 51.125 25.703125 53.5625 \nQ 30.078125 56 35.796875 56 \nQ 45.21875 56 50.046875 50.171875 \nQ 54.890625 44.34375 54.890625 33.015625 \nz\n\" id=\"DejaVuSans-110\"/>\n     <path d=\"M 18.3125 70.21875 \nL 18.3125 54.6875 \nL 36.8125 54.6875 \nL 36.8125 47.703125 \nL 18.3125 47.703125 \nL 18.3125 18.015625 \nQ 18.3125 11.328125 20.140625 9.421875 \nQ 21.96875 7.515625 27.59375 7.515625 \nL 36.8125 7.515625 \nL 36.8125 0 \nL 27.59375 0 \nQ 17.1875 0 13.234375 3.875 \nQ 9.28125 7.765625 9.28125 18.015625 \nL 9.28125 47.703125 \nL 2.6875 47.703125 \nL 2.6875 54.6875 \nL 9.28125 54.6875 \nL 9.28125 70.21875 \nz\n\" id=\"DejaVuSans-116\"/>\n    </defs>\n    <g transform=\"translate(140.660522 111.979167)scale(0.1 -0.1)\">\n     <use xlink:href=\"#DejaVuSans-115\"/>\n     <use x=\"52.099609\" xlink:href=\"#DejaVuSans-97\"/>\n     <use x=\"113.378906\" xlink:href=\"#DejaVuSans-100\"/>\n     <use x=\"176.855469\" xlink:href=\"#DejaVuSans-100\"/>\n     <use x=\"240.332031\" xlink:href=\"#DejaVuSans-108\"/>\n     <use x=\"268.115234\" xlink:href=\"#DejaVuSans-101\"/>\n     <use x=\"329.638672\" xlink:href=\"#DejaVuSans-32\"/>\n     <use x=\"361.425781\" xlink:href=\"#DejaVuSans-112\"/>\n     <use x=\"424.902344\" xlink:href=\"#DejaVuSans-111\"/>\n     <use x=\"486.083984\" xlink:href=\"#DejaVuSans-105\"/>\n     <use x=\"513.867188\" xlink:href=\"#DejaVuSans-110\"/>\n     <use x=\"577.246094\" xlink:href=\"#DejaVuSans-116\"/>\n    </g>\n   </g>\n  </g>\n </g>\n <defs>\n  <clipPath id=\"pcc0cef18bd\">\n   <rect height=\"135.9\" width=\"251.1\" x=\"42.620313\" y=\"7.2\"/>\n  </clipPath>\n </defs>\n</svg>\n"
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wL6pxgd3VRaT",
        "colab_type": "text"
      },
      "source": [
        "再举个定义在二维空间的函数的例子，例如：\n",
        "$$\n",
        "f(x, y) = x^2 - y^2.\n",
        "$$\n",
        "我们可以找出该函数的鞍点位置。也许你已经发现了，该函数看起来像一个马鞍，而鞍点恰好是马鞍上可坐区域的中心。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_vS-6VXv_Nef",
        "colab_type": "code",
        "outputId": "a2668187-dc0a-4a28-a744-8b0768103681",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 220
        }
      },
      "source": [
        "x, y = np.mgrid[-1: 1: 31j, -1: 1: 31j]\n",
        "z = x**2 - y**2\n",
        "\n",
        "ax = plt.figure().add_subplot(111, projection='3d')\n",
        "ax.plot_wireframe(x, y, z, **{'rstride': 2, 'cstride': 2})\n",
        "ax.plot([0], [0], [0], 'rx')\n",
        "ticks = [-1,  0, 1]\n",
        "plt.xticks(ticks)\n",
        "plt.yticks(ticks)\n",
        "ax.set_zticks(ticks)\n",
        "plt.xlabel('x')\n",
        "plt.ylabel('y');"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 324x180 with 1 Axes>"
            ],
            "image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<!-- Created with matplotlib (https://matplotlib.org/) -->\n<svg height=\"150.3pt\" version=\"1.1\" viewBox=\"0 0 265.5 150.3\" width=\"265.5pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n <defs>\n  <style type=\"text/css\">\n*{stroke-linecap:butt;stroke-linejoin:round;}\n  </style>\n </defs>\n <g id=\"figure_1\">\n  <g id=\"patch_1\">\n   <path d=\"M 0 150.3 \nL 265.5 150.3 \nL 265.5 0 \nL 0 0 \nz\n\" style=\"fill:none;\"/>\n  </g>\n  <g id=\"patch_2\">\n   <path d=\"M 7.2 143.1 \nL 258.3 143.1 \nL 258.3 7.2 \nL 7.2 7.2 \nz\n\" style=\"fill:#ffffff;\"/>\n  </g>\n  <g id=\"pane3d_1\">\n   <g id=\"patch_3\">\n    <path d=\"M 40.487185 113.070889 \nL 112.353319 79.751289 \nL 111.156824 16.083 \nL 35.283682 46.019976 \n\" style=\"fill:#f2f2f2;opacity:0.5;stroke:#f2f2f2;stroke-linejoin:miter;\"/>\n   </g>\n  </g>\n  <g id=\"pane3d_2\">\n   <g id=\"patch_4\">\n    <path d=\"M 112.353319 79.751289 \nL 228.769699 98.379096 \nL 233.640421 32.790916 \nL 111.156824 16.083 \n\" style=\"fill:#e6e6e6;opacity:0.5;stroke:#e6e6e6;stroke-linejoin:miter;\"/>\n   </g>\n  </g>\n  <g id=\"pane3d_3\">\n   <g id=\"patch_5\">\n    <path d=\"M 40.487185 113.070889 \nL 162.972947 134.766006 \nL 228.769699 98.379096 \nL 112.353319 79.751289 \n\" style=\"fill:#ececec;opacity:0.5;stroke:#ececec;stroke-linejoin:miter;\"/>\n   </g>\n  </g>\n  <g id=\"axis3d_1\">\n   <g id=\"line2d_1\">\n    <path d=\"M 40.487185 113.070889 \nL 162.972947 134.766006 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"text_1\">\n    <!-- x -->\n    <defs>\n     <path d=\"M 54.890625 54.6875 \nL 35.109375 28.078125 \nL 55.90625 0 \nL 45.3125 0 \nL 29.390625 21.484375 \nL 13.484375 0 \nL 2.875 0 \nL 24.125 28.609375 \nL 4.6875 54.6875 \nL 15.28125 54.6875 \nL 29.78125 35.203125 \nL 44.28125 54.6875 \nz\n\" id=\"DejaVuSans-120\"/>\n    </defs>\n    <g transform=\"translate(79.250785 151.909667)scale(0.1 -0.1)\">\n     <use xlink:href=\"#DejaVuSans-120\"/>\n    </g>\n   </g>\n   <g id=\"Line3DCollection_1\">\n    <path d=\"M 47.954708 114.393564 \nL 119.473198 80.890542 \nL 118.632462 17.102748 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 100.33065 123.670578 \nL 169.328996 88.867978 \nL 171.035205 24.250975 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 154.838065 133.325126 \nL 221.062087 97.145799 \nL 225.514604 31.682478 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"xtick_1\">\n    <g id=\"line2d_2\">\n     <path d=\"M 48.570877 114.104918 \nL 46.720073 114.971932 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n    <g id=\"text_2\">\n     <!-- −1 -->\n     <defs>\n      <path d=\"M 10.59375 35.5 \nL 73.1875 35.5 \nL 73.1875 27.203125 \nL 10.59375 27.203125 \nz\n\" id=\"DejaVuSans-8722\"/>\n      <path d=\"M 12.40625 8.296875 \nL 28.515625 8.296875 \nL 28.515625 63.921875 \nL 10.984375 60.40625 \nL 10.984375 69.390625 \nL 28.421875 72.90625 \nL 38.28125 72.90625 \nL 38.28125 8.296875 \nL 54.390625 8.296875 \nL 54.390625 0 \nL 12.40625 0 \nz\n\" id=\"DejaVuSans-49\"/>\n     </defs>\n     <g transform=\"translate(32.285551 133.414626)scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-8722\"/>\n      <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"xtick_2\">\n    <g id=\"line2d_3\">\n     <path d=\"M 100.925952 123.370309 \nL 99.13778 124.272259 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n    <g id=\"text_3\">\n     <!-- 0 -->\n     <defs>\n      <path d=\"M 31.78125 66.40625 \nQ 24.171875 66.40625 20.328125 58.90625 \nQ 16.5 51.421875 16.5 36.375 \nQ 16.5 21.390625 20.328125 13.890625 \nQ 24.171875 6.390625 31.78125 6.390625 \nQ 39.453125 6.390625 43.28125 13.890625 \nQ 47.125 21.390625 47.125 36.375 \nQ 47.125 51.421875 43.28125 58.90625 \nQ 39.453125 66.40625 31.78125 66.40625 \nz\nM 31.78125 74.21875 \nQ 44.046875 74.21875 50.515625 64.515625 \nQ 56.984375 54.828125 56.984375 36.375 \nQ 56.984375 17.96875 50.515625 8.265625 \nQ 44.046875 -1.421875 31.78125 -1.421875 \nQ 19.53125 -1.421875 13.0625 8.265625 \nQ 6.59375 17.96875 6.59375 36.375 \nQ 6.59375 54.828125 13.0625 64.515625 \nQ 19.53125 74.21875 31.78125 74.21875 \nz\n\" id=\"DejaVuSans-48\"/>\n     </defs>\n     <g transform=\"translate(88.843524 142.948134)scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-48\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"xtick_3\">\n    <g id=\"line2d_4\">\n     <path d=\"M 155.410277 133.012517 \nL 153.69142 133.951558 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n    <g id=\"text_4\">\n     <!-- 1 -->\n     <g transform=\"translate(143.349377 152.87081)scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-49\"/>\n     </g>\n    </g>\n   </g>\n  </g>\n  <g id=\"axis3d_2\">\n   <g id=\"line2d_5\">\n    <path d=\"M 228.769699 98.379096 \nL 162.972947 134.766006 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"text_5\">\n    <!-- y -->\n    <defs>\n     <path d=\"M 32.171875 -5.078125 \nQ 28.375 -14.84375 24.75 -17.8125 \nQ 21.140625 -20.796875 15.09375 -20.796875 \nL 7.90625 -20.796875 \nL 7.90625 -13.28125 \nL 13.1875 -13.28125 \nQ 16.890625 -13.28125 18.9375 -11.515625 \nQ 21 -9.765625 23.484375 -3.21875 \nL 25.09375 0.875 \nL 2.984375 54.6875 \nL 12.5 54.6875 \nL 29.59375 11.921875 \nL 46.6875 54.6875 \nL 56.203125 54.6875 \nz\n\" id=\"DejaVuSans-121\"/>\n    </defs>\n    <g transform=\"translate(224.303284 139.624581)scale(0.1 -0.1)\">\n     <use xlink:href=\"#DejaVuSans-121\"/>\n    </g>\n   </g>\n   <g id=\"Line3DCollection_2\">\n    <path d=\"M 40.478971 43.970091 \nL 45.390254 110.797659 \nL 167.475981 132.275739 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 74.707339 30.464735 \nL 77.75884 95.790474 \nL 197.151948 115.86434 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 106.671251 17.852856 \nL 108.089022 81.728363 \nL 224.877837 100.531372 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"xtick_4\">\n    <g id=\"line2d_6\">\n     <path d=\"M 166.454231 132.095986 \nL 169.521765 132.635646 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n    <g id=\"text_6\">\n     <!-- −1 -->\n     <g transform=\"translate(174.48021 149.869862)scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-8722\"/>\n      <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"xtick_5\">\n    <g id=\"line2d_7\">\n     <path d=\"M 196.154229 115.696591 \nL 199.149539 116.2002 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n    <g id=\"text_7\">\n     <!-- 0 -->\n     <g transform=\"translate(207.834516 133.132167)scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-48\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"xtick_6\">\n    <g id=\"line2d_8\">\n     <path d=\"M 223.90325 100.374464 \nL 226.829048 100.845517 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n    <g id=\"text_8\">\n     <!-- 1 -->\n     <g transform=\"translate(235.083983 117.493501)scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-49\"/>\n     </g>\n    </g>\n   </g>\n  </g>\n  <g id=\"axis3d_3\">\n   <g id=\"line2d_9\">\n    <path d=\"M 228.769699 98.379096 \nL 233.640421 32.790916 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"Line3DCollection_3\">\n    <path d=\"M 228.862339 97.131618 \nL 112.330513 78.537737 \nL 40.388382 111.797743 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 231.142669 66.425148 \nL 111.769747 48.698043 \nL 37.954325 80.433148 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 233.537992 34.170208 \nL 111.181932 17.419065 \nL 35.393294 47.432407 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"xtick_7\">\n    <g id=\"line2d_10\">\n     <path d=\"M 227.890048 96.976479 \nL 230.808945 97.44222 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n    <g id=\"text_9\">\n     <!-- −1 -->\n     <g transform=\"translate(242.240836 103.115424)scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-8722\"/>\n      <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"xtick_8\">\n    <g id=\"line2d_11\">\n     <path d=\"M 230.145664 66.277091 \nL 233.138804 66.721578 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n    <g id=\"text_10\">\n     <!-- 0 -->\n     <g transform=\"translate(249.211018 72.482787)scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-48\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"xtick_9\">\n    <g id=\"line2d_12\">\n     <path d=\"M 232.514986 34.030153 \nL 235.586238 34.450623 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n    <g id=\"text_11\">\n     <!-- 1 -->\n     <g transform=\"translate(252.131033 40.311251)scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-49\"/>\n     </g>\n    </g>\n   </g>\n  </g>\n  <g id=\"axes_1\">\n   <g id=\"line2d_13\">\n    <defs>\n     <path d=\"M -3 3 \nL 3 -3 \nM -3 -3 \nL 3 3 \n\" id=\"m6345b32f4b\" style=\"stroke:#ff0000;\"/>\n    </defs>\n    <g clip-path=\"url(#p6a4ab83ea0)\">\n     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"136.143243\" xlink:href=\"#m6345b32f4b\" y=\"73.313514\"/>\n    </g>\n   </g>\n   <g id=\"Line3DCollection_4\">\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 50.633789 79.513765 \nL 52.639919 74.377046 \nL 54.668328 69.518701 \nL 56.71769 64.942641 \nL 58.786647 60.652404 \nL 60.873813 56.651146 \nL 62.977777 52.941634 \nL 65.097104 49.526238 \nL 67.230341 46.406927 \nL 69.376017 43.585267 \nL 71.532652 41.062414 \nL 73.698755 38.839118 \nL 75.872827 36.915722 \nL 78.05337 35.292162 \nL 80.238887 33.967973 \nL 82.427882 32.94229 \nL 84.618869 32.213859 \nL 86.810374 31.781039 \nL 89.000935 31.641811 \nL 91.189107 31.793791 \nL 93.373466 32.234237 \nL 95.552611 32.960061 \nL 97.725166 33.967842 \nL 99.889782 35.253839 \nL 102.045142 36.814003 \nL 104.189962 38.643996 \nL 106.322991 40.739201 \nL 108.443017 43.094743 \nL 110.548865 45.705501 \nL 112.6394 48.566124 \nL 114.713531 51.671053 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 58.146625 88.619798 \nL 60.15267 83.510529 \nL 62.179222 78.676611 \nL 64.224956 74.121918 \nL 66.288521 69.849956 \nL 68.368537 65.863853 \nL 70.463603 62.166357 \nL 72.572295 58.759822 \nL 74.693176 55.646208 \nL 76.824791 52.827075 \nL 78.96568 50.303584 \nL 81.114371 48.076492 \nL 83.269392 46.146154 \nL 85.429268 44.512527 \nL 87.59253 43.175169 \nL 89.757713 42.133247 \nL 91.923361 41.385539 \nL 94.088033 40.930447 \nL 96.250301 40.765997 \nL 98.408758 40.889854 \nL 100.562017 41.299331 \nL 102.708714 41.991399 \nL 104.847515 42.962699 \nL 106.97711 44.209557 \nL 109.096226 45.727995 \nL 111.20362 47.513747 \nL 113.298086 49.562274 \nL 115.378454 51.868778 \nL 117.443595 54.428221 \nL 119.492419 57.235335 \nL 121.523878 60.284647 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 65.541795 96.541427 \nL 67.549442 91.453716 \nL 69.575882 86.638857 \nL 71.619791 82.100694 \nL 73.67982 77.842708 \nL 75.754594 73.86801 \nL 77.842717 70.179329 \nL 79.942778 66.779009 \nL 82.053348 63.669004 \nL 84.172989 60.850872 \nL 86.300255 58.325776 \nL 88.433694 56.09448 \nL 90.571854 54.157352 \nL 92.713282 52.514365 \nL 94.856534 51.165097 \nL 97.00017 50.10874 \nL 99.142764 49.344103 \nL 101.282905 48.869619 \nL 103.419195 48.683353 \nL 105.550261 48.783009 \nL 107.67475 49.165947 \nL 109.791334 49.829183 \nL 111.898714 50.769412 \nL 113.995623 51.983013 \nL 116.080822 53.466064 \nL 118.153111 55.214361 \nL 120.211322 57.223425 \nL 122.25433 59.488523 \nL 124.281044 62.004683 \nL 126.290419 64.766705 \nL 128.281449 67.769186 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 72.840106 103.309598 \nL 74.851052 98.237687 \nL 76.879127 93.436635 \nL 78.923004 88.910265 \nL 80.981334 84.662041 \nL 83.052745 80.695059 \nL 85.135845 77.012037 \nL 87.229228 73.615313 \nL 89.331477 70.506837 \nL 91.441164 67.688167 \nL 93.556856 65.160469 \nL 95.677117 62.924515 \nL 97.800512 60.980683 \nL 99.925609 59.328958 \nL 102.050983 57.968936 \nL 104.175221 56.899831 \nL 106.29692 56.120473 \nL 108.414695 55.629323 \nL 110.527179 55.424474 \nL 112.633028 55.503666 \nL 114.730921 55.86429 \nL 116.819563 56.503402 \nL 118.89769 57.417736 \nL 120.964068 58.603713 \nL 123.017497 60.057458 \nL 125.056813 61.774811 \nL 127.080887 63.751343 \nL 129.088633 65.98237 \nL 131.079001 68.462971 \nL 133.050983 71.188 \nL 135.003618 74.152106 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 80.061656 108.948787 \nL 82.077618 103.887018 \nL 84.109084 99.094606 \nL 86.154723 94.575362 \nL 88.213182 90.33274 \nL 90.28309 86.369828 \nL 92.363058 82.689338 \nL 94.451683 79.293606 \nL 96.547554 76.184579 \nL 98.649252 73.363819 \nL 100.755356 70.832495 \nL 102.86444 68.591386 \nL 104.975087 66.640878 \nL 107.085879 64.980969 \nL 109.195413 63.611268 \nL 111.302294 62.531003 \nL 113.405143 61.739025 \nL 115.502599 61.233812 \nL 117.593321 61.013482 \nL 119.675992 61.075797 \nL 121.74932 61.418175 \nL 123.812043 62.037701 \nL 125.862926 62.931136 \nL 127.900769 64.094933 \nL 129.924407 65.52525 \nL 131.93271 67.21796 \nL 133.924588 69.168669 \nL 135.898988 71.372729 \nL 137.8549 73.825258 \nL 139.791356 76.521146 \nL 141.707431 79.455082 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 87.226022 113.477284 \nL 89.248743 108.420057 \nL 91.285375 103.63117 \nL 93.334578 99.114427 \nL 95.394996 94.87328 \nL 97.465254 90.910814 \nL 99.543963 87.229742 \nL 101.629723 83.832399 \nL 103.721125 80.720736 \nL 105.816757 77.896316 \nL 107.915205 75.360316 \nL 110.015055 73.113519 \nL 112.1149 71.15632 \nL 114.213339 69.488724 \nL 116.308983 68.110353 \nL 118.400456 67.020445 \nL 120.4864 66.217864 \nL 122.565474 65.701102 \nL 124.636362 65.468291 \nL 126.697772 65.517211 \nL 128.748438 65.845295 \nL 130.787126 66.449646 \nL 132.812633 67.327047 \nL 134.823788 68.473968 \nL 136.819459 69.886589 \nL 138.798549 71.560803 \nL 140.760003 73.49224 \nL 142.702803 75.676274 \nL 144.625975 78.108045 \nL 146.528589 80.782469 \nL 148.409757 83.694258 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 94.352434 116.907381 \nL 96.383695 111.849123 \nL 98.427294 107.058663 \nL 100.481884 102.539811 \nL 102.5461 98.296019 \nL 104.618564 94.330379 \nL 106.697881 90.645606 \nL 108.782653 87.244042 \nL 110.871471 84.127641 \nL 112.962927 81.297973 \nL 115.055611 78.75622 \nL 117.148119 76.503172 \nL 119.239052 74.539229 \nL 121.327022 72.864404 \nL 123.410653 71.478325 \nL 125.488586 70.380239 \nL 127.559478 69.569014 \nL 129.62201 69.043154 \nL 131.674887 68.800797 \nL 133.716839 68.83973 \nL 135.746627 69.157396 \nL 137.763041 69.750906 \nL 139.764906 70.617051 \nL 141.751081 71.752312 \nL 143.720464 73.152876 \nL 145.671991 74.814647 \nL 147.604637 76.733261 \nL 149.517421 78.904106 \nL 151.409403 81.322328 \nL 153.279688 83.982853 \nL 155.127425 86.880404 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 101.459949 119.245487 \nL 103.501576 114.180612 \nL 105.553982 109.383471 \nL 107.61581 104.857882 \nL 109.685685 100.607311 \nL 111.76222 96.634858 \nL 113.844017 92.943248 \nL 115.929672 89.534831 \nL 118.017777 86.41157 \nL 120.106924 83.575042 \nL 122.195706 81.026435 \nL 124.282725 78.766545 \nL 126.366588 76.795779 \nL 128.445918 75.114153 \nL 130.519348 73.7213 \nL 132.585535 72.616467 \nL 134.64315 71.798528 \nL 136.690893 71.265986 \nL 138.727487 71.016981 \nL 140.751683 71.049301 \nL 142.762265 71.360388 \nL 144.758047 71.947353 \nL 146.737881 72.806983 \nL 148.700654 73.935759 \nL 150.645292 75.329865 \nL 152.570761 76.985201 \nL 154.476069 78.897402 \nL 156.360267 81.061849 \nL 158.222449 83.473686 \nL 160.061754 86.127835 \nL 161.877367 89.019012 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 108.567624 120.492157 \nL 110.621497 115.415034 \nL 112.684598 110.606056 \nL 114.755554 106.069064 \nL 116.832979 101.807539 \nL 118.915474 97.824596 \nL 121.001635 94.122977 \nL 123.09005 90.705043 \nL 125.179309 87.57277 \nL 127.268002 84.727742 \nL 129.354722 82.171156 \nL 131.438074 79.903814 \nL 133.516672 77.926125 \nL 135.589142 76.23811 \nL 137.65413 74.8394 \nL 139.710302 73.729243 \nL 141.756344 72.90651 \nL 143.79097 72.369698 \nL 145.812921 72.116943 \nL 147.820968 72.146022 \nL 149.813915 72.454371 \nL 151.790599 73.039089 \nL 153.749897 73.896953 \nL 155.690722 75.024428 \nL 157.612027 76.417686 \nL 159.512808 78.07261 \nL 161.392104 79.984821 \nL 163.248997 82.14968 \nL 165.082613 84.562315 \nL 166.892128 87.217629 \nL 168.67676 90.110317 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 115.694682 120.642048 \nL 117.762751 115.546961 \nL 119.838492 110.720916 \nL 121.920516 106.167781 \nL 124.007422 101.891061 \nL 126.097799 97.893896 \nL 128.190231 94.179044 \nL 130.283298 90.748885 \nL 132.375585 87.605408 \nL 134.465675 84.75021 \nL 136.552162 82.184495 \nL 138.63365 79.909071 \nL 140.708756 77.924351 \nL 142.776111 76.230354 \nL 144.834368 74.826711 \nL 146.882201 73.712662 \nL 148.918309 72.88707 \nL 150.94142 72.348423 \nL 152.950288 72.094842 \nL 154.943704 72.12409 \nL 156.92049 72.433584 \nL 158.879507 73.020402 \nL 160.819652 73.881299 \nL 162.739864 75.012717 \nL 164.639124 76.4108 \nL 166.516455 78.071405 \nL 168.370926 79.990121 \nL 170.201649 82.16228 \nL 172.007786 84.582978 \nL 173.788542 87.247083 \nL 175.543173 90.149259 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 122.860695 119.68379 \nL 124.944986 114.564903 \nL 127.035383 109.716449 \nL 129.130476 105.14233 \nL 131.228848 100.846086 \nL 133.329071 96.830883 \nL 135.429715 93.099507 \nL 137.529352 89.654356 \nL 139.626555 86.497438 \nL 141.719902 83.630364 \nL 143.807982 81.054346 \nL 145.889397 78.770198 \nL 147.962764 76.778336 \nL 150.026717 75.078775 \nL 152.079915 73.67114 \nL 154.121038 72.554662 \nL 156.148796 71.728191 \nL 158.161926 71.190196 \nL 160.159198 70.938777 \nL 162.139419 70.971675 \nL 164.101428 71.286278 \nL 166.044107 71.879635 \nL 167.966375 72.748467 \nL 169.867196 73.88918 \nL 171.745576 75.297879 \nL 173.600565 76.970382 \nL 175.431261 78.902233 \nL 177.236809 81.088721 \nL 179.0164 83.524894 \nL 180.769275 86.205572 \nL 182.494723 89.12537 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 130.085763 117.599775 \nL 132.188394 112.451091 \nL 134.295545 107.574737 \nL 136.405782 102.974663 \nL 138.517668 98.654447 \nL 140.629756 94.617291 \nL 142.740601 90.86601 \nL 144.84876 87.403031 \nL 146.952794 84.230379 \nL 149.051274 81.349682 \nL 151.14278 78.762164 \nL 153.22591 76.468642 \nL 155.299279 74.469533 \nL 157.361523 72.764849 \nL 159.411302 71.354203 \nL 161.447302 70.236813 \nL 163.46824 69.411508 \nL 165.472864 68.876735 \nL 167.459958 68.630565 \nL 169.42834 68.670706 \nL 171.376869 68.994508 \nL 173.304445 69.598981 \nL 175.210008 70.4808 \nL 177.092546 71.636324 \nL 178.951089 73.061606 \nL 180.784715 74.75241 \nL 182.592552 76.704223 \nL 184.373772 78.912275 \nL 186.127602 81.371551 \nL 187.853314 84.076809 \nL 189.550234 87.022597 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 137.390712 114.365856 \nL 139.513908 109.181172 \nL 141.640006 104.271245 \nL 143.76755 99.640077 \nL 145.895074 95.291295 \nL 148.021114 91.228143 \nL 150.144205 87.453473 \nL 152.262886 83.969741 \nL 154.375704 80.778998 \nL 156.481218 77.882888 \nL 158.577999 75.282647 \nL 160.664638 72.979098 \nL 162.739746 70.972657 \nL 164.801957 69.263327 \nL 166.849932 67.850708 \nL 168.882361 66.734001 \nL 170.897966 65.912007 \nL 172.895504 65.383142 \nL 174.873769 65.145442 \nL 176.831595 65.196571 \nL 178.767855 65.533834 \nL 180.681467 66.154188 \nL 182.571394 67.054251 \nL 184.436643 68.230323 \nL 186.276272 69.67839 \nL 188.089385 71.394149 \nL 189.875137 73.373016 \nL 191.632734 75.610144 \nL 193.361432 78.100442 \nL 195.06054 80.838588 \nL 196.729416 83.819047 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 144.797308 109.95094 \nL 146.943414 104.723805 \nL 149.090766 99.774404 \nL 151.237878 95.106802 \nL 153.383258 90.724683 \nL 155.525417 86.63134 \nL 157.662868 82.829669 \nL 159.794129 79.322159 \nL 161.917731 76.110891 \nL 164.032218 73.197529 \nL 166.136149 70.58332 \nL 168.228106 68.269093 \nL 170.306693 66.255261 \nL 172.370539 64.541817 \nL 174.418306 63.128344 \nL 176.448684 62.014017 \nL 178.460401 61.197607 \nL 180.452221 60.67749 \nL 182.422946 60.451657 \nL 184.371423 60.51772 \nL 186.296542 60.872926 \nL 188.197236 61.514168 \nL 190.072489 62.437997 \nL 191.921331 63.640635 \nL 193.742844 65.117993 \nL 195.536159 66.865682 \nL 197.300461 68.879031 \nL 199.034985 71.153103 \nL 200.739022 73.682711 \nL 202.411914 76.462436 \nL 204.053058 79.486644 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 152.328489 104.316478 \nL 154.499989 99.04014 \nL 156.671028 94.045093 \nL 158.840088 89.335474 \nL 161.005647 84.915035 \nL 163.166187 80.787127 \nL 165.320195 76.954693 \nL 167.466167 73.420264 \nL 169.602613 70.185952 \nL 171.728059 67.253443 \nL 173.84105 64.623999 \nL 175.940157 62.298453 \nL 178.023973 60.277212 \nL 180.091123 58.560258 \nL 182.140264 57.147152 \nL 184.170087 56.037038 \nL 186.179323 55.228647 \nL 188.166741 54.72031 \nL 190.131153 54.509962 \nL 192.071417 54.595153 \nL 193.986435 54.973062 \nL 195.875161 55.640503 \nL 197.736596 56.593946 \nL 199.569794 57.829524 \nL 201.373861 59.343053 \nL 203.147955 61.130044 \nL 204.891291 63.185723 \nL 206.603136 65.505043 \nL 208.282814 68.082706 \nL 209.929704 70.913177 \nL 211.543239 73.990704 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 160.008622 97.415821 \nL 162.208158 92.083173 \nL 164.405466 87.035985 \nL 166.59899 82.278484 \nL 168.787174 77.814494 \nL 170.968467 73.647433 \nL 173.141328 69.780302 \nL 175.304226 66.215676 \nL 177.455647 62.955703 \nL 179.594098 60.002094 \nL 181.718106 57.356125 \nL 183.826225 55.018635 \nL 185.91704 52.990024 \nL 187.989166 51.270257 \nL 190.041256 49.858869 \nL 192.071999 48.754966 \nL 194.080125 47.957234 \nL 196.064411 47.463947 \nL 198.023674 47.272975 \nL 199.956785 47.381796 \nL 201.862659 47.787504 \nL 203.740265 48.486825 \nL 205.588627 49.47613 \nL 207.406819 50.751449 \nL 209.193973 52.308487 \nL 210.949276 54.142636 \nL 212.671972 56.249001 \nL 214.361362 58.622408 \nL 216.016805 61.257427 \nL 217.637716 64.148386 \nL 219.22357 67.289397 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 50.633789 79.513765 \nL 54.406249 84.217028 \nL 58.146625 88.619798 \nL 61.857599 92.726574 \nL 65.541795 96.541427 \nL 69.201787 100.068017 \nL 72.840106 103.309598 \nL 76.459242 106.269029 \nL 80.061656 108.948787 \nL 83.649779 111.350964 \nL 87.226022 113.477284 \nL 90.792779 115.329095 \nL 94.352434 116.907381 \nL 97.907365 118.212761 \nL 101.459949 119.245487 \nL 105.012571 120.005446 \nL 108.567624 120.492157 \nL 112.127517 120.704767 \nL 115.694682 120.642048 \nL 119.271578 120.302388 \nL 122.860695 119.68379 \nL 126.464565 118.783854 \nL 130.085763 117.599775 \nL 133.726916 116.128329 \nL 137.390712 114.365856 \nL 141.079901 112.308251 \nL 144.797308 109.95094 \nL 148.54584 107.28887 \nL 152.328489 104.316478 \nL 156.14835 101.027676 \nL 160.008622 97.415821 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 54.668328 69.518701 \nL 58.439387 74.249329 \nL 62.179222 78.676611 \nL 65.890511 82.805085 \nL 69.575882 86.638857 \nL 73.237909 90.181614 \nL 76.879127 93.436635 \nL 80.50203 96.406802 \nL 84.109084 99.094606 \nL 87.702726 101.502152 \nL 91.285375 103.63117 \nL 94.859433 105.483012 \nL 98.427294 107.058663 \nL 101.991347 108.358735 \nL 105.553982 109.383471 \nL 109.117596 110.132744 \nL 112.684598 110.606056 \nL 116.257414 110.802532 \nL 119.838492 110.720916 \nL 123.430311 110.359567 \nL 127.035383 109.716449 \nL 130.656261 108.789123 \nL 134.295545 107.574737 \nL 137.955888 106.070017 \nL 141.640006 104.271245 \nL 145.35068 102.174255 \nL 149.090766 99.774404 \nL 152.863205 97.066564 \nL 156.671028 94.045093 \nL 160.517368 90.703812 \nL 164.405466 87.035985 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 58.786647 60.652404 \nL 62.552737 65.404008 \nL 66.288521 69.849956 \nL 69.996675 73.994812 \nL 73.67982 77.842708 \nL 77.340529 81.397352 \nL 80.981334 84.662041 \nL 84.60473 87.63967 \nL 88.213182 90.33274 \nL 91.809131 92.743366 \nL 95.394996 94.87328 \nL 98.973187 96.723838 \nL 102.5461 98.296019 \nL 106.116134 99.590432 \nL 109.685685 100.607311 \nL 113.25716 101.346518 \nL 116.832979 101.807539 \nL 120.415579 101.989479 \nL 124.007422 101.891061 \nL 127.611002 101.510619 \nL 131.228848 100.846086 \nL 134.86353 99.894992 \nL 138.517668 98.654447 \nL 142.193937 97.121135 \nL 145.895074 95.291295 \nL 149.623887 93.160709 \nL 153.383258 90.724683 \nL 157.176157 87.978027 \nL 161.005647 84.915035 \nL 164.874893 81.529462 \nL 168.787174 77.814494 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 62.977777 52.941634 \nL 66.735355 57.707701 \nL 70.463603 62.166357 \nL 74.165186 66.322185 \nL 77.842717 70.179329 \nL 81.498762 73.741511 \nL 85.135845 77.012037 \nL 88.756456 79.993811 \nL 92.363058 82.689338 \nL 95.958087 85.100738 \nL 99.543963 87.229742 \nL 103.123095 89.077706 \nL 106.697881 90.645606 \nL 110.270721 91.934045 \nL 113.844017 92.943248 \nL 117.42018 93.673068 \nL 121.001635 94.122977 \nL 124.590828 94.292068 \nL 128.190231 94.179044 \nL 131.802346 93.78222 \nL 135.429715 93.099507 \nL 139.074922 92.128409 \nL 142.740601 90.86601 \nL 146.429444 89.308963 \nL 150.144205 87.453473 \nL 153.887711 85.295286 \nL 157.662868 82.829669 \nL 161.472666 80.051388 \nL 165.320195 76.954693 \nL 169.208646 73.533286 \nL 173.141328 69.780302 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 67.230341 46.406927 \nL 70.975909 51.180868 \nL 74.693176 55.646208 \nL 78.384789 59.807536 \nL 82.053348 63.669004 \nL 85.701406 67.234334 \nL 89.331477 70.506837 \nL 92.946042 73.489416 \nL 96.547554 76.184579 \nL 100.138444 78.594442 \nL 103.721125 80.720736 \nL 107.298002 82.564811 \nL 110.871471 84.127641 \nL 114.443929 85.40982 \nL 118.017777 86.41157 \nL 121.595428 87.132732 \nL 125.179309 87.57277 \nL 128.771869 87.730764 \nL 132.375585 87.605408 \nL 135.992964 87.195 \nL 139.626555 86.497438 \nL 143.27895 85.510209 \nL 146.952794 84.230379 \nL 150.65079 82.65458 \nL 154.375704 80.778998 \nL 158.130378 78.599354 \nL 161.917731 76.110891 \nL 165.740774 73.308351 \nL 169.602613 70.185952 \nL 173.506461 66.737369 \nL 177.455647 62.955703 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 71.532652 41.062414 \nL 75.262781 45.837608 \nL 78.96568 50.303584 \nL 82.643978 54.464925 \nL 86.300255 58.325776 \nL 89.937049 61.889856 \nL 93.556856 65.160469 \nL 97.162145 68.140515 \nL 100.755356 70.832495 \nL 104.338907 73.23852 \nL 107.915205 75.360316 \nL 111.486643 77.199227 \nL 115.055611 78.75622 \nL 118.6245 80.031885 \nL 122.195706 81.026435 \nL 125.771639 81.739707 \nL 129.354722 82.171156 \nL 132.947405 82.319857 \nL 136.552162 82.184495 \nL 140.171505 81.763361 \nL 143.807982 81.054346 \nL 147.464191 80.054928 \nL 151.14278 78.762164 \nL 154.846458 77.172678 \nL 158.577999 75.282647 \nL 162.340253 73.087783 \nL 166.136149 70.58332 \nL 169.968709 67.763988 \nL 173.84105 64.623999 \nL 177.756401 61.157014 \nL 181.718106 57.356125 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 75.872827 36.915722 \nL 79.584169 41.685572 \nL 83.269392 46.146154 \nL 86.931101 50.302038 \nL 90.571854 54.157352 \nL 94.194164 57.715803 \nL 97.800512 60.980683 \nL 101.393345 63.954877 \nL 104.975087 66.640878 \nL 108.548142 69.040787 \nL 112.1149 71.15632 \nL 115.677744 72.988814 \nL 119.239052 74.539229 \nL 122.801204 75.808148 \nL 126.366588 76.795779 \nL 129.937605 77.501954 \nL 133.516672 77.926125 \nL 137.106231 78.067363 \nL 140.708756 77.924351 \nL 144.32675 77.495379 \nL 147.962764 76.778336 \nL 151.619391 75.770701 \nL 155.299279 74.469533 \nL 159.005139 72.871459 \nL 162.739746 70.972657 \nL 166.505953 68.768845 \nL 170.306693 66.255261 \nL 174.144991 63.426644 \nL 178.023973 60.277212 \nL 181.946872 56.800638 \nL 185.91704 52.990024 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 80.238887 33.967973 \nL 83.928195 38.725945 \nL 87.59253 43.175169 \nL 91.234466 47.320184 \nL 94.856534 51.165097 \nL 98.461221 54.713591 \nL 102.050983 57.968936 \nL 105.628246 60.934004 \nL 109.195413 63.611268 \nL 112.754868 66.002816 \nL 116.308983 68.110353 \nL 119.860124 69.935206 \nL 123.410653 71.478325 \nL 126.962937 72.740289 \nL 130.519348 73.7213 \nL 134.082277 74.421186 \nL 137.65413 74.8394 \nL 141.23734 74.975013 \nL 144.834368 74.826711 \nL 148.447713 74.392788 \nL 152.079915 73.67114 \nL 155.733564 72.659255 \nL 159.411302 71.354203 \nL 163.115834 69.752622 \nL 166.849932 67.850708 \nL 170.616445 65.644196 \nL 174.418306 63.128344 \nL 178.258538 60.297915 \nL 182.140264 57.147152 \nL 186.066719 53.669759 \nL 190.041256 49.858869 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 84.618869 32.213859 \nL 88.283017 36.953536 \nL 91.923361 41.385539 \nL 95.542444 45.514371 \nL 99.142764 49.344103 \nL 102.726781 52.878389 \nL 106.29692 56.120473 \nL 109.855581 59.0732 \nL 113.405143 61.739025 \nL 116.947965 64.120016 \nL 120.4864 66.217864 \nL 124.022789 68.033883 \nL 127.559478 69.569014 \nL 131.098812 70.82383 \nL 134.64315 71.798528 \nL 138.194864 72.492937 \nL 141.756344 72.90651 \nL 145.33001 73.038323 \nL 148.918309 72.88707 \nL 152.523729 72.451056 \nL 156.148796 71.728191 \nL 159.796089 70.715978 \nL 163.46824 69.411508 \nL 167.167945 67.811444 \nL 170.897966 65.912007 \nL 174.661143 63.708963 \nL 178.460401 61.197607 \nL 182.298756 58.372739 \nL 186.179323 55.228647 \nL 190.105331 51.759082 \nL 194.080125 47.957234 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 89.000935 31.641811 \nL 92.636924 36.356932 \nL 96.250301 40.765997 \nL 99.843571 44.873461 \nL 103.419195 48.683353 \nL 106.9796 52.199287 \nL 110.527179 55.424474 \nL 114.064303 58.36173 \nL 117.593321 61.013482 \nL 121.116566 63.381779 \nL 124.636362 65.468291 \nL 128.155029 67.27432 \nL 131.674887 68.800797 \nL 135.198261 70.048285 \nL 138.727487 71.016981 \nL 142.264916 71.706715 \nL 145.812921 72.116943 \nL 149.373902 72.24675 \nL 152.950288 72.094842 \nL 156.54455 71.659541 \nL 160.159198 70.938777 \nL 163.796795 69.930079 \nL 167.459958 68.630565 \nL 171.151366 67.036933 \nL 174.873769 65.145442 \nL 178.629993 62.951902 \nL 182.422946 60.451657 \nL 186.255632 57.63956 \nL 190.131153 54.509962 \nL 194.052723 51.05668 \nL 198.023674 47.272975 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 93.373466 32.234237 \nL 96.978446 36.91874 \nL 100.562017 41.299331 \nL 104.126644 45.380407 \nL 107.67475 49.165947 \nL 111.208722 52.659517 \nL 114.730921 55.86429 \nL 118.243681 58.783043 \nL 121.74932 61.418175 \nL 125.250141 63.771709 \nL 128.748438 65.845295 \nL 132.246502 67.640217 \nL 135.746627 69.157396 \nL 139.251111 70.397388 \nL 142.762265 71.360388 \nL 146.282416 72.046227 \nL 149.813915 72.454371 \nL 153.359136 72.583916 \nL 156.92049 72.433584 \nL 160.500424 72.001719 \nL 164.101428 71.286278 \nL 167.726044 70.284822 \nL 171.376869 68.994508 \nL 175.056563 67.412076 \nL 178.767855 65.533834 \nL 182.513551 63.35565 \nL 186.296542 60.872926 \nL 190.119809 58.080588 \nL 193.986435 54.973062 \nL 197.899615 51.544249 \nL 201.862659 47.787504 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 97.725166 33.967842 \nL 101.296438 38.615903 \nL 104.847515 42.962699 \nL 108.380813 47.012563 \nL 111.898714 50.769412 \nL 115.403567 54.236762 \nL 118.89769 57.417736 \nL 122.383382 60.315073 \nL 125.862926 62.931136 \nL 129.338588 65.267918 \nL 132.812633 67.327047 \nL 136.287318 69.109789 \nL 139.764906 70.617051 \nL 143.247666 71.849385 \nL 146.737881 72.806983 \nL 150.237851 73.489681 \nL 153.749897 73.896953 \nL 157.27637 74.027911 \nL 160.819652 73.881299 \nL 164.382165 73.455488 \nL 167.966375 72.748467 \nL 171.574799 71.757837 \nL 175.210008 70.4808 \nL 178.874638 68.914148 \nL 182.571394 67.054251 \nL 186.303056 64.897041 \nL 190.072489 62.437997 \nL 193.88265 59.672126 \nL 197.736596 56.593946 \nL 201.637494 53.197459 \nL 205.588627 49.47613 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 102.045142 36.814003 \nL 105.580176 41.420067 \nL 109.096226 45.727995 \nL 112.595663 49.742044 \nL 116.080822 53.466064 \nL 119.554007 56.903512 \nL 123.017497 60.057458 \nL 126.473549 62.930596 \nL 129.924407 65.52525 \nL 133.372302 67.84338 \nL 136.819459 69.886589 \nL 140.268104 71.656122 \nL 143.720464 73.152876 \nL 147.178777 74.377393 \nL 150.645292 75.329865 \nL 154.122278 76.010133 \nL 157.612027 76.417686 \nL 161.116858 76.551651 \nL 164.639124 76.4108 \nL 168.181217 75.993533 \nL 171.745576 75.297879 \nL 175.334685 74.321485 \nL 178.951089 73.061606 \nL 182.597393 71.515095 \nL 186.276272 69.67839 \nL 189.990478 67.547501 \nL 193.742844 65.117993 \nL 197.536297 62.38497 \nL 201.373861 59.343053 \nL 205.258669 55.986361 \nL 209.193973 52.308487 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 106.322991 40.739201 \nL 109.819427 45.298016 \nL 113.298086 49.562274 \nL 116.761289 53.536151 \nL 120.211322 57.223425 \nL 123.650444 60.627484 \nL 127.080887 63.751343 \nL 130.504868 66.597644 \nL 133.924588 69.168669 \nL 137.342237 71.466343 \nL 140.760003 73.49224 \nL 144.180072 75.247586 \nL 147.604637 76.733261 \nL 151.035898 77.949803 \nL 154.476069 78.897402 \nL 157.927386 79.575908 \nL 161.392104 79.984821 \nL 164.872511 80.123292 \nL 168.370926 79.990121 \nL 171.889708 79.583743 \nL 175.431261 78.902233 \nL 178.998039 77.943288 \nL 182.592552 76.704223 \nL 186.217371 75.181961 \nL 189.875137 73.373016 \nL 193.568568 71.273485 \nL 197.300461 68.879031 \nL 201.073706 66.184864 \nL 204.891291 63.185723 \nL 208.756309 59.875857 \nL 212.671972 56.249001 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 110.548865 45.705501 \nL 114.00452 50.212137 \nL 117.443595 54.428221 \nL 120.868358 58.357837 \nL 124.281044 62.004683 \nL 127.683863 65.372077 \nL 131.079001 68.462971 \nL 134.468627 71.279952 \nL 137.8549 73.825258 \nL 141.239968 76.100774 \nL 144.625975 78.108045 \nL 148.01507 79.848274 \nL 151.409403 81.322328 \nL 154.811137 82.530734 \nL 158.222449 83.473686 \nL 161.645534 84.151041 \nL 165.082613 84.562315 \nL 168.535936 84.706684 \nL 172.007786 84.582978 \nL 175.500485 84.189674 \nL 179.0164 83.524894 \nL 182.557948 82.586392 \nL 186.127602 81.371551 \nL 189.727896 79.877367 \nL 193.361432 78.100442 \nL 197.030888 76.036967 \nL 200.739022 73.682711 \nL 204.488683 71.033 \nL 208.282814 68.082706 \nL 212.124467 64.826218 \nL 216.016805 61.257427 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n    <path clip-path=\"url(#p6a4ab83ea0)\" d=\"M 114.713531 51.671053 \nL 118.126404 56.120929 \nL 121.523878 60.284647 \nL 124.908166 64.166195 \nL 128.281449 67.769186 \nL 131.645887 71.096862 \nL 135.003618 74.152106 \nL 138.356762 76.937451 \nL 141.707431 79.455082 \nL 145.057729 81.706847 \nL 148.409757 83.694258 \nL 151.765619 85.418493 \nL 155.127425 86.880404 \nL 158.497296 88.080512 \nL 161.877367 89.019012 \nL 165.269795 89.695769 \nL 168.67676 90.110317 \nL 172.100471 90.261859 \nL 175.543173 90.149259 \nL 179.007148 89.771038 \nL 182.494723 89.12537 \nL 186.008274 88.210072 \nL 189.550234 87.022597 \nL 193.123094 85.560025 \nL 196.729416 83.819047 \nL 200.371832 81.795959 \nL 204.053058 79.486644 \nL 207.775894 76.886556 \nL 211.543239 73.990704 \nL 215.358093 70.793632 \nL 219.22357 67.289397 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n   </g>\n  </g>\n </g>\n <defs>\n  <clipPath id=\"p6a4ab83ea0\">\n   <rect height=\"135.9\" width=\"251.1\" x=\"7.2\" y=\"7.2\"/>\n  </clipPath>\n </defs>\n</svg>\n"
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PXqOqOZn_wF5",
        "colab_type": "text"
      },
      "source": [
        "在图的鞍点位置，目标函数在 $x$ 轴方向上是局部最小值，但在$ y$ 轴方向上是局部最大值。\n",
        "\n",
        "假设一个函数的输入为 $k$ 维向量，输出为标量，那么它的海森矩阵（Hessian matrix）有 $k$ 个特征值（参见附录中“数学基础”一节）。该函数在梯度为0的位置上可能是局部最小值、局部最大值或者鞍点。\n",
        "\n",
        "* 当函数的海森矩阵在梯度为零的位置上的特征值全为正时，该函数得到局部最小值。\n",
        "* 当函数的海森矩阵在梯度为零的位置上的特征值全为负时，该函数得到局部最大值。\n",
        "* 当函数的海森矩阵在梯度为零的位置上的特征值有正有负时，该函数得到鞍点。\n",
        "随机矩阵理论告诉我们，对于一个大的高斯随机矩阵来说，任一特征值是正或者是负的概率都是0.5 [1]。那么，以上第一种情况的概率为  $0.5^k$ 。由于深度学习模型参数通常都是高维的（ $k$ 很大），目标函数的鞍点通常比局部最小值更常见。\n",
        "\n",
        "在深度学习中，虽然找到目标函数的全局最优解很难，但这并非必要。我们将在本章接下来的几节中逐一介绍深度学习中常用的优化算法，它们在很多实际问题中都能够训练出十分有效的深度学习模型。\n",
        "\n",
        "###7.1.3. 小结\n",
        "* 由于优化算法的目标函数通常是一个基于训练数据集的损失函数，优化的目标在于降低训练误差。\n",
        "* 由于深度学习模型参数通常都是高维的，目标函数的鞍点通常比局部最小值更常见。"
      ]
    }
  ]
}