Basic-Tutorials/handson201809-vcp/201-GPUインスタンスの利用.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea488498-c2e9-11e8-9925-0242ac110002",
     "next": "ea4886aa-c2e9-11e8-9925-0242ac110002",
     "previous": null
    }
   },
   "source": [
    "# GPUの利用"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea4886aa-c2e9-11e8-9925-0242ac110002",
     "next": "ea488786-c2e9-11e8-9925-0242ac110002",
     "previous": "ea488498-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "## GPU利用のために必要となる設定"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea488786-c2e9-11e8-9925-0242ac110002",
     "next": "ea488826-c2e9-11e8-9925-0242ac110002",
     "previous": "ea4886aa-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "### インスタンスタイプ、VMサイズ\n",
    "\n",
    "GPUが利用できるEC2インスタンスタイプ(Amazon)、VMサイズ(Microsoft Azure)を指定する必要があります。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea488826-c2e9-11e8-9925-0242ac110002",
     "next": "ea4888c6-c2e9-11e8-9925-0242ac110002",
     "previous": "ea488786-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "Amazon EC2 であれば、以下のものが候補になります。\n",
    "\n",
    "* [p3インスタンス](https://aws.amazon.com/jp/ec2/instance-types/p3/)\n",
    "* [p2インスタンス](https://aws.amazon.com/jp/ec2/instance-types/p2/)\n",
    "* [g3インスタンス](https://aws.amazon.com/jp/ec2/instance-types/g3/)\n",
    "\n",
    "Microsoft Azure VM であれば、以下のものが候補になります。\n",
    "\n",
    "* [NVシリーズ](https://docs.microsoft.com/ja-jp/azure/virtual-machines/windows/sizes-gpu#nv-series)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea4888c6-c2e9-11e8-9925-0242ac110002",
     "next": "ea488966-c2e9-11e8-9925-0242ac110002",
     "previous": "ea488826-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "### ドライバ、ライブラリなど"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea488966-c2e9-11e8-9925-0242ac110002",
     "next": "ea4889fc-c2e9-11e8-9925-0242ac110002",
     "previous": "ea4888c6-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "GPUを利用するためには、GPUドライバ、ライブラリなどを準備する必要があります。\n",
    "\n",
    "VCPではGPUドライバなどをVCノードの構成に合わせた形でセットアップしたものを事前に用意してあります。そのため通常のVCノードを利用する場合に`spec`指定を行ったのと同様の手順で、いくつかのパラメータを `spec`に指定するだけでGPUを利用することが出来ます。\n",
    "\n",
    "GPUを利用しない場合（下図左側）と、GPUを利用する場合（下図右側）のVCノードの構成を以下に示します。\n",
    "\n",
    "![GPUコンテナ](images/201-001.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea4889fc-c2e9-11e8-9925-0242ac110002",
     "next": "ea488a9c-c2e9-11e8-9925-0242ac110002",
     "previous": "ea488966-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "GPUを利用しない場合との違いを以下に示します。\n",
    "* OS（マシンイメージ）に nvidia ドライバをインストールしている\n",
    "* Baseコンテナで[NVIDIA Container Runtime for Docker(nvidia-docker)](https://github.com/NVIDIA/nvidia-docker)を利用できるようにセットアップしている\n",
    "\n",
    "nvidia ドライバはカーネルモジュールになっているため、VMのカーネルバージョンに強く依存しています。そのためnvidiaドライバはBaseコンテナではなくOS（マシンイメージ）にインストールしています。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea488a9c-c2e9-11e8-9925-0242ac110002",
     "next": "ea488b3c-c2e9-11e8-9925-0242ac110002",
     "previous": "ea4889fc-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "## 操作手順"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea488b3c-c2e9-11e8-9925-0242ac110002",
     "next": "ea488bd2-c2e9-11e8-9925-0242ac110002",
     "previous": "ea488a9c-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "この Notebook では以下の操作を行います。\n",
    "\n",
    "1. GPU用のVCノードを起動する\n",
    "1. GPUを利用するアプリケーションコンテナを起動する"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea488bd2-c2e9-11e8-9925-0242ac110002",
     "next": "ea488c68-c2e9-11e8-9925-0242ac110002",
     "previous": "ea488b3c-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "# GPU用VCノードを起動する\n",
    "\n",
    "GPUを利用するために必要となるパラメータを `spec`に指定して、VCノードを起動する。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea488c68-c2e9-11e8-9925-0242ac110002",
     "next": "ea488d08-c2e9-11e8-9925-0242ac110002",
     "previous": "ea488bd2-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "## アクセストークンの入力"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea488d08-c2e9-11e8-9925-0242ac110002",
     "next": "ea488dda-c2e9-11e8-9925-0242ac110002",
     "previous": "ea488c68-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "VCP SDKを利用するにはVC Controllerのアクセストークンが必要となります。次のセルを実行すると入力枠が表示されるのでアクセストークンの値を入力してください。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea488dda-c2e9-11e8-9925-0242ac110002",
     "next": "ea488e70-c2e9-11e8-9925-0242ac110002",
     "previous": "ea488d08-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "········\n"
     ]
    }
   ],
   "source": [
    "from getpass import getpass\n",
    "vcc_access_token = getpass()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea488e70-c2e9-11e8-9925-0242ac110002",
     "next": "ea488f10-c2e9-11e8-9925-0242ac110002",
     "previous": "ea488dda-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "## VCP SDKの初期化\n",
    "\n",
    "VCP SDKの初期化を行います。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea488f10-c2e9-11e8-9925-0242ac110002",
     "next": "ea488fa6-c2e9-11e8-9925-0242ac110002",
     "previous": "ea488e70-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [],
   "source": [
    "from common import logsetting\n",
    "from vcpsdk.vcpsdk import VcpSDK\n",
    "\n",
    "# VCの管理オブジェクトの作成\n",
    "vcp = VcpSDK(\n",
    "    vcc_access_token,  # VCCのアクセストークン\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea488fa6-c2e9-11e8-9925-0242ac110002",
     "next": "ea489046-c2e9-11e8-9925-0242ac110002",
     "previous": "ea488f10-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "上のセルの実行結果がエラーとなり以下のようなメッセージが表示されている場合は、入力されたアクセストークンに誤りがあります。\n",
    "\n",
    "```\n",
    "2018-09-XX XX:XX:XX,XXX - ERROR - config vc failed: http_status(403)\n",
    "2018-09-XX XX:XX:XX,XXX - ERROR - 2018/XX/XX XX:XX:XX UTC: VCPAuthException: xxxxxxx:token lookup is failed: permission denied\n",
    "```\n",
    "\n",
    "この場合はアクセストークンの入力からやり直してください。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea489046-c2e9-11e8-9925-0242ac110002",
     "next": "ea4890e6-c2e9-11e8-9925-0242ac110002",
     "previous": "ea488fa6-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "## VCノードのspecを指定する\n",
    "\n",
    "GPUを利用するVCノードのspecを指定します。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea4890e6-c2e9-11e8-9925-0242ac110002",
     "next": "ea489186-c2e9-11e8-9925-0242ac110002",
     "previous": "ea489046-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "GPUを利用するためには`spec`に以下の設定を行う必要があります。\n",
    "\n",
    "* GPUを利用できるインスタンスタイプを指定する\n",
    "* GPU用のBaseコンテナイメージを指定する\n",
    "* GPU用の仮想マシンのイメージを指定する"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea489186-c2e9-11e8-9925-0242ac110002",
     "next": "ea48921c-c2e9-11e8-9925-0242ac110002",
     "previous": "ea4890e6-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "### specの取得"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48921c-c2e9-11e8-9925-0242ac110002",
     "next": "ea4892b2-c2e9-11e8-9925-0242ac110002",
     "previous": "ea489186-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "プロバイダと `flavor`を指定して `spec`オブジェクトを取得します。ここでは以下の値を指定します。\n",
    "\n",
    "* プロバイダ: `aws`\n",
    "* flavor: `medium`\n",
    "\n",
    "> GPUを利用する `flavor` は定義されていないので、後で`spec`オブジェクトに直接インスタンスタイプの指定を行います。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea4892b2-c2e9-11e8-9925-0242ac110002",
     "next": "ea48933e-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48921c-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [],
   "source": [
    "spec = VcpSDK.get_spec(\n",
    "    'aws',    # プロバイダ\n",
    "    'medium'  # flavor\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48933e-c2e9-11e8-9925-0242ac110002",
     "next": "ea4893d4-c2e9-11e8-9925-0242ac110002",
     "previous": "ea4892b2-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "### インスタンスタイプの設定\n",
    "\n",
    "GPU を利用できるインスタンスタイプを指定します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea4893d4-c2e9-11e8-9925-0242ac110002",
     "next": "ea48946a-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48933e-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [],
   "source": [
    "spec.instance_type = 'p2.xlarge'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48946a-c2e9-11e8-9925-0242ac110002",
     "next": "ea48950a-c2e9-11e8-9925-0242ac110002",
     "previous": "ea4893d4-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "### Baseコンテナイメージの設定\n",
    "\n",
    "GPU用のBaseコンテナイメージを`spec`オブジェクトに指定します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea48950a-c2e9-11e8-9925-0242ac110002",
     "next": "ea4895aa-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48946a-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [],
   "source": [
    "spec.image = 'gpu-base:nvidia-384'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea4895aa-c2e9-11e8-9925-0242ac110002",
     "next": "ea48965e-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48950a-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "この`gpu-base:nvidia-384`を利用すると、Baseコンテナで[nvidia-docker2](https://github.com/NVIDIA/nvidia-docker)が利用できるようになります。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48965e-c2e9-11e8-9925-0242ac110002",
     "next": "ea4896fe-c2e9-11e8-9925-0242ac110002",
     "previous": "ea4895aa-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "### 仮想マシンイメージの設定\n",
    "\n",
    "GPU用の仮想マシンイメージを`spec`オブジェクトに指定します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea4896fe-c2e9-11e8-9925-0242ac110002",
     "next": "ea489794-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48965e-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [],
   "source": [
    "spec.cloud_image = 'niivcp-gpu-20180903'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea489794-c2e9-11e8-9925-0242ac110002",
     "next": "ea489820-c2e9-11e8-9925-0242ac110002",
     "previous": "ea4896fe-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "GPUを利用するための nvidiaドライバはカーネルモジュールを含んでいます。そのためnvidiaドライバは仮想マシンのカーネルバージョンに合ったものが必要となります。そこで仮想マシン環境に合ったnvidiaドライバを事前にインストールした仮想マシンイメージを用意しました。ここではその仮想マシンイメージ名を指定しています。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea489820-c2e9-11e8-9925-0242ac110002",
     "next": "ea4898b6-c2e9-11e8-9925-0242ac110002",
     "previous": "ea489794-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "### sshの鍵ファイルを設定する\n",
    "\n",
    "VCノードにsshでログインするためには事前に公開鍵認証の鍵を VCノードに登録しておく必要があり、そのための設定をここで行います。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea4898b6-c2e9-11e8-9925-0242ac110002",
     "next": "ea48994c-c2e9-11e8-9925-0242ac110002",
     "previous": "ea489820-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "VCノードに登録する公開鍵認証の**秘密鍵**のパスを次のセルで設定してください。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea48994c-c2e9-11e8-9925-0242ac110002",
     "next": "ea4899ec-c2e9-11e8-9925-0242ac110002",
     "previous": "ea4898b6-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "ssh_private_key = os.path.expanduser('~/.ssh/id_rsa')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea4899ec-c2e9-11e8-9925-0242ac110002",
     "next": "ea489a82-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48994c-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "spec に設定が必要となるのは公開鍵なので `ssh-keygen -y` コマンドで秘密鍵から公開鍵を取り出します。さらに公開鍵を`spec`が要求するフォーマット(Base64)に変換して `spec` オブジェクトに設定しています。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea489a82-c2e9-11e8-9925-0242ac110002",
     "next": "ea489b0e-c2e9-11e8-9925-0242ac110002",
     "previous": "ea4899ec-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [],
   "source": [
    "out = !ssh-keygen -y -f {ssh_private_key} | base64 -w 0\n",
    "b64_ssh_public_key = out[0]\n",
    "\n",
    "# specオブジェクトに設定する\n",
    "spec.params_e = ['AUTHORIZED_KEYS={}'.format(b64_ssh_public_key)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea489b0e-c2e9-11e8-9925-0242ac110002",
     "next": "ea489ba4-c2e9-11e8-9925-0242ac110002",
     "previous": "ea489a82-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "### specの設定内容の確認\n",
    "\n",
    "ここまで `spec` に設定した内容を確認してみます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea489ba4-c2e9-11e8-9925-0242ac110002",
     "next": "ea489c3a-c2e9-11e8-9925-0242ac110002",
     "previous": "ea489b0e-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "========================\n",
      "aws\n",
      "------------------------\n",
      "\n",
      "    cci_version: \"1.0\"\n",
      "    name: \n",
      "    description: \n",
      "    private_network: default\n",
      "    num_nodes: 1\n",
      "    image: gpu-base:nvidia-384\n",
      "    recovery_mode: none\n",
      "    params:\n",
      "        {'e': ['AUTHORIZED_KEYS=c3NoLXJzYSBBQUFBQjNOemFDMXljMkVBQUFBREFRQUJBQUFCQVFDdE5KYjBIWkU0SHR0ZjV3WWhhbkQyR2RBZm9Wc0o2eVZOR3JMOWFUQ040SWtXUGZ1a1ZOOFdUTldldHp4T3cwOXlrZlZqQlFZUG9JOWhzOHYzd2hjaHRqZlFneHEvWTBvTWFLNlQ1bmdybmtMZFd6VFcxSFhhZ3A0SFdBTGtvVGNtQkhUaG9lWitKdGdkbS9ham8ySHVNblhqbHY0Y215WDVQZ3JhdU5IVDZoTEJUc2k2NzBkNWFSTTZLY3B1VVc0RS8xMHF1WklQcXhaMzJvQnFxU2V6c1gzVlV2Nnk2Um1Ldzg3eC9FUU1XVUtlemR3bHJuTWpwY3p0TTRSQzVrc0s1QWp6bC9oY3FaOGVOMjhaanVxM3RPRHNWY05SVHdzajdGWTZjaXVsbFhURndzZHhjNjM5U290VmpwWWNZSUpqWGhZTW1mTkNpdFFRamxaNVNVdUgK'], 'v': []}\n",
      "    instance_type: p2.xlarge\n",
      "    volume_size: 40\n",
      "    volume_type: standard\n",
      "    volume_id: []\n",
      "    cloud_image: niivcp-gpu-20180903\n",
      "    tags: {}\n",
      "========================\n"
     ]
    }
   ],
   "source": [
    "print(spec)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "lc_cell_meme": {
     "current": "ea489c3a-c2e9-11e8-9925-0242ac110002",
     "next": "ea489cd0-c2e9-11e8-9925-0242ac110002",
     "previous": "ea489ba4-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "## VCノードの起動\n",
    "\n",
    "VCノードの起動と、VCノードを操作するためのAnsible設定を行います。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea489cd0-c2e9-11e8-9925-0242ac110002",
     "next": "ea489d66-c2e9-11e8-9925-0242ac110002",
     "previous": "ea489c3a-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "### Unitの作成とVCノードの起動\n",
    "\n",
    "Unitの作成とVCノードの起動を行います。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea489d66-c2e9-11e8-9925-0242ac110002",
     "next": "ea489dfc-c2e9-11e8-9925-0242ac110002",
     "previous": "ea489cd0-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "まずはUnitGroupを作成します。UnitGroupの名前は `handson201`とします。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "deletable": false,
    "editable": false,
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea489dfc-c2e9-11e8-9925-0242ac110002",
     "next": "ea489e92-c2e9-11e8-9925-0242ac110002",
     "previous": "ea489d66-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [],
   "source": [
    "unit_group = vcp.create_ugroup(\n",
    "    'handson201'                   # UnitGroupの名前\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea489e92-c2e9-11e8-9925-0242ac110002",
     "next": "ea489f28-c2e9-11e8-9925-0242ac110002",
     "previous": "ea489dfc-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "VCノードを起動する前のUnitとVCノードの状態を確認しておきます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "deletable": false,
    "editable": false,
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea489f28-c2e9-11e8-9925-0242ac110002",
     "next": "ea489fb4-c2e9-11e8-9925-0242ac110002",
     "previous": "ea489e92-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>vctype</th>\n",
       "      <th>vcno</th>\n",
       "      <th>vcname</th>\n",
       "      <th>vc_state</th>\n",
       "      <th>vcid</th>\n",
       "      <th>cdate</th>\n",
       "      <th>unit_name</th>\n",
       "      <th>unit_state</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [vctype, vcno, vcname, vc_state, vcid, cdate, unit_name, unit_state]\n",
       "Index: []"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "None"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.display import display\n",
    "\n",
    "# Unitの一覧を DataFrame で表示する\n",
    "display(unit_group.df_units())\n",
    "\n",
    "# VCノードの一覧を DataFrame で表示する\n",
    "display(unit_group.df_nodes())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea489fb4-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a04a-c2e9-11e8-9925-0242ac110002",
     "previous": "ea489f28-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "Unitの作成とVCノードの起動を行います。\n",
    "\n",
    "> 処理が完了するまで2分～7分程度かかります。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "deletable": false,
    "editable": false,
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea48a04a-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a0ea-c2e9-11e8-9925-0242ac110002",
     "previous": "ea489fb4-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2018-09-28 15:45:43,908 - INFO - BOOTING ... 0 sec\n",
      "2018-09-28 15:45:49,113 - INFO - BOOTING ... 5 sec\n",
      "2018-09-28 15:45:54,309 - INFO - BOOTING ... 10 sec\n",
      "2018-09-28 15:45:59,494 - INFO - BOOTING ... 15 sec\n",
      "2018-09-28 15:46:04,680 - INFO - BOOTING ... 20 sec\n",
      "2018-09-28 15:46:09,867 - INFO - BOOTING ... 25 sec\n",
      "2018-09-28 15:46:15,068 - INFO - BOOTING ... 30 sec\n",
      "2018-09-28 15:46:20,275 - INFO - BOOTING ... 35 sec\n",
      "2018-09-28 15:46:25,485 - INFO - BOOTING ... 40 sec\n",
      "2018-09-28 15:46:30,694 - INFO - BOOTING ... 45 sec\n",
      "2018-09-28 15:46:35,896 - INFO - BOOTING ... 50 sec\n",
      "2018-09-28 15:46:41,104 - INFO - BOOTING ... 55 sec\n",
      "2018-09-28 15:46:46,316 - INFO - BOOTING ... 60 sec\n",
      "2018-09-28 15:46:51,518 - INFO - BOOTING ... 65 sec\n",
      "2018-09-28 15:46:56,731 - INFO - BOOTING ... 70 sec\n",
      "2018-09-28 15:47:01,930 - INFO - BOOTING ... 75 sec\n",
      "2018-09-28 15:47:07,139 - INFO - BOOTING ... 80 sec\n",
      "2018-09-28 15:47:12,347 - INFO - BOOTING ... 85 sec\n",
      "2018-09-28 15:47:17,559 - INFO - BOOTING ... 90 sec\n",
      "2018-09-28 15:47:22,771 - INFO - BOOTING ... 95 sec\n",
      "2018-09-28 15:47:28,047 - INFO - BOOTING ... 100 sec\n",
      "2018-09-28 15:47:33,268 - INFO - BOOTING ... 105 sec\n",
      "2018-09-28 15:47:38,492 - INFO - BOOTING ... 110 sec\n",
      "2018-09-28 15:47:43,699 - INFO - BOOTING ... 115 sec\n",
      "2018-09-28 15:47:48,898 - INFO - BOOTING ... 120 sec\n",
      "2018-09-28 15:47:54,105 - INFO - BOOTING ... 125 sec\n",
      "2018-09-28 15:47:59,310 - INFO - BOOTING ... 130 sec\n",
      "2018-09-28 15:48:04,514 - INFO - BOOTING ... 135 sec\n",
      "2018-09-28 15:48:09,722 - INFO - BOOTING ... 140 sec\n",
      "2018-09-28 15:48:14,922 - INFO - BOOTING ... 145 sec\n",
      "2018-09-28 15:48:20,127 - INFO - BOOTING ... 150 sec\n",
      "2018-09-28 15:48:25,340 - INFO - BOOTING ... 155 sec\n",
      "2018-09-28 15:48:30,543 - INFO - BOOTING ... 160 sec\n",
      "2018-09-28 15:48:35,748 - INFO - BOOTING ... 165 sec\n",
      "2018-09-28 15:48:40,951 - INFO - BOOTING ... 170 sec\n",
      "2018-09-28 15:48:46,164 - INFO - BOOTING ... 175 sec\n",
      "2018-09-28 15:48:51,366 - INFO - BOOTING ... 180 sec\n",
      "2018-09-28 15:48:56,573 - INFO - BOOTING ... 185 sec\n",
      "2018-09-28 15:49:01,772 - INFO - BOOTING ... 190 sec\n",
      "2018-09-28 15:49:06,971 - INFO - BOOTING ... 195 sec\n",
      "2018-09-28 15:49:12,191 - INFO - BOOTING ... 200 sec\n",
      "2018-09-28 15:49:17,404 - INFO - BOOTING ... 205 sec\n",
      "2018-09-28 15:49:22,620 - INFO - unit gpu is RUNNING\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<vcpsdk.vcpsdk.VcpUnitClass at 0x7fbad5b1a3c8>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Unitの作成（同時に VCノードが作成される）\n",
    "unit_group.create_unit(\n",
    "    'gpu',         # Unit名\n",
    "    spec\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea48a0ea-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a180-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a04a-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "起動したUnit, VCノードの一覧を表示します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "deletable": false,
    "editable": false,
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea48a180-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a220-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a0ea-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>vctype</th>\n",
       "      <th>vcno</th>\n",
       "      <th>vcname</th>\n",
       "      <th>vc_state</th>\n",
       "      <th>vcid</th>\n",
       "      <th>cdate</th>\n",
       "      <th>unit_name</th>\n",
       "      <th>unit_state</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>compute</td>\n",
       "      <td>42</td>\n",
       "      <td>handson201</td>\n",
       "      <td>RUNNING</td>\n",
       "      <td>975de972...</td>\n",
       "      <td>2018/09/28 06:44:22 UTC</td>\n",
       "      <td>gpu</td>\n",
       "      <td>RUNNING</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    vctype  vcno      vcname vc_state         vcid                    cdate  \\\n",
       "0  compute    42  handson201  RUNNING  975de972...  2018/09/28 06:44:22 UTC   \n",
       "\n",
       "  unit_name unit_state  \n",
       "0       gpu    RUNNING  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>vcno</th>\n",
       "      <th>vcname</th>\n",
       "      <th>unit_name</th>\n",
       "      <th>unit_state</th>\n",
       "      <th>node_no</th>\n",
       "      <th>node_id</th>\n",
       "      <th>node_state</th>\n",
       "      <th>cloud_instance_address</th>\n",
       "      <th>cloud_instance_id</th>\n",
       "      <th>cloud_instance_name</th>\n",
       "      <th>volumes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>42</td>\n",
       "      <td>handson201</td>\n",
       "      <td>gpu</td>\n",
       "      <td>RUNNING</td>\n",
       "      <td>1</td>\n",
       "      <td>e3ce219d...</td>\n",
       "      <td>RUNNING</td>\n",
       "      <td>172.30.2.125</td>\n",
       "      <td>i-00c607c9b70af63bb</td>\n",
       "      <td>VCP-020661f0-975de972</td>\n",
       "      <td>none</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   vcno      vcname unit_name unit_state  node_no      node_id node_state  \\\n",
       "0    42  handson201       gpu    RUNNING        1  e3ce219d...    RUNNING   \n",
       "\n",
       "  cloud_instance_address    cloud_instance_id    cloud_instance_name volumes  \n",
       "0           172.30.2.125  i-00c607c9b70af63bb  VCP-020661f0-975de972    none  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Unitの一覧を DataFrame で表示する\n",
    "display(unit_group.df_units())\n",
    "\n",
    "# VCノードの一覧を DataFrame で表示する\n",
    "display(unit_group.df_nodes())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea48a220-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a2b6-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a180-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "### Ansible の設定\n",
    "\n",
    "VCノードの操作には Ansible を用いるので、その設定を行います。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea48a2b6-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a34c-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a220-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "まず、起動したVCノードのIPアドレスをインベントリに登録します。グループ名は `handson201` とします。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "deletable": false,
    "editable": false,
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea48a34c-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a3e2-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a2b6-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[handson201]\r\n",
      "172.30.2.125\r\n"
     ]
    }
   ],
   "source": [
    "# ./hosts に追記する\n",
    "with open('hosts', 'a') as f:\n",
    "    f.write(\"[{}]\\n\".format(unit_group.name))\n",
    "    for addr in unit_group.find_ip_addresses():\n",
    "        f.write(\"{}\\n\".format(addr))\n",
    "\n",
    "# ./hosts の内容を表示する\n",
    "!cat hosts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea48a3e2-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a478-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a34c-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "インベントリの内容は、次のセルを実行すると表示されるリンクから編集することができます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "deletable": false,
    "editable": false,
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea48a478-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a50e-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a3e2-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<a href=\"/red/edit/handson/vcp-tutorial/hosts\" target=\"_blank\">インベントリ</a>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%run scripts/nbutils.py\n",
    "nb_html_link(os.path.abspath('hosts'), 'インベントリ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea48a50e-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a5a4-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a478-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "カレントディレクトリの `hosts`ファイルをインベントリとして ansible が認識するようにする設定を `ansible.cfg`に書き込みます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "deletable": false,
    "editable": false,
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea48a5a4-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a63a-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a50e-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[defaults]\r\n",
      "inventory = /notebooks/notebook/handson/vcp-tutorial/hosts\r\n"
     ]
    }
   ],
   "source": [
    "if not os.path.exists('ansible.cfg'):\n",
    "    # ansible.cfgが存在しない場合のみ作成する\n",
    "    with open('ansible.cfg', 'w') as f:\n",
    "        f.write('''[defaults]\n",
    "inventory = {}\n",
    "'''.format(os.path.abspath('./hosts')))\n",
    "    \n",
    "!cat ansible.cfg"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea48a63a-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a6da-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a5a4-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "VCノードに接続する際のsshの設定を ansible の変数ファイルに格納しておきます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "deletable": false,
    "editable": false,
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea48a6da-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a766-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a63a-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ansible_ssh_extra_args: -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null\r\n",
      "ansible_ssh_private_key_file: /home/bit_kun/.ssh/id_rsa\r\n",
      "ansible_user: root\r\n"
     ]
    }
   ],
   "source": [
    "import yaml\n",
    "\n",
    "# ansibleでVCノードに接続する際の設定\n",
    "ansible_params = {\n",
    "    'ansible_user': 'root',\n",
    "    'ansible_ssh_extra_args': '-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null',\n",
    "    'ansible_ssh_private_key_file': os.path.abspath(ssh_private_key),\n",
    "}\n",
    "\n",
    "# group_vars への書き込み\n",
    "!mkdir -p group_vars\n",
    "vc_group_vars = os.path.abspath('group_vars/{}.yml'.format(unit_group.name))\n",
    "with open(vc_group_vars, 'w') as f:\n",
    "    yaml.dump(ansible_params, stream=f, default_flow_style=False)\n",
    "\n",
    "!cat {vc_group_vars}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea48a766-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a7fc-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a6da-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "疎通確認を行います。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "deletable": false,
    "editable": false,
    "hidden": true,
    "lc_cell_meme": {
     "current": "ea48a7fc-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a892-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a766-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32m172.30.2.125 | SUCCESS => {\u001b[0m\r\n",
      "\u001b[0;32m    \"changed\": false, \u001b[0m\r\n",
      "\u001b[0;32m    \"failed\": false, \u001b[0m\r\n",
      "\u001b[0;32m    \"ping\": \"pong\"\u001b[0m\r\n",
      "\u001b[0;32m}\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible {unit_group.name} -m ping"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48a892-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a928-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a7fc-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "# GPU環境の確認\n",
    "\n",
    "NVIDIA GPUではGPUデバイスを管理、監視するためのコマンド[nvidia-smi](https://developer.nvidia.com/nvidia-system-management-interface)が提供されています。ここでは、起動したVCノードのGPUに関する状態を取得するために`nvidia-smi`コマンドを実行してみます。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48a928-c2e9-11e8-9925-0242ac110002",
     "next": "ea48a9c8-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a892-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "`nvidia-smi` コマンドを利用するとGPU名やドライババージョン、GPUメモリ使用量、GPU使用率、GPUの温度などのGPUに関する情報を取得することができます。`nvidia-smi`コマンドが表示するサマリ情報の例を以下に示します。\n",
    "\n",
    "```\n",
    "+-----------------------------------------------------------------------------+\n",
    "| NVIDIA-SMI 384.130                Driver Version: 384.130                   |\n",
    "|-------------------------------+----------------------+----------------------+\n",
    "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
    "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
    "|===============================+======================+======================|\n",
    "|   0  Tesla K80           Off  | 00000000:00:1E.0 Off |                    0 |\n",
    "| N/A   51C    P0    59W / 149W |      0MiB / 11439MiB |     96%      Default |\n",
    "+-------------------------------+----------------------+----------------------+\n",
    "                                                                               \n",
    "+-----------------------------------------------------------------------------+\n",
    "| Processes:                                                       GPU Memory |\n",
    "|  GPU       PID   Type   Process name                             Usage      |\n",
    "|=============================================================================|\n",
    "|  No running processes found                                                 |\n",
    "+-----------------------------------------------------------------------------+\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48a9c8-c2e9-11e8-9925-0242ac110002",
     "next": "ea48aa5e-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a928-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "`nvidia-smi`コマンドは[CUDAコンテナ](https://hub.docker.com/r/nvidia/cuda/)の中で実行するので、まずコンテナイメージの取得を行います。\n",
    "\n",
    "> コンテナイメージの取得に１～３分程度かかります。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea48aa5e-c2e9-11e8-9925-0242ac110002",
     "next": "ea48aaf4-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48a9c8-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32m172.30.2.125 | SUCCESS | rc=0 >>\u001b[0m\r\n",
      "\u001b[0;32m9.0-base: Pulling from nvidia/cuda\u001b[0m\r\n",
      "\u001b[0;32m3b37166ec614: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32m504facff238f: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32mebbcacd28e10: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32mc7fb3351ecad: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32m2e3debadcbf7: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32m328373d1cf96: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32ma9379f1c8963: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32mcc8317b7e8dd: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32m2e3debadcbf7: Waiting\u001b[0m\r\n",
      "\u001b[0;32m328373d1cf96: Waiting\u001b[0m\r\n",
      "\u001b[0;32ma9379f1c8963: Waiting\u001b[0m\r\n",
      "\u001b[0;32mcc8317b7e8dd: Waiting\u001b[0m\r\n",
      "\u001b[0;32mc7fb3351ecad: Waiting\u001b[0m\r\n",
      "\u001b[0;32mebbcacd28e10: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32mebbcacd28e10: Download complete\u001b[0m\r\n",
      "\u001b[0;32m504facff238f: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32m504facff238f: Download complete\u001b[0m\r\n",
      "\u001b[0;32mc7fb3351ecad: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32mc7fb3351ecad: Download complete\u001b[0m\r\n",
      "\u001b[0;32m2e3debadcbf7: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32m2e3debadcbf7: Download complete\u001b[0m\r\n",
      "\u001b[0;32m3b37166ec614: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32m3b37166ec614: Download complete\u001b[0m\r\n",
      "\u001b[0;32m328373d1cf96: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32m328373d1cf96: Download complete\u001b[0m\r\n",
      "\u001b[0;32ma9379f1c8963: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32ma9379f1c8963: Download complete\u001b[0m\r\n",
      "\u001b[0;32mcc8317b7e8dd: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32mcc8317b7e8dd: Download complete\u001b[0m\r\n",
      "\u001b[0;32m3b37166ec614: Pull complete\u001b[0m\r\n",
      "\u001b[0;32m504facff238f: Pull complete\u001b[0m\r\n",
      "\u001b[0;32mebbcacd28e10: Pull complete\u001b[0m\r\n",
      "\u001b[0;32mc7fb3351ecad: Pull complete\u001b[0m\r\n",
      "\u001b[0;32m2e3debadcbf7: Pull complete\u001b[0m\r\n",
      "\u001b[0;32m328373d1cf96: Pull complete\u001b[0m\r\n",
      "\u001b[0;32ma9379f1c8963: Pull complete\u001b[0m\r\n",
      "\u001b[0;32mcc8317b7e8dd: Pull complete\u001b[0m\r\n",
      "\u001b[0;32mDigest: sha256:8cd42ff61ccd3b2129e59f487cfeb4039f398cd1ca5e6937a3deab78058f537c\u001b[0m\r\n",
      "\u001b[0;32mStatus: Downloaded newer image for nvidia/cuda:9.0-base\u001b[0m\r\n",
      "\u001b[0;32m\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible {unit_group.name} -a 'docker pull nvidia/cuda:9.0-base'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48aaf4-c2e9-11e8-9925-0242ac110002",
     "next": "ea48ab80-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48aa5e-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "実際にCUDAコンテナから`nvidia-smi` コマンドを実行してGPUに関するサマリ情報が表示させてみます。\n",
    "\n",
    "> コンテナを実行する際に`docker run`コマンドの引数に`--runtime=nvidia`オプションを追加しています。このオプションを指定することで、コンテナ内からGPUを利用するための処理が [nvidia-docker2](https://github.com/NVIDIA/nvidia-docker) によって実行されています。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea48ab80-c2e9-11e8-9925-0242ac110002",
     "next": "ea48ac16-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48aaf4-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32m172.30.2.125 | SUCCESS | rc=0 >>\u001b[0m\r\n",
      "\u001b[0;32mFri Sep 28 06:49:50 2018       \u001b[0m\r\n",
      "\u001b[0;32m+-----------------------------------------------------------------------------+\u001b[0m\r\n",
      "\u001b[0;32m| NVIDIA-SMI 384.130                Driver Version: 384.130                   |\u001b[0m\r\n",
      "\u001b[0;32m|-------------------------------+----------------------+----------------------+\u001b[0m\r\n",
      "\u001b[0;32m| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\u001b[0m\r\n",
      "\u001b[0;32m| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\u001b[0m\r\n",
      "\u001b[0;32m|===============================+======================+======================|\u001b[0m\r\n",
      "\u001b[0;32m|   0  Tesla K80           Off  | 00000000:00:1E.0 Off |                    0 |\u001b[0m\r\n",
      "\u001b[0;32m| N/A   40C    P0    80W / 149W |      0MiB / 11439MiB |     96%      Default |\u001b[0m\r\n",
      "\u001b[0;32m+-------------------------------+----------------------+----------------------+\u001b[0m\r\n",
      "\u001b[0;32m                                                                               \u001b[0m\r\n",
      "\u001b[0;32m+-----------------------------------------------------------------------------+\u001b[0m\r\n",
      "\u001b[0;32m| Processes:                                                       GPU Memory |\u001b[0m\r\n",
      "\u001b[0;32m|  GPU       PID   Type   Process name                             Usage      |\u001b[0m\r\n",
      "\u001b[0;32m|=============================================================================|\u001b[0m\r\n",
      "\u001b[0;32m|  No running processes found                                                 |\u001b[0m\r\n",
      "\u001b[0;32m+-----------------------------------------------------------------------------+\u001b[0m\r\n",
      "\u001b[0;32m\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible {unit_group.name} -a 'docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48ac16-c2e9-11e8-9925-0242ac110002",
     "next": "ea48acac-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48ab80-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "GPUに関する詳細情報を取得するために `--query`オプションを指定して `nvidia-smi` コマンドを実行してみます。\n",
    "\n",
    "> `nvidia-smi`コマンドの他のオプションについては [nvidia-smi documentation](http://developer.download.nvidia.com/compute/DCGM/docs/nvidia-smi-367.38.pdf)を参照してください。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea48acac-c2e9-11e8-9925-0242ac110002",
     "next": "ea48ad42-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48ac16-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32m172.30.2.125 | SUCCESS | rc=0 >>\u001b[0m\r\n",
      "\u001b[0;32m\u001b[0m\r\n",
      "\u001b[0;32m==============NVSMI LOG==============\u001b[0m\r\n",
      "\u001b[0;32m\u001b[0m\r\n",
      "\u001b[0;32mTimestamp                           : Fri Sep 28 06:49:57 2018\u001b[0m\r\n",
      "\u001b[0;32mDriver Version                      : 384.130\u001b[0m\r\n",
      "\u001b[0;32m\u001b[0m\r\n",
      "\u001b[0;32mAttached GPUs                       : 1\u001b[0m\r\n",
      "\u001b[0;32mGPU 00000000:00:1E.0\u001b[0m\r\n",
      "\u001b[0;32m    Product Name                    : Tesla K80\u001b[0m\r\n",
      "\u001b[0;32m    Product Brand                   : Tesla\u001b[0m\r\n",
      "\u001b[0;32m    Display Mode                    : Disabled\u001b[0m\r\n",
      "\u001b[0;32m    Display Active                  : Disabled\u001b[0m\r\n",
      "\u001b[0;32m    Persistence Mode                : Disabled\u001b[0m\r\n",
      "\u001b[0;32m    Accounting Mode                 : Disabled\u001b[0m\r\n",
      "\u001b[0;32m    Accounting Mode Buffer Size     : 1920\u001b[0m\r\n",
      "\u001b[0;32m    Driver Model\u001b[0m\r\n",
      "\u001b[0;32m        Current                     : N/A\u001b[0m\r\n",
      "\u001b[0;32m        Pending                     : N/A\u001b[0m\r\n",
      "\u001b[0;32m    Serial Number                   : 0320217038173\u001b[0m\r\n",
      "\u001b[0;32m    GPU UUID                        : GPU-cdfd3eb7-1561-4d38-f417-431b93da5d08\u001b[0m\r\n",
      "\u001b[0;32m    Minor Number                    : 0\u001b[0m\r\n",
      "\u001b[0;32m    VBIOS Version                   : 80.21.1F.00.02\u001b[0m\r\n",
      "\u001b[0;32m    MultiGPU Board                  : Yes\u001b[0m\r\n",
      "\u001b[0;32m    Board ID                        : 0x1e\u001b[0m\r\n",
      "\u001b[0;32m    GPU Part Number                 : 900-22080-0000-000\u001b[0m\r\n",
      "\u001b[0;32m    Inforom Version\u001b[0m\r\n",
      "\u001b[0;32m        Image Version               : 2080.0200.00.04\u001b[0m\r\n",
      "\u001b[0;32m        OEM Object                  : 1.1\u001b[0m\r\n",
      "\u001b[0;32m        ECC Object                  : 3.0\u001b[0m\r\n",
      "\u001b[0;32m        Power Management Object     : N/A\u001b[0m\r\n",
      "\u001b[0;32m    GPU Operation Mode\u001b[0m\r\n",
      "\u001b[0;32m        Current                     : N/A\u001b[0m\r\n",
      "\u001b[0;32m        Pending                     : N/A\u001b[0m\r\n",
      "\u001b[0;32m    GPU Virtualization Mode\u001b[0m\r\n",
      "\u001b[0;32m        Virtualization mode         : Pass-Through\u001b[0m\r\n",
      "\u001b[0;32m    PCI\u001b[0m\r\n",
      "\u001b[0;32m        Bus                         : 0x00\u001b[0m\r\n",
      "\u001b[0;32m        Device                      : 0x1E\u001b[0m\r\n",
      "\u001b[0;32m        Domain                      : 0x0000\u001b[0m\r\n",
      "\u001b[0;32m        Device Id                   : 0x102D10DE\u001b[0m\r\n",
      "\u001b[0;32m        Bus Id                      : 00000000:00:1E.0\u001b[0m\r\n",
      "\u001b[0;32m        Sub System Id               : 0x106C10DE\u001b[0m\r\n",
      "\u001b[0;32m        GPU Link Info\u001b[0m\r\n",
      "\u001b[0;32m            PCIe Generation\u001b[0m\r\n",
      "\u001b[0;32m                Max                 : 3\u001b[0m\r\n",
      "\u001b[0;32m                Current             : 3\u001b[0m\r\n",
      "\u001b[0;32m            Link Width\u001b[0m\r\n",
      "\u001b[0;32m                Max                 : 16x\u001b[0m\r\n",
      "\u001b[0;32m                Current             : 16x\u001b[0m\r\n",
      "\u001b[0;32m        Bridge Chip\u001b[0m\r\n",
      "\u001b[0;32m            Type                    : N/A\u001b[0m\r\n",
      "\u001b[0;32m            Firmware                : N/A\u001b[0m\r\n",
      "\u001b[0;32m        Replays since reset         : 0\u001b[0m\r\n",
      "\u001b[0;32m        Tx Throughput               : N/A\u001b[0m\r\n",
      "\u001b[0;32m        Rx Throughput               : N/A\u001b[0m\r\n",
      "\u001b[0;32m    Fan Speed                       : N/A\u001b[0m\r\n",
      "\u001b[0;32m    Performance State               : P0\u001b[0m\r\n",
      "\u001b[0;32m    Clocks Throttle Reasons\u001b[0m\r\n",
      "\u001b[0;32m        Idle                        : Not Active\u001b[0m\r\n",
      "\u001b[0;32m        Applications Clocks Setting : Not Active\u001b[0m\r\n",
      "\u001b[0;32m        SW Power Cap                : Not Active\u001b[0m\r\n",
      "\u001b[0;32m        HW Slowdown                 : Not Active\u001b[0m\r\n",
      "\u001b[0;32m        Sync Boost                  : Not Active\u001b[0m\r\n",
      "\u001b[0;32m        SW Thermal Slowdown         : Not Active\u001b[0m\r\n",
      "\u001b[0;32m    FB Memory Usage\u001b[0m\r\n",
      "\u001b[0;32m        Total                       : 11439 MiB\u001b[0m\r\n",
      "\u001b[0;32m        Used                        : 0 MiB\u001b[0m\r\n",
      "\u001b[0;32m        Free                        : 11439 MiB\u001b[0m\r\n",
      "\u001b[0;32m    BAR1 Memory Usage\u001b[0m\r\n",
      "\u001b[0;32m        Total                       : 16384 MiB\u001b[0m\r\n",
      "\u001b[0;32m        Used                        : 2 MiB\u001b[0m\r\n",
      "\u001b[0;32m        Free                        : 16382 MiB\u001b[0m\r\n",
      "\u001b[0;32m    Compute Mode                    : Default\u001b[0m\r\n",
      "\u001b[0;32m    Utilization\u001b[0m\r\n",
      "\u001b[0;32m        Gpu                         : 98 %\u001b[0m\r\n",
      "\u001b[0;32m        Memory                      : 6 %\u001b[0m\r\n",
      "\u001b[0;32m        Encoder                     : 0 %\u001b[0m\r\n",
      "\u001b[0;32m        Decoder                     : 0 %\u001b[0m\r\n",
      "\u001b[0;32m    Encoder Stats\u001b[0m\r\n",
      "\u001b[0;32m        Active Sessions             : 0\u001b[0m\r\n",
      "\u001b[0;32m        Average FPS                 : 0\u001b[0m\r\n",
      "\u001b[0;32m        Average Latency             : 0\u001b[0m\r\n",
      "\u001b[0;32m    Ecc Mode\u001b[0m\r\n",
      "\u001b[0;32m        Current                     : Enabled\u001b[0m\r\n",
      "\u001b[0;32m        Pending                     : Enabled\u001b[0m\r\n",
      "\u001b[0;32m    ECC Errors\u001b[0m\r\n",
      "\u001b[0;32m        Volatile\u001b[0m\r\n",
      "\u001b[0;32m            Single Bit            \u001b[0m\r\n",
      "\u001b[0;32m                Device Memory       : 0\u001b[0m\r\n",
      "\u001b[0;32m                Register File       : 0\u001b[0m\r\n",
      "\u001b[0;32m                L1 Cache            : 0\u001b[0m\r\n",
      "\u001b[0;32m                L2 Cache            : 0\u001b[0m\r\n",
      "\u001b[0;32m                Texture Memory      : 0\u001b[0m\r\n",
      "\u001b[0;32m                Texture Shared      : N/A\u001b[0m\r\n",
      "\u001b[0;32m                CBU                 : N/A\u001b[0m\r\n",
      "\u001b[0;32m                Total               : 0\u001b[0m\r\n",
      "\u001b[0;32m            Double Bit            \u001b[0m\r\n",
      "\u001b[0;32m                Device Memory       : 0\u001b[0m\r\n",
      "\u001b[0;32m                Register File       : 0\u001b[0m\r\n",
      "\u001b[0;32m                L1 Cache            : 0\u001b[0m\r\n",
      "\u001b[0;32m                L2 Cache            : 0\u001b[0m\r\n",
      "\u001b[0;32m                Texture Memory      : 0\u001b[0m\r\n",
      "\u001b[0;32m                Texture Shared      : N/A\u001b[0m\r\n",
      "\u001b[0;32m                CBU                 : N/A\u001b[0m\r\n",
      "\u001b[0;32m                Total               : 0\u001b[0m\r\n",
      "\u001b[0;32m        Aggregate\u001b[0m\r\n",
      "\u001b[0;32m            Single Bit            \u001b[0m\r\n",
      "\u001b[0;32m                Device Memory       : 0\u001b[0m\r\n",
      "\u001b[0;32m                Register File       : 0\u001b[0m\r\n",
      "\u001b[0;32m                L1 Cache            : 0\u001b[0m\r\n",
      "\u001b[0;32m                L2 Cache            : 0\u001b[0m\r\n",
      "\u001b[0;32m                Texture Memory      : 0\u001b[0m\r\n",
      "\u001b[0;32m                Texture Shared      : N/A\u001b[0m\r\n",
      "\u001b[0;32m                CBU                 : N/A\u001b[0m\r\n",
      "\u001b[0;32m                Total               : 0\u001b[0m\r\n",
      "\u001b[0;32m            Double Bit            \u001b[0m\r\n",
      "\u001b[0;32m                Device Memory       : 0\u001b[0m\r\n",
      "\u001b[0;32m                Register File       : 0\u001b[0m\r\n",
      "\u001b[0;32m                L1 Cache            : 0\u001b[0m\r\n",
      "\u001b[0;32m                L2 Cache            : 0\u001b[0m\r\n",
      "\u001b[0;32m                Texture Memory      : 0\u001b[0m\r\n",
      "\u001b[0;32m                Texture Shared      : N/A\u001b[0m\r\n",
      "\u001b[0;32m                CBU                 : N/A\u001b[0m\r\n",
      "\u001b[0;32m                Total               : 0\u001b[0m\r\n",
      "\u001b[0;32m    Retired Pages\u001b[0m\r\n",
      "\u001b[0;32m        Single Bit ECC              : 0\u001b[0m\r\n",
      "\u001b[0;32m        Double Bit ECC              : 0\u001b[0m\r\n",
      "\u001b[0;32m        Pending                     : No\u001b[0m\r\n",
      "\u001b[0;32m    Temperature\u001b[0m\r\n",
      "\u001b[0;32m        GPU Current Temp            : 41 C\u001b[0m\r\n",
      "\u001b[0;32m        GPU Shutdown Temp           : 110 C\u001b[0m\r\n",
      "\u001b[0;32m        GPU Slowdown Temp           : 88 C\u001b[0m\r\n",
      "\u001b[0;32m        GPU Max Operating Temp      : N/A\u001b[0m\r\n",
      "\u001b[0;32m        Memory Current Temp         : N/A\u001b[0m\r\n",
      "\u001b[0;32m        Memory Max Operating Temp   : N/A\u001b[0m\r\n",
      "\u001b[0;32m    Power Readings\u001b[0m\r\n",
      "\u001b[0;32m        Power Management            : Supported\u001b[0m\r\n",
      "\u001b[0;32m        Power Draw                  : 82.59 W\u001b[0m\r\n",
      "\u001b[0;32m        Power Limit                 : 149.00 W\u001b[0m\r\n",
      "\u001b[0;32m        Default Power Limit         : 149.00 W\u001b[0m\r\n",
      "\u001b[0;32m        Enforced Power Limit        : 149.00 W\u001b[0m\r\n",
      "\u001b[0;32m        Min Power Limit             : 100.00 W\u001b[0m\r\n",
      "\u001b[0;32m        Max Power Limit             : 175.00 W\u001b[0m\r\n",
      "\u001b[0;32m    Clocks\u001b[0m\r\n",
      "\u001b[0;32m        Graphics                    : 875 MHz\u001b[0m\r\n",
      "\u001b[0;32m        SM                          : 875 MHz\u001b[0m\r\n",
      "\u001b[0;32m        Memory                      : 2505 MHz\u001b[0m\r\n",
      "\u001b[0;32m        Video                       : 540 MHz\u001b[0m\r\n",
      "\u001b[0;32m    Applications Clocks\u001b[0m\r\n",
      "\u001b[0;32m        Graphics                    : 562 MHz\u001b[0m\r\n",
      "\u001b[0;32m        Memory                      : 2505 MHz\u001b[0m\r\n",
      "\u001b[0;32m    Default Applications Clocks\u001b[0m\r\n",
      "\u001b[0;32m        Graphics                    : 562 MHz\u001b[0m\r\n",
      "\u001b[0;32m        Memory                      : 2505 MHz\u001b[0m\r\n",
      "\u001b[0;32m    Max Clocks\u001b[0m\r\n",
      "\u001b[0;32m        Graphics                    : 875 MHz\u001b[0m\r\n",
      "\u001b[0;32m        SM                          : 875 MHz\u001b[0m\r\n",
      "\u001b[0;32m        Memory                      : 2505 MHz\u001b[0m\r\n",
      "\u001b[0;32m        Video                       : 540 MHz\u001b[0m\r\n",
      "\u001b[0;32m    Max Customer Boost Clocks\u001b[0m\r\n",
      "\u001b[0;32m        Graphics                    : N/A\u001b[0m\r\n",
      "\u001b[0;32m    Clock Policy\u001b[0m\r\n",
      "\u001b[0;32m        Auto Boost                  : On\u001b[0m\r\n",
      "\u001b[0;32m        Auto Boost Default          : On\u001b[0m\r\n",
      "\u001b[0;32m    Processes                       : None\u001b[0m\r\n",
      "\u001b[0;32m\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible {unit_group.name} -a 'docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi --query'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48ad42-c2e9-11e8-9925-0242ac110002",
     "next": "ea48add8-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48acac-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "# GPU環境の利用"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48add8-c2e9-11e8-9925-0242ac110002",
     "next": "ea48ae6e-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48ad42-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "## chainerコンテナの実行\n",
    "\n",
    "[chainer コンテナ](https://hub.docker.com/r/chainer/chainer/)をアプリケーションコンテナとして実行します。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48ae6e-c2e9-11e8-9925-0242ac110002",
     "next": "ea48af04-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48add8-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "まずコンテナイメージを取得します。\n",
    "\n",
    "> コンテナイメージの取得に２～３分程度かかります。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea48af04-c2e9-11e8-9925-0242ac110002",
     "next": "ea48afae-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48ae6e-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32m172.30.2.125 | SUCCESS | rc=0 >>\u001b[0m\r\n",
      "\u001b[0;32mv4.4.0-python3: Pulling from chainer/chainer\u001b[0m\r\n",
      "\u001b[0;32m8ee29e426c26: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32m6e83b260b73b: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32me26b65fd1143: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32m40dca07f8222: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32mb420ae9e10b3: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32md96ac2114ef0: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32me1e52a706380: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32m8c519f84b56d: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32m03bdaf98f8ba: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32m44afa2e72753: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32mb9bd6c9f8288: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32m5daa41137c52: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32m5e74e353d49f: Pulling fs layer\u001b[0m\r\n",
      "\u001b[0;32mb420ae9e10b3: Waiting\u001b[0m\r\n",
      "\u001b[0;32m03bdaf98f8ba: Waiting\u001b[0m\r\n",
      "\u001b[0;32md96ac2114ef0: Waiting\u001b[0m\r\n",
      "\u001b[0;32m5daa41137c52: Waiting\u001b[0m\r\n",
      "\u001b[0;32m44afa2e72753: Waiting\u001b[0m\r\n",
      "\u001b[0;32me1e52a706380: Waiting\u001b[0m\r\n",
      "\u001b[0;32m5e74e353d49f: Waiting\u001b[0m\r\n",
      "\u001b[0;32m8c519f84b56d: Waiting\u001b[0m\r\n",
      "\u001b[0;32mb9bd6c9f8288: Waiting\u001b[0m\r\n",
      "\u001b[0;32m40dca07f8222: Waiting\u001b[0m\r\n",
      "\u001b[0;32m6e83b260b73b: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32m6e83b260b73b: Download complete\u001b[0m\r\n",
      "\u001b[0;32me26b65fd1143: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32me26b65fd1143: Download complete\u001b[0m\r\n",
      "\u001b[0;32m8ee29e426c26: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32m8ee29e426c26: Download complete\u001b[0m\r\n",
      "\u001b[0;32m40dca07f8222: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32m40dca07f8222: Download complete\u001b[0m\r\n",
      "\u001b[0;32mb420ae9e10b3: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32mb420ae9e10b3: Download complete\u001b[0m\r\n",
      "\u001b[0;32md96ac2114ef0: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32md96ac2114ef0: Download complete\u001b[0m\r\n",
      "\u001b[0;32m8c519f84b56d: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32m8c519f84b56d: Download complete\u001b[0m\r\n",
      "\u001b[0;32m44afa2e72753: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32m44afa2e72753: Download complete\u001b[0m\r\n",
      "\u001b[0;32m8ee29e426c26: Pull complete\u001b[0m\r\n",
      "\u001b[0;32m6e83b260b73b: Pull complete\u001b[0m\r\n",
      "\u001b[0;32me26b65fd1143: Pull complete\u001b[0m\r\n",
      "\u001b[0;32m40dca07f8222: Pull complete\u001b[0m\r\n",
      "\u001b[0;32mb420ae9e10b3: Pull complete\u001b[0m\r\n",
      "\u001b[0;32md96ac2114ef0: Pull complete\u001b[0m\r\n",
      "\u001b[0;32mb9bd6c9f8288: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32mb9bd6c9f8288: Download complete\u001b[0m\r\n",
      "\u001b[0;32m5daa41137c52: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32m5daa41137c52: Download complete\u001b[0m\r\n",
      "\u001b[0;32m5e74e353d49f: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32m5e74e353d49f: Download complete\u001b[0m\r\n",
      "\u001b[0;32me1e52a706380: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32me1e52a706380: Download complete\u001b[0m\r\n",
      "\u001b[0;32me1e52a706380: Pull complete\u001b[0m\r\n",
      "\u001b[0;32m8c519f84b56d: Pull complete\u001b[0m\r\n",
      "\u001b[0;32m03bdaf98f8ba: Verifying Checksum\u001b[0m\r\n",
      "\u001b[0;32m03bdaf98f8ba: Download complete\u001b[0m\r\n",
      "\u001b[0;32m03bdaf98f8ba: Pull complete\u001b[0m\r\n",
      "\u001b[0;32m44afa2e72753: Pull complete\u001b[0m\r\n",
      "\u001b[0;32mb9bd6c9f8288: Pull complete\u001b[0m\r\n",
      "\u001b[0;32m5daa41137c52: Pull complete\u001b[0m\r\n",
      "\u001b[0;32m5e74e353d49f: Pull complete\u001b[0m\r\n",
      "\u001b[0;32mDigest: sha256:487dca7e3ff7b65ff85335cf6d34de9bf4b8c58a35468352305823b3cab6f414\u001b[0m\r\n",
      "\u001b[0;32mStatus: Downloaded newer image for chainer/chainer:v4.4.0-python3\u001b[0m\r\n",
      "\u001b[0;32m\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible {unit_group.name} -m shell \\\n",
    "    -a 'docker pull  chainer/chainer:v4.4.0-python3'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48afae-c2e9-11e8-9925-0242ac110002",
     "next": "ea48b04e-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48af04-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "実際に[chainer コンテナ](https://hub.docker.com/r/chainer/chainer/)を実行します。コンテナからGPUを利用するので、先ほどCUDAコンテナを実行した場合と同様に `--runtime=nvidia`オプションを指定します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea48b04e-c2e9-11e8-9925-0242ac110002",
     "next": "ea48b0e4-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48afae-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32m172.30.2.125 | SUCCESS | rc=0 >>\u001b[0m\r\n",
      "\u001b[0;32m07965e90148bacd4f1b0f727ac93cf1583939d06afe094eed37b03c4f42cad8c\u001b[0m\r\n",
      "\u001b[0;32m\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible {unit_group.name} -m shell \\\n",
    "    -a 'docker run --runtime=nvidia -td --name chainer --rm \\\n",
    "    -v $HOME/chainer:/opt chainer/chainer:v4.4.0-python3'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48b0e4-c2e9-11e8-9925-0242ac110002",
     "next": "ea48b17a-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48b04e-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "## train_mnist.py の実行\n",
    "\n",
    "[chainer コンテナ](https://hub.docker.com/r/chainer/chainer/)を実行している環境で `train_mnist.py`を実行してみます。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48b17a-c2e9-11e8-9925-0242ac110002",
     "next": "ea48b210-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48b0e4-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    " まず `train_mnist.py` のソースを取得するために https://github.com/chainer/chainer.git のソースツリーを取得します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea48b210-c2e9-11e8-9925-0242ac110002",
     "next": "ea48b2a6-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48b17a-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;33m172.30.2.125 | SUCCESS => {\u001b[0m\r\n",
      "\u001b[0;33m    \"after\": \"d20fb546e255abedb8edc0a93584b290a1a548d2\", \u001b[0m\r\n",
      "\u001b[0;33m    \"before\": null, \u001b[0m\r\n",
      "\u001b[0;33m    \"changed\": true, \u001b[0m\r\n",
      "\u001b[0;33m    \"failed\": false\u001b[0m\r\n",
      "\u001b[0;33m}\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible {unit_group.name} -m git \\\n",
    "    -a 'repo=https://github.com/chainer/chainer.git \\\n",
    "    dest=$HOME/chainer version=v4.4.0'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48b2a6-c2e9-11e8-9925-0242ac110002",
     "next": "ea48b33c-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48b210-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "`train_mnist.py`を実行するのに必要となるライブラリを chainerコンテナの環境にインストールします。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea48b33c-c2e9-11e8-9925-0242ac110002",
     "next": "ea48b3d2-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48b2a6-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32m172.30.2.125 | SUCCESS | rc=0 >>\u001b[0m\r\n",
      "\u001b[0;32mCollecting matplotlib\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m  Downloading https://files.pythonhosted.org/packages/7b/ca/8b55a66b7ce426329ab16419a7eee4eb35b5a3fbe0d002434b339a4a7b09/matplotlib-3.0.0-cp35-cp35m-manylinux1_x86_64.whl (12.8MB)\r",
      "\u001b[0m\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32m\u001b[?25l\r",
      "\u001b[K    0% |                                | 10kB 6.7MB/s eta 0:00:02\r",
      "\u001b[K    0% |                                | 20kB 2.6MB/s eta 0:00:05\r",
      "\u001b[K    0% |                                | 30kB 3.3MB/s eta 0:00:04\r",
      "\u001b[K    0% |                                | 40kB 3.1MB/s eta 0:00:05\r",
      "\u001b[K    0% |                                | 51kB 3.6MB/s eta 0:00:04\r",
      "\u001b[K    0% |                                | 61kB 4.3MB/s eta 0:00:04\r",
      "\u001b[K    0% |                                | 71kB 4.1MB/s eta 0:00:04\r",
      "\u001b[K    0% |                                | 81kB 4.6MB/s eta 0:00:03\r",
      "\u001b[K    0% |                                | 92kB 4.7MB/s eta 0:00:03\r",
      "\u001b[K    0% |                                | 102kB 4.9MB/s eta 0:00:03\r",
      "\u001b[K    0% |                                | 112kB 5.2MB/s eta 0:00:03\r",
      "\u001b[K    0% |                                | 122kB 6.5MB/s eta 0:00:02\r",
      "\u001b[K    1% |                                | 133kB 6.5MB/s eta 0:00:02\r",
      "\u001b[K    1% |                                | 143kB 8.6MB/s eta 0:00:02\r",
      "\u001b[K    1% |                                | 153kB 8.2MB/s eta 0:00:02\r",
      "\u001b[K    1% |                                | 163kB 8.2MB/s eta 0:00:02\r",
      "\u001b[K    1% |                                | 174kB 8.9MB/s eta 0:00:02\r",
      "\u001b[K    1% |                                | 184kB 7.9MB/s eta 0:00:02\r",
      "\u001b[K    1% |                                | 194kB 8.9MB/s eta 0:00:02\r",
      "\u001b[K    1% |                                | 204kB 8.2MB/s eta 0:00:02\r",
      "\u001b[K    1% |                                | 215kB 7.8MB/s eta 0:00:02\r",
      "\u001b[K    1% |                                | 225kB 9.5MB/s eta 0:00:02\r",
      "\u001b[K    1% |                                | 235kB 9.2MB/s eta 0:00:02\r",
      "\u001b[K    1% |                                | 245kB 9.2MB/s eta 0:00:02\r",
      "\u001b[K    1% |                                | 256kB 8.5MB/s eta 0:00:02\r",
      "\u001b[K    2% |                                | 266kB 7.8MB/s eta 0:00:02\r",
      "\u001b[K    2% |                                | 276kB 9.1MB/s eta 0:00:02\r",
      "\u001b[K    2% |                                | 286kB 9.4MB/s eta 0:00:02\r",
      "\u001b[K    2% |                                | 296kB 8.8MB/s eta 0:00:02\r",
      "\u001b[K    2% |                                | 307kB 11.0MB/s eta 0:00:02\r",
      "\u001b[K    2% |                                | 317kB 9.6MB/s eta 0:00:02\r",
      "\u001b[K    2% |                                | 327kB 9.6MB/s eta 0:00:02\r",
      "\u001b[K    2% |                                | 337kB 9.1MB/s eta 0:00:02\r",
      "\u001b[K    2% |                                | 348kB 8.9MB/s eta 0:00:02\r",
      "\u001b[K    2% |                                | 358kB 11.1MB/s eta 0:00:02\r",
      "\u001b[K    2% |                                | 368kB 10.4MB/s eta 0:00:02\r",
      "\u001b[K    2% |                                | 378kB 9.1MB/s eta 0:00:02\r",
      "\u001b[K    3% |                                | 389kB 10.1MB/s eta 0:00:02\r",
      "\u001b[K    3% |                                | 399kB 9.8MB/s eta 0:00:02\r",
      "\u001b[K    3% |#                               | 409kB 8.3MB/s eta 0:00:02\r",
      "\u001b[K    3% |#                               | 419kB 9.8MB/s eta 0:00:02\r",
      "\u001b[K    3% |#                               | 430kB 9.0MB/s eta 0:00:02\r",
      "\u001b[K    3% |#                               | 440kB 11.0MB/s eta 0:00:02\r",
      "\u001b[K    3% |#                               | 450kB 10.1MB/s eta 0:00:02\r",
      "\u001b[K    3% |#                               | 460kB 9.5MB/s eta 0:00:02\r",
      "\u001b[K    3% |#                               | 471kB 11.3MB/s eta 0:00:02\r",
      "\u001b[K    3% |#                               | 481kB 10.7MB/s eta 0:00:02\r",
      "\u001b[K    3% |#                               | 491kB 10.3MB/s eta 0:00:02\r",
      "\u001b[K    3% |#                               | 501kB 11.4MB/s eta 0:00:02\r",
      "\u001b[K    3% |#                               | 512kB 12.1MB/s eta 0:00:02\r",
      "\u001b[K    4% |#                               | 522kB 12.1MB/s eta 0:00:02\r",
      "\u001b[K    4% |#                               | 532kB 11.1MB/s eta 0:00:02\r",
      "\u001b[K    4% |#                               | 542kB 9.9MB/s eta 0:00:02\r",
      "\u001b[K    4% |#                               | 552kB 11.1MB/s eta 0:00:02\r",
      "\u001b[K    4% |#                               | 563kB 11.8MB/s eta 0:00:02\r",
      "\u001b[K    4% |#                               | 573kB 10.3MB/s eta 0:00:02\r",
      "\u001b[K    4% |#                               | 583kB 13.1MB/s eta 0:00:01\r",
      "\u001b[K    4% |#                               | 593kB 11.1MB/s eta 0:00:02\r",
      "\u001b[K    4% |#                               | 604kB 11.1MB/s eta 0:00:02\r",
      "\u001b[K    4% |#                               | 614kB 12.0MB/s eta 0:00:02\r",
      "\u001b[K    4% |#                               | 624kB 11.3MB/s eta 0:00:02\r",
      "\u001b[K    4% |#                               | 634kB 13.9MB/s eta 0:00:01\r",
      "\u001b[K    5% |#                               | 645kB 11.0MB/s eta 0:00:02\r",
      "\u001b[K    5% |#                               | 655kB 10.6MB/s eta 0:00:02\r",
      "\u001b[K    5% |#                               | 665kB 10.7MB/s eta 0:00:02\r",
      "\u001b[K    5% |#                               | 675kB 11.5MB/s eta 0:00:02\r",
      "\u001b[K    5% |#                               | 686kB 11.5MB/s eta 0:00:02\r",
      "\u001b[K    5% |#                               | 696kB 13.4MB/s eta 0:00:01\r",
      "\u001b[K    5% |#                               | 706kB 9.9MB/s eta 0:00:02\r",
      "\u001b[K    5% |#                               | 716kB 10.7MB/s eta 0:00:02\r",
      "\u001b[K    5% |#                               | 727kB 11.2MB/s eta 0:00:02\r",
      "\u001b[K    5% |#                               | 737kB 9.7MB/s eta 0:00:02\r",
      "\u001b[K    5% |#                               | 747kB 13.5MB/s eta 0:00:01\r",
      "\u001b[K    5% |#                               | 757kB 14.1MB/s eta 0:00:01\r",
      "\u001b[K    5% |#                               | 768kB 14.0MB/s eta 0:00:01\r",
      "\u001b[K    6% |#                               | 778kB 14.2MB/s eta 0:00:01\r",
      "\u001b[K    6% |#                               | 788kB 10.9MB/s eta 0:00:02\r",
      "\u001b[K    6% |#                               | 798kB 11.6MB/s eta 0:00:02\r",
      "\u001b[K    6% |##                              | 808kB 16.3MB/s eta 0:00:01\r",
      "\u001b[K    6% |##                              | 819kB 13.1MB/s eta 0:00:01\r",
      "\u001b[K    6% |##                              | 829kB 13.3MB/s eta 0:00:01\r",
      "\u001b[K    6% |##                              | 839kB 15.3MB/s eta 0:00:01\r",
      "\u001b[K    6% |##                              | 849kB 15.5MB/s eta 0:00:01\r",
      "\u001b[K    6% |##                              | 860kB 12.1MB/s eta 0:00:01\r",
      "\u001b[K    6% |##                              | 870kB 11.6MB/s eta 0:00:02\r",
      "\u001b[K    6% |##                              | 880kB 12.3MB/s eta 0:00:01\r",
      "\u001b[K    6% |##                              | 890kB 15.0MB/s eta 0:00:01\r",
      "\u001b[K    7% |##                              | 901kB 14.9MB/s eta 0:00:01\r",
      "\u001b[K    7% |##                              | 911kB 15.3MB/s eta 0:00:01\r",
      "\u001b[K    7% |##                              | 921kB 17.7MB/s eta 0:00:01\r",
      "\u001b[K    7% |##                              | 931kB 17.7MB/s eta 0:00:01\r",
      "\u001b[K    7% |##                              | 942kB 17.0MB/s eta 0:00:01\r",
      "\u001b[K    7% |##                              | 952kB 12.2MB/s eta 0:00:01\r",
      "\u001b[K    7% |##                              | 962kB 15.6MB/s eta 0:00:01\r",
      "\u001b[K    7% |##                              | 972kB 14.4MB/s eta 0:00:01\r",
      "\u001b[K    7% |##                              | 983kB 14.2MB/s eta 0:00:01\r",
      "\u001b[K    7% |##                              | 993kB 15.7MB/s eta 0:00:01\r",
      "\u001b[K    7% |##                              | 1.0MB 14.4MB/s eta 0:00:01\r",
      "\u001b[K    7% |##                              | 1.0MB 14.2MB/s eta 0:00:01\r",
      "\u001b[K    7% |##                              | 1.0MB 12.5MB/s eta 0:00:01\r",
      "\u001b[K    8% |##                              | 1.0MB 12.4MB/s eta 0:00:01\r",
      "\u001b[K    8% |##                              | 1.0MB 13.4MB/s eta 0:00:01\r",
      "\u001b[K    8% |##                              | 1.1MB 16.9MB/s eta 0:00:01\r",
      "\u001b[K    8% |##                              | 1.1MB 15.0MB/s eta 0:00:01\r",
      "\u001b[K    8% |##                              | 1.1MB 17.7MB/s eta 0:00:01\r",
      "\u001b[K    8% |##                              | 1.1MB 15.1MB/s eta 0:00:01\r",
      "\u001b[K    8% |##                              | 1.1MB 14.8MB/s eta 0:00:01\r",
      "\u001b[K    8% |##                              | 1.1MB 16.3MB/s eta 0:00:01\r",
      "\u001b[K    8% |##                              | 1.1MB 13.1MB/s eta 0:00:01\r",
      "\u001b[K    8% |##                              | 1.1MB 16.4MB/s eta 0:00:01\r",
      "\u001b[K    8% |##                              | 1.1MB 16.5MB/s eta 0:00:01\r",
      "\u001b[K    8% |##                              | 1.1MB 13.3MB/s eta 0:00:01\r",
      "\u001b[K    9% |##                              | 1.2MB 14.9MB/s eta 0:00:01\r",
      "\u001b[K    9% |##                              | 1.2MB 16.4MB/s eta 0:00:01\r",
      "\u001b[K    9% |##                              | 1.2MB 16.5MB/s eta 0:00:01\r",
      "\u001b[K    9% |##                              | 1.2MB 17.0MB/s eta 0:00:01\r",
      "\u001b[K    9% |##                              | 1.2MB 17.3MB/s eta 0:00:01\r",
      "\u001b[K    9% |###                             | 1.2MB 17.4MB/s eta 0:00:01\r",
      "\u001b[K    9% |###                             | 1.2MB 17.8MB/s eta 0:00:01\r",
      "\u001b[K    9% |###                             | 1.2MB 17.8MB/s eta 0:00:01\r",
      "\u001b[K    9% |###                             | 1.2MB 17.8MB/s eta 0:00:01\r",
      "\u001b[K    9% |###                             | 1.2MB 19.1MB/s eta 0:00:01\r",
      "\u001b[K    9% |###                             | 1.3MB 19.0MB/s eta 0:00:01\r",
      "\u001b[K    9% |###                             | 1.3MB 18.7MB/s eta 0:00:01\r",
      "\u001b[K    9% |###                             | 1.3MB 15.5MB/s eta 0:00:01\r",
      "\u001b[K    10% |###                             | 1.3MB 17.8MB/s eta 0:00:01\r",
      "\u001b[K    10% |###                             | 1.3MB 17.8MB/s eta 0:00:01\r",
      "\u001b[K    10% |###                             | 1.3MB 14.4MB/s eta 0:00:01\r",
      "\u001b[K    10% |###                             | 1.3MB 18.1MB/s eta 0:00:01\r",
      "\u001b[K    10% |###                             | 1.3MB 15.0MB/s eta 0:00:01\r",
      "\u001b[K    10% |###                             | 1.3MB 15.0MB/s eta 0:00:01\r",
      "\u001b[K    10% |###                             | 1.4MB 17.2MB/s eta 0:00:01\r",
      "\u001b[K    10% |###                             | 1.4MB 17.2MB/s eta 0:00:01\r",
      "\u001b[K    10% |###                             | 1.4MB 17.8MB/s eta 0:00:01\r",
      "\u001b[K    10% |###                             | 1.4MB 18.8MB/s eta 0:00:01\r",
      "\u001b[K    10% |###                             | 1.4MB 17.8MB/s eta 0:00:01\r",
      "\u001b[K    10% |###                             | 1.4MB 17.8MB/s eta 0:00:01\r",
      "\u001b[K    11% |###                             | 1.4MB 18.8MB/s eta 0:00:01\r",
      "\u001b[K    11% |###                             | 1.4MB 18.9MB/s eta 0:00:01\r",
      "\u001b[K    11% |###                             | 1.4MB 24.2MB/s eta 0:00:01\r",
      "\u001b[K    11% |###                             | 1.4MB 17.5MB/s eta 0:00:01\r",
      "\u001b[K    11% |###                             | 1.5MB 18.3MB/s eta 0:00:01\r",
      "\u001b[K    11% |###                             | 1.5MB 18.1MB/s eta 0:00:01\r",
      "\u001b[K    11% |###                             | 1.5MB 15.7MB/s eta 0:00:01\r",
      "\u001b[K    11% |###                             | 1.5MB 18.0MB/s eta 0:00:01\r",
      "\u001b[K    11% |###                             | 1.5MB 19.1MB/s eta 0:00:01\r",
      "\u001b[K    11% |###                             | 1.5MB 19.0MB/s eta 0:00:01\r",
      "\u001b[K    11% |###                             | 1.5MB 22.8MB/s eta 0:00:01\r",
      "\u001b[K    11% |###                             | 1.5MB 18.4MB/s eta 0:00:01\r",
      "\u001b[K    11% |###                             | 1.5MB 18.4MB/s eta 0:00:01\r",
      "\u001b[K    12% |###                             | 1.5MB 17.7MB/s eta 0:00:01\r",
      "\u001b[K    12% |###                             | 1.6MB 17.6MB/s eta 0:00:01\r",
      "\u001b[K    12% |###                             | 1.6MB 17.3MB/s eta 0:00:01\r",
      "\u001b[K    12% |###                             | 1.6MB 20.1MB/s eta 0:00:01\r",
      "\u001b[K    12% |###                             | 1.6MB 20.1MB/s eta 0:00:01\r",
      "\u001b[K    12% |###                             | 1.6MB 18.8MB/s eta 0:00:01\r",
      "\u001b[K    12% |####                            | 1.6MB 17.4MB/s eta 0:00:01\r",
      "\u001b[K    12% |####                            | 1.6MB 17.9MB/s eta 0:00:01\r",
      "\u001b[K    12% |####                            | 1.6MB 21.8MB/s eta 0:00:01\r",
      "\u001b[K    12% |####                            | 1.6MB 19.3MB/s eta 0:00:01\r",
      "\u001b[K    12% |####                            | 1.6MB 29.6MB/s eta 0:00:01\r",
      "\u001b[K    12% |####                            | 1.7MB 20.1MB/s eta 0:00:01\r",
      "\u001b[K    13% |####                            | 1.7MB 20.8MB/s eta 0:00:01\r",
      "\u001b[K    13% |####                            | 1.7MB 18.4MB/s eta 0:00:01\r",
      "\u001b[K    13% |####                            | 1.7MB 18.1MB/s eta 0:00:01\r",
      "\u001b[K    13% |####                            | 1.7MB 18.4MB/s eta 0:00:01\r",
      "\u001b[K    13% |####                            | 1.7MB 20.1MB/s eta 0:00:01\r",
      "\u001b[K    13% |####                            | 1.7MB 18.7MB/s eta 0:00:01\r",
      "\u001b[K    13% |####                            | 1.7MB 18.7MB/s eta 0:00:01\r",
      "\u001b[K    13% |####                            | 1.7MB 19.2MB/s eta 0:00:01\r",
      "\u001b[K    13% |####                            | 1.8MB 19.2MB/s eta 0:00:01\r",
      "\u001b[K    13% |####                            | 1.8MB 27.9MB/s eta 0:00:01\r",
      "\u001b[K    13% |####                            | 1.8MB 19.9MB/s eta 0:00:01\r",
      "\u001b[K    13% |####                            | 1.8MB 22.9MB/s eta 0:00:01\r",
      "\u001b[K    13% |####                            | 1.8MB 20.4MB/s eta 0:00:01\r",
      "\u001b[K    14% |####                            | 1.8MB 18.2MB/s eta 0:00:01\r",
      "\u001b[K    14% |####                            | 1.8MB 18.3MB/s eta 0:00:01\r",
      "\u001b[K    14% |####                            | 1.8MB 19.1MB/s eta 0:00:01\r",
      "\u001b[K    14% |####                            | 1.8MB 19.1MB/s eta 0:00:01\r",
      "\u001b[K    14% |####                            | 1.8MB 20.9MB/s eta 0:00:01\r",
      "\u001b[K    14% |####                            | 1.9MB 19.5MB/s eta 0:00:01\r",
      "\u001b[K    14% |####                            | 1.9MB 19.5MB/s eta 0:00:01\r",
      "\u001b[K    14% |####                            | 1.9MB 19.3MB/s eta 0:00:01\r",
      "\u001b[K    14% |####                            | 1.9MB 19.2MB/s eta 0:00:01\r",
      "\u001b[K    14% |####                            | 1.9MB 21.9MB/s eta 0:00:01\r",
      "\u001b[K    14% |####                            | 1.9MB 23.1MB/s eta 0:00:01\r",
      "\u001b[K    14% |####                            | 1.9MB 23.1MB/s eta 0:00:01\r",
      "\u001b[K    15% |####                            | 1.9MB 23.4MB/s eta 0:00:01\r",
      "\u001b[K    15% |####                            | 1.9MB 19.8MB/s eta 0:00:01\r",
      "\u001b[K    15% |####                            | 1.9MB 19.5MB/s eta 0:00:01\r",
      "\u001b[K    15% |####                            | 2.0MB 20.9MB/s eta 0:00:01\r",
      "\u001b[K    15% |####                            | 2.0MB 19.9MB/s eta 0:00:01\r",
      "\u001b[K    15% |####                            | 2.0MB 28.3MB/s eta 0:00:01\r",
      "\u001b[K    15% |####                            | 2.0MB 28.1MB/s eta 0:00:01\r",
      "\u001b[K    15% |####                            | 2.0MB 28.2MB/s eta 0:00:01\r",
      "\u001b[K    15% |#####                           | 2.0MB 22.6MB/s eta 0:00:01\r",
      "\u001b[K    15% |#####                           | 2.0MB 22.4MB/s eta 0:00:01\r",
      "\u001b[K    15% |#####                           | 2.0MB 23.0MB/s eta 0:00:01\r",
      "\u001b[K    15% |#####                           | 2.0MB 23.3MB/s eta 0:00:01\r",
      "\u001b[K    15% |#####                           | 2.0MB 23.5MB/s eta 0:00:01\r",
      "\u001b[K    16% |#####                           | 2.1MB 23.7MB/s eta 0:00:01\r",
      "\u001b[K    16% |#####                           | 2.1MB 20.1MB/s eta 0:00:01\r",
      "\u001b[K    16% |#####                           | 2.1MB 19.4MB/s eta 0:00:01\r",
      "\u001b[K    16% |#####                           | 2.1MB 19.4MB/s eta 0:00:01\r",
      "\u001b[K    16% |#####                           | 2.1MB 19.3MB/s eta 0:00:01\r",
      "\u001b[K    16% |#####                           | 2.1MB 26.7MB/s eta 0:00:01\r",
      "\u001b[K    16% |#####                           | 2.1MB 26.4MB/s eta 0:00:01\r",
      "\u001b[K    16% |#####                           | 2.1MB 20.8MB/s eta 0:00:01\r",
      "\u001b[K    16% |#####                           | 2.1MB 24.5MB/s eta 0:00:01\r",
      "\u001b[K    16% |#####                           | 2.2MB 21.2MB/s eta 0:00:01\r",
      "\u001b[K    16% |#####                           | 2.2MB 21.2MB/s eta 0:00:01\r",
      "\u001b[K    16% |#####                           | 2.2MB 26.5MB/s eta 0:00:01\r",
      "\u001b[K    16% |#####                           | 2.2MB 27.2MB/s eta 0:00:01\r",
      "\u001b[K    17% |#####                           | 2.2MB 27.9MB/s eta 0:00:01\r",
      "\u001b[K    17% |#####                           | 2.2MB 22.2MB/s eta 0:00:01\r",
      "\u001b[K    17% |#####                           | 2.2MB 19.6MB/s eta 0:00:01\r",
      "\u001b[K    17% |#####                           | 2.2MB 20.0MB/s eta 0:00:01\r",
      "\u001b[K    17% |#####                           | 2.2MB 24.4MB/s eta 0:00:01\r",
      "\u001b[K    17% |#####                           | 2.2MB 24.6MB/s eta 0:00:01\r",
      "\u001b[K    17% |#####                           | 2.3MB 29.5MB/s eta 0:00:01\r",
      "\u001b[K    17% |#####                           | 2.3MB 23.5MB/s eta 0:00:01\r",
      "\u001b[K    17% |#####                           | 2.3MB 23.6MB/s eta 0:00:01\r",
      "\u001b[K    17% |#####                           | 2.3MB 23.9MB/s eta 0:00:01\r",
      "\u001b[K    17% |#####                           | 2.3MB 19.9MB/s eta 0:00:01\r",
      "\u001b[K    17% |#####                           | 2.3MB 24.3MB/s eta 0:00:01\r",
      "\u001b[K    18% |#####                           | 2.3MB 28.3MB/s eta 0:00:01\r",
      "\u001b[K    18% |#####                           | 2.3MB 28.4MB/s eta 0:00:01\r",
      "\u001b[K    18% |#####                           | 2.3MB 23.8MB/s eta 0:00:01\r",
      "\u001b[K    18% |#####                           | 2.3MB 23.4MB/s eta 0:00:01\r",
      "\u001b[K    18% |#####                           | 2.4MB 23.5MB/s eta 0:00:01\r",
      "\u001b[K    18% |#####                           | 2.4MB 24.9MB/s eta 0:00:01\r",
      "\u001b[K    18% |#####                           | 2.4MB 24.9MB/s eta 0:00:01\r",
      "\u001b[K    18% |#####                           | 2.4MB 24.9MB/s eta 0:00:01\r",
      "\u001b[K    18% |#####                           | 2.4MB 26.5MB/s eta 0:00:01\r",
      "\u001b[K    18% |######                          | 2.4MB 26.8MB/s eta 0:00:01\r",
      "\u001b[K    18% |######                          | 2.4MB 26.7MB/s eta 0:00:01\r",
      "\u001b[K    18% |######                          | 2.4MB 21.1MB/s eta 0:00:01\r",
      "\u001b[K    18% |######                          | 2.4MB 25.2MB/s eta 0:00:01\r",
      "\u001b[K    19% |######                          | 2.4MB 25.4MB/s eta 0:00:01\r",
      "\u001b[K    19% |######                          | 2.5MB 20.1MB/s eta 0:00:01\r",
      "\u001b[K    19% |######                          | 2.5MB 23.0MB/s eta 0:00:01\r",
      "\u001b[K    19% |######                          | 2.5MB 23.2MB/s eta 0:00:01\r",
      "\u001b[K    19% |######                          | 2.5MB 23.4MB/s eta 0:00:01\r",
      "\u001b[K    19% |######                          | 2.5MB 24.4MB/s eta 0:00:01\r",
      "\u001b[K    19% |######                          | 2.5MB 24.2MB/s eta 0:00:01\r",
      "\u001b[K    19% |######                          | 2.5MB 24.4MB/s eta 0:00:01\r",
      "\u001b[K    19% |######                          | 2.5MB 27.7MB/s eta 0:00:01\r",
      "\u001b[K    19% |######                          | 2.5MB 27.7MB/s eta 0:00:01\r",
      "\u001b[K    19% |######                          | 2.5MB 27.3MB/s eta 0:00:01\r",
      "\u001b[K    19% |######                          | 2.6MB 36.4MB/s eta 0:00:01\r",
      "\u001b[K    20% |######                          | 2.6MB 36.2MB/s eta 0:00:01\r",
      "\u001b[K    20% |######                          | 2.6MB 36.5MB/s eta 0:00:01\r",
      "\u001b[K    20% |######                          | 2.6MB 27.4MB/s eta 0:00:01\r",
      "\u001b[K    20% |######                          | 2.6MB 30.4MB/s eta 0:00:01\r",
      "\u001b[K    20% |######                          | 2.6MB 30.5MB/s eta 0:00:01\r",
      "\u001b[K    20% |######                          | 2.6MB 22.9MB/s eta 0:00:01\r",
      "\u001b[K    20% |######                          | 2.6MB 25.9MB/s eta 0:00:01\r",
      "\u001b[K    20% |######                          | 2.6MB 26.0MB/s eta 0:00:01\r",
      "\u001b[K    20% |######                          | 2.7MB 26.5MB/s eta 0:00:01\r",
      "\u001b[K    20% |######                          | 2.7MB 25.8MB/s eta 0:00:01\r",
      "\u001b[K    20% |######                          | 2.7MB 25.4MB/s eta 0:00:01\r",
      "\u001b[K    20% |######                          | 2.7MB 25.5MB/s eta 0:00:01\r",
      "\u001b[K    20% |######                          | 2.7MB 29.0MB/s eta 0:00:01\r",
      "\u001b[K    21% |######                          | 2.7MB 28.8MB/s eta 0:00:01\r",
      "\u001b[K    21% |######                          | 2.7MB 27.9MB/s eta 0:00:01\r",
      "\u001b[K    21% |######                          | 2.7MB 38.5MB/s eta 0:00:01\r",
      "\u001b[K    21% |######                          | 2.7MB 38.6MB/s eta 0:00:01\r",
      "\u001b[K    21% |######                          | 2.7MB 38.9MB/s eta 0:00:01\r",
      "\u001b[K    21% |######                          | 2.8MB 28.5MB/s eta 0:00:01\r",
      "\u001b[K    21% |######                          | 2.8MB 30.6MB/s eta 0:00:01\r",
      "\u001b[K    21% |######                          | 2.8MB 31.1MB/s eta 0:00:01\r",
      "\u001b[K    21% |######                          | 2.8MB 22.4MB/s eta 0:00:01\r",
      "\u001b[K    21% |######                          | 2.8MB 2.3MB/s eta 0:00:05\r",
      "\u001b[K    21% |######                          | 2.8MB 2.3MB/s eta 0:00:05\r",
      "\u001b[K    21% |#######                         | 2.8MB 2.3MB/s eta 0:00:05\r",
      "\u001b[K    22% |#######                         | 2.8MB 2.3MB/s eta 0:00:05\r",
      "\u001b[K    22% |#######                         | 2.8MB 2.3MB/s eta 0:00:05\r",
      "\u001b[K    22% |#######                         | 2.8MB 2.3MB/s eta 0:00:05\r",
      "\u001b[K    22% |#######                         | 2.9MB 2.3MB/s eta 0:00:05\r",
      "\u001b[K    22% |#######                         | 2.9MB 2.3MB/s eta 0:00:05\r",
      "\u001b[K    22% |#######                         | 2.9MB 2.3MB/s eta 0:00:05\r",
      "\u001b[K    22% |#######                         | 2.9MB 2.3MB/s eta 0:00:05\r",
      "\u001b[K    22% |#######                         | 2.9MB 32.6MB/s eta 0:00:01\r",
      "\u001b[K    22% |#######                         | 2.9MB 32.4MB/s eta 0:00:01\r",
      "\u001b[K    22% |#######                         | 2.9MB 31.6MB/s eta 0:00:01\r",
      "\u001b[K    22% |#######                         | 2.9MB 30.1MB/s eta 0:00:01\r",
      "\u001b[K    22% |#######                         | 2.9MB 30.2MB/s eta 0:00:01\r",
      "\u001b[K    22% |#######                         | 2.9MB 30.1MB/s eta 0:00:01\r",
      "\u001b[K    23% |#######                         | 3.0MB 28.5MB/s eta 0:00:01\r",
      "\u001b[K    23% |#######                         | 3.0MB 28.3MB/s eta 0:00:01\r",
      "\u001b[K    23% |#######                         | 3.0MB 28.3MB/s eta 0:00:01\r",
      "\u001b[K    23% |#######                         | 3.0MB 28.3MB/s eta 0:00:01\r",
      "\u001b[K    23% |#######                         | 3.0MB 29.4MB/s eta 0:00:01\r",
      "\u001b[K    23% |#######                         | 3.0MB 28.9MB/s eta 0:00:01\r",
      "\u001b[K    23% |#######                         | 3.0MB 27.7MB/s eta 0:00:01\r",
      "\u001b[K    23% |#######                         | 3.0MB 28.8MB/s eta 0:00:01\r",
      "\u001b[K    23% |#######                         | 3.0MB 28.4MB/s eta 0:00:01\r",
      "\u001b[K    23% |#######                         | 3.1MB 27.3MB/s eta 0:00:01\r",
      "\u001b[K    23% |#######                         | 3.1MB 29.1MB/s eta 0:00:01\r",
      "\u001b[K    23% |#######                         | 3.1MB 27.5MB/s eta 0:00:01\r",
      "\u001b[K    24% |#######                         | 3.1MB 27.0MB/s eta 0:00:01\r",
      "\u001b[K    24% |#######                         | 3.1MB 27.5MB/s eta 0:00:01\r",
      "\u001b[K    24% |#######                         | 3.1MB 27.9MB/s eta 0:00:01\r",
      "\u001b[K    24% |#######                         | 3.1MB 28.7MB/s eta 0:00:01\r",
      "\u001b[K    24% |#######                         | 3.1MB 28.9MB/s eta 0:00:01\r",
      "\u001b[K    24% |#######                         | 3.1MB 29.7MB/s eta 0:00:01\r",
      "\u001b[K    24% |#######                         | 3.1MB 31.5MB/s eta 0:00:01\r",
      "\u001b[K    24% |#######                         | 3.2MB 33.5MB/s eta 0:00:01\r",
      "\u001b[K    24% |#######                         | 3.2MB 32.0MB/s eta 0:00:01\r",
      "\u001b[K    24% |#######                         | 3.2MB 35.8MB/s eta 0:00:01\r",
      "\u001b[K    24% |#######                         | 3.2MB 37.1MB/s eta 0:00:01\r",
      "\u001b[K    24% |#######                         | 3.2MB 38.7MB/s eta 0:00:01\r",
      "\u001b[K    24% |#######                         | 3.2MB 39.8MB/s eta 0:00:01\r",
      "\u001b[K    25% |########                        | 3.2MB 40.1MB/s eta 0:00:01\r",
      "\u001b[K    25% |########                        | 3.2MB 41.1MB/s eta 0:00:01\r",
      "\u001b[K    25% |########                        | 3.2MB 39.5MB/s eta 0:00:01\r",
      "\u001b[K    25% |########                        | 3.2MB 39.0MB/s eta 0:00:01\r",
      "\u001b[K    25% |########                        | 3.3MB 37.0MB/s eta 0:00:01\r",
      "\u001b[K    25% |########                        | 3.3MB 39.3MB/s eta 0:00:01\r",
      "\u001b[K    25% |########                        | 3.3MB 35.2MB/s eta 0:00:01\r",
      "\u001b[K    25% |########                        | 3.3MB 35.0MB/s eta 0:00:01\r",
      "\u001b[K    25% |########                        | 3.3MB 35.9MB/s eta 0:00:01\r",
      "\u001b[K    25% |########                        | 3.3MB 35.5MB/s eta 0:00:01\r",
      "\u001b[K    25% |########                        | 3.3MB 35.4MB/s eta 0:00:01\r",
      "\u001b[K    25% |########                        | 3.3MB 31.6MB/s eta 0:00:01\r",
      "\u001b[K    26% |########                        | 3.3MB 32.0MB/s eta 0:00:01\r",
      "\u001b[K    26% |########                        | 3.3MB 31.1MB/s eta 0:00:01\r",
      "\u001b[K    26% |########                        | 3.4MB 28.6MB/s eta 0:00:01\r",
      "\u001b[K    26% |########                        | 3.4MB 28.7MB/s eta 0:00:01\r",
      "\u001b[K    26% |########                        | 3.4MB 31.0MB/s eta 0:00:01\r",
      "\u001b[K    26% |########                        | 3.4MB 30.0MB/s eta 0:00:01\r",
      "\u001b[K    26% |########                        | 3.4MB 29.4MB/s eta 0:00:01\r",
      "\u001b[K    26% |########                        | 3.4MB 29.2MB/s eta 0:00:01\r",
      "\u001b[K    26% |########                        | 3.4MB 28.9MB/s eta 0:00:01\r",
      "\u001b[K    26% |########                        | 3.4MB 31.7MB/s eta 0:00:01\r",
      "\u001b[K    26% |########                        | 3.4MB 31.6MB/s eta 0:00:01\r",
      "\u001b[K    26% |########                        | 3.5MB 32.4MB/s eta 0:00:01\r",
      "\u001b[K    26% |########                        | 3.5MB 35.0MB/s eta 0:00:01\r",
      "\u001b[K    27% |########                        | 3.5MB 34.7MB/s eta 0:00:01\r",
      "\u001b[K    27% |########                        | 3.5MB 34.3MB/s eta 0:00:01\r",
      "\u001b[K    27% |########                        | 3.5MB 32.2MB/s eta 0:00:01\r",
      "\u001b[K    27% |########                        | 3.5MB 32.6MB/s eta 0:00:01\r",
      "\u001b[K    27% |########                        | 3.5MB 32.6MB/s eta 0:00:01\r",
      "\u001b[K    27% |########                        | 3.5MB 30.7MB/s eta 0:00:01\r",
      "\u001b[K    27% |########                        | 3.5MB 32.3MB/s eta 0:00:01\r",
      "\u001b[K    27% |########                        | 3.5MB 32.5MB/s eta 0:00:01\r",
      "\u001b[K    27% |########                        | 3.6MB 30.5MB/s eta 0:00:01\r",
      "\u001b[K    27% |########                        | 3.6MB 32.3MB/s eta 0:00:01\r",
      "\u001b[K    27% |########                        | 3.6MB 32.4MB/s eta 0:00:01\r",
      "\u001b[K    27% |########                        | 3.6MB 32.7MB/s eta 0:00:01\r",
      "\u001b[K    28% |########                        | 3.6MB 35.2MB/s eta 0:00:01\r",
      "\u001b[K    28% |########                        | 3.6MB 34.1MB/s eta 0:00:01\r",
      "\u001b[K    28% |#########                       | 3.6MB 33.1MB/s eta 0:00:01\r",
      "\u001b[K    28% |#########                       | 3.6MB 33.7MB/s eta 0:00:01\r",
      "\u001b[K    28% |#########                       | 3.6MB 34.2MB/s eta 0:00:01\r",
      "\u001b[K    28% |#########                       | 3.6MB 34.3MB/s eta 0:00:01\r",
      "\u001b[K    28% |#########                       | 3.7MB 36.9MB/s eta 0:00:01\r",
      "\u001b[K    28% |#########                       | 3.7MB 35.2MB/s eta 0:00:01\r",
      "\u001b[K    28% |#########                       | 3.7MB 35.2MB/s eta 0:00:01\r",
      "\u001b[K    28% |#########                       | 3.7MB 30.2MB/s eta 0:00:01\r",
      "\u001b[K    28% |#########                       | 3.7MB 30.0MB/s eta 0:00:01\r",
      "\u001b[K    28% |#########                       | 3.7MB 31.0MB/s eta 0:00:01\r",
      "\u001b[K    28% |#########                       | 3.7MB 31.7MB/s eta 0:00:01\r",
      "\u001b[K    29% |#########                       | 3.7MB 33.5MB/s eta 0:00:01\r",
      "\u001b[K    29% |#########                       | 3.7MB 31.7MB/s eta 0:00:01\r",
      "\u001b[K    29% |#########                       | 3.7MB 30.9MB/s eta 0:00:01\r",
      "\u001b[K    29% |#########                       | 3.8MB 29.4MB/s eta 0:00:01\r",
      "\u001b[K    29% |#########                       | 3.8MB 30.6MB/s eta 0:00:01\r",
      "\u001b[K    29% |#########                       | 3.8MB 31.0MB/s eta 0:00:01\r",
      "\u001b[K    29% |#########                       | 3.8MB 36.2MB/s eta 0:00:01\r",
      "\u001b[K    29% |#########                       | 3.8MB 38.4MB/s eta 0:00:01\r",
      "\u001b[K    29% |#########                       | 3.8MB 39.5MB/s eta 0:00:01\r",
      "\u001b[K    29% |#########                       | 3.8MB 36.4MB/s eta 0:00:01\r",
      "\u001b[K    29% |#########                       | 3.8MB 35.2MB/s eta 0:00:01\r",
      "\u001b[K    29% |#########                       | 3.8MB 36.7MB/s eta 0:00:01\r",
      "\u001b[K    30% |#########                       | 3.9MB 36.9MB/s eta 0:00:01\r",
      "\u001b[K    30% |#########                       | 3.9MB 39.6MB/s eta 0:00:01\r",
      "\u001b[K    30% |#########                       | 3.9MB 39.6MB/s eta 0:00:01\r",
      "\u001b[K    30% |#########                       | 3.9MB 39.7MB/s eta 0:00:01\r",
      "\u001b[K    30% |#########                       | 3.9MB 31.5MB/s eta 0:00:01\r",
      "\u001b[K    30% |#########                       | 3.9MB 29.6MB/s eta 0:00:01\r",
      "\u001b[K    30% |#########                       | 3.9MB 29.6MB/s eta 0:00:01\r",
      "\u001b[K    30% |#########                       | 3.9MB 31.9MB/s eta 0:00:01\r",
      "\u001b[K    30% |#########                       | 3.9MB 33.3MB/s eta 0:00:01\r",
      "\u001b[K    30% |#########                       | 3.9MB 32.7MB/s eta 0:00:01\r",
      "\u001b[K    30% |#########                       | 4.0MB 31.2MB/s eta 0:00:01\r",
      "\u001b[K    30% |#########                       | 4.0MB 31.0MB/s eta 0:00:01\r",
      "\u001b[K    30% |#########                       | 4.0MB 31.1MB/s eta 0:00:01\r",
      "\u001b[K    31% |#########                       | 4.0MB 30.7MB/s eta 0:00:01\r",
      "\u001b[K    31% |#########                       | 4.0MB 36.1MB/s eta 0:00:01\r",
      "\u001b[K    31% |#########                       | 4.0MB 37.8MB/s eta 0:00:01\r",
      "\u001b[K    31% |##########                      | 4.0MB 34.3MB/s eta 0:00:01\r",
      "\u001b[K    31% |##########                      | 4.0MB 34.7MB/s eta 0:00:01\r",
      "\u001b[K    31% |##########                      | 4.0MB 34.4MB/s eta 0:00:01\r",
      "\u001b[K    31% |##########                      | 4.0MB 35.8MB/s eta 0:00:01\r",
      "\u001b[K    31% |##########                      | 4.1MB 39.6MB/s eta 0:00:01\r",
      "\u001b[K    31% |##########                      | 4.1MB 39.8MB/s eta 0:00:01\r",
      "\u001b[K    31% |##########                      | 4.1MB 40.6MB/s eta 0:00:01\r",
      "\u001b[K    31% |##########                      | 4.1MB 34.3MB/s eta 0:00:01\r",
      "\u001b[K    31% |##########                      | 4.1MB 33.9MB/s eta 0:00:01\r",
      "\u001b[K    31% |##########                      | 4.1MB 33.2MB/s eta 0:00:01\r",
      "\u001b[K    32% |##########                      | 4.1MB 34.4MB/s eta 0:00:01\r",
      "\u001b[K    32% |##########                      | 4.1MB 34.3MB/s eta 0:00:01\r",
      "\u001b[K    32% |##########                      | 4.1MB 32.4MB/s eta 0:00:01\r",
      "\u001b[K    32% |##########                      | 4.1MB 31.7MB/s eta 0:00:01\r",
      "\u001b[K    32% |##########                      | 4.2MB 31.5MB/s eta 0:00:01\r",
      "\u001b[K    32% |##########                      | 4.2MB 30.2MB/s eta 0:00:01\r",
      "\u001b[K    32% |##########                      | 4.2MB 27.7MB/s eta 0:00:01\r",
      "\u001b[K    32% |##########                      | 4.2MB 31.0MB/s eta 0:00:01\r",
      "\u001b[K    32% |##########                      | 4.2MB 33.0MB/s eta 0:00:01\r",
      "\u001b[K    32% |##########                      | 4.2MB 34.3MB/s eta 0:00:01\r",
      "\u001b[K    32% |##########                      | 4.2MB 32.7MB/s eta 0:00:01\r",
      "\u001b[K    32% |##########                      | 4.2MB 32.0MB/s eta 0:00:01\r",
      "\u001b[K    33% |##########                      | 4.2MB 34.2MB/s eta 0:00:01\r",
      "\u001b[K    33% |##########                      | 4.2MB 34.3MB/s eta 0:00:01\r",
      "\u001b[K    33% |##########                      | 4.3MB 34.4MB/s eta 0:00:01\r",
      "\u001b[K    33% |##########                      | 4.3MB 36.2MB/s eta 0:00:01\r",
      "\u001b[K    33% |##########                      | 4.3MB 39.9MB/s eta 0:00:01\r",
      "\u001b[K    33% |##########                      | 4.3MB 41.3MB/s eta 0:00:01\r",
      "\u001b[K    33% |##########                      | 4.3MB 38.4MB/s eta 0:00:01\r",
      "\u001b[K    33% |##########                      | 4.3MB 34.6MB/s eta 0:00:01\r",
      "\u001b[K    33% |##########                      | 4.3MB 37.9MB/s eta 0:00:01\r",
      "\u001b[K    33% |##########                      | 4.3MB 37.2MB/s eta 0:00:01\r",
      "\u001b[K    33% |##########                      | 4.3MB 33.2MB/s eta 0:00:01\r",
      "\u001b[K    33% |##########                      | 4.4MB 32.8MB/s eta 0:00:01\r",
      "\u001b[K    33% |##########                      | 4.4MB 32.0MB/s eta 0:00:01\r",
      "\u001b[K    34% |##########                      | 4.4MB 31.0MB/s eta 0:00:01\r",
      "\u001b[K    34% |##########                      | 4.4MB 28.6MB/s eta 0:00:01\r",
      "\u001b[K    34% |##########                      | 4.4MB 27.7MB/s eta 0:00:01\r",
      "\u001b[K    34% |##########                      | 4.4MB 28.6MB/s eta 0:00:01\r",
      "\u001b[K    34% |###########                     | 4.4MB 30.8MB/s eta 0:00:01\r",
      "\u001b[K    34% |###########                     | 4.4MB 31.2MB/s eta 0:00:01\r",
      "\u001b[K    34% |###########                     | 4.4MB 32.5MB/s eta 0:00:01\r",
      "\u001b[K    34% |###########                     | 4.4MB 36.0MB/s eta 0:00:01\r",
      "\u001b[K    34% |###########                     | 4.5MB 37.1MB/s eta 0:00:01\r",
      "\u001b[K    34% |###########                     | 4.5MB 37.9MB/s eta 0:00:01\r",
      "\u001b[K    34% |###########                     | 4.5MB 38.8MB/s eta 0:00:01\r",
      "\u001b[K    34% |###########                     | 4.5MB 40.1MB/s eta 0:00:01\r",
      "\u001b[K    35% |###########                     | 4.5MB 39.0MB/s eta 0:00:01\r",
      "\u001b[K    35% |###########                     | 4.5MB 40.4MB/s eta 0:00:01\r",
      "\u001b[K    35% |###########                     | 4.5MB 39.4MB/s eta 0:00:01\r",
      "\u001b[K    35% |###########                     | 4.5MB 36.6MB/s eta 0:00:01\r",
      "\u001b[K    35% |###########                     | 4.5MB 35.2MB/s eta 0:00:01\r",
      "\u001b[K    35% |###########                     | 4.5MB 33.8MB/s eta 0:00:01\r",
      "\u001b[K    35% |###########                     | 4.6MB 31.4MB/s eta 0:00:01\r",
      "\u001b[K    35% |###########                     | 4.6MB 31.7MB/s eta 0:00:01\r",
      "\u001b[K    35% |###########                     | 4.6MB 30.6MB/s eta 0:00:01\r",
      "\u001b[K    35% |###########                     | 4.6MB 29.1MB/s eta 0:00:01\r",
      "\u001b[K    35% |###########                     | 4.6MB 30.6MB/s eta 0:00:01\r",
      "\u001b[K    35% |###########                     | 4.6MB 29.4MB/s eta 0:00:01\r",
      "\u001b[K    35% |###########                     | 4.6MB 30.6MB/s eta 0:00:01\r",
      "\u001b[K    36% |###########                     | 4.6MB 32.5MB/s eta 0:00:01\r",
      "\u001b[K    36% |###########                     | 4.6MB 33.2MB/s eta 0:00:01\r",
      "\u001b[K    36% |###########                     | 4.6MB 34.6MB/s eta 0:00:01\r",
      "\u001b[K    36% |###########                     | 4.7MB 35.8MB/s eta 0:00:01\r",
      "\u001b[K    36% |###########                     | 4.7MB 34.2MB/s eta 0:00:01\r",
      "\u001b[K    36% |###########                     | 4.7MB 33.5MB/s eta 0:00:01\r",
      "\u001b[K    36% |###########                     | 4.7MB 36.1MB/s eta 0:00:01\r",
      "\u001b[K    36% |###########                     | 4.7MB 36.1MB/s eta 0:00:01\r",
      "\u001b[K    36% |###########                     | 4.7MB 38.1MB/s eta 0:00:01\r",
      "\u001b[K    36% |###########                     | 4.7MB 37.4MB/s eta 0:00:01\r",
      "\u001b[K    36% |###########                     | 4.7MB 36.1MB/s eta 0:00:01\r",
      "\u001b[K    36% |###########                     | 4.7MB 33.6MB/s eta 0:00:01\r",
      "\u001b[K    37% |###########                     | 4.8MB 32.0MB/s eta 0:00:01\r",
      "\u001b[K    37% |###########                     | 4.8MB 32.0MB/s eta 0:00:01\r",
      "\u001b[K    37% |###########                     | 4.8MB 33.0MB/s eta 0:00:01\r",
      "\u001b[K    37% |###########                     | 4.8MB 35.3MB/s eta 0:00:01\r",
      "\u001b[K    37% |###########                     | 4.8MB 33.9MB/s eta 0:00:01\r",
      "\u001b[K    37% |###########                     | 4.8MB 32.8MB/s eta 0:00:01\r",
      "\u001b[K    37% |############                    | 4.8MB 31.9MB/s eta 0:00:01\r",
      "\u001b[K    37% |############                    | 4.8MB 31.2MB/s eta 0:00:01\r",
      "\u001b[K    37% |############                    | 4.8MB 32.3MB/s eta 0:00:01\r",
      "\u001b[K    37% |############                    | 4.8MB 35.0MB/s eta 0:00:01\r",
      "\u001b[K    37% |############                    | 4.9MB 36.9MB/s eta 0:00:01\r",
      "\u001b[K    37% |############                    | 4.9MB 38.3MB/s eta 0:00:01\r",
      "\u001b[K    37% |############                    | 4.9MB 38.4MB/s eta 0:00:01\r",
      "\u001b[K    38% |############                    | 4.9MB 37.3MB/s eta 0:00:01\r",
      "\u001b[K    38% |############                    | 4.9MB 38.0MB/s eta 0:00:01\r",
      "\u001b[K    38% |############                    | 4.9MB 37.5MB/s eta 0:00:01\r",
      "\u001b[K    38% |############                    | 4.9MB 38.5MB/s eta 0:00:01\r",
      "\u001b[K    38% |############                    | 4.9MB 39.7MB/s eta 0:00:01\r",
      "\u001b[K    38% |############                    | 4.9MB 36.1MB/s eta 0:00:01\r",
      "\u001b[K    38% |############                    | 4.9MB 34.7MB/s eta 0:00:01\r",
      "\u001b[K    38% |############                    | 5.0MB 34.7MB/s eta 0:00:01\r",
      "\u001b[K    38% |############                    | 5.0MB 31.2MB/s eta 0:00:01\r",
      "\u001b[K    38% |############                    | 5.0MB 30.9MB/s eta 0:00:01\r",
      "\u001b[K    38% |############                    | 5.0MB 31.2MB/s eta 0:00:01\r",
      "\u001b[K    38% |############                    | 5.0MB 32.1MB/s eta 0:00:01\r",
      "\u001b[K    39% |############                    | 5.0MB 32.3MB/s eta 0:00:01\r",
      "\u001b[K    39% |############                    | 5.0MB 31.4MB/s eta 0:00:01\r",
      "\u001b[K    39% |############                    | 5.0MB 31.4MB/s eta 0:00:01\r",
      "\u001b[K    39% |############                    | 5.0MB 33.8MB/s eta 0:00:01\r",
      "\u001b[K    39% |############                    | 5.0MB 34.5MB/s eta 0:00:01\r",
      "\u001b[K    39% |############                    | 5.1MB 35.1MB/s eta 0:00:01\r",
      "\u001b[K    39% |############                    | 5.1MB 39.1MB/s eta 0:00:01\r",
      "\u001b[K    39% |############                    | 5.1MB 39.2MB/s eta 0:00:01\r",
      "\u001b[K    39% |############                    | 5.1MB 36.6MB/s eta 0:00:01\r",
      "\u001b[K    39% |############                    | 5.1MB 35.7MB/s eta 0:00:01\r",
      "\u001b[K    39% |############                    | 5.1MB 37.4MB/s eta 0:00:01\r",
      "\u001b[K    39% |############                    | 5.1MB 36.6MB/s eta 0:00:01\r",
      "\u001b[K    39% |############                    | 5.1MB 33.3MB/s eta 0:00:01\r",
      "\u001b[K    40% |############                    | 5.1MB 33.8MB/s eta 0:00:01\r",
      "\u001b[K    40% |############                    | 5.2MB 32.6MB/s eta 0:00:01\r",
      "\u001b[K    40% |############                    | 5.2MB 30.8MB/s eta 0:00:01\r",
      "\u001b[K    40% |############                    | 5.2MB 30.3MB/s eta 0:00:01\r",
      "\u001b[K    40% |############                    | 5.2MB 30.6MB/s eta 0:00:01\r",
      "\u001b[K    40% |############                    | 5.2MB 32.8MB/s eta 0:00:01\r",
      "\u001b[K    40% |############                    | 5.2MB 33.8MB/s eta 0:00:01\r",
      "\u001b[K    40% |############                    | 5.2MB 31.3MB/s eta 0:00:01\r",
      "\u001b[K    40% |#############                   | 5.2MB 32.7MB/s eta 0:00:01\r",
      "\u001b[K    40% |#############                   | 5.2MB 35.2MB/s eta 0:00:01\r",
      "\u001b[K    40% |#############                   | 5.2MB 35.1MB/s eta 0:00:01\r",
      "\u001b[K    40% |#############                   | 5.3MB 37.2MB/s eta 0:00:01\r",
      "\u001b[K    41% |#############                   | 5.3MB 39.2MB/s eta 0:00:01\r",
      "\u001b[K    41% |#############                   | 5.3MB 40.4MB/s eta 0:00:01\r",
      "\u001b[K    41% |#############                   | 5.3MB 40.7MB/s eta 0:00:01\r",
      "\u001b[K    41% |#############                   | 5.3MB 37.1MB/s eta 0:00:01\r",
      "\u001b[K    41% |#############                   | 5.3MB 36.6MB/s eta 0:00:01\r",
      "\u001b[K    41% |#############                   | 5.3MB 38.9MB/s eta 0:00:01\r",
      "\u001b[K    41% |#############                   | 5.3MB 38.4MB/s eta 0:00:01\r",
      "\u001b[K    41% |#############                   | 5.3MB 36.7MB/s eta 0:00:01\r",
      "\u001b[K    41% |#############                   | 5.3MB 34.3MB/s eta 0:00:01\r",
      "\u001b[K    41% |#############                   | 5.4MB 31.7MB/s eta 0:00:01\r",
      "\u001b[K    41% |#############                   | 5.4MB 31.2MB/s eta 0:00:01\r",
      "\u001b[K    41% |#############                   | 5.4MB 31.1MB/s eta 0:00:01\r",
      "\u001b[K    41% |#############                   | 5.4MB 31.4MB/s eta 0:00:01\r",
      "\u001b[K    42% |#############                   | 5.4MB 33.6MB/s eta 0:00:01\r",
      "\u001b[K    42% |#############                   | 5.4MB 35.2MB/s eta 0:00:01\r",
      "\u001b[K    42% |#############                   | 5.4MB 33.6MB/s eta 0:00:01\r",
      "\u001b[K    42% |#############                   | 5.4MB 33.0MB/s eta 0:00:01\r",
      "\u001b[K    42% |#############                   | 5.4MB 35.0MB/s eta 0:00:01\r",
      "\u001b[K    42% |#############                   | 5.4MB 37.7MB/s eta 0:00:01\r",
      "\u001b[K    42% |#############                   | 5.5MB 41.0MB/s eta 0:00:01\r",
      "\u001b[K    42% |#############                   | 5.5MB 43.3MB/s eta 0:00:01\r",
      "\u001b[K    42% |#############                   | 5.5MB 43.0MB/s eta 0:00:01\r",
      "\u001b[K    42% |#############                   | 5.5MB 41.3MB/s eta 0:00:01\r",
      "\u001b[K    42% |#############                   | 5.5MB 39.7MB/s eta 0:00:01\r",
      "\u001b[K    42% |#############                   | 5.5MB 37.3MB/s eta 0:00:01\r",
      "\u001b[K    43% |#############                   | 5.5MB 36.4MB/s eta 0:00:01\r",
      "\u001b[K    43% |#############                   | 5.5MB 36.9MB/s eta 0:00:01\r",
      "\u001b[K    43% |#############                   | 5.5MB 34.7MB/s eta 0:00:01\r",
      "\u001b[K    43% |#############                   | 5.6MB 32.8MB/s eta 0:00:01\r",
      "\u001b[K    43% |#############                   | 5.6MB 32.7MB/s eta 0:00:01\r",
      "\u001b[K    43% |#############                   | 5.6MB 30.1MB/s eta 0:00:01\r",
      "\u001b[K    43% |#############                   | 5.6MB 29.8MB/s eta 0:00:01\r",
      "\u001b[K    43% |#############                   | 5.6MB 30.3MB/s eta 0:00:01\r",
      "\u001b[K    43% |#############                   | 5.6MB 31.6MB/s eta 0:00:01\r",
      "\u001b[K    43% |#############                   | 5.6MB 33.3MB/s eta 0:00:01\r",
      "\u001b[K    43% |##############                  | 5.6MB 35.6MB/s eta 0:00:01\r",
      "\u001b[K    43% |##############                  | 5.6MB 34.5MB/s eta 0:00:01\r",
      "\u001b[K    43% |##############                  | 5.6MB 35.5MB/s eta 0:00:01\r",
      "\u001b[K    44% |##############                  | 5.7MB 37.3MB/s eta 0:00:01\r",
      "\u001b[K    44% |##############                  | 5.7MB 37.5MB/s eta 0:00:01\r",
      "\u001b[K    44% |##############                  | 5.7MB 40.7MB/s eta 0:00:01\r",
      "\u001b[K    44% |##############                  | 5.7MB 42.1MB/s eta 0:00:01\r",
      "\u001b[K    44% |##############                  | 5.7MB 42.5MB/s eta 0:00:01\r",
      "\u001b[K    44% |##############                  | 5.7MB 42.0MB/s eta 0:00:01\r",
      "\u001b[K    44% |##############                  | 5.7MB 42.5MB/s eta 0:00:01\r",
      "\u001b[K    44% |##############                  | 5.7MB 43.6MB/s eta 0:00:01\r",
      "\u001b[K    44% |##############                  | 5.7MB 47.3MB/s eta 0:00:01\r",
      "\u001b[K    44% |##############                  | 5.7MB 46.3MB/s eta 0:00:01\r",
      "\u001b[K    44% |##############                  | 5.8MB 43.6MB/s eta 0:00:01\r",
      "\u001b[K    44% |##############                  | 5.8MB 41.9MB/s eta 0:00:01\r",
      "\u001b[K    45% |##############                  | 5.8MB 40.3MB/s eta 0:00:01\r",
      "\u001b[K    45% |##############                  | 5.8MB 39.4MB/s eta 0:00:01\r",
      "\u001b[K    45% |##############                  | 5.8MB 37.8MB/s eta 0:00:01\r",
      "\u001b[K    45% |##############                  | 5.8MB 36.9MB/s eta 0:00:01\r",
      "\u001b[K    45% |##############                  | 5.8MB 36.7MB/s eta 0:00:01\r",
      "\u001b[K    45% |##############                  | 5.8MB 37.1MB/s eta 0:00:01\r",
      "\u001b[K    45% |##############                  | 5.8MB 36.3MB/s eta 0:00:01\r",
      "\u001b[K    45% |##############                  | 5.8MB 38.1MB/s eta 0:00:01\r",
      "\u001b[K    45% |##############                  | 5.9MB 40.6MB/s eta 0:00:01\r",
      "\u001b[K    45% |##############                  | 5.9MB 42.6MB/s eta 0:00:01\r",
      "\u001b[K    45% |##############                  | 5.9MB 45.3MB/s eta 0:00:01\r",
      "\u001b[K    45% |##############                  | 5.9MB 46.7MB/s eta 0:00:01\r",
      "\u001b[K    45% |##############                  | 5.9MB 49.8MB/s eta 0:00:01\r",
      "\u001b[K    46% |##############                  | 5.9MB 52.5MB/s eta 0:00:01\r",
      "\u001b[K    46% |##############                  | 5.9MB 52.7MB/s eta 0:00:01\r",
      "\u001b[K    46% |##############                  | 5.9MB 53.4MB/s eta 0:00:01\r",
      "\u001b[K    46% |##############                  | 5.9MB 55.9MB/s eta 0:00:01\r",
      "\u001b[K    46% |##############                  | 5.9MB 56.8MB/s eta 0:00:01\r",
      "\u001b[K    46% |##############                  | 6.0MB 58.7MB/s eta 0:00:01\r",
      "\u001b[K    46% |##############                  | 6.0MB 58.8MB/s eta 0:00:01\r",
      "\u001b[K    46% |##############                  | 6.0MB 50.6MB/s eta 0:00:01\r",
      "\u001b[K    46% |##############                  | 6.0MB 47.8MB/s eta 0:00:01\r",
      "\u001b[K    46% |##############                  | 6.0MB 44.4MB/s eta 0:00:01\r",
      "\u001b[K    46% |##############                  | 6.0MB 41.8MB/s eta 0:00:01\r",
      "\u001b[K    46% |###############                 | 6.0MB 41.4MB/s eta 0:00:01\r",
      "\u001b[K    46% |###############                 | 6.0MB 40.7MB/s eta 0:00:01\r",
      "\u001b[K    47% |###############                 | 6.0MB 41.1MB/s eta 0:00:01\r",
      "\u001b[K    47% |###############                 | 6.1MB 41.1MB/s eta 0:00:01\r",
      "\u001b[K    47% |###############                 | 6.1MB 40.5MB/s eta 0:00:01\r",
      "\u001b[K    47% |###############                 | 6.1MB 40.8MB/s eta 0:00:01\r",
      "\u001b[K    47% |###############                 | 6.1MB 44.2MB/s eta 0:00:01\r",
      "\u001b[K    47% |###############                 | 6.1MB 44.3MB/s eta 0:00:01\r",
      "\u001b[K    47% |###############                 | 6.1MB 44.0MB/s eta 0:00:01\r",
      "\u001b[K    47% |###############                 | 6.1MB 46.1MB/s eta 0:00:01\r",
      "\u001b[K    47% |###############                 | 6.1MB 46.6MB/s eta 0:00:01\r",
      "\u001b[K    47% |###############                 | 6.1MB 46.4MB/s eta 0:00:01\r",
      "\u001b[K    47% |###############                 | 6.1MB 45.5MB/s eta 0:00:01\r",
      "\u001b[K    47% |###############                 | 6.2MB 45.7MB/s eta 0:00:01\r",
      "\u001b[K    48% |###############                 | 6.2MB 45.5MB/s eta 0:00:01\r",
      "\u001b[K    48% |###############                 | 6.2MB 45.0MB/s eta 0:00:01\r",
      "\u001b[K    48% |###############                 | 6.2MB 46.4MB/s eta 0:00:01\r",
      "\u001b[K    48% |###############                 | 6.2MB 48.4MB/s eta 0:00:01\r",
      "\u001b[K    48% |###############                 | 6.2MB 52.8MB/s eta 0:00:01\r",
      "\u001b[K    48% |###############                 | 6.2MB 53.7MB/s eta 0:00:01\r",
      "\u001b[K    48% |###############                 | 6.2MB 51.5MB/s eta 0:00:01\r",
      "\u001b[K    48% |###############                 | 6.2MB 48.0MB/s eta 0:00:01\r",
      "\u001b[K    48% |###############                 | 6.2MB 47.4MB/s eta 0:00:01\r",
      "\u001b[K    48% |###############                 | 6.3MB 43.2MB/s eta 0:00:01\r",
      "\u001b[K    48% |###############                 | 6.3MB 38.6MB/s eta 0:00:01\r",
      "\u001b[K    48% |###############                 | 6.3MB 36.7MB/s eta 0:00:01\r",
      "\u001b[K    48% |###############                 | 6.3MB 36.9MB/s eta 0:00:01\r",
      "\u001b[K    49% |###############                 | 6.3MB 31.5MB/s eta 0:00:01\r",
      "\u001b[K    49% |###############                 | 6.3MB 29.1MB/s eta 0:00:01\r",
      "\u001b[K    49% |###############                 | 6.3MB 28.0MB/s eta 0:00:01\r",
      "\u001b[K    49% |###############                 | 6.3MB 26.9MB/s eta 0:00:01\r",
      "\u001b[K    49% |###############                 | 6.3MB 27.9MB/s eta 0:00:01\r",
      "\u001b[K    49% |###############                 | 6.3MB 25.3MB/s eta 0:00:01\r",
      "\u001b[K    49% |###############                 | 6.4MB 24.9MB/s eta 0:00:01\r",
      "\u001b[K    49% |###############                 | 6.4MB 26.0MB/s eta 0:00:01\r",
      "\u001b[K    49% |###############                 | 6.4MB 23.7MB/s eta 0:00:01\r",
      "\u001b[K    49% |###############                 | 6.4MB 23.4MB/s eta 0:00:01\r",
      "\u001b[K    49% |###############                 | 6.4MB 26.3MB/s eta 0:00:01\r",
      "\u001b[K    49% |###############                 | 6.4MB 26.3MB/s eta 0:00:01\r",
      "\u001b[K    50% |################                | 6.4MB 26.9MB/s eta 0:00:01\r",
      "\u001b[K    50% |################                | 6.4MB 28.3MB/s eta 0:00:01\r",
      "\u001b[K    50% |################                | 6.4MB 27.0MB/s eta 0:00:01\r",
      "\u001b[K    50% |################                | 6.5MB 30.3MB/s eta 0:00:01\r",
      "\u001b[K    50% |################                | 6.5MB 32.6MB/s eta 0:00:01\r",
      "\u001b[K    50% |################                | 6.5MB 33.8MB/s eta 0:00:01\r",
      "\u001b[K    50% |################                | 6.5MB 40.9MB/s eta 0:00:01\r",
      "\u001b[K    50% |################                | 6.5MB 41.9MB/s eta 0:00:01\r",
      "\u001b[K    50% |################                | 6.5MB 42.1MB/s eta 0:00:01\r",
      "\u001b[K    50% |################                | 6.5MB 45.1MB/s eta 0:00:01\r",
      "\u001b[K    50% |################                | 6.5MB 41.8MB/s eta 0:00:01\r",
      "\u001b[K    50% |################                | 6.5MB 40.3MB/s eta 0:00:01\r",
      "\u001b[K    50% |################                | 6.5MB 41.2MB/s eta 0:00:01\r",
      "\u001b[K    51% |################                | 6.6MB 40.5MB/s eta 0:00:01\r",
      "\u001b[K    51% |################                | 6.6MB 39.8MB/s eta 0:00:01\r",
      "\u001b[K    51% |################                | 6.6MB 37.1MB/s eta 0:00:01\r",
      "\u001b[K    51% |################                | 6.6MB 35.4MB/s eta 0:00:01\r",
      "\u001b[K    51% |################                | 6.6MB 34.7MB/s eta 0:00:01\r",
      "\u001b[K    51% |################                | 6.6MB 33.6MB/s eta 0:00:01\r",
      "\u001b[K    51% |################                | 6.6MB 34.5MB/s eta 0:00:01\r",
      "\u001b[K    51% |################                | 6.6MB 36.6MB/s eta 0:00:01\r",
      "\u001b[K    51% |################                | 6.6MB 38.2MB/s eta 0:00:01\r",
      "\u001b[K    51% |################                | 6.6MB 40.9MB/s eta 0:00:01\r",
      "\u001b[K    51% |################                | 6.7MB 41.9MB/s eta 0:00:01\r",
      "\u001b[K    51% |################                | 6.7MB 43.0MB/s eta 0:00:01\r",
      "\u001b[K    52% |################                | 6.7MB 46.7MB/s eta 0:00:01\r",
      "\u001b[K    52% |################                | 6.7MB 48.9MB/s eta 0:00:01\r",
      "\u001b[K    52% |################                | 6.7MB 50.5MB/s eta 0:00:01\r",
      "\u001b[K    52% |################                | 6.7MB 52.2MB/s eta 0:00:01\r",
      "\u001b[K    52% |################                | 6.7MB 52.6MB/s eta 0:00:01\r",
      "\u001b[K    52% |################                | 6.7MB 54.1MB/s eta 0:00:01\r",
      "\u001b[K    52% |################                | 6.7MB 52.8MB/s eta 0:00:01\r",
      "\u001b[K    52% |################                | 6.7MB 48.3MB/s eta 0:00:01\r",
      "\u001b[K    52% |################                | 6.8MB 44.1MB/s eta 0:00:01\r",
      "\u001b[K    52% |################                | 6.8MB 39.3MB/s eta 0:00:01\r",
      "\u001b[K    52% |################                | 6.8MB 39.5MB/s eta 0:00:01\r",
      "\u001b[K    52% |################                | 6.8MB 39.4MB/s eta 0:00:01\r",
      "\u001b[K    52% |################                | 6.8MB 39.0MB/s eta 0:00:01\r",
      "\u001b[K    53% |################                | 6.8MB 39.2MB/s eta 0:00:01\r",
      "\u001b[K    53% |#################               | 6.8MB 39.2MB/s eta 0:00:01\r",
      "\u001b[K    53% |#################               | 6.8MB 38.3MB/s eta 0:00:01\r",
      "\u001b[K    53% |#################               | 6.8MB 39.2MB/s eta 0:00:01\r",
      "\u001b[K    53% |#################               | 6.9MB 40.4MB/s eta 0:00:01\r",
      "\u001b[K    53% |#################               | 6.9MB 41.1MB/s eta 0:00:01\r",
      "\u001b[K    53% |#################               | 6.9MB 43.8MB/s eta 0:00:01\r",
      "\u001b[K    53% |#################               | 6.9MB 41.7MB/s eta 0:00:01\r",
      "\u001b[K    53% |#################               | 6.9MB 42.3MB/s eta 0:00:01\r",
      "\u001b[K    53% |#################               | 6.9MB 42.0MB/s eta 0:00:01\r",
      "\u001b[K    53% |#################               | 6.9MB 42.1MB/s eta 0:00:01\r",
      "\u001b[K    53% |#################               | 6.9MB 42.1MB/s eta 0:00:01\r",
      "\u001b[K    54% |#################               | 6.9MB 42.7MB/s eta 0:00:01\r",
      "\u001b[K    54% |#################               | 6.9MB 42.6MB/s eta 0:00:01\r",
      "\u001b[K    54% |#################               | 7.0MB 40.8MB/s eta 0:00:01\r",
      "\u001b[K    54% |#################               | 7.0MB 39.2MB/s eta 0:00:01\r",
      "\u001b[K    54% |#################               | 7.0MB 38.4MB/s eta 0:00:01\r",
      "\u001b[K    54% |#################               | 7.0MB 39.0MB/s eta 0:00:01\r",
      "\u001b[K    54% |#################               | 7.0MB 38.5MB/s eta 0:00:01\r",
      "\u001b[K    54% |#################               | 7.0MB 35.4MB/s eta 0:00:01\r",
      "\u001b[K    54% |#################               | 7.0MB 34.3MB/s eta 0:00:01\r",
      "\u001b[K    54% |#################               | 7.0MB 34.7MB/s eta 0:00:01\r",
      "\u001b[K    54% |#################               | 7.0MB 34.5MB/s eta 0:00:01\r",
      "\u001b[K    54% |#################               | 7.0MB 34.2MB/s eta 0:00:01\r",
      "\u001b[K    54% |#################               | 7.1MB 36.8MB/s eta 0:00:01\r",
      "\u001b[K    55% |#################               | 7.1MB 40.0MB/s eta 0:00:01\r",
      "\u001b[K    55% |#################               | 7.1MB 40.8MB/s eta 0:00:01\r",
      "\u001b[K    55% |#################               | 7.1MB 38.4MB/s eta 0:00:01\r",
      "\u001b[K    55% |#################               | 7.1MB 33.7MB/s eta 0:00:01\r",
      "\u001b[K    55% |#################               | 7.1MB 36.7MB/s eta 0:00:01\r",
      "\u001b[K    55% |#################               | 7.1MB 37.2MB/s eta 0:00:01\r",
      "\u001b[K    55% |#################               | 7.1MB 36.0MB/s eta 0:00:01\r",
      "\u001b[K    55% |#################               | 7.1MB 35.0MB/s eta 0:00:01\r",
      "\u001b[K    55% |#################               | 7.1MB 33.6MB/s eta 0:00:01\r",
      "\u001b[K    55% |#################               | 7.2MB 33.5MB/s eta 0:00:01\r",
      "\u001b[K    55% |#################               | 7.2MB 33.3MB/s eta 0:00:01\r",
      "\u001b[K    55% |#################               | 7.2MB 34.5MB/s eta 0:00:01\r",
      "\u001b[K    56% |#################               | 7.2MB 37.3MB/s eta 0:00:01\r",
      "\u001b[K    56% |#################               | 7.2MB 40.7MB/s eta 0:00:01\r",
      "\u001b[K    56% |#################               | 7.2MB 38.2MB/s eta 0:00:01\r",
      "\u001b[K    56% |##################              | 7.2MB 39.0MB/s eta 0:00:01\r",
      "\u001b[K    56% |##################              | 7.2MB 39.2MB/s eta 0:00:01\r",
      "\u001b[K    56% |##################              | 7.2MB 41.2MB/s eta 0:00:01\r",
      "\u001b[K    56% |##################              | 7.2MB 43.4MB/s eta 0:00:01\r",
      "\u001b[K    56% |##################              | 7.3MB 40.8MB/s eta 0:00:01\r",
      "\u001b[K    56% |##################              | 7.3MB 40.0MB/s eta 0:00:01\r",
      "\u001b[K    56% |##################              | 7.3MB 39.3MB/s eta 0:00:01\r",
      "\u001b[K    56% |##################              | 7.3MB 35.6MB/s eta 0:00:01\r",
      "\u001b[K    56% |##################              | 7.3MB 36.7MB/s eta 0:00:01\r",
      "\u001b[K    56% |##################              | 7.3MB 38.6MB/s eta 0:00:01\r",
      "\u001b[K    57% |##################              | 7.3MB 38.5MB/s eta 0:00:01\r",
      "\u001b[K    57% |##################              | 7.3MB 39.2MB/s eta 0:00:01\r",
      "\u001b[K    57% |##################              | 7.3MB 37.1MB/s eta 0:00:01\r",
      "\u001b[K    57% |##################              | 7.4MB 37.6MB/s eta 0:00:01\r",
      "\u001b[K    57% |##################              | 7.4MB 38.9MB/s eta 0:00:01\r",
      "\u001b[K    57% |##################              | 7.4MB 39.8MB/s eta 0:00:01\r",
      "\u001b[K    57% |##################              | 7.4MB 41.2MB/s eta 0:00:01\r",
      "\u001b[K    57% |##################              | 7.4MB 44.9MB/s eta 0:00:01\r",
      "\u001b[K    57% |##################              | 7.4MB 46.4MB/s eta 0:00:01\r",
      "\u001b[K    57% |##################              | 7.4MB 46.9MB/s eta 0:00:01\r",
      "\u001b[K    57% |##################              | 7.4MB 46.0MB/s eta 0:00:01\r",
      "\u001b[K    57% |##################              | 7.4MB 41.9MB/s eta 0:00:01\r",
      "\u001b[K    58% |##################              | 7.4MB 42.4MB/s eta 0:00:01\r",
      "\u001b[K    58% |##################              | 7.5MB 41.7MB/s eta 0:00:01\r",
      "\u001b[K    58% |##################              | 7.5MB 42.6MB/s eta 0:00:01\r",
      "\u001b[K    58% |##################              | 7.5MB 39.4MB/s eta 0:00:01\r",
      "\u001b[K    58% |##################              | 7.5MB 36.9MB/s eta 0:00:01\r",
      "\u001b[K    58% |##################              | 7.5MB 37.0MB/s eta 0:00:01\r",
      "\u001b[K    58% |##################              | 7.5MB 35.1MB/s eta 0:00:01\r",
      "\u001b[K    58% |##################              | 7.5MB 33.0MB/s eta 0:00:01\r",
      "\u001b[K    58% |##################              | 7.5MB 33.0MB/s eta 0:00:01\r",
      "\u001b[K    58% |##################              | 7.5MB 34.5MB/s eta 0:00:01\r",
      "\u001b[K    58% |##################              | 7.5MB 34.4MB/s eta 0:00:01\r",
      "\u001b[K    58% |##################              | 7.6MB 34.3MB/s eta 0:00:01\r",
      "\u001b[K    58% |##################              | 7.6MB 34.2MB/s eta 0:00:01\r",
      "\u001b[K    59% |##################              | 7.6MB 36.3MB/s eta 0:00:01\r",
      "\u001b[K    59% |##################              | 7.6MB 38.2MB/s eta 0:00:01\r",
      "\u001b[K    59% |##################              | 7.6MB 38.9MB/s eta 0:00:01\r",
      "\u001b[K    59% |##################              | 7.6MB 40.6MB/s eta 0:00:01\r",
      "\u001b[K    59% |##################              | 7.6MB 43.5MB/s eta 0:00:01\r",
      "\u001b[K    59% |###################             | 7.6MB 44.2MB/s eta 0:00:01\r",
      "\u001b[K    59% |###################             | 7.6MB 44.2MB/s eta 0:00:01\r",
      "\u001b[K    59% |###################             | 7.6MB 43.3MB/s eta 0:00:01\r",
      "\u001b[K    59% |###################             | 7.7MB 40.1MB/s eta 0:00:01\r",
      "\u001b[K    59% |###################             | 7.7MB 36.5MB/s eta 0:00:01\r",
      "\u001b[K    59% |###################             | 7.7MB 33.8MB/s eta 0:00:01\r",
      "\u001b[K    59% |###################             | 7.7MB 32.9MB/s eta 0:00:01\r",
      "\u001b[K    60% |###################             | 7.7MB 32.6MB/s eta 0:00:01\r",
      "\u001b[K    60% |###################             | 7.7MB 32.7MB/s eta 0:00:01\r",
      "\u001b[K    60% |###################             | 7.7MB 33.0MB/s eta 0:00:01\r",
      "\u001b[K    60% |###################             | 7.7MB 32.6MB/s eta 0:00:01\r",
      "\u001b[K    60% |###################             | 7.7MB 33.5MB/s eta 0:00:01\r",
      "\u001b[K    60% |###################             | 7.8MB 35.0MB/s eta 0:00:01\r",
      "\u001b[K    60% |###################             | 7.8MB 38.0MB/s eta 0:00:01\r",
      "\u001b[K    60% |###################             | 7.8MB 41.8MB/s eta 0:00:01\r",
      "\u001b[K    60% |###################             | 7.8MB 46.2MB/s eta 0:00:01\r",
      "\u001b[K    60% |###################             | 7.8MB 49.2MB/s eta 0:00:01\r",
      "\u001b[K    60% |###################             | 7.8MB 49.2MB/s eta 0:00:01\r",
      "\u001b[K    60% |###################             | 7.8MB 50.1MB/s eta 0:00:01\r",
      "\u001b[K    60% |###################             | 7.8MB 49.4MB/s eta 0:00:01\r",
      "\u001b[K    61% |###################             | 7.8MB 49.4MB/s eta 0:00:01\r",
      "\u001b[K    61% |###################             | 7.8MB 50.3MB/s eta 0:00:01\r",
      "\u001b[K    61% |###################             | 7.9MB 45.9MB/s eta 0:00:01\r",
      "\u001b[K    61% |###################             | 7.9MB 39.7MB/s eta 0:00:01\r",
      "\u001b[K    61% |###################             | 7.9MB 36.4MB/s eta 0:00:01\r",
      "\u001b[K    61% |###################             | 7.9MB 34.4MB/s eta 0:00:01\r",
      "\u001b[K    61% |###################             | 7.9MB 33.1MB/s eta 0:00:01\r",
      "\u001b[K    61% |###################             | 7.9MB 32.6MB/s eta 0:00:01\r",
      "\u001b[K    61% |###################             | 7.9MB 32.3MB/s eta 0:00:01\r",
      "\u001b[K    61% |###################             | 7.9MB 32.7MB/s eta 0:00:01\r",
      "\u001b[K    61% |###################             | 7.9MB 32.8MB/s eta 0:00:01\r",
      "\u001b[K    61% |###################             | 7.9MB 32.4MB/s eta 0:00:01\r",
      "\u001b[K    62% |###################             | 8.0MB 34.8MB/s eta 0:00:01\r",
      "\u001b[K    62% |###################             | 8.0MB 39.2MB/s eta 0:00:01\r",
      "\u001b[K    62% |###################             | 8.0MB 43.3MB/s eta 0:00:01\r",
      "\u001b[K    62% |###################             | 8.0MB 46.6MB/s eta 0:00:01\r",
      "\u001b[K    62% |###################             | 8.0MB 48.2MB/s eta 0:00:01\r",
      "\u001b[K    62% |###################             | 8.0MB 50.4MB/s eta 0:00:01\r",
      "\u001b[K    62% |###################             | 8.0MB 48.0MB/s eta 0:00:01\r",
      "\u001b[K    62% |####################            | 8.0MB 47.4MB/s eta 0:00:01\r",
      "\u001b[K    62% |####################            | 8.0MB 48.1MB/s eta 0:00:01\r",
      "\u001b[K    62% |####################            | 8.0MB 46.8MB/s eta 0:00:01\r",
      "\u001b[K    62% |####################            | 8.1MB 44.1MB/s eta 0:00:01\r",
      "\u001b[K    62% |####################            | 8.1MB 40.5MB/s eta 0:00:01\r",
      "\u001b[K    62% |####################            | 8.1MB 36.1MB/s eta 0:00:01\r",
      "\u001b[K    63% |####################            | 8.1MB 34.7MB/s eta 0:00:01\r",
      "\u001b[K    63% |####################            | 8.1MB 33.6MB/s eta 0:00:01\r",
      "\u001b[K    63% |####################            | 8.1MB 32.8MB/s eta 0:00:01\r",
      "\u001b[K    63% |####################            | 8.1MB 33.3MB/s eta 0:00:01\r",
      "\u001b[K    63% |####################            | 8.1MB 32.9MB/s eta 0:00:01\r",
      "\u001b[K    63% |####################            | 8.1MB 32.8MB/s eta 0:00:01\r",
      "\u001b[K    63% |####################            | 8.2MB 33.4MB/s eta 0:00:01\r",
      "\u001b[K    63% |####################            | 8.2MB 34.5MB/s eta 0:00:01\r",
      "\u001b[K    63% |####################            | 8.2MB 37.5MB/s eta 0:00:01\r",
      "\u001b[K    63% |####################            | 8.2MB 41.9MB/s eta 0:00:01\r",
      "\u001b[K    63% |####################            | 8.2MB 44.2MB/s eta 0:00:01\r",
      "\u001b[K    63% |####################            | 8.2MB 46.6MB/s eta 0:00:01\r",
      "\u001b[K    63% |####################            | 8.2MB 47.1MB/s eta 0:00:01\r",
      "\u001b[K    64% |####################            | 8.2MB 49.1MB/s eta 0:00:01\r",
      "\u001b[K    64% |####################            | 8.2MB 49.9MB/s eta 0:00:01\r",
      "\u001b[K    64% |####################            | 8.2MB 49.1MB/s eta 0:00:01\r",
      "\u001b[K    64% |####################            | 8.3MB 49.5MB/s eta 0:00:01\r",
      "\u001b[K    64% |####################            | 8.3MB 46.4MB/s eta 0:00:01\r",
      "\u001b[K    64% |####################            | 8.3MB 41.7MB/s eta 0:00:01\r",
      "\u001b[K    64% |####################            | 8.3MB 39.3MB/s eta 0:00:01\r",
      "\u001b[K    64% |####################            | 8.3MB 38.0MB/s eta 0:00:01\r",
      "\u001b[K    64% |####################            | 8.3MB 35.9MB/s eta 0:00:01\r",
      "\u001b[K    64% |####################            | 8.3MB 33.5MB/s eta 0:00:01\r",
      "\u001b[K    64% |####################            | 8.3MB 32.6MB/s eta 0:00:01\r",
      "\u001b[K    64% |####################            | 8.3MB 33.0MB/s eta 0:00:01\r",
      "\u001b[K    65% |####################            | 8.3MB 31.8MB/s eta 0:00:01\r",
      "\u001b[K    65% |####################            | 8.4MB 31.8MB/s eta 0:00:01\r",
      "\u001b[K    65% |####################            | 8.4MB 33.7MB/s eta 0:00:01\r",
      "\u001b[K    65% |####################            | 8.4MB 36.0MB/s eta 0:00:01\r",
      "\u001b[K    65% |####################            | 8.4MB 38.6MB/s eta 0:00:01\r",
      "\u001b[K    65% |####################            | 8.4MB 39.6MB/s eta 0:00:01\r",
      "\u001b[K    65% |####################            | 8.4MB 41.9MB/s eta 0:00:01\r",
      "\u001b[K    65% |####################            | 8.4MB 46.5MB/s eta 0:00:01\r",
      "\u001b[K    65% |#####################           | 8.4MB 47.6MB/s eta 0:00:01\r",
      "\u001b[K    65% |#####################           | 8.4MB 47.0MB/s eta 0:00:01\r",
      "\u001b[K    65% |#####################           | 8.4MB 50.0MB/s eta 0:00:01\r",
      "\u001b[K    65% |#####################           | 8.5MB 50.0MB/s eta 0:00:01\r",
      "\u001b[K    65% |#####################           | 8.5MB 47.9MB/s eta 0:00:01\r",
      "\u001b[K    66% |#####################           | 8.5MB 44.4MB/s eta 0:00:01\r",
      "\u001b[K    66% |#####################           | 8.5MB 41.0MB/s eta 0:00:01\r",
      "\u001b[K    66% |#####################           | 8.5MB 38.0MB/s eta 0:00:01\r",
      "\u001b[K    66% |#####################           | 8.5MB 34.9MB/s eta 0:00:01\r",
      "\u001b[K    66% |#####################           | 8.5MB 33.1MB/s eta 0:00:01\r",
      "\u001b[K    66% |#####################           | 8.5MB 30.4MB/s eta 0:00:01\r",
      "\u001b[K    66% |#####################           | 8.5MB 29.2MB/s eta 0:00:01\r",
      "\u001b[K    66% |#####################           | 8.6MB 28.0MB/s eta 0:00:01\r",
      "\u001b[K    66% |#####################           | 8.6MB 27.6MB/s eta 0:00:01\r",
      "\u001b[K    66% |#####################           | 8.6MB 28.0MB/s eta 0:00:01\r",
      "\u001b[K    66% |#####################           | 8.6MB 29.7MB/s eta 0:00:01\r",
      "\u001b[K    66% |#####################           | 8.6MB 31.0MB/s eta 0:00:01\r",
      "\u001b[K    67% |#####################           | 8.6MB 33.2MB/s eta 0:00:01\r",
      "\u001b[K    67% |#####################           | 8.6MB 36.0MB/s eta 0:00:01\r",
      "\u001b[K    67% |#####################           | 8.6MB 37.9MB/s eta 0:00:01\r",
      "\u001b[K    67% |#####################           | 8.6MB 42.6MB/s eta 0:00:01\r",
      "\u001b[K    67% |#####################           | 8.6MB 43.9MB/s eta 0:00:01\r",
      "\u001b[K    67% |#####################           | 8.7MB 46.8MB/s eta 0:00:01\r",
      "\u001b[K    67% |#####################           | 8.7MB 45.6MB/s eta 0:00:01\r",
      "\u001b[K    67% |#####################           | 8.7MB 44.0MB/s eta 0:00:01\r",
      "\u001b[K    67% |#####################           | 8.7MB 37.7MB/s eta 0:00:01\r",
      "\u001b[K    67% |#####################           | 8.7MB 38.1MB/s eta 0:00:01\r",
      "\u001b[K    67% |#####################           | 8.7MB 37.9MB/s eta 0:00:01\r",
      "\u001b[K    67% |#####################           | 8.7MB 33.7MB/s eta 0:00:01\r",
      "\u001b[K    67% |#####################           | 8.7MB 31.9MB/s eta 0:00:01\r",
      "\u001b[K    68% |#####################           | 8.7MB 31.1MB/s eta 0:00:01\r",
      "\u001b[K    68% |#####################           | 8.7MB 31.4MB/s eta 0:00:01\r",
      "\u001b[K    68% |#####################           | 8.8MB 29.1MB/s eta 0:00:01\r",
      "\u001b[K    68% |#####################           | 8.8MB 26.5MB/s eta 0:00:01\r",
      "\u001b[K    68% |#####################           | 8.8MB 25.7MB/s eta 0:00:01\r",
      "\u001b[K    68% |#####################           | 8.8MB 28.2MB/s eta 0:00:01\r",
      "\u001b[K    68% |#####################           | 8.8MB 28.1MB/s eta 0:00:01\r",
      "\u001b[K    68% |#####################           | 8.8MB 28.1MB/s eta 0:00:01\r",
      "\u001b[K    68% |#####################           | 8.8MB 30.8MB/s eta 0:00:01\r",
      "\u001b[K    68% |######################          | 8.8MB 32.7MB/s eta 0:00:01\r",
      "\u001b[K    68% |######################          | 8.8MB 33.2MB/s eta 0:00:01\r",
      "\u001b[K    68% |######################          | 8.8MB 33.6MB/s eta 0:00:01\r",
      "\u001b[K    69% |######################          | 8.9MB 37.0MB/s eta 0:00:01\r",
      "\u001b[K    69% |######################          | 8.9MB 44.1MB/s eta 0:00:01\r",
      "\u001b[K    69% |######################          | 8.9MB 48.2MB/s eta 0:00:01\r",
      "\u001b[K    69% |######################          | 8.9MB 49.0MB/s eta 0:00:01\r",
      "\u001b[K    69% |######################          | 8.9MB 46.4MB/s eta 0:00:01\r",
      "\u001b[K    69% |######################          | 8.9MB 45.6MB/s eta 0:00:01\r",
      "\u001b[K    69% |######################          | 8.9MB 41.3MB/s eta 0:00:01\r",
      "\u001b[K    69% |######################          | 8.9MB 39.7MB/s eta 0:00:01\r",
      "\u001b[K    69% |######################          | 8.9MB 40.2MB/s eta 0:00:01\r",
      "\u001b[K    69% |######################          | 8.9MB 37.4MB/s eta 0:00:01\r",
      "\u001b[K    69% |######################          | 9.0MB 36.7MB/s eta 0:00:01\r",
      "\u001b[K    69% |######################          | 9.0MB 36.4MB/s eta 0:00:01\r",
      "\u001b[K    69% |######################          | 9.0MB 36.3MB/s eta 0:00:01\r",
      "\u001b[K    70% |######################          | 9.0MB 36.8MB/s eta 0:00:01\r",
      "\u001b[K    70% |######################          | 9.0MB 38.1MB/s eta 0:00:01\r",
      "\u001b[K    70% |######################          | 9.0MB 38.8MB/s eta 0:00:01\r",
      "\u001b[K    70% |######################          | 9.0MB 40.0MB/s eta 0:00:01\r",
      "\u001b[K    70% |######################          | 9.0MB 37.8MB/s eta 0:00:01\r",
      "\u001b[K    70% |######################          | 9.0MB 36.8MB/s eta 0:00:01\r",
      "\u001b[K    70% |######################          | 9.1MB 39.3MB/s eta 0:00:01\r",
      "\u001b[K    70% |######################          | 9.1MB 39.9MB/s eta 0:00:01\r",
      "\u001b[K    70% |######################          | 9.1MB 40.7MB/s eta 0:00:01\r",
      "\u001b[K    70% |######################          | 9.1MB 40.8MB/s eta 0:00:01\r",
      "\u001b[K    70% |######################          | 9.1MB 39.8MB/s eta 0:00:01\r",
      "\u001b[K    70% |######################          | 9.1MB 38.7MB/s eta 0:00:01\r",
      "\u001b[K    71% |######################          | 9.1MB 36.3MB/s eta 0:00:01\r",
      "\u001b[K    71% |######################          | 9.1MB 38.5MB/s eta 0:00:01\r",
      "\u001b[K    71% |######################          | 9.1MB 41.6MB/s eta 0:00:01\r",
      "\u001b[K    71% |######################          | 9.1MB 39.5MB/s eta 0:00:01\r",
      "\u001b[K    71% |######################          | 9.2MB 38.0MB/s eta 0:00:01\r",
      "\u001b[K    71% |######################          | 9.2MB 36.9MB/s eta 0:00:01\r",
      "\u001b[K    71% |######################          | 9.2MB 35.0MB/s eta 0:00:01\r",
      "\u001b[K    71% |######################          | 9.2MB 35.5MB/s eta 0:00:01\r",
      "\u001b[K    71% |######################          | 9.2MB 35.3MB/s eta 0:00:01\r",
      "\u001b[K    71% |######################          | 9.2MB 36.7MB/s eta 0:00:01\r",
      "\u001b[K    71% |######################          | 9.2MB 39.3MB/s eta 0:00:01\r",
      "\u001b[K    71% |#######################         | 9.2MB 39.3MB/s eta 0:00:01\r",
      "\u001b[K    71% |#######################         | 9.2MB 40.2MB/s eta 0:00:01\r",
      "\u001b[K    72% |#######################         | 9.2MB 41.5MB/s eta 0:00:01\r",
      "\u001b[K    72% |#######################         | 9.3MB 40.2MB/s eta 0:00:01\r",
      "\u001b[K    72% |#######################         | 9.3MB 42.2MB/s eta 0:00:01\r",
      "\u001b[K    72% |#######################         | 9.3MB 44.1MB/s eta 0:00:01\r",
      "\u001b[K    72% |#######################         | 9.3MB 44.2MB/s eta 0:00:01\r",
      "\u001b[K    72% |#######################         | 9.3MB 45.3MB/s eta 0:00:01\r",
      "\u001b[K    72% |#######################         | 9.3MB 44.8MB/s eta 0:00:01\r",
      "\u001b[K    72% |#######################         | 9.3MB 41.7MB/s eta 0:00:01\r",
      "\u001b[K    72% |#######################         | 9.3MB 39.1MB/s eta 0:00:01\r",
      "\u001b[K    72% |#######################         | 9.3MB 36.5MB/s eta 0:00:01\r",
      "\u001b[K    72% |#######################         | 9.3MB 35.8MB/s eta 0:00:01\r",
      "\u001b[K    72% |#######################         | 9.4MB 38.4MB/s eta 0:00:01\r",
      "\u001b[K    73% |#######################         | 9.4MB 38.5MB/s eta 0:00:01\r",
      "\u001b[K    73% |#######################         | 9.4MB 38.8MB/s eta 0:00:01\r",
      "\u001b[K    73% |#######################         | 9.4MB 38.2MB/s eta 0:00:01\r",
      "\u001b[K    73% |#######################         | 9.4MB 38.8MB/s eta 0:00:01\r",
      "\u001b[K    73% |#######################         | 9.4MB 38.6MB/s eta 0:00:01\r",
      "\u001b[K    73% |#######################         | 9.4MB 41.6MB/s eta 0:00:01\r",
      "\u001b[K    73% |#######################         | 9.4MB 44.8MB/s eta 0:00:01\r",
      "\u001b[K    73% |#######################         | 9.4MB 48.6MB/s eta 0:00:01\r",
      "\u001b[K    73% |#######################         | 9.5MB 51.8MB/s eta 0:00:01\r",
      "\u001b[K    73% |#######################         | 9.5MB 52.5MB/s eta 0:00:01\r",
      "\u001b[K    73% |#######################         | 9.5MB 51.2MB/s eta 0:00:01\r",
      "\u001b[K    73% |#######################         | 9.5MB 50.1MB/s eta 0:00:01\r",
      "\u001b[K    73% |#######################         | 9.5MB 45.1MB/s eta 0:00:01\r",
      "\u001b[K    74% |#######################         | 9.5MB 41.2MB/s eta 0:00:01\r",
      "\u001b[K    74% |#######################         | 9.5MB 40.0MB/s eta 0:00:01\r",
      "\u001b[K    74% |#######################         | 9.5MB 37.4MB/s eta 0:00:01\r",
      "\u001b[K    74% |#######################         | 9.5MB 35.2MB/s eta 0:00:01\r",
      "\u001b[K    74% |#######################         | 9.5MB 35.0MB/s eta 0:00:01\r",
      "\u001b[K    74% |#######################         | 9.6MB 35.4MB/s eta 0:00:01\r",
      "\u001b[K    74% |#######################         | 9.6MB 35.4MB/s eta 0:00:01\r",
      "\u001b[K    74% |#######################         | 9.6MB 35.6MB/s eta 0:00:01\r",
      "\u001b[K    74% |#######################         | 9.6MB 33.7MB/s eta 0:00:01\r",
      "\u001b[K    74% |#######################         | 9.6MB 35.7MB/s eta 0:00:01\r",
      "\u001b[K    74% |#######################         | 9.6MB 37.8MB/s eta 0:00:01\r",
      "\u001b[K    74% |#######################         | 9.6MB 39.4MB/s eta 0:00:01\r",
      "\u001b[K    75% |########################        | 9.6MB 42.0MB/s eta 0:00:01\r",
      "\u001b[K    75% |########################        | 9.6MB 45.1MB/s eta 0:00:01\r",
      "\u001b[K    75% |########################        | 9.6MB 46.2MB/s eta 0:00:01\r",
      "\u001b[K    75% |########################        | 9.7MB 45.5MB/s eta 0:00:01\r",
      "\u001b[K    75% |########################        | 9.7MB 45.3MB/s eta 0:00:01\r",
      "\u001b[K    75% |########################        | 9.7MB 45.9MB/s eta 0:00:01\r",
      "\u001b[K    75% |########################        | 9.7MB 50.6MB/s eta 0:00:01\r",
      "\u001b[K    75% |########################        | 9.7MB 49.9MB/s eta 0:00:01\r",
      "\u001b[K    75% |########################        | 9.7MB 46.2MB/s eta 0:00:01\r",
      "\u001b[K    75% |########################        | 9.7MB 42.2MB/s eta 0:00:01\r",
      "\u001b[K    75% |########################        | 9.7MB 41.0MB/s eta 0:00:01\r",
      "\u001b[K    75% |########################        | 9.7MB 37.3MB/s eta 0:00:01\r",
      "\u001b[K    75% |########################        | 9.7MB 36.4MB/s eta 0:00:01\r",
      "\u001b[K    76% |########################        | 9.8MB 37.1MB/s eta 0:00:01\r",
      "\u001b[K    76% |########################        | 9.8MB 36.8MB/s eta 0:00:01\r",
      "\u001b[K    76% |########################        | 9.8MB 36.9MB/s eta 0:00:01\r",
      "\u001b[K    76% |########################        | 9.8MB 36.9MB/s eta 0:00:01\r",
      "\u001b[K    76% |########################        | 9.8MB 38.3MB/s eta 0:00:01\r",
      "\u001b[K    76% |########################        | 9.8MB 41.6MB/s eta 0:00:01\r",
      "\u001b[K    76% |########################        | 9.8MB 41.0MB/s eta 0:00:01\r",
      "\u001b[K    76% |########################        | 9.8MB 39.0MB/s eta 0:00:01\r",
      "\u001b[K    76% |########################        | 9.8MB 43.6MB/s eta 0:00:01\r",
      "\u001b[K    76% |########################        | 9.9MB 42.9MB/s eta 0:00:01\r",
      "\u001b[K    76% |########################        | 9.9MB 43.2MB/s eta 0:00:01\r",
      "\u001b[K    76% |########################        | 9.9MB 43.4MB/s eta 0:00:01\r",
      "\u001b[K    77% |########################        | 9.9MB 42.5MB/s eta 0:00:01\r",
      "\u001b[K    77% |########################        | 9.9MB 43.3MB/s eta 0:00:01\r",
      "\u001b[K    77% |########################        | 9.9MB 43.0MB/s eta 0:00:01\r",
      "\u001b[K    77% |########################        | 9.9MB 41.5MB/s eta 0:00:01\r",
      "\u001b[K    77% |########################        | 9.9MB 42.9MB/s eta 0:00:01\r",
      "\u001b[K    77% |########################        | 9.9MB 42.2MB/s eta 0:00:01\r",
      "\u001b[K    77% |########################        | 9.9MB 39.5MB/s eta 0:00:01\r",
      "\u001b[K    77% |########################        | 10.0MB 39.7MB/s eta 0:00:01\r",
      "\u001b[K    77% |########################        | 10.0MB 37.7MB/s eta 0:00:01\r",
      "\u001b[K    77% |########################        | 10.0MB 36.3MB/s eta 0:00:01\r",
      "\u001b[K    77% |########################        | 10.0MB 36.3MB/s eta 0:00:01\r",
      "\u001b[K    77% |########################        | 10.0MB 35.6MB/s eta 0:00:01\r",
      "\u001b[K    77% |########################        | 10.0MB 36.3MB/s eta 0:00:01\r",
      "\u001b[K    78% |########################        | 10.0MB 36.6MB/s eta 0:00:01\r",
      "\u001b[K    78% |########################        | 10.0MB 39.0MB/s eta 0:00:01\r",
      "\u001b[K    78% |#########################       | 10.0MB 43.0MB/s eta 0:00:01\r",
      "\u001b[K    78% |#########################       | 10.0MB 44.4MB/s eta 0:00:01\r",
      "\u001b[K    78% |#########################       | 10.1MB 44.4MB/s eta 0:00:01\r",
      "\u001b[K    78% |#########################       | 10.1MB 45.8MB/s eta 0:00:01\r",
      "\u001b[K    78% |#########################       | 10.1MB 45.9MB/s eta 0:00:01\r",
      "\u001b[K    78% |#########################       | 10.1MB 43.4MB/s eta 0:00:01\r",
      "\u001b[K    78% |#########################       | 10.1MB 42.3MB/s eta 0:00:01\r",
      "\u001b[K    78% |#########################       | 10.1MB 41.7MB/s eta 0:00:01\r",
      "\u001b[K    78% |#########################       | 10.1MB 41.1MB/s eta 0:00:01\r",
      "\u001b[K    78% |#########################       | 10.1MB 38.5MB/s eta 0:00:01\r",
      "\u001b[K    78% |#########################       | 10.1MB 37.4MB/s eta 0:00:01\r",
      "\u001b[K    79% |#########################       | 10.1MB 35.3MB/s eta 0:00:01\r",
      "\u001b[K    79% |#########################       | 10.2MB 33.0MB/s eta 0:00:01\r",
      "\u001b[K    79% |#########################       | 10.2MB 33.6MB/s eta 0:00:01\r",
      "\u001b[K    79% |#########################       | 10.2MB 30.2MB/s eta 0:00:01\r",
      "\u001b[K    79% |#########################       | 10.2MB 30.8MB/s eta 0:00:01\r",
      "\u001b[K    79% |#########################       | 10.2MB 31.5MB/s eta 0:00:01\r",
      "\u001b[K    79% |#########################       | 10.2MB 31.2MB/s eta 0:00:01\r",
      "\u001b[K    79% |#########################       | 10.2MB 32.0MB/s eta 0:00:01\r",
      "\u001b[K    79% |#########################       | 10.2MB 33.4MB/s eta 0:00:01\r",
      "\u001b[K    79% |#########################       | 10.2MB 34.3MB/s eta 0:00:01\r",
      "\u001b[K    79% |#########################       | 10.3MB 37.6MB/s eta 0:00:01\r",
      "\u001b[K    79% |#########################       | 10.3MB 41.3MB/s eta 0:00:01\r",
      "\u001b[K    80% |#########################       | 10.3MB 41.3MB/s eta 0:00:01\r",
      "\u001b[K    80% |#########################       | 10.3MB 50.3MB/s eta 0:00:01\r",
      "\u001b[K    80% |#########################       | 10.3MB 51.9MB/s eta 0:00:01\r",
      "\u001b[K    80% |#########################       | 10.3MB 53.0MB/s eta 0:00:01\r",
      "\u001b[K    80% |#########################       | 10.3MB 53.7MB/s eta 0:00:01\r",
      "\u001b[K    80% |#########################       | 10.3MB 51.0MB/s eta 0:00:01\r",
      "\u001b[K    80% |#########################       | 10.3MB 46.5MB/s eta 0:00:01\r",
      "\u001b[K    80% |#########################       | 10.3MB 41.7MB/s eta 0:00:01\r",
      "\u001b[K    80% |#########################       | 10.4MB 37.6MB/s eta 0:00:01\r",
      "\u001b[K    80% |#########################       | 10.4MB 33.4MB/s eta 0:00:01\r",
      "\u001b[K    80% |#########################       | 10.4MB 31.9MB/s eta 0:00:01\r",
      "\u001b[K    80% |#########################       | 10.4MB 32.2MB/s eta 0:00:01\r",
      "\u001b[K    80% |#########################       | 10.4MB 32.1MB/s eta 0:00:01\r",
      "\u001b[K    81% |#########################       | 10.4MB 31.7MB/s eta 0:00:01\r",
      "\u001b[K    81% |#########################       | 10.4MB 30.7MB/s eta 0:00:01\r",
      "\u001b[K    81% |#########################       | 10.4MB 30.4MB/s eta 0:00:01\r",
      "\u001b[K    81% |##########################      | 10.4MB 32.7MB/s eta 0:00:01\r",
      "\u001b[K    81% |##########################      | 10.4MB 35.5MB/s eta 0:00:01\r",
      "\u001b[K    81% |##########################      | 10.5MB 38.6MB/s eta 0:00:01\r",
      "\u001b[K    81% |##########################      | 10.5MB 44.1MB/s eta 0:00:01\r",
      "\u001b[K    81% |##########################      | 10.5MB 47.4MB/s eta 0:00:01\r",
      "\u001b[K    81% |##########################      | 10.5MB 46.3MB/s eta 0:00:01\r",
      "\u001b[K    81% |##########################      | 10.5MB 47.4MB/s eta 0:00:01\r",
      "\u001b[K    81% |##########################      | 10.5MB 47.2MB/s eta 0:00:01\r",
      "\u001b[K    81% |##########################      | 10.5MB 50.6MB/s eta 0:00:01\r",
      "\u001b[K    82% |##########################      | 10.5MB 53.6MB/s eta 0:00:01\r",
      "\u001b[K    82% |##########################      | 10.5MB 52.7MB/s eta 0:00:01\r",
      "\u001b[K    82% |##########################      | 10.5MB 49.3MB/s eta 0:00:01\r",
      "\u001b[K    82% |##########################      | 10.6MB 43.9MB/s eta 0:00:01\r",
      "\u001b[K    82% |##########################      | 10.6MB 39.8MB/s eta 0:00:01\r",
      "\u001b[K    82% |##########################      | 10.6MB 36.6MB/s eta 0:00:01\r",
      "\u001b[K    82% |##########################      | 10.6MB 35.9MB/s eta 0:00:01\r",
      "\u001b[K    82% |##########################      | 10.6MB 31.6MB/s eta 0:00:01\r",
      "\u001b[K    82% |##########################      | 10.6MB 28.9MB/s eta 0:00:01\r",
      "\u001b[K    82% |##########################      | 10.6MB 27.3MB/s eta 0:00:01\r",
      "\u001b[K    82% |##########################      | 10.6MB 26.2MB/s eta 0:00:01\r",
      "\u001b[K    82% |##########################      | 10.6MB 26.0MB/s eta 0:00:01\r",
      "\u001b[K    82% |##########################      | 10.6MB 26.8MB/s eta 0:00:01\r",
      "\u001b[K    83% |##########################      | 10.7MB 28.9MB/s eta 0:00:01\r",
      "\u001b[K    83% |##########################      | 10.7MB 31.1MB/s eta 0:00:01\r",
      "\u001b[K    83% |##########################      | 10.7MB 33.1MB/s eta 0:00:01\r",
      "\u001b[K    83% |##########################      | 10.7MB 33.8MB/s eta 0:00:01\r",
      "\u001b[K    83% |##########################      | 10.7MB 38.3MB/s eta 0:00:01\r",
      "\u001b[K    83% |##########################      | 10.7MB 43.6MB/s eta 0:00:01\r",
      "\u001b[K    83% |##########################      | 10.7MB 47.0MB/s eta 0:00:01\r",
      "\u001b[K    83% |##########################      | 10.7MB 50.2MB/s eta 0:00:01\r",
      "\u001b[K    83% |##########################      | 10.7MB 51.4MB/s eta 0:00:01\r",
      "\u001b[K    83% |##########################      | 10.8MB 51.6MB/s eta 0:00:01\r",
      "\u001b[K    83% |##########################      | 10.8MB 51.6MB/s eta 0:00:01\r",
      "\u001b[K    83% |##########################      | 10.8MB 49.2MB/s eta 0:00:01\r",
      "\u001b[K    84% |##########################      | 10.8MB 43.6MB/s eta 0:00:01\r",
      "\u001b[K    84% |##########################      | 10.8MB 40.6MB/s eta 0:00:01\r",
      "\u001b[K    84% |##########################      | 10.8MB 39.9MB/s eta 0:00:01\r",
      "\u001b[K    84% |##########################      | 10.8MB 37.4MB/s eta 0:00:01\r",
      "\u001b[K    84% |##########################      | 10.8MB 35.7MB/s eta 0:00:01\r",
      "\u001b[K    84% |###########################     | 10.8MB 34.8MB/s eta 0:00:01\r",
      "\u001b[K    84% |###########################     | 10.8MB 32.6MB/s eta 0:00:01\r",
      "\u001b[K    84% |###########################     | 10.9MB 31.5MB/s eta 0:00:01\r",
      "\u001b[K    84% |###########################     | 10.9MB 29.7MB/s eta 0:00:01\r",
      "\u001b[K    84% |###########################     | 10.9MB 30.9MB/s eta 0:00:01\r",
      "\u001b[K    84% |###########################     | 10.9MB 33.3MB/s eta 0:00:01\r",
      "\u001b[K    84% |###########################     | 10.9MB 35.8MB/s eta 0:00:01\r",
      "\u001b[K    84% |###########################     | 10.9MB 36.2MB/s eta 0:00:01\r",
      "\u001b[K    85% |###########################     | 10.9MB 37.9MB/s eta 0:00:01\r",
      "\u001b[K    85% |###########################     | 10.9MB 40.1MB/s eta 0:00:01\r",
      "\u001b[K    85% |###########################     | 10.9MB 41.6MB/s eta 0:00:01\r",
      "\u001b[K    85% |###########################     | 10.9MB 44.7MB/s eta 0:00:01\r",
      "\u001b[K    85% |###########################     | 11.0MB 47.1MB/s eta 0:00:01\r",
      "\u001b[K    85% |###########################     | 11.0MB 51.2MB/s eta 0:00:01\r",
      "\u001b[K    85% |###########################     | 11.0MB 48.7MB/s eta 0:00:01\r",
      "\u001b[K    85% |###########################     | 11.0MB 45.6MB/s eta 0:00:01\r",
      "\u001b[K    85% |###########################     | 11.0MB 42.7MB/s eta 0:00:01\r",
      "\u001b[K    85% |###########################     | 11.0MB 40.8MB/s eta 0:00:01\r",
      "\u001b[K    85% |###########################     | 11.0MB 39.5MB/s eta 0:00:01\r",
      "\u001b[K    85% |###########################     | 11.0MB 36.2MB/s eta 0:00:01\r",
      "\u001b[K    86% |###########################     | 11.0MB 34.8MB/s eta 0:00:01\r",
      "\u001b[K    86% |###########################     | 11.0MB 33.0MB/s eta 0:00:01\r",
      "\u001b[K    86% |###########################     | 11.1MB 32.8MB/s eta 0:00:01\r",
      "\u001b[K    86% |###########################     | 11.1MB 32.9MB/s eta 0:00:01\r",
      "\u001b[K    86% |###########################     | 11.1MB 32.9MB/s eta 0:00:01\r",
      "\u001b[K    86% |###########################     | 11.1MB 34.7MB/s eta 0:00:01\r",
      "\u001b[K    86% |###########################     | 11.1MB 35.9MB/s eta 0:00:01\r",
      "\u001b[K    86% |###########################     | 11.1MB 37.2MB/s eta 0:00:01\r",
      "\u001b[K    86% |###########################     | 11.1MB 39.2MB/s eta 0:00:01\r",
      "\u001b[K    86% |###########################     | 11.1MB 42.3MB/s eta 0:00:01\r",
      "\u001b[K    86% |###########################     | 11.1MB 44.3MB/s eta 0:00:01\r",
      "\u001b[K    86% |###########################     | 11.2MB 48.0MB/s eta 0:00:01\r",
      "\u001b[K    86% |###########################     | 11.2MB 48.4MB/s eta 0:00:01\r",
      "\u001b[K    87% |###########################     | 11.2MB 48.1MB/s eta 0:00:01\r",
      "\u001b[K    87% |###########################     | 11.2MB 48.6MB/s eta 0:00:01\r",
      "\u001b[K    87% |###########################     | 11.2MB 43.6MB/s eta 0:00:01\r",
      "\u001b[K    87% |###########################     | 11.2MB 41.6MB/s eta 0:00:01\r",
      "\u001b[K    87% |###########################     | 11.2MB 38.7MB/s eta 0:00:01\r",
      "\u001b[K    87% |###########################     | 11.2MB 35.2MB/s eta 0:00:01\r",
      "\u001b[K    87% |############################    | 11.2MB 34.7MB/s eta 0:00:01\r",
      "\u001b[K    87% |############################    | 11.2MB 34.6MB/s eta 0:00:01\r",
      "\u001b[K    87% |############################    | 11.3MB 34.6MB/s eta 0:00:01\r",
      "\u001b[K    87% |############################    | 11.3MB 34.3MB/s eta 0:00:01\r",
      "\u001b[K    87% |############################    | 11.3MB 34.2MB/s eta 0:00:01\r",
      "\u001b[K    87% |############################    | 11.3MB 35.0MB/s eta 0:00:01\r",
      "\u001b[K    88% |############################    | 11.3MB 38.0MB/s eta 0:00:01\r",
      "\u001b[K    88% |############################    | 11.3MB 40.1MB/s eta 0:00:01\r",
      "\u001b[K    88% |############################    | 11.3MB 43.8MB/s eta 0:00:01\r",
      "\u001b[K    88% |############################    | 11.3MB 48.8MB/s eta 0:00:01\r",
      "\u001b[K    88% |############################    | 11.3MB 51.6MB/s eta 0:00:01\r",
      "\u001b[K    88% |############################    | 11.3MB 53.0MB/s eta 0:00:01\r",
      "\u001b[K    88% |############################    | 11.4MB 52.0MB/s eta 0:00:01\r",
      "\u001b[K    88% |############################    | 11.4MB 50.3MB/s eta 0:00:01\r",
      "\u001b[K    88% |############################    | 11.4MB 46.6MB/s eta 0:00:01\r",
      "\u001b[K    88% |############################    | 11.4MB 45.1MB/s eta 0:00:01\r",
      "\u001b[K    88% |############################    | 11.4MB 41.8MB/s eta 0:00:01\r",
      "\u001b[K    88% |############################    | 11.4MB 39.1MB/s eta 0:00:01\r",
      "\u001b[K    88% |############################    | 11.4MB 38.1MB/s eta 0:00:01\r",
      "\u001b[K    89% |############################    | 11.4MB 35.5MB/s eta 0:00:01\r",
      "\u001b[K    89% |############################    | 11.4MB 33.7MB/s eta 0:00:01\r",
      "\u001b[K    89% |############################    | 11.4MB 33.5MB/s eta 0:00:01\r",
      "\u001b[K    89% |############################    | 11.5MB 32.5MB/s eta 0:00:01\r",
      "\u001b[K    89% |############################    | 11.5MB 32.0MB/s eta 0:00:01\r",
      "\u001b[K    89% |############################    | 11.5MB 34.1MB/s eta 0:00:01\r",
      "\u001b[K    89% |############################    | 11.5MB 34.6MB/s eta 0:00:01\r",
      "\u001b[K    89% |############################    | 11.5MB 37.5MB/s eta 0:00:01\r",
      "\u001b[K    89% |############################    | 11.5MB 39.6MB/s eta 0:00:01\r",
      "\u001b[K    89% |############################    | 11.5MB 40.7MB/s eta 0:00:01\r",
      "\u001b[K    89% |############################    | 11.5MB 44.1MB/s eta 0:00:01\r",
      "\u001b[K    89% |############################    | 11.5MB 46.1MB/s eta 0:00:01\r",
      "\u001b[K    90% |############################    | 11.6MB 45.3MB/s eta 0:00:01\r",
      "\u001b[K    90% |############################    | 11.6MB 47.8MB/s eta 0:00:01\r",
      "\u001b[K    90% |############################    | 11.6MB 46.0MB/s eta 0:00:01\r",
      "\u001b[K    90% |############################    | 11.6MB 43.5MB/s eta 0:00:01\r",
      "\u001b[K    90% |############################    | 11.6MB 41.1MB/s eta 0:00:01\r",
      "\u001b[K    90% |############################    | 11.6MB 36.5MB/s eta 0:00:01\r",
      "\u001b[K    90% |############################    | 11.6MB 34.7MB/s eta 0:00:01\r",
      "\u001b[K    90% |############################    | 11.6MB 33.8MB/s eta 0:00:01\r",
      "\u001b[K    90% |#############################   | 11.6MB 28.9MB/s eta 0:00:01\r",
      "\u001b[K    90% |#############################   | 11.6MB 28.2MB/s eta 0:00:01\r",
      "\u001b[K    90% |#############################   | 11.7MB 28.2MB/s eta 0:00:01\r",
      "\u001b[K    90% |#############################   | 11.7MB 27.3MB/s eta 0:00:01\r",
      "\u001b[K    90% |#############################   | 11.7MB 27.0MB/s eta 0:00:01\r",
      "\u001b[K    91% |#############################   | 11.7MB 27.5MB/s eta 0:00:01\r",
      "\u001b[K    91% |#############################   | 11.7MB 28.8MB/s eta 0:00:01\r",
      "\u001b[K    91% |#############################   | 11.7MB 31.1MB/s eta 0:00:01\r",
      "\u001b[K    91% |#############################   | 11.7MB 32.7MB/s eta 0:00:01\r",
      "\u001b[K    91% |#############################   | 11.7MB 33.5MB/s eta 0:00:01\r",
      "\u001b[K    91% |#############################   | 11.7MB 39.5MB/s eta 0:00:01\r",
      "\u001b[K    91% |#############################   | 11.7MB 39.9MB/s eta 0:00:01\r",
      "\u001b[K    91% |#############################   | 11.8MB 39.9MB/s eta 0:00:01\r",
      "\u001b[K    91% |#############################   | 11.8MB 40.7MB/s eta 0:00:01\r",
      "\u001b[K    91% |#############################   | 11.8MB 42.9MB/s eta 0:00:01\r",
      "\u001b[K    91% |#############################   | 11.8MB 40.8MB/s eta 0:00:01\r",
      "\u001b[K    91% |#############################   | 11.8MB 37.5MB/s eta 0:00:01\r",
      "\u001b[K    92% |#############################   | 11.8MB 35.9MB/s eta 0:00:01\r",
      "\u001b[K    92% |#############################   | 11.8MB 35.4MB/s eta 0:00:01\r",
      "\u001b[K    92% |#############################   | 11.8MB 35.4MB/s eta 0:00:01\r",
      "\u001b[K    92% |#############################   | 11.8MB 36.0MB/s eta 0:00:01\r",
      "\u001b[K    92% |#############################   | 11.8MB 36.7MB/s eta 0:00:01\r",
      "\u001b[K    92% |#############################   | 11.9MB 37.4MB/s eta 0:00:01\r",
      "\u001b[K    92% |#############################   | 11.9MB 36.4MB/s eta 0:00:01\r",
      "\u001b[K    92% |#############################   | 11.9MB 35.4MB/s eta 0:00:01\r",
      "\u001b[K    92% |#############################   | 11.9MB 37.7MB/s eta 0:00:01\r",
      "\u001b[K    92% |#############################   | 11.9MB 40.5MB/s eta 0:00:01\r",
      "\u001b[K    92% |#############################   | 11.9MB 43.0MB/s eta 0:00:01\r",
      "\u001b[K    92% |#############################   | 11.9MB 43.5MB/s eta 0:00:01\r",
      "\u001b[K    92% |#############################   | 11.9MB 42.9MB/s eta 0:00:01\r",
      "\u001b[K    93% |#############################   | 11.9MB 43.1MB/s eta 0:00:01\r",
      "\u001b[K    93% |#############################   | 12.0MB 43.0MB/s eta 0:00:01\r",
      "\u001b[K    93% |#############################   | 12.0MB 42.4MB/s eta 0:00:01\r",
      "\u001b[K    93% |#############################   | 12.0MB 45.3MB/s eta 0:00:01\r",
      "\u001b[K    93% |#############################   | 12.0MB 48.3MB/s eta 0:00:01\r",
      "\u001b[K    93% |#############################   | 12.0MB 43.0MB/s eta 0:00:01\r",
      "\u001b[K    93% |#############################   | 12.0MB 40.4MB/s eta 0:00:01\r",
      "\u001b[K    93% |#############################   | 12.0MB 37.3MB/s eta 0:00:01\r",
      "\u001b[K    93% |#############################   | 12.0MB 35.4MB/s eta 0:00:01\r",
      "\u001b[K    93% |##############################  | 12.0MB 34.7MB/s eta 0:00:01\r",
      "\u001b[K    93% |##############################  | 12.0MB 34.5MB/s eta 0:00:01\r",
      "\u001b[K    93% |##############################  | 12.1MB 34.8MB/s eta 0:00:01\r",
      "\u001b[K    93% |##############################  | 12.1MB 34.8MB/s eta 0:00:01\r",
      "\u001b[K    94% |##############################  | 12.1MB 32.6MB/s eta 0:00:01\r",
      "\u001b[K    94% |##############################  | 12.1MB 31.8MB/s eta 0:00:01\r",
      "\u001b[K    94% |##############################  | 12.1MB 34.3MB/s eta 0:00:01\r",
      "\u001b[K    94% |##############################  | 12.1MB 36.8MB/s eta 0:00:01\r",
      "\u001b[K    94% |##############################  | 12.1MB 39.9MB/s eta 0:00:01\r",
      "\u001b[K    94% |##############################  | 12.1MB 42.9MB/s eta 0:00:01\r",
      "\u001b[K    94% |##############################  | 12.1MB 44.5MB/s eta 0:00:01\r",
      "\u001b[K    94% |##############################  | 12.1MB 44.0MB/s eta 0:00:01\r",
      "\u001b[K    94% |##############################  | 12.2MB 41.4MB/s eta 0:00:01\r",
      "\u001b[K    94% |##############################  | 12.2MB 41.2MB/s eta 0:00:01\r",
      "\u001b[K    94% |##############################  | 12.2MB 44.1MB/s eta 0:00:01\r",
      "\u001b[K    94% |##############################  | 12.2MB 44.0MB/s eta 0:00:01\r",
      "\u001b[K    95% |##############################  | 12.2MB 40.9MB/s eta 0:00:01\r",
      "\u001b[K    95% |##############################  | 12.2MB 38.2MB/s eta 0:00:01\r",
      "\u001b[K    95% |##############################  | 12.2MB 35.6MB/s eta 0:00:01\r",
      "\u001b[K    95% |##############################  | 12.2MB 33.4MB/s eta 0:00:01\r",
      "\u001b[K    95% |##############################  | 12.2MB 33.5MB/s eta 0:00:01\r",
      "\u001b[K    95% |##############################  | 12.2MB 33.4MB/s eta 0:00:01\r",
      "\u001b[K    95% |##############################  | 12.3MB 35.0MB/s eta 0:00:01\r",
      "\u001b[K    95% |##############################  | 12.3MB 35.5MB/s eta 0:00:01\r",
      "\u001b[K    95% |##############################  | 12.3MB 32.6MB/s eta 0:00:01\r",
      "\u001b[K    95% |##############################  | 12.3MB 30.7MB/s eta 0:00:01\r",
      "\u001b[K    95% |##############################  | 12.3MB 31.9MB/s eta 0:00:01\r",
      "\u001b[K    95% |##############################  | 12.3MB 33.1MB/s eta 0:00:01\r",
      "\u001b[K    95% |##############################  | 12.3MB 35.3MB/s eta 0:00:01\r",
      "\u001b[K    96% |##############################  | 12.3MB 37.6MB/s eta 0:00:01\r",
      "\u001b[K    96% |##############################  | 12.3MB 35.9MB/s eta 0:00:01\r",
      "\u001b[K    96% |##############################  | 12.3MB 35.4MB/s eta 0:00:01\r",
      "\u001b[K    96% |##############################  | 12.4MB 35.0MB/s eta 0:00:01\r",
      "\u001b[K    96% |##############################  | 12.4MB 34.1MB/s eta 0:00:01\r",
      "\u001b[K    96% |##############################  | 12.4MB 37.8MB/s eta 0:00:01\r",
      "\u001b[K    96% |##############################  | 12.4MB 43.5MB/s eta 0:00:01\r",
      "\u001b[K    96% |##############################  | 12.4MB 44.9MB/s eta 0:00:01\r",
      "\u001b[K    96% |##############################  | 12.4MB 46.1MB/s eta 0:00:01\r",
      "\u001b[K    96% |##############################  | 12.4MB 46.0MB/s eta 0:00:01\r",
      "\u001b[K    96% |##############################  | 12.4MB 46.8MB/s eta 0:00:01\r",
      "\u001b[K    96% |############################### | 12.4MB 49.8MB/s eta 0:00:01\r",
      "\u001b[K    97% |############################### | 12.5MB 51.9MB/s eta 0:00:01\r",
      "\u001b[K    97% |############################### | 12.5MB 54.2MB/s eta 0:00:01\r",
      "\u001b[K    97% |############################### | 12.5MB 55.8MB/s eta 0:00:01\r",
      "\u001b[K    97% |############################### | 12.5MB 56.5MB/s eta 0:00:01\r",
      "\u001b[K    97% |############################### | 12.5MB 57.2MB/s eta 0:00:01\r",
      "\u001b[K    97% |############################### | 12.5MB 57.0MB/s eta 0:00:01\r",
      "\u001b[K    97% |############################### | 12.5MB 57.6MB/s eta 0:00:01\r",
      "\u001b[K    97% |############################### | 12.5MB 57.5MB/s eta 0:00:01\r",
      "\u001b[K    97% |############################### | 12.5MB 56.8MB/s eta 0:00:01\r",
      "\u001b[K    97% |############################### | 12.5MB 57.2MB/s eta 0:00:01\r",
      "\u001b[K    97% |############################### | 12.6MB 56.3MB/s eta 0:00:01\r",
      "\u001b[K    97% |############################### | 12.6MB 55.8MB/s eta 0:00:01\r",
      "\u001b[K    97% |############################### | 12.6MB 55.3MB/s eta 0:00:01\r",
      "\u001b[K    98% |############################### | 12.6MB 54.0MB/s eta 0:00:01\r",
      "\u001b[K    98% |############################### | 12.6MB 53.4MB/s eta 0:00:01\r",
      "\u001b[K    98% |############################### | 12.6MB 52.9MB/s eta 0:00:01\r",
      "\u001b[K    98% |############################### | 12.6MB 51.7MB/s eta 0:00:01\r",
      "\u001b[K    98% |############################### | 12.6MB 51.7MB/s eta 0:00:01\r",
      "\u001b[K    98% |############################### | 12.6MB 51.2MB/s eta 0:00:01\r",
      "\u001b[K    98% |############################### | 12.6MB 51.0MB/s eta 0:00:01\r",
      "\u001b[K    98% |############################### | 12.7MB 51.1MB/s eta 0:00:01\r",
      "\u001b[K    98% |############################### | 12.7MB 49.4MB/s eta 0:00:01\r",
      "\u001b[K    98% |############################### | 12.7MB 50.5MB/s eta 0:00:01\r",
      "\u001b[K    98% |############################### | 12.7MB 50.1MB/s eta 0:00:01\r",
      "\u001b[K    98% |############################### | 12.7MB 50.2MB/s eta 0:00:01\r",
      "\u001b[K    99% |############################### | 12.7MB 50.7MB/s eta 0:00:01\r",
      "\u001b[K    99% |############################### | 12.7MB 50.6MB/s eta 0:00:01\r",
      "\u001b[K    99% |############################### | 12.7MB 50.6MB/s eta 0:00:01\r",
      "\u001b[K    99% |############################### | 12.7MB 50.5MB/s eta 0:00:01\r",
      "\u001b[K    99% |############################### | 12.7MB 49.6MB/s eta 0:00:01\r",
      "\u001b[K    99% |############################### | 12.8MB 50.5MB/s eta 0:00:01\r",
      "\u001b[K    99% |############################### | 12.8MB 50.9MB/s eta 0:00:01\r",
      "\u001b[K    99% |############################### | 12.8MB 50.2MB/s eta 0:00:01\r",
      "\u001b[K    99% |############################### | 12.8MB 51.0MB/s eta 0:00:01\r",
      "\u001b[K    99% |############################### | 12.8MB 50.5MB/s eta 0:00:01\r",
      "\u001b[K    99% |############################### | 12.8MB 48.9MB/s eta 0:00:01\r",
      "\u001b[K    99% |############################### | 12.8MB 49.0MB/s eta 0:00:01\r",
      "\u001b[K    99% |############################### | 12.8MB 48.7MB/s eta 0:00:01\r",
      "\u001b[K    100% |################################| 12.8MB 117kB/s \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[?25hCollecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 (from matplotlib)\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m  Downloading https://files.pythonhosted.org/packages/42/47/e6d51aef3d0393f7d343592d63a73beee2a8d3d69c22b053e252c6cfacd5/pyparsing-2.2.1-py2.py3-none-any.whl (57kB)\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[?25l\r",
      "\u001b[K    17% |#####                           | 10kB 42.7MB/s eta 0:00:01\r",
      "\u001b[K    35% |###########                     | 20kB 39.7MB/s eta 0:00:01\r",
      "\u001b[K    53% |#################               | 30kB 41.7MB/s eta 0:00:01\r",
      "\u001b[K    71% |######################          | 40kB 42.1MB/s eta 0:00:01\r",
      "\u001b[K    88% |############################    | 51kB 42.8MB/s eta 0:00:01\r",
      "\u001b[K    100% |################################| 61kB 11.0MB/s \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[?25hRequirement already satisfied (use --upgrade to upgrade): numpy>=1.10.0 in /usr/local/lib/python3.5/dist-packages (from matplotlib)\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mCollecting cycler>=0.10 (from matplotlib)\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m  Downloading https://files.pythonhosted.org/packages/f7/d2/e07d3ebb2bd7af696440ce7e754c59dd546ffe1bbe732c8ab68b9c834e61/cycler-0.10.0-py2.py3-none-any.whl\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mCollecting python-dateutil>=2.1 (from matplotlib)\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m  Downloading https://files.pythonhosted.org/packages/cf/f5/af2b09c957ace60dcfac112b669c45c8c97e32f94aa8b56da4c6d1682825/python_dateutil-2.7.3-py2.py3-none-any.whl (211kB)\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[?25l\r",
      "\u001b[K    4% |#                               | 10kB 16.8MB/s eta 0:00:01\r",
      "\u001b[K    9% |###                             | 20kB 23.6MB/s eta 0:00:01\r",
      "\u001b[K    14% |####                            | 30kB 28.4MB/s eta 0:00:01\r",
      "\u001b[K    19% |######                          | 40kB 30.9MB/s eta 0:00:01\r",
      "\u001b[K    24% |#######                         | 51kB 33.3MB/s eta 0:00:01\r",
      "\u001b[K    29% |#########                       | 61kB 36.1MB/s eta 0:00:01\r",
      "\u001b[K    33% |##########                      | 71kB 37.5MB/s eta 0:00:01\r",
      "\u001b[K    38% |############                    | 81kB 39.1MB/s eta 0:00:01\r",
      "\u001b[K    43% |#############                   | 92kB 40.4MB/s eta 0:00:01\r",
      "\u001b[K    48% |###############                 | 102kB 41.5MB/s eta 0:00:01\r",
      "\u001b[K    53% |#################               | 112kB 49.9MB/s eta 0:00:01\r",
      "\u001b[K    58% |##################              | 122kB 51.4MB/s eta 0:00:01\r",
      "\u001b[K    62% |####################            | 133kB 51.4MB/s eta 0:00:01\r",
      "\u001b[K    67% |#####################           | 143kB 53.1MB/s eta 0:00:01\r",
      "\u001b[K    72% |#######################         | 153kB 53.2MB/s eta 0:00:01\r",
      "\u001b[K    77% |########################        | 163kB 53.3MB/s eta 0:00:01\r",
      "\u001b[K    82% |##########################      | 174kB 53.3MB/s eta 0:00:01\r",
      "\u001b[K    87% |###########################     | 184kB 51.6MB/s eta 0:00:01\r",
      "\u001b[K    92% |#############################   | 194kB 51.4MB/s eta 0:00:01\r",
      "\u001b[K    96% |##############################  | 204kB 50.8MB/s eta 0:00:01\r",
      "\u001b[K    100% |################################| 215kB 6.5MB/s \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[?25hCollecting kiwisolver>=1.0.1 (from matplotlib)\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m  Downloading https://files.pythonhosted.org/packages/7e/31/d6fedd4fb2c94755cd101191e581af30e1650ccce7a35bddb7930fed6574/kiwisolver-1.0.1-cp35-cp35m-manylinux1_x86_64.whl (949kB)\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[?25l\r",
      "\u001b[K    1% |                                | 10kB 25.4MB/s eta 0:00:01\r",
      "\u001b[K    2% |                                | 20kB 22.5MB/s eta 0:00:01\r",
      "\u001b[K    3% |#                               | 30kB 20.4MB/s eta 0:00:01\r",
      "\u001b[K    4% |#                               | 40kB 21.8MB/s eta 0:00:01\r",
      "\u001b[K    5% |#                               | 51kB 22.1MB/s eta 0:00:01\r",
      "\u001b[K    6% |##                              | 61kB 22.0MB/s eta 0:00:01\r",
      "\u001b[K    7% |##                              | 71kB 23.6MB/s eta 0:00:01\r",
      "\u001b[K    8% |##                              | 81kB 25.1MB/s eta 0:00:01\r",
      "\u001b[K    9% |###                             | 92kB 26.7MB/s eta 0:00:01\r",
      "\u001b[K    10% |###                             | 102kB 27.7MB/s eta 0:00:01\r",
      "\u001b[K    11% |###                             | 112kB 27.8MB/s eta 0:00:01\r",
      "\u001b[K    12% |####                            | 122kB 30.3MB/s eta 0:00:01\r",
      "\u001b[K    14% |####                            | 133kB 34.4MB/s eta 0:00:01\r",
      "\u001b[K    15% |####                            | 143kB 37.1MB/s eta 0:00:01\r",
      "\u001b[K    16% |#####                           | 153kB 40.2MB/s eta 0:00:01\r",
      "\u001b[K    17% |#####                           | 163kB 45.6MB/s eta 0:00:01\r",
      "\u001b[K    18% |#####                           | 174kB 47.9MB/s eta 0:00:01\r",
      "\u001b[K    19% |######                          | 184kB 48.0MB/s eta 0:00:01\r",
      "\u001b[K    20% |######                          | 194kB 48.9MB/s eta 0:00:01\r",
      "\u001b[K    21% |######                          | 204kB 50.9MB/s eta 0:00:01\r",
      "\u001b[K    22% |#######                         | 215kB 57.1MB/s eta 0:00:01\r",
      "\u001b[K    23% |#######                         | 225kB 56.5MB/s eta 0:00:01\r",
      "\u001b[K    24% |#######                         | 235kB 47.7MB/s eta 0:00:01\r",
      "\u001b[K    25% |########                        | 245kB 38.9MB/s eta 0:00:01\r",
      "\u001b[K    26% |########                        | 256kB 34.6MB/s eta 0:00:01\r",
      "\u001b[K    28% |########                        | 266kB 32.2MB/s eta 0:00:01\r",
      "\u001b[K    29% |#########                       | 276kB 29.2MB/s eta 0:00:01\r",
      "\u001b[K    30% |#########                       | 286kB 28.3MB/s eta 0:00:01\r",
      "\u001b[K    31% |##########                      | 296kB 25.6MB/s eta 0:00:01\r",
      "\u001b[K    32% |##########                      | 307kB 24.3MB/s eta 0:00:01\r",
      "\u001b[K    33% |##########                      | 317kB 23.2MB/s eta 0:00:01\r",
      "\u001b[K    34% |###########                     | 327kB 21.4MB/s eta 0:00:01\r",
      "\u001b[K    35% |###########                     | 337kB 22.8MB/s eta 0:00:01\r",
      "\u001b[K    36% |###########                     | 348kB 25.1MB/s eta 0:00:01\r",
      "\u001b[K    37% |############                    | 358kB 27.8MB/s eta 0:00:01\r",
      "\u001b[K    38% |############                    | 368kB 29.4MB/s eta 0:00:01\r",
      "\u001b[K    39% |############                    | 378kB 32.2MB/s eta 0:00:01\r",
      "\u001b[K    40% |#############                   | 389kB 34.2MB/s eta 0:00:01\r",
      "\u001b[K    42% |#############                   | 399kB 37.9MB/s eta 0:00:01\r",
      "\u001b[K    43% |#############                   | 409kB 38.2MB/s eta 0:00:01\r",
      "\u001b[K    44% |##############                  | 419kB 41.4MB/s eta 0:00:01\r",
      "\u001b[K    45% |##############                  | 430kB 49.2MB/s eta 0:00:01\r",
      "\u001b[K    46% |##############                  | 440kB 50.7MB/s eta 0:00:01\r",
      "\u001b[K    47% |###############                 | 450kB 51.7MB/s eta 0:00:01\r",
      "\u001b[K    48% |###############                 | 460kB 50.9MB/s eta 0:00:01\r",
      "\u001b[K    49% |###############                 | 471kB 51.7MB/s eta 0:00:01\r",
      "\u001b[K    50% |################                | 481kB 50.4MB/s eta 0:00:01\r",
      "\u001b[K    51% |################                | 491kB 49.5MB/s eta 0:00:01\r",
      "\u001b[K    52% |################                | 501kB 51.3MB/s eta 0:00:01\r",
      "\u001b[K    53% |#################               | 512kB 55.3MB/s eta 0:00:01\r",
      "\u001b[K    55% |#################               | 522kB 55.4MB/s eta 0:00:01\r",
      "\u001b[K    56% |#################               | 532kB 55.6MB/s eta 0:00:01\r",
      "\u001b[K    57% |##################              | 542kB 52.7MB/s eta 0:00:01\r",
      "\u001b[K    58% |##################              | 552kB 53.9MB/s eta 0:00:01\r",
      "\u001b[K    59% |##################              | 563kB 52.5MB/s eta 0:00:01\r",
      "\u001b[K    60% |###################             | 573kB 51.9MB/s eta 0:00:01\r",
      "\u001b[K    61% |###################             | 583kB 53.4MB/s eta 0:00:01\r",
      "\u001b[K    62% |####################            | 593kB 52.8MB/s eta 0:00:01\r",
      "\u001b[K    63% |####################            | 604kB 53.1MB/s eta 0:00:01\r",
      "\u001b[K    64% |####################            | 614kB 53.0MB/s eta 0:00:01\r",
      "\u001b[K    65% |#####################           | 624kB 52.4MB/s eta 0:00:01\r",
      "\u001b[K    66% |#####################           | 634kB 52.8MB/s eta 0:00:01\r",
      "\u001b[K    67% |#####################           | 645kB 55.7MB/s eta 0:00:01\r",
      "\u001b[K    69% |######################          | 655kB 54.2MB/s eta 0:00:01\r",
      "\u001b[K    70% |######################          | 665kB 56.8MB/s eta 0:00:01\r",
      "\u001b[K    71% |######################          | 675kB 56.3MB/s eta 0:00:01\r",
      "\u001b[K    72% |#######################         | 686kB 56.4MB/s eta 0:00:01\r",
      "\u001b[K    73% |#######################         | 696kB 56.7MB/s eta 0:00:01\r",
      "\u001b[K    74% |#######################         | 706kB 55.3MB/s eta 0:00:01\r",
      "\u001b[K    75% |########################        | 716kB 57.4MB/s eta 0:00:01\r",
      "\u001b[K    76% |########################        | 727kB 56.7MB/s eta 0:00:01\r",
      "\u001b[K    77% |########################        | 737kB 54.8MB/s eta 0:00:01\r",
      "\u001b[K    78% |#########################       | 747kB 55.6MB/s eta 0:00:01\r",
      "\u001b[K    79% |#########################       | 757kB 56.9MB/s eta 0:00:01\r",
      "\u001b[K    80% |#########################       | 768kB 56.2MB/s eta 0:00:01\r",
      "\u001b[K    81% |##########################      | 778kB 56.1MB/s eta 0:00:01\r",
      "\u001b[K    83% |##########################      | 788kB 54.5MB/s eta 0:00:01\r",
      "\u001b[K    84% |##########################      | 798kB 55.5MB/s eta 0:00:01\r",
      "\u001b[K    85% |###########################     | 808kB 55.9MB/s eta 0:00:01\r",
      "\u001b[K    86% |###########################     | 819kB 54.8MB/s eta 0:00:01\r",
      "\u001b[K    87% |###########################     | 829kB 55.7MB/s eta 0:00:01\r",
      "\u001b[K    88% |############################    | 839kB 56.5MB/s eta 0:00:01\r",
      "\u001b[K    89% |############################    | 849kB 56.7MB/s eta 0:00:01\r",
      "\u001b[K    90% |############################    | 860kB 55.5MB/s eta 0:00:01\r",
      "\u001b[K    91% |#############################   | 870kB 54.4MB/s eta 0:00:01\r",
      "\u001b[K    92% |#############################   | 880kB 55.9MB/s eta 0:00:01\r",
      "\u001b[K    93% |##############################  | 890kB 55.5MB/s eta 0:00:01\r",
      "\u001b[K    94% |##############################  | 901kB 54.3MB/s eta 0:00:01\r",
      "\u001b[K    95% |##############################  | 911kB 55.1MB/s eta 0:00:01\r",
      "\u001b[K    97% |############################### | 921kB 55.1MB/s eta 0:00:01\r",
      "\u001b[K    98% |############################### | 931kB 54.3MB/s eta 0:00:01\r",
      "\u001b[K    99% |############################### | 942kB 55.0MB/s eta 0:00:01\r",
      "\u001b[K    100% |################################| 952kB 1.6MB/s \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[?25hRequirement already satisfied (use --upgrade to upgrade): six in /usr/local/lib/python3.5/dist-packages (from cycler>=0.10->matplotlib)\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mRequirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python3/dist-packages (from kiwisolver>=1.0.1->matplotlib)\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mInstalling collected packages: pyparsing, cycler, python-dateutil, kiwisolver, matplotlib\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mSuccessfully installed cycler-0.10.0 kiwisolver-1.0.1 matplotlib-3.0.0 pyparsing-2.2.1 python-dateutil-2.7.3\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[33mYou are using pip version 8.1.1, however version 18.0 is available.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mYou should consider upgrading via the 'pip install --upgrade pip' command.\u001b[0m\u001b[0m\r\n",
      "\u001b[0;32m\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible {unit_group.name} -a 'docker exec -t chainer pip3 install matplotlib'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48b3d2-c2e9-11e8-9925-0242ac110002",
     "next": "ea48b468-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48b33c-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "`train_mnist.py` を実行します。\n",
    "\n",
    "> 処理が完了するまで４分～５分程度かかります。\n",
    "\n",
    "`train_mnist.py`の引数で `--gpu 0` を指定しているのでGPUを利用して処理が実行されています。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea48b468-c2e9-11e8-9925-0242ac110002",
     "next": "ea48b4fe-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48b3d2-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32m172.30.2.125 | SUCCESS | rc=0 >>\u001b[0m\r\n",
      "\u001b[0;32mGPU: 0\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m# unit: 1000\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m# Minibatch-size: 100\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m# epoch: 20\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mDownloading from http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz...\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mDownloading from http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz...\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mDownloading from http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz...\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mDownloading from http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz...\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mepoch       main/loss   validation/main/loss  main/accuracy  validation/main/accuracy  elapsed_time\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [..................................................]  0.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m       100 iter, 0 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m       inf iters/sec. Estimated time to finish: 0:00:00.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [..................................................]  1.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m       200 iter, 0 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    214.46 iters/sec. Estimated time to finish: 0:00:55.021291.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#.................................................]  2.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m       300 iter, 0 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    213.19 iters/sec. Estimated time to finish: 0:00:54.880181.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#.................................................]  3.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m       400 iter, 0 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    213.75 iters/sec. Estimated time to finish: 0:00:54.269204.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##................................................]  4.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m       500 iter, 0 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    213.98 iters/sec. Estimated time to finish: 0:00:53.744380.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J1           0.191597    0.0964438             0.942267       0.969                     11.8135       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [##................................................]  5.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m       600 iter, 1 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    174.18 iters/sec. Estimated time to finish: 0:01:05.449821.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##................................................]  5.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m       700 iter, 1 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    179.64 iters/sec. Estimated time to finish: 0:01:02.904665.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###...............................................]  6.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m       800 iter, 1 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    183.69 iters/sec. Estimated time to finish: 0:01:00.971699.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###...............................................]  7.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m       900 iter, 1 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    187.09 iters/sec. Estimated time to finish: 0:00:59.328256.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [####..............................................]  8.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      1000 iter, 1 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    189.71 iters/sec. Estimated time to finish: 0:00:57.982912.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [####..............................................]  9.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      1100 iter, 1 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    191.88 iters/sec. Estimated time to finish: 0:00:56.806779.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J2           0.0729999   0.0791599             0.977049       0.9729                    14.8464       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [#####.............................................] 10.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      1200 iter, 2 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    186.32 iters/sec. Estimated time to finish: 0:00:57.963309.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#####.............................................] 10.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      1300 iter, 2 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    188.44 iters/sec. Estimated time to finish: 0:00:56.781089.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#####.............................................] 11.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      1400 iter, 2 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    190.14 iters/sec. Estimated time to finish: 0:00:55.747787.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [######............................................] 12.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      1500 iter, 2 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    191.71 iters/sec. Estimated time to finish: 0:00:54.769140.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [######............................................] 13.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      1600 iter, 2 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    193.11 iters/sec. Estimated time to finish: 0:00:53.854580.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#######...........................................] 14.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      1700 iter, 2 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    194.35 iters/sec. Estimated time to finish: 0:00:52.995998.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J3           0.0496285   0.0749612             0.983515       0.9778                    17.8648       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [#######...........................................] 15.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      1800 iter, 3 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    190.54 iters/sec. Estimated time to finish: 0:00:53.532392.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#######...........................................] 15.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      1900 iter, 3 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    191.67 iters/sec. Estimated time to finish: 0:00:52.693430.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      2000 iter, 3 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    192.79 iters/sec. Estimated time to finish: 0:00:51.871158.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [########..........................................] 17.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      2100 iter, 3 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    193.75 iters/sec. Estimated time to finish: 0:00:51.096979.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#########.........................................] 18.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      2200 iter, 3 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    194.61 iters/sec. Estimated time to finish: 0:00:50.358364.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#########.........................................] 19.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      2300 iter, 3 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.41 iters/sec. Estimated time to finish: 0:00:49.640058.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J4           0.0350352   0.0919421             0.988965       0.9751                    20.8944       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [##########........................................] 20.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      2400 iter, 4 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    192.44 iters/sec. Estimated time to finish: 0:00:49.885661.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##########........................................] 20.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      2500 iter, 4 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    193.22 iters/sec. Estimated time to finish: 0:00:49.165846.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##########........................................] 21.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      2600 iter, 4 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    193.97 iters/sec. Estimated time to finish: 0:00:48.461104.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###########.......................................] 22.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      2700 iter, 4 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    194.65 iters/sec. Estimated time to finish: 0:00:47.778685.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###########.......................................] 23.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      2800 iter, 4 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.31 iters/sec. Estimated time to finish: 0:00:47.105580.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [############......................................] 24.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      2900 iter, 4 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.93 iters/sec. Estimated time to finish: 0:00:46.445977.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J5           0.028976    0.0839051             0.990816       0.9795                    23.9292       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [############......................................] 25.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      3000 iter, 5 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    193.51 iters/sec. Estimated time to finish: 0:00:46.509821.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [############......................................] 25.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      3100 iter, 5 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    194.14 iters/sec. Estimated time to finish: 0:00:45.843501.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#############.....................................] 26.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      3200 iter, 5 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     194.7 iters/sec. Estimated time to finish: 0:00:45.197966.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#############.....................................] 27.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      3300 iter, 5 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.23 iters/sec. Estimated time to finish: 0:00:44.561708.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##############....................................] 28.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      3400 iter, 5 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.73 iters/sec. Estimated time to finish: 0:00:43.937072.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##############....................................] 29.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      3500 iter, 5 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.22 iters/sec. Estimated time to finish: 0:00:43.317987.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J6           0.0241202   0.0775701             0.992048       0.9782                    26.9659       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [###############...................................] 30.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      3600 iter, 6 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    194.19 iters/sec. Estimated time to finish: 0:00:43.255859.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###############...................................] 30.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      3700 iter, 6 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    194.69 iters/sec. Estimated time to finish: 0:00:42.631847.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###############...................................] 31.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      3800 iter, 6 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.16 iters/sec. Estimated time to finish: 0:00:42.017470.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [################..................................] 32.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      3900 iter, 6 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.61 iters/sec. Estimated time to finish: 0:00:41.409866.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      4000 iter, 6 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.02 iters/sec. Estimated time to finish: 0:00:40.811331.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#################.................................] 34.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      4100 iter, 6 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.42 iters/sec. Estimated time to finish: 0:00:40.219117.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J7           0.0205676   0.0801511             0.993432       0.9805                    29.9978       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [#################.................................] 35.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      4200 iter, 7 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    194.73 iters/sec. Estimated time to finish: 0:00:40.056302.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#################.................................] 35.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      4300 iter, 7 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.15 iters/sec. Estimated time to finish: 0:00:39.457155.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##################................................] 36.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      4400 iter, 7 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.53 iters/sec. Estimated time to finish: 0:00:38.868396.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##################................................] 37.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      4500 iter, 7 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.89 iters/sec. Estimated time to finish: 0:00:38.287127.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###################...............................] 38.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      4600 iter, 7 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.23 iters/sec. Estimated time to finish: 0:00:37.711105.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###################...............................] 39.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      4700 iter, 7 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.57 iters/sec. Estimated time to finish: 0:00:37.136970.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J8           0.017744    0.0878126             0.994549       0.9816                    33.037        \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [####################..............................] 40.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      4800 iter, 8 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.07 iters/sec. Estimated time to finish: 0:00:36.910660.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [####################..............................] 40.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      4900 iter, 8 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.41 iters/sec. Estimated time to finish: 0:00:36.333619.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [####################..............................] 41.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      5000 iter, 8 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.74 iters/sec. Estimated time to finish: 0:00:35.761709.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#####################.............................] 42.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      5100 iter, 8 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.06 iters/sec. Estimated time to finish: 0:00:35.193472.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#####################.............................] 43.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      5200 iter, 8 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.36 iters/sec. Estimated time to finish: 0:00:34.630078.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [######################............................] 44.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      5300 iter, 8 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.67 iters/sec. Estimated time to finish: 0:00:34.067153.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J9           0.0185273   0.0956357             0.994133       0.9793                    36.0751       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [######################............................] 45.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      5400 iter, 9 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.34 iters/sec. Estimated time to finish: 0:00:33.787682.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [######################............................] 45.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      5500 iter, 9 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.65 iters/sec. Estimated time to finish: 0:00:33.223268.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#######################...........................] 46.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      5600 iter, 9 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.95 iters/sec. Estimated time to finish: 0:00:32.661681.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#######################...........................] 47.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      5700 iter, 9 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.24 iters/sec. Estimated time to finish: 0:00:32.103015.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [########################..........................] 48.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      5800 iter, 9 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.54 iters/sec. Estimated time to finish: 0:00:31.545870.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [########################..........................] 49.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      5900 iter, 9 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.79 iters/sec. Estimated time to finish: 0:00:30.997382.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J10          0.0167872   0.0903217             0.994732       0.9811                    39.1063       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      6000 iter, 10 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     195.6 iters/sec. Estimated time to finish: 0:00:30.675046.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#########################.........................] 50.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      6100 iter, 10 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.88 iters/sec. Estimated time to finish: 0:00:30.121133.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#########################.........................] 51.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      6200 iter, 10 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.14 iters/sec. Estimated time to finish: 0:00:29.570512.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##########################........................] 52.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      6300 iter, 10 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.38 iters/sec. Estimated time to finish: 0:00:29.025011.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##########################........................] 53.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      6400 iter, 10 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.62 iters/sec. Estimated time to finish: 0:00:28.481855.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###########################.......................] 54.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      6500 iter, 10 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.86 iters/sec. Estimated time to finish: 0:00:27.938581.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J11          0.0112759   0.106473              0.996365       0.9788                    42.1471       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [###########################.......................] 55.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      6600 iter, 11 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.76 iters/sec. Estimated time to finish: 0:00:27.585356.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###########################.......................] 55.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      6700 iter, 11 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m       196 iters/sec. Estimated time to finish: 0:00:27.040932.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [############################......................] 56.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      6800 iter, 11 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.25 iters/sec. Estimated time to finish: 0:00:26.497364.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [############################......................] 57.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      6900 iter, 11 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.48 iters/sec. Estimated time to finish: 0:00:25.957245.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#############################.....................] 58.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      7000 iter, 11 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.71 iters/sec. Estimated time to finish: 0:00:25.418457.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#############################.....................] 59.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      7100 iter, 11 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.91 iters/sec. Estimated time to finish: 0:00:24.884162.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J12          0.0146664   0.112049              0.995415       0.9778                    45.187        \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [##############################....................] 60.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      7200 iter, 12 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    195.89 iters/sec. Estimated time to finish: 0:00:24.503403.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##############################....................] 60.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      7300 iter, 12 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.11 iters/sec. Estimated time to finish: 0:00:23.966696.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##############################....................] 61.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      7400 iter, 12 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.32 iters/sec. Estimated time to finish: 0:00:23.430623.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###############################...................] 62.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      7500 iter, 12 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.53 iters/sec. Estimated time to finish: 0:00:22.896770.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###############################...................] 63.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      7600 iter, 12 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.73 iters/sec. Estimated time to finish: 0:00:22.365508.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [################################..................] 64.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      7700 iter, 12 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.94 iters/sec. Estimated time to finish: 0:00:21.834506.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J13          0.011369    0.0832606             0.996366       0.9828                    48.2282       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [################################..................] 65.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      7800 iter, 13 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m       196 iters/sec. Estimated time to finish: 0:00:21.428605.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [################################..................] 65.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      7900 iter, 13 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     196.2 iters/sec. Estimated time to finish: 0:00:20.897364.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      8000 iter, 13 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     196.4 iters/sec. Estimated time to finish: 0:00:20.366894.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#################################.................] 67.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      8100 iter, 13 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.59 iters/sec. Estimated time to finish: 0:00:19.838127.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##################################................] 68.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      8200 iter, 13 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.78 iters/sec. Estimated time to finish: 0:00:19.310944.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##################################................] 69.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      8300 iter, 13 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.96 iters/sec. Estimated time to finish: 0:00:18.785956.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J14          0.0120257   0.0842915             0.996016       0.984                     51.2697       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [###################################...............] 70.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      8400 iter, 14 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.09 iters/sec. Estimated time to finish: 0:00:18.358858.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###################################...............] 70.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      8500 iter, 14 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.28 iters/sec. Estimated time to finish: 0:00:17.831649.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###################################...............] 71.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      8600 iter, 14 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.47 iters/sec. Estimated time to finish: 0:00:17.305504.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [####################################..............] 72.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      8700 iter, 14 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.64 iters/sec. Estimated time to finish: 0:00:16.781538.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [####################################..............] 73.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      8800 iter, 14 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     196.8 iters/sec. Estimated time to finish: 0:00:16.260087.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#####################################.............] 74.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      8900 iter, 14 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.96 iters/sec. Estimated time to finish: 0:00:15.738837.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J15          0.00868902  0.0952061             0.997315       0.9812                    54.3145       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [#####################################.............] 75.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      9000 iter, 15 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.16 iters/sec. Estimated time to finish: 0:00:15.293995.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#####################################.............] 75.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      9100 iter, 15 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.32 iters/sec. Estimated time to finish: 0:00:14.771618.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [######################################............] 76.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      9200 iter, 15 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.49 iters/sec. Estimated time to finish: 0:00:14.250154.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [######################################............] 77.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      9300 iter, 15 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.65 iters/sec. Estimated time to finish: 0:00:13.729659.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#######################################...........] 78.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      9400 iter, 15 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.81 iters/sec. Estimated time to finish: 0:00:13.211044.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#######################################...........] 79.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      9500 iter, 15 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.96 iters/sec. Estimated time to finish: 0:00:12.692793.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J16          0.00934882  0.104617              0.997199       0.9806                    57.3633       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [########################################..........] 80.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      9600 iter, 16 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     196.2 iters/sec. Estimated time to finish: 0:00:12.232670.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [########################################..........] 80.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      9700 iter, 16 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.36 iters/sec. Estimated time to finish: 0:00:11.712967.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [########################################..........] 81.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      9800 iter, 16 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.53 iters/sec. Estimated time to finish: 0:00:11.194476.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#########################################.........] 82.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m      9900 iter, 16 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.69 iters/sec. Estimated time to finish: 0:00:10.676854.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     10000 iter, 16 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.84 iters/sec. Estimated time to finish: 0:00:10.160444.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##########################################........] 84.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     10100 iter, 16 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.99 iters/sec. Estimated time to finish: 0:00:09.644956.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J17          0.0112478   0.114741              0.996866       0.9807                    60.3992       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [##########################################........] 85.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     10200 iter, 17 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.11 iters/sec. Estimated time to finish: 0:00:09.178307.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##########################################........] 85.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     10300 iter, 17 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.12 iters/sec. Estimated time to finish: 0:00:08.668072.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###########################################.......] 86.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     10400 iter, 17 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     196.1 iters/sec. Estimated time to finish: 0:00:08.159041.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###########################################.......] 87.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     10500 iter, 17 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    196.08 iters/sec. Estimated time to finish: 0:00:07.649858.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [############################################......] 88.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     10600 iter, 17 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    198.14 iters/sec. Estimated time to finish: 0:00:07.065639.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [############################################......] 89.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     10700 iter, 17 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    198.14 iters/sec. Estimated time to finish: 0:00:06.561034.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J18          0.00981509  0.110915              0.997249       0.9815                    63.4461       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [#############################################.....] 90.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     10800 iter, 18 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    197.27 iters/sec. Estimated time to finish: 0:00:06.083169.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#############################################.....] 90.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     10900 iter, 18 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    197.25 iters/sec. Estimated time to finish: 0:00:05.576685.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#############################################.....] 91.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     11000 iter, 18 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    197.24 iters/sec. Estimated time to finish: 0:00:05.069926.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##############################################....] 92.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     11100 iter, 18 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    197.22 iters/sec. Estimated time to finish: 0:00:04.563343.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [##############################################....] 93.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     11200 iter, 18 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     198.1 iters/sec. Estimated time to finish: 0:00:04.038458.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###############################################...] 94.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     11300 iter, 18 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    198.06 iters/sec. Estimated time to finish: 0:00:03.534208.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J19          0.0107425   0.103733              0.996916       0.9837                    66.5017       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [###############################################...] 95.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     11400 iter, 19 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    197.15 iters/sec. Estimated time to finish: 0:00:03.043348.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [###############################################...] 95.83%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [########..........................................] 16.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     11500 iter, 19 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    197.14 iters/sec. Estimated time to finish: 0:00:02.536317.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [################################################..] 96.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [################..................................] 33.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     11600 iter, 19 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    197.12 iters/sec. Estimated time to finish: 0:00:02.029266.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [################################################..] 97.50%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################.........................] 50.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     11700 iter, 19 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     197.1 iters/sec. Estimated time to finish: 0:00:01.522091.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#################################################.] 98.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#################################.................] 66.67%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     11800 iter, 19 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    197.95 iters/sec. Estimated time to finish: 0:00:01.010373.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J     total [#################################################.] 99.17%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [#########################################.........] 83.33%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     11900 iter, 19 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    197.93 iters/sec. Estimated time to finish: 0:00:00.505226.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J20          0.0104862   0.0929223             0.997366       0.9832                    69.5505       \r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[J     total [##################################################] 100.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32mthis epoch [..................................................]  0.00%\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m     12000 iter, 20 epoch / 20 epochs\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m    197.03 iters/sec. Estimated time to finish: 0:00:00.\r",
      "\u001b[0m\r\n",
      "\u001b[0;32m\u001b[4A\u001b[J\u001b[0m\r\n",
      "\u001b[0;32m\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible {unit_group.name} -a 'docker exec -t chainer python3 \\\n",
    "    /opt/examples/mnist/train_mnist.py \\\n",
    "    --gpu 0 --noplot --out /opt/output/out'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48b4fe-c2e9-11e8-9925-0242ac110002",
     "next": "ea48b59e-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48b468-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "[Grafana](/grafana/d/handson/vcp-metrics?refresh=5s)でVCノードの利用状況（CPU、メモリ、ネットワーク）を確認してみます。\n",
    "\n",
    "> GrafanaでのGPUメトリクス表示は未対応です。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48b59e-c2e9-11e8-9925-0242ac110002",
     "next": "ea48b62a-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48b4fe-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "# 後始末"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48b62a-c2e9-11e8-9925-0242ac110002",
     "next": "ea48b6ca-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48b59e-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "全てのリソースを削除します。\n",
    "\n",
    "> 処理が完了するまで1分～2分程度かかります。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea48b6ca-c2e9-11e8-9925-0242ac110002",
     "next": "ea48b774-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48b62a-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2018-09-28 15:55:41,175 - INFO - gpu is DELETING ... 0 sec\n",
      "2018-09-28 15:55:46,420 - INFO - gpu is DELETING ... 5 sec\n",
      "2018-09-28 15:55:51,675 - INFO - gpu is DELETING ... 10 sec\n",
      "2018-09-28 15:55:56,923 - INFO - gpu is DELETING ... 15 sec\n",
      "2018-09-28 15:56:02,177 - INFO - gpu is DELETING ... 20 sec\n",
      "2018-09-28 15:56:07,420 - INFO - gpu is DELETING ... 25 sec\n",
      "2018-09-28 15:56:12,687 - INFO - gpu is DELETING ... 30 sec\n",
      "2018-09-28 15:56:17,937 - INFO - gpu is DELETING ... 35 sec\n",
      "2018-09-28 15:56:23,196 - INFO - gpu is DELETING ... 40 sec\n",
      "2018-09-28 15:56:28,436 - INFO - gpu is DELETING ... 45 sec\n",
      "2018-09-28 15:56:33,699 - INFO - gpu is DELETING ... 50 sec\n",
      "2018-09-28 15:56:38,959 - INFO - gpu is DELETING ... 55 sec\n",
      "2018-09-28 15:56:44,222 - INFO - gpu is DELETING ... 60 sec\n",
      "2018-09-28 15:56:49,473 - INFO - gpu is DELETING ... 65 sec\n",
      "2018-09-28 15:56:54,726 - INFO - gpu is DELETING ... 70 sec\n",
      "2018-09-28 15:56:59,969 - INFO - gpu is DELETING ... 75 sec\n",
      "2018-09-28 15:57:05,228 - INFO - gpu is DELETING ... 80 sec\n",
      "2018-09-28 15:57:10,472 - INFO - gpu is DELETING ... 85 sec\n",
      "2018-09-28 15:57:15,726 - INFO - gpu is DELETING ... 90 sec\n",
      "2018-09-28 15:57:20,997 - INFO - gpu is DELETING ... 95 sec\n",
      "2018-09-28 15:57:26,213 - INFO - cleanup completed. vc handson201 is cleanup(no unit)\n"
     ]
    }
   ],
   "source": [
    "unit_group.cleanup()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "ea48b774-c2e9-11e8-9925-0242ac110002",
     "next": "ea48b80a-c2e9-11e8-9925-0242ac110002",
     "previous": "ea48b6ca-c2e9-11e8-9925-0242ac110002"
    }
   },
   "source": [
    "不要になったAnsibleのインベントリファイルを削除します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "deletable": false,
    "editable": false,
    "lc_cell_meme": {
     "current": "ea48b80a-c2e9-11e8-9925-0242ac110002",
     "next": null,
     "previous": "ea48b774-c2e9-11e8-9925-0242ac110002"
    },
    "run_through_control": {
     "frozen": true
    }
   },
   "outputs": [],
   "source": [
    "!rm hosts"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  },
  "lc_notebook_meme": {
   "current": "ea4882e0-c2e9-11e8-9925-0242ac110002",
   "lc_server_signature": {
    "current": {
     "notebook_dir": "/notebooks/notebook",
     "notebook_path": "/handson/vcp-tutorial",
     "server_url": "http://13.231.115.112:8888/red/",
     "signature_id": "9618e102-9304-11e8-ab91-0242ac110002"
    },
    "history": []
   }
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "toc_cell": false,
   "toc_position": {
    "height": "748px",
    "left": "0px",
    "right": "1118.08px",
    "top": "111px",
    "width": "212px"
   },
   "toc_section_display": "block",
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}