notebooks/llmu/Fine_Tuning_for_Chat.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/cohere-ai/notebooks/blob/main/notebooks/llmu/Fine_Tuning_for_Chat.ipynb\">\n",
    "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
    "</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HZ_ZFietTx7o"
   },
   "source": [
    "# Fine-Tuning for Chat\n",
    "\n",
    "Our ready-to-use large language models, such as [Command](https://cohere.com/models/command), are very good at producing responses to natural language prompts. However, there are many cases in which getting the best model performance requires performing an additional round of training on custom user data. Creating a custom model using this process is called **fine-tuning**.\n",
    "\n",
    "Fine-tuning is recommended when you want to teach the model a new task, or leverage your company's unique knowledge base. Fine-tuning models is also helpful for generating a specific writing style or format, or leveraging a new data type.\n",
    "\n",
    "In this notebook, you will fine-tune a chatbot on custom conversational data to improve its performance at a specific task.\n",
    "\n",
    "_Read the [accompanying blog post here](https://docs.cohere.com/docs/fine-tuning-for-chat)._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll do the following steps:\n",
    "- **Step 1: Prepare the Dataset** - Download the dataset, select a subset, and prepare it for the Chat endpoint.\n",
    "- **Step 2: Fine-Tune the Model** - Kick off a fine-tuning job, and confirm when the model has completed training.\n",
    "- **Step 3: Use/Evaluate the Fine-Tuned Model** - Evaluate the fine-tuned model's performance on the test dataset, and confirm it is a competent participant in multi-turn conversations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "We'll start by installing the tools we'll need and then importing them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "DcGt_h00Tx7q",
    "outputId": "d28e180f-87b6-407e-c727-85b6cd3ced99"
   },
   "outputs": [],
   "source": [
    "! pip install cohere jsonlines -q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#@title Enable text wrapping in Google Colab\n",
    "\n",
    "from IPython.display import HTML, display\n",
    "\n",
    "def set_css():\n",
    "  display(HTML('''\n",
    "  <style>\n",
    "    pre {\n",
    "        white-space: pre-wrap;\n",
    "    }\n",
    "  </style>\n",
    "  '''))\n",
    "get_ipython().events.register('pre_run_cell', set_css)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "DPbi4T0qTx7r"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import json\n",
    "import jsonlines\n",
    "import cohere"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fill in your Cohere API key in the next cell. To do this, begin by [signing up to Cohere](https://os.cohere.ai/) (for free!) if you haven't yet. Then get your API key [here](https://dashboard.cohere.com/api-keys)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <style>\n",
       "    pre {\n",
       "        white-space: pre-wrap;\n",
       "    }\n",
       "  </style>\n",
       "  "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Paste your API key here. Remember to not share publicly\n",
    "co = cohere.Client(\"COHERE_API_KEY\") "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MX02UXCFTx7r"
   },
   "source": [
    "## Step 1: Prepare and Validate the Dataset\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qOdgYOPfT98F"
   },
   "source": [
    "### Download the dataset\n",
    "\n",
    "We will work with the [CoEdIT dataset](https://huggingface.co/datasets/grammarly/coedit) of text editing examples (Raheja, et al). In each example, the user asks a writing assistant to rewrite text to suit a specific task (editing fluency, coherence, clarity, or style) and receives a response. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 573
    },
    "id": "sCdFjMBGTx7t",
    "outputId": "5f567449-22d2-4545-b0cf-9b59fa3a0ce7"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <style>\n",
       "    pre {\n",
       "        white-space: pre-wrap;\n",
       "    }\n",
       "  </style>\n",
       "  "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2024-03-21 13:29:29--  https://huggingface.co/datasets/grammarly/coedit/resolve/main/train.jsonl\n",
      "Resolving huggingface.co (huggingface.co)... 3.161.4.106, 3.161.4.111, 3.161.4.115, ...\n",
      "Connecting to huggingface.co (huggingface.co)|3.161.4.106|:443... connected.\n",
      "HTTP request sent, awaiting response... 302 Found\n",
      "Location: https://cdn-lfs.huggingface.co/repos/30/91/3091c2c741f77a2f5aa8986b13e4fb2c3658ab3ebc30ecaa5f6890e60939bdf9/2913249158d6a178dc638e870212ff8a432d128eb6b4bdbe969ee805e6063ce3?response-content-disposition=attachment%3B+filename*%3DUTF-8%27%27train.jsonl%3B+filename%3D%22train.jsonl%22%3B&Expires=1711308569&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTcxMTMwODU2OX19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy5odWdnaW5nZmFjZS5jby9yZXBvcy8zMC85MS8zMDkxYzJjNzQxZjc3YTJmNWFhODk4NmIxM2U0ZmIyYzM2NThhYjNlYmMzMGVjYWE1ZjY4OTBlNjA5MzliZGY5LzI5MTMyNDkxNThkNmExNzhkYzYzOGU4NzAyMTJmZjhhNDMyZDEyOGViNmI0YmRiZTk2OWVlODA1ZTYwNjNjZTM%7EcmVzcG9uc2UtY29udGVudC1kaXNwb3NpdGlvbj0qIn1dfQ__&Signature=o4vxvJruLayMUe37m5puwa3ni4UqKKqAvWNjyd28fZsDX95Z1N8SwYh3CXUCe-T%7ElTsKpdvt7lHhoW7p2J8ZmhnTDQu-Sb8QPXpj4JE0oJgzJQrLdbbYpv%7ETKEgw5PfW%7ECYXeJ2Rz2YmgjFttH3jMc3bRC8J379fgfgGxU%7ER8fPHDb5oznqLKGwbLWkRjnRHQTMzU9lv9m78XHTW6glGN3X2LzXgH5F2n1KZtQ34x4C%7E2HxHRZx5aEhumgS893pwc8ayStKdfODPd-8yTQ6gRu%7Eud9u762SoV3HIVQ-zvYZD6SmZOaiCK-gaMciYzs01L%7EQAg7dp1AWk41h6swQh1g__&Key-Pair-Id=KVTP0A1DKRTAX [following]\n",
      "--2024-03-21 13:29:29--  https://cdn-lfs.huggingface.co/repos/30/91/3091c2c741f77a2f5aa8986b13e4fb2c3658ab3ebc30ecaa5f6890e60939bdf9/2913249158d6a178dc638e870212ff8a432d128eb6b4bdbe969ee805e6063ce3?response-content-disposition=attachment%3B+filename*%3DUTF-8%27%27train.jsonl%3B+filename%3D%22train.jsonl%22%3B&Expires=1711308569&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTcxMTMwODU2OX19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy5odWdnaW5nZmFjZS5jby9yZXBvcy8zMC85MS8zMDkxYzJjNzQxZjc3YTJmNWFhODk4NmIxM2U0ZmIyYzM2NThhYjNlYmMzMGVjYWE1ZjY4OTBlNjA5MzliZGY5LzI5MTMyNDkxNThkNmExNzhkYzYzOGU4NzAyMTJmZjhhNDMyZDEyOGViNmI0YmRiZTk2OWVlODA1ZTYwNjNjZTM%7EcmVzcG9uc2UtY29udGVudC1kaXNwb3NpdGlvbj0qIn1dfQ__&Signature=o4vxvJruLayMUe37m5puwa3ni4UqKKqAvWNjyd28fZsDX95Z1N8SwYh3CXUCe-T%7ElTsKpdvt7lHhoW7p2J8ZmhnTDQu-Sb8QPXpj4JE0oJgzJQrLdbbYpv%7ETKEgw5PfW%7ECYXeJ2Rz2YmgjFttH3jMc3bRC8J379fgfgGxU%7ER8fPHDb5oznqLKGwbLWkRjnRHQTMzU9lv9m78XHTW6glGN3X2LzXgH5F2n1KZtQ34x4C%7E2HxHRZx5aEhumgS893pwc8ayStKdfODPd-8yTQ6gRu%7Eud9u762SoV3HIVQ-zvYZD6SmZOaiCK-gaMciYzs01L%7EQAg7dp1AWk41h6swQh1g__&Key-Pair-Id=KVTP0A1DKRTAX\n",
      "Resolving cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)... 18.160.109.120, 18.160.109.61, 18.160.109.59, ...\n",
      "Connecting to cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)|18.160.109.120|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 19695735 (19M) [binary/octet-stream]\n",
      "Saving to: ‘train.jsonl’\n",
      "\n",
      "train.jsonl         100%[===================>]  18.78M  46.9MB/s    in 0.4s    \n",
      "\n",
      "2024-03-21 13:29:30 (46.9 MB/s) - ‘train.jsonl’ saved [19695735/19695735]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Download the dataset\n",
    "! wget \"https://huggingface.co/datasets/grammarly/coedit/resolve/main/train.jsonl\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PAnUk94HTx7t"
   },
   "source": [
    "### Get a subset of the dataset\n",
    "\n",
    "Instead of using the full dataset, we will use a subset focused on making text coherent: 927 total conversations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 71
    },
    "id": "rtduxw5QTx7u",
    "outputId": "20df8e3f-b92b-41c3-91d5-71bc8d7441c5"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <style>\n",
       "    pre {\n",
       "        white-space: pre-wrap;\n",
       "    }\n",
       "  </style>\n",
       "  "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of examples: 927\n",
      "Number of examples in training set: 800\n",
      "Number of examples in the test set: 127\n"
     ]
    }
   ],
   "source": [
    "# we will use subset of the dataset focused on making text more coherent\n",
    "phrase = \"coherent\"\n",
    "\n",
    "# instantiate python list where we will store correct subset of dataset\n",
    "dataset_list = []\n",
    "\n",
    "# create subset of dataset\n",
    "with jsonlines.open('train.jsonl') as f:\n",
    "    for line in f.iter():\n",
    "        if phrase in line['src'].split(\":\")[0]:\n",
    "            dataset_list.append(line)\n",
    "\n",
    "# Split data into training and test\n",
    "dataset_list_train = dataset_list[:800]\n",
    "dataset_list_test = dataset_list[800:]\n",
    "\n",
    "print(\"Total number of examples:\", len(dataset_list))\n",
    "print(\"Number of examples in training set:\", len(dataset_list_train))\n",
    "print(\"Number of examples in the test set:\", len(dataset_list_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2cQqlA2gTx7v"
   },
   "source": [
    "### Preview the dataset\n",
    "\n",
    "We will use the `src` and `tgt` fields from each example, which correspond to the user’s prompt and the writing assistant’s response, respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "jNhdIa8OTx7v",
    "outputId": "40c01c60-eec3-42f7-b714-49948eec989d"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <style>\n",
       "    pre {\n",
       "        white-space: pre-wrap;\n",
       "    }\n",
       "  </style>\n",
       "  "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Make the text coherent: The Bank's main strategy is to further expand its network and increase its lending activities with particular focus on the SME sector. The EBRD helps Bank, by developing and financing Bank's portfolio of and strengthening the bank's funding base.\n",
      "The Bank's main strategy is to further expand its network and increase its lending activities with particular focus on the SME sector. The EBRD helps Union Bank, by developing and financing its portfolio of and strengthening the bank's funding base.\n",
      "--------------------------------------------------\n",
      "Make the text coherent: It was not illegal under international law ; captured foreign sailors were released. Confederates went to prison camps.\n",
      "It was not illegal under international law ; captured foreign sailors were released, while Confederates went to prison camps.\n",
      "--------------------------------------------------\n",
      "Make the text coherent: The Union blockade was a powerful weapon that eventually ruined the Southern economy, at the cost of very few lives. The measure of the blockade's success was not the few ships that slipped through, but the thousands that never tried Union.\n",
      "The Union blockade was a powerful weapon that eventually ruined the Southern economy, at the cost of very few lives. The measure of the blockade's success was not the few ships that slipped through, but the thousands that never tried it.\n",
      "--------------------------------------------------\n",
      "Make the text more coherent: It lasted for 60 minutes. It featured the three men taking questions from a studio audience.\n",
      "Lasting for 60 minutes, it featured the three men taking questions from a studio audience.\n",
      "--------------------------------------------------\n",
      "Make the text more coherent: The Security Council could not decide on a Secretary-General. The Third World countries would not nominate any other candidates as long as Salim remained in the race.\n",
      "The Security Council could not decide on a Secretary-General, but the Third World countries would not nominate any other candidates as long as Salim remained in the race.\n",
      "--------------------------------------------------\n",
      "Make the text coherent: All of the 2011 inductees lost their lives in the 1961 crash of Sabena Flight 548, considered to be the most tragic event in figure skating history. inductees were honored posthumously in observance of the fiftieth anniversary of the tragedy.\n",
      "All of the 2011 inductees lost their lives in the 1961 crash of Sabena Flight 548, considered to be the most tragic event in figure skating history. They were honored posthumously in observance of the fiftieth anniversary of the tragedy.\n",
      "--------------------------------------------------\n",
      "Make the text more coherent: Foreign Service personnel stationed in nations with inadequate public infrastructure also face greater risk of injury or death due to fire, traffic accidents, and natural disasters. An FSO was one of the first identified victims of the 2010 Haiti earthquake.\n",
      "Foreign Service personnel stationed in nations with inadequate public infrastructure also face greater risk of injury or death due to fire, traffic accidents, and natural disasters. For instance, an FSO was one of the first identified victims of the 2010 Haiti earthquake.\n",
      "--------------------------------------------------\n",
      "Make the text more coherent: The Federalist Party made a relatively strong showing, winning seats in both chambers while supporting a competitive challenge to the incumbent Democratic-Republican President. The Democratic-Republican continued Democratic-Republican's control of the Presidency and both houses of Congress.\n",
      "The Federalist Party made a relatively strong showing, winning seats in both chambers while supporting a competitive challenge to the incumbent Democratic-Republican President. However, the Democratic-Republican Party continued its control of the Presidency and both houses of Congress.\n",
      "--------------------------------------------------\n",
      "Make the text coherent: Since the 1990s, Loughborough University operated a satellite higher education campus in Peterborough. This closed in 2003, leaving the city as one of the largest urban areas in the country without a dedicated provision of higher education.\n",
      "Since the 1990s, Loughborough University operated a satellite higher education campus in Peterborough. However, this closed in 2003, leaving the city as one of the largest urban areas in the country without a dedicated provision of higher education.\n",
      "--------------------------------------------------\n",
      "Make the text coherent: The cancer center is named after Monroe Dunaway Anderson, a banker and cotton trader from Jackson, Tennessee. Monroe Dunaway Anderson was a banker of a business partnership with Monroe Dunaway Anderson's brother-in-law Will Clayton.\n",
      "The cancer center is named after Monroe Dunaway Anderson, a banker and cotton trader from Jackson, Tennessee. He was a member of a business partnership with his brother-in-law Will Clayton.\n",
      "--------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "# print the first ten prompts and corresponding responses\n",
    "for item in dataset_list_train[:10]:\n",
    "    print(item[\"src\"])\n",
    "    print(item[\"tgt\"])\n",
    "    print(\"-\"*50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Du8c0guSTx7v"
   },
   "source": [
    "### Prepare the dataset for Cohere's Chat endpoint\n",
    "\n",
    "To format the dataset for the Chat endpoint, we create a `.jsonl` where each JSON object is a conversation containing a series of messages.\n",
    "- A `System` message in the beginning, acting as the preamble that guides the whole conversation\n",
    "- Multiple pairs of `User` and `Chatbot` messages, representing the conversation that takes place between a human user and a chatbot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 321
    },
    "id": "wgg438PATx7x",
    "outputId": "55d52c19-6723-4a57-9f4f-0bb7ed79346e"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <style>\n",
       "    pre {\n",
       "        white-space: pre-wrap;\n",
       "    }\n",
       "  </style>\n",
       "  "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'messages': [{'role': 'System', 'content': 'You are a writing assistant that helps the user write coherent text.'}, {'role': 'User', 'content': \"Make the text coherent: The Bank's main strategy is to further expand its network and increase its lending activities with particular focus on the SME sector. The EBRD helps Bank, by developing and financing Bank's portfolio of and strengthening the bank's funding base.\"}, {'role': 'Chatbot', 'content': \"The Bank's main strategy is to further expand its network and increase its lending activities with particular focus on the SME sector. The EBRD helps Union Bank, by developing and financing its portfolio of and strengthening the bank's funding base.\"}]}\n",
      "{'messages': [{'role': 'System', 'content': 'You are a writing assistant that helps the user write coherent text.'}, {'role': 'User', 'content': 'Make the text coherent: It was not illegal under international law ; captured foreign sailors were released. Confederates went to prison camps.'}, {'role': 'Chatbot', 'content': 'It was not illegal under international law ; captured foreign sailors were released, while Confederates went to prison camps.'}]}\n",
      "{'messages': [{'role': 'System', 'content': 'You are a writing assistant that helps the user write coherent text.'}, {'role': 'User', 'content': \"Make the text coherent: The Union blockade was a powerful weapon that eventually ruined the Southern economy, at the cost of very few lives. The measure of the blockade's success was not the few ships that slipped through, but the thousands that never tried Union.\"}, {'role': 'Chatbot', 'content': \"The Union blockade was a powerful weapon that eventually ruined the Southern economy, at the cost of very few lives. The measure of the blockade's success was not the few ships that slipped through, but the thousands that never tried it.\"}]}\n"
     ]
    }
   ],
   "source": [
    "# arranges the data to suit Cohere's format\n",
    "def create_chat_ft_data(preamble, user_message, chatbot_message):\n",
    "    formatted_data = {\n",
    "        \"messages\": [\n",
    "            {\n",
    "                \"role\": \"System\",\n",
    "                \"content\": preamble\n",
    "            },\n",
    "            {\n",
    "                \"role\": \"User\",\n",
    "                \"content\": user_message\n",
    "            },\n",
    "            {\n",
    "                \"role\": \"Chatbot\",\n",
    "                \"content\": chatbot_message\n",
    "            }\n",
    "        ]\n",
    "    }\n",
    "\n",
    "    return formatted_data\n",
    "\n",
    "preamble = \"You are a writing assistant that helps the user write coherent text.\"\n",
    "\n",
    "# creates jsonl file from list of examples\n",
    "def create_jsonl_from_list(file_name, dataset_segment, preamble):\n",
    "    path = f'{file_name}.jsonl'\n",
    "    if not os.path.isfile(path):\n",
    "        with open(path, 'w+') as file:\n",
    "            for item in dataset_segment:\n",
    "                user_message = item[\"src\"]\n",
    "                chatbot_message = item[\"tgt\"]\n",
    "                formatted_data = create_chat_ft_data(preamble, user_message, chatbot_message)\n",
    "                file.write(json.dumps(formatted_data) + '\\n')\n",
    "            file.close()\n",
    "\n",
    "# Create training jsonl file\n",
    "file_name = \"coedit_coherence_train\"\n",
    "create_jsonl_from_list(file_name, dataset_list_train, preamble)\n",
    "\n",
    "# List the first 3 items in the JSONL file\n",
    "with jsonlines.open(f'{file_name}.jsonl') as f:\n",
    "    [print(line) for _, line in zip(range(3), f)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "u3sVbOAfTx7y"
   },
   "source": [
    "## Step 2: Fine-Tune the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "A_kESxXgUCFt"
   },
   "source": [
    "We kick off a fine-tuning job by navigating to the [fine-tuning tab of the Dashboard](https://dashboard.cohere.com/fine-tuning).  Under \"Chat\", click on \"Create a Chat model\".\n",
    "\n",
    "<img src=\"https://files.readme.io/48dad78-cohere_dashboard.png\">\n",
    "\n",
    "Next, upload the `.jsonl` file you just created as the training set by clicking on the \"TRAINING SET\" button. When ready, click on \"Review data\" to proceed to the next step.\n",
    "\n",
    "<img src=\"https://files.readme.io/82e3691-image_2.png\">\n",
    "\n",
    "Then, you'll see a preview of how the model will ingest your data. If anything is wrong with the data, the page will also provide suggested changes to fix the training file. Otherwise, if everything looks good, you can proceed to the next step.\n",
    "\n",
    "<img src=\"https://files.readme.io/fbce852-image_3.png\">\n",
    "\n",
    "Finally, you'll see an estimated cost of fine-tuning, followed by a page where you'll provide a nickname to your model. We used `coedit-coherence-ft` as the nickname for our model. This page also allows you to provide custom values for the hyperparameters used during training, but we'll keep them at the default values for now. \n",
    "\n",
    "<img src=\"https://files.readme.io/801e93a-name_model.png\">\n",
    "\n",
    "Once you have filled in a name, click on \"Start training\" to kick off the fine-tuning process. This will navigate you to a page where you can monitor the status of the model. A model that has finished fine-tuning will show the status as `READY`.\n",
    "\n",
    "<img src=\"https://files.readme.io/dd0d48b-ready_model.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FgtQcmzmTx7z"
   },
   "source": [
    "## Step 3: Use/Evaluate the Fine-Tuned Model\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the model has completed the fine-tuning process, it’s time to evaluate its performance. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "R2eVldwyjDD3"
   },
   "source": [
    "### With Test Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you're ready to use the fine-tuned model, navigate to the API tab. There, you'll see the model ID that you should use when calling `co.chat()`.\n",
    "\n",
    "<img src=\"https://files.readme.io/82c726e-get_model_id.png\">\n",
    "\n",
    "In the following code, we supply the first three messages from the test dataset to both the pre-trained and fine-tuned models for comparison."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "qTbH0eeCTx7z",
    "outputId": "5f48f63b-b2e1-402e-d915-6b9efca35cdd"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <style>\n",
       "    pre {\n",
       "        white-space: pre-wrap;\n",
       "    }\n",
       "  </style>\n",
       "  "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "User: Make the text more coherent: We do know that at the end of the Muromachi period it stopped appearing in written records. That Muromachi burned down many times, the last we know of in 1405. \n",
      "-----\n",
      "Desired response: We do know that at the end of the Muromachi period it stopped appearing in written records and that it burned down many times, the last we know of in 1405. \n",
      "-----\n",
      "Default model's response: We have knowledge that towards the end of the Muromachi period, mentions of it in written records ceased. This period's namesake, Muromachi, faced numerous fires that ravaged the area. The last of these disasters occurred in 1405, after which the district's name disappeared from the historical record. \n",
      "-----\n",
      "Fine-tuned model's response: We do know that at the end of the Muromachi period it stopped appearing in written records, although it burned down many times, the last we know of in 1405.\n",
      "---------------------------------------------------------------------------------------------------- \n",
      "\n",
      "\n",
      "User: Make the text coherent: Pimelodella kronei is a species of three-barbeled catfish endemic to Brazil. Discovered by the German naturalist Sigismund Ernst Richard Krone, Pimelodella kronei was the first troglobitic species described in Brazil, but several others have been described later. \n",
      "-----\n",
      "Desired response: Pimelodella kronei is a species of three-barbeled catfish endemic to Brazil. Discovered by the German naturalist Sigismund Ernst Richard Krone, it was the first troglobitic fish described in Brazil, but several others have been described later. \n",
      "-----\n",
      "Default model's response: Pimelodella kronei is a three-bearded catfish species native only to Brazil. This species was discovered by German naturalist Sigismund Ernst Richard Krone, making it the first troglobitic species discovered in Brazil. Since then, several other troglobitic species have been discovered in the country. \n",
      "-----\n",
      "Fine-tuned model's response: Pimelodella kronei is a species of three-barbeled catfish endemic to Brazil. Discovered by the German naturalist Sigismund Ernst Richard Krone, it was the first troglobitic species described in Brazil, but several others have been described later.\n",
      "---------------------------------------------------------------------------------------------------- \n",
      "\n",
      "\n",
      "User: Make the text more coherent: The Radio City Music Hall was designed by architect Edward Durell Stone and interior designer Donald Deskey in the Art Deco style. architect used Indiana Limestone for the facade, as with all the other buildings in Rockefeller Center, but Edward Durell Stone also included some distinguishing features. \n",
      "-----\n",
      "Desired response: The Radio City Music Hall was designed by architect Edward Durell Stone and interior designer Donald Deskey in the Art Deco style. Stone used Indiana Limestone for the facade, as with all the other buildings in Rockefeller Center, but he also included some distinguishing features. \n",
      "-----\n",
      "Default model's response: The Radio City Music Hall, an iconic Art Deco masterpiece, was designed in collaboration by the architect Edward Durell Stone and interior designer Donald Deskey. While Stone clad the facade in Indiana Limestone, matching the other buildings in Rockefeller Center, he also incorporated unique features that set the Music Hall apart. \n",
      "-----\n",
      "Fine-tuned model's response: The Radio City Music Hall, designed by architect Edward Durell Stone and interior designer Donald Deskey in the Art Deco style, was built in Indiana Limestone for the facade, as with all the other buildings in Rockefeller Center, but Stone also included some distinguishing features.\n",
      "---------------------------------------------------------------------------------------------------- \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for item in dataset_list_test[:3]:\n",
    "    # User prompt\n",
    "    user_message = item[\"src\"]\n",
    "    # Desired/target response from dataset\n",
    "    tgt_message = item[\"tgt\"]\n",
    "\n",
    "    # Get default model response\n",
    "    response_pretrained=co.chat(\n",
    "        message=user_message,\n",
    "        preamble=preamble,\n",
    "        )\n",
    "\n",
    "    # Get fine-tuned model response\n",
    "    response_finetuned = co.chat(\n",
    "        message=user_message,\n",
    "        model='acb944bb-fb49-4c29-a15b-e6a245a7bdf9-ft',\n",
    "        preamble=preamble,\n",
    "        )\n",
    "\n",
    "    print(f\"User: {user_message}\",\"\\n-----\")\n",
    "    print(f\"Desired response: {tgt_message}\",\"\\n-----\")\n",
    "    print(f\"Default model's response: {response_pretrained.text}\",\"\\n-----\")\n",
    "    print(f\"Fine-tuned model's response: {response_finetuned.text}\")\n",
    "\n",
    "\n",
    "    print(\"-\"*100,\"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example, both models provide reasonable answers that are an improvement over the user’s original text. However, the fine-tuned model’s response better matches the style of the fine-tuning data, because it is more succinct. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bRkgtYAHTx7z"
   },
   "source": [
    "### In the Chat Context"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have demonstrated that the fine-tuned model can provide good answers to individual questions. But it is also a competent participant in longer, multi-turn conversations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 535
    },
    "id": "ds2t26xxTx70",
    "outputId": "0901987a-441b-42d8-fd70-285770f889b7"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <style>\n",
       "    pre {\n",
       "        white-space: pre-wrap;\n",
       "    }\n",
       "  </style>\n",
       "  "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting the chat. Type \"quit\" to end.\n",
      "\n",
      "User: Hello\n",
      "Chatbot: Hello! How can I help you today?\n",
      "--------------------------------------------------\n",
      "\n",
      "User: I'm fine. Can I ask you for help with some tasks?\n",
      "Chatbot: Of course! I'm here to help you with any tasks you need assistance with.\n",
      "--------------------------------------------------\n",
      "\n",
      "User: Make this more coherent: Manuel now has to decide-will he let his best friend be happy with her Prince Charming. Or will he fight for the love that has kept him alive for the last 16 years?\n",
      "Chatbot: Manuel now has to decide-will he let his best friend be happy with her Prince Charming, or will he fight for the love that has kept him alive for the last 16 years?\n",
      "--------------------------------------------------\n",
      "\n",
      "User: Help me with this one - She left Benaras. Conditions back home were bad.\n",
      "Chatbot: She left Benaras because conditions back home were bad.\n",
      "--------------------------------------------------\n",
      "\n",
      "User: What's a good time to visit London\n",
      "Chatbot: A good time to visit London is in the spring or fall, when the weather is mild and the city is not too crowded. However, if you want to experience the city in all its glory, you should visit in the summer or winter, when the weather is colder and the city is more empty.\n",
      "--------------------------------------------------\n",
      "\n",
      "User: Could you help with this please: Make the text coherent: Critically the album has not been as well received as other Browne recordings. It remains his only album to date to reach number 1 on the Billboard chart. \n",
      "Chatbot: Critically the album has not been as well received as other Browne recordings, but it remains his only album to date to reach number 1 on the Billboard chart.\n",
      "--------------------------------------------------\n",
      "\n",
      "User: quit\n",
      "Ending chat.\n"
     ]
    }
   ],
   "source": [
    "# Create a conversation ID\n",
    "import uuid\n",
    "conversation_id = str(uuid.uuid4())\n",
    "\n",
    "print('Starting the chat. Type \"quit\" to end.\\n')\n",
    "\n",
    "while True:\n",
    "\n",
    "    # User message\n",
    "    message = input(\"User: \")\n",
    "\n",
    "    # Typing \"quit\" ends the conversation\n",
    "    if message.lower() == 'quit':\n",
    "        print(\"Ending chat.\")\n",
    "        break\n",
    "\n",
    "    # Chatbot response\n",
    "    stream = co.chat_stream(message=message,\n",
    "                              model='acb944bb-fb49-4c29-a15b-e6a245a7bdf9-ft',\n",
    "                              preamble=preamble,\n",
    "                              conversation_id=conversation_id)\n",
    "\n",
    "    print(\"Chatbot: \", end='')\n",
    "\n",
    "    for event in stream:\n",
    "        if event.event_type == \"text-generation\":\n",
    "            print(event.text, end='')\n",
    "\n",
    "    print(f\"\\n{'-'*50}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note the fine-tuned model is still able to respond to prompts like “Hello”, “I’m fine. Can I ask you for help with some tasks?”, and “What’s a good time to visit London” instead of strictly following the fine-tuning objective of editing text.\n",
    "\n",
    "The model also did a good job with context switching; it can hold a conversation when the user switches from friendly greetings, to a request for writing help, to travel planning, and finally back to writing assistance. It can also infer when the user is asking for help with making a text coherent, even if it is not explicitly stated (e.g., “Help me with this one”) or if the request is buried slightly (e.g., with “Could you help me with this please”)."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}