intro2lt_assignment_3.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "intro2lt_assignment_3.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/TurkuNLP/intro-to-nlp/blob/master/intro2lt_assignment_3.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<h3>Introduction</h3>\n",
        "\n",
        "This notebook is the template for **project assignment 3**. Please base your submission on this template. Further information on submission is at the end of the notebook.\n",
        "\n",
        "In this assignment, we will use the dataset we analyzed in assignment 2, train our first baselines on it, and test few simple techniques to see if we can improve upon our initial results. There are three broad exercises in this assignment:\n",
        "\n",
        "*   Training a linear SVC classifier on the data\n",
        "*   Testing hyperparametres of the training\n",
        "*   Think about aspect-based classification\n",
        "\n",
        "We provide textual guidelines for the tasks and your task is to turn these instructions into code. In many cases, there are many ways a task can be achieved. Our tips and instructions are for beginning coders, and if you know what you are doing, you can choose whatever methods that you are comfortable with. Please ask for help in case you run into issues.\n",
        "\n",
        "### Data\n",
        "\n",
        "**Finnish:** you use the data created in week 1 as usual, there is enough of it\n",
        "**English:** use the data created in week 1 and mix it with this additional data to have enough for decent results, remember to shuffle! http://dl.turkunlp.org/ABSA16/ABSA16_Restaurants_Train_SB1_v2.tsv\n",
        "\n",
        "### Contributions\n",
        "\n",
        "In case the assignment is completed as **group work**, list all students and their contributions here.\n",
        "\n",
        "* Names:\n",
        "* Contributions: "
      ],
      "metadata": {
        "id": "3mvdzdpexOZu"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<h2>Questions 1-3: Data loading and basic classification</h2> \n",
        "\n",
        "(1) **Load the data**: Use your solution to Assignment 2 to load the data. As you saw in the lecture notebook here https://colab.research.google.com/github/TurkuNLP/intro-to-nlp/blob/master/bow_classifier.ipynb it is an advantage to load the data such that texts and labels are in two same-length lists. This is the format in which many of the `sklearn` functions need the data. Since you will be classifying the data, you should split it randomly to train and test sections.\n",
        "\n",
        "(2) **Classify the data**: You can freely base this solution on the `bow_classifier.ipynb` notebook. Classify the data using a linear SVC classifier and report on its performance. Of course if you want and know how, you can also experiment with other methods and libraries. But you should run the SVC to get an idea of what the baseline numbers are. It is OK not to take the target of the sentiment specifically into account, we will do that later.\n",
        "\n",
        "(3) **Hyperparameters**: The SVC classifier is quite sensitive to the regularization hyperparameter `C`. There is no good default value, you need to try what works on the very dataset you use. The value of C is typically optimized on a powers-of-10 scale, so a good start is something like 0.001, 0.01, 0.1, 1. Also note, that the higher the value, the longer the training takes. If you have extra time left, you can optionally also divide the data into train - development - test and optimize the hyperparameter C on the development data and report the difference in performance on development and test data. If there is any difference, discuss where it stems from.\n",
        "\n"
      ],
      "metadata": {
        "id": "DNZ3yCmG1z7s"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Answers to questions (1-3)\n",
        "\n",
        "Summarize here what you learned about the basic performance of the SVC classifier, and how you interpret your findings on hyperparameter optimization. Do you think the results are any good?\n"
      ],
      "metadata": {
        "id": "LRciFeyHb8t5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Your code for (1-3) goes here, divided to as many cells as you like"
      ],
      "metadata": {
        "id": "PuYaWdzhb485"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<h1> Question 4: Target</h1>\n",
        "\n",
        "You trained a linear SVC classifier on the data. So far, we have not taken into account the target of the sentiment in any way (this we will do next week).\n",
        "\n",
        "Considering that our overall objective is to build a decent target sentiment classifier, discuss briefly about ways in which you think the target of the sentiment could be taken into account and represented for the classifier. \n",
        "\n",
        "<h1> Answer to question 4:</h1>\n",
        "\n"
      ],
      "metadata": {
        "id": "EJgwlHOYvdtW"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Your answer to Question 4 goes here."
      ],
      "metadata": {
        "id": "ONMLzCJYwySw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<h3>Submission information</h3>\n",
        "\n",
        "**Please submit a .ipynb or .pdf version of the notebook to Moodle.** You can obtain pdf on Colab through File > Print > Save to pdf, but you can also submit an .ipynb (note: we won't execute the code). Please check that the submission is of reasonable quality representing your effort. E.g. all the cells are executed and their outputs are visible (this holds for both .pdf and .ipynb)"
      ],
      "metadata": {
        "id": "mb47GSKrEOdL"
      }
    }
  ]
}