Lab1_Basic_Image_Processing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hf7bOCqRoGHP"
   },
   "source": [
    "# Getting started with OpenCV\n",
    "OpenCV is the most widely used framework for computer vision tasks. It supports a wide variety of algorithms related to computer vision and machine learning and it is expanding day-by-day.\n",
    "\n",
    "The purpose if this exercise is for you to get some hands-on experience with OpenCV by solving some basic image processing tasks.\n",
    "\n",
    "The best place to start is here: https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_tutorials.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rN3woPvKrQCq"
   },
   "source": [
    "## Mount your Google Drive\n",
    "Make a copy of this notebook in your Google Drive. And yes, you need to mount your Drive every time you start working on a notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 121
    },
    "colab_type": "code",
    "id": "8h-_g9o3riJV",
    "outputId": "4ba34dfe-5297-4c89-bd11-8fc024016eac"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=email%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdocs.test%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.photos.readonly%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fpeopleapi.readonly&response_type=code\n",
      "\n",
      "Enter your authorization code:\n",
      "··········\n",
      "Mounted at /content/gdrive\n"
     ]
    }
   ],
   "source": [
    "from google.colab import drive\n",
    "drive.mount('/content/gdrive')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fi07kNQ8qpzo"
   },
   "source": [
    "## Task 1 - Core operations\n",
    "You will learn some basic operations with OpenCV.\n",
    "\n",
    "First, lets download an image to work on:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "G35gaf4pv9JQ",
    "outputId": "89041964-b205-49ec-861c-0ea78987efa1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'/content/gdrive/My Drive/cat.jpg'\n"
     ]
    }
   ],
   "source": [
    "import urllib.request\n",
    "\n",
    "url = \"https://github.com/aivclab/dlcourse/raw/master/data/cat.jpg\"\n",
    "urllib.request.urlretrieve(url,'/content/gdrive/My Drive/cat.jpg')\n",
    "\n",
    "# Check that the file is in your Drive\n",
    "!ls \"/content/gdrive/My Drive/cat.jpg\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "P-JA6QLYrCz6"
   },
   "outputs": [],
   "source": [
    "import cv2\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "prLT8sTmru-q"
   },
   "source": [
    "### Task 1.1\n",
    "Load the image from your Drive using OpenCV's imread function (not Pillow!).\n",
    "\n",
    "Then display the image using Matplotlib (OpenCV's imshow function will not work in a notebook)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "m-2-NqMYtCaG"
   },
   "outputs": [],
   "source": [
    "# Read image using OpenCV\n",
    "#img = <your code goes here>\n",
    "\n",
    "# Display image using Matplotlib\n",
    "plt.imshow(img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "CPegGAS8uOlY"
   },
   "source": [
    "### Questions 1.1\n",
    "1. What is the size of the image?\n",
    "2. How many color channels does it have?\n",
    "3. Why do the colors look so weird?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ZP3H7gXrvgsM"
   },
   "source": [
    "### Tasks 1.2\n",
    "Run this piece of code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "0rF6jBPCtTOd"
   },
   "outputs": [],
   "source": [
    "b,g,r = cv2.split(img)\n",
    "rgb = cv2.merge((r,g,b))\n",
    "plt.imshow(rgb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9E4a8j8Ywsk9"
   },
   "source": [
    "### Questions 1.2\n",
    "1. What just happened?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Vk_xozm3yJ19"
   },
   "source": [
    "### Task 1.3\n",
    "Let's do some pixel operations:\n",
    "\n",
    "(Need help? OpenCV images are simply Numpy arrays. Look under indexing in the Numpy tutorial: https://github.com/aivclab/dlcourse/blob/master/Lab1_Tutorial.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ZrCW86y5wrKw"
   },
   "outputs": [],
   "source": [
    "# What is the color [b,g,r] of the pixel at location [100,100]?\n",
    "#<your code goes here>\n",
    "\n",
    "# Set the color the pixel at location [100,100] to the value [255,255,255]\n",
    "#<your code goes here>\n",
    "\n",
    "# Make the color of all pixels in the region [380:480, 710:810] white\n",
    "#<your code goes here>\n",
    "\n",
    "# Display\n",
    "plt.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ojMeTM6O1Do0"
   },
   "source": [
    "### Questions 1.3\n",
    "1. Where is the origin of the image coordinate system?\n",
    "2. What is the direction of the first coordinate/axis?\n",
    "3. What is the direction of the second coordinate/axis?\n",
    "4. Oh, and what does the function cv2.cvtColor do?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "4ktIAmWl1rjM"
   },
   "source": [
    "## Task 2\n",
    "You will learn some basic image processing.\n",
    "\n",
    "Where to find inspiration:\n",
    "* https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_core/py_basic_ops/py_basic_ops.html#basic-ops\n",
    "* https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_colorspaces/py_colorspaces.html\n",
    "* https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_thresholding/py_thresholding.html\n",
    "* https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_geometric_transformations/py_geometric_transformations.html\n",
    "* https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_contours/py_table_of_contents_contours/py_table_of_contents_contours.html\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_p7xsST65iwD"
   },
   "source": [
    "### Task 2.1\n",
    "Lets do some simple geometric transformations of the cat image:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Ta9kGCjvyyol"
   },
   "outputs": [],
   "source": [
    "#Reload cat image\n",
    "img = cv2.imread('/content/gdrive/My Drive/cat.jpg')\n",
    "\n",
    "# Convert to grayscale\n",
    "img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n",
    "\n",
    "# Resize img to size 100 by 100\n",
    "#img = <your code goes here>\n",
    "\n",
    "# Now, rotate the resized image 35 degrees counter clockwise\n",
    "angle = 35\n",
    "rows,cols = img.shape\n",
    "M = cv2.getRotationMatrix2D((cols/2,rows/2),angle,1)\n",
    "img = cv2.warpAffine(img,M,(cols,rows))\n",
    "\n",
    "# Display\n",
    "plt.imshow(img, cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "sUhC6cB_3Vzg"
   },
   "source": [
    "### Questions 2.1\n",
    "1. How would you use cv2.resize to scale the image to half the original size?\n",
    "2. What do you think the matrix M does? Hint: See [here](https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_geometric_transformations/py_geometric_transformations.html)\n",
    "3. What happens if you delete cmap='gray' in plt.imshow?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "yoA0VXIU6ABf"
   },
   "source": [
    "### Task 2.2\n",
    "Let's try another image:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "3uJpEZp52N_6",
    "outputId": "e738e28f-0d2d-458c-cff7-0be504cc0772"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'/content/gdrive/My Drive/sudoku.jpg'\n"
     ]
    }
   ],
   "source": [
    "import urllib.request\n",
    "\n",
    "url = \"https://github.com/aivclab/dlcourse/raw/master/data/sudoku.jpg\"\n",
    "urllib.request.urlretrieve(url,'/content/gdrive/My Drive/sudoku.jpg')\n",
    "\n",
    "# Check that the file is in your Drive\n",
    "!ls \"/content/gdrive/My Drive/sudoku.jpg\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "y8Hgu4Xp6Z5y"
   },
   "source": [
    "Display the image:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Ic7kSgv16Fns"
   },
   "outputs": [],
   "source": [
    "# Load sudoku image\n",
    "img = cv2.imread('/content/gdrive/My Drive/sudoku.jpg')\n",
    "\n",
    "# Convert to grayscale\n",
    "img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n",
    "\n",
    "# Display\n",
    "plt.imshow(img, cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Te1t8IpD7F1B"
   },
   "source": [
    "Do some magic!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "QBH_LQ0e6W5M"
   },
   "outputs": [],
   "source": [
    "rows,cols = img.shape\n",
    "\n",
    "pts1 = np.float32([[56,65],[368,52],[28,387],[389,390]])\n",
    "pts2 = np.float32([[0,0],[300,0],[0,300],[300,300]])\n",
    "\n",
    "M = cv2.getPerspectiveTransform(pts1,pts2)\n",
    "\n",
    "img_aligned = cv2.warpPerspective(img,M,(300,300))\n",
    "\n",
    "plt.subplot(121),plt.imshow(img,cmap='gray'),plt.title('Input')\n",
    "plt.subplot(122),plt.imshow(img_aligned,cmap='gray'),plt.title('Output')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VzI10b-T7I-b"
   },
   "source": [
    "### Questions 2.2\n",
    "1. Can you figure out how the coordinates ``pts1`` were selected?\n",
    "2. What does the transform ``warpPerspective`` do?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "O35KEL8O-cAP"
   },
   "source": [
    "### Task 2.3\n",
    "Suppose, you wanted to detect and recognize the individual numbers from the cells on the sudoku field. A first step towards achieving this could be to\n",
    "\n",
    "1. make the image binary, such that pixels are either black or white\n",
    "2. group black pixels into connected clusters (also called connected components)\n",
    "3. recognize digits by performing some sort of template matching on each component against a database of digits\n",
    "\n",
    "The first step is called thresholding:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "XbwL-iGg6kxz"
   },
   "outputs": [],
   "source": [
    "ret,thresh1 = cv2.threshold(img_aligned,100,255,cv2.THRESH_BINARY)\n",
    "thresh2 = cv2.adaptiveThreshold(img_aligned,255,cv2.ADAPTIVE_THRESH_MEAN_C,cv2.THRESH_BINARY,11,7)\n",
    "plt.subplot(121),plt.imshow(thresh1,cmap='gray'),plt.title('thresh1')\n",
    "plt.subplot(122),plt.imshow(thresh2,cmap='gray'),plt.title('thresh2')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "sWWWCP4KB1c-"
   },
   "source": [
    "### Questions 2.3\n",
    "1. Explain how thresh1 is calculated\n",
    "2. Explain how thresh2 is calculated"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ugMIVK57qEsQ"
   },
   "source": [
    "### Task 2.4\n",
    "We can use OpenCV findContours to find alle the connected components in a binary image. The algorithm looks for connected white pixels that are connected, so we first have to invert the binary image:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "5JwyN6hZuxHg"
   },
   "outputs": [],
   "source": [
    "thresh2_inv = cv2.bitwise_not(thresh2)\n",
    "plt.imshow(thresh2_inv,cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Ofi7LqZAu5pv"
   },
   "source": [
    "Next, let's run findContours and look at the output:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Bn_lMwCh-6ET"
   },
   "outputs": [],
   "source": [
    "contours, hierarchy = cv2.findContours(thresh2_inv,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)\n",
    "\n",
    "drawing_contours = np.zeros((img_aligned.shape[0],img_aligned.shape[1],3),dtype=np.uint8)\n",
    "drawing_contours = cv2.drawContours(drawing_contours, contours, -1, (255,255,255), 1)\n",
    "\n",
    "drawning_boxes = cv2.cvtColor(img_aligned,cv2.COLOR_GRAY2BGR)\n",
    "for cnt in contours:\n",
    "  x,y,w,h = cv2.boundingRect(cnt)\n",
    "  drawning_boxes = cv2.rectangle(drawning_boxes,(x,y),(x+w,y+h),(0,255,0),2)\n",
    "\n",
    "plt.figure(figsize=(10,10))\n",
    "plt.subplot(121); plt.imshow(drawing_contours,cmap='gray')\n",
    "plt.subplot(122); plt.imshow(drawning_boxes,cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "WEd1hWZ2wSYu"
   },
   "source": [
    "### Questions 2.4\n",
    "``contours`` is a Python list of all the contours in the image.\n",
    "1. How many contours are there?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hSKmYVlMx5P3"
   },
   "source": [
    "### Task 2.5\n",
    "Based on the output of findContours, see if you can generate a new image, containing just the numbers. You should remove the gridlines of the sudoku field, as well as noise."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9TY6O0n_1K32"
   },
   "source": [
    "## Task 3\n",
    "You will learn about image gradients and smoothing.\n",
    "\n",
    "Where to find inspiration:\n",
    "* https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_gradients/py_gradients.html\n",
    "* https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_filtering/py_filtering.html\n",
    "* https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_canny/py_canny.html\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "igadgQgYDnhq"
   },
   "source": [
    "Let's continue working on the sudoku image:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "AG7DQWR3zPhX"
   },
   "outputs": [],
   "source": [
    "# Load sudoku image\n",
    "img = cv2.imread('/content/gdrive/My Drive/sudoku.jpg')\n",
    "\n",
    "# Convert to grayscale\n",
    "img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n",
    "\n",
    "# Display\n",
    "plt.imshow(img, cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "AOf2SEF5D9XG"
   },
   "source": [
    "### Task 3.1\n",
    "Recall that in vector calculus, the gradient is a multi-variable generalization of the derivative. Thinking of an image as a 2D function, the gradient at each position (pixel) indicates how much the intensity varies along the X- and Y-direction.\n",
    "\n",
    "A simple way to approximate the derivate along X and Y is to simply take pixel differences:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "74u66D_CDssH"
   },
   "outputs": [],
   "source": [
    "# First convert to float and normalize intensities to range 0.0 to 1.0\n",
    "img_float = img.astype(np.float32) / 255.\n",
    "\n",
    "# Approximate derivate along X\n",
    "Ix = np.diff(img_float, axis=1)\n",
    "\n",
    "# Approximate derivate along Y\n",
    "Iy = np.diff(img_float, axis=0)\n",
    "\n",
    "plt.figure(figsize=(12,12))\n",
    "plt.subplot(121);\n",
    "plt.imshow(Ix,cmap='gray',vmin=-0.5,vmax=0.5), plt.title('Ix: Derivate along X')\n",
    "plt.subplot(122);\n",
    "plt.imshow(Iy,cmap='gray',vmin=-0.5,vmax=0.5), plt.title('Iy: Derivate along Y')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot image intensities (img_float) and derivate (Ix) along row 200.\n",
    "\n",
    "Also, display Ix again but this time with amplified intensities (using vmin and vmax)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(12,12))\n",
    "plt.subplot(211); plt.plot(img_float[200,:]); plt.plot(Ix[200,:]), plt.legend(('Image intenstities','Derivate'))\n",
    "plt.subplot(212); plt.imshow(Ix,cmap='gray',vmin=-0.05,vmax=0.05)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "HxSkAlqDHNlH"
   },
   "source": [
    "### Questions 3.1\n",
    "1. Explain what you see in the two images above (Ix and Iy)?\n",
    "2. What is the difference between the two images?\n",
    "3. Explain what you see in the plot of Ix along row 200 and amplified Ix image? Why is the image so noisy?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "nO7N9NaII2rw"
   },
   "source": [
    "### Task 3.2\n",
    "Let's try to smoothen the input image and then recalculate the gradients:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "HSIbYz9rGCi8"
   },
   "outputs": [],
   "source": [
    "kernel = np.ones((5,5),np.float32)/25\n",
    "img_float_smoothed = cv2.filter2D(img_float,-1,kernel)\n",
    "\n",
    "# Approximate derivate along X\n",
    "Ix = np.diff(img_float_smoothed, axis=1)\n",
    "\n",
    "# Approximate derivate along Y\n",
    "Iy = np.diff(img_float_smoothed, axis=0)\n",
    "\n",
    "plt.figure(figsize=(12,12))\n",
    "plt.subplot(221);\n",
    "plt.imshow(img_float,cmap='gray'), plt.title('Original image')\n",
    "plt.subplot(222);\n",
    "plt.imshow(img_float_smoothed,cmap='gray'), plt.title('Smoothed image')\n",
    "plt.subplot(223);\n",
    "plt.imshow(Ix,cmap='gray',vmin=-0.5,vmax=0.5), plt.title('Ix: Derivate along X')\n",
    "plt.subplot(224);\n",
    "plt.imshow(Iy,cmap='gray',vmin=-0.5,vmax=0.5), plt.title('Iy: Derivate along Y')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "kRd83qEoJkce"
   },
   "outputs": [],
   "source": [
    "plt.figure(figsize=(12,12))\n",
    "plt.subplot(211); plt.plot(img_float_smoothed[200,:]); plt.plot(Ix[200,:]), plt.legend(('Image intenstities','Derivate'))\n",
    "plt.subplot(212); plt.imshow(Ix,cmap='gray',vmin=-0.05,vmax=0.05)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "dGs1C132JyU3"
   },
   "source": [
    "Notice any difference?\n",
    "\n",
    "### Questions 3.2\n",
    "1. See if you can figure out how filtering works (see [here](https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_filtering/py_filtering.html))\n",
    "2. Why does the filtering smoothen the image?\n",
    "3. Can you figure out a way to calculate the image gradients using filtering? (Hint: You need to design a filter on your own!)\n",
    "4. Can you calculate an image that displays the magnitude of the gradient at each pixel location?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "p9cWzgNL8g8g"
   },
   "source": [
    "## Task 4\n",
    "You will learn about template matching.\n",
    "\n",
    "Where to find inspiration:\n",
    "* https://docs.opencv.org/master/d4/dc6/tutorial_py_template_matching.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Odnwy2DI8m3j"
   },
   "source": [
    "### Task 4.1\n",
    "Inspect the cat image and create a template by cropping out an image region corresponding to the cat's left eye (your right-hand side). The eye is roughly at ``[380:480, 710:810]``. You may need to fine-tune the cooordinates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "hs1QcFgw8lvd"
   },
   "outputs": [],
   "source": [
    "# Load cat image\n",
    "bgr = cv2.imread('/content/gdrive/My Drive/cat.jpg')\n",
    "\n",
    "# Convert to grayscale\n",
    "img = cv2.cvtColor(bgr, cv2.COLOR_BGR2GRAY)\n",
    "\n",
    "# Display\n",
    "plt.imshow(img, cmap='gray')\n",
    "\n",
    "# Create template\n",
    "# template = <Your code goes here>\n",
    "\n",
    "# Display the image and the template\n",
    "plt.subplot(121),plt.imshow(img,cmap = 'gray')\n",
    "plt.title('Input image'), plt.xticks([]), plt.yticks([])\n",
    "plt.subplot(122);plt.imshow(template, cmap='gray')\n",
    "plt.title('Template'), plt.xticks([]), plt.yticks([])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "896klQtE8tLA"
   },
   "source": [
    "### Task 4.2\n",
    "Your task is to detect the other eye using template matching (i.e., detect cat's right eye by using the left eye as a kind of \"search template\"). Use this as inspiration:\n",
    "https://docs.opencv.org/master/d4/dc6/tutorial_py_template_matching.html\n",
    "\n",
    "Use the TM_CCOEFF_NORMED distance metric.\n",
    "\n",
    "You may find the ``where`` function af numpy useful: https://docs.scipy.org/doc/numpy/reference/generated/numpy.where.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sP-hiv2I9J2H"
   },
   "outputs": [],
   "source": [
    "# Apply template Matching\n",
    "res = cv2.matchTemplate(img,template,cv2.TM_CCOEFF_NORMED)\n",
    "\n",
    "# <Your code goes here>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "e1F-adDiQjHw"
   },
   "source": [
    "## Optional tasks\n",
    "If you have more time, feel free to look into [feature detection and matching](https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_feature2d/py_table_of_contents_feature2d/py_table_of_contents_feature2d.html):\n",
    "* https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_feature2d/py_features_meaning/py_features_meaning.html#features-meaning\n",
    "* https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_feature2d/py_features_harris/py_features_harris.html#harris-corners\n",
    "* https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_feature2d/py_matcher/py_matcher.html#matcher\n",
    "* https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_feature2d/py_feature_homography/py_feature_homography.html#feature-homography"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Lab1_Basic_Image_Processing.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}