02_Licence-plate-recognition.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we'll learn how to **recognize and extract licence plate numbers from car pictures**.\n",
    "\n",
    "This will allow you to discover, step by step, how you can create the code doing the recognition. In the last part of the workshop, this exact same code will be **packaged to create a service** that you can query from any application!\n",
    "\n",
    "We will use a **pre-trained model** as it takes a **long time** and **lots of data** to train such a model. However, you'll find the code to train the model as a reference in this repository if you're interested to learn more.\n",
    "\n",
    "Ready? Let's go!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Environment initialization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Libraries\n",
    "First, we'll need to **install some libraries** that are not part of our container image. Normally, **Red Hat OpenShift Data Science** is already taking care of this for you, based on what it detects in the code. **Red Hat OpenShift Data Science** will reinstall all those libraries for you every time you launch the notebook!\n",
    "\n",
    "In case you're using this notebook in a different environment, or just to make sure everything is ready, you can run the following cell to install OpenCV (a library to work with images) and Keras (an abstraction layer over Tensorflow)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "nbstart = time.time()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-04-06T19:29:11.979775Z",
     "start_time": "2021-04-06T19:29:02.801991Z"
    }
   },
   "outputs": [],
   "source": [
    "!pip install opencv-python-headless==4.5.* keras==2.4.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports\n",
    "Of course, we'll need to import various packages. They are either built in the notebook image you are running, or have been installed in the previous step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-04-06T19:29:28.412759Z",
     "start_time": "2021-04-06T19:29:24.929377Z"
    }
   },
   "outputs": [],
   "source": [
    "import glob\n",
    "import json\n",
    "import os\n",
    "from os.path import splitext,basename\n",
    "\n",
    "import tensorflow as tf\n",
    "tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)\n",
    "tf.get_logger().setLevel('ERROR')\n",
    "\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras.models  import model_from_json\n",
    "from keras.preprocessing.image import load_img, img_to_array\n",
    "from keras.applications.mobilenet_v2 import preprocess_input\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "import cv2\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.gridspec as gridspec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Finding Nemo (aka, the plate)...\n",
    "Now that we have loaded the tools we need, the first step in our journey is to be able to **find a plate within a car picture**.\n",
    "\n",
    "For us mere humans this may seem easy: it's a rectangle shape with some inscriptions on it, and we pretty much know that it's supposed to be in the front or the rear of the car.\n",
    "\n",
    "However, when you don't have any understanding of the concept of a rectangle, even less a car, that's something else. Fortunately, we have pre-trained a model to recognize this specific pattern withing a picture (a rectangle shape with some inscriptions).\n",
    "\n",
    "In this section, we'll see how we can use it to extract this specific part of the image."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Helper functions\n",
    "Those are some simple, mostly geometric functions and classes that will be used later on to process the image, resize it, calculate intersections,... Those functions are not really documented.\n",
    "\n",
    "You can simply run the cell below to initialize those functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-04-06T19:29:32.298098Z",
     "start_time": "2021-04-06T19:29:32.273339Z"
    }
   },
   "outputs": [],
   "source": [
    "class Label:\n",
    "    def __init__(self, cl=-1, tl=np.array([0., 0.]), br=np.array([0., 0.]), prob=None):\n",
    "        self.__tl = tl\n",
    "        self.__br = br\n",
    "        self.__cl = cl\n",
    "        self.__prob = prob\n",
    "\n",
    "    def __str__(self):\n",
    "        return 'Class: %d, top left(x: %f, y: %f), bottom right(x: %f, y: %f)' % (\n",
    "        self.__cl, self.__tl[0], self.__tl[1], self.__br[0], self.__br[1])\n",
    "\n",
    "    def copy(self):\n",
    "        return Label(self.__cl, self.__tl, self.__br)\n",
    "\n",
    "    def wh(self): return self.__br - self.__tl\n",
    "\n",
    "    def cc(self): return self.__tl + self.wh() / 2\n",
    "\n",
    "    def tl(self): return self.__tl\n",
    "\n",
    "    def br(self): return self.__br\n",
    "\n",
    "    def tr(self): return np.array([self.__br[0], self.__tl[1]])\n",
    "\n",
    "    def bl(self): return np.array([self.__tl[0], self.__br[1]])\n",
    "\n",
    "    def cl(self): return self.__cl\n",
    "\n",
    "    def area(self): return np.prod(self.wh())\n",
    "\n",
    "    def prob(self): return self.__prob\n",
    "\n",
    "    def set_class(self, cl):\n",
    "        self.__cl = cl\n",
    "\n",
    "    def set_tl(self, tl):\n",
    "        self.__tl = tl\n",
    "\n",
    "    def set_br(self, br):\n",
    "        self.__br = br\n",
    "\n",
    "    def set_wh(self, wh):\n",
    "        cc = self.cc()\n",
    "        self.__tl = cc - .5 * wh\n",
    "        self.__br = cc + .5 * wh\n",
    "\n",
    "    def set_prob(self, prob):\n",
    "        self.__prob = prob\n",
    "\n",
    "class DLabel(Label):\n",
    "    def __init__(self, cl, pts, prob):\n",
    "        self.pts = pts\n",
    "        tl = np.amin(pts, axis=1)\n",
    "        br = np.amax(pts, axis=1)\n",
    "        Label.__init__(self, cl, tl, br, prob)\n",
    "\n",
    "def getWH(shape):\n",
    "    return np.array(shape[1::-1]).astype(float)\n",
    "\n",
    "def IOU(tl1, br1, tl2, br2):\n",
    "    wh1, wh2 = br1-tl1, br2-tl2\n",
    "    assert((wh1 >= 0).all() and (wh2 >= 0).all())\n",
    "    \n",
    "    intersection_wh = np.maximum(np.minimum(br1, br2) - np.maximum(tl1, tl2), 0)\n",
    "    intersection_area = np.prod(intersection_wh)\n",
    "    area1, area2 = (np.prod(wh1), np.prod(wh2))\n",
    "    union_area = area1 + area2 - intersection_area\n",
    "    return intersection_area/union_area\n",
    "\n",
    "def IOU_labels(l1, l2):\n",
    "    return IOU(l1.tl(), l1.br(), l2.tl(), l2.br())\n",
    "\n",
    "def nms(Labels, iou_threshold=0.5):\n",
    "    SelectedLabels = []\n",
    "    Labels.sort(key=lambda l: l.prob(), reverse=True)\n",
    "    \n",
    "    for label in Labels:\n",
    "        non_overlap = True\n",
    "        for sel_label in SelectedLabels:\n",
    "            if IOU_labels(label, sel_label) > iou_threshold:\n",
    "               # print(label)\n",
    "                #print(sel_label)\n",
    "                non_overlap = False\n",
    "                break\n",
    "        if non_overlap:\n",
    "            SelectedLabels.append(label)\n",
    "    return SelectedLabels\n",
    "\n",
    "def find_T_matrix(pts, t_pts):\n",
    "    A = np.zeros((8, 9))\n",
    "    for i in range(0, 4):\n",
    "        xi = pts[:, i]\n",
    "        xil = t_pts[:, i]\n",
    "        xi = xi.T\n",
    "        \n",
    "        A[i*2, 3:6] = -xil[2]*xi\n",
    "        A[i*2, 6:] = xil[1]*xi\n",
    "        A[i*2+1, :3] = xil[2]*xi\n",
    "        A[i*2+1, 6:] = -xil[0]*xi\n",
    "\n",
    "    [U, S, V] = np.linalg.svd(A)\n",
    "    H = V[-1, :].reshape((3, 3))\n",
    "    return H\n",
    "\n",
    "def getRectPts(tlx, tly, brx, bry):\n",
    "    return np.matrix([[tlx, brx, brx, tlx], [tly, tly, bry, bry], [1, 1, 1, 1]], dtype=float)\n",
    "\n",
    "def normal(pts, side, mn, MN):\n",
    "    pts_MN_center_mn = pts * side\n",
    "    pts_MN = pts_MN_center_mn + mn.reshape((2, 1))\n",
    "    pts_prop = pts_MN / MN.reshape((2, 1))\n",
    "    return pts_prop"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Processing functions\n",
    "Now, the more interesting part. Those are the **main functions** that will be used in our detection pipeline. Each one is executing a step in the process (see comments for each function).\n",
    "\n",
    "Run the cell below to intialize the functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-04-06T19:29:39.888759Z",
     "start_time": "2021-04-06T19:29:39.864565Z"
    }
   },
   "outputs": [],
   "source": [
    "#######################################\n",
    "# Loads a model given a specific path #\n",
    "#######################################\n",
    "def load_model(path):\n",
    "    try:\n",
    "        path = splitext(path)[0]\n",
    "        with open('%s.json' % path, 'r') as json_file:\n",
    "            model_json = json_file.read()\n",
    "        model = model_from_json(model_json, custom_objects={})\n",
    "        model.load_weights('%s.h5' % path)\n",
    "        print(\"Model Loaded successfully...\")\n",
    "        return model\n",
    "    except Exception as e:\n",
    "        print(e)\n",
    "\n",
    "######################################################################################\n",
    "# Converts colors from BGR (as read by OpenCV) to RGB (so that we can display them), #\n",
    "# also eventually resizes the image to fit the size the model has been trained on    #\n",
    "######################################################################################\n",
    "def preprocess_image(image_path,resize=False):\n",
    "    img = cv2.imread(image_path)\n",
    "    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n",
    "    img = img / 255\n",
    "    if resize:\n",
    "        img = cv2.resize(img, (224,224))\n",
    "    return img\n",
    "\n",
    "#########################################################################\n",
    "# Reconstructs the image from detected pattern into plate cropped image #\n",
    "#########################################################################\n",
    "def reconstruct(I, Iresized, Yr, lp_threshold):\n",
    "    # 4 max-pooling layers, stride = 2\n",
    "    net_stride = 2**4\n",
    "    side = ((208 + 40)/2)/net_stride\n",
    "\n",
    "    # one line and two lines license plate size\n",
    "    one_line = (470, 110)\n",
    "    two_lines = (280, 200)\n",
    "\n",
    "    Probs = Yr[..., 0]\n",
    "    Affines = Yr[..., 2:]\n",
    "\n",
    "    xx, yy = np.where(Probs > lp_threshold)\n",
    "    # CNN input image size \n",
    "    WH = getWH(Iresized.shape)\n",
    "    # output feature map size\n",
    "    MN = WH/net_stride\n",
    "\n",
    "    vxx = vyy = 0.5 #alpha\n",
    "    base = lambda vx, vy: np.matrix([[-vx, -vy, 1], [vx, -vy, 1], [vx, vy, 1], [-vx, vy, 1]]).T\n",
    "    labels = []\n",
    "    labels_frontal = []\n",
    "\n",
    "    for i in range(len(xx)):\n",
    "        x, y = xx[i], yy[i]\n",
    "        affine = Affines[x, y]\n",
    "        prob = Probs[x, y]\n",
    "\n",
    "        mn = np.array([float(y) + 0.5, float(x) + 0.5])\n",
    "\n",
    "        # affine transformation matrix\n",
    "        A = np.reshape(affine, (2, 3))\n",
    "        A[0, 0] = max(A[0, 0], 0)\n",
    "        A[1, 1] = max(A[1, 1], 0)\n",
    "        # identity transformation\n",
    "        B = np.zeros((2, 3))\n",
    "        B[0, 0] = max(A[0, 0], 0)\n",
    "        B[1, 1] = max(A[1, 1], 0)\n",
    "\n",
    "        pts = np.array(A*base(vxx, vyy))\n",
    "        pts_frontal = np.array(B*base(vxx, vyy))\n",
    "\n",
    "        pts_prop = normal(pts, side, mn, MN)\n",
    "        frontal = normal(pts_frontal, side, mn, MN)\n",
    "\n",
    "        labels.append(DLabel(0, pts_prop, prob))\n",
    "        labels_frontal.append(DLabel(0, frontal, prob))\n",
    "        \n",
    "    final_labels = nms(labels, 0.1)\n",
    "    final_labels_frontal = nms(labels_frontal, 0.1)\n",
    "    \n",
    "    assert final_labels_frontal, \"No License plate is founded!\"\n",
    "\n",
    "    # LP size and type\n",
    "    out_size, lp_type = (two_lines, 2) if ((final_labels_frontal[0].wh()[0] / final_labels_frontal[0].wh()[1]) < 1.7) else (one_line, 1)\n",
    "\n",
    "    TLp = []\n",
    "    Cor = []\n",
    "    if len(final_labels):\n",
    "        final_labels.sort(key=lambda x: x.prob(), reverse=True)\n",
    "        for _, label in enumerate(final_labels):\n",
    "            t_ptsh = getRectPts(0, 0, out_size[0], out_size[1])\n",
    "            ptsh = np.concatenate((label.pts * getWH(I.shape).reshape((2, 1)), np.ones((1, 4))))\n",
    "            H = find_T_matrix(ptsh, t_ptsh)\n",
    "            Ilp = cv2.warpPerspective(I, H, out_size, borderValue=0)\n",
    "            TLp.append(Ilp)\n",
    "            Cor.append(ptsh)\n",
    "    return final_labels, TLp, lp_type, Cor\n",
    "\n",
    "\n",
    "#####################################################\n",
    "# Detects a licence plate in an image using a model #\n",
    "#####################################################\n",
    "def detect_lp(model, I, max_dim, lp_threshold):\n",
    "    min_dim_img = min(I.shape[:2])\n",
    "    factor = float(max_dim) / min_dim_img\n",
    "    w, h = (np.array(I.shape[1::-1], dtype=float) * factor).astype(int).tolist()\n",
    "    Iresized = cv2.resize(I, (w, h))\n",
    "    T = Iresized.copy()\n",
    "    T = T.reshape((1, T.shape[0], T.shape[1], T.shape[2]))\n",
    "    Yr = model.predict(T)\n",
    "    Yr = np.squeeze(Yr)\n",
    "    L, TLp, lp_type, Cor = reconstruct(I, Iresized, Yr, lp_threshold)\n",
    "    return L, TLp, lp_type, Cor\n",
    "\n",
    "##############################################################################\n",
    "# Returns the image of the car (vehicle) and the Licence plate image (LpImg) #\n",
    "##############################################################################\n",
    "def get_plate(image_path, Dmax=608, Dmin = 608):\n",
    "    vehicle = preprocess_image(image_path)\n",
    "    ratio = float(max(vehicle.shape[:2])) / min(vehicle.shape[:2])\n",
    "    side = int(ratio * Dmin)\n",
    "    bound_dim = min(side, Dmax)\n",
    "    _ , LpImg, _, cor = detect_lp(wpod_net, vehicle, bound_dim, lp_threshold=0.5)\n",
    "    return vehicle, LpImg, cor\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's make a detection!\n",
    "Now that we have all our functions ready, we can load our model, and try to detect a licence plate from a test image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-04-06T19:29:45.230209Z",
     "start_time": "2021-04-06T19:29:42.080364Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Load the model\n",
    "wpod_net_path = \"models/wpod-net.json\"\n",
    "wpod_net = load_model(wpod_net_path)\n",
    "\n",
    "# Get the vehicle image (vehicle) and the plate (LpImg)\n",
    "test_image_path = \"dataset/images/Cars374.png\"\n",
    "vehicle, LpImg, cor = get_plate(test_image_path)\n",
    "\n",
    "# Display the result\n",
    "fig = plt.figure(figsize=(12,6))\n",
    "grid = gridspec.GridSpec(ncols=2,nrows=1,figure=fig)\n",
    "fig.add_subplot(grid[0])\n",
    "plt.axis(False)\n",
    "plt.imshow(vehicle);\n",
    "grid = gridspec.GridSpec(ncols=2,nrows=1,figure=fig)\n",
    "fig.add_subplot(grid[1])\n",
    "plt.axis(False)\n",
    "plt.imshow(LpImg[0]);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great, we now know how to detect where the licence plate is located in an image!\n",
    "We're ready for the next step..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Recognizing the licence plate number\n",
    "Now that we have the image of the licence plate itself, we can create new functions to \"read\" what is written on it. Let's go!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## More processing functions\n",
    "First, we need to define other functions that will perform the rest of the recognition steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-04-06T19:29:58.475533Z",
     "start_time": "2021-04-06T19:29:58.336890Z"
    }
   },
   "outputs": [],
   "source": [
    "######################################################\n",
    "# Grabs the contour of each digit from left to right #\n",
    "######################################################\n",
    "def sort_contours(cnts,reverse = False):\n",
    "    i = 0\n",
    "    boundingBoxes = [cv2.boundingRect(c) for c in cnts]\n",
    "    (cnts, boundingBoxes) = zip(*sorted(zip(cnts, boundingBoxes),\n",
    "                                        key=lambda b: b[1][i], reverse=reverse))\n",
    "    return cnts\n",
    "\n",
    "###############################################\n",
    "# Recognizes a single character from an image #\n",
    "###############################################\n",
    "def predict_characters_from_model(image):\n",
    "    image = cv2.resize(image,(80,80))\n",
    "    image = np.stack((image,)*3, axis=-1)\n",
    "    prediction = labels.inverse_transform([np.argmax(character_model.predict(image[np.newaxis,:]))])\n",
    "    return prediction\n",
    "\n",
    "####################################################\n",
    "# Ties all the steps together in a single function #\n",
    "####################################################\n",
    "def  lpr_process(input_image_path):\n",
    "    # Get licence plate image\n",
    "    vehicle, LpImg, cor = get_plate(input_image_path)\n",
    "    \n",
    "    # Preprocess the LP image\n",
    "    plate_image = cv2.convertScaleAbs(LpImg[0], alpha=(255.0))\n",
    "    gray = cv2.cvtColor(plate_image, cv2.COLOR_BGR2GRAY)\n",
    "    blur = cv2.GaussianBlur(gray,(7,7),0)\n",
    "     # Applied inversed thresh_binary \n",
    "    binary = cv2.threshold(blur, 180, 255,cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]\n",
    "    kernel3 = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 3))\n",
    "    thre_mor = cv2.morphologyEx(binary, cv2.MORPH_DILATE, kernel3)\n",
    "    \n",
    "    # Find the contours of the characters using cv2\n",
    "    cont, _  = cv2.findContours(binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)\n",
    "    # creat a copy version \"test_roi\" of plat_image to draw bounding box\n",
    "    test_roi = plate_image.copy()\n",
    "    \n",
    "    # Initialize a list which will be used to append charater image\n",
    "    crop_characters = []\n",
    "    \n",
    "    # Define standard width and height of character\n",
    "    digit_w, digit_h = 30, 60\n",
    "    \n",
    "    # Validate found characters\n",
    "    for c in sort_contours(cont):\n",
    "        (x, y, w, h) = cv2.boundingRect(c)\n",
    "        ratio = h/w\n",
    "        if 1<=ratio<=3.5: # Only select contour with defined ratio\n",
    "            if h/plate_image.shape[0]>=0.5: # Select contour which has the height larger than 50% of the plate\n",
    "                # Draw bounding box arroung digit number\n",
    "                cv2.rectangle(test_roi, (x, y), (x + w, y + h), (0, 255,0), 2)\n",
    "                # Sperate number and gibe prediction\n",
    "                curr_num = thre_mor[y:y+h,x:x+w]\n",
    "                curr_num = cv2.resize(curr_num, dsize=(digit_w, digit_h))\n",
    "                _, curr_num = cv2.threshold(curr_num, 220, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)\n",
    "                crop_characters.append(curr_num)\n",
    "    \n",
    "    cols = len(crop_characters)\n",
    "    \n",
    "    # Recognize each character\n",
    "    license_plate_string =  \"\"\n",
    "    for i,character in enumerate(crop_characters):\n",
    "        title = np.array2string(predict_characters_from_model(character))\n",
    "        license_plate_string+=title.strip(\"'[]\")\n",
    "    \n",
    "    # Print results\n",
    "    if len(license_plate_string) >= 3 :\n",
    "        result = {\n",
    "            \"license_plate_number_detection_status\": \"Successful\",\n",
    "            \"detected_license_plate_number\": license_plate_string,\n",
    "            \"input_image_name\": input_image_path\n",
    "        }\n",
    "        #print(json.dumps(result))\n",
    "        return vehicle, LpImg, license_plate_string\n",
    "    else:\n",
    "        result = {\n",
    "            \"license_plate_number_detection_status\": \"Failed\",\n",
    "            \"reason\": \"Not able to read license plate, it could be blurred or complex image\",\n",
    "            \"input_image_name\": input_image_path\n",
    "        }\n",
    "        #print(json.dumps(result))\n",
    "        return vehicle, LpImg, 'Not able to read license plate'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading the new model\n",
    "We're almost there! We must now **load** the other **pre-trained model**, the one that is able to recognize letters and numbers. We are also loading character classes to classify them appropriately."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-04-06T19:30:03.222615Z",
     "start_time": "2021-04-06T19:30:01.515362Z"
    }
   },
   "outputs": [],
   "source": [
    "# Load the character recognition model\n",
    "character_net_path = 'models/character_recoginition/MobileNets_character_recognition.json'\n",
    "character_model = load_model(character_net_path)\n",
    "print(\"[INFO] Model loaded successfully...\")\n",
    "\n",
    "# Load the character classes\n",
    "labels = LabelEncoder()\n",
    "labels.classes_ = np.load('models/character_recoginition/license_character_classes.npy')\n",
    "print(\"[INFO] Labels loaded successfully...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Detecting the licence plate number!\n",
    "Everything is ready, we only have to process an example image from our dataset, and display the result:\n",
    "- The number that has been detected,\n",
    "- The original picture,\n",
    "- The licence plate picture."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-04-06T19:30:09.527911Z",
     "start_time": "2021-04-06T19:30:06.227234Z"
    }
   },
   "outputs": [],
   "source": [
    "vehicle, LpImg, license_plate_string = lpr_process(\"dataset/images/plate4.jpeg\")\n",
    "\n",
    "# Display the result\n",
    "fig = plt.figure(figsize=(12,6))\n",
    "grid = gridspec.GridSpec(ncols=2,nrows=1,figure=fig)\n",
    "fig.add_subplot(grid[0])\n",
    "plt.axis(False)\n",
    "plt.imshow(vehicle);\n",
    "grid = gridspec.GridSpec(ncols=2,nrows=1,figure=fig)\n",
    "fig.add_subplot(grid[1])\n",
    "plt.axis(False)\n",
    "plt.imshow(LpImg[0]);\n",
    "print('Detected plate number: ' + license_plate_string)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the result is not perfect, a 'B' has been mistaken for an '8', and a 'M' for an 'N'. It shows why there is always work to do to improve a model (still not too bad for  difficult plate seen from the side). But that's another story..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Processing a set of images\n",
    "Now that we have functions that we can directly call to do the processing, let's do this on all the test images that we have!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-04-06T19:30:49.048961Z",
     "start_time": "2021-04-06T19:30:21.132468Z"
    }
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "images_path = 'dataset/images'\n",
    "for filename in os.listdir(images_path):\n",
    "    if filename.endswith(\".jpg\") or filename.endswith(\".jpeg\") or filename.endswith(\".png\"):\n",
    "        vehicle, LpImg, license_plate_string = lpr_process(os.path.join(images_path, filename))\n",
    "        # Display the result\n",
    "        fig = plt.figure(figsize=(12,6))\n",
    "        fig.add_subplot(1,2,1)\n",
    "        plt.axis(False)\n",
    "        plt.imshow(vehicle);\n",
    "        fig.add_subplot(1,2,2)\n",
    "        plt.axis(False)\n",
    "        plt.imshow(LpImg[0]);\n",
    "        print('File Name:' + images_path + filename)\n",
    "        plt.show()\n",
    "        print('Detected plate number: ' + license_plate_string)\n",
    "        print('\\n-------------------------------------\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nbend = time.time()\n",
    "dur = nbend - nbstart\n",
    "print (\"elapsed seconds:\" + str(int(dur)))"
   ]
  }
 ],
 "metadata": {
  "finalized": {
   "timestamp": 1617733038100,
   "trusted": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  },
  "requirements": {
   "aliases": {},
   "dev-packages": {},
   "packages": {
    "keras": "==2.4.3",
    "opencv-python-headless": "==4.5.1.48"
   },
   "requires": {
    "python_version": "3.6"
   },
   "sources": [
    {
     "name": "pypi",
     "url": "https://pypi.org/simple",
     "verify_ssl": true
    }
   ]
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}