10_aggResults.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Final Results {#sec-final-res}\n",
    "\n",
    "We only used 10 iterations in the previous three chapters to demonstrate how our algorithm works. However, to get them to really work, we need to run several thousand iterations.\n",
    "\n",
    "Also, to cancel out noises and randomness, we need to use all the fifty variations of data.\n",
    "\n",
    "With the help of [The Center for High Throughput Computing](https://chtc.cs.wisc.edu/) at the University of Wisconsin-Madison, we were able to run these tests. In the following, we present our results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Results of Hard K-Means](img/box_plot_hk.png){#fig-res-hard-kmeans}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Results of Soft K-Means](img/box_plot_sk.png){#fig-res-soft-kmeans}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Results of Conjugate Priors](img/box_plot_cp.png){#fig-res-conjugate-priors}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the two results, we are able to see that it is better to have more participants, because that offers more information for our models. In terms of healthy ratio, it seems $50\\%$ is a sweet spot.\n",
    "\n",
    "Also, we notice that conjugate priors perform better than soft K-Means."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also tested the two algorithms developed by [UCL POND group](https://ucl-pond.github.io/) with the same 750 datasets: [EBM Basic](https://github.com/ucl-pond/ebm) and [KDE EBM](https://github.com/ucl-pond/kde_ebm). See the following for results:\n",
    "\n",
    "![Results of EBM](img/ebm-basic.png){#fig-res-ebm-basic}\n",
    "\n",
    "The parameters we used in the package of [ebm](https://github.com/ucl-pond/ebm) are:\n",
    "\n",
    "- `n_iter = 10000`\n",
    "- `greedy_n_iter=10`\n",
    "- `greedy_n_init=5`\n",
    "\n",
    "More specific configurations can be found at https://github.com/hongtaoh/ucl_ebm/blob/master/implement/calc_tau_basic.ipynb\n",
    "\n",
    "![Results of KDE EBM](img/ebm-kde.png){#fig-res-ebm-kde}\n",
    "\n",
    "The parameters we used in the package of [kde-ebm](https://github.com/ucl-pond/kde_ebm) are:\n",
    "\n",
    "- `n_iter = 10000`\n",
    "- `greedy_n_iter=10`\n",
    "- `greedy_n_init=5`\n",
    "\n",
    "More specific configurations can be found at https://github.com/hongtaoh/ucl_kde_ebm/blob/master/implement/calc_tau_basic.ipynb\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the performance of KDE EBM is more stable but EBM basic performs well when the healthy ratio is below $50\\%$.\n",
    "\n",
    "Neither of these methods had results as good as conjugate priors. We have to point out that, even though their accuracy is not as high, their speed is really high. Both of these two algorithms can generate results with a single laptop GPU in just one hour; however, it might take days for our algorithms."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "bayes",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.19"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}