Seaborn_Matplotlib_solutions_and_demos.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Matplotlib and Seaborn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Handy hints "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some Jupyter shortcut keys:\n",
    "\n",
    "- Esc : get into Command mode (leaves Edit mode)\n",
    "- Enter : edit a cell (puts you in Edit mode)\n",
    "- h : see help (see all commands)\n",
    "- Shift+Enter or Ctrl+Enter : run the code in the cell\n",
    "- a / b : add new cell above/below\n",
    "- m : turn current cell into a Markdown cell\n",
    "- y : turn current cell into a Code cell\n",
    "- 1-6 : turn current cell into a heading (a type of Markdown cell)\n",
    "\n",
    "You can get help on Python objects and functions with `help()` or the `?` operator.\n",
    "\n",
    "In general, we are using plotting libraries that return objects encapsulating the plot. You can check the type of these returned objects with `type()`. \n",
    "\n",
    "Some of the plotting libraries we use need to communicate a lot of data to the browser. Current versions of Jupyter shouldn't have a problem, but if you are running an older version, you may need to launch this notebook with a higher data rate limit: `jupyter notebook --NotebookApp.iopub_data_rate_limit=10000000`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-14T23:56:53.109645Z",
     "start_time": "2018-01-14T23:56:53.106000Z"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-14T23:56:53.128661Z",
     "start_time": "2018-01-14T23:56:53.118959Z"
    }
   },
   "outputs": [],
   "source": [
    "# This causes Jupyter to display any matplotlib plots directly in the notebook\n",
    "# It also works for seaborn, since seaborn uses matplotlib to render plots\n",
    "%matplotlib inline\n",
    "from matplotlib import pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Be aware that Seaborn automatically changes Matplotlib's defaults *on import*. Not only your Seaborn plots, but also your Matplotlib plots, will look different once Seaborn is imported. If you don't want this behaviour, you can call `sns.reset_orig()` after import, or `import seaborn.apionly as sns` in the first place."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-14T23:56:53.136630Z",
     "start_time": "2018-01-14T23:56:53.132215Z"
    }
   },
   "outputs": [],
   "source": [
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this workshop we're going to load in two main datasets:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LTEE data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This data is from the [Long Term Evolution Experiment (LTEE)](http://myxo.css.msu.edu/ecoli/). This experiment has been running for over 30 years and over 50,000 E. coli generations, and is still ongoing. Twelve separate populations of E. coli have been propagated for the life of the experiment. Every 500 generations, each population is cloned and stored, allowing researchers to study evolutionary behaviour over the long term, and to \"rewind and replay\" alternate evolutionary trajectories by propagating from an earlier clone. \n",
    "\n",
    "Several interesting events have occurred during the experiment. Some populations have spontaneously developed hypermutator phenotypes. In addition, around generation 31,000 one population, Ara-3, spontaneously developed a rare and novel Cit+ mutation, giving it the ability to metabolise citrate in the substrate.\n",
    "\n",
    "There have been many publications from this experiment. A handful:\n",
    "\n",
    "- [Blount et al 2008: Historical contingency and the evolution of a key innovation in an experimental population of Escherichia coli](https://www.pnas.org/content/105/23/7899) - on the spontaneous development of citrate metabolisation and on potentiating mutations\n",
    "- [Tenaillon et al 2016: Tempo and mode of genome evolution in a 50,000-generation experiment](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4988878) - various investigations by sequencing and variant-calling over 50,000 generations of clones, including discussion of hypermutator phenotypes and genetic drift vs natural selection.\n",
    "\n",
    "Sequence data from clones is available, but for this workshop we'll just be using some published tabular data.\n",
    "\n",
    "A version of this dataset is also used by the [Data Carpentry lessons on Genomics](https://datacarpentry.org/genomics-workshop/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this lesson we'll use a large flat file containing both sample metadata on each clone, and information on observed mutations in their genomes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# If you have done the pandas and data tidying workshop and saved the final file,\n",
    "# you can try reading that file in here if you prefer\n",
    "ltee = pd.read_csv('ltee_merged.csv',\n",
    "                   index_col = 'Strain ID')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ltee.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ltee.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### House sales data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This data represents house sales in Seattle in 2014 and 2015. We have information on the houses themselves: location, size, quality, view, and whether the house is tagged as 'waterfront' or not. We also have information on the date and price of each sale."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-14T23:56:53.195524Z",
     "start_time": "2018-01-14T23:56:53.140978Z"
    }
   },
   "outputs": [],
   "source": [
    "sales = pd.read_csv(\"housing-data-10000.csv\", \n",
    "                    usecols=['id','date','price','lat','long', 'zipcode',\n",
    "                             'waterfront','view','grade','sqft_living'],\n",
    "                    parse_dates=['date'], \n",
    "                    dtype={'zipcode': 'category',\n",
    "                           'waterfront': 'bool'})\n",
    "\n",
    "# We'll downsample in order to make plots a bit simpler and smaller to store\n",
    "# In particular Altair defaults to a limit of 5000 observations\n",
    "sales = sales.sample(3000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sales.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sales.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sales.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Demo data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll read in the classic `iris` and `mtcars` datasets for demo purposes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris = pd.read_csv('iris.csv')\n",
    "cars = pd.read_csv('mtcars.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "cars.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Matplotlib "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Matplotlib is the oldest and still the fundamental plotting library in Python. It has a huge range of capabilities. Many other libraries (including Seaborn) use Matplotlib as a back-end renderer.\n",
    "\n",
    "Today we're focussing on plotting tabular data. We won't touch on all Matplotlib's capabilities. If you want to see more of the range of things Matplotlib can do, you can look through the [Matplotlib gallery](https://matplotlib.org/gallery.html.), or try out this excellent [Matplotlib tutorial](https://www.labri.fr/perso/nrougier/teaching/matplotlib/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some simple Matplotlib plots, inline in the notebook:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.scatter(x=[1,2,3],y=[5,6,8])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.scatter(iris['petal_length'], iris['petal_width'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's better to use the object-oriented Axes methods to draw the plot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "\n",
    "ax.scatter(iris['petal_length'], iris['petal_width'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(fig)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An example Matplotlib plot with legend and annotation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-14T05:36:41.975324Z",
     "start_time": "2018-01-14T05:36:41.805467Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "x = [1,2,3,4,5]\n",
    "y = [2,5,10,17,26]\n",
    "y2 = [1,4,9,11,9]\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "ax.plot(x, y, c='blue', label='Projected')\n",
    "ax.scatter(x, y2, c='red', label='Actual')\n",
    "fig.legend()\n",
    "ax.annotate(\"where it all went wrong\", \n",
    "                                 xy=(3,10), xytext=(1,12),\n",
    "                                 arrowprops={'width':2})\n",
    "\n",
    "fig.savefig('example_matplotilb.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Seaborn "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Seaborn builds on Matplotlib. Some nice features are:\n",
    "\n",
    "- works directly with Pandas dataframes, concise syntax\n",
    "- lots of plot types, including some more advanced options\n",
    "- statistical plotting: many plots do summary statistics for you\n",
    "- good default aesthetics and easy control of aesthetics\n",
    "- uses Matplotlib, so can use all Matplotlib backends (incl lots of image file formats)\n",
    "- underlying Matplotlib objects can be tweaked directly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For completeness, here's the plot we made before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame({\n",
    "    'Time': [1,2,3,4,5],\n",
    "    'Projected': [2,5,10,17,26],\n",
    "    'Actual': [1,4,9,11,9]\n",
    "})\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "sns.scatterplot(data=df, x='Time', y='Actual', color='red', ax=ax)\n",
    "sns.lineplot(data=df, x='Time', y='Projected', color='blue', ax=ax)\n",
    "\n",
    "ax.set_ylabel('Huge profits')\n",
    "\n",
    "ax.annotate(\"where it all went wrong\", \n",
    "                                 xy=(3,10), xytext=(1,12),\n",
    "                                 arrowprops={'width':2})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that we can add changes like annotations in exactly the same way, as we have Matplotlib Figure and Axes objects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Demo: seaborn scatterplot can do hue as just another channel\n",
    "sns.scatterplot(data=iris, \n",
    "                x='petal_length', \n",
    "                y='petal_width',\n",
    "                hue='species',\n",
    "                size='sepal_width')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn and Pandas "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In some cases we can use Seaborn by passing in lists (or arrays or series) directly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-14T23:56:54.489607Z",
     "start_time": "2018-01-14T23:56:53.283680Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "sns.barplot(x=['A','B','C'], y=[33,44,20])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However Seaborn is aware of Pandas and it is very common to use Seaborn directly with DataFrames. Plotting functions can take a DataFrame as their `data` parameter and then refer directly to column names:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-14T23:56:54.706924Z",
     "start_time": "2018-01-14T23:56:54.492344Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "sns.barplot(data=iris, x='species', y='petal_length')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here Seaborn has interpreted the `x` and `y` arguments as field names in the supplied DataFrame. Notice also that Seaborn has performed the summary statistics for us - in this case, using the default `estimator`, which is `mean()`. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice also what happens if we simply swap the `x` and `y` parameters. Seaborn will automatically deduce that the categorical or string-like variable must be the bar labels, and the numeric variable must be the numeric axis:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-14T23:56:54.706924Z",
     "start_time": "2018-01-14T23:56:54.492344Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "sns.barplot(data=iris, y='species', x='petal_length')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What happens if the grouping is not obviously categorical? How could we fix this?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.barplot(data=cars, x='acceleration', y='origin')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cars.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# parameter orient=\"h\", or appropriate Pandas dtype\n",
    "cars['origin'] = cars['origin'].astype('category')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercises: \n",
    "\n",
    "**1:** Create a count plot using `sns.countplot()` on the `ltee` data, showing how many clones have each `Mutator` phenotype. Note that you do not need to specify the `y` axis variable for a countplot, just the `x` axis variable (i.e. category).\n",
    "\n",
    "**2:** Create a (vertical) bar plot using the `sales` data, showing how house prices vary with the value of the property `grade`.\n",
    "\n",
    "Bar plots are often deplored as a way of showing statistical estimates, as only the top of the bar is really important, and the bar itself is a visual distraction. A point plot is an alternative, and plots like box plots can show more information. Several other plot types also show distributional information within categories.\n",
    "\n",
    "**3:** Reproduce the plot you just made, using instead each of the Seaborn functions:\n",
    "\n",
    "- pointplot()\n",
    "- boxplot()\n",
    "- violinplot()  (try the `scale` parameter)\n",
    "- boxenplot()\n",
    "- stripplot() [SEE WARNING]  (try the `jitter` parameter)\n",
    "- swarmplot() [SEE WARNING]\n",
    "\n",
    "Note what sort of information about the distribution is shown by each.\n",
    "\n",
    "WARNING: `stripplot()` and `swarmplot()` will plot individual data points. There are too many house sales to easily display in this way - you may want to subsample the dataframe with e.g.  `data=sales.sample(100)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.countplot(data=ltee, x='Mutator')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.barplot(data=sales, x='grade', y='price')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "?sns.violinplot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.violinplot(data=sales, x='grade', y='price', scale='width')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hue "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Many Seaborn plotting functions take a `hue` parameter. This colours the plot elements by some categorical variable, but more than this, summary statistics are calculated for each level of the hue variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "sns.scatterplot(data=ltee, \n",
    "                x='Generation', \n",
    "                y='Synonymous Base Substitutions', \n",
    "                palette='bright', hue='Population')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "sns.lineplot(data=ltee, \n",
    "            x='Generation', \n",
    "            y='Synonymous Base Substitutions', \n",
    "            palette='bright', hue='Population')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "sns.lineplot(data=ltee, \n",
    "            x='Generation', \n",
    "            y='Relative fitness', \n",
    "            palette='bright', hue='Population')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:09.975033Z",
     "start_time": "2018-01-15T00:00:09.367327Z"
    }
   },
   "outputs": [],
   "source": [
    "sns.lmplot(data=sales, x='sqft_living', y='price', hue='waterfront')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercises:\n",
    "\n",
    "1. Subset the `ltee` dataset to get only the clones where the `Mutator` value is \"PM\"\n",
    "2. Create an `lmplot` of `Total Mutations` over time (i.e. against `Generation`). Do this without a `hue` parameter, then add in `Population` as the `hue` parameter. \n",
    "3. Try adding the `hue` parameter to one of your previous plots of some other type - for instance, a box plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "subset = ltee[ ltee['Mutator']==\"PM\" ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# or, convenient but less flexible\n",
    "subset = ltee.query('Mutator==\"PM\"')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.lmplot(data=subset, x='Generation', y='Total Mutations', \n",
    "           hue='Population')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we'd wanted to colour scatter points by some continuous variable, `hue` can be made to work, but doesn't really make sense as it is intended for discrete values. In this specific case, we could pass our colour variable down to the underlying Matplotlib scatter call via the `scatter_kws` parameter. We'll look more at this later. Or, if we're not trying to fit a linear model, we could just use Seaborn aesthetics around a Matplot scatter plot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:10.985210Z",
     "start_time": "2018-01-15T00:00:09.978297Z"
    }
   },
   "outputs": [],
   "source": [
    "sample = sales.sample(200)\n",
    "\n",
    "with sns.axes_style('white'):\n",
    "    fig, ax = plt.subplots(figsize=(8,8))\n",
    "    ax.scatter(x=sales['long'], y=sales['lat'], c=sales['price'].apply(np.log), \n",
    "               alpha=0.5, s=7, cmap='Reds')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For discrete color palettes, as needed by the `hue` parameter, Seaborn has a `color_palette()` function to generate a useful range of palettes. You can find [a tutorial on choosing color palettes here](https://seaborn.pydata.org/tutorial/color_palettes.html). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compound plots "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Seaborn has some plotting functions which create more complex figures made of multiple subplots. These include `cat()`, `catplot()`, `jointplot()`, `lmplot()` and `clustermap()`. Let's see a few examples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:11.855411Z",
     "start_time": "2018-01-15T00:00:10.988170Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# jointplot shows a scatter or density plot, with marginal distributions\n",
    "sns.jointplot(data=sales, x='sqft_living', y='price') #, kind='reg')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:12.995121Z",
     "start_time": "2018-01-15T00:00:11.858767Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# pairplot shows pairwise relationships between variables\n",
    "sns.pairplot(data=ltee,\n",
    "             vars=['Generation', 'Total Mutations', 'Nonsynonymous Base Substitutions', 'IS Element Insertions'])\n",
    "             #hue='Mutator')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:12.995121Z",
     "start_time": "2018-01-15T00:00:11.858767Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# pairplot shows pairwise relationships between variables\n",
    "sns.pairplot(data=ltee,\n",
    "             vars=['Generation', \n",
    "                   'Total Mutations', \n",
    "                   'Nonsynonymous Base Substitutions', \n",
    "                   'IS Element Insertions'],\n",
    "             hue='Mutator')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:13.461175Z",
     "start_time": "2018-01-15T00:00:12.998412Z"
    }
   },
   "outputs": [],
   "source": [
    "# catplot conditions different subplots on different categorical variable values\n",
    "# we map variables to row and column of a grid of plots (as well as to hue)\n",
    "# in this example, we just use columns, and so have only one row\n",
    "sns.catplot(data=cars, kind='box',\n",
    "               x='year', y='mpg', \n",
    "               row='origin',\n",
    "               aspect=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# vs a single (unfaceted) boxplot\n",
    "sns.boxplot(data=cars, x='year', y='mpg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise:\n",
    "\n",
    "Create a joint plot using the `ltee` data showing `Cell size` against `Relative fitness`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.jointplot(data=ltee,\n",
    "              x='Relative fitness',\n",
    "              y='Cell size')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-11T11:39:43.169894Z",
     "start_time": "2018-01-11T11:39:43.013278Z"
    }
   },
   "source": [
    "#### Exercise: \n",
    "\n",
    "Design a plot using `sns.catplot()`, to show the relationship between house price and: grade, view, and whether the property is waterfront. Available channels of information are:\n",
    "\n",
    "- x and y coordinates\n",
    "- hue\n",
    "- row and column of subplot (`row` and `col`)\n",
    "\n",
    "You can set the `kind` parameter to the kind of plot you want to make: point, bar, count, box, violin, or stripplot.\n",
    "\n",
    "You do not have to use all of these channels - in fact your plot may be difficult to take in if you do."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Colour and Palettes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Seaborn has good colour options. There are a few ways we could want to use access colours:\n",
    "\n",
    "* Specify an individual colour for some plot element. Matplotlib named colours can be used, or rgb values specified. Also check out the `sns.xkcd_rgb` dictionary of 954 named colours from the XKCD colour survey - for instance, `sns.xkcd['fire engine red']` is a colour.\n",
    "* Specify a colormap, for mapping a continuous value to colour. All Matplotlib colormaps can be used by name. You can see these under the `plt.cm` module. Seaborn's `light_palette()` and `dark_palette()` functions can also generate custom colourmaps easily.\n",
    "* Specify a discrete colour palette (a list of colours), for mapping a discrete or categorical variable to colour. In Seaborn, there is a distinction between colour palettes and colormaps. In general, you can create a colour palette by explicitly listing some colours, or by selecting a series of colours along some colormap. There are several functions, including `color_palette()`, `light_palette()`, `dark_palette()`, `diverging_palette()` and `xkcd_palette()`, which can produce many discrete colour palettes of whatever size you need. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:08.929020Z",
     "start_time": "2018-01-15T00:00:08.834779Z"
    }
   },
   "outputs": [],
   "source": [
    "# An example discrete colour palette of 7 colours, based on the XKCD colour \"denim blue\"\n",
    "# palplot is a function to visualise a palette\n",
    "palette = sns.light_palette(\"denim blue\", n_colors=7, input='xkcd')\n",
    "sns.palplot(palette)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:09.043908Z",
     "start_time": "2018-01-15T00:00:08.932500Z"
    }
   },
   "outputs": [],
   "source": [
    "# Equivalently (to illustrate that we can use an rgb value directly)\n",
    "denim_blue = sns.xkcd_rgb[\"denim blue\"]\n",
    "print(denim_blue)\n",
    "palette = sns.light_palette(denim_blue, n_colors=7)\n",
    "print(palette)\n",
    "sns.palplot(palette)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Try out the Seaborn function `choose_diverging_palette()` in your notebook (it requires no arguments). You can assign the result to a variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p = sns.choose_diverging_palette()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.xkcd_rgb['denim blue']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try a heatmap. Unlike most Seaborn functions, heatmaps take data in wide form. We'll need to pivot our long-form data to get a table of numbers, indexed by two variables. The heatmap function will then transform each number to a colour via a continuous colourmap.\n",
    "\n",
    "#### Exercise: \n",
    "\n",
    "Use `ltee.pivot_table()` to produce a table showing the (average) number of mutations per Population and Generation. The x-axis (column headers) should be the values of the `Generation` variable, and the rows (index) should be the values of the `Population` variable. If you're new to pivot tables in Pandas, check the example below that uses the `cars` dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mutations_table = ltee.pivot_table(index='Population', \n",
    "                                   columns='Generation',\n",
    "                                   values='Total Mutations',\n",
    "                                   aggfunc='mean')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:09.103356Z",
     "start_time": "2018-01-15T00:00:09.048120Z"
    }
   },
   "outputs": [],
   "source": [
    "# Here's an example using the toy dataset\n",
    "# Avoid looking at the details first if you want to solve the above without hints!\n",
    "\n",
    "speed_table = cars.pivot_table(index='origin', columns='year', values='mpg', aggfunc=np.mean)\n",
    "speed_table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:09.363724Z",
     "start_time": "2018-01-15T00:00:09.106153Z"
    }
   },
   "outputs": [],
   "source": [
    "sns.heatmap(speed_table, vmin=0, annot=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercises:\n",
    "\n",
    "* If you haven't already, produce a heatmap with the `ltee` pivot table you produced above. You will probably want to leave off the annotations unless your plot is very large. \n",
    "* Specify a different colourmap using the `cmap` parameter to `heatmap`. An ascending (not diverging) colourmap is appropriate for counts that are all positive.\n",
    "* Some populations have far more mutations than others, and so our heatmap doesn't really show detail for the lower end of the scale. Try to plot a heatmap where colour is based on the *log* of the mutation count.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:09.363724Z",
     "start_time": "2018-01-15T00:00:09.106153Z"
    }
   },
   "outputs": [],
   "source": [
    "sns.heatmap(np.log(mutations_table), vmin=0, cmap='Greens')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Seaborn and Matplotlib "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Seaborn plotting functions call Matplotlib plotting functions, and we can pass arguments down to these underlying functions.\n",
    "\n",
    "For instance, `lmplot()` calls `scatter()` and `plot()` to draw points and lines. We can pass arguments down using `scatter_kws` and `line_kws`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:14.587879Z",
     "start_time": "2018-01-15T00:00:13.464146Z"
    }
   },
   "outputs": [],
   "source": [
    "sns.lmplot(data=sales, x='grade', y='price',\n",
    "           scatter_kws={'alpha': 0.3},\n",
    "           line_kws={'linestyle': 'dashed', 'color': 'red'})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Which underlying plotting function is called by your `catplot` above? You can try passing arguments to it with catplot's `kwargs`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at the object returned by a Seaborn plot function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:14.737395Z",
     "start_time": "2018-01-15T00:00:14.590597Z"
    }
   },
   "outputs": [],
   "source": [
    "g = sns.countplot(ltee['Mutator'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:14.745309Z",
     "start_time": "2018-01-15T00:00:14.740201Z"
    }
   },
   "outputs": [],
   "source": [
    "type(g)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a Matplotlib `Axes` object. We can use all the usual `Axes` methods to tweak the plot. What's more, if we have an existing `Axes` object, we can ask Seaborn to draw the plot onto it by passing it in as the `ax` parameter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:14.909998Z",
     "start_time": "2018-01-15T00:00:14.748778Z"
    }
   },
   "outputs": [],
   "source": [
    "fig, myaxes = plt.subplots(figsize=(7,5), facecolor=sns.xkcd_rgb['pale pink'])\n",
    "\n",
    "sns.countplot(ltee['Mutator'], color='purple', ax=myaxes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the plot is still attached to `fig`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:14.995755Z",
     "start_time": "2018-01-15T00:00:14.912758Z"
    }
   },
   "outputs": [],
   "source": [
    "fig"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The simpler Seaborn plotting functions return `Axes` objects, and can take an `Axes` as a parameter. More complex functions like `jointplot()` and `catplot()` need to make multiple subplots. What do they return?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:16.110729Z",
     "start_time": "2018-01-15T00:00:14.998478Z"
    }
   },
   "outputs": [],
   "source": [
    "g = sns.pairplot(data=ltee,\n",
    "             vars=['Generation', 'Total Mutations', 'Nonsynonymous Base Substitutions', 'IS Element Insertions'],\n",
    "             hue='Mutator')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:16.120201Z",
     "start_time": "2018-01-15T00:00:16.113887Z"
    }
   },
   "outputs": [],
   "source": [
    "type(g)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:16.132657Z",
     "start_time": "2018-01-15T00:00:16.123808Z"
    }
   },
   "outputs": [],
   "source": [
    "type(g.fig)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:16.145801Z",
     "start_time": "2018-01-15T00:00:16.135870Z"
    }
   },
   "outputs": [],
   "source": [
    "g.axes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:16.585296Z",
     "start_time": "2018-01-15T00:00:16.150407Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "g.fig"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, these more complex functions produce entire Matplotlib `Figure` objects, which can contain multiple `Axes`. The `Figure`, however, comes wrapped in a Seaborn class, which provides some convenient functions to manipulate the figure properties."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-01-15T00:00:18.201406Z",
     "start_time": "2018-01-15T00:00:16.588311Z"
    }
   },
   "outputs": [],
   "source": [
    "# g.set sets a property on all Axes in the Figure\n",
    "# set x-axis to log scale:\n",
    "g.set(xscale='log')\n",
    "g.fig"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Seaborn classes commonly returned are:\n",
    "\n",
    "- JointGrid : a central bivariate plot with two marginal univariate plots. Used by `jointplot()`. \n",
    "- PairGrid : a grid of subplots for plotting pairwise relationships. Has convenience functions for mapping plots onto diagonal and non-diagonal elements. Used by `pairplot()`.\n",
    "- FacetGrid : a grid of subplots showing the same relationship, conditioned on some variable across different subplots. Designed to map fields of a Pandas DataFrame to different rows, columns, and hues. Used by `catplot()` and `lmplot()`.\n",
    "\n",
    "It's possible to instantiate these classes yourself for custom plots."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {
    "height": "138px",
    "width": "252px"
   },
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": "block",
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}