diff --git a/docs/html/.doctrees/environment.pickle b/docs/html/.doctrees/environment.pickle
index 8530386a..4386c851 100644
Binary files a/docs/html/.doctrees/environment.pickle and b/docs/html/.doctrees/environment.pickle differ
diff --git a/docs/html/.doctrees/exercises/exercise_power_calculations.doctree b/docs/html/.doctrees/exercises/exercise_power_calculations.doctree
new file mode 100644
index 00000000..9d39ee40
Binary files /dev/null and b/docs/html/.doctrees/exercises/exercise_power_calculations.doctree differ
diff --git a/docs/html/.doctrees/nbsphinx/exercises/exercise_power_calculations.ipynb b/docs/html/.doctrees/nbsphinx/exercises/exercise_power_calculations.ipynb
new file mode 100644
index 00000000..942a31f4
--- /dev/null
+++ b/docs/html/.doctrees/nbsphinx/exercises/exercise_power_calculations.ipynb
@@ -0,0 +1,271 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Power Calculations and Experiment Planning\n",
+ "\n",
+ "When you read about them in a book, power calculations can feel very abstract. To make the concept more concrete, and to give you a chance to familiarize yourself with power calculation tools and the considerations that go into power calculations, in this exercise you will play the role of an experiment planner hired by a Non-Governmental Organization (NGO) named [Bandhan](https://www.bandhan.org/) in West Bengal, India. Bandhan is planning to roll out a program that provides livestock, cash, and some basic training to households in extreme poverty. They hope to demonstrate the value of cash and asset transfers, and so wish their program to take the form of a randomized experiment, with data being collected on both control and treatment households.\n",
+ "\n",
+ "Your job will be to help them estimate how many households they should enroll in the program.\n",
+ "\n",
+ "This was, in fact, a real program, and so the data you will be working with comes from the actual program (conducted in 2007, with followup data collection in 2010, 2017, and 2020), allowing us to do some retrospective comparison of your power calculations and what actually occurred."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## The Context\n",
+ "\n",
+ "> Development economics has long posited that the poor may be poor for no good reason other than the fact that they started poor. This is the idea of a poverty trap, which has the implication that a one-time capital grant that makes very poor households significantly less poor (\"big push\") might set off a virtuous cycle that takes them out of poverty. Forty-three countries now embrace some version of this idea and make large transfers to over 3.1 million of their poorest households. In particular, the \"Targeting the Ultra Poor\" (TUP) intervention, pioneered by BRAC in Bangladesh, employs a multifaceted approach, offering poor households an asset transfer, consumption support, savings, and training in the hopes of unlocking a poverty trap.\n",
+ "\n",
+ "- [*Long-Term Effects of the Targeting the Ultra Poor Program*](https://economics.mit.edu/research/publications/long-term-effects-targeting-ultra-poor-program), Abhijit Banerjee, Esther Duflo, and Garima Sharma, 2021.\n",
+ "\n",
+ "In 2007, Bandhan in West Bengal, India created a pilot program to provide direct, unconditional transfers of productive livestock and a 30 or 40 week stipend of 90 rupees a week (about 7 US dollars a week using the [Purchasing Power Parity (PPP)](https://en.wikipedia.org/wiki/Purchasing_power_parity) exchange rate). The program targeted the poorest households in these villages based on range of criteria, and the average eligible household was later estimated to have a consumption level of about 1.35 2018 US Dollars per capita per day in PPP terms $.^1$\n",
+ "\n",
+ "Because the goal of the program was, in significant part, to demonstrate the effect of direct asset transfers, Bandhan identified twice as many eligible households as it could support in the pilot and randomly selected half of the households to act as controls and half to be treated $.^2$ Treated households were offered assets from a menu of options, from the most common choice was productive livestock (e.g., cows and goats). $^3$ Bandhan's contact with the households came to an end 18 months after the households were initially provided their livestock and cash transfers began.\n",
+ "\n",
+ "Although data was collected on a huge range of attributes of the households in this program, our focus will be on per capita household expenditures in 2018 US Dollars (PPP). \n",
+ "\n",
+ "(Note that these households certainly were not buying their food with dollars, and in most cases they weren't even buying most of their food with rupees — consumption estimates in this type of study are calculated by collecting detailed data on what household members have consumed in the past week, then estimating the price one would pay to buy those goods in local markets and using [PPP conversion rates](https://en.wikipedia.org/wiki/Purchasing_power_parity) conversion rates to convert that into US Dollars).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## Exercises\n",
+ "\n",
+ "### Exercise 1\n",
+ "\n",
+ "Load a thinned version of household-level survey data from [Banerjee, Duflo, and Sharma (2021)'s](https://economics.mit.edu/research/publications/long-term-effects-targeting-ultra-poor-program) evaluation of the Bandham program [here](https://github.com/nickeubank/MIDS_Data/tree/master/cash_transfers). \n",
+ "\n",
+ "This is just a version of the replication data for that paper ([which can be found here](https://www.openicpsr.org/openicpsr/project/130362)) with only the variables that are relevant for our analysis."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 2\n",
+ "\n",
+ "The first step in doing any power calculations is to establish an estimate of the baseline level of variation that exists in the dependent variable you wish to study. In this case, that variable is Per Capita Monthly Expenditures, and the baseline survey values of expenditures are in `pc_exp_month_bl` (the `_bl` stands for \"baseline\", meaning this data was collected prior to any households receiving asset transfers and indeed any households being assigned to treatment or control).\n",
+ "\n",
+ "What is the mean and standard deviation of `pc_exp_month_bl` in the baseline survey? Divide by 30 to confirm our average household has the expected per capita expenditure level of about US Dollar 1.35 (PPP) a day.\n",
+ "\n",
+ "You'll use these quantites a lot, so assign them to variables."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 3\n",
+ "\n",
+ "For these exercises, we'll use the `power` module in `statsmodels`. [You can read about it here.](https://www.statsmodels.org/stable/stats.html#power-and-sample-size-calculations)\n",
+ "\n",
+ "Since we're comparing means in a continuous variable (expenditures) from two samples of households, we will use `TTestIndPower` in `statsmodels.stats.power`. Import this class and instantiate a new instance (for some reason this is class based, so you have to start of with a command like `my_power = TTestIndPower()`). \n",
+ "\n",
+ "Note that a common situation in data science is testing a difference in *proportions*, as in situations where your dependent variable is binary and each group's mean is the share for whom the binary variable is 1. This comes up a lot with apps and websites — e.g., \"clicked an ad,\" \"subscribed,\" \"made a purchase.\" For that reason, there's actually a full sub-class of power calculating tools for [proportions you can read about here.](https://www.statsmodels.org/stable/stats.html#proportion) Basically, because the standard deviation of a binary variable is just $\\sqrt{p * (1-p)}$, power calculations become really simple."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 4\n",
+ "\n",
+ "[TTestIndPower has three methods](https://www.statsmodels.org/stable/generated/statsmodels.stats.power.TTestIndPower.html#statsmodels.stats.power.TTestIndPower) — `plot_power`, `power`, and `solve_power` — but `solve_power` does everything `power` does, so there are basically two methods.\n",
+ "\n",
+ "The idea of `solve_power` is that you give it all but one parameter of a power calculation — minimum detectable effect, number of observations, power, and p-value $\\alpha$ threshold — and it will solve for the omitted variable.\n",
+ "\n",
+ "To get a quick flavor for how this works, set: \n",
+ "\n",
+ "- Minimal Detectable Effect (`effect_size`) to `0.5` (we'll take about the units of that argument soon), \n",
+ "- `alpha` to `0.05`, \n",
+ "- `ratio` to `1` (this is the ratio of between the number of observations being treated and the number in control — in this study, the target was to have these be equal, as that provides the best statistical power for a given total number of observations).\n",
+ "- `power` to `0.8`\n",
+ "- `alternative` to `two-sided` (we want to do a two-tailed t-test, as you could imagine social unrest caused by transfers could make people worse off).\n",
+ "- `nobs1` to `None`. By passing `None`, we're telling `solve_power` this is the quantity we want it to solve for.\n",
+ "\n",
+ "So in other words, we're asking:\n",
+ "\n",
+ "- how many observations do we need to have in the treatment arm,\n",
+ "- assuming we'll have the same number also in the control arm,\n",
+ "- to have an 80% chance\n",
+ "- of rejecting the null hypothesis of no-effect at a significance threshold of 0.05\n",
+ "- if the true effect is of size 0.5?\n",
+ "\n",
+ "What's the answer?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 5\n",
+ "\n",
+ "Of all these quantities, perhaps the most obscure is the Minimal Detectable Effect (`effect_size`). What is this 0.5? Where did it come from? And how did you do all that without telling it that standard deviation you had me calculate?\n",
+ "\n",
+ "If you check the docs for `solve_power`, you'll see that you're supposed to pass `effect_size` the \"standardized effect size, difference between the two means divided by the standard deviation. `effect_size` has to be positive.\"\n",
+ "\n",
+ "In other words, when we passed 0.5, we weren't saying we wanted to be able to detect an effect of 0.5 2018 US Dollars (PPP) per capita per month, we were saying we wanted to detect an effect of 1/2 standard deviation. \n",
+ "\n",
+ "Given the standard deviation in our baseline survey was about 24 Dollars, that means we were saying we wanted to detect an effect size of about 12 US Dollars. \n",
+ "\n",
+ "Is that a lot? Well, the average per capita monthly expenditures in the baseline survey was about 41 Dollars, so it's an increase in expenditures of about 25%. \n",
+ "\n",
+ "OK, but is that what we should use?\n",
+ "\n",
+ "Well, the idea of a Minimal Detectable Effect is that you want to set it at a level where (a) if this *was* the real effect when you rejected the null, you'd be happy you did the experiment and, presumably, would be comfortable scaling up the treatment, but also (b) if the true effect were any smaller, you wouldn't want to reject the null hypothesis and scale up the treatment.\n",
+ "\n",
+ "Basically, if you set this too high, you may run an under-powered experiment where you miss out on learning that your treatment was effective; if you set this too low, you're spending time and money collecting more data than is probably necessary given you might reject the null at a point estimate where you still wouldn't scale up the treatment.\n",
+ "\n",
+ "So what value makes sense here? That's a question you'd want to start off by asking the stakeholders. Since Bandhan wants to use this to promote cash transfers, they probably need the effect to be relatively large to be convincing. Let's assume that, three years after the intervention, they feel they need to show that incomes have increased by at least 30%. \n",
+ "\n",
+ "What value does that imply should be passed to `effect_size`? What is the new implied number of treated households they'll need? How many households total?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 6\n",
+ "\n",
+ "Let's also suppose that because Bandhan really wants to sell their result, they also want to ensure no one claims it might have arisen by chance. What parameter would you adjust, and in what direction? Try modifying it and get a new observation requirement (can you predict what the change in num of observations will be before you do it?)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 7\n",
+ "\n",
+ "Now suppose that your stakeholder is budget constrained to only being able to enroll a total of 100 (50 in each arm). Assuming an alpha of `0.05` and a power of `0.8`, what is their minimal detectable effect size (in dollars)?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 8\n",
+ "\n",
+ "Suppose your stakeholder wants to see a few different scenarios to see how different experiment sizes would impact power given different effect sizes. Let's use `plot_power`. Pass it an array of treatment arm observation counts you want evaluated (I'd recommend `np.arange()`) between 20 and 100, along with effect sizes of 0.2, 0.4, and 0.6 (again, use `np.arange()`).\n",
+ "\n",
+ "If the stakeholder decided they wanted power above 90% — they're only gonna get a chance to do this once, after all! — what kind of sample size would they need with a minimal detectable effect size of 0.6? (Approximately).\n",
+ "\n",
+ "You may need to check the docs to figure out how to use it."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Extra Credit Extension"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 9\n",
+ "\n",
+ "As noted above, this study actually was conducted, so we know the actual treatment effect. Calculate that treatment effect 3 years after these families first received asset transfers by calculating the difference in `pc_exp_month_el2` between the `treatment == 1` and `treatment == 0` groups.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 10\n",
+ "\n",
+ "What was the statistical power of the actual test? That is, what was the probability, given this effect size and the number of observations in the study, that they would reject the null hypothesis of no effect at alpha=0.05 and given the actual final sample sizes?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 11\n",
+ "\n",
+ "What would the power be if Bandhan had only enrolled 200 households per arm? \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 13\n",
+ "\n",
+ "The idea of the \"power\" of a test is that, when we draw a sample from a population, there's some variation in who happens to end up in that sample. Consequently, even if the *true* effect size in the population is equal to our Minimal Detectable Effect, there's some probability that when we compare treated and untreated outcomes for the $N$ people who actually end up in the study, that Minimal Detectable Effect may not be evident in that sample (at the level of statistical significance $\\alpha$ we have chosen).\n",
+ "\n",
+ "Given that, one way to think of power is: \"If the true effect in the population is our Minimal Detectable Effect, then if I were able to re-run this experiment over and over — drawing new people into the study and testing the difference in outcomes between the control and treated sample each time — then in what percentage of those instances of the experiment would I reject the null hypothesis of no effect?\" Power of 80% means that we'd expect to reject the null of no effect in 80% of those many experiments.\n",
+ "\n",
+ "Well, we can basically do that thought experiment with this data! \n",
+ "\n",
+ "To do the experiment you did the power calculations for in Exercise 11, we just need to sample 200 observations from the treated group and 200 from the control group, treat that as our experimental sample, and estimate the difference in per capita month expenditures between those two groups (along with the associated p-value).\n",
+ "\n",
+ "Then we can repeat that over and over to simulate \"re-running\" the experiment, each time drawing a new sample of 200 treated observations and 200 control observations. Then we can store the p-values from all these \"re-run\" experiments and see how often we reject the null of no effect!\n",
+ "\n",
+ "Note that when you do this experiment, we have to sample our 200 observations from each treatment arm *with replacement*, just as you would when bootstrapping (if that means anything to you — if it doesn't, don't worry about it). \n",
+ "\n",
+ "So: write a loop where, on each pass, you draw 200 observations (with replacement) from treatment and 200 from control, then calculate the treatment effect and p-value for that sample. Repeat this 10,000 times. \n",
+ "\n",
+ "In what share of cases would you reject the null of no effect at alpha = 0.05?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 14\n",
+ "\n",
+ "Repeat 12 and 13 with 300 observations per arm."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 15\n",
+ "\n",
+ "What did you learn from Exercises 9-14?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Footnotes\n",
+ "\n",
+ "1. The poorest households were identified in two steps. First, residents across 120 village hamlets ranked households into five wealth quintiles. Among households ranked in the bottom quintile, Bandhan then verified eligibility per seven criteria: (i) presence of an able-bodied female member (to manage the asset), (ii) no credit access, (iii) landholding below 0.2 acres, (iv) no ownership of productive assets, (v) no able-bodied male member, (vi) presence of school-aged children who were working instead of attending school, and (vii) primary source of income being informal labor or begging. Households had to meet the first two criteria and at least three of the remaining five in order to be eligible for the TUP intervention.\n",
+ "2. The [ethics of randomization in these types of programs](https://www.povertyactionlab.org/resource/ethical-conduct-randomized-evaluations) is subject of a rich literature, and there are currently an extensive set of guidelines used by researchers developing these types of programs.\n",
+ "3. Of the 514 offered the livestock and cash transfers, only 266 accepted. The treatment estimates that follow are thus estimates of the effect of *offering* these transfers. Households that accepted the offers might differ systematically from those that do not, so the study simply compares those *offered* the transfer to those who did not). This is what's called an estimate of the \"intention to treat\" effect, and is thus *likely* an under-estimate of the effect of cash transfers that would be observed if uptake were greater."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.11.8"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/docs/html/.doctrees/nbsphinx/exercises_exercise_power_calculations_18_0.png b/docs/html/.doctrees/nbsphinx/exercises_exercise_power_calculations_18_0.png
new file mode 100644
index 00000000..de08431a
Binary files /dev/null and b/docs/html/.doctrees/nbsphinx/exercises_exercise_power_calculations_18_0.png differ
diff --git a/docs/html/.doctrees/nbsphinx/exercises_exercise_power_calculations_18_1.png b/docs/html/.doctrees/nbsphinx/exercises_exercise_power_calculations_18_1.png
new file mode 100644
index 00000000..de08431a
Binary files /dev/null and b/docs/html/.doctrees/nbsphinx/exercises_exercise_power_calculations_18_1.png differ
diff --git a/docs/html/_images/exercises_exercise_power_calculations_18_0.png b/docs/html/_images/exercises_exercise_power_calculations_18_0.png
new file mode 100644
index 00000000..de08431a
Binary files /dev/null and b/docs/html/_images/exercises_exercise_power_calculations_18_0.png differ
diff --git a/docs/html/_images/exercises_exercise_power_calculations_18_1.png b/docs/html/_images/exercises_exercise_power_calculations_18_1.png
new file mode 100644
index 00000000..de08431a
Binary files /dev/null and b/docs/html/_images/exercises_exercise_power_calculations_18_1.png differ
diff --git a/docs/html/_sources/exercises/exercise_power_calculations.ipynb.txt b/docs/html/_sources/exercises/exercise_power_calculations.ipynb.txt
new file mode 100644
index 00000000..942a31f4
--- /dev/null
+++ b/docs/html/_sources/exercises/exercise_power_calculations.ipynb.txt
@@ -0,0 +1,271 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Power Calculations and Experiment Planning\n",
+ "\n",
+ "When you read about them in a book, power calculations can feel very abstract. To make the concept more concrete, and to give you a chance to familiarize yourself with power calculation tools and the considerations that go into power calculations, in this exercise you will play the role of an experiment planner hired by a Non-Governmental Organization (NGO) named [Bandhan](https://www.bandhan.org/) in West Bengal, India. Bandhan is planning to roll out a program that provides livestock, cash, and some basic training to households in extreme poverty. They hope to demonstrate the value of cash and asset transfers, and so wish their program to take the form of a randomized experiment, with data being collected on both control and treatment households.\n",
+ "\n",
+ "Your job will be to help them estimate how many households they should enroll in the program.\n",
+ "\n",
+ "This was, in fact, a real program, and so the data you will be working with comes from the actual program (conducted in 2007, with followup data collection in 2010, 2017, and 2020), allowing us to do some retrospective comparison of your power calculations and what actually occurred."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## The Context\n",
+ "\n",
+ "> Development economics has long posited that the poor may be poor for no good reason other than the fact that they started poor. This is the idea of a poverty trap, which has the implication that a one-time capital grant that makes very poor households significantly less poor (\"big push\") might set off a virtuous cycle that takes them out of poverty. Forty-three countries now embrace some version of this idea and make large transfers to over 3.1 million of their poorest households. In particular, the \"Targeting the Ultra Poor\" (TUP) intervention, pioneered by BRAC in Bangladesh, employs a multifaceted approach, offering poor households an asset transfer, consumption support, savings, and training in the hopes of unlocking a poverty trap.\n",
+ "\n",
+ "- [*Long-Term Effects of the Targeting the Ultra Poor Program*](https://economics.mit.edu/research/publications/long-term-effects-targeting-ultra-poor-program), Abhijit Banerjee, Esther Duflo, and Garima Sharma, 2021.\n",
+ "\n",
+ "In 2007, Bandhan in West Bengal, India created a pilot program to provide direct, unconditional transfers of productive livestock and a 30 or 40 week stipend of 90 rupees a week (about 7 US dollars a week using the [Purchasing Power Parity (PPP)](https://en.wikipedia.org/wiki/Purchasing_power_parity) exchange rate). The program targeted the poorest households in these villages based on range of criteria, and the average eligible household was later estimated to have a consumption level of about 1.35 2018 US Dollars per capita per day in PPP terms $.^1$\n",
+ "\n",
+ "Because the goal of the program was, in significant part, to demonstrate the effect of direct asset transfers, Bandhan identified twice as many eligible households as it could support in the pilot and randomly selected half of the households to act as controls and half to be treated $.^2$ Treated households were offered assets from a menu of options, from the most common choice was productive livestock (e.g., cows and goats). $^3$ Bandhan's contact with the households came to an end 18 months after the households were initially provided their livestock and cash transfers began.\n",
+ "\n",
+ "Although data was collected on a huge range of attributes of the households in this program, our focus will be on per capita household expenditures in 2018 US Dollars (PPP). \n",
+ "\n",
+ "(Note that these households certainly were not buying their food with dollars, and in most cases they weren't even buying most of their food with rupees — consumption estimates in this type of study are calculated by collecting detailed data on what household members have consumed in the past week, then estimating the price one would pay to buy those goods in local markets and using [PPP conversion rates](https://en.wikipedia.org/wiki/Purchasing_power_parity) conversion rates to convert that into US Dollars).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## Exercises\n",
+ "\n",
+ "### Exercise 1\n",
+ "\n",
+ "Load a thinned version of household-level survey data from [Banerjee, Duflo, and Sharma (2021)'s](https://economics.mit.edu/research/publications/long-term-effects-targeting-ultra-poor-program) evaluation of the Bandham program [here](https://github.com/nickeubank/MIDS_Data/tree/master/cash_transfers). \n",
+ "\n",
+ "This is just a version of the replication data for that paper ([which can be found here](https://www.openicpsr.org/openicpsr/project/130362)) with only the variables that are relevant for our analysis."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 2\n",
+ "\n",
+ "The first step in doing any power calculations is to establish an estimate of the baseline level of variation that exists in the dependent variable you wish to study. In this case, that variable is Per Capita Monthly Expenditures, and the baseline survey values of expenditures are in `pc_exp_month_bl` (the `_bl` stands for \"baseline\", meaning this data was collected prior to any households receiving asset transfers and indeed any households being assigned to treatment or control).\n",
+ "\n",
+ "What is the mean and standard deviation of `pc_exp_month_bl` in the baseline survey? Divide by 30 to confirm our average household has the expected per capita expenditure level of about US Dollar 1.35 (PPP) a day.\n",
+ "\n",
+ "You'll use these quantites a lot, so assign them to variables."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 3\n",
+ "\n",
+ "For these exercises, we'll use the `power` module in `statsmodels`. [You can read about it here.](https://www.statsmodels.org/stable/stats.html#power-and-sample-size-calculations)\n",
+ "\n",
+ "Since we're comparing means in a continuous variable (expenditures) from two samples of households, we will use `TTestIndPower` in `statsmodels.stats.power`. Import this class and instantiate a new instance (for some reason this is class based, so you have to start of with a command like `my_power = TTestIndPower()`). \n",
+ "\n",
+ "Note that a common situation in data science is testing a difference in *proportions*, as in situations where your dependent variable is binary and each group's mean is the share for whom the binary variable is 1. This comes up a lot with apps and websites — e.g., \"clicked an ad,\" \"subscribed,\" \"made a purchase.\" For that reason, there's actually a full sub-class of power calculating tools for [proportions you can read about here.](https://www.statsmodels.org/stable/stats.html#proportion) Basically, because the standard deviation of a binary variable is just $\\sqrt{p * (1-p)}$, power calculations become really simple."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 4\n",
+ "\n",
+ "[TTestIndPower has three methods](https://www.statsmodels.org/stable/generated/statsmodels.stats.power.TTestIndPower.html#statsmodels.stats.power.TTestIndPower) — `plot_power`, `power`, and `solve_power` — but `solve_power` does everything `power` does, so there are basically two methods.\n",
+ "\n",
+ "The idea of `solve_power` is that you give it all but one parameter of a power calculation — minimum detectable effect, number of observations, power, and p-value $\\alpha$ threshold — and it will solve for the omitted variable.\n",
+ "\n",
+ "To get a quick flavor for how this works, set: \n",
+ "\n",
+ "- Minimal Detectable Effect (`effect_size`) to `0.5` (we'll take about the units of that argument soon), \n",
+ "- `alpha` to `0.05`, \n",
+ "- `ratio` to `1` (this is the ratio of between the number of observations being treated and the number in control — in this study, the target was to have these be equal, as that provides the best statistical power for a given total number of observations).\n",
+ "- `power` to `0.8`\n",
+ "- `alternative` to `two-sided` (we want to do a two-tailed t-test, as you could imagine social unrest caused by transfers could make people worse off).\n",
+ "- `nobs1` to `None`. By passing `None`, we're telling `solve_power` this is the quantity we want it to solve for.\n",
+ "\n",
+ "So in other words, we're asking:\n",
+ "\n",
+ "- how many observations do we need to have in the treatment arm,\n",
+ "- assuming we'll have the same number also in the control arm,\n",
+ "- to have an 80% chance\n",
+ "- of rejecting the null hypothesis of no-effect at a significance threshold of 0.05\n",
+ "- if the true effect is of size 0.5?\n",
+ "\n",
+ "What's the answer?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 5\n",
+ "\n",
+ "Of all these quantities, perhaps the most obscure is the Minimal Detectable Effect (`effect_size`). What is this 0.5? Where did it come from? And how did you do all that without telling it that standard deviation you had me calculate?\n",
+ "\n",
+ "If you check the docs for `solve_power`, you'll see that you're supposed to pass `effect_size` the \"standardized effect size, difference between the two means divided by the standard deviation. `effect_size` has to be positive.\"\n",
+ "\n",
+ "In other words, when we passed 0.5, we weren't saying we wanted to be able to detect an effect of 0.5 2018 US Dollars (PPP) per capita per month, we were saying we wanted to detect an effect of 1/2 standard deviation. \n",
+ "\n",
+ "Given the standard deviation in our baseline survey was about 24 Dollars, that means we were saying we wanted to detect an effect size of about 12 US Dollars. \n",
+ "\n",
+ "Is that a lot? Well, the average per capita monthly expenditures in the baseline survey was about 41 Dollars, so it's an increase in expenditures of about 25%. \n",
+ "\n",
+ "OK, but is that what we should use?\n",
+ "\n",
+ "Well, the idea of a Minimal Detectable Effect is that you want to set it at a level where (a) if this *was* the real effect when you rejected the null, you'd be happy you did the experiment and, presumably, would be comfortable scaling up the treatment, but also (b) if the true effect were any smaller, you wouldn't want to reject the null hypothesis and scale up the treatment.\n",
+ "\n",
+ "Basically, if you set this too high, you may run an under-powered experiment where you miss out on learning that your treatment was effective; if you set this too low, you're spending time and money collecting more data than is probably necessary given you might reject the null at a point estimate where you still wouldn't scale up the treatment.\n",
+ "\n",
+ "So what value makes sense here? That's a question you'd want to start off by asking the stakeholders. Since Bandhan wants to use this to promote cash transfers, they probably need the effect to be relatively large to be convincing. Let's assume that, three years after the intervention, they feel they need to show that incomes have increased by at least 30%. \n",
+ "\n",
+ "What value does that imply should be passed to `effect_size`? What is the new implied number of treated households they'll need? How many households total?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 6\n",
+ "\n",
+ "Let's also suppose that because Bandhan really wants to sell their result, they also want to ensure no one claims it might have arisen by chance. What parameter would you adjust, and in what direction? Try modifying it and get a new observation requirement (can you predict what the change in num of observations will be before you do it?)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 7\n",
+ "\n",
+ "Now suppose that your stakeholder is budget constrained to only being able to enroll a total of 100 (50 in each arm). Assuming an alpha of `0.05` and a power of `0.8`, what is their minimal detectable effect size (in dollars)?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 8\n",
+ "\n",
+ "Suppose your stakeholder wants to see a few different scenarios to see how different experiment sizes would impact power given different effect sizes. Let's use `plot_power`. Pass it an array of treatment arm observation counts you want evaluated (I'd recommend `np.arange()`) between 20 and 100, along with effect sizes of 0.2, 0.4, and 0.6 (again, use `np.arange()`).\n",
+ "\n",
+ "If the stakeholder decided they wanted power above 90% — they're only gonna get a chance to do this once, after all! — what kind of sample size would they need with a minimal detectable effect size of 0.6? (Approximately).\n",
+ "\n",
+ "You may need to check the docs to figure out how to use it."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Extra Credit Extension"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 9\n",
+ "\n",
+ "As noted above, this study actually was conducted, so we know the actual treatment effect. Calculate that treatment effect 3 years after these families first received asset transfers by calculating the difference in `pc_exp_month_el2` between the `treatment == 1` and `treatment == 0` groups.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 10\n",
+ "\n",
+ "What was the statistical power of the actual test? That is, what was the probability, given this effect size and the number of observations in the study, that they would reject the null hypothesis of no effect at alpha=0.05 and given the actual final sample sizes?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 11\n",
+ "\n",
+ "What would the power be if Bandhan had only enrolled 200 households per arm? \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 13\n",
+ "\n",
+ "The idea of the \"power\" of a test is that, when we draw a sample from a population, there's some variation in who happens to end up in that sample. Consequently, even if the *true* effect size in the population is equal to our Minimal Detectable Effect, there's some probability that when we compare treated and untreated outcomes for the $N$ people who actually end up in the study, that Minimal Detectable Effect may not be evident in that sample (at the level of statistical significance $\\alpha$ we have chosen).\n",
+ "\n",
+ "Given that, one way to think of power is: \"If the true effect in the population is our Minimal Detectable Effect, then if I were able to re-run this experiment over and over — drawing new people into the study and testing the difference in outcomes between the control and treated sample each time — then in what percentage of those instances of the experiment would I reject the null hypothesis of no effect?\" Power of 80% means that we'd expect to reject the null of no effect in 80% of those many experiments.\n",
+ "\n",
+ "Well, we can basically do that thought experiment with this data! \n",
+ "\n",
+ "To do the experiment you did the power calculations for in Exercise 11, we just need to sample 200 observations from the treated group and 200 from the control group, treat that as our experimental sample, and estimate the difference in per capita month expenditures between those two groups (along with the associated p-value).\n",
+ "\n",
+ "Then we can repeat that over and over to simulate \"re-running\" the experiment, each time drawing a new sample of 200 treated observations and 200 control observations. Then we can store the p-values from all these \"re-run\" experiments and see how often we reject the null of no effect!\n",
+ "\n",
+ "Note that when you do this experiment, we have to sample our 200 observations from each treatment arm *with replacement*, just as you would when bootstrapping (if that means anything to you — if it doesn't, don't worry about it). \n",
+ "\n",
+ "So: write a loop where, on each pass, you draw 200 observations (with replacement) from treatment and 200 from control, then calculate the treatment effect and p-value for that sample. Repeat this 10,000 times. \n",
+ "\n",
+ "In what share of cases would you reject the null of no effect at alpha = 0.05?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 14\n",
+ "\n",
+ "Repeat 12 and 13 with 300 observations per arm."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 15\n",
+ "\n",
+ "What did you learn from Exercises 9-14?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Footnotes\n",
+ "\n",
+ "1. The poorest households were identified in two steps. First, residents across 120 village hamlets ranked households into five wealth quintiles. Among households ranked in the bottom quintile, Bandhan then verified eligibility per seven criteria: (i) presence of an able-bodied female member (to manage the asset), (ii) no credit access, (iii) landholding below 0.2 acres, (iv) no ownership of productive assets, (v) no able-bodied male member, (vi) presence of school-aged children who were working instead of attending school, and (vii) primary source of income being informal labor or begging. Households had to meet the first two criteria and at least three of the remaining five in order to be eligible for the TUP intervention.\n",
+ "2. The [ethics of randomization in these types of programs](https://www.povertyactionlab.org/resource/ethical-conduct-randomized-evaluations) is subject of a rich literature, and there are currently an extensive set of guidelines used by researchers developing these types of programs.\n",
+ "3. Of the 514 offered the livestock and cash transfers, only 266 accepted. The treatment estimates that follow are thus estimates of the effect of *offering* these transfers. Households that accepted the offers might differ systematically from those that do not, so the study simply compares those *offered* the transfer to those who did not). This is what's called an estimate of the \"intention to treat\" effect, and is thus *likely* an under-estimate of the effect of cash transfers that would be observed if uptake were greater."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.11.8"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/docs/html/_static/basic.css b/docs/html/_static/basic.css
index 8fef7354..617082b9 100644
--- a/docs/html/_static/basic.css
+++ b/docs/html/_static/basic.css
@@ -1 +1 @@
-@charset "utf-8";div.clearer{clear:both}div.section::after{display:block;content:'';clear:left}div.related{width:100%;font-size:90%}div.related h3{display:none}div.related ul{margin:0;padding:0 0 0 10px;list-style:none}div.related li{display:inline}div.related li.right{float:right;margin-right:5px}div.sphinxsidebarwrapper{padding:10px 5px 0 10px}div.sphinxsidebar{float:left;width:230px;margin-left:-100%;font-size:90%;word-wrap:break-word;overflow-wrap:break-word}div.sphinxsidebar ul{list-style:none}div.sphinxsidebar ul ul,div.sphinxsidebar ul.want-points{margin-left:20px;list-style:square}div.sphinxsidebar ul ul{margin-top:0;margin-bottom:0}div.sphinxsidebar form{margin-top:10px}div.sphinxsidebar input{border:1px solid #98dbcc;font-family:sans-serif;font-size:1em}div.sphinxsidebar #searchbox form.search{overflow:hidden}div.sphinxsidebar #searchbox input[type=text]{float:left;width:80%;padding:.25em;box-sizing:border-box}div.sphinxsidebar #searchbox input[type=submit]{float:left;width:20%;border-left:none;padding:.25em;box-sizing:border-box}img{border:0;max-width:100%}ul.search{margin:10px 0 0 20px;padding:0}ul.search li{padding:5px 0 5px 20px;background-image:url(file.png);background-repeat:no-repeat;background-position:0 7px}ul.search li a{font-weight:bold}ul.search li p.context{color:#888;margin:2px 0 0 30px;text-align:left}ul.keywordmatches li.goodmatch a{font-weight:bold}table.contentstable{width:90%;margin-left:auto;margin-right:auto}table.contentstable p.biglink{line-height:150%}a.biglink{font-size:1.3em}span.linkdescr{font-style:italic;padding-top:5px;font-size:90%}table.indextable{width:100%}table.indextable td{text-align:left;vertical-align:top}table.indextable ul{margin-top:0;margin-bottom:0;list-style-type:none}table.indextable>tbody>tr>td>ul{padding-left:0}table.indextable tr.pcap{height:10px}table.indextable tr.cap{margin-top:10px;background-color:#f2f2f2}img.toggler{margin-right:3px;margin-top:3px;cursor:pointer}div.modindex-jumpbox{border-top:1px solid #ddd;border-bottom:1px solid #ddd;margin:1em 0 1em 0;padding:.4em}div.genindex-jumpbox{border-top:1px solid #ddd;border-bottom:1px solid #ddd;margin:1em 0 1em 0;padding:.4em}table.modindextable td{padding:2px;border-collapse:collapse}div.body{min-width:360px;max-width:800px}div.body p,div.body dd,div.body li,div.body blockquote{-moz-hyphens:auto;-ms-hyphens:auto;-webkit-hyphens:auto;hyphens:auto}a.headerlink{visibility:hidden}a:visited{color:#551A8B}h1:hover>a.headerlink,h2:hover>a.headerlink,h3:hover>a.headerlink,h4:hover>a.headerlink,h5:hover>a.headerlink,h6:hover>a.headerlink,dt:hover>a.headerlink,caption:hover>a.headerlink,p.caption:hover>a.headerlink,div.code-block-caption:hover>a.headerlink{visibility:visible}div.body p.caption{text-align:inherit}div.body td{text-align:left}.first{margin-top:0 !important}p.rubric{margin-top:30px;font-weight:bold}img.align-left,figure.align-left,.figure.align-left,object.align-left{clear:left;float:left;margin-right:1em}img.align-right,figure.align-right,.figure.align-right,object.align-right{clear:right;float:right;margin-left:1em}img.align-center,figure.align-center,.figure.align-center,object.align-center{display:block;margin-left:auto;margin-right:auto}img.align-default,figure.align-default,.figure.align-default{display:block;margin-left:auto;margin-right:auto}.align-left{text-align:left}.align-center{text-align:center}.align-default{text-align:center}.align-right{text-align:right}div.sidebar,aside.sidebar{margin:0 0 .5em 1em;border:1px solid #ddb;padding:7px;background-color:#ffe;width:40%;float:right;clear:right;overflow-x:auto}p.sidebar-title{font-weight:bold}nav.contents,aside.topic,div.admonition,div.topic,blockquote{clear:left}nav.contents,aside.topic,div.topic{border:1px solid #ccc;padding:7px;margin:10px 0 10px 0}p.topic-title{font-size:1.1em;font-weight:700;margin-top:10px}div.admonition{margin-top:10px;margin-bottom:10px;padding:7px}div.admonition dt{font-weight:bold}p.admonition-title{margin:0 10px 5px 0;font-weight:bold}div.body p.centered{text-align:center;margin-top:25px}div.sidebar>:last-child,aside.sidebar>:last-child,nav.contents>:last-child,aside.topic>:last-child,div.topic>:last-child,div.admonition>:last-child{margin-bottom:0}div.sidebar::after,aside.sidebar::after,nav.contents::after,aside.topic::after,div.topic::after,div.admonition::after,blockquote::after{display:block;content:'';clear:both}table.docutils{margin-top:10px;margin-bottom:10px;border:0;border-collapse:collapse}table.align-center{margin-left:auto;margin-right:auto}table.align-default{margin-left:auto;margin-right:auto}table caption span.caption-number{font-style:italic}table caption span.caption-text{}table.docutils td,table.docutils th{padding:1px 8px 1px 5px;border-top:0;border-left:0;border-right:0;border-bottom:1px solid #aaa}th{text-align:left;padding-right:5px}table.citation{border-left:solid 1px gray;margin-left:1px}table.citation td{border-bottom:none}th>:first-child,td>:first-child{margin-top:0}th>:last-child,td>:last-child{margin-bottom:0}div.figure,figure{margin:.5em;padding:.5em}div.figure p.caption,figcaption{padding:.3em}div.figure p.caption span.caption-number,figcaption span.caption-number{font-style:italic}div.figure p.caption span.caption-text,figcaption span.caption-text{}table.field-list td,table.field-list th{border:0 !important}.field-list ul{margin:0;padding-left:1em}.field-list p{margin:0}.field-name{-moz-hyphens:manual;-ms-hyphens:manual;-webkit-hyphens:manual;hyphens:manual}table.hlist{margin:1em 0}table.hlist td{vertical-align:top}.sig{font-family:'Consolas','Menlo','DejaVu Sans Mono','Bitstream Vera Sans Mono',monospace}.sig-name,code.descname{background-color:transparent;font-weight:bold}.sig-name{font-size:1.1em}code.descname{font-size:1.2em}.sig-prename,code.descclassname{background-color:transparent}.optional{font-size:1.3em}.sig-paren{font-size:larger}.sig-param.n{font-style:italic}.sig-inline.c-texpr,.sig-inline.cpp-texpr{font-family:unset}.sig.c .k,.sig.c .kt,.sig.cpp .k,.sig.cpp .kt{color:#0033B3}.sig.c .m,.sig.cpp .m{color:#1750EB}.sig.c .s,.sig.c .sc,.sig.cpp .s,.sig.cpp .sc{color:#067D17}ol.arabic{list-style:decimal}ol.loweralpha{list-style:lower-alpha}ol.upperalpha{list-style:upper-alpha}ol.lowerroman{list-style:lower-roman}ol.upperroman{list-style:upper-roman}:not(li)>ol>li:first-child>:first-child,:not(li)>ul>li:first-child>:first-child{margin-top:0}:not(li)>ol>li:last-child>:last-child,:not(li)>ul>li:last-child>:last-child{margin-bottom:0}ol.simple ol p,ol.simple ul p,ul.simple ol p,ul.simple ul p{margin-top:0}ol.simple>li:not(:first-child)>p,ul.simple>li:not(:first-child)>p{margin-top:0}ol.simple p,ul.simple p{margin-bottom:0}aside.footnote>span,div.citation>span{float:left}aside.footnote>span:last-of-type,div.citation>span:last-of-type{padding-right:.5em}aside.footnote>p{margin-left:2em}div.citation>p{margin-left:4em}aside.footnote>p:last-of-type,div.citation>p:last-of-type{margin-bottom:0}aside.footnote>p:last-of-type:after,div.citation>p:last-of-type:after{content:"";clear:both}dl.field-list{display:grid;grid-template-columns:fit-content(30%) auto}dl.field-list>dt{font-weight:700;word-break:break-word;padding-left:.5em;padding-right:5px}dl.field-list>dd{padding-left:.5em;margin-top:0;margin-left:0;margin-bottom:0}dl{margin-bottom:15px}dd>:first-child{margin-top:0}dd ul,dd table{margin-bottom:10px}dd{margin-top:3px;margin-bottom:10px;margin-left:30px}.sig dd{margin-top:0;margin-bottom:0}.sig dl{margin-top:0;margin-bottom:0}dl>dd:last-child,dl>dd:last-child>:last-child{margin-bottom:0}dt:target,span.highlighted{background-color:#fbe54e}rect.highlighted{fill:#fbe54e}dl.glossary dt{font-weight:700;font-size:1.1em}.versionmodified{font-style:italic}.system-message{background-color:#fda;padding:5px;border:3px solid red}.footnote:target{background-color:#ffa}.line-block{display:block;margin-top:1em;margin-bottom:1em}.line-block .line-block{margin-top:0;margin-bottom:0;margin-left:1.5em}.guilabel,.menuselection{font-family:sans-serif}.accelerator{text-decoration:underline}.classifier{font-style:oblique}.classifier:before{font-style:normal;margin:0 .5em;content:":";display:inline-block}abbr,acronym{border-bottom:dotted 1px;cursor:help}.translated{background-color:rgba(207,255,207,0.2)}.untranslated{background-color:rgba(255,207,207,0.2)}pre{overflow:auto;overflow-y:hidden}pre,div[class*="highlight-"]{clear:both}span.pre{-moz-hyphens:none;-ms-hyphens:none;-webkit-hyphens:none;hyphens:none;white-space:nowrap}div[class*="highlight-"]{margin:1em 0}td.linenos pre{border:0;background-color:transparent;color:#aaa}table.highlighttable{display:block}table.highlighttable tbody{display:block}table.highlighttable tr{display:flex}table.highlighttable td{margin:0;padding:0}table.highlighttable td.linenos{padding-right:.5em}table.highlighttable td.code{flex:1;overflow:hidden}.highlight .hll{display:block}div.highlight pre,table.highlighttable pre{margin:0}div.code-block-caption + div{margin-top:0}div.code-block-caption{margin-top:1em;padding:2px 5px;font-size:small}div.code-block-caption code{background-color:transparent}table.highlighttable td.linenos,span.linenos,div.highlight span.gp{user-select:none;-webkit-user-select:text;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none}div.code-block-caption span.caption-number{padding:.1em .3em;font-style:italic}div.code-block-caption span.caption-text{}div.literal-block-wrapper{margin:1em 0}code.xref,a code{background-color:transparent;font-weight:bold}h1 code,h2 code,h3 code,h4 code,h5 code,h6 code{background-color:transparent}.viewcode-link{float:right}.viewcode-back{float:right;font-family:sans-serif}div.viewcode-block:target{margin:-1px -10px;padding:0 10px}img.math{vertical-align:middle}div.body div.math p{text-align:center}span.eqno{float:right}span.eqno a.headerlink{position:absolute;z-index:1}div.math:hover a.headerlink{visibility:visible}@media print{div.document,div.documentwrapper,div.bodywrapper{margin:0 !important;width:100%}div.sphinxsidebar,div.related,div.footer,#top-link{display:none}}
\ No newline at end of file
+@charset "utf-8"; div.clearer{clear:both}div.section::after{display:block;content:'';clear:left}div.related{width:100%;font-size:90%}div.related h3{display:none}div.related ul{margin:0;padding:0 0 0 10px;list-style:none}div.related li{display:inline}div.related li.right{float:right;margin-right:5px}div.sphinxsidebarwrapper{padding:10px 5px 0 10px}div.sphinxsidebar{float:left;width:230px;margin-left:-100%;font-size:90%;word-wrap:break-word;overflow-wrap:break-word}div.sphinxsidebar ul{list-style:none}div.sphinxsidebar ul ul,div.sphinxsidebar ul.want-points{margin-left:20px;list-style:square}div.sphinxsidebar ul ul{margin-top:0;margin-bottom:0}div.sphinxsidebar form{margin-top:10px}div.sphinxsidebar input{border:1px solid #98dbcc;font-family:sans-serif;font-size:1em}div.sphinxsidebar #searchbox form.search{overflow:hidden}div.sphinxsidebar #searchbox input[type=text]{float:left;width:80%;padding:.25em;box-sizing:border-box}div.sphinxsidebar #searchbox input[type=submit]{float:left;width:20%;border-left:none;padding:.25em;box-sizing:border-box}img{border:0;max-width:100%}ul.search{margin:10px 0 0 20px;padding:0}ul.search li{padding:5px 0 5px 20px;background-image:url(file.png);background-repeat:no-repeat;background-position:0 7px}ul.search li a{font-weight:bold}ul.search li p.context{color:#888;margin:2px 0 0 30px;text-align:left}ul.keywordmatches li.goodmatch a{font-weight:bold}table.contentstable{width:90%;margin-left:auto;margin-right:auto}table.contentstable p.biglink{line-height:150%}a.biglink{font-size:1.3em}span.linkdescr{font-style:italic;padding-top:5px;font-size:90%}table.indextable{width:100%}table.indextable td{text-align:left;vertical-align:top}table.indextable ul{margin-top:0;margin-bottom:0;list-style-type:none}table.indextable>tbody>tr>td>ul{padding-left:0}table.indextable tr.pcap{height:10px}table.indextable tr.cap{margin-top:10px;background-color:#f2f2f2}img.toggler{margin-right:3px;margin-top:3px;cursor:pointer}div.modindex-jumpbox{border-top:1px solid #ddd;border-bottom:1px solid #ddd;margin:1em 0 1em 0;padding:.4em}div.genindex-jumpbox{border-top:1px solid #ddd;border-bottom:1px solid #ddd;margin:1em 0 1em 0;padding:.4em}table.modindextable td{padding:2px;border-collapse:collapse}div.body{min-width:360px;max-width:800px}div.body p,div.body dd,div.body li,div.body blockquote{-moz-hyphens:auto;-ms-hyphens:auto;-webkit-hyphens:auto;hyphens:auto}a.headerlink{visibility:hidden}a:visited{color:#551A8B}h1:hover>a.headerlink,h2:hover>a.headerlink,h3:hover>a.headerlink,h4:hover>a.headerlink,h5:hover>a.headerlink,h6:hover>a.headerlink,dt:hover>a.headerlink,caption:hover>a.headerlink,p.caption:hover>a.headerlink,div.code-block-caption:hover>a.headerlink{visibility:visible}div.body p.caption{text-align:inherit}div.body td{text-align:left}.first{margin-top:0 !important}p.rubric{margin-top:30px;font-weight:bold}img.align-left,figure.align-left,.figure.align-left,object.align-left{clear:left;float:left;margin-right:1em}img.align-right,figure.align-right,.figure.align-right,object.align-right{clear:right;float:right;margin-left:1em}img.align-center,figure.align-center,.figure.align-center,object.align-center{display:block;margin-left:auto;margin-right:auto}img.align-default,figure.align-default,.figure.align-default{display:block;margin-left:auto;margin-right:auto}.align-left{text-align:left}.align-center{text-align:center}.align-default{text-align:center}.align-right{text-align:right}div.sidebar,aside.sidebar{margin:0 0 .5em 1em;border:1px solid #ddb;padding:7px;background-color:#ffe;width:40%;float:right;clear:right;overflow-x:auto}p.sidebar-title{font-weight:bold}nav.contents,aside.topic,div.admonition,div.topic,blockquote{clear:left}nav.contents,aside.topic,div.topic{border:1px solid #ccc;padding:7px;margin:10px 0 10px 0}p.topic-title{font-size:1.1em;font-weight:bold;margin-top:10px}div.admonition{margin-top:10px;margin-bottom:10px;padding:7px}div.admonition dt{font-weight:bold}p.admonition-title{margin:0 10px 5px 0;font-weight:bold}div.body p.centered{text-align:center;margin-top:25px}div.sidebar>:last-child,aside.sidebar>:last-child,nav.contents>:last-child,aside.topic>:last-child,div.topic>:last-child,div.admonition>:last-child{margin-bottom:0}div.sidebar::after,aside.sidebar::after,nav.contents::after,aside.topic::after,div.topic::after,div.admonition::after,blockquote::after{display:block;content:'';clear:both}table.docutils{margin-top:10px;margin-bottom:10px;border:0;border-collapse:collapse}table.align-center{margin-left:auto;margin-right:auto}table.align-default{margin-left:auto;margin-right:auto}table caption span.caption-number{font-style:italic}table caption span.caption-text{}table.docutils td,table.docutils th{padding:1px 8px 1px 5px;border-top:0;border-left:0;border-right:0;border-bottom:1px solid #aaa}th{text-align:left;padding-right:5px}table.citation{border-left:solid 1px gray;margin-left:1px}table.citation td{border-bottom:none}th>:first-child,td>:first-child{margin-top:0}th>:last-child,td>:last-child{margin-bottom:0}div.figure,figure{margin:.5em;padding:.5em}div.figure p.caption,figcaption{padding:.3em}div.figure p.caption span.caption-number,figcaption span.caption-number{font-style:italic}div.figure p.caption span.caption-text,figcaption span.caption-text{}table.field-list td,table.field-list th{border:0 !important}.field-list ul{margin:0;padding-left:1em}.field-list p{margin:0}.field-name{-moz-hyphens:manual;-ms-hyphens:manual;-webkit-hyphens:manual;hyphens:manual}table.hlist{margin:1em 0}table.hlist td{vertical-align:top}.sig{font-family:'Consolas','Menlo','DejaVu Sans Mono','Bitstream Vera Sans Mono',monospace}.sig-name,code.descname{background-color:transparent;font-weight:bold}.sig-name{font-size:1.1em}code.descname{font-size:1.2em}.sig-prename,code.descclassname{background-color:transparent}.optional{font-size:1.3em}.sig-paren{font-size:larger}.sig-param.n{font-style:italic}.sig-inline.c-texpr,.sig-inline.cpp-texpr{font-family:unset}.sig.c .k,.sig.c .kt,.sig.cpp .k,.sig.cpp .kt{color:#0033B3}.sig.c .m,.sig.cpp .m{color:#1750EB}.sig.c .s,.sig.c .sc,.sig.cpp .s,.sig.cpp .sc{color:#067D17}ol.arabic{list-style:decimal}ol.loweralpha{list-style:lower-alpha}ol.upperalpha{list-style:upper-alpha}ol.lowerroman{list-style:lower-roman}ol.upperroman{list-style:upper-roman}:not(li)>ol>li:first-child>:first-child,:not(li)>ul>li:first-child>:first-child{margin-top:0}:not(li)>ol>li:last-child>:last-child,:not(li)>ul>li:last-child>:last-child{margin-bottom:0}ol.simple ol p,ol.simple ul p,ul.simple ol p,ul.simple ul p{margin-top:0}ol.simple>li:not(:first-child)>p,ul.simple>li:not(:first-child)>p{margin-top:0}ol.simple p,ul.simple p{margin-bottom:0}aside.footnote>span,div.citation>span{float:left}aside.footnote>span:last-of-type,div.citation>span:last-of-type{padding-right:.5em}aside.footnote>p{margin-left:2em}div.citation>p{margin-left:4em}aside.footnote>p:last-of-type,div.citation>p:last-of-type{margin-bottom:0}aside.footnote>p:last-of-type:after,div.citation>p:last-of-type:after{content:"";clear:both}dl.field-list{display:grid;grid-template-columns:fit-content(30%) auto}dl.field-list>dt{font-weight:bold;word-break:break-word;padding-left:.5em;padding-right:5px}dl.field-list>dd{padding-left:.5em;margin-top:0;margin-left:0;margin-bottom:0}dl{margin-bottom:15px}dd>:first-child{margin-top:0}dd ul,dd table{margin-bottom:10px}dd{margin-top:3px;margin-bottom:10px;margin-left:30px}.sig dd{margin-top:0;margin-bottom:0}.sig dl{margin-top:0;margin-bottom:0}dl>dd:last-child,dl>dd:last-child>:last-child{margin-bottom:0}dt:target,span.highlighted{background-color:#fbe54e}rect.highlighted{fill:#fbe54e}dl.glossary dt{font-weight:bold;font-size:1.1em}.versionmodified{font-style:italic}.system-message{background-color:#fda;padding:5px;border:3px solid red}.footnote:target{background-color:#ffa}.line-block{display:block;margin-top:1em;margin-bottom:1em}.line-block .line-block{margin-top:0;margin-bottom:0;margin-left:1.5em}.guilabel,.menuselection{font-family:sans-serif}.accelerator{text-decoration:underline}.classifier{font-style:oblique}.classifier:before{font-style:normal;margin:0 .5em;content:":";display:inline-block}abbr,acronym{border-bottom:dotted 1px;cursor:help}.translated{background-color:rgba(207,255,207,0.2)}.untranslated{background-color:rgba(255,207,207,0.2)}pre{overflow:auto;overflow-y:hidden}pre,div[class*="highlight-"]{clear:both}span.pre{-moz-hyphens:none;-ms-hyphens:none;-webkit-hyphens:none;hyphens:none;white-space:nowrap}div[class*="highlight-"]{margin:1em 0}td.linenos pre{border:0;background-color:transparent;color:#aaa}table.highlighttable{display:block}table.highlighttable tbody{display:block}table.highlighttable tr{display:flex}table.highlighttable td{margin:0;padding:0}table.highlighttable td.linenos{padding-right:.5em}table.highlighttable td.code{flex:1;overflow:hidden}.highlight .hll{display:block}div.highlight pre,table.highlighttable pre{margin:0}div.code-block-caption + div{margin-top:0}div.code-block-caption{margin-top:1em;padding:2px 5px;font-size:small}div.code-block-caption code{background-color:transparent}table.highlighttable td.linenos,span.linenos,div.highlight span.gp{user-select:none;-webkit-user-select:text;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none}div.code-block-caption span.caption-number{padding:.1em .3em;font-style:italic}div.code-block-caption span.caption-text{}div.literal-block-wrapper{margin:1em 0}code.xref,a code{background-color:transparent;font-weight:bold}h1 code,h2 code,h3 code,h4 code,h5 code,h6 code{background-color:transparent}.viewcode-link{float:right}.viewcode-back{float:right;font-family:sans-serif}div.viewcode-block:target{margin:-1px -10px;padding:0 10px}img.math{vertical-align:middle}div.body div.math p{text-align:center}span.eqno{float:right}span.eqno a.headerlink{position:absolute;z-index:1}div.math:hover a.headerlink{visibility:visible}@media print{div.document,div.documentwrapper,div.bodywrapper{margin:0 !important;width:100%}div.sphinxsidebar,div.related,div.footer,#top-link{display:none}}
\ No newline at end of file
diff --git a/docs/html/_static/nbsphinx-code-cells.css b/docs/html/_static/nbsphinx-code-cells.css
index 15008205..d4bcdac4 100644
--- a/docs/html/_static/nbsphinx-code-cells.css
+++ b/docs/html/_static/nbsphinx-code-cells.css
@@ -1 +1 @@
-@charset "utf-8";div.nbinput.container div.prompt *,div.nboutput.container div.prompt *,div.nbinput.container div.input_area pre,div.nboutput.container div.output_area pre,div.nbinput.container div.input_area .highlight,div.nboutput.container div.output_area .highlight{border:0;padding:0;margin:0;box-shadow:none}div.nbinput.container>div[class*=highlight],div.nboutput.container>div[class*=highlight]{margin:0}div.nbinput.container div.prompt *,div.nboutput.container div.prompt *{background:none}div.nboutput.container div.output_area .highlight,div.nboutput.container div.output_area pre{background:unset}div.nboutput.container div.output_area div.highlight{color:unset}div.nboutput.container div[class*=highlight] pre{line-height:normal}div.nbinput.container,div.nboutput.container{display:-webkit-flex;display:flex;align-items:flex-start;margin:0;width:100%}@media (max-width:540px){div.nbinput.container,div.nboutput.container{flex-direction:column}}div.nbinput.container{padding-top:5px}div.nblast.container{padding-bottom:5px}div.nbinput.container div.prompt pre,div.nbinput.container div.prompt pre>code{color:#307FC1}div.nboutput.container div.prompt pre,div.nboutput.container div.prompt pre>code{color:#BF5B3D}div.nbinput.container div.prompt,div.nboutput.container div.prompt{width:4.5ex;padding-top:5px;position:relative;user-select:none}div.nbinput.container div.prompt>div,div.nboutput.container div.prompt>div{position:absolute;right:0;margin-right:.3ex}@media (max-width:540px){div.nbinput.container div.prompt,div.nboutput.container div.prompt{width:unset;text-align:left;padding:.4em}div.nboutput.container div.prompt.empty{padding:0}div.nbinput.container div.prompt>div,div.nboutput.container div.prompt>div{position:unset}}div.nbinput.container div.prompt pre,div.nboutput.container div.prompt pre{overflow:hidden;white-space:pre}div.nbinput.container div.input_area,div.nboutput.container div.output_area{-webkit-flex:1;flex:1;overflow:auto}@media (max-width:540px){div.nbinput.container div.input_area,div.nboutput.container div.output_area{width:100%}}div.nbinput.container div.input_area{border:1px solid #e0e0e0;border-radius:2px}div.nboutput.container div[class*=MathJax]{text-align:left !important}div.nboutput.container div.math p{text-align:left}div.nboutput.container div.output_area.stderr{background:#fdd}.ansi-black-fg{color:#3E424D}.ansi-black-bg{background-color:#3E424D}.ansi-black-intense-fg{color:#282C36}.ansi-black-intense-bg{background-color:#282C36}.ansi-red-fg{color:#E75C58}.ansi-red-bg{background-color:#E75C58}.ansi-red-intense-fg{color:#B22B31}.ansi-red-intense-bg{background-color:#B22B31}.ansi-green-fg{color:#00A250}.ansi-green-bg{background-color:#00A250}.ansi-green-intense-fg{color:#007427}.ansi-green-intense-bg{background-color:#007427}.ansi-yellow-fg{color:#DDB62B}.ansi-yellow-bg{background-color:#DDB62B}.ansi-yellow-intense-fg{color:#B27D12}.ansi-yellow-intense-bg{background-color:#B27D12}.ansi-blue-fg{color:#208FFB}.ansi-blue-bg{background-color:#208FFB}.ansi-blue-intense-fg{color:#0065CA}.ansi-blue-intense-bg{background-color:#0065CA}.ansi-magenta-fg{color:#D160C4}.ansi-magenta-bg{background-color:#D160C4}.ansi-magenta-intense-fg{color:#A03196}.ansi-magenta-intense-bg{background-color:#A03196}.ansi-cyan-fg{color:#60C6C8}.ansi-cyan-bg{background-color:#60C6C8}.ansi-cyan-intense-fg{color:#258F8F}.ansi-cyan-intense-bg{background-color:#258F8F}.ansi-white-fg{color:#C5C1B4}.ansi-white-bg{background-color:#C5C1B4}.ansi-white-intense-fg{color:#A1A6B2}.ansi-white-intense-bg{background-color:#A1A6B2}.ansi-default-inverse-fg{color:#FFF}.ansi-default-inverse-bg{background-color:#000}.ansi-bold{font-weight:bold}.ansi-underline{text-decoration:underline}div.nbinput.container div.input_area div[class*=highlight]>pre,div.nboutput.container div.output_area div[class*=highlight]>pre,div.nboutput.container div.output_area div[class*=highlight].math,div.nboutput.container div.output_area.rendered_html,div.nboutput.container div.output_area>div.output_javascript,div.nboutput.container div.output_area:not(.rendered_html)>img{padding:5px;margin:0}div.nbinput.container div.input_area>div[class^='highlight'],div.nboutput.container div.output_area>div[class^='highlight']{overflow-y:hidden}.prompt .copybtn,.prompt .md-clipboard.md-icon{display:none}.jp-RenderedHTMLCommon table,div.rendered_html table{border:0;border-collapse:collapse;border-spacing:0;color:black;font-size:12px;table-layout:fixed}.jp-RenderedHTMLCommon thead,div.rendered_html thead{border-bottom:1px solid black;vertical-align:bottom}.jp-RenderedHTMLCommon tr,.jp-RenderedHTMLCommon th,.jp-RenderedHTMLCommon td,div.rendered_html tr,div.rendered_html th,div.rendered_html td{text-align:right;vertical-align:middle;padding:.5em .5em;line-height:normal;white-space:normal;max-width:none;border:none}.jp-RenderedHTMLCommon th,div.rendered_html th{font-weight:bold}.jp-RenderedHTMLCommon tbody tr:nth-child(odd),div.rendered_html tbody tr:nth-child(odd){background:#f5f5f5}.jp-RenderedHTMLCommon tbody tr:hover,div.rendered_html tbody tr:hover{background:rgba(66,165,245,0.2)}
\ No newline at end of file
+@charset "utf-8"; div.nbinput.container div.prompt *,div.nboutput.container div.prompt *,div.nbinput.container div.input_area pre,div.nboutput.container div.output_area pre,div.nbinput.container div.input_area .highlight,div.nboutput.container div.output_area .highlight{border:0;padding:0;margin:0;box-shadow:none}div.nbinput.container>div[class*=highlight],div.nboutput.container>div[class*=highlight]{margin:0}div.nbinput.container div.prompt *,div.nboutput.container div.prompt *{background:none}div.nboutput.container div.output_area .highlight,div.nboutput.container div.output_area pre{background:unset}div.nboutput.container div.output_area div.highlight{color:unset}div.nboutput.container div[class*=highlight] pre{line-height:normal}div.nbinput.container,div.nboutput.container{display:-webkit-flex;display:flex;align-items:flex-start;margin:0;width:100%}@media (max-width:540px){div.nbinput.container,div.nboutput.container{flex-direction:column}}div.nbinput.container{padding-top:5px}div.nblast.container{padding-bottom:5px}div.nbinput.container div.prompt pre,div.nbinput.container div.prompt pre>code{color:#307FC1}div.nboutput.container div.prompt pre,div.nboutput.container div.prompt pre>code{color:#BF5B3D}div.nbinput.container div.prompt,div.nboutput.container div.prompt{width:4.5ex;padding-top:5px;position:relative;user-select:none}div.nbinput.container div.prompt>div,div.nboutput.container div.prompt>div{position:absolute;right:0;margin-right:.3ex}@media (max-width:540px){div.nbinput.container div.prompt,div.nboutput.container div.prompt{width:unset;text-align:left;padding:.4em}div.nboutput.container div.prompt.empty{padding:0}div.nbinput.container div.prompt>div,div.nboutput.container div.prompt>div{position:unset}}div.nbinput.container div.prompt pre,div.nboutput.container div.prompt pre{overflow:hidden;white-space:pre}div.nbinput.container div.input_area,div.nboutput.container div.output_area{-webkit-flex:1;flex:1;overflow:auto}@media (max-width:540px){div.nbinput.container div.input_area,div.nboutput.container div.output_area{width:100%}}div.nbinput.container div.input_area{border:1px solid #e0e0e0;border-radius:2px}div.nboutput.container div[class*=MathJax]{text-align:left !important}div.nboutput.container div.math p{text-align:left}div.nboutput.container div.output_area.stderr{background:#fdd}.ansi-black-fg{color:#3E424D}.ansi-black-bg{background-color:#3E424D}.ansi-black-intense-fg{color:#282C36}.ansi-black-intense-bg{background-color:#282C36}.ansi-red-fg{color:#E75C58}.ansi-red-bg{background-color:#E75C58}.ansi-red-intense-fg{color:#B22B31}.ansi-red-intense-bg{background-color:#B22B31}.ansi-green-fg{color:#00A250}.ansi-green-bg{background-color:#00A250}.ansi-green-intense-fg{color:#007427}.ansi-green-intense-bg{background-color:#007427}.ansi-yellow-fg{color:#DDB62B}.ansi-yellow-bg{background-color:#DDB62B}.ansi-yellow-intense-fg{color:#B27D12}.ansi-yellow-intense-bg{background-color:#B27D12}.ansi-blue-fg{color:#208FFB}.ansi-blue-bg{background-color:#208FFB}.ansi-blue-intense-fg{color:#0065CA}.ansi-blue-intense-bg{background-color:#0065CA}.ansi-magenta-fg{color:#D160C4}.ansi-magenta-bg{background-color:#D160C4}.ansi-magenta-intense-fg{color:#A03196}.ansi-magenta-intense-bg{background-color:#A03196}.ansi-cyan-fg{color:#60C6C8}.ansi-cyan-bg{background-color:#60C6C8}.ansi-cyan-intense-fg{color:#258F8F}.ansi-cyan-intense-bg{background-color:#258F8F}.ansi-white-fg{color:#C5C1B4}.ansi-white-bg{background-color:#C5C1B4}.ansi-white-intense-fg{color:#A1A6B2}.ansi-white-intense-bg{background-color:#A1A6B2}.ansi-default-inverse-fg{color:#FFF}.ansi-default-inverse-bg{background-color:#000}.ansi-bold{font-weight:bold}.ansi-underline{text-decoration:underline}div.nbinput.container div.input_area div[class*=highlight]>pre,div.nboutput.container div.output_area div[class*=highlight]>pre,div.nboutput.container div.output_area div[class*=highlight].math,div.nboutput.container div.output_area.rendered_html,div.nboutput.container div.output_area>div.output_javascript,div.nboutput.container div.output_area:not(.rendered_html)>img{padding:5px;margin:0}div.nbinput.container div.input_area>div[class^='highlight'],div.nboutput.container div.output_area>div[class^='highlight']{overflow-y:hidden}.prompt .copybtn,.prompt .md-clipboard.md-icon{display:none}.jp-RenderedHTMLCommon table,div.rendered_html table{border:0;border-collapse:collapse;border-spacing:0;color:black;font-size:12px;table-layout:fixed}.jp-RenderedHTMLCommon thead,div.rendered_html thead{border-bottom:1px solid black;vertical-align:bottom}.jp-RenderedHTMLCommon tr,.jp-RenderedHTMLCommon th,.jp-RenderedHTMLCommon td,div.rendered_html tr,div.rendered_html th,div.rendered_html td{text-align:right;vertical-align:middle;padding:.5em .5em;line-height:normal;white-space:normal;max-width:none;border:none}.jp-RenderedHTMLCommon th,div.rendered_html th{font-weight:bold}.jp-RenderedHTMLCommon tbody tr:nth-child(odd),div.rendered_html tbody tr:nth-child(odd){background:#f5f5f5}.jp-RenderedHTMLCommon tbody tr:hover,div.rendered_html tbody tr:hover{background:rgba(66,165,245,0.2)}
\ No newline at end of file
diff --git a/docs/html/_static/pygments.css b/docs/html/_static/pygments.css
index 6f8505d9..8dadad3f 100644
--- a/docs/html/_static/pygments.css
+++ b/docs/html/_static/pygments.css
@@ -1 +1 @@
-@charset "utf-8";pre{line-height:125%}td.linenos .normal{color:#666;background-color:transparent;padding-left:5px;padding-right:5px}span.linenos{color:#666;background-color:transparent;padding-left:5px;padding-right:5px}td.linenos .special{color:#000;background-color:#ffffc0;padding-left:5px;padding-right:5px}span.linenos.special{color:#000;background-color:#ffffc0;padding-left:5px;padding-right:5px}.highlight .hll{background-color:#ffc}.highlight{background:#f0f0f0}.highlight .c{color:#60a0b0;font-style:italic}.highlight .err{border:1px solid #F00}.highlight .k{color:#007020;font-weight:bold}.highlight .o{color:#666}.highlight .ch{color:#60a0b0;font-style:italic}.highlight .cm{color:#60a0b0;font-style:italic}.highlight .cp{color:#007020}.highlight .cpf{color:#60a0b0;font-style:italic}.highlight .c1{color:#60a0b0;font-style:italic}.highlight .cs{color:#60a0b0;background-color:#fff0f0}.highlight .gd{color:#A00000}.highlight .ge{font-style:italic}.highlight .ges{font-weight:700;font-style:italic}.highlight .gr{color:#F00}.highlight .gh{color:#000080;font-weight:bold}.highlight .gi{color:#00A000}.highlight .go{color:#888}.highlight .gp{color:#c65d09;font-weight:bold}.highlight .gs{font-weight:bold}.highlight .gu{color:#800080;font-weight:bold}.highlight .gt{color:#04D}.highlight .kc{color:#007020;font-weight:bold}.highlight .kd{color:#007020;font-weight:bold}.highlight .kn{color:#007020;font-weight:bold}.highlight .kp{color:#007020}.highlight .kr{color:#007020;font-weight:bold}.highlight .kt{color:#902000}.highlight .m{color:#40a070}.highlight .s{color:#4070a0}.highlight .na{color:#4070a0}.highlight .nb{color:#007020}.highlight .nc{color:#0e84b5;font-weight:bold}.highlight .no{color:#60add5}.highlight .nd{color:#555;font-weight:bold}.highlight .ni{color:#d55537;font-weight:bold}.highlight .ne{color:#007020}.highlight .nf{color:#06287e}.highlight .nl{color:#002070;font-weight:bold}.highlight .nn{color:#0e84b5;font-weight:bold}.highlight .nt{color:#062873;font-weight:bold}.highlight .nv{color:#bb60d5}.highlight .ow{color:#007020;font-weight:bold}.highlight .w{color:#bbb}.highlight .mb{color:#40a070}.highlight .mf{color:#40a070}.highlight .mh{color:#40a070}.highlight .mi{color:#40a070}.highlight .mo{color:#40a070}.highlight .sa{color:#4070a0}.highlight .sb{color:#4070a0}.highlight .sc{color:#4070a0}.highlight .dl{color:#4070a0}.highlight .sd{color:#4070a0;font-style:italic}.highlight .s2{color:#4070a0}.highlight .se{color:#4070a0;font-weight:bold}.highlight .sh{color:#4070a0}.highlight .si{color:#70a0d0;font-style:italic}.highlight .sx{color:#c65d09}.highlight .sr{color:#235388}.highlight .s1{color:#4070a0}.highlight .ss{color:#517918}.highlight .bp{color:#007020}.highlight .fm{color:#06287e}.highlight .vc{color:#bb60d5}.highlight .vg{color:#bb60d5}.highlight .vi{color:#bb60d5}.highlight .vm{color:#bb60d5}.highlight .il{color:#40a070}
\ No newline at end of file
+@charset "utf-8";pre{line-height:125%}td.linenos .normal{color:#666;background-color:transparent;padding-left:5px;padding-right:5px}span.linenos{color:#666;background-color:transparent;padding-left:5px;padding-right:5px}td.linenos .special{color:#000;background-color:#ffffc0;padding-left:5px;padding-right:5px}span.linenos.special{color:#000;background-color:#ffffc0;padding-left:5px;padding-right:5px}.highlight .hll{background-color:#ffc}.highlight{background:#f0f0f0}.highlight .c{color:#60a0b0;font-style:italic}.highlight .err{border:1px solid #F00}.highlight .k{color:#007020;font-weight:bold}.highlight .o{color:#666}.highlight .ch{color:#60a0b0;font-style:italic}.highlight .cm{color:#60a0b0;font-style:italic}.highlight .cp{color:#007020}.highlight .cpf{color:#60a0b0;font-style:italic}.highlight .c1{color:#60a0b0;font-style:italic}.highlight .cs{color:#60a0b0;background-color:#fff0f0}.highlight .gd{color:#A00000}.highlight .ge{font-style:italic}.highlight .ges{font-weight:bold;font-style:italic}.highlight .gr{color:#F00}.highlight .gh{color:#000080;font-weight:bold}.highlight .gi{color:#00A000}.highlight .go{color:#888}.highlight .gp{color:#c65d09;font-weight:bold}.highlight .gs{font-weight:bold}.highlight .gu{color:#800080;font-weight:bold}.highlight .gt{color:#04D}.highlight .kc{color:#007020;font-weight:bold}.highlight .kd{color:#007020;font-weight:bold}.highlight .kn{color:#007020;font-weight:bold}.highlight .kp{color:#007020}.highlight .kr{color:#007020;font-weight:bold}.highlight .kt{color:#902000}.highlight .m{color:#40a070}.highlight .s{color:#4070a0}.highlight .na{color:#4070a0}.highlight .nb{color:#007020}.highlight .nc{color:#0e84b5;font-weight:bold}.highlight .no{color:#60add5}.highlight .nd{color:#555;font-weight:bold}.highlight .ni{color:#d55537;font-weight:bold}.highlight .ne{color:#007020}.highlight .nf{color:#06287e}.highlight .nl{color:#002070;font-weight:bold}.highlight .nn{color:#0e84b5;font-weight:bold}.highlight .nt{color:#062873;font-weight:bold}.highlight .nv{color:#bb60d5}.highlight .ow{color:#007020;font-weight:bold}.highlight .w{color:#bbb}.highlight .mb{color:#40a070}.highlight .mf{color:#40a070}.highlight .mh{color:#40a070}.highlight .mi{color:#40a070}.highlight .mo{color:#40a070}.highlight .sa{color:#4070a0}.highlight .sb{color:#4070a0}.highlight .sc{color:#4070a0}.highlight .dl{color:#4070a0}.highlight .sd{color:#4070a0;font-style:italic}.highlight .s2{color:#4070a0}.highlight .se{color:#4070a0;font-weight:bold}.highlight .sh{color:#4070a0}.highlight .si{color:#70a0d0;font-style:italic}.highlight .sx{color:#c65d09}.highlight .sr{color:#235388}.highlight .s1{color:#4070a0}.highlight .ss{color:#517918}.highlight .bp{color:#007020}.highlight .fm{color:#06287e}.highlight .vc{color:#bb60d5}.highlight .vg{color:#bb60d5}.highlight .vi{color:#bb60d5}.highlight .vm{color:#bb60d5}.highlight .il{color:#40a070}
\ No newline at end of file
diff --git a/docs/html/_static/stylesheets/application-fixes.css b/docs/html/_static/stylesheets/application-fixes.css
index 4176634c..bbe6ad0e 100644
--- a/docs/html/_static/stylesheets/application-fixes.css
+++ b/docs/html/_static/stylesheets/application-fixes.css
@@ -1 +1 @@
-@charset "utf-8";.md-nav--primary ul,.md-nav--primary ul li ul,.md-nav--secondary ul,.md-nav--secondary ul li ul{margin:0;padding:0;list-style:none}.md-nav--primary ul li,.md-nav--primary ul li ul li,.md-nav--secondary ul li,.md-nav--secondary ul li ul li{padding:0 .6rem}.md-nav--primary a.reference,.md-nav--secondary a.reference{display:block;margin-top:.625em;transition:color .125s;text-overflow:ellipsis;cursor:pointer;overflow:hidden}.md-typeset td p,.md-typeset th p{margin:0}.md-typeset .admonition,.md-typeset details{font-size:.8rem}.classifier:before{font-style:normal;margin:.5em;content:":"}dl.footnote>dt,dl.citation>dt{float:left}code.xref{background-color:transparent;font-weight:bold}table.docutils{width:100%}.longtable tr td:first-child{width:50%;white-space:nowrap}dt:target{margin-top:-3.55rem;padding-top:3.45rem}.md-typeset code{margin:0}ul.search li div.context{color:#888;margin:2px 0 0 30px;text-align:left}span.highlighted{background-color:#fbe54e}p.rubric{margin-top:1rem;font-weight:bold}dl.field-list>dt{font-weight:700;word-break:break-word;padding-left:.5em;padding-right:5px}table.longtable{border-collapse:collapse}.longtable tr{border:solid;border-width:1px 0}.longtable tr:first-child{border-top:none}.md-tabs code,kbd,pre,.md-footer-nav code,kbd,pre{color:#fff}.toctree-wrapper.compound li{list-style:none;padding:0;margin:0 0 .1rem .2rem}table.indextable.genindextable li{margin:0 .5rem}table.indextable.genindextable li ul li{list-style:none}p.highlight-link{margin:.625rem 0 0 0}table.longtable.docutils.align-default{width:100%;overflow-x:auto;-webkit-overflow-scrolling:touch}@media only screen and (max-width:40em){table.longtable.docutils.align-default{display:block}.longtable.docutils.align-default td{padding-right:1rem}}.md-nav__extra_link:after{font-family:Material Icons;font-style:normal;font-variant:normal;font-weight:400;line-height:1;text-transform:none;white-space:nowrap;speak:none;word-wrap:normal;direction:ltr}.md-nav__extra_link{display:block;margin-top:.625em;transition:color .125s;text-overflow:ellipsis;cursor:pointer;overflow:hidden}.md-nav__extra_link:active{color:#3f51b5}.md-nav__extra_link:focus,.md-nav__extra_link:hover{color:#536dfe}@media only screen and (max-width:76.1875em){.md-nav--primary .md-nav--secondary .md-nav__extra_link{position:static}.md-nav--primary .md-nav--secondary .md-nav .md-nav__extra_link{padding-left:1.4rem}.md-nav--primary .md-nav--secondary .md-nav .md-nav .md-nav .md-nav__extra_link{padding-left:2.6rem}}[data-md-color-primary=red] .md-nav__extra_link:active{color:#ef5350}[data-md-color-primary=red] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=pink] .md-nav__extra_link:active{color:#e91e63}[data-md-color-primary=pink] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=purple] .md-nav__extra_link:active{color:#ab47bc}[data-md-color-primary=purple] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=deep-purple] .md-nav__extra_link:active{color:#7e57c2}[data-md-color-primary=deep-purple] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=indigo] .md-nav__extra_link:active{color:#3f51b5}[data-md-color-primary=indigo] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=blue] .md-nav__extra_link:active{color:#2196f3}[data-md-color-primary=blue] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=light-blue] .md-nav__extra_link:active{color:#03a9f4}[data-md-color-primary=light-blue] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=cyan] .md-nav__extra_link:active{color:#00bcd4}[data-md-color-primary=cyan] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=teal] .md-nav__extra_link:active{color:#009688}[data-md-color-primary=teal] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=green] .md-nav__extra_link:active{color:#4caf50}[data-md-color-primary=green] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=light-green] .md-nav__extra_link:active{color:#7cb342}[data-md-color-primary=light-green] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=lime] .md-nav__extra_link:active{color:#c0ca33}[data-md-color-primary=lime] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=yellow] .md-nav__extra_link:active{color:#f9a825}[data-md-color-primary=yellow] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=amber] .md-nav__extra_link:active{color:#ffa000}[data-md-color-primary=amber] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=orange] .md-nav__extra_link:active{color:#fb8c00}[data-md-color-primary=orange] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=deep-orange] .md-nav__extra_link:active{color:#ff7043}[data-md-color-primary=deep-orange] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=brown] .md-nav__extra_link:active{color:#795548}[data-md-color-primary=brown] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=grey] .md-nav__extra_link:active{color:#757575}[data-md-color-primary=grey] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=blue-grey] .md-nav__extra_link:active{color:#546e7a}[data-md-color-primary=blue-grey] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-accent=red] .md-nav__extra_link:focus,[data-md-color-accent=red] .md-nav__extra_link:hover{color:#ff1744}[data-md-color-accent=pink] .md-nav__extra_link:focus,[data-md-color-accent=pink] .md-nav__extra_link:hover{color:#f50057}[data-md-color-accent=purple] .md-nav__extra_link:focus,[data-md-color-accent=purple] .md-nav__extra_link:hover{color:#e040fb}[data-md-color-accent=deep-purple] .md-nav__extra_link:focus,[data-md-color-accent=deep-purple] .md-nav__extra_link:hover{color:#7c4dff}[data-md-color-accent=indigo] .md-nav__extra_link:focus,[data-md-color-accent=indigo] .md-nav__extra_link:hover{color:#536dfe}[data-md-color-accent=blue] .md-nav__extra_link:focus,[data-md-color-accent=blue] .md-nav__extra_link:hover{color:#448aff}[data-md-color-accent=light-blue] .md-nav__extra_link:focus,[data-md-color-accent=light-blue] .md-nav__extra_link:hover{color:#0091ea}[data-md-color-accent=cyan] .md-nav__extra_link:focus,[data-md-color-accent=cyan] .md-nav__extra_link:hover{color:#00b8d4}[data-md-color-accent=teal] .md-nav__extra_link:focus,[data-md-color-accent=teal] .md-nav__extra_link:hover{color:#00bfa5}[data-md-color-accent=green] .md-nav__extra_link:focus,[data-md-color-accent=green] .md-nav__extra_link:hover{color:#00c853}[data-md-color-accent=light-green] .md-nav__extra_link:focus,[data-md-color-accent=light-green] .md-nav__extra_link:hover{color:#64dd17}[data-md-color-accent=lime] .md-nav__extra_link:focus,[data-md-color-accent=lime] .md-nav__extra_link:hover{color:#aeea00}[data-md-color-accent=yellow] .md-nav__extra_link:focus,[data-md-color-accent=yellow] .md-nav__extra_link:hover{color:#ffd600}[data-md-color-accent=amber] .md-nav__extra_link:focus,[data-md-color-accent=amber] .md-nav__extra_link:hover{color:#ffab00}[data-md-color-accent=orange] .md-nav__extra_link:focus,[data-md-color-accent=orange] .md-nav__extra_link:hover{color:#ff9100}[data-md-color-accent=deep-orange] .md-nav__extra_link:focus,[data-md-color-accent=deep-orange] .md-nav__extra_link:hover{color:#ff6e40}div.rendered_html table{font-size:.8rem !important}.md-nav span.caption{font-weight:700;pointer-events:none}.md-nav span.caption:hover,.md-nav span.caption:active{color:#000}.md-typeset img.align-right{clear:right;float:right;margin-left:1em}.md-typeset img.align-center{display:block;margin-left:auto;margin-right:auto}dl.citation dt span.brackets{margin-right:.3rem}
\ No newline at end of file
+@charset "utf-8";.md-nav--primary ul,.md-nav--primary ul li ul,.md-nav--secondary ul,.md-nav--secondary ul li ul{margin:0;padding:0;list-style:none}.md-nav--primary ul li,.md-nav--primary ul li ul li,.md-nav--secondary ul li,.md-nav--secondary ul li ul li{padding:0 .6rem}.md-nav--primary a.reference,.md-nav--secondary a.reference{display:block;margin-top:.625em;transition:color .125s;text-overflow:ellipsis;cursor:pointer;overflow:hidden}.md-typeset td p,.md-typeset th p{margin:0}.md-typeset .admonition,.md-typeset details{font-size:.8rem}.classifier:before{font-style:normal;margin:.5em;content:":"}dl.footnote>dt,dl.citation>dt{float:left}code.xref{background-color:transparent;font-weight:bold}table.docutils{width:100%}.longtable tr td:first-child{width:50%;white-space:nowrap}dt:target{margin-top:-3.55rem;padding-top:3.45rem}.md-typeset code{margin:0}ul.search li div.context{color:#888;margin:2px 0 0 30px;text-align:left}span.highlighted{background-color:#fbe54e}p.rubric{margin-top:1rem;font-weight:bold}dl.field-list>dt{font-weight:bold;word-break:break-word;padding-left:.5em;padding-right:5px}table.longtable{border-collapse:collapse}.longtable tr{border:solid;border-width:1px 0}.longtable tr:first-child{border-top:none}.md-tabs code,kbd,pre,.md-footer-nav code,kbd,pre{color:#fff}.toctree-wrapper.compound li{list-style:none;padding:0;margin:0 0 .1rem .2rem}table.indextable.genindextable li{margin:0 .5rem}table.indextable.genindextable li ul li{list-style:none}p.highlight-link{margin:.625rem 0 0 0}table.longtable.docutils.align-default{width:100%;overflow-x:auto;-webkit-overflow-scrolling:touch}@media only screen and (max-width:40em){table.longtable.docutils.align-default{display:block}.longtable.docutils.align-default td{padding-right:1rem}}.md-nav__extra_link:after{font-family:Material Icons;font-style:normal;font-variant:normal;font-weight:400;line-height:1;text-transform:none;white-space:nowrap;speak:none;word-wrap:normal;direction:ltr}.md-nav__extra_link{display:block;margin-top:.625em;transition:color .125s;text-overflow:ellipsis;cursor:pointer;overflow:hidden}.md-nav__extra_link:active{color:#3f51b5}.md-nav__extra_link:focus,.md-nav__extra_link:hover{color:#536dfe}@media only screen and (max-width:76.1875em){.md-nav--primary .md-nav--secondary .md-nav__extra_link{position:static}.md-nav--primary .md-nav--secondary .md-nav .md-nav__extra_link{padding-left:1.4rem}.md-nav--primary .md-nav--secondary .md-nav .md-nav .md-nav .md-nav__extra_link{padding-left:2.6rem}}[data-md-color-primary=red] .md-nav__extra_link:active{color:#ef5350}[data-md-color-primary=red] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=pink] .md-nav__extra_link:active{color:#e91e63}[data-md-color-primary=pink] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=purple] .md-nav__extra_link:active{color:#ab47bc}[data-md-color-primary=purple] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=deep-purple] .md-nav__extra_link:active{color:#7e57c2}[data-md-color-primary=deep-purple] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=indigo] .md-nav__extra_link:active{color:#3f51b5}[data-md-color-primary=indigo] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=blue] .md-nav__extra_link:active{color:#2196f3}[data-md-color-primary=blue] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=light-blue] .md-nav__extra_link:active{color:#03a9f4}[data-md-color-primary=light-blue] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=cyan] .md-nav__extra_link:active{color:#00bcd4}[data-md-color-primary=cyan] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=teal] .md-nav__extra_link:active{color:#009688}[data-md-color-primary=teal] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=green] .md-nav__extra_link:active{color:#4caf50}[data-md-color-primary=green] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=light-green] .md-nav__extra_link:active{color:#7cb342}[data-md-color-primary=light-green] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=lime] .md-nav__extra_link:active{color:#c0ca33}[data-md-color-primary=lime] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=yellow] .md-nav__extra_link:active{color:#f9a825}[data-md-color-primary=yellow] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=amber] .md-nav__extra_link:active{color:#ffa000}[data-md-color-primary=amber] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=orange] .md-nav__extra_link:active{color:#fb8c00}[data-md-color-primary=orange] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=deep-orange] .md-nav__extra_link:active{color:#ff7043}[data-md-color-primary=deep-orange] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=brown] .md-nav__extra_link:active{color:#795548}[data-md-color-primary=brown] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=grey] .md-nav__extra_link:active{color:#757575}[data-md-color-primary=grey] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-primary=blue-grey] .md-nav__extra_link:active{color:#546e7a}[data-md-color-primary=blue-grey] .md-nav__item--nested>.md-nav__extra_link{color:inherit}[data-md-color-accent=red] .md-nav__extra_link:focus,[data-md-color-accent=red] .md-nav__extra_link:hover{color:#ff1744}[data-md-color-accent=pink] .md-nav__extra_link:focus,[data-md-color-accent=pink] .md-nav__extra_link:hover{color:#f50057}[data-md-color-accent=purple] .md-nav__extra_link:focus,[data-md-color-accent=purple] .md-nav__extra_link:hover{color:#e040fb}[data-md-color-accent=deep-purple] .md-nav__extra_link:focus,[data-md-color-accent=deep-purple] .md-nav__extra_link:hover{color:#7c4dff}[data-md-color-accent=indigo] .md-nav__extra_link:focus,[data-md-color-accent=indigo] .md-nav__extra_link:hover{color:#536dfe}[data-md-color-accent=blue] .md-nav__extra_link:focus,[data-md-color-accent=blue] .md-nav__extra_link:hover{color:#448aff}[data-md-color-accent=light-blue] .md-nav__extra_link:focus,[data-md-color-accent=light-blue] .md-nav__extra_link:hover{color:#0091ea}[data-md-color-accent=cyan] .md-nav__extra_link:focus,[data-md-color-accent=cyan] .md-nav__extra_link:hover{color:#00b8d4}[data-md-color-accent=teal] .md-nav__extra_link:focus,[data-md-color-accent=teal] .md-nav__extra_link:hover{color:#00bfa5}[data-md-color-accent=green] .md-nav__extra_link:focus,[data-md-color-accent=green] .md-nav__extra_link:hover{color:#00c853}[data-md-color-accent=light-green] .md-nav__extra_link:focus,[data-md-color-accent=light-green] .md-nav__extra_link:hover{color:#64dd17}[data-md-color-accent=lime] .md-nav__extra_link:focus,[data-md-color-accent=lime] .md-nav__extra_link:hover{color:#aeea00}[data-md-color-accent=yellow] .md-nav__extra_link:focus,[data-md-color-accent=yellow] .md-nav__extra_link:hover{color:#ffd600}[data-md-color-accent=amber] .md-nav__extra_link:focus,[data-md-color-accent=amber] .md-nav__extra_link:hover{color:#ffab00}[data-md-color-accent=orange] .md-nav__extra_link:focus,[data-md-color-accent=orange] .md-nav__extra_link:hover{color:#ff9100}[data-md-color-accent=deep-orange] .md-nav__extra_link:focus,[data-md-color-accent=deep-orange] .md-nav__extra_link:hover{color:#ff6e40}div.rendered_html table{font-size:.8rem !important}.md-nav span.caption{font-weight:700;pointer-events:none}.md-nav span.caption:hover,.md-nav span.caption:active{color:#000}.md-typeset img.align-right{clear:right;float:right;margin-left:1em}.md-typeset img.align-center{display:block;margin-left:auto;margin-right:auto}dl.citation dt span.brackets{margin-right:.3rem}
\ No newline at end of file
diff --git a/docs/html/exercises/exercise_power_calculations.html b/docs/html/exercises/exercise_power_calculations.html
new file mode 100644
index 00000000..7ad2e647
--- /dev/null
+++ b/docs/html/exercises/exercise_power_calculations.html
@@ -0,0 +1 @@
+
Power Calculations and Experiment Planning — Unifying Data Science Skip to content Power Calculations and Experiment Planning When you read about them in a book, power calculations can feel very abstract. To make the concept more concrete, and to give you a chance to familiarize yourself with power calculation tools and the considerations that go into power calculations, in this exercise you will play the role of an experiment planner hired by a Non-Governmental Organization (NGO) named Bandhan in West Bengal, India. Bandhan is planning to roll out a program that provides livestock, cash, and some basic training to households in extreme poverty. They hope to demonstrate the value of cash and asset transfers, and so wish their program to take the form of a randomized experiment, with data being collected on both control and treatment households.
Your job will be to help them estimate how many households they should enroll in the program.
This was, in fact, a real program, and so the data you will be working with comes from the actual program (conducted in 2007, with followup data collection in 2010, 2017, and 2020), allowing us to do some retrospective comparison of your power calculations and what actually occurred.
The Context Development economics has long posited that the poor may be poor for no good reason other than the fact that they started poor. This is the idea of a poverty trap, which has the implication that a one-time capital grant that makes very poor households significantly less poor (“big push”) might set off a virtuous cycle that takes them out of poverty. Forty-three countries now embrace some version of this idea and make large transfers to over 3.1 million of their poorest households. In particular, the “Targeting the Ultra Poor” (TUP) intervention, pioneered by BRAC in Bangladesh, employs a multifaceted approach, offering poor households an asset transfer, consumption support, savings, and training in the hopes of unlocking a poverty trap.
In 2007, Bandhan in West Bengal, India created a pilot program to provide direct, unconditional transfers of productive livestock and a 30 or 40 week stipend of 90 rupees a week (about 7 US dollars a week using the Purchasing Power Parity (PPP) exchange rate). The program targeted the poorest households in these villages based on range of criteria, and the average eligible household was later estimated to have a consumption level of about 1.35 2018 US Dollars per capita per day in PPP terms \(.^1\)
Because the goal of the program was, in significant part, to demonstrate the effect of direct asset transfers, Bandhan identified twice as many eligible households as it could support in the pilot and randomly selected half of the households to act as controls and half to be treated \(.^2\) Treated households were offered assets from a menu of options, from the most common choice was productive livestock (e.g., cows and goats). \(^3\) Bandhan’s contact with the households came to an end 18 months after the households were initially provided their livestock and cash transfers began.
Although data was collected on a huge range of attributes of the households in this program, our focus will be on per capita household expenditures in 2018 US Dollars (PPP).
(Note that these households certainly were not buying their food with dollars, and in most cases they weren’t even buying most of their food with rupees — consumption estimates in this type of study are calculated by collecting detailed data on what household members have consumed in the past week, then estimating the price one would pay to buy those goods in local markets and using PPP conversion rates conversion rates to convert that into US Dollars).
Exercises Exercise 2 The first step in doing any power calculations is to establish an estimate of the baseline level of variation that exists in the dependent variable you wish to study. In this case, that variable is Per Capita Monthly Expenditures, and the baseline survey values of expenditures are in pc_exp_month_bl
(the _bl
stands for “baseline”, meaning this data was collected prior to any households receiving asset transfers and indeed any households being assigned to treatment or control).
What is the mean and standard deviation of pc_exp_month_bl
in the baseline survey? Divide by 30 to confirm our average household has the expected per capita expenditure level of about US Dollar 1.35 (PPP) a day.
You’ll use these quantites a lot, so assign them to variables.
Exercise 3 For these exercises, we’ll use the power
module in statsmodels
. You can read about it here.
Since we’re comparing means in a continuous variable (expenditures) from two samples of households, we will use TTestIndPower
in statsmodels.stats.power
. Import this class and instantiate a new instance (for some reason this is class based, so you have to start of with a command like my_power = TTestIndPower()
).
Note that a common situation in data science is testing a difference in proportions , as in situations where your dependent variable is binary and each group’s mean is the share for whom the binary variable is 1. This comes up a lot with apps and websites — e.g., “clicked an ad,” “subscribed,” “made a purchase.” For that reason, there’s actually a full sub-class of power calculating tools for proportions you can read about here. Basically, because the standard deviation of a binary variable is just \(\sqrt{p * (1-p)}\) , power calculations become really simple.
Exercise 4 TTestIndPower has three methods — plot_power
, power
, and solve_power
— but solve_power
does everything power
does, so there are basically two methods.
The idea of solve_power
is that you give it all but one parameter of a power calculation — minimum detectable effect, number of observations, power, and p-value \(\alpha\) threshold — and it will solve for the omitted variable.
To get a quick flavor for how this works, set:
Minimal Detectable Effect (effect_size
) to 0.5
(we’ll take about the units of that argument soon),
alpha
to 0.05
,
ratio
to 1
(this is the ratio of between the number of observations being treated and the number in control — in this study, the target was to have these be equal, as that provides the best statistical power for a given total number of observations).
power
to 0.8
alternative
to two-sided
(we want to do a two-tailed t-test, as you could imagine social unrest caused by transfers could make people worse off).
nobs1
to None
. By passing None
, we’re telling solve_power
this is the quantity we want it to solve for.
So in other words, we’re asking:
how many observations do we need to have in the treatment arm,
assuming we’ll have the same number also in the control arm,
to have an 80% chance
of rejecting the null hypothesis of no-effect at a significance threshold of 0.05
if the true effect is of size 0.5?
What’s the answer?
Exercise 5 Of all these quantities, perhaps the most obscure is the Minimal Detectable Effect (effect_size
). What is this 0.5? Where did it come from? And how did you do all that without telling it that standard deviation you had me calculate?
If you check the docs for solve_power
, you’ll see that you’re supposed to pass effect_size
the “standardized effect size, difference between the two means divided by the standard deviation. effect_size
has to be positive.”
In other words, when we passed 0.5, we weren’t saying we wanted to be able to detect an effect of 0.5 2018 US Dollars (PPP) per capita per month, we were saying we wanted to detect an effect of 1/2 standard deviation.
Given the standard deviation in our baseline survey was about 24 Dollars, that means we were saying we wanted to detect an effect size of about 12 US Dollars.
Is that a lot? Well, the average per capita monthly expenditures in the baseline survey was about 41 Dollars, so it’s an increase in expenditures of about 25%.
OK, but is that what we should use?
Well, the idea of a Minimal Detectable Effect is that you want to set it at a level where (a) if this was the real effect when you rejected the null, you’d be happy you did the experiment and, presumably, would be comfortable scaling up the treatment, but also (b) if the true effect were any smaller, you wouldn’t want to reject the null hypothesis and scale up the treatment.
Basically, if you set this too high, you may run an under-powered experiment where you miss out on learning that your treatment was effective; if you set this too low, you’re spending time and money collecting more data than is probably necessary given you might reject the null at a point estimate where you still wouldn’t scale up the treatment.
So what value makes sense here? That’s a question you’d want to start off by asking the stakeholders. Since Bandhan wants to use this to promote cash transfers, they probably need the effect to be relatively large to be convincing. Let’s assume that, three years after the intervention, they feel they need to show that incomes have increased by at least 30%.
What value does that imply should be passed to effect_size
? What is the new implied number of treated households they’ll need? How many households total?
Exercise 6 Let’s also suppose that because Bandhan really wants to sell their result, they also want to ensure no one claims it might have arisen by chance. What parameter would you adjust, and in what direction? Try modifying it and get a new observation requirement (can you predict what the change in num of observations will be before you do it?).
Exercise 7 Now suppose that your stakeholder is budget constrained to only being able to enroll a total of 100 (50 in each arm). Assuming an alpha of 0.05
and a power of 0.8
, what is their minimal detectable effect size (in dollars)?
Exercise 8 Suppose your stakeholder wants to see a few different scenarios to see how different experiment sizes would impact power given different effect sizes. Let’s use plot_power
. Pass it an array of treatment arm observation counts you want evaluated (I’d recommend np.arange()
) between 20 and 100, along with effect sizes of 0.2, 0.4, and 0.6 (again, use np.arange()
).
If the stakeholder decided they wanted power above 90% — they’re only gonna get a chance to do this once, after all! — what kind of sample size would they need with a minimal detectable effect size of 0.6? (Approximately).
You may need to check the docs to figure out how to use it.
\ No newline at end of file
diff --git a/docs/html/exercises/exercise_power_calculations.ipynb b/docs/html/exercises/exercise_power_calculations.ipynb
new file mode 100644
index 00000000..942a31f4
--- /dev/null
+++ b/docs/html/exercises/exercise_power_calculations.ipynb
@@ -0,0 +1,271 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Power Calculations and Experiment Planning\n",
+ "\n",
+ "When you read about them in a book, power calculations can feel very abstract. To make the concept more concrete, and to give you a chance to familiarize yourself with power calculation tools and the considerations that go into power calculations, in this exercise you will play the role of an experiment planner hired by a Non-Governmental Organization (NGO) named [Bandhan](https://www.bandhan.org/) in West Bengal, India. Bandhan is planning to roll out a program that provides livestock, cash, and some basic training to households in extreme poverty. They hope to demonstrate the value of cash and asset transfers, and so wish their program to take the form of a randomized experiment, with data being collected on both control and treatment households.\n",
+ "\n",
+ "Your job will be to help them estimate how many households they should enroll in the program.\n",
+ "\n",
+ "This was, in fact, a real program, and so the data you will be working with comes from the actual program (conducted in 2007, with followup data collection in 2010, 2017, and 2020), allowing us to do some retrospective comparison of your power calculations and what actually occurred."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## The Context\n",
+ "\n",
+ "> Development economics has long posited that the poor may be poor for no good reason other than the fact that they started poor. This is the idea of a poverty trap, which has the implication that a one-time capital grant that makes very poor households significantly less poor (\"big push\") might set off a virtuous cycle that takes them out of poverty. Forty-three countries now embrace some version of this idea and make large transfers to over 3.1 million of their poorest households. In particular, the \"Targeting the Ultra Poor\" (TUP) intervention, pioneered by BRAC in Bangladesh, employs a multifaceted approach, offering poor households an asset transfer, consumption support, savings, and training in the hopes of unlocking a poverty trap.\n",
+ "\n",
+ "- [*Long-Term Effects of the Targeting the Ultra Poor Program*](https://economics.mit.edu/research/publications/long-term-effects-targeting-ultra-poor-program), Abhijit Banerjee, Esther Duflo, and Garima Sharma, 2021.\n",
+ "\n",
+ "In 2007, Bandhan in West Bengal, India created a pilot program to provide direct, unconditional transfers of productive livestock and a 30 or 40 week stipend of 90 rupees a week (about 7 US dollars a week using the [Purchasing Power Parity (PPP)](https://en.wikipedia.org/wiki/Purchasing_power_parity) exchange rate). The program targeted the poorest households in these villages based on range of criteria, and the average eligible household was later estimated to have a consumption level of about 1.35 2018 US Dollars per capita per day in PPP terms $.^1$\n",
+ "\n",
+ "Because the goal of the program was, in significant part, to demonstrate the effect of direct asset transfers, Bandhan identified twice as many eligible households as it could support in the pilot and randomly selected half of the households to act as controls and half to be treated $.^2$ Treated households were offered assets from a menu of options, from the most common choice was productive livestock (e.g., cows and goats). $^3$ Bandhan's contact with the households came to an end 18 months after the households were initially provided their livestock and cash transfers began.\n",
+ "\n",
+ "Although data was collected on a huge range of attributes of the households in this program, our focus will be on per capita household expenditures in 2018 US Dollars (PPP). \n",
+ "\n",
+ "(Note that these households certainly were not buying their food with dollars, and in most cases they weren't even buying most of their food with rupees — consumption estimates in this type of study are calculated by collecting detailed data on what household members have consumed in the past week, then estimating the price one would pay to buy those goods in local markets and using [PPP conversion rates](https://en.wikipedia.org/wiki/Purchasing_power_parity) conversion rates to convert that into US Dollars).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## Exercises\n",
+ "\n",
+ "### Exercise 1\n",
+ "\n",
+ "Load a thinned version of household-level survey data from [Banerjee, Duflo, and Sharma (2021)'s](https://economics.mit.edu/research/publications/long-term-effects-targeting-ultra-poor-program) evaluation of the Bandham program [here](https://github.com/nickeubank/MIDS_Data/tree/master/cash_transfers). \n",
+ "\n",
+ "This is just a version of the replication data for that paper ([which can be found here](https://www.openicpsr.org/openicpsr/project/130362)) with only the variables that are relevant for our analysis."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 2\n",
+ "\n",
+ "The first step in doing any power calculations is to establish an estimate of the baseline level of variation that exists in the dependent variable you wish to study. In this case, that variable is Per Capita Monthly Expenditures, and the baseline survey values of expenditures are in `pc_exp_month_bl` (the `_bl` stands for \"baseline\", meaning this data was collected prior to any households receiving asset transfers and indeed any households being assigned to treatment or control).\n",
+ "\n",
+ "What is the mean and standard deviation of `pc_exp_month_bl` in the baseline survey? Divide by 30 to confirm our average household has the expected per capita expenditure level of about US Dollar 1.35 (PPP) a day.\n",
+ "\n",
+ "You'll use these quantites a lot, so assign them to variables."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 3\n",
+ "\n",
+ "For these exercises, we'll use the `power` module in `statsmodels`. [You can read about it here.](https://www.statsmodels.org/stable/stats.html#power-and-sample-size-calculations)\n",
+ "\n",
+ "Since we're comparing means in a continuous variable (expenditures) from two samples of households, we will use `TTestIndPower` in `statsmodels.stats.power`. Import this class and instantiate a new instance (for some reason this is class based, so you have to start of with a command like `my_power = TTestIndPower()`). \n",
+ "\n",
+ "Note that a common situation in data science is testing a difference in *proportions*, as in situations where your dependent variable is binary and each group's mean is the share for whom the binary variable is 1. This comes up a lot with apps and websites — e.g., \"clicked an ad,\" \"subscribed,\" \"made a purchase.\" For that reason, there's actually a full sub-class of power calculating tools for [proportions you can read about here.](https://www.statsmodels.org/stable/stats.html#proportion) Basically, because the standard deviation of a binary variable is just $\\sqrt{p * (1-p)}$, power calculations become really simple."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 4\n",
+ "\n",
+ "[TTestIndPower has three methods](https://www.statsmodels.org/stable/generated/statsmodels.stats.power.TTestIndPower.html#statsmodels.stats.power.TTestIndPower) — `plot_power`, `power`, and `solve_power` — but `solve_power` does everything `power` does, so there are basically two methods.\n",
+ "\n",
+ "The idea of `solve_power` is that you give it all but one parameter of a power calculation — minimum detectable effect, number of observations, power, and p-value $\\alpha$ threshold — and it will solve for the omitted variable.\n",
+ "\n",
+ "To get a quick flavor for how this works, set: \n",
+ "\n",
+ "- Minimal Detectable Effect (`effect_size`) to `0.5` (we'll take about the units of that argument soon), \n",
+ "- `alpha` to `0.05`, \n",
+ "- `ratio` to `1` (this is the ratio of between the number of observations being treated and the number in control — in this study, the target was to have these be equal, as that provides the best statistical power for a given total number of observations).\n",
+ "- `power` to `0.8`\n",
+ "- `alternative` to `two-sided` (we want to do a two-tailed t-test, as you could imagine social unrest caused by transfers could make people worse off).\n",
+ "- `nobs1` to `None`. By passing `None`, we're telling `solve_power` this is the quantity we want it to solve for.\n",
+ "\n",
+ "So in other words, we're asking:\n",
+ "\n",
+ "- how many observations do we need to have in the treatment arm,\n",
+ "- assuming we'll have the same number also in the control arm,\n",
+ "- to have an 80% chance\n",
+ "- of rejecting the null hypothesis of no-effect at a significance threshold of 0.05\n",
+ "- if the true effect is of size 0.5?\n",
+ "\n",
+ "What's the answer?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 5\n",
+ "\n",
+ "Of all these quantities, perhaps the most obscure is the Minimal Detectable Effect (`effect_size`). What is this 0.5? Where did it come from? And how did you do all that without telling it that standard deviation you had me calculate?\n",
+ "\n",
+ "If you check the docs for `solve_power`, you'll see that you're supposed to pass `effect_size` the \"standardized effect size, difference between the two means divided by the standard deviation. `effect_size` has to be positive.\"\n",
+ "\n",
+ "In other words, when we passed 0.5, we weren't saying we wanted to be able to detect an effect of 0.5 2018 US Dollars (PPP) per capita per month, we were saying we wanted to detect an effect of 1/2 standard deviation. \n",
+ "\n",
+ "Given the standard deviation in our baseline survey was about 24 Dollars, that means we were saying we wanted to detect an effect size of about 12 US Dollars. \n",
+ "\n",
+ "Is that a lot? Well, the average per capita monthly expenditures in the baseline survey was about 41 Dollars, so it's an increase in expenditures of about 25%. \n",
+ "\n",
+ "OK, but is that what we should use?\n",
+ "\n",
+ "Well, the idea of a Minimal Detectable Effect is that you want to set it at a level where (a) if this *was* the real effect when you rejected the null, you'd be happy you did the experiment and, presumably, would be comfortable scaling up the treatment, but also (b) if the true effect were any smaller, you wouldn't want to reject the null hypothesis and scale up the treatment.\n",
+ "\n",
+ "Basically, if you set this too high, you may run an under-powered experiment where you miss out on learning that your treatment was effective; if you set this too low, you're spending time and money collecting more data than is probably necessary given you might reject the null at a point estimate where you still wouldn't scale up the treatment.\n",
+ "\n",
+ "So what value makes sense here? That's a question you'd want to start off by asking the stakeholders. Since Bandhan wants to use this to promote cash transfers, they probably need the effect to be relatively large to be convincing. Let's assume that, three years after the intervention, they feel they need to show that incomes have increased by at least 30%. \n",
+ "\n",
+ "What value does that imply should be passed to `effect_size`? What is the new implied number of treated households they'll need? How many households total?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 6\n",
+ "\n",
+ "Let's also suppose that because Bandhan really wants to sell their result, they also want to ensure no one claims it might have arisen by chance. What parameter would you adjust, and in what direction? Try modifying it and get a new observation requirement (can you predict what the change in num of observations will be before you do it?)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 7\n",
+ "\n",
+ "Now suppose that your stakeholder is budget constrained to only being able to enroll a total of 100 (50 in each arm). Assuming an alpha of `0.05` and a power of `0.8`, what is their minimal detectable effect size (in dollars)?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 8\n",
+ "\n",
+ "Suppose your stakeholder wants to see a few different scenarios to see how different experiment sizes would impact power given different effect sizes. Let's use `plot_power`. Pass it an array of treatment arm observation counts you want evaluated (I'd recommend `np.arange()`) between 20 and 100, along with effect sizes of 0.2, 0.4, and 0.6 (again, use `np.arange()`).\n",
+ "\n",
+ "If the stakeholder decided they wanted power above 90% — they're only gonna get a chance to do this once, after all! — what kind of sample size would they need with a minimal detectable effect size of 0.6? (Approximately).\n",
+ "\n",
+ "You may need to check the docs to figure out how to use it."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Extra Credit Extension"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 9\n",
+ "\n",
+ "As noted above, this study actually was conducted, so we know the actual treatment effect. Calculate that treatment effect 3 years after these families first received asset transfers by calculating the difference in `pc_exp_month_el2` between the `treatment == 1` and `treatment == 0` groups.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 10\n",
+ "\n",
+ "What was the statistical power of the actual test? That is, what was the probability, given this effect size and the number of observations in the study, that they would reject the null hypothesis of no effect at alpha=0.05 and given the actual final sample sizes?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 11\n",
+ "\n",
+ "What would the power be if Bandhan had only enrolled 200 households per arm? \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 13\n",
+ "\n",
+ "The idea of the \"power\" of a test is that, when we draw a sample from a population, there's some variation in who happens to end up in that sample. Consequently, even if the *true* effect size in the population is equal to our Minimal Detectable Effect, there's some probability that when we compare treated and untreated outcomes for the $N$ people who actually end up in the study, that Minimal Detectable Effect may not be evident in that sample (at the level of statistical significance $\\alpha$ we have chosen).\n",
+ "\n",
+ "Given that, one way to think of power is: \"If the true effect in the population is our Minimal Detectable Effect, then if I were able to re-run this experiment over and over — drawing new people into the study and testing the difference in outcomes between the control and treated sample each time — then in what percentage of those instances of the experiment would I reject the null hypothesis of no effect?\" Power of 80% means that we'd expect to reject the null of no effect in 80% of those many experiments.\n",
+ "\n",
+ "Well, we can basically do that thought experiment with this data! \n",
+ "\n",
+ "To do the experiment you did the power calculations for in Exercise 11, we just need to sample 200 observations from the treated group and 200 from the control group, treat that as our experimental sample, and estimate the difference in per capita month expenditures between those two groups (along with the associated p-value).\n",
+ "\n",
+ "Then we can repeat that over and over to simulate \"re-running\" the experiment, each time drawing a new sample of 200 treated observations and 200 control observations. Then we can store the p-values from all these \"re-run\" experiments and see how often we reject the null of no effect!\n",
+ "\n",
+ "Note that when you do this experiment, we have to sample our 200 observations from each treatment arm *with replacement*, just as you would when bootstrapping (if that means anything to you — if it doesn't, don't worry about it). \n",
+ "\n",
+ "So: write a loop where, on each pass, you draw 200 observations (with replacement) from treatment and 200 from control, then calculate the treatment effect and p-value for that sample. Repeat this 10,000 times. \n",
+ "\n",
+ "In what share of cases would you reject the null of no effect at alpha = 0.05?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 14\n",
+ "\n",
+ "Repeat 12 and 13 with 300 observations per arm."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 15\n",
+ "\n",
+ "What did you learn from Exercises 9-14?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Footnotes\n",
+ "\n",
+ "1. The poorest households were identified in two steps. First, residents across 120 village hamlets ranked households into five wealth quintiles. Among households ranked in the bottom quintile, Bandhan then verified eligibility per seven criteria: (i) presence of an able-bodied female member (to manage the asset), (ii) no credit access, (iii) landholding below 0.2 acres, (iv) no ownership of productive assets, (v) no able-bodied male member, (vi) presence of school-aged children who were working instead of attending school, and (vii) primary source of income being informal labor or begging. Households had to meet the first two criteria and at least three of the remaining five in order to be eligible for the TUP intervention.\n",
+ "2. The [ethics of randomization in these types of programs](https://www.povertyactionlab.org/resource/ethical-conduct-randomized-evaluations) is subject of a rich literature, and there are currently an extensive set of guidelines used by researchers developing these types of programs.\n",
+ "3. Of the 514 offered the livestock and cash transfers, only 266 accepted. The treatment estimates that follow are thus estimates of the effect of *offering* these transfers. Households that accepted the offers might differ systematically from those that do not, so the study simply compares those *offered* the transfer to those who did not). This is what's called an estimate of the \"intention to treat\" effect, and is thus *likely* an under-estimate of the effect of cash transfers that would be observed if uptake were greater."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.11.8"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/docs/html/objects.inv b/docs/html/objects.inv
index c673224d..7a59e1d6 100644
Binary files a/docs/html/objects.inv and b/docs/html/objects.inv differ
diff --git a/docs/html/searchindex.js b/docs/html/searchindex.js
index b678ab27..d67ca0a6 100644
--- a/docs/html/searchindex.js
+++ b/docs/html/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"docnames": ["DRAFT_ml_bias", "UDS_midterm_part2_2022", "backwards_design", "causal_inference_beyond_ab_testing", "class_schedule", "descriptive_questions", "endogenous_stopping", "ethical_ml_recommendations", "evaluating_real_studies", "exercises/discussion_exploratory", "exercises/discussion_regressions_incomeineq", "exercises/exercise_abtesting", "exercises/exercise_counterfactuals", "exercises/exercise_diffindiff", "exercises/exercise_evaluating_studies", "exercises/exercise_exploratory", "exercises/exercise_first_class_capstone_proposal", "exercises/exercise_generating_questions", "exercises/exercise_ghostmap_1", "exercises/exercise_ghostmap_2", "exercises/exercise_indicators", "exercises/exercise_interpretable", "exercises/exercise_matching", "exercises/exercise_optimalABthresholds", "exercises/exercise_panel", "exercises/exercise_passive_prediction", "exercises/exercise_potential_outcomes1", "exercises/exercise_potential_outcomes2", "exercises/exercise_regressions_incomeineq", "exercises/exercise_resume_experiment", "exercises/exercise_stakeholder_management", "exercises/exercise_taxonomy_of_questions", "exercises/setup_diffindiff", "exercises/solutions_descriptive", "exercises/solutions_diffindiff", "exercises/solutions_first_class_capstone_proposal", "exercises/solutions_indicators", "exercises/solutions_matching", "exercises/solutions_panel", "exercises/solutions_potential_outcomes1", "exercises/solutions_regressions_incomeineq", "exercises/solutions_resume", "fixed_effects", "fixed_effects_and_causal_inference", "fixed_effects_v_hierarchical", "how_to_read", "index", "internal_v_external_validity", "interpreting_indicator_vars", "limitations_of_ATE", "matching_how", "matching_why", "moving_from_problems_to_questions", "nick_ab_testing_checklist", "public_data", "pvalues_and_decision_making", "reading_reflections", "solutions", "solutions_warning", "taxonomy_of_questions", "team_assignments/UDS_TeamProject_Exploratory", "writing_to_stakeholders"], "filenames": ["DRAFT_ml_bias.ipynb", "UDS_midterm_part2_2022.ipynb", "backwards_design.ipynb", "causal_inference_beyond_ab_testing.ipynb", "class_schedule.rst", "descriptive_questions.ipynb", "endogenous_stopping.ipynb", "ethical_ml_recommendations.ipynb", "evaluating_real_studies.ipynb", "exercises/discussion_exploratory.ipynb", "exercises/discussion_regressions_incomeineq.ipynb", "exercises/exercise_abtesting.ipynb", "exercises/exercise_counterfactuals.ipynb", "exercises/exercise_diffindiff.ipynb", "exercises/exercise_evaluating_studies.ipynb", "exercises/exercise_exploratory.ipynb", "exercises/exercise_first_class_capstone_proposal.ipynb", "exercises/exercise_generating_questions.ipynb", "exercises/exercise_ghostmap_1.ipynb", "exercises/exercise_ghostmap_2.ipynb", "exercises/exercise_indicators.ipynb", "exercises/exercise_interpretable.ipynb", "exercises/exercise_matching.ipynb", "exercises/exercise_optimalABthresholds.ipynb", "exercises/exercise_panel.ipynb", "exercises/exercise_passive_prediction.ipynb", "exercises/exercise_potential_outcomes1.ipynb", "exercises/exercise_potential_outcomes2.ipynb", "exercises/exercise_regressions_incomeineq.ipynb", "exercises/exercise_resume_experiment.ipynb", "exercises/exercise_stakeholder_management.ipynb", "exercises/exercise_taxonomy_of_questions.ipynb", "exercises/setup_diffindiff.ipynb", "exercises/solutions_descriptive.ipynb", "exercises/solutions_diffindiff.ipynb", "exercises/solutions_first_class_capstone_proposal.ipynb", "exercises/solutions_indicators.ipynb", "exercises/solutions_matching.ipynb", "exercises/solutions_panel.ipynb", "exercises/solutions_potential_outcomes1.ipynb", "exercises/solutions_regressions_incomeineq.ipynb", "exercises/solutions_resume.ipynb", "fixed_effects.ipynb", "fixed_effects_and_causal_inference.ipynb", "fixed_effects_v_hierarchical.ipynb", "how_to_read.ipynb", "index.rst", "internal_v_external_validity.ipynb", "interpreting_indicator_vars.ipynb", "limitations_of_ATE.ipynb", "matching_how.ipynb", "matching_why.ipynb", "moving_from_problems_to_questions.ipynb", "nick_ab_testing_checklist.ipynb", "public_data.ipynb", "pvalues_and_decision_making.ipynb", "reading_reflections.ipynb", "solutions.ipynb", "solutions_warning.ipynb", "taxonomy_of_questions.ipynb", "team_assignments/UDS_TeamProject_Exploratory.ipynb", "writing_to_stakeholders.ipynb"], "titles": ["Backwards Design", "UDS Midterm 2022, Part 2", "Backwards Design", "Beyond The Experiment", "Class Schedule", "Researcher Discretion in Descriptive Analysis", "Peeking / Endogenous Stopping", "Recommendations for Responsible Machine Learning", "Evaluating Real Studies", "Discussion of Descriptive Exercises", "Interpreting Causal Effects of Gender", "A/B Testing the Udacity Website", "Counter-Factuals and Experimental Ideals", "Marijuana Legalization and Violent Crime", "Evaluating Studies", "Crime and Policing Expenditures Exploratory Questions", "Your First Stakeholder", "Converting Stakeholder Prompts into Actionable Questions", "Causality in the Time of Cholera", "Causality in the Time of Cholera", "Interpreting Indicator Variables", "Interpretable Modelling of Credit Risk", "Matching Exercise", "Optimal AB Testing Design", "Traffic Death Analysis", "Predicting Mortgage Delinquency Risk", "Making Potential Outcomes Concrete", "Making Potential Outcomes Concrete 2", "Estimating Gender Discrimination in the Workplace", "Resume Experiment Analysis", "Soliciting Information From Your Stakeholder", "Generating and Classifying Questions", "Groupby and Arrest Data", "Crime and Poverty Descriptive Analysis", "Groupby and Arrest Data", "Acme Corportation Solution", "Interpreting Indicator Variables", "Matching Exercise", "Traffic Death Analysis", "Making Potential Outcomes Concrete", "Estimating Gender Discrimination in the Workplace", "Resume Experiment Analysis", "Fixed Effects: Indicator Variables for Groups", "Fixed Effects and Causal Inference", "Fixed Effects and Hierarchical Models", "How to Read (Academic Edition)", "Welcome to Solving Problems with Data!", "Internal and External Validity", "Using and Interpreting Indicator (Dummy) Variables", "Limitations of Experiments (and Average Treatment Effects)", "How Matching is Done (A Summary)", "Matching", "Converting Stakeholder Prompts into Questions", "Nick\u2019s Things to Think About Before A/B Testing", "Where To Find Public Data", "<no title>", "Reading Reflections", "Solutions", "Looking for the solutions?", "Taxonomy of Questions", "Using Exploratory Questions to Better Understanding Your Problem", "Writing Data Science Report for Non-Technical Audiences"], "terms": {"i": [0, 1, 3, 5, 6, 9, 10, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 51, 53, 54, 56, 58, 60, 61], "wai": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 16, 22, 24, 25, 26, 27, 29, 30, 32, 35, 37, 38, 39, 41, 42, 43, 45, 47, 48, 49, 50, 51, 52, 53, 54, 58, 59, 61], "develop": [0, 2, 5, 7, 17, 18, 19, 21, 22, 37, 40, 45, 46, 47, 49, 52, 56, 59], "an": [0, 1, 3, 5, 6, 8, 9, 11, 12, 13, 14, 15, 16, 17, 21, 22, 23, 24, 25, 26, 27, 28, 29, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 51, 52, 53, 56, 58, 59, 60, 61], "effici": [0, 2, 3, 7, 22, 24, 28, 29, 31, 37, 38, 40, 44, 45], "strategi": [0, 2, 11, 16, 24, 38, 44, 49, 50, 52, 53, 61], "complet": [0, 2, 11, 16, 18, 19, 21, 22, 25, 29, 37, 41, 51, 52, 61], "new": [0, 2, 3, 5, 7, 8, 11, 13, 14, 20, 21, 22, 23, 25, 30, 34, 35, 36, 37, 40, 43, 45, 47, 48, 49, 53, 59, 61], "data": [0, 3, 4, 5, 6, 8, 9, 13, 14, 15, 16, 17, 18, 20, 23, 24, 26, 27, 28, 29, 33, 35, 36, 38, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 51, 52, 53, 59], "scienc": [0, 2, 3, 4, 5, 7, 13, 16, 18, 26, 27, 30, 34, 39, 45, 46, 49, 52, 54, 59], "project": [0, 2, 3, 4, 11, 16, 21, 26, 27, 35, 39, 46, 54, 56, 60, 61], "my": [0, 2, 4, 5, 7, 21, 22, 25, 32, 36, 37, 41, 47, 48, 49, 51, 52, 59], "view": [0, 2, 7, 8, 11, 23, 48], "one": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 13, 15, 16, 21, 22, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 56, 59, 60, 61], "most": [0, 2, 5, 7, 13, 16, 21, 22, 29, 30, 32, 34, 37, 41, 43, 44, 46, 47, 48, 49, 50, 51, 52, 54, 56, 59, 61], "import": [0, 1, 3, 4, 5, 7, 13, 16, 21, 22, 24, 28, 32, 33, 34, 35, 36, 37, 38, 40, 41, 42, 43, 45, 47, 48, 49, 51, 52, 59, 60, 61], "skill": [0, 2, 3, 13, 15, 29, 33, 34, 41, 45, 46, 54, 56, 58, 59, 61], "profession": [0, 2, 3, 40, 46], "scientist": [0, 2, 3, 5, 7, 16, 22, 30, 37, 42, 44, 45, 46, 48, 52, 61], "If": [0, 1, 2, 3, 5, 6, 7, 8, 11, 12, 18, 19, 21, 22, 24, 25, 29, 32, 33, 36, 37, 38, 40, 42, 43, 44, 45, 46, 48, 49, 50, 51, 52, 53, 54], "don": [0, 1, 2, 4, 5, 6, 7, 8, 9, 11, 15, 18, 19, 20, 21, 22, 23, 24, 25, 28, 32, 33, 34, 35, 36, 37, 39, 40, 41, 42, 43, 44, 45, 47, 48, 50, 51, 52, 53, 59, 60, 61], "t": [0, 1, 2, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 50, 51, 52, 53, 54, 58, 60, 61], "have": [0, 1, 3, 5, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 34, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 56, 58, 59, 60, 61], "lot": [0, 2, 3, 5, 6, 7, 8, 10, 11, 14, 21, 22, 28, 37, 39, 40, 42, 44, 45, 47, 49, 51, 52, 53, 54, 59, 60, 61], "experi": [0, 2, 4, 5, 6, 7, 8, 40, 43, 45, 46, 47, 51, 53, 57, 59, 60, 61], "mai": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 14, 16, 20, 21, 22, 24, 25, 26, 27, 28, 29, 31, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 47, 48, 49, 51, 52, 53, 54, 56, 58, 59, 60, 61], "obviou": [0, 2, 5, 7, 41, 43, 47, 48], "why": [0, 1, 3, 4, 5, 6, 7, 11, 21, 22, 25, 26, 27, 29, 30, 35, 37, 39, 41, 45, 46, 48, 49, 50, 51, 56, 60, 61], "thi": [0, 1, 2, 3, 5, 6, 7, 9, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 54, 56, 58, 60, 61], "even": [0, 2, 3, 6, 7, 8, 11, 12, 13, 14, 22, 23, 25, 29, 32, 34, 37, 39, 41, 42, 44, 46, 47, 49, 59], "call": [0, 1, 2, 3, 6, 7, 8, 11, 12, 13, 22, 24, 25, 28, 29, 30, 32, 34, 37, 38, 40, 41, 42, 44, 47, 48, 50, 51, 53, 54, 59], "That": [0, 3, 5, 7, 8, 11, 12, 18, 19, 22, 25, 29, 36, 37, 40, 43, 45, 47, 48, 49, 50, 51, 59, 60, 61], "": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 13, 14, 15, 16, 17, 18, 19, 20, 24, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 50, 51, 54, 58, 59, 60, 61], "becaus": [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 21, 22, 24, 26, 27, 28, 29, 32, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 49, 50, 51, 52, 54, 59, 60, 61], "student": [0, 2, 5, 7, 11, 13, 16, 26, 29, 30, 34, 39, 41, 42, 44, 45, 47, 54, 58, 59], "come": [0, 1, 2, 3, 5, 7, 11, 14, 21, 22, 25, 28, 37, 40, 42, 45, 46, 47, 48, 49, 51, 52, 54, 59, 60], "from": [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 56, 59, 60, 61], "classroom": [0, 2, 12, 44, 46], "exercis": [0, 1, 2, 4, 10, 16, 18, 19, 23, 26, 27, 32, 39, 45, 46, 58], "site": [0, 2, 7, 8, 11, 23, 30, 32, 40, 42, 46, 48, 52, 54, 58, 60], "like": [0, 1, 3, 5, 6, 7, 8, 11, 12, 14, 15, 16, 17, 18, 19, 21, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 35, 37, 38, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 50, 51, 52, 53, 54, 56, 58, 59, 60, 61], "kaggl": [0, 2, 7, 60], "These": [0, 2, 11, 12, 20, 22, 36, 37, 54, 56], "type": [0, 2, 3, 7, 8, 11, 15, 16, 20, 24, 25, 28, 29, 30, 32, 33, 34, 36, 37, 38, 40, 41, 42, 43, 45, 47, 48, 51, 52, 59, 60, 61], "ar": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 49, 50, 51, 52, 56, 59, 60, 61], "excel": [0, 2, 7, 21], "opportun": [0, 2, 3, 13, 34, 45, 56], "usual": [0, 2, 3, 11, 12, 25, 28, 29, 40, 43, 46, 47, 49, 54, 59, 61], "case": [0, 2, 3, 5, 7, 11, 12, 18, 19, 21, 22, 24, 25, 26, 27, 29, 30, 32, 37, 38, 39, 41, 42, 44, 46, 47, 48, 49, 52, 59, 61], "unbeknownst": [0, 2], "been": [0, 1, 2, 5, 7, 9, 11, 12, 13, 16, 17, 22, 24, 25, 26, 27, 29, 30, 31, 33, 34, 37, 38, 39, 41, 43, 45, 47, 50, 51, 52, 59, 61], "carefulli": [0, 2, 32, 47, 61], "tailor": [0, 2, 61], "clearli": [0, 2, 5, 7, 8, 12, 48, 51, 59], "goal": [0, 2, 3, 5, 7, 11, 16, 22, 26, 27, 37, 39, 45, 50, 56, 58, 60, 61], "thei": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 20, 21, 22, 23, 25, 26, 27, 28, 29, 30, 34, 35, 36, 37, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 54, 58, 59, 61], "set": [0, 2, 5, 7, 8, 9, 11, 13, 14, 21, 22, 24, 25, 29, 32, 34, 37, 38, 41, 44, 45, 46, 47, 48, 49, 50, 59, 60], "clean": [0, 1, 2, 17, 22, 32, 37, 40, 60], "filter": [0, 2, 5], "provid": [0, 1, 2, 3, 5, 7, 8, 11, 16, 21, 25, 30, 32, 41, 45, 46, 49, 52, 54, 58, 59, 60, 61], "onli": [0, 1, 2, 3, 5, 8, 9, 10, 11, 12, 13, 17, 18, 20, 21, 22, 24, 25, 27, 28, 29, 32, 34, 35, 36, 37, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 54, 56, 58, 59], "relev": [0, 2, 3, 22, 31, 37, 45, 53, 61], "variabl": [0, 1, 2, 4, 5, 6, 7, 9, 11, 12, 13, 16, 21, 22, 24, 25, 28, 29, 32, 34, 37, 38, 40, 41, 43, 44, 50, 51, 52, 54, 59, 60], "done": [0, 2, 5, 7, 12, 13, 28, 31, 34, 38, 40, 44, 45, 53, 59, 60, 61], "good": [0, 3, 4, 6, 7, 8, 11, 20, 21, 22, 23, 25, 30, 36, 37, 40, 41, 42, 44, 45, 46, 47, 49, 50, 51, 53, 56, 59, 61], "reason": [0, 2, 3, 5, 7, 8, 9, 10, 12, 14, 21, 22, 23, 25, 26, 27, 28, 29, 30, 35, 37, 39, 40, 41, 42, 44, 45, 47, 49, 51, 53, 56, 59, 61], "instructor": [0, 2, 5, 56], "focus": [0, 1, 2, 15, 28, 29, 33, 40, 47], "attent": [0, 1, 2, 22, 25, 37, 45, 61], "try": [0, 2, 3, 7, 8, 11, 12, 17, 18, 19, 21, 22, 25, 26, 27, 28, 37, 39, 40, 42, 43, 45, 47, 48, 49, 51, 52, 59, 60, 61], "model": [0, 1, 2, 4, 5, 11, 13, 16, 20, 22, 24, 28, 29, 34, 35, 36, 37, 38, 40, 41, 42, 43, 45, 46, 47, 48, 59, 61], "select": [0, 2, 6, 7, 11, 21, 29, 41], "interpret": [0, 1, 2, 5, 13, 16, 17, 23, 24, 26, 28, 29, 34, 38, 39, 40, 41, 46, 47, 49, 61], "But": [0, 2, 3, 5, 6, 7, 8, 10, 13, 15, 21, 22, 26, 29, 32, 33, 34, 35, 36, 37, 39, 41, 42, 43, 44, 45, 46, 47, 48, 49, 51, 52, 59, 61], "result": [0, 2, 5, 6, 7, 9, 10, 11, 15, 16, 18, 20, 22, 23, 24, 25, 28, 29, 33, 34, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 51, 52, 54, 59], "often": [0, 1, 2, 3, 5, 6, 7, 21, 24, 28, 30, 32, 38, 40, 42, 44, 45, 46, 47, 48, 49, 51, 52, 54, 56, 59, 60, 61], "awai": [0, 2, 9, 23, 36, 50, 52, 59, 61], "impress": [0, 2, 15, 33], "what": [0, 1, 3, 4, 6, 7, 9, 10, 11, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 42, 43, 44, 45, 47, 49, 50, 51, 53, 54, 59, 60, 61], "do": [0, 1, 4, 5, 6, 7, 8, 9, 11, 13, 15, 16, 18, 19, 20, 21, 23, 24, 25, 26, 27, 29, 30, 31, 32, 33, 34, 35, 36, 38, 39, 41, 42, 44, 47, 48, 49, 50, 51, 54, 56, 58, 59, 60, 61], "work": [0, 1, 2, 3, 5, 7, 9, 10, 11, 12, 14, 16, 17, 18, 20, 21, 22, 24, 27, 28, 30, 32, 36, 37, 38, 40, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 53, 54, 56, 58, 59, 60, 61], "statist": [0, 1, 2, 5, 6, 7, 9, 11, 12, 15, 16, 20, 22, 23, 25, 29, 33, 34, 36, 37, 38, 40, 41, 42, 44, 46, 48, 49, 50, 51, 52, 53, 59, 61], "In": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 56, 59, 61], "realiti": [0, 2, 7, 47, 48, 50], "howev": [0, 1, 2, 6, 7, 11, 18, 21, 22, 24, 25, 32, 37, 38, 42, 43, 44, 46, 48, 49, 50, 59], "thing": [0, 2, 3, 5, 7, 8, 9, 10, 11, 14, 17, 18, 19, 21, 22, 23, 26, 27, 28, 32, 37, 39, 40, 41, 42, 43, 45, 46, 47, 49, 51, 52, 54, 56, 59, 61], "doe": [0, 1, 2, 3, 5, 7, 10, 11, 12, 13, 15, 21, 22, 23, 24, 28, 29, 30, 32, 33, 34, 36, 37, 38, 40, 41, 42, 43, 44, 45, 48, 49, 51, 52, 53, 59], "articul": [0, 2], "concret": [0, 2, 7, 8, 16, 30, 40, 48, 61], "feasibl": [0, 2, 18, 19, 22, 37, 49, 53], "object": [0, 2, 3, 5, 8, 11, 22, 32, 33, 37], "b": [0, 2, 4, 5, 6, 7, 12, 13, 23, 24, 29, 31, 32, 34, 38, 44, 46, 49, 56], "achiev": [0, 2, 3, 11, 30, 53], "And": [0, 2, 3, 5, 7, 8, 9, 12, 22, 24, 25, 28, 29, 35, 37, 38, 40, 41, 42, 43, 45, 46, 47, 48, 49, 51, 52, 58, 59, 61], "best": [0, 2, 3, 5, 7, 8, 9, 12, 16, 22, 25, 30, 37, 45, 47, 49, 50, 51, 52, 54, 58, 59], "go": [0, 1, 2, 5, 7, 8, 9, 10, 13, 15, 18, 19, 20, 21, 22, 24, 25, 29, 30, 33, 34, 36, 37, 38, 41, 44, 45, 47, 48, 50, 54], "about": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 54, 56, 59, 60, 61], "accomplish": [0, 2, 29, 44], "both": [0, 1, 2, 3, 5, 7, 8, 10, 13, 15, 16, 18, 19, 22, 24, 25, 26, 27, 29, 30, 31, 32, 33, 34, 37, 38, 39, 41, 42, 44, 47, 48, 49, 50, 51, 53, 54, 59, 60, 61], "As": [0, 2, 3, 5, 6, 7, 8, 10, 11, 16, 17, 18, 21, 22, 25, 26, 27, 28, 29, 31, 32, 37, 39, 40, 41, 42, 43, 44, 47, 48, 49, 50, 51, 52, 54, 59, 60, 61], "name": [0, 1, 2, 11, 13, 16, 20, 21, 22, 24, 25, 29, 30, 32, 34, 36, 37, 38, 41, 42, 43, 48, 52, 54], "impli": [0, 2, 10, 15, 24, 28, 29, 33, 38, 40, 41, 42, 44, 52, 59], "idea": [0, 2, 6, 7, 13, 34, 42, 44, 45, 47, 49, 50, 51, 52, 54], "start": [0, 2, 3, 4, 6, 7, 8, 11, 21, 22, 23, 25, 28, 29, 37, 40, 45, 47, 48, 50, 51, 52, 53, 59, 61], "where": [0, 1, 3, 5, 6, 7, 8, 9, 11, 12, 13, 21, 22, 24, 25, 27, 30, 32, 34, 37, 38, 39, 41, 42, 43, 44, 45, 46, 49, 50, 51, 52, 53, 56, 59, 61], "want": [0, 1, 2, 5, 6, 7, 8, 11, 15, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 29, 32, 33, 36, 37, 38, 39, 41, 42, 43, 44, 45, 47, 48, 49, 50, 51, 52, 53, 54, 58, 59, 60, 61], "end": [0, 2, 3, 4, 5, 6, 7, 8, 11, 16, 21, 22, 24, 25, 29, 30, 35, 37, 38, 41, 48, 50, 51, 53, 59], "up": [0, 4, 5, 7, 8, 11, 14, 15, 16, 17, 21, 22, 25, 28, 29, 32, 33, 35, 36, 37, 40, 41, 42, 45, 47, 49, 50, 51, 59, 61], "figur": [0, 1, 2, 6, 8, 22, 25, 32, 37, 45, 48, 51, 52, 58, 59, 60], "out": [0, 1, 2, 3, 4, 5, 7, 8, 11, 12, 14, 16, 21, 22, 25, 29, 30, 32, 36, 37, 41, 42, 45, 46, 47, 48, 50, 51, 52, 53, 56, 58, 59, 60, 61], "exactli": [0, 2, 5, 6, 10, 11, 16, 22, 26, 27, 29, 30, 32, 37, 39, 41, 47, 48, 50, 51, 61], "need": [0, 3, 4, 7, 13, 16, 17, 21, 23, 25, 28, 30, 32, 34, 35, 38, 40, 48, 50, 53, 54, 60, 61], "get": [0, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 20, 21, 23, 25, 28, 29, 33, 34, 36, 38, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 56, 58, 59, 60, 61], "actual": [0, 1, 2, 3, 5, 7, 8, 9, 11, 13, 16, 21, 22, 25, 26, 29, 30, 32, 34, 35, 36, 37, 39, 41, 42, 43, 44, 45, 46, 47, 48, 49, 51, 52, 54, 59, 61], "common": [0, 2, 3, 7, 13, 34, 42, 44, 47, 49, 50, 59, 60], "manag": [0, 2, 4, 5, 7, 40, 46, 56, 61], "rang": [0, 2, 3, 23, 34, 37, 45, 51], "differ": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 24, 25, 26, 27, 28, 29, 30, 33, 34, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 60, 61], "domain": [0, 2, 7, 8, 21, 31, 56, 59, 60], "so": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 32, 33, 34, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 59, 60, 61], "alreadi": [0, 2, 7, 8, 16, 34, 47, 50, 52, 53, 59, 60], "familiar": [0, 2, 13, 21, 22, 34, 37, 44, 46, 59], "broad": [0, 2, 5, 18, 19, 21, 45, 61], "term": [0, 1, 2, 3, 4, 5, 6, 7, 8, 11, 13, 16, 20, 21, 22, 23, 25, 26, 27, 28, 29, 30, 34, 36, 37, 39, 40, 41, 43, 44, 45, 47, 48, 50, 51, 53, 59, 60, 61], "class": [0, 2, 3, 5, 7, 8, 10, 13, 16, 21, 28, 34, 40, 45, 46, 47, 48, 51, 52, 54, 56, 59, 60, 61], "we": [0, 1, 2, 3, 5, 6, 7, 10, 11, 12, 13, 15, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 30, 32, 33, 34, 36, 37, 38, 39, 40, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 54, 56, 59, 60, 61], "focu": [0, 2, 5, 22, 28, 29, 30, 37, 40, 42, 43, 45, 46, 47, 51, 59], "five": [0, 2, 20], "step": [0, 2, 8, 22, 25, 29, 30, 37, 41, 50, 54, 59], "solv": [0, 2, 4, 7, 16, 30, 51, 52, 58, 60], "help": [0, 2, 3, 5, 7, 11, 12, 16, 17, 18, 21, 24, 25, 26, 28, 29, 30, 31, 38, 39, 40, 41, 44, 47, 49, 51, 52, 54, 56, 59, 60, 61], "would": [0, 1, 3, 5, 6, 7, 8, 11, 12, 13, 14, 15, 16, 17, 19, 21, 22, 23, 25, 26, 27, 28, 29, 30, 31, 33, 34, 35, 37, 39, 40, 41, 44, 47, 49, 50, 51, 52, 54, 59, 60, 61], "look": [0, 1, 3, 5, 6, 7, 8, 11, 12, 16, 21, 22, 23, 24, 25, 27, 29, 30, 32, 37, 38, 40, 41, 42, 47, 48, 49, 50, 51, 52, 53, 54, 59, 60, 61], "determin": [0, 1, 2, 5, 7, 11, 12, 16, 25, 29, 41, 47, 49, 52, 60], "order": [0, 2, 7, 16, 18, 19, 22, 26, 27, 30, 35, 37, 39, 40, 45], "gener": [0, 2, 4, 5, 7, 11, 13, 21, 22, 23, 25, 28, 29, 32, 34, 37, 40, 41, 42, 44, 45, 46, 47, 49, 50, 51, 52, 53, 54, 56, 59, 60, 61], "identifi": [0, 1, 2, 7, 11, 13, 17, 18, 19, 21, 22, 25, 32, 34, 37, 42, 47, 48, 52, 54, 59, 60], "those": [0, 1, 2, 3, 5, 6, 7, 8, 11, 13, 16, 18, 22, 25, 26, 27, 28, 29, 30, 32, 34, 37, 39, 40, 42, 43, 44, 45, 47, 48, 49, 50, 51, 52, 58, 59, 60, 61], "bring": [0, 2, 7, 25, 35, 42, 46, 47, 52], "them": [0, 2, 3, 5, 7, 8, 11, 12, 16, 17, 21, 22, 26, 27, 30, 37, 39, 42, 44, 45, 47, 48, 50, 51, 52, 54, 59, 60, 61], "togeth": [0, 2, 11], "first": [0, 1, 2, 3, 7, 8, 9, 11, 12, 13, 15, 17, 21, 22, 24, 25, 27, 29, 30, 31, 32, 33, 34, 36, 37, 38, 41, 42, 45, 46, 47, 48, 50, 51, 52, 53, 54, 59, 61], "should": [0, 2, 3, 5, 7, 8, 9, 11, 13, 15, 16, 21, 22, 23, 25, 26, 29, 31, 33, 34, 37, 38, 40, 41, 44, 45, 46, 47, 48, 49, 51, 52, 59, 60, 61], "straightforward": [0, 2, 25], "yet": [0, 2, 3, 9, 13, 21, 22, 34, 37, 48, 52, 59], "surpris": [0, 2, 41, 47], "how": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 24, 25, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 40, 41, 42, 43, 44, 46, 47, 48, 51, 52, 54, 58, 59, 60, 61], "never": [0, 2, 3, 5, 7, 8, 9, 11, 13, 22, 34, 37, 45, 51, 52, 56, 59, 61], "explicitli": [0, 2, 7, 49, 59], "address": [0, 2, 7, 11, 16, 17, 25, 26, 30, 39, 44, 46, 47, 51, 52, 60, 61], "peopl": [0, 2, 3, 5, 7, 8, 11, 12, 13, 14, 17, 18, 21, 22, 26, 27, 28, 29, 30, 31, 34, 37, 39, 40, 41, 42, 45, 47, 48, 49, 51, 52, 54, 56, 58, 59, 60, 61], "excit": [0, 2, 17, 49], "sai": [0, 2, 3, 5, 6, 7, 8, 11, 21, 22, 26, 27, 29, 32, 35, 36, 37, 39, 41, 42, 43, 47, 48, 49, 51, 52, 59, 61], "some": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 14, 17, 21, 22, 23, 24, 25, 28, 29, 34, 37, 38, 39, 40, 41, 42, 46, 47, 48, 49, 50, 51, 52, 54, 56, 58, 59, 60, 61], "alwai": [0, 2, 5, 8, 12, 13, 16, 21, 22, 23, 26, 27, 29, 34, 37, 39, 41, 43, 44, 45, 47, 48, 49, 52, 53, 54, 56, 59, 60], "when": [0, 1, 2, 3, 5, 6, 8, 9, 10, 11, 12, 13, 17, 18, 21, 22, 23, 24, 25, 26, 27, 28, 29, 32, 34, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 49, 51, 52, 54, 59, 60, 61], "make": [0, 1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 15, 16, 18, 19, 20, 21, 22, 23, 24, 25, 28, 29, 30, 31, 32, 33, 34, 36, 37, 38, 40, 41, 42, 43, 44, 47, 48, 49, 50, 51, 53, 55, 59, 60, 61], "sure": [0, 2, 3, 5, 8, 11, 15, 21, 22, 23, 24, 25, 26, 29, 31, 32, 33, 37, 38, 39, 41, 46, 47, 48, 49, 52, 59, 61], "can": [0, 1, 3, 5, 6, 7, 8, 9, 11, 12, 13, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 34, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 51, 52, 53, 54, 56, 58, 59, 60, 61], "addit": [0, 2, 7, 8, 11, 13, 21, 22, 28, 29, 30, 34, 36, 37, 40, 41, 42, 46, 48, 49, 61], "stakehold": [0, 2, 4, 5, 7, 21, 25, 46, 60, 61], "agre": [0, 2, 47, 60], "There": [0, 2, 6, 8, 11, 12, 16, 21, 22, 28, 37, 40, 44, 45, 47, 49, 50, 54, 56], "noth": [0, 2, 11, 41, 53, 59], "wors": [0, 2, 42, 45, 47], "than": [0, 2, 3, 5, 6, 7, 8, 10, 11, 14, 20, 21, 23, 24, 25, 26, 27, 28, 29, 30, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 49, 51, 53, 54, 56, 58, 59, 61], "spend": [0, 2, 3, 7, 8, 12, 26, 27, 39, 49, 51, 52, 58], "week": [0, 2, 11, 16, 23, 28, 32, 39, 40, 43, 47, 51, 53, 56], "discov": [0, 2, 7, 23, 47, 52, 59], "valu": [0, 1, 2, 6, 7, 8, 11, 13, 16, 20, 21, 22, 23, 24, 25, 29, 32, 34, 36, 37, 38, 41, 42, 43, 48, 49, 51, 53, 55, 59], "here": [0, 1, 2, 3, 4, 5, 7, 8, 10, 11, 13, 16, 21, 22, 24, 25, 29, 30, 32, 34, 37, 38, 41, 42, 45, 46, 47, 48, 49, 50, 51, 52, 54, 58, 59, 60, 61], "few": [0, 2, 3, 7, 13, 21, 24, 27, 32, 34, 37, 38, 41, 45, 46, 47, 49, 50, 54, 59, 61], "exampl": [0, 2, 3, 4, 5, 7, 8, 11, 12, 14, 20, 22, 25, 26, 27, 28, 29, 31, 32, 36, 37, 39, 40, 42, 44, 47, 48, 49, 50, 51, 52, 60, 61], "know": [0, 2, 3, 5, 6, 7, 8, 10, 11, 14, 15, 16, 20, 21, 23, 25, 26, 27, 28, 29, 30, 33, 36, 39, 40, 41, 42, 44, 45, 47, 48, 51, 52, 53, 54, 59, 60, 61], "reduc": [0, 2, 3, 5, 7, 11, 12, 13, 14, 15, 17, 23, 24, 29, 31, 33, 34, 38, 49, 51, 52, 59, 60], "mass": [0, 2, 26, 39], "incarcer": [0, 2, 4], "busi": [0, 2, 3, 5, 8, 13, 34, 40, 42, 46, 47, 53, 54, 56, 59], "potenti": [0, 1, 2, 4, 7, 8, 11, 16, 18, 19, 21, 24, 25, 28, 29, 30, 38, 40, 47, 49, 51, 53, 57, 61], "custom": [0, 2, 3, 8, 12, 17, 47, 51, 59, 61], "who": [0, 1, 2, 3, 5, 7, 8, 10, 11, 14, 16, 17, 18, 19, 21, 22, 26, 27, 28, 29, 30, 31, 37, 39, 40, 41, 44, 45, 47, 48, 51, 54, 58, 59, 60, 61], "alzheim": [0, 2, 59], "test": [0, 2, 4, 5, 6, 7, 8, 13, 16, 18, 19, 21, 22, 25, 29, 34, 36, 37, 38, 41, 42, 44, 45, 46, 48, 49, 51, 61], "earli": [0, 2, 4, 6, 18, 19, 30, 52, 53, 61], "intervent": [0, 2, 3, 11, 47, 49, 59], "although": [0, 2, 11, 12], "everyon": [0, 1, 2, 11, 25, 28, 39, 40, 44, 47, 49, 54, 59], "u": [0, 1, 2, 4, 7, 8, 11, 13, 16, 17, 21, 22, 25, 28, 29, 30, 31, 32, 34, 37, 40, 41, 42, 47, 48, 49, 51, 52, 54, 59, 61], "fundament": [0, 2, 5, 18, 19, 22, 31, 37, 46, 47, 51, 59], "practic": [0, 2, 3, 4, 5, 6, 7, 13, 15, 33, 34, 45, 46, 48, 49, 53, 59], "us": [0, 1, 2, 3, 4, 5, 8, 9, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 31, 32, 33, 34, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 49, 51, 52, 54, 56, 59, 61], "quantifi": [0, 2], "world": [0, 1, 2, 3, 5, 7, 8, 12, 25, 27, 29, 36, 39, 40, 41, 43, 45, 46, 47, 49, 51, 53, 54, 59, 60], "For": [0, 2, 3, 7, 8, 11, 12, 13, 14, 16, 22, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 36, 37, 39, 40, 41, 42, 43, 44, 47, 48, 49, 50, 51, 52, 59, 61], "ran": [0, 2, 3, 5, 6, 8, 11, 13, 22, 34, 37, 43, 51, 53], "our": [0, 1, 2, 3, 5, 6, 8, 10, 11, 12, 13, 16, 17, 18, 20, 21, 22, 24, 25, 28, 29, 31, 32, 34, 36, 37, 38, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 56, 59, 61], "regress": [0, 1, 2, 5, 7, 8, 9, 10, 11, 13, 16, 20, 22, 24, 28, 29, 34, 36, 37, 38, 40, 41, 42, 43, 48, 50, 51, 52, 57, 61], "birth": [0, 2, 29, 41], "weight": [0, 2, 20, 21, 22, 26, 28, 36, 37, 39, 40, 42, 53], "variou": [0, 2, 29, 41], "demograph": [0, 2, 11, 30, 47, 54], "whether": [0, 1, 2, 3, 5, 6, 7, 8, 11, 12, 13, 14, 15, 20, 21, 22, 25, 26, 28, 29, 30, 32, 33, 34, 36, 37, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 50, 51, 52, 53, 59, 60, 61], "mother": [0, 2], "smoke": [0, 2, 7, 59], "dure": [0, 2, 5, 12, 13, 18, 25, 34, 51], "pregnanc": [0, 2], "were": [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 13, 18, 19, 20, 25, 27, 28, 29, 32, 34, 36, 39, 40, 41, 42, 47, 48, 49, 51, 52, 53, 59, 61], "matern": [0, 2], "associ": [0, 1, 2, 25, 29, 41, 48, 59], "lower": [0, 2, 7, 10, 14, 16, 28, 29, 34, 36, 38, 39, 40, 42, 48, 61], "signific": [0, 1, 2, 6, 11, 13, 22, 29, 34, 37, 41, 46, 48, 53, 59], "level": [0, 1, 2, 7, 11, 12, 15, 21, 24, 25, 29, 32, 33, 38, 40, 41, 42, 43, 44, 45, 47, 48, 50, 51, 54, 61], "after": [0, 1, 2, 5, 7, 8, 11, 13, 19, 21, 22, 23, 24, 25, 29, 34, 36, 37, 38, 41, 42, 43, 47, 48, 49, 51, 52, 58, 59], "control": [0, 1, 2, 3, 4, 5, 10, 11, 12, 13, 20, 22, 24, 26, 27, 28, 29, 34, 36, 37, 38, 39, 40, 41, 42, 43, 44, 47, 48, 49, 50, 51, 52, 53], "other": [0, 2, 3, 5, 7, 8, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 23, 24, 25, 28, 29, 30, 31, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 56, 59, 60, 61], "confound": [0, 2, 3, 18, 19], "someon": [0, 1, 2, 3, 5, 7, 12, 26, 29, 31, 39, 41, 42, 49, 58, 60, 61], "run": [0, 2, 3, 5, 6, 7, 8, 11, 13, 14, 21, 22, 23, 25, 28, 29, 32, 34, 37, 38, 40, 42, 44, 47, 48, 49, 50, 52, 54, 61], "commerc": [0, 2, 23], "websit": [0, 2, 6, 8, 23, 29, 41, 59], "user": [0, 2, 3, 4, 7, 8, 11, 13, 23, 32, 34, 41, 42, 47, 48, 49, 50, 53, 59, 60], "visit": [0, 2, 8, 11], "randomli": [0, 2, 3, 8, 11, 12, 29, 41, 49, 61], "assign": [0, 1, 2, 3, 4, 7, 8, 11, 22, 25, 29, 37, 41, 42, 43, 45, 46, 47, 49, 53, 56, 61], "see": [0, 1, 2, 3, 5, 6, 7, 8, 11, 13, 14, 17, 18, 19, 22, 25, 27, 29, 32, 33, 34, 36, 37, 39, 41, 42, 43, 44, 45, 46, 47, 48, 49, 51, 53, 54, 58, 59, 61], "two": [0, 2, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 33, 34, 37, 38, 39, 40, 41, 42, 44, 47, 49, 50, 51, 52, 56, 59, 61], "version": [0, 2, 9, 11, 12, 21, 22, 37, 47], "land": [0, 2, 3, 52], "page": [0, 2, 3, 11, 15, 23, 30, 33, 59, 61], "track": [0, 2, 11, 13, 17, 34, 47], "purchas": [0, 2, 25], "behavior": [0, 2, 3, 7, 8, 11, 16, 22, 32, 37, 47, 49, 59, 61], "analyz": [0, 2, 6, 11, 13, 24, 29, 34, 38, 41, 49, 51, 59, 60, 61], "re": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 21, 22, 23, 24, 26, 27, 28, 29, 30, 32, 33, 34, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 50, 51, 52, 53, 54, 58, 59, 61], "which": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 34, 37, 38, 39, 40, 41, 42, 44, 47, 48, 49, 50, 51, 52, 53, 54, 56, 59, 60, 61], "more": [0, 1, 2, 3, 5, 7, 8, 11, 12, 13, 14, 16, 17, 18, 20, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 33, 34, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 49, 50, 51, 52, 53, 54, 56, 58, 59, 60, 61], "effect": [0, 2, 3, 4, 6, 7, 12, 13, 14, 16, 21, 22, 23, 26, 27, 28, 34, 35, 37, 39, 40, 45, 46, 47, 51, 52, 53, 57, 59, 60, 61], "bui": [0, 2, 8, 25, 52, 59, 61], "train": [0, 2, 5, 25, 40, 46, 47, 52, 59, 60, 61], "algorithm": [0, 2, 5, 7, 22, 37, 45, 47, 50, 51, 52, 59], "cancer": [0, 2, 59], "x": [0, 2, 7, 9, 12, 21, 24, 32, 33, 34, 36, 37, 38, 48, 51, 59], "rai": [0, 2], "evalu": [0, 2, 3, 4, 5, 7, 11, 22, 23, 25, 30, 37, 44, 46, 47, 51, 53, 56, 59, 61], "its": [0, 2, 5, 7, 8, 13, 16, 21, 22, 23, 25, 27, 30, 34, 37, 42, 45, 47, 51, 52, 59], "perform": [0, 5, 7, 16, 25, 29, 35, 38, 40, 41, 42, 44, 45, 60], "patient": [0, 2, 5, 6, 7, 17, 27, 29, 47, 59], "well": [0, 2, 3, 5, 6, 7, 10, 13, 22, 23, 24, 29, 33, 34, 35, 37, 38, 41, 45, 46, 48, 50, 51, 52, 54, 59, 61], "next": [0, 2, 6, 7, 11, 22, 25, 28, 37, 40, 44, 47, 48], "ask": [0, 2, 7, 8, 11, 18, 25, 29, 30, 31, 41, 45, 47, 49, 51, 54, 56, 59, 60, 61], "yourself": [0, 2, 3, 5, 7, 9, 20, 22, 24, 37, 45, 47], "motiv": [0, 2, 3, 21, 46, 61], "The": [0, 2, 4, 5, 6, 7, 12, 13, 14, 18, 19, 20, 21, 22, 24, 28, 29, 30, 31, 32, 34, 35, 36, 37, 38, 40, 41, 42, 43, 44, 45, 46, 47, 50, 54, 56, 58, 59, 60, 61], "part": [0, 2, 6, 7, 11, 18, 21, 23, 28, 29, 35, 40, 41, 42, 45, 46, 47, 50, 59, 61], "character": [0, 5, 21], "predict": [0, 2, 4, 14, 16, 17, 21, 22, 37, 47, 48, 51, 52, 54, 60], "label": [0, 1, 2, 7, 15, 21, 22, 37, 52, 54, 59], "unlabel": [0, 2], "form": [0, 5, 9, 12, 21, 23, 25, 28, 40, 42, 51, 52, 59], "think": [0, 2, 3, 5, 7, 8, 11, 14, 16, 18, 20, 22, 24, 25, 26, 27, 29, 30, 31, 32, 36, 37, 38, 39, 41, 42, 43, 44, 45, 47, 48, 49, 50, 51, 52, 56, 58, 59, 60, 61], "entiti": [0, 2, 7, 13, 24, 34, 38, 42, 43, 47], "observ": [0, 1, 2, 3, 7, 11, 12, 13, 16, 18, 19, 21, 22, 24, 25, 29, 32, 34, 36, 37, 38, 40, 41, 42, 43, 44, 46, 48, 49, 50, 51, 53, 59, 61], "all": [0, 1, 2, 3, 5, 7, 9, 11, 13, 14, 15, 18, 19, 20, 21, 22, 23, 25, 26, 27, 28, 29, 32, 33, 34, 36, 37, 38, 39, 40, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 54, 58, 59, 60, 61], "replic": [0, 2, 7, 11, 25], "gave": [0, 2, 7, 8, 17, 22, 31, 37, 49], "rise": [0, 2, 7, 8, 59], "due": [0, 2, 3, 10, 25, 39, 40, 44, 56, 60, 61], "pictur": [0, 2, 40], "anim": [0, 2, 40], "feed": [0, 2, 3, 7, 50], "bunch": [0, 2, 47], "undergradu": [0, 1, 2, 47], "american": [0, 1, 2, 5, 29, 41, 42, 47, 54], "univers": [0, 2, 16, 29, 30, 40, 41, 47, 48, 59], "photo": [0, 2, 5], "everi": [0, 1, 2, 3, 7, 23, 25, 26, 32, 36, 49, 59, 61], "time": [0, 1, 2, 3, 6, 7, 8, 9, 11, 13, 16, 20, 21, 22, 23, 24, 25, 26, 28, 29, 30, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 50, 51, 53, 61], "photograph": [0, 2], "obvious": [0, 2, 3, 7, 9, 11, 13, 15, 16, 21, 22, 25, 29, 33, 34, 37, 41, 48, 52, 61], "success": [0, 2, 7, 11, 47, 56], "depend": [0, 2, 3, 5, 12, 16, 21, 24, 29, 30, 37, 38, 41, 42, 43, 47, 48, 49, 53, 56, 59, 61], "context": [0, 2, 26, 27, 28, 39, 40, 47, 53, 54, 59, 61], "much": [0, 1, 2, 3, 7, 11, 13, 17, 20, 21, 22, 26, 28, 29, 34, 36, 37, 39, 40, 41, 42, 44, 45, 46, 47, 48, 51, 52, 54, 59, 60], "studi": [0, 1, 2, 3, 5, 6, 7, 12, 17, 29, 30, 41, 42, 45, 46, 47, 51, 53, 54, 59, 61], "cours": [0, 2, 3, 4, 5, 11, 22, 37, 45, 46, 47, 48, 50, 53, 56, 59, 60, 61], "aspir": [0, 2], "bit": [0, 2, 36, 38, 41, 59, 61], "recogn": [0, 2, 22, 37, 44, 45, 48, 59], "understand": [0, 2, 3, 5, 6, 7, 16, 18, 23, 24, 30, 32, 38, 42, 43, 44, 45, 46, 47, 49, 51, 59, 61], "limit": [0, 2, 5, 7, 12, 13, 21, 34, 41, 42, 43, 44, 45, 47, 51, 61], "surprisingli": [0, 2], "long": [0, 1, 2, 3, 4, 6, 7, 13, 16, 30, 34, 35, 36, 42, 45, 47, 53, 56], "thought": [0, 2, 3, 5, 7, 30, 42, 45, 47, 56], "incap": [0, 2], "harbor": [0, 2, 12], "racial": [0, 1, 2, 5, 7, 22, 29, 32, 37, 41], "sexist": [0, 2], "prejudic": [0, 2], "just": [0, 1, 2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20, 21, 22, 23, 24, 25, 28, 32, 33, 34, 36, 37, 38, 40, 42, 43, 44, 45, 47, 48, 49, 50, 51, 53, 54, 58, 59, 60, 61], "built": [0, 2, 54], "math": [0, 2, 6, 10, 23, 44, 45, 48, 49], "racist": [0, 2, 5, 47], "compani": [0, 2, 3, 5, 7, 11, 12, 14, 16, 21, 25, 27, 30, 40, 47, 49, 52, 54, 60], "amazon": [0, 2], "tri": [0, 2, 22, 37, 47, 51], "build": [0, 2, 7, 16, 25, 31, 40, 52, 59], "decid": [0, 1, 2, 3, 5, 11, 13, 26, 34, 39, 47, 50, 52, 53, 61], "hire": [0, 2, 3, 5, 7, 12, 16, 17, 25, 29, 31, 41, 61], "though": [0, 1, 2, 3, 5, 7, 11, 13, 15, 21, 22, 24, 25, 33, 34, 37, 38, 41, 42, 43, 45, 46, 48, 50, 59, 60], "human": [0, 2, 3, 7, 18, 19, 40, 47, 59], "employe": [0, 2, 3, 27, 28, 29, 40], "had": [0, 2, 6, 8, 11, 12, 13, 22, 25, 34, 37, 40, 41, 43, 46, 47, 48, 49, 59, 60], "past": [0, 2, 3, 7, 8, 11, 13, 22, 23, 27, 29, 34, 37, 45, 52, 54, 59, 60], "subject": [0, 2, 3, 11, 13, 25, 28, 34, 40, 44, 45, 46, 47, 53], "made": [0, 2, 5, 7, 8, 18, 19, 22, 37, 46, 51, 56], "supervisor": [0, 2], "resum": [0, 2, 57], "itself": [0, 1, 2, 5, 16, 23, 56, 58, 59], "veri": [0, 1, 2, 3, 6, 7, 9, 12, 13, 20, 25, 28, 30, 33, 34, 36, 37, 40, 41, 42, 45, 47, 49, 50, 51, 52, 54, 56, 59, 61], "staff": [0, 2], "person": [0, 1, 2, 3, 5, 7, 11, 16, 21, 27, 29, 30, 32, 40, 41, 47, 48, 49, 60, 61], "inherit": [0, 2], "bias": [0, 2, 5, 7, 44], "oop": [0, 2], "suddenli": [0, 2, 40, 47], "realiz": [0, 2, 7, 9, 18, 25, 47, 59], "recruit": [0, 2, 47], "engin": [0, 2, 21, 25, 40, 45], "didn": [0, 2, 4, 7, 11, 29, 41, 47, 52, 59], "women": [0, 2, 5, 7, 10, 28, 29, 40, 41, 47, 48, 49], "abandon": [0, 2], "ok": [0, 2, 3, 5, 7, 11, 22, 37, 38, 42, 44, 47, 48, 50, 51, 61], "bia": [0, 1, 2, 4, 5, 7, 22, 37, 42], "now": [0, 1, 2, 3, 5, 7, 8, 9, 11, 13, 15, 17, 21, 22, 24, 26, 27, 28, 29, 32, 33, 34, 37, 38, 39, 40, 41, 42, 43, 45, 48, 49, 50, 51, 52, 59, 61], "open": [1, 3, 11, 16, 29, 41, 54], "book": [1, 23, 40, 45, 59], "commun": [1, 2, 5, 7, 12, 30, 40, 45, 47, 49, 54, 61], "survei": [1, 2, 14, 22, 28, 29, 37, 40, 41, 42, 48, 51, 54], "ac": [1, 54], "relationship": [1, 9, 12, 13, 15, 21, 22, 24, 28, 32, 33, 34, 37, 38, 40, 42, 43, 48, 51, 59], "between": [1, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 24, 25, 28, 29, 30, 32, 33, 34, 36, 37, 38, 39, 40, 42, 43, 45, 49, 50, 51, 53, 56, 59, 61], "attain": [1, 28, 40], "wage": [1, 10, 28, 40, 42, 59], "conduct": [1, 30, 47], "unit": [1, 5, 11, 12, 13, 18, 19, 20, 22, 24, 25, 28, 29, 32, 34, 36, 37, 38, 40, 41, 49, 52, 53, 54, 59], "state": [1, 2, 5, 12, 13, 20, 24, 25, 28, 29, 30, 32, 34, 36, 38, 40, 41, 52, 54, 59, 60], "censu": [1, 2, 32, 54], "bureau": [1, 2, 31, 54], "count": [1, 5, 11, 13, 25, 29, 32, 33, 34, 37, 41, 49, 59], "take": [1, 3, 5, 7, 8, 9, 10, 11, 12, 20, 21, 22, 23, 24, 25, 36, 37, 38, 42, 44, 45, 46, 48, 49, 50, 51, 53, 59, 61], "place": [1, 2, 5, 7, 11, 13, 18, 19, 21, 25, 28, 29, 30, 32, 34, 40, 42, 46, 47, 49, 50, 51, 52, 54, 58, 59], "10": [1, 4, 5, 8, 15, 24, 26, 27, 32, 33, 34, 36, 38, 39, 42, 48], "year": [1, 5, 13, 17, 22, 23, 24, 25, 27, 28, 29, 30, 32, 33, 34, 37, 38, 40, 41, 43, 45, 46, 48, 49, 51, 54, 59], "numer": [1, 11, 18, 19, 22, 24, 36, 37, 38, 42], "featur": [1, 2, 5, 16, 21, 22, 25, 29, 37, 41, 49, 52, 59, 60], "popul": [1, 2, 8, 13, 22, 24, 26, 27, 28, 30, 32, 34, 37, 38, 39, 40, 41, 42, 47, 49, 51, 53, 60], "includ": [1, 5, 7, 8, 11, 13, 15, 16, 21, 22, 24, 25, 26, 28, 29, 30, 33, 34, 35, 37, 38, 40, 41, 42, 43, 48, 49, 50, 51, 54, 59, 60, 61], "100": [1, 5, 8, 13, 15, 21, 23, 33, 34, 37, 41, 42], "2017": [1, 32, 34, 37], "sampl": [1, 2, 7, 8, 13, 14, 16, 21, 22, 25, 28, 29, 30, 32, 34, 36, 37, 40, 41, 42, 44, 47, 49, 51, 53, 59, 61], "1": [1, 4, 5, 8, 23, 26, 27, 32, 39, 42, 43, 47, 48, 49, 51, 56], "0": [1, 5, 8, 13, 15, 20, 21, 22, 23, 25, 26, 27, 29, 32, 33, 34, 36, 37, 38, 39, 40, 41, 42, 43, 48, 56], "ipum": [1, 54], "tool": [1, 2, 3, 5, 11, 16, 21, 22, 23, 24, 37, 38, 40, 42, 43, 46, 48, 52, 56, 59], "major": [1, 5, 7, 16, 29, 41, 45, 46, 48, 53, 59, 60], "dataset": [1, 2, 5, 9, 11, 13, 16, 20, 24, 25, 28, 29, 32, 34, 36, 38, 40, 41, 42, 44, 48, 50, 51, 54], "govern": [1, 2, 3, 20, 30, 36, 40, 47, 52, 54, 59], "agenc": [1, 52], "real": [1, 4, 5, 7, 9, 10, 12, 16, 21, 25, 29, 30, 40, 41, 45, 46, 47, 49, 59, 60, 61], "mean": [1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 51, 52, 54, 56, 59, 61], "you": [1, 5, 6, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 49, 50, 51, 52, 54, 56, 59, 60, 61], "being": [1, 2, 3, 5, 7, 11, 13, 16, 21, 22, 25, 26, 27, 28, 29, 34, 35, 36, 37, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 51, 59, 60, 61], "document": [1, 16, 20, 21, 25, 36, 59], "found": [1, 3, 7, 11, 13, 14, 17, 21, 22, 25, 26, 28, 34, 37, 39, 40, 44, 45, 46, 47, 56, 59], "either": [1, 2, 3, 5, 11, 21, 22, 30, 37, 43, 46, 47, 49, 51, 54, 61], "search": [1, 21, 29, 40, 41, 54], "someth": [1, 3, 5, 7, 8, 11, 13, 18, 19, 21, 22, 25, 34, 35, 37, 43, 45, 47, 48, 49, 50, 51, 52, 53, 54, 56, 58, 59, 60, 61], "right": [1, 4, 7, 19, 20, 22, 30, 32, 36, 37, 44, 47, 48, 50, 51, 52, 56, 59, 61], "within": [1, 3, 10, 23, 24, 25, 28, 32, 34, 38, 40, 42, 44, 49, 56, 59], "inform": [1, 2, 5, 9, 11, 18, 19, 21, 22, 25, 33, 37, 40, 45, 47, 52, 54, 56, 60], "background": [1, 7, 45], "statu": [1, 25, 60], "repres": [1, 2, 5, 7, 11, 12, 32, 41, 45, 46, 47, 52], "colleg": [1, 22, 29, 37, 40, 41, 42], "degre": [1, 22, 29, 37, 41, 47], "ones": [1, 7, 37], "abil": [1, 7, 16, 33, 45], "job": [1, 3, 5, 7, 22, 26, 27, 28, 29, 32, 37, 39, 40, 41, 52, 56, 61], "download": [1, 2, 13, 15, 20, 22, 24, 25, 28, 29, 32, 33, 34, 36, 37, 38, 40, 41], "us_acs_2017_10pct_sampl": 1, "dta": [1, 20, 22, 28, 29, 36, 37, 40, 41, 42], "panda": [1, 11, 32, 33, 34, 36, 37, 38, 40, 41, 42, 48], "datafram": [1, 22, 32, 37], "pd": [1, 22, 29, 32, 33, 34, 36, 37, 38, 40, 41, 42, 48], "read_stata": [1, 22, 36, 37, 40, 41, 42], "normal": [1, 5, 6, 8, 11, 22, 23, 24, 37, 38, 42, 50, 54], "d": [1, 2, 5, 6, 8, 11, 12, 13, 21, 22, 23, 25, 27, 32, 34, 37, 39, 40, 42, 43, 47, 48, 49, 50, 51, 52, 61], "tell": [1, 5, 7, 8, 11, 13, 17, 22, 26, 27, 28, 32, 34, 36, 37, 39, 40, 42, 47, 48, 49, 52, 54, 59, 61], "dive": [1, 2, 5, 7, 52, 59], "mid": [1, 16, 30, 44], "ll": [1, 2, 3, 5, 7, 8, 11, 13, 15, 20, 21, 22, 23, 24, 28, 29, 32, 33, 34, 36, 37, 38, 40, 41, 42, 43, 44, 46, 47, 48, 49, 50, 51, 52, 54, 59, 61], "spare": 1, "wrang": 1, "analysi": [1, 3, 7, 8, 11, 13, 18, 19, 21, 22, 25, 28, 34, 37, 40, 49, 51, 53, 59, 60, 61], "ag": [1, 2, 5, 10, 11, 21, 22, 28, 37, 40, 42, 43, 45, 47, 48], "empstat": 1, "inctot": 1, "educd": 1, "statefip": 1, "countyfip": 1, "sex": [1, 21, 28, 40, 42], "race": [1, 7, 22, 32, 37, 42, 48, 54], "hispan": [1, 22, 32, 37], "thin": [1, 36], "3": [1, 4, 8, 17, 26, 27, 32, 39, 42, 48, 49], "creat": [1, 5, 6, 7, 13, 15, 16, 20, 21, 22, 23, 25, 29, 33, 34, 36, 37, 41, 42, 45, 47, 48, 59, 61], "indic": [1, 2, 4, 7, 11, 13, 24, 25, 28, 29, 32, 34, 38, 40, 41, 47, 53, 57], "respond": [1, 8, 28, 29, 39, 40, 42, 47, 48, 54], "color": [1, 3, 20, 23, 30, 34, 36, 47], "A": [1, 4, 5, 6, 8, 12, 14, 16, 21, 23, 25, 41, 45, 46, 47, 49, 52, 54, 56, 59, 61], "NOT": [1, 34, 48, 54], "white": [1, 5, 7, 22, 29, 32, 37, 41, 42, 47, 48], "non": [1, 2, 3, 5, 7, 16, 18, 20, 22, 25, 28, 30, 31, 36, 37, 40, 42, 43, 48], "individu": [1, 11, 12, 16, 18, 19, 29, 40, 41, 42, 43, 44, 48, 49, 54], "again": [1, 3, 8, 11, 15, 22, 24, 25, 29, 33, 37, 38, 41, 42, 47, 48, 49, 52], "poc": 1, "categori": [1, 13, 20, 21, 22, 25, 29, 32, 36, 37, 41], "note": [1, 2, 4, 7, 11, 12, 13, 18, 20, 21, 22, 23, 24, 25, 28, 29, 32, 34, 36, 37, 38, 40, 41, 42, 44, 45, 46, 48, 49, 50, 51, 52, 54, 56, 60, 61], "pleas": [1, 11, 16, 22, 25, 26, 27, 28, 29, 30, 32, 37, 39, 40, 41, 45, 46, 50, 56, 59, 61], "excus": [1, 49, 56], "harmon": 1, "code": [1, 7, 9, 11, 16, 20, 21, 22, 24, 25, 28, 29, 32, 36, 37, 40, 41, 42, 46, 48, 54, 58], "across": [1, 3, 8, 10, 11, 13, 18, 19, 22, 28, 29, 30, 34, 37, 38, 40, 41, 42, 43, 49, 53, 56, 59, 61], "period": [1, 13, 21, 24, 25, 34, 38, 42, 43, 45, 47, 59], "list": [1, 7, 22, 25, 34, 35, 37, 42, 59, 60], "black": [1, 4, 5, 7, 21, 22, 29, 32, 37, 38, 41, 42, 47, 59], "appropri": [1, 11, 25, 32, 46, 50, 60, 61], "2018": [1, 13, 28, 32, 34, 40, 49, 59], "4": [1, 4, 17, 26, 27, 30, 32, 39, 42, 48, 49, 56], "interest": [1, 2, 5, 7, 11, 12, 14, 17, 18, 19, 22, 25, 28, 29, 31, 35, 37, 40, 41, 42, 43, 44, 45, 46, 47, 48, 50, 52, 53, 56, 59, 60, 61], "emploi": [1, 10, 22, 28, 37, 40, 42, 46, 59], "seek": [1, 27, 29, 30, 41, 61], "restrict": [1, 22, 35, 37, 61], "unemploi": 1, "exclud": [1, 21, 22, 37, 45], "anyon": [1, 2, 3, 5, 7, 18, 19, 28, 29, 40, 41, 48, 49, 58], "doesn": [1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 21, 22, 32, 37, 39, 40, 42, 48, 49, 51, 53, 59, 61], "answer": [1, 4, 5, 6, 8, 10, 11, 12, 17, 20, 22, 25, 26, 27, 28, 29, 30, 31, 32, 34, 36, 37, 39, 40, 41, 43, 44, 45, 46, 47, 48, 49, 51, 52, 56, 58, 59, 61], "aren": [1, 2, 3, 5, 6, 7, 8, 11, 21, 22, 23, 25, 26, 28, 32, 37, 39, 40, 42, 44, 45, 49, 50, 51, 52, 54, 59, 61], "5": [1, 4, 5, 17, 23, 26, 27, 32, 39, 42, 48, 51, 56], "also": [1, 2, 3, 5, 7, 11, 12, 13, 14, 16, 17, 18, 19, 20, 21, 22, 24, 28, 29, 30, 32, 34, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 56, 59, 60, 61], "least": [1, 2, 3, 21, 22, 25, 26, 28, 30, 34, 36, 37, 38, 40, 41, 42, 46, 47, 48, 50, 53, 56, 59, 60], "requir": [1, 4, 5, 8, 11, 13, 16, 18, 21, 27, 30, 34, 42, 44, 45, 46, 48, 49, 52, 53, 54, 56, 59, 60, 61], "whose": [1, 8, 13, 31, 34, 41, 59], "record": [1, 7, 20, 25, 36, 40], "90": [1, 5, 21, 23, 25, 32, 36, 37, 48], "1980": [1, 32, 34, 40], "1990": [1, 15, 32, 33, 37, 40], "wa": [1, 3, 5, 6, 7, 11, 12, 13, 15, 16, 18, 19, 21, 22, 25, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 40, 41, 42, 43, 47, 49, 52, 53, 59, 61], "might": [1, 2, 3, 5, 7, 8, 11, 12, 16, 20, 22, 25, 26, 27, 29, 30, 31, 32, 33, 36, 37, 39, 40, 41, 42, 43, 44, 47, 49, 52, 53, 59, 60, 61], "old": [1, 11, 22, 37, 42, 48, 51, 52], "census": [1, 54], "older": [1, 5, 28, 40, 47, 48], "e": [1, 3, 5, 7, 8, 11, 12, 13, 15, 16, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 43, 44, 45, 46, 47, 51, 53, 54, 59, 60, 61], "ani": [1, 3, 5, 7, 8, 11, 12, 13, 16, 21, 22, 23, 25, 28, 29, 32, 34, 37, 40, 41, 42, 43, 45, 47, 49, 50, 51, 52, 53, 54, 56, 59, 60, 61], "abov": [1, 2, 7, 8, 12, 13, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 34, 37, 38, 39, 40, 41, 42, 43, 47, 48, 50, 51, 52, 54, 59, 60], "deal": [1, 3, 23, 36, 47, 48, 52, 58, 60], "worri": [1, 7, 8, 11, 12, 18, 19, 21, 23, 26, 29, 37, 39, 41, 52, 59], "parenthes": 1, "except": [1, 5, 7, 12, 22, 32, 37, 40, 42, 44, 48, 50, 53, 56], "insofar": 1, "littl": [1, 5, 14, 21, 22, 32, 37, 38, 41, 43, 47, 48, 49, 51], "complic": [1, 3, 6, 7, 56, 59], "6": [1, 4, 23, 26, 27, 32, 39, 42, 48, 49, 50], "categor": [1, 21, 22, 24, 25, 29, 37, 38, 41, 42, 48], "highest": [1, 42], "high": [1, 7, 10, 12, 13, 14, 16, 21, 24, 25, 28, 29, 34, 38, 40, 41, 43, 44, 47, 50, 51, 61], "school": [1, 7, 10, 28, 29, 30, 40, 41, 42, 43, 44, 59], "diploma": 1, "equival": [1, 11, 44], "ged": 1, "altern": [1, 7, 18, 19, 61], "credenti": 1, "better": [1, 3, 6, 7, 22, 25, 29, 30, 37, 41, 43, 44, 45, 46, 47, 50, 56, 61], "ha": [1, 2, 3, 5, 7, 9, 11, 13, 14, 16, 21, 22, 25, 27, 28, 29, 30, 32, 33, 34, 37, 40, 41, 42, 43, 44, 45, 47, 48, 49, 50, 51, 52, 53, 54, 59, 60, 61], "neither": [1, 39, 48], "nor": [1, 45], "bachelor": [1, 29, 41], "assum": [1, 5, 7, 8, 14, 16, 21, 23, 24, 25, 28, 29, 36, 37, 38, 40, 41, 42, 43, 44, 46, 51, 52, 59, 61], "advanc": [1, 3, 7, 16, 22, 37, 46, 49, 53, 56], "master": [1, 15, 22, 28, 33, 34, 36, 37, 38, 40, 41, 42, 46, 47, 48], "doctor": [1, 27], "while": [1, 3, 5, 7, 8, 9, 11, 12, 13, 16, 21, 25, 29, 30, 31, 34, 37, 42, 45, 46, 47, 48, 49, 51, 54, 56, 59, 60], "number": [1, 11, 12, 13, 15, 16, 21, 22, 24, 25, 28, 29, 30, 32, 33, 34, 36, 37, 38, 40, 41, 42, 47], "encod": [1, 21, 25, 32, 33, 34, 36, 37, 42, 48], "implement": [1, 13, 21, 22, 24, 26, 29, 30, 34, 37, 38, 39, 40, 41, 50, 51], "properli": [1, 33, 47, 52, 59], "below": [1, 5, 7, 14, 16, 18, 19, 21, 22, 25, 26, 27, 28, 37, 39, 40, 42, 43, 47, 48, 51, 54, 59], "fine": [1, 5, 6, 7, 13, 34, 45, 47, 51, 56], "suggest": [1, 11, 23, 26, 39, 40, 41, 53], "singl": [1, 3, 9, 11, 13, 21, 22, 23, 24, 25, 32, 34, 37, 38, 44, 48, 52, 60, 61], "three": [1, 2, 13, 21, 23, 25, 31, 34, 48, 50, 54, 59, 60, 61], "each": [1, 2, 6, 9, 11, 12, 13, 14, 15, 22, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 37, 38, 39, 40, 41, 42, 43, 44, 45, 49, 50, 52, 54, 56, 59], "group": [1, 2, 8, 11, 12, 13, 14, 17, 18, 19, 22, 24, 26, 27, 28, 29, 30, 31, 34, 37, 38, 39, 40, 41, 43, 44, 45, 47, 48, 49, 51, 52, 59], "refer": [1, 11, 12, 13, 25, 34, 35, 48, 50, 51, 56, 59, 61], "graduat": [1, 28, 29, 40, 41], "interchang": 1, "post": [1, 2, 3, 13, 31, 34, 48, 59, 60], "treatment": [1, 3, 5, 6, 11, 12, 13, 14, 18, 19, 22, 26, 27, 29, 34, 37, 39, 40, 41, 43, 46, 47, 48, 50, 51, 53, 59], "outcom": [1, 2, 3, 4, 7, 8, 11, 12, 13, 14, 17, 18, 19, 22, 25, 28, 34, 37, 40, 43, 44, 45, 47, 49, 51, 53, 57, 59, 61], "monei": [1, 3, 8, 14, 23, 27, 49, 52], "mechan": [1, 16, 28, 40, 42], "phd": [1, 21], "got": [1, 8, 22, 23, 29, 37, 47, 51, 61], "under": [1, 3, 11, 22, 23, 25, 29, 32, 37, 44, 48, 52, 60], "sinc": [1, 2, 7, 8, 11, 13, 22, 28, 29, 30, 32, 34, 37, 40, 41, 42, 43, 44, 47, 48, 49, 51, 54, 59, 61], "without": [1, 2, 5, 7, 11, 14, 20, 22, 24, 25, 29, 35, 36, 37, 38, 40, 42, 46, 49, 53, 61], "given": [1, 2, 5, 7, 8, 11, 12, 13, 14, 15, 16, 18, 19, 21, 22, 23, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 37, 39, 40, 41, 43, 44, 45, 46, 47, 48, 49, 52, 53, 56, 59], "gone": [1, 7, 42, 54], "7": [1, 4, 23, 26, 27, 32, 39, 42, 48, 49], "simpl": [1, 4, 5, 7, 9, 11, 13, 17, 24, 26, 27, 34, 38, 39, 42, 43, 45, 48, 50, 51, 59], "likelihood": [1, 7, 25, 29, 34, 36, 37, 38, 40, 41, 42, 48, 52, 59], "linear": [1, 5, 7, 9, 11, 16, 21, 22, 24, 28, 29, 37, 38, 40, 41, 42, 43, 46, 48, 51], "probabl": [1, 5, 7, 8, 9, 11, 21, 22, 23, 25, 28, 29, 32, 37, 40, 41, 44, 47, 48, 49, 50, 51, 52, 53, 54, 59, 61], "squar": [1, 22, 28, 29, 34, 36, 37, 38, 40, 41, 42, 43, 48, 50], "updat": [1, 22, 32, 37], "languag": [1, 11, 18, 19, 20, 21, 36, 45, 46, 51, 59, 61], "oppos": [1, 28, 29, 40, 41, 60], "could": [1, 2, 5, 7, 8, 11, 14, 16, 18, 19, 22, 25, 28, 29, 30, 31, 32, 37, 39, 40, 41, 42, 47, 48, 50, 52, 59, 61], "8": [1, 4, 26, 27, 32, 34, 39, 42, 48, 49], "add": [1, 5, 10, 11, 16, 21, 24, 28, 29, 32, 38, 40, 41, 42, 43, 48, 50, 60], "appar": [1, 6, 7, 22, 37, 45, 51], "compar": [1, 2, 3, 8, 10, 11, 13, 16, 18, 19, 22, 24, 25, 26, 28, 29, 30, 34, 35, 37, 38, 39, 40, 41, 49, 53, 59, 61], "9": [1, 4, 23, 26, 27, 32, 34, 36, 39, 48], "formula": [1, 34, 36, 37, 38, 40, 41, 42, 48], "omit": [1, 42, 48], "coeffici": [1, 2, 10, 20, 21, 22, 28, 29, 36, 37, 40, 42, 43, 48], "neg": [1, 2, 6, 11, 21, 25, 26, 29, 39, 48, 51], "chang": [1, 5, 11, 13, 15, 21, 22, 23, 26, 28, 30, 32, 33, 34, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 49, 51, 53, 59, 60, 61], "correl": [1, 5, 7, 8, 9, 12, 14, 15, 21, 24, 33, 34, 36, 38, 43, 44, 59], "explain": [1, 16, 20, 21, 24, 28, 29, 36, 38, 40, 42, 52, 56, 59, 61], "learn": [1, 3, 4, 5, 11, 13, 16, 17, 18, 21, 22, 30, 34, 37, 44, 45, 46, 47, 48, 52, 53, 56, 58, 59, 61], "wrong": [1, 5, 7, 8, 9, 21, 29, 32, 41, 48, 51, 56], "assumpt": [1, 3, 4, 7, 18, 19, 44, 47, 49, 56], "causal": [1, 3, 4, 8, 12, 15, 17, 22, 28, 33, 37, 40, 45, 46, 47, 56, 60, 61], "violat": [1, 8, 49], "11": [1, 4, 26, 27, 32, 33, 34, 36, 38, 39, 46, 48], "final": [2, 3, 5, 7, 11, 12, 13, 15, 21, 22, 23, 25, 28, 33, 34, 37, 40, 45, 46, 48, 50, 56, 59, 60, 61], "narrow": [2, 22, 30, 37], "small": [2, 3, 7, 11, 13, 22, 25, 29, 33, 34, 37, 41, 44, 47, 49, 51, 53, 54], "conceptu": [2, 18, 26, 27, 39, 49], "framework": [2, 4, 7, 8, 11, 18, 19, 21, 29, 43, 49], "kei": [2, 11, 16, 25, 29, 30, 32, 41, 45, 47, 61], "tractabl": 2, "meet": [2, 16, 17, 21, 26, 27, 30, 35, 39, 45, 46], "criteria": [2, 4, 49], "directli": [2, 3, 11, 18, 22, 25, 26, 27, 29, 37, 39, 41, 44, 48, 59], "approach": [2, 16, 24, 26, 38, 39, 42, 45, 46, 51, 52, 56, 59, 61], "seem": [2, 5, 7, 11, 12, 17, 22, 33, 37, 41, 45, 47, 48, 49, 51, 52, 53, 59], "vagu": [2, 52], "immedi": [2, 21, 28, 31, 40, 52, 59], "collect": [2, 3, 5, 11, 25, 26, 39, 48, 50, 52, 59], "To": [2, 4, 6, 8, 11, 13, 14, 20, 21, 26, 28, 29, 30, 32, 34, 36, 39, 40, 42, 46, 47, 48, 49, 50, 51, 56, 59, 61], "illustr": [2, 5, 6, 7, 9, 42, 47, 50, 51, 59], "bad": [2, 3, 6, 17, 22, 37, 47, 51, 59], "describ": [2, 7, 11, 12, 13, 16, 18, 19, 22, 23, 26, 27, 29, 30, 34, 37, 39, 41, 42, 47, 59], "polici": [2, 5, 13, 14, 26, 31, 34, 39, 45, 47, 59, 61], "me": [2, 7, 21, 26, 27, 38, 39, 44, 48, 54], "By": [2, 3, 9, 22, 24, 30, 37, 38, 48, 49, 59], "contrast": [2, 5, 47, 48, 49, 59], "avail": [2, 3, 5, 7, 11, 16, 18, 19, 22, 23, 25, 30, 37, 46, 47, 52, 54, 56], "grand": 2, "juri": 2, "longer": [2, 24, 34, 38, 42, 43, 49, 53], "sentenc": [2, 45, 56], "pool": [2, 24, 38], "citizen": [2, 37], "prosecutor": 2, "guidanc": [2, 11, 16, 25, 29], "theori": [2, 4, 13, 18, 19, 34, 51, 59], "suppos": [2, 3, 5, 7, 8, 11, 21, 22, 23, 37, 42, 43, 44, 47, 48, 51, 52, 59], "hold": [2, 3, 8, 26, 27, 29, 39, 41, 47], "account": [2, 3, 7, 11, 18, 19, 22, 23, 28, 36, 37, 40, 42, 43, 44, 48], "said": [2, 8, 41, 43, 47, 48, 49, 59, 60, 61], "prosector": 2, "shape": [2, 7, 21, 25, 30, 59], "reach": [2, 7, 49, 52, 61], "recommend": [2, 11, 23, 32, 47, 61], "begin": [2, 7, 8, 11, 13, 15, 16, 22, 24, 25, 28, 30, 33, 34, 37, 38, 40, 42, 46, 59, 60, 61], "attribut": [2, 8, 11, 29, 36, 41, 47], "lab": [2, 22, 37, 47], "later": [2, 5, 21, 28, 31, 40, 41, 42, 51, 59, 61], "diagnos": [2, 9], "mortem": 2, "ve": [2, 5, 7, 8, 9, 11, 14, 15, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 28, 29, 33, 37, 39, 40, 41, 42, 44, 45, 47, 48, 49, 50, 51, 52, 53, 54, 58, 59, 61], "clear": [2, 5, 6, 7, 11, 13, 22, 30, 31, 34, 37, 42, 46, 49, 52, 54, 56, 59, 60, 61], "second": [2, 3, 7, 9, 16, 21, 22, 25, 28, 29, 32, 37, 40, 41, 42, 44, 45, 47, 48, 52, 60, 61], "find": [2, 3, 5, 7, 8, 11, 12, 13, 14, 18, 19, 21, 22, 23, 25, 28, 29, 34, 37, 39, 40, 41, 43, 44, 45, 47, 48, 49, 51, 52, 56, 59, 60], "current": [2, 11, 14, 16, 22, 23, 25, 28, 37, 40, 42, 59], "comparison": [2, 7, 13, 16, 18, 19, 28, 34, 40], "last": [2, 11, 13, 22, 25, 29, 32, 34, 35, 37, 44, 50], "moreov": [2, 5, 7, 23, 24, 38, 45, 47, 48, 56, 58], "imagin": [2, 5, 8, 11, 16, 47, 59], "crime": [2, 9, 31, 32, 34, 57], "commit": [2, 11, 31, 52], "tabl": [2, 28, 29, 37, 40, 41, 52, 61], "characterist": [2, 20, 29, 36, 41, 50], "It": [2, 3, 4, 7, 8, 9, 13, 16, 17, 22, 26, 27, 29, 32, 33, 34, 37, 39, 41, 42, 43, 44, 45, 46, 47, 48, 49, 51, 53, 54, 59, 61], "save": [2, 7, 8, 9, 22, 24, 27, 37, 38, 40], "lost": 2, "energi": [2, 31, 54], "Being": 2, "abl": [2, 5, 7, 8, 11, 16, 18, 21, 22, 25, 29, 32, 37, 41, 45, 46, 51, 54, 59, 61], "allow": [2, 3, 11, 13, 18, 21, 22, 25, 27, 29, 34, 37, 41, 42, 44, 45, 47, 48, 53, 60], "statehold": 2, "happen": [2, 3, 7, 8, 10, 11, 16, 19, 22, 24, 28, 37, 38, 40, 42, 43, 44, 45, 47, 49, 52, 59], "serious": [2, 5, 52], "Not": [2, 17, 37, 40, 49], "abstractli": 2, "draw": [2, 7, 24, 38, 45, 51, 56, 61], "graph": [2, 20, 36, 52], "column": [2, 11, 16, 21, 22, 24, 25, 32, 34, 37, 38, 40, 42, 54], "predictor": [2, 7, 16, 25], "diagnost": [2, 52, 61], "liter": [2, 5, 43, 48, 58], "ax": [2, 52], "etc": [2, 7, 12, 16, 22, 25, 26, 27, 37, 39, 52, 53, 54, 59, 61], "turn": [2, 3, 7, 16, 20, 30, 36, 40, 42, 47, 48, 51, 59], "wasn": [2, 22, 29, 37, 41, 47], "suffici": [2, 8, 28, 40, 53], "show": [2, 3, 5, 6, 7, 8, 11, 21, 22, 24, 25, 30, 36, 37, 38, 41, 47, 48, 58, 59, 61], "ensur": [2, 5, 7, 11, 12, 16, 21, 22, 25, 29, 30, 37, 41, 45, 46, 47, 49, 53], "constitut": [2, 5, 8, 24, 29, 30, 32, 38, 41, 42, 52, 56, 59], "avoid": [2, 7, 21, 23, 25, 26, 27, 39, 47, 51, 52, 59], "told": [2, 5, 8, 18, 19, 52], "clearer": [2, 11], "written": [2, 12, 45, 59, 61], "hypothesi": [2, 11, 41, 48], "follow": [2, 3, 4, 5, 7, 11, 12, 16, 17, 20, 21, 24, 25, 28, 29, 30, 32, 36, 38, 40, 41, 42, 48, 49, 50, 51, 56, 59, 61], "true": [2, 5, 7, 8, 10, 11, 12, 21, 22, 26, 27, 28, 29, 32, 33, 34, 36, 37, 38, 39, 40, 41, 42, 44, 45, 48, 49, 52, 61], "fals": [2, 5, 21, 22, 25, 32, 33, 34, 36, 37, 48], "consid": [2, 5, 7, 12, 13, 21, 25, 26, 30, 32, 34, 39, 47, 49, 51, 59], "sentec": 2, "insul": 2, "counti": [2, 9, 13, 15, 22, 30, 32, 33, 34, 37, 42, 54, 59], "along": [2, 16, 25, 42, 47], "posit": [2, 7, 12, 13, 29, 34, 36, 40, 51], "presenc": [2, 12, 30, 32, 52, 59], "zero": [2, 7, 32, 33, 34, 36, 37, 41, 48, 49], "plan": [2, 23, 26, 30, 39, 47, 59, 61], "congratul": [2, 17, 22, 37, 48], "realli": [2, 3, 5, 6, 7, 8, 9, 11, 12, 17, 20, 22, 24, 30, 32, 37, 42, 44, 45, 47, 49, 50, 51, 53, 54, 59], "hard": [2, 3, 5, 7, 16, 17, 21, 22, 23, 30, 31, 37, 45, 46, 47, 54], "easi": [2, 3, 5, 6, 7, 21, 32, 37, 42, 48], "stuff": [2, 4, 45, 54], "let": [2, 5, 6, 8, 11, 14, 17, 24, 25, 28, 29, 32, 36, 38, 40, 41, 42, 45, 47, 48, 49, 51, 59], "same": [2, 4, 6, 7, 8, 9, 10, 11, 12, 13, 18, 21, 22, 25, 28, 29, 30, 32, 34, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 49, 50, 51, 54, 59, 61], "big": [2, 3, 5, 7, 21, 22, 23, 37, 45, 47, 49, 51, 53, 60], "spender": 2, "variat": [2, 3, 5, 7, 9, 11, 24, 28, 29, 37, 38, 40, 42, 43, 53], "public": [2, 3, 5, 18, 19, 30, 40, 45, 47, 51, 52, 59, 60], "whom": [2, 21, 26, 43, 48, 51, 59, 61], "measur": [2, 3, 7, 8, 12, 22, 26, 27, 33, 37, 39, 40, 43, 44, 47, 53, 54, 59], "combin": [2, 4, 25, 40, 53], "similar": [2, 3, 7, 8, 11, 15, 16, 22, 28, 29, 32, 33, 37, 40, 41, 44, 45, 47, 48, 51, 53, 60, 61], "resourc": [2, 3, 7, 18, 19, 30, 40, 54, 58], "annual": [2, 22, 28, 37, 40, 42], "match": [2, 4, 32, 42, 47, 49, 57], "gender": [2, 5, 7, 11, 22, 29, 32, 37, 41, 42, 47, 48], "live": [2, 5, 8, 12, 49, 52], "nearli": [2, 7, 18, 19, 29, 30, 41], "guarante": [2, 7, 8, 59], "still": [2, 5, 7, 8, 11, 21, 22, 25, 29, 37, 39, 41, 42, 52, 54], "wrangl": 2, "ahead": [2, 7, 21, 30, 54], "won": [2, 5, 7, 21, 23, 28, 38, 40, 47, 48, 49, 50], "templat": 2, "great": [2, 5, 6, 7, 8, 12, 16, 18, 22, 37, 45, 47, 50, 53, 56, 59], "superiv": 2, "access": [2, 7, 11, 29, 41, 54, 59], "pretti": [2, 3, 12, 17, 28, 40, 41], "nuanc": [2, 45, 61], "super": [2, 3, 41, 42, 60], "role": [3, 5, 28, 30, 40, 49, 51, 59], "infer": [3, 4, 5, 8, 12, 15, 18, 19, 22, 28, 32, 33, 37, 40, 42, 45, 46, 47, 56, 61], "industri": [3, 10, 11, 18, 28, 40, 42, 45, 47, 49], "question": [3, 4, 5, 7, 8, 11, 12, 22, 25, 26, 27, 28, 29, 30, 34, 37, 39, 40, 41, 43, 44, 45, 46, 47, 54, 56, 61], "tech": [3, 5], "sector": [3, 28, 40, 42], "todai": [3, 13, 17, 22, 34, 37, 46, 59], "constantli": [3, 48], "refin": [3, 21, 40], "product": [3, 5, 8, 12, 25, 40, 47, 61], "target": [3, 5, 12, 16, 25, 30, 51, 59], "ad": [3, 8, 11, 24, 28, 29, 37, 38, 40, 41, 42, 45, 47, 49, 60], "increment": [3, 5], "innov": [3, 5, 7, 17, 22, 37, 45, 59], "definit": [3, 5, 21, 47, 49, 59, 61], "toolbox": 3, "read": [3, 4, 5, 6, 7, 8, 15, 18, 19, 22, 25, 26, 27, 28, 29, 33, 37, 39, 40, 46, 47, 50, 51, 59, 61], "discuss": [3, 5, 7, 8, 15, 16, 17, 28, 32, 33, 40, 42, 44, 45, 47, 48, 49, 51, 59, 61], "critic": [3, 5, 7, 8, 29, 37, 41, 45, 47, 48, 56, 58, 61], "gather": 3, "passiv": [3, 4, 17, 45, 58, 59, 60], "somebodi": [3, 7, 53], "els": [3, 18, 19, 58, 61], "rather": [3, 7, 8, 9, 24, 25, 30, 38, 42, 43, 44, 48, 54, 56, 59, 61], "manipul": [3, 7, 11, 46, 54], "random": [3, 4, 6, 7, 8, 11, 14, 21, 25, 29, 41, 44, 46, 49, 51, 53, 61], "arm": [3, 11, 29, 40, 41, 54], "low": [3, 5, 14, 21, 22, 25, 29, 37, 41, 43, 44, 47, 49], "cost": [3, 14, 17, 20, 22, 23, 27, 36, 37, 52, 53], "expens": [3, 6, 36, 53, 54], "worth": [3, 5, 7, 21, 22, 23, 29, 36, 37, 41, 44, 48, 51, 59], "possibl": [3, 5, 7, 8, 14, 16, 18, 19, 25, 26, 27, 29, 30, 39, 41, 44, 45, 51, 53, 56, 59, 61], "decis": [3, 4, 5, 7, 21, 30, 49, 50, 51, 52, 53, 55, 61], "hand": [3, 5, 24, 38, 40, 44, 45, 47, 49, 52], "situat": [3, 7, 8, 41, 44, 47, 50, 51, 52, 59], "specif": [3, 5, 7, 8, 11, 16, 24, 26, 27, 30, 31, 38, 39, 43, 45, 47, 51, 53, 59, 60, 61], "facebook": [3, 8, 49], "twitter": 3, "satisfact": 3, "choos": [3, 6, 10, 21, 22, 37, 39, 42, 51, 53], "certain": [3, 7, 8, 21, 22, 30, 37, 49, 51, 52, 59, 61], "function": [3, 5, 9, 11, 21, 22, 24, 25, 29, 32, 37, 38, 41, 42, 48, 51], "your": [3, 4, 5, 6, 8, 9, 12, 13, 14, 15, 17, 18, 19, 20, 21, 23, 24, 25, 26, 27, 28, 29, 31, 33, 34, 35, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 51, 54, 56, 58, 59], "ideal": [3, 21, 53, 59], "block": [3, 18, 49, 54], "One": [3, 11, 12, 16, 21, 22, 26, 27, 28, 37, 38, 39, 40, 41, 42, 43, 52, 60, 61], "shot": 3, "bowl": [3, 40], "store": [3, 8, 11, 14, 16, 25, 29, 40, 42, 43, 44, 47, 51, 59, 61], "n": [3, 5, 24, 34, 36, 38, 40, 41], "event": [3, 14, 59], "strateg": 3, "scheme": 3, "anoth": [3, 5, 7, 8, 16, 17, 21, 22, 28, 37, 40, 46, 49, 54, 59, 60, 61], "entir": [3, 5, 8, 12, 16, 47, 59, 61], "price": [3, 20, 36], "wouldn": [3, 7, 31, 37, 61], "stand": [3, 30, 45, 48, 49, 59], "similarli": [3, 7, 8, 49, 54, 59], "sometim": [3, 5, 7, 11, 25, 45, 48, 51], "rollout": 3, "launch": [3, 30, 52], "titl": [3, 29, 33, 41], "netflix": [3, 4, 6], "premier": 3, "movi": 3, "interfer": 3, "abstract": [3, 26, 27, 39], "talk": [3, 6, 7, 8, 13, 14, 22, 34, 35, 37, 42, 47, 48, 50, 59, 61], "constraint": [3, 21, 25, 35, 42, 48], "horizon": 3, "too": [3, 4, 5, 7, 9, 22, 26, 31, 37, 39, 41, 42, 46, 47, 50, 52, 53, 59, 60, 61], "popular": [3, 21, 44], "quickli": [3, 7, 20, 22, 36, 37, 45, 52, 61], "short": [3, 8, 23, 47, 53, 56], "countri": [3, 12, 30, 47, 49, 54, 59], "put": [3, 8, 14, 22, 23, 24, 26, 27, 37, 38, 39, 43, 48, 51, 52, 59, 61], "pick": [3, 7, 21, 22, 37, 48, 50, 51, 53, 60], "locat": [3, 5, 18, 19, 43, 44, 52, 54], "setup": 3, "shop": [3, 21, 40], "wait": [3, 5, 7, 51], "sale": [3, 8, 13, 23, 34, 43, 44, 47, 49, 61], "sens": [3, 5, 7, 15, 16, 21, 22, 24, 25, 29, 33, 37, 38, 41, 43, 46, 51, 53, 54, 59, 60], "kohavi": [3, 4, 11], "tang": [3, 4, 11], "xu": [3, 4, 11], "overal": [3, 4, 11, 13, 16, 29, 30, 34, 38, 41, 42, 53], "criterion": [3, 11, 53], "oec": [3, 11, 53], "durat": [3, 59], "believ": [3, 7, 24, 38, 47, 53], "drive": [3, 7, 12, 16, 17, 24, 38, 47, 59], "shortest": 3, "meaning": [3, 5, 8, 21, 56], "option": [3, 4, 11, 14, 30, 42], "mani": [3, 7, 11, 12, 13, 19, 21, 22, 23, 24, 25, 28, 30, 34, 37, 38, 40, 42, 44, 45, 46, 47, 48, 51, 54, 56, 59, 60, 61], "face": [3, 5, 7, 10, 17, 30, 45, 47, 52], "social": [3, 7, 22, 30, 32, 37, 40, 42, 44, 45, 47, 49, 53, 54, 59, 60], "media": [3, 33, 34, 38], "harass": 3, "retent": 3, "mind": [3, 5, 7, 14, 21, 27, 28, 29, 40, 41, 44, 45, 47, 48, 56, 61], "2012": [3, 32, 36], "emot": 3, "valenc": 3, "saw": [3, 8, 11, 48, 59], "happi": [3, 8], "sad": 3, "mood": 3, "spill": 3, "over": [3, 5, 6, 7, 8, 13, 16, 20, 22, 23, 24, 29, 32, 34, 36, 37, 38, 41, 42, 43, 44, 47, 48, 50, 53, 54, 59, 60, 61], "network": [3, 17, 29, 41, 53], "toi": [3, 40], "furiou": 3, "everyth": [3, 4, 7, 11, 22, 23, 37, 42, 44, 45, 47, 53, 59, 61], "consum": [3, 8, 16, 20, 27, 36, 40, 45, 47, 51, 54, 61], "mayb": [3, 4, 11, 41, 51, 61], "deliber": [3, 6], "harm": 3, "huge": [3, 5, 22, 37, 45, 47, 54, 59], "experiment": [3, 11, 41], "matter": [3, 5, 8, 21, 22, 37, 39, 43, 45, 51, 53, 60, 61], "issu": [3, 5, 7, 11, 20, 21, 23, 24, 25, 26, 29, 36, 38, 39, 45, 47, 49, 59, 60], "okcupid": 3, "publish": [3, 20, 36, 40, 59], "research": [3, 7, 8, 10, 16, 21, 22, 30, 31, 35, 37, 40, 45, 47, 49, 51, 52, 54, 58, 59, 61], "date": [3, 4, 11, 25, 34, 36, 37, 38, 40, 41, 42, 48, 49], "profil": 3, "honestli": [3, 59], "standard": [3, 5, 9, 11, 12, 13, 15, 21, 22, 23, 24, 25, 29, 33, 34, 36, 37, 38, 40, 41, 42, 47, 48, 49, 50, 53, 61], "mess": 3, "heart": [3, 14, 47], "care": [3, 7, 8, 11, 21, 27, 28, 29, 35, 40, 41, 42, 47, 49, 51, 53, 54, 59, 60, 61], "improv": [3, 7, 11, 16, 28, 29, 31, 40, 41, 49, 59, 60], "younger": [3, 28, 40], "woker": 3, "offer": [3, 7, 8, 11, 12, 14, 23, 27, 28, 32, 40, 44, 45, 47, 54, 59], "childcar": [3, 40], "subsidi": 3, "workforc": [3, 5, 28, 40], "union": [3, 12, 40], "benefit": [3, 29, 49, 53], "contract": [3, 18], "scale": [3, 15, 29, 33, 34, 36, 37, 47], "larger": [3, 8, 11, 13, 28, 30, 34, 39, 40, 43, 44, 48, 54], "larg": [3, 6, 8, 11, 12, 24, 27, 29, 30, 31, 36, 38, 41, 44, 45, 47, 49, 51, 52], "area": [3, 7, 14, 16, 17, 25, 31, 44, 50, 51, 52, 54, 56, 60], "increasingli": 3, "outsid": [3, 14, 51], "administ": 3, "program": [3, 16, 27, 30, 40, 46, 47, 52, 58, 60], "servic": [3, 11, 16, 18, 25, 40, 42, 47, 54, 60], "pai": [3, 10, 11, 12, 13, 14, 23, 28, 34, 40, 47], "vendor": 3, "base": [3, 6, 7, 11, 15, 20, 21, 22, 25, 33, 36, 37, 42, 45, 47, 53, 54], "input": [3, 7, 16, 22, 32, 37, 42, 59], "hour": [3, 4, 11, 28, 40, 58], "spent": [3, 7, 8, 9, 20, 22, 24, 35, 37], "encourag": [3, 4, 16], "paid": [3, 11, 25, 28, 40], "basi": [3, 5, 7, 13, 16, 25, 34, 59], "incent": 3, "cut": [3, 7, 21, 22, 37, 40, 45], "risk": [3, 7, 8, 16, 17, 29, 54], "arrang": [3, 40], "hospit": [3, 5, 17, 40, 42], "infect": [3, 5, 12, 18, 19], "reduct": [3, 5, 31, 49], "minim": [3, 5, 7, 21, 53, 60], "downtim": 3, "intern": [3, 4, 12, 17, 29, 40, 41, 52, 54, 56, 59], "ty": 3, "compens": [3, 28, 40], "parti": [3, 30, 41, 48], "incentiv": 3, "precis": [3, 7, 11, 13, 18, 25, 34, 36, 39, 40, 41, 44, 47, 61], "effort": [3, 5, 13, 21, 23, 24, 34, 38, 52, 59], "averag": [3, 5, 7, 8, 10, 11, 12, 13, 15, 16, 23, 24, 26, 27, 28, 29, 30, 33, 34, 36, 38, 39, 40, 41, 43, 44, 47, 48, 51, 61], "suit": 3, "circumst": 3, "unfortun": [3, 7, 25], "befor": [3, 4, 5, 7, 8, 11, 13, 14, 22, 25, 28, 29, 32, 34, 35, 37, 40, 42, 48, 49, 50, 51, 52, 59, 61], "finish": [3, 4, 37, 40, 50], "natur": [3, 5, 30, 40, 47, 52, 54, 60], "season": [3, 23, 53], "expect": [3, 8, 11, 13, 16, 18, 19, 21, 23, 26, 28, 29, 34, 37, 39, 40, 41, 46, 56, 61], "vari": [3, 11, 25, 28, 32, 38, 40, 43, 48, 49, 61], "anyth": [3, 5, 7, 12, 18, 19, 21, 32, 42, 43, 44, 50, 53, 56, 61], "design": [3, 4, 8, 11, 13, 17, 21, 22, 28, 29, 34, 37, 40, 41, 42, 44, 46, 47, 51, 52, 56, 59], "g": [3, 5, 7, 8, 11, 12, 13, 15, 16, 21, 22, 25, 29, 33, 34, 36, 37, 40, 41, 45, 46, 47, 53, 54, 59, 61], "upward": 3, "analyst": [3, 11], "extrem": [3, 7, 26, 39, 41, 43, 44, 45, 49, 51, 59], "friendli": 3, "easier": [3, 7, 13, 21, 23, 34, 56, 60], "duck": 3, "goos": 3, "rel": [3, 5, 25, 29, 35, 41, 49, 51, 59, 61], "difficulti": [3, 29, 39, 41, 46], "task": [3, 11, 18, 19, 22, 25, 37], "length": [3, 20, 22, 32, 36, 37, 42, 49, 56], "extern": [3, 4, 29, 41, 59, 61], "valid": [3, 4, 8, 16, 18, 19, 23, 25, 29, 32, 34, 41, 59, 61], "estim": [3, 5, 7, 8, 10, 13, 16, 21, 22, 23, 24, 26, 27, 34, 37, 38, 39, 42, 43, 44, 47, 48, 49, 51, 54, 59], "wide": [3, 45], "thu": [3, 5, 7, 11, 12, 17, 21, 32, 39, 40, 43, 47, 59, 61], "differenti": [3, 7, 12, 14, 25, 28, 29, 40, 41, 44], "market": [3, 16, 40, 59], "shouldn": [3, 7, 11, 22, 29, 32, 37, 41, 44, 51, 59], "appli": [3, 5, 11, 13, 15, 32, 33, 34, 35, 44, 45, 46, 47, 51, 59], "logic": [3, 12, 29], "almost": [3, 5, 7, 18, 19, 22, 29, 37, 41, 44, 47, 49, 50, 52, 54, 56, 59], "inevit": 3, "interact": [3, 13, 21, 23, 29, 34, 41, 47, 49, 53], "consider": [3, 7, 52, 59], "colleagu": [3, 14], "tuesdai": 4, "thursdai": 4, "25": [4, 22, 23, 32, 36, 37, 38, 40, 47], "2": [4, 8, 26, 32, 39, 42, 49, 51], "40pm": 4, "offic": [4, 13, 26, 30, 32, 39, 40, 42], "nick": [4, 32, 48], "00": [4, 33, 34, 36, 37, 38, 40, 41, 42, 48], "00am": 4, "http": [4, 13, 15, 20, 22, 24, 28, 32, 33, 34, 36, 37, 38, 40, 41, 42, 48], "duke": [4, 5, 14, 16, 22, 26, 29, 30, 37, 39, 41, 46, 47, 54], "zoom": [4, 27], "nickeubank": [4, 13, 15, 20, 22, 24, 28, 33, 34, 36, 37, 38, 40, 41, 42, 48], "gross": 4, "hall": [4, 26, 39], "231": [4, 36, 37, 40], "echo": 4, "fridai": 4, "3pm": 4, "kashaf": 4, "4pm": 4, "pragya": 4, "wednesdai": 4, "topic": [4, 7, 51, 56, 59], "thr": 4, "jan": 4, "overview": [4, 11, 61], "acm": [4, 16], "propos": [4, 11, 13, 17, 34, 35, 52], "tue": [4, 36], "16": [4, 32, 33, 34, 36, 38, 40, 41, 42, 47, 48, 51], "problem": [4, 5, 7, 8, 9, 11, 12, 16, 17, 21, 25, 30, 35, 36, 40, 43, 45, 47, 49, 51, 53, 58, 59, 61], "sign": [4, 7, 47], "submit": [4, 7, 11, 16, 25, 29, 30, 56], "syllabu": 4, "gradescop": 4, "18": [4, 20, 22, 32, 34, 36, 37, 38, 40, 41, 42, 48], "histori": [4, 5, 30, 59], "30": [4, 8, 11, 22, 25, 30, 32, 34, 37, 38, 40, 49], "min": [4, 34, 37, 38, 42], "video": [4, 5, 7, 11, 21, 40, 47, 50], "23": [4, 15, 33, 34, 37, 38, 40, 41, 48], "proscript": [4, 59], "v": [4, 7, 30], "descript": [4, 11, 15, 22, 35, 37, 52], "versu": [4, 16, 22, 35, 37], "prescript": [4, 59], "exploratori": [4, 17, 59], "eda": 4, "write": [4, 9, 16, 30, 32, 45, 46, 48, 52, 56, 60], "notebook": [4, 11, 25, 29], "team": [4, 12, 16, 30, 35, 46, 60], "explor": [4, 5, 9, 11, 16, 51, 54, 59], "feb": [4, 36, 42, 48], "aristotl": 4, "edmondson": 4, "fearless": 4, "organ": [4, 11, 12, 13, 18, 19, 32, 34, 40, 42, 45, 52, 54, 61], "chpt": 4, "canva": 4, "review": [4, 5, 11, 16, 30, 48, 53, 59], "charter": 4, "cognit": 4, "barrier": [4, 30], "p": [4, 6, 11, 23, 29, 34, 36, 37, 38, 40, 41, 42, 48, 53, 55], "82": [4, 32, 37, 40], "chapter": [4, 47], "foster": 4, "psycholog": 4, "safeti": [4, 17, 40], "tip": 4, "ex": 4, "13": [4, 9, 10, 26, 27, 32, 33, 34, 36, 38, 39, 41, 48], "stop": [4, 5, 22, 37, 46, 47, 51, 52, 53], "box": [4, 21, 22, 30, 37, 40, 48, 59], "trust": [4, 9, 11, 21, 40, 44, 48, 61], "contrain": 4, "artifici": [4, 46, 47], "stupid": [4, 52], "explicit": [4, 32, 48], "scientif": [4, 7, 40, 59], "ml": [4, 7, 16, 21], "15": [4, 5, 15, 32, 33, 34, 36, 38, 40, 41, 48, 49, 50], "gam": [4, 21], "silver": 4, "bullet": 4, "tour": [4, 59], "pygam": [4, 21], "20": [4, 21, 22, 23, 32, 33, 34, 36, 37, 40, 41, 42, 48, 50], "22": [4, 23, 30, 36, 37, 40, 41, 42], "27": [4, 33, 37], "cunningham": 4, "pp": 4, "135": [4, 32, 36, 37, 40], "independ": [4, 12, 20, 36, 40, 41, 46, 48, 49, 51, 53], "148": [4, 37, 38, 40], "sdo": 4, "widehat": 4, "ATE": [4, 8, 11, 29, 37, 41], "present": [4, 8, 16, 25, 49, 52, 56, 60, 61], "29": [4, 22, 32, 37, 38, 48], "kohvani": 4, "19": [4, 21, 22, 32, 34, 36, 37, 38, 40, 41, 42], "21": [4, 23, 32, 33, 36, 37, 40, 41], "guardrail": [11, 53], "metric": [11, 47, 53], "threat": [4, 29, 41, 43], "42": [4, 25, 37, 38, 40], "47": [4, 32, 33, 34, 36, 37, 38, 40, 48], "ktx": [], "mar": [4, 34, 36, 37, 38, 40, 41], "54": [4, 33, 36, 37, 38, 42], "kohvai": 4, "primaci": [4, 53], "decai": 4, "39": [22, 32, 34, 37, 38, 40, 41, 42, 48], "unless": [6, 7, 8, 11, 53], "method": [5, 7, 16, 20, 21, 22, 23, 24, 25, 28, 32, 34, 36, 37, 38, 40, 41, 42, 48, 51, 53], "power": [4, 7, 11, 22, 24, 37, 38, 40, 41, 42, 43, 48, 49, 50, 53, 59], "calcul": [4, 5, 11, 13, 15, 22, 23, 25, 28, 29, 33, 34, 37, 40, 41, 42, 44, 47, 48, 49, 50, 53], "12": [4, 26, 27, 32, 33, 34, 36, 38, 39, 42, 48, 51], "NO": [4, 48], "14": [4, 26, 27, 30, 32, 34, 36, 38, 39, 41, 42, 48, 50], "bayesian": 4, "ab": [4, 6, 11, 40, 53], "midterm": [4, 61], "26": [4, 32, 33, 34, 36, 37], "28": [4, 32, 36, 37, 38, 41, 51], "fix": [4, 13, 28, 34, 40, 47, 57], "apr": 4, "panel": [4, 24, 38, 42, 43], "angrist": 4, "pisck": 4, "mm": 4, "178": [4, 32, 37, 40], "208": [4, 34, 36, 37, 40], "diff": [4, 13, 34, 57], "dont": 4, "enfranchis": 4, "ethic": [4, 7, 53], "mayson": 4, "2281": 4, "extra": [4, 48, 59], "ai": [4, 46], "misalign": 4, "adversari": [4, 7], "fool": [4, 7], "openai": 4, "robograd": 4, "text": [4, 7, 21, 38, 45, 46], "referenc": 4, "trustworthi": 4, "onlin": [4, 11, 23, 29, 41, 54], "guid": [4, 5, 60], "analys": [5, 9, 28, 40, 51, 53, 54, 59, 60, 61], "extract": [5, 40], "pattern": [5, 29, 41, 59], "otherwis": [5, 20, 21, 36, 42, 48, 50, 59], "incomprehensibli": 5, "messi": [5, 28, 40], "reader": [5, 8, 61], "digest": 5, "manner": [5, 12, 13, 25, 34, 45, 56, 59], "intrins": 5, "report": [5, 7, 8, 11, 20, 22, 25, 26, 28, 29, 30, 35, 36, 37, 39, 40, 41, 47, 48, 49, 54, 60], "safe": [5, 21, 25, 29], "discard": 5, "word": [5, 7, 8, 10, 11, 12, 13, 20, 21, 25, 26, 27, 28, 30, 32, 34, 36, 37, 39, 40, 42, 43, 48, 49, 50, 51, 59, 61], "entail": [5, 16, 21, 59], "asid": [5, 49], "origin": [5, 7, 11, 20, 21, 22, 25, 29, 36, 37, 41, 45, 47, 50, 51], "summar": [5, 61], "summari": [5, 9, 21, 22, 34, 36, 37, 38, 40, 41, 42, 45, 48, 58], "plot": [5, 6, 7, 9, 13, 15, 16, 20, 21, 22, 24, 33, 34, 36, 37, 38, 52], "distribut": [5, 7, 13, 21, 22, 23, 34, 37, 38, 40, 42, 49, 50, 51], "sophist": [5, 47, 54, 59], "cluster": [5, 13, 18, 19, 22, 24, 34, 37, 38], "dimension": [5, 50], "therefor": [5, 13, 34], "herein": [5, 49], "li": [5, 46, 49], "hazard": 5, "presum": [5, 11], "irrelev": 5, "incumb": 5, "upon": [5, 56, 59], "highlight": [5, 45], "struggl": [5, 9, 20, 22, 24, 27, 30, 37, 48, 61], "deviat": [5, 9, 15, 22, 23, 33, 37, 43, 50, 61], "unsupervis": [5, 59], "machin": [5, 16, 17, 18, 22, 37, 40, 46, 47, 48, 52, 59], "principl": [5, 28, 40, 42, 44], "compon": [5, 13, 18, 19, 26, 27, 34, 39, 40, 52, 60], "pca": 5, "academ": [5, 29, 30, 41, 42, 47], "peer": [5, 30], "must": [5, 7, 12, 18, 19, 21, 26, 27, 28, 39, 40, 52, 59, 60], "frac": [5, 24, 38], "sigma": 5, "x_i": 5, "central": [5, 14, 30, 59], "tendenc": 5, "paper": [5, 7, 21, 22, 29, 37, 40, 45, 47, 48, 52, 58], "neutral": 5, "pipelin": [5, 16, 40], "deliv": [5, 46], "choic": [5, 11, 22, 24, 37, 38, 48, 51], "taken": [5, 11, 23, 25, 41, 43, 52], "signal": [5, 7], "implicitli": [5, 18, 19, 28, 40], "With": [5, 7, 11, 13, 14, 16, 21, 25, 27, 34, 41, 44, 45, 47, 48, 56, 60, 61], "whenev": [5, 7, 11, 47, 59], "am": [5, 52], "faithfulli": 5, "full": [5, 8, 11, 16, 24, 28, 29, 38, 40, 41, 42, 46, 51, 52, 53, 54, 61], "incomprehens": 5, "raw": [5, 13, 22, 28, 32, 36, 37, 40, 41, 42, 48, 60], "eventu": [5, 11, 13, 30, 34], "essenti": [5, 10, 52], "themselv": [5, 42], "give": [5, 7, 8, 12, 14, 15, 16, 17, 21, 22, 24, 26, 27, 30, 33, 37, 38, 39, 40, 44, 46, 47, 48, 49, 50, 54, 56, 59, 60, 61], "receiv": [5, 7, 8, 11, 12, 29, 30, 47, 60, 61], "professor": [5, 61], "exam": 5, "score": [5, 7, 15, 25, 26, 33, 39, 42, 43, 44, 46, 50], "grade": [5, 7, 11, 25, 29, 42, 56, 59], "85": [5, 32, 34, 37, 41, 48], "sound": [5, 13, 29, 34, 36, 40, 41], "dig": [5, 8, 43, 52], "bimod": 5, "argu": [5, 11, 13, 25, 34, 47, 52, 59, 61], "somewhat": [5, 20, 36, 48, 50], "around": [5, 7, 18, 19, 21, 24, 29, 36, 38, 40, 47, 49, 59, 60], "occur": [5, 11, 12, 16, 25, 30, 32, 59, 60, 61], "teach": [5, 7], "suspect": [5, 20, 36, 42], "fail": [5, 11, 22, 37, 46, 59], "worst": 5, "incom": [5, 7, 8, 21, 25, 28, 38, 40, 42, 43, 44, 46, 49, 52, 54, 57, 59, 61], "rose": 5, "earn": [5, 22, 28, 37, 40, 42, 54], "000": [5, 8, 13, 21, 22, 23, 24, 34, 36, 37, 38, 40, 41, 42, 47, 48, 49], "hear": [5, 7, 44, 51], "x_2": 5, "bin": [5, 33, 37, 59], "y_2": 5, "thousand": 5, "dollar": [5, 8, 23, 27, 28, 36, 40], "yup": [5, 8, 9, 37, 59], "fit": [5, 7, 11, 14, 22, 24, 25, 34, 36, 37, 38, 40, 41, 42, 46, 48, 51, 59], "technic": [5, 25, 40, 42, 44, 49, 51, 56], "accur": [5, 22, 28, 37, 40, 47, 49, 59], "captur": [5, 16, 36, 53], "trend": [5, 13, 34, 61], "ten": 5, "monoton": [5, 7, 16, 21, 35], "protect": [5, 7, 30, 54, 60], "kind": [5, 7, 8, 11, 12, 30, 42, 43, 47, 48, 50, 51, 54], "error": [5, 7, 11, 13, 21, 22, 24, 25, 29, 34, 36, 37, 38, 40, 41, 42, 47, 48, 51, 53], "residu": [5, 11, 29, 34, 36, 37, 38, 40, 41, 42, 48], "histogram": [5, 15, 33], "rule": [5, 11, 30, 48, 53, 59, 60], "thumb": 5, "beyond": [5, 18, 19, 30, 60], "median": [5, 13, 34], "radic": 5, "check": [5, 6, 7, 11, 21, 25, 32], "primari": [5, 30, 40, 42, 59], "stori": [5, 7, 8, 26, 27, 39, 47, 51, 61], "miss": [5, 7, 21, 22, 25, 28, 36, 37, 40, 42], "point": [5, 6, 7, 8, 11, 13, 15, 21, 22, 23, 25, 29, 30, 33, 34, 37, 41, 42, 45, 48, 50, 51, 56, 61], "substant": [5, 7, 21, 45, 46, 48], "behind": [5, 21, 52], "notic": [5, 11, 12, 18, 56, 61], "properti": [5, 25, 32], "haven": [5, 8, 21, 41, 42, 50, 59], "defin": [5, 13, 14, 16, 24, 26, 27, 34, 38, 39, 49], "system": [5, 7, 13, 17, 25, 34, 40, 42, 59], "antibiot": 5, "resist": [5, 45], "brought": [5, 51, 59], "facil": [5, 40, 42], "director": 5, "remov": [5, 7, 25, 26, 34, 39, 43], "fabric": [5, 40], "chair": 5, "disinfect": 5, "prescrib": 5, "impact": [5, 7, 11, 13, 16, 22, 24, 34, 37, 38, 39, 42, 47, 49, 51, 52, 61], "instead": [5, 7, 11, 22, 23, 25, 26, 27, 37, 39, 42, 43, 47, 48, 49, 51, 52, 59], "futur": [5, 30, 59], "respons": [5, 8, 14, 16, 18, 19, 29, 30, 41, 47, 49, 56], "biggest": [5, 13, 34, 45, 49, 50], "contributor": 5, "rais": [5, 22, 32, 37, 56], "livestock": 5, "societi": 5, "subtl": [5, 45, 59], "less": [5, 7, 8, 13, 14, 20, 21, 22, 25, 26, 27, 28, 29, 34, 36, 37, 39, 40, 41, 44, 45, 47, 48, 49, 50, 51, 61], "maker": [5, 49, 61], "co_2": 5, "emiss": 5, "95": [5, 21, 23, 33, 37, 56], "unemploy": [5, 38, 43, 59], "profit": [5, 23, 30, 37, 46, 60], "million": [5, 22, 23, 37], "tax": [5, 24, 31, 38, 40, 49, 59], "moder": 5, "200": [5, 37, 40], "poverti": 5, "line": [5, 9, 21, 22, 24, 37, 38], "proport": [5, 10, 22, 37], "tend": [5, 7, 10, 12, 13, 17, 21, 22, 23, 29, 34, 36, 37, 39, 40, 41, 42, 43, 44, 49, 59, 60], "feel": [5, 7, 9, 11, 13, 25, 32, 34, 35, 44, 45, 48, 58, 59], "lack": [5, 18, 19, 30, 56], "divers": [5, 30, 45], "men": [5, 7, 10, 28, 29, 40, 41, 47, 48, 49, 59], "misogynist": 5, "life": [5, 7, 18, 19, 21, 49], "influenc": [5, 48, 60], "male": [5, 28, 32, 40, 42, 48], "infam": 5, "thankfulli": [5, 7, 22, 37, 42, 61], "stake": [5, 7, 21, 52], "failur": [5, 46], "watch": [5, 6, 7, 29, 41, 50], "camera": 5, "went": [5, 8, 23, 40, 61], "hp": 5, "did": [5, 11, 13, 18, 21, 23, 25, 32, 34, 35, 46, 51, 53, 56, 59, 61], "ship": [5, 40, 54], "silli": [5, 36], "web": [5, 40], "cam": 5, "forefront": [5, 7], "technolog": 5, "seriou": [5, 17, 47], "mistak": 5, "oh": [5, 8, 38, 54, 59], "googl": [5, 47, 54], "Their": 5, "tag": [5, 24, 32, 38], "dark": 5, "skin": 5, "gorilla": 5, "releas": [5, 22, 31, 37, 49], "reflect": [5, 22, 25, 37, 42, 44, 45, 49, 54, 61], "No": [5, 8, 29, 34, 36, 37, 38, 40, 41, 42, 48, 52, 53, 61], "hei": 5, "roll": [5, 6, 8, 11, 40, 53], "behav": [5, 7], "predominantli": [5, 47], "light": [5, 30, 40, 59], "ye": [5, 8, 21, 41, 51], "sadli": 5, "unfairli": 5, "sent": [5, 29, 41, 47, 61], "jail": [5, 13, 34], "discrimin": [5, 7, 10, 22, 29, 30, 37, 41, 42], "against": [5, 7, 11, 20, 22, 23, 25, 29, 36, 37, 41, 49], "per": [5, 8, 11, 13, 20, 24, 25, 28, 32, 34, 36, 38, 40, 43, 48, 56], "se": [5, 34, 42, 48, 56], "press": [5, 7, 16, 61], "especi": [5, 7, 9, 21, 22, 25, 32, 37, 44, 45, 47, 49, 59, 61], "salienc": 5, "rememb": [5, 7, 8, 11, 25, 26, 27, 29, 39, 47, 48, 49, 53, 59, 60, 61], "odd": [5, 8, 49, 50], "temptat": [6, 45], "known": [6, 7, 11, 39], "boss": [6, 12, 47], "yesterdai": [6, 17], "medic": [6, 7, 14, 27, 29, 40, 47, 49, 54], "trial": [6, 11, 47, 49], "soon": [6, 7, 8, 47, 58], "experienc": [6, 10, 29, 33], "side": [6, 7, 12, 16, 22, 24, 37, 38], "intermedi": 6, "render": [6, 33], "invalid": 6, "basic": [6, 7, 13, 21, 22, 24, 28, 34, 37, 38, 40, 41, 42, 46, 50, 51, 59, 61], "fluctuat": [6, 13, 34], "enough": [6, 8, 11, 14, 22, 23, 31, 37, 45, 47, 51, 53, 56, 59], "probabilist": 6, "outlier": [6, 9], "moment": [6, 21, 28, 29, 40, 41, 45, 49, 59], "ramesh": 6, "johari": 6, "leo": 6, "peke": 6, "david": 6, "walsh": 6, "fake": [6, 30], "condit": [6, 8, 11, 12, 20, 23, 26, 27, 30, 36, 39, 40, 43, 47, 51], "sever": [6, 13, 16, 24, 34, 38, 42, 43, 44, 52, 59, 60], "dai": [4, 6, 7, 11, 25, 26, 30, 35, 40, 47, 49, 53, 56], "chosen": [6, 10, 28, 30, 40], "hit": [6, 22, 37], "deep": [6, 16, 22, 37], "troubl": [6, 51], "fact": [6, 7, 9, 12, 13, 18, 19, 21, 22, 28, 32, 34, 36, 37, 40, 42, 43, 44, 47, 49, 52, 61], "supervis": [7, 52, 59], "discourag": [7, 10], "inde": [7, 9, 21, 44, 45, 48, 59], "cover": [7, 21, 46, 51, 54, 59, 61], "construct": [7, 9, 16, 24, 31, 38, 40], "action": [7, 16, 49, 53, 59], "deploi": 7, "discriminatori": 7, "uncontest": 7, "grip": [7, 11], "approxim": [7, 12, 42], "doubt": 7, "previou": [7, 28, 29, 34, 40, 41], "emul": 7, "process": [7, 11, 18, 22, 37, 40, 45, 47, 50, 52, 56, 58, 59], "attempt": [7, 11, 12, 16, 22, 37, 59], "systemat": [7, 46], "inaccur": [7, 21], "relat": [7, 8, 12, 14, 22, 29, 37, 40, 43, 45, 60, 61], "thoughtfulli": [7, 56], "down": [7, 12, 16, 21, 22, 30, 32, 37, 40, 44, 50, 52, 53, 61], "downstream": 7, "leav": [7, 12, 22, 37], "fight": 7, "own": [7, 11, 12, 21, 25, 45, 46, 52, 58, 59, 60], "flavor": [7, 25], "h": 7, "michael": 7, "akand": 7, "unforeseen": 7, "uneth": [7, 12], "recapitul": [7, 61], "fulli": [7, 13, 21, 34, 45, 47, 49, 53, 59], "underli": [7, 11, 22, 37, 43], "advantag": [7, 16, 22, 37], "intepret": 7, "eleph": 7, "room": [7, 40], "trade": [7, 18, 22, 37, 40, 42, 54, 60], "off": [7, 8, 21, 22, 29, 37, 41, 44, 59, 61], "simplic": [7, 43], "accuraci": [7, 21], "tree": [7, 28, 40], "nice": [7, 15, 22, 33, 37, 48], "compet": 7, "vast": 7, "fanci": [7, 9, 14], "substanti": [7, 14, 15, 27, 40, 44], "applic": [7, 25, 29, 41, 47, 50, 51, 53, 61], "claim": [7, 47], "crack": 7, "formal": [7, 8], "unsolv": 7, "win": 7, "crazi": 7, "complex": [7, 22, 37, 45, 56], "articl": [7, 29, 41, 61], "self": [7, 37], "car": [7, 17, 20, 24, 30, 36, 38, 40, 47, 48, 60], "protein": 7, "fold": [7, 59], "edg": [7, 21], "comput": [7, 11, 21, 22, 24, 26, 29, 37, 38, 39, 40, 41, 42, 45, 48, 54], "vision": 7, "necessari": [7, 16, 21, 27, 47, 50, 59], "folk": [7, 54], "competit": [7, 16, 21, 60], "consult": [7, 12, 16, 40], "necessarili": [7, 10, 45, 54], "career": [7, 56], "bigger": [7, 36, 39, 51], "maxim": [7, 21, 47], "bear": [7, 28, 29, 40, 41, 45, 46, 47, 56, 59, 61], "maintain": [7, 25], "chanc": [7, 11], "direct": [7, 11, 16, 20, 30, 36, 40, 50], "mainten": [7, 40], "weird": [7, 25, 47, 51], "educ": [7, 10, 11, 22, 28, 29, 37, 40, 41, 42, 47, 49, 50, 51, 52, 54], "highli": [7, 16, 51], "imposs": [7, 12], "spit": 7, "nonsens": 7, "danger": [7, 21], "financ": [7, 8, 40], "healthcar": [7, 14, 27, 40, 42], "contain": [7, 21, 24, 25, 38, 40, 42, 60], "virtual": 7, "bomb": [7, 12], "trigger": 7, "relatedli": [7, 47], "difficult": [7, 59], "adopt": [7, 21, 46], "auto": [7, 12, 40], "grader": [7, 34], "certainli": [7, 33, 52], "net": [7, 12], "essai": [7, 59], "free": [7, 8, 11, 12, 23, 26, 39, 54], "teacher": 7, "deepli": 7, "manipulat": 7, "particular": [7, 8, 11, 15, 21, 23, 40, 42, 45, 56, 58, 59, 61], "appear": [7, 11, 22, 37, 45, 49], "mostli": [7, 11, 18, 25, 59, 61], "onc": [7, 22, 28, 29, 31, 32, 37, 39, 40, 41, 58, 59, 60], "quit": [7, 36, 54, 60], "incredibli": [7, 9, 59], "mad": 7, "financi": [7, 8, 16, 35, 40, 54, 59, 61], "game": [7, 49], "engag": [7, 16, 45, 47, 56, 61], "isn": [7, 11, 21, 22, 28, 37, 40, 41, 42, 43, 45, 47, 49, 51, 53, 54, 60], "hu": 7, "et": [7, 20, 36], "al": [7, 20, 36, 38], "replac": [7, 22, 26, 37, 39], "proprietari": 7, "court": [7, 30], "advis": [7, 12, 30], "judg": 7, "grant": [7, 30], "bail": 7, "share": [7, 11, 15, 16, 21, 22, 25, 29, 33, 37, 41, 47, 49, 54, 60, 61], "juvenil": 7, "arrest": [7, 13], "adult": [7, 29, 41], "assess": [7, 45, 52], "seal": 7, "belief": [7, 32], "childhood": 7, "fate": 7, "transpar": [7, 16, 59], "nlp": 7, "headach": [7, 49], "unusu": [7, 44], "bet": [7, 21], "easili": [7, 9, 30, 48, 49, 54], "analyt": [7, 23], "parametr": 7, "cardiac": 7, "cholesterol": [7, 14, 49], "credit": [7, 11, 16, 25, 31, 40], "worthi": [7, 16], "wish": [7, 25, 31, 45, 48, 58, 60, 61], "ldl": 7, "cholesterdol": 7, "regular": [7, 37, 39, 48, 51, 54, 61], "prevent": [7, 12, 21, 45, 59], "forest": 7, "isoton": 7, "vincent": 7, "warmerdam": 7, "health": [7, 14, 26, 27, 39, 40, 42, 47, 54, 59, 60], "y": [7, 8, 9, 11, 12, 13, 18, 19, 22, 24, 26, 33, 34, 36, 37, 38, 43, 48, 51, 59], "axi": [7, 15, 22, 24, 32, 33, 34, 37, 38], "regressor": 7, "sklearn": [7, 22, 25, 37], "xgboost": 7, "tensor": 7, "flow": 7, "goe": [7, 22, 37, 48, 61], "subpopul": [7, 41, 53], "concern": [7, 8, 11, 12, 25, 47, 54], "dalit": 7, "brahmin": 7, "christian": 7, "muslim": 7, "coupl": [7, 22, 37], "disproportion": [7, 47, 49], "rate": [7, 12, 13, 14, 21, 24, 25, 28, 29, 34, 38, 40, 41, 43, 47, 48, 59], "tail": [7, 11], "co": [7, 38, 43, 47], "enter": [7, 11, 21, 25, 28, 40], "through": [7, 8, 11, 16, 18, 19, 20, 22, 37, 42, 45, 46, 48, 54, 56, 58, 60], "swap": [7, 40], "imput": 7, "factor": [7, 12, 21, 24, 36, 38, 59], "Of": [7, 22, 37, 47, 49, 50, 59], "third": [7, 9, 11, 16, 25, 29, 51, 59], "unbias": [7, 44], "regardless": [7, 48, 51], "religion": 7, "sickl": 7, "cell": [7, 60], "anemia": 7, "symptom": [7, 49], "african": 7, "descent": 7, "allel": 7, "malaria": [7, 12], "pressur": [7, 28, 40], "region": 7, "endem": 7, "carri": [7, 18, 50], "gene": [7, 47], "strong": [7, 33, 45, 46, 51], "At": [7, 8, 11, 21, 22, 25, 37, 39, 40, 42, 46, 50, 52, 59, 61], "recent": [7, 13, 17, 22, 32, 34, 37, 45, 46, 52], "kidnei": 7, "transplant": 7, "correct": [7, 18, 19, 21, 42, 44, 53, 56], "blood": 7, "flag": [7, 11, 25, 61], "prioriti": [7, 16, 30], "unlik": [7, 8, 41, 51, 52, 56], "adjust": [7, 21, 42, 44], "explan": [7, 21, 44], "histor": [7, 40], "succe": 7, "biolog": 7, "until": [7, 9, 11, 20, 22, 24, 37, 41, 50, 51], "rigor": [7, 13, 34, 59, 61], "emphas": [7, 51, 59, 61], "solut": [7, 11, 17, 25, 29, 52], "proxi": 7, "crude": [7, 13, 34], "initi": [7, 21, 22, 25, 27, 30, 37, 47, 48, 52, 59], "screen": 7, "drop": [7, 8, 21, 22, 25, 28, 30, 32, 34, 37, 40, 41, 42, 51], "margin": [7, 45], "weakli": 7, "genet": 7, "ever": [7, 8, 22, 25, 29, 37, 41, 49], "evid": [7, 8, 13, 18, 19, 26, 29, 34, 39, 41, 53, 59, 61], "dynam": [7, 59], "plai": [7, 21, 22, 23, 37, 51, 59, 60], "greater": [7, 11, 23, 25, 29, 36, 40, 41, 59], "knowledg": [7, 21, 31, 56, 59], "remind": 7, "expert": 7, "quantit": [7, 25, 30, 31], "expertis": [7, 16], "terribl": [7, 42], "su": [7, 17], "unus": 7, "imperfect": 7, "dramat": [7, 19, 22, 37, 51], "campaign": 7, "constant": [7, 9, 42], "toward": [7, 21, 59], "hurdl": [7, 60], "Then": [7, 12, 13, 21, 22, 34, 37, 42, 48, 50, 52, 59], "accept": [7, 8, 11, 16, 20, 22, 28, 36, 37, 40], "charg": [7, 11], "lawsuit": 7, "pr": 7, "disast": [7, 54], "optim": [7, 16, 21, 60], "spars": 7, "scalabl": 7, "fall": [7, 13, 17, 22, 23, 25, 34, 37, 49, 51], "triag": [7, 27], "notori": 7, "concept": [7, 11, 45, 46, 56, 61], "whiten": 7, "hybrid": 7, "portion": [7, 11, 28, 40], "flexibl": [7, 13, 21, 34], "machineri": [7, 24, 38, 40], "lattic": 7, "counterfactu": [7, 59], "visual": [7, 21, 46], "dtreeviz": 7, "lda": 7, "detail": [8, 16, 21, 22, 28, 30, 37, 40, 44, 48, 50, 54, 59, 60, 61], "deriv": [8, 13, 17, 26, 27, 39, 49], "mathemat": [8, 20, 36, 42, 43, 44, 45, 56], "advertis": [8, 40, 47], "discount": [8, 40, 42], "amount": [8, 13, 16, 25, 34, 43, 47, 48, 49, 59, 61], "treat": [8, 11, 13, 22, 23, 25, 28, 29, 32, 34, 37, 40, 41, 42, 44, 49, 50, 51, 59], "untreat": [8, 22, 37, 50, 51], "seen": [8, 15, 21, 29, 32, 33, 47, 58, 59], "y_0": [8, 22, 37, 43], "y_1": [8, 22, 37], "met": [8, 26, 27, 39, 46, 47], "sell": [8, 13, 14, 34, 40, 47, 49, 61], "sunglass": 8, "pawda": 8, "sourc": [8, 11, 18, 19, 43, 51, 52, 60], "promot": [8, 14, 51], "coupon": [8, 51, 61], "york": [8, 43, 47], "ny": [8, 37, 38, 43], "came": [8, 20, 36, 47, 51], "revenu": [8, 44, 60], "instagram": 8, "nytim": 8, "elsewher": 8, "tempt": 8, "vacuous": 8, "willing": [8, 28, 40, 50], "higher": [8, 10, 14, 22, 23, 26, 29, 37, 39, 40, 41, 52, 53], "bar": [8, 13, 34], "affirm": [8, 26], "wealthier": [8, 47], "relativelei": 8, "dispos": 8, "split": [8, 13, 16, 21, 25, 34, 41, 49], "weren": [8, 39, 47, 51], "height": [8, 26, 39], "cm": 8, "happier": 8, "absent": [8, 28, 40, 59], "balanc": [8, 11, 22, 25, 37, 51, 53, 61], "size": [8, 16, 21, 22, 30, 37, 41, 42, 44, 49, 53], "lln": 8, "kick": [8, 14], "kinda": [8, 49], "att": 8, "yai": 8, "platform": [8, 49], "exist": [8, 30, 44, 47, 52, 54, 59, 60], "everywher": [8, 47, 54], "convinc": [8, 18, 19, 61], "ourselv": [8, 22, 37], "increas": [8, 11, 12, 16, 22, 23, 26, 36, 37, 39, 40, 42, 47, 48, 49, 59], "relativlei": 8, "understim": 8, "BUT": [8, 61], "obsess": 8, "plausibl": 8, "dunno": 8, "dozen": [8, 9, 50, 59], "far": [8, 13, 21, 22, 29, 34, 37, 41, 47, 49, 51, 54], "perfect": [8, 21, 22, 25, 28, 37, 40, 47], "absolut": [8, 12, 21, 29, 43, 58, 60, 61], "further": [8, 23, 30, 39], "investig": [8, 16, 40, 52], "advic": [8, 23, 27, 45], "strongli": 8, "friend": [8, 49], "flip": 8, "flop": 8, "sandal": 8, "cat": [8, 42], "meow": 8, "dal": 8, "referr": 8, "none": 8, "forward": [8, 25, 50, 51, 52, 58, 59], "welcom": [8, 11, 16], "certainti": 8, "hopefulli": [8, 10, 18, 22, 37, 41, 48, 59, 61], "polic": [9, 33, 47, 57, 60], "expenditur": [9, 33, 57], "massachusett": [9, 15, 29, 33, 41], "lead": [9, 12, 16, 17, 20, 21, 22, 26, 30, 31, 36, 37, 39, 47, 61], "loop": [9, 15, 33, 50], "policeexpenditur": [9, 15, 33], "crimeindex": [9, 15, 33], "separ": [9, 15, 20, 33, 43, 45, 49], "remark": [9, 29, 41, 47, 59], "cute": 9, "anscomb": 9, "quartet": 9, "four": [9, 11, 13, 16, 25, 29, 34, 60], "bi": [9, 16, 37], "varianc": [9, 22, 37, 44], "fourth": [9, 16], "betrai": 9, "explanatori": [9, 44, 51], "forev": 9, "rich": [9, 47, 54, 56], "link": [9, 20, 21, 22, 23, 24, 28, 36, 37, 38, 40, 42, 50, 54], "cheat": [9, 20, 22, 24, 37], "preced": [10, 13, 21, 25, 34], "establish": [10, 11, 40, 47], "gap": [10, 28, 32, 40, 46], "sort": [10, 11, 25, 40, 42, 47, 51, 52, 60], "femal": [10, 22, 28, 29, 37, 40, 41, 42, 48], "field": [10, 22, 23, 30, 37, 40, 45, 46], "remain": [10, 13, 22, 25, 34, 37, 52], "stem": 10, "decompos": 10, "man": [10, 28, 36, 40], "inequ": [10, 40, 57], "onboard": 11, "app": [11, 47, 49], "apach": 11, "licens": 11, "writeup": 11, "fair": [11, 21, 40], "materi": [11, 40, 42, 45, 46, 51, 56, 59], "click": [11, 23], "card": [11, 21], "enrol": [11, 21, 27, 46, 47], "automat": [11, 21, 34, 42, 48], "cancel": 11, "quizz": 11, "coach": 11, "support": [11, 12, 16, 18, 19, 30, 40, 42, 50], "verifi": 11, "certif": [11, 29, 31, 41], "feedback": [11, 16, 60], "devot": 11, "checkout": 11, "fewer": [11, 12, 25, 29, 42, 49, 50, 59], "messag": [11, 33], "continu": [11, 21, 22, 25, 37, 43, 47, 48], "screenshot": 11, "hope": [11, 14, 15, 23, 25, 30, 46, 49, 52, 56, 59], "upfront": 11, "frustrat": [11, 45], "left": [11, 29, 32, 43, 50, 51], "significantli": [11, 41, 45], "held": [11, 28, 40], "capac": 11, "www": [13, 20, 23, 24, 34, 36, 38, 41], "github": [11, 13, 15, 20, 22, 24, 28, 29, 32, 33, 34, 36, 37, 38, 40, 41, 42, 46], "mids_data": [13, 15, 20, 22, 24, 28, 32, 33, 34, 36, 37, 38, 40, 41, 42], "udacity_ab_test": [], "folder": [25, 28, 34, 40, 41], "visitor": 11, "roughli": [11, 16, 20, 36, 41], "row": [11, 13, 24, 32, 34, 37, 38, 40, 42, 54], "easiest": [11, 42, 59], "stack": 11, "pageview": 11, "uniqu": [11, 21, 22, 25, 32, 37], "homepag": 11, "payment": [11, 25], "Or": [11, 25, 29, 36, 37, 39, 40, 41, 42, 47], "m": [11, 15, 25, 33, 34, 38, 42, 49, 51, 54], "phrase": 11, "terminologi": 11, "encount": [11, 45, 59], "unintend": 11, "consequ": [11, 12, 26, 39, 49, 59], "correctli": [11, 29, 36, 37, 38, 40, 41, 42, 47, 48, 53, 61], "deem": 11, "half": [11, 13, 21, 29, 34, 51, 52, 61], "shown": [11, 46], "invari": [11, 13, 34], "affect": [11, 24, 30, 38, 42, 43, 44, 48, 52], "pre": [11, 13, 22, 34, 37, 47, 50, 60], "analog": [11, 42], "previous": [11, 13, 22, 34, 37, 45, 48, 52, 59], "qualiti": [11, 12, 22, 36, 37, 40, 54], "tricki": 11, "divid": [11, 16, 28, 40, 44, 50], "emerg": [11, 17, 40], "ttest": 11, "scipi": [11, 37, 38, 41], "pass": [11, 13, 21, 25, 34, 36, 42], "numpi": [11, 24, 37, 38, 40], "vector": [11, 22, 37, 42], "oper": [11, 12, 16, 21, 25, 40], "df": [11, 22, 32, 33, 34, 36, 37, 38, 40, 41, 42, 48], "my_column": 11, "93": [32, 33, 36, 37, 40], "consist": [11, 25, 59], "magic": [11, 43, 49, 52, 59], "bivari": 11, "close": [11, 22, 28, 30, 37, 40, 51], "float": [11, 42], "decreas": [11, 12, 13, 19, 23, 34], "covari": [22, 24, 29, 34, 36, 37, 38, 40, 41, 42, 44, 48, 53], "perfectli": [11, 48], "mondai": [], "adapt": [12, 22, 37, 51, 53], "permiss": [12, 23], "adrian": 12, "fresh": 12, "caus": [11, 12, 19, 25, 31, 43, 45, 47, 52, 53, 59], "statement": [12, 15, 17, 32, 59], "underlin": 12, "embed": 12, "gold": [12, 21], "typic": [12, 16], "law": [12, 13, 16, 29, 34, 41, 59], "cool": [12, 17, 49, 50], "absenc": 12, "arisen": 12, "instruct": [12, 18, 19, 40], "c": [12, 20, 22, 24, 33, 34, 36, 37, 38, 40, 41, 42, 48, 51], "infeas": 12, "briefli": 12, "nato": 12, "militari": 12, "command": [12, 32], "defeat": 12, "insurg": 12, "afghanistan": 12, "kill": [12, 60], "villag": [12, 49], "upset": 12, "civilian": [12, 40, 42], "hous": [12, 40], "member": [12, 14, 16, 44, 50], "afghani": 12, "coalit": [12, 30], "forc": [12, 40, 45], "war": [12, 13, 34, 59], "break": [12, 22, 23, 37, 44, 52], "hypothet": [12, 14], "farfetch": 12, "wildli": 12, "baselin": [11, 12, 14, 16, 22, 26, 27, 29, 37, 39, 41, 43, 51, 61], "milit": 12, "insan": [12, 37, 54], "unanswer": 12, "jason": 12, "lyall": 12, "insur": [12, 25, 27, 40, 47], "bought": [12, 52], "devic": 12, "alert": 12, "driver": [12, 17, 60], "unsaf": 12, "accid": [12, 14, 24, 38, 60], "wonder": [12, 22, 37], "britain": 12, "vote": [12, 48, 59], "european": [12, 47], "uk": 12, "brexit": 12, "immigr": 12, "attitud": 12, "proxim": 12, "favor": 12, "eu": 12, "2016": [12, 13, 32, 34], "aid": [12, 21, 24, 29, 38, 52, 56, 60, 61], "priorit": [12, 22, 37], "bed": 12, "famili": [12, 25, 40], "rural": [12, 47, 49, 52], "kenya": 12, "drug": [13, 32, 34, 40, 47, 49, 59, 60], "dealer": [13, 34, 40], "decad": [13, 22, 34, 37, 54, 59], "california": [13, 32, 34], "lessen": [13, 34], "penal": [13, 21, 34], "possess": [13, 34], "arguabl": [13, 34], "2010": [13, 21, 32, 34], "penalti": [13, 21, 25, 29, 34, 40, 41], "crimin": [13, 34], "civil": [13, 34], "guilti": [13, 34], "produc": [13, 34], "quantiti": [13, 26, 27, 34, 39, 41, 48], "illeg": [13, 34], "recreat": [13, 34, 40], "regulatori": [13, 16, 21, 34], "propon": [13, 34], "contribut": [13, 22, 30, 34, 37, 51, 59], "violenc": [13, 34], "gang": [13, 34], "battl": [13, 34], "territori": [13, 34], "accord": [13, 30, 34, 36, 56], "argument": [13, 22, 25, 32, 34, 37, 51, 59], "simpli": [13, 34, 46, 59], "semest": [13, 34, 60], "com": [13, 15, 20, 22, 23, 24, 28, 33, 34, 36, 37, 38, 40, 41, 42, 48], "uds_arrest_data": [13, 32, 34], "csv": [13, 15, 24, 32, 33, 34, 38, 48], "attornei": [13, 32], "offens": [13, 34], "f_drugoff": [13, 32, 34], "feloni": [13, 34], "total_popul": [13, 32, 34], "total": [11, 13, 22, 24, 25, 29, 32, 34, 37, 38, 48, 56], "began": [13, 34, 47, 59], "liber": [13, 34], "2007": [13, 25, 32, 34, 37, 51], "2009": [13, 21, 25, 32, 34], "violent_r": [13, 34], "among": [13, 16, 28, 29, 31, 34, 40, 41, 42, 44, 45, 47, 48, 49, 52], "2008": [13, 25, 32, 34], "special": [13, 21, 34, 40, 42, 47, 54, 59], "y_": [13, 27, 34, 39], "avg": [13, 34, 38, 40, 42], "hat": [13, 34], "delta": [13, 34], "_": [13, 24, 34, 38], "took": [13, 30, 34, 59], "ignor": [11, 13, 21, 28, 34, 40, 41, 42], "middl": [13, 34, 37, 43, 48, 51, 59], "decrimin": [13, 34], "parallel": [13, 34, 61], "2000": [13, 32, 34, 40, 47], "inclus": [13, 22, 34, 37], "geometr": [13, 34], "confid": [13, 21, 23, 34, 41, 42, 47, 48, 51, 61], "panelol": [13, 24, 34, 38, 42], "lfe": [13, 24, 34, 38], "r": [13, 21, 24, 34, 36, 38, 40, 41, 42, 46, 48], "client": [14, 16], "offset": 14, "worthwhil": [14, 59], "durham": [14, 59], "nc": [14, 38], "index": [14, 15, 22, 26, 32, 33, 37, 39, 42], "rest": [14, 47, 51, 59, 60], "consumpt": [14, 20, 24, 27, 31, 36, 38], "healthier": 14, "rental": [14, 40], "renter": 14, "deduct": 14, "fee": 14, "pocket": 14, "lose": [14, 28, 40, 42, 51, 61], "hassl": 14, "fleet": 14, "agreement": [14, 21], "tesla": 14, "dealership": 14, "hillsboro": 14, "town": 14, "heard": 14, "center": [14, 40, 46, 52], "examin": [15, 22, 32, 33, 37, 59], "blob": [15, 22, 33, 36, 37, 40, 41, 42], "descriptive_exercis": [15, 33], "crime_expend_ma": [15, 33], "monthli": [15, 25, 33], "budget": [15, 33], "late": [15, 25, 33, 47], "2001": [15, 29, 32, 33, 34, 41], "county_cod": [15, 33], "print": [15, 23, 33, 34, 36, 38, 40, 41], "output": [15, 16, 24, 33, 38, 42, 48], "gonna": [15, 33, 36, 61], "75": [15, 32, 33, 37, 38], "83": [15, 32, 33, 37, 41], "62": [15, 33, 34, 37, 41], "55": [15, 33, 36, 37, 38], "88": [15, 32, 33, 37], "format": [15, 54], "guess": [15, 20, 23, 33, 36, 41, 47, 52, 59], "knew": [15, 33, 61], "till": [15, 33, 42, 50], "scatter": [15, 20, 24, 33, 34, 36, 38], "advisori": 16, "firm": [16, 25, 48, 54], "compliant": 16, "authent": 16, "capston": [16, 30, 35], "undertaken": [16, 30], "corpor": [16, 30], "intersect": 16, "top": [16, 52, 61], "bank": [16, 40], "fintech": 16, "lend": 16, "core": [16, 17, 30, 51, 52, 59], "capabl": 16, "lender": 16, "led": [16, 30], "reli": 16, "challeng": [16, 46, 47], "tradit": [16, 30], "partial": [16, 21, 22, 37], "shaplei": 16, "regul": [16, 39, 59], "delinqu": 16, "loan": [16, 25, 35, 54], "burden": 16, "champion": 16, "gradient": [16, 21], "boost": [16, 21], "gbm": [16, 35], "viabil": 16, "ebm": [16, 21, 35], "publicli": [16, 52, 54], "complianc": [16, 47, 53], "overlai": 16, "enforc": 16, "kdd": 16, "cup": 16, "191k": 16, "481": [16, 40, 41], "mail": [16, 40, 47], "creation": [16, 21, 30], "weekli": [16, 28, 40, 42, 43], "stretch": 16, "methodologi": 16, "advers": 16, "satisfi": [16, 49], "contact": 16, "redact": 16, "outlin": [16, 61], "undertak": [16, 53, 59, 61], "deliver": [16, 30], "uglier": 17, "realist": 17, "motor": [17, 40, 52], "conglomer": 17, "america": [17, 52], "manufactur": [17, 20, 36, 40], "announc": [17, 54, 61], "enhanc": 17, "brake": 17, "lane": 17, "departur": 17, "warn": [17, 21, 42, 48, 58], "keep": [17, 21, 22, 25, 35, 37, 45, 47, 50, 51, 56, 59, 61], "drowsi": 17, "ey": [17, 32, 37], "squirt": 17, "water": [17, 18, 19, 40], "wake": 17, "fortun": 17, "weaker": 17, "prone": 17, "injuri": 17, "activ": [17, 26, 39, 40, 54, 56, 58, 59], "tracker": 17, "sheet": 18, "aloud": [18, 19], "devis": [18, 19], "waterborn": [18, 19], "pathogen": [18, 19], "technologi": [18, 19], "falsifi": [18, 19], "inconsist": [18, 19, 48], "germ": [18, 19], "spread": [18, 19], "worksheet": [18, 19], "walk": [18, 19], "john": [18, 19, 59], "snow": [18, 19], "london": [18, 19], "diseas": [18, 19, 49], "miasma": 18, "lesson": [18, 22, 37, 50], "1854": [18, 19, 48], "street": [18, 19, 61], "pump": [18, 19], "neighborhood": [18, 19, 47, 54, 59], "soho": [18, 19], "suppli": [18, 19, 40], "drink": [18, 19, 26, 39, 40], "cook": [18, 19], "household": [18, 19, 40, 42, 49], "chore": [18, 19], "indoor": [18, 19], "plumb": [18, 19], "unheard": [18, 19], "meant": [11, 18, 19, 54, 59], "obtain": [18, 19, 52], "corner": [18, 19], "throughout": [18, 19, 25, 43, 59], "citi": [18, 19, 25, 42, 43, 44, 59, 60], "despit": [18, 47, 59], "worker": [18, 28, 40], "breweri": 18, "outbreak": 18, "food": [18, 40], "bakeri": [18, 40], "exhibit": 18, "involv": [18, 60, 61], "retail": [18, 40, 42], "drank": [18, 26], "beer": [18, 24, 38, 40], "hydrat": 18, "brew": 18, "boil": 18, "famous": [18, 59], "map": [18, 45], "counter": [18, 28, 40, 45, 59], "factual": [18, 28, 40, 59], "Will": [18, 19], "space": [18, 19, 20, 21, 30, 36, 40, 42, 50, 54, 60], "incorrect": [18, 19, 21], "brewer": 18, "theoriz": 19, "contamin": 19, "author": 19, "dismantl": 19, "handl": [19, 32], "septemb": 19, "subsequ": [19, 22, 25, 32, 37, 45, 60], "7th": 19, "canon": [20, 36], "mileag": [20, 36], "74": [20, 32, 36, 37], "automobil": [20, 36, 40], "april": [20, 36], "1979": [20, 36, 37], "epa": [20, 36], "fuel": [20, 36, 40], "compil": [20, 36, 54, 59], "chamber": [20, 36], "1983": [20, 32, 34, 36, 38], "automobile_dataset": [20, 36], "file": [11, 20, 25, 28, 29, 32, 36, 40, 48, 54], "internet": [20, 36, 40], "codebook": [20, 36], "mpg": [20, 36], "rep78": [20, 36], "repair": [20, 36, 40], "1978": [20, 36], "headroom": [20, 36], "trunk": [20, 36], "cu": [20, 36], "ft": [20, 36], "lb": [20, 36], "circl": [20, 36, 61], "displac": [20, 36], "gear_ratio": [20, 36], "gear": [20, 36], "ratio": [11, 20, 25, 36, 53], "foreign": [20, 36], "guzzler": [20, 36], "mile": [20, 36], "gallon": [20, 36], "ga": [20, 36, 38, 40], "guzzl": [20, 36], "marker": [20, 36], "red": [20, 36, 51], "green": [20, 36], "confirm": [20, 36, 48], "q3": [20, 36], "poorli": [20, 36, 47, 52, 54], "poor": [20, 36, 44, 47], "respect": [20, 21, 36, 41, 47, 48, 53], "abroad": [20, 36], "grandma": [20, 36], "domest": [20, 36], "main": [20, 24, 36, 38, 51, 61], "layman": [20, 36], "cynthia": 21, "rudin": 21, "commentari": 21, "arxiv": 21, "plethora": 21, "alter": 21, "appeal": 21, "inher": 21, "flaw": [21, 59], "favorit": 21, "passag": [21, 45], "her": 21, "faith": 21, "cannot": [21, 22, 37, 47, 48, 59], "fidel": 21, "equal": [21, 25, 36, 48], "represent": 21, "extens": 21, "tenth": 21, "14th": 21, "pacif": [21, 47], "asia": [21, 52], "discoveri": [21, 59], "mine": [21, 40], "confer": [21, 30], "pakdd": 21, "2006": [21, 25, 32, 34], "privat": [21, 37, 40], "brazilian": 21, "partner": [21, 60], "timi": 21, "pure": 21, "coincid": 21, "dissert": 21, "confus": [21, 25, 48, 56, 59], "glm": 21, "mvgam": 21, "pioneer": 21, "statsmodel": [21, 22, 29, 34, 36, 37, 38, 40, 41, 42, 48], "hide": [21, 49], "plain": [21, 26, 27, 39, 47], "sight": 21, "meta": 21, "prophet": 21, "seri": [21, 32], "forecast": 21, "librari": [21, 22, 25, 29, 37, 40, 41, 42, 48], "padd": 21, "repositori": [21, 54], "pakdd2010_variableslist": 21, "xl": 21, "pakdd2010_modeling_data": 21, "txt": [21, 25, 28, 40], "string": [21, 22, 25, 29, 37, 42], "load": [11, 21, 22, 23, 24, 25, 29, 37, 38, 41, 42, 48, 60], "latin": [21, 32, 52], "latin8": 21, "prior": [21, 46], "utf": 21, "hint": [21, 22, 24, 25, 32, 37, 38, 41], "strictli": [21, 56], "stage": 21, "missing": 21, "quant_depend": 21, "quant_car": 21, "months_in_resid": 21, "personal_monthly_incom": 21, "quant_banking_account": 21, "marital_statu": 21, "occupation_typ": 21, "residence_typ": 21, "residencial_st": 21, "residencial_c": 21, "residencial_borough": 21, "residencial_zip_3": 21, "banana": 21, "log": [21, 29, 34, 36, 37, 38, 40, 41, 42, 48, 51], "geograph": [21, 30, 54], "segreg": 21, "resid": [21, 59], "resolv": 21, "residencial_": 21, "train_test_split": [21, 25], "80": [21, 32, 37, 40], "target_label_bad": 21, "classif": [21, 25], "gridsearch": 21, "spline": 21, "slow": [21, 42], "50": [21, 27, 32, 33, 36, 37, 38, 41, 42, 48, 49], "residencial_zip": 21, "minut": [21, 50], "configur": 21, "naiv": [21, 41], "matrix": [21, 24, 25, 36, 37, 38, 40, 42, 44], "extend": [21, 43], "default": [11, 21, 22, 25, 32, 35, 37], "omiss": [21, 25], "predict_proba": [21, 25], "shift": 21, "emphasi": 21, "threshold": [21, 22, 23, 25, 37, 50], "toler": 21, "interv": [21, 23, 48], "partial_depend": 21, "pdep": 21, "confi": 21, "xx": 21, "width": 21, "valueerror": [21, 22, 37, 42], "recod": 21, "refit": 21, "impos": [21, 35, 60], "concav": 21, "convex": 21, "overfit": [11, 21], "smooth": [21, 48], "paramet": [21, 29, 34, 38, 42], "lambda": [21, 32], "push": [21, 49], "smoother": [21, 48], "grid": 21, "doc": 21, "explainableboostingclassifi": 21, "classmat": 21, "prefer": [21, 22, 29, 37, 39, 41, 44, 46, 59, 61], "glassbox": 21, "x_train": 21, "y_train": 21, "catch_warn": 21, "simplefilt": 21, "ebm_glob": 21, "explain_glob": 21, "ebm_loc": 21, "explain_loc": 21, "matchit": [22, 37], "rpy2": [22, 37], "statistician": [22, 37, 44], "awar": [22, 37, 49, 59, 60], "matur": [22, 37], "purpos": [22, 37, 42, 44, 59], "techniqu": [22, 37, 46], "frontier": [22, 37], "hyper": [22, 37], "pip": [22, 37, 42], "conda": [22, 37, 42], "exact": [22, 37, 47], "pair": [22, 32, 37, 50, 51, 60], "repeat": [22, 37], "trick": [22, 37], "massiv": [22, 37, 47, 59], "computation": [22, 24, 37, 38], "amaz": [22, 29, 37, 41, 54, 59], "month": [22, 25, 27, 33, 37, 47, 49], "cp": [22, 28, 37, 40, 42], "current_population_survei": [22, 28, 37, 40, 42], "cps_for_match": [22, 37], "content": [22, 37, 39, 46], "overlap": [22, 37, 50, 51], "unsur": [11, 22, 37], "imbal": [22, 29, 37, 50], "annual_earn": [22, 37, 40, 42], "has_colleg": [22, 37], "plotnin": [22, 37, 48], "geom_smooth": [22, 37, 48], "lowess": [22, 37, 48], "altair": [22, 33, 34, 36, 37], "transform_loess": [22, 37], "tutori": [22, 37, 48], "speak": [22, 37, 45], "discret": [22, 37, 48, 52], "employ": [22, 28, 29, 37, 40, 41], "merg": [22, 25, 32, 34, 37, 54], "covert": [22, 37], "convert": [22, 25, 37, 42, 48], "class94": [22, 37], "interg": [22, 37], "belong": [22, 28, 37, 40, 42, 43, 48, 59], "passion": [22, 37], "var": [22, 34, 37, 57], "pull": [22, 37, 42], "integ": [22, 37, 48], "reset": [22, 37], "dame_flam": [22, 37], "syntax": [22, 37, 48], "my_data": [22, 37], "verbos": [22, 37], "want_p": [22, 37], "treatment_column_nam": [22, 37], "outcome_column_nam": [22, 37], "printout": [22, 37], "iter": [22, 37], "unmatch": [22, 37], "empti": [22, 37], "hasn": [22, 37, 59], "370": [22, 36, 37, 40, 48], "644": [22, 37], "1150": [22, 37], "3187": [22, 37], "4365": [22, 37], "1684": [22, 37], "1199312680": [22, 37], "0957854": [22, 37], "684": [22, 37], "subset": [22, 25, 28, 37, 40], "secret": [22, 37], "sauc": [22, 37], "ridg": [22, 37], "elimin": [22, 37, 51], "626": [22, 36, 37, 40], "494": [22, 37], "180": [22, 37], "3626": [22, 37], "5310": [22, 37], "frozenset": [22, 37], "1199421883": [22, 37], "1095908": [22, 37], "recal": [11, 22, 23, 29, 37, 47], "becom": [22, 37, 43, 47, 48, 59], "manual": [22, 36, 37], "early_stop_iter": [22, 37], "evolv": [22, 37, 59], "pe_each_it": [22, 37], "pe": [22, 37], "falloff": [22, 37], "complaint": [22, 37], "entri": [22, 32, 37, 42], "simplified_rac": [22, 37], "discretized_ag": [22, 37], "24": [22, 25, 32, 33, 36, 37, 40, 41, 51], "copi": [22, 37, 42], "def": [22, 37], "get_datafram": [22, 37], "result_of_fit": [22, 37], "input_data": [22, 37], "loc": [22, 33, 34, 37, 41], "is_uniqu": [22, 37], "match_group": [22, 37], "np": [22, 25, 36, 37, 38, 40], "nan": [22, 25, 32, 36, 37, 40, 42], "match_group_s": [22, 37], "idx": [22, 37], "enumer": [22, 37], "units_per_group": [22, 37], "len": [22, 37], "t_in_group": [22, 37], "groupbi": [22, 37, 38], "transform": [22, 37, 38], "sum": [22, 32, 37], "treament": [22, 37], "num": [22, 37], "control_weight": [22, 37], "num_control_ob": [22, 37], "drop_dupl": [22, 37], "renorm": [22, 37], "assert": [11, 22, 25, 29, 32, 34, 37, 38, 59], "notnul": [22, 37, 42], "return": [22, 31, 32, 37, 40, 46, 47, 59], "wl": [22, 37], "matched_data": [22, 37], "smf": [22, 34, 36, 37, 38, 40, 41, 42, 48], "educt": [22, 37], "ba": [22, 28, 37, 40], "tast": [22, 37], "permut": [22, 37], "closest": [22, 37], "multipl": [22, 24, 25, 37, 38, 43, 46, 47, 60], "twice": [22, 29, 37, 39, 43], "malt": [22, 37], "mahalanobi": [22, 37, 50], "distanc": [22, 30, 37, 50], "ulik": [22, 37], "clever": [22, 37], "arbitrari": [22, 37], "explod": [22, 37, 59], "aeml": [22, 37], "planner": 23, "georgi": 23, "georgiev": [4, 23], "hi": [23, 36, 51, 59], "toolkit": 23, "processor": 23, "latenc": 23, "delai": 23, "chip": 23, "justifi": 23, "05": [23, 36, 37, 40, 41, 48], "millisecond": 23, "smaller": [23, 29, 34, 41, 42, 44, 50, 51, 52], "god": 23, "forbid": 23, "500": [23, 34, 40], "costli": [11, 23, 53], "revers": 23, "labor": [23, 40], "50m": 23, "15m": 23, "wanna": [23, 48], "sequenti": 23, "agil": 23, "valuabl": [23, 51], "5m": 23, "hover": 23, "interpet": 23, "99": [23, 37, 42, 49], "densiti": 23, "lift": 23, "f": [23, 33, 34, 36, 37, 38, 40, 41, 42, 48], "alcohol": [24, 38, 40, 60], "fatal": [24, 38, 59, 60], "336": [24, 34, 37, 38, 40], "34": [24, 34, 36, 37, 38, 48], "beertax": [24, 38], "drunk": [24, 38], "intuit": [24, 38, 44, 45], "us_driving_fat": [24, 38], "frame": [24, 38], "fat_rat": [24, 38], "pop": [24, 28, 38, 40, 48], "python": [24, 36, 38, 41, 44, 46, 60], "navig": [24, 38, 40], "ol": [24, 34, 36, 37, 38, 40, 41, 42, 48], "align": [24, 38], "fatalityrate_i": [24, 38], "beta_0": [24, 38, 48], "beta_1": [24, 38, 48, 51], "beertax_i": [24, 38], "notat": [24, 38, 45, 48], "whei": [24, 38], "fatalityrate_": [24, 38], "alpha": [24, 38, 51, 53], "beta": [24, 38, 53], "beertax_": [24, 38], "psi": [24, 38], "z_i": [24, 38], "epsilon_": [24, 38], "sum_": [24, 38], "overlin": [24, 38], "fatalityr": [24, 38], "_i": [24, 38], "epsilon": [24, 38, 48, 51], "substract": [24, 38], "equat": [24, 38, 48, 59], "yield": [24, 38], "overset": [24, 38], "sim": [24, 38, 42], "x_": [24, 38], "intercept": [24, 34, 36, 37, 38, 40, 41, 42, 44, 48], "from_formula": [24, 34, 38, 42], "arrai": [24, 32, 38, 42], "phenomenon": [24, 38, 59], "state_i": [24, 38], "year_t": [24, 38], "homeown": 25, "newli": 25, "home": [25, 40, 47], "freddi": 25, "mac": 25, "calendar": 25, "dictionari": [11, 25, 29], "exercise_passive_predict": 25, "ipynb": [11, 25, 29], "upload": [11, 25, 29], "ex2_merge_typ": 25, "ex4_num_mortgag": 25, "ex5_num_ob": 25, "ex7_num_mortgag": 25, "ex7_share_delinqu": 25, "ex10_num_ob": [11, 25], "ex12_roc_auc": 25, "ex14_false_omission_r": 25, "ex16_num_ob": 25, "ex16_share_delinqu": 25, "ex17_false_omission_r": 25, "clariti": [11, 25, 29], "supplement": 25, "scope": 25, "sample_orig_2004": 25, "2004": [25, 32, 34], "sample_svcg_2004orig_3year": 25, "2005": [25, 32, 34], "url": [25, 29], "AND": [25, 36, 47], "Be": 25, "keyword": [25, 32, 42], "inner": 25, "join": 25, "quarter": 25, "vanilla": 25, "sequenc": [25, 52, 59], "behalf": 25, "element": [25, 26, 27, 39], "cycl": 25, "perceiv": [25, 29, 41], "anomali": 25, "mismatch": [25, 53], "investor": 25, "commonli": [25, 42, 50], "section": [25, 59, 61], "cleanli": 25, "correspond": [25, 26, 27, 29, 39], "borrow": [25, 59], "instal": [25, 42], "ddlpi": 25, "banker": 25, "mba": 25, "acquir": 25, "reo": 25, "59": [25, 32, 33, 36, 37, 38], "60": [25, 34, 37, 38, 48], "89dai": 25, "119dai": 25, "ra": 25, "acquisit": 25, "issuanc": 25, "tidi": 25, "holder": [25, 30, 52], "metropolitan": 25, "msa": 25, "divis": 25, "deploy": 25, "homebuy": 25, "percentag": [11, 25, 28, 29, 40, 41, 48], "mi": [25, 38, 40, 42, 47, 52], "occup": [25, 40, 42], "debt": 25, "dti": 25, "upb": 25, "ltv": 25, "channel": 25, "prepay": 25, "ppm": 25, "amort": 25, "formerli": 25, "o": 25, "distinct": [25, 51, 59], "hot": [25, 48], "seed": [25, 54], "fan": 25, "patsi": 25, "dmatric": 25, "binari": [25, 48], "number_of_borrow": 25, "number_of_unit": 25, "model_select": 25, "test_siz": 25, "random_st": 25, "gradientboostingclassifi": 25, "ensembl": 25, "roc_auc_scor": 25, "roc": 25, "auc": [25, 46], "round": [11, 25, 29], "decim": [11, 25, 29, 51], "instabl": 25, "classifi": 25, "viabl": 25, "riski": 25, "08": [25, 36, 38, 48], "saniti": [11, 25], "cutoff": 25, "retrospect": 25, "move": [25, 26, 27, 39, 40, 42, 48, 51, 59], "stai": [25, 49], "onto": [26, 27, 39, 45], "wellb": [26, 39], "sit": [26, 39], "front": [26, 39, 61], "bodi": [26, 30, 39, 40, 61], "bmi": [26, 39], "soda": [26, 39, 57], "dine": [26, 39], "diet": [26, 39, 57], "sugar": [26, 39, 40], "healthi": [26, 39, 59], "sugari": [26, 39], "campu": [26, 29, 30, 39, 41], "0_i": 26, "1_i": 26, "d_i": 26, "english": [26, 27, 39, 45], "neq": [26, 27, 39, 43], "hr": 27, "depart": [27, 40, 42], "wemakewidget": 27, "llc": 27, "skyrocket": 27, "wmw": 27, "nurs": [27, 40, 42, 47], "via": [27, 30], "send": [27, 47, 61], "exchang": 27, "appoint": 27, "bonu": 27, "mandatori": 27, "enrolle": 27, "woman": [28, 40, 48], "morg18": [28, 40, 42], "stata": [28, 40], "readm": [28, 40, 54], "nation": [28, 40, 52, 59], "lfsr94": [28, 40, 42], "uhours": [28, 40, 42], "35": [28, 37, 38, 40, 41, 42, 48], "couldn": [28, 40, 59], "chose": [28, 29, 40, 47, 61], "earnwk": [28, 40, 42], "hourli": [28, 40], "48": [28, 32, 34, 36, 37, 38, 40, 42], "implicit": [28, 40, 49, 51], "pretend": [28, 40], "grade92": [28, 40, 42], "nest": [28, 40, 42, 43, 44, 59], "structur": [28, 30, 40, 42, 44, 47, 59, 61], "ind02": [28, 40, 42], "hierarch": [28, 40, 42], "serv": [28, 30, 40, 60], "slightli": [28, 37, 40, 42, 61], "wherea": [28, 40], "introduc": [28, 40, 47, 59], "harder": [29, 41, 56], "express": [29, 41], "870": [29, 32, 37, 40, 41], "fictici": [29, 41], "advert": [29, 41], "boston": [29, 41], "chicago": [29, 41], "alloc": [29, 41], "children": [29, 41, 42, 44, 49, 52], "subscrib": 29, "kwame": 29, "anthoni": 29, "appiah": 29, "capit": 29, "w": 29, "journal": [29, 41, 61], "exercise_resume_experi": 29, "ex2_pvalue_computerskil": 29, "ex2_pvalue_femal": 29, "ex2_pvalue_yearsexp": 29, "ex3_pvalue_educ": 29, "ex4_valid": 29, "ex5_pvalu": 29, "ex5_white_advantage_perc": 29, "ex5_white_advantage_percentage_point": 29, "ex6_black_pvalu": 29, "ex8_black_colleg": 29, "ex8_black_nocolleg": 29, "ex8_college_heterogen": 29, "ex9_gender_and_discrimin": 29, "ex10_experiment_v_u": 29, "profoundli": [29, 41], "outset": [29, 41], "unbalanc": [29, 41], "resume_experi": [29, 41], "fictiti": [29, 41], "interview": [29, 41], "yearsexp": [29, 41], "computerskil": [29, 41], "ofjob": [29, 41], "qualif": [29, 41], "magnitud": [29, 30, 41, 44], "tabul": [29, 41], "dropout": [29, 41], "chi": [29, 41], "conting": [29, 41, 56], "crosstab": [29, 37, 41], "markdown": 29, "back": [29, 41, 43, 51, 52, 61], "heteroskedast": [29, 41, 48], "robust": [29, 30, 34, 38, 41, 42, 44, 48, 61], "hc3": [29, 41, 48], "upsid": [29, 45], "downsid": [29, 45], "particularli": [29, 45, 59], "migrain": [29, 49], "aimovig": [29, 49], "denot": [11, 29], "lookup": [29, 41], "paus": [29, 41, 59, 61], "followup": [29, 30, 41], "ident": [29, 41, 50], "piec": [29, 40, 41, 45, 52], "crucial": [29, 41, 59, 61], "successfulli": [30, 52], "csr": 30, "2020": [30, 48], "partnership": 30, "expand": 30, "summer": 30, "41": [30, 34, 37], "leader": 30, "elect": [30, 54], "poll": 30, "ballot": 30, "satellit": 30, "thank": [30, 37], "incred": 30, "500k": 30, "cast": 30, "2013": [30, 32], "suprem": 30, "shelbi": 30, "struck": 30, "provis": [30, 60], "1965": 30, "act": [30, 47, 53], "voter": [30, 48], "preclear": 30, "feder": 30, "2019": [30, 32, 34, 42, 59], "leadership": 30, "1600": [30, 40], "becam": 30, "closur": 30, "comprehens": [30, 59], "sponsor": 30, "aim": [30, 59], "eas": 30, "thereof": 30, "dropbox": [30, 32], "youth": 30, "equip": [30, 40], "young": [30, 45, 53, 59], "backup": 30, "advocaci": [30, 40], "surround": 30, "shine": 30, "brighter": 30, "greatest": 30, "proactiv": 30, "2022": [30, 36], "institut": [30, 40, 54], "qualit": 30, "extent": 30, "hbcu": 30, "msi": 30, "primarili": [30, 46, 51, 54, 59], "minor": [30, 42], "mere": 30, "travel": [30, 40], "nearest": 30, "booth": 30, "foot": 30, "transport": [30, 40, 42], "landscap": [30, 40, 52], "dichotomi": 30, "tank": [31, 40], "combat": 31, "global": [31, 46, 60], "warm": 31, "implic": [31, 61], "leed": 31, "prison": 31, "recidiv": 31, "former": 31, "inmat": 31, "curiou": 31, "violent": [32, 34], "scroll": [32, 47], "mb": 32, "43": [32, 36, 37, 40, 41, 42], "read_csv": [32, 33, 34, 38, 48], "openjustic": 32, "doj": 32, "ca": [32, 37, 38], "gov": [32, 54], "06": [32, 33, 34, 40, 41, 42, 48], "onlinearrestdata1980": 32, "44": [32, 37, 40], "head": [32, 33, 34, 37, 38, 40, 41, 42, 48, 52], "age_group": 32, "f_sexoff": 32, "f_alloth": 32, "f_total": 32, "m_total": 32, "s_total": 32, "alameda": [32, 34], "505": [32, 37, 40], "1351": 32, "188": [32, 37, 38], "79": [32, 37, 40, 42], "2149": 32, "2286": 32, "295": [32, 37, 40], "butt": [32, 34], "calavera": [32, 34], "contra": [32, 34], "costa": [32, 34], "116": [32, 37, 40], "446": [32, 37, 40, 48], "37": [32, 36, 37, 38, 40, 42, 48], "629": [32, 36, 37, 40], "557": [32, 36], "31": [32, 37, 40, 48], "el": [32, 34], "dorado": [32, 34], "45": [32, 34, 37, 38, 40, 41, 42, 50], "value_count": [32, 34, 37, 48], "26590": 32, "23767": 32, "23328": 32, "21188": 32, "dtype": [32, 34, 37, 42, 48], "int64": [32, 34, 37, 48], "46": [32, 36, 37, 41], "17438": 32, "17258": 32, "40": [32, 34, 36, 37, 40, 48, 49], "69": [32, 34, 36, 37, 38, 40], "17082": 32, "16305": 32, "16040": 32, "70": [32, 36, 37], "10750": 32, "duplic": 32, "social_security_numb": 32, "111111111": 32, "222222222": 32, "333333333": 32, "second_column": 32, "bool": 32, "instanc": [32, 42], "repit": 32, "sub": [32, 49, 53], "49": [32, 36, 37], "secur": [32, 40], "boolean": [32, 48], "idenfi": 32, "backward": 32, "51": [32, 37, 48], "assertionerror": 32, "traceback": 32, "lt": [32, 33, 38, 48], "ipython": 32, "f30d4b630726": 32, "gt": [32, 33, 38, 48], "modul": 32, "56": [32, 37, 38], "collaps": 32, "aggreg": [32, 54], "57": [32, 34, 37, 38, 40], "arrests_collaps": 32, "as_index": [32, 34], "4504": [32, 34], "9377": 32, "3569": [32, 34], "257": [32, 34, 36, 37], "1651": 32, "19358": 32, "71342": 32, "2338": [32, 40], "alpin": [32, 34], "119": [32, 37, 40], "amador": [32, 34], "305": [32, 37], "306": [32, 37, 40], "683": [32, 34, 37, 40], "149": [32, 37, 38, 40, 48], "207": [32, 34, 37], "1365": [32, 36], "52": [32, 37, 38], "130": [32, 37, 40, 42, 48], "297": [32, 37], "679": [32, 36], "ball": 32, "sort_valu": [32, 37], "colusa": [32, 34], "89": [32, 36, 37], "138": [32, 37, 40], "655": [32, 40], "1596": 32, "3851": 32, "1089": 32, "929": 32, "7539": 32, "25788": 32, "590": [32, 37], "del": [32, 34], "nort": [32, 34], "183": [32, 37, 40], "357": [32, 37, 48], "1078": 32, "198": [32, 37], "574": [32, 48], "118": [32, 36, 37], "1080": 32, "3266": 32, "fresno": [32, 34], "2367": [32, 40], "3855": 32, "804": [32, 40], "105": [32, 37], "719": [32, 37, 40], "7850": 32, "37907": 32, "1178": [32, 41], "glenn": [32, 34], "67": [32, 36, 37, 48], "111": [32, 36, 37, 40], "38": [32, 33, 37, 41, 42], "250": [32, 37, 40, 41], "766": [32, 37, 40, 42], "humboldt": [32, 34], "211": [32, 34, 36, 37, 38, 40], "593": [32, 36, 38, 40], "196": [32, 37, 38, 40], "33": [32, 36, 37, 38, 40, 42], "212": [32, 38], "1245": 32, "5953": 32, "197": [32, 34, 36, 37], "imperi": [32, 34], "565": [32, 36, 40], "1045": 32, "97": [32, 37, 38], "163": [32, 37, 38, 40], "1886": 32, "6512": 32, "inyo": [32, 34], "158": [32, 37, 40, 42], "315": [32, 37, 40], "663": [32, 36, 40], "kern": [32, 34], "1732": 32, "3530": 32, "1098": 32, "401": [32, 38], "6896": 32, "26372": 32, "890": [32, 40], "63": [32, 37, 42], "64": [32, 37, 38, 40, 41], "mids_data_prep": 32, "county_demograph": 32, "census_ts_nominal_counti": 32, "gisjoin": [32, 54], "statefp": 32, "statenh": 32, "countyfp": 32, "countynh": 32, "b18aa": 32, "cv4aa": 32, "cv4ab": 32, "cv4ac": 32, "cv4ad": 32, "cv4ae": 32, "cv4af": 32, "cv4ag": 32, "cv4ah": 32, "cv4ai": 32, "cv4aj": 32, "g0100010": 32, "1970": 32, "alabama": 32, "autauga": 32, "17511": 32, "g0100030": 32, "baldwin": 32, "48650": 32, "g0100050": 32, "barbour": 32, "12134": 32, "g0100070": 32, "bibb": 32, "9953": 32, "g0100090": 32, "blount": 32, "26205": 32, "65": [32, 33, 37], "66": [32, 37, 38, 40], "b18ab": 32, "b18ac": 32, "b18ad": 32, "b18ae": 32, "dropna": 32, "514559": 32, "265294": 32, "17996": 32, "103958": 32, "41887": 32, "744282": 32, "130795": 32, "244309": 32, "21154": 32, "884": [32, 37], "232": [32, 34, 37, 40], "6471": [32, 37], "73615": 32, "2622": 32, "1859": 32, "31207": 32, "31742": 32, "2272": 32, "3195": 32, "159": [32, 37, 38, 42], "41873": 32, "350": [32, 37, 40, 41, 42], "799": [32, 37], "28913": 32, "3357": 32, "san": [32, 34], "joaquin": [32, 34], "266886": 32, "19288": 32, "4474": [32, 40], "21738": 32, "237799": 32, "18443": 32, "23765": 32, "631": 32, "29087": 32, "845": [32, 37, 38, 40], "2447": 32, "34325": 32, "9608": 32, "621490": 32, "88813": 32, "5830": 32, "183969": 32, "48714": 32, "549409": 32, "86851": 32, "109486": 32, "2636": 32, "32658": 32, "72081": 32, "1962": 32, "3803": 32, "73874": 32, "16056": 32, "12800": 32, "ventura": [32, 34], "565804": 32, "15163": 32, "8068": 32, "197342": 32, "36941": 32, "400868": 32, "13082": 32, "57841": 32, "1371": [32, 40], "18589": 32, "164936": 32, "2081": 32, "7316": 32, "138882": 32, "18352": 32, "12764": 32, "madera": [32, 34], "94456": 32, "5629": 32, "4136": 32, "40344": 32, "6300": 32, "57380": 32, "5009": 32, "4430": 32, "649": [32, 37, 40], "2405": 32, "37076": 32, "620": [32, 37, 40], "2670": 32, "36731": 32, "3895": [32, 40], "9635": 32, "sacramento": [32, 34], "783240": 32, "121804": 32, "13359": 32, "233704": 32, "71392": 32, "706655": 32, "118073": 32, "148459": 32, "3406": 32, "51016": 32, "76585": 32, "3731": [32, 40], "7063": 32, "88135": 32, "20376": 32, "9606": 32, "36982": 32, "304": [32, 37, 38], "705": [32, 36, 37, 40, 41], "1222": 32, "1341": 32, "35465": 32, "300": [32, 37, 40, 44], "965": [32, 34, 37, 40], "1019": 32, "1517": [32, 37, 40], "123": [32, 37, 38, 40], "322": [32, 34, 37, 40], "6510": 32, "tehama": [32, 34], "45593": 32, "256": [32, 37], "966": [32, 37], "2810": 32, "43049": 32, "246": [32, 37, 38, 40], "1174": 32, "32": [32, 33, 37, 38, 40], "2544": [32, 37, 40], "145": [32, 37, 41], "2425": 32, "3321": [32, 40], "18469": 32, "177": [32, 36, 37, 38], "299": [32, 37, 38], "17811": 32, "363": [32, 38, 40], "658": [32, 36], "255": [32, 37, 40], "12792": 32, "solano": [32, 34], "210751": 32, "60750": 32, "3212": [32, 37, 40], "107273": 32, "31358": 32, "168628": 32, "58743": 32, "64134": 32, "1463": [32, 40], "21020": 32, "42123": 32, "3115": [32, 40], "41773": 32, "10338": 32, "68": [32, 33, 36, 37, 40, 42], "race_table_var": 32, "fillna": 32, "71": [32, 36, 37, 48], "72": [32, 36, 37, 40, 41], "3319": 32, "1105379": [32, 34], "3320": 32, "1097": [32, 38], "19314": 32, "3322": [32, 40], "143851": 32, "3323": [32, 40], "20710": 32, "float64": [32, 34, 37], "73": [32, 36, 37, 40], "58": [32, 34, 36, 37, 38, 40], "arrests_w_pop": 32, "_merg": [32, 34], "left_onli": 32, "2030": 32, "right_onli": 32, "76": [32, 37, 38, 41], "fill": [32, 46], "interpol": 32, "wave": [32, 59], "1995": [32, 40], "78": [32, 37], "ascend": 32, "interpolated_total_popul": 32, "1981": [32, 34], "4699": [32, 34, 37], "3926": [32, 34], "1122759": [32, 34], "1982": [32, 34, 38, 40], "4389": [32, 34], "4436": [32, 34], "1140139": [32, 34], "174": [32, 37, 38, 40], "4500": [32, 34], "5086": [32, 34, 40], "1157519": [32, 34], "1984": [32, 34, 38], "3714": [32, 34], "5878": [32, 34], "1174900": [32, 34], "white_arrests_w_pop": 32, "miniconda3": [32, 42, 48], "lib": [32, 42, 48], "python3": [32, 42, 48], "packag": [32, 42, 48], "ipykernel_launch": 32, "py": [32, 34, 42, 48], "userwarn": 32, "reindex": 32, "290": [32, 34, 37, 38], "1985": [32, 38], "4199": 32, "7139": [32, 37], "1192280": 32, "348": [32, 34, 36, 37, 38], "1986": [32, 38], "4915": [32, 38], "7720": 32, "1209660": 32, "406": [32, 37, 40, 42], "1987": 32, "5137": 32, "10333": 32, "1227041": 32, "464": [32, 37, 40], "1988": [32, 38], "5087": 32, "12574": 32, "1244421": 32, "522": [32, 37, 40, 41], "1989": 32, "5551": 32, "14056": 32, "1261801": 32, "580": [32, 37, 40], "6156": 32, "12378": 32, "1279182": 32, "638": [32, 34, 37, 38, 40], "1991": 32, "5942": 32, "11408": 32, "1295637": 32, "696": [32, 34, 37, 40], "1992": 32, "6162": [32, 40], "10712": 32, "1312093": 32, "754": [32, 34, 37], "1993": 32, "9656": 32, "1328549": 32, "812": [32, 40, 48], "1994": 32, "5911": 32, "9764": 32, "1345005": 32, "3844": 32, "3265": 32, "1361461": 32, "928": [32, 37, 40, 42], "1996": 32, "5610": 32, "7322": [32, 40], "1377917": 32, "986": 32, "1997": 32, "5976": 32, "7646": 32, "1394373": 32, "1044": 32, "1998": 32, "5636": 32, "7852": 32, "1410829": 32, "1102": 32, "1999": [32, 40], "5168": 32, "8985": 32, "1427285": 32, "1160": 32, "4356": 32, "7414": 32, "1443741": 32, "1218": 32, "4478": 32, "6486": 32, "1450394": 32, "1276": [32, 36], "2002": [32, 34], "4424": 32, "6414": 32, "1457047": 32, "1334": [32, 36, 40], "2003": [32, 34], "4749": 32, "6708": 32, "1463700": 32, "1392": 32, "4301": 32, "6886": [32, 40], "1470353": 32, "1450": 32, "4090": 32, "6328": 32, "1477006": 32, "1508": [32, 40], "4042": 32, "5879": 32, "1483659": 32, "1566": 32, "4443": 32, "6071": 32, "1490312": 32, "1624": 32, "4336": 32, "5893": 32, "1496965": 32, "1682": [32, 37], "4318": 32, "5749": 32, "1503618": 32, "1740": 32, "3823": 32, "5357": [32, 40], "1510271": 32, "1798": 32, "2011": [32, 59], "3504": [32, 41], "3957": 32, "1856": 32, "3686": 32, "1914": 32, "3183": 32, "4337": 32, "1972": [32, 40], "2014": 32, "3491": 32, "4805": 32, "2015": [32, 37], "3452": [32, 37], "1946": 32, "2088": [32, 36], "3513": 32, "1762": [32, 40], "2146": 32, "3965": 32, "1279": 32, "2204": 32, "4132": 32, "1062": 32, "lo": [32, 34], "angel": [32, 34], "37964": 32, "29319": 32, "7477503": 32, "37610": 32, "27427": 32, "7616069": 32, "134": [32, 37, 38, 40], "36648": 32, "28807": 32, "7754635": 32, "192": [32, 34, 37, 40], "33858": 32, "36565": 32, "7893201": 32, "35303": 32, "44331": 32, "8031767": 32, "308": [32, 37, 40], "35659": 32, "51015": 32, "8170333": 32, "366": [32, 40], "45254": 32, "61035": 32, "8308899": 32, "424": [32, 37, 38, 40], "46020": 32, "61713": 32, "8447465": 32, "482": [32, 37, 40, 42], "51021": 32, "69363": 32, "8586031": 32, "540": [32, 40, 42], "59012": 32, "69667": 32, "8724597": 32, "598": [32, 38, 40], "64790": 32, "55218": 32, "8863164": 32, "656": [32, 37, 40], "58701": 32, "42583": 32, "8928781": 32, "714": [32, 48], "59215": 32, "43212": 32, "8994398": 32, "772": [32, 37, 48], "55079": 32, "43406": 32, "9060016": 32, "830": 32, "53467": 32, "48161": 32, "9125633": 32, "888": [32, 38, 40], "53218": 32, "45535": 32, "9191251": 32, "946": [32, 37], "51277": 32, "44102": 32, "9256868": 32, "1004": [32, 40], "50573": 32, "46951": 32, "9322485": 32, "46886": 32, "43947": 32, "9388103": 32, "1120": [32, 41], "43655": 32, "38253": 32, "9453720": 32, "40440": 32, "33499": 32, "9519338": 32, "1236": [32, 37], "42095": 32, "33570": 32, "9549264": 32, "1294": 32, "39768": 32, "36060": 32, "9579191": 32, "1352": [32, 36], "39633": 32, "42002": 32, "9609118": 32, "1410": 32, "37397": 32, "46060": 32, "9639044": 32, "1468": 32, "36322": 32, "48411": 32, "9668971": 32, "1526": 32, "35384": 32, "47549": 32, "9698898": 32, "1584": 32, "36250": 32, "45112": 32, "9728824": 32, "1642": 32, "36452": 32, "40928": 32, "9758751": 32, "1700": 32, "35319": 32, "32193": 32, "9788678": 32, "1758": 32, "33147": 32, "33920": 32, "9818605": 32, "1816": 32, "30818": 32, "33244": 32, "1874": 32, "29601": 32, "33775": 32, "1932": 32, "27159": 32, "35972": 32, "28251": 32, "36372": 32, "2048": 32, "28194": 32, "9753": 32, "2106": [32, 40], "27804": 32, "8970": 32, "2164": 32, "28746": 32, "7302": [32, 37], "2222": 32, "28865": 32, "6412": 32, "86": [32, 36, 37, 41], "renam": [32, 54], "to_csv": 32, "githubusercont": [33, 34, 38, 48], "61": [33, 37, 40], "411101": 33, "331110": 33, "92": [33, 37, 41], "779361": 33, "342067": 33, "222701": 33, "481508": 33, "588374": 33, "815540": 33, "472719": 33, "337757": 33, "2f": [33, 36, 40, 41], "77": [33, 36, 37, 41, 48], "std": [33, 34, 36, 37, 38, 40, 41, 42, 48], "expend": 33, "94": [33, 37], "corr": 33, "iloc": 33, "alt": [33, 34, 36, 37], "enabl": [33, 34, 37], "mimetyp": 33, "grumpi": 33, "displai": 33, "hist": 33, "dict": 33, "chart": [33, 34, 36, 37], "mark_bar": 33, "q": [33, 34], "vegalit": 33, "frontend": 33, "viz": 33, "io": 33, "user_guid": 33, "troubleshoot": 33, "html": 33, "17": [33, 34, 36, 37, 38, 40, 41, 42, 48], "mark_point": [33, 34, 36], "legal": [34, 40, 53], "marijuana": 34, "drug_rat": 34, "100_000": 34, "isin": 34, "394": 34, "457331": 34, "551801": 34, "289": [34, 37, 40], "027292": 34, "271": [34, 37, 38, 40], "709690": 34, "312": [34, 36, 37, 40], "159811": 34, "209": [34, 37, 40], "450372": 34, "276": [34, 37, 38, 40], "018946": 34, "358": [34, 37, 41], "305449": 34, "228": [34, 36, 37], "874748": 34, "473": [34, 37, 40, 42], "876866": 34, "411": [34, 37, 38, 40], "831533": 34, "499": [34, 37, 40], "572904": 34, "460": [34, 40], "694485": 34, "380": [34, 36, 37], "164074": 34, "432": [34, 37, 40], "659812": 34, "king": [34, 50, 51], "035930": 34, "lake": 34, "369": [34, 37], "871380": 34, "lassen": 34, "147": [34, 37, 41], "165387": 34, "403": [34, 37, 38, 40], "990696": 34, "261": [34, 36, 37, 40], "117203": 34, "marin": 34, "190": [34, 36, 37, 38], "030996": 34, "mariposa": 34, "969880": 34, "mendocino": 34, "599": [34, 36, 40], "799767": 34, "merc": 34, "441": [34, 37], "290221": 34, "modoc": 34, "204960": 34, "mono": 34, "272": [34, 37, 40], "602759": 34, "monterei": 34, "248": [34, 37, 40], "094595": 34, "napa": 34, "247": [34, 36, 37, 38, 40], "823080": 34, "nevada": 34, "235": [34, 36, 37, 40], "403323": 34, "orang": 34, "277": [34, 37, 40], "267304": 34, "placer": 34, "583530": 34, "pluma": 34, "492": [34, 40], "429787": 34, "riversid": 34, "291": [34, 36, 37, 38, 40], "458787": 34, "375": [34, 36, 37, 38, 56], "134854": 34, "benito": 34, "210": [34, 36, 37, 38, 40], "849718": 34, "bernardino": 34, "465": [34, 37, 38, 40], "333881": 34, "diego": 34, "249": [34, 37, 38, 40], "177208": 34, "francisco": 34, "873": [34, 37], "961738": 34, "824184": 34, "lui": 34, "obispo": 34, "583757": 34, "mateo": 34, "214": [34, 37, 40], "668466": 34, "santa": 34, "barbara": 34, "225": [34, 36, 37, 40, 41], "267192": 34, "clara": 34, "348166": 34, "cruz": 34, "577393": 34, "shasta": 34, "470436": 34, "sierra": 34, "221": [34, 37, 40], "364059": 34, "siskiy": 34, "352": [34, 36, 37, 40], "795109": 34, "388": [34, 37, 38, 40, 41, 48], "929454": 34, "sonoma": 34, "331": [34, 37, 40], "905273": 34, "stanislau": 34, "457": [34, 37, 40], "198700": 34, "sutter": 34, "695398": 34, "585": [34, 36, 38, 40], "300555": 34, "triniti": 34, "427": [34, 37, 40], "859162": 34, "tular": 34, "447": [34, 37, 40], "774248": 34, "tuolumn": 34, "405": 34, "897681": 34, "264": [34, 37, 38, 40], "878553": 34, "yolo": 34, "333": [34, 37, 40], "788690": 34, "yuba": 34, "351": [34, 37], "561100": 34, "301": [34, 37, 40], "8092992253924": 34, "astyp": [34, 36, 40, 42, 48], "int": [34, 36, 40, 42, 48], "left_on": 34, "right_index": 34, "outer": 34, "875683": 34, "407": [34, 40, 42], "462056": 34, "349": 34, "674236": 34, "418": [34, 36, 37], "522474": 34, "389": [34, 37, 40, 48], "075162": 34, "384": [34, 36], "952860": 34, "439": [34, 40], "387694": 34, "762215": 34, "297813": 34, "316": [34, 37, 38, 42], "111956": 34, "post_2010": 34, "arrests_sub": 34, "y_t1_post": 34, "y_t1_pre": 34, "y_t0_post": 34, "y_t0_pre": 34, "799650070477355": 34, "418061484004568": 34, "api": [34, 36, 37, 38, 40, 41, 42, 48], "get_robustcov_result": [34, 41, 42, 48], "cov_typ": [34, 38, 41, 42], "dep": [34, 36, 37, 38, 40, 41, 42, 48], "adj": [34, 36, 37, 38, 40, 41, 42, 48], "sun": [34, 42, 48], "2023": [34, 36, 37, 38, 40, 41, 42], "prob": [34, 36, 37, 38, 40, 41, 42, 48], "45e": [34, 37, 40], "2094": 34, "aic": [34, 36, 37, 38, 40, 41, 42, 48], "4196": 34, "344": [34, 37, 40, 48], "bic": [34, 36, 37, 38, 40, 41, 42, 48], "4212": 34, "coef": [34, 36, 37, 38, 40, 41, 42, 48], "err": [34, 36, 37, 38, 40, 41, 42, 48], "025": [34, 36, 37, 38, 40, 41, 42, 48], "975": [34, 36, 37, 38, 40, 41, 42, 48], "319": [34, 36, 37], "7820": 34, "131": [34, 37, 40], "284": [34, 36, 37, 40], "463": 34, "355": [34, 37, 40], "101": [11, 34, 37, 38, 40, 42], "3816": 34, "892": [34, 37, 40], "959": 34, "055": [34, 36, 37, 38], "189": [34, 37, 38, 40], "426": [34, 40], "106": [34, 37, 38, 40, 41], "8289": 34, "385": 34, "568": [34, 40], "001": [34, 37, 38, 40, 41, 48], "153": [34, 36, 37, 40], "657": [34, 37, 40, 41], "4181": 34, "869": [34, 37], "393": [34, 40, 41], "203": [34, 37, 40], "367": [34, 37], "omnibu": [34, 36, 37, 38, 40, 41, 42, 48], "53": [34, 36, 37, 38, 40, 42, 48], "945": [34, 37, 40], "durbin": [34, 36, 37, 38, 40, 41, 42, 48], "watson": [34, 36, 37, 38, 40, 41, 42, 48], "741": [34, 40], "jarqu": [34, 36, 37, 38, 40, 41, 42, 48], "bera": [34, 36, 37, 38, 40, 41, 42, 48], "jb": [34, 36, 37, 38, 40, 41, 42, 48], "81": [34, 36, 37, 41, 42, 48], "621": [34, 37, 40, 41], "skew": [34, 36, 37, 38, 40, 41, 42, 48], "89e": [34, 36, 37, 40], "kurtosi": [34, 36, 37, 38, 40, 41, 42, 48], "cond": [34, 36, 37, 38, 40, 41, 42, 48], "data_transform": [34, 37], "data_serv": [34, 37], "arrests_long": 34, "grouped_mean": 34, "loess": [34, 37], "lightblu": 34, "transform_regress": 34, "mark_lin": [34, 37], "append": 34, "layer": 34, "TO": 34, "eq": 34, "hte": 34, "_thought_": 34, "cluster_ent": [34, 38, 42], "mislead": 34, "fault": 34, "linearmodel": [34, 38, 42], "arrests_for_panelol": 34, "set_index": [34, 38, 42], "mod": [34, 38, 42], "entityeffect": [34, 38, 42], "timeeffect": [34, 38], "drop_absorb": 34, "cluster_tim": 34, "h_8_rwsn5hvg9mhp0txgc_s9v6191b": 34, "ipykernel_84356": 34, "3310218310": 34, "absorbingeffectwarn": 34, "absorb": [34, 38], "0013": 34, "0109": 34, "0155": 34, "0104": 34, "1858": 34, "cov": [34, 38, 42], "3829": 34, "5366": [34, 40], "ob": [34, 38, 42], "0000": [34, 36, 38, 42], "max": [34, 37, 38, 42], "1899": 34, "6633": 34, "stat": [11, 34, 37, 38, 41, 42, 48], "ci": [34, 38, 42], "upper": [34, 38, 42], "023": [34, 36, 40, 41], "4358": 34, "925": [34, 36, 37, 38, 40], "089": [34, 37, 40], "poolabl": [34, 38, 42], "282p": 34, "0000distribut": [34, 42], "0097": 34, "0168": 34, "0588": 34, "0150": 34, "4108": 34, "6568": 34, "0101": 34, "6634": 34, "1031": 34, "362": [34, 36, 37, 40], "6320": 34, "078": [34, 37, 40], "3540": 34, "619": [34, 36, 40], "3654453326": 34, "unconstrain": 35, "kupp": 35, "constrain": 35, "umbrella": 35, "dry": 35, "sunni": 35, "pont": 36, "le": 36, "4723": 36, "3200": 36, "199": [36, 37, 40, 41], "379": [36, 37], "38e": [36, 37], "09": [36, 37, 38, 48], "678": [36, 37, 40], "1360": 36, "nonrobust": [36, 37, 38, 40, 42], "5143": 36, "0893": 36, "807": [36, 37, 40], "442": [36, 37, 40], "4519": 36, "521": [36, 40], "5766": 36, "4202": 36, "2440": 36, "634": [36, 37, 40, 48], "243": [36, 37], "2937": 36, "904": [36, 40], "5466": 36, "584": [36, 40], "244": [36, 37, 40, 41], "04e": [36, 37, 40, 41], "126": [36, 37, 40], "specifi": [36, 37, 38, 40, 42, 47, 51, 52, 61], "202": [36, 37], "596": 36, "566": [36, 37, 38], "04": [36, 37, 38, 40, 41, 42, 48], "14e": [36, 37, 40], "662": [36, 37, 40], "1336": [36, 40], "1350": 36, "782": [36, 37, 40], "5353": 36, "1612": 36, "628": [36, 40], "485": [36, 37, 40], "4000": 36, "484": [36, 38, 40, 42], "2435": [36, 37], "414": [36, 38, 40], "3278": 36, "9827": 36, "671": [36, 37], "826": [36, 37, 40], "881": [36, 40], "1938": 36, "4619": [36, 40], "591": [36, 37, 38, 40], "1977": 36, "1796": 36, "711": [36, 38, 40], "781": [36, 37], "007": [36, 37, 40, 41, 48], "558": [36, 37], "3396": 36, "068": [36, 37, 41, 48], "9634": [36, 40], "702": [36, 40], "797": [36, 38, 40], "563": [36, 40], "364": [36, 40], "736": [36, 37, 38], "7997": 36, "309": [36, 37, 38, 40], "009": [36, 37, 40, 41, 48], "020": [36, 37, 38, 40, 48], "1353": [36, 37], "120": [36, 37, 38, 40], "182": [36, 37, 40, 42], "9667": 36, "819": [36, 37, 40], "541": [36, 37], "128": [36, 37], "646": [36, 37, 40, 41], "579": [36, 37, 40], "179": [36, 37, 38, 40], "409": [36, 37, 40], "01e": [36, 37, 40, 42], "36e": [36, 37, 40], "arestrong": [36, 40], "multicollinear": [36, 40], "bewar": 36, "inadvert": [36, 61], "air": [36, 40, 47, 54], "531": [36, 37, 40], "11e": [36, 37, 40], "616": [36, 40, 41], "1254": [36, 40], "1852": 36, "4340": 36, "2284": 36, "811": [36, 37, 40], "421": 36, "6423": 36, "964": [36, 37, 40, 41], "2719": 36, "096": [36, 37, 40, 41], "951": [36, 37, 41], "3308": 36, "1676": 36, "704": [36, 37, 41], "567": [36, 37, 40], "573": [36, 40], "2403": 36, "746": [36, 37], "4306": 36, "408": [36, 37], "9162": 36, "1539": 36, "879": [36, 37], "383": [36, 37, 40], "1728": 36, "4434": 36, "011": [36, 37, 40, 41], "994": [36, 37, 40], "5742": [36, 38], "1609": 36, "415": 36, "618": [36, 37, 41], "539": [36, 38, 40], "2225": 36, "857": [36, 40], "4215": 36, "006": [36, 37, 38, 40, 41, 48], "1325": 36, "1545": [36, 40], "1705": 36, "943": [36, 40], "777": [36, 37, 38, 40], "440": [36, 37, 40, 41], "429": [36, 37], "4738": [36, 40], "738": [36, 40], "3240": [36, 40], "0049": 36, "012": [36, 37, 40, 41, 48], "015": [36, 40, 41, 48], "1625": 36, "4854": [36, 40], "833": [36, 37, 40], "1644": 36, "4650": [36, 37], "769": [36, 37, 40], "136": [36, 37, 40], "037": [36, 37, 38, 40, 41, 48], "103": [36, 37], "848": [36, 37, 40], "3185": 36, "082": [36, 40, 41], "7544": 36, "883": 36, "988": [36, 40], "051": [36, 37, 40], "520": [36, 40], "755": [36, 37, 41], "1823": 36, "341": [36, 37, 38], "695": [36, 37, 40], "031": [36, 40, 41], "1438": 36, "912": [36, 40], "452": [36, 37], "0412": [36, 41], "472": [36, 37, 40, 42], "612": [36, 37, 40], "112": [36, 37, 38, 40, 41], "910": [36, 37, 38, 40], "992": [36, 40], "610": [36, 37], "430": [36, 37, 40], "697": [36, 40], "181": [36, 37, 38, 40], "82e": [36, 37, 40, 48], "478": [36, 37, 40], "54e": [36, 37, 40], "353": [36, 40, 42], "390": [36, 37, 42], "20e": 36, "675": [36, 37, 40], "1369": 36, "4925": 36, "378": [36, 37, 40], "835": [36, 40], "4169": 36, "438": [36, 37, 40], "5680": 36, "562": [36, 37, 40, 41, 48], "642": [36, 37, 40], "7895": 36, "650": [36, 37, 42], "326": [36, 37, 40, 48], "654": [36, 40], "1939": 36, "931": [36, 37, 40], "3977": 36, "7333": 36, "639": [36, 40, 48], "2570": 36, "953": [36, 37, 40], "5384": 36, "513": [36, 37, 41, 42], "8105": 36, "1595": [36, 40], "941": [36, 37], "1170": 36, "193": [36, 37], "5195": 36, "814": [36, 40], "269": [36, 37, 38, 40, 42], "940": [36, 37, 40], "79e": [36, 37, 40], "635": [36, 37, 40], "01": [36, 41, 48], "21e": [36, 37, 40], "660": [36, 37, 40, 41], "391": 36, "7038": 36, "1588": 36, "834": [36, 37, 40], "806": [36, 40, 41, 48], "3563": 36, "030": [36, 37, 40, 41], "2779": 36, "622": [36, 37, 40, 42], "2929": 36, "3402": 36, "1573": [36, 40], "413": [36, 40], "4285": 36, "267": [36, 37, 38, 40], "1354": 36, "9011": 36, "760": [36, 40], "079": [36, 37, 41, 48], "162": [36, 37, 40], "552": [36, 37, 40], "2872": [36, 40], "354": [36, 37], "2797": 36, "6787": 36, "1381": [36, 40], "501": [36, 37, 38, 40], "047": [36, 37, 40, 41, 48], "5555": 36, "167": [36, 37, 38, 40], "6417": 36, "330": [36, 37, 40], "048": [36, 37, 40, 41], "8717": 36, "302": [36, 37], "195": [36, 37, 40], "017": [36, 37, 40, 41, 48], "1340": 36, "056": [36, 37, 48], "133": [36, 37, 40], "688": [36, 37], "6296": 36, "972": [36, 37, 41], "115": [36, 37, 40, 48], "038": [36, 37, 38, 41], "710": [36, 37, 38], "549": 36, "874": [36, 37, 40, 41], "311": [36, 37, 38, 40], "63e": [36, 37, 40], "885": [36, 37], "39e": [36, 37, 40], "grandmoth": 36, "ago": 36, "mom": 36, "admittedli": [36, 37], "databas": [36, 54], "param": [36, 40], "4152": 36, "m2": 36, "564": [36, 37], "12e": [36, 37, 40], "661": [36, 40], "1338": [36, 40], "232e": 36, "4465": 36, "758": [36, 37], "3404": 36, "206": [36, 37, 41], "176e": 36, "2796": 36, "6184": [36, 37], "73e": [36, 37, 40], "259": [36, 37, 40, 42], "8139": [36, 40], "109": [36, 37, 38], "998": [36, 37, 38, 40, 41], "021": [36, 37, 40], "479": [36, 37, 40], "371": 36, "314": [36, 37, 40], "4806": 36, "360": [36, 37, 38, 48], "876": [36, 37], "005": [36, 37, 38, 41], "532": [36, 40], "764": [36, 40, 48], "96": [36, 37], "4327": 36, "856": [36, 37, 40], "008": [36, 37, 41], "723": [36, 40], "142": [36, 37, 40, 41], "5821": 36, "958": [36, 37, 40], "515": [36, 40], "1123": 36, "222": [36, 37, 40, 42], "154": [36, 37, 40], "058": [36, 37, 40, 41], "4670": 36, "839": [36, 40], "016": [36, 37, 38, 40, 41, 48], "813": [36, 37], "121": [36, 37, 40, 41], "595": [36, 37], "07e": [36, 37, 40], "204": [36, 37, 38, 40], "premium": 36, "milag": 36, "151404": 37, "wv": [37, 38], "123453": 37, "251": [37, 41], "tx": [37, 38], "187982": 37, "ma": [37, 38], "unincorpor": 37, "122356": 37, "tn": [37, 38], "nonprofit": 37, "210750": 37, "42900": 37, "ia": [37, 38], "063": 37, "sat": [37, 38, 40, 41], "56e": [37, 40], "63018": 37, "5515": 37, "260e": 37, "5513": 37, "261e": 37, "887e": 37, "669": 37, "95e": [37, 40], "416e": 37, "735": [37, 40], "820": [37, 40], "242": [37, 38, 40], "27e": [37, 40], "2214": 37, "974": [37, 40, 48], "10578": 37, "287": [37, 38], "608": 37, "iffi": 37, "frequenc": [37, 49, 59], "chi2": [37, 41], "_i_": 37, "freq": 37, "032": [37, 38, 40], "122": [37, 40, 42], "74e": [37, 40, 42], "7675": [37, 38], "11150": 37, "536e": 37, "11146": 37, "539e": 37, "4382": 37, "420": [37, 40, 42, 48], "449": [37, 40], "1206": 37, "507": [37, 40], "152": 37, "2398": 37, "014": [37, 41], "682": 37, "266": [37, 40], "213": [37, 40, 41], "0367": 37, "024": 37, "069": [37, 40], "46681": 37, "1670": 37, "377": [37, 40], "438205": 37, "317647": 37, "198413": 37, "474900": 37, "ctab": [37, 41], "4282": 37, "3340": 37, "324": [37, 40], "1212": 37, "523": [37, 40], "dof": [37, 41], "chi2_conting": [37, 41], "2993875943569016e": 37, "mo": [37, 38], "000000": 37, "700": [37, 40], "va": [37, 38], "fl": [37, 38], "md": [37, 38], "882353": 37, "332106": 37, "nj": [37, 38], "IN": 37, "171": [37, 40, 48], "mn": [37, 38], "390058": 37, "207616": 37, "262319": 37, "375000": 37, "500000": 37, "grad": [37, 41], "remot": 37, "quadrat": [37, 48, 51], "2760": [37, 40], "2551": 37, "2397": 37, "173": [37, 38, 40], "tini": 37, "576": [37, 38], "275": [37, 40, 41], "230": [37, 40], "223": [37, 38, 40], "263": [37, 38, 40], "285": [37, 40], "7809": 37, "740": [37, 38, 40], "706": [37, 38, 40], "615": [37, 40], "387": [37, 38, 40], "337": [37, 40, 48], "for_match": 37, "reset_index": 37, "31200": 37, "20020": 37, "22859": 37, "73860": 37, "5510": 37, "33800": 37, "5511": 37, "23920": 37, "5512": 37, "37440": 37, "5514": 37, "26000": 37, "1204727749": 37, "8949614": 37, "129": [37, 38, 40], "5378": 37, "1204742613": 37, "479154": 37, "1205072671": 37, "3262901": 37, "508": [37, 48], "5390": 37, "1205171280": 37, "4727237": 37, "509": 37, "5392": 37, "1210524158": 37, "7436352": 37, "511": [37, 40], "110": [37, 38, 40, 41], "5405": 37, "1210539313": 37, "933855": 37, "for_p": 37, "199313e": 37, "199422e": 37, "204728e": 37, "204743e": 37, "205073e": 37, "205171e": 37, "210524e": 37, "210539e": 37, "fast": 37, "5509": 37, "930000": 37, "108": [37, 40, 41], "070189": 37, "240000": 37, "134e": 37, "3736": 37, "7480": 37, "5306": 37, "7506": 37, "2119": 37, "469e": 37, "018": [37, 40, 41, 48], "92e": [37, 40], "036": [37, 40, 41], "378e": 37, "019": [37, 38, 40, 48], "86e": [37, 40], "18e": [37, 40], "83e": 37, "040": [37, 40, 41], "860": [37, 40], "227": [37, 40], "234": 37, "41e": [37, 40], "294": [37, 38, 40], "851": [37, 40], "057": [37, 41], "19e": [37, 40], "61753": 37, "235e": 37, "5308": 37, "909e": 37, "293": 37, "84e": [37, 40], "98e": 37, "374e": [37, 40], "763": [37, 40], "003": [37, 38, 40, 41, 48], "22e": [37, 40], "52e": [37, 40], "2934": 37, "035": [37, 40, 41], "33100": 37, "529": 37, "util": [37, 40], "post_process": 37, "matching_object": 37, "36": [37, 38], "238": [37, 40], "786": [37, 38, 40, 41, 48], "62e": 37, "132": [37, 41], "61189": 37, "230e": 37, "4984": 37, "252e": 37, "325": [37, 40], "761e": 37, "2429": [37, 40], "28e": [37, 40], "24e": [37, 40], "8344": 37, "9150": 37, "1067": 37, "818": 37, "6252": [37, 40], "476": [37, 38], "6753": 37, "9175": [37, 38], "1140": 37, "922": [37, 40], "8989": 37, "844": [37, 38], "4517": 37, "991": [37, 48], "3220": 37, "6308": 37, "1202": 37, "997": [37, 40], "677": [37, 38, 40], "5579": [37, 40], "862": [37, 40], "8584": 37, "0505": 37, "868": 37, "889": [37, 38, 40], "6882": 37, "03e": [37, 40], "251e": 37, "923": [37, 40], "43e": [37, 42], "266e": 37, "08e": [37, 38, 40], "46e": [37, 40], "9235": 37, "0616": 37, "1189": 37, "062": [37, 38], "767": [37, 38], "6903": [37, 40], "976": [37, 40, 41, 48], "16e": [37, 40], "347e": 37, "2975": 37, "342": [37, 40], "528": [37, 40], "7639": 37, "93e": 37, "114e": 37, "3231": [37, 40], "653": [37, 38, 40], "75e": [37, 40], "4799": 37, "550": [37, 38, 40, 41], "279e": 37, "3245": 37, "734": [37, 38, 40], "942": [37, 40], "6430": 37, "9142": 37, "9921": 37, "771": [37, 38], "4363": 37, "2990": 37, "164": [37, 38, 40], "23e": 37, "609": 37, "6378": 37, "3577": 37, "4178": 37, "527": [37, 40], "127": [37, 42], "1812": 37, "617": 37, "627e": [37, 40], "66e": [37, 40], "4103": 37, "023e": [37, 40], "3835": 37, "666": 37, "77e": [37, 40], "2706": [37, 40], "153e": 37, "3175": 37, "916": [37, 40], "78e": [37, 40], "5300": 37, "382e": 37, "5130": 37, "694": 37, "3762": 37, "501e": 37, "3427": 37, "926": [37, 38, 40], "17e": [37, 40], "8293": 37, "693": [37, 40], "418e": 37, "3040": [37, 40], "664": 37, "8218": 37, "6849": 37, "3954": 37, "3121": [37, 40], "194": [37, 40], "028": [37, 40, 48], "3e": [37, 40], "730": [37, 38, 40], "303": [37, 40, 42], "381e": 37, "3368": 37, "7209": 37, "369e": 37, "3918": 37, "748": [37, 40], "493": [37, 40, 41], "6007": 37, "9190": [37, 40], "4689": 37, "915": [37, 40], "218": [37, 38], "895": 37, "061e": 37, "3616": 37, "372": [37, 40, 41], "935": 37, "3523": [37, 40], "982": [37, 40], "601e": 37, "7731": 37, "556": 37, "070": [37, 40, 41], "909": [37, 40], "1788": 37, "7046": 37, "4787": 37, "690": 37, "374": 37, "709": [37, 40], "7597": 37, "684e": 37, "6555": 37, "900": [37, 41], "569": [37, 40], "010": [37, 38, 40, 41, 48], "97e": 37, "3989": 37, "067": 37, "4e": [37, 40], "4328": 37, "098": [37, 38, 40], "25e": [37, 40], "3776": 37, "3595": 37, "547": 37, "720": [37, 40, 48], "085": [37, 40, 41], "32e": [37, 40], "864": [37, 41], "826e": 37, "3694": 37, "55e": 37, "1e": [37, 40, 42], "242e": 37, "3244": 37, "837": [37, 40, 41], "828": 37, "88e": [37, 40], "6059": 37, "104e": 37, "3171": 37, "4824": 37, "506e": 37, "3496": 37, "8211": 37, "049": [37, 40, 41], "391e": 37, "3137": 37, "745": [37, 40], "7755": 37, "337e": [37, 40], "3345": [37, 40], "99e": [37, 40], "6814": 37, "297e": [37, 40], "5671": 37, "022": [37, 41], "1853": [37, 40], "4845": [37, 38], "0790": 37, "5203": 37, "125": [37, 40, 41], "5e": [37, 40], "5355": 37, "335": 37, "2248": 37, "8602": [37, 40], "5067": 37, "398": 37, "444": 37, "7685": 37, "469": [37, 40], "155e": 37, "4710": 37, "979": [37, 40], "2313": 37, "897": [37, 40], "7114": 37, "5598": 37, "3632": 37, "908": [37, 40], "050": [37, 38, 40, 48], "42e": 37, "538": [37, 40], "272e": 37, "3067": 37, "87e": [37, 40], "6711": 37, "3443": 37, "339": [37, 40], "6447": 37, "428": [37, 40], "9324": 37, "3384": 37, "603": 37, "6e": 37, "2688": 37, "932": [37, 40], "505e": 37, "3658": [37, 40], "7881": 37, "76e": [37, 40, 41, 48], "2856": 37, "308e": 37, "3310": 37, "96e": [37, 38, 40, 41], "6590": 37, "177e": [37, 40], "3252": [37, 40], "81e": [37, 38, 40, 48], "5391": 37, "516e": 37, "3677": 37, "7954": 37, "9224": 37, "6049": 37, "2690": 37, "524": [37, 40], "3949": 37, "365e": 37, "3460": 37, "095": [37, 38, 40], "944": 37, "6864": 37, "165e": 37, "3698": 37, "161": [37, 38, 40], "151": [37, 40], "002": [37, 40, 41, 48], "4404": 37, "402": [37, 38, 40], "2961": 37, "611": 37, "65e": [37, 40], "4889": 37, "7052": 37, "4496": 37, "3127": 37, "920": 37, "437": [37, 40], "9064": 37, "0230": 37, "3380": 37, "497": [37, 40], "681": [37, 40], "57e": [37, 40], "2436": 37, "761": [37, 40], "681e": 37, "3013": [37, 40], "09e": [37, 40], "8897": 37, "8731": 37, "3296": [37, 40], "699": [37, 40], "774": [37, 38, 40], "125e": 37, "7993": 37, "659": 37, "5580": 37, "69e": [37, 40], "13e": [37, 40, 42], "046": [37, 41], "459": [37, 40], "5880": [37, 40], "043": [37, 38, 40, 41], "703": [37, 40], "26e": [37, 38, 40], "887": [37, 38, 40], "270": [37, 40], "006e": 37, "8297": 37, "064": [37, 40, 41], "6201": 37, "7396": 37, "8710": 37, "376": [37, 40, 41], "707": [37, 38], "59e": [37, 40], "349e": 37, "8853": 37, "843": [37, 38, 40], "3865": 37, "6016": 37, "8132": [37, 40], "9691": 37, "581": [37, 40, 42], "535": [37, 38, 40], "7435": 37, "2407": 37, "7281": 37, "280": [37, 38, 40], "307": [37, 38, 40], "6839": 37, "8434": 37, "9441": 37, "604": [37, 40], "546": [37, 38, 40], "58e": [37, 40, 42], "5918": 37, "5652": 37, "5278": 37, "262": [37, 42], "4429": 37, "631e": 37, "9894": [37, 40], "838": [37, 40], "099": [37, 40, 48], "3086": 37, "114": [37, 40, 41], "499e": 37, "6719": 37, "026": [37, 38, 41], "1819": 37, "273": [37, 38], "8015": 37, "9218": [37, 40], "732": [37, 38, 40], "34e": [37, 40], "171e": [37, 40], "124e": 37, "8626": 37, "461": [37, 40], "8596": 37, "970": 37, "1156": [37, 40], "964e": 37, "9254": 37, "939": [37, 38, 40], "034": [37, 38, 40], "1498": 37, "286e": 37, "47e": [37, 40], "6046": 37, "434": [37, 38, 40], "53e": [37, 40], "265": [37, 38, 40], "8046": 37, "9594": [37, 40], "978": [37, 40], "85e": [37, 40], "91e": 37, "9042": 37, "15e": [37, 40], "3909": 37, "886": [37, 38, 40], "086e": [37, 40], "33e": [37, 40], "117": [37, 38, 40], "7e": [37, 40, 42], "5231": 37, "914e": 37, "8999": [37, 40], "435": [37, 40], "033": [37, 40], "68e": [37, 40], "1501": [37, 40], "949": [37, 38], "203e": 37, "630": [37, 40], "4151": 37, "5319": 37, "5648": 37, "737": [37, 40], "462": [37, 40], "6922": 37, "341e": 37, "7534": 37, "780": [37, 40], "075": 37, "1357": 37, "560": [37, 40], "719e": 37, "3924": 37, "381": 37, "49e": [37, 40], "9500": [37, 40], "746e": 37, "237": [37, 40, 48], "8624": 37, "6897": 37, "6770": 37, "72e": [37, 40], "146e": 37, "8462": 37, "899": [37, 40, 41], "8e": [37, 40], "741e": 37, "5589": 37, "6454": 37, "143e": 37, "5721": 37, "800": 37, "216": [37, 40], "658e": 37, "61e": [37, 40], "307e": [37, 40], "840": [37, 40, 41], "066": [37, 40, 41], "1514": 37, "84": [37, 48], "9730": 37, "48e": [37, 40], "3419": 37, "8384": 37, "9046": 37, "824e": [37, 40], "279": [37, 40], "201": [37, 40, 42], "9720": 37, "87": 37, "403e": 37, "205": [37, 40, 41], "027": [37, 40], "2667": 37, "326e": 37, "445": [37, 40], "574e": 37, "8531": [37, 40], "065": [37, 38, 40, 41], "985": [37, 38, 40], "816": [37, 40], "8870": 37, "8793": 37, "4968": [37, 40], "614": [37, 40], "785": [37, 40, 48], "074": [37, 38, 40, 41], "791": 37, "91": 37, "9791": 37, "880": [37, 38, 40, 41], "288": [37, 40], "8799": 37, "744": [37, 40], "687e": 37, "150": [37, 40], "6076": 37, "4593": 37, "8182": 37, "9052": 37, "5470": 37, "9761": 37, "637": [37, 40], "995": 37, "199e": 37, "5085": 37, "2e": [37, 40, 42], "219": [37, 38, 40], "7572": 37, "457e": 37, "9432": 37, "605": [37, 40], "31e": [37, 38, 40], "6080": [37, 40], "404": [37, 40], "98": [37, 40], "2197": 37, "3011": 37, "144": [37, 48], "9289": 37, "2093": 37, "3098": 37, "282": [37, 40], "3215": 37, "121e": 37, "7876": 37, "423": 37, "155": [37, 40], "4235": [37, 40], "9598": 37, "102": [37, 38, 40, 48, 54], "2597": 37, "8743": 37, "35e": [37, 40], "847": [37, 40], "014e": 37, "9066": 37, "7638": 37, "542": [37, 38, 40], "104": [37, 41], "252": [37, 40], "2631": [37, 40], "983": [37, 38, 40], "145e": 37, "107": [37, 38, 40, 41], "364e": 37, "71e": [37, 40], "962": 37, "4795": [37, 40], "8027": 37, "8034": 37, "597": [37, 38, 40], "551": [37, 38, 40, 41], "05e": [37, 40], "8716": 37, "5745": 37, "06e": [37, 40, 41], "822": 37, "946e": 37, "7351": 37, "516": [37, 40], "606": [37, 38, 40], "113": [37, 40, 41, 48], "293e": 37, "882": [37, 40], "908e": 37, "185": [37, 40], "236": [37, 38], "2798": 37, "0105": 37, "8785": 37, "2988": 37, "473e": 37, "577": [37, 40], "698e": 37, "124": [37, 38, 40, 41, 48], "4669": 37, "861": [37, 38, 40, 48], "704e": 37, "8908": 37, "767e": 37, "093": [37, 38, 40], "1747": 37, "291e": 37, "5091": 37, "347": [37, 40], "536": 37, "29e": [37, 40], "2931": 37, "559": [37, 40], "209e": 37, "795e": 37, "6490": 37, "6948": 37, "1460": 37, "9793": 37, "4111": [37, 40], "7736": 37, "668": [37, 41], "407e": 37, "338e": 37, "094": [37, 38], "274": [37, 40], "958e": 37, "5205": 37, "9462": [37, 40], "1890": [37, 48], "7889": 37, "2394": 37, "5638": [37, 40], "399": [37, 40], "3164": 37, "2175": [37, 38], "4884": 37, "51e": [37, 40], "467e": 37, "045": [37, 38, 40], "083e": [37, 40], "6228": 37, "141": [37, 40, 42], "4420": 37, "8608": 37, "5997": 37, "7337": 37, "137": [37, 40, 41, 48], "4087": 37, "2387": 37, "8100": 37, "7939": 37, "1789": 37, "8029": 37, "739": [37, 40], "989": 37, "323": [37, 40], "37e": [37, 40], "7802": [37, 40], "643": [37, 48], "139": [37, 38, 40], "692": [37, 38, 40], "3178": [37, 40], "02e": [37, 40, 41], "94e": [37, 40], "140": [37, 48], "651e": 37, "229": [37, 40], "8251": [37, 40], "4595": 37, "676": [37, 40], "1373": [37, 40], "8309": 37, "948": [37, 41], "143": [37, 48], "5545": 37, "8845": 37, "5862": 37, "5947": 37, "5771": 37, "921": [37, 40], "9556": 37, "0440": 37, "783": [37, 38], "146": [37, 40], "903e": 37, "1707": 37, "239": [37, 42], "012e": 37, "670": [37, 40], "097": [37, 41], "052e": 37, "157": [37, 38, 40], "867": 37, "249e": 37, "7332": 37, "914": 37, "1887": 37, "074e": 37, "174e": 37, "373": [37, 40], "379e": 37, "076": [37, 41], "2454": 37, "878": [37, 38, 40], "004": [37, 40, 41, 48], "3961": 37, "2072": [37, 40], "67e": [37, 40, 48], "160": [37, 41, 42], "811e": 37, "8356": 37, "451": 37, "1729": 37, "6091": 37, "1910": 37, "8574": 37, "477": [37, 40], "9087": 37, "0830": 37, "586": 37, "133e": 37, "054": [37, 40], "292": [37, 40], "283e": 37, "6795": 37, "165": [37, 38, 41, 48], "166": [37, 40], "551e": 37, "3002": 37, "990": [37, 38, 40], "947e": 37, "168": [37, 40, 48], "007e": 37, "7103": 37, "417": [37, 40], "156": [37, 40, 48], "3857": 37, "169": [37, 38], "9600": 37, "934": [37, 40], "170": [37, 40, 41, 42], "615e": 37, "7177": 37, "9576": 37, "172": 37, "626e": 37, "629e": 37, "314e": 37, "458": [37, 40, 41], "7983": 37, "176": 37, "8331": 37, "5679": 37, "7787": 37, "9729": 37, "6943": 37, "898": [37, 40], "5825": 37, "8426": 37, "9047": 37, "9527": 37, "061": [37, 48], "5061": 37, "4958": 37, "9108": 37, "578": [37, 40], "9156": 37, "0300": 37, "7253": 37, "999": [37, 40], "5065": 37, "6042": 37, "8525": 37, "602": 37, "8180": 37, "956": [37, 40], "1688": 37, "4654": 37, "4051": 37, "6253": 37, "894": 37, "9630": 37, "825": [37, 40], "184": [37, 38, 40], "566e": 37, "3739": 37, "789": [37, 40, 48], "8333": 37, "281": [37, 38], "158e": 37, "186": 37, "8134": [37, 40], "088": [37, 38], "187": [37, 38], "6652": 37, "8781": 37, "3548": [37, 40], "268": [37, 38, 40], "053": [37, 40, 41], "3878": [37, 40], "254": [37, 38, 40], "122e": 37, "5407": 37, "661e": 37, "803": [37, 40], "071": [37, 40], "2321": 37, "191": [37, 38, 40], "3036": 37, "3270": 37, "6940": 37, "9690": 37, "9281": 37, "647": 37, "455": [37, 42], "9444": 37, "8116": 37, "902": [37, 48], "2498": 37, "3312": [37, 40], "853": 37, "9e": [37, 40], "9595": 37, "1493": 37, "798": [37, 40], "2753": 37, "6229": 37, "6585": 37, "5757": 37, "7328": 37, "9890": 37, "7484": [37, 40], "328": [37, 40], "7343": 37, "801": [37, 40], "136e": 37, "8594": 37, "013": [37, 38, 40, 41, 48], "4510": 37, "7508": 37, "9207": 37, "3016": 37, "489": 37, "1594": 37, "737e": 37, "2251": 37, "1237": 37, "8204": 37, "6666": 37, "464e": 37, "9850": 37, "486": [37, 40], "4674": 37, "706e": 37, "6442": 37, "648": 37, "4427": 37, "733e": 37, "7640": 37, "6087": 37, "6907": [37, 40], "651": [37, 40, 48], "578e": 37, "973": [37, 40, 48], "1093": 37, "8388": 37, "7382": 37, "081": [37, 40], "351e": 37, "858": 37, "1299": 37, "215": [37, 40], "488": [37, 40], "5271": 37, "613": [37, 40], "8944": [37, 40], "5337": 37, "794": [37, 40], "090": [37, 40, 41], "1411": 37, "217": [37, 38], "9643": 37, "642e": 37, "987": [37, 40], "9429": 37, "8190": 37, "059": [37, 40, 41, 42], "220": [37, 42], "038e": 37, "7271": 37, "3058": 37, "526": [37, 38, 40], "622e": 37, "588": 37, "3804": 37, "878e": 37, "5199": 37, "8586": 37, "224": [37, 38, 40], "9192": 37, "1466": 37, "636": [37, 38, 41], "525": [37, 40], "2641": 37, "2389": 37, "226": [37, 40, 41], "8830": [37, 40], "2891": 37, "6639": 37, "4186": [37, 40], "397": [37, 40], "233e": [37, 40], "1135": [37, 40], "428e": 37, "454": [37, 40], "6264": 37, "041": [37, 38, 40, 41, 42], "517": [37, 38, 40], "8271": 37, "2748": 37, "7245": 37, "759": [37, 48], "5933": 37, "299e": 37, "8519": 37, "6289": 37, "960": [37, 38, 40], "233": [37, 40], "285e": 37, "8491": 37, "3796": 37, "735e": 37, "766e": 37, "217e": 37, "8349": 37, "4200": 37, "141e": [37, 40], "4505": [37, 40], "533": [37, 40], "2581": 37, "613e": 37, "258": [37, 38, 40, 42], "2127": 37, "712e": 37, "240": [37, 38, 48], "6376": 37, "9154": 37, "44e": [37, 40], "241": [37, 40], "3359": [37, 40], "7031": 37, "6189": 37, "543": [37, 40], "587": [37, 40], "8774": 37, "753": [37, 40, 48], "8724": 37, "775": [37, 38, 40], "788e": 37, "286": [37, 38], "9383": 37, "071e": 37, "2703": 37, "321": [37, 38, 40], "245": [37, 38, 40, 41], "8153": 37, "4624": 37, "5281": 37, "9970": 37, "4036": 37, "2630": 37, "9765": 37, "4698": 37, "5639": 37, "083": [37, 40], "1290": 37, "0204": 37, "8692": 37, "476e": 37, "616e": [37, 40], "4398": 37, "9569": 37, "726": [37, 40], "084": [37, 40, 48], "2247": 37, "701": [37, 46, 48], "253": [37, 38], "801e": 37, "5819": 37, "5313": 37, "470": [37, 40], "9352": 37, "6145": 37, "712": [37, 40], "64e": [37, 40], "474e": 37, "5626": 37, "575": [37, 40], "6945": 37, "662e": 37, "260": 37, "075e": 37, "7093": 37, "214e": 37, "9508": 37, "6499": 37, "474": [37, 40], "094e": 37, "872": 37, "2479": 37, "7477": 37, "6210": 37, "2882": 37, "138e": 37, "097e": 37, "8629": 37, "836e": 37, "7924": 37, "317": 37, "2824": 37, "361e": 37, "204e": 37, "485e": 37, "8880": 37, "672": [37, 40], "2561": 37, "149e": 37, "9891": 37, "3781": 37, "4694": 37, "687": [37, 38], "5173": 37, "7930": 37, "966e": 37, "402e": 37, "278": [37, 40], "2322": 37, "2439": 37, "903": [37, 40], "119e": 37, "345e": 37, "6306": 37, "1085": 37, "930": 37, "8403": 37, "4354": 37, "778": 37, "3681": 37, "0429": 37, "283": [37, 38], "5339": 37, "8789": 37, "9936": 37, "537": 37, "452e": 37, "8179": 37, "1519": [37, 40], "7684": 37, "0660": 37, "775e": 37, "5134": 37, "756": 37, "456": [37, 40], "7680": [37, 40], "2038": 37, "8041": 37, "0146": 37, "468": [37, 40], "581e": 37, "7099": 37, "1889": 37, "7663": 37, "1814": 37, "571": [37, 40], "567e": 37, "7319": 37, "1323": [37, 42], "652": [37, 38, 40, 41], "2555": 37, "433": [37, 38], "665": [37, 40, 41, 48], "9840": 37, "996": [37, 40, 41], "296": [37, 42], "1096": 37, "1249": 37, "4502": 37, "2586": 37, "298": [37, 40], "8732": 37, "4700": 37, "583": [37, 40, 42], "986e": [37, 40], "907": [37, 40], "329e": 37, "4581": 37, "0131": [37, 41], "7593": [37, 38], "484e": [37, 40], "7473": 37, "222e": 37, "8772": 37, "5022": 37, "2514": 37, "2672": 37, "7217": 37, "728": [37, 40], "336e": 37, "116e": [37, 40], "9796": 37, "8045": 37, "973e": 37, "179e": [37, 42], "5120": 37, "504": [37, 40], "310": [37, 38], "268e": 37, "624e": 37, "3524": 37, "9191": [37, 40], "1488": [37, 40], "9873": 37, "313": [37, 38, 40], "765e": 37, "412": 37, "680": 37, "793e": 37, "7170": 37, "3875": [37, 40], "4379": 37, "933": [37, 40], "5807": [37, 40], "9640": 37, "6758": 37, "859": [37, 40], "7442": 37, "794e": 37, "9335": 37, "924": [37, 38, 40], "318": [37, 41], "4980": 37, "2142": 37, "8582": 37, "1051": 37, "3237": 37, "320": [37, 38, 40], "431e": 37, "6968": 37, "416": [37, 38, 40], "3176": 37, "6523": 37, "846": [37, 40], "950": [37, 40], "853e": 37, "5726": 37, "625": 37, "2501": 37, "1256": 37, "7866": 37, "751": 37, "951e": [37, 40], "325e": 37, "742": 37, "8657": 37, "2875": 37, "9851": 37, "7463": 37, "2414": 37, "039": [37, 40, 41, 48], "981": [37, 40], "22245": 37, "534": 37, "model2": 37, "result2": 37, "matched_data2": 37, "19512": 37, "904e": 37, "386": [37, 40, 42], "128e": 37, "8904": 37, "805": [37, 40, 42], "855": [37, 38, 40, 41], "6653": 37, "matplotlib": 38, "pyplot": 38, "plt": 38, "unnam": 38, "spirit": 38, "unemp": 38, "emppop": 38, "baptist": 38, "mormon": 38, "nfatal2124": 38, "afat": 38, "pop1517": 38, "pop1820": 38, "pop2124": 38, "milestot": 38, "unempu": 38, "emppopu": 38, "gsp": 38, "10544": 38, "152344": 38, "692039": 38, "539379": 38, "355700": 38, "32829": 38, "437988": 38, "3942002": 38, "208999": 38, "593750": 38, "221553": 38, "43750": 38, "290000": 38, "06250": 38, "28516": 38, "799999": 38, "022125": 38, "10732": 38, "797852": 38, "147030": 38, "788991": 38, "333599": 38, "34341": 38, "834015": 38, "3960008": 38, "202000": 38, "078125": 38, "219125": 38, "46875": 38, "15625": 38, "31032": 38, "900002": 38, "046558": 38, "11108": 38, "791016": 38, "168087": 38, "714286": 38, "311501": 38, "35924": 38, "872009": 38, "3988991": 38, "196999": 38, "968750": 38, "216724": 38, "09375": 38, "288000": 38, "32961": 38, "500004": 38, "062798": 38, "11332": 38, "626953": 38, "271137": 38, "652542": 38, "289499": 38, "37579": 38, "742004": 38, "4021007": 38, "194999": 38, "734375": 38, "214349": 38, "03125": 38, "284000": 38, "31250": 38, "35091": 38, "100002": 38, "027490": 38, "11661": 38, "506836": 38, "514496": 38, "609907": 38, "267401": 38, "39311": 38, "716003": 38, "4049993": 38, "203999": 38, "890625": 38, "212000": 38, "00000": 38, "263000": 38, "28125": 38, "36259": 38, "700001": 38, "032143": 38, "nuniqu": 38, "10000": 38, "polynomi": 38, "polyfit": 38, "scatterplot": 38, "line2d": 38, "0x16c6a9b40": 38, "quad": 38, "091": 38, "334": [38, 40], "553": [38, 40], "8533": 38, "044": [38, 40], "768": [38, 40], "3646": 38, "865": 38, "487": [38, 40], "905": 38, "4776": 38, "6550": 38, "086": [38, 40], "az": 38, "5677": 38, "5095": 38, "4843": [38, 40], "919": [38, 40, 41], "ct": 38, "8623": 38, "de": 38, "3076": 38, "448": [38, 40, 41], "729": 38, "2681": 38, "5246": 38, "852": [38, 40], "5439": 38, "092": [38, 41], "id": [38, 46], "6690": 38, "il": 38, "9616": 38, "4615": 38, "k": 38, "2232": 38, "984": 38, "ky": 38, "la": [38, 54], "8471": 38, "490": [38, 40, 41], "641": [38, 40], "7064": 38, "1079": 38, "8972": 38, "419": [38, 40, 42], "2963": [38, 40], "821": [38, 40], "0291": 38, "mt": 38, "3604": 38, "365": [38, 40], "2905": 38, "nd": 38, "6234": 38, "396": [38, 40], "ne": 38, "5222": [38, 40], "nh": 38, "2545": 38, "667": [38, 40], "842": [38, 40], "1057": 38, "nm": 38, "4264": [38, 40], "927": [38, 40, 48], "nv": 38, "6008": 38, "1867": 38, "6744": 38, "175": [38, 40], "5451": 38, "1680": 38, "pa": 38, "ri": 38, "2651": 38, "sc": 38, "5572": 38, "sd": 38, "0037": 38, "788": [38, 40], "8757": 38, "ut": 38, "1640": 38, "2902": 38, "vt": 38, "9660": 38, "382": 38, "6595": 38, "854": [38, 40, 41], "wi": 38, "338": 38, "8968": 38, "wy": 38, "2285": 38, "466": 38, "6559": 38, "491": [38, 40], "863": [38, 40], "beertax_dm": 38, "fat_rate_dm": 38, "000196": 38, "168e": [38, 42], "0407": 38, "3805": 38, "3775": 38, "unadjust": 38, "0006": [38, 48], "1878": 38, "0256": 38, "2861": 38, "2939": 38, "8925": 38, "2563": 38, "8295": 38, "060": [38, 40], "0162": 38, "2245": 38, "4582": 38, "5398": 38, "3521": 38, "5330": 38, "1264": [38, 40], "2329": 38, "1533": 38, "0951": 38, "0160": 38, "9261": 38, "1267": 38, "0635": 38, "530": [38, 40], "covid": 39, "pandem": 39, "atc": 39, "q4": 39, "q5": 39, "nobodi": 39, "calori": 39, "intak": 39, "beverag": [39, 40], "hhid": [40, 42], "intmonth": [40, 42], "hurespli": [40, 42], "hrhtype": [40, 42], "minsamp": [40, 42], "hrlonglk": [40, 42], "hrsampl": [40, 42], "hrhhid2": [40, 42], "serial": [40, 42], "hhnum": [40, 42], "ym_file": 40, "ym": 40, "ch02": 40, "ch35": [40, 42], "ch613": [40, 42], "ch1417": [40, 42], "ch05": [40, 42], "ihigrdc": [40, 42], "docc00": [40, 42], "dind02": [40, 42], "000110339935453": [40, 42], "januari": [40, 42], "unmarri": [40, 42], "fam": [40, 42], "0701": [40, 42], "07011": [40, 42], "administr": [40, 42, 47], "000110359424339": [40, 42], "0711": [40, 42], "07111": [40, 42], "practition": [40, 42], "000110651278174": [40, 42], "0601": [40, 42], "06011": [40, 42], "wareh": [40, 42], "007680515071194": [40, 42], "0611": [40, 42], "06112": [40, 42], "implied_hourly_wag": 40, "mean_femal": 40, "mean_mal": 40, "1204": 40, "On": [40, 47, 49], "827": 40, "514": [40, 41], "age_squar": 40, "2625": 40, "4441e": 40, "122603": [40, 42], "888e": 40, "122599": [40, 42], "833e": 40, "062e": 40, "6349": 40, "443": 40, "2740": 40, "18297": 40, "733": [40, 48], "27620": 40, "OF": 40, "THE": 40, "has_highschool": 40, "has_ba": 40, "m_educ": 40, "8573": 40, "4296e": 40, "859e": 40, "122597": 40, "579e": 40, "373e": 40, "756e": 40, "937": 40, "496": 40, "9737": 40, "3958": 40, "359": 40, "555": 40, "14257": 40, "849": 40, "20565": 40, "673": [40, 41], "m_industri": 40, "4253e": 40, "851e": 40, "122339": 40, "854e": 40, "1138": 40, "8766": 40, "1701": 40, "4193": 40, "2477": 40, "forestri": 40, "1131": 40, "1132": 40, "8666": 40, "3722": 40, "793": 40, "6372": 40, "8220": 40, "1133": 40, "5656": 40, "2100": 40, "2999": 40, "fish": 40, "hunt": 40, "trap": 40, "3510": 40, "2980": 40, "4528": 40, "918": 40, "agricultur": 40, "6207": 40, "2024": 40, "796": 40, "750": [40, 41], "oil": 40, "392e": 40, "3072": 40, "coal": 40, "2121": 40, "528e": 40, "2550": 40, "metal": 40, "ore": 40, "2122": 40, "384e": 40, "4184": 40, "087": 40, "nonmetal": 40, "miner": 40, "quarri": 40, "2123": 40, "538e": 40, "2391": [40, 48], "495": 40, "431": 40, "773e": 40, "1513": 40, "electr": 40, "transmiss": 40, "pt": 40, "2211": 40, "162e": 40, "1437": 40, "s2212": 40, "588e": 40, "2480": 40, "2212": 40, "2685": 40, "steam": 40, "irrig": 40, "22131": 40, "22133": 40, "5944": 40, "3370": 40, "2059": 40, "9829": 40, "sewag": 40, "22132": 40, "8219": 40, "6616": 40, "3000": 40, "8332": 40, "056e": 40, "1126": 40, "8347": 40, "grain": [40, 41], "oilse": 40, "mill": 40, "3111": 40, "3112": 40, "2351": 40, "5394": 40, "confectioneri": 40, "3113": 40, "8933": 40, "3087": 40, "1430": 40, "fruit": 40, "veget": 40, "preserv": [40, 52], "specialti": 40, "3114": 40, "2238": 40, "7515": 40, "2458": 40, "725": 40, "911": 40, "2580": 40, "7057": 40, "dairi": 40, "6859": 40, "2203": 40, "2541": 40, "slaughter": 40, "3116": 40, "8284": 40, "2715": 40, "3343": 40, "311811": 40, "1774": 40, "8294": 40, "2453": 40, "6583": 40, "957": 40, "3034": 40, "3118": 40, "exc": 40, "4588": 40, "2350": 40, "395": [40, 41, 42], "2607": 40, "392": 40, "6608": 40, "seafood": 40, "miscellan": 40, "3117": 40, "3119": 40, "9160": 40, "3042": 40, "2207": 40, "4833": 40, "2859": 40, "0902": 40, "3663": 40, "4321": 40, "9058": 40, "1956": 40, "5223": 40, "tobacco": 40, "3122": 40, "7325": 40, "1456": 40, "fiber": 40, "yarn": 40, "thread": 40, "3131": 40, "5169": 40, "8050": 40, "8640": 40, "knit": 40, "3132": 40, "31324": 40, "3205": 40, "4471": 40, "3167": 40, "3003": 40, "9414": 40, "textil": 40, "coat": 40, "3133": 40, "8272": 40, "8244": [40, 42], "carpet": 40, "rug": 40, "31411": 40, "1664": 40, "3750": 40, "5371": 40, "9331": 40, "3206": 40, "0346": 40, "3374": 40, "3407": 40, "9819": 40, "3151": 40, "5465": 40, "8039": 40, "6312": 40, "866": [40, 41], "sew": 40, "apparel": 40, "3152": 40, "3536": 40, "0852": 40, "2485": 40, "422": 40, "8408": 40, "accessori": 40, "3159": 40, "1030": 40, "1607": 40, "9100": 40, "footwear": 40, "3162": 40, "478e": 40, "5328": 40, "4338": 40, "963": [40, 42], "leather": 40, "tan": 40, "3161": 40, "3169": 40, "2374": 40, "0553": 40, "6157": 40, "9694": 40, "pulp": 40, "paperboard": 40, "3221": 40, "264e": 40, "8510": 40, "32221": 40, "9113": 40, "32222": 40, "32223": 40, "32229": 40, "7723": 40, "9346": 40, "3395": 40, "1068": 40, "3648": 40, "2945": 40, "1712": 40, "7005": 40, "548": 40, "petroleum": 40, "32411": 40, "739e": 40, "2234": 40, "32419": 40, "8945": 40, "7389": 40, "5426": 40, "1689": 40, "resin": 40, "synthet": 40, "rubber": 40, "filament": 40, "7943": 40, "2560": 40, "2408": 40, "3222": 40, "chemic": 40, "3253": 40, "754e": 40, "8765": 40, "pharmaceut": [40, 49], "medicin": [40, 47, 49], "3254": 40, "377e": 40, "paint": 40, "adhes": 40, "b46": 40, "3255": 40, "586e": 40, "3584": 40, "8831": 40, "soap": 40, "compound": 40, "cosmet": 40, "3256": 40, "137e": 40, "2837": 40, "5809": 40, "3251": 40, "3259": 40, "plastic": 40, "3261": 40, "4034": 40, "0463": 40, "1804": 40, "7571": 40, "686": 40, "tire": 40, "32621": 40, "298e": 40, "3637": 40, "570": 40, "5854": 40, "32622": 40, "32629": 40, "3975": 40, "2268": 40, "3134": 40, "2168": 40, "potteri": 40, "ceram": 40, "32711": 40, "2984": 40, "4031": 40, "6013": 40, "8802": 40, "clai": 40, "32712": 40, "2829": 40, "502": 40, "8221": 40, "glass": 40, "3272": 40, "2191": 40, "7411": 40, "2614": 40, "2932": 40, "7315": 40, "cement": 40, "lime": 40, "gypsum": 40, "3273": 40, "3274": 40, "6765": 40, "1364": 40, "2187": 40, "080": [40, 41], "2478": 40, "3279": 40, "3390": 40, "5201": 40, "2515": 40, "9296": 40, "iron": 40, "steel": 40, "3311": 40, "098e": 40, "1963": 40, "7132": 40, "aluminum": 40, "3313": 40, "3558": 40, "601": 40, "674": 40, "6098": 40, "nonferr": 40, "3314": 40, "9701": 40, "8019": 40, "3984": 40, "1891": 40, "foundri": 40, "3315": 40, "5445": 40, "2055": 40, "3417": 40, "1252": 40, "forg": 40, "stamp": 40, "1715": 40, "0691": 40, "3721": 40, "645": 40, "9009": 40, "cutleri": 40, "8808": 40, "3925": 40, "4913": 40, "073": [40, 48], "3324": 40, "7136": 40, "9478": 40, "1831": 40, "896": 40, "3546": 40, "screw": 40, "nut": [40, 61], "bolt": 40, "3327": 40, "3821": 40, "1968": 40, "052": 40, "824": 40, "7679": 40, "engrav": 40, "heat": 40, "alli": 40, "3328": 40, "5874": 40, "3997": 40, "3353": 40, "752": 40, "623": 40, "ordnanc": 40, "332992": 40, "332995": 40, "139e": 40, "3325": 40, "3326": 40, "3329": 40, "332993": 40, "332994": 40, "5931": 40, "6094": 40, "1880": 40, "2244": 40, "9618": 40, "332": 40, "4324": 40, "5660": 40, "7461": [40, 48], "33311": 40, "009e": 40, "2528": 40, "5132": 40, "33312": 40, "33313": 40, "421e": 40, "2319": [40, 48], "9664": 40, "3095": 40, "3463": 40, "4982": 40, "metalwork": 40, "3335": 40, "4830": 40, "2648": 40, "600": 40, "1694": 40, "turbin": 40, "3336": 40, "8186": 40, "2951": 40, "3879": 40, "582": 40, "955": 40, "3332": 40, "3334": 40, "3339": 40, "9758": 40, "8177": 40, "6801": 40, "3365": [40, 48], "389e": 40, "2296": 40, "audio": 40, "3342": 40, "728e": 40, "2782": 40, "electromed": 40, "instrument": 40, "876e": 40, "2124": 40, "831": [40, 41], "electron": 40, "3344": 40, "3346": 40, "893e": 40, "1535": 40, "329": 40, "applianc": 40, "3352": 40, "9869": 40, "3063": 40, "3295": 40, "3351": 40, "092e": 40, "7284": 40, "vehicl": 40, "3361": 40, "3362": 40, "3363": 40, "8727": 40, "1319": 40, "6359": 40, "aircraft": 40, "336411": 40, "336413": 40, "999e": 40, "1454": 40, "743": 40, "aerospac": 40, "336414": 40, "336415": 40, "336419": 40, "932e": 40, "3846": 40, "railroad": 40, "stock": [40, 59], "102e": 40, "5066": 40, "1084": 40, "boat": 40, "3366": 40, "779": 40, "7078": 40, "3369": 40, "8638": 40, "4587": 40, "8438": 40, "9542": 40, "sawmil": 40, "wood": 40, "3211": 40, "4960": 40, "2021": 40, "9599": 40, "veneer": 40, "plywood": 40, "5189": 40, "9036": 40, "4529": 40, "3687": 40, "prefabr": 40, "mobil": [40, 47], "321991": 40, "321992": 40, "5435": 40, "6788": 40, "4140": 40, "2680": 40, "3936": 40, "2236": 40, "7907": 40, "2156": 40, "9836": 40, "1298": 40, "5612": 40, "3391": 40, "411e": 40, "1565": 40, "amus": 40, "sport": 40, "33992": 40, "33993": 40, "2138": 40, "4970": 40, "2826": 40, "757": [40, 41, 48], "3401": 40, "7678": 40, "3399": 40, "4899": 40, "8913": 40, "1693": 40, "893": 40, "1580": 40, "1163": 40, "6909": 40, "merchant": 40, "wholesal": 40, "4231": 40, "8269": 40, "6204": 40, "2427": 40, "3511": 40, "furnitur": 40, "furnish": 40, "4232": 40, "7603": 40, "698": 40, "4895": 40, "lumber": 40, "4233": 40, "7793": 40, "7344": 40, "2415": 40, "3060": 40, "042": [40, 41, 48], "commerci": [40, 60], "4234": 40, "1896": 40, "519": [40, 48], "2031": 40, "4274": 40, "9198": 40, "823": 40, "7558": 40, "4195": 40, "076e": 40, "2213": 40, "6418": 40, "4265": 40, "7256": 40, "2269": 40, "4238": 40, "8008": 40, "recycl": 40, "42393": 40, "9854": 40, "3166": 40, "6248": 40, "6166": 40, "durabl": 40, "4239": 40, "9834": 40, "9299": 40, "3813": 40, "2360": 40, "4241": 40, "9631": 40, "5826": 40, "4061": 40, "1671": 40, "sundri": 40, "wholesalerss": 40, "4242": 40, "4246": 40, "663e": 40, "2150": 40, "731": 40, "notion": 40, "4243": 40, "8318": 40, "9813": 40, "1937": 40, "groceri": 40, "4244": 40, "5788": 40, "8515": 40, "1461": 40, "2923": 40, "8653": 40, "farm": 40, "4245": 40, "2950": [40, 41], "5444": 40, "3014": 40, "2957": 40, "8858": 40, "4247": 40, "555e": 40, "2434": 40, "368": 40, "4248": 40, "5327": 40, "6504": 40, "2713": 40, "42491": 40, "3704": 40, "nondur": 40, "4249": 40, "5692": 40, "3790": 40, "2543": 40, "agent": 40, "broker": 40, "4251": 40, "1193": 40, "5585": 40, "3485": 40, "8025": 40, "3868": 40, "4101": 40, "7998": 40, "8077": 40, "4411": 40, "8101": 40, "2705": 40, "1331": 40, "817": 40, "5490": 40, "4412": 40, "7648": 40, "0634": 40, "2446": 40, "850": [40, 42], "2852": 40, "4413": 40, "1683": [40, 42], "3592": 40, "3007": 40, "1227": 40, "7563": 40, "1726": 40, "498": 40, "4611": 40, "443111": 40, "5737": 40, "7708": 40, "6879": 40, "7784": 40, "0374": 40, "1730": 40, "4392": 40, "4441": 40, "44413": 40, "1277": 40, "4693": 40, "1425": 40, "4072": 40, "hardwar": 40, "2386": 40, "327": 40, "5456": 40, "3896": 40, "721": 40, "lawn": 40, "garden": 40, "4442": 40, "2166": 40, "2082": 40, "1915": 40, "4451": 40, "1616": 40, "4452": 40, "2629": 40, "8689": 40, "2684": 40, "980": 40, "7891": 40, "077": 40, "wine": 40, "liquor": 40, "4453": 40, "1459": 40, "4922": 40, "7841": 40, "689": 40, "pharmaci": 40, "44611": 40, "4348": 40, "1489": 40, "512": 40, "8283": 40, "4951": 40, "1951": 40, "4459": 40, "gasolin": 40, "station": 40, "1103": 40, "5162": 40, "1800": 40, "4633": 40, "2426": 40, "cloth": 40, "shoe": 40, "44821": 40, "4483": 40, "2557": 40, "3712": 40, "1658": 40, "1941": 40, "5779": 40, "3418": 40, "4758": 40, "8641": 40, "jewelri": 40, "luggag": 40, "2215": 40, "2885": 40, "6310": 40, "4999": 40, "5275": 40, "2639": 40, "7650": 40, "1114": 40, "6393": 40, "836": 40, "needlework": 40, "45113": 40, "668e": 40, "7083": 40, "2800": [40, 42], "5295": 40, "1476": 40, "5755": 40, "8103": 40, "45121": 40, "2523": 40, "8203": 40, "450": 40, "5079": 40, "s45211": [40, 42], "2063": 40, "7215": 40, "1315": [40, 41], "4641": 40, "merchandis": 40, "1100": 40, "1773": 40, "2375": 40, "518": 40, "4577": 40, "florist": 40, "4531": 40, "3223": 40, "4774": 40, "6135": 40, "stationeri": 40, "45321": 40, "5986": 40, "0326": 40, "2732": 40, "4533": 40, "8427": 40, "7584": 40, "2908": 40, "2727": 40, "gift": 40, "novelti": 40, "souvenir": 40, "45322": 40, "3744": 40, "5060": 40, "3044": 40, "4539": 40, "4518": 40, "5257": 40, "1904": 40, "454111": 40, "8738": 40, "4826": 40, "auction": 40, "454112": 40, "6663": 40, "3104": 40, "454113": 40, "5161": 40, "2790": 40, "vend": 40, "4542": 40, "7982": 40, "6144": 40, "1008": 40, "45431": 40, "2986": 40, "3140": 40, "545": 40, "8566": 40, "45439": 40, "4085": 40, "6284": 40, "3751": 40, "717": 40, "3267": 40, "4490": 40, "2046": 40, "3477": 40, "4544": 40, "475": [40, 56], "208e": 40, "1622": 40, "8898": 40, "rail": 40, "1954": 40, "483": 40, "727e": 40, "truck": [40, 42], "7630": 40, "0133": 40, "1291": 40, "5098": 40, "bu": 40, "urban": [40, 52], "transit": [40, 61], "4851": 40, "4852": 40, "4855": 40, "4859": 40, "9145": 40, "1719": 40, "5829": 40, "taxi": 40, "limousin": 40, "4853": 40, "8472": 40, "8000": 40, "673e": 40, "480": 40, "scenic": 40, "sightse": 40, "2172": 40, "4842": 40, "8118": 40, "incident": 40, "6852": 40, "5261": 40, "postal": 40, "6028": 40, "1553": 40, "9072": 40, "courier": 40, "messeng": 40, "7625": 40, "2671": 40, "4540": 40, "storag": 40, "2103": 40, "1611": 40, "554": [40, 48], "891": 40, "2205": 40, "newspap": 40, "51111": 40, "1191": 40, "2376": 40, "3465": 40, "5848": 40, "softwar": 40, "5111": 40, "2354": 40, "685": 40, "346": [40, 48], "5617": 40, "5112": 40, "952": 40, "motion": 40, "5121": 40, "2032": 40, "5122": 40, "6398": 40, "3136": 40, "radio": 40, "televis": 40, "broadcast": 40, "cabl": 40, "5151": 40, "5152": 40, "5175": 40, "1620": 40, "936": 40, "8595": 40, "portal": 40, "51913": 40, "618e": 40, "2601": 40, "wire": 40, "telecommun": 40, "carrier": 40, "5171": 40, "524e": 40, "515e": 40, "1711": 40, "host": [40, 46, 60], "5182": 40, "263e": 40, "3120": 40, "6518": 40, "archiv": 40, "51912": 40, "4295": 40, "1435": 40, "2627": 40, "9445": 40, "5191": 40, "52211": 40, "52219": 40, "436e": 40, "1242": 40, "561": 40, "52212": 40, "52213": 40, "5783": 40, "7155": 40, "2066": 40, "depositori": 40, "814e": 40, "1415": 40, "810": 40, "commod": 40, "fund": [40, 47], "invest": [40, 46, 53, 61], "1217": 40, "9944": 40, "estat": [40, 52], "2280": 40, "1274": 40, "6691": 40, "automot": 40, "leas": 40, "5321": 40, "2069": 40, "0345": 40, "2606": 40, "7178": 40, "tape": 40, "disk": 40, "53223": 40, "6701": 40, "784": 40, "53221": 40, "53222": 40, "53229": 40, "5323": 40, "1012": 40, "0036": 40, "3219": 40, "5298": 40, "intang": 40, "asset": 40, "5324": 40, "446e": 40, "2815": 40, "8942": 40, "607": 40, "5411": 40, "004e": 40, "1333": 40, "029": [40, 48], "prepar": 40, "bookkeep": 40, "payrol": 40, "5412": 40, "386e": 40, "1453": 40, "architectur": 40, "5413": 40, "585e": 40, "5414": 40, "6793": 40, "0642": 40, "2382": 40, "5415": 40, "509e": 40, "1205": 40, "5416": 40, "045e": 40, "5417": 40, "997e": 40, "1515": 40, "5418": 40, "1681": 40, "veterinari": 40, "54194": 40, "5952": 40, "7522": 40, "1921": 40, "2186": 40, "809": 40, "9718": 40, "5419": 40, "226e": 40, "8199": 40, "enterpris": 40, "7092": 40, "9170": 40, "2364": 40, "5613": 40, "5211": 40, "0769": 40, "1475": 40, "2318": 40, "5614": [40, 42], "2759": 40, "4388": 40, "1603": 40, "633": 40, "5902": 40, "reserv": 40, "5615": 40, "8363": 40, "5104": 40, "2115": 40, "954": 40, "4217": 40, "5616": 40, "6381": 40, "1510": 40, "2835": 40, "3085": 40, "dwell": 40, "56173": 40, "2131": 40, "7452": 40, "1447": 40, "2910": 40, "1473": 40, "5799": 40, "5611": 40, "5619": 40, "3873": 40, "7759": 40, "wast": 40, "remedi": 40, "6367": 40, "7567": 40, "1659": 40, "724": 40, "9620": 40, "elementari": 40, "secondari": 40, "6111": 40, "0227": 40, "1556": 40, "2906": 40, "junior": 40, "6112": 40, "6113": 40, "6187": 40, "1199": 40, "6114": 40, "6115": 40, "8042": 40, "8479": 40, "6116": 40, "6117": 40, "7034": 40, "8796": 40, "1821": 40, "physician": 40, "6211": 40, "1328": 40, "8253": 40, "dentist": 40, "6212": 40, "7495": 40, "9965": 40, "4369": 40, "chiropractor": 40, "62131": 40, "2199": 40, "3757": 40, "3638": 40, "4932": 40, "optometrist": 40, "62132": 40, "3748": 40, "7228": 40, "2592": 40, "6213": 40, "4644": 40, "4511": 40, "2678": 40, "outpati": 40, "6214": 40, "6209": 40, "1278": 40, "3703": 40, "8715": 40, "6216": 40, "1292": 40, "3263": 40, "1418": 40, "1487": 40, "6215": 40, "6219": 40, "9278": 40, "7353": 40, "1326": 40, "6678": 40, "1143": 40, "8956": 40, "6231": 40, "1585": 40, "6208": 40, "1312": 40, "4158": 40, "residenti": [40, 42, 52], "6232": [40, 42], "6233": [40, 42], "6239": [40, 42], "0647": 40, "1470": 40, "2775": 40, "2989": 40, "6241": 40, "1112": 40, "7260": 40, "3728": 40, "1503": 40, "6242": 40, "3755": 40, "2794": 40, "1722": 40, "9234": 40, "vocat": 40, "rehabilit": 40, "6243": 40, "1419": 40, "3375": 40, "716": 40, "1185": 40, "child": [40, 44, 52], "6244": 40, "3177": 40, "7836": 40, "1434": 40, "5989": 40, "artist": 40, "art": [40, 59], "spectat": 40, "5905": 40, "7767": 40, "1907": 40, "2167": 40, "9644": 40, "museum": 40, "galleri": 40, "4334": 40, "4099": 40, "71395": 40, "8548": 40, "7123": 40, "5529": 40, "2288": 40, "gambl": 40, "713": 40, "8607": 40, "1324": 40, "4289": 40, "901": [40, 41], "accommod": [40, 46], "7211": 40, "2165": [40, 41], "2804": 40, "2442": 40, "506": 40, "park": 40, "camp": 40, "board": [40, 54], "7212": 40, "7213": 40, "6767": 40, "restaur": 40, "722": 40, "7224": 40, "3845": 40, "3157": 40, "1378": 40, "6277": 40, "0522": 40, "2704": 40, "977": 40, "8111": 40, "811192": 40, "4106": 40, "0577": 40, "1389": 40, "1383": 40, "6828": 40, "wash": 40, "3871": 40, "3298": 40, "2919": 40, "1851": 40, "8112": 40, "8832": 40, "3202": 40, "9258": 40, "8113": 40, "5347": 40, "4836": 40, "8114": 40, "81143": 40, "4203": 40, "0746": 40, "3150": 40, "barber": 40, "812111": 40, "9681": 40, "4421": 40, "9867": 40, "7467": 40, "beauti": 40, "salon": 40, "812112": 40, "1777": 40, "2348": 40, "nail": 40, "812113": 40, "81219": 40, "8031": 40, "2039": 40, "dryclean": 40, "laundri": 40, "8123": 40, "2282": 40, "7262": 40, "2206": 40, "938": 40, "2042": 40, "funer": 40, "cemeteri": 40, "crematori": 40, "8122": 40, "2626": 40, "3135": 40, "8771": 40, "3518": 40, "8129": 40, "2217": 40, "4546": 40, "4147": 40, "877": 40, "religi": 40, "8131": 40, "2863": 40, "7055": 40, "5780": 40, "civic": 40, "grantmak": 40, "8133": 40, "8513": 40, "9769": 40, "961": 40, "5644": 40, "81393": 40, "934e": 40, "polit": [40, 47, 48], "646e": 40, "2179": 40, "3089": 40, "5909": 40, "1865": [40, 48], "765": 40, "6746": 40, "execut": 40, "legisl": [40, 61], "92111": 40, "92112": 40, "92114": 40, "92115": 40, "6798": 40, "3444": 40, "1320": 40, "4210": 40, "9385": 40, "92113": 40, "795": [40, 41], "92119": 40, "9480": 40, "2676": 40, "justic": 40, "8924": 40, "4905": 40, "1197": 40, "6577": 40, "787": 40, "6796": 40, "9979": 40, "1359": 40, "708": 40, "4131": 40, "environment": [40, 54], "8922": 40, "1751": 40, "econom": [40, 47, 49, 54, 59], "1490": 40, "affair": 40, "755e": 40, "1346": 40, "991e": 40, "762": 40, "815": 40, "3093": 40, "2084": 40, "14403": 40, "21479": 40, "smallest": [40, 53], "eigenvalu": 40, "singular": 40, "fell": 40, "12995": 40, "10860": 40, "2134": 40, "read_dta": 41, "sm": 41, "pvalu": [29, 41], "ttest_ind": 41, "03": 41, "statisticallli": 41, "spuriou": 41, "quick": [41, 47, 48, 59], "dirti": 41, "1744": 41, "1760": 41, "4917640058792273": 41, "darn": 41, "isol": [41, 49, 53], "ness": 41, "t2": 41, "p2": 41, "callback": 41, "Thats": 41, "1f": 41, "3f": 41, "4870": 41, "1128": 41, "4868": 41, "1141": 41, "0965": 41, "0320": 41, "2969": 41, "18927": 41, "heteroscedast": [41, 48], "02": [41, 48], "4861": 41, "0821": 41, "0017": 41, "953e": 41, "0025": 41, "0047": 41, "0316": 41, "0032": 41, "0186": 41, "0112": 41, "18631": 41, "non_college_grad": 41, "1116": 41, "4863": 41, "1161": 41, "0759": 41, "0090": 41, "0281": 41, "0123": 41, "0188": 41, "0110": 41, "18624": 41, "noncolleg": 41, "salt": 41, "underpow": 41, "reject": [41, 50], "null": 41, "college_grad": 41, "00198": 41, "3499": 41, "0736": 41, "0286": 41, "0019": 41, "0108": 41, "0197": 41, "14157": 41, "90e": 41, "1366": 41, "1361": 41, "0408": 41, "0062": 41, "592": 41, "0460": 41, "4394": 41, "1122": 41, "4860": 41, "1187": 41, "0807": 41, "0021": 41, "971": 41, "0001": 41, "0026": 41, "947": 41, "0048": 41, "0287": 41, "0038": 41, "18630": 41, "insignific": 41, "fifteen": 41, "percent": [41, 48], "caveat": 41, "acknowledg": 41, "workplac": [41, 42], "unrel": 41, "broader": 41, "perhap": [42, 49, 52], "intra": 42, "regard": 42, "has_college_educ": 42, "8393": 42, "4365e": 42, "873e": 42, "639e": 42, "7144": 42, "054e": 42, "18030": 42, "27634": 42, "semi": 42, "colon": 42, "suppress": 42, "voil\u00e0": 42, "plm": 42, "whatev": [42, 59, 61], "typeerror": 42, "str": 42, "fe_group": 42, "group_cod": 42, "int16": 42, "semicolon": 42, "nce8": 42, "opt": 42, "1871": 42, "valuewarn": 42, "rank": 42, "bottom": [42, 50], "ordinari": 42, "dimens": 42, "catch": [42, 61], "multiindex": 42, "hate": 42, "multi": [42, 53], "43344": 42, "19200": 42, "60000": 42, "32640": 42, "38400": 42, "cps_w_multiindex": 42, "1183": 42, "missingvaluewarn": 42, "__init__": 42, "exog": 42, "check_rank": 42, "1414": 42, "6716": 42, "122341": 42, "747": 42, "593e": 42, "886e": 42, "065e": 42, "9511": 42, "665e": 42, "405e": 42, "924e": 42, "365p": 42, "entityid": 42, "0x2f61875e0": 42, "kid": [43, 44, 47], "perspect": [43, 46], "stabl": [43, 49], "uh": 43, "local": [43, 54], "sensit": [43, 51, 60], "ton": [43, 54], "nyc": 43, "demean": 43, "y_i": 43, "hurrican": 43, "dip": 43, "disrupt": 43, "stick": 43, "algebra": 43, "slope": [44, 48], "shock": 44, "chief": 44, "uncorrel": 44, "leverag": 44, "htose": 44, "higherarch": 44, "hierarhc": 44, "hausman": 44, "accompanyingli": 44, "frequent": 44, "wavi": 44, "confess": 44, "diagon": 44, "smarter": 44, "princip": 44, "execis": 44, "mix": 44, "intens": 45, "wrestl": [45, 58, 59, 60], "aris": 45, "unavoid": 45, "exposit": 45, "comment": 45, "facet": 45, "embrac": 45, "urg": 45, "emblemat": 45, "intellectu": 45, "arbitrag": 45, "port": 45, "insight": [45, 56], "richli": 45, "unfamiliar": 45, "orient": 45, "infuriatingli": 45, "salient": 45, "consolid": 45, "transcrib": 45, "concis": 45, "depriv": 45, "connect": [45, 56], "bedrock": 45, "lectur": [45, 51], "prolong": 45, "modern": 45, "flit": 45, "cultiv": 45, "hello": 46, "promis": [46, 48, 53], "throw": [46, 50], "intellig": [46, 54], "ticket": 46, "pilot": [46, 52], "reap": 46, "sizeabl": 46, "burgeon": 46, "pitfal": 46, "endeavor": [46, 59], "curricula": 46, "hardest": 46, "deliveri": 46, "unifi": 46, "complement": 46, "depth": [46, 51], "introduct": 46, "heavili": 46, "interdisciplinari": 46, "grasp": 46, "inferenti": 46, "concurr": 46, "comfort": 46, "facilit": 46, "collabor": [46, 60], "bilingu": 46, "git": 46, "pickup": 46, "workshop": 46, "preliminari": [46, 61], "schedul": [46, 52], "constrast": 47, "stream": 47, "subscript": [47, 54], "signup": 47, "hing": 47, "email": 47, "music": 47, "unverifi": 47, "banal": 47, "textbook": 47, "sutva": [47, 53], "generaliz": 47, "elit": 47, "univer": 47, "emori": 47, "vanderbilt": 47, "unc": 47, "whole": [47, 49, 51, 54, 61], "luxuri": 47, "detroit": 47, "wealth": 47, "cultur": 47, "brand": 47, "imag": [47, 54], "unrepres": 47, "unsurprisingli": 47, "dose": 47, "asian": 47, "island": 47, "sclerosi": 47, "mutat": 47, "descend": 47, "longest": 47, "psychologi": 47, "volunt": 47, "western": 47, "democrat": [47, 48, 54, 59], "facial": 47, "recognit": 47, "overlook": 47, "declar": 47, "billboard": 47, "infinit": [11, 47], "perman": 47, "tv": 47, "000th": 47, "famou": 47, "india": 47, "clinic": [47, 49], "absente": 47, "ngo": 47, "mit": 47, "economist": 47, "clock": 47, "punish": 47, "attend": 47, "doubl": 47, "exempt": 47, "equilibrium": 47, "angu": 47, "deaton": 47, "nanci": 47, "cartwright": 47, "rct": 47, "monitor": 47, "environ": [47, 54], "rand": 47, "particip": 47, "catastroph": 47, "coverag": 47, "oregon": 47, "ohp": 47, "medicaid": 47, "lotteri": 47, "elig": [47, 60], "winner": 47, "uninsur": 47, "expans": 47, "tension": 47, "interfac": 47, "dog": 47, "bark": 47, "hurri": 47, "gloss": 48, "deserv": 48, "registr": 48, "republican": 48, "tenur": 48, "faculti": 48, "turnout": 48, "north": [48, 59], "carolina": [48, 59], "dichotom": 48, "problemat": [48, 49, 53], "css_tutori": 48, "exercise_data": 48, "voter_turnout": 48, "unaffili": 48, "7416": 48, "07": 48, "5768": 48, "9919": 48, "154e": 48, "9917": 48, "156e": 48, "0075": 48, "4426": 48, "2128": 48, "5748": 48, "150e": 48, "151e": 48, "7754": 48, "0562": 48, "2361": 48, "343": 48, "5735": 48, "148e": 48, "9916": 48, "7988": 48, "0797": 48, "0606": 48, "2341": 48, "dem": 48, "linearhypothesi": 48, "t_test": 48, "contrastresult": 48, "c0": 48, "0191": 48, "voila": 48, "7383": 48, "436": 48, "pro": [48, 51], "alon": 48, "age_x_femal": 48, "917": 48, "5764": 48, "9915": 48, "157e": 48, "6961": 48, "0933": 48, "0009": 48, "0015": 48, "1883": 48, "2388": 48, "340": 48, "declin": 48, "plu": 48, "simplest": 48, "beta_2": [48, 51], "beta_3": [48, 51], "voted_": 48, "linearli": 48, "old_x_femal": 48, "5717": 48, "144e": 48, "147e": 48, "6761": 48, "1015": 48, "0138": 48, "0111": 48, "1820": 48, "peak": 48, "ggplot": 48, "ae": 48, "plotninewarn": 48, "implementedfor": 48, "9223363270683622540": 48, "lingo": 49, "homogen": 49, "heterogen": 49, "fda": 49, "approv": 49, "chronic": [49, 59], "hail": 49, "changer": 49, "undergo": 49, "minu": 49, "crippl": 49, "disabl": 49, "bucket": 49, "proportion": 49, "latino": 49, "indirect": 49, "spillov": 49, "vaccin": 49, "immun": 49, "sick": 49, "indirectli": 49, "embodi": 49, "newsfe": 49, "apart": 49, "youtub": [50, 51], "nitti": 50, "gritti": 50, "winnow": 50, "proce": [50, 61], "dissimilar": 50, "unpair": 50, "thrown": 50, "root": 50, "euclidean": 50, "strength": 50, "weak": 50, "strike": 50, "strict": 50, "maximium": 50, "gari": [50, 51], "minunt": 50, "ho": 51, "imai": 51, "stuart": 51, "streamlin": 51, "concepu": 51, "framwork": 51, "insert": 51, "tremend": 51, "02315": 51, "extrapol": [51, 59], "funni": 51, "imbalanc": 51, "blue": 51, "badli": 51, "uncomfort": 51, "drew": 51, "parabola": 51, "invert": 51, "theoret": [51, 61], "nervou": 51, "grei": 51, "determinist": 51, "situt": 51, "con": 51, "fashion": 51, "impov": 51, "charl": 52, "ketter": 52, "1920": 52, "1947": 52, "invok": 52, "kingdom": 52, "fundat": 52, "ancient": 52, "ruin": [52, 59], "poster": 52, "pile": 52, "archeologist": 52, "bone": 52, "toothbrush": 52, "detect": 52, "archeolog": 52, "ground": 52, "penetr": 52, "radar": 52, "scan": [52, 59], "foundat": [52, 59], "spot": 52, "glanc": 52, "feasabl": 52, "excav": 52, "importantli": 52, "sorri": 52, "topologi": 52, "hydrologi": 52, "soil": 52, "geographi": [52, 59], "emmin": 52, "doabl": 52, "africa": 52, "infant": 52, "mortal": 52, "migrant": 52, "literatur": 53, "warrant": 53, "unexpect": 53, "srm": 53, "optimist": 53, "hmmm": 53, "rock": 53, "solid": [53, 56, 61], "benjamini": 53, "hochberg": 53, "westfal": 53, "bonferroni": 53, "conserv": 53, "unrealist": 53, "exhaust": [54, 60], "ethnic": 54, "tabular": 54, "shapefil": 54, "shorter": 54, "nhgi": 54, "stabil": 54, "wharton": 54, "membership": 54, "\u00e1": 54, "cart": 54, "microsoft": 54, "planetari": 54, "wildlif": 54, "aw": 54, "registri": 54, "genom": 54, "nih": 54, "telescop": 54, "mortgag": 54, "overst": 54, "freeli": 54, "conflict": 54, "terror": 54, "pollut": 54, "flood": 54, "freedom": 54, "airbnb": [54, 60], "traffic": 54, "underpin": [54, 61], "geospati": 54, "ihgi": 54, "internation": 54, "privaci": [11, 54], "withheld": 54, "raster": 54, "nasa": 54, "elev": 54, "plant": 54, "grow": 54, "wavelength": 54, "crop": 54, "imageri": 54, "infrastructur": 54, "factori": 54, "offici": [54, 59], "obscen": 54, "noaa": 54, "govt": 54, "awesom": 54, "skeptic": [56, 61], "mindset": 56, "untest": 56, "8am": 56, "morn": 56, "annoi": 56, "rubric": 56, "sparingli": 56, "exception": 56, "425": 56, "demonstr": [56, 61], "meati": 56, "paragraph": [56, 61], "succinct": 56, "superfici": 56, "skate": 56, "surfac": 56, "dedic": 56, "prompt": [56, 59], "stuck": 58, "illus": 58, "debug": 58, "unse": 58, "prece": 58, "fragment": 59, "disciplin": 59, "myopic": 59, "strive": 59, "merit": 59, "inquiri": 59, "unseen": 59, "unobserv": 59, "desir": 59, "opioid": 59, "overdos": 59, "pain": 59, "enact": 59, "ger": 59, "aspect": [59, 61], "phenomena": 59, "centuri": 59, "appreci": 59, "arros": 59, "carmen": 59, "reinhart": 59, "kenneth": 59, "rogoff": 59, "eight": 59, "folli": 59, "hundr": 59, "sixti": 59, "pundit": 59, "regularli": 59, "decri": 59, "crise": [59, 60], "feroc": 59, "wipe": 59, "death": 59, "hiv": 59, "gai": 59, "dy": 59, "cigarett": 59, "lung": 59, "climat": 59, "quot": 59, "temperatur": 59, "earth": 59, "unfair": 59, "noodl": 59, "casual": 59, "fancier": 59, "necessit": 59, "novel": 59, "shed": 59, "latent": 59, "rare": 59, "vera": 59, "rubin": 59, "rotat": 59, "star": 59, "galaxi": 59, "ken": [59, 61], "crisi": 59, "truth": 59, "demand": 59, "judgement": 59, "curv": 59, "physic": 59, "particl": 59, "she": 59, "cold": 59, "minimum": 59, "iowa": 59, "presidenti": 59, "causus": 59, "politician": 59, "presid": 59, "awfulli": 59, "occasion": 59, "stumbl": 59, "reiter": 59, "circumstanti": 59, "nutshel": [59, 61], "father": 59, "causat": 59, "simplif": 59, "wealthi": 59, "caucus": 59, "trendi": 59, "mammogram": 59, "radiologist": 59, "tumor": 59, "tomorrow": 59, "mri": 59, "sml": 59, "garbag": 59, "analogu": 59, "ridicul": 59, "benign": 59, "meaningless": 59, "contextu": 59, "februari": 60, "13th": 60, "12pm": 60, "reinvent": 60, "wheel": 60, "revis": 60, "pedestrian": 60, "freewai": 60, "bureaucrat": 60, "unclear": 60, "fraud": 60, "deter": 60, "entitl": 60, "recipi": 60, "largest": 60, "shoot": 60, "mental": 60, "societ": 60, "tenni": 60, "hinder": 60, "tournament": 60, "phone": 60, "churn": 60, "yourselv": 60, "curat": 60, "translat": 61, "guidelin": 61, "policymak": 61, "slowli": 61, "lai": 61, "graf": 61, "journalist": 61, "wall": 61, "farther": 61, "broadli": 61, "conclud": 61, "brief": 61, "drown": 61, "bog": 61, "linchpin": 61, "hook": 61, "caught": 61, "player": 61, "laid": 61, "fellow": 61, "cross": 61, "dot": 61, "masteri": 61, "deligi": 61, "erron": 61, "homework": 61, "afford": 61, "honest": 61, "humil": 61, "sought": 61, "appendic": 61, "bore": 61, "opinion": 61, "exercise_abtest": 11, "ex4_avg_oec": 11, "ex5_avg_guardrail": 11, "ex7_ttest_pvalu": 11, "ex9_ttest_pvalue_click": 11, "ex11_guard_": 11, "ex11_guard_pvalu": 11, "ex11_oec_": 11, "ex11_oec_pvalu": 11, "ex14_se_treat": 11, "assist": 11, "finit": 11}, "objects": {}, "objtypes": {}, "objnames": {}, "titleterms": {"backward": [0, 2], "design": [0, 2, 23, 61], "overview": [0, 2], "1": [0, 2, 7, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 24, 25, 28, 29, 30, 31, 33, 34, 36, 37, 38, 40, 41, 53, 59], "defin": [0, 2, 53], "your": [0, 1, 2, 7, 11, 16, 22, 30, 32, 37, 50, 52, 53, 60, 61], "problem": [0, 2, 46, 52, 60], "2": [0, 1, 2, 7, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 23, 24, 25, 27, 28, 29, 30, 31, 33, 34, 36, 37, 38, 40, 41, 48, 53, 60], "question": [0, 2, 15, 17, 18, 19, 23, 31, 52, 59, 60], "you": [0, 2, 3, 7, 15, 33, 48, 53, 58], "wish": [0, 2], "answer": [0, 2, 15, 33, 60], "A": [0, 2, 3, 11, 30, 50, 53], "digress": [0, 2], "supervis": [0, 2], "machin": [0, 2, 7], "learn": [0, 2, 7, 32], "ud": 1, "midterm": 1, "2022": 1, "part": [1, 60], "estim": [1, 11, 28, 29, 40, 41], "The": [1, 3, 8, 11, 16, 23, 25, 26, 27, 39, 48, 49, 51, 52], "effect": [1, 8, 10, 11, 20, 24, 29, 36, 38, 41, 42, 43, 44, 48, 49], "educ": 1, "employ": 1, "get": [1, 2, 22, 37], "know": [1, 22, 37], "data": [1, 2, 7, 11, 21, 22, 25, 30, 32, 34, 37, 46, 50, 54, 60, 61], "subset": 1, "peopl": 1, "over": 1, "25": 1, "labor": 1, "forc": 1, "measur": [1, 11, 29, 41, 50], "good": [2, 54], "why": [2, 8, 59], "i": [2, 7, 8, 12, 50, 52, 59], "have": [2, 7, 15, 33], "import": [2, 11], "3": [2, 7, 11, 12, 13, 14, 15, 18, 19, 20, 21, 22, 23, 24, 25, 28, 29, 30, 33, 34, 36, 37, 38, 40, 41, 53, 59, 60], "write": [2, 61], "down": 2, "what": [2, 5, 8, 12, 29, 41, 48, 52], "an": [2, 7], "would": 2, "look": [2, 58], "like": 2, "falsifi": 2, "4": [2, 7, 11, 13, 15, 18, 19, 20, 21, 22, 24, 25, 28, 29, 33, 34, 36, 37, 38, 40, 41, 53, 59], "do": [2, 3, 22, 37, 45, 52, 53], "need": [2, 8, 9, 20, 22, 24, 37, 52], "5": [2, 7, 11, 13, 15, 18, 19, 20, 21, 22, 24, 25, 28, 29, 33, 34, 36, 37, 38, 40, 41, 53], "where": [2, 54], "can": [2, 50], "That": [2, 54], "wrap": 2, "up": 2, "beyond": 3, "experi": [3, 11, 12, 23, 29, 41, 49], "b": [3, 11, 53], "test": [3, 11, 23, 32, 53], "isn": [3, 59], "t": [3, 59], "alwai": [3, 7], "feasibl": 3, "legal": [3, 13], "ethic": 3, "go": 3, "futur": [3, 25], "don": 3, "want": 3, "limit": [3, 11, 25, 29, 49], "Not": 3, "enough": 3, "abl": 3, "must": [3, 48], "explain": 3, "class": 4, "schedul": 4, "research": 5, "discret": 5, "descript": [5, 9, 33, 59], "analysi": [5, 24, 29, 30, 33, 38, 41], "faith": 5, "represent": 5, "choos": [5, 7, 60], "To": [5, 7, 22, 25, 37, 44, 52, 54], "ask": [5, 52], "present": 5, "context": [5, 30], "valu": 5, "peek": 6, "endogen": 6, "stop": 6, "recommend": 7, "respons": 7, "train": 7, "carefulli": 7, "onli": 7, "us": [7, 48, 50, 60], "opaqu": 7, "model": [7, 21, 25, 44, 50, 51], "when": [7, 48, 50], "function": 7, "form": [7, 22, 37], "constraint": 7, "compar": 7, "predict": [7, 25, 31, 59], "sub": 7, "popul": 7, "No": 7, "option": 7, "tool": 7, "interpret": [7, 10, 20, 21, 22, 36, 37, 48, 53], "constrain": 7, "neural": 7, "network": 7, "introspect": 7, "evalu": [8, 14], "real": [8, 52], "studi": [8, 14, 26, 27, 39], "sun": 8, "bright": 8, "dog": 8, "too": 8, "baselin": 8, "differ": [8, 45, 59], "we": [8, 29, 41], "all": 8, "three": 8, "evaul": 8, "differenti": 8, "treatment": [8, 49], "come": 8, "next": [8, 52], "well": 8, "thi": [8, 21, 53, 59], "ha": 8, "been": 8, "unsatisfi": 8, "discuss": 9, "exercis": [9, 11, 12, 13, 15, 17, 20, 21, 22, 24, 25, 28, 29, 30, 31, 33, 34, 36, 37, 38, 40, 41], "8": [9, 11, 15, 18, 19, 20, 21, 22, 24, 25, 28, 29, 36, 37, 38, 40, 41], "takeawai": 9, "absolut": [9, 20, 22, 24, 37], "posit": [9, 20, 22, 24, 37, 59], "solut": [9, 20, 22, 24, 35, 37, 57, 58], "causal": [10, 18, 19, 26, 27, 31, 39, 43, 51, 59], "gender": [10, 28, 40], "udac": 11, "websit": 11, "": [11, 21, 22, 37, 49, 52, 53], "pick": 11, "valid": [11, 47, 53], "6": [11, 13, 15, 18, 19, 20, 21, 22, 24, 25, 28, 29, 33, 34, 36, 37, 38, 40, 41, 53], "7": [11, 13, 15, 18, 19, 20, 21, 22, 24, 25, 28, 29, 33, 34, 36, 37, 38, 40, 41], "9": [11, 18, 19, 21, 22, 25, 28, 29, 37, 38, 40, 41], "10": [11, 21, 22, 25, 28, 29, 37, 40, 41], "11": [11, 21, 22, 25, 28, 29, 37, 40, 41], "12": [11, 21, 22, 25, 28, 29, 37, 40, 41], "13": [11, 21, 22, 25, 28, 29, 37, 40], "14": [11, 21, 22, 25, 28, 37, 40], "counter": 12, "factual": 12, "experiment": 12, "ideal": 12, "counterfactu": 12, "random": 12, "identifi": [12, 61], "marijuana": 13, "violent": 13, "crime": [13, 15, 33], "gym": 14, "membership": 14, "car": 14, "insur": 14, "billboard": 14, "polic": 15, "expenditur": 15, "exploratori": [15, 31, 60], "after": [15, 33], "first": [16, 60], "stakehold": [16, 17, 30, 52], "propos": [16, 30], "task": [16, 30], "convert": [17, 52], "prompt": [17, 52], "action": [17, 52], "time": [18, 19, 59], "cholera": [18, 19], "indic": [20, 36, 42, 48], "variabl": [20, 36, 42, 48], "omit": [20, 36], "bia": [20, 36], "interact": [20, 36, 48], "credit": 21, "risk": [21, 25], "prep": 21, "fit": 21, "let": [21, 22, 37], "see": 21, "15": [11, 21, 22, 25, 37], "16": [21, 22, 25, 37], "17": [21, 25], "match": [22, 37, 50, 51], "packag": [22, 37], "python": [22, 37], "v": [22, 37], "r": [22, 37], "instal": [22, 37], "dame": [22, 37], "flame": [22, 37], "setup": [22, 37], "output": [22, 37], "back": [22, 37], "dataset": [22, 37], "check": [22, 29, 37, 41, 50, 53], "analyz": [22, 37, 50], "other": [22, 37, 54], "optim": 23, "ab": 23, "traffic": [24, 38], "death": [24, 38], "fix": [24, 38, 42, 43, 44, 48], "demean": [24, 38], "mortgag": 25, "delinqu": 25, "gradescop": [11, 25, 29], "autograd": [11, 25, 29], "submiss": [11, 25, 29], "clean": 25, "organ": 25, "now": 25, "18": 25, "make": [26, 27, 39, 52], "potenti": [26, 27, 39], "outcom": [26, 27, 39], "concret": [26, 27, 39], "map": [26, 27, 39], "framework": [26, 27, 39], "observ": [26, 27, 39], "infer": [26, 27, 39, 43, 51, 59], "discrimin": [28, 40], "workplac": [28, 40], "resum": [29, 41], "balanc": [29, 41, 50], "race": [29, 41], "heterogen": [29, 41], "did": [29, 41], "just": [29, 41], "solicit": 30, "inform": 30, "from": [30, 45], "project": 30, "vote": 30, "access": 30, "colleg": 30, "campus": 30, "background": [30, 61], "object": 30, "our": 30, "approach": 30, "gener": 31, "classifi": 31, "passiv": 31, "groupbi": [32, 34], "arrest": [32, 34], "group": [32, 42], "structur": 32, "assumpt": [32, 51], "poverti": 33, "acm": 35, "corport": 35, "implement": [42, 53], "cluster": [42, 44], "computation": 42, "effici": 42, "hierarch": 44, "standard": 44, "error": 44, "thing": [44, 48, 53], "rememb": 44, "how": [45, 50], "read": [45, 52, 56], "academ": 45, "edit": 45, "activ": 45, "Be": 45, "patient": 45, "exampl": [45, 59], "domain": 45, "NOT": 45, "summar": 45, "llm": 45, "welcom": 46, "solv": 46, "pre": 46, "requisit": 46, "non": [46, 61], "mid": 46, "student": 46, "syllabu": 46, "intern": [47, 53], "extern": [47, 53], "consider": 47, "trade": 47, "off": 47, "between": [47, 48], "conclus": [47, 59, 61], "dummi": 48, "ar": [48, 53, 54, 58], "ONE": 48, "understand": [48, 60], "two": 48, "categori": 48, "more": 48, "than": 48, "constant": 48, "keep": 48, "straight": 48, "oper": 48, "multipl": 48, "averag": 49, "who": 49, "fine": 49, "print": 49, "ATE": 49, "sutva": 49, "done": 50, "summari": [50, 51, 61], "prune": [50, 51], "similar": 50, "should": [50, 53], "specif": [50, 52], "depend": 51, "imbal": 51, "better": [51, 60], "thought": [51, 59], "cost": 51, "In": 51, "mani": 52, "nick": 53, "think": 53, "about": 53, "befor": 53, "step": 53, "0": 53, "sure": 53, "goal": 53, "plan": 53, "run": 53, "result": [53, 61], "consid": 53, "find": 54, "public": 54, "start": 54, "point": 54, "interest": [54, 58], "sourc": 54, "spatial": 54, "gi": 54, "compon": 54, "satellit": 54, "list": 54, "great": 54, "reflect": 56, "ok": 58, "still": 58, "taxonomi": 59, "versu": 59, "norm": 59, "nope": 59, "diseas": 59, "surveil": 59, "global": 59, "warm": 59, "eda": 59, "last": 59, "manipul": 59, "hard": 59, "classif": 59, "without": 59, "assign": 60, "specifi": 60, "scienc": 61, "report": 61, "technic": 61, "audienc": 61, "introduct": 61, "execut": 61, "appendix": 61}, "envversion": {"sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "nbsphinx": 4, "sphinx": 60}, "alltitles": {"Backwards Design": [[0, "Backwards-Design"], [2, "Backwards-Design"]], "Overview": [[0, "Overview"], [2, "Overview"]], "1) Define Your Problem": [[0, "1)-Define-Your-Problem"], [2, "1)-Define-Your-Problem"]], "2) Define the Question You Wish to Answer": [[0, "2)-Define-the-Question-You-Wish-to-Answer"], [2, "2)-Define-the-Question-You-Wish-to-Answer"]], "A Digression on Supervised Machine Learning": [[0, "A-Digression-on-Supervised-Machine-Learning"], [2, "A-Digression-on-Supervised-Machine-Learning"]], "UDS Midterm 2022, Part 2": [[1, "UDS-Midterm-2022,-Part-2"]], "Estimating The Effect of Education on Employment": [[1, "Estimating-The-Effect-of-Education-on-Employment"]], "Getting to Know Your Data": [[1, "Getting-to-Know-Your-Data"]], "Subsetting for People over 25 in the Labor Force": [[1, "Subsetting-for-People-over-25-in-the-Labor-Force"]], "Measuring Education": [[1, "Measuring-Education"]], "Education and Employment": [[1, "Education-and-Employment"]], "Defining a Good Question": [[2, "Defining-a-Good-Question"]], "Why is Having a Good Question Important?": [[2, "Why-is-Having-a-Good-Question-Important?"]], "3) Write Down What An Answer Would Look Like": [[2, "3)-Write-Down-What-An-Answer-Would-Look-Like"]], "Falsifiability": [[2, "Falsifiability"]], "4) What Data Do You Need?": [[2, "4)-What-Data-Do-You-Need?"]], "5) Where Can You Get That Data?": [[2, "5)-Where-Can-You-Get-That-Data?"]], "Wrapping Up": [[2, "Wrapping-Up"]], "Beyond The Experiment": [[3, "Beyond-The-Experiment"]], "A/B Testing Isn\u2019t Always Feasible": [[3, "A/B-Testing-Isn't-Always-Feasible"]], "A/B Testing Isn\u2019t Always Legal or Ethical": [[3, "A/B-Testing-Isn't-Always-Legal-or-Ethical"]], "Going Beyond Testing for the Future": [[3, "Going-Beyond-Testing-for-the-Future"]], "Don\u2019t want to be limited": [[3, "Don't-want-to-be-limited"]], "Not enough to be able to do it; you must be able to explain it": [[3, "Not-enough-to-be-able-to-do-it;-you-must-be-able-to-explain-it"]], "Researcher Discretion in Descriptive Analysis": [[5, "Researcher-Discretion-in-Descriptive-Analysis"]], "Faithful Representation": [[5, "Faithful-Representation"]], "Choosing What To Ask and Present": [[5, "Choosing-What-To-Ask-and-Present"]], "Context": [[5, "Context"], [30, "Context"]], "Values": [[5, "Values"]], "Peeking / Endogenous Stopping": [[6, "Peeking-/-Endogenous-Stopping"]], "Recommendations for Responsible Machine Learning": [[7, "Recommendations-for-Responsible-Machine-Learning"]], "1. Choose Your Training Data Carefully": [[7, "1.-Choose-Your-Training-Data-Carefully"]], "2. Only Use Opaque Models When You Have To": [[7, "2.-Only-Use-Opaque-Models-When-You-Have-To"]], "3. Use Models with Functional Form Constraints": [[7, "3.-Use-Models-with-Functional-Form-Constraints"]], "4. Compare Predictions for Sub-Populations": [[7, "4.-Compare-Predictions-for-Sub-Populations"]], "5. \u201cNo\u201d is Always An Option": [[7, "5.-%22No%22-is-Always-An-Option"]], "Tools for Interpretable / Constrained Machine Learning": [[7, "Tools-for-Interpretable-/-Constrained-Machine-Learning"]], "Interpretable Models": [[7, "Interpretable-Models"]], "Tools for Neural Networks": [[7, "Tools-for-Neural-Networks"]], "Constrained Models": [[7, "Constrained-Models"]], "Tools for Introspection": [[7, "Tools-for-Introspection"]], "Evaluating Real Studies": [[8, "Evaluating-Real-Studies"]], "The Sun Is Bright for Dogs Too": [[8, "The-Sun-Is-Bright-for-Dogs-Too"]], "Evaluating Baseline Differences": [[8, "Evaluating-Baseline-Differences"]], "Why we need all three": [[8, "Why-we-need-all-three"]], "Evaulating Differential Treatment Effects": [[8, "Evaulating-Differential-Treatment-Effects"]], "What Comes Next?": [[8, "What-Comes-Next?"]], "Well this has been unsatisfying": [[8, "Well-this-has-been-unsatisfying"]], "Discussion of Descriptive Exercises": [[9, "Discussion-of-Descriptive-Exercises"]], "Exercise 8": [[9, "Exercise-8"], [15, "Exercise-8"], [20, "Exercise-8"], [21, "Exercise-8"], [22, "Exercise-8"], [24, "Exercise-8"], [25, "Exercise-8"], [28, "Exercise-8"], [36, "Exercise-8"], [37, "Exercise-8"], [38, "Exercise-8"], [40, "Exercise-8"], [41, "Exercise-8"], [29, "Exercise-8"], [11, "Exercise-8"]], "Takeaways": [[9, "Takeaways"]], "Absolutely positively need the solutions?": [[9, "Absolutely-positively-need-the-solutions?"], [20, "Absolutely-positively-need-the-solutions?"], [22, "Absolutely-positively-need-the-solutions?"], [24, "Absolutely-positively-need-the-solutions?"], [37, "Absolutely-positively-need-the-solutions?"]], "Interpreting Causal Effects of Gender": [[10, "Interpreting-Causal-Effects-of-Gender"]], "Counter-Factuals and Experimental Ideals": [[12, "Counter-Factuals-and-Experimental-Ideals"]], "What is a Counterfactual?": [[12, "What-is-a-Counterfactual?"]], "What is a randomized experiment?": [[12, "What-is-a-randomized-experiment?"]], "Identifying Counter-Factuals": [[12, "Identifying-Counter-Factuals"]], "Exercise 1:": [[12, "Exercise-1:"], [28, "Exercise-1:"], [40, "Exercise-1:"]], "Exercise 2:": [[12, "Exercise-2:"], [17, "Exercise-2:"], [28, "Exercise-2:"], [40, "Exercise-2:"]], "Exercise 3:": [[12, "Exercise-3:"]], "Exercise 1": [[13, "Exercise-1"], [15, "Exercise-1"], [17, "Exercise-1"], [20, "Exercise-1"], [21, "Exercise-1"], [22, "Exercise-1"], [24, "Exercise-1"], [25, "Exercise-1"], [30, "Exercise-1"], [31, "Exercise-1"], [33, "Exercise-1"], [34, "Exercise-1"], [36, "Exercise-1"], [37, "Exercise-1"], [38, "Exercise-1"], [41, "Exercise-1"], [29, "Exercise-1"], [11, "Exercise-1"]], "Exercise 2": [[13, "Exercise-2"], [15, "Exercise-2"], [20, "Exercise-2"], [21, "Exercise-2"], [22, "Exercise-2"], [24, "Exercise-2"], [25, "Exercise-2"], [30, "Exercise-2"], [31, "Exercise-2"], [33, "Exercise-2"], [33, "id1"], [34, "Exercise-2"], [36, "Exercise-2"], [37, "Exercise-2"], [38, "Exercise-2"], [41, "Exercise-2"], [29, "Exercise-2"], [11, "Exercise-2"]], "Exercise 3": [[13, "Exercise-3"], [15, "Exercise-3"], [20, "Exercise-3"], [21, "Exercise-3"], [22, "Exercise-3"], [24, "Exercise-3"], [25, "Exercise-3"], [28, "Exercise-3"], [30, "Exercise-3"], [33, "Exercise-3"], [34, "Exercise-3"], [36, "Exercise-3"], [37, "Exercise-3"], [38, "Exercise-3"], [40, "Exercise-3"], [41, "Exercise-3"], [29, "Exercise-3"], [11, "Exercise-3"]], "Exercise 4": [[13, "Exercise-4"], [15, "Exercise-4"], [20, "Exercise-4"], [21, "Exercise-4"], [22, "Exercise-4"], [24, "Exercise-4"], [25, "Exercise-4"], [28, "Exercise-4"], [33, "Exercise-4"], [34, "Exercise-4"], [36, "Exercise-4"], [37, "Exercise-4"], [38, "Exercise-4"], [40, "Exercise-4"], [41, "Exercise-4"], [29, "Exercise-4"], [11, "Exercise-4"]], "Exercise 5": [[13, "Exercise-5"], [15, "Exercise-5"], [20, "Exercise-5"], [21, "Exercise-5"], [22, "Exercise-5"], [24, "Exercise-5"], [25, "Exercise-5"], [28, "Exercise-5"], [33, "Exercise-5"], [34, "Exercise-5"], [36, "Exercise-5"], [37, "Exercise-5"], [38, "Exercise-5"], [40, "Exercise-5"], [41, "Exercise-5"], [29, "Exercise-5"], [11, "Exercise-5"]], "Exercise 6": [[13, "Exercise-6"], [15, "Exercise-6"], [20, "Exercise-6"], [21, "Exercise-6"], [22, "Exercise-6"], [24, "Exercise-6"], [24, "id1"], [25, "Exercise-6"], [28, "Exercise-6"], [33, "Exercise-6"], [34, "Exercise-6"], [36, "Exercise-6"], [37, "Exercise-6"], [38, "Exercise-6"], [40, "Exercise-6"], [41, "Exercise-6"], [29, "Exercise-6"], [11, "Exercise-6"]], "Exercise 7": [[13, "Exercise-7"], [15, "Exercise-7"], [20, "Exercise-7"], [21, "Exercise-7"], [22, "Exercise-7"], [24, "Exercise-7"], [25, "Exercise-7"], [28, "Exercise-7"], [33, "Exercise-7"], [34, "Exercise-7"], [36, "Exercise-7"], [37, "Exercise-7"], [38, "Exercise-7"], [40, "Exercise-7"], [41, "Exercise-7"], [29, "Exercise-7"], [11, "Exercise-7"]], "Marijuana Legalization and Violent Crime": [[13, "Marijuana-Legalization-and-Violent-Crime"]], "Evaluating Studies": [[14, "Evaluating-Studies"]], "Study 1: Gym Memberships": [[14, "Study-1:-Gym-Memberships"]], "Study 2: Car Insurance": [[14, "Study-2:-Car-Insurance"]], "Study 3: Billboard": [[14, "Study-3:-Billboard"]], "Crime and Policing Expenditures Exploratory Questions": [[15, "Crime-and-Policing-Expenditures-Exploratory-Questions"]], "Exercises": [[15, "Exercises"]], "After you have answered\u2026": [[15, "After-you-have-answered..."], [33, "After-you-have-answered..."]], "Your First Stakeholder": [[16, "Your-First-Stakeholder"]], "The Proposal": [[16, "The-Proposal"]], "Your Task": [[16, "Your-Task"], [30, "Your-Task"]], "Converting Stakeholder Prompts into Actionable Questions": [[17, "Converting-Stakeholder-Prompts-into-Actionable-Questions"]], "Causality in the Time of Cholera": [[18, "Causality-in-the-Time-of-Cholera"], [19, "Causality-in-the-Time-of-Cholera"]], "Question 1:": [[18, "Question-1:"], [19, "Question-1:"]], "Question 2:": [[18, "Question-2:"], [19, "Question-2:"]], "Question 3:": [[18, "Question-3:"], [19, "Question-3:"]], "Question 4": [[18, "Question-4"], [19, "Question-4"]], "Question 5": [[18, "Question-5"], [19, "Question-5"]], "Question 6": [[18, "Question-6"], [19, "Question-6"]], "Question 7:": [[18, "Question-7:"], [19, "Question-7:"]], "Question 8:": [[18, "Question-8:"], [19, "Question-8:"]], "Question 9:": [[18, "Question-9:"], [19, "Question-9:"]], "Interpreting Indicator Variables": [[20, "Interpreting-Indicator-Variables"], [36, "Interpreting-Indicator-Variables"]], "Indicator Variables and Omitted Variable Bias": [[20, "Indicator-Variables-and-Omitted-Variable-Bias"], [36, "Indicator-Variables-and-Omitted-Variable-Bias"]], "Interaction Effects": [[20, "Interaction-Effects"], [36, "Interaction-Effects"]], "Exercise 9": [[21, "Exercise-9"], [22, "Exercise-9"], [25, "Exercise-9"], [28, "Exercise-9"], [37, "Exercise-9"], [38, "Exercise-9"], [40, "Exercise-9"], [41, "Exercise-9"], [29, "Exercise-9"], [11, "Exercise-9"]], "Exercise 10": [[21, "Exercise-10"], [22, "Exercise-10"], [25, "Exercise-10"], [28, "Exercise-10"], [37, "Exercise-10"], [40, "Exercise-10"], [41, "Exercise-10"], [29, "Exercise-10"], [11, "Exercise-10"]], "Exercise 11": [[21, "Exercise-11"], [22, "Exercise-11"], [25, "Exercise-11"], [28, "Exercise-11"], [37, "Exercise-11"], [40, "Exercise-11"], [41, "Exercise-11"], [29, "Exercise-11"], [11, "Exercise-11"]], "Exercise 12": [[21, "Exercise-12"], [22, "Exercise-12"], [25, "Exercise-12"], [28, "Exercise-12"], [37, "Exercise-12"], [40, "Exercise-12"], [41, "Exercise-12"], [29, "Exercise-12"], [11, "Exercise-12"]], "Exercise 13": [[21, "Exercise-13"], [22, "Exercise-13"], [25, "Exercise-13"], [28, "Exercise-13"], [37, "Exercise-13"], [40, "Exercise-13"], [29, "Exercise-13"], [11, "Exercise-13"]], "Exercise 14": [[21, "Exercise-14"], [22, "Exercise-14"], [25, "Exercise-14"], [28, "Exercise-14"], [37, "Exercise-14"], [40, "Exercise-14"], [11, "Exercise-14"]], "Interpretable Modelling of Credit Risk": [[21, "Interpretable-Modelling-of-Credit-Risk"]], "Data Prep": [[21, "Data-Prep"]], "Model Fitting": [[21, "Model-Fitting"]], "Let\u2019s See This Interpretability!": [[21, "Let's-See-This-Interpretability!"]], "Exercise 15": [[21, "Exercise-15"], [22, "Exercise-15"], [25, "Exercise-15"], [37, "Exercise-15"], [11, "Exercise-15"]], "Exercise 16": [[21, "Exercise-16"], [22, "Exercise-16"], [25, "Exercise-16"], [37, "Exercise-16"]], "Exercise 17": [[21, "Exercise-17"], [25, "Exercise-17"]], "Matching Exercise": [[22, "Matching-Exercise"], [37, "Matching-Exercise"]], "Matching Packages: Python v. R": [[22, "Matching-Packages:-Python-v.-R"], [37, "Matching-Packages:-Python-v.-R"]], "Installing dame-flame.": [[22, "Installing-dame-flame."], [37, "Installing-dame-flame."]], "Data Setup": [[22, "Data-Setup"], [37, "Data-Setup"]], "Getting To Know Your Data": [[22, "Getting-To-Know-Your-Data"], [37, "Getting-To-Know-Your-Data"]], "Matching!": [[22, "Matching!"], [37, "Matching!"]], "Let\u2019s Do Matching with DAME": [[22, "Let's-Do-Matching-with-DAME"], [37, "Let's-Do-Matching-with-DAME"]], "Interpreting DAME output": [[22, "Interpreting-DAME-output"], [37, "Interpreting-DAME-output"]], "Getting Back a Dataset": [[22, "Getting-Back-a-Dataset"], [37, "Getting-Back-a-Dataset"]], "Check Your Matches and Analyze": [[22, "Check-Your-Matches-and-Analyze"], [37, "Check-Your-Matches-and-Analyze"]], "Other Forms of Matching": [[22, "Other-Forms-of-Matching"], [37, "Other-Forms-of-Matching"]], "Optimal AB Testing Design": [[23, "Optimal-AB-Testing-Design"]], "The Experiment": [[23, "The-Experiment"]], "Question 2": [[23, "Question-2"]], "Question 3": [[23, "Question-3"]], "Traffic Death Analysis": [[24, "Traffic-Death-Analysis"], [38, "Traffic-Death-Analysis"]], "Fixed Effects by Demeaning": [[24, "Fixed-Effects-by-Demeaning"], [38, "Fixed-Effects-by-Demeaning"]], "Predicting Mortgage Delinquency Risk": [[25, "Predicting-Mortgage-Delinquency-Risk"]], "Gradescope Autograding": [[25, "Gradescope-Autograding"], [29, "Gradescope-Autograding"], [11, "Gradescope-Autograding"]], "Submission Limits": [[25, "Submission-Limits"], [29, "Submission-Limits"], [11, "Submission-Limits"]], "Data Cleaning and Organization": [[25, "Data-Cleaning-and-Organization"]], "Modelling Delinquency Risk": [[25, "Modelling-Delinquency-Risk"]], "Now To The Future": [[25, "Now-To-The-Future"]], "Exercise 18": [[25, "Exercise-18"]], "Making Potential Outcomes Concrete": [[26, "Making-Potential-Outcomes-Concrete"], [39, "Making-Potential-Outcomes-Concrete"]], "The Study": [[26, "The-Study"], [27, "The-Study"], [39, "The-Study"]], "Mapping to the Potential Outcomes Framework": [[26, "Mapping-to-the-Potential-Outcomes-Framework"], [27, "Mapping-to-the-Potential-Outcomes-Framework"], [39, "Mapping-to-the-Potential-Outcomes-Framework"]], "Observability": [[26, "Observability"], [27, "Observability"], [39, "Observability"]], "Causal Inference": [[26, "Causal-Inference"], [27, "Causal-Inference"], [39, "Causal-Inference"]], "Making Potential Outcomes Concrete 2": [[27, "Making-Potential-Outcomes-Concrete-2"]], "Estimating Gender Discrimination in the Workplace": [[28, "Estimating-Gender-Discrimination-in-the-Workplace"], [40, "Estimating-Gender-Discrimination-in-the-Workplace"]], "Soliciting Information From Your Stakeholder": [[30, "Soliciting-Information-From-Your-Stakeholder"]], "Stakeholder Project Proposal": [[30, "Stakeholder-Project-Proposal"]], "A Data Analysis of Voting Access on College Campuses": [[30, "A-Data-Analysis-of-Voting-Access-on-College-Campuses"]], "Background": [[30, "Background"], [61, "Background"]], "Objectives": [[30, "Objectives"]], "Our Approach": [[30, "Our-Approach"]], "Generating and Classifying Questions": [[31, "Generating-and-Classifying-Questions"]], "Exploratory Question:": [[31, "Exploratory-Question:"], [31, "id1"]], "Passive-Predictive Question:": [[31, "Passive-Predictive-Question:"], [31, "id2"]], "Causal Question:": [[31, "Causal-Question:"], [31, "id3"]], "Groupby and Arrest Data": [[32, "Groupby-and-Arrest-Data"], [34, "Groupby-and-Arrest-Data"]], "Learning the Group Structure of Your Data": [[32, "Learning-the-Group-Structure-of-Your-Data"]], "Testing Your Assumptions": [[32, "Testing-Your-Assumptions"]], "Crime and Poverty Descriptive Analysis": [[33, "Crime-and-Poverty-Descriptive-Analysis"]], "Acme Corportation Solution": [[35, "Acme-Corportation-Solution"]], "Resume Experiment Analysis": [[41, "Resume-Experiment-Analysis"], [29, "Resume-Experiment-Analysis"]], "Checking for Balance": [[41, "Checking-for-Balance"], [29, "Checking-for-Balance"]], "Estimating Effect of Race": [[41, "Estimating-Effect-of-Race"], [29, "Estimating-Effect-of-Race"]], "Estimating Heterogeneous Effects": [[41, "Estimating-Heterogeneous-Effects"], [29, "Estimating-Heterogeneous-Effects"]], "What Did We Just Measure?": [[41, "What-Did-We-Just-Measure?"], [29, "What-Did-We-Just-Measure?"]], "Fixed Effects: Indicator Variables for Groups": [[42, "Fixed-Effects:-Indicator-Variables-for-Groups"]], "Implementing Fixed Effects": [[42, "Implementing-Fixed-Effects"]], "Clustering": [[42, "Clustering"]], "Computationally Efficient Fixed Effects": [[42, "Computationally-Efficient-Fixed-Effects"]], "Fixed Effects and Causal Inference": [[43, "Fixed-Effects-and-Causal-Inference"]], "Fixed Effects and Hierarchical Models": [[44, "Fixed-Effects-and-Hierarchical-Models"]], "Fixed Effects & Clustered Standard Errors": [[44, "Fixed-Effects-&-Clustered-Standard-Errors"]], "Things To Remember:": [[44, "Things-To-Remember:"]], "How to Read (Academic Edition)": [[45, "How-to-Read-(Academic-Edition)"]], "Read Actively": [[45, "Read-Actively"]], "Be Patient with Examples From Different Domains": [[45, "Be-Patient-with-Examples-From-Different-Domains"]], "Do NOT Summarize with LLMs": [[45, "Do-NOT-Summarize-with-LLMs"]], "Welcome to Solving Problems with Data!": [[46, "welcome-to-solving-problems-with-data"]], "Pre-Requisites for Non-MIDS Students": [[46, "pre-requisites-for-non-mids-students"]], "Syllabus": [[46, "syllabus"]], "Internal and External Validity": [[47, "Internal-and-External-Validity"]], "Internal Validity": [[47, "Internal-Validity"]], "External Validity": [[47, "External-Validity"]], "External Validity Considerations": [[47, "External-Validity-Considerations"]], "Trade-Offs Between Internal and External Validity": [[47, "Trade-Offs-Between-Internal-and-External-Validity"]], "Conclusion": [[47, "Conclusion"], [59, "Conclusion"]], "Using and Interpreting Indicator (Dummy) Variables": [[48, "Using-and-Interpreting-Indicator-(Dummy)-Variables"]], "What are indicator variables?": [[48, "What-are-indicator-variables?"]], "The ONE thing that you must understand when using indicator variables:": [[48, "The-ONE-thing-that-you-must-understand-when-using-indicator-variables:"]], "Indicator Variables with Two Category Variable": [[48, "Indicator-Variables-with-Two-Category-Variable"]], "Indicator Variables for variables with more than 2 categories": [[48, "Indicator-Variables-for-variables-with-more-than-2-categories"]], "Interactions with Constant Variables": [[48, "Interactions-with-Constant-Variables"]], "Keeping Things Straight": [[48, "Keeping-Things-Straight"]], "The * operator": [[48, "The-*-operator"]], "Interactions Between Multiple Indicator Variables": [[48, "Interactions-Between-Multiple-Indicator-Variables"]], "Fixed Effects": [[48, "Fixed-Effects"]], "Limitations of Experiments (and Average Treatment Effects)": [[49, "Limitations-of-Experiments-(and-Average-Treatment-Effects)"]], "Who\u2019s Average?": [[49, "Who's-Average?"]], "The Fine Print of ATE": [[49, "The-Fine-Print-of-ATE"]], "SUTVA": [[49, "SUTVA"]], "How Matching is Done (A Summary)": [[50, "How-Matching-is-Done-(A-Summary)"]], "Pruning Your Data": [[50, "Pruning-Your-Data"]], "Measuring Similarity": [[50, "Measuring-Similarity"]], "When Can / Should I Use Matching?": [[50, "When-Can-/-Should-I-Use-Matching?"]], "Checking Balance": [[50, "Checking-Balance"]], "Analyze!": [[50, "Analyze!"]], "Specific Models": [[50, "Specific-Models"]], "Matching": [[51, "Matching"]], "Model Dependency": [[51, "Model-Dependency"]], "Model Dependency, Causal Inference, and Imbalance": [[51, "Model-Dependency,-Causal-Inference,-and-Imbalance"]], "Matching: Better thought of as pruning": [[51, "Matching:-Better-thought-of-as-pruning"]], "The Cost of Matching": [[51, "The-Cost-of-Matching"]], "Causal Inference Assumptions": [[51, "Causal-Inference-Assumptions"]], "In Summary": [[51, "In-Summary"]], "Converting Stakeholder Prompts into Questions": [[52, "Converting-Stakeholder-Prompts-into-Questions"]], "What\u2019s The Real Problem / Need": [[52, "What's-The-Real-Problem-/-Need"]], "What Questions Do I Need To Ask?": [[52, "What-Questions-Do-I-Need-To-Ask?"]], "Make Your Questions Specific and Actionable": [[52, "Make-Your-Questions-Specific-and-Actionable"]], "Many Questions?": [[52, "Many-Questions?"]], "What to Read Next": [[52, "What-to-Read-Next"]], "Nick\u2019s Things to Think About Before A/B Testing": [[53, "Nick's-Things-to-Think-About-Before-A/B-Testing"]], "Step 0: Are You Sure You Should Do This?": [[53, "Step-0:-Are-You-Sure-You-Should-Do-This?"]], "Step 1: Define Your Goals": [[53, "Step-1:-Define-Your-Goals"]], "Step 2: Plan Implementation": [[53, "Step-2:-Plan-Implementation"]], "Step 3: Run!": [[53, "Step-3:-Run!"]], "Step 4: Check Internal Validity": [[53, "Step-4:-Check-Internal-Validity"]], "Step 5: Interpret Results": [[53, "Step-5:-Interpret-Results"]], "Step 6: Consider External Validity": [[53, "Step-6:-Consider-External-Validity"]], "Reading Reflections": [[56, "Reading-Reflections"]], "Solutions": [[57, "Solutions"]], "Looking for the solutions?": [[58, "Looking-for-the-solutions?"]], "OK, if you are still interested in looking at the solutions\u2026": [[58, "OK,-if-you-are-still-interested-in-looking-at-the-solutions..."]], "Taxonomy of Questions": [[59, "Taxonomy-of-Questions"]], "Positive versus Normative Questions": [[59, "Positive-versus-Normative-Questions"]], "Descriptive Questions": [[59, "Descriptive-Questions"]], "Descriptive Example 1: Nope, This Time Isn\u2019t Different": [[59, "Descriptive-Example-1:-Nope,-This-Time-Isn't-Different"]], "Descriptive Example 3: Disease Surveillance": [[59, "Descriptive-Example-3:-Disease-Surveillance"]], "Descriptive Example 4: Global Warming": [[59, "Descriptive-Example-4:-Global-Warming"]], "Descriptive Questions and EDA": [[59, "Descriptive-Questions-and-EDA"]], "Last Thoughts": [[59, "Last-Thoughts"]], "Causal Questions and Prediction-with-Manipulation": [[59, "Causal-Questions-and-Prediction-with-Manipulation"]], "Why Causal Inference is Hard": [[59, "Why-Causal-Inference-is-Hard"]], "Causal Inference, Manipulations, and Prediction": [[59, "Causal-Inference,-Manipulations,-and-Prediction"]], "Classification Questions / Prediction-Without-Manipulation": [[59, "Classification-Questions-/-Prediction-Without-Manipulation"]], "\u201cWithout Manipulation\u201d?": [[59, "%22Without-Manipulation%22?"]], "Using Exploratory Questions to Better Understanding Your Problem": [[60, "Using-Exploratory-Questions-to-Better-Understanding-Your-Problem"]], "Your First Assignment": [[60, "Your-First-Assignment"]], "Part 2: Choosing a Problem": [[60, "Part-2:-Choosing-a-Problem"]], "Part 3: Specifying and Answering Questions": [[60, "Part-3:-Specifying-and-Answering-Questions"]], "Data": [[60, "Data"]], "Writing Data Science Report for Non-Technical Audiences": [[61, "Writing-Data-Science-Report-for-Non-Technical-Audiences"]], "Identify your audience": [[61, "Identify-your-audience"]], "Introduction / Executive Summary": [[61, "Introduction-/-Executive-Summary"]], "Your Design": [[61, "Your-Design"]], "Your Results": [[61, "Your-Results"]], "Conclusions": [[61, "Conclusions"]], "Appendix": [[61, "Appendix"]], "A/B Testing the Udacity Website": [[11, "A/B-Testing-the-Udacity-Website"]], "Udacity\u2019s Test": [[11, "Udacity's-Test"]], "Import the Data": [[11, "Import-the-Data"]], "Pick your measures": [[11, "Pick-your-measures"]], "Validating The Data": [[11, "Validating-The-Data"]], "Estimating the Effect of Experiment": [[11, "Estimating-the-Effect-of-Experiment"]], "Where To Find Public Data": [[54, "Where-To-Find-Public-Data"]], "Good Starting Points": [[54, "Good-Starting-Points"]], "Other Interesting Data Sources": [[54, "Other-Interesting-Data-Sources"]], "Data with a Spatial/GIS Component": [[54, "Data-with-a-Spatial/GIS-Component"]], "Public Satellite Data": [[54, "Public-Satellite-Data"]], "Other Lists of Data That Are Great": [[54, "Other-Lists-of-Data-That-Are-Great"]], "Class Schedule": [[4, "class-schedule"]]}, "indexentries": {}})
\ No newline at end of file
+Search.setIndex({"docnames": ["DRAFT_ml_bias", "UDS_midterm_part2_2022", "backwards_design", "causal_inference_beyond_ab_testing", "class_schedule", "descriptive_questions", "endogenous_stopping", "ethical_ml_recommendations", "evaluating_real_studies", "exercises/discussion_exploratory", "exercises/discussion_regressions_incomeineq", "exercises/exercise_abtesting", "exercises/exercise_counterfactuals", "exercises/exercise_diffindiff", "exercises/exercise_evaluating_studies", "exercises/exercise_exploratory", "exercises/exercise_first_class_capstone_proposal", "exercises/exercise_generating_questions", "exercises/exercise_ghostmap_1", "exercises/exercise_ghostmap_2", "exercises/exercise_indicators", "exercises/exercise_interpretable", "exercises/exercise_matching", "exercises/exercise_optimalABthresholds", "exercises/exercise_panel", "exercises/exercise_passive_prediction", "exercises/exercise_potential_outcomes1", "exercises/exercise_potential_outcomes2", "exercises/exercise_power_calculations", "exercises/exercise_regressions_incomeineq", "exercises/exercise_resume_experiment", "exercises/exercise_stakeholder_management", "exercises/exercise_taxonomy_of_questions", "exercises/setup_diffindiff", "exercises/solutions_descriptive", "exercises/solutions_diffindiff", "exercises/solutions_first_class_capstone_proposal", "exercises/solutions_indicators", "exercises/solutions_matching", "exercises/solutions_panel", "exercises/solutions_potential_outcomes1", "exercises/solutions_regressions_incomeineq", "exercises/solutions_resume", "fixed_effects", "fixed_effects_and_causal_inference", "fixed_effects_v_hierarchical", "how_to_read", "index", "internal_v_external_validity", "interpreting_indicator_vars", "limitations_of_ATE", "matching_how", "matching_why", "moving_from_problems_to_questions", "nick_ab_testing_checklist", "public_data", "pvalues_and_decision_making", "reading_reflections", "solutions", "solutions_warning", "taxonomy_of_questions", "team_assignments/UDS_TeamProject_Exploratory", "writing_to_stakeholders"], "filenames": ["DRAFT_ml_bias.ipynb", "UDS_midterm_part2_2022.ipynb", "backwards_design.ipynb", "causal_inference_beyond_ab_testing.ipynb", "class_schedule.rst", "descriptive_questions.ipynb", "endogenous_stopping.ipynb", "ethical_ml_recommendations.ipynb", "evaluating_real_studies.ipynb", "exercises/discussion_exploratory.ipynb", "exercises/discussion_regressions_incomeineq.ipynb", "exercises/exercise_abtesting.ipynb", "exercises/exercise_counterfactuals.ipynb", "exercises/exercise_diffindiff.ipynb", "exercises/exercise_evaluating_studies.ipynb", "exercises/exercise_exploratory.ipynb", "exercises/exercise_first_class_capstone_proposal.ipynb", "exercises/exercise_generating_questions.ipynb", "exercises/exercise_ghostmap_1.ipynb", "exercises/exercise_ghostmap_2.ipynb", "exercises/exercise_indicators.ipynb", "exercises/exercise_interpretable.ipynb", "exercises/exercise_matching.ipynb", "exercises/exercise_optimalABthresholds.ipynb", "exercises/exercise_panel.ipynb", "exercises/exercise_passive_prediction.ipynb", "exercises/exercise_potential_outcomes1.ipynb", "exercises/exercise_potential_outcomes2.ipynb", "exercises/exercise_power_calculations.ipynb", "exercises/exercise_regressions_incomeineq.ipynb", "exercises/exercise_resume_experiment.ipynb", "exercises/exercise_stakeholder_management.ipynb", "exercises/exercise_taxonomy_of_questions.ipynb", "exercises/setup_diffindiff.ipynb", "exercises/solutions_descriptive.ipynb", "exercises/solutions_diffindiff.ipynb", "exercises/solutions_first_class_capstone_proposal.ipynb", "exercises/solutions_indicators.ipynb", "exercises/solutions_matching.ipynb", "exercises/solutions_panel.ipynb", "exercises/solutions_potential_outcomes1.ipynb", "exercises/solutions_regressions_incomeineq.ipynb", "exercises/solutions_resume.ipynb", "fixed_effects.ipynb", "fixed_effects_and_causal_inference.ipynb", "fixed_effects_v_hierarchical.ipynb", "how_to_read.ipynb", "index.rst", "internal_v_external_validity.ipynb", "interpreting_indicator_vars.ipynb", "limitations_of_ATE.ipynb", "matching_how.ipynb", "matching_why.ipynb", "moving_from_problems_to_questions.ipynb", "nick_ab_testing_checklist.ipynb", "public_data.ipynb", "pvalues_and_decision_making.ipynb", "reading_reflections.ipynb", "solutions.ipynb", "solutions_warning.ipynb", "taxonomy_of_questions.ipynb", "team_assignments/UDS_TeamProject_Exploratory.ipynb", "writing_to_stakeholders.ipynb"], "titles": ["Backwards Design", "UDS Midterm 2022, Part 2", "Backwards Design", "Beyond The Experiment", "Class Schedule", "Researcher Discretion in Descriptive Analysis", "Peeking / Endogenous Stopping", "Recommendations for Responsible Machine Learning", "Evaluating Real Studies", "Discussion of Descriptive Exercises", "Interpreting Causal Effects of Gender", "A/B Testing the Udacity Website", "Counter-Factuals and Experimental Ideals", "Marijuana Legalization and Violent Crime", "Evaluating Studies", "Crime and Policing Expenditures Exploratory Questions", "Your First Stakeholder", "Converting Stakeholder Prompts into Actionable Questions", "Causality in the Time of Cholera", "Causality in the Time of Cholera", "Interpreting Indicator Variables", "Interpretable Modelling of Credit Risk", "Matching Exercise", "Optimal AB Testing Design", "Traffic Death Analysis", "Predicting Mortgage Delinquency Risk", "Making Potential Outcomes Concrete", "Making Potential Outcomes Concrete 2", "Power Calculations and Experiment Planning", "Estimating Gender Discrimination in the Workplace", "Resume Experiment Analysis", "Soliciting Information From Your Stakeholder", "Generating and Classifying Questions", "Groupby and Arrest Data", "Crime and Poverty Descriptive Analysis", "Groupby and Arrest Data", "Acme Corportation Solution", "Interpreting Indicator Variables", "Matching Exercise", "Traffic Death Analysis", "Making Potential Outcomes Concrete", "Estimating Gender Discrimination in the Workplace", "Resume Experiment Analysis", "Fixed Effects: Indicator Variables for Groups", "Fixed Effects and Causal Inference", "Fixed Effects and Hierarchical Models", "How to Read (Academic Edition)", "Welcome to Solving Problems with Data!", "Internal and External Validity", "Using and Interpreting Indicator (Dummy) Variables", "Limitations of Experiments (and Average Treatment Effects)", "How Matching is Done (A Summary)", "Matching", "Converting Stakeholder Prompts into Questions", "Nick\u2019s Things to Think About Before A/B Testing", "Where To Find Public Data", "<no title>", "Reading Reflections", "Solutions", "Looking for the solutions?", "Taxonomy of Questions", "Using Exploratory Questions to Better Understanding Your Problem", "Writing Data Science Report for Non-Technical Audiences"], "terms": {"i": [0, 1, 3, 5, 6, 9, 10, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 54, 55, 57, 59, 61, 62], "wai": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 16, 22, 24, 25, 26, 27, 28, 30, 31, 33, 36, 38, 39, 40, 42, 43, 44, 46, 48, 49, 50, 51, 52, 53, 54, 55, 59, 60, 62], "develop": [0, 2, 5, 7, 17, 18, 19, 21, 22, 28, 38, 41, 46, 47, 48, 50, 53, 57, 60], "an": [0, 1, 3, 5, 6, 8, 9, 11, 12, 13, 14, 15, 16, 17, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 53, 54, 57, 59, 60, 61, 62], "effici": [0, 2, 3, 7, 22, 24, 29, 30, 32, 38, 39, 41, 45, 46], "strategi": [0, 2, 11, 16, 24, 39, 45, 50, 51, 53, 54, 62], "complet": [0, 2, 11, 16, 18, 19, 21, 22, 25, 30, 38, 42, 52, 53, 62], "new": [0, 2, 3, 5, 7, 8, 11, 13, 14, 20, 21, 22, 23, 25, 28, 31, 35, 36, 37, 38, 41, 44, 46, 48, 49, 50, 54, 60, 62], "data": [0, 3, 4, 5, 6, 8, 9, 13, 14, 15, 16, 17, 18, 20, 23, 24, 26, 27, 28, 29, 30, 34, 36, 37, 39, 40, 41, 42, 43, 44, 45, 46, 48, 49, 50, 52, 53, 54, 60], "scienc": [0, 2, 3, 4, 5, 7, 13, 16, 18, 26, 27, 28, 31, 35, 40, 46, 47, 50, 53, 55, 60], "project": [0, 2, 3, 4, 11, 16, 21, 26, 27, 36, 40, 47, 55, 57, 61, 62], "my": [0, 2, 4, 5, 7, 21, 22, 25, 33, 37, 38, 42, 48, 49, 50, 52, 53, 60], "view": [0, 2, 7, 8, 11, 23, 49], "one": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 13, 15, 16, 21, 22, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 57, 60, 61, 62], "most": [0, 2, 5, 7, 13, 16, 21, 22, 28, 30, 31, 33, 35, 38, 42, 44, 45, 47, 48, 49, 50, 51, 52, 53, 55, 57, 60, 62], "import": [0, 1, 3, 4, 5, 7, 13, 16, 21, 22, 24, 28, 29, 33, 34, 35, 36, 37, 38, 39, 41, 42, 43, 44, 46, 48, 49, 50, 52, 53, 60, 61, 62], "skill": [0, 2, 3, 13, 15, 30, 34, 35, 42, 46, 47, 55, 57, 59, 60, 62], "profession": [0, 2, 3, 41, 47], "scientist": [0, 2, 3, 5, 7, 16, 22, 31, 38, 43, 45, 46, 47, 49, 53, 62], "If": [0, 1, 2, 3, 5, 6, 7, 8, 11, 12, 18, 19, 21, 22, 24, 25, 28, 30, 33, 34, 37, 38, 39, 41, 43, 44, 45, 46, 47, 49, 50, 51, 52, 53, 54, 55], "don": [0, 1, 2, 4, 5, 6, 7, 8, 9, 11, 15, 18, 19, 20, 21, 22, 23, 24, 25, 28, 29, 33, 34, 35, 36, 37, 38, 40, 41, 42, 43, 44, 45, 46, 48, 49, 51, 52, 53, 54, 60, 61, 62], "t": [0, 1, 2, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 48, 49, 50, 51, 52, 53, 54, 55, 59, 61, 62], "have": [0, 1, 3, 5, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 35, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 57, 59, 60, 61, 62], "lot": [0, 2, 3, 5, 6, 7, 8, 10, 11, 14, 21, 22, 28, 29, 38, 40, 41, 43, 45, 46, 48, 50, 52, 53, 54, 55, 60, 61, 62], "experi": [0, 2, 4, 5, 6, 7, 8, 41, 44, 46, 47, 48, 52, 54, 58, 60, 61, 62], "mai": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 14, 16, 20, 21, 22, 24, 25, 26, 27, 28, 29, 30, 32, 36, 37, 38, 39, 40, 41, 42, 43, 45, 46, 48, 49, 50, 52, 53, 54, 55, 57, 59, 60, 61, 62], "obviou": [0, 2, 5, 7, 42, 44, 48, 49], "why": [0, 1, 3, 4, 5, 6, 7, 11, 21, 22, 25, 26, 27, 30, 31, 36, 38, 40, 42, 46, 47, 49, 50, 51, 52, 57, 61, 62], "thi": [0, 1, 2, 3, 5, 6, 7, 9, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 55, 57, 59, 61, 62], "even": [0, 2, 3, 6, 7, 8, 11, 12, 13, 14, 22, 23, 25, 28, 30, 33, 35, 38, 40, 42, 43, 45, 47, 48, 50, 60], "call": [0, 1, 2, 3, 6, 7, 8, 11, 12, 13, 22, 24, 25, 28, 29, 30, 31, 33, 35, 38, 39, 41, 42, 43, 45, 48, 49, 51, 52, 54, 55, 60], "That": [0, 3, 5, 7, 8, 11, 12, 18, 19, 22, 25, 28, 30, 37, 38, 41, 44, 46, 48, 49, 50, 51, 52, 60, 61, 62], "": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 13, 14, 15, 16, 17, 18, 19, 20, 24, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 51, 52, 55, 59, 60, 61, 62], "becaus": [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 21, 22, 24, 26, 27, 28, 29, 30, 33, 37, 38, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 50, 51, 52, 53, 55, 60, 61, 62], "student": [0, 2, 5, 7, 11, 13, 16, 26, 30, 31, 35, 40, 42, 43, 45, 46, 48, 55, 59, 60], "come": [0, 1, 2, 3, 5, 7, 11, 14, 21, 22, 25, 28, 29, 38, 41, 43, 46, 47, 48, 49, 50, 52, 53, 55, 60, 61], "from": [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 50, 51, 52, 53, 54, 55, 57, 60, 61, 62], "classroom": [0, 2, 12, 45, 47], "exercis": [0, 1, 2, 4, 10, 16, 18, 19, 23, 26, 27, 33, 40, 46, 47, 59], "site": [0, 2, 7, 8, 11, 23, 31, 33, 41, 43, 47, 49, 53, 55, 59, 61], "like": [0, 1, 3, 5, 6, 7, 8, 11, 12, 14, 15, 16, 17, 18, 19, 21, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 48, 49, 50, 51, 52, 53, 54, 55, 57, 59, 60, 61, 62], "kaggl": [0, 2, 7, 61], "These": [0, 2, 11, 12, 20, 22, 37, 38, 55, 57], "type": [0, 2, 3, 7, 8, 11, 15, 16, 20, 24, 25, 28, 29, 30, 31, 33, 34, 35, 37, 38, 39, 41, 42, 43, 44, 46, 48, 49, 52, 53, 60, 61, 62], "ar": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 50, 51, 52, 53, 57, 60, 61, 62], "excel": [0, 2, 7, 21], "opportun": [0, 2, 3, 13, 35, 46, 57], "usual": [0, 2, 3, 11, 12, 25, 29, 30, 41, 44, 47, 48, 50, 55, 60, 62], "case": [0, 2, 3, 5, 7, 11, 12, 18, 19, 21, 22, 24, 25, 26, 27, 28, 30, 31, 33, 38, 39, 40, 42, 43, 45, 47, 48, 49, 50, 53, 60, 62], "unbeknownst": [0, 2], "been": [0, 1, 2, 5, 7, 9, 11, 12, 13, 16, 17, 22, 24, 25, 26, 27, 30, 31, 32, 34, 35, 38, 39, 40, 42, 44, 46, 48, 51, 52, 53, 60, 62], "carefulli": [0, 2, 33, 48, 62], "tailor": [0, 2, 62], "clearli": [0, 2, 5, 7, 8, 12, 49, 52, 60], "goal": [0, 2, 3, 5, 7, 11, 16, 22, 26, 27, 28, 38, 40, 46, 51, 57, 59, 61, 62], "thei": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 20, 21, 22, 23, 25, 26, 27, 28, 29, 30, 31, 35, 36, 37, 38, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 55, 59, 60, 62], "set": [0, 2, 5, 7, 8, 9, 11, 13, 14, 21, 22, 24, 25, 28, 30, 33, 35, 38, 39, 42, 45, 46, 47, 48, 49, 50, 51, 60, 61], "clean": [0, 1, 2, 17, 22, 33, 38, 41, 61], "filter": [0, 2, 5], "provid": [0, 1, 2, 3, 5, 7, 8, 11, 16, 21, 25, 28, 31, 33, 42, 46, 47, 50, 53, 55, 59, 60, 61, 62], "onli": [0, 1, 2, 3, 5, 8, 9, 10, 11, 12, 13, 17, 18, 20, 21, 22, 24, 25, 27, 28, 29, 30, 33, 35, 36, 37, 38, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 55, 57, 59, 60], "relev": [0, 2, 3, 22, 28, 32, 38, 46, 54, 62], "variabl": [0, 1, 2, 4, 5, 6, 7, 9, 11, 12, 13, 16, 21, 22, 24, 25, 28, 29, 30, 33, 35, 38, 39, 41, 42, 44, 45, 51, 52, 53, 55, 60, 61], "done": [0, 2, 5, 7, 12, 13, 29, 32, 35, 39, 41, 45, 46, 54, 60, 61, 62], "good": [0, 3, 4, 6, 7, 8, 11, 20, 21, 22, 23, 25, 28, 31, 37, 38, 41, 42, 43, 45, 46, 47, 48, 50, 51, 52, 54, 57, 60, 62], "reason": [0, 2, 3, 5, 7, 8, 9, 10, 12, 14, 21, 22, 23, 25, 26, 27, 28, 29, 30, 31, 36, 38, 40, 41, 42, 43, 45, 46, 48, 50, 52, 54, 57, 60, 62], "instructor": [0, 2, 5, 57], "focus": [0, 1, 2, 15, 29, 30, 34, 41, 48], "attent": [0, 1, 2, 22, 25, 38, 46, 62], "try": [0, 2, 3, 7, 8, 11, 12, 17, 18, 19, 21, 22, 25, 26, 27, 28, 29, 38, 40, 41, 43, 44, 46, 48, 49, 50, 52, 53, 60, 61, 62], "model": [0, 1, 2, 4, 5, 11, 13, 16, 20, 22, 24, 29, 30, 35, 36, 37, 38, 39, 41, 42, 43, 44, 46, 47, 48, 49, 60, 62], "select": [0, 2, 6, 7, 11, 21, 28, 30, 42], "interpret": [0, 1, 2, 5, 13, 16, 17, 23, 24, 26, 29, 30, 35, 39, 40, 41, 42, 47, 48, 50, 62], "But": [0, 2, 3, 5, 6, 7, 8, 10, 13, 15, 21, 22, 26, 30, 33, 34, 35, 36, 37, 38, 40, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 53, 60, 62], "result": [0, 2, 5, 6, 7, 9, 10, 11, 15, 16, 18, 20, 22, 23, 24, 25, 28, 29, 30, 34, 35, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 53, 55, 60], "often": [0, 1, 2, 3, 5, 6, 7, 21, 24, 28, 29, 31, 33, 39, 41, 43, 45, 46, 47, 48, 49, 50, 52, 53, 55, 57, 60, 61, 62], "awai": [0, 2, 9, 23, 37, 51, 53, 60, 62], "impress": [0, 2, 15, 34], "what": [0, 1, 3, 4, 6, 7, 9, 10, 11, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 43, 44, 45, 46, 48, 50, 51, 52, 54, 55, 60, 61, 62], "do": [0, 1, 4, 5, 6, 7, 8, 9, 11, 13, 15, 16, 18, 19, 20, 21, 23, 24, 25, 26, 27, 28, 30, 31, 32, 33, 34, 35, 36, 37, 39, 40, 42, 43, 45, 48, 49, 50, 51, 52, 55, 57, 59, 60, 61, 62], "work": [0, 1, 2, 3, 5, 7, 9, 10, 11, 12, 14, 16, 17, 18, 20, 21, 22, 24, 27, 28, 29, 31, 33, 37, 38, 39, 41, 43, 44, 45, 46, 47, 48, 49, 50, 51, 53, 54, 55, 57, 59, 60, 61, 62], "statist": [0, 1, 2, 5, 6, 7, 9, 11, 12, 15, 16, 20, 22, 23, 25, 28, 30, 34, 35, 37, 38, 39, 41, 42, 43, 45, 47, 49, 50, 51, 52, 53, 54, 60, 62], "In": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 53, 57, 60, 62], "realiti": [0, 2, 7, 48, 49, 51], "howev": [0, 1, 2, 6, 7, 11, 18, 21, 22, 24, 25, 33, 38, 39, 43, 44, 45, 47, 49, 50, 51, 60], "thing": [0, 2, 3, 5, 7, 8, 9, 10, 11, 14, 17, 18, 19, 21, 22, 23, 26, 27, 29, 33, 38, 40, 41, 42, 43, 44, 46, 47, 48, 50, 52, 53, 55, 57, 60, 62], "doe": [0, 1, 2, 3, 5, 7, 10, 11, 12, 13, 15, 21, 22, 23, 24, 28, 29, 30, 31, 33, 34, 35, 37, 38, 39, 41, 42, 43, 44, 45, 46, 49, 50, 52, 53, 54, 60], "articul": [0, 2], "concret": [0, 2, 7, 8, 16, 28, 31, 41, 49, 62], "feasibl": [0, 2, 18, 19, 22, 38, 50, 54], "object": [0, 2, 3, 5, 8, 11, 22, 33, 34, 38], "b": [0, 2, 4, 5, 6, 7, 12, 13, 23, 24, 28, 30, 32, 33, 35, 39, 45, 47, 50, 57], "achiev": [0, 2, 3, 11, 31, 54], "And": [0, 2, 3, 5, 7, 8, 9, 12, 22, 24, 25, 28, 29, 30, 36, 38, 39, 41, 42, 43, 44, 46, 47, 48, 49, 50, 52, 53, 59, 60, 62], "best": [0, 2, 3, 5, 7, 8, 9, 12, 16, 22, 25, 28, 31, 38, 46, 48, 50, 51, 52, 53, 55, 59, 60], "go": [0, 1, 2, 5, 7, 8, 9, 10, 13, 15, 18, 19, 20, 21, 22, 24, 25, 28, 30, 31, 34, 35, 37, 38, 39, 42, 45, 46, 48, 49, 51, 55], "about": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 55, 57, 60, 61, 62], "accomplish": [0, 2, 30, 45], "both": [0, 1, 2, 3, 5, 7, 8, 10, 13, 15, 16, 18, 19, 22, 24, 25, 26, 27, 28, 30, 31, 32, 33, 34, 35, 38, 39, 40, 42, 43, 45, 48, 49, 50, 51, 52, 54, 55, 60, 61, 62], "As": [0, 2, 3, 5, 6, 7, 8, 10, 11, 16, 17, 18, 21, 22, 25, 26, 27, 28, 29, 30, 32, 33, 38, 40, 41, 42, 43, 44, 45, 48, 49, 50, 51, 52, 53, 55, 60, 61, 62], "name": [0, 1, 2, 11, 13, 16, 20, 21, 22, 24, 25, 28, 30, 31, 33, 35, 37, 38, 39, 42, 43, 44, 49, 53, 55], "impli": [0, 2, 10, 15, 24, 28, 29, 30, 34, 39, 41, 42, 43, 45, 53, 60], "idea": [0, 2, 6, 7, 13, 28, 35, 43, 45, 46, 48, 50, 51, 52, 53, 55], "start": [0, 2, 3, 4, 6, 7, 8, 11, 21, 22, 23, 25, 28, 29, 30, 38, 41, 46, 48, 49, 51, 52, 53, 54, 60, 62], "where": [0, 1, 3, 5, 6, 7, 8, 9, 11, 12, 13, 21, 22, 24, 25, 27, 28, 31, 33, 35, 38, 39, 40, 42, 43, 44, 45, 46, 47, 50, 51, 52, 53, 54, 57, 60, 62], "want": [0, 1, 2, 5, 6, 7, 8, 11, 15, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 30, 33, 34, 37, 38, 39, 40, 42, 43, 44, 45, 46, 48, 49, 50, 51, 52, 53, 54, 55, 59, 60, 61, 62], "end": [0, 2, 3, 4, 5, 6, 7, 8, 11, 16, 21, 22, 24, 25, 28, 30, 31, 36, 38, 39, 42, 49, 51, 52, 54, 60], "up": [0, 4, 5, 7, 8, 11, 14, 15, 16, 17, 21, 22, 25, 28, 29, 30, 33, 34, 36, 37, 38, 41, 42, 43, 46, 48, 50, 51, 52, 60, 62], "figur": [0, 1, 2, 6, 8, 22, 25, 28, 33, 38, 46, 49, 52, 53, 59, 60, 61], "out": [0, 1, 2, 3, 4, 5, 7, 8, 11, 12, 14, 16, 21, 22, 25, 28, 30, 31, 33, 37, 38, 42, 43, 46, 47, 48, 49, 51, 52, 53, 54, 57, 59, 60, 61, 62], "exactli": [0, 2, 5, 6, 10, 11, 16, 22, 26, 27, 30, 31, 33, 38, 40, 42, 48, 49, 51, 52, 62], "need": [0, 3, 4, 7, 13, 16, 17, 21, 23, 25, 28, 29, 31, 33, 35, 36, 39, 41, 49, 51, 54, 55, 61, 62], "get": [0, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 20, 21, 23, 25, 28, 29, 30, 34, 35, 37, 39, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 57, 59, 60, 61, 62], "actual": [0, 1, 2, 3, 5, 7, 8, 9, 11, 13, 16, 21, 22, 25, 26, 28, 30, 31, 33, 35, 36, 37, 38, 40, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 53, 55, 60, 62], "common": [0, 2, 3, 7, 13, 28, 35, 43, 45, 48, 50, 51, 60, 61], "manag": [0, 2, 4, 5, 7, 28, 41, 47, 57, 62], "rang": [0, 2, 3, 23, 28, 35, 38, 46, 52], "differ": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 24, 25, 26, 27, 28, 29, 30, 31, 34, 35, 37, 38, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 50, 51, 52, 53, 54, 55, 61, 62], "domain": [0, 2, 7, 8, 21, 32, 57, 60, 61], "so": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 33, 34, 35, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 59, 60, 61, 62], "alreadi": [0, 2, 7, 8, 16, 35, 48, 51, 53, 54, 60, 61], "familiar": [0, 2, 13, 21, 22, 28, 35, 38, 45, 47, 60], "broad": [0, 2, 5, 18, 19, 21, 46, 62], "term": [0, 1, 2, 3, 4, 5, 6, 7, 8, 11, 13, 16, 20, 21, 22, 23, 25, 26, 27, 28, 29, 30, 31, 35, 37, 38, 40, 41, 42, 44, 45, 46, 48, 49, 51, 52, 54, 60, 61, 62], "class": [0, 2, 3, 5, 7, 8, 10, 13, 16, 21, 28, 29, 35, 41, 46, 47, 48, 49, 52, 53, 55, 57, 60, 61, 62], "we": [0, 1, 2, 3, 5, 6, 7, 10, 11, 12, 13, 15, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 31, 33, 34, 35, 37, 38, 39, 40, 41, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 55, 57, 60, 61, 62], "focu": [0, 2, 5, 22, 28, 29, 30, 31, 38, 41, 43, 44, 46, 47, 48, 52, 60], "five": [0, 2, 20, 28], "step": [0, 2, 8, 22, 25, 28, 30, 31, 38, 42, 51, 55, 60], "solv": [0, 2, 4, 7, 16, 28, 31, 52, 53, 59, 61], "help": [0, 2, 3, 5, 7, 11, 12, 16, 17, 18, 21, 24, 25, 26, 28, 29, 30, 31, 32, 39, 40, 41, 42, 45, 48, 50, 52, 53, 55, 57, 60, 61, 62], "would": [0, 1, 3, 5, 6, 7, 8, 11, 12, 13, 14, 15, 16, 17, 19, 21, 22, 23, 25, 26, 27, 28, 29, 30, 31, 32, 34, 35, 36, 38, 40, 41, 42, 45, 48, 50, 51, 52, 53, 55, 60, 61, 62], "look": [0, 1, 3, 5, 6, 7, 8, 11, 12, 16, 21, 22, 23, 24, 25, 27, 30, 31, 33, 38, 39, 41, 42, 43, 48, 49, 50, 51, 52, 53, 54, 55, 60, 61, 62], "determin": [0, 1, 2, 5, 7, 11, 12, 16, 25, 30, 42, 48, 50, 53, 61], "order": [0, 2, 7, 16, 18, 19, 22, 26, 27, 28, 31, 36, 38, 40, 41, 46], "gener": [0, 2, 4, 5, 7, 11, 13, 21, 22, 23, 25, 29, 30, 33, 35, 38, 41, 42, 43, 45, 46, 47, 48, 50, 51, 52, 53, 54, 55, 57, 60, 61, 62], "identifi": [0, 1, 2, 7, 11, 13, 17, 18, 19, 21, 22, 25, 28, 33, 35, 38, 43, 48, 49, 53, 55, 60, 61], "those": [0, 1, 2, 3, 5, 6, 7, 8, 11, 13, 16, 18, 22, 25, 26, 27, 28, 29, 30, 31, 33, 35, 38, 40, 41, 43, 44, 45, 46, 48, 49, 50, 51, 52, 53, 59, 60, 61, 62], "bring": [0, 2, 7, 25, 36, 43, 47, 48, 53], "them": [0, 2, 3, 5, 7, 8, 11, 12, 16, 17, 21, 22, 26, 27, 28, 31, 38, 40, 43, 45, 46, 48, 49, 51, 52, 53, 55, 60, 61, 62], "togeth": [0, 2, 11], "first": [0, 1, 2, 3, 7, 8, 9, 11, 12, 13, 15, 17, 21, 22, 24, 25, 27, 28, 30, 31, 32, 33, 34, 35, 37, 38, 39, 42, 43, 46, 47, 48, 49, 51, 52, 53, 54, 55, 60, 62], "should": [0, 2, 3, 5, 7, 8, 9, 11, 13, 15, 16, 21, 22, 23, 25, 26, 28, 30, 32, 34, 35, 38, 39, 41, 42, 45, 46, 47, 48, 49, 50, 52, 53, 60, 61, 62], "straightforward": [0, 2, 25], "yet": [0, 2, 3, 9, 13, 21, 22, 35, 38, 49, 53, 60], "surpris": [0, 2, 42, 48], "how": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 24, 25, 28, 29, 30, 31, 33, 34, 35, 36, 37, 38, 39, 41, 42, 43, 44, 45, 47, 48, 49, 52, 53, 55, 59, 60, 61, 62], "never": [0, 2, 3, 5, 7, 8, 9, 11, 13, 22, 35, 38, 46, 52, 53, 57, 60, 62], "explicitli": [0, 2, 7, 50, 60], "address": [0, 2, 7, 11, 16, 17, 25, 26, 31, 40, 45, 47, 48, 52, 53, 61, 62], "peopl": [0, 2, 3, 5, 7, 8, 11, 12, 13, 14, 17, 18, 21, 22, 26, 27, 28, 29, 30, 31, 32, 35, 38, 40, 41, 42, 43, 46, 48, 49, 50, 52, 53, 55, 57, 59, 60, 61, 62], "excit": [0, 2, 17, 50], "sai": [0, 2, 3, 5, 6, 7, 8, 11, 21, 22, 26, 27, 28, 30, 33, 36, 37, 38, 40, 42, 43, 44, 48, 49, 50, 52, 53, 60, 62], "some": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 14, 17, 21, 22, 23, 24, 25, 28, 29, 30, 35, 38, 39, 40, 41, 42, 43, 47, 48, 49, 50, 51, 52, 53, 55, 57, 59, 60, 61, 62], "alwai": [0, 2, 5, 8, 12, 13, 16, 21, 22, 23, 26, 27, 30, 35, 38, 40, 42, 44, 45, 46, 48, 49, 50, 53, 54, 55, 57, 60, 61], "when": [0, 1, 2, 3, 5, 6, 8, 9, 10, 11, 12, 13, 17, 18, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 33, 35, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 50, 52, 53, 55, 60, 61, 62], "make": [0, 1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 15, 16, 18, 19, 20, 21, 22, 23, 24, 25, 28, 29, 30, 31, 32, 33, 34, 35, 37, 38, 39, 41, 42, 43, 44, 45, 48, 49, 50, 51, 52, 54, 56, 60, 61, 62], "sure": [0, 2, 3, 5, 8, 11, 15, 21, 22, 23, 24, 25, 26, 30, 32, 33, 34, 38, 39, 40, 42, 47, 48, 49, 50, 53, 60, 62], "can": [0, 1, 3, 5, 6, 7, 8, 9, 11, 12, 13, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 35, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 53, 54, 55, 57, 59, 60, 61, 62], "addit": [0, 2, 7, 8, 11, 13, 21, 22, 29, 30, 31, 35, 37, 38, 41, 42, 43, 47, 49, 50, 62], "stakehold": [0, 2, 4, 5, 7, 21, 25, 28, 47, 61, 62], "agre": [0, 2, 48, 61], "There": [0, 2, 6, 8, 11, 12, 16, 21, 22, 29, 38, 41, 45, 46, 48, 50, 51, 55, 57], "noth": [0, 2, 11, 42, 54, 60], "wors": [0, 2, 28, 43, 46, 48], "than": [0, 2, 3, 5, 6, 7, 8, 10, 11, 14, 20, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 37, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 50, 52, 54, 55, 57, 59, 60, 62], "spend": [0, 2, 3, 7, 8, 12, 26, 27, 28, 40, 50, 52, 53, 59], "week": [0, 2, 11, 16, 23, 28, 29, 33, 40, 41, 44, 48, 52, 54, 57], "discov": [0, 2, 7, 23, 48, 53, 60], "valu": [0, 1, 2, 6, 7, 8, 11, 13, 16, 20, 21, 22, 23, 24, 25, 28, 30, 33, 35, 37, 38, 39, 42, 43, 44, 49, 50, 52, 54, 56, 60], "here": [0, 1, 2, 3, 4, 5, 7, 8, 10, 11, 13, 16, 21, 22, 24, 25, 28, 30, 31, 33, 35, 38, 39, 42, 43, 46, 47, 48, 49, 50, 51, 52, 53, 55, 59, 60, 61, 62], "few": [0, 2, 3, 7, 13, 21, 24, 27, 28, 33, 35, 38, 39, 42, 46, 47, 48, 50, 51, 55, 60, 62], "exampl": [0, 2, 3, 4, 5, 7, 8, 11, 12, 14, 20, 22, 25, 26, 27, 29, 30, 32, 33, 37, 38, 40, 41, 43, 45, 48, 49, 50, 51, 52, 53, 61, 62], "know": [0, 2, 3, 5, 6, 7, 8, 10, 11, 14, 15, 16, 20, 21, 23, 25, 26, 27, 28, 29, 30, 31, 34, 37, 40, 41, 42, 43, 45, 46, 48, 49, 52, 53, 54, 55, 60, 61, 62], "reduc": [0, 2, 3, 5, 7, 11, 12, 13, 14, 15, 17, 23, 24, 30, 32, 34, 35, 39, 50, 52, 53, 60, 61], "mass": [0, 2, 26, 40], "incarcer": [0, 2, 4], "busi": [0, 2, 3, 5, 8, 13, 35, 41, 43, 47, 48, 54, 55, 57, 60], "potenti": [0, 1, 2, 4, 7, 8, 11, 16, 18, 19, 21, 24, 25, 29, 30, 31, 39, 41, 48, 50, 52, 54, 58, 62], "custom": [0, 2, 3, 8, 12, 17, 48, 52, 60, 62], "who": [0, 1, 2, 3, 5, 7, 8, 10, 11, 14, 16, 17, 18, 19, 21, 22, 26, 27, 28, 29, 30, 31, 32, 38, 40, 41, 42, 45, 46, 48, 49, 52, 55, 59, 60, 61, 62], "alzheim": [0, 2, 60], "test": [0, 2, 4, 5, 6, 7, 8, 13, 16, 18, 19, 21, 22, 25, 28, 30, 35, 37, 38, 39, 42, 43, 45, 46, 47, 49, 50, 52, 62], "earli": [0, 2, 4, 6, 18, 19, 31, 53, 54, 62], "intervent": [0, 2, 3, 11, 28, 48, 50, 60], "although": [0, 2, 11, 12, 28], "everyon": [0, 1, 2, 11, 25, 29, 40, 41, 45, 48, 50, 55, 60], "u": [0, 1, 2, 4, 7, 8, 11, 13, 16, 17, 21, 22, 25, 28, 29, 30, 31, 32, 33, 35, 38, 41, 42, 43, 48, 49, 50, 52, 53, 55, 60, 62], "fundament": [0, 2, 5, 18, 19, 22, 32, 38, 47, 48, 52, 60], "practic": [0, 2, 3, 4, 5, 6, 7, 13, 15, 34, 35, 46, 47, 49, 50, 54, 60], "us": [0, 1, 2, 3, 4, 5, 8, 9, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 50, 52, 53, 55, 57, 60, 62], "quantifi": [0, 2], "world": [0, 1, 2, 3, 5, 7, 8, 12, 25, 27, 30, 37, 40, 41, 42, 44, 46, 47, 48, 50, 52, 54, 55, 60, 61], "For": [0, 2, 3, 7, 8, 11, 12, 13, 14, 16, 22, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 37, 38, 40, 41, 42, 43, 44, 45, 48, 49, 50, 51, 52, 53, 60, 62], "ran": [0, 2, 3, 5, 6, 8, 11, 13, 22, 35, 38, 44, 52, 54], "our": [0, 1, 2, 3, 5, 6, 8, 10, 11, 12, 13, 16, 17, 18, 20, 21, 22, 24, 25, 28, 29, 30, 32, 33, 35, 37, 38, 39, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 57, 60, 62], "regress": [0, 1, 2, 5, 7, 8, 9, 10, 11, 13, 16, 20, 22, 24, 29, 30, 35, 37, 38, 39, 41, 42, 43, 44, 49, 51, 52, 53, 58, 62], "birth": [0, 2, 30, 42], "weight": [0, 2, 20, 21, 22, 26, 29, 37, 38, 40, 41, 43, 54], "variou": [0, 2, 30, 42], "demograph": [0, 2, 11, 31, 48, 55], "whether": [0, 1, 2, 3, 5, 6, 7, 8, 11, 12, 13, 14, 15, 20, 21, 22, 25, 26, 29, 30, 31, 33, 34, 35, 37, 38, 40, 41, 42, 43, 44, 45, 46, 48, 49, 50, 51, 52, 53, 54, 60, 61, 62], "mother": [0, 2], "smoke": [0, 2, 7, 60], "dure": [0, 2, 5, 12, 13, 18, 25, 35, 52], "pregnanc": [0, 2], "were": [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 13, 18, 19, 20, 25, 27, 28, 29, 30, 33, 35, 37, 40, 41, 42, 43, 48, 49, 50, 52, 53, 54, 60, 62], "matern": [0, 2], "associ": [0, 1, 2, 25, 28, 30, 42, 49, 60], "lower": [0, 2, 7, 10, 14, 16, 29, 30, 35, 37, 39, 40, 41, 43, 49, 62], "signific": [0, 1, 2, 6, 11, 13, 22, 28, 30, 35, 38, 42, 47, 49, 54, 60], "level": [0, 1, 2, 7, 11, 12, 15, 21, 24, 25, 28, 30, 33, 34, 39, 41, 42, 43, 44, 45, 46, 48, 49, 51, 52, 55, 62], "after": [0, 1, 2, 5, 7, 8, 11, 13, 19, 21, 22, 23, 24, 25, 28, 30, 35, 37, 38, 39, 42, 43, 44, 48, 49, 50, 52, 53, 59, 60], "control": [0, 1, 2, 3, 4, 5, 10, 11, 12, 13, 20, 22, 24, 26, 27, 28, 29, 30, 35, 37, 38, 39, 40, 41, 42, 43, 44, 45, 48, 49, 50, 51, 52, 53, 54], "other": [0, 2, 3, 5, 7, 8, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 23, 24, 25, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 57, 60, 61, 62], "confound": [0, 2, 3, 18, 19], "someon": [0, 1, 2, 3, 5, 7, 12, 26, 30, 32, 40, 42, 43, 50, 59, 61, 62], "run": [0, 2, 3, 5, 6, 7, 8, 11, 13, 14, 21, 22, 23, 25, 28, 29, 30, 33, 35, 38, 39, 41, 43, 45, 48, 49, 50, 51, 53, 55, 62], "commerc": [0, 2, 23], "websit": [0, 2, 6, 8, 23, 28, 30, 42, 60], "user": [0, 2, 3, 4, 7, 8, 11, 13, 23, 33, 35, 42, 43, 48, 49, 50, 51, 54, 60, 61], "visit": [0, 2, 8, 11], "randomli": [0, 2, 3, 8, 11, 12, 28, 30, 42, 50, 62], "assign": [0, 1, 2, 3, 4, 7, 8, 11, 22, 25, 28, 30, 38, 42, 43, 44, 46, 47, 48, 50, 54, 57, 62], "see": [0, 1, 2, 3, 5, 6, 7, 8, 11, 13, 14, 17, 18, 19, 22, 25, 27, 28, 30, 33, 34, 35, 37, 38, 40, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 54, 55, 59, 60, 62], "two": [0, 2, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 34, 35, 38, 39, 40, 41, 42, 43, 45, 48, 50, 51, 52, 53, 57, 60, 62], "version": [0, 2, 9, 11, 12, 21, 22, 28, 38, 48], "land": [0, 2, 3, 53], "page": [0, 2, 3, 11, 15, 23, 31, 34, 60, 62], "track": [0, 2, 11, 13, 17, 35, 48], "purchas": [0, 2, 25, 28], "behavior": [0, 2, 3, 7, 8, 11, 16, 22, 33, 38, 48, 50, 60, 62], "analyz": [0, 2, 6, 11, 13, 24, 30, 35, 39, 42, 50, 52, 60, 61, 62], "re": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 21, 22, 23, 24, 26, 27, 28, 29, 30, 31, 33, 34, 35, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 48, 49, 50, 51, 52, 53, 54, 55, 59, 60, 62], "which": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 33, 35, 38, 39, 40, 41, 42, 43, 45, 48, 49, 50, 51, 52, 53, 54, 55, 57, 60, 61, 62], "more": [0, 1, 2, 3, 5, 7, 8, 11, 12, 13, 14, 16, 17, 18, 20, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 32, 34, 35, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 50, 51, 52, 53, 54, 55, 57, 59, 60, 61, 62], "effect": [0, 2, 3, 4, 6, 7, 12, 13, 14, 16, 21, 22, 23, 26, 27, 28, 29, 35, 36, 38, 40, 41, 46, 47, 48, 52, 53, 54, 58, 60, 61, 62], "bui": [0, 2, 8, 25, 28, 53, 60, 62], "train": [0, 2, 5, 25, 28, 41, 47, 48, 53, 60, 61, 62], "algorithm": [0, 2, 5, 7, 22, 38, 46, 48, 51, 52, 53, 60], "cancer": [0, 2, 60], "x": [0, 2, 7, 9, 12, 21, 24, 33, 34, 35, 37, 38, 39, 49, 52, 60], "rai": [0, 2], "evalu": [0, 2, 3, 4, 5, 7, 11, 22, 23, 25, 28, 31, 38, 45, 47, 48, 52, 54, 57, 60, 62], "its": [0, 2, 5, 7, 8, 13, 16, 21, 22, 23, 25, 27, 31, 35, 38, 43, 46, 48, 52, 53, 60], "perform": [0, 5, 7, 16, 25, 30, 36, 39, 41, 42, 43, 45, 46, 61], "patient": [0, 2, 5, 6, 7, 17, 27, 30, 48, 60], "well": [0, 2, 3, 5, 6, 7, 10, 13, 22, 23, 24, 28, 30, 34, 35, 36, 38, 39, 42, 46, 47, 49, 51, 52, 53, 55, 60, 62], "next": [0, 2, 6, 7, 11, 22, 25, 29, 38, 41, 45, 48, 49], "ask": [0, 2, 7, 8, 11, 18, 25, 28, 30, 31, 32, 42, 46, 48, 50, 52, 55, 57, 60, 61, 62], "yourself": [0, 2, 3, 5, 7, 9, 20, 22, 24, 28, 38, 46, 48], "motiv": [0, 2, 3, 21, 47, 62], "The": [0, 2, 4, 5, 6, 7, 12, 13, 14, 18, 19, 20, 21, 22, 24, 29, 30, 31, 32, 33, 35, 36, 37, 38, 39, 41, 42, 43, 44, 45, 46, 47, 48, 51, 55, 57, 59, 60, 61, 62], "part": [0, 2, 6, 7, 11, 18, 21, 23, 28, 29, 30, 36, 41, 42, 43, 46, 47, 48, 51, 60, 62], "character": [0, 5, 21], "predict": [0, 2, 4, 14, 16, 17, 21, 22, 28, 38, 48, 49, 52, 53, 55, 61], "label": [0, 1, 2, 7, 15, 21, 22, 38, 53, 55, 60], "unlabel": [0, 2], "form": [0, 5, 9, 12, 21, 23, 25, 28, 29, 41, 43, 52, 53, 60], "think": [0, 2, 3, 5, 7, 8, 11, 14, 16, 18, 20, 22, 24, 25, 26, 27, 28, 30, 31, 32, 33, 37, 38, 39, 40, 42, 43, 44, 45, 46, 48, 49, 50, 51, 52, 53, 57, 59, 60, 61, 62], "entiti": [0, 2, 7, 13, 24, 35, 39, 43, 44, 48], "observ": [0, 1, 2, 3, 7, 11, 12, 13, 16, 18, 19, 21, 22, 24, 25, 28, 30, 33, 35, 37, 38, 39, 41, 42, 43, 44, 45, 47, 49, 50, 51, 52, 54, 60, 62], "all": [0, 1, 2, 3, 5, 7, 9, 11, 13, 14, 15, 18, 19, 20, 21, 22, 23, 25, 26, 27, 28, 29, 30, 33, 34, 35, 37, 38, 39, 40, 41, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 55, 59, 60, 61, 62], "replic": [0, 2, 7, 11, 25, 28], "gave": [0, 2, 7, 8, 17, 22, 32, 38, 50], "rise": [0, 2, 7, 8, 60], "due": [0, 2, 3, 10, 25, 40, 41, 45, 57, 61, 62], "pictur": [0, 2, 41], "anim": [0, 2, 41], "feed": [0, 2, 3, 7, 51], "bunch": [0, 2, 48], "undergradu": [0, 1, 2, 48], "american": [0, 1, 2, 5, 30, 42, 43, 48, 55], "univers": [0, 2, 16, 30, 31, 41, 42, 48, 49, 60], "photo": [0, 2, 5], "everi": [0, 1, 2, 3, 7, 23, 25, 26, 33, 37, 50, 60, 62], "time": [0, 1, 2, 3, 6, 7, 8, 9, 11, 13, 16, 20, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 49, 51, 52, 54, 62], "photograph": [0, 2], "obvious": [0, 2, 3, 7, 9, 11, 13, 15, 16, 21, 22, 25, 30, 34, 35, 38, 42, 49, 53, 62], "success": [0, 2, 7, 11, 48, 57], "depend": [0, 2, 3, 5, 12, 16, 21, 24, 28, 30, 31, 38, 39, 42, 43, 44, 48, 49, 50, 54, 57, 60, 62], "context": [0, 2, 26, 27, 29, 40, 41, 48, 54, 55, 60, 62], "much": [0, 1, 2, 3, 7, 11, 13, 17, 20, 21, 22, 26, 29, 30, 35, 37, 38, 40, 41, 42, 43, 45, 46, 47, 48, 49, 52, 53, 55, 60, 61], "studi": [0, 1, 2, 3, 5, 6, 7, 12, 17, 28, 30, 31, 42, 43, 46, 47, 48, 52, 54, 55, 60, 62], "cours": [0, 2, 3, 4, 5, 11, 22, 38, 46, 47, 48, 49, 51, 54, 57, 60, 61, 62], "aspir": [0, 2], "bit": [0, 2, 37, 39, 42, 60, 62], "recogn": [0, 2, 22, 38, 45, 46, 49, 60], "understand": [0, 2, 3, 5, 6, 7, 16, 18, 23, 24, 31, 33, 39, 43, 44, 45, 46, 47, 48, 50, 52, 60, 62], "limit": [0, 2, 5, 7, 12, 13, 21, 35, 42, 43, 44, 45, 46, 48, 52, 62], "surprisingli": [0, 2], "long": [0, 1, 2, 3, 4, 6, 7, 13, 16, 28, 31, 35, 36, 37, 43, 46, 48, 54, 57], "thought": [0, 2, 3, 5, 7, 28, 31, 43, 46, 48, 57], "incap": [0, 2], "harbor": [0, 2, 12], "racial": [0, 1, 2, 5, 7, 22, 30, 33, 38, 42], "sexist": [0, 2], "prejudic": [0, 2], "just": [0, 1, 2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20, 21, 22, 23, 24, 25, 28, 29, 33, 34, 35, 37, 38, 39, 41, 43, 44, 45, 46, 48, 49, 50, 51, 52, 54, 55, 59, 60, 61, 62], "built": [0, 2, 55], "math": [0, 2, 6, 10, 23, 45, 46, 49, 50], "racist": [0, 2, 5, 48], "compani": [0, 2, 3, 5, 7, 11, 12, 14, 16, 21, 25, 27, 31, 41, 48, 50, 53, 55, 61], "amazon": [0, 2], "tri": [0, 2, 22, 38, 48, 52], "build": [0, 2, 7, 16, 25, 32, 41, 53, 60], "decid": [0, 1, 2, 3, 5, 11, 13, 26, 28, 35, 40, 48, 51, 53, 54, 62], "hire": [0, 2, 3, 5, 7, 12, 16, 17, 25, 28, 30, 32, 42, 62], "though": [0, 1, 2, 3, 5, 7, 11, 13, 15, 21, 22, 24, 25, 34, 35, 38, 39, 42, 43, 44, 46, 47, 49, 51, 60, 61], "human": [0, 2, 3, 7, 18, 19, 41, 48, 60], "employe": [0, 2, 3, 27, 29, 30, 41], "had": [0, 2, 6, 8, 11, 12, 13, 22, 25, 28, 35, 38, 41, 42, 44, 47, 48, 49, 50, 60, 61], "past": [0, 2, 3, 7, 8, 11, 13, 22, 23, 27, 28, 30, 35, 38, 46, 53, 55, 60, 61], "subject": [0, 2, 3, 11, 13, 25, 28, 29, 35, 41, 45, 46, 47, 48, 54], "made": [0, 2, 5, 7, 8, 18, 19, 22, 28, 38, 47, 52, 57], "supervisor": [0, 2], "resum": [0, 2, 58], "itself": [0, 1, 2, 5, 16, 23, 57, 59, 60], "veri": [0, 1, 2, 3, 6, 7, 9, 12, 13, 20, 25, 28, 29, 31, 34, 35, 37, 38, 41, 42, 43, 46, 48, 50, 51, 52, 53, 55, 57, 60, 62], "staff": [0, 2], "person": [0, 1, 2, 3, 5, 7, 11, 16, 21, 27, 30, 31, 33, 41, 42, 48, 49, 50, 61, 62], "inherit": [0, 2], "bias": [0, 2, 5, 7, 45], "oop": [0, 2], "suddenli": [0, 2, 41, 48], "realiz": [0, 2, 7, 9, 18, 25, 48, 60], "recruit": [0, 2, 48], "engin": [0, 2, 21, 25, 41, 46], "didn": [0, 2, 4, 7, 11, 30, 42, 48, 53, 60], "women": [0, 2, 5, 7, 10, 29, 30, 41, 42, 48, 49, 50], "abandon": [0, 2], "ok": [0, 2, 3, 5, 7, 11, 22, 28, 38, 39, 43, 45, 48, 49, 51, 52, 62], "bia": [0, 1, 2, 4, 5, 7, 22, 38, 43], "now": [0, 1, 2, 3, 5, 7, 8, 9, 11, 13, 15, 17, 21, 22, 24, 26, 27, 28, 29, 30, 33, 34, 35, 38, 39, 40, 41, 42, 43, 44, 46, 49, 50, 51, 52, 53, 60, 62], "open": [1, 3, 11, 16, 30, 42, 55], "book": [1, 23, 28, 41, 46, 60], "commun": [1, 2, 5, 7, 12, 31, 41, 46, 48, 50, 55, 62], "survei": [1, 2, 14, 22, 28, 29, 30, 38, 41, 42, 43, 49, 52, 55], "ac": [1, 55], "relationship": [1, 9, 12, 13, 15, 21, 22, 24, 29, 33, 34, 35, 38, 39, 41, 43, 44, 49, 52, 60], "between": [1, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 24, 25, 28, 29, 30, 31, 33, 34, 35, 37, 38, 39, 40, 41, 43, 44, 46, 50, 51, 52, 54, 57, 60, 62], "attain": [1, 29, 41], "wage": [1, 10, 29, 41, 43, 60], "conduct": [1, 28, 31, 48], "unit": [1, 5, 11, 12, 13, 18, 19, 20, 22, 24, 25, 28, 29, 30, 33, 35, 37, 38, 39, 41, 42, 50, 53, 54, 55, 60], "state": [1, 2, 5, 12, 13, 20, 24, 25, 29, 30, 31, 33, 35, 37, 39, 41, 42, 53, 55, 60, 61], "censu": [1, 2, 33, 55], "bureau": [1, 2, 32, 55], "count": [1, 5, 11, 13, 25, 28, 30, 33, 34, 35, 38, 42, 50, 60], "take": [1, 3, 5, 7, 8, 9, 10, 11, 12, 20, 21, 22, 23, 24, 25, 28, 37, 38, 39, 43, 45, 46, 47, 49, 50, 51, 52, 54, 60, 62], "place": [1, 2, 5, 7, 11, 13, 18, 19, 21, 25, 29, 30, 31, 33, 35, 41, 43, 47, 48, 50, 51, 52, 53, 55, 59, 60], "10": [1, 4, 5, 8, 15, 24, 26, 27, 33, 34, 35, 37, 39, 40, 43, 49], "year": [1, 5, 13, 17, 22, 23, 24, 25, 27, 28, 29, 30, 31, 33, 34, 35, 38, 39, 41, 42, 44, 46, 47, 49, 50, 52, 55, 60], "numer": [1, 11, 18, 19, 22, 24, 37, 38, 39, 43], "featur": [1, 2, 5, 16, 21, 22, 25, 30, 38, 42, 50, 53, 60, 61], "popul": [1, 2, 8, 13, 22, 24, 26, 27, 28, 29, 31, 33, 35, 38, 39, 40, 41, 42, 43, 48, 50, 52, 54, 61], "includ": [1, 5, 7, 8, 11, 13, 15, 16, 21, 22, 24, 25, 26, 29, 30, 31, 34, 35, 36, 38, 39, 41, 42, 43, 44, 49, 50, 51, 52, 55, 60, 61, 62], "100": [1, 5, 8, 13, 15, 21, 23, 28, 34, 35, 38, 42, 43], "2017": [1, 28, 33, 35, 38], "sampl": [1, 2, 7, 8, 13, 14, 16, 21, 22, 25, 28, 29, 30, 31, 33, 35, 37, 38, 41, 42, 43, 45, 48, 50, 52, 54, 60, 62], "1": [1, 4, 5, 8, 23, 26, 27, 33, 40, 43, 44, 48, 49, 50, 52, 57], "0": [1, 5, 8, 13, 15, 20, 21, 22, 23, 25, 26, 27, 28, 30, 33, 34, 35, 37, 38, 39, 40, 41, 42, 43, 44, 49, 57], "ipum": [1, 55], "tool": [1, 2, 3, 5, 11, 16, 21, 22, 23, 24, 28, 38, 39, 41, 43, 44, 47, 49, 53, 57, 60], "major": [1, 5, 7, 16, 30, 42, 46, 47, 49, 54, 60, 61], "dataset": [1, 2, 5, 9, 11, 13, 16, 20, 24, 25, 29, 30, 33, 35, 37, 39, 41, 42, 43, 45, 49, 51, 52, 55], "govern": [1, 2, 3, 20, 31, 37, 41, 48, 53, 55, 60], "agenc": [1, 53], "real": [1, 4, 5, 7, 9, 10, 12, 16, 21, 25, 28, 30, 31, 41, 42, 46, 47, 48, 50, 60, 61, 62], "mean": [1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 48, 49, 50, 52, 53, 55, 57, 60, 62], "you": [1, 5, 6, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 50, 51, 52, 53, 55, 57, 60, 61, 62], "being": [1, 2, 3, 5, 7, 11, 13, 16, 21, 22, 25, 26, 27, 28, 29, 30, 35, 36, 37, 38, 40, 41, 42, 43, 44, 46, 47, 48, 49, 50, 52, 60, 61, 62], "document": [1, 16, 20, 21, 25, 37, 60], "found": [1, 3, 7, 11, 13, 14, 17, 21, 22, 25, 26, 28, 29, 35, 38, 40, 41, 45, 46, 47, 48, 57, 60], "either": [1, 2, 3, 5, 11, 21, 22, 31, 38, 44, 47, 48, 50, 52, 55, 62], "search": [1, 21, 30, 41, 42, 55], "someth": [1, 3, 5, 7, 8, 11, 13, 18, 19, 21, 22, 25, 35, 36, 38, 44, 46, 48, 49, 50, 51, 52, 53, 54, 55, 57, 59, 60, 61, 62], "right": [1, 4, 7, 19, 20, 22, 31, 33, 37, 38, 45, 48, 49, 51, 52, 53, 57, 60, 62], "within": [1, 3, 10, 23, 24, 25, 29, 33, 35, 39, 41, 43, 45, 50, 57, 60], "inform": [1, 2, 5, 9, 11, 18, 19, 21, 22, 25, 28, 34, 38, 41, 46, 48, 53, 55, 57, 61], "background": [1, 7, 46], "statu": [1, 25, 61], "repres": [1, 2, 5, 7, 11, 12, 33, 42, 46, 47, 48, 53], "colleg": [1, 22, 30, 38, 41, 42, 43], "degre": [1, 22, 30, 38, 42, 48], "ones": [1, 7, 38], "abil": [1, 7, 16, 34, 46], "job": [1, 3, 5, 7, 22, 26, 27, 28, 29, 30, 33, 38, 40, 41, 42, 53, 57, 62], "download": [1, 2, 13, 15, 20, 22, 24, 25, 29, 30, 33, 34, 35, 37, 38, 39, 41, 42], "us_acs_2017_10pct_sampl": 1, "dta": [1, 20, 22, 29, 30, 37, 38, 41, 42, 43], "panda": [1, 11, 33, 34, 35, 37, 38, 39, 41, 42, 43, 49], "datafram": [1, 22, 33, 38], "pd": [1, 22, 30, 33, 34, 35, 37, 38, 39, 41, 42, 43, 49], "read_stata": [1, 22, 37, 38, 41, 42, 43], "normal": [1, 5, 6, 8, 11, 22, 23, 24, 38, 39, 43, 51, 55], "d": [1, 2, 5, 6, 8, 11, 12, 13, 21, 22, 23, 25, 27, 28, 33, 35, 38, 40, 41, 43, 44, 48, 49, 50, 51, 52, 53, 62], "tell": [1, 5, 7, 8, 11, 13, 17, 22, 26, 27, 28, 29, 33, 35, 37, 38, 40, 41, 43, 48, 49, 50, 53, 55, 60, 62], "dive": [1, 2, 5, 7, 53, 60], "mid": [1, 16, 31, 45], "ll": [1, 2, 3, 5, 7, 8, 11, 13, 15, 20, 21, 22, 23, 24, 28, 29, 30, 33, 34, 35, 37, 38, 39, 41, 42, 43, 44, 45, 47, 48, 49, 50, 51, 52, 53, 55, 60, 62], "spare": 1, "wrang": 1, "analysi": [1, 3, 7, 8, 11, 13, 18, 19, 21, 22, 25, 28, 29, 35, 38, 41, 50, 52, 54, 60, 61, 62], "ag": [1, 2, 5, 10, 11, 21, 22, 28, 29, 38, 41, 43, 44, 46, 48, 49], "empstat": 1, "inctot": 1, "educd": 1, "statefip": 1, "countyfip": 1, "sex": [1, 21, 29, 41, 43], "race": [1, 7, 22, 33, 38, 43, 49, 55], "hispan": [1, 22, 33, 38], "thin": [1, 28, 37], "3": [1, 4, 8, 17, 26, 27, 33, 40, 43, 49, 50], "creat": [1, 5, 6, 7, 13, 15, 16, 20, 21, 22, 23, 25, 28, 30, 34, 35, 37, 38, 42, 43, 46, 48, 49, 60, 62], "indic": [1, 2, 4, 7, 11, 13, 24, 25, 29, 30, 33, 35, 39, 41, 42, 48, 54, 58], "respond": [1, 8, 29, 30, 40, 41, 43, 48, 49, 55], "color": [1, 3, 20, 23, 31, 35, 37, 48], "A": [1, 4, 5, 6, 8, 12, 14, 16, 21, 23, 25, 42, 46, 47, 48, 50, 53, 55, 57, 60, 62], "NOT": [1, 35, 49, 55], "white": [1, 5, 7, 22, 30, 33, 38, 42, 43, 48, 49], "non": [1, 2, 3, 5, 7, 16, 18, 20, 22, 25, 28, 29, 31, 32, 37, 38, 41, 43, 44, 49], "individu": [1, 11, 12, 16, 18, 19, 30, 41, 42, 43, 44, 45, 49, 50, 55], "again": [1, 3, 8, 11, 15, 22, 24, 25, 28, 30, 34, 38, 39, 42, 43, 48, 49, 50, 53], "poc": 1, "categori": [1, 13, 20, 21, 22, 25, 30, 33, 37, 38, 42], "note": [1, 2, 4, 7, 11, 12, 13, 18, 20, 21, 22, 23, 24, 25, 28, 29, 30, 33, 35, 37, 38, 39, 41, 42, 43, 45, 46, 47, 49, 50, 51, 52, 53, 55, 57, 61, 62], "pleas": [1, 11, 16, 22, 25, 26, 27, 29, 30, 31, 33, 38, 40, 41, 42, 46, 47, 51, 57, 60, 62], "excus": [1, 50, 57], "harmon": 1, "code": [1, 7, 9, 11, 16, 20, 21, 22, 24, 25, 29, 30, 33, 37, 38, 41, 42, 43, 47, 49, 55, 59], "across": [1, 3, 8, 10, 11, 13, 18, 19, 22, 28, 29, 30, 31, 35, 38, 39, 41, 42, 43, 44, 50, 54, 57, 60, 62], "period": [1, 13, 21, 24, 25, 35, 39, 43, 44, 46, 48, 60], "list": [1, 7, 22, 25, 35, 36, 38, 43, 60, 61], "black": [1, 4, 5, 7, 21, 22, 30, 33, 38, 39, 42, 43, 48, 60], "appropri": [1, 11, 25, 33, 47, 51, 61, 62], "2018": [1, 13, 28, 29, 33, 35, 41, 50, 60], "4": [1, 4, 17, 26, 27, 31, 33, 40, 43, 49, 50, 57], "interest": [1, 2, 5, 7, 11, 12, 14, 17, 18, 19, 22, 25, 29, 30, 32, 36, 38, 41, 42, 43, 44, 45, 46, 47, 48, 49, 51, 53, 54, 57, 60, 61, 62], "emploi": [1, 10, 22, 28, 29, 38, 41, 43, 47, 60], "seek": [1, 27, 30, 31, 42, 62], "restrict": [1, 22, 36, 38, 62], "unemploi": 1, "exclud": [1, 21, 22, 38, 46], "anyon": [1, 2, 3, 5, 7, 18, 19, 29, 30, 41, 42, 49, 50, 59], "doesn": [1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 21, 22, 28, 33, 38, 40, 41, 43, 49, 50, 52, 54, 60, 62], "answer": [1, 4, 5, 6, 8, 10, 11, 12, 17, 20, 22, 25, 26, 27, 28, 29, 30, 31, 32, 33, 35, 37, 38, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 52, 53, 57, 59, 60, 62], "aren": [1, 2, 3, 5, 6, 7, 8, 11, 21, 22, 23, 25, 26, 29, 33, 38, 40, 41, 43, 45, 46, 50, 51, 52, 53, 55, 60, 62], "5": [1, 4, 5, 17, 23, 26, 27, 33, 40, 43, 49, 52, 57], "also": [1, 2, 3, 5, 7, 11, 12, 13, 14, 16, 17, 18, 19, 20, 21, 22, 24, 28, 29, 30, 31, 33, 35, 37, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 57, 60, 61, 62], "least": [1, 2, 3, 21, 22, 25, 26, 28, 29, 31, 35, 37, 38, 39, 41, 42, 43, 47, 48, 49, 51, 54, 57, 60, 61], "requir": [1, 4, 5, 8, 11, 13, 16, 18, 21, 27, 28, 31, 35, 43, 45, 46, 47, 49, 50, 53, 54, 55, 57, 60, 61, 62], "whose": [1, 8, 13, 32, 35, 42, 60], "record": [1, 7, 20, 25, 37, 41], "90": [1, 5, 21, 23, 25, 28, 33, 37, 38, 49], "1980": [1, 33, 35, 41], "1990": [1, 15, 33, 34, 38, 41], "wa": [1, 3, 5, 6, 7, 11, 12, 13, 15, 16, 18, 19, 21, 22, 25, 28, 29, 30, 31, 33, 34, 35, 36, 37, 38, 39, 41, 42, 43, 44, 48, 50, 53, 54, 60, 62], "might": [1, 2, 3, 5, 7, 8, 11, 12, 16, 20, 22, 25, 26, 27, 28, 30, 31, 32, 33, 34, 37, 38, 40, 41, 42, 43, 44, 45, 48, 50, 53, 54, 60, 61, 62], "old": [1, 11, 22, 38, 43, 49, 52, 53], "census": [1, 55], "older": [1, 5, 29, 41, 48, 49], "e": [1, 3, 5, 7, 8, 11, 12, 13, 15, 16, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 52, 54, 55, 60, 61, 62], "ani": [1, 3, 5, 7, 8, 11, 12, 13, 16, 21, 22, 23, 25, 28, 29, 30, 33, 35, 38, 41, 42, 43, 44, 46, 48, 50, 51, 52, 53, 54, 55, 57, 60, 61, 62], "abov": [1, 2, 7, 8, 12, 13, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 35, 38, 39, 40, 41, 42, 43, 44, 48, 49, 51, 52, 53, 55, 60, 61], "deal": [1, 3, 23, 37, 48, 49, 53, 59, 61], "worri": [1, 7, 8, 11, 12, 18, 19, 21, 23, 26, 28, 30, 38, 40, 42, 53, 60], "parenthes": 1, "except": [1, 5, 7, 12, 22, 33, 38, 41, 43, 45, 49, 51, 54, 57], "insofar": 1, "littl": [1, 5, 14, 21, 22, 33, 38, 39, 42, 44, 48, 49, 50, 52], "complic": [1, 3, 6, 7, 57, 60], "6": [1, 4, 23, 26, 27, 33, 40, 43, 49, 50, 51], "categor": [1, 21, 22, 24, 25, 30, 38, 39, 42, 43, 49], "highest": [1, 43], "high": [1, 7, 10, 12, 13, 14, 16, 21, 24, 25, 28, 29, 30, 35, 39, 41, 42, 44, 45, 48, 51, 52, 62], "school": [1, 7, 10, 28, 29, 30, 31, 41, 42, 43, 44, 45, 60], "diploma": 1, "equival": [1, 11, 45], "ged": 1, "altern": [1, 7, 18, 19, 28, 62], "credenti": 1, "better": [1, 3, 6, 7, 22, 25, 30, 31, 38, 42, 44, 45, 46, 47, 48, 51, 57, 62], "ha": [1, 2, 3, 5, 7, 9, 11, 13, 14, 16, 21, 22, 25, 27, 28, 29, 30, 31, 33, 34, 35, 38, 41, 42, 43, 44, 45, 46, 48, 49, 50, 51, 52, 53, 54, 55, 60, 61, 62], "neither": [1, 40, 49], "nor": [1, 46], "bachelor": [1, 30, 42], "assum": [1, 5, 7, 8, 14, 16, 21, 23, 24, 25, 28, 29, 30, 37, 38, 39, 41, 42, 43, 44, 45, 47, 52, 53, 60, 62], "advanc": [1, 3, 7, 16, 22, 38, 47, 50, 54, 57], "master": [1, 15, 22, 29, 34, 35, 37, 38, 39, 41, 42, 43, 47, 48, 49], "doctor": [1, 27], "while": [1, 3, 5, 7, 8, 9, 11, 12, 13, 16, 21, 25, 30, 31, 32, 35, 38, 43, 46, 47, 48, 49, 50, 52, 55, 57, 60, 61], "number": [1, 11, 12, 13, 15, 16, 21, 22, 24, 25, 28, 29, 30, 31, 33, 34, 35, 37, 38, 39, 41, 42, 43, 48], "encod": [1, 21, 25, 33, 34, 35, 37, 38, 43, 49], "implement": [1, 13, 21, 22, 24, 26, 30, 31, 35, 38, 39, 40, 41, 42, 51, 52], "properli": [1, 34, 48, 53, 60], "below": [1, 5, 7, 14, 16, 18, 19, 21, 22, 25, 26, 27, 28, 29, 38, 40, 41, 43, 44, 48, 49, 52, 55, 60], "fine": [1, 5, 6, 7, 13, 35, 46, 48, 52, 57], "suggest": [1, 11, 23, 26, 40, 41, 42, 54], "singl": [1, 3, 9, 11, 13, 21, 22, 23, 24, 25, 33, 35, 38, 39, 45, 49, 53, 61, 62], "three": [1, 2, 13, 21, 23, 25, 28, 32, 35, 49, 51, 55, 60, 61, 62], "each": [1, 2, 6, 9, 11, 12, 13, 14, 15, 22, 24, 25, 26, 27, 28, 29, 30, 31, 33, 34, 35, 38, 39, 40, 41, 42, 43, 44, 45, 46, 50, 51, 53, 55, 57, 60], "group": [1, 2, 8, 11, 12, 13, 14, 17, 18, 19, 22, 24, 26, 27, 28, 29, 30, 31, 32, 35, 38, 39, 40, 41, 42, 44, 45, 46, 48, 49, 50, 52, 53, 60], "refer": [1, 11, 12, 13, 25, 35, 36, 49, 51, 52, 57, 60, 62], "graduat": [1, 29, 30, 41, 42], "interchang": 1, "post": [1, 2, 3, 13, 32, 35, 49, 60, 61], "treatment": [1, 3, 5, 6, 11, 12, 13, 14, 18, 19, 22, 26, 27, 28, 30, 35, 38, 40, 41, 42, 44, 47, 48, 49, 51, 52, 54, 60], "outcom": [1, 2, 3, 4, 7, 8, 11, 12, 13, 14, 17, 18, 19, 22, 25, 28, 29, 35, 38, 41, 44, 45, 46, 48, 50, 52, 54, 58, 60, 62], "monei": [1, 3, 8, 14, 23, 27, 28, 50, 53], "mechan": [1, 16, 29, 41, 43], "phd": [1, 21], "got": [1, 8, 22, 23, 30, 38, 48, 52, 62], "under": [1, 3, 11, 22, 23, 25, 28, 30, 33, 38, 45, 49, 53, 61], "sinc": [1, 2, 7, 8, 11, 13, 22, 28, 29, 30, 31, 33, 35, 38, 41, 42, 43, 44, 45, 48, 49, 50, 52, 55, 60, 62], "without": [1, 2, 5, 7, 11, 14, 20, 22, 24, 25, 28, 30, 36, 37, 38, 39, 41, 43, 47, 50, 54, 62], "given": [1, 2, 5, 7, 8, 11, 12, 13, 14, 15, 16, 18, 19, 21, 22, 23, 25, 26, 27, 28, 29, 30, 31, 33, 34, 35, 36, 38, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 53, 54, 57, 60], "gone": [1, 7, 43, 55], "7": [1, 4, 23, 26, 27, 33, 40, 43, 49, 50], "simpl": [1, 4, 5, 7, 9, 11, 13, 17, 24, 26, 27, 28, 35, 39, 40, 43, 44, 46, 49, 51, 52, 60], "likelihood": [1, 7, 25, 30, 35, 37, 38, 39, 41, 42, 43, 49, 53, 60], "linear": [1, 5, 7, 9, 11, 16, 21, 22, 24, 29, 30, 38, 39, 41, 42, 43, 44, 47, 49, 52], "probabl": [1, 5, 7, 8, 9, 11, 21, 22, 23, 25, 28, 29, 30, 33, 38, 41, 42, 45, 48, 49, 50, 51, 52, 53, 54, 55, 60, 62], "squar": [1, 22, 29, 30, 35, 37, 38, 39, 41, 42, 43, 44, 49, 51], "updat": [1, 22, 33, 38], "languag": [1, 11, 18, 19, 20, 21, 37, 46, 47, 52, 60, 62], "oppos": [1, 29, 30, 41, 42, 61], "could": [1, 2, 5, 7, 8, 11, 14, 16, 18, 19, 22, 25, 28, 29, 30, 31, 32, 33, 38, 40, 41, 42, 43, 48, 49, 51, 53, 60, 62], "8": [1, 4, 26, 27, 33, 35, 40, 43, 49, 50], "add": [1, 5, 10, 11, 16, 21, 24, 29, 30, 33, 39, 41, 42, 43, 44, 49, 51, 61], "appar": [1, 6, 7, 22, 38, 46, 52], "compar": [1, 2, 3, 8, 10, 11, 13, 16, 18, 19, 22, 24, 25, 26, 28, 29, 30, 31, 35, 36, 38, 39, 40, 41, 42, 50, 54, 60, 62], "9": [1, 4, 23, 26, 27, 33, 35, 37, 40, 49], "formula": [1, 35, 37, 38, 39, 41, 42, 43, 49], "omit": [1, 28, 43, 49], "coeffici": [1, 2, 10, 20, 21, 22, 29, 30, 37, 38, 41, 43, 44, 49], "neg": [1, 2, 6, 11, 21, 25, 26, 30, 40, 49, 52], "chang": [1, 5, 11, 13, 15, 21, 22, 23, 26, 28, 29, 31, 33, 34, 35, 37, 38, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 50, 52, 54, 60, 61, 62], "correl": [1, 5, 7, 8, 9, 12, 14, 15, 21, 24, 34, 35, 37, 39, 44, 45, 60], "explain": [1, 16, 20, 21, 24, 29, 30, 37, 39, 41, 43, 53, 57, 60, 62], "learn": [1, 3, 4, 5, 11, 13, 16, 17, 18, 21, 22, 28, 31, 35, 38, 45, 46, 47, 48, 49, 53, 54, 57, 59, 60, 62], "wrong": [1, 5, 7, 8, 9, 21, 30, 33, 42, 49, 52, 57], "assumpt": [1, 3, 4, 7, 18, 19, 45, 48, 50, 57], "causal": [1, 3, 4, 8, 12, 15, 17, 22, 29, 34, 38, 41, 46, 47, 48, 57, 61, 62], "violat": [1, 8, 50], "11": [1, 4, 26, 27, 33, 34, 35, 37, 39, 40, 47, 49], "final": [2, 3, 5, 7, 11, 12, 13, 15, 21, 22, 23, 25, 28, 29, 34, 35, 38, 41, 46, 47, 49, 51, 57, 60, 61, 62], "narrow": [2, 22, 31, 38], "small": [2, 3, 7, 11, 13, 22, 25, 30, 34, 35, 38, 42, 45, 48, 50, 52, 54, 55], "conceptu": [2, 18, 26, 27, 40, 50], "framework": [2, 4, 7, 8, 11, 18, 19, 21, 30, 44, 50], "kei": [2, 11, 16, 25, 30, 31, 33, 42, 46, 48, 62], "tractabl": 2, "meet": [2, 16, 17, 21, 26, 27, 28, 31, 36, 40, 46, 47], "criteria": [2, 4, 28, 50], "directli": [2, 3, 11, 18, 22, 25, 26, 27, 30, 38, 40, 42, 45, 49, 60], "approach": [2, 16, 24, 26, 28, 39, 40, 43, 46, 47, 52, 53, 57, 60, 62], "seem": [2, 5, 7, 11, 12, 17, 22, 34, 38, 42, 46, 48, 49, 50, 52, 53, 54, 60], "vagu": [2, 53], "immedi": [2, 21, 29, 32, 41, 53, 60], "collect": [2, 3, 5, 11, 25, 26, 28, 40, 49, 51, 53, 60], "To": [2, 4, 6, 8, 11, 13, 14, 20, 21, 26, 28, 29, 30, 31, 33, 35, 37, 40, 41, 43, 47, 48, 49, 50, 51, 52, 57, 60, 62], "illustr": [2, 5, 6, 7, 9, 43, 48, 51, 52, 60], "bad": [2, 3, 6, 17, 22, 38, 48, 52, 60], "describ": [2, 7, 11, 12, 13, 16, 18, 19, 22, 23, 26, 27, 30, 31, 35, 38, 40, 42, 43, 48, 60], "polici": [2, 5, 13, 14, 26, 32, 35, 40, 46, 48, 60, 62], "me": [2, 7, 21, 26, 27, 28, 39, 40, 45, 49, 55], "By": [2, 3, 9, 22, 24, 28, 31, 38, 39, 49, 50, 60], "contrast": [2, 5, 48, 49, 50, 60], "avail": [2, 3, 5, 7, 11, 16, 18, 19, 22, 23, 25, 31, 38, 47, 48, 53, 55, 57], "grand": 2, "juri": 2, "longer": [2, 24, 35, 39, 43, 44, 50, 54], "sentenc": [2, 46, 57], "pool": [2, 24, 39], "citizen": [2, 38], "prosecutor": 2, "guidanc": [2, 11, 16, 25, 30], "theori": [2, 4, 13, 18, 19, 35, 52, 60], "suppos": [2, 3, 5, 7, 8, 11, 21, 22, 23, 28, 38, 43, 44, 45, 48, 49, 52, 53, 60], "hold": [2, 3, 8, 26, 27, 30, 40, 42, 48], "account": [2, 3, 7, 11, 18, 19, 22, 23, 29, 37, 38, 41, 43, 44, 45, 49], "said": [2, 8, 42, 44, 48, 49, 50, 60, 61, 62], "prosector": 2, "shape": [2, 7, 21, 25, 31, 60], "reach": [2, 7, 50, 53, 62], "recommend": [2, 11, 23, 28, 33, 48, 62], "begin": [2, 7, 8, 11, 13, 15, 16, 22, 24, 25, 29, 31, 34, 35, 38, 39, 41, 43, 47, 60, 61, 62], "attribut": [2, 8, 11, 28, 30, 37, 42, 48], "lab": [2, 22, 38, 48], "later": [2, 5, 21, 28, 29, 32, 41, 42, 43, 52, 60, 62], "diagnos": [2, 9], "mortem": 2, "ve": [2, 5, 7, 8, 9, 11, 14, 15, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 29, 30, 34, 38, 40, 41, 42, 43, 45, 46, 48, 49, 50, 51, 52, 53, 54, 55, 59, 60, 62], "clear": [2, 5, 6, 7, 11, 13, 22, 31, 32, 35, 38, 43, 47, 50, 53, 55, 57, 60, 61, 62], "second": [2, 3, 7, 9, 16, 21, 22, 25, 29, 30, 33, 38, 41, 42, 43, 45, 46, 48, 49, 53, 61, 62], "find": [2, 3, 5, 7, 8, 11, 12, 13, 14, 18, 19, 21, 22, 23, 25, 29, 30, 35, 38, 40, 41, 42, 44, 45, 46, 48, 49, 50, 52, 53, 57, 60, 61], "current": [2, 11, 14, 16, 22, 23, 25, 28, 29, 38, 41, 43, 60], "comparison": [2, 7, 13, 16, 18, 19, 28, 29, 35, 41], "last": [2, 11, 13, 22, 25, 30, 33, 35, 36, 38, 45, 51], "moreov": [2, 5, 7, 23, 24, 39, 46, 48, 49, 57, 59], "imagin": [2, 5, 8, 11, 16, 28, 48, 60], "crime": [2, 9, 32, 33, 35, 58], "commit": [2, 11, 32, 53], "tabl": [2, 29, 30, 38, 41, 42, 53, 62], "characterist": [2, 20, 30, 37, 42, 51], "It": [2, 3, 4, 7, 8, 9, 13, 16, 17, 22, 26, 27, 30, 33, 34, 35, 38, 40, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 54, 55, 60, 62], "save": [2, 7, 8, 9, 22, 24, 27, 28, 38, 39, 41], "lost": 2, "energi": [2, 32, 55], "Being": 2, "abl": [2, 5, 7, 8, 11, 16, 18, 21, 22, 25, 28, 30, 33, 38, 42, 46, 47, 52, 55, 60, 62], "allow": [2, 3, 11, 13, 18, 21, 22, 25, 27, 28, 30, 35, 38, 42, 43, 45, 46, 48, 49, 54, 61], "statehold": 2, "happen": [2, 3, 7, 8, 10, 11, 16, 19, 22, 24, 28, 29, 38, 39, 41, 43, 44, 45, 46, 48, 50, 53, 60], "serious": [2, 5, 53], "Not": [2, 17, 38, 41, 50], "abstractli": 2, "draw": [2, 7, 24, 28, 39, 46, 52, 57, 62], "graph": [2, 20, 37, 53], "column": [2, 11, 16, 21, 22, 24, 25, 33, 35, 38, 39, 41, 43, 55], "predictor": [2, 7, 16, 25], "diagnost": [2, 53, 62], "liter": [2, 5, 44, 49, 59], "ax": [2, 53], "etc": [2, 7, 12, 16, 22, 25, 26, 27, 38, 40, 53, 54, 55, 60, 62], "turn": [2, 3, 7, 16, 20, 31, 37, 41, 43, 48, 49, 52, 60], "wasn": [2, 22, 30, 38, 42, 48], "suffici": [2, 8, 29, 41, 54], "show": [2, 3, 5, 6, 7, 8, 11, 21, 22, 24, 25, 28, 31, 37, 38, 39, 42, 48, 49, 59, 60, 62], "ensur": [2, 5, 7, 11, 12, 16, 21, 22, 25, 28, 30, 31, 38, 42, 46, 47, 48, 50, 54], "constitut": [2, 5, 8, 24, 30, 31, 33, 39, 42, 43, 53, 57, 60], "avoid": [2, 7, 21, 23, 25, 26, 27, 40, 48, 52, 53, 60], "told": [2, 5, 8, 18, 19, 53], "clearer": [2, 11], "written": [2, 12, 46, 60, 62], "hypothesi": [2, 11, 28, 42, 49], "follow": [2, 3, 4, 5, 7, 11, 12, 16, 17, 20, 21, 24, 25, 28, 29, 30, 31, 33, 37, 39, 41, 42, 43, 49, 50, 51, 52, 57, 60, 62], "true": [2, 5, 7, 8, 10, 11, 12, 21, 22, 26, 27, 28, 29, 30, 33, 34, 35, 37, 38, 39, 40, 41, 42, 43, 45, 46, 49, 50, 53, 62], "fals": [2, 5, 21, 22, 25, 33, 34, 35, 37, 38, 49], "consid": [2, 5, 7, 12, 13, 21, 25, 26, 31, 33, 35, 40, 48, 50, 52, 60], "sentec": 2, "insul": 2, "counti": [2, 9, 13, 15, 22, 31, 33, 34, 35, 38, 43, 55, 60], "along": [2, 16, 25, 28, 43, 48], "posit": [2, 7, 12, 13, 28, 30, 35, 37, 41, 52], "presenc": [2, 12, 28, 31, 33, 53, 60], "zero": [2, 7, 33, 34, 35, 37, 38, 42, 49, 50], "plan": [2, 23, 26, 31, 40, 48, 60, 62], "congratul": [2, 17, 22, 38, 49], "realli": [2, 3, 5, 6, 7, 8, 9, 11, 12, 17, 20, 22, 24, 28, 31, 33, 38, 43, 45, 46, 48, 50, 51, 52, 54, 55, 60], "hard": [2, 3, 5, 7, 16, 17, 21, 22, 23, 31, 32, 38, 46, 47, 48, 55], "easi": [2, 3, 5, 6, 7, 21, 33, 38, 43, 49], "stuff": [2, 4, 46, 55], "let": [2, 5, 6, 8, 11, 14, 17, 24, 25, 28, 29, 30, 33, 37, 39, 41, 42, 43, 46, 48, 49, 50, 52, 60], "same": [2, 4, 6, 7, 8, 9, 10, 11, 12, 13, 18, 21, 22, 25, 28, 29, 30, 31, 33, 35, 38, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 50, 51, 52, 55, 60, 62], "big": [2, 3, 5, 7, 21, 22, 23, 28, 38, 46, 48, 50, 52, 54, 61], "spender": 2, "variat": [2, 3, 5, 7, 9, 11, 24, 28, 29, 30, 38, 39, 41, 43, 44, 54], "public": [2, 3, 5, 18, 19, 31, 41, 46, 48, 52, 53, 60, 61], "whom": [2, 21, 26, 28, 44, 49, 52, 60, 62], "measur": [2, 3, 7, 8, 12, 22, 26, 27, 34, 38, 40, 41, 44, 45, 48, 54, 55, 60], "combin": [2, 4, 25, 41, 54], "similar": [2, 3, 7, 8, 11, 15, 16, 22, 29, 30, 33, 34, 38, 41, 42, 45, 46, 48, 49, 52, 54, 61, 62], "resourc": [2, 3, 7, 18, 19, 31, 41, 55, 59], "annual": [2, 22, 29, 38, 41, 43], "match": [2, 4, 33, 43, 48, 50, 58], "gender": [2, 5, 7, 11, 22, 30, 33, 38, 42, 43, 48, 49], "live": [2, 5, 8, 12, 50, 53], "nearli": [2, 7, 18, 19, 30, 31, 42], "guarante": [2, 7, 8, 60], "still": [2, 5, 7, 8, 11, 21, 22, 25, 28, 30, 38, 40, 42, 43, 53, 55], "wrangl": 2, "ahead": [2, 7, 21, 31, 55], "won": [2, 5, 7, 21, 23, 29, 39, 41, 48, 49, 50, 51], "templat": 2, "great": [2, 5, 6, 7, 8, 12, 16, 18, 22, 38, 46, 48, 51, 54, 57, 60], "superiv": 2, "access": [2, 7, 11, 28, 30, 42, 55, 60], "pretti": [2, 3, 12, 17, 29, 41, 42], "nuanc": [2, 46, 62], "super": [2, 3, 42, 43, 61], "role": [3, 5, 28, 29, 31, 41, 50, 52, 60], "infer": [3, 4, 5, 8, 12, 15, 18, 19, 22, 29, 33, 34, 38, 41, 43, 46, 47, 48, 57, 62], "industri": [3, 10, 11, 18, 29, 41, 43, 46, 48, 50], "question": [3, 4, 5, 7, 8, 11, 12, 22, 25, 26, 27, 28, 29, 30, 31, 35, 38, 40, 41, 42, 44, 45, 46, 47, 48, 55, 57, 62], "tech": [3, 5], "sector": [3, 29, 41, 43], "todai": [3, 13, 17, 22, 35, 38, 47, 60], "constantli": [3, 49], "refin": [3, 21, 41], "product": [3, 5, 8, 12, 25, 28, 41, 48, 62], "target": [3, 5, 12, 16, 25, 28, 31, 52, 60], "ad": [3, 8, 11, 24, 28, 29, 30, 38, 39, 41, 42, 43, 46, 48, 50, 61], "increment": [3, 5], "innov": [3, 5, 7, 17, 22, 38, 46, 60], "definit": [3, 5, 21, 48, 50, 60, 62], "toolbox": 3, "read": [3, 4, 5, 6, 7, 8, 15, 18, 19, 22, 25, 26, 27, 28, 29, 30, 34, 38, 40, 41, 47, 48, 51, 52, 60, 62], "discuss": [3, 5, 7, 8, 15, 16, 17, 29, 33, 34, 41, 43, 45, 46, 48, 49, 50, 52, 60, 62], "critic": [3, 5, 7, 8, 30, 38, 42, 46, 48, 49, 57, 59, 62], "gather": 3, "passiv": [3, 4, 17, 46, 59, 60, 61], "somebodi": [3, 7, 54], "els": [3, 18, 19, 59, 62], "rather": [3, 7, 8, 9, 24, 25, 31, 39, 43, 44, 45, 49, 55, 57, 60, 62], "manipul": [3, 7, 11, 47, 55], "random": [3, 4, 6, 7, 8, 11, 14, 21, 25, 28, 30, 42, 45, 47, 50, 52, 54, 62], "arm": [3, 11, 28, 30, 41, 42, 55], "low": [3, 5, 14, 21, 22, 25, 28, 30, 38, 42, 44, 45, 48, 50], "cost": [3, 14, 17, 20, 22, 23, 27, 37, 38, 53, 54], "expens": [3, 6, 37, 54, 55], "worth": [3, 5, 7, 21, 22, 23, 30, 37, 38, 42, 45, 49, 52, 60], "possibl": [3, 5, 7, 8, 14, 16, 18, 19, 25, 26, 27, 30, 31, 40, 42, 45, 46, 52, 54, 57, 60, 62], "decis": [3, 4, 5, 7, 21, 31, 50, 51, 52, 53, 54, 56, 62], "hand": [3, 5, 24, 39, 41, 45, 46, 48, 50, 53], "situat": [3, 7, 8, 28, 42, 45, 48, 51, 52, 53, 60], "specif": [3, 5, 7, 8, 11, 16, 24, 26, 27, 31, 32, 39, 40, 44, 46, 48, 52, 54, 60, 61, 62], "facebook": [3, 8, 50], "twitter": 3, "satisfact": 3, "choos": [3, 6, 10, 21, 22, 38, 40, 43, 52, 54], "certain": [3, 7, 8, 21, 22, 31, 38, 50, 52, 53, 60, 62], "function": [3, 5, 9, 11, 21, 22, 24, 25, 30, 33, 38, 39, 42, 43, 49, 52], "your": [3, 4, 5, 6, 8, 9, 12, 13, 14, 15, 17, 18, 19, 20, 21, 23, 24, 25, 26, 27, 28, 29, 30, 32, 34, 35, 36, 37, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 55, 57, 59, 60], "ideal": [3, 21, 54, 60], "block": [3, 18, 50, 55], "One": [3, 11, 12, 16, 21, 22, 26, 27, 29, 38, 39, 40, 41, 42, 43, 44, 53, 61, 62], "shot": 3, "bowl": [3, 41], "store": [3, 8, 11, 14, 16, 25, 28, 30, 41, 43, 44, 45, 48, 52, 60, 62], "n": [3, 5, 24, 28, 35, 37, 39, 41, 42], "event": [3, 14, 60], "strateg": 3, "scheme": 3, "anoth": [3, 5, 7, 8, 16, 17, 21, 22, 29, 38, 41, 47, 50, 55, 60, 61, 62], "entir": [3, 5, 8, 12, 16, 48, 60, 62], "price": [3, 20, 28, 37], "wouldn": [3, 7, 28, 32, 38, 62], "stand": [3, 28, 31, 46, 49, 50, 60], "similarli": [3, 7, 8, 50, 55, 60], "sometim": [3, 5, 7, 11, 25, 46, 49, 52], "rollout": 3, "launch": [3, 31, 53], "titl": [3, 30, 34, 42], "netflix": [3, 4, 6], "premier": 3, "movi": 3, "interfer": 3, "abstract": [3, 26, 27, 28, 40], "talk": [3, 6, 7, 8, 13, 14, 22, 35, 36, 38, 43, 48, 49, 51, 60, 62], "constraint": [3, 21, 25, 36, 43, 49], "horizon": 3, "too": [3, 4, 5, 7, 9, 22, 26, 28, 32, 38, 40, 42, 43, 47, 48, 51, 53, 54, 60, 61, 62], "popular": [3, 21, 45], "quickli": [3, 7, 20, 22, 37, 38, 46, 53, 62], "short": [3, 8, 23, 48, 54, 57], "countri": [3, 12, 28, 31, 48, 50, 55, 60], "put": [3, 8, 14, 22, 23, 24, 26, 27, 38, 39, 40, 44, 49, 52, 53, 60, 62], "pick": [3, 7, 21, 22, 38, 49, 51, 52, 54, 61], "locat": [3, 5, 18, 19, 44, 45, 53, 55], "setup": 3, "shop": [3, 21, 41], "wait": [3, 5, 7, 52], "sale": [3, 8, 13, 23, 35, 44, 45, 48, 50, 62], "sens": [3, 5, 7, 15, 16, 21, 22, 24, 25, 28, 30, 34, 38, 39, 42, 44, 47, 52, 54, 55, 60, 61], "kohavi": [3, 4, 11], "tang": [3, 4, 11], "xu": [3, 4, 11], "overal": [3, 4, 11, 13, 16, 30, 31, 35, 39, 42, 43, 54], "criterion": [3, 11, 54], "oec": [3, 11, 54], "durat": [3, 60], "believ": [3, 7, 24, 39, 48, 54], "drive": [3, 7, 12, 16, 17, 24, 39, 48, 60], "shortest": 3, "meaning": [3, 5, 8, 21, 57], "option": [3, 4, 11, 14, 28, 31, 43], "mani": [3, 7, 11, 12, 13, 19, 21, 22, 23, 24, 25, 28, 29, 31, 35, 38, 39, 41, 43, 45, 46, 47, 48, 49, 52, 55, 57, 60, 61, 62], "face": [3, 5, 7, 10, 17, 31, 46, 48, 53], "social": [3, 7, 22, 28, 31, 33, 38, 41, 43, 45, 46, 48, 50, 54, 55, 60, 61], "media": [3, 34, 35, 39], "harass": 3, "retent": 3, "mind": [3, 5, 7, 14, 21, 27, 29, 30, 41, 42, 45, 46, 48, 49, 57, 62], "2012": [3, 33, 37], "emot": 3, "valenc": 3, "saw": [3, 8, 11, 49, 60], "happi": [3, 8, 28], "sad": 3, "mood": 3, "spill": 3, "over": [3, 5, 6, 7, 8, 13, 16, 20, 22, 23, 24, 28, 30, 33, 35, 37, 38, 39, 42, 43, 44, 45, 48, 49, 51, 54, 55, 60, 61, 62], "network": [3, 17, 30, 42, 54], "toi": [3, 41], "furiou": 3, "everyth": [3, 4, 7, 11, 22, 23, 28, 38, 43, 45, 46, 48, 54, 60, 62], "consum": [3, 8, 16, 20, 27, 28, 37, 41, 46, 48, 52, 55, 62], "mayb": [3, 4, 11, 42, 52, 62], "deliber": [3, 6], "harm": 3, "huge": [3, 5, 22, 28, 38, 46, 48, 55, 60], "experiment": [3, 11, 28, 42], "matter": [3, 5, 8, 21, 22, 38, 40, 44, 46, 52, 54, 61, 62], "issu": [3, 5, 7, 11, 20, 21, 23, 24, 25, 26, 30, 37, 39, 40, 46, 48, 50, 60, 61], "okcupid": 3, "publish": [3, 20, 37, 41, 60], "research": [3, 7, 8, 10, 16, 21, 22, 28, 31, 32, 36, 38, 41, 46, 48, 50, 52, 53, 55, 59, 60, 62], "date": [3, 4, 11, 25, 35, 37, 38, 39, 41, 42, 43, 49, 50], "profil": 3, "honestli": [3, 60], "standard": [3, 5, 9, 11, 12, 13, 15, 21, 22, 23, 24, 25, 28, 30, 34, 35, 37, 38, 39, 41, 42, 43, 48, 49, 50, 51, 54, 62], "mess": 3, "heart": [3, 14, 48], "care": [3, 7, 8, 11, 21, 27, 29, 30, 36, 41, 42, 43, 48, 50, 52, 54, 55, 60, 61, 62], "improv": [3, 7, 11, 16, 29, 30, 32, 41, 42, 50, 60, 61], "younger": [3, 29, 41], "woker": 3, "offer": [3, 7, 8, 11, 12, 14, 23, 27, 28, 29, 33, 41, 45, 46, 48, 55, 60], "childcar": [3, 41], "subsidi": 3, "workforc": [3, 5, 29, 41], "union": [3, 12, 41], "benefit": [3, 30, 50, 54], "contract": [3, 18], "scale": [3, 15, 28, 30, 34, 35, 37, 38, 48], "larger": [3, 8, 11, 13, 29, 31, 35, 40, 41, 44, 45, 49, 55], "larg": [3, 6, 8, 11, 12, 24, 27, 28, 30, 31, 32, 37, 39, 42, 45, 46, 48, 50, 52, 53], "area": [3, 7, 14, 16, 17, 25, 32, 45, 51, 52, 53, 55, 57, 61], "increasingli": 3, "outsid": [3, 14, 52], "administ": 3, "program": [3, 16, 27, 28, 31, 41, 47, 48, 53, 59, 61], "servic": [3, 11, 16, 18, 25, 41, 43, 48, 55, 61], "pai": [3, 10, 11, 12, 13, 14, 23, 28, 29, 35, 41, 48], "vendor": 3, "base": [3, 6, 7, 11, 15, 20, 21, 22, 25, 28, 34, 37, 38, 43, 46, 48, 54, 55], "input": [3, 7, 16, 22, 33, 38, 43, 60], "hour": [3, 4, 11, 29, 41, 59], "spent": [3, 7, 8, 9, 20, 22, 24, 36, 38], "encourag": [3, 4, 16], "paid": [3, 11, 25, 29, 41], "basi": [3, 5, 7, 13, 16, 25, 35, 60], "incent": 3, "cut": [3, 7, 21, 22, 38, 41, 46], "risk": [3, 7, 8, 16, 17, 30, 55], "arrang": [3, 41], "hospit": [3, 5, 17, 41, 43], "infect": [3, 5, 12, 18, 19], "reduct": [3, 5, 32, 50], "minim": [3, 5, 7, 21, 28, 54, 61], "downtim": 3, "intern": [3, 4, 12, 17, 30, 41, 42, 53, 55, 57, 60], "ty": 3, "compens": [3, 29, 41], "parti": [3, 31, 42, 49], "incentiv": 3, "precis": [3, 7, 11, 13, 18, 25, 35, 37, 40, 41, 42, 45, 48, 62], "effort": [3, 5, 13, 21, 23, 24, 35, 39, 53, 60], "averag": [3, 5, 7, 8, 10, 11, 12, 13, 15, 16, 23, 24, 26, 27, 28, 29, 30, 31, 34, 35, 37, 39, 40, 41, 42, 44, 45, 48, 49, 52, 62], "suit": 3, "circumst": 3, "unfortun": [3, 7, 25], "befor": [3, 4, 5, 7, 8, 11, 13, 14, 22, 25, 28, 29, 30, 33, 35, 36, 38, 41, 43, 49, 50, 51, 52, 53, 60, 62], "finish": [3, 4, 38, 41, 51], "natur": [3, 5, 31, 41, 48, 53, 55, 61], "season": [3, 23, 54], "expect": [3, 8, 11, 13, 16, 18, 19, 21, 23, 26, 28, 29, 30, 35, 38, 40, 41, 42, 47, 57, 62], "vari": [3, 11, 25, 29, 33, 39, 41, 44, 49, 50, 62], "anyth": [3, 5, 7, 12, 18, 19, 21, 28, 33, 43, 44, 45, 51, 54, 57, 62], "design": [3, 4, 8, 11, 13, 17, 21, 22, 29, 30, 35, 38, 41, 42, 43, 45, 47, 48, 52, 53, 57, 60], "g": [3, 5, 7, 8, 11, 12, 13, 15, 16, 21, 22, 25, 28, 30, 34, 35, 37, 38, 41, 42, 46, 47, 48, 54, 55, 60, 62], "upward": 3, "analyst": [3, 11], "extrem": [3, 7, 26, 28, 40, 42, 44, 45, 46, 50, 52, 60], "friendli": 3, "easier": [3, 7, 13, 21, 23, 35, 57, 61], "duck": 3, "goos": 3, "rel": [3, 5, 25, 28, 30, 36, 42, 50, 52, 60, 62], "difficulti": [3, 30, 40, 42, 47], "task": [3, 11, 18, 19, 22, 25, 38], "length": [3, 20, 22, 33, 37, 38, 43, 50, 57], "extern": [3, 4, 30, 42, 60, 62], "valid": [3, 4, 8, 16, 18, 19, 23, 25, 30, 33, 35, 42, 60, 62], "estim": [3, 5, 7, 8, 10, 13, 16, 21, 22, 23, 24, 26, 27, 28, 35, 38, 39, 40, 43, 44, 45, 48, 49, 50, 52, 55, 60], "wide": [3, 46], "thu": [3, 5, 7, 11, 12, 17, 21, 28, 33, 40, 41, 44, 48, 60, 62], "differenti": [3, 7, 12, 14, 25, 29, 30, 41, 42, 45], "market": [3, 16, 28, 41, 60], "shouldn": [3, 7, 11, 22, 30, 33, 38, 42, 45, 52, 60], "appli": [3, 5, 11, 13, 15, 33, 34, 35, 36, 45, 46, 47, 48, 52, 60], "logic": [3, 12, 30], "almost": [3, 5, 7, 18, 19, 22, 30, 38, 42, 45, 48, 50, 51, 53, 55, 57, 60], "inevit": 3, "interact": [3, 13, 21, 23, 30, 35, 42, 48, 50, 54], "consider": [3, 7, 28, 53, 60], "colleagu": [3, 14], "tuesdai": 4, "thursdai": 4, "25": [4, 22, 23, 28, 33, 37, 38, 39, 41, 48], "2": [4, 8, 26, 33, 40, 43, 50, 52], "40pm": 4, "offic": [4, 13, 26, 31, 33, 40, 41, 43], "nick": [4, 33, 49], "00": [4, 34, 35, 37, 38, 39, 41, 42, 43, 49], "00am": 4, "http": [4, 13, 15, 20, 22, 24, 29, 33, 34, 35, 37, 38, 39, 41, 42, 43, 49], "duke": [4, 5, 14, 16, 22, 26, 30, 31, 38, 40, 42, 47, 48, 55], "zoom": [4, 27], "nickeubank": [4, 13, 15, 20, 22, 24, 29, 34, 35, 37, 38, 39, 41, 42, 43, 49], "gross": 4, "hall": [4, 26, 40], "231": [4, 37, 38, 41], "echo": 4, "fridai": 4, "3pm": 4, "kashaf": 4, "4pm": 4, "pragya": 4, "wednesdai": 4, "topic": [4, 7, 52, 57, 60], "thr": 4, "jan": 4, "overview": [4, 11, 62], "acm": [4, 16], "propos": [4, 11, 13, 17, 35, 36, 53], "tue": [4, 37], "16": [4, 33, 34, 35, 37, 39, 41, 42, 43, 48, 49, 52], "problem": [4, 5, 7, 8, 9, 11, 12, 16, 17, 21, 25, 31, 36, 37, 41, 44, 46, 48, 50, 52, 54, 59, 60, 62], "sign": [4, 7, 48], "submit": [4, 7, 11, 16, 25, 30, 31, 57], "syllabu": 4, "gradescop": 4, "18": [4, 20, 22, 28, 33, 35, 37, 38, 39, 41, 42, 43, 49], "histori": [4, 5, 31, 60], "30": [4, 8, 11, 22, 25, 28, 31, 33, 35, 38, 39, 41, 50], "min": [4, 35, 38, 39, 43], "video": [4, 5, 7, 11, 21, 41, 48, 51], "23": [4, 15, 34, 35, 38, 39, 41, 42, 49], "proscript": [4, 60], "v": [4, 7, 28, 31], "descript": [4, 11, 15, 22, 36, 38, 53], "versu": [4, 16, 22, 36, 38], "prescript": [4, 60], "exploratori": [4, 17, 60], "eda": 4, "write": [4, 9, 16, 28, 31, 33, 46, 47, 49, 53, 57, 61], "notebook": [4, 11, 25, 30], "team": [4, 12, 16, 31, 36, 47, 61], "explor": [4, 5, 9, 11, 16, 52, 55, 60], "feb": [4, 37, 43, 49], "aristotl": 4, "edmondson": 4, "fearless": 4, "organ": [4, 11, 12, 13, 18, 19, 28, 33, 35, 41, 43, 46, 53, 55, 62], "chpt": 4, "canva": 4, "review": [4, 5, 11, 16, 31, 49, 54, 60], "charter": 4, "cognit": 4, "barrier": [4, 31], "p": [4, 6, 11, 23, 28, 30, 35, 37, 38, 39, 41, 42, 43, 49, 54, 56], "82": [4, 33, 38, 41], "chapter": [4, 48], "foster": 4, "psycholog": 4, "safeti": [4, 17, 41], "tip": 4, "ex": 4, "13": [4, 9, 10, 26, 27, 33, 34, 35, 37, 39, 40, 42, 49], "stop": [4, 5, 22, 38, 47, 48, 52, 53, 54], "box": [4, 21, 22, 31, 38, 41, 49, 60], "trust": [4, 9, 11, 21, 41, 45, 49, 62], "contrain": 4, "artifici": [4, 47, 48], "stupid": [4, 53], "explicit": [4, 33, 49], "scientif": [4, 7, 41, 60], "ml": [4, 7, 16, 21], "15": [4, 5, 15, 33, 34, 35, 37, 39, 41, 42, 49, 50, 51], "gam": [4, 21], "silver": 4, "bullet": 4, "tour": [4, 60], "pygam": [4, 21], "20": [4, 21, 22, 23, 28, 33, 34, 35, 37, 38, 41, 42, 43, 49, 51], "22": [4, 23, 31, 37, 38, 41, 42, 43], "27": [4, 34, 38], "cunningham": 4, "pp": 4, "135": [4, 33, 37, 38, 41], "independ": [4, 12, 20, 37, 41, 42, 47, 49, 50, 52, 54], "148": [4, 38, 39, 41], "sdo": 4, "widehat": 4, "ATE": [4, 8, 11, 30, 38, 42], "present": [4, 8, 16, 25, 50, 53, 57, 61, 62], "29": [4, 22, 33, 38, 39, 49], "kohvani": 4, "19": [4, 21, 22, 33, 35, 37, 38, 39, 41, 42, 43], "21": [4, 23, 33, 34, 37, 38, 41, 42], "guardrail": [11, 54], "metric": [11, 48, 54], "threat": [4, 30, 42, 44], "42": [4, 25, 38, 39, 41], "47": [4, 33, 34, 35, 37, 38, 39, 41, 49], "ktx": [], "mar": [4, 35, 37, 38, 39, 41, 42], "54": [4, 34, 37, 38, 39, 43], "kohvai": 4, "primaci": [4, 54], "decai": 4, "39": [22, 33, 35, 38, 39, 41, 42, 43, 49], "unless": [6, 7, 8, 11, 54], "method": [5, 7, 16, 20, 21, 22, 23, 24, 25, 28, 29, 33, 35, 37, 38, 39, 41, 42, 43, 49, 52, 54], "power": [4, 7, 11, 22, 24, 38, 39, 41, 42, 43, 44, 49, 50, 51, 54, 60], "calcul": [4, 5, 11, 13, 15, 22, 23, 25, 29, 30, 34, 35, 38, 41, 42, 43, 45, 48, 49, 50, 51, 54], "12": [4, 26, 27, 28, 33, 34, 35, 37, 39, 40, 43, 49, 52], "NO": [4, 49], "14": [4, 26, 27, 31, 33, 35, 37, 39, 40, 42, 43, 49, 51], "bayesian": 4, "ab": [4, 6, 11, 41, 54], "midterm": [4, 62], "26": [4, 33, 34, 35, 37, 38], "28": [4, 33, 37, 38, 39, 42, 52], "fix": [4, 13, 29, 35, 41, 48, 58], "apr": 4, "panel": [4, 24, 39, 43, 44], "angrist": 4, "pisck": 4, "mm": 4, "178": [4, 33, 38, 41], "208": [4, 35, 37, 38, 41], "diff": [4, 13, 35, 58], "dont": 4, "enfranchis": 4, "ethic": [4, 7, 28, 54], "mayson": 4, "2281": 4, "extra": [4, 49, 60], "ai": [4, 47], "misalign": 4, "adversari": [4, 7], "fool": [4, 7], "openai": 4, "robograd": 4, "text": [4, 7, 21, 39, 46, 47], "referenc": 4, "trustworthi": 4, "onlin": [4, 11, 23, 30, 42, 55], "guid": [4, 5, 61], "analys": [5, 9, 29, 41, 52, 54, 55, 60, 61, 62], "extract": [5, 41], "pattern": [5, 30, 42, 60], "otherwis": [5, 20, 21, 37, 43, 49, 51, 60], "incomprehensibli": 5, "messi": [5, 29, 41], "reader": [5, 8, 62], "digest": 5, "manner": [5, 12, 13, 25, 35, 46, 57, 60], "intrins": 5, "report": [5, 7, 8, 11, 20, 22, 25, 26, 29, 30, 31, 36, 37, 38, 40, 41, 42, 48, 49, 50, 55, 61], "safe": [5, 21, 25, 30], "discard": 5, "word": [5, 7, 8, 10, 11, 12, 13, 20, 21, 25, 26, 27, 28, 29, 31, 33, 35, 37, 38, 40, 41, 43, 44, 49, 50, 51, 52, 60, 62], "entail": [5, 16, 21, 60], "asid": [5, 50], "origin": [5, 7, 11, 20, 21, 22, 25, 30, 37, 38, 42, 46, 48, 51, 52], "summar": [5, 62], "summari": [5, 9, 21, 22, 35, 37, 38, 39, 41, 42, 43, 46, 49, 59], "plot": [5, 6, 7, 9, 13, 15, 16, 20, 21, 22, 24, 34, 35, 37, 38, 39, 53], "distribut": [5, 7, 13, 21, 22, 23, 35, 38, 39, 41, 43, 50, 51, 52], "sophist": [5, 48, 55, 60], "cluster": [5, 13, 18, 19, 22, 24, 35, 38, 39], "dimension": [5, 51], "therefor": [5, 13, 35], "herein": [5, 50], "li": [5, 47, 50], "hazard": 5, "presum": [5, 11, 28], "irrelev": 5, "incumb": 5, "upon": [5, 57, 60], "highlight": [5, 46], "struggl": [5, 9, 20, 22, 24, 27, 31, 38, 49, 62], "deviat": [5, 9, 15, 22, 23, 28, 34, 38, 44, 51, 62], "unsupervis": [5, 60], "machin": [5, 16, 17, 18, 22, 38, 41, 47, 48, 49, 53, 60], "principl": [5, 29, 41, 43, 45], "compon": [5, 13, 18, 19, 26, 27, 35, 40, 41, 53, 61], "pca": 5, "academ": [5, 30, 31, 42, 43, 48], "peer": [5, 31], "must": [5, 7, 12, 18, 19, 21, 26, 27, 29, 40, 41, 53, 60, 61], "frac": [5, 24, 39], "sigma": 5, "x_i": 5, "central": [5, 14, 31, 60], "tendenc": 5, "paper": [5, 7, 21, 22, 28, 30, 38, 41, 46, 48, 49, 53, 59], "neutral": 5, "pipelin": [5, 16, 41], "deliv": [5, 47], "choic": [5, 11, 22, 24, 28, 38, 39, 49, 52], "taken": [5, 11, 23, 25, 42, 44, 53], "signal": [5, 7], "implicitli": [5, 18, 19, 29, 41], "With": [5, 7, 11, 13, 14, 16, 21, 25, 27, 35, 42, 45, 46, 48, 49, 57, 61, 62], "whenev": [5, 7, 11, 48, 60], "am": [5, 53], "faithfulli": 5, "full": [5, 8, 11, 16, 24, 28, 29, 30, 39, 41, 42, 43, 47, 52, 53, 54, 55, 62], "incomprehens": 5, "raw": [5, 13, 22, 29, 33, 37, 38, 41, 42, 43, 49, 61], "eventu": [5, 11, 13, 31, 35], "essenti": [5, 10, 53], "themselv": [5, 43], "give": [5, 7, 8, 12, 14, 15, 16, 17, 21, 22, 24, 26, 27, 28, 31, 34, 38, 39, 40, 41, 45, 47, 48, 49, 50, 51, 55, 57, 60, 61, 62], "receiv": [5, 7, 8, 11, 12, 28, 30, 31, 48, 61, 62], "professor": [5, 62], "exam": 5, "score": [5, 7, 15, 25, 26, 34, 40, 43, 44, 45, 47, 51], "grade": [5, 7, 11, 25, 30, 43, 57, 60], "85": [5, 33, 35, 38, 42, 49], "sound": [5, 13, 30, 35, 37, 41, 42], "dig": [5, 8, 44, 53], "bimod": 5, "argu": [5, 11, 13, 25, 35, 48, 53, 60, 62], "somewhat": [5, 20, 37, 49, 51], "around": [5, 7, 18, 19, 21, 24, 30, 37, 39, 41, 48, 50, 60, 61], "occur": [5, 11, 12, 16, 25, 28, 31, 33, 60, 61, 62], "teach": [5, 7], "suspect": [5, 20, 37, 43], "fail": [5, 11, 22, 38, 47, 60], "worst": 5, "incom": [5, 7, 8, 21, 25, 28, 29, 39, 41, 43, 44, 45, 47, 50, 53, 55, 58, 60, 62], "rose": 5, "earn": [5, 22, 29, 38, 41, 43, 55], "000": [5, 8, 13, 21, 22, 23, 24, 28, 35, 37, 38, 39, 41, 42, 43, 48, 49, 50], "hear": [5, 7, 45, 52], "x_2": 5, "bin": [5, 34, 38, 60], "y_2": 5, "thousand": 5, "dollar": [5, 8, 23, 27, 28, 29, 37, 41], "yup": [5, 8, 9, 38, 60], "fit": [5, 7, 11, 14, 22, 24, 25, 35, 37, 38, 39, 41, 42, 43, 47, 49, 52, 60], "technic": [5, 25, 41, 43, 45, 50, 52, 57], "accur": [5, 22, 29, 38, 41, 48, 50, 60], "captur": [5, 16, 37, 54], "trend": [5, 13, 35, 62], "ten": 5, "monoton": [5, 7, 16, 21, 36], "protect": [5, 7, 31, 55, 61], "kind": [5, 7, 8, 11, 12, 28, 31, 43, 44, 48, 49, 51, 52, 55], "error": [5, 7, 11, 13, 21, 22, 24, 25, 30, 35, 37, 38, 39, 41, 42, 43, 48, 49, 52, 54], "residu": [5, 11, 30, 35, 37, 38, 39, 41, 42, 43, 49], "histogram": [5, 15, 34], "rule": [5, 11, 31, 49, 54, 60, 61], "thumb": 5, "beyond": [5, 18, 19, 31, 61], "median": [5, 13, 35], "radic": 5, "check": [5, 6, 7, 11, 21, 25, 28, 33], "primari": [5, 28, 31, 41, 43, 60], "stori": [5, 7, 8, 26, 27, 40, 48, 52, 62], "miss": [5, 7, 21, 22, 25, 28, 29, 37, 38, 41, 43], "point": [5, 6, 7, 8, 11, 13, 15, 21, 22, 23, 25, 28, 30, 31, 34, 35, 38, 42, 43, 46, 49, 51, 52, 57, 62], "substant": [5, 7, 21, 46, 47, 49], "behind": [5, 21, 53], "notic": [5, 11, 12, 18, 57, 62], "properti": [5, 25, 33], "haven": [5, 8, 21, 42, 43, 51, 60], "defin": [5, 13, 14, 16, 24, 26, 27, 35, 39, 40, 50], "system": [5, 7, 13, 17, 25, 35, 41, 43, 60], "antibiot": 5, "resist": [5, 46], "brought": [5, 52, 60], "facil": [5, 41, 43], "director": 5, "remov": [5, 7, 25, 26, 35, 40, 44], "fabric": [5, 41], "chair": 5, "disinfect": 5, "prescrib": 5, "impact": [5, 7, 11, 13, 16, 22, 24, 28, 35, 38, 39, 40, 43, 48, 50, 52, 53, 62], "instead": [5, 7, 11, 22, 23, 25, 26, 27, 28, 38, 40, 43, 44, 48, 49, 50, 52, 53, 60], "futur": [5, 31, 60], "respons": [5, 8, 14, 16, 18, 19, 30, 31, 42, 48, 50, 57], "biggest": [5, 13, 35, 46, 50, 51], "contributor": 5, "rais": [5, 22, 33, 38, 57], "livestock": [5, 28], "societi": 5, "subtl": [5, 46, 60], "less": [5, 7, 8, 13, 14, 20, 21, 22, 25, 26, 27, 28, 29, 30, 35, 37, 38, 40, 41, 42, 45, 46, 48, 49, 50, 51, 52, 62], "maker": [5, 50, 62], "co_2": 5, "emiss": 5, "95": [5, 21, 23, 34, 38, 57], "unemploy": [5, 39, 44, 60], "profit": [5, 23, 31, 38, 47, 61], "million": [5, 22, 23, 28, 38], "tax": [5, 24, 32, 39, 41, 50, 60], "moder": 5, "200": [5, 28, 38, 41], "poverti": [5, 28], "line": [5, 9, 21, 22, 24, 38, 39], "proport": [5, 10, 22, 28, 38], "tend": [5, 7, 10, 12, 13, 17, 21, 22, 23, 30, 35, 37, 38, 40, 41, 42, 43, 44, 45, 50, 60, 61], "feel": [5, 7, 9, 11, 13, 25, 28, 33, 35, 36, 45, 46, 49, 59, 60], "lack": [5, 18, 19, 31, 57], "divers": [5, 31, 46], "men": [5, 7, 10, 29, 30, 41, 42, 48, 49, 50, 60], "misogynist": 5, "life": [5, 7, 18, 19, 21, 50], "influenc": [5, 49, 61], "male": [5, 28, 29, 33, 41, 43, 49], "infam": 5, "thankfulli": [5, 7, 22, 38, 43, 62], "stake": [5, 7, 21, 53], "failur": [5, 47], "watch": [5, 6, 7, 30, 42, 51], "camera": 5, "went": [5, 8, 23, 41, 62], "hp": 5, "did": [5, 11, 13, 18, 21, 23, 25, 28, 33, 35, 36, 47, 52, 54, 57, 60, 62], "ship": [5, 41, 55], "silli": [5, 37], "web": [5, 41], "cam": 5, "forefront": [5, 7], "technolog": 5, "seriou": [5, 17, 48], "mistak": 5, "oh": [5, 8, 39, 55, 60], "googl": [5, 48, 55], "Their": 5, "tag": [5, 24, 33, 39], "dark": 5, "skin": 5, "gorilla": 5, "releas": [5, 22, 32, 38, 50], "reflect": [5, 22, 25, 38, 43, 45, 46, 50, 55, 62], "No": [5, 8, 30, 35, 37, 38, 39, 41, 42, 43, 49, 53, 54, 62], "hei": 5, "roll": [5, 6, 8, 11, 28, 41, 54], "behav": [5, 7], "predominantli": [5, 48], "light": [5, 31, 41, 60], "ye": [5, 8, 21, 42, 52], "sadli": 5, "unfairli": 5, "sent": [5, 30, 42, 48, 62], "jail": [5, 13, 35], "discrimin": [5, 7, 10, 22, 30, 31, 38, 42, 43], "against": [5, 7, 11, 20, 22, 23, 25, 30, 37, 38, 42, 50], "per": [5, 8, 11, 13, 20, 24, 25, 28, 29, 33, 35, 37, 39, 41, 44, 49, 57], "se": [5, 35, 43, 49, 57], "press": [5, 7, 16, 62], "especi": [5, 7, 9, 21, 22, 25, 33, 38, 45, 46, 48, 50, 60, 62], "salienc": 5, "rememb": [5, 7, 8, 11, 25, 26, 27, 30, 40, 48, 49, 50, 54, 60, 61, 62], "odd": [5, 8, 50, 51], "temptat": [6, 46], "known": [6, 7, 11, 40], "boss": [6, 12, 48], "yesterdai": [6, 17], "medic": [6, 7, 14, 27, 30, 41, 48, 50, 55], "trial": [6, 11, 48, 50], "soon": [6, 7, 8, 28, 48, 59], "experienc": [6, 10, 30, 34], "side": [6, 7, 12, 16, 22, 24, 28, 38, 39], "intermedi": 6, "render": [6, 34], "invalid": 6, "basic": [6, 7, 13, 21, 22, 24, 28, 29, 35, 38, 39, 41, 42, 43, 47, 51, 52, 60, 62], "fluctuat": [6, 13, 35], "enough": [6, 8, 11, 14, 22, 23, 32, 38, 46, 48, 52, 54, 57, 60], "probabilist": 6, "outlier": [6, 9], "moment": [6, 21, 29, 30, 41, 42, 46, 50, 60], "ramesh": 6, "johari": 6, "leo": 6, "peke": 6, "david": 6, "walsh": 6, "fake": [6, 31], "condit": [6, 8, 11, 12, 20, 23, 26, 27, 31, 37, 40, 41, 44, 48, 52], "sever": [6, 13, 16, 24, 35, 39, 43, 44, 45, 53, 60, 61], "dai": [4, 6, 7, 11, 25, 26, 28, 31, 36, 41, 48, 50, 54, 57], "chosen": [6, 10, 28, 29, 31, 41], "hit": [6, 22, 38], "deep": [6, 16, 22, 38], "troubl": [6, 52], "fact": [6, 7, 9, 12, 13, 18, 19, 21, 22, 28, 29, 33, 35, 37, 38, 41, 43, 44, 45, 48, 50, 53, 62], "supervis": [7, 53, 60], "discourag": [7, 10], "inde": [7, 9, 21, 28, 45, 46, 49, 60], "cover": [7, 21, 47, 52, 55, 60, 62], "construct": [7, 9, 16, 24, 32, 39, 41], "action": [7, 16, 50, 54, 60], "deploi": 7, "discriminatori": 7, "uncontest": 7, "grip": [7, 11], "approxim": [7, 12, 28, 43], "doubt": 7, "previou": [7, 29, 30, 35, 41, 42], "emul": 7, "process": [7, 11, 18, 22, 38, 41, 46, 48, 51, 53, 57, 59, 60], "attempt": [7, 11, 12, 16, 22, 38, 60], "systemat": [7, 28, 47], "inaccur": [7, 21], "relat": [7, 8, 12, 14, 22, 30, 38, 41, 44, 46, 61, 62], "thoughtfulli": [7, 57], "down": [7, 12, 16, 21, 22, 31, 33, 38, 41, 45, 51, 53, 54, 62], "downstream": 7, "leav": [7, 12, 22, 38], "fight": 7, "own": [7, 11, 12, 21, 25, 46, 47, 53, 59, 60, 61], "flavor": [7, 25, 28], "h": 7, "michael": 7, "akand": 7, "unforeseen": 7, "uneth": [7, 12], "recapitul": [7, 62], "fulli": [7, 13, 21, 35, 46, 48, 50, 54, 60], "underli": [7, 11, 22, 38, 44], "advantag": [7, 16, 22, 38], "intepret": 7, "eleph": 7, "room": [7, 41], "trade": [7, 18, 22, 38, 41, 43, 55, 61], "off": [7, 8, 21, 22, 28, 30, 38, 42, 45, 60, 62], "simplic": [7, 44], "accuraci": [7, 21], "tree": [7, 29, 41], "nice": [7, 15, 22, 34, 38, 49], "compet": 7, "vast": 7, "fanci": [7, 9, 14], "substanti": [7, 14, 15, 27, 41, 45], "applic": [7, 25, 30, 42, 48, 51, 52, 54, 62], "claim": [7, 28, 48], "crack": 7, "formal": [7, 8], "unsolv": 7, "win": 7, "crazi": 7, "complex": [7, 22, 38, 46, 57], "articl": [7, 30, 42, 62], "self": [7, 38], "car": [7, 17, 20, 24, 31, 37, 39, 41, 48, 49, 61], "protein": 7, "fold": [7, 60], "edg": [7, 21], "comput": [7, 11, 21, 22, 24, 26, 30, 38, 39, 40, 41, 42, 43, 46, 49, 55], "vision": 7, "necessari": [7, 16, 21, 27, 28, 48, 51, 60], "folk": [7, 55], "competit": [7, 16, 21, 61], "consult": [7, 12, 16, 41], "necessarili": [7, 10, 46, 55], "career": [7, 57], "bigger": [7, 37, 40, 52], "maxim": [7, 21, 48], "bear": [7, 29, 30, 41, 42, 46, 47, 48, 57, 60, 62], "maintain": [7, 25], "chanc": [7, 11, 28], "direct": [7, 11, 16, 20, 28, 31, 37, 41, 51], "mainten": [7, 41], "weird": [7, 25, 48, 52], "educ": [7, 10, 11, 22, 29, 30, 38, 41, 42, 43, 48, 50, 51, 52, 53, 55], "highli": [7, 16, 52], "imposs": [7, 12], "spit": 7, "nonsens": 7, "danger": [7, 21], "financ": [7, 8, 41], "healthcar": [7, 14, 27, 41, 43], "contain": [7, 21, 24, 25, 39, 41, 43, 61], "virtual": 7, "bomb": [7, 12], "trigger": 7, "relatedli": [7, 48], "difficult": [7, 60], "adopt": [7, 21, 47], "auto": [7, 12, 41], "grader": [7, 35], "certainli": [7, 28, 34, 53], "net": [7, 12], "essai": [7, 60], "free": [7, 8, 11, 12, 23, 26, 40, 55], "teacher": 7, "deepli": 7, "manipulat": 7, "particular": [7, 8, 11, 15, 21, 23, 28, 41, 43, 46, 57, 59, 60, 62], "appear": [7, 11, 22, 38, 46, 50], "mostli": [7, 11, 18, 25, 60, 62], "onc": [7, 22, 28, 29, 30, 32, 33, 38, 40, 41, 42, 59, 60, 61], "quit": [7, 37, 55, 61], "incredibli": [7, 9, 60], "mad": 7, "financi": [7, 8, 16, 36, 41, 55, 60, 62], "game": [7, 50], "engag": [7, 16, 46, 48, 57, 62], "isn": [7, 11, 21, 22, 29, 38, 41, 42, 43, 44, 46, 48, 50, 52, 54, 55, 61], "hu": 7, "et": [7, 20, 37], "al": [7, 20, 37, 39], "replac": [7, 22, 26, 28, 38, 40], "proprietari": 7, "court": [7, 31], "advis": [7, 12, 31], "judg": 7, "grant": [7, 28, 31], "bail": 7, "share": [7, 11, 15, 16, 21, 22, 25, 28, 30, 34, 38, 42, 48, 50, 55, 61, 62], "juvenil": 7, "arrest": [7, 13], "adult": [7, 30, 42], "assess": [7, 46, 53], "seal": 7, "belief": [7, 33], "childhood": 7, "fate": 7, "transpar": [7, 16, 60], "nlp": 7, "headach": [7, 50], "unusu": [7, 45], "bet": [7, 21], "easili": [7, 9, 31, 49, 50, 55], "analyt": [7, 23], "parametr": 7, "cardiac": 7, "cholesterol": [7, 14, 50], "credit": [7, 11, 16, 25, 32, 41], "worthi": [7, 16], "wish": [7, 25, 28, 32, 46, 49, 59, 61, 62], "ldl": 7, "cholesterdol": 7, "regular": [7, 38, 40, 49, 52, 55, 62], "prevent": [7, 12, 21, 46, 60], "forest": 7, "isoton": 7, "vincent": 7, "warmerdam": 7, "health": [7, 14, 26, 27, 40, 41, 43, 48, 55, 60, 61], "y": [7, 8, 9, 11, 12, 13, 18, 19, 22, 24, 26, 34, 35, 37, 38, 39, 44, 49, 52, 60], "axi": [7, 15, 22, 24, 33, 34, 35, 38, 39], "regressor": 7, "sklearn": [7, 22, 25, 38], "xgboost": 7, "tensor": 7, "flow": 7, "goe": [7, 22, 38, 49, 62], "subpopul": [7, 42, 54], "concern": [7, 8, 11, 12, 25, 48, 55], "dalit": 7, "brahmin": 7, "christian": 7, "muslim": 7, "coupl": [7, 22, 38], "disproportion": [7, 48, 50], "rate": [7, 12, 13, 14, 21, 24, 25, 28, 29, 30, 35, 39, 41, 42, 44, 48, 49, 60], "tail": [7, 11, 28], "co": [7, 39, 44, 48], "enter": [7, 11, 21, 25, 29, 41], "through": [7, 8, 11, 16, 18, 19, 20, 22, 38, 43, 46, 47, 49, 55, 57, 59, 61], "swap": [7, 41], "imput": 7, "factor": [7, 12, 21, 24, 37, 39, 60], "Of": [7, 22, 28, 38, 48, 50, 51, 60], "third": [7, 9, 11, 16, 25, 30, 52, 60], "unbias": [7, 45], "regardless": [7, 49, 52], "religion": 7, "sickl": 7, "cell": [7, 61], "anemia": 7, "symptom": [7, 50], "african": 7, "descent": 7, "allel": 7, "malaria": [7, 12], "pressur": [7, 29, 41], "region": 7, "endem": 7, "carri": [7, 18, 51], "gene": [7, 48], "strong": [7, 34, 46, 47, 52], "At": [7, 8, 11, 21, 22, 25, 38, 40, 41, 43, 47, 51, 53, 60, 62], "recent": [7, 13, 17, 22, 33, 35, 38, 46, 47, 53], "kidnei": 7, "transplant": 7, "correct": [7, 18, 19, 21, 43, 45, 54, 57], "blood": 7, "flag": [7, 11, 25, 62], "prioriti": [7, 16, 31], "unlik": [7, 8, 42, 52, 53, 57], "adjust": [7, 21, 28, 43, 45], "explan": [7, 21, 45], "histor": [7, 41], "succe": 7, "biolog": 7, "until": [7, 9, 11, 20, 22, 24, 38, 42, 51, 52], "rigor": [7, 13, 35, 60, 62], "emphas": [7, 52, 60, 62], "solut": [7, 11, 17, 25, 30, 53], "proxi": 7, "crude": [7, 13, 35], "initi": [7, 21, 22, 25, 27, 28, 31, 38, 48, 49, 53, 60], "screen": 7, "drop": [7, 8, 21, 22, 25, 29, 31, 33, 35, 38, 41, 42, 43, 52], "margin": [7, 46], "weakli": 7, "genet": 7, "ever": [7, 8, 22, 25, 30, 38, 42, 50], "evid": [7, 8, 13, 18, 19, 26, 28, 30, 35, 40, 42, 54, 60, 62], "dynam": [7, 60], "plai": [7, 21, 22, 23, 28, 38, 52, 60, 61], "greater": [7, 11, 23, 25, 28, 30, 37, 41, 42, 60], "knowledg": [7, 21, 32, 57, 60], "remind": 7, "expert": 7, "quantit": [7, 25, 28, 31, 32], "expertis": [7, 16], "terribl": [7, 43], "su": [7, 17], "unus": 7, "imperfect": 7, "dramat": [7, 19, 22, 38, 52], "campaign": 7, "constant": [7, 9, 43], "toward": [7, 21, 60], "hurdl": [7, 61], "Then": [7, 12, 13, 21, 22, 28, 35, 38, 43, 49, 51, 53, 60], "accept": [7, 8, 11, 16, 20, 22, 28, 29, 37, 38, 41], "charg": [7, 11], "lawsuit": 7, "pr": 7, "disast": [7, 55], "optim": [7, 16, 21, 61], "spars": 7, "scalabl": 7, "fall": [7, 13, 17, 22, 23, 25, 35, 38, 50, 52], "triag": [7, 27], "notori": 7, "concept": [7, 11, 28, 46, 47, 57, 62], "whiten": 7, "hybrid": 7, "portion": [7, 11, 29, 41], "flexibl": [7, 13, 21, 35], "machineri": [7, 24, 39, 41], "lattic": 7, "counterfactu": [7, 60], "visual": [7, 21, 47], "dtreeviz": 7, "lda": 7, "detail": [8, 16, 21, 22, 28, 29, 31, 38, 41, 45, 49, 51, 55, 60, 61, 62], "deriv": [8, 13, 17, 26, 27, 40, 50], "mathemat": [8, 20, 37, 43, 44, 45, 46, 57], "advertis": [8, 41, 48], "discount": [8, 41, 43], "amount": [8, 13, 16, 25, 35, 44, 48, 49, 50, 60, 62], "treat": [8, 11, 13, 22, 23, 25, 28, 29, 30, 33, 35, 38, 41, 42, 43, 45, 50, 51, 52, 60], "untreat": [8, 22, 28, 38, 51, 52], "seen": [8, 15, 21, 30, 33, 34, 48, 59, 60], "y_0": [8, 22, 38, 44], "y_1": [8, 22, 38], "met": [8, 26, 27, 40, 47, 48], "sell": [8, 13, 14, 28, 35, 41, 48, 50, 62], "sunglass": 8, "pawda": 8, "sourc": [8, 11, 18, 19, 28, 44, 52, 53, 61], "promot": [8, 14, 28, 52], "coupon": [8, 52, 62], "york": [8, 44, 48], "ny": [8, 38, 39, 44], "came": [8, 20, 28, 37, 48, 52], "revenu": [8, 45, 61], "instagram": 8, "nytim": 8, "elsewher": 8, "tempt": 8, "vacuous": 8, "willing": [8, 29, 41, 51], "higher": [8, 10, 14, 22, 23, 26, 30, 38, 40, 41, 42, 53, 54], "bar": [8, 13, 35], "affirm": [8, 26], "wealthier": [8, 48], "relativelei": 8, "dispos": 8, "split": [8, 13, 16, 21, 25, 35, 42, 50], "weren": [8, 28, 40, 48, 52], "height": [8, 26, 40], "cm": 8, "happier": 8, "absent": [8, 29, 41, 60], "balanc": [8, 11, 22, 25, 38, 52, 54, 62], "size": [8, 16, 21, 22, 28, 31, 38, 42, 43, 45, 50, 54], "lln": 8, "kick": [8, 14], "kinda": [8, 50], "att": 8, "yai": 8, "platform": [8, 50], "exist": [8, 28, 31, 45, 48, 53, 55, 60, 61], "everywher": [8, 48, 55], "convinc": [8, 18, 19, 28, 62], "ourselv": [8, 22, 38], "increas": [8, 11, 12, 16, 22, 23, 26, 28, 37, 38, 40, 41, 43, 48, 49, 50, 60], "relativlei": 8, "understim": 8, "BUT": [8, 62], "obsess": 8, "plausibl": 8, "dunno": 8, "dozen": [8, 9, 51, 60], "far": [8, 13, 21, 22, 30, 35, 38, 42, 48, 50, 52, 55], "perfect": [8, 21, 22, 25, 29, 38, 41, 48], "absolut": [8, 12, 21, 30, 44, 59, 61, 62], "further": [8, 23, 31, 40], "investig": [8, 16, 41, 53], "advic": [8, 23, 27, 46], "strongli": 8, "friend": [8, 50], "flip": 8, "flop": 8, "sandal": 8, "cat": [8, 43], "meow": 8, "dal": 8, "referr": 8, "none": [8, 28], "forward": [8, 25, 51, 52, 53, 59, 60], "welcom": [8, 11, 16], "certainti": 8, "hopefulli": [8, 10, 18, 22, 38, 42, 49, 60, 62], "polic": [9, 34, 48, 58, 61], "expenditur": [9, 28, 34, 58], "massachusett": [9, 15, 30, 34, 42], "lead": [9, 12, 16, 17, 20, 21, 22, 26, 31, 32, 37, 38, 40, 48, 62], "loop": [9, 15, 28, 34, 51], "policeexpenditur": [9, 15, 34], "crimeindex": [9, 15, 34], "separ": [9, 15, 20, 34, 44, 46, 50], "remark": [9, 30, 42, 48, 60], "cute": 9, "anscomb": 9, "quartet": 9, "four": [9, 11, 13, 16, 25, 30, 35, 61], "bi": [9, 16, 38], "varianc": [9, 22, 38, 45], "fourth": [9, 16], "betrai": 9, "explanatori": [9, 45, 52], "forev": 9, "rich": [9, 28, 48, 55, 57], "link": [9, 20, 21, 22, 23, 24, 29, 37, 38, 39, 41, 43, 51, 55], "cheat": [9, 20, 22, 24, 38], "preced": [10, 13, 21, 25, 35], "establish": [10, 11, 28, 41, 48], "gap": [10, 29, 33, 41, 47], "sort": [10, 11, 25, 41, 43, 48, 52, 53, 61], "femal": [10, 22, 28, 29, 30, 38, 41, 42, 43, 49], "field": [10, 22, 23, 31, 38, 41, 46, 47], "remain": [10, 13, 22, 25, 28, 35, 38, 53], "stem": 10, "decompos": 10, "man": [10, 29, 37, 41], "inequ": [10, 41, 58], "onboard": 11, "app": [11, 28, 48, 50], "apach": 11, "licens": 11, "writeup": 11, "fair": [11, 21, 41], "materi": [11, 41, 43, 46, 47, 52, 57, 60], "click": [11, 23, 28], "card": [11, 21], "enrol": [11, 21, 27, 28, 47, 48], "automat": [11, 21, 35, 43, 49], "cancel": 11, "quizz": 11, "coach": 11, "support": [11, 12, 16, 18, 19, 28, 31, 41, 43, 51], "verifi": [11, 28], "certif": [11, 30, 32, 42], "feedback": [11, 16, 61], "devot": 11, "checkout": 11, "fewer": [11, 12, 25, 30, 43, 50, 51, 60], "messag": [11, 34], "continu": [11, 21, 22, 25, 28, 38, 44, 48, 49], "screenshot": 11, "hope": [11, 14, 15, 23, 25, 28, 31, 47, 50, 53, 57, 60], "upfront": 11, "frustrat": [11, 46], "left": [11, 30, 33, 44, 51, 52], "significantli": [11, 28, 42, 46], "held": [11, 29, 41], "capac": 11, "www": [13, 20, 23, 24, 35, 37, 39, 42], "github": [11, 13, 15, 20, 22, 24, 29, 30, 33, 34, 35, 37, 38, 39, 41, 42, 43, 47], "mids_data": [13, 15, 20, 22, 24, 29, 33, 34, 35, 37, 38, 39, 41, 42, 43], "udacity_ab_test": [], "folder": [25, 29, 35, 41, 42], "visitor": 11, "roughli": [11, 16, 20, 37, 42], "row": [11, 13, 24, 33, 35, 38, 39, 41, 43, 55], "easiest": [11, 43, 60], "stack": 11, "pageview": 11, "uniqu": [11, 21, 22, 25, 33, 38], "homepag": 11, "payment": [11, 25], "Or": [11, 25, 30, 37, 38, 40, 41, 42, 43, 48], "m": [11, 15, 25, 34, 35, 39, 43, 50, 52, 55], "phrase": 11, "terminologi": 11, "encount": [11, 46, 60], "unintend": 11, "consequ": [11, 12, 26, 28, 40, 50, 60], "correctli": [11, 30, 37, 38, 39, 41, 42, 43, 48, 49, 54, 62], "deem": 11, "half": [11, 13, 21, 28, 30, 35, 52, 53, 62], "shown": [11, 47], "invari": [11, 13, 35], "affect": [11, 24, 31, 39, 43, 44, 45, 49, 53], "pre": [11, 13, 22, 35, 38, 48, 51, 61], "analog": [11, 43], "previous": [11, 13, 22, 35, 38, 46, 49, 53, 60], "qualiti": [11, 12, 22, 37, 38, 41, 55], "tricki": 11, "divid": [11, 16, 28, 29, 41, 45, 51], "emerg": [11, 17, 41], "ttest": 11, "scipi": [11, 38, 39, 42], "pass": [11, 13, 21, 25, 28, 35, 37, 43], "numpi": [11, 24, 38, 39, 41], "vector": [11, 22, 38, 43], "oper": [11, 12, 16, 21, 25, 41], "df": [11, 22, 33, 34, 35, 37, 38, 39, 41, 42, 43, 49], "my_column": 11, "93": [33, 34, 37, 38, 41], "consist": [11, 25, 60], "magic": [11, 44, 50, 53, 60], "bivari": 11, "close": [11, 22, 29, 31, 38, 41, 52], "float": [11, 43], "decreas": [11, 12, 13, 19, 23, 35], "covari": [22, 24, 30, 35, 37, 38, 39, 41, 42, 43, 45, 49, 54], "perfectli": [11, 49], "mondai": [], "adapt": [12, 22, 38, 52, 54], "permiss": [12, 23], "adrian": 12, "fresh": 12, "caus": [11, 12, 19, 25, 28, 32, 44, 46, 48, 53, 54, 60], "statement": [12, 15, 17, 33, 60], "underlin": 12, "embed": 12, "gold": [12, 21], "typic": [12, 16], "law": [12, 13, 16, 30, 35, 42, 60], "cool": [12, 17, 50, 51], "absenc": 12, "arisen": [12, 28], "instruct": [12, 18, 19, 41], "c": [12, 20, 22, 24, 34, 35, 37, 38, 39, 41, 42, 43, 49, 52], "infeas": 12, "briefli": 12, "nato": 12, "militari": 12, "command": [12, 28, 33], "defeat": 12, "insurg": 12, "afghanistan": 12, "kill": [12, 61], "villag": [12, 28, 50], "upset": 12, "civilian": [12, 41, 43], "hous": [12, 41], "member": [12, 14, 16, 28, 45, 51], "afghani": 12, "coalit": [12, 31], "forc": [12, 41, 46], "war": [12, 13, 35, 60], "break": [12, 22, 23, 38, 45, 53], "hypothet": [12, 14], "farfetch": 12, "wildli": 12, "baselin": [11, 12, 14, 16, 22, 26, 27, 28, 30, 38, 40, 42, 44, 52, 62], "milit": 12, "insan": [12, 38, 55], "unanswer": 12, "jason": 12, "lyall": 12, "insur": [12, 25, 27, 41, 48], "bought": [12, 53], "devic": 12, "alert": 12, "driver": [12, 17, 61], "unsaf": 12, "accid": [12, 14, 24, 39, 61], "wonder": [12, 22, 38], "britain": 12, "vote": [12, 49, 60], "european": [12, 48], "uk": 12, "brexit": 12, "immigr": 12, "attitud": 12, "proxim": 12, "favor": 12, "eu": 12, "2016": [12, 13, 33, 35], "aid": [12, 21, 24, 30, 39, 53, 57, 61, 62], "priorit": [12, 22, 38], "bed": 12, "famili": [12, 25, 28, 41], "rural": [12, 48, 50, 53], "kenya": 12, "drug": [13, 33, 35, 41, 48, 50, 60, 61], "dealer": [13, 35, 41], "decad": [13, 22, 35, 38, 55, 60], "california": [13, 33, 35], "lessen": [13, 35], "penal": [13, 21, 35], "possess": [13, 35], "arguabl": [13, 35], "2010": [13, 21, 28, 33, 35], "penalti": [13, 21, 25, 30, 35, 41, 42], "crimin": [13, 35], "civil": [13, 35], "guilti": [13, 35], "produc": [13, 35], "quantiti": [13, 26, 27, 28, 35, 40, 42, 49], "illeg": [13, 35], "recreat": [13, 35, 41], "regulatori": [13, 16, 21, 35], "propon": [13, 35], "contribut": [13, 22, 31, 35, 38, 52, 60], "violenc": [13, 35], "gang": [13, 35], "battl": [13, 35], "territori": [13, 35], "accord": [13, 31, 35, 37, 57], "argument": [13, 22, 25, 28, 33, 35, 38, 52, 60], "simpli": [13, 28, 35, 47, 60], "semest": [13, 35, 61], "com": [13, 15, 20, 22, 23, 24, 29, 34, 35, 37, 38, 39, 41, 42, 43, 49], "uds_arrest_data": [13, 33, 35], "csv": [13, 15, 24, 33, 34, 35, 39, 49], "attornei": [13, 33], "offens": [13, 35], "f_drugoff": [13, 33, 35], "feloni": [13, 35], "total_popul": [13, 33, 35], "total": [11, 13, 22, 24, 25, 28, 30, 33, 35, 38, 39, 49, 57], "began": [13, 28, 35, 48, 60], "liber": [13, 35], "2007": [13, 25, 28, 33, 35, 38, 52], "2009": [13, 21, 25, 33, 35], "violent_r": [13, 35], "among": [13, 16, 28, 29, 30, 32, 35, 41, 42, 43, 45, 46, 48, 49, 50, 53], "2008": [13, 25, 33, 35], "special": [13, 21, 35, 41, 43, 48, 55, 60], "y_": [13, 27, 35, 40], "avg": [13, 35, 39, 41, 43], "hat": [13, 35], "delta": [13, 35], "_": [13, 24, 35, 39], "took": [13, 31, 35, 60], "ignor": [11, 13, 21, 29, 35, 41, 42, 43], "middl": [13, 35, 38, 44, 49, 52, 60], "decrimin": [13, 35], "parallel": [13, 35, 62], "2000": [13, 33, 35, 41, 48], "inclus": [13, 22, 35, 38], "geometr": [13, 35], "confid": [13, 21, 23, 35, 42, 43, 48, 49, 52, 62], "panelol": [13, 24, 35, 39, 43], "lfe": [13, 24, 35, 39], "r": [13, 21, 24, 35, 37, 39, 41, 42, 43, 47, 49], "client": [14, 16], "offset": 14, "worthwhil": [14, 60], "durham": [14, 60], "nc": [14, 39], "index": [14, 15, 22, 26, 33, 34, 38, 40, 43], "rest": [14, 48, 52, 60, 61], "consumpt": [14, 20, 24, 27, 28, 32, 37, 39], "healthier": 14, "rental": [14, 41], "renter": 14, "deduct": 14, "fee": 14, "pocket": 14, "lose": [14, 29, 41, 43, 52, 62], "hassl": 14, "fleet": 14, "agreement": [14, 21], "tesla": 14, "dealership": 14, "hillsboro": 14, "town": 14, "heard": 14, "center": [14, 41, 47, 53], "examin": [15, 22, 33, 34, 38, 60], "blob": [15, 22, 34, 37, 38, 41, 42, 43], "descriptive_exercis": [15, 34], "crime_expend_ma": [15, 34], "monthli": [15, 25, 28, 34], "budget": [15, 28, 34], "late": [15, 25, 34, 48], "2001": [15, 30, 33, 34, 35, 42], "county_cod": [15, 34], "print": [15, 23, 34, 35, 37, 39, 41, 42], "output": [15, 16, 24, 34, 39, 43, 49], "gonna": [15, 28, 34, 37, 62], "75": [15, 33, 34, 38, 39], "83": [15, 33, 34, 38, 42], "62": [15, 34, 35, 38, 42], "55": [15, 34, 37, 38, 39], "88": [15, 33, 34, 38], "format": [15, 55], "guess": [15, 20, 23, 34, 37, 42, 48, 53, 60], "knew": [15, 34, 62], "till": [15, 34, 43, 51], "scatter": [15, 20, 24, 34, 35, 37, 39], "advisori": 16, "firm": [16, 25, 49, 55], "compliant": 16, "authent": 16, "capston": [16, 31, 36], "undertaken": [16, 31], "corpor": [16, 31], "intersect": 16, "top": [16, 53, 62], "bank": [16, 41], "fintech": 16, "lend": 16, "core": [16, 17, 31, 52, 53, 60], "capabl": 16, "lender": 16, "led": [16, 31], "reli": 16, "challeng": [16, 47, 48], "tradit": [16, 31], "partial": [16, 21, 22, 38], "shaplei": 16, "regul": [16, 40, 60], "delinqu": 16, "loan": [16, 25, 36, 55], "burden": 16, "champion": 16, "gradient": [16, 21], "boost": [16, 21], "gbm": [16, 36], "viabil": 16, "ebm": [16, 21, 36], "publicli": [16, 53, 55], "complianc": [16, 48, 54], "overlai": 16, "enforc": 16, "kdd": 16, "cup": 16, "191k": 16, "481": [16, 41, 42], "mail": [16, 41, 48], "creation": [16, 21, 31], "weekli": [16, 29, 41, 43, 44], "stretch": 16, "methodologi": 16, "advers": 16, "satisfi": [16, 50], "contact": [16, 28], "redact": 16, "outlin": [16, 62], "undertak": [16, 54, 60, 62], "deliver": [16, 31], "uglier": 17, "realist": 17, "motor": [17, 41, 53], "conglomer": 17, "america": [17, 53], "manufactur": [17, 20, 37, 41], "announc": [17, 55, 62], "enhanc": 17, "brake": 17, "lane": 17, "departur": 17, "warn": [17, 21, 43, 49, 59], "keep": [17, 21, 22, 25, 36, 38, 46, 48, 51, 52, 57, 60, 62], "drowsi": 17, "ey": [17, 33, 38], "squirt": 17, "water": [17, 18, 19, 41], "wake": 17, "fortun": 17, "weaker": 17, "prone": 17, "injuri": 17, "activ": [17, 26, 40, 41, 55, 57, 59, 60], "tracker": 17, "sheet": 18, "aloud": [18, 19], "devis": [18, 19], "waterborn": [18, 19], "pathogen": [18, 19], "technologi": [18, 19], "falsifi": [18, 19], "inconsist": [18, 19, 49], "germ": [18, 19], "spread": [18, 19], "worksheet": [18, 19], "walk": [18, 19], "john": [18, 19, 60], "snow": [18, 19], "london": [18, 19], "diseas": [18, 19, 50], "miasma": 18, "lesson": [18, 22, 38, 51], "1854": [18, 19, 49], "street": [18, 19, 62], "pump": [18, 19], "neighborhood": [18, 19, 48, 55, 60], "soho": [18, 19], "suppli": [18, 19, 41], "drink": [18, 19, 26, 40, 41], "cook": [18, 19], "household": [18, 19, 28, 41, 43, 50], "chore": [18, 19], "indoor": [18, 19], "plumb": [18, 19], "unheard": [18, 19], "meant": [11, 18, 19, 55, 60], "obtain": [18, 19, 53], "corner": [18, 19], "throughout": [18, 19, 25, 44, 60], "citi": [18, 19, 25, 43, 44, 45, 60, 61], "despit": [18, 48, 60], "worker": [18, 29, 41], "breweri": 18, "outbreak": 18, "food": [18, 28, 41], "bakeri": [18, 41], "exhibit": 18, "involv": [18, 61, 62], "retail": [18, 41, 43], "drank": [18, 26], "beer": [18, 24, 39, 41], "hydrat": 18, "brew": 18, "boil": 18, "famous": [18, 60], "map": [18, 46], "counter": [18, 29, 41, 46, 60], "factual": [18, 29, 41, 60], "Will": [18, 19], "space": [18, 19, 20, 21, 31, 37, 41, 43, 51, 55, 61], "incorrect": [18, 19, 21], "brewer": 18, "theoriz": 19, "contamin": 19, "author": 19, "dismantl": 19, "handl": [19, 33], "septemb": 19, "subsequ": [19, 22, 25, 33, 38, 46, 61], "7th": 19, "canon": [20, 37], "mileag": [20, 37], "74": [20, 33, 37, 38], "automobil": [20, 37, 41], "april": [20, 37], "1979": [20, 37, 38], "epa": [20, 37], "fuel": [20, 37, 41], "compil": [20, 37, 55, 60], "chamber": [20, 37], "1983": [20, 33, 35, 37, 39], "automobile_dataset": [20, 37], "file": [11, 20, 25, 29, 30, 33, 37, 41, 49, 55], "internet": [20, 37, 41], "codebook": [20, 37], "mpg": [20, 37], "rep78": [20, 37], "repair": [20, 37, 41], "1978": [20, 37], "headroom": [20, 37], "trunk": [20, 37], "cu": [20, 37], "ft": [20, 37], "lb": [20, 37], "circl": [20, 37, 62], "displac": [20, 37], "gear_ratio": [20, 37], "gear": [20, 37], "ratio": [11, 20, 25, 28, 37, 54], "foreign": [20, 37], "guzzler": [20, 37], "mile": [20, 37], "gallon": [20, 37], "ga": [20, 37, 39, 41], "guzzl": [20, 37], "marker": [20, 37], "red": [20, 37, 52], "green": [20, 37], "confirm": [20, 28, 37, 49], "q3": [20, 37], "poorli": [20, 37, 48, 53, 55], "poor": [20, 28, 37, 45, 48], "respect": [20, 21, 37, 42, 48, 49, 54], "abroad": [20, 37], "grandma": [20, 37], "domest": [20, 37], "main": [20, 24, 37, 39, 52, 62], "layman": [20, 37], "cynthia": 21, "rudin": 21, "commentari": 21, "arxiv": 21, "plethora": 21, "alter": 21, "appeal": 21, "inher": 21, "flaw": [21, 60], "favorit": 21, "passag": [21, 46], "her": 21, "faith": 21, "cannot": [21, 22, 38, 48, 49, 60], "fidel": 21, "equal": [21, 25, 28, 37, 49], "represent": 21, "extens": 21, "tenth": 21, "14th": 21, "pacif": [21, 48], "asia": [21, 53], "discoveri": [21, 60], "mine": [21, 41], "confer": [21, 31], "pakdd": 21, "2006": [21, 25, 33, 35], "privat": [21, 38, 41], "brazilian": 21, "partner": [21, 61], "timi": 21, "pure": 21, "coincid": 21, "dissert": 21, "confus": [21, 25, 49, 57, 60], "glm": 21, "mvgam": 21, "pioneer": [21, 28], "statsmodel": [21, 22, 28, 30, 35, 37, 38, 39, 41, 42, 43, 49], "hide": [21, 50], "plain": [21, 26, 27, 40, 48], "sight": 21, "meta": 21, "prophet": 21, "seri": [21, 33], "forecast": 21, "librari": [21, 22, 25, 30, 38, 41, 42, 43, 49], "padd": 21, "repositori": [21, 55], "pakdd2010_variableslist": 21, "xl": 21, "pakdd2010_modeling_data": 21, "txt": [21, 25, 29, 41], "string": [21, 22, 25, 30, 38, 43], "load": [11, 21, 22, 23, 24, 25, 28, 30, 38, 39, 42, 43, 49, 61], "latin": [21, 33, 53], "latin8": 21, "prior": [21, 28, 47], "utf": 21, "hint": [21, 22, 24, 25, 33, 38, 39, 42], "strictli": [21, 57], "stage": 21, "missing": 21, "quant_depend": 21, "quant_car": 21, "months_in_resid": 21, "personal_monthly_incom": 21, "quant_banking_account": 21, "marital_statu": 21, "occupation_typ": 21, "residence_typ": 21, "residencial_st": 21, "residencial_c": 21, "residencial_borough": 21, "residencial_zip_3": 21, "banana": 21, "log": [21, 30, 35, 37, 38, 39, 41, 42, 43, 49, 52], "geograph": [21, 31, 55], "segreg": 21, "resid": [21, 28, 60], "resolv": 21, "residencial_": 21, "train_test_split": [21, 25], "80": [21, 28, 33, 38, 41], "target_label_bad": 21, "classif": [21, 25], "gridsearch": 21, "spline": 21, "slow": [21, 43], "50": [21, 27, 28, 33, 34, 37, 38, 39, 42, 43, 49, 50], "residencial_zip": 21, "minut": [21, 51], "configur": 21, "naiv": [21, 42], "matrix": [21, 24, 25, 37, 38, 39, 41, 43, 45], "extend": [21, 44], "default": [11, 21, 22, 25, 33, 36, 38], "omiss": [21, 25], "predict_proba": [21, 25], "shift": 21, "emphasi": 21, "threshold": [21, 22, 23, 25, 28, 38, 51], "toler": 21, "interv": [21, 23, 49], "partial_depend": 21, "pdep": 21, "confi": 21, "xx": 21, "width": 21, "valueerror": [21, 22, 38, 43], "recod": 21, "refit": 21, "impos": [21, 36, 61], "concav": 21, "convex": 21, "overfit": [11, 21], "smooth": [21, 49], "paramet": [21, 28, 30, 35, 39, 43], "lambda": [21, 33], "push": [21, 28, 50], "smoother": [21, 49], "grid": 21, "doc": [21, 28], "explainableboostingclassifi": 21, "classmat": 21, "prefer": [21, 22, 30, 38, 40, 42, 45, 47, 60, 62], "glassbox": 21, "x_train": 21, "y_train": 21, "catch_warn": 21, "simplefilt": 21, "ebm_glob": 21, "explain_glob": 21, "ebm_loc": 21, "explain_loc": 21, "matchit": [22, 38], "rpy2": [22, 38], "statistician": [22, 38, 45], "awar": [22, 38, 50, 60, 61], "matur": [22, 38], "purpos": [22, 38, 43, 45, 60], "techniqu": [22, 38, 47], "frontier": [22, 38], "hyper": [22, 38], "pip": [22, 38, 43], "conda": [22, 38, 43], "exact": [22, 38, 48], "pair": [22, 33, 38, 51, 52, 61], "repeat": [22, 28, 38], "trick": [22, 38], "massiv": [22, 38, 48, 60], "computation": [22, 24, 38, 39], "amaz": [22, 30, 38, 42, 55, 60], "month": [22, 25, 27, 28, 34, 38, 48, 50], "cp": [22, 29, 38, 41, 43], "current_population_survei": [22, 29, 38, 41, 43], "cps_for_match": [22, 38], "content": [22, 38, 40, 47], "overlap": [22, 38, 51, 52], "unsur": [11, 22, 38], "imbal": [22, 30, 38, 51], "annual_earn": [22, 38, 41, 43], "has_colleg": [22, 38], "plotnin": [22, 38, 49], "geom_smooth": [22, 38, 49], "lowess": [22, 38, 49], "altair": [22, 34, 35, 37, 38], "transform_loess": [22, 38], "tutori": [22, 38, 49], "speak": [22, 38, 46], "discret": [22, 38, 49, 53], "employ": [22, 29, 30, 38, 41, 42], "merg": [22, 25, 33, 35, 38, 55], "covert": [22, 38], "convert": [22, 25, 28, 38, 43, 49], "class94": [22, 38], "interg": [22, 38], "belong": [22, 29, 38, 41, 43, 44, 49, 60], "passion": [22, 38], "var": [22, 35, 38, 58], "pull": [22, 38, 43], "integ": [22, 38, 49], "reset": [22, 38], "dame_flam": [22, 38], "syntax": [22, 38, 49], "my_data": [22, 38], "verbos": [22, 38], "want_p": [22, 38], "treatment_column_nam": [22, 38], "outcome_column_nam": [22, 38], "printout": [22, 38], "iter": [22, 38], "unmatch": [22, 38], "empti": [22, 38], "hasn": [22, 38, 60], "370": [22, 37, 38, 41, 49], "644": [22, 38], "1150": [22, 38], "3187": [22, 38], "4365": [22, 38], "1684": [22, 38], "1199312680": [22, 38], "0957854": [22, 38], "684": [22, 38], "subset": [22, 25, 29, 38, 41], "secret": [22, 38], "sauc": [22, 38], "ridg": [22, 38], "elimin": [22, 38, 52], "626": [22, 37, 38, 41], "494": [22, 38], "180": [22, 38], "3626": [22, 38], "5310": [22, 38], "frozenset": [22, 38], "1199421883": [22, 38], "1095908": [22, 38], "recal": [11, 22, 23, 30, 38, 48], "becom": [22, 28, 38, 44, 48, 49, 60], "manual": [22, 37, 38], "early_stop_iter": [22, 38], "evolv": [22, 38, 60], "pe_each_it": [22, 38], "pe": [22, 38], "falloff": [22, 38], "complaint": [22, 38], "entri": [22, 33, 38, 43], "simplified_rac": [22, 38], "discretized_ag": [22, 38], "24": [22, 25, 28, 33, 34, 37, 38, 41, 42, 52], "copi": [22, 38, 43], "def": [22, 38], "get_datafram": [22, 38], "result_of_fit": [22, 38], "input_data": [22, 38], "loc": [22, 34, 35, 38, 42], "is_uniqu": [22, 38], "match_group": [22, 38], "np": [22, 25, 28, 37, 38, 39, 41], "nan": [22, 25, 33, 37, 38, 41, 43], "match_group_s": [22, 38], "idx": [22, 38], "enumer": [22, 38], "units_per_group": [22, 38], "len": [22, 38], "t_in_group": [22, 38], "groupbi": [22, 38, 39], "transform": [22, 38, 39], "sum": [22, 33, 38], "treament": [22, 38], "num": [22, 28, 38], "control_weight": [22, 38], "num_control_ob": [22, 38], "drop_dupl": [22, 38], "renorm": [22, 38], "assert": [11, 22, 25, 30, 33, 35, 38, 39, 60], "notnul": [22, 38, 43], "return": [22, 32, 33, 38, 41, 47, 48, 60], "wl": [22, 38], "matched_data": [22, 38], "smf": [22, 35, 37, 38, 39, 41, 42, 43, 49], "educt": [22, 38], "ba": [22, 29, 38, 41], "tast": [22, 38], "permut": [22, 38], "closest": [22, 38], "multipl": [22, 24, 25, 38, 39, 44, 47, 48, 61], "twice": [22, 28, 30, 38, 40, 44], "malt": [22, 38], "mahalanobi": [22, 38, 51], "distanc": [22, 31, 38, 51], "ulik": [22, 38], "clever": [22, 38], "arbitrari": [22, 38], "explod": [22, 38, 60], "aeml": [22, 38], "planner": [23, 28], "georgi": 23, "georgiev": [4, 23], "hi": [23, 37, 52, 60], "toolkit": 23, "processor": 23, "latenc": 23, "delai": 23, "chip": 23, "justifi": 23, "05": [23, 28, 37, 38, 41, 42, 49], "millisecond": 23, "smaller": [23, 28, 30, 35, 42, 43, 45, 51, 52, 53], "god": 23, "forbid": 23, "500": [23, 35, 41], "costli": [11, 23, 54], "revers": 23, "labor": [23, 28, 41], "50m": 23, "15m": 23, "wanna": [23, 49], "sequenti": 23, "agil": 23, "valuabl": [23, 52], "5m": 23, "hover": 23, "interpet": 23, "99": [23, 38, 43, 50], "densiti": 23, "lift": 23, "f": [23, 34, 35, 37, 38, 39, 41, 42, 43, 49], "alcohol": [24, 39, 41, 61], "fatal": [24, 39, 60, 61], "336": [24, 35, 38, 39, 41], "34": [24, 35, 37, 38, 39, 49], "beertax": [24, 39], "drunk": [24, 39], "intuit": [24, 39, 45, 46], "us_driving_fat": [24, 39], "frame": [24, 39], "fat_rat": [24, 39], "pop": [24, 29, 39, 41, 49], "python": [24, 37, 39, 42, 45, 47, 61], "navig": [24, 39, 41], "ol": [24, 35, 37, 38, 39, 41, 42, 43, 49], "align": [24, 39], "fatalityrate_i": [24, 39], "beta_0": [24, 39, 49], "beta_1": [24, 39, 49, 52], "beertax_i": [24, 39], "notat": [24, 39, 46, 49], "whei": [24, 39], "fatalityrate_": [24, 39], "alpha": [24, 28, 39, 52, 54], "beta": [24, 39, 54], "beertax_": [24, 39], "psi": [24, 39], "z_i": [24, 39], "epsilon_": [24, 39], "sum_": [24, 39], "overlin": [24, 39], "fatalityr": [24, 39], "_i": [24, 39], "epsilon": [24, 39, 49, 52], "substract": [24, 39], "equat": [24, 39, 49, 60], "yield": [24, 39], "overset": [24, 39], "sim": [24, 39, 43], "x_": [24, 39], "intercept": [24, 35, 37, 38, 39, 41, 42, 43, 45, 49], "from_formula": [24, 35, 39, 43], "arrai": [24, 28, 33, 39, 43], "phenomenon": [24, 39, 60], "state_i": [24, 39], "year_t": [24, 39], "homeown": 25, "newli": 25, "home": [25, 41, 48], "freddi": 25, "mac": 25, "calendar": 25, "dictionari": [11, 25, 30], "exercise_passive_predict": 25, "ipynb": [11, 25, 30], "upload": [11, 25, 30], "ex2_merge_typ": 25, "ex4_num_mortgag": 25, "ex5_num_ob": 25, "ex7_num_mortgag": 25, "ex7_share_delinqu": 25, "ex10_num_ob": [11, 25], "ex12_roc_auc": 25, "ex14_false_omission_r": 25, "ex16_num_ob": 25, "ex16_share_delinqu": 25, "ex17_false_omission_r": 25, "clariti": [11, 25, 30], "supplement": 25, "scope": 25, "sample_orig_2004": 25, "2004": [25, 33, 35], "sample_svcg_2004orig_3year": 25, "2005": [25, 33, 35], "url": [25, 30], "AND": [25, 37, 48], "Be": 25, "keyword": [25, 33, 43], "inner": 25, "join": 25, "quarter": 25, "vanilla": 25, "sequenc": [25, 53, 60], "behalf": 25, "element": [25, 26, 27, 40], "cycl": [25, 28], "perceiv": [25, 30, 42], "anomali": 25, "mismatch": [25, 54], "investor": 25, "commonli": [25, 43, 51], "section": [25, 60, 62], "cleanli": 25, "correspond": [25, 26, 27, 30, 40], "borrow": [25, 60], "instal": [25, 43], "ddlpi": 25, "banker": 25, "mba": 25, "acquir": 25, "reo": 25, "59": [25, 33, 34, 37, 38, 39], "60": [25, 35, 38, 39, 49], "89dai": 25, "119dai": 25, "ra": 25, "acquisit": 25, "issuanc": 25, "tidi": 25, "holder": [25, 31, 53], "metropolitan": 25, "msa": 25, "divis": 25, "deploy": 25, "homebuy": 25, "percentag": [11, 25, 28, 29, 30, 41, 42, 49], "mi": [25, 39, 41, 43, 48, 53], "occup": [25, 41, 43], "debt": 25, "dti": 25, "upb": 25, "ltv": 25, "channel": 25, "prepay": 25, "ppm": 25, "amort": 25, "formerli": 25, "o": 25, "distinct": [25, 52, 60], "hot": [25, 49], "seed": [25, 55], "fan": 25, "patsi": 25, "dmatric": 25, "binari": [25, 28, 49], "number_of_borrow": 25, "number_of_unit": 25, "model_select": 25, "test_siz": 25, "random_st": 25, "gradientboostingclassifi": 25, "ensembl": 25, "roc_auc_scor": 25, "roc": 25, "auc": [25, 47], "round": [11, 25, 30], "decim": [11, 25, 30, 52], "instabl": 25, "classifi": 25, "viabl": 25, "riski": 25, "08": [25, 37, 39, 49], "saniti": [11, 25], "cutoff": 25, "retrospect": [25, 28], "move": [25, 26, 27, 40, 41, 43, 49, 52, 60], "stai": [25, 50], "onto": [26, 27, 40, 46], "wellb": [26, 40], "sit": [26, 40], "front": [26, 40, 62], "bodi": [26, 28, 31, 40, 41, 62], "bmi": [26, 40], "soda": [26, 40, 58], "dine": [26, 40], "diet": [26, 40, 58], "sugar": [26, 40, 41], "healthi": [26, 40, 60], "sugari": [26, 40], "campu": [26, 30, 31, 40, 42], "0_i": 26, "1_i": 26, "d_i": 26, "english": [26, 27, 40, 46], "neq": [26, 27, 40, 44], "hr": 27, "depart": [27, 41, 43], "wemakewidget": 27, "llc": 27, "skyrocket": 27, "wmw": 27, "nurs": [27, 41, 43, 48], "via": [27, 31], "send": [27, 48, 62], "exchang": [27, 28], "appoint": 27, "bonu": 27, "mandatori": 27, "enrolle": 27, "woman": [29, 41, 49], "morg18": [29, 41, 43], "stata": [29, 41], "readm": [29, 41, 55], "nation": [29, 41, 53, 60], "lfsr94": [29, 41, 43], "uhours": [29, 41, 43], "35": [28, 29, 38, 39, 41, 42, 43, 49], "couldn": [29, 41, 60], "chose": [29, 30, 41, 48, 62], "earnwk": [29, 41, 43], "hourli": [29, 41], "48": [29, 33, 35, 37, 38, 39, 41, 43], "implicit": [29, 41, 50, 52], "pretend": [29, 41], "grade92": [29, 41, 43], "nest": [29, 41, 43, 44, 45, 60], "structur": [29, 31, 41, 43, 45, 48, 60, 62], "ind02": [29, 41, 43], "hierarch": [29, 41, 43], "serv": [29, 31, 41, 61], "slightli": [29, 38, 41, 43, 62], "wherea": [29, 41], "introduc": [29, 41, 48, 60], "harder": [30, 42, 57], "express": [30, 42], "870": [30, 33, 38, 41, 42], "fictici": [30, 42], "advert": [30, 42], "boston": [30, 42], "chicago": [30, 42], "alloc": [30, 42], "children": [28, 30, 42, 43, 45, 50, 53], "subscrib": [28, 30], "kwame": 30, "anthoni": 30, "appiah": 30, "capit": [28, 30], "w": 30, "journal": [30, 42, 62], "exercise_resume_experi": 30, "ex2_pvalue_computerskil": 30, "ex2_pvalue_femal": 30, "ex2_pvalue_yearsexp": 30, "ex3_pvalue_educ": 30, "ex4_valid": 30, "ex5_pvalu": 30, "ex5_white_advantage_perc": 30, "ex5_white_advantage_percentage_point": 30, "ex6_black_pvalu": 30, "ex8_black_colleg": 30, "ex8_black_nocolleg": 30, "ex8_college_heterogen": 30, "ex9_gender_and_discrimin": 30, "ex10_experiment_v_u": 30, "profoundli": [30, 42], "outset": [30, 42], "unbalanc": [30, 42], "resume_experi": [30, 42], "fictiti": [30, 42], "interview": [30, 42], "yearsexp": [30, 42], "computerskil": [30, 42], "ofjob": [30, 42], "qualif": [30, 42], "magnitud": [30, 31, 42, 45], "tabul": [30, 42], "dropout": [30, 42], "chi": [30, 42], "conting": [30, 42, 57], "crosstab": [30, 38, 42], "markdown": 30, "back": [30, 42, 44, 52, 53, 62], "heteroskedast": [30, 42, 49], "robust": [30, 31, 35, 39, 42, 43, 45, 49, 62], "hc3": [30, 42, 49], "upsid": [30, 46], "downsid": [30, 46], "particularli": [30, 46, 60], "migrain": [30, 50], "aimovig": [30, 50], "denot": [11, 30], "lookup": [30, 42], "paus": [30, 42, 60, 62], "followup": [28, 30, 31, 42], "ident": [30, 42, 51], "piec": [30, 41, 42, 46, 53], "crucial": [30, 42, 60, 62], "successfulli": [31, 53], "csr": 31, "2020": [28, 31, 49], "partnership": 31, "expand": 31, "summer": 31, "41": [28, 31, 35, 38], "leader": 31, "elect": [31, 55], "poll": 31, "ballot": 31, "satellit": 31, "thank": [31, 38], "incred": 31, "500k": 31, "cast": 31, "2013": [31, 33], "suprem": 31, "shelbi": 31, "struck": 31, "provis": [31, 61], "1965": 31, "act": [28, 31, 48, 54], "voter": [31, 49], "preclear": 31, "feder": 31, "2019": [31, 33, 35, 43, 60], "leadership": 31, "1600": [31, 41], "becam": 31, "closur": 31, "comprehens": [31, 60], "sponsor": 31, "aim": [31, 60], "eas": 31, "thereof": 31, "dropbox": [31, 33], "youth": 31, "equip": [31, 41], "young": [31, 46, 54, 60], "backup": 31, "advocaci": [31, 41], "surround": 31, "shine": 31, "brighter": 31, "greatest": 31, "proactiv": 31, "2022": [31, 37], "institut": [31, 41, 55], "qualit": 31, "extent": 31, "hbcu": 31, "msi": 31, "primarili": [31, 47, 52, 55, 60], "minor": [31, 43], "mere": 31, "travel": [31, 41], "nearest": 31, "booth": 31, "foot": 31, "transport": [31, 41, 43], "landscap": [31, 41, 53], "dichotomi": 31, "tank": [32, 41], "combat": 32, "global": [32, 47, 61], "warm": 32, "implic": [28, 32, 62], "leed": 32, "prison": 32, "recidiv": 32, "former": 32, "inmat": 32, "curiou": 32, "violent": [33, 35], "scroll": [33, 48], "mb": 33, "43": [33, 37, 38, 41, 42, 43], "read_csv": [33, 34, 35, 39, 49], "openjustic": 33, "doj": 33, "ca": [33, 38, 39], "gov": [33, 55], "06": [33, 34, 35, 41, 42, 43, 49], "onlinearrestdata1980": 33, "44": [33, 38, 41], "head": [33, 34, 35, 38, 39, 41, 42, 43, 49, 53], "age_group": 33, "f_sexoff": 33, "f_alloth": 33, "f_total": 33, "m_total": 33, "s_total": 33, "alameda": [33, 35], "505": [33, 38, 41], "1351": 33, "188": [33, 38, 39], "79": [33, 38, 41, 43], "2149": 33, "2286": 33, "295": [33, 38, 41], "butt": [33, 35], "calavera": [33, 35], "contra": [33, 35], "costa": [33, 35], "116": [33, 38, 41], "446": [33, 38, 41, 49], "37": [33, 37, 38, 39, 41, 43, 49], "629": [33, 37, 38, 41], "557": [33, 37], "31": [33, 38, 41, 49], "el": [33, 35], "dorado": [33, 35], "45": [33, 35, 38, 39, 41, 42, 43, 51], "value_count": [33, 35, 38, 49], "26590": 33, "23767": 33, "23328": 33, "21188": 33, "dtype": [33, 35, 38, 43, 49], "int64": [33, 35, 38, 49], "46": [33, 37, 38, 42], "17438": 33, "17258": 33, "40": [28, 33, 35, 37, 38, 41, 49, 50], "69": [33, 35, 37, 38, 39, 41], "17082": 33, "16305": 33, "16040": 33, "70": [33, 37, 38], "10750": 33, "duplic": 33, "social_security_numb": 33, "111111111": 33, "222222222": 33, "333333333": 33, "second_column": 33, "bool": 33, "instanc": [28, 33, 43], "repit": 33, "sub": [28, 33, 50, 54], "49": [33, 37, 38], "secur": [33, 41], "boolean": [33, 49], "idenfi": 33, "backward": 33, "51": [33, 38, 49], "assertionerror": 33, "traceback": 33, "lt": [33, 34, 39, 49], "ipython": 33, "f30d4b630726": 33, "gt": [33, 34, 39, 49], "modul": [28, 33], "56": [33, 38, 39], "collaps": 33, "aggreg": [33, 55], "57": [33, 35, 38, 39, 41], "arrests_collaps": 33, "as_index": [33, 35], "4504": [33, 35], "9377": 33, "3569": [33, 35], "257": [33, 35, 37, 38], "1651": 33, "19358": 33, "71342": 33, "2338": [33, 41], "alpin": [33, 35], "119": [33, 38, 41], "amador": [33, 35], "305": [33, 38], "306": [33, 38, 41], "683": [33, 35, 38, 41], "149": [33, 38, 39, 41, 49], "207": [33, 35, 38], "1365": [33, 37], "52": [33, 38, 39], "130": [33, 38, 41, 43, 49], "297": [33, 38], "679": [33, 37], "ball": 33, "sort_valu": [33, 38], "colusa": [33, 35], "89": [33, 37, 38], "138": [33, 38, 41], "655": [33, 41], "1596": 33, "3851": 33, "1089": 33, "929": 33, "7539": 33, "25788": 33, "590": [33, 38], "del": [33, 35], "nort": [33, 35], "183": [33, 38, 41], "357": [33, 38, 49], "1078": 33, "198": [33, 38], "574": [33, 49], "118": [33, 37, 38], "1080": 33, "3266": 33, "fresno": [33, 35], "2367": [33, 41], "3855": 33, "804": [33, 41], "105": [33, 38], "719": [33, 38, 41], "7850": 33, "37907": 33, "1178": [33, 42], "glenn": [33, 35], "67": [33, 37, 38, 49], "111": [33, 37, 38, 41], "38": [33, 34, 38, 42, 43], "250": [33, 38, 41, 42], "766": [33, 38, 41, 43], "humboldt": [33, 35], "211": [33, 35, 37, 38, 39, 41], "593": [33, 37, 39, 41], "196": [33, 38, 39, 41], "33": [33, 37, 38, 39, 41, 43], "212": [33, 39], "1245": 33, "5953": 33, "197": [33, 35, 37, 38], "imperi": [33, 35], "565": [33, 37, 41], "1045": 33, "97": [33, 38, 39], "163": [33, 38, 39, 41], "1886": 33, "6512": 33, "inyo": [33, 35], "158": [33, 38, 41, 43], "315": [33, 38, 41], "663": [33, 37, 41], "kern": [33, 35], "1732": 33, "3530": 33, "1098": 33, "401": [33, 39], "6896": 33, "26372": 33, "890": [33, 41], "63": [33, 38, 43], "64": [33, 38, 39, 41, 42], "mids_data_prep": 33, "county_demograph": 33, "census_ts_nominal_counti": 33, "gisjoin": [33, 55], "statefp": 33, "statenh": 33, "countyfp": 33, "countynh": 33, "b18aa": 33, "cv4aa": 33, "cv4ab": 33, "cv4ac": 33, "cv4ad": 33, "cv4ae": 33, "cv4af": 33, "cv4ag": 33, "cv4ah": 33, "cv4ai": 33, "cv4aj": 33, "g0100010": 33, "1970": 33, "alabama": 33, "autauga": 33, "17511": 33, "g0100030": 33, "baldwin": 33, "48650": 33, "g0100050": 33, "barbour": 33, "12134": 33, "g0100070": 33, "bibb": 33, "9953": 33, "g0100090": 33, "blount": 33, "26205": 33, "65": [33, 34, 38], "66": [33, 38, 39, 41], "b18ab": 33, "b18ac": 33, "b18ad": 33, "b18ae": 33, "dropna": 33, "514559": 33, "265294": 33, "17996": 33, "103958": 33, "41887": 33, "744282": 33, "130795": 33, "244309": 33, "21154": 33, "884": [33, 38], "232": [33, 35, 38, 41], "6471": [33, 38], "73615": 33, "2622": 33, "1859": 33, "31207": 33, "31742": 33, "2272": 33, "3195": 33, "159": [33, 38, 39, 43], "41873": 33, "350": [33, 38, 41, 42, 43], "799": [33, 38], "28913": 33, "3357": 33, "san": [33, 35], "joaquin": [33, 35], "266886": 33, "19288": 33, "4474": [33, 41], "21738": 33, "237799": 33, "18443": 33, "23765": 33, "631": 33, "29087": 33, "845": [33, 38, 39, 41], "2447": 33, "34325": 33, "9608": 33, "621490": 33, "88813": 33, "5830": 33, "183969": 33, "48714": 33, "549409": 33, "86851": 33, "109486": 33, "2636": 33, "32658": 33, "72081": 33, "1962": 33, "3803": 33, "73874": 33, "16056": 33, "12800": 33, "ventura": [33, 35], "565804": 33, "15163": 33, "8068": 33, "197342": 33, "36941": 33, "400868": 33, "13082": 33, "57841": 33, "1371": [33, 41], "18589": 33, "164936": 33, "2081": 33, "7316": 33, "138882": 33, "18352": 33, "12764": 33, "madera": [33, 35], "94456": 33, "5629": 33, "4136": 33, "40344": 33, "6300": 33, "57380": 33, "5009": 33, "4430": 33, "649": [33, 38, 41], "2405": 33, "37076": 33, "620": [33, 38, 41], "2670": 33, "36731": 33, "3895": [33, 41], "9635": 33, "sacramento": [33, 35], "783240": 33, "121804": 33, "13359": 33, "233704": 33, "71392": 33, "706655": 33, "118073": 33, "148459": 33, "3406": 33, "51016": 33, "76585": 33, "3731": [33, 41], "7063": 33, "88135": 33, "20376": 33, "9606": 33, "36982": 33, "304": [33, 38, 39], "705": [33, 37, 38, 41, 42], "1222": 33, "1341": 33, "35465": 33, "300": [28, 33, 38, 41, 45], "965": [33, 35, 38, 41], "1019": 33, "1517": [33, 38, 41], "123": [33, 38, 39, 41], "322": [33, 35, 38, 41], "6510": 33, "tehama": [33, 35], "45593": 33, "256": [33, 38], "966": [33, 38], "2810": 33, "43049": 33, "246": [33, 38, 39, 41], "1174": 33, "32": [33, 34, 38, 39, 41], "2544": [33, 38, 41], "145": [33, 38, 42], "2425": 33, "3321": [33, 41], "18469": 33, "177": [33, 37, 38, 39], "299": [33, 38, 39], "17811": 33, "363": [33, 39, 41], "658": [33, 37], "255": [33, 38, 41], "12792": 33, "solano": [33, 35], "210751": 33, "60750": 33, "3212": [33, 38, 41], "107273": 33, "31358": 33, "168628": 33, "58743": 33, "64134": 33, "1463": [33, 41], "21020": 33, "42123": 33, "3115": [33, 41], "41773": 33, "10338": 33, "68": [33, 34, 37, 38, 41, 43], "race_table_var": 33, "fillna": 33, "71": [33, 37, 38, 49], "72": [33, 37, 38, 41, 42], "3319": 33, "1105379": [33, 35], "3320": 33, "1097": [33, 39], "19314": 33, "3322": [33, 41], "143851": 33, "3323": [33, 41], "20710": 33, "float64": [33, 35, 38], "73": [33, 37, 38, 41], "58": [33, 35, 37, 38, 39, 41], "arrests_w_pop": 33, "_merg": [33, 35], "left_onli": 33, "2030": 33, "right_onli": 33, "76": [33, 38, 39, 42], "fill": [33, 47], "interpol": 33, "wave": [33, 60], "1995": [33, 41], "78": [33, 38], "ascend": 33, "interpolated_total_popul": 33, "1981": [33, 35], "4699": [33, 35, 38], "3926": [33, 35], "1122759": [33, 35], "1982": [33, 35, 39, 41], "4389": [33, 35], "4436": [33, 35], "1140139": [33, 35], "174": [33, 38, 39, 41], "4500": [33, 35], "5086": [33, 35, 41], "1157519": [33, 35], "1984": [33, 35, 39], "3714": [33, 35], "5878": [33, 35], "1174900": [33, 35], "white_arrests_w_pop": 33, "miniconda3": [33, 43, 49], "lib": [33, 43, 49], "python3": [33, 43, 49], "packag": [33, 43, 49], "ipykernel_launch": 33, "py": [33, 35, 43, 49], "userwarn": 33, "reindex": 33, "290": [33, 35, 38, 39], "1985": [33, 39], "4199": 33, "7139": [33, 38], "1192280": 33, "348": [33, 35, 37, 38, 39], "1986": [33, 39], "4915": [33, 39], "7720": 33, "1209660": 33, "406": [33, 38, 41, 43], "1987": 33, "5137": 33, "10333": 33, "1227041": 33, "464": [33, 38, 41], "1988": [33, 39], "5087": 33, "12574": 33, "1244421": 33, "522": [33, 38, 41, 42], "1989": 33, "5551": 33, "14056": 33, "1261801": 33, "580": [33, 38, 41], "6156": 33, "12378": 33, "1279182": 33, "638": [33, 35, 38, 39, 41], "1991": 33, "5942": 33, "11408": 33, "1295637": 33, "696": [33, 35, 38, 41], "1992": 33, "6162": [33, 41], "10712": 33, "1312093": 33, "754": [33, 35, 38], "1993": 33, "9656": 33, "1328549": 33, "812": [33, 41, 49], "1994": 33, "5911": 33, "9764": 33, "1345005": 33, "3844": 33, "3265": 33, "1361461": 33, "928": [33, 38, 41, 43], "1996": 33, "5610": 33, "7322": [33, 41], "1377917": 33, "986": 33, "1997": 33, "5976": 33, "7646": 33, "1394373": 33, "1044": 33, "1998": 33, "5636": 33, "7852": 33, "1410829": 33, "1102": 33, "1999": [33, 41], "5168": 33, "8985": 33, "1427285": 33, "1160": 33, "4356": 33, "7414": 33, "1443741": 33, "1218": 33, "4478": 33, "6486": 33, "1450394": 33, "1276": [33, 37], "2002": [33, 35], "4424": 33, "6414": 33, "1457047": 33, "1334": [33, 37, 41], "2003": [33, 35], "4749": 33, "6708": 33, "1463700": 33, "1392": 33, "4301": 33, "6886": [33, 41], "1470353": 33, "1450": 33, "4090": 33, "6328": 33, "1477006": 33, "1508": [33, 41], "4042": 33, "5879": 33, "1483659": 33, "1566": 33, "4443": 33, "6071": 33, "1490312": 33, "1624": 33, "4336": 33, "5893": 33, "1496965": 33, "1682": [33, 38], "4318": 33, "5749": 33, "1503618": 33, "1740": 33, "3823": 33, "5357": [33, 41], "1510271": 33, "1798": 33, "2011": [33, 60], "3504": [33, 42], "3957": 33, "1856": 33, "3686": 33, "1914": 33, "3183": 33, "4337": 33, "1972": [33, 41], "2014": 33, "3491": 33, "4805": 33, "2015": [33, 38], "3452": [33, 38], "1946": 33, "2088": [33, 37], "3513": 33, "1762": [33, 41], "2146": 33, "3965": 33, "1279": 33, "2204": 33, "4132": 33, "1062": 33, "lo": [33, 35], "angel": [33, 35], "37964": 33, "29319": 33, "7477503": 33, "37610": 33, "27427": 33, "7616069": 33, "134": [33, 38, 39, 41], "36648": 33, "28807": 33, "7754635": 33, "192": [33, 35, 38, 41], "33858": 33, "36565": 33, "7893201": 33, "35303": 33, "44331": 33, "8031767": 33, "308": [33, 38, 41], "35659": 33, "51015": 33, "8170333": 33, "366": [33, 41], "45254": 33, "61035": 33, "8308899": 33, "424": [33, 38, 39, 41], "46020": 33, "61713": 33, "8447465": 33, "482": [33, 38, 41, 43], "51021": 33, "69363": 33, "8586031": 33, "540": [33, 41, 43], "59012": 33, "69667": 33, "8724597": 33, "598": [33, 39, 41], "64790": 33, "55218": 33, "8863164": 33, "656": [33, 38, 41], "58701": 33, "42583": 33, "8928781": 33, "714": [33, 49], "59215": 33, "43212": 33, "8994398": 33, "772": [33, 38, 49], "55079": 33, "43406": 33, "9060016": 33, "830": 33, "53467": 33, "48161": 33, "9125633": 33, "888": [33, 39, 41], "53218": 33, "45535": 33, "9191251": 33, "946": [33, 38], "51277": 33, "44102": 33, "9256868": 33, "1004": [33, 41], "50573": 33, "46951": 33, "9322485": 33, "46886": 33, "43947": 33, "9388103": 33, "1120": [33, 42], "43655": 33, "38253": 33, "9453720": 33, "40440": 33, "33499": 33, "9519338": 33, "1236": [33, 38], "42095": 33, "33570": 33, "9549264": 33, "1294": 33, "39768": 33, "36060": 33, "9579191": 33, "1352": [33, 37], "39633": 33, "42002": 33, "9609118": 33, "1410": 33, "37397": 33, "46060": 33, "9639044": 33, "1468": 33, "36322": 33, "48411": 33, "9668971": 33, "1526": 33, "35384": 33, "47549": 33, "9698898": 33, "1584": 33, "36250": 33, "45112": 33, "9728824": 33, "1642": 33, "36452": 33, "40928": 33, "9758751": 33, "1700": 33, "35319": 33, "32193": 33, "9788678": 33, "1758": 33, "33147": 33, "33920": 33, "9818605": 33, "1816": 33, "30818": 33, "33244": 33, "1874": 33, "29601": 33, "33775": 33, "1932": 33, "27159": 33, "35972": 33, "28251": 33, "36372": 33, "2048": 33, "28194": 33, "9753": 33, "2106": [33, 41], "27804": 33, "8970": 33, "2164": 33, "28746": 33, "7302": [33, 38], "2222": 33, "28865": 33, "6412": 33, "86": [33, 37, 38, 42], "renam": [33, 55], "to_csv": 33, "githubusercont": [34, 35, 39, 49], "61": [34, 38, 41], "411101": 34, "331110": 34, "92": [34, 38, 42], "779361": 34, "342067": 34, "222701": 34, "481508": 34, "588374": 34, "815540": 34, "472719": 34, "337757": 34, "2f": [34, 37, 41, 42], "77": [34, 37, 38, 42, 49], "std": [34, 35, 37, 38, 39, 41, 42, 43, 49], "expend": 34, "94": [34, 38], "corr": 34, "iloc": 34, "alt": [34, 35, 37, 38], "enabl": [34, 35, 38], "mimetyp": 34, "grumpi": 34, "displai": 34, "hist": 34, "dict": 34, "chart": [34, 35, 37, 38], "mark_bar": 34, "q": [34, 35], "vegalit": 34, "frontend": 34, "viz": 34, "io": 34, "user_guid": 34, "troubleshoot": 34, "html": 34, "17": [34, 35, 37, 38, 39, 41, 42, 43, 49], "mark_point": [34, 35, 37], "legal": [35, 41, 54], "marijuana": 35, "drug_rat": 35, "100_000": 35, "isin": 35, "394": 35, "457331": 35, "551801": 35, "289": [35, 38, 41], "027292": 35, "271": [35, 38, 39, 41], "709690": 35, "312": [35, 37, 38, 41], "159811": 35, "209": [35, 38, 41], "450372": 35, "276": [35, 38, 39, 41], "018946": 35, "358": [35, 38, 42], "305449": 35, "228": [35, 37, 38], "874748": 35, "473": [35, 38, 41, 43], "876866": 35, "411": [35, 38, 39, 41], "831533": 35, "499": [35, 38, 41], "572904": 35, "460": [35, 41], "694485": 35, "380": [35, 37, 38], "164074": 35, "432": [35, 38, 41], "659812": 35, "king": [35, 51, 52], "035930": 35, "lake": 35, "369": [35, 38], "871380": 35, "lassen": 35, "147": [35, 38, 42], "165387": 35, "403": [35, 38, 39, 41], "990696": 35, "261": [35, 37, 38, 41], "117203": 35, "marin": 35, "190": [35, 37, 38, 39], "030996": 35, "mariposa": 35, "969880": 35, "mendocino": 35, "599": [35, 37, 41], "799767": 35, "merc": 35, "441": [35, 38], "290221": 35, "modoc": 35, "204960": 35, "mono": 35, "272": [35, 38, 41], "602759": 35, "monterei": 35, "248": [35, 38, 41], "094595": 35, "napa": 35, "247": [35, 37, 38, 39, 41], "823080": 35, "nevada": 35, "235": [35, 37, 38, 41], "403323": 35, "orang": 35, "277": [35, 38, 41], "267304": 35, "placer": 35, "583530": 35, "pluma": 35, "492": [35, 41], "429787": 35, "riversid": 35, "291": [35, 37, 38, 39, 41], "458787": 35, "375": [35, 37, 38, 39, 57], "134854": 35, "benito": 35, "210": [35, 37, 38, 39, 41], "849718": 35, "bernardino": 35, "465": [35, 38, 39, 41], "333881": 35, "diego": 35, "249": [35, 38, 39, 41], "177208": 35, "francisco": 35, "873": [35, 38], "961738": 35, "824184": 35, "lui": 35, "obispo": 35, "583757": 35, "mateo": 35, "214": [35, 38, 41], "668466": 35, "santa": 35, "barbara": 35, "225": [35, 37, 38, 41, 42], "267192": 35, "clara": 35, "348166": 35, "cruz": 35, "577393": 35, "shasta": 35, "470436": 35, "sierra": 35, "221": [35, 38, 41], "364059": 35, "siskiy": 35, "352": [35, 37, 38, 41], "795109": 35, "388": [35, 38, 39, 41, 42, 49], "929454": 35, "sonoma": 35, "331": [35, 38, 41], "905273": 35, "stanislau": 35, "457": [35, 38, 41], "198700": 35, "sutter": 35, "695398": 35, "585": [35, 37, 39, 41], "300555": 35, "triniti": 35, "427": [35, 38, 41], "859162": 35, "tular": 35, "447": [35, 38, 41], "774248": 35, "tuolumn": 35, "405": 35, "897681": 35, "264": [35, 38, 39, 41], "878553": 35, "yolo": 35, "333": [35, 38, 41], "788690": 35, "yuba": 35, "351": [35, 38], "561100": 35, "301": [35, 38, 41], "8092992253924": 35, "astyp": [35, 37, 41, 43, 49], "int": [35, 37, 41, 43, 49], "left_on": 35, "right_index": 35, "outer": 35, "875683": 35, "407": [35, 41, 43], "462056": 35, "349": 35, "674236": 35, "418": [35, 37, 38], "522474": 35, "389": [35, 38, 41, 49], "075162": 35, "384": [35, 37], "952860": 35, "439": [35, 41], "387694": 35, "762215": 35, "297813": 35, "316": [35, 38, 39, 43], "111956": 35, "post_2010": 35, "arrests_sub": 35, "y_t1_post": 35, "y_t1_pre": 35, "y_t0_post": 35, "y_t0_pre": 35, "799650070477355": 35, "418061484004568": 35, "api": [35, 37, 38, 39, 41, 42, 43, 49], "get_robustcov_result": [35, 42, 43, 49], "cov_typ": [35, 39, 42, 43], "dep": [35, 37, 38, 39, 41, 42, 43, 49], "adj": [35, 37, 38, 39, 41, 42, 43, 49], "sun": [35, 43, 49], "2023": [35, 37, 38, 39, 41, 42, 43], "prob": [35, 37, 38, 39, 41, 42, 43, 49], "45e": [35, 38, 41], "2094": 35, "aic": [35, 37, 38, 39, 41, 42, 43, 49], "4196": 35, "344": [35, 38, 41, 49], "bic": [35, 37, 38, 39, 41, 42, 43, 49], "4212": 35, "coef": [35, 37, 38, 39, 41, 42, 43, 49], "err": [35, 37, 38, 39, 41, 42, 43, 49], "025": [35, 37, 38, 39, 41, 42, 43, 49], "975": [35, 37, 38, 39, 41, 42, 43, 49], "319": [35, 37, 38], "7820": 35, "131": [35, 38, 41], "284": [35, 37, 38, 41], "463": 35, "355": [35, 38, 41], "101": [11, 35, 38, 39, 41, 43], "3816": 35, "892": [35, 38, 41], "959": 35, "055": [35, 37, 38, 39], "189": [35, 38, 39, 41], "426": [35, 41], "106": [35, 38, 39, 41, 42], "8289": 35, "385": 35, "568": [35, 41], "001": [35, 38, 39, 41, 42, 49], "153": [35, 37, 38, 41], "657": [35, 38, 41, 42], "4181": 35, "869": [35, 38], "393": [35, 41, 42], "203": [35, 38, 41], "367": [35, 38], "omnibu": [35, 37, 38, 39, 41, 42, 43, 49], "53": [35, 37, 38, 39, 41, 43, 49], "945": [35, 38, 41], "durbin": [35, 37, 38, 39, 41, 42, 43, 49], "watson": [35, 37, 38, 39, 41, 42, 43, 49], "741": [35, 41], "jarqu": [35, 37, 38, 39, 41, 42, 43, 49], "bera": [35, 37, 38, 39, 41, 42, 43, 49], "jb": [35, 37, 38, 39, 41, 42, 43, 49], "81": [35, 37, 38, 42, 43, 49], "621": [35, 38, 41, 42], "skew": [35, 37, 38, 39, 41, 42, 43, 49], "89e": [35, 37, 38, 41], "kurtosi": [35, 37, 38, 39, 41, 42, 43, 49], "cond": [35, 37, 38, 39, 41, 42, 43, 49], "data_transform": [35, 38], "data_serv": [35, 38], "arrests_long": 35, "grouped_mean": 35, "loess": [35, 38], "lightblu": 35, "transform_regress": 35, "mark_lin": [35, 38], "append": 35, "layer": 35, "TO": 35, "eq": 35, "hte": 35, "_thought_": 35, "cluster_ent": [35, 39, 43], "mislead": 35, "fault": 35, "linearmodel": [35, 39, 43], "arrests_for_panelol": 35, "set_index": [35, 39, 43], "mod": [35, 39, 43], "entityeffect": [35, 39, 43], "timeeffect": [35, 39], "drop_absorb": 35, "cluster_tim": 35, "h_8_rwsn5hvg9mhp0txgc_s9v6191b": 35, "ipykernel_84356": 35, "3310218310": 35, "absorbingeffectwarn": 35, "absorb": [35, 39], "0013": 35, "0109": 35, "0155": 35, "0104": 35, "1858": 35, "cov": [35, 39, 43], "3829": 35, "5366": [35, 41], "ob": [35, 39, 43], "0000": [35, 37, 39, 43], "max": [35, 38, 39, 43], "1899": 35, "6633": 35, "stat": [11, 28, 35, 38, 39, 42, 43, 49], "ci": [35, 39, 43], "upper": [35, 39, 43], "023": [35, 37, 41, 42], "4358": 35, "925": [35, 37, 38, 39, 41], "089": [35, 38, 41], "poolabl": [35, 39, 43], "282p": 35, "0000distribut": [35, 43], "0097": 35, "0168": 35, "0588": 35, "0150": 35, "4108": 35, "6568": 35, "0101": 35, "6634": 35, "1031": 35, "362": [35, 37, 38, 41], "6320": 35, "078": [35, 38, 41], "3540": 35, "619": [35, 37, 41], "3654453326": 35, "unconstrain": 36, "kupp": 36, "constrain": [28, 36], "umbrella": 36, "dry": 36, "sunni": 36, "pont": 37, "le": 37, "4723": 37, "3200": 37, "199": [37, 38, 41, 42], "379": [37, 38], "38e": [37, 38], "09": [37, 38, 39, 49], "678": [37, 38, 41], "1360": 37, "nonrobust": [37, 38, 39, 41, 43], "5143": 37, "0893": 37, "807": [37, 38, 41], "442": [37, 38, 41], "4519": 37, "521": [37, 41], "5766": 37, "4202": 37, "2440": 37, "634": [37, 38, 41, 49], "243": [37, 38], "2937": 37, "904": [37, 41], "5466": 37, "584": [37, 41], "244": [37, 38, 41, 42], "04e": [37, 38, 41, 42], "126": [37, 38, 41], "specifi": [37, 38, 39, 41, 43, 48, 52, 53, 62], "202": [37, 38], "596": 37, "566": [37, 38, 39], "04": [37, 38, 39, 41, 42, 43, 49], "14e": [37, 38, 41], "662": [37, 38, 41], "1336": [37, 41], "1350": 37, "782": [37, 38, 41], "5353": 37, "1612": 37, "628": [37, 41], "485": [37, 38, 41], "4000": 37, "484": [37, 39, 41, 43], "2435": [37, 38], "414": [37, 39, 41], "3278": 37, "9827": 37, "671": [37, 38], "826": [37, 38, 41], "881": [37, 41], "1938": 37, "4619": [37, 41], "591": [37, 38, 39, 41], "1977": 37, "1796": 37, "711": [37, 39, 41], "781": [37, 38], "007": [37, 38, 41, 42, 49], "558": [37, 38], "3396": 37, "068": [37, 38, 42, 49], "9634": [37, 41], "702": [37, 41], "797": [37, 39, 41], "563": [37, 41], "364": [37, 41], "736": [37, 38, 39], "7997": 37, "309": [37, 38, 39, 41], "009": [37, 38, 41, 42, 49], "020": [37, 38, 39, 41, 49], "1353": [37, 38], "120": [28, 37, 38, 39, 41], "182": [37, 38, 41, 43], "9667": 37, "819": [37, 38, 41], "541": [37, 38], "128": [37, 38], "646": [37, 38, 41, 42], "579": [37, 38, 41], "179": [37, 38, 39, 41], "409": [37, 38, 41], "01e": [37, 38, 41, 43], "36e": [37, 38, 41], "arestrong": [37, 41], "multicollinear": [37, 41], "bewar": 37, "inadvert": [37, 62], "air": [37, 41, 48, 55], "531": [37, 38, 41], "11e": [37, 38, 41], "616": [37, 41, 42], "1254": [37, 41], "1852": 37, "4340": 37, "2284": 37, "811": [37, 38, 41], "421": 37, "6423": 37, "964": [37, 38, 41, 42], "2719": 37, "096": [37, 38, 41, 42], "951": [37, 38, 42], "3308": 37, "1676": 37, "704": [37, 38, 42], "567": [37, 38, 41], "573": [37, 41], "2403": 37, "746": [37, 38], "4306": 37, "408": [37, 38], "9162": 37, "1539": 37, "879": [37, 38], "383": [37, 38, 41], "1728": 37, "4434": 37, "011": [37, 38, 41, 42], "994": [37, 38, 41], "5742": [37, 39], "1609": 37, "415": 37, "618": [37, 38, 42], "539": [37, 39, 41], "2225": 37, "857": [37, 41], "4215": 37, "006": [37, 38, 39, 41, 42, 49], "1325": 37, "1545": [37, 41], "1705": 37, "943": [37, 41], "777": [37, 38, 39, 41], "440": [37, 38, 41, 42], "429": [37, 38], "4738": [37, 41], "738": [37, 41], "3240": [37, 41], "0049": 37, "012": [37, 38, 41, 42, 49], "015": [37, 41, 42, 49], "1625": 37, "4854": [37, 41], "833": [37, 38, 41], "1644": 37, "4650": [37, 38], "769": [37, 38, 41], "136": [37, 38, 41], "037": [37, 38, 39, 41, 42, 49], "103": [37, 38], "848": [37, 38, 41], "3185": 37, "082": [37, 41, 42], "7544": 37, "883": 37, "988": [37, 41], "051": [37, 38, 41], "520": [37, 41], "755": [37, 38, 42], "1823": 37, "341": [37, 38, 39], "695": [37, 38, 41], "031": [37, 41, 42], "1438": 37, "912": [37, 41], "452": [37, 38], "0412": [37, 42], "472": [37, 38, 41, 43], "612": [37, 38, 41], "112": [37, 38, 39, 41, 42], "910": [37, 38, 39, 41], "992": [37, 41], "610": [37, 38], "430": [37, 38, 41], "697": [37, 41], "181": [37, 38, 39, 41], "82e": [37, 38, 41, 49], "478": [37, 38, 41], "54e": [37, 38, 41], "353": [37, 41, 43], "390": [37, 38, 43], "20e": 37, "675": [37, 38, 41], "1369": 37, "4925": 37, "378": [37, 38, 41], "835": [37, 41], "4169": 37, "438": [37, 38, 41], "5680": 37, "562": [37, 38, 41, 42, 49], "642": [37, 38, 41], "7895": 37, "650": [37, 38, 43], "326": [37, 38, 41, 49], "654": [37, 41], "1939": 37, "931": [37, 38, 41], "3977": 37, "7333": 37, "639": [37, 41, 49], "2570": 37, "953": [37, 38, 41], "5384": 37, "513": [37, 38, 42, 43], "8105": 37, "1595": [37, 41], "941": [37, 38], "1170": 37, "193": [37, 38], "5195": 37, "814": [37, 41], "269": [37, 38, 39, 41, 43], "940": [37, 38, 41], "79e": [37, 38, 41], "635": [37, 38, 41], "01": [37, 42, 49], "21e": [37, 38, 41], "660": [37, 38, 41, 42], "391": 37, "7038": 37, "1588": 37, "834": [37, 38, 41], "806": [37, 41, 42, 49], "3563": 37, "030": [37, 38, 41, 42], "2779": 37, "622": [37, 38, 41, 43], "2929": 37, "3402": 37, "1573": [37, 41], "413": [37, 41], "4285": 37, "267": [37, 38, 39, 41], "1354": 37, "9011": 37, "760": [37, 41], "079": [37, 38, 42, 49], "162": [37, 38, 41], "552": [37, 38, 41], "2872": [37, 41], "354": [37, 38], "2797": 37, "6787": 37, "1381": [37, 41], "501": [37, 38, 39, 41], "047": [37, 38, 41, 42, 49], "5555": 37, "167": [37, 38, 39, 41], "6417": 37, "330": [37, 38, 41], "048": [37, 38, 41, 42], "8717": 37, "302": [37, 38], "195": [37, 38, 41], "017": [37, 38, 41, 42, 49], "1340": 37, "056": [37, 38, 49], "133": [37, 38, 41], "688": [37, 38], "6296": 37, "972": [37, 38, 42], "115": [37, 38, 41, 49], "038": [37, 38, 39, 42], "710": [37, 38, 39], "549": 37, "874": [37, 38, 41, 42], "311": [37, 38, 39, 41], "63e": [37, 38, 41], "885": [37, 38], "39e": [37, 38, 41], "grandmoth": 37, "ago": 37, "mom": 37, "admittedli": [37, 38], "databas": [37, 55], "param": [37, 41], "4152": 37, "m2": 37, "564": [37, 38], "12e": [37, 38, 41], "661": [37, 41], "1338": [37, 41], "232e": 37, "4465": 37, "758": [37, 38], "3404": 37, "206": [37, 38, 42], "176e": 37, "2796": 37, "6184": [37, 38], "73e": [37, 38, 41], "259": [37, 38, 41, 43], "8139": [37, 41], "109": [37, 38, 39], "998": [37, 38, 39, 41, 42], "021": [37, 38, 41], "479": [37, 38, 41], "371": 37, "314": [37, 38, 41], "4806": 37, "360": [37, 38, 39, 49], "876": [37, 38], "005": [37, 38, 39, 42], "532": [37, 41], "764": [37, 41, 49], "96": [37, 38], "4327": 37, "856": [37, 38, 41], "008": [37, 38, 42], "723": [37, 41], "142": [37, 38, 41, 42], "5821": 37, "958": [37, 38, 41], "515": [37, 41], "1123": 37, "222": [37, 38, 41, 43], "154": [37, 38, 41], "058": [37, 38, 41, 42], "4670": 37, "839": [37, 41], "016": [37, 38, 39, 41, 42, 49], "813": [37, 38], "121": [37, 38, 41, 42], "595": [37, 38], "07e": [37, 38, 41], "204": [37, 38, 39, 41], "premium": 37, "milag": 37, "151404": 38, "wv": [38, 39], "123453": 38, "251": [38, 42], "tx": [38, 39], "187982": 38, "ma": [38, 39], "unincorpor": 38, "122356": 38, "tn": [38, 39], "nonprofit": 38, "210750": 38, "42900": 38, "ia": [38, 39], "063": 38, "sat": [38, 39, 41, 42], "56e": [38, 41], "63018": 38, "5515": 38, "260e": 38, "5513": 38, "261e": 38, "887e": 38, "669": 38, "95e": [38, 41], "416e": 38, "735": [38, 41], "820": [38, 41], "242": [38, 39, 41], "27e": [38, 41], "2214": 38, "974": [38, 41, 49], "10578": 38, "287": [38, 39], "608": 38, "iffi": 38, "frequenc": [38, 50, 60], "chi2": [38, 42], "_i_": 38, "freq": 38, "032": [38, 39, 41], "122": [38, 41, 43], "74e": [38, 41, 43], "7675": [38, 39], "11150": 38, "536e": 38, "11146": 38, "539e": 38, "4382": 38, "420": [38, 41, 43, 49], "449": [38, 41], "1206": 38, "507": [38, 41], "152": 38, "2398": 38, "014": [38, 42], "682": 38, "266": [28, 38, 41], "213": [38, 41, 42], "0367": 38, "024": 38, "069": [38, 41], "46681": 38, "1670": 38, "377": [38, 41], "438205": 38, "317647": 38, "198413": 38, "474900": 38, "ctab": [38, 42], "4282": 38, "3340": 38, "324": [38, 41], "1212": 38, "523": [38, 41], "dof": [38, 42], "chi2_conting": [38, 42], "2993875943569016e": 38, "mo": [38, 39], "000000": 38, "700": [38, 41], "va": [38, 39], "fl": [38, 39], "md": [38, 39], "882353": 38, "332106": 38, "nj": [38, 39], "IN": 38, "171": [38, 41, 49], "mn": [38, 39], "390058": 38, "207616": 38, "262319": 38, "375000": 38, "500000": 38, "grad": [38, 42], "remot": 38, "quadrat": [38, 49, 52], "2760": [38, 41], "2551": 38, "2397": 38, "173": [38, 39, 41], "tini": 38, "576": [38, 39], "275": [38, 41, 42], "230": [38, 41], "223": [38, 39, 41], "263": [38, 39, 41], "285": [38, 41], "7809": 38, "740": [38, 39, 41], "706": [38, 39, 41], "615": [38, 41], "387": [38, 39, 41], "337": [38, 41, 49], "for_match": 38, "reset_index": 38, "31200": 38, "20020": 38, "22859": 38, "73860": 38, "5510": 38, "33800": 38, "5511": 38, "23920": 38, "5512": 38, "37440": 38, "5514": 38, "26000": 38, "1204727749": 38, "8949614": 38, "129": [38, 39, 41], "5378": 38, "1204742613": 38, "479154": 38, "1205072671": 38, "3262901": 38, "508": [38, 49], "5390": 38, "1205171280": 38, "4727237": 38, "509": 38, "5392": 38, "1210524158": 38, "7436352": 38, "511": [38, 41], "110": [38, 39, 41, 42], "5405": 38, "1210539313": 38, "933855": 38, "for_p": 38, "199313e": 38, "199422e": 38, "204728e": 38, "204743e": 38, "205073e": 38, "205171e": 38, "210524e": 38, "210539e": 38, "fast": 38, "5509": 38, "930000": 38, "108": [38, 41, 42], "070189": 38, "240000": 38, "134e": 38, "3736": 38, "7480": 38, "5306": 38, "7506": 38, "2119": 38, "469e": 38, "018": [38, 41, 42, 49], "92e": [38, 41], "036": [38, 41, 42], "378e": 38, "019": [38, 39, 41, 49], "86e": [38, 41], "18e": [38, 41], "83e": 38, "040": [38, 41, 42], "860": [38, 41], "227": [38, 41], "234": 38, "41e": [38, 41], "294": [38, 39, 41], "851": [38, 41], "057": [38, 42], "19e": [38, 41], "61753": 38, "235e": 38, "5308": 38, "909e": 38, "293": 38, "84e": [38, 41], "98e": 38, "374e": [38, 41], "763": [38, 41], "003": [38, 39, 41, 42, 49], "22e": [38, 41], "52e": [38, 41], "2934": 38, "035": [38, 41, 42], "33100": 38, "529": 38, "util": [38, 41], "post_process": 38, "matching_object": 38, "36": [38, 39], "238": [38, 41], "786": [38, 39, 41, 42, 49], "62e": 38, "132": [38, 42], "61189": 38, "230e": 38, "4984": 38, "252e": 38, "325": [38, 41], "761e": 38, "2429": [38, 41], "28e": [38, 41], "24e": [38, 41], "8344": 38, "9150": 38, "1067": 38, "818": 38, "6252": [38, 41], "476": [38, 39], "6753": 38, "9175": [38, 39], "1140": 38, "922": [38, 41], "8989": 38, "844": [38, 39], "4517": 38, "991": [38, 49], "3220": 38, "6308": 38, "1202": 38, "997": [38, 41], "677": [38, 39, 41], "5579": [38, 41], "862": [38, 41], "8584": 38, "0505": 38, "868": 38, "889": [38, 39, 41], "6882": 38, "03e": [38, 41], "251e": 38, "923": [38, 41], "43e": [38, 43], "266e": 38, "08e": [38, 39, 41], "46e": [38, 41], "9235": 38, "0616": 38, "1189": 38, "062": [38, 39], "767": [38, 39], "6903": [38, 41], "976": [38, 41, 42, 49], "16e": [38, 41], "347e": 38, "2975": 38, "342": [38, 41], "528": [38, 41], "7639": 38, "93e": 38, "114e": 38, "3231": [38, 41], "653": [38, 39, 41], "75e": [38, 41], "4799": 38, "550": [38, 39, 41, 42], "279e": 38, "3245": 38, "734": [38, 39, 41], "942": [38, 41], "6430": 38, "9142": 38, "9921": 38, "771": [38, 39], "4363": 38, "2990": 38, "164": [38, 39, 41], "23e": 38, "609": 38, "6378": 38, "3577": 38, "4178": 38, "527": [38, 41], "127": [38, 43], "1812": 38, "617": 38, "627e": [38, 41], "66e": [38, 41], "4103": 38, "023e": [38, 41], "3835": 38, "666": 38, "77e": [38, 41], "2706": [38, 41], "153e": 38, "3175": 38, "916": [38, 41], "78e": [38, 41], "5300": 38, "382e": 38, "5130": 38, "694": 38, "3762": 38, "501e": 38, "3427": 38, "926": [38, 39, 41], "17e": [38, 41], "8293": 38, "693": [38, 41], "418e": 38, "3040": [38, 41], "664": 38, "8218": 38, "6849": 38, "3954": 38, "3121": [38, 41], "194": [38, 41], "028": [38, 41, 49], "3e": [38, 41], "730": [38, 39, 41], "303": [38, 41, 43], "381e": 38, "3368": 38, "7209": 38, "369e": 38, "3918": 38, "748": [38, 41], "493": [38, 41, 42], "6007": 38, "9190": [38, 41], "4689": 38, "915": [38, 41], "218": [38, 39], "895": 38, "061e": 38, "3616": 38, "372": [38, 41, 42], "935": 38, "3523": [38, 41], "982": [38, 41], "601e": 38, "7731": 38, "556": 38, "070": [38, 41, 42], "909": [38, 41], "1788": 38, "7046": 38, "4787": 38, "690": 38, "374": 38, "709": [38, 41], "7597": 38, "684e": 38, "6555": 38, "900": [38, 42], "569": [38, 41], "010": [38, 39, 41, 42, 49], "97e": 38, "3989": 38, "067": 38, "4e": [38, 41], "4328": 38, "098": [38, 39, 41], "25e": [38, 41], "3776": 38, "3595": 38, "547": 38, "720": [38, 41, 49], "085": [38, 41, 42], "32e": [38, 41], "864": [38, 42], "826e": 38, "3694": 38, "55e": 38, "1e": [38, 41, 43], "242e": 38, "3244": 38, "837": [38, 41, 42], "828": 38, "88e": [38, 41], "6059": 38, "104e": 38, "3171": 38, "4824": 38, "506e": 38, "3496": 38, "8211": 38, "049": [38, 41, 42], "391e": 38, "3137": 38, "745": [38, 41], "7755": 38, "337e": [38, 41], "3345": [38, 41], "99e": [38, 41], "6814": 38, "297e": [38, 41], "5671": 38, "022": [38, 42], "1853": [38, 41], "4845": [38, 39], "0790": 38, "5203": 38, "125": [38, 41, 42], "5e": [38, 41], "5355": 38, "335": 38, "2248": 38, "8602": [38, 41], "5067": 38, "398": 38, "444": 38, "7685": 38, "469": [38, 41], "155e": 38, "4710": 38, "979": [38, 41], "2313": 38, "897": [38, 41], "7114": 38, "5598": 38, "3632": 38, "908": [38, 41], "050": [38, 39, 41, 49], "42e": 38, "538": [38, 41], "272e": 38, "3067": 38, "87e": [38, 41], "6711": 38, "3443": 38, "339": [38, 41], "6447": 38, "428": [38, 41], "9324": 38, "3384": 38, "603": 38, "6e": 38, "2688": 38, "932": [38, 41], "505e": 38, "3658": [38, 41], "7881": 38, "76e": [38, 41, 42, 49], "2856": 38, "308e": 38, "3310": 38, "96e": [38, 39, 41, 42], "6590": 38, "177e": [38, 41], "3252": [38, 41], "81e": [38, 39, 41, 49], "5391": 38, "516e": 38, "3677": 38, "7954": 38, "9224": 38, "6049": 38, "2690": 38, "524": [38, 41], "3949": 38, "365e": 38, "3460": 38, "095": [38, 39, 41], "944": 38, "6864": 38, "165e": 38, "3698": 38, "161": [38, 39, 41], "151": [38, 41], "002": [38, 41, 42, 49], "4404": 38, "402": [38, 39, 41], "2961": 38, "611": 38, "65e": [38, 41], "4889": 38, "7052": 38, "4496": 38, "3127": 38, "920": 38, "437": [38, 41], "9064": 38, "0230": 38, "3380": 38, "497": [38, 41], "681": [38, 41], "57e": [38, 41], "2436": 38, "761": [38, 41], "681e": 38, "3013": [38, 41], "09e": [38, 41], "8897": 38, "8731": 38, "3296": [38, 41], "699": [38, 41], "774": [38, 39, 41], "125e": 38, "7993": 38, "659": 38, "5580": 38, "69e": [38, 41], "13e": [38, 41, 43], "046": [38, 42], "459": [38, 41], "5880": [38, 41], "043": [38, 39, 41, 42], "703": [38, 41], "26e": [38, 39, 41], "887": [38, 39, 41], "270": [38, 41], "006e": 38, "8297": 38, "064": [38, 41, 42], "6201": 38, "7396": 38, "8710": 38, "376": [38, 41, 42], "707": [38, 39], "59e": [38, 41], "349e": 38, "8853": 38, "843": [38, 39, 41], "3865": 38, "6016": 38, "8132": [38, 41], "9691": 38, "581": [38, 41, 43], "535": [38, 39, 41], "7435": 38, "2407": 38, "7281": 38, "280": [38, 39, 41], "307": [38, 39, 41], "6839": 38, "8434": 38, "9441": 38, "604": [38, 41], "546": [38, 39, 41], "58e": [38, 41, 43], "5918": 38, "5652": 38, "5278": 38, "262": [38, 43], "4429": 38, "631e": 38, "9894": [38, 41], "838": [38, 41], "099": [38, 41, 49], "3086": 38, "114": [38, 41, 42], "499e": 38, "6719": 38, "026": [38, 39, 42], "1819": 38, "273": [38, 39], "8015": 38, "9218": [38, 41], "732": [38, 39, 41], "34e": [38, 41], "171e": [38, 41], "124e": 38, "8626": 38, "461": [38, 41], "8596": 38, "970": 38, "1156": [38, 41], "964e": 38, "9254": 38, "939": [38, 39, 41], "034": [38, 39, 41], "1498": 38, "286e": 38, "47e": [38, 41], "6046": 38, "434": [38, 39, 41], "53e": [38, 41], "265": [38, 39, 41], "8046": 38, "9594": [38, 41], "978": [38, 41], "85e": [38, 41], "91e": 38, "9042": 38, "15e": [38, 41], "3909": 38, "886": [38, 39, 41], "086e": [38, 41], "33e": [38, 41], "117": [38, 39, 41], "7e": [38, 41, 43], "5231": 38, "914e": 38, "8999": [38, 41], "435": [38, 41], "033": [38, 41], "68e": [38, 41], "1501": [38, 41], "949": [38, 39], "203e": 38, "630": [38, 41], "4151": 38, "5319": 38, "5648": 38, "737": [38, 41], "462": [38, 41], "6922": 38, "341e": 38, "7534": 38, "780": [38, 41], "075": 38, "1357": 38, "560": [38, 41], "719e": 38, "3924": 38, "381": 38, "49e": [38, 41], "9500": [38, 41], "746e": 38, "237": [38, 41, 49], "8624": 38, "6897": 38, "6770": 38, "72e": [38, 41], "146e": 38, "8462": 38, "899": [38, 41, 42], "8e": [38, 41], "741e": 38, "5589": 38, "6454": 38, "143e": 38, "5721": 38, "800": 38, "216": [38, 41], "658e": 38, "61e": [38, 41], "307e": [38, 41], "840": [38, 41, 42], "066": [38, 41, 42], "1514": 38, "84": [38, 49], "9730": 38, "48e": [38, 41], "3419": 38, "8384": 38, "9046": 38, "824e": [38, 41], "279": [38, 41], "201": [38, 41, 43], "9720": 38, "87": 38, "403e": 38, "205": [38, 41, 42], "027": [38, 41], "2667": 38, "326e": 38, "445": [38, 41], "574e": 38, "8531": [38, 41], "065": [38, 39, 41, 42], "985": [38, 39, 41], "816": [38, 41], "8870": 38, "8793": 38, "4968": [38, 41], "614": [38, 41], "785": [38, 41, 49], "074": [38, 39, 41, 42], "791": 38, "91": 38, "9791": 38, "880": [38, 39, 41, 42], "288": [38, 41], "8799": 38, "744": [38, 41], "687e": 38, "150": [38, 41], "6076": 38, "4593": 38, "8182": 38, "9052": 38, "5470": 38, "9761": 38, "637": [38, 41], "995": 38, "199e": 38, "5085": 38, "2e": [38, 41, 43], "219": [38, 39, 41], "7572": 38, "457e": 38, "9432": 38, "605": [38, 41], "31e": [38, 39, 41], "6080": [38, 41], "404": [38, 41], "98": [38, 41], "2197": 38, "3011": 38, "144": [38, 49], "9289": 38, "2093": 38, "3098": 38, "282": [38, 41], "3215": 38, "121e": 38, "7876": 38, "423": 38, "155": [38, 41], "4235": [38, 41], "9598": 38, "102": [38, 39, 41, 49, 55], "2597": 38, "8743": 38, "35e": [38, 41], "847": [38, 41], "014e": 38, "9066": 38, "7638": 38, "542": [38, 39, 41], "104": [38, 42], "252": [38, 41], "2631": [38, 41], "983": [38, 39, 41], "145e": 38, "107": [38, 39, 41, 42], "364e": 38, "71e": [38, 41], "962": 38, "4795": [38, 41], "8027": 38, "8034": 38, "597": [38, 39, 41], "551": [38, 39, 41, 42], "05e": [38, 41], "8716": 38, "5745": 38, "06e": [38, 41, 42], "822": 38, "946e": 38, "7351": 38, "516": [38, 41], "606": [38, 39, 41], "113": [38, 41, 42, 49], "293e": 38, "882": [38, 41], "908e": 38, "185": [38, 41], "236": [38, 39], "2798": 38, "0105": 38, "8785": 38, "2988": 38, "473e": 38, "577": [38, 41], "698e": 38, "124": [38, 39, 41, 42, 49], "4669": 38, "861": [38, 39, 41, 49], "704e": 38, "8908": 38, "767e": 38, "093": [38, 39, 41], "1747": 38, "291e": 38, "5091": 38, "347": [38, 41], "536": 38, "29e": [38, 41], "2931": 38, "559": [38, 41], "209e": 38, "795e": 38, "6490": 38, "6948": 38, "1460": 38, "9793": 38, "4111": [38, 41], "7736": 38, "668": [38, 42], "407e": 38, "338e": 38, "094": [38, 39], "274": [38, 41], "958e": 38, "5205": 38, "9462": [38, 41], "1890": [38, 49], "7889": 38, "2394": 38, "5638": [38, 41], "399": [38, 41], "3164": 38, "2175": [38, 39], "4884": 38, "51e": [38, 41], "467e": 38, "045": [38, 39, 41], "083e": [38, 41], "6228": 38, "141": [38, 41, 43], "4420": 38, "8608": 38, "5997": 38, "7337": 38, "137": [38, 41, 42, 49], "4087": 38, "2387": 38, "8100": 38, "7939": 38, "1789": 38, "8029": 38, "739": [38, 41], "989": 38, "323": [38, 41], "37e": [38, 41], "7802": [38, 41], "643": [38, 49], "139": [38, 39, 41], "692": [38, 39, 41], "3178": [38, 41], "02e": [38, 41, 42], "94e": [38, 41], "140": [38, 49], "651e": 38, "229": [38, 41], "8251": [38, 41], "4595": 38, "676": [38, 41], "1373": [38, 41], "8309": 38, "948": [38, 42], "143": [38, 49], "5545": 38, "8845": 38, "5862": 38, "5947": 38, "5771": 38, "921": [38, 41], "9556": 38, "0440": 38, "783": [38, 39], "146": [38, 41], "903e": 38, "1707": 38, "239": [38, 43], "012e": 38, "670": [38, 41], "097": [38, 42], "052e": 38, "157": [38, 39, 41], "867": 38, "249e": 38, "7332": 38, "914": 38, "1887": 38, "074e": 38, "174e": 38, "373": [38, 41], "379e": 38, "076": [38, 42], "2454": 38, "878": [38, 39, 41], "004": [38, 41, 42, 49], "3961": 38, "2072": [38, 41], "67e": [38, 41, 49], "160": [38, 42, 43], "811e": 38, "8356": 38, "451": 38, "1729": 38, "6091": 38, "1910": 38, "8574": 38, "477": [38, 41], "9087": 38, "0830": 38, "586": 38, "133e": 38, "054": [38, 41], "292": [38, 41], "283e": 38, "6795": 38, "165": [38, 39, 42, 49], "166": [38, 41], "551e": 38, "3002": 38, "990": [38, 39, 41], "947e": 38, "168": [38, 41, 49], "007e": 38, "7103": 38, "417": [38, 41], "156": [38, 41, 49], "3857": 38, "169": [38, 39], "9600": 38, "934": [38, 41], "170": [38, 41, 42, 43], "615e": 38, "7177": 38, "9576": 38, "172": 38, "626e": 38, "629e": 38, "314e": 38, "458": [38, 41, 42], "7983": 38, "176": 38, "8331": 38, "5679": 38, "7787": 38, "9729": 38, "6943": 38, "898": [38, 41], "5825": 38, "8426": 38, "9047": 38, "9527": 38, "061": [38, 49], "5061": 38, "4958": 38, "9108": 38, "578": [38, 41], "9156": 38, "0300": 38, "7253": 38, "999": [38, 41], "5065": 38, "6042": 38, "8525": 38, "602": 38, "8180": 38, "956": [38, 41], "1688": 38, "4654": 38, "4051": 38, "6253": 38, "894": 38, "9630": 38, "825": [38, 41], "184": [38, 39, 41], "566e": 38, "3739": 38, "789": [38, 41, 49], "8333": 38, "281": [38, 39], "158e": 38, "186": 38, "8134": [38, 41], "088": [38, 39], "187": [38, 39], "6652": 38, "8781": 38, "3548": [38, 41], "268": [38, 39, 41], "053": [38, 41, 42], "3878": [38, 41], "254": [38, 39, 41], "122e": 38, "5407": 38, "661e": 38, "803": [38, 41], "071": [38, 41], "2321": 38, "191": [38, 39, 41], "3036": 38, "3270": 38, "6940": 38, "9690": 38, "9281": 38, "647": 38, "455": [38, 43], "9444": 38, "8116": 38, "902": [38, 49], "2498": 38, "3312": [38, 41], "853": 38, "9e": [38, 41], "9595": 38, "1493": 38, "798": [38, 41], "2753": 38, "6229": 38, "6585": 38, "5757": 38, "7328": 38, "9890": 38, "7484": [38, 41], "328": [38, 41], "7343": 38, "801": [38, 41], "136e": 38, "8594": 38, "013": [38, 39, 41, 42, 49], "4510": 38, "7508": 38, "9207": 38, "3016": 38, "489": 38, "1594": 38, "737e": 38, "2251": 38, "1237": 38, "8204": 38, "6666": 38, "464e": 38, "9850": 38, "486": [38, 41], "4674": 38, "706e": 38, "6442": 38, "648": 38, "4427": 38, "733e": 38, "7640": 38, "6087": 38, "6907": [38, 41], "651": [38, 41, 49], "578e": 38, "973": [38, 41, 49], "1093": 38, "8388": 38, "7382": 38, "081": [38, 41], "351e": 38, "858": 38, "1299": 38, "215": [38, 41], "488": [38, 41], "5271": 38, "613": [38, 41], "8944": [38, 41], "5337": 38, "794": [38, 41], "090": [38, 41, 42], "1411": 38, "217": [38, 39], "9643": 38, "642e": 38, "987": [38, 41], "9429": 38, "8190": 38, "059": [38, 41, 42, 43], "220": [38, 43], "038e": 38, "7271": 38, "3058": 38, "526": [38, 39, 41], "622e": 38, "588": 38, "3804": 38, "878e": 38, "5199": 38, "8586": 38, "224": [38, 39, 41], "9192": 38, "1466": 38, "636": [38, 39, 42], "525": [38, 41], "2641": 38, "2389": 38, "226": [38, 41, 42], "8830": [38, 41], "2891": 38, "6639": 38, "4186": [38, 41], "397": [38, 41], "233e": [38, 41], "1135": [38, 41], "428e": 38, "454": [38, 41], "6264": 38, "041": [38, 39, 41, 42, 43], "517": [38, 39, 41], "8271": 38, "2748": 38, "7245": 38, "759": [38, 49], "5933": 38, "299e": 38, "8519": 38, "6289": 38, "960": [38, 39, 41], "233": [38, 41], "285e": 38, "8491": 38, "3796": 38, "735e": 38, "766e": 38, "217e": 38, "8349": 38, "4200": 38, "141e": [38, 41], "4505": [38, 41], "533": [38, 41], "2581": 38, "613e": 38, "258": [38, 39, 41, 43], "2127": 38, "712e": 38, "240": [38, 39, 49], "6376": 38, "9154": 38, "44e": [38, 41], "241": [38, 41], "3359": [38, 41], "7031": 38, "6189": 38, "543": [38, 41], "587": [38, 41], "8774": 38, "753": [38, 41, 49], "8724": 38, "775": [38, 39, 41], "788e": 38, "286": [38, 39], "9383": 38, "071e": 38, "2703": 38, "321": [38, 39, 41], "245": [38, 39, 41, 42], "8153": 38, "4624": 38, "5281": 38, "9970": 38, "4036": 38, "2630": 38, "9765": 38, "4698": 38, "5639": 38, "083": [38, 41], "1290": 38, "0204": 38, "8692": 38, "476e": 38, "616e": [38, 41], "4398": 38, "9569": 38, "726": [38, 41], "084": [38, 41, 49], "2247": 38, "701": [38, 47, 49], "253": [38, 39], "801e": 38, "5819": 38, "5313": 38, "470": [38, 41], "9352": 38, "6145": 38, "712": [38, 41], "64e": [38, 41], "474e": 38, "5626": 38, "575": [38, 41], "6945": 38, "662e": 38, "260": 38, "075e": 38, "7093": 38, "214e": 38, "9508": 38, "6499": 38, "474": [38, 41], "094e": 38, "872": 38, "2479": 38, "7477": 38, "6210": 38, "2882": 38, "138e": 38, "097e": 38, "8629": 38, "836e": 38, "7924": 38, "317": 38, "2824": 38, "361e": 38, "204e": 38, "485e": 38, "8880": 38, "672": [38, 41], "2561": 38, "149e": 38, "9891": 38, "3781": 38, "4694": 38, "687": [38, 39], "5173": 38, "7930": 38, "966e": 38, "402e": 38, "278": [38, 41], "2322": 38, "2439": 38, "903": [38, 41], "119e": 38, "345e": 38, "6306": 38, "1085": 38, "930": 38, "8403": 38, "4354": 38, "778": 38, "3681": 38, "0429": 38, "283": [38, 39], "5339": 38, "8789": 38, "9936": 38, "537": 38, "452e": 38, "8179": 38, "1519": [38, 41], "7684": 38, "0660": 38, "775e": 38, "5134": 38, "756": 38, "456": [38, 41], "7680": [38, 41], "2038": 38, "8041": 38, "0146": 38, "468": [38, 41], "581e": 38, "7099": 38, "1889": 38, "7663": 38, "1814": 38, "571": [38, 41], "567e": 38, "7319": 38, "1323": [38, 43], "652": [38, 39, 41, 42], "2555": 38, "433": [38, 39], "665": [38, 41, 42, 49], "9840": 38, "996": [38, 41, 42], "296": [38, 43], "1096": 38, "1249": 38, "4502": 38, "2586": 38, "298": [38, 41], "8732": 38, "4700": 38, "583": [38, 41, 43], "986e": [38, 41], "907": [38, 41], "329e": 38, "4581": 38, "0131": [38, 42], "7593": [38, 39], "484e": [38, 41], "7473": 38, "222e": 38, "8772": 38, "5022": 38, "2514": 38, "2672": 38, "7217": 38, "728": [38, 41], "336e": 38, "116e": [38, 41], "9796": 38, "8045": 38, "973e": 38, "179e": [38, 43], "5120": 38, "504": [38, 41], "310": [38, 39], "268e": 38, "624e": 38, "3524": 38, "9191": [38, 41], "1488": [38, 41], "9873": 38, "313": [38, 39, 41], "765e": 38, "412": 38, "680": 38, "793e": 38, "7170": 38, "3875": [38, 41], "4379": 38, "933": [38, 41], "5807": [38, 41], "9640": 38, "6758": 38, "859": [38, 41], "7442": 38, "794e": 38, "9335": 38, "924": [38, 39, 41], "318": [38, 42], "4980": 38, "2142": 38, "8582": 38, "1051": 38, "3237": 38, "320": [38, 39, 41], "431e": 38, "6968": 38, "416": [38, 39, 41], "3176": 38, "6523": 38, "846": [38, 41], "950": [38, 41], "853e": 38, "5726": 38, "625": 38, "2501": 38, "1256": 38, "7866": 38, "751": 38, "951e": [38, 41], "325e": 38, "742": 38, "8657": 38, "2875": 38, "9851": 38, "7463": 38, "2414": 38, "039": [38, 41, 42, 49], "981": [38, 41], "22245": 38, "534": 38, "model2": 38, "result2": 38, "matched_data2": 38, "19512": 38, "904e": 38, "386": [38, 41, 43], "128e": 38, "8904": 38, "805": [38, 41, 43], "855": [38, 39, 41, 42], "6653": 38, "matplotlib": 39, "pyplot": 39, "plt": 39, "unnam": 39, "spirit": 39, "unemp": 39, "emppop": 39, "baptist": 39, "mormon": 39, "nfatal2124": 39, "afat": 39, "pop1517": 39, "pop1820": 39, "pop2124": 39, "milestot": 39, "unempu": 39, "emppopu": 39, "gsp": 39, "10544": 39, "152344": 39, "692039": 39, "539379": 39, "355700": 39, "32829": 39, "437988": 39, "3942002": 39, "208999": 39, "593750": 39, "221553": 39, "43750": 39, "290000": 39, "06250": 39, "28516": 39, "799999": 39, "022125": 39, "10732": 39, "797852": 39, "147030": 39, "788991": 39, "333599": 39, "34341": 39, "834015": 39, "3960008": 39, "202000": 39, "078125": 39, "219125": 39, "46875": 39, "15625": 39, "31032": 39, "900002": 39, "046558": 39, "11108": 39, "791016": 39, "168087": 39, "714286": 39, "311501": 39, "35924": 39, "872009": 39, "3988991": 39, "196999": 39, "968750": 39, "216724": 39, "09375": 39, "288000": 39, "32961": 39, "500004": 39, "062798": 39, "11332": 39, "626953": 39, "271137": 39, "652542": 39, "289499": 39, "37579": 39, "742004": 39, "4021007": 39, "194999": 39, "734375": 39, "214349": 39, "03125": 39, "284000": 39, "31250": 39, "35091": 39, "100002": 39, "027490": 39, "11661": 39, "506836": 39, "514496": 39, "609907": 39, "267401": 39, "39311": 39, "716003": 39, "4049993": 39, "203999": 39, "890625": 39, "212000": 39, "00000": 39, "263000": 39, "28125": 39, "36259": 39, "700001": 39, "032143": 39, "nuniqu": 39, "10000": 39, "polynomi": 39, "polyfit": 39, "scatterplot": 39, "line2d": 39, "0x16c6a9b40": 39, "quad": 39, "091": 39, "334": [39, 41], "553": [39, 41], "8533": 39, "044": [39, 41], "768": [39, 41], "3646": 39, "865": 39, "487": [39, 41], "905": 39, "4776": 39, "6550": 39, "086": [39, 41], "az": 39, "5677": 39, "5095": 39, "4843": [39, 41], "919": [39, 41, 42], "ct": 39, "8623": 39, "de": 39, "3076": 39, "448": [39, 41, 42], "729": 39, "2681": 39, "5246": 39, "852": [39, 41], "5439": 39, "092": [39, 42], "id": [39, 47], "6690": 39, "il": 39, "9616": 39, "4615": 39, "k": 39, "2232": 39, "984": 39, "ky": 39, "la": [39, 55], "8471": 39, "490": [39, 41, 42], "641": [39, 41], "7064": 39, "1079": 39, "8972": 39, "419": [39, 41, 43], "2963": [39, 41], "821": [39, 41], "0291": 39, "mt": 39, "3604": 39, "365": [39, 41], "2905": 39, "nd": 39, "6234": 39, "396": [39, 41], "ne": 39, "5222": [39, 41], "nh": 39, "2545": 39, "667": [39, 41], "842": [39, 41], "1057": 39, "nm": 39, "4264": [39, 41], "927": [39, 41, 49], "nv": 39, "6008": 39, "1867": 39, "6744": 39, "175": [39, 41], "5451": 39, "1680": 39, "pa": 39, "ri": 39, "2651": 39, "sc": 39, "5572": 39, "sd": 39, "0037": 39, "788": [39, 41], "8757": 39, "ut": 39, "1640": 39, "2902": 39, "vt": 39, "9660": 39, "382": 39, "6595": 39, "854": [39, 41, 42], "wi": 39, "338": 39, "8968": 39, "wy": 39, "2285": 39, "466": 39, "6559": 39, "491": [39, 41], "863": [39, 41], "beertax_dm": 39, "fat_rate_dm": 39, "000196": 39, "168e": [39, 43], "0407": 39, "3805": 39, "3775": 39, "unadjust": 39, "0006": [39, 49], "1878": 39, "0256": 39, "2861": 39, "2939": 39, "8925": 39, "2563": 39, "8295": 39, "060": [39, 41], "0162": 39, "2245": 39, "4582": 39, "5398": 39, "3521": 39, "5330": 39, "1264": [39, 41], "2329": 39, "1533": 39, "0951": 39, "0160": 39, "9261": 39, "1267": 39, "0635": 39, "530": [39, 41], "covid": 40, "pandem": 40, "atc": 40, "q4": 40, "q5": 40, "nobodi": 40, "calori": 40, "intak": 40, "beverag": [40, 41], "hhid": [41, 43], "intmonth": [41, 43], "hurespli": [41, 43], "hrhtype": [41, 43], "minsamp": [41, 43], "hrlonglk": [41, 43], "hrsampl": [41, 43], "hrhhid2": [41, 43], "serial": [41, 43], "hhnum": [41, 43], "ym_file": 41, "ym": 41, "ch02": 41, "ch35": [41, 43], "ch613": [41, 43], "ch1417": [41, 43], "ch05": [41, 43], "ihigrdc": [41, 43], "docc00": [41, 43], "dind02": [41, 43], "000110339935453": [41, 43], "januari": [41, 43], "unmarri": [41, 43], "fam": [41, 43], "0701": [41, 43], "07011": [41, 43], "administr": [41, 43, 48], "000110359424339": [41, 43], "0711": [41, 43], "07111": [41, 43], "practition": [41, 43], "000110651278174": [41, 43], "0601": [41, 43], "06011": [41, 43], "wareh": [41, 43], "007680515071194": [41, 43], "0611": [41, 43], "06112": [41, 43], "implied_hourly_wag": 41, "mean_femal": 41, "mean_mal": 41, "1204": 41, "On": [41, 48, 50], "827": 41, "514": [28, 41, 42], "age_squar": 41, "2625": 41, "4441e": 41, "122603": [41, 43], "888e": 41, "122599": [41, 43], "833e": 41, "062e": 41, "6349": 41, "443": 41, "2740": 41, "18297": 41, "733": [41, 49], "27620": 41, "OF": 41, "THE": 41, "has_highschool": 41, "has_ba": 41, "m_educ": 41, "8573": 41, "4296e": 41, "859e": 41, "122597": 41, "579e": 41, "373e": 41, "756e": 41, "937": 41, "496": 41, "9737": 41, "3958": 41, "359": 41, "555": 41, "14257": 41, "849": 41, "20565": 41, "673": [41, 42], "m_industri": 41, "4253e": 41, "851e": 41, "122339": 41, "854e": 41, "1138": 41, "8766": 41, "1701": 41, "4193": 41, "2477": 41, "forestri": 41, "1131": 41, "1132": 41, "8666": 41, "3722": 41, "793": 41, "6372": 41, "8220": 41, "1133": 41, "5656": 41, "2100": 41, "2999": 41, "fish": 41, "hunt": 41, "trap": [28, 41], "3510": 41, "2980": 41, "4528": 41, "918": 41, "agricultur": 41, "6207": 41, "2024": 41, "796": 41, "750": [41, 42], "oil": 41, "392e": 41, "3072": 41, "coal": 41, "2121": 41, "528e": 41, "2550": 41, "metal": 41, "ore": 41, "2122": 41, "384e": 41, "4184": 41, "087": 41, "nonmetal": 41, "miner": 41, "quarri": 41, "2123": 41, "538e": 41, "2391": [41, 49], "495": 41, "431": 41, "773e": 41, "1513": 41, "electr": 41, "transmiss": 41, "pt": 41, "2211": 41, "162e": 41, "1437": 41, "s2212": 41, "588e": 41, "2480": 41, "2212": 41, "2685": 41, "steam": 41, "irrig": 41, "22131": 41, "22133": 41, "5944": 41, "3370": 41, "2059": 41, "9829": 41, "sewag": 41, "22132": 41, "8219": 41, "6616": 41, "3000": 41, "8332": 41, "056e": 41, "1126": 41, "8347": 41, "grain": [41, 42], "oilse": 41, "mill": 41, "3111": 41, "3112": 41, "2351": 41, "5394": 41, "confectioneri": 41, "3113": 41, "8933": 41, "3087": 41, "1430": 41, "fruit": 41, "veget": 41, "preserv": [41, 53], "specialti": 41, "3114": 41, "2238": 41, "7515": 41, "2458": 41, "725": 41, "911": 41, "2580": 41, "7057": 41, "dairi": 41, "6859": 41, "2203": 41, "2541": 41, "slaughter": 41, "3116": 41, "8284": 41, "2715": 41, "3343": 41, "311811": 41, "1774": 41, "8294": 41, "2453": 41, "6583": 41, "957": 41, "3034": 41, "3118": 41, "exc": 41, "4588": 41, "2350": 41, "395": [41, 42, 43], "2607": 41, "392": 41, "6608": 41, "seafood": 41, "miscellan": 41, "3117": 41, "3119": 41, "9160": 41, "3042": 41, "2207": 41, "4833": 41, "2859": 41, "0902": 41, "3663": 41, "4321": 41, "9058": 41, "1956": 41, "5223": 41, "tobacco": 41, "3122": 41, "7325": 41, "1456": 41, "fiber": 41, "yarn": 41, "thread": 41, "3131": 41, "5169": 41, "8050": 41, "8640": 41, "knit": 41, "3132": 41, "31324": 41, "3205": 41, "4471": 41, "3167": 41, "3003": 41, "9414": 41, "textil": 41, "coat": 41, "3133": 41, "8272": 41, "8244": [41, 43], "carpet": 41, "rug": 41, "31411": 41, "1664": 41, "3750": 41, "5371": 41, "9331": 41, "3206": 41, "0346": 41, "3374": 41, "3407": 41, "9819": 41, "3151": 41, "5465": 41, "8039": 41, "6312": 41, "866": [41, 42], "sew": 41, "apparel": 41, "3152": 41, "3536": 41, "0852": 41, "2485": 41, "422": 41, "8408": 41, "accessori": 41, "3159": 41, "1030": 41, "1607": 41, "9100": 41, "footwear": 41, "3162": 41, "478e": 41, "5328": 41, "4338": 41, "963": [41, 43], "leather": 41, "tan": 41, "3161": 41, "3169": 41, "2374": 41, "0553": 41, "6157": 41, "9694": 41, "pulp": 41, "paperboard": 41, "3221": 41, "264e": 41, "8510": 41, "32221": 41, "9113": 41, "32222": 41, "32223": 41, "32229": 41, "7723": 41, "9346": 41, "3395": 41, "1068": 41, "3648": 41, "2945": 41, "1712": 41, "7005": 41, "548": 41, "petroleum": 41, "32411": 41, "739e": 41, "2234": 41, "32419": 41, "8945": 41, "7389": 41, "5426": 41, "1689": 41, "resin": 41, "synthet": 41, "rubber": 41, "filament": 41, "7943": 41, "2560": 41, "2408": 41, "3222": 41, "chemic": 41, "3253": 41, "754e": 41, "8765": 41, "pharmaceut": [41, 50], "medicin": [41, 48, 50], "3254": 41, "377e": 41, "paint": 41, "adhes": 41, "b46": 41, "3255": 41, "586e": 41, "3584": 41, "8831": 41, "soap": 41, "compound": 41, "cosmet": 41, "3256": 41, "137e": 41, "2837": 41, "5809": 41, "3251": 41, "3259": 41, "plastic": 41, "3261": 41, "4034": 41, "0463": 41, "1804": 41, "7571": 41, "686": 41, "tire": 41, "32621": 41, "298e": 41, "3637": 41, "570": 41, "5854": 41, "32622": 41, "32629": 41, "3975": 41, "2268": 41, "3134": 41, "2168": 41, "potteri": 41, "ceram": 41, "32711": 41, "2984": 41, "4031": 41, "6013": 41, "8802": 41, "clai": 41, "32712": 41, "2829": 41, "502": 41, "8221": 41, "glass": 41, "3272": 41, "2191": 41, "7411": 41, "2614": 41, "2932": 41, "7315": 41, "cement": 41, "lime": 41, "gypsum": 41, "3273": 41, "3274": 41, "6765": 41, "1364": 41, "2187": 41, "080": [41, 42], "2478": 41, "3279": 41, "3390": 41, "5201": 41, "2515": 41, "9296": 41, "iron": 41, "steel": 41, "3311": 41, "098e": 41, "1963": 41, "7132": 41, "aluminum": 41, "3313": 41, "3558": 41, "601": 41, "674": 41, "6098": 41, "nonferr": 41, "3314": 41, "9701": 41, "8019": 41, "3984": 41, "1891": 41, "foundri": 41, "3315": 41, "5445": 41, "2055": 41, "3417": 41, "1252": 41, "forg": 41, "stamp": 41, "1715": 41, "0691": 41, "3721": 41, "645": 41, "9009": 41, "cutleri": 41, "8808": 41, "3925": 41, "4913": 41, "073": [41, 49], "3324": 41, "7136": 41, "9478": 41, "1831": 41, "896": 41, "3546": 41, "screw": 41, "nut": [41, 62], "bolt": 41, "3327": 41, "3821": 41, "1968": 41, "052": 41, "824": 41, "7679": 41, "engrav": 41, "heat": 41, "alli": 41, "3328": 41, "5874": 41, "3997": 41, "3353": 41, "752": 41, "623": 41, "ordnanc": 41, "332992": 41, "332995": 41, "139e": 41, "3325": 41, "3326": 41, "3329": 41, "332993": 41, "332994": 41, "5931": 41, "6094": 41, "1880": 41, "2244": 41, "9618": 41, "332": 41, "4324": 41, "5660": 41, "7461": [41, 49], "33311": 41, "009e": 41, "2528": 41, "5132": 41, "33312": 41, "33313": 41, "421e": 41, "2319": [41, 49], "9664": 41, "3095": 41, "3463": 41, "4982": 41, "metalwork": 41, "3335": 41, "4830": 41, "2648": 41, "600": 41, "1694": 41, "turbin": 41, "3336": 41, "8186": 41, "2951": 41, "3879": 41, "582": 41, "955": 41, "3332": 41, "3334": 41, "3339": 41, "9758": 41, "8177": 41, "6801": 41, "3365": [41, 49], "389e": 41, "2296": 41, "audio": 41, "3342": 41, "728e": 41, "2782": 41, "electromed": 41, "instrument": 41, "876e": 41, "2124": 41, "831": [41, 42], "electron": 41, "3344": 41, "3346": 41, "893e": 41, "1535": 41, "329": 41, "applianc": 41, "3352": 41, "9869": 41, "3063": 41, "3295": 41, "3351": 41, "092e": 41, "7284": 41, "vehicl": 41, "3361": 41, "3362": 41, "3363": 41, "8727": 41, "1319": 41, "6359": 41, "aircraft": 41, "336411": 41, "336413": 41, "999e": 41, "1454": 41, "743": 41, "aerospac": 41, "336414": 41, "336415": 41, "336419": 41, "932e": 41, "3846": 41, "railroad": 41, "stock": [41, 60], "102e": 41, "5066": 41, "1084": 41, "boat": 41, "3366": 41, "779": 41, "7078": 41, "3369": 41, "8638": 41, "4587": 41, "8438": 41, "9542": 41, "sawmil": 41, "wood": 41, "3211": 41, "4960": 41, "2021": [28, 41], "9599": 41, "veneer": 41, "plywood": 41, "5189": 41, "9036": 41, "4529": 41, "3687": 41, "prefabr": 41, "mobil": [41, 48], "321991": 41, "321992": 41, "5435": 41, "6788": 41, "4140": 41, "2680": 41, "3936": 41, "2236": 41, "7907": 41, "2156": 41, "9836": 41, "1298": 41, "5612": 41, "3391": 41, "411e": 41, "1565": 41, "amus": 41, "sport": 41, "33992": 41, "33993": 41, "2138": 41, "4970": 41, "2826": 41, "757": [41, 42, 49], "3401": 41, "7678": 41, "3399": 41, "4899": 41, "8913": 41, "1693": 41, "893": 41, "1580": 41, "1163": 41, "6909": 41, "merchant": 41, "wholesal": 41, "4231": 41, "8269": 41, "6204": 41, "2427": 41, "3511": 41, "furnitur": 41, "furnish": 41, "4232": 41, "7603": 41, "698": 41, "4895": 41, "lumber": 41, "4233": 41, "7793": 41, "7344": 41, "2415": 41, "3060": 41, "042": [41, 42, 49], "commerci": [41, 61], "4234": 41, "1896": 41, "519": [41, 49], "2031": 41, "4274": 41, "9198": 41, "823": 41, "7558": 41, "4195": 41, "076e": 41, "2213": 41, "6418": 41, "4265": 41, "7256": 41, "2269": 41, "4238": 41, "8008": 41, "recycl": 41, "42393": 41, "9854": 41, "3166": 41, "6248": 41, "6166": 41, "durabl": 41, "4239": 41, "9834": 41, "9299": 41, "3813": 41, "2360": 41, "4241": 41, "9631": 41, "5826": 41, "4061": 41, "1671": 41, "sundri": 41, "wholesalerss": 41, "4242": 41, "4246": 41, "663e": 41, "2150": 41, "731": 41, "notion": 41, "4243": 41, "8318": 41, "9813": 41, "1937": 41, "groceri": 41, "4244": 41, "5788": 41, "8515": 41, "1461": 41, "2923": 41, "8653": 41, "farm": 41, "4245": 41, "2950": [41, 42], "5444": 41, "3014": 41, "2957": 41, "8858": 41, "4247": 41, "555e": 41, "2434": 41, "368": 41, "4248": 41, "5327": 41, "6504": 41, "2713": 41, "42491": 41, "3704": 41, "nondur": 41, "4249": 41, "5692": 41, "3790": 41, "2543": 41, "agent": 41, "broker": 41, "4251": 41, "1193": 41, "5585": 41, "3485": 41, "8025": 41, "3868": 41, "4101": 41, "7998": 41, "8077": 41, "4411": 41, "8101": 41, "2705": 41, "1331": 41, "817": 41, "5490": 41, "4412": 41, "7648": 41, "0634": 41, "2446": 41, "850": [41, 43], "2852": 41, "4413": 41, "1683": [41, 43], "3592": 41, "3007": 41, "1227": 41, "7563": 41, "1726": 41, "498": 41, "4611": 41, "443111": 41, "5737": 41, "7708": 41, "6879": 41, "7784": 41, "0374": 41, "1730": 41, "4392": 41, "4441": 41, "44413": 41, "1277": 41, "4693": 41, "1425": 41, "4072": 41, "hardwar": 41, "2386": 41, "327": 41, "5456": 41, "3896": 41, "721": 41, "lawn": 41, "garden": 41, "4442": 41, "2166": 41, "2082": 41, "1915": 41, "4451": 41, "1616": 41, "4452": 41, "2629": 41, "8689": 41, "2684": 41, "980": 41, "7891": 41, "077": 41, "wine": 41, "liquor": 41, "4453": 41, "1459": 41, "4922": 41, "7841": 41, "689": 41, "pharmaci": 41, "44611": 41, "4348": 41, "1489": 41, "512": 41, "8283": 41, "4951": 41, "1951": 41, "4459": 41, "gasolin": 41, "station": 41, "1103": 41, "5162": 41, "1800": 41, "4633": 41, "2426": 41, "cloth": 41, "shoe": 41, "44821": 41, "4483": 41, "2557": 41, "3712": 41, "1658": 41, "1941": 41, "5779": 41, "3418": 41, "4758": 41, "8641": 41, "jewelri": 41, "luggag": 41, "2215": 41, "2885": 41, "6310": 41, "4999": 41, "5275": 41, "2639": 41, "7650": 41, "1114": 41, "6393": 41, "836": 41, "needlework": 41, "45113": 41, "668e": 41, "7083": 41, "2800": [41, 43], "5295": 41, "1476": 41, "5755": 41, "8103": 41, "45121": 41, "2523": 41, "8203": 41, "450": 41, "5079": 41, "s45211": [41, 43], "2063": 41, "7215": 41, "1315": [41, 42], "4641": 41, "merchandis": 41, "1100": 41, "1773": 41, "2375": 41, "518": 41, "4577": 41, "florist": 41, "4531": 41, "3223": 41, "4774": 41, "6135": 41, "stationeri": 41, "45321": 41, "5986": 41, "0326": 41, "2732": 41, "4533": 41, "8427": 41, "7584": 41, "2908": 41, "2727": 41, "gift": 41, "novelti": 41, "souvenir": 41, "45322": 41, "3744": 41, "5060": 41, "3044": 41, "4539": 41, "4518": 41, "5257": 41, "1904": 41, "454111": 41, "8738": 41, "4826": 41, "auction": 41, "454112": 41, "6663": 41, "3104": 41, "454113": 41, "5161": 41, "2790": 41, "vend": 41, "4542": 41, "7982": 41, "6144": 41, "1008": 41, "45431": 41, "2986": 41, "3140": 41, "545": 41, "8566": 41, "45439": 41, "4085": 41, "6284": 41, "3751": 41, "717": 41, "3267": 41, "4490": 41, "2046": 41, "3477": 41, "4544": 41, "475": [41, 57], "208e": 41, "1622": 41, "8898": 41, "rail": 41, "1954": 41, "483": 41, "727e": 41, "truck": [41, 43], "7630": 41, "0133": 41, "1291": 41, "5098": 41, "bu": 41, "urban": [41, 53], "transit": [41, 62], "4851": 41, "4852": 41, "4855": 41, "4859": 41, "9145": 41, "1719": 41, "5829": 41, "taxi": 41, "limousin": 41, "4853": 41, "8472": 41, "8000": 41, "673e": 41, "480": 41, "scenic": 41, "sightse": 41, "2172": 41, "4842": 41, "8118": 41, "incident": 41, "6852": 41, "5261": 41, "postal": 41, "6028": 41, "1553": 41, "9072": 41, "courier": 41, "messeng": 41, "7625": 41, "2671": 41, "4540": 41, "storag": 41, "2103": 41, "1611": 41, "554": [41, 49], "891": 41, "2205": 41, "newspap": 41, "51111": 41, "1191": 41, "2376": 41, "3465": 41, "5848": 41, "softwar": 41, "5111": 41, "2354": 41, "685": 41, "346": [41, 49], "5617": 41, "5112": 41, "952": 41, "motion": 41, "5121": 41, "2032": 41, "5122": 41, "6398": 41, "3136": 41, "radio": 41, "televis": 41, "broadcast": 41, "cabl": 41, "5151": 41, "5152": 41, "5175": 41, "1620": 41, "936": 41, "8595": 41, "portal": 41, "51913": 41, "618e": 41, "2601": 41, "wire": 41, "telecommun": 41, "carrier": 41, "5171": 41, "524e": 41, "515e": 41, "1711": 41, "host": [41, 47, 61], "5182": 41, "263e": 41, "3120": 41, "6518": 41, "archiv": 41, "51912": 41, "4295": 41, "1435": 41, "2627": 41, "9445": 41, "5191": 41, "52211": 41, "52219": 41, "436e": 41, "1242": 41, "561": 41, "52212": 41, "52213": 41, "5783": 41, "7155": 41, "2066": 41, "depositori": 41, "814e": 41, "1415": 41, "810": 41, "commod": 41, "fund": [41, 48], "invest": [41, 47, 54, 62], "1217": 41, "9944": 41, "estat": [41, 53], "2280": 41, "1274": 41, "6691": 41, "automot": 41, "leas": 41, "5321": 41, "2069": 41, "0345": 41, "2606": 41, "7178": 41, "tape": 41, "disk": 41, "53223": 41, "6701": 41, "784": 41, "53221": 41, "53222": 41, "53229": 41, "5323": 41, "1012": 41, "0036": 41, "3219": 41, "5298": 41, "intang": 41, "asset": [28, 41], "5324": 41, "446e": 41, "2815": 41, "8942": 41, "607": 41, "5411": 41, "004e": 41, "1333": 41, "029": [41, 49], "prepar": 41, "bookkeep": 41, "payrol": 41, "5412": 41, "386e": 41, "1453": 41, "architectur": 41, "5413": 41, "585e": 41, "5414": 41, "6793": 41, "0642": 41, "2382": 41, "5415": 41, "509e": 41, "1205": 41, "5416": 41, "045e": 41, "5417": 41, "997e": 41, "1515": 41, "5418": 41, "1681": 41, "veterinari": 41, "54194": 41, "5952": 41, "7522": 41, "1921": 41, "2186": 41, "809": 41, "9718": 41, "5419": 41, "226e": 41, "8199": 41, "enterpris": 41, "7092": 41, "9170": 41, "2364": 41, "5613": 41, "5211": 41, "0769": 41, "1475": 41, "2318": 41, "5614": [41, 43], "2759": 41, "4388": 41, "1603": 41, "633": 41, "5902": 41, "reserv": 41, "5615": 41, "8363": 41, "5104": 41, "2115": 41, "954": 41, "4217": 41, "5616": 41, "6381": 41, "1510": 41, "2835": 41, "3085": 41, "dwell": 41, "56173": 41, "2131": 41, "7452": 41, "1447": 41, "2910": 41, "1473": 41, "5799": 41, "5611": 41, "5619": 41, "3873": 41, "7759": 41, "wast": 41, "remedi": 41, "6367": 41, "7567": 41, "1659": 41, "724": 41, "9620": 41, "elementari": 41, "secondari": 41, "6111": 41, "0227": 41, "1556": 41, "2906": 41, "junior": 41, "6112": 41, "6113": 41, "6187": 41, "1199": 41, "6114": 41, "6115": 41, "8042": 41, "8479": 41, "6116": 41, "6117": 41, "7034": 41, "8796": 41, "1821": 41, "physician": 41, "6211": 41, "1328": 41, "8253": 41, "dentist": 41, "6212": 41, "7495": 41, "9965": 41, "4369": 41, "chiropractor": 41, "62131": 41, "2199": 41, "3757": 41, "3638": 41, "4932": 41, "optometrist": 41, "62132": 41, "3748": 41, "7228": 41, "2592": 41, "6213": 41, "4644": 41, "4511": 41, "2678": 41, "outpati": 41, "6214": 41, "6209": 41, "1278": 41, "3703": 41, "8715": 41, "6216": 41, "1292": 41, "3263": 41, "1418": 41, "1487": 41, "6215": 41, "6219": 41, "9278": 41, "7353": 41, "1326": 41, "6678": 41, "1143": 41, "8956": 41, "6231": 41, "1585": 41, "6208": 41, "1312": 41, "4158": 41, "residenti": [41, 43, 53], "6232": [41, 43], "6233": [41, 43], "6239": [41, 43], "0647": 41, "1470": 41, "2775": 41, "2989": 41, "6241": 41, "1112": 41, "7260": 41, "3728": 41, "1503": 41, "6242": 41, "3755": 41, "2794": 41, "1722": 41, "9234": 41, "vocat": 41, "rehabilit": 41, "6243": 41, "1419": 41, "3375": 41, "716": 41, "1185": 41, "child": [41, 45, 53], "6244": 41, "3177": 41, "7836": 41, "1434": 41, "5989": 41, "artist": 41, "art": [41, 60], "spectat": 41, "5905": 41, "7767": 41, "1907": 41, "2167": 41, "9644": 41, "museum": 41, "galleri": 41, "4334": 41, "4099": 41, "71395": 41, "8548": 41, "7123": 41, "5529": 41, "2288": 41, "gambl": 41, "713": 41, "8607": 41, "1324": 41, "4289": 41, "901": [41, 42], "accommod": [41, 47], "7211": 41, "2165": [41, 42], "2804": 41, "2442": 41, "506": 41, "park": 41, "camp": 41, "board": [41, 55], "7212": 41, "7213": 41, "6767": 41, "restaur": 41, "722": 41, "7224": 41, "3845": 41, "3157": 41, "1378": 41, "6277": 41, "0522": 41, "2704": 41, "977": 41, "8111": 41, "811192": 41, "4106": 41, "0577": 41, "1389": 41, "1383": 41, "6828": 41, "wash": 41, "3871": 41, "3298": 41, "2919": 41, "1851": 41, "8112": 41, "8832": 41, "3202": 41, "9258": 41, "8113": 41, "5347": 41, "4836": 41, "8114": 41, "81143": 41, "4203": 41, "0746": 41, "3150": 41, "barber": 41, "812111": 41, "9681": 41, "4421": 41, "9867": 41, "7467": 41, "beauti": 41, "salon": 41, "812112": 41, "1777": 41, "2348": 41, "nail": 41, "812113": 41, "81219": 41, "8031": 41, "2039": 41, "dryclean": 41, "laundri": 41, "8123": 41, "2282": 41, "7262": 41, "2206": 41, "938": 41, "2042": 41, "funer": 41, "cemeteri": 41, "crematori": 41, "8122": 41, "2626": 41, "3135": 41, "8771": 41, "3518": 41, "8129": 41, "2217": 41, "4546": 41, "4147": 41, "877": 41, "religi": 41, "8131": 41, "2863": 41, "7055": 41, "5780": 41, "civic": 41, "grantmak": 41, "8133": 41, "8513": 41, "9769": 41, "961": 41, "5644": 41, "81393": 41, "934e": 41, "polit": [41, 48, 49], "646e": 41, "2179": 41, "3089": 41, "5909": 41, "1865": [41, 49], "765": 41, "6746": 41, "execut": 41, "legisl": [41, 62], "92111": 41, "92112": 41, "92114": 41, "92115": 41, "6798": 41, "3444": 41, "1320": 41, "4210": 41, "9385": 41, "92113": 41, "795": [41, 42], "92119": 41, "9480": 41, "2676": 41, "justic": 41, "8924": 41, "4905": 41, "1197": 41, "6577": 41, "787": 41, "6796": 41, "9979": 41, "1359": 41, "708": 41, "4131": 41, "environment": [41, 55], "8922": 41, "1751": 41, "econom": [28, 41, 48, 50, 55, 60], "1490": 41, "affair": 41, "755e": 41, "1346": 41, "991e": 41, "762": 41, "815": 41, "3093": 41, "2084": 41, "14403": 41, "21479": 41, "smallest": [41, 54], "eigenvalu": 41, "singular": 41, "fell": 41, "12995": 41, "10860": 41, "2134": 41, "read_dta": 42, "sm": 42, "pvalu": [30, 42], "ttest_ind": 42, "03": 42, "statisticallli": 42, "spuriou": 42, "quick": [28, 42, 48, 49, 60], "dirti": 42, "1744": 42, "1760": 42, "4917640058792273": 42, "darn": 42, "isol": [42, 50, 54], "ness": 42, "t2": 42, "p2": 42, "callback": 42, "Thats": 42, "1f": 42, "3f": 42, "4870": 42, "1128": 42, "4868": 42, "1141": 42, "0965": 42, "0320": 42, "2969": 42, "18927": 42, "heteroscedast": [42, 49], "02": [42, 49], "4861": 42, "0821": 42, "0017": 42, "953e": 42, "0025": 42, "0047": 42, "0316": 42, "0032": 42, "0186": 42, "0112": 42, "18631": 42, "non_college_grad": 42, "1116": 42, "4863": 42, "1161": 42, "0759": 42, "0090": 42, "0281": 42, "0123": 42, "0188": 42, "0110": 42, "18624": 42, "noncolleg": 42, "salt": 42, "underpow": 42, "reject": [28, 42, 51], "null": [28, 42], "college_grad": 42, "00198": 42, "3499": 42, "0736": 42, "0286": 42, "0019": 42, "0108": 42, "0197": 42, "14157": 42, "90e": 42, "1366": 42, "1361": 42, "0408": 42, "0062": 42, "592": 42, "0460": 42, "4394": 42, "1122": 42, "4860": 42, "1187": 42, "0807": 42, "0021": 42, "971": 42, "0001": 42, "0026": 42, "947": 42, "0048": 42, "0287": 42, "0038": 42, "18630": 42, "insignific": 42, "fifteen": 42, "percent": [42, 49], "caveat": 42, "acknowledg": 42, "workplac": [42, 43], "unrel": 42, "broader": 42, "perhap": [28, 43, 50, 53], "intra": 43, "regard": 43, "has_college_educ": 43, "8393": 43, "4365e": 43, "873e": 43, "639e": 43, "7144": 43, "054e": 43, "18030": 43, "27634": 43, "semi": 43, "colon": 43, "suppress": 43, "voil\u00e0": 43, "plm": 43, "whatev": [43, 60, 62], "typeerror": 43, "str": 43, "fe_group": 43, "group_cod": 43, "int16": 43, "semicolon": 43, "nce8": 43, "opt": 43, "1871": 43, "valuewarn": 43, "rank": [28, 43], "bottom": [28, 43, 51], "ordinari": 43, "dimens": 43, "catch": [43, 62], "multiindex": 43, "hate": 43, "multi": [43, 54], "43344": 43, "19200": 43, "60000": 43, "32640": 43, "38400": 43, "cps_w_multiindex": 43, "1183": 43, "missingvaluewarn": 43, "__init__": 43, "exog": 43, "check_rank": 43, "1414": 43, "6716": 43, "122341": 43, "747": 43, "593e": 43, "886e": 43, "065e": 43, "9511": 43, "665e": 43, "405e": 43, "924e": 43, "365p": 43, "entityid": 43, "0x2f61875e0": 43, "kid": [44, 45, 48], "perspect": [44, 47], "stabl": [44, 50], "uh": 44, "local": [28, 44, 55], "sensit": [44, 52, 61], "ton": [44, 55], "nyc": 44, "demean": 44, "y_i": 44, "hurrican": 44, "dip": 44, "disrupt": 44, "stick": 44, "algebra": 44, "slope": [45, 49], "shock": 45, "chief": 45, "uncorrel": 45, "leverag": 45, "htose": 45, "higherarch": 45, "hierarhc": 45, "hausman": 45, "accompanyingli": 45, "frequent": 45, "wavi": 45, "confess": 45, "diagon": 45, "smarter": 45, "princip": 45, "execis": 45, "mix": 45, "intens": 46, "wrestl": [46, 59, 60, 61], "aris": 46, "unavoid": 46, "exposit": 46, "comment": 46, "facet": 46, "embrac": [28, 46], "urg": 46, "emblemat": 46, "intellectu": 46, "arbitrag": 46, "port": 46, "insight": [46, 57], "richli": 46, "unfamiliar": 46, "orient": 46, "infuriatingli": 46, "salient": 46, "consolid": 46, "transcrib": 46, "concis": 46, "depriv": 46, "connect": [46, 57], "bedrock": 46, "lectur": [46, 52], "prolong": 46, "modern": 46, "flit": 46, "cultiv": 46, "hello": 47, "promis": [47, 49, 54], "throw": [47, 51], "intellig": [47, 55], "ticket": 47, "pilot": [28, 47, 53], "reap": 47, "sizeabl": 47, "burgeon": 47, "pitfal": 47, "endeavor": [47, 60], "curricula": 47, "hardest": 47, "deliveri": 47, "unifi": 47, "complement": 47, "depth": [47, 52], "introduct": 47, "heavili": 47, "interdisciplinari": 47, "grasp": 47, "inferenti": 47, "concurr": 47, "comfort": [28, 47], "facilit": 47, "collabor": [47, 61], "bilingu": 47, "git": 47, "pickup": 47, "workshop": 47, "preliminari": [47, 62], "schedul": [47, 53], "constrast": 48, "stream": 48, "subscript": [48, 55], "signup": 48, "hing": 48, "email": 48, "music": 48, "unverifi": 48, "banal": 48, "textbook": 48, "sutva": [48, 54], "generaliz": 48, "elit": 48, "univer": 48, "emori": 48, "vanderbilt": 48, "unc": 48, "whole": [48, 50, 52, 55, 62], "luxuri": 48, "detroit": 48, "wealth": [28, 48], "cultur": 48, "brand": 48, "imag": [48, 55], "unrepres": 48, "unsurprisingli": 48, "dose": 48, "asian": 48, "island": 48, "sclerosi": 48, "mutat": 48, "descend": 48, "longest": 48, "psychologi": 48, "volunt": 48, "western": 48, "democrat": [48, 49, 55, 60], "facial": 48, "recognit": 48, "overlook": 48, "declar": 48, "billboard": 48, "infinit": [11, 48], "perman": 48, "tv": 48, "000th": 48, "famou": 48, "india": [28, 48], "clinic": [48, 50], "absente": 48, "ngo": [28, 48], "mit": 48, "economist": 48, "clock": 48, "punish": 48, "attend": [28, 48], "doubl": 48, "exempt": 48, "equilibrium": 48, "angu": 48, "deaton": 48, "nanci": 48, "cartwright": 48, "rct": 48, "monitor": 48, "environ": [48, 55], "rand": 48, "particip": 48, "catastroph": 48, "coverag": 48, "oregon": 48, "ohp": 48, "medicaid": 48, "lotteri": 48, "elig": [28, 48, 61], "winner": 48, "uninsur": 48, "expans": 48, "tension": 48, "interfac": 48, "dog": 48, "bark": 48, "hurri": 48, "gloss": 49, "deserv": 49, "registr": 49, "republican": 49, "tenur": 49, "faculti": 49, "turnout": 49, "north": [49, 60], "carolina": [49, 60], "dichotom": 49, "problemat": [49, 50, 54], "css_tutori": 49, "exercise_data": 49, "voter_turnout": 49, "unaffili": 49, "7416": 49, "07": 49, "5768": 49, "9919": 49, "154e": 49, "9917": 49, "156e": 49, "0075": 49, "4426": 49, "2128": 49, "5748": 49, "150e": 49, "151e": 49, "7754": 49, "0562": 49, "2361": 49, "343": 49, "5735": 49, "148e": 49, "9916": 49, "7988": 49, "0797": 49, "0606": 49, "2341": 49, "dem": 49, "linearhypothesi": 49, "t_test": 49, "contrastresult": 49, "c0": 49, "0191": 49, "voila": 49, "7383": 49, "436": 49, "pro": [49, 52], "alon": 49, "age_x_femal": 49, "917": 49, "5764": 49, "9915": 49, "157e": 49, "6961": 49, "0933": 49, "0009": 49, "0015": 49, "1883": 49, "2388": 49, "340": 49, "declin": 49, "plu": 49, "simplest": 49, "beta_2": [49, 52], "beta_3": [49, 52], "voted_": 49, "linearli": 49, "old_x_femal": 49, "5717": 49, "144e": 49, "147e": 49, "6761": 49, "1015": 49, "0138": 49, "0111": 49, "1820": 49, "peak": 49, "ggplot": 49, "ae": 49, "plotninewarn": 49, "implementedfor": 49, "9223363270683622540": 49, "lingo": 50, "homogen": 50, "heterogen": 50, "fda": 50, "approv": 50, "chronic": [50, 60], "hail": 50, "changer": 50, "undergo": 50, "minu": 50, "crippl": 50, "disabl": 50, "bucket": 50, "proportion": 50, "latino": 50, "indirect": 50, "spillov": 50, "vaccin": 50, "immun": 50, "sick": 50, "indirectli": 50, "embodi": 50, "newsfe": 50, "apart": 50, "youtub": [51, 52], "nitti": 51, "gritti": 51, "winnow": 51, "proce": [51, 62], "dissimilar": 51, "unpair": 51, "thrown": 51, "root": 51, "euclidean": 51, "strength": 51, "weak": 51, "strike": 51, "strict": 51, "maximium": 51, "gari": [51, 52], "minunt": 51, "ho": 52, "imai": 52, "stuart": 52, "streamlin": 52, "concepu": 52, "framwork": 52, "insert": 52, "tremend": 52, "02315": 52, "extrapol": [52, 60], "funni": 52, "imbalanc": 52, "blue": 52, "badli": 52, "uncomfort": 52, "drew": 52, "parabola": 52, "invert": 52, "theoret": [52, 62], "nervou": 52, "grei": 52, "determinist": 52, "situt": 52, "con": 52, "fashion": 52, "impov": 52, "charl": 53, "ketter": 53, "1920": 53, "1947": 53, "invok": 53, "kingdom": 53, "fundat": 53, "ancient": 53, "ruin": [53, 60], "poster": 53, "pile": 53, "archeologist": 53, "bone": 53, "toothbrush": 53, "detect": [28, 53], "archeolog": 53, "ground": 53, "penetr": 53, "radar": 53, "scan": [53, 60], "foundat": [53, 60], "spot": 53, "glanc": 53, "feasabl": 53, "excav": 53, "importantli": 53, "sorri": 53, "topologi": 53, "hydrologi": 53, "soil": 53, "geographi": [53, 60], "emmin": 53, "doabl": 53, "africa": 53, "infant": 53, "mortal": 53, "migrant": 53, "literatur": [28, 54], "warrant": 54, "unexpect": 54, "srm": 54, "optimist": 54, "hmmm": 54, "rock": 54, "solid": [54, 57, 62], "benjamini": 54, "hochberg": 54, "westfal": 54, "bonferroni": 54, "conserv": 54, "unrealist": 54, "exhaust": [55, 61], "ethnic": 55, "tabular": 55, "shapefil": 55, "shorter": 55, "nhgi": 55, "stabil": 55, "wharton": 55, "membership": 55, "\u00e1": 55, "cart": 55, "microsoft": 55, "planetari": 55, "wildlif": 55, "aw": 55, "registri": 55, "genom": 55, "nih": 55, "telescop": 55, "mortgag": 55, "overst": 55, "freeli": 55, "conflict": 55, "terror": 55, "pollut": 55, "flood": 55, "freedom": 55, "airbnb": [55, 61], "traffic": 55, "underpin": [55, 62], "geospati": 55, "ihgi": 55, "internation": 55, "privaci": [11, 55], "withheld": 55, "raster": 55, "nasa": 55, "elev": 55, "plant": 55, "grow": 55, "wavelength": 55, "crop": 55, "imageri": 55, "infrastructur": 55, "factori": 55, "offici": [55, 60], "obscen": 55, "noaa": 55, "govt": 55, "awesom": 55, "skeptic": [57, 62], "mindset": 57, "untest": 57, "8am": 57, "morn": 57, "annoi": 57, "rubric": 57, "sparingli": 57, "exception": 57, "425": 57, "demonstr": [28, 57, 62], "meati": 57, "paragraph": [57, 62], "succinct": 57, "superfici": 57, "skate": 57, "surfac": 57, "dedic": 57, "prompt": [57, 60], "stuck": 59, "illus": 59, "debug": 59, "unse": 59, "prece": 59, "fragment": 60, "disciplin": 60, "myopic": 60, "strive": 60, "merit": 60, "inquiri": 60, "unseen": 60, "unobserv": 60, "desir": 60, "opioid": 60, "overdos": 60, "pain": 60, "enact": 60, "ger": 60, "aspect": [60, 62], "phenomena": 60, "centuri": 60, "appreci": 60, "arros": 60, "carmen": 60, "reinhart": 60, "kenneth": 60, "rogoff": 60, "eight": 60, "folli": 60, "hundr": 60, "sixti": 60, "pundit": 60, "regularli": 60, "decri": 60, "crise": [60, 61], "feroc": 60, "wipe": 60, "death": 60, "hiv": 60, "gai": 60, "dy": 60, "cigarett": 60, "lung": 60, "climat": 60, "quot": 60, "temperatur": 60, "earth": 60, "unfair": 60, "noodl": 60, "casual": 60, "fancier": 60, "necessit": 60, "novel": 60, "shed": 60, "latent": 60, "rare": 60, "vera": 60, "rubin": 60, "rotat": 60, "star": 60, "galaxi": 60, "ken": [60, 62], "crisi": 60, "truth": 60, "demand": 60, "judgement": 60, "curv": 60, "physic": 60, "particl": 60, "she": 60, "cold": 60, "minimum": [28, 60], "iowa": 60, "presidenti": 60, "causus": 60, "politician": 60, "presid": 60, "awfulli": 60, "occasion": 60, "stumbl": 60, "reiter": 60, "circumstanti": 60, "nutshel": [60, 62], "father": 60, "causat": 60, "simplif": 60, "wealthi": 60, "caucus": 60, "trendi": 60, "mammogram": 60, "radiologist": 60, "tumor": 60, "tomorrow": 60, "mri": 60, "sml": 60, "garbag": 60, "analogu": 60, "ridicul": 60, "benign": 60, "meaningless": 60, "contextu": 60, "februari": 61, "13th": 61, "12pm": 61, "reinvent": 61, "wheel": 61, "revis": 61, "pedestrian": 61, "freewai": 61, "bureaucrat": 61, "unclear": 61, "fraud": 61, "deter": 61, "entitl": 61, "recipi": 61, "largest": 61, "shoot": 61, "mental": 61, "societ": 61, "tenni": 61, "hinder": 61, "tournament": 61, "phone": 61, "churn": 61, "yourselv": 61, "curat": 61, "translat": 62, "guidelin": [28, 62], "policymak": 62, "slowli": 62, "lai": 62, "graf": 62, "journalist": 62, "wall": 62, "farther": 62, "broadli": 62, "conclud": 62, "brief": 62, "drown": 62, "bog": 62, "linchpin": 62, "hook": 62, "caught": 62, "player": 62, "laid": 62, "fellow": 62, "cross": 62, "dot": 62, "masteri": 62, "deligi": 62, "erron": 62, "homework": 62, "afford": 62, "honest": 62, "humil": 62, "sought": 62, "appendic": 62, "bore": 62, "opinion": 62, "exercise_abtest": 11, "ex4_avg_oec": 11, "ex5_avg_guardrail": 11, "ex7_ttest_pvalu": 11, "ex9_ttest_pvalue_click": 11, "ex11_guard_": 11, "ex11_guard_pvalu": 11, "ex11_oec_": 11, "ex11_oec_pvalu": 11, "ex14_se_treat": 11, "assist": 11, "finit": 11, "government": 28, "bandhan": 28, "west": 28, "bengal": 28, "cash": 28, "transfer": 28, "virtuou": 28, "forti": 28, "poorest": 28, "ultra": 28, "tup": 28, "brac": 28, "bangladesh": 28, "multifacet": 28, "unlock": 28, "abhijit": 28, "banerje": 28, "esther": 28, "duflo": 28, "garima": 28, "sharma": 28, "uncondit": 28, "stipend": 28, "rupe": 28, "pariti": 28, "ppp": 28, "capita": 28, "menu": 28, "cow": 28, "goat": 28, "convers": 28, "bandham": 28, "set_opt": [], "mode": [], "copy_on_writ": [], "hh": [], "cash_transf": [], "tup_cash_transf": [], "pc_exp_month_bl": 28, "pc_exp_month_el1": [], "pc_exp_month_el2": 28, "pc_exp_month_el3": [], "152084": [], "816093": [], "286613": [], "728490": [], "388021": [], "816334": [], "899952": [], "277290": [], "698770": [], "222298": [], "547070": [], "601420": [], "839483": [], "802390": [], "763836": [], "879470": [], "511020": [], "258686": [], "585903": [], "133247": [], "930300": [], "072914": [], "625847": [], "612293": [], "582490": [], "765170": [], "787525": [], "300316": [], "075207": [], "112366": [], "774239": [], "511593": [], "183617": [], "493570": [], "494990": [], "819830": [], "082634": [], "052082": [], "_bl": 28, "expend_avg": [], "expend_std": [], "ttestindpow": 28, "instanti": 28, "my_pow": 28, "sqrt": 28, "plot_pow": 28, "solve_pow": 28, "effect_s": 28, "unrest": 28, "nobs1": 28, "treat_ob": [], "0f": [], "obscur": 28, "thirty_percent_lift": [], "nthat": [], "modifi": 28, "bump": [], "min_effect": [], "usd": [], "pct": [], "scenario": 28, "arang": 28, "nob": [], "ate_2": [], "ate_2_in_std": [], "num_treat": [], "num_control": [], "final_pow": [], "recalcul": [], "10_000": [], "ate": [], "hh_treated_sampl": [], "hh_control_sampl": [], "ttest_result": [], "ballpark": [], "lean": [], "hamlet": 28, "quintil": 28, "seven": 28, "ii": 28, "iii": 28, "landhold": 28, "acr": 28, "iv": 28, "ownership": 28, "vi": 28, "vii": 28, "beg": 28, "intent": 28, "uptak": 28, "simul": 28, "bootstrap": 28}, "objects": {}, "objtypes": {}, "objnames": {}, "titleterms": {"backward": [0, 2], "design": [0, 2, 23, 62], "overview": [0, 2], "1": [0, 2, 7, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 24, 25, 28, 29, 30, 31, 32, 34, 35, 37, 38, 39, 41, 42, 54, 60], "defin": [0, 2, 54], "your": [0, 1, 2, 7, 11, 16, 22, 31, 33, 38, 51, 53, 54, 61, 62], "problem": [0, 2, 47, 53, 61], "2": [0, 1, 2, 7, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 23, 24, 25, 27, 28, 29, 30, 31, 32, 34, 35, 37, 38, 39, 41, 42, 49, 54, 61], "question": [0, 2, 15, 17, 18, 19, 23, 32, 53, 60, 61], "you": [0, 2, 3, 7, 15, 34, 49, 54, 59], "wish": [0, 2], "answer": [0, 2, 15, 34, 61], "A": [0, 2, 3, 11, 31, 51, 54], "digress": [0, 2], "supervis": [0, 2], "machin": [0, 2, 7], "learn": [0, 2, 7, 33], "ud": 1, "midterm": 1, "2022": 1, "part": [1, 61], "estim": [1, 11, 29, 30, 41, 42], "The": [1, 3, 8, 11, 16, 23, 25, 26, 27, 28, 40, 49, 50, 52, 53], "effect": [1, 8, 10, 11, 20, 24, 30, 37, 39, 42, 43, 44, 45, 49, 50], "educ": 1, "employ": 1, "get": [1, 2, 22, 38], "know": [1, 22, 38], "data": [1, 2, 7, 11, 21, 22, 25, 31, 33, 35, 38, 47, 51, 55, 61, 62], "subset": 1, "peopl": 1, "over": 1, "25": 1, "labor": 1, "forc": 1, "measur": [1, 11, 30, 42, 51], "good": [2, 55], "why": [2, 8, 60], "i": [2, 7, 8, 12, 51, 53, 60], "have": [2, 7, 15, 34], "import": [2, 11], "3": [2, 7, 11, 12, 13, 14, 15, 18, 19, 20, 21, 22, 23, 24, 25, 28, 29, 30, 31, 34, 35, 37, 38, 39, 41, 42, 54, 60, 61], "write": [2, 62], "down": 2, "what": [2, 5, 8, 12, 30, 42, 49, 53], "an": [2, 7], "would": 2, "look": [2, 59], "like": 2, "falsifi": 2, "4": [2, 7, 11, 13, 15, 18, 19, 20, 21, 22, 24, 25, 28, 29, 30, 34, 35, 37, 38, 39, 41, 42, 54, 60], "do": [2, 3, 22, 38, 46, 53, 54], "need": [2, 8, 9, 20, 22, 24, 38, 53], "5": [2, 7, 11, 13, 15, 18, 19, 20, 21, 22, 24, 25, 28, 29, 30, 34, 35, 37, 38, 39, 41, 42, 54], "where": [2, 55], "can": [2, 51], "That": [2, 55], "wrap": 2, "up": 2, "beyond": 3, "experi": [3, 11, 12, 23, 28, 30, 42, 50], "b": [3, 11, 54], "test": [3, 11, 23, 33, 54], "isn": [3, 60], "t": [3, 60], "alwai": [3, 7], "feasibl": 3, "legal": [3, 13], "ethic": 3, "go": 3, "futur": [3, 25], "don": 3, "want": 3, "limit": [3, 11, 25, 30, 50], "Not": 3, "enough": 3, "abl": 3, "must": [3, 49], "explain": 3, "class": 4, "schedul": 4, "research": 5, "discret": 5, "descript": [5, 9, 34, 60], "analysi": [5, 24, 30, 31, 34, 39, 42], "faith": 5, "represent": 5, "choos": [5, 7, 61], "To": [5, 7, 22, 25, 38, 45, 53, 55], "ask": [5, 53], "present": 5, "context": [5, 28, 31], "valu": 5, "peek": 6, "endogen": 6, "stop": 6, "recommend": 7, "respons": 7, "train": 7, "carefulli": 7, "onli": 7, "us": [7, 49, 51, 61], "opaqu": 7, "model": [7, 21, 25, 45, 51, 52], "when": [7, 49, 51], "function": 7, "form": [7, 22, 38], "constraint": 7, "compar": 7, "predict": [7, 25, 32, 60], "sub": 7, "popul": 7, "No": 7, "option": 7, "tool": 7, "interpret": [7, 10, 20, 21, 22, 37, 38, 49, 54], "constrain": 7, "neural": 7, "network": 7, "introspect": 7, "evalu": [8, 14], "real": [8, 53], "studi": [8, 14, 26, 27, 40], "sun": 8, "bright": 8, "dog": 8, "too": 8, "baselin": 8, "differ": [8, 46, 60], "we": [8, 30, 42], "all": 8, "three": 8, "evaul": 8, "differenti": 8, "treatment": [8, 50], "come": 8, "next": [8, 53], "well": 8, "thi": [8, 21, 54, 60], "ha": 8, "been": 8, "unsatisfi": 8, "discuss": 9, "exercis": [9, 11, 12, 13, 15, 17, 20, 21, 22, 24, 25, 28, 29, 30, 31, 32, 34, 35, 37, 38, 39, 41, 42], "8": [9, 11, 15, 18, 19, 20, 21, 22, 24, 25, 28, 29, 30, 37, 38, 39, 41, 42], "takeawai": 9, "absolut": [9, 20, 22, 24, 38], "posit": [9, 20, 22, 24, 38, 60], "solut": [9, 20, 22, 24, 36, 38, 58, 59], "causal": [10, 18, 19, 26, 27, 32, 40, 44, 52, 60], "gender": [10, 29, 41], "udac": 11, "websit": 11, "": [11, 21, 22, 38, 50, 53, 54], "pick": 11, "valid": [11, 48, 54], "6": [11, 13, 15, 18, 19, 20, 21, 22, 24, 25, 28, 29, 30, 34, 35, 37, 38, 39, 41, 42, 54], "7": [11, 13, 15, 18, 19, 20, 21, 22, 24, 25, 28, 29, 30, 34, 35, 37, 38, 39, 41, 42], "9": [11, 18, 19, 21, 22, 25, 28, 29, 30, 38, 39, 41, 42], "10": [11, 21, 22, 25, 28, 29, 30, 38, 41, 42], "11": [11, 21, 22, 25, 28, 29, 30, 38, 41, 42], "12": [11, 21, 22, 25, 29, 30, 38, 41, 42], "13": [11, 21, 22, 25, 28, 29, 30, 38, 41], "14": [11, 21, 22, 25, 28, 29, 38, 41], "counter": 12, "factual": 12, "experiment": 12, "ideal": 12, "counterfactu": 12, "random": 12, "identifi": [12, 62], "marijuana": 13, "violent": 13, "crime": [13, 15, 34], "gym": 14, "membership": 14, "car": 14, "insur": 14, "billboard": 14, "polic": 15, "expenditur": 15, "exploratori": [15, 32, 61], "after": [15, 34], "first": [16, 61], "stakehold": [16, 17, 31, 53], "propos": [16, 31], "task": [16, 31], "convert": [17, 53], "prompt": [17, 53], "action": [17, 53], "time": [18, 19, 60], "cholera": [18, 19], "indic": [20, 37, 43, 49], "variabl": [20, 37, 43, 49], "omit": [20, 37], "bia": [20, 37], "interact": [20, 37, 49], "credit": [21, 28], "risk": [21, 25], "prep": 21, "fit": 21, "let": [21, 22, 38], "see": 21, "15": [11, 21, 22, 25, 28, 38], "16": [21, 22, 25, 38], "17": [21, 25], "match": [22, 38, 51, 52], "packag": [22, 38], "python": [22, 38], "v": [22, 38], "r": [22, 38], "instal": [22, 38], "dame": [22, 38], "flame": [22, 38], "setup": [22, 38], "output": [22, 38], "back": [22, 38], "dataset": [22, 38], "check": [22, 30, 38, 42, 51, 54], "analyz": [22, 38, 51], "other": [22, 38, 55], "optim": 23, "ab": 23, "traffic": [24, 39], "death": [24, 39], "fix": [24, 39, 43, 44, 45, 49], "demean": [24, 39], "mortgag": 25, "delinqu": 25, "gradescop": [11, 25, 30], "autograd": [11, 25, 30], "submiss": [11, 25, 30], "clean": 25, "organ": 25, "now": 25, "18": 25, "make": [26, 27, 40, 53], "potenti": [26, 27, 40], "outcom": [26, 27, 40], "concret": [26, 27, 40], "map": [26, 27, 40], "framework": [26, 27, 40], "observ": [26, 27, 40], "infer": [26, 27, 40, 44, 52, 60], "discrimin": [29, 41], "workplac": [29, 41], "resum": [30, 42], "balanc": [30, 42, 51], "race": [30, 42], "heterogen": [30, 42], "did": [30, 42], "just": [30, 42], "solicit": 31, "inform": 31, "from": [31, 46], "project": 31, "vote": 31, "access": 31, "colleg": 31, "campus": 31, "background": [31, 62], "object": 31, "our": 31, "approach": 31, "gener": 32, "classifi": 32, "passiv": 32, "groupbi": [33, 35], "arrest": [33, 35], "group": [33, 43], "structur": 33, "assumpt": [33, 52], "poverti": 34, "acm": 36, "corport": 36, "implement": [43, 54], "cluster": [43, 45], "computation": 43, "effici": 43, "hierarch": 45, "standard": 45, "error": 45, "thing": [45, 49, 54], "rememb": 45, "how": [46, 51], "read": [46, 53, 57], "academ": 46, "edit": 46, "activ": 46, "Be": 46, "patient": 46, "exampl": [46, 60], "domain": 46, "NOT": 46, "summar": 46, "llm": 46, "welcom": 47, "solv": 47, "pre": 47, "requisit": 47, "non": [47, 62], "mid": 47, "student": 47, "syllabu": 47, "intern": [48, 54], "extern": [48, 54], "consider": 48, "trade": 48, "off": 48, "between": [48, 49], "conclus": [48, 60, 62], "dummi": 49, "ar": [49, 54, 55, 59], "ONE": 49, "understand": [49, 61], "two": 49, "categori": 49, "more": 49, "than": 49, "constant": 49, "keep": 49, "straight": 49, "oper": 49, "multipl": 49, "averag": 50, "who": 50, "fine": 50, "print": 50, "ATE": 50, "sutva": 50, "done": 51, "summari": [51, 52, 62], "prune": [51, 52], "similar": 51, "should": [51, 54], "specif": [51, 53], "depend": 52, "imbal": 52, "better": [52, 61], "thought": [52, 60], "cost": 52, "In": 52, "mani": 53, "nick": 54, "think": 54, "about": 54, "befor": 54, "step": 54, "0": 54, "sure": 54, "goal": 54, "plan": [28, 54], "run": 54, "result": [54, 62], "consid": 54, "find": 55, "public": 55, "start": 55, "point": 55, "interest": [55, 59], "sourc": 55, "spatial": 55, "gi": 55, "compon": 55, "satellit": 55, "list": 55, "great": 55, "reflect": 57, "ok": 59, "still": 59, "taxonomi": 60, "versu": 60, "norm": 60, "nope": 60, "diseas": 60, "surveil": 60, "global": 60, "warm": 60, "eda": 60, "last": 60, "manipul": 60, "hard": 60, "classif": 60, "without": 60, "assign": 61, "specifi": 61, "scienc": 62, "report": 62, "technic": 62, "audienc": 62, "introduct": 62, "execut": 62, "appendix": 62, "power": 28, "calcul": 28, "extra": 28, "extens": 28, "footnot": 28}, "envversion": {"sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "nbsphinx": 4, "sphinx": 60}, "alltitles": {"Backwards Design": [[0, "Backwards-Design"], [2, "Backwards-Design"]], "Overview": [[0, "Overview"], [2, "Overview"]], "1) Define Your Problem": [[0, "1)-Define-Your-Problem"], [2, "1)-Define-Your-Problem"]], "2) Define the Question You Wish to Answer": [[0, "2)-Define-the-Question-You-Wish-to-Answer"], [2, "2)-Define-the-Question-You-Wish-to-Answer"]], "A Digression on Supervised Machine Learning": [[0, "A-Digression-on-Supervised-Machine-Learning"], [2, "A-Digression-on-Supervised-Machine-Learning"]], "UDS Midterm 2022, Part 2": [[1, "UDS-Midterm-2022,-Part-2"]], "Estimating The Effect of Education on Employment": [[1, "Estimating-The-Effect-of-Education-on-Employment"]], "Getting to Know Your Data": [[1, "Getting-to-Know-Your-Data"]], "Subsetting for People over 25 in the Labor Force": [[1, "Subsetting-for-People-over-25-in-the-Labor-Force"]], "Measuring Education": [[1, "Measuring-Education"]], "Education and Employment": [[1, "Education-and-Employment"]], "Defining a Good Question": [[2, "Defining-a-Good-Question"]], "Why is Having a Good Question Important?": [[2, "Why-is-Having-a-Good-Question-Important?"]], "3) Write Down What An Answer Would Look Like": [[2, "3)-Write-Down-What-An-Answer-Would-Look-Like"]], "Falsifiability": [[2, "Falsifiability"]], "4) What Data Do You Need?": [[2, "4)-What-Data-Do-You-Need?"]], "5) Where Can You Get That Data?": [[2, "5)-Where-Can-You-Get-That-Data?"]], "Wrapping Up": [[2, "Wrapping-Up"]], "Beyond The Experiment": [[3, "Beyond-The-Experiment"]], "A/B Testing Isn\u2019t Always Feasible": [[3, "A/B-Testing-Isn't-Always-Feasible"]], "A/B Testing Isn\u2019t Always Legal or Ethical": [[3, "A/B-Testing-Isn't-Always-Legal-or-Ethical"]], "Going Beyond Testing for the Future": [[3, "Going-Beyond-Testing-for-the-Future"]], "Don\u2019t want to be limited": [[3, "Don't-want-to-be-limited"]], "Not enough to be able to do it; you must be able to explain it": [[3, "Not-enough-to-be-able-to-do-it;-you-must-be-able-to-explain-it"]], "Researcher Discretion in Descriptive Analysis": [[5, "Researcher-Discretion-in-Descriptive-Analysis"]], "Faithful Representation": [[5, "Faithful-Representation"]], "Choosing What To Ask and Present": [[5, "Choosing-What-To-Ask-and-Present"]], "Context": [[5, "Context"], [31, "Context"]], "Values": [[5, "Values"]], "Peeking / Endogenous Stopping": [[6, "Peeking-/-Endogenous-Stopping"]], "Recommendations for Responsible Machine Learning": [[7, "Recommendations-for-Responsible-Machine-Learning"]], "1. Choose Your Training Data Carefully": [[7, "1.-Choose-Your-Training-Data-Carefully"]], "2. Only Use Opaque Models When You Have To": [[7, "2.-Only-Use-Opaque-Models-When-You-Have-To"]], "3. Use Models with Functional Form Constraints": [[7, "3.-Use-Models-with-Functional-Form-Constraints"]], "4. Compare Predictions for Sub-Populations": [[7, "4.-Compare-Predictions-for-Sub-Populations"]], "5. \u201cNo\u201d is Always An Option": [[7, "5.-%22No%22-is-Always-An-Option"]], "Tools for Interpretable / Constrained Machine Learning": [[7, "Tools-for-Interpretable-/-Constrained-Machine-Learning"]], "Interpretable Models": [[7, "Interpretable-Models"]], "Tools for Neural Networks": [[7, "Tools-for-Neural-Networks"]], "Constrained Models": [[7, "Constrained-Models"]], "Tools for Introspection": [[7, "Tools-for-Introspection"]], "Evaluating Real Studies": [[8, "Evaluating-Real-Studies"]], "The Sun Is Bright for Dogs Too": [[8, "The-Sun-Is-Bright-for-Dogs-Too"]], "Evaluating Baseline Differences": [[8, "Evaluating-Baseline-Differences"]], "Why we need all three": [[8, "Why-we-need-all-three"]], "Evaulating Differential Treatment Effects": [[8, "Evaulating-Differential-Treatment-Effects"]], "What Comes Next?": [[8, "What-Comes-Next?"]], "Well this has been unsatisfying": [[8, "Well-this-has-been-unsatisfying"]], "Discussion of Descriptive Exercises": [[9, "Discussion-of-Descriptive-Exercises"]], "Exercise 8": [[9, "Exercise-8"], [15, "Exercise-8"], [20, "Exercise-8"], [21, "Exercise-8"], [22, "Exercise-8"], [24, "Exercise-8"], [25, "Exercise-8"], [29, "Exercise-8"], [37, "Exercise-8"], [38, "Exercise-8"], [39, "Exercise-8"], [41, "Exercise-8"], [42, "Exercise-8"], [30, "Exercise-8"], [11, "Exercise-8"], [28, "Exercise-8"]], "Takeaways": [[9, "Takeaways"]], "Absolutely positively need the solutions?": [[9, "Absolutely-positively-need-the-solutions?"], [20, "Absolutely-positively-need-the-solutions?"], [22, "Absolutely-positively-need-the-solutions?"], [24, "Absolutely-positively-need-the-solutions?"], [38, "Absolutely-positively-need-the-solutions?"]], "Interpreting Causal Effects of Gender": [[10, "Interpreting-Causal-Effects-of-Gender"]], "Counter-Factuals and Experimental Ideals": [[12, "Counter-Factuals-and-Experimental-Ideals"]], "What is a Counterfactual?": [[12, "What-is-a-Counterfactual?"]], "What is a randomized experiment?": [[12, "What-is-a-randomized-experiment?"]], "Identifying Counter-Factuals": [[12, "Identifying-Counter-Factuals"]], "Exercise 1:": [[12, "Exercise-1:"], [29, "Exercise-1:"], [41, "Exercise-1:"]], "Exercise 2:": [[12, "Exercise-2:"], [17, "Exercise-2:"], [29, "Exercise-2:"], [41, "Exercise-2:"]], "Exercise 3:": [[12, "Exercise-3:"]], "Exercise 1": [[13, "Exercise-1"], [15, "Exercise-1"], [17, "Exercise-1"], [20, "Exercise-1"], [21, "Exercise-1"], [22, "Exercise-1"], [24, "Exercise-1"], [25, "Exercise-1"], [31, "Exercise-1"], [32, "Exercise-1"], [34, "Exercise-1"], [35, "Exercise-1"], [37, "Exercise-1"], [38, "Exercise-1"], [39, "Exercise-1"], [42, "Exercise-1"], [30, "Exercise-1"], [11, "Exercise-1"], [28, "Exercise-1"]], "Exercise 2": [[13, "Exercise-2"], [15, "Exercise-2"], [20, "Exercise-2"], [21, "Exercise-2"], [22, "Exercise-2"], [24, "Exercise-2"], [25, "Exercise-2"], [31, "Exercise-2"], [32, "Exercise-2"], [34, "Exercise-2"], [34, "id1"], [35, "Exercise-2"], [37, "Exercise-2"], [38, "Exercise-2"], [39, "Exercise-2"], [42, "Exercise-2"], [30, "Exercise-2"], [11, "Exercise-2"], [28, "Exercise-2"]], "Exercise 3": [[13, "Exercise-3"], [15, "Exercise-3"], [20, "Exercise-3"], [21, "Exercise-3"], [22, "Exercise-3"], [24, "Exercise-3"], [25, "Exercise-3"], [29, "Exercise-3"], [31, "Exercise-3"], [34, "Exercise-3"], [35, "Exercise-3"], [37, "Exercise-3"], [38, "Exercise-3"], [39, "Exercise-3"], [41, "Exercise-3"], [42, "Exercise-3"], [30, "Exercise-3"], [11, "Exercise-3"], [28, "Exercise-3"]], "Exercise 4": [[13, "Exercise-4"], [15, "Exercise-4"], [20, "Exercise-4"], [21, "Exercise-4"], [22, "Exercise-4"], [24, "Exercise-4"], [25, "Exercise-4"], [29, "Exercise-4"], [34, "Exercise-4"], [35, "Exercise-4"], [37, "Exercise-4"], [38, "Exercise-4"], [39, "Exercise-4"], [41, "Exercise-4"], [42, "Exercise-4"], [30, "Exercise-4"], [11, "Exercise-4"], [28, "Exercise-4"]], "Exercise 5": [[13, "Exercise-5"], [15, "Exercise-5"], [20, "Exercise-5"], [21, "Exercise-5"], [22, "Exercise-5"], [24, "Exercise-5"], [25, "Exercise-5"], [29, "Exercise-5"], [34, "Exercise-5"], [35, "Exercise-5"], [37, "Exercise-5"], [38, "Exercise-5"], [39, "Exercise-5"], [41, "Exercise-5"], [42, "Exercise-5"], [30, "Exercise-5"], [11, "Exercise-5"], [28, "Exercise-5"]], "Exercise 6": [[13, "Exercise-6"], [15, "Exercise-6"], [20, "Exercise-6"], [21, "Exercise-6"], [22, "Exercise-6"], [24, "Exercise-6"], [24, "id1"], [25, "Exercise-6"], [29, "Exercise-6"], [34, "Exercise-6"], [35, "Exercise-6"], [37, "Exercise-6"], [38, "Exercise-6"], [39, "Exercise-6"], [41, "Exercise-6"], [42, "Exercise-6"], [30, "Exercise-6"], [11, "Exercise-6"], [28, "Exercise-6"]], "Exercise 7": [[13, "Exercise-7"], [15, "Exercise-7"], [20, "Exercise-7"], [21, "Exercise-7"], [22, "Exercise-7"], [24, "Exercise-7"], [25, "Exercise-7"], [29, "Exercise-7"], [34, "Exercise-7"], [35, "Exercise-7"], [37, "Exercise-7"], [38, "Exercise-7"], [39, "Exercise-7"], [41, "Exercise-7"], [42, "Exercise-7"], [30, "Exercise-7"], [11, "Exercise-7"], [28, "Exercise-7"]], "Marijuana Legalization and Violent Crime": [[13, "Marijuana-Legalization-and-Violent-Crime"]], "Evaluating Studies": [[14, "Evaluating-Studies"]], "Study 1: Gym Memberships": [[14, "Study-1:-Gym-Memberships"]], "Study 2: Car Insurance": [[14, "Study-2:-Car-Insurance"]], "Study 3: Billboard": [[14, "Study-3:-Billboard"]], "Crime and Policing Expenditures Exploratory Questions": [[15, "Crime-and-Policing-Expenditures-Exploratory-Questions"]], "Exercises": [[15, "Exercises"], [28, "Exercises"]], "After you have answered\u2026": [[15, "After-you-have-answered..."], [34, "After-you-have-answered..."]], "Your First Stakeholder": [[16, "Your-First-Stakeholder"]], "The Proposal": [[16, "The-Proposal"]], "Your Task": [[16, "Your-Task"], [31, "Your-Task"]], "Converting Stakeholder Prompts into Actionable Questions": [[17, "Converting-Stakeholder-Prompts-into-Actionable-Questions"]], "Causality in the Time of Cholera": [[18, "Causality-in-the-Time-of-Cholera"], [19, "Causality-in-the-Time-of-Cholera"]], "Question 1:": [[18, "Question-1:"], [19, "Question-1:"]], "Question 2:": [[18, "Question-2:"], [19, "Question-2:"]], "Question 3:": [[18, "Question-3:"], [19, "Question-3:"]], "Question 4": [[18, "Question-4"], [19, "Question-4"]], "Question 5": [[18, "Question-5"], [19, "Question-5"]], "Question 6": [[18, "Question-6"], [19, "Question-6"]], "Question 7:": [[18, "Question-7:"], [19, "Question-7:"]], "Question 8:": [[18, "Question-8:"], [19, "Question-8:"]], "Question 9:": [[18, "Question-9:"], [19, "Question-9:"]], "Interpreting Indicator Variables": [[20, "Interpreting-Indicator-Variables"], [37, "Interpreting-Indicator-Variables"]], "Indicator Variables and Omitted Variable Bias": [[20, "Indicator-Variables-and-Omitted-Variable-Bias"], [37, "Indicator-Variables-and-Omitted-Variable-Bias"]], "Interaction Effects": [[20, "Interaction-Effects"], [37, "Interaction-Effects"]], "Exercise 9": [[21, "Exercise-9"], [22, "Exercise-9"], [25, "Exercise-9"], [29, "Exercise-9"], [38, "Exercise-9"], [39, "Exercise-9"], [41, "Exercise-9"], [42, "Exercise-9"], [30, "Exercise-9"], [11, "Exercise-9"], [28, "Exercise-9"]], "Exercise 10": [[21, "Exercise-10"], [22, "Exercise-10"], [25, "Exercise-10"], [29, "Exercise-10"], [38, "Exercise-10"], [41, "Exercise-10"], [42, "Exercise-10"], [30, "Exercise-10"], [11, "Exercise-10"], [28, "Exercise-10"]], "Exercise 11": [[21, "Exercise-11"], [22, "Exercise-11"], [25, "Exercise-11"], [29, "Exercise-11"], [38, "Exercise-11"], [41, "Exercise-11"], [42, "Exercise-11"], [30, "Exercise-11"], [11, "Exercise-11"], [28, "Exercise-11"]], "Exercise 12": [[21, "Exercise-12"], [22, "Exercise-12"], [25, "Exercise-12"], [29, "Exercise-12"], [38, "Exercise-12"], [41, "Exercise-12"], [42, "Exercise-12"], [30, "Exercise-12"], [11, "Exercise-12"]], "Exercise 13": [[21, "Exercise-13"], [22, "Exercise-13"], [25, "Exercise-13"], [29, "Exercise-13"], [38, "Exercise-13"], [41, "Exercise-13"], [30, "Exercise-13"], [11, "Exercise-13"], [28, "Exercise-13"]], "Exercise 14": [[21, "Exercise-14"], [22, "Exercise-14"], [25, "Exercise-14"], [29, "Exercise-14"], [38, "Exercise-14"], [41, "Exercise-14"], [11, "Exercise-14"], [28, "Exercise-14"]], "Interpretable Modelling of Credit Risk": [[21, "Interpretable-Modelling-of-Credit-Risk"]], "Data Prep": [[21, "Data-Prep"]], "Model Fitting": [[21, "Model-Fitting"]], "Let\u2019s See This Interpretability!": [[21, "Let's-See-This-Interpretability!"]], "Exercise 15": [[21, "Exercise-15"], [22, "Exercise-15"], [25, "Exercise-15"], [38, "Exercise-15"], [11, "Exercise-15"], [28, "Exercise-15"]], "Exercise 16": [[21, "Exercise-16"], [22, "Exercise-16"], [25, "Exercise-16"], [38, "Exercise-16"]], "Exercise 17": [[21, "Exercise-17"], [25, "Exercise-17"]], "Matching Exercise": [[22, "Matching-Exercise"], [38, "Matching-Exercise"]], "Matching Packages: Python v. R": [[22, "Matching-Packages:-Python-v.-R"], [38, "Matching-Packages:-Python-v.-R"]], "Installing dame-flame.": [[22, "Installing-dame-flame."], [38, "Installing-dame-flame."]], "Data Setup": [[22, "Data-Setup"], [38, "Data-Setup"]], "Getting To Know Your Data": [[22, "Getting-To-Know-Your-Data"], [38, "Getting-To-Know-Your-Data"]], "Matching!": [[22, "Matching!"], [38, "Matching!"]], "Let\u2019s Do Matching with DAME": [[22, "Let's-Do-Matching-with-DAME"], [38, "Let's-Do-Matching-with-DAME"]], "Interpreting DAME output": [[22, "Interpreting-DAME-output"], [38, "Interpreting-DAME-output"]], "Getting Back a Dataset": [[22, "Getting-Back-a-Dataset"], [38, "Getting-Back-a-Dataset"]], "Check Your Matches and Analyze": [[22, "Check-Your-Matches-and-Analyze"], [38, "Check-Your-Matches-and-Analyze"]], "Other Forms of Matching": [[22, "Other-Forms-of-Matching"], [38, "Other-Forms-of-Matching"]], "Optimal AB Testing Design": [[23, "Optimal-AB-Testing-Design"]], "The Experiment": [[23, "The-Experiment"]], "Question 2": [[23, "Question-2"]], "Question 3": [[23, "Question-3"]], "Traffic Death Analysis": [[24, "Traffic-Death-Analysis"], [39, "Traffic-Death-Analysis"]], "Fixed Effects by Demeaning": [[24, "Fixed-Effects-by-Demeaning"], [39, "Fixed-Effects-by-Demeaning"]], "Predicting Mortgage Delinquency Risk": [[25, "Predicting-Mortgage-Delinquency-Risk"]], "Gradescope Autograding": [[25, "Gradescope-Autograding"], [30, "Gradescope-Autograding"], [11, "Gradescope-Autograding"]], "Submission Limits": [[25, "Submission-Limits"], [30, "Submission-Limits"], [11, "Submission-Limits"]], "Data Cleaning and Organization": [[25, "Data-Cleaning-and-Organization"]], "Modelling Delinquency Risk": [[25, "Modelling-Delinquency-Risk"]], "Now To The Future": [[25, "Now-To-The-Future"]], "Exercise 18": [[25, "Exercise-18"]], "Making Potential Outcomes Concrete": [[26, "Making-Potential-Outcomes-Concrete"], [40, "Making-Potential-Outcomes-Concrete"]], "The Study": [[26, "The-Study"], [27, "The-Study"], [40, "The-Study"]], "Mapping to the Potential Outcomes Framework": [[26, "Mapping-to-the-Potential-Outcomes-Framework"], [27, "Mapping-to-the-Potential-Outcomes-Framework"], [40, "Mapping-to-the-Potential-Outcomes-Framework"]], "Observability": [[26, "Observability"], [27, "Observability"], [40, "Observability"]], "Causal Inference": [[26, "Causal-Inference"], [27, "Causal-Inference"], [40, "Causal-Inference"]], "Making Potential Outcomes Concrete 2": [[27, "Making-Potential-Outcomes-Concrete-2"]], "Estimating Gender Discrimination in the Workplace": [[29, "Estimating-Gender-Discrimination-in-the-Workplace"], [41, "Estimating-Gender-Discrimination-in-the-Workplace"]], "Soliciting Information From Your Stakeholder": [[31, "Soliciting-Information-From-Your-Stakeholder"]], "Stakeholder Project Proposal": [[31, "Stakeholder-Project-Proposal"]], "A Data Analysis of Voting Access on College Campuses": [[31, "A-Data-Analysis-of-Voting-Access-on-College-Campuses"]], "Background": [[31, "Background"], [62, "Background"]], "Objectives": [[31, "Objectives"]], "Our Approach": [[31, "Our-Approach"]], "Generating and Classifying Questions": [[32, "Generating-and-Classifying-Questions"]], "Exploratory Question:": [[32, "Exploratory-Question:"], [32, "id1"]], "Passive-Predictive Question:": [[32, "Passive-Predictive-Question:"], [32, "id2"]], "Causal Question:": [[32, "Causal-Question:"], [32, "id3"]], "Groupby and Arrest Data": [[33, "Groupby-and-Arrest-Data"], [35, "Groupby-and-Arrest-Data"]], "Learning the Group Structure of Your Data": [[33, "Learning-the-Group-Structure-of-Your-Data"]], "Testing Your Assumptions": [[33, "Testing-Your-Assumptions"]], "Crime and Poverty Descriptive Analysis": [[34, "Crime-and-Poverty-Descriptive-Analysis"]], "Acme Corportation Solution": [[36, "Acme-Corportation-Solution"]], "Resume Experiment Analysis": [[42, "Resume-Experiment-Analysis"], [30, "Resume-Experiment-Analysis"]], "Checking for Balance": [[42, "Checking-for-Balance"], [30, "Checking-for-Balance"]], "Estimating Effect of Race": [[42, "Estimating-Effect-of-Race"], [30, "Estimating-Effect-of-Race"]], "Estimating Heterogeneous Effects": [[42, "Estimating-Heterogeneous-Effects"], [30, "Estimating-Heterogeneous-Effects"]], "What Did We Just Measure?": [[42, "What-Did-We-Just-Measure?"], [30, "What-Did-We-Just-Measure?"]], "Fixed Effects: Indicator Variables for Groups": [[43, "Fixed-Effects:-Indicator-Variables-for-Groups"]], "Implementing Fixed Effects": [[43, "Implementing-Fixed-Effects"]], "Clustering": [[43, "Clustering"]], "Computationally Efficient Fixed Effects": [[43, "Computationally-Efficient-Fixed-Effects"]], "Fixed Effects and Causal Inference": [[44, "Fixed-Effects-and-Causal-Inference"]], "Fixed Effects and Hierarchical Models": [[45, "Fixed-Effects-and-Hierarchical-Models"]], "Fixed Effects & Clustered Standard Errors": [[45, "Fixed-Effects-&-Clustered-Standard-Errors"]], "Things To Remember:": [[45, "Things-To-Remember:"]], "How to Read (Academic Edition)": [[46, "How-to-Read-(Academic-Edition)"]], "Read Actively": [[46, "Read-Actively"]], "Be Patient with Examples From Different Domains": [[46, "Be-Patient-with-Examples-From-Different-Domains"]], "Do NOT Summarize with LLMs": [[46, "Do-NOT-Summarize-with-LLMs"]], "Welcome to Solving Problems with Data!": [[47, "welcome-to-solving-problems-with-data"]], "Pre-Requisites for Non-MIDS Students": [[47, "pre-requisites-for-non-mids-students"]], "Syllabus": [[47, "syllabus"]], "Internal and External Validity": [[48, "Internal-and-External-Validity"]], "Internal Validity": [[48, "Internal-Validity"]], "External Validity": [[48, "External-Validity"]], "External Validity Considerations": [[48, "External-Validity-Considerations"]], "Trade-Offs Between Internal and External Validity": [[48, "Trade-Offs-Between-Internal-and-External-Validity"]], "Conclusion": [[48, "Conclusion"], [60, "Conclusion"]], "Using and Interpreting Indicator (Dummy) Variables": [[49, "Using-and-Interpreting-Indicator-(Dummy)-Variables"]], "What are indicator variables?": [[49, "What-are-indicator-variables?"]], "The ONE thing that you must understand when using indicator variables:": [[49, "The-ONE-thing-that-you-must-understand-when-using-indicator-variables:"]], "Indicator Variables with Two Category Variable": [[49, "Indicator-Variables-with-Two-Category-Variable"]], "Indicator Variables for variables with more than 2 categories": [[49, "Indicator-Variables-for-variables-with-more-than-2-categories"]], "Interactions with Constant Variables": [[49, "Interactions-with-Constant-Variables"]], "Keeping Things Straight": [[49, "Keeping-Things-Straight"]], "The * operator": [[49, "The-*-operator"]], "Interactions Between Multiple Indicator Variables": [[49, "Interactions-Between-Multiple-Indicator-Variables"]], "Fixed Effects": [[49, "Fixed-Effects"]], "Limitations of Experiments (and Average Treatment Effects)": [[50, "Limitations-of-Experiments-(and-Average-Treatment-Effects)"]], "Who\u2019s Average?": [[50, "Who's-Average?"]], "The Fine Print of ATE": [[50, "The-Fine-Print-of-ATE"]], "SUTVA": [[50, "SUTVA"]], "How Matching is Done (A Summary)": [[51, "How-Matching-is-Done-(A-Summary)"]], "Pruning Your Data": [[51, "Pruning-Your-Data"]], "Measuring Similarity": [[51, "Measuring-Similarity"]], "When Can / Should I Use Matching?": [[51, "When-Can-/-Should-I-Use-Matching?"]], "Checking Balance": [[51, "Checking-Balance"]], "Analyze!": [[51, "Analyze!"]], "Specific Models": [[51, "Specific-Models"]], "Matching": [[52, "Matching"]], "Model Dependency": [[52, "Model-Dependency"]], "Model Dependency, Causal Inference, and Imbalance": [[52, "Model-Dependency,-Causal-Inference,-and-Imbalance"]], "Matching: Better thought of as pruning": [[52, "Matching:-Better-thought-of-as-pruning"]], "The Cost of Matching": [[52, "The-Cost-of-Matching"]], "Causal Inference Assumptions": [[52, "Causal-Inference-Assumptions"]], "In Summary": [[52, "In-Summary"]], "Converting Stakeholder Prompts into Questions": [[53, "Converting-Stakeholder-Prompts-into-Questions"]], "What\u2019s The Real Problem / Need": [[53, "What's-The-Real-Problem-/-Need"]], "What Questions Do I Need To Ask?": [[53, "What-Questions-Do-I-Need-To-Ask?"]], "Make Your Questions Specific and Actionable": [[53, "Make-Your-Questions-Specific-and-Actionable"]], "Many Questions?": [[53, "Many-Questions?"]], "What to Read Next": [[53, "What-to-Read-Next"]], "Nick\u2019s Things to Think About Before A/B Testing": [[54, "Nick's-Things-to-Think-About-Before-A/B-Testing"]], "Step 0: Are You Sure You Should Do This?": [[54, "Step-0:-Are-You-Sure-You-Should-Do-This?"]], "Step 1: Define Your Goals": [[54, "Step-1:-Define-Your-Goals"]], "Step 2: Plan Implementation": [[54, "Step-2:-Plan-Implementation"]], "Step 3: Run!": [[54, "Step-3:-Run!"]], "Step 4: Check Internal Validity": [[54, "Step-4:-Check-Internal-Validity"]], "Step 5: Interpret Results": [[54, "Step-5:-Interpret-Results"]], "Step 6: Consider External Validity": [[54, "Step-6:-Consider-External-Validity"]], "Reading Reflections": [[57, "Reading-Reflections"]], "Solutions": [[58, "Solutions"]], "Looking for the solutions?": [[59, "Looking-for-the-solutions?"]], "OK, if you are still interested in looking at the solutions\u2026": [[59, "OK,-if-you-are-still-interested-in-looking-at-the-solutions..."]], "Taxonomy of Questions": [[60, "Taxonomy-of-Questions"]], "Positive versus Normative Questions": [[60, "Positive-versus-Normative-Questions"]], "Descriptive Questions": [[60, "Descriptive-Questions"]], "Descriptive Example 1: Nope, This Time Isn\u2019t Different": [[60, "Descriptive-Example-1:-Nope,-This-Time-Isn't-Different"]], "Descriptive Example 3: Disease Surveillance": [[60, "Descriptive-Example-3:-Disease-Surveillance"]], "Descriptive Example 4: Global Warming": [[60, "Descriptive-Example-4:-Global-Warming"]], "Descriptive Questions and EDA": [[60, "Descriptive-Questions-and-EDA"]], "Last Thoughts": [[60, "Last-Thoughts"]], "Causal Questions and Prediction-with-Manipulation": [[60, "Causal-Questions-and-Prediction-with-Manipulation"]], "Why Causal Inference is Hard": [[60, "Why-Causal-Inference-is-Hard"]], "Causal Inference, Manipulations, and Prediction": [[60, "Causal-Inference,-Manipulations,-and-Prediction"]], "Classification Questions / Prediction-Without-Manipulation": [[60, "Classification-Questions-/-Prediction-Without-Manipulation"]], "\u201cWithout Manipulation\u201d?": [[60, "%22Without-Manipulation%22?"]], "Using Exploratory Questions to Better Understanding Your Problem": [[61, "Using-Exploratory-Questions-to-Better-Understanding-Your-Problem"]], "Your First Assignment": [[61, "Your-First-Assignment"]], "Part 2: Choosing a Problem": [[61, "Part-2:-Choosing-a-Problem"]], "Part 3: Specifying and Answering Questions": [[61, "Part-3:-Specifying-and-Answering-Questions"]], "Data": [[61, "Data"]], "Writing Data Science Report for Non-Technical Audiences": [[62, "Writing-Data-Science-Report-for-Non-Technical-Audiences"]], "Identify your audience": [[62, "Identify-your-audience"]], "Introduction / Executive Summary": [[62, "Introduction-/-Executive-Summary"]], "Your Design": [[62, "Your-Design"]], "Your Results": [[62, "Your-Results"]], "Conclusions": [[62, "Conclusions"]], "Appendix": [[62, "Appendix"]], "A/B Testing the Udacity Website": [[11, "A/B-Testing-the-Udacity-Website"]], "Udacity\u2019s Test": [[11, "Udacity's-Test"]], "Import the Data": [[11, "Import-the-Data"]], "Pick your measures": [[11, "Pick-your-measures"]], "Validating The Data": [[11, "Validating-The-Data"]], "Estimating the Effect of Experiment": [[11, "Estimating-the-Effect-of-Experiment"]], "Where To Find Public Data": [[55, "Where-To-Find-Public-Data"]], "Good Starting Points": [[55, "Good-Starting-Points"]], "Other Interesting Data Sources": [[55, "Other-Interesting-Data-Sources"]], "Data with a Spatial/GIS Component": [[55, "Data-with-a-Spatial/GIS-Component"]], "Public Satellite Data": [[55, "Public-Satellite-Data"]], "Other Lists of Data That Are Great": [[55, "Other-Lists-of-Data-That-Are-Great"]], "Class Schedule": [[4, "class-schedule"]], "Power Calculations and Experiment Planning": [[28, "Power-Calculations-and-Experiment-Planning"]], "The Context": [[28, "The-Context"]], "Extra Credit Extension": [[28, "Extra-Credit-Extension"]], "Footnotes": [[28, "Footnotes"]]}, "indexentries": {}})
\ No newline at end of file
diff --git a/docs/html/sitemap.xml b/docs/html/sitemap.xml
index 4b674beb..54e15aef 100644
--- a/docs/html/sitemap.xml
+++ b/docs/html/sitemap.xml
@@ -1 +1 @@
-http://unifyingdatascience.orgclass_schedule.html http://unifyingdatascience.orgindex.html http://unifyingdatascience.orggenindex.html http://unifyingdatascience.orgsearch.html
\ No newline at end of file
+http://unifyingdatascience.orgexercises/exercise_power_calculations.html http://unifyingdatascience.orgindex.html http://unifyingdatascience.orggenindex.html http://unifyingdatascience.orgsearch.html
\ No newline at end of file
diff --git a/source/exercises/exercise_power_calculations.ipynb b/source/exercises/exercise_power_calculations.ipynb
new file mode 100644
index 00000000..942a31f4
--- /dev/null
+++ b/source/exercises/exercise_power_calculations.ipynb
@@ -0,0 +1,271 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Power Calculations and Experiment Planning\n",
+ "\n",
+ "When you read about them in a book, power calculations can feel very abstract. To make the concept more concrete, and to give you a chance to familiarize yourself with power calculation tools and the considerations that go into power calculations, in this exercise you will play the role of an experiment planner hired by a Non-Governmental Organization (NGO) named [Bandhan](https://www.bandhan.org/) in West Bengal, India. Bandhan is planning to roll out a program that provides livestock, cash, and some basic training to households in extreme poverty. They hope to demonstrate the value of cash and asset transfers, and so wish their program to take the form of a randomized experiment, with data being collected on both control and treatment households.\n",
+ "\n",
+ "Your job will be to help them estimate how many households they should enroll in the program.\n",
+ "\n",
+ "This was, in fact, a real program, and so the data you will be working with comes from the actual program (conducted in 2007, with followup data collection in 2010, 2017, and 2020), allowing us to do some retrospective comparison of your power calculations and what actually occurred."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## The Context\n",
+ "\n",
+ "> Development economics has long posited that the poor may be poor for no good reason other than the fact that they started poor. This is the idea of a poverty trap, which has the implication that a one-time capital grant that makes very poor households significantly less poor (\"big push\") might set off a virtuous cycle that takes them out of poverty. Forty-three countries now embrace some version of this idea and make large transfers to over 3.1 million of their poorest households. In particular, the \"Targeting the Ultra Poor\" (TUP) intervention, pioneered by BRAC in Bangladesh, employs a multifaceted approach, offering poor households an asset transfer, consumption support, savings, and training in the hopes of unlocking a poverty trap.\n",
+ "\n",
+ "- [*Long-Term Effects of the Targeting the Ultra Poor Program*](https://economics.mit.edu/research/publications/long-term-effects-targeting-ultra-poor-program), Abhijit Banerjee, Esther Duflo, and Garima Sharma, 2021.\n",
+ "\n",
+ "In 2007, Bandhan in West Bengal, India created a pilot program to provide direct, unconditional transfers of productive livestock and a 30 or 40 week stipend of 90 rupees a week (about 7 US dollars a week using the [Purchasing Power Parity (PPP)](https://en.wikipedia.org/wiki/Purchasing_power_parity) exchange rate). The program targeted the poorest households in these villages based on range of criteria, and the average eligible household was later estimated to have a consumption level of about 1.35 2018 US Dollars per capita per day in PPP terms $.^1$\n",
+ "\n",
+ "Because the goal of the program was, in significant part, to demonstrate the effect of direct asset transfers, Bandhan identified twice as many eligible households as it could support in the pilot and randomly selected half of the households to act as controls and half to be treated $.^2$ Treated households were offered assets from a menu of options, from the most common choice was productive livestock (e.g., cows and goats). $^3$ Bandhan's contact with the households came to an end 18 months after the households were initially provided their livestock and cash transfers began.\n",
+ "\n",
+ "Although data was collected on a huge range of attributes of the households in this program, our focus will be on per capita household expenditures in 2018 US Dollars (PPP). \n",
+ "\n",
+ "(Note that these households certainly were not buying their food with dollars, and in most cases they weren't even buying most of their food with rupees — consumption estimates in this type of study are calculated by collecting detailed data on what household members have consumed in the past week, then estimating the price one would pay to buy those goods in local markets and using [PPP conversion rates](https://en.wikipedia.org/wiki/Purchasing_power_parity) conversion rates to convert that into US Dollars).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## Exercises\n",
+ "\n",
+ "### Exercise 1\n",
+ "\n",
+ "Load a thinned version of household-level survey data from [Banerjee, Duflo, and Sharma (2021)'s](https://economics.mit.edu/research/publications/long-term-effects-targeting-ultra-poor-program) evaluation of the Bandham program [here](https://github.com/nickeubank/MIDS_Data/tree/master/cash_transfers). \n",
+ "\n",
+ "This is just a version of the replication data for that paper ([which can be found here](https://www.openicpsr.org/openicpsr/project/130362)) with only the variables that are relevant for our analysis."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 2\n",
+ "\n",
+ "The first step in doing any power calculations is to establish an estimate of the baseline level of variation that exists in the dependent variable you wish to study. In this case, that variable is Per Capita Monthly Expenditures, and the baseline survey values of expenditures are in `pc_exp_month_bl` (the `_bl` stands for \"baseline\", meaning this data was collected prior to any households receiving asset transfers and indeed any households being assigned to treatment or control).\n",
+ "\n",
+ "What is the mean and standard deviation of `pc_exp_month_bl` in the baseline survey? Divide by 30 to confirm our average household has the expected per capita expenditure level of about US Dollar 1.35 (PPP) a day.\n",
+ "\n",
+ "You'll use these quantites a lot, so assign them to variables."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 3\n",
+ "\n",
+ "For these exercises, we'll use the `power` module in `statsmodels`. [You can read about it here.](https://www.statsmodels.org/stable/stats.html#power-and-sample-size-calculations)\n",
+ "\n",
+ "Since we're comparing means in a continuous variable (expenditures) from two samples of households, we will use `TTestIndPower` in `statsmodels.stats.power`. Import this class and instantiate a new instance (for some reason this is class based, so you have to start of with a command like `my_power = TTestIndPower()`). \n",
+ "\n",
+ "Note that a common situation in data science is testing a difference in *proportions*, as in situations where your dependent variable is binary and each group's mean is the share for whom the binary variable is 1. This comes up a lot with apps and websites — e.g., \"clicked an ad,\" \"subscribed,\" \"made a purchase.\" For that reason, there's actually a full sub-class of power calculating tools for [proportions you can read about here.](https://www.statsmodels.org/stable/stats.html#proportion) Basically, because the standard deviation of a binary variable is just $\\sqrt{p * (1-p)}$, power calculations become really simple."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 4\n",
+ "\n",
+ "[TTestIndPower has three methods](https://www.statsmodels.org/stable/generated/statsmodels.stats.power.TTestIndPower.html#statsmodels.stats.power.TTestIndPower) — `plot_power`, `power`, and `solve_power` — but `solve_power` does everything `power` does, so there are basically two methods.\n",
+ "\n",
+ "The idea of `solve_power` is that you give it all but one parameter of a power calculation — minimum detectable effect, number of observations, power, and p-value $\\alpha$ threshold — and it will solve for the omitted variable.\n",
+ "\n",
+ "To get a quick flavor for how this works, set: \n",
+ "\n",
+ "- Minimal Detectable Effect (`effect_size`) to `0.5` (we'll take about the units of that argument soon), \n",
+ "- `alpha` to `0.05`, \n",
+ "- `ratio` to `1` (this is the ratio of between the number of observations being treated and the number in control — in this study, the target was to have these be equal, as that provides the best statistical power for a given total number of observations).\n",
+ "- `power` to `0.8`\n",
+ "- `alternative` to `two-sided` (we want to do a two-tailed t-test, as you could imagine social unrest caused by transfers could make people worse off).\n",
+ "- `nobs1` to `None`. By passing `None`, we're telling `solve_power` this is the quantity we want it to solve for.\n",
+ "\n",
+ "So in other words, we're asking:\n",
+ "\n",
+ "- how many observations do we need to have in the treatment arm,\n",
+ "- assuming we'll have the same number also in the control arm,\n",
+ "- to have an 80% chance\n",
+ "- of rejecting the null hypothesis of no-effect at a significance threshold of 0.05\n",
+ "- if the true effect is of size 0.5?\n",
+ "\n",
+ "What's the answer?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 5\n",
+ "\n",
+ "Of all these quantities, perhaps the most obscure is the Minimal Detectable Effect (`effect_size`). What is this 0.5? Where did it come from? And how did you do all that without telling it that standard deviation you had me calculate?\n",
+ "\n",
+ "If you check the docs for `solve_power`, you'll see that you're supposed to pass `effect_size` the \"standardized effect size, difference between the two means divided by the standard deviation. `effect_size` has to be positive.\"\n",
+ "\n",
+ "In other words, when we passed 0.5, we weren't saying we wanted to be able to detect an effect of 0.5 2018 US Dollars (PPP) per capita per month, we were saying we wanted to detect an effect of 1/2 standard deviation. \n",
+ "\n",
+ "Given the standard deviation in our baseline survey was about 24 Dollars, that means we were saying we wanted to detect an effect size of about 12 US Dollars. \n",
+ "\n",
+ "Is that a lot? Well, the average per capita monthly expenditures in the baseline survey was about 41 Dollars, so it's an increase in expenditures of about 25%. \n",
+ "\n",
+ "OK, but is that what we should use?\n",
+ "\n",
+ "Well, the idea of a Minimal Detectable Effect is that you want to set it at a level where (a) if this *was* the real effect when you rejected the null, you'd be happy you did the experiment and, presumably, would be comfortable scaling up the treatment, but also (b) if the true effect were any smaller, you wouldn't want to reject the null hypothesis and scale up the treatment.\n",
+ "\n",
+ "Basically, if you set this too high, you may run an under-powered experiment where you miss out on learning that your treatment was effective; if you set this too low, you're spending time and money collecting more data than is probably necessary given you might reject the null at a point estimate where you still wouldn't scale up the treatment.\n",
+ "\n",
+ "So what value makes sense here? That's a question you'd want to start off by asking the stakeholders. Since Bandhan wants to use this to promote cash transfers, they probably need the effect to be relatively large to be convincing. Let's assume that, three years after the intervention, they feel they need to show that incomes have increased by at least 30%. \n",
+ "\n",
+ "What value does that imply should be passed to `effect_size`? What is the new implied number of treated households they'll need? How many households total?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 6\n",
+ "\n",
+ "Let's also suppose that because Bandhan really wants to sell their result, they also want to ensure no one claims it might have arisen by chance. What parameter would you adjust, and in what direction? Try modifying it and get a new observation requirement (can you predict what the change in num of observations will be before you do it?)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 7\n",
+ "\n",
+ "Now suppose that your stakeholder is budget constrained to only being able to enroll a total of 100 (50 in each arm). Assuming an alpha of `0.05` and a power of `0.8`, what is their minimal detectable effect size (in dollars)?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 8\n",
+ "\n",
+ "Suppose your stakeholder wants to see a few different scenarios to see how different experiment sizes would impact power given different effect sizes. Let's use `plot_power`. Pass it an array of treatment arm observation counts you want evaluated (I'd recommend `np.arange()`) between 20 and 100, along with effect sizes of 0.2, 0.4, and 0.6 (again, use `np.arange()`).\n",
+ "\n",
+ "If the stakeholder decided they wanted power above 90% — they're only gonna get a chance to do this once, after all! — what kind of sample size would they need with a minimal detectable effect size of 0.6? (Approximately).\n",
+ "\n",
+ "You may need to check the docs to figure out how to use it."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Extra Credit Extension"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 9\n",
+ "\n",
+ "As noted above, this study actually was conducted, so we know the actual treatment effect. Calculate that treatment effect 3 years after these families first received asset transfers by calculating the difference in `pc_exp_month_el2` between the `treatment == 1` and `treatment == 0` groups.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 10\n",
+ "\n",
+ "What was the statistical power of the actual test? That is, what was the probability, given this effect size and the number of observations in the study, that they would reject the null hypothesis of no effect at alpha=0.05 and given the actual final sample sizes?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 11\n",
+ "\n",
+ "What would the power be if Bandhan had only enrolled 200 households per arm? \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 13\n",
+ "\n",
+ "The idea of the \"power\" of a test is that, when we draw a sample from a population, there's some variation in who happens to end up in that sample. Consequently, even if the *true* effect size in the population is equal to our Minimal Detectable Effect, there's some probability that when we compare treated and untreated outcomes for the $N$ people who actually end up in the study, that Minimal Detectable Effect may not be evident in that sample (at the level of statistical significance $\\alpha$ we have chosen).\n",
+ "\n",
+ "Given that, one way to think of power is: \"If the true effect in the population is our Minimal Detectable Effect, then if I were able to re-run this experiment over and over — drawing new people into the study and testing the difference in outcomes between the control and treated sample each time — then in what percentage of those instances of the experiment would I reject the null hypothesis of no effect?\" Power of 80% means that we'd expect to reject the null of no effect in 80% of those many experiments.\n",
+ "\n",
+ "Well, we can basically do that thought experiment with this data! \n",
+ "\n",
+ "To do the experiment you did the power calculations for in Exercise 11, we just need to sample 200 observations from the treated group and 200 from the control group, treat that as our experimental sample, and estimate the difference in per capita month expenditures between those two groups (along with the associated p-value).\n",
+ "\n",
+ "Then we can repeat that over and over to simulate \"re-running\" the experiment, each time drawing a new sample of 200 treated observations and 200 control observations. Then we can store the p-values from all these \"re-run\" experiments and see how often we reject the null of no effect!\n",
+ "\n",
+ "Note that when you do this experiment, we have to sample our 200 observations from each treatment arm *with replacement*, just as you would when bootstrapping (if that means anything to you — if it doesn't, don't worry about it). \n",
+ "\n",
+ "So: write a loop where, on each pass, you draw 200 observations (with replacement) from treatment and 200 from control, then calculate the treatment effect and p-value for that sample. Repeat this 10,000 times. \n",
+ "\n",
+ "In what share of cases would you reject the null of no effect at alpha = 0.05?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 14\n",
+ "\n",
+ "Repeat 12 and 13 with 300 observations per arm."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exercise 15\n",
+ "\n",
+ "What did you learn from Exercises 9-14?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Footnotes\n",
+ "\n",
+ "1. The poorest households were identified in two steps. First, residents across 120 village hamlets ranked households into five wealth quintiles. Among households ranked in the bottom quintile, Bandhan then verified eligibility per seven criteria: (i) presence of an able-bodied female member (to manage the asset), (ii) no credit access, (iii) landholding below 0.2 acres, (iv) no ownership of productive assets, (v) no able-bodied male member, (vi) presence of school-aged children who were working instead of attending school, and (vii) primary source of income being informal labor or begging. Households had to meet the first two criteria and at least three of the remaining five in order to be eligible for the TUP intervention.\n",
+ "2. The [ethics of randomization in these types of programs](https://www.povertyactionlab.org/resource/ethical-conduct-randomized-evaluations) is subject of a rich literature, and there are currently an extensive set of guidelines used by researchers developing these types of programs.\n",
+ "3. Of the 514 offered the livestock and cash transfers, only 266 accepted. The treatment estimates that follow are thus estimates of the effect of *offering* these transfers. Households that accepted the offers might differ systematically from those that do not, so the study simply compares those *offered* the transfer to those who did not). This is what's called an estimate of the \"intention to treat\" effect, and is thus *likely* an under-estimate of the effect of cash transfers that would be observed if uptake were greater."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.11.8"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}