Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding example for testing proportions #581

Merged
merged 4 commits into from
Apr 30, 2020

Conversation

freddyaboulton
Copy link
Contributor

In reference to issue #189, I added an example for testing if two proportions are equal using logistic regression. This example uses a self-contained, simulated dataset.

@fehiepsi
Copy link
Member

Thanks for contributing this nice example, @freddyaboulton ! I'll review this in detail tomorrow.

examples/proportion_test.py Outdated Show resolved Hide resolved
examples/proportion_test.py Outdated Show resolved Hide resolved
examples/proportion_test.py Outdated Show resolved Hide resolved
examples/proportion_test.py Outdated Show resolved Hide resolved
impact_on_probability = response_with_calls - baseline_response

print(f"There is a {interval_size * 100}% probability that calling customers "
"increases the chance they'll make a purchase by "
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me since we constructed the dataset, but users extrapolating it to more complex examples with multiple predictors might be problematic. @fehiepsi, @martinjankowiak: WDYT?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this is fine for this self-contained example 👍

examples/proportion_test.py Outdated Show resolved Hide resolved
examples/proportion_test.py Outdated Show resolved Hide resolved
examples/proportion_test.py Outdated Show resolved Hide resolved
impact_on_probability = response_with_calls - baseline_response

print(f"There is a {interval_size * 100}% probability that calling customers "
"increases the chance they'll make a purchase by "
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this is fine for this self-contained example 👍

:param design_matrix: Covariates. All categorical variables have been one-hot
encoded.
:param outcome: Binary response variable. In this case, whether or not the
customer made a purchase.
"""

# Use multivariate normal prior in case we want to add more covariates later.

beta = numpyro.sample('coefficients', dist.MultivariateNormal(loc=0.,
covariance_matrix=np.eye(design_matrix.shape[1])))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this can be faster if you replace covariance_matrix by scale_tril. It is still fine if you want to leave it as-is.


)

print("There is a 95% probability the effect of gender on the log odds of conversion "
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto: 0.95 -> interval_size

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still not addressed yet.

@freddyaboulton
Copy link
Contributor Author

@fehiepsi @martinjankowiak @neerajprad

I think I've addressed your comments! Please let me know if you need me to change anything else.

fehiepsi
fehiepsi previously approved these changes Apr 29, 2020
Copy link
Member

@fehiepsi fehiepsi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after the following nit. Nice work, @freddyaboulton !


)

print("There is a 95% probability the effect of gender on the log odds of conversion "
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still not addressed yet.

@freddyaboulton
Copy link
Contributor Author

@fehiepsi Thank you for the help! Should be all good now.

@fehiepsi fehiepsi merged commit 25d055a into pyro-ppl:master Apr 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants