Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ascending parameter causal validation #220

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

MarianaBlaz
Copy link
Contributor

Instructions

  • Follow the instructions in README.md
  • Delete everything between parenthesis (...)
  • Remove the sections that are not relevant

Status

READY

Todo list

Background context

In the causal validation module and the curves file, it would be useful to add an ascending parameter for the cumulative effect and cumulative gain curves.

The current state is to order predictions descending:

ordered_df = df.sort_values(prediction, ascending=False).reset_index(drop=True)
If we add an ascending: bool = False argument to the cumulative_effect_curve, cumulative_gain_curve, relative_cumulative_gain_curve, and effect_curves, a user could modify how these effects are computed, whether to do them ascending or descending by the prediction column.

Description of the changes proposed in the pull request

A model could output a prediction that is not necessarily positively related to the effect to be computed, so adding an option to order this relationship differently allows for effects and gains with negatively related predictions and outcomes to be computed adequately.

The changes are applied to curves.py and also on auc.py on the causal-effect module.

Where should the reviewer start?

Reviewing causal-effect/curves as there are the definition of the functions from which all ordering behavior is propagated.

@MarianaBlaz MarianaBlaz requested a review from a team as a code owner December 20, 2022 15:43
@MarianaBlaz MarianaBlaz added the review-request Waiting to be reviewed label Jan 17, 2023
Copy link

@raphael-bruce raphael-bruce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
review-request Waiting to be reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add an option to order by ascending/descending prediction in cumulative effect curves
3 participants