Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fixing Test-Cases] Pandas/Numpy use different stdev estimators (mean_std_column) #124

Open
ian-coccimiglio opened this issue Sep 9, 2024 · 2 comments · Fixed by #126
Open

Comments

@ian-coccimiglio
Copy link
Contributor

ian-coccimiglio commented Sep 9, 2024

Pandas and numpy use different approximations for standard deviation. As such, this explains why this test gets such a low score. Related to #76 and one of the not-improving tests in #118

I think we should either specify the desired estimator in the prompt (unbiased or biased) or make the test flexible enough to accept either answer (the majority of the models use the builtin pandas stdev, but we only accept the numpy estimator).

This is our reference test function

def mean_std_column(dataframe, column:str):
    """
    Computes the mean average and standard deviation of a specified column 
    in a given dataframe and returns these two values.
    """
    import numpy as np
    data = dataframe[column]
    return np.mean(data), np.std(data)

And this is what most models provide:

import pandas as pd

def mean_std_column(dataframe, column:str):
    """
    Computes the mean average and standard deviation of a specified column 
    in a given dataframe and returns these two values.
    """
    mean_value = dataframe[column].mean()
    std_value = dataframe[column].std()
    return (mean_value, std_value)
@haesleinhuepf
Copy link
Owner

make the test flexible enough to accept either answer (the majority of the models use the builtin pandas stdev, but we only accept the numpy estimator).

I vote for flexibility. Both solutions should be detected as correct.

@ian-coccimiglio
Copy link
Contributor Author

Done! I put in a PR for this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants