Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow extra argument type=1, 2 or 3 for eta_squared function & change default to type=2 #526

Open
tomwenseleers opened this issue Oct 24, 2022 · 8 comments
Assignees
Labels
enhancement 🔥 New feature or request
Milestone

Comments

@tomwenseleers
Copy link

Describe the solution you'd like
The current eta_squared function appears to return partial eta squared values based on type I SS. It is generally reckoned those are not very useful for factorial designs or models with interaction effects. It also leads to confusion, as the partial eta squared values of different packages have different defaults & options in this respect, sometimes using type I SS and sometimes type II or III, https://stats.stackexchange.com/questions/552451/which-r-functions-are-correct-for-estimating-partial-eta-squared-for-effects-in. At the very least an extra argument type=XX would seem to be in order, as in the DescTools::EtaSq function.

How could we do it?
To get partial eta squared values using type I, II or III SS I figured that with the effectsize::eta_squared function one would have to do
effectsize::eta_squared(fit) or equivalently effectsize::eta_squared(anova(fit)) # using type I tests
effectsize::eta_squared(car::Anova(fit, type=2)) # using type II tests
effectsize::eta_squared(car::Anova(fit, type=3)) # using type III tests, requires using effect coding / sum contrasts, cf afex::set_sum_contrasts() and all continuous covariates need to be centered using scale()

It would be nice if these could be obtained using effectsize::eta_squared(fit, type=1, 2 or 3)

@mattansb mattansb self-assigned this Oct 27, 2022
@mattansb mattansb added the enhancement 🔥 New feature or request label Oct 27, 2022
@mattansb
Copy link
Member

This is definitely something I want to implement - I would have to see how to consistently do this across model types.

Thanks for the suggestion.

@mattansb
Copy link
Member

mattansb commented Nov 6, 2022

This will require a re-write of a lot of code.
Essentially we would need to have (internally) two types of processing functions:

  1. Those that deal with ANOVA tables as inputs. These DON'T not get a "type" argument, since that is a characteristic of the table.
  2. Those the deal with models as inputs. These DO get a "type" argument, and pass the resulting ANOVA table to the functions that deal with ANOVA tables as inputs (1).

For each model class, the "type" argument would have to be checked against what is possible for that model, and which packages are available. For example:

  • For aov, type 1 is default, while types 2/3 require car::Anova
  • For aovlist only type 1 is supported
  • For merMod models, type 1 is from lme4:::anova.merMod while types 2/3 are from lmerTest:::anova.merModLmerTest
    • (For merModLmerTest models, type 1 is not possible)
  • Etc...

@strengejacke I know some of this is available in parameters - do you know what might be missing to that effectsize can completely rely on it?

@strengejacke
Copy link
Member

You mean in terms of detecting the type?

@mattansb
Copy link
Member

mattansb commented Nov 7, 2022

I think the type detection is fully implemented in parameters for ANOVA tables.

For aov / maov, what type is returned by parameters by default? Can I request all types?
For Anova.mlm what is the default - univariate of multivariate tests?

@tomwenseleers
Copy link
Author

tomwenseleers commented Nov 15, 2022

If it would be too much hassle to rework large parts of the function, maybe just a little note in the help could be good saying that if you would like effect sizes for Anova tables (of whatever fit supported by https://rdrr.io/cran/car/man/Anova.html, i.e. lm, glm, multinom, polr, coxph, coxme, svyglm, svycoxph, rlm, lmer or lme), effectsize::eta_squared(car::Anova(fit, type=3)) is probably what you want (provided you used sum/effect coding using afex::set_sum_contrasts() and that all continuous covariates were centered first using scale())?

@mattansb
Copy link
Member

Well, we already have that:

  1. In the function help page - an explanation about type 2/3 under "Type of Sums of Squares" section, and in the examples it says "Recommended: Type-2 or -3 effect sizes + effects coding", both with examples of use of car::Anova.
  2. The ANOVA vignette goes into great detail about using car::Anova or afex

@tomwenseleers
Copy link
Author

tomwenseleers commented Dec 29, 2022

Thanks - I only just noticed those bits now. I guess it comes down to it being a bit unfortunate that the R folks only implemented type I SSs in their default anova() method, which you then use for most models "For all other model, effect sizes are approximated via test statistic conversion of the omnibus F statistic provided by the appropriate anova() method". Whereas it's Anova(type="III") results one would typically like to see for any model. But as long as this caveat is pointed out somewhere & how the proper effect sizes using type III SSs can be obtained I'm fine with that... I just thought an extra argument type=1, 2 or 3 could make this a bit more explicit... (As this is also the way this is done in some other effect size packages)

@mattansb
Copy link
Member

mattansb commented Dec 7, 2024

Coming back to this - perhaps we can give a "warning" when type 1 SSs are used (or when the type could not be detected). This will be (1) easier to code (🙃) but also (2) will not enforce any choice (2 vs 3) on our part on the user.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement 🔥 New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants