From 89a634ec89c2ab5ab720a87493e143e076373403 Mon Sep 17 00:00:00 2001 From: Daniel Date: Tue, 3 Sep 2024 17:42:00 +0200 Subject: [PATCH] refs in readme --- README.Rmd | 8 ++++---- README.md | 29 ++++++++++++++--------------- paper/paper.bib | 27 +++++++++++++-------------- 3 files changed, 31 insertions(+), 33 deletions(-) diff --git a/README.Rmd b/README.Rmd index efa2ee881..6cde9fb81 100644 --- a/README.Rmd +++ b/README.Rmd @@ -142,9 +142,9 @@ lm(disp ~ ., data = mtcars) |> ## Statistical inference - how to quantify evidence -There is no standardized approach to drawing conclusions based on the available data and statistical models. A frequently chosen but also much criticized approach is to evaluate results based on their statistical significance ([@amrhein_earth_2017]). +There is no standardized approach to drawing conclusions based on the available data and statistical models. A frequently chosen but also much criticized approach is to evaluate results based on their statistical significance [@amrhein_earth_2017]. -A more sophisticated way would be to test whether estimated effects exceed the "smallest effect size of interest", to avoid even the smallest effects being considered relevant simply because they are statistically significant, but clinically or practically irrelevant [@lakens2020equivalence;@lakens_improving_2022]. A rather unconventional approach, which is nevertheless advocated by various authors, is to interpret results from classical regression models in terms of probabilities, similar to the usual approach in Bayesian statistics ([@greenland_aid_2022;@rafi_semantic_2020;@schweder_confidence_2018;@schweder_frequentist_2003;@vos_frequentist_2022]). +A more sophisticated way would be to test whether estimated effects exceed the "smallest effect size of interest", to avoid even the smallest effects being considered relevant simply because they are statistically significant, but clinically or practically irrelevant [@lakens2020equivalence;@lakens_improving_2022]. A rather unconventional approach, which is nevertheless advocated by various authors, is to interpret results from classical regression models in terms of probabilities, similar to the usual approach in Bayesian statistics [@greenland_aid_2022;@rafi_semantic_2020;@schweder_confidence_2018;@schweder_frequentist_2003;@vos_frequentist_2022]. The _parameters_ package provides several options or functions to aid statistical inference. These are, for example: @@ -152,10 +152,10 @@ The _parameters_ package provides several options or functions to aid statistica - [`p_significance()`](https://easystats.github.io/parameters/reference/p_significance.lm.html), to compute the probability of *practical significance*, which can be conceptualized as a unidirectional equivalence test - [`p_function()`](https://easystats.github.io/parameters/reference/p_function.html), or _consonance function_, to compute p-values and compatibility (confidence) intervals for statistical models - the `pd` argument (setting `pd = TRUE`) in `model_parameters()` includes a column with the *probability of direction*, i.e. the probability that a parameter is strictly positive or negative. See [`bayestestR::p_direction()`](https://easystats.github.io/bayestestR/reference/p_direction.html) for details. -- the `s_value` argument (setting `s_value = TRUE`) in `model_parameters()` replaces the p-values with their related _S_-values (*Rafi and Greenland 2020*) +- the `s_value` argument (setting `s_value = TRUE`) in `model_parameters()` replaces the p-values with their related _S_-values [@@rafi_semantic_2020] - finally, it is possible to generate distributions of model coefficients by generating bootstrap-samples (setting `bootstrap = TRUE`) or simulating draws from model coefficients using [`simulate_model()`](https://easystats.github.io/parameters/reference/simulate_model.html). These samples can then be treated as "posterior samples" and used in many functions from the **bayestestR** package. -Most of the above shown options or functions derive from methods originally implemented for Bayesian models ([@makowski2019bayetestR]). However, assuming that model assumptions are met (which means, the model fits well to the data, the correct model is chosen that reflects the data generating process (distributional model family) etc.), it seems appropriate to interpret results from classical frequentist models in a "Bayesian way" (more details: documentation in [`p_function()`](https://easystats.github.io/parameters/reference/p_function.html)). +Most of the above shown options or functions derive from methods originally implemented for Bayesian models [@makowski_indices_2019]. However, assuming that model assumptions are met (which means, the model fits well to the data, the correct model is chosen that reflects the data generating process (distributional model family) etc.), it seems appropriate to interpret results from classical frequentist models in a "Bayesian way" (more details: documentation in [`p_function()`](https://easystats.github.io/parameters/reference/p_function.html)). ## Citation diff --git a/README.md b/README.md index 96aec0f2c..9e2ca989e 100644 --- a/README.md +++ b/README.md @@ -235,7 +235,7 @@ lm(disp ~ ., data = mtcars) |> There is no standardized approach to drawing conclusions based on the available data and statistical models. A frequently chosen but also much criticized approach is to evaluate results based on their statistical -significance ((Amrhein, Korner-Nievergelt, & Roth, 2017)). +significance (Amrhein, Korner-Nievergelt, & Roth, 2017). A more sophisticated way would be to test whether estimated effects exceed the “smallest effect size of interest”, to avoid even the @@ -245,8 +245,8 @@ statistically significant, but clinically or practically irrelevant approach, which is nevertheless advocated by various authors, is to interpret results from classical regression models in terms of probabilities, similar to the usual approach in Bayesian statistics -((Greenland, Rafi, Matthews, & Higgs, 2022; Rafi & Greenland, 2020; -Schweder, 2018; Schweder & Hjort, 2003; Vos & Holbert, 2022)). +(Greenland, Rafi, Matthews, & Higgs, 2022; Rafi & Greenland, 2020; +Schweder, 2018; Schweder & Hjort, 2003; Vos & Holbert, 2022). The *parameters* package provides several options or functions to aid statistical inference. These are, for example: @@ -266,7 +266,7 @@ statistical inference. These are, for example: for details. - the `s_value` argument (setting `s_value = TRUE`) in `model_parameters()` replaces the p-values with their related - *S*-values (*Rafi and Greenland 2020*) + *S*-values (@ Rafi & Greenland, 2020) - finally, it is possible to generate distributions of model coefficients by generating bootstrap-samples (setting `bootstrap = TRUE`) or simulating draws from model coefficients using @@ -275,11 +275,11 @@ statistical inference. These are, for example: many functions from the **bayestestR** package. Most of the above shown options or functions derive from methods -originally implemented for Bayesian models ((Makowski, Ben-Shachar, & -Lüdecke, 2019)). However, assuming that model assumptions are met (which -means, the model fits well to the data, the correct model is chosen that -reflects the data generating process (distributional model family) -etc.), it seems appropriate to interpret results from classical +originally implemented for Bayesian models (Makowski, Ben-Shachar, Chen, +& Lüdecke, 2019). However, assuming that model assumptions are met +(which means, the model fits well to the data, the correct model is +chosen that reflects the data generating process (distributional model +family) etc.), it seems appropriate to interpret results from classical frequentist models in a “Bayesian way” (more details: documentation in [`p_function()`](https://easystats.github.io/parameters/reference/p_function.html)). @@ -354,13 +354,12 @@ Practices in Psychological Science*, *1*(2), 259–269. -
+
-Makowski, D., Ben-Shachar, M., & Lüdecke, D. (2019). -bayestestR: Describing effects and their -uncertainty, existence and significance within the bayesian framework. -*Journal of Open Source Software*, *4*(40), 1541. - +Makowski, D., Ben-Shachar, M. S., Chen, S. H. A., & Lüdecke, D. (2019). +Indices of Effect Existence and Significance in the Bayesian Framework. +*Frontiers in Psychology*, *10*, 2767. +
diff --git a/paper/paper.bib b/paper/paper.bib index c8f61d639..e9a408f58 100644 --- a/paper/paper.bib +++ b/paper/paper.bib @@ -10,8 +10,6 @@ @Article{makowski2019bayetestR journal = {Journal of Open Source Software} } - - @Article{ludecke2019insight, title = {{insight}: A Unified Interface to Access Information from Model Objects in {R}}, volume = {4}, @@ -23,7 +21,6 @@ @Article{ludecke2019insight pages = {1412} } - @Manual{rcore, title = {{R}: A Language and Environment for Statistical Computing}, author = {{R Core Team}}, @@ -33,7 +30,6 @@ @Manual{rcore url = {https://www.R-project.org/} } - @Article{zeileis2006, title = {Object-Oriented Computation of Sandwich Estimators}, author = {Achim Zeileis}, @@ -45,7 +41,6 @@ @Article{zeileis2006 doi = {10.18637/jss.v016.i09} } - @Manual{pustejovsky2020, title = {{clubSandwich}: Cluster-Robust (Sandwich) Variance Estimators with Small-Sample Corrections}, @@ -55,8 +50,6 @@ @Manual{pustejovsky2020 url = {https://CRAN.R-project.org/package=clubSandwich} } - - @article{lakens2020equivalence, title = {Equivalence Testing for Psychological Research: A Tutorial}, volume = {1}, @@ -72,7 +65,6 @@ @article{lakens2020equivalence pages = {259--269} } - @Article{luedecke2020performance, title = {{performance}: Assessment of Regression Models Performance}, author = {Daniel Lüdecke and Dominique Makowski and Philip Waggoner and Indrajeet Patil}, @@ -83,7 +75,6 @@ @Article{luedecke2020performance url = {https://easystats.github.io/performance} } - @Manual{robinson_broom_2020, title = {{broom}: Convert Statistical Objects into Tidy Tibbles}, author = {David Robinson and Alex Hayes and Simon Couch}, @@ -92,7 +83,6 @@ @Manual{robinson_broom_2020 url = {https://CRAN.R-project.org/package=broom} } - @Manual{hlavac_stargazer_2018, title = {{stargazer}: Well-Formatted Regression and Summary Statistics Tables}, author = {Marek Hlavac}, @@ -103,7 +93,6 @@ @Manual{hlavac_stargazer_2018 url = {https://CRAN.R-project.org/package=stargazer}, } - @Manual{harrison2020finalfit, title = {{finalfit}: Quickly Create Elegant Regression Results Tables and Plots when Modeling}, @@ -113,7 +102,6 @@ @Manual{harrison2020finalfit url = {https://CRAN.R-project.org/package=finalfit}, } - @Article{ludecke2020see, title = {{see}: Visualisation Toolbox for 'easystats' and Extra Geoms, Themes and Color Palettes for 'ggplot2'}, author = {Daniel Lüdecke and Mattan S. Ben-Shachar and Philip Waggoner and Dominique Makowski}, @@ -124,7 +112,6 @@ @Article{ludecke2020see url = {https://easystats.github.io/see} } - @Manual{revelle_psych_2019, title = {{psych}: Procedures for Psychological, Psychometric, and Personality Research}, author = {William Revelle}, @@ -135,7 +122,6 @@ @Manual{revelle_psych_2019 url = {https://CRAN.R-project.org/package=psych} } - @Article{saefken_caic4_2018, title = {Conditional Model Selection in Mixed-Effects Models with {cAIC4}}, author = {Benjamin Saefken and David Ruegamer and Thomas Kneib and Sonja Greven}, @@ -259,3 +245,16 @@ @article{vos_frequentist_2022 year = {2022}, pages = {89} } + +@article{makowski_indices_2019, + title = {Indices of {Effect} {Existence} and {Significance} in the {Bayesian} {Framework}}, + volume = {10}, + issn = {1664-1078}, + url = {https://www.frontiersin.org/article/10.3389/fpsyg.2019.02767}, + doi = {10.3389/fpsyg.2019.02767}, + abstract = {Turmoil has engulfed psychological science. Causes and consequences of the reproducibility crisis are in dispute. With the hope of addressing some of its aspects, Bayesian methods are gaining increasing attention in psychological science. Some of their advantages, as opposed to the frequentist framework, are the ability to describe parameters in probabilistic terms and explicitly incorporate prior knowledge about them into the model. These issues are crucial in particular regarding the current debate about statistical significance. Bayesian methods are not necessarily the only remedy against incorrect interpretations or wrong conclusions, but there is an increasing agreement that they are one of the keys to avoid such fallacies. Nevertheless, its flexible nature is its power and weakness, for there is no agreement about what indices of “significance” should be computed or reported. This lack of a consensual index or guidelines, such as the frequentist p-value, further contributes to the unnecessary opacity that many non-familiar readers perceive in Bayesian statistics. Thus, this study describes and compares several Bayesian indices, provide intuitive visual representation of their “behavior” in relationship with common sources of variance such as sample size, magnitude of effects and also frequentist significance. The results contribute to the development of an intuitive understanding of the values that researchers report, allowing to draw sensible recommendations for Bayesian statistics description, critical for the standardization of scientific reporting.}, + journal = {Frontiers in Psychology}, + author = {Makowski, Dominique and Ben-Shachar, Mattan S. and Chen, S. H. Annabel and Lüdecke, Daniel}, + year = {2019}, + pages = {2767} +}