Skip to content

Commit

Permalink
refs in readme
Browse files Browse the repository at this point in the history
  • Loading branch information
strengejacke committed Sep 3, 2024
1 parent 54a4360 commit 89a634e
Show file tree
Hide file tree
Showing 3 changed files with 31 additions and 33 deletions.
8 changes: 4 additions & 4 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -142,20 +142,20 @@ lm(disp ~ ., data = mtcars) |>


## Statistical inference - how to quantify evidence
There is no standardized approach to drawing conclusions based on the available data and statistical models. A frequently chosen but also much criticized approach is to evaluate results based on their statistical significance ([@amrhein_earth_2017]).
There is no standardized approach to drawing conclusions based on the available data and statistical models. A frequently chosen but also much criticized approach is to evaluate results based on their statistical significance [@amrhein_earth_2017].

A more sophisticated way would be to test whether estimated effects exceed the "smallest effect size of interest", to avoid even the smallest effects being considered relevant simply because they are statistically significant, but clinically or practically irrelevant [@lakens2020equivalence;@lakens_improving_2022]. A rather unconventional approach, which is nevertheless advocated by various authors, is to interpret results from classical regression models in terms of probabilities, similar to the usual approach in Bayesian statistics ([@greenland_aid_2022;@rafi_semantic_2020;@schweder_confidence_2018;@schweder_frequentist_2003;@vos_frequentist_2022]).
A more sophisticated way would be to test whether estimated effects exceed the "smallest effect size of interest", to avoid even the smallest effects being considered relevant simply because they are statistically significant, but clinically or practically irrelevant [@lakens2020equivalence;@lakens_improving_2022]. A rather unconventional approach, which is nevertheless advocated by various authors, is to interpret results from classical regression models in terms of probabilities, similar to the usual approach in Bayesian statistics [@greenland_aid_2022;@rafi_semantic_2020;@schweder_confidence_2018;@schweder_frequentist_2003;@vos_frequentist_2022].

The _parameters_ package provides several options or functions to aid statistical inference. These are, for example:

- [`equivalence_test()`](https://easystats.github.io/parameters/reference/equivalence_test.lm.html), to compute the (conditional) equivalence test for frequentist models
- [`p_significance()`](https://easystats.github.io/parameters/reference/p_significance.lm.html), to compute the probability of *practical significance*, which can be conceptualized as a unidirectional equivalence test
- [`p_function()`](https://easystats.github.io/parameters/reference/p_function.html), or _consonance function_, to compute p-values and compatibility (confidence) intervals for statistical models
- the `pd` argument (setting `pd = TRUE`) in `model_parameters()` includes a column with the *probability of direction*, i.e. the probability that a parameter is strictly positive or negative. See [`bayestestR::p_direction()`](https://easystats.github.io/bayestestR/reference/p_direction.html) for details.
- the `s_value` argument (setting `s_value = TRUE`) in `model_parameters()` replaces the p-values with their related _S_-values (*Rafi and Greenland 2020*)
- the `s_value` argument (setting `s_value = TRUE`) in `model_parameters()` replaces the p-values with their related _S_-values [@@rafi_semantic_2020]
- finally, it is possible to generate distributions of model coefficients by generating bootstrap-samples (setting `bootstrap = TRUE`) or simulating draws from model coefficients using [`simulate_model()`](https://easystats.github.io/parameters/reference/simulate_model.html). These samples can then be treated as "posterior samples" and used in many functions from the **bayestestR** package.

Most of the above shown options or functions derive from methods originally implemented for Bayesian models ([@makowski2019bayetestR]). However, assuming that model assumptions are met (which means, the model fits well to the data, the correct model is chosen that reflects the data generating process (distributional model family) etc.), it seems appropriate to interpret results from classical frequentist models in a "Bayesian way" (more details: documentation in [`p_function()`](https://easystats.github.io/parameters/reference/p_function.html)).
Most of the above shown options or functions derive from methods originally implemented for Bayesian models [@makowski_indices_2019]. However, assuming that model assumptions are met (which means, the model fits well to the data, the correct model is chosen that reflects the data generating process (distributional model family) etc.), it seems appropriate to interpret results from classical frequentist models in a "Bayesian way" (more details: documentation in [`p_function()`](https://easystats.github.io/parameters/reference/p_function.html)).

## Citation

Expand Down
29 changes: 14 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,7 @@ lm(disp ~ ., data = mtcars) |>
There is no standardized approach to drawing conclusions based on the
available data and statistical models. A frequently chosen but also much
criticized approach is to evaluate results based on their statistical
significance ((Amrhein, Korner-Nievergelt, & Roth, 2017)).
significance (Amrhein, Korner-Nievergelt, & Roth, 2017).

A more sophisticated way would be to test whether estimated effects
exceed the “smallest effect size of interest”, to avoid even the
Expand All @@ -245,8 +245,8 @@ statistically significant, but clinically or practically irrelevant
approach, which is nevertheless advocated by various authors, is to
interpret results from classical regression models in terms of
probabilities, similar to the usual approach in Bayesian statistics
((Greenland, Rafi, Matthews, & Higgs, 2022; Rafi & Greenland, 2020;
Schweder, 2018; Schweder & Hjort, 2003; Vos & Holbert, 2022)).
(Greenland, Rafi, Matthews, & Higgs, 2022; Rafi & Greenland, 2020;
Schweder, 2018; Schweder & Hjort, 2003; Vos & Holbert, 2022).

The *parameters* package provides several options or functions to aid
statistical inference. These are, for example:
Expand All @@ -266,7 +266,7 @@ statistical inference. These are, for example:
for details.
- the `s_value` argument (setting `s_value = TRUE`) in
`model_parameters()` replaces the p-values with their related
*S*-values (*Rafi and Greenland 2020*)
*S*-values (@ Rafi & Greenland, 2020)
- finally, it is possible to generate distributions of model
coefficients by generating bootstrap-samples (setting
`bootstrap = TRUE`) or simulating draws from model coefficients using
Expand All @@ -275,11 +275,11 @@ statistical inference. These are, for example:
many functions from the **bayestestR** package.

Most of the above shown options or functions derive from methods
originally implemented for Bayesian models ((Makowski, Ben-Shachar, &
Lüdecke, 2019)). However, assuming that model assumptions are met (which
means, the model fits well to the data, the correct model is chosen that
reflects the data generating process (distributional model family)
etc.), it seems appropriate to interpret results from classical
originally implemented for Bayesian models (Makowski, Ben-Shachar, Chen,
& Lüdecke, 2019). However, assuming that model assumptions are met
(which means, the model fits well to the data, the correct model is
chosen that reflects the data generating process (distributional model
family) etc.), it seems appropriate to interpret results from classical
frequentist models in a “Bayesian way” (more details: documentation in
[`p_function()`](https://easystats.github.io/parameters/reference/p_function.html)).

Expand Down Expand Up @@ -354,13 +354,12 @@ Practices in Psychological Science*, *1*(2), 259–269.

</div>

<div id="ref-makowski2019bayetestR" class="csl-entry">
<div id="ref-makowski_indices_2019" class="csl-entry">

Makowski, D., Ben-Shachar, M., & Lüdecke, D. (2019).
<span class="nocase">bayestestR</span>: Describing effects and their
uncertainty, existence and significance within the bayesian framework.
*Journal of Open Source Software*, *4*(40), 1541.
<https://doi.org/10.21105/joss.01541>
Makowski, D., Ben-Shachar, M. S., Chen, S. H. A., & Lüdecke, D. (2019).
Indices of Effect Existence and Significance in the Bayesian Framework.
*Frontiers in Psychology*, *10*, 2767.
<https://doi.org/10.3389/fpsyg.2019.02767>

</div>

Expand Down
27 changes: 13 additions & 14 deletions paper/paper.bib
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,6 @@ @Article{makowski2019bayetestR
journal = {Journal of Open Source Software}
}



@Article{ludecke2019insight,
title = {{insight}: A Unified Interface to Access Information from Model Objects in {R}},
volume = {4},
Expand All @@ -23,7 +21,6 @@ @Article{ludecke2019insight
pages = {1412}
}


@Manual{rcore,
title = {{R}: A Language and Environment for Statistical Computing},
author = {{R Core Team}},
Expand All @@ -33,7 +30,6 @@ @Manual{rcore
url = {https://www.R-project.org/}
}


@Article{zeileis2006,
title = {Object-Oriented Computation of Sandwich Estimators},
author = {Achim Zeileis},
Expand All @@ -45,7 +41,6 @@ @Article{zeileis2006
doi = {10.18637/jss.v016.i09}
}


@Manual{pustejovsky2020,
title = {{clubSandwich}: Cluster-Robust (Sandwich) Variance Estimators with Small-Sample
Corrections},
Expand All @@ -55,8 +50,6 @@ @Manual{pustejovsky2020
url = {https://CRAN.R-project.org/package=clubSandwich}
}



@article{lakens2020equivalence,
title = {Equivalence Testing for Psychological Research: A Tutorial},
volume = {1},
Expand All @@ -72,7 +65,6 @@ @article{lakens2020equivalence
pages = {259--269}
}


@Article{luedecke2020performance,
title = {{performance}: Assessment of Regression Models Performance},
author = {Daniel Lüdecke and Dominique Makowski and Philip Waggoner and Indrajeet Patil},
Expand All @@ -83,7 +75,6 @@ @Article{luedecke2020performance
url = {https://easystats.github.io/performance}
}


@Manual{robinson_broom_2020,
title = {{broom}: Convert Statistical Objects into Tidy Tibbles},
author = {David Robinson and Alex Hayes and Simon Couch},
Expand All @@ -92,7 +83,6 @@ @Manual{robinson_broom_2020
url = {https://CRAN.R-project.org/package=broom}
}


@Manual{hlavac_stargazer_2018,
title = {{stargazer}: Well-Formatted Regression and Summary Statistics Tables},
author = {Marek Hlavac},
Expand All @@ -103,7 +93,6 @@ @Manual{hlavac_stargazer_2018
url = {https://CRAN.R-project.org/package=stargazer},
}


@Manual{harrison2020finalfit,
title = {{finalfit}: Quickly Create Elegant Regression Results Tables and Plots when
Modeling},
Expand All @@ -113,7 +102,6 @@ @Manual{harrison2020finalfit
url = {https://CRAN.R-project.org/package=finalfit},
}


@Article{ludecke2020see,
title = {{see}: Visualisation Toolbox for 'easystats' and Extra Geoms, Themes and Color Palettes for 'ggplot2'},
author = {Daniel Lüdecke and Mattan S. Ben-Shachar and Philip Waggoner and Dominique Makowski},
Expand All @@ -124,7 +112,6 @@ @Article{ludecke2020see
url = {https://easystats.github.io/see}
}


@Manual{revelle_psych_2019,
title = {{psych}: Procedures for Psychological, Psychometric, and Personality Research},
author = {William Revelle},
Expand All @@ -135,7 +122,6 @@ @Manual{revelle_psych_2019
url = {https://CRAN.R-project.org/package=psych}
}


@Article{saefken_caic4_2018,
title = {Conditional Model Selection in Mixed-Effects Models with {cAIC4}},
author = {Benjamin Saefken and David Ruegamer and Thomas Kneib and Sonja Greven},
Expand Down Expand Up @@ -259,3 +245,16 @@ @article{vos_frequentist_2022
year = {2022},
pages = {89}
}

@article{makowski_indices_2019,
title = {Indices of {Effect} {Existence} and {Significance} in the {Bayesian} {Framework}},
volume = {10},
issn = {1664-1078},
url = {https://www.frontiersin.org/article/10.3389/fpsyg.2019.02767},
doi = {10.3389/fpsyg.2019.02767},
abstract = {Turmoil has engulfed psychological science. Causes and consequences of the reproducibility crisis are in dispute. With the hope of addressing some of its aspects, Bayesian methods are gaining increasing attention in psychological science. Some of their advantages, as opposed to the frequentist framework, are the ability to describe parameters in probabilistic terms and explicitly incorporate prior knowledge about them into the model. These issues are crucial in particular regarding the current debate about statistical significance. Bayesian methods are not necessarily the only remedy against incorrect interpretations or wrong conclusions, but there is an increasing agreement that they are one of the keys to avoid such fallacies. Nevertheless, its flexible nature is its power and weakness, for there is no agreement about what indices of “significance” should be computed or reported. This lack of a consensual index or guidelines, such as the frequentist p-value, further contributes to the unnecessary opacity that many non-familiar readers perceive in Bayesian statistics. Thus, this study describes and compares several Bayesian indices, provide intuitive visual representation of their “behavior” in relationship with common sources of variance such as sample size, magnitude of effects and also frequentist significance. The results contribute to the development of an intuitive understanding of the values that researchers report, allowing to draw sensible recommendations for Bayesian statistics description, critical for the standardization of scientific reporting.},
journal = {Frontiers in Psychology},
author = {Makowski, Dominique and Ben-Shachar, Mattan S. and Chen, S. H. Annabel and Lüdecke, Daniel},
year = {2019},
pages = {2767}
}

0 comments on commit 89a634e

Please sign in to comment.