16_methods-assessment.Rmd

# Methods assessment

**Learning objectives:**

- learn good practices in model evaluation

## Overview {-}

Determining predictive performance of a spatial interpolation method requires:

- a method to obtain training and testing datasets
- a choice of the predictive performance measure (applied to each testing dataset)

## Obtaining training and testing datasets {-}

- K-fold or leave-one-out (LOOCV) cross validation (see chapter 13 for example code)
- splitting is preferably done by random sampling (Wadoux et al., 2021)

## Predictive performance measures {-}

- _not_ taking into account uncertainty of predictions:
  - MAE (mean absolute error)
  - RMSE (root mean square error)
- taking into account uncertainty of predictions:
  - 95% CP (coverage probability): proportion of observations within the 95% prediction intervals
  - CRPS (Continuous Ranked Probability Score): integration on the response scale of the squared difference between the predicted and the observed (degenarate) CDF.

## Meeting Videos {-}

### Cohort 1 {-}

`r knitr::include_url("https://www.youtube.com/embed/URL")`

<details>
<summary> Meeting chat log </summary>

```
LOG
```
</details>