Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

472: fix TMB option warning #473

Merged
merged 10 commits into from
Sep 27, 2024
1 change: 0 additions & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,6 @@ Imports:
parallel,
Rcpp,
Rdpack,
rlang,
stats,
stringr,
tibble,
Expand Down
6 changes: 6 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

### Bug Fixes

- Previously, when the tape optimizer from `TMB` was switched on a warning would be given, instructing to turn off the tape optimizer. However, this is not necessary for reproducible results. Instead now it is checked whether the deterministic hash for the TMB tape optimizer is used, and a warning is issued otherwise.
danielinteractive marked this conversation as resolved.
Show resolved Hide resolved

# mmrm 0.3.13

### Bug Fixes

- When running with `TMB` package versions below 1.9.15, MMRM fit results are not completely reproducible. While this may not be relevant for most applications, because the numerical differences are very small, we now issue a warning to the user if this is the case. We advise users to upgrade their `TMB` package versions to 1.9.15 or higher to ensure reproducibility.
- Previously, `mmrm` ignored contrasts defined for covariates in the input data set. This is fixed now.
- Previously, `predict` always required the response to be valid, even for unconditional predictions. This is fixed now and unconditional prediction does not require the response to be valid or present any longer.
Expand Down
1 change: 1 addition & 0 deletions R/fit.R
Original file line number Diff line number Diff line change
Expand Up @@ -422,6 +422,7 @@ mmrm <- function(formula,
h_valid_formula(formula)
covariance <- h_reconcile_cov_struct(formula, covariance)
formula_parts <- h_mmrm_tmb_formula_parts(formula, covariance)
h_tmb_warn_non_deterministic()

if (!missing(data)) {
attr(data, which = "dataname") <- toString(match.call()$data)
Expand Down
2 changes: 0 additions & 2 deletions R/tmb.R
Original file line number Diff line number Diff line change
Expand Up @@ -537,8 +537,6 @@ fit_mmrm <- function(formula,
}
tmb_parameters <- h_mmrm_tmb_parameters(formula_parts, tmb_data, start = control$start, n_groups = tmb_data$n_groups)

h_tmb_warn_optimization()

tmb_object <- TMB::MakeADFun(
data = tmb_data,
parameters = tmb_parameters,
Expand Down
23 changes: 14 additions & 9 deletions R/utils.R
Original file line number Diff line number Diff line change
Expand Up @@ -524,22 +524,27 @@ h_drop_levels <- function(data, subject_var, visit_var, except) {
data
}

#' Warn if TMB is Configured to Optimize Instantly
#' Warn if TMB is Configured to Use Non-Deterministic Hash for Tape Optimizer
#'
#' This function checks the TMB configuration for the `optimize.instantly` setting.
#' If it is set to `TRUE`, a warning is issued indicating that this may lead to
#' This function checks the TMB configuration for the `tmbad_deterministic_hash` setting
#' If it is set to `FALSE`, a warning is issued indicating that this may lead to
#' unreproducible results.
#'
#' @return No return value, called for side effects.
#' @keywords internal
h_tmb_warn_optimization <- function() {
tmb_config <- TMB::config("optimize.instantly", DLL = "mmrm")
if (tmb_config$optimize.instantly) {
h_tmb_warn_non_deterministic <- function() {
if (packageVersion("TMB") < "1.9.15") {
return()
}
tmb_config <- TMB::config(DLL = "mmrm")
tape_deterministic <- tmb_config$tmbad_deterministic_hash
if (!tape_deterministic) {
msg <- paste(
"TMB is configured to optimize instantly, this may lead to unreproducible results.",
"To disable this behavior, use `TMB::config(optimize.instantly = 0)`.",
"TMB is configured to use a non-deterministic hash for its tape optimizer,",
"and this may lead to unreproducible results.",
"To disable this behavior, use `TMB::config(tmbad_deterministic_hash = 1)`.",
sep = "\n"
)
rlang::warn(msg, .frequency = "once", .frequency_id = "tmb_warn_optimization")
warning(msg)
}
}
17 changes: 17 additions & 0 deletions man/h_tmb_warn_non_deterministic.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 0 additions & 17 deletions man/h_tmb_warn_optimization.Rd

This file was deleted.

4 changes: 2 additions & 2 deletions tests/testthat/helper-examples.R
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# We need to disable instant optimization to run tests.
TMB::config(optimize.instantly = 0, DLL = "mmrm")
# We need to make sure to have deterministic hash to run tests.
TMB::config(tmbad_deterministic_hash = 1, DLL = "mmrm")

.tmb_formula <- FEV1 ~ RACE + us(AVISIT | USUBJID)
.mmrm_tmb_example <- fit_mmrm(.tmb_formula, fev_data, weights = rep(1, nrow(fev_data)))
Expand Down
18 changes: 15 additions & 3 deletions tests/testthat/test-fit.R
Original file line number Diff line number Diff line change
Expand Up @@ -402,9 +402,9 @@ test_that("mmrm works if formula contains variables not in data", {

test_that("mmrm works for specific small data example", {
small_dat <- data.frame(
FEV1 = c(1, 2, 3, 4, 5, 6),
AVISIT = factor(c("V1", "V1", "V2", "V3", "V3", "V4")),
USUBJID = c("A", "B", "A", "C", "D", "A")
FEV1 = c(1, 2, 3, 4, 5, 6, 7, 8),
AVISIT = factor(c("V1", "V1", "V2", "V3", "V3", "V4", "V4", "V4")),
USUBJID = c("A", "B", "A", "C", "D", "A", "B", "C")
)
vars <- list(
response = "FEV1",
Expand Down Expand Up @@ -933,3 +933,15 @@ test_that("mmrm fails for vcov: Jackknife and method: Kenward-Roger", {
"Kenward-Roger degrees of freedom must work together with Kenward-Roger or Kenward-Roger-Linear covariance!"
)
})

test_that("mmrm gives warning if not reproducible TMB option is used", {
TMB::config(tmbad_deterministic_hash = 0, DLL = "mmrm")
expect_warning(
mmrm(
formula = FEV1 ~ ARMCD + ar1(AVISIT | USUBJID),
data = fev_data
),
"TMB is configured to use a non-deterministic hash"
)
TMB::config(tmbad_deterministic_hash = 1, DLL = "mmrm")
})
15 changes: 8 additions & 7 deletions tests/testthat/test-utils.R
Original file line number Diff line number Diff line change
Expand Up @@ -323,16 +323,17 @@ test_that("h_drop_levels works as expected", {
)
})

# h_tmb_warn_optimization ----
# h_tmb_warn_non_deterministic ----

test_that("h_tmb_warn_optimization works as expected", {
TMB::config(optimize.instantly = 1, DLL = "mmrm")
test_that("h_tmb_warn_non_deterministic works as expected", {
skip_if(packageVersion("TMB") < "1.9.15")
TMB::config(tmbad_deterministic_hash = 0, DLL = "mmrm")
expect_warning(
h_tmb_warn_optimization(),
"TMB is configured to optimize instantly"
h_tmb_warn_non_deterministic(),
"TMB is configured to use a non-deterministic hash"
)
TMB::config(optimize.instantly = 0, DLL = "mmrm")
expect_silent(h_tmb_warn_optimization())
TMB::config(tmbad_deterministic_hash = 1, DLL = "mmrm")
expect_silent(h_tmb_warn_non_deterministic())
})

# h_get_na_action ----
Expand Down
Loading