Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

temporary suggestion for testing simulate for design class #802

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
200 changes: 200 additions & 0 deletions tests/testthat/test-helpers_design.R
Original file line number Diff line number Diff line change
Expand Up @@ -205,3 +205,203 @@ test_that("h_determine_dlts returns correctly updated data object for placebo =
expect_s4_class(result, "Data")
expect_equal(result, expected_result)
})

test_that("simulate for the class design works end to end", {
emptydata <- Data(doseGrid = c(1, 3, 5, 10, 15, 20, 25, 40, 50, 80, 100))

model <- LogisticLogNormal(
mean = c(-0.85, 1),
cov =
matrix(c(1, -0.5, -0.5, 1),
nrow = 2
),
ref_dose = 56
)

myNextBest <- NextBestNCRM(
target = c(0.2, 0.35),
overdose = c(0.35, 1),
max_overdose_prob = 0.25
)

mySize1 <- CohortSizeRange(
intervals = c(0, 30),
cohort_size = c(1, 3)
)
mySize2 <- CohortSizeDLT(
intervals = c(0, 1),
cohort_size = c(1, 3)
)
mySize <- maxSize(mySize1, mySize2)

myStopping1 <- StoppingMinCohorts(nCohorts = 3)
myStopping2 <- StoppingTargetProb(
target = c(0.2, 0.35),
prob = 0.5
)
myStopping3 <- StoppingMinPatients(nPatients = 20)
myStopping <- (myStopping1 & myStopping2) | myStopping3

myIncrements <- IncrementsRelative(
intervals = c(0, 20),
increments = c(1, 0.33)
)

design <- Design(
model = model,
nextBest = myNextBest,
stopping = myStopping,
increments = myIncrements,
cohort_size = mySize,
data = emptydata,
startingDose = 3
) # check if default design constructor is enough to test simulate method

myTruth <- probFunction(model, alpha0 = 7, alpha1 = 8)

options <- McmcOptions(
burnin = 100,
step = 2,
samples = 1000,
rng_kind = "Mersenne-Twister",
rng_seed = 1234
)
time <- system.time(mySims <- simulate(design,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to time this simulation run? (not sure, because time is also not used below and would anyway differ between platforms)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fully agree - was a bit lazy from me taking that over from the examples...

args = NULL,
truth = myTruth,
nsim = 1,
seed = 819,
mcmcOptions = options,
parallel = FALSE,
derive = list(
max_mtd = max,
mean_mtd = mean,
median_mtd = median
),
))[3]

expected <-
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is too detailed / too much maintenance when numbers change etc.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for guidance! I will adopt.

Copy link
Collaborator Author

@robertadamsbayer robertadamsbayer Mar 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@danielinteractive Hi Daniel, I put the new "end-to-end" test for simulate in "test-helpers_design.R". When moving it over to "test-Design-methods.R" I realized that there are already tests (using snapshots) for simulate (and different classes) already implemented. Those use fixed rng seeds, therefore "mimicking" quite closely what you suggested not to do. I don´t know what to keep and what to get rid of there?!

image

image

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would just leave existing tests. More coverage does not hurt us

new(
"Simulations",
fit = list(structure(
list(
middle = c(
0.0278840989442296,
0.0585187075243448,
0.0861763463009416,
0.152046506082125,
0.215028653301285,
0.27436876755974,
0.329137865843746,
0.463222935947319,
0.529835624620552,
0.658215004013923,
0.709356365120325
),
lower = c(
0.000124727518807684,
0.00169457982279485,
0.00571810889381897,
0.0262840977019272,
0.0631561484944847,
0.108053895434153,
0.14935814438918,
0.254733947607829,
0.299313351022304,
0.377782206208248,
0.41692555950274
),
upper = c(
0.173968745159563,
0.236079098676,
0.285833540365002,
0.35945722660184,
0.41825266097964,
0.461892665717615,
0.517432747132171,
0.666615508629213,
0.750262264008957,
0.886377462207185,
0.928390287149569
)
),
row.names = c(NA, 11L),
class = "data.frame"
)),
stop_report = structure(
c(
TRUE, TRUE,
TRUE, TRUE, FALSE
),
dim = c(1L, 5L),
dimnames = list(
NULL,
c(
NA_character_,
NA_character_,
NA_character_,
NA_character_,
NA_character_
)
)
),
stop_reasons = list(
list(
list(
"Number of cohorts is 9 and thus reached the prespecified minimum number 3",
"Probability for target toxicity is 53 % for dose 20 and thus above the required 50 %"
),
"Number of patients is 19 and thus below the prespecified minimum number 20"
)
),
additional_stats = list(
list(
max_mtd = 70.6519400150433,
mean_mtd = 21.3728034044661,
median_mtd = 20.2435513052115
)
),
data = list(
new(
"Data",
x = c(
3, 5, 10, 20, 20, 20, 20, 25,
25, 25, 25, 25, 25, 25, 25, 25, 15, 15, 15
),
y = c(
0L, 0L,
0L, 1L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 1L, 0L, 1L, 1L, 1L, 0L,
0L, 0L
),
doseGrid = c(
1, 3, 5, 10, 15, 20, 25, 40, 50, 80,
100
),
nGrid = 11L,
xLevel = c(
2L, 3L, 4L, 6L, 6L, 6L, 6L,
7L, 7L, 7L, 7L, 7L, 7L, 7L, 7L, 7L, 5L, 5L, 5L
),
placebo = FALSE,
ID = 1:19,
cohort = c(
1L, 2L, 3L, 4L, 5L, 5L, 5L, 6L,
6L, 6L, 7L, 7L, 7L, 8L, 8L, 8L, 9L, 9L, 9L
),
nObs = 19L
)
),
doses = 20,
seed = 819L
)

expect_equal(mySims, expected)

expect_class(mySims, "Simulations")

expect_equal(any(sapply(mySims@fit[[1]], is.numeric)), TRUE) # check if all elements in mySims@fit are numeric

expect_equal(length(mySims@stop_report), 5) # check for length

expect_logical(mySims@stop_report) # check for stop_report to be logical vector
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something like this would be sufficient

})