-
Notifications
You must be signed in to change notification settings - Fork 5
User specified cognitive data generation
For the following example, we will generate item response data for fifteen 3PL items taken by 40 test takers based on user-specified block and booklet constructions.
Table 1 indicates the item parameters for the 15 items.
| Item | Difficulty | Discrimination | Pseudo-guessing |
|---|---|---|---|
| i1 | -2 | 1.5 | 0.1 |
| i2 | -1.71 | 1.5 | 0.1 |
| i3 | -1.43 | 1.5 | 0.1 |
| i4 | -1.14 | 1.5 | 0.1 |
| i5 | -0.86 | 1.5 | 0.1 |
| i6 | -0.57 | 2 | 0.15 |
| i7 | -0.29 | 2 | 0.15 |
| i8 | 0 | 2 | 0.15 |
| i9 | 0.29 | 2 | 0.15 |
| i10 | 0.57 | 2 | 0.15 |
| i11 | 0.86 | 2.5 | 0.2 |
| i12 | 1.14 | 2.5 | 0.2 |
| i13 | 1.43 | 2.5 | 0.2 |
| i14 | 1.71 | 2.5 | 0.2 |
| i15 | 2 | 2.5 | 0.2 |
Table 2 indicates four blocks and the assignment of items to these blocks. Items 1 through 4 are assigned to block 1, items 5 though 8 are assigned to block 2, items 9 through 12 are assigned to block 3, and items 13 though 15 are assigned to block 4.
| b1 | b2 | b3 | b4 |
|---|---|---|---|
| i1 | i5 | i9 | i13 |
| i2 | i6 | i10 | i14 |
| i3 | i7 | i11 | i15 |
| i4 | i8 | i12 | 0 |
Table 3 indicates six booklets and the assignment of blocks to these booklets. Booklet 1 includes items from blocks 1, 3, and 4. Booklet 2 includes items from block 2. Booklet 3 includes items from blocks 3 and 4. Booklet 4 includes items from block 4. Booklet 5 includes items from blocks 2, 3, and 4. Booklet 6 includes items from block 1.
| B1 | B2 | B3 | B4 | B5 | B6 |
|---|---|---|---|---|---|
| b1 | b2 | b3 | b4 | b2 | b1 |
| b3 | - | b4 | - | b3 | - |
| b4 | - | - | - | b4 | - |
We will start our data generation by specifying item parameters. First, we create the item pool invloving 15 items and import their item parameters. b, a, and c indicate difficulty, discrimination, and pseudo-guessung parameters respectively.
item_pool <- as.data.frame(matrix(NA, nrow = 15, ncol = 4))
colnames(item_pool) <- c("item", "b", "a", "c")
item_pool$item <- c(1:15)
item_pool$b <- round(seq(-2, 2, length.out=15), 2)
item_pool$a <- c(rep(1.5, 5), rep(2.0, 5), rep(2.5, 5))
item_pool$c <- c(rep(0.10, 5), rep(0.15, 5), rep(0.20, 5))print(item_pool)## item b a c
## 1 1 -2.00 1.5 0.10
## 2 2 -1.71 1.5 0.10
## 3 3 -1.43 1.5 0.10
## 4 4 -1.14 1.5 0.10
## 5 5 -0.86 1.5 0.10
## 6 6 -0.57 2.0 0.15
## 7 7 -0.29 2.0 0.15
## 8 8 0.00 2.0 0.15
## 9 9 0.29 2.0 0.15
## 10 10 0.57 2.0 0.15
## 11 11 0.86 2.5 0.20
## 12 12 1.14 2.5 0.20
## 13 13 1.43 2.5 0.20
## 14 14 1.71 2.5 0.20
## 15 15 2.00 2.5 0.20
Second, we specify an indicator matrix where the number of columns equal the number of blocks and the number of rows equal the number of items in the item pool. The 1s indicate which items belong to which blocks. Given the example above, we assign items 1 through 4 to block 1, items 5 though 8 to block 2, items 9 through 12 to block 3 and items 13 though 15 to block 4.
item_block_design <- matrix(c(1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1),
ncol = 4, nrow = 15)
print(item_block_design)## [,1] [,2] [,3] [,4]
## [1,] 1 0 0 0
## [2,] 1 0 0 0
## [3,] 1 0 0 0
## [4,] 1 0 0 0
## [5,] 0 1 0 0
## [6,] 0 1 0 0
## [7,] 0 1 0 0
## [8,] 0 1 0 0
## [9,] 0 0 1 0
## [10,] 0 0 1 0
## [11,] 0 0 1 0
## [12,] 0 0 1 0
## [13,] 0 0 0 1
## [14,] 0 0 0 1
## [15,] 0 0 0 1
The block_design function facilitates the determination of the number of blocks and the assignment of items to those blocks. The n_blocks argument specifies four (K = 4) blocks. The item_block_matrix argument takes the indicator matrix of item_block design. The item_parameters argument takes the data frame of item parameters.
block_ex <- block_design(n_blocks = 4,
item_block_matrix = item_block_design,
item_parameters = item_pool)The output of block_ex is a list that contains two elements. The first element, block_assignment, is a matrix that that reflects the specified item-block design. The column names of block_assignment begin with b to indicate block while the rows begin with i to indicate item.
The second element in block_ex is a table of descriptive statistics for each block. This table indicates the number of items in each block and the average difficulty for each block. Again, notice blocks 1 through 3 each have 4 items while block 4 only has 3 items. Furthermore, the easiest block is b1 with an average difficulty of -1.570 while the most difficult block is b4 with an average difficulty of 1.713.
print(block_ex)## $block_assignment
## b1 b2 b3 b4
## i1 1 5 9 13
## i2 2 6 10 14
## i3 3 7 11 15
## i4 4 8 12 0
##
## $block_descriptives
## block length average difficulty
## b1 4 -1.570
## b2 4 -0.430
## b3 4 0.715
## b4 3 1.713
Third, in the code below, we create a matrix, block_book_design, which specifies six test booklets. The 1s indicate which blocks belong to which booklets. For example, Booklet 1 (row 1) includes items from blocks 1, 3 and 4 while booklet 6 (row 6) includes items from block 1 only.
block_book_design <- matrix(c(1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1,
0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0),
ncol = 4, byrow = TRUE)
print(block_book_design)## [,1] [,2] [,3] [,4]
## [1,] 1 0 1 1
## [2,] 0 1 0 0
## [3,] 0 0 1 1
## [4,] 0 0 0 1
## [5,] 0 1 1 1
## [6,] 1 0 0 0
The booklet_design function facilitates the assignment of blocks to those booklets. The item_block_assignment argument takes the block_assignment matrix from block_ex. The book_design argument takes the indicator matrix block_book_design.
book_ex <- booklet_design(item_block_assignment = block_ex$block_assignment,
book_design = block_book_design)The output of book_ex reflects the specified block-booklet design. B1 include the items from block 1 (1, 2, 3, 4), items from block 3 (9, 10, 11, 12), and items from block 4 (13, 14, 15). Book B2 includes the items from block 2 (5, 6, 7, 8). Book B3 includes the items from block 3 and block 4. Book B4 includes the items from block 4. Book B5 includes the items from block 2, 3 and 4. Book B6 includes the items from block 1.
print(book_ex)## B1 B2 B3 B4 B5 B6
## i1 1 5 9 13 5 1
## i2 2 6 10 14 6 2
## i3 3 7 11 15 7 3
## i4 4 8 12 0 8 4
## i5 9 0 13 0 9 0
## i6 10 0 14 0 10 0
## i7 11 0 15 0 11 0
## i8 12 0 0 0 12 0
## i9 13 0 0 0 13 0
## i10 14 0 0 0 14 0
## i11 15 0 0 0 15 0
Next, we generate examinees' latent abilities from a standard normal distribution with mean 0 and standard deviation 1. The function booklet_sample facilitates the distribution of test booklets to 40 examinees.
n_examinees <- 40
examinees_theta <- rnorm(n_examinees, 0, 1)
book_admin <- booklet_sample(n_subj = n_examinees,
book_item_design = book_ex)For example, we see that Subject 1 is assigned booklet 4.
book_admin[book_admin$subject==1,]## subject book item
## 1 1 4 13
## 2 1 4 14
## 3 1 4 15
And, Subject 2 is assigned booklet 5.
book_admin[book_admin$subject==2,]## subject book item
## 12 2 5 5
## 13 2 5 6
## 14 2 5 7
## 15 2 5 8
## 16 2 5 9
## 17 2 5 10
## 18 2 5 11
## 19 2 5 12
## 20 2 5 13
## 21 2 5 14
## 22 2 5 15
Finally, the response_gen function generates item responses. The resulting object, ex_cognitive_data, is a data frame with performance on 15 items and subject's ID. Items that are not administered are labeled as (NA).
We print the first two subjects' response data below. Subject 1 takes booklet 4 and only items 13, 14, and 15 are administered, while Subject 2 takes booklet 5 and items 5 through 15 are administered.
ex_cognitive_data <- response_gen(subject = book_admin$subject,
item = book_admin$item,
theta = examinees_theta,
a_par = item_pool$a,
b_par = item_pool$b,
c_par = item_pool$c)
print(ex_cognitive_data[1:2,])## i001 i002 i003 i004 i005 i006 i007 i008 i009 i010 i011 i012 i013 i014 i015 subject
## 1 NA NA NA NA NA NA NA NA NA NA NA NA 0 0 0 1
## 2 NA NA NA NA 0 0 1 1 0 0 0 0 0 0 0 2