Typical Use of eatATA: a Minimal Example

Benjamin Becker

2021-02-04

eatATA efficiently translates test design requirements for Automated Test Assembly (ATA) into constraints for a Mixed Integer Linear Programming Model (MILP). A number of efficient and user-friendly functions are available that translate conceptual test assembly constraints to constraint objects for MILP solvers, like the GLPK solver. In the remainder of this vignette we will illustrate the use of eatATA using a minimal example. A general overview over eatATA can be found in the vignette Overview of eatATA Functionality.

Setup

The eatATA package can be installed from CRAN.

install.packages("eatATA")

Item Pool

First, eatATA is loaded into your R session. In this vignette we use a small simulated item pool, items_sim. The goal will be to assemble a single test form consisting of ten items, an average test time of eight minutes and maximum TIF at medium ability. We therefore calculate the IIF at medium ability and append it to the item pool using the calculateIFF() function.

# loading eatATA
library(eatATA)

# item pool structure
str(items_sim)
#> 'data.frame':    30 obs. of  4 variables:
#>  $ id        : int  1 2 3 4 5 6 7 8 9 10 ...
#>  $ format    : chr  "mc" "mc" "mc" "mc" ...
#>  $ mean_time : num  29.5 25 31 37.6 27.3 ...
#>  $ difficulty: num  0.0302 -0.5803 1.7906 -0.4644 -0.5265 ...

# calculate and append IIF
items_sim[, "IIF"] <- calculateIIF(B = items_sim$difficulty, theta = 0)

Objective Function

Next, the objective function is defined: The TIF should be maximized at medium ability. For this, we use the maxConstraint() function.

testInfo <- maxConstraint(nForms = 1, itemValues = items_sim$IIF,
                          itemIDs = items_sim$id)

Constraints

Our further, fixed constraints are defined as additional constraint objects.

itemNumber <- itemsPerFormConstraint(nForms = 1, operator = "=", 
                                     targetValue = 10, 
                                     itemIDs = items_sim$id)

itemUsage <- itemUsageConstraint(nForms = 1, operator = "<=", 
                                 targetValue = 1, 
                                 itemIDs = items_sim$id)

testTime <- itemValuesDeviation(nForms = 1, 
                                itemValues = items_sim$mean_time,
                                targetValue = 8 * 60, 
                                allowedDeviation = 5,
                                relative = FALSE, 
                                itemIDs = items_sim$id)

Alternatively, we could determine the appropriate test time based on the item pool using the autoItemValuesMinMax() function.

testTime2 <- autoItemValuesMinMax(nForms = 1, 
                                itemValues = items_sim$mean_time,
                                testLength = 10, 
                                allowedDeviation = 5,
                                relative = FALSE, 
                                itemIDs = items_sim$id)
#> The minimum and maximum values per test form are: min = 415.26 - max = 425.26

Solver usage

To automatically assemble the test form based on our constraints, we call the useSolver() function. In this function we define which solver should be used as back end. As a default solver, we recommend GLPK, which is automatically installed alongside this package.

solver_out <- useSolver(list(itemNumber, itemUsage, testTime, testInfo),
                        solver = "GLPK")
#> GLPK Simplex Optimizer, v4.47
#> 34 rows, 31 columns, 151 non-zeros
#>       0: obj =  0.000000000e+000  infeas = 4.850e+002 (1)
#> *    14: obj =  0.000000000e+000  infeas = 4.441e-016 (0)
#> *    34: obj =  6.719704076e+000  infeas = 0.000e+000 (0)
#> OPTIMAL SOLUTION FOUND
#> GLPK Integer Optimizer, v4.47
#> 34 rows, 31 columns, 151 non-zeros
#> 30 integer variables, all of which are binary
#> Integer optimization begins...
#> +    34: mip =     not found yet <=              +inf        (1; 0)
#> +    51: >>>>>  6.345959248e+000 <=  6.716532093e+000   5.8% (10; 0)
#> +    56: >>>>>  6.712004599e+000 <=  6.712004599e+000   0.0% (10; 7)
#> +    56: mip =  6.712004599e+000 <=     tree is empty   0.0% (0; 27)
#> INTEGER OPTIMAL SOLUTION FOUND

Inspect solution

The solution can be inspected directly via inspectSolution() or appended to the item pool via appendSolution(). Using the inspectSolution() function an additional row is created that calculates the column sums for all numeric variables.

inspectSolution(solver_out, items = items_sim, idCol = "id")
#> $form_1
#>      id format mean_time  difficulty   theta=0
#> 1     1     mc  29.54138  0.03019108 0.7220244
#> 13   13   open  38.81086 -0.33968639 0.6654634
#> 15   15   open  25.30197 -0.19227341 0.7035404
#> 16   16   open  69.92575 -0.65288754 0.5388101
#> 18   18   open  44.58367  0.04727869 0.7213344
#> 19   19   open  62.42280  0.33837743 0.6658788
#> 20   20   open  86.91120 -0.44677154 0.6275614
#> 22   22  order  40.40665  0.37084738 0.6552107
#> 24   24  order  30.83387 -0.17304691 0.7070911
#> 27   27  order  48.69160  0.18411508 0.7050898
#> Sum 175   <NA> 477.42976 -0.83385614 6.7120046
appendSolution(solver_out, items = items_sim, idCol = "id")
#>    id format mean_time  difficulty   theta=0 form_1
#> 1   1     mc 29.541377  0.03019108 0.7220244      1
#> 2   2     mc 25.040617 -0.58029174 0.5717449      0
#> 3   3     mc 30.952331  1.79059604 0.1254506      0
#> 4   4     mc 37.554676 -0.46439976 0.6206603      0
#> 5   5     mc 27.345274 -0.52650959 0.5951304      0
#> 6   6     mc 24.960790 -0.28648377 0.6812958      0
#> 7   7     mc 22.449372  0.29689958 0.6783710      0
#> 8   8     mc 21.648790 -0.87213286 0.4357964      0
#> 9   9     mc 24.044253  1.07571901 0.3445996      0
#> 10 10     mc 21.594588 -0.25230142 0.6902641      0
#> 11 11   open 82.922469  1.13700527 0.3191851      0
#> 12 12   open 85.285402 -0.76846443 0.4846056      0
#> 13 13   open 38.810862 -0.33968639 0.6654634      1
#> 14 14   open 65.312240 -1.25831226 0.2723894      0
#> 15 15   open 25.301969 -0.19227341 0.7035404      1
#> 16 16   open 69.925753 -0.65288754 0.5388101      1
#> 17 17   open 74.558120  1.03858357 0.3605237      0
#> 18 18   open 44.583666  0.04727869 0.7213344      1
#> 19 19   open 62.422802  0.33837743 0.6658788      1
#> 20 20   open 86.911204 -0.44677154 0.6275614      1
#> 21 21  order 33.284070  0.81977272 0.4603699      0
#> 22 22  order 40.406652  0.37084738 0.6552107      1
#> 23 23  order 38.419331 -1.17782688 0.3028989      0
#> 24 24  order 30.833874 -0.17304691 0.7070911      1
#> 25 25  order 36.477034 -0.92835911 0.4097768      0
#> 26 26  order  7.130145  0.96061901 0.3950889      0
#> 27 27  order 48.691605  0.18411508 0.7050898      1
#> 28 28  order 49.235210 -1.82611791 0.1187312      0
#> 29 29  order 31.675337  0.67348514 0.5292601      0
#> 30 30  order 43.453273  1.01279383 0.3717969      0