Author: Julian Urban, Markus D. Feuchter, Vsevolod Scherrer, Moritz L. Breit, and Franzis Preckel

# A Tutorial for Mahalanobis distance matching with MAGMA.R

#### Julian Urban, Markus D. Feuchter, Vsevolod Scherrer, Moritz L. Breit, and Franzis Preckel

# Summary

This vignette complements the main MAGMA.R vignette by introducing Mahalanobis Distance Matching (MDM). Unlike Propensity Score Matching (PSM), MDM matches participants directly on multiple continuous covariates without requiring the estimation of propensity scores. Furthermore, MAGMA.R supports weighted Mahalanobis distance matching, allowing researchers to assign greater importance to selected covariates during the matching process. Importantly, Mahalanobis distance matching integrates into the MAGMA framework. Consequently, all features of MAGMA.R - including matching for up to four groups (and 2×2 designs), exact matching, and the comprehensive post-matching balance evaluation framework - are fully supported when using MDM.

# When to use MDM instead of PSM?

PSM offers several advantages. Propensity scores can incorporate covariates measured on different scale levels, can account for nonlinear relationships and interactions between covariates and - depending on the method with which Propensity Scores are estimated - cases with missing values can be kept in the data set (Ridgeway et al., 2015). However, because a propensity score summarizes information from multiple covariates into a single value, individuals with identical propensity scores may still differ in their observed covariate values (Guo et al., 2020). This limitation is often less problematic, as comparisons take place at the sample level, where the joint distribution of covariates is independent of the treatment assignment after conducting PSM (Guo et al., 2020). However, when the matched sample size is small, differences between groups may remain. As MDM directly optimizes similarity across covariates, it can provide closer matches at the individual level and may therefore be advantageous in small samples.

Despite this advantage, MDM has several limitations. First, standard MDM cannot directly handle categorical covariates or missing data. Second, its performance may decrease when covariates deviate from normality (Gu & Rosenbaum, 1993). Third, MDM can become less effective when many covariates are included, partly because all covariates contribute equally to the distance calculation (Stuart, 2010). To address this issue, weighted MDM can be used to assign greater importance to selected covariates. Nevertheless, computational demands increase with the number of covariates and are typically higher for MDM than for PSM. In summary, MDM may be advantagous when PSM fails to achieve sufficient balance, the number of (metric) covariates is not too high, and the amount of missing data on the covariates is low.

# Two-Group Example

This example illustrates MDM for two groups using the simulated data set `MAGMA_sim_data` included in this package. The grouping variable is `gifted_support`. The covariates used for MDM are `GPA_school`, `IQ_score`, and `Motivation`. To facilitate later estimations, we define a vector containing these covariates:

    covariates <- c("GPA_school",
                           "IQ_score",
                           "Motivation")

Before matching, we examine descriptive statistics and initial unbalance.

    # Estimate overall and group specific descriptive statistics and Cohen’s d
    descs_gifted_pre <- MAGMA_desc(Data = MAGMA_sim_data,
                                   group = "gifted_support",
                                   covariates = covariates,
                                   filename = "stats_gifted_pre.docx")
    descs_gifted_pre %>%
      purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
                         "No Support N", "No Support Mean", "No Support SD",
                         "Support N", "Support Mean", "Support SD",
                         "d"))
    #>                Overall N Overall Mean Overall SD No Support N No Support Mean
    #> gifted_support       800         0.38       0.49          495            0.00
    #> GPA_school           800         3.29       0.93          495            3.13
    #> IQ_score             800       100.51      15.00          495           96.32
    #> Motivation           800         3.82       0.89          495            3.81
    #>                No Support SD Support N Support Mean Support SD     d
    #> gifted_support          0.00       305         1.00       0.00  -Inf
    #> GPA_school              0.91       305         3.56       0.89 -0.48
    #> IQ_score               14.20       305       107.30      13.75 -0.78
    #> Motivation              0.90       305         3.83       0.88 -0.02

    # Estimating the four balance criteria
    unbalance_gifted <- initial_unbalance(Data = MAGMA_sim_data,
                                          group = "gifted_support",
                                          covariates = covariates,
                                          round = 3)
    #> Mean g was computed using random effects meta-analysis with metafor.
    unbalance_gifted
    #>           Pillai's Trace d-ratio Mean g adj. d-ratio
    #> Unbalance          0.129   0.333  0.427        0.331

As evident from the descriptive statistics - particulary Cohen’s d - and the initial imbalance assessment, the two groups differ with respect to the covariates. Therefore, we conduct two MDM matchings. First, a standard matching in which all cases from group one can be matched to all cases from group two. Second, an exact matching in which only cases sharing the same value on the variable `gender` can be matched (for more details on exact matching, see the main MAGMA.R vignette).

## Standard Matching

The function `MAGMA()` enables standard MDM. The argument `Data` specifies the data set, the argument `group` specifies the grouping variable as a character, and the argument `covs` defines the covariates used for MDM as a character vector. Optional arguments can be used to further customize the matching procedure. For weighted MDM, the argument `weights` assigns weights to the covariates specified under `covs`. This argument must be a numeric vector with the same length as `covs`. The argument `cores` enables parallel computation, and the argument `verbose` controls console output. Although the matching algorithm differs from PSM, the output structure of `MAGMA()` remains the same.

    # Conducting matching for gifted support
    MAGMA_sim_data_gifted <- MAGMA(Data = MAGMA_sim_data,
                                   group = "gifted_support",
                                   covs = covariates,
                                   weights = c(1, 1.5, 0.5),
                                   cores = 2,
                                   verbose = TRUE)

### Balance Estimation

The procedure for post-matching evaluation is identical to the procedure used for PSM. Balance can be estimated using the function `Balance_MAGMA()`. Subsequently, balance results for specific sample sizes can be extracted using `Balance_extract`, the balance trend across different sample sizes can be examined using `Plot_MAGMA`, and sample sizes at which each balance criterion is optimized can be displayed using `Table_MAGMA`. Finally, post-matching descriptive statistics can be computed.

    # Estimating the four balance criteria iteratively over possible sample sizes
    Balance_gifted <- Balance_MAGMA(Data = MAGMA_sim_data_gifted,
                                    group = "gifted_support",
                                    covariates = covariates,
                                    step = "step")

    # Extracting balance criteria for 100 cases per group
    Balance_100_criteria <- Balance_extract(Balance = Balance_gifted,
                                            samplesize = 100,
                                            effects = FALSE)
    Balance_100_criteria
    #> Pillai's Trace        d-ratio         mean g   adj. d-ratio
    #>           0.00           1.00           0.01           0.91

    # Extracting pairwise effects for 100 cases per group
    Balance_100_effects <- Balance_extract(Balance = Balance_gifted,
                                           samplesize = 100,
                                           effects = TRUE)
    Balance_100_effects
    #> GPA_school   IQ_score Motivation
    #>       0.02       0.01      -0.01

    # Plotting balance trend over sample size
    Plot_MAGMA(Balance = Balance_gifted,
               criterion = c("Pillai", "d_ratio", "mean_g", "Adj_d_ratio"),
               print = FALSE)
    #> $Pillai

    #>
    #> $`d-ratio`

    #>
    #> $`mean g`

    #>
    #> $`adj-d-ratio`

    # Sample Sizes with optimized balance criteria
    Table_MAGMA(Balance = Balance_gifted,
                round = 3)
    #> Balance table successfully created!
    #> # A tibble: 4 × 6
    #>   Criterion_optimized Pillai_Trace d_ratio mean_g adjusted_d_ratio n_per_group
    #>   <chr>                      <dbl>   <dbl>  <dbl>            <dbl>       <int>
    #> 1 Best Pillai                 0          1  0.002            0.947         132
    #> 2 Best mean g                 0          1  0.002            0.947         132
    #> 3 Best adj. d-ratio           0          1  0.018            0.979         255
    #> 4 Best d-ratio                0.01       1  0.099            0.803         295

    # Computing descriptive statistics and pairwise effects for 100 cases per group
    descs_gifted_post <- MAGMA_desc(Data = MAGMA_sim_data_gifted,
                                    group = "gifted_support",
                                    covariates = covariates,
                                    step_num = 255,
                                    step_var = "step")

    # Displaying the table with defined colum names
    descs_gifted_post %>%
      purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
                         "No Support N", "No Support Mean", "No Support SD",
                         "Support N", "Support Mean", "Support SD",
                         "d"))
    #>                Overall N Overall Mean Overall SD No Support N No Support Mean
    #> gifted_support       510         0.50       0.50          255            0.00
    #> GPA_school           510         3.48       0.85          255            3.47
    #> IQ_score             510       104.04      12.39          255          103.82
    #> Motivation           510         3.86       0.84          255            3.86
    #>                No Support SD Support N Support Mean Support SD     d
    #> gifted_support          0.00       255         1.00       0.00  -Inf
    #> GPA_school              0.85       255         3.48       0.86 -0.01
    #> IQ_score               12.32       255       104.27      12.48 -0.04
    #> Motivation              0.83       255         3.86       0.86  0.00

The post-matching balance and descriptive statistics show that the initial imbalance between the groups was substantially reduced. In the solution with the best adjusted d-ratio (n = 255 cases per group), Pillai’s Trace decreased from .129 to < .001, d-ratio increased from .333 to 1, mean g decreased from 0.427 to 0.018, and adjusted d-ratio increased from .331 to .979. Thus, MDM was able to identify a solution in which the distributions of the covariates were comparable between the two groups.

## Exact Matching

In exact matching, only cases sharing the same value on a variable - here `gender` can be matched. The function `MAGMA_exact()` enables standard MDM. This function has an additional argument, where the name of the exact variable must be defined as a character.

    # Conducting matching for gifted support
    MAGMA_sim_data_gifted_exact <- MAGMA_exact(Data = MAGMA_sim_data,
                                               group = "gifted_support",
                                               covs = covariates,
                                               weights = c(1, 1.5, 0.5),
                                               cores = 2,
                                               exact = "gender")

### Balance Estimation

After the exaxt matching, the same procedure as for standard matching can be applied to evaluate the balance.

    # Estimating the four balance criteria iteratively over possible sample sizes
    Balance_gifted_exact <- Balance_MAGMA(Data = MAGMA_sim_data_gifted_exact,
                                          group = "gifted_support",
                                          covariates = covariates,
                                          step = "step")

    # Extracting balance criteria for 100 cases per group
    Balance_100_criteria_exact <- Balance_extract(Balance = Balance_gifted_exact,
                                                  samplesize = 100,
                                                  effects = FALSE)
    Balance_100_criteria_exact
    #> Pillai's Trace        d-ratio         mean g   adj. d-ratio
    #>           0.00           1.00           0.01           0.91

    # Extracting pairwise effects for 100 cases per group
    Balance_100_effects_exact <- Balance_extract(Balance = Balance_gifted_exact,
                                                 samplesize = 100,
                                                 effects = TRUE)
    Balance_100_effects_exact
    #> GPA_school   IQ_score Motivation
    #>      -0.02      -0.02       0.00

    # Plotting balance trend over sample size
    Plot_MAGMA(Balance = Balance_gifted_exact,
               criterion = c("Pillai", "d_ratio", "mean_g", "Adj_d_ratio"),
               print = FALSE)
    #> $Pillai

    #>
    #> $`d-ratio`

    #>
    #> $`mean g`

    #>
    #> $`adj-d-ratio`

    # Sample Sizes with optimized balance criteria
    Table_MAGMA(Balance = Balance_gifted_exact,
                round = 3)
    #> Balance table successfully created!
    #> # A tibble: 4 × 6
    #>   Criterion_optimized Pillai_Trace d_ratio mean_g adjusted_d_ratio n_per_group
    #>   <chr>                      <dbl>   <dbl>  <dbl>            <dbl>       <int>
    #> 1 Best Pillai                 0          1  0.001            0.758          24
    #> 2 Best mean g                 0          1  0.001            0.758          24
    #> 3 Best adj. d-ratio           0          1  0.016            0.96          179
    #> 4 Best d-ratio                0.01       1  0.112            0.792         292

    # Computing descriptive statistics and pairwise effects for 100 cases per group
    descs_gifted_post_exact <- MAGMA_desc(Data = MAGMA_sim_data_gifted_exact,
                                          group = "gifted_support",
                                          covariates = covariates,
                                          step_num = 179,
                                          step_var = "step")

    # Displaying the table with defined colum names
    descs_gifted_post_exact %>%
      purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
                         "No Support N", "No Support Mean", "No Support SD",
                         "Support N", "Support Mean", "Support SD",
                         "d"))
    #>                Overall N Overall Mean Overall SD No Support N No Support Mean
    #> gifted_support       358         0.50       0.50          179            0.00
    #> GPA_school           358         3.37       0.78          179            3.37
    #> IQ_score             358       103.11      12.04          179          103.01
    #> Motivation           358         3.79       0.77          179            3.78
    #>                No Support SD Support N Support Mean Support SD     d
    #> gifted_support          0.00       179         1.00       0.00  -Inf
    #> GPA_school              0.78       179         3.38       0.78 -0.01
    #> IQ_score               12.08       179       103.22      12.04 -0.02
    #> Motivation              0.74       179         3.80       0.80 -0.03

The post-matching balance and descriptive statistics show that the initial imbalance between the groups was substantially reduced. In the solution with the best adjusted d-ratio (n = 179 cases per group), Pillai’s Trace decreased from .129 to < .001, d-ratio increased from .333 to 1, mean g decreased from 0.427 to 0.016, and adjusted d-ratio increased from .331 to .96. Thus, exact MDM was also able to identify a solution in which the distributions of the covariates were comparable between the two groups.

# Three-Group Example

This example illustrates MDM for three groups using the same data set (`MAGMA_sim_data`) and covariates (`GPA_school`, `IQ_score`, and `Motivation`) as the two-group example. As the grouping variable changes to `teacher_ability_rating` (below average[BA], average [A], above average[AA]), we start by estimating descriptive statistics and initial unbalance.

    # Computing descriptive statistics and all pairwise effects for three groups
    descs_tar_pre <- MAGMA_desc(Data = MAGMA_sim_data,
                                group = "teacher_ability_rating",
                                covariates = covariates)

    descs_tar_pre %>%
      purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
                         "BA N", "BA Support Mean", "BA Support SD",
                         "A N", "A Mean", "A SD",
                         "AA N", "AA Mean", "AA SD",
                         "d BA-A", "d BA-AA", "d A-AA"))
    #>                        Overall N Overall Mean Overall SD BA N BA Support Mean
    #> teacher_ability_rating       800         2.02       0.80  243            1.00
    #> GPA_school                   800         3.29       0.93  243            2.76
    #> IQ_score                     800       100.51      15.00  243           92.93
    #> Motivation                   800         3.82       0.89  243            3.77
    #>                        BA Support SD A N A Mean  A SD AA N AA Mean AA SD d BA-A
    #> teacher_ability_rating          0.00 294   2.00  0.00  263    3.00  0.00   -Inf
    #> GPA_school                      0.82 294   3.33  0.85  263    3.73  0.86  -0.68
    #> IQ_score                       14.63 294 101.20 12.84  263  106.73 14.54  -0.60
    #> Motivation                      0.88 294   3.72  0.88  263    3.98  0.90   0.06
    #>                        d BA-AA d A-AA
    #> teacher_ability_rating    -Inf   -Inf
    #> GPA_school               -1.15  -0.47
    #> IQ_score                 -0.95  -0.40
    #> Motivation               -0.24  -0.29

    # Estimating and printing initial unbalance for teacher rated ability
    unbalance_tar <- initial_unbalance(Data = MAGMA_sim_data,
                                       group = "teacher_ability_rating",
                                       covariates = covariates,
                                       round = 3)
    #> Mean g was computed using robust variance meta-analysis with robumeta.
    unbalance_tar
    #>           Pillai's Trace d-ratio Mean g adj. d-ratio
    #> Unbalance          0.215   0.111  0.537        0.158

Similar as above, there are some substantial group difference, wherefore we conduct a standard and an exact matching.

## Standard Matching

To conduct standard MDM, we only need to adapt the grouping variable compared to the two-group example.

    # Conducting matching for gifted support
    MAGMA_sim_data_tar <- MAGMA(Data = MAGMA_sim_data,
                                group = "teacher_ability_rating",
                                covs = covariates,
                                weights = c(1, 1.5, 0.5),
                                cores = 2)

### Balance Estimation

To evaluate the balance, we only need to change the specific object names and grouping variable compared to the two group example. Otherwise, the process of balance estimation and visualization remains the same.

    # Estimating the four balance criteria iteratively over possible sample sizes
    Balance_tar <- Balance_MAGMA(Data = MAGMA_sim_data_tar,
                                    group = "teacher_ability_rating",
                                    covariates = covariates,
                                    step = "step")

    # Extracting balance criteria for 100 cases per group
    Balance_100_criteria_tar <- Balance_extract(Balance = Balance_tar,
                                                samplesize = 100,
                                                effects = FALSE)
    Balance_100_criteria_tar
    #> Pillai's Trace        d-ratio         mean g   adj. d-ratio
    #>           0.00           1.00           0.04           0.87

    # Extracting pairwise effects for 100 cases per group
    Balance_100_effects_tar <- Balance_extract(Balance = Balance_tar,
                                               samplesize = 100,
                                               effects = TRUE)
    Balance_100_effects_tar
    #> GPA_school GPA_school GPA_school   IQ_score   IQ_score   IQ_score Motivation
    #>       0.00       0.01       0.03      -0.03      -0.03      -0.07      -0.03
    #> Motivation Motivation
    #>      -0.03      -0.10

    # Plotting balance trend over sample size
    Plot_MAGMA(Balance = Balance_tar,
               criterion = c("Pillai", "d_ratio", "mean_g", "Adj_d_ratio"),
               print = FALSE)
    #> $Pillai

    #>
    #> $`d-ratio`

    #>
    #> $`mean g`

    #>
    #> $`adj-d-ratio`

    # Sample Sizes with optimized balance criteria
    Table_MAGMA(Balance = Balance_tar,
                round = 3)
    #> Balance table successfully created!
    #> # A tibble: 4 × 6
    #>   Criterion_optimized Pillai_Trace d_ratio mean_g adjusted_d_ratio n_per_group
    #>   <chr>                      <dbl>   <dbl>  <dbl>            <dbl>       <int>
    #> 1 Best mean g                0           1  0.02             0.77           33
    #> 2 Best Pillai                0           1  0.028            0.786          42
    #> 3 Best adj. d-ratio          0.002       1  0.043            0.882         120
    #> 4 Best d-ratio               0.008       1  0.086            0.822         148

    # Computing descriptive statistics and pairwise effects for 100 cases per group
    descs_tar_post <- MAGMA_desc(Data = MAGMA_sim_data_tar,
                                 group = "teacher_ability_rating",
                                 covariates = covariates,
                                 step_num = 120,
                                 step_var = "step")

    # Displaying the table with defined colum names
    descs_tar_post %>%
      purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
                         "BA N", "BA Support Mean", "BA Support SD",
                         "A N", "A Mean", "A SD",
                         "AA N", "AA Mean", "AA SD",
                         "d BA-A", "d BA-AA", "d A-AA"))
    #>                        Overall N Overall Mean Overall SD BA N BA Support Mean
    #> teacher_ability_rating       360         2.00       0.82  120            1.00
    #> GPA_school                   360         3.18       0.64  120            3.15
    #> IQ_score                     360        99.77      12.30  120           99.50
    #> Motivation                   360         3.81       0.80  120            3.76
    #>                        BA Support SD A N A Mean  A SD AA N AA Mean AA SD d BA-A
    #> teacher_ability_rating          0.00 120   2.00  0.00  120    3.00  0.00   -Inf
    #> GPA_school                      0.66 120   3.18  0.65  120    3.20  0.62  -0.05
    #> IQ_score                       12.45 120  99.91 12.24  120   99.90 12.32  -0.03
    #> Motivation                      0.85 120   3.83  0.74  120    3.83  0.82  -0.09
    #>                        d BA-AA d A-AA
    #> teacher_ability_rating    -Inf   -Inf
    #> GPA_school               -0.08  -0.03
    #> IQ_score                 -0.03   0.00
    #> Motivation               -0.08   0.00

The post-matching balance and descriptive statistics show that the initial imbalance between the groups was substantially reduced. In the solution with the best adjusted d-ratio (n = 120 cases per group), Pillai’s Trace decreased from .215 to .002, d-ratio increased from .111 to 1, mean g decreased from 0.537 to 0.043, and adjusted d-ratio increased from .158 to .882. Thus, MDM was able to identify a solution in which the distributions of the covariates were comparable between the three groups.

## Exaxt Matching

Similar as for the two-group example, we need to use the function `MAGMA_exact()` for exact MDM and specify the argument `exact`.

    # Conducting matching for gifted support
    MAGMA_sim_data_tar_exact <- MAGMA_exact(Data = MAGMA_sim_data,
                                            group = "teacher_ability_rating",
                                            covs = covariates,
                                            weights = c(1, 1.5, 0.5),
                                            cores = 2,
                                            exact = "gender")

### Balance Estimation

To evaluate the balance, we only need to change the specific object names and grouping variable compared to the standard matching. Otherwise, the process of balance estimation and visualization remains the same.

    # Estimating the four balance criteria iteratively over possible sample sizes
    Balance_tar_exact <- Balance_MAGMA(Data = MAGMA_sim_data_tar_exact,
                                       group = "teacher_ability_rating",
                                       covariates = covariates,
                                       step = "step")

    # Extracting balance criteria for 100 cases per group
    Balance_100_criteria_tar_exact <- Balance_extract(Balance = Balance_tar_exact,
                                                      samplesize = 100,
                                                      effects = FALSE)
    Balance_100_criteria_tar_exact
    #> Pillai's Trace        d-ratio         mean g   adj. d-ratio
    #>           0.00           1.00           0.05           0.86

    # Extracting pairwise effects for 100 cases per group
    Balance_100_effects_tar_exact <- Balance_extract(Balance = Balance_tar_exact,
                                                     samplesize = 100,
                                                     effects = TRUE)
    Balance_100_effects_tar_exact
    #> GPA_school GPA_school GPA_school   IQ_score   IQ_score   IQ_score Motivation
    #>      -0.01      -0.01       0.04      -0.08      -0.04      -0.04      -0.07
    #> Motivation Motivation
    #>      -0.03      -0.08

    # Plotting balance trend over sample size
    Plot_MAGMA(Balance = Balance_tar_exact,
               criterion = c("Pillai", "d_ratio", "mean_g", "Adj_d_ratio"),
               print = FALSE)
    #> $Pillai

    #>
    #> $`d-ratio`

    #>
    #> $`mean g`

    #>
    #> $`adj-d-ratio`

    # Sample Sizes with optimized balance criteria
    Table_MAGMA(Balance = Balance_tar_exact,
                round = 3)
    #> Balance table successfully created!
    #> # A tibble: 4 × 6
    #>   Criterion_optimized Pillai_Trace d_ratio mean_g adjusted_d_ratio n_per_group
    #>   <chr>                      <dbl>   <dbl>  <dbl>            <dbl>       <int>
    #> 1 Best mean g                0.001       1  0.026            0.827          59
    #> 2 Best Pillai                0           1  0.036            0.851          81
    #> 3 Best adj. d-ratio          0.002       1  0.041            0.882         116
    #> 4 Best d-ratio               0.009       1  0.094            0.801         145

    # Computing descriptive statistics and pairwise effects for 100 cases per group
    descs_tar_post_exact <- MAGMA_desc(Data = MAGMA_sim_data_tar_exact,
                                       group = "teacher_ability_rating",
                                       covariates = covariates,
                                       step_num = 116,
                                       step_var = "step")

    # Displaying the table with defined colum names
    descs_tar_post_exact %>%
      purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
                         "BA N", "BA Support Mean", "BA Support SD",
                         "A N", "A Mean", "A SD",
                         "AA N", "AA Mean", "AA SD",
                         "d BA-A", "d BA-AA", "d A-AA"))
    #>                        Overall N Overall Mean Overall SD BA N BA Support Mean
    #> teacher_ability_rating       348         2.00       0.82  116            1.00
    #> GPA_school                   348         3.18       0.62  116            3.15
    #> IQ_score                     348        99.54      12.13  116           99.31
    #> Motivation                   348         3.79       0.81  116            3.76
    #>                        BA Support SD A N A Mean  A SD AA N AA Mean AA SD d BA-A
    #> teacher_ability_rating          0.00 116   2.00  0.00  116    3.00  0.00   -Inf
    #> GPA_school                      0.66 116   3.19  0.59  116    3.21  0.60  -0.06
    #> IQ_score                       12.37 116  99.56 11.96  116   99.74 12.16  -0.02
    #> Motivation                      0.86 116   3.80  0.74  116    3.80  0.82  -0.05
    #>                        d BA-AA d A-AA
    #> teacher_ability_rating    -Inf   -Inf
    #> GPA_school               -0.10  -0.03
    #> IQ_score                 -0.04  -0.01
    #> Motivation               -0.05   0.00

The post-matching balance and descriptive statistics show that the initial imbalance between the groups was substantially reduced. In the solution with the best adjusted d-ratio (n = 116 cases per group), Pillai’s Trace decreased from .215 to .002, d-ratio increased from .111 to 1, mean g decreased from 0.537 to 0.041, and adjusted d-ratio increased from .158 to .882. Thus, exact MDM was also able to identify a solution in which the distributions of the covariates were comparable between the three groups.

# 2x2-design Example

This example illustrates MDM for a 2x2 designs using the same data set (`MAGMA_sim_data`) and covariates (`GPA_school`, `IQ_score`, and `Motivation`) as the previous examples. The major change is that now both `gifted_support` and `enrichment` are used as grouping variable.

    # Computing descriptive statistics and all pairwise effects for three groups
    descs_2x2 <- MAGMA_desc(Data = MAGMA_sim_data,
                            group = c("gifted_support", "enrichment"),
                            covariates = covariates)
    #> 2x2 groups are represented as 4 groups.

    descs_2x2  %>%
      purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
                         "Sup & No En N", "Sup & No En Mean",
                         "Sup & No En SD",
                         "Sup & En N", "Sup & En Mean", "Sup & En SD",
                         "No Sup & No En N", "No Sup & No En Mean", "No Sup & No En SD",
                         "No Sup & En N", "No Sup & En Mean", "No Sup & En SD",
                         "d YesNo-YesYes", "d YesNo-NoNo", "d YesNo-NoYes",
                         "d YesYes-NoNo", "d YesYes-YNoYes",
                         "d NoNo-NoYes"))
    #>            Overall N Overall Mean Overall SD Sup & No En N Sup & No En Mean
    #> group_long       800         2.63       1.07           175             1.00
    #> GPA_school       800         3.29       0.93           175             3.44
    #> IQ_score         800       100.51      15.00           175           105.09
    #> Motivation       800         3.82       0.89           175             3.79
    #>            Sup & No En SD Sup & En N Sup & En Mean Sup & En SD No Sup & No En N
    #> group_long           0.00        130          2.00        0.00              309
    #> GPA_school           0.88        130          3.71        0.88              309
    #> IQ_score            14.09        130        110.26       12.74              309
    #> Motivation           0.90        130          3.89        0.86              309
    #>            No Sup & No En Mean No Sup & No En SD No Sup & En N No Sup & En Mean
    #> group_long                3.00              0.00           186             4.00
    #> GPA_school                3.01              0.93           186             3.31
    #> IQ_score                 94.27             13.30           186            99.74
    #> Motivation                3.75              0.89           186             3.92
    #>            No Sup & En SD d YesNo-YesYes d YesNo-NoNo d YesNo-NoYes
    #> group_long           0.00           -Inf         -Inf          -Inf
    #> GPA_school           0.86          -0.31         0.47          0.15
    #> IQ_score            15.01          -0.38         0.80          0.37
    #> Motivation           0.90          -0.11         0.04         -0.14
    #>            d YesYes-NoNo d YesYes-YNoYes d NoNo-NoYes
    #> group_long          -Inf            -Inf         -Inf
    #> GPA_school          0.76            0.46        -0.33
    #> IQ_score            1.22            0.74        -0.39
    #> Motivation          0.16           -0.03        -0.19

    # Estimating and printing initial unbalance for teacher rated ability
    unbalance_2x2 <- initial_unbalance(Data = MAGMA_sim_data,
                                       group = c("gifted_support", "enrichment"),
                                       covariates = covariates,
                                       round = 3)
    #> Mean g was computed using robust variance meta-analysis with robumeta.
    unbalance_2x2
    #>           Pillai's Trace gifted_support Pillai's Trace enrichment
    #> Unbalance                         0.133                     0.044
    #>           Pillai's Trace IA d-ratio Mean g adj. d-ratio
    #> Unbalance             0.001   0.389  0.391        0.314

Similar as above, there are some substantial group difference, wherefore we conduct a standard and an exact matching.

## Standard Matching

To conduct standard MDM, we only need to adapt the grouping variable compared to the previous examples.

    # Conducting matching for gifted support
    MAGMA_sim_data_2x2 <- MAGMA(Data = MAGMA_sim_data,
                                group = c("gifted_support", "enrichment"),
                                covs = covariates,
                                weights = c(1, 1.5, 0.5),
                                cores = 2)

### Balance Estimation

To evaluate the balance, we only need to change the specific object names and grouping variable compared to the previous examples. Otherwise, the process of balance estimation and visualization remains the same.

    # Estimating the four balance criteria iteratively over possible sample sizes
    Balance_2x2 <- Balance_MAGMA(Data = MAGMA_sim_data_2x2,
                                    group = c("gifted_support", "enrichment"),
                                    covariates = covariates,
                                    step = "step")

    # Extracting balance criteria for 100 cases per group
    Balance_100_criteria_2x2 <- Balance_extract(Balance = Balance_2x2,
                                                samplesize = 100,
                                                effects = FALSE)
    Balance_100_criteria_2x2
    #> Pillai's Trace ME 1 Pillai's Trace ME 2   Pillai's Trace IA             d-ratio
    #>                0.00                0.00                0.00                1.00
    #>              mean g        adj. d-ratio
    #>                0.07                0.82

    # Extracting pairwise effects for 100 cases per group
    Balance_100_effects_2x2 <- Balance_extract(Balance = Balance_2x2,
                                               samplesize = 100,
                                               effects = TRUE)
    Balance_100_effects_2x2
    #> GPA_school GPA_school GPA_school GPA_school GPA_school GPA_school   IQ_score
    #>      -0.09      -0.08      -0.08       0.01       0.03       0.05      -0.05
    #>   IQ_score   IQ_score   IQ_score   IQ_score   IQ_score Motivation Motivation
    #>      -0.04      -0.07       0.09       0.12       0.14       0.04       0.04
    #> Motivation Motivation Motivation Motivation
    #>       0.02      -0.05      -0.07      -0.13

    # Plotting balance trend over sample size
    Plot_MAGMA(Balance = Balance_2x2,
               criterion = c("Pillai", "d_ratio", "mean_g", "Adj_d_ratio"),
               print = FALSE)
    #> $Pillai
    #> $Pillai[[1]]

    #>
    #> $Pillai[[2]]

    #>
    #> $Pillai[[3]]

    #>
    #>
    #> $`d-ratio`

    #>
    #> $`mean g`

    #>
    #> $`adj-d-ratio`

    # Sample Sizes with optimized balance criteria
    Table_MAGMA(Balance = Balance_2x2,
                round = 3)
    #> Balance table successfully created!
    #> # A tibble: 6 × 8
    #>   Criterion_optimized Pillai_Trace_ME1 Pillai_Trace_ME2 Pillai_Trace_IA d_ratio
    #>   <chr>                          <dbl>            <dbl>           <dbl>   <dbl>
    #> 1 Best Pillai ME 2               0.009            0               0.002   0.889
    #> 2 Best mean g                    0.001            0.002           0       1
    #> 3 Best Pillai ME 1               0                0.003           0       1
    #> 4 Best Pillai IA                 0.002            0.004           0       1
    #> 5 Best adj. d-ratio              0.001            0.002           0       1
    #> 6 Best d-ratio                   0.001            0.005           0.001   1
    #>   mean_g adjusted_d_ratio n_per_group
    #>    <dbl>            <dbl>       <int>
    #> 1  0.075            0.705          39
    #> 2  0.046            0.836          83
    #> 3  0.049            0.833          84
    #> 4  0.069            0.824         106
    #> 5  0.055            0.855         112
    #> 6  0.076            0.829         139

    # Computing descriptive statistics and pairwise effects for 100 cases per group
    descs_2x2_post <- MAGMA_desc(Data = MAGMA_sim_data_2x2,
                                 group = c("gifted_support", "enrichment"),
                                 covariates = covariates,
                                 step_num = 112,
                                 step_var = "step")
    #> 2x2 groups are represented as 4 groups.

    # Displaying the table with defined colum names
    descs_2x2_post %>%
      purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
                         "Sup & No En N", "Sup & No En Mean",
                         "Sup & No En SD",
                         "Sup & En N", "Sup & En Mean", "Sup & En SD",
                         "No Sup & No En N", "No Sup & No En Mean", "No Sup & No En SD",
                         "No Sup & En N", "No Sup & En Mean", "No Sup & En SD",
                         "d YesNo-YesYes", "d YesNo-NoNo", "d YesNo-NoYes",
                         "d YesYes-NoNo", "d YesYes-YNoYes",
                         "d NoNo-NoYes"))
    #>            Overall N Overall Mean Overall SD Sup & No En N Sup & No En Mean
    #> group_long       336         2.50       1.12            84             1.00
    #> GPA_school       336         3.53       0.74            84             3.58
    #> IQ_score         336       104.87      10.41            84           105.58
    #> Motivation       336         3.87       0.74            84             3.90
    #>            Sup & No En SD Sup & En N Sup & En Mean Sup & En SD No Sup & No En N
    #> group_long           0.00         84          2.00        0.00               84
    #> GPA_school           0.78         84          3.51        0.73               84
    #> IQ_score            10.67         84        104.74       10.56               84
    #> Motivation           0.77         84          3.85        0.74               84
    #>            No Sup & No En Mean No Sup & No En SD No Sup & En N No Sup & En Mean
    #> group_long                3.00              0.00            84             4.00
    #> GPA_school                3.53              0.71            84             3.50
    #> IQ_score                104.87             10.06            84           104.30
    #> Motivation                3.88              0.74            84             3.83
    #>            No Sup & En SD d YesNo-YesYes d YesNo-NoNo d YesNo-NoYes
    #> group_long           0.00           -Inf         -Inf          -Inf
    #> GPA_school           0.73           0.09         0.07          0.11
    #> IQ_score            10.48           0.08         0.07          0.12
    #> Motivation           0.73           0.07         0.03          0.09
    #>            d YesYes-NoNo d YesYes-YNoYes d NoNo-NoYes
    #> group_long          -Inf            -Inf         -Inf
    #> GPA_school         -0.03            0.01         0.04
    #> IQ_score           -0.01            0.04         0.06
    #> Motivation         -0.04            0.03         0.07

The post-matching balance and descriptive statistics show that the initial imbalance between the groups was substantially reduced. In the solution with the best adjusted d-ratio (n = 112 cases per group), Pillai’s Trace (gifted_support) decreased from .133 to .001, Pillai’s Trace (enrichment) decreased from .044 to .002, Pillai’s Trace (interaction) remained at < .001, d-ratio increased from .389 to 1, mean g decreased from 0.391 to 0.055, and adjusted d-ratio increased from .314 to .855. Thus, MDM was able to identify a solution in which the distributions of the covariates were comparable for the 2x2 design.

## Exaxt Matching

Similar as for the previous examples, we need to use the function `MAGMA_exact()` for exact MDM and specify the argument `exact`.

    # Conducting matching for gifted support
    MAGMA_sim_data_2x2_exact <- MAGMA_exact(Data = MAGMA_sim_data,
                                            group = c("gifted_support", "enrichment"),
                                            covs = covariates,
                                            weights = c(1, 1.5, 0.5),
                                            cores = 2,
                                            exact = "gender")

### Balance Estimation

To evaluate the balance, we only need to change the specific object names and grouping variable compared to the standard matching. Otherwise, the process of balance estimation and visualization remains the same.

    # Estimating the four balance criteria iteratively over possible sample sizes
    Balance_2x2_exact <- Balance_MAGMA(Data = MAGMA_sim_data_2x2_exact,
                                       group = c("gifted_support", "enrichment"),
                                       covariates = covariates,
                                       step = "step")

    # Extracting balance criteria for 100 cases per group
    Balance_100_criteria_2x2_exact <- Balance_extract(Balance = Balance_2x2_exact,
                                                      samplesize = 100,
                                                      effects = FALSE)
    Balance_100_criteria_2x2_exact
    #> Pillai's Trace ME 1 Pillai's Trace ME 2   Pillai's Trace IA             d-ratio
    #>                0.00                0.00                0.00                1.00
    #>              mean g        adj. d-ratio
    #>                0.06                0.83

    # Extracting pairwise effects for 100 cases per group
    Balance_100_effects_2x2_exact <- Balance_extract(Balance = Balance_2x2_exact,
                                                     samplesize = 100,
                                                     effects = TRUE)
    Balance_100_effects_2x2_exact
    #> GPA_school GPA_school GPA_school GPA_school GPA_school GPA_school   IQ_score
    #>      -0.02      -0.05      -0.09       0.06       0.09      -0.03       0.03
    #>   IQ_score   IQ_score   IQ_score   IQ_score   IQ_score Motivation Motivation
    #>       0.04       0.05       0.08       0.14       0.07       0.05       0.09
    #> Motivation Motivation Motivation Motivation
    #>       0.13      -0.03      -0.05       0.07

    # Plotting balance trend over sample size
    Plot_MAGMA(Balance = Balance_2x2_exact,
               criterion = c("Pillai", "d_ratio", "mean_g", "Adj_d_ratio"),
               print = FALSE)
    #> $Pillai
    #> $Pillai[[1]]

    #>
    #> $Pillai[[2]]

    #>
    #> $Pillai[[3]]

    #>
    #>
    #> $`d-ratio`

    #>
    #> $`mean g`

    #>
    #> $`adj-d-ratio`

    # Sample Sizes with optimized balance criteria
    Table_MAGMA(Balance = Balance_2x2_exact,
                round = 3)
    #> Balance table successfully created!
    #> # A tibble: 6 × 8
    #>   Criterion_optimized Pillai_Trace_ME1 Pillai_Trace_ME2 Pillai_Trace_IA d_ratio
    #>   <chr>                          <dbl>            <dbl>           <dbl>   <dbl>
    #> 1 Best mean g                   NA               NA              NA       1
    #> 2 Best Pillai IA                 0.005            0               0       1
    #> 3 Best Pillai ME 2               0.003            0               0.004   0.944
    #> 4 Best Pillai ME 1               0.001            0               0.002   1
    #> 5 Best adj. d-ratio              0.001            0               0.002   1
    #> 6 Best d-ratio                   0.003            0.003           0.002   1
    #>   mean_g adjusted_d_ratio n_per_group
    #>    <dbl>            <dbl>       <int>
    #> 1  0.044            0.692          20
    #> 2  0.055            0.723          34
    #> 3  0.052            0.811          77
    #> 4  0.044            0.842          86
    #> 5  0.046            0.845          90
    #> 6  0.075            0.808         109

    # Computing descriptive statistics and pairwise effects for 100 cases per group
    descs_2x2_post_exact <- MAGMA_desc(Data = MAGMA_sim_data_2x2_exact,
                                       group = c("gifted_support", "enrichment"),
                                       covariates = covariates,
                                       step_num = 116,
                                       step_var = "step")
    #> 2x2 groups are represented as 4 groups.

    # Displaying the table with defined colum names
    descs_2x2_post_exact %>%
      purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
                         "Sup & No En N", "Sup & No En Mean",
                         "Sup & No En SD",
                         "Sup & En N", "Sup & En Mean", "Sup & En SD",
                         "No Sup & No En N", "No Sup & No En Mean", "No Sup & No En SD",
                         "No Sup & En N", "No Sup & En Mean", "No Sup & En SD",
                         "d YesNo-YesYes", "d YesNo-NoNo", "d YesNo-NoYes",
                         "d YesYes-NoNo", "d YesYes-YNoYes",
                         "d NoNo-NoYes"))
    #>            Overall N Overall Mean Overall SD Sup & No En N Sup & No En Mean
    #> group_long       464         2.50       1.12           116             1.00
    #> GPA_school       464         3.58       0.83           116             3.64
    #> IQ_score         464       106.51      10.53           116           107.92
    #> Motivation       464         3.85       0.81           116             3.91
    #>            Sup & No En SD Sup & En N Sup & En Mean Sup & En SD No Sup & No En N
    #> group_long           0.00        116          2.00        0.00              116
    #> GPA_school           0.85        116          3.58        0.86              116
    #> IQ_score            11.30        116        106.57       10.53              116
    #> Motivation           0.86        116          3.82        0.77              116
    #>            No Sup & No En Mean No Sup & No En SD No Sup & En N No Sup & En Mean
    #> group_long                3.00              0.00           116             4.00
    #> GPA_school                3.58              0.78           116             3.50
    #> IQ_score                106.42             10.64           116           105.14
    #> Motivation                3.82              0.83           116             3.84
    #>            No Sup & En SD d YesNo-YesYes d YesNo-NoNo d YesNo-NoYes
    #> group_long           0.00           -Inf         -Inf          -Inf
    #> GPA_school           0.85           0.07         0.07          0.16
    #> IQ_score             9.54           0.12         0.14          0.27
    #> Motivation           0.79           0.11         0.11          0.08
    #>            d YesYes-NoNo d YesYes-YNoYes d NoNo-NoYes
    #> group_long          -Inf            -Inf         -Inf
    #> GPA_school          0.00            0.09         0.10
    #> IQ_score            0.01            0.14         0.13
    #> Motivation          0.00           -0.03        -0.02

The post-matching balance and descriptive statistics show that the initial imbalance between the groups was substantially reduced. In the solution with the best adjusted d-ratio (n = 112 cases per group), Pillai’s Trace (gifted_support) decreased from .133 to .001, Pillai’s Trace (enrichment) decreased from .044 to < .001, Pillai’s Trace (interaction) remained low at .002, d-ratio increased from .389 to 1, mean g decreased from 0.391 to 0.046, and adjusted d-ratio increased from .314 to .845. Thus, exact MDM was also able to identify a solution in which the distributions of the covariates were comparable for the 2x2 design.

# References

Gu, X. S., & Rosenbaum, P. R. (1993). Comparison of multivariate matching methods: Structures, distances, and algorithms. Journal of Computational and Graphical Statistics, 2(4), 405-420. https://doi.org/10.1080/10618600.1993.10474623

Guo, S., Fraser, M., & Chen, Q. (2020). Propensity Score Analysis: Recent Debate and Discussion. Journal of the Society for Social Work and Research, 11(3), 463–482. https://doi.org/10.1086/711393

Ridgeway, G., McCaffrey, D. F., Morral, A. R., Cefalu, M., Burgette, L. F., Pane, J. D., & Griffin, B. A. (2015). Toolkit for Weighting and Analysis of Nonequivalent Groups: A tutorial for the twang package. Rand.

Stuart, E. A. (2010). Matching Methods for Causal Inference: A Review and a Look Forward. Statistical Science, 25(1). https://doi.org/10.1214/09-STS313