# Diagnostics and Model Selection

Before the final table goes into a manuscript, check the model. These helpers keep diagnostics close to the regression workflow. They are screening aids: interpret them with the study design, clinical or subject-matter judgement, and the diagnostics from the fitted model.

    library(gtregression)
    library(dplyr)

    data("data_birthwt", package = "gtregression")

    birthwt_data <- data_birthwt |>
      mutate(
        race = factor(race, levels = c(1, 2, 3),
                      labels = c("White", "Black", "Other")),
        smoke = factor(smoke, levels = c(0, 1), labels = c("No", "Yes")),
        ht = factor(ht, levels = c(0, 1), labels = c("No", "Yes")),
        ui = factor(ui, levels = c(0, 1), labels = c("No", "Yes")),
        low = factor(low, levels = c(0, 1), labels = c("Normal BW", "Low BW")),
        ptl_cat = factor(ifelse(ptl > 0, "Yes", "No"), levels = c("No", "Yes"))
      )

    exposures <- c("age", "lwt", "race", "smoke", "ht", "ui", "ptl_cat")

## Convergence Screening

Use `check_convergence()` before interpreting model estimates, especially for log-binomial and small or sparse binary-outcome models. A non-converged model is a fitting warning, not a finding.

    check_convergence(
      data = birthwt_data,
      exposures = exposures,
      outcome = low,
      approach = logit,
      multivariate = TRUE
    )

Convergence check
Exposure | Model | Converged | Max fitted value
--- | --- | --- | ---
age + lwt + race + smoke + ht + ui + ptl_cat | logit | Yes | 0.880
Screening aid only; inspect non-convergence, impossible fitted values, and model specification before interpreting estimates.

For risk-ratio workflows, this same check helps users decide whether a log-binomial model fitted cleanly or whether a robust Poisson approach may be a more practical sensitivity analysis.

    check_convergence(
      data = birthwt_data,
      exposures = c("smoke", "ht", "ui", "ptl_cat"),
      outcome = low,
      approach = logbinomial,
      multivariate = TRUE
    )

Convergence check
Exposure | Model | Converged | Max fitted value
--- | --- | --- | ---
smoke + ht + ui + ptl_cat | logbinomial | No |
Screening aid only; inspect non-convergence, impossible fitted values, and model specification before interpreting estimates.

## Collinearity Screening

`check_collinearity()` reports VIF-style diagnostics for multivariable models. High VIF values are prompts to inspect coding, overlap between predictors, and the scientific purpose of the model.

    birthwt_multi <- multi_reg(
      data = birthwt_data,
      outcome = low,
      exposures = exposures,
      approach = logit
    )

    check_collinearity(birthwt_multi, format = gt)

Collinearity check
Variable | VIF | Interpretation
--- | --- | ---
age | 1.13 | No collinearity
lwt | 1.24 | No collinearity
race | 1.08 | No collinearity
smoke | 1.21 | No collinearity
ht | 1.08 | No collinearity
ui | 1.07 | No collinearity
ptl_cat | 1.12 | No collinearity
Screening aid only; interpret VIF with model purpose, coding choices, sample size, and subject-matter knowledge.

Adjusted-mode `multi_reg()` objects contain one model per exposure. The collinearity output keeps that list structure so each model can be inspected separately.

    birthwt_adjusted <- multi_reg(
      data = birthwt_data,
      outcome = low,
      exposures = c("smoke", "ht", "ui", "ptl_cat"),
      adjust_for = c("age", "lwt", "race"),
      approach = logit
    )

    check_collinearity(birthwt_adjusted, format = tibble)

    ## $smoke
    ## # A tibble: 4 × 3
    ##   Variable   VIF Interpretation
    ##   <chr>    <dbl> <chr>
    ## 1 smoke     1.15 No collinearity
    ## 2 age       1.09 No collinearity
    ## 3 lwt       1.14 No collinearity
    ## 4 race      1.07 No collinearity
    ##
    ## $ht
    ## # A tibble: 4 × 3
    ##   Variable   VIF Interpretation
    ##   <chr>    <dbl> <chr>
    ## 1 ht        1.07 No collinearity
    ## 2 age       1.09 No collinearity
    ## 3 lwt       1.2  No collinearity
    ## 4 race      1.03 No collinearity
    ##
    ## $ui
    ## # A tibble: 4 × 3
    ##   Variable   VIF Interpretation
    ##   <chr>    <dbl> <chr>
    ## 1 ui        1.03 No collinearity
    ## 2 age       1.09 No collinearity
    ## 3 lwt       1.15 No collinearity
    ## 4 race      1.03 No collinearity
    ##
    ## $ptl_cat
    ## # A tibble: 4 × 3
    ##   Variable   VIF Interpretation
    ##   <chr>    <dbl> <chr>
    ## 1 ptl_cat   1.03 No collinearity
    ## 2 age       1.1  No collinearity
    ## 3 lwt       1.15 No collinearity
    ## 4 race      1.04 No collinearity

## Model Fit Plots

`plot_model_fit()` turns fitted models into quick diagnostic plots. It accepts raw `lm()` and `glm()` objects, and it also works with models saved inside `uni_reg()` and `multi_reg()` results.

For logistic regression, the calibration plot compares predicted probabilities with observed event proportions. Points close to the diagonal line suggest that the model predictions are reasonably aligned with the observed data. Calibration is usually most useful for multivariable models because predicted probabilities vary across many people.

    plot_model_fit(
      birthwt_multi,
      type = calibration,
      bins = 6
    )

When a `uni_reg()` object contains several models, use `model_name` to choose the exposure you want to inspect. For a simple binary exposure, calibration may only show two points because the model has only two fitted probabilities; in that situation, residual and influence plots are usually more useful.

    birthwt_uni <- uni_reg(
      data = birthwt_data,
      outcome = low,
      exposures = c("age", "lwt", "smoke"),
      approach = logit
    )

    plot_model_fit(
      birthwt_uni,
      model_name = smoke,
      type = residual
    )

For logistic models, residual plots often form two visible bands. That is a normal consequence of a 0/1 outcome and should be interpreted as a screening plot rather than a linear-model residual plot.

For linear regression, `type = all` shows the classic residual, Q-Q, scale-location, and Cook’s distance views.

    fit_lm <- lm(bwt ~ age + lwt, data = birthwt_data)
    plot_model_fit(fit_lm)

## Proportional Hazards Screening

For Cox models, use `check_ph()` before treating hazard ratios as final. It reports Schoenfeld residual tests from `survival::cox.zph()`, including a global test. Small p-values suggest possible non-proportional hazards and should be reviewed with plots, follow-up pattern, and clinical judgement.

    data("data_lungcancer", package = "gtregression")

    lung_data <- data_lungcancer |>
      mutate(
        trt = factor(trt, levels = c(1, 2),
                     labels = c("Standard treatment", "Test treatment")),
        prior = factor(prior, levels = c(0, 10), labels = c("No", "Yes")),
        celltype = factor(
          celltype,
          levels = c("squamous", "smallcell", "adeno", "large"),
          labels = c("Squamous", "Small cell", "Adenocarcinoma", "Large cell")
        )
      )

    cox_fit <- cox_reg(
      data = lung_data,
      time = time,
      event = status,
      exposures = c(trt, celltype, prior),
      adjust_for = c(age, karno)
    )

    check_ph(cox_fit, format = gt)

Proportional hazards check
Model | Term | Test | Chi-square | df | p-value | Interpretation
--- | --- | --- | --- | --- | --- | ---
trt | trt | Term | 0.28 | 1 | 0.594 | No evidence of PH violation
trt | age | Term | 2.10 | 1 | 0.147 | No evidence of PH violation
trt | karno | Term | 12.00 | 1 | <0.001 | Possible PH violation
trt | GLOBAL | Global | 19.10 | 3 | <0.001 | Possible PH violation
celltype | celltype | Term | 14.15 | 3 | 0.003 | Possible PH violation
celltype | age | Term | 1.99 | 1 | 0.158 | No evidence of PH violation
celltype | karno | Term | 14.77 | 1 | <0.001 | Possible PH violation
celltype | GLOBAL | Global | 30.18 | 5 | <0.001 | Possible PH violation
prior | prior | Term | 1.87 | 1 | 0.171 | No evidence of PH violation
prior | age | Term | 2.18 | 1 | 0.140 | No evidence of PH violation
prior | karno | Term | 13.31 | 1 | <0.001 | Possible PH violation
prior | GLOBAL | Global | 22.29 | 3 | <0.001 | Possible PH violation
Screening aid only. Small p-values suggest possible non-proportional hazards; interpret with Schoenfeld residual plots, follow-up pattern, clinical context, and model purpose. alpha = 0.05; transform = km.

Use `format = tibble` when you want to inspect or filter the diagnostic results.

    check_ph(cox_fit, transform = rank, format = tibble)

    ## # A tibble: 12 × 7
    ##    Model    Term     Test   Chi.square    df   p.value Interpretation
    ##    <chr>    <chr>    <chr>       <dbl> <dbl>     <dbl> <chr>
    ##  1 trt      trt      Term        0.278     1 0.598     No evidence of PH violat…
    ##  2 trt      age      Term        2.04      1 0.153     No evidence of PH violat…
    ##  3 trt      karno    Term       12.6       1 0.000377  Possible PH violation
    ##  4 trt      GLOBAL   Global     19.7       3 0.000191  Possible PH violation
    ##  5 celltype celltype Term       14.3       3 0.00249   Possible PH violation
    ##  6 celltype age      Term        1.93      1 0.164     No evidence of PH violat…
    ##  7 celltype karno    Term       15.4       1 0.0000877 Possible PH violation
    ##  8 celltype GLOBAL   Global     30.6       5 0.0000114 Possible PH violation
    ##  9 prior    prior    Term        1.92      1 0.166     No evidence of PH violat…
    ## 10 prior    age      Term        2.13      1 0.145     No evidence of PH violat…
    ## 11 prior    karno    Term       14.0       1 0.000187  Possible PH violation
    ## 12 prior    GLOBAL   Global     23.0       3 0.0000401 Possible PH violation

## Stepwise Model Selection

`compare_models()` is for prespecified candidate models that have already been fitted with gtregression. It answers a different question from stepwise selection: “How do these planned models compare?” The inputs should be `multi_reg()`, `cox_reg()`, or `surv_reg()` outputs, not raw `lm()`, `glm()`, `coxph()`, or `survreg()` objects. This keeps the workflow consistent with the publication-ready tables created by the package.

    logit_m0 <- multi_reg(
      data = birthwt_data,
      outcome = low,
      exposures = smoke,
      approach = logit
    )

    logit_m1 <- multi_reg(
      data = birthwt_data,
      outcome = low,
      exposures = c(smoke, age, lwt),
      approach = logit
    )

    logit_m2 <- multi_reg(
      data = birthwt_data,
      outcome = low,
      exposures = c(smoke, age, lwt, race, ht, ui),
      approach = logit
    )

    logit_model_comparison <- compare_models(
      logit_m0,
      logit_m1,
      logit_m2,
      model_names = c(
        "Smoking only",
        "Add age and weight",
        "Full clinical model"
      ),
      primary_exposure = smoke,
      format = gt
    )

    logit_model_comparison$table

Model comparison
Model | Variables | N | Parameters | AIC | BIC | Best AIC | Best BIC | Log-likelihood | LR chi-square | df | p-value | Primary estimate | Change from first
--- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---
Smoking only | smoke | 189 | 2 | 233.80 | 240.29 | No | Yes | -114.90 |  |  |  | 2.02 | 0.00%
Add age and weight | smoke + age + lwt | 189 | 4 | 230.88 | 243.85 | No | No | -111.44 | 6.93 | 2 | 0.031 | 1.96 | -4.73%
Full clinical model | smoke + age + lwt + race + ht + ui | 189 | 8 | 219.95 | 245.88 | Yes | No | -101.97 | 18.93 | 4 | <0.001 | 2.79 | 45.95%
Comparison status: Same analysis sample. Same analysis sample; assessed using retained model row identifiers. Nested-model status: sequential models are nested.
Compare prespecified candidate models; lower AIC or BIC indicates better relative fit among the compared models. Models were fitted to the same analysis sample. AIC, BIC, log-likelihood and likelihood-ratio tests may be interpreted as formal model-comparison statistics when the models are nested as required. Primary estimate change is calculated on the coefficient/log-effect scale before exponentiation and can help assess robustness across candidate models.

The table reports N, number of parameters, AIC, BIC, log-likelihood, and likelihood-ratio comparisons when `nested = TRUE`. Lower AIC or BIC identifies better relative fit among the compared models. When `primary_exposure` is supplied, the table also tracks that effect estimate and the percentage change across models.

`compare_models()` automatically checks whether the candidate models appear to use the same analysis sample. It uses retained row identifiers when the fitted model stores them; otherwise it compares N and event counts. If the models use different complete-case samples, the table still displays AIC, BIC, log-likelihood, and likelihood-ratio statistics for transparency, but the footer warns that these values should not be interpreted as formal model-selection criteria across different datasets. In that situation, use the primary exposure estimate, percentage change, confidence intervals, and clinical or epidemiological reasoning to judge robustness.

### Build Candidates in the App

The function API intentionally accepts fitted gtregression objects. In the gtregression app, Advanced > Compare models provides a guided layer before that API: choose the shared outcome and approach, name two to six candidates, and select each candidate’s exposures, adjustment variables, and optional interaction. The app fits `multi_reg()`, `cox_reg()`, or `surv_reg()` objects and then calls `compare_models()` with those objects.

For the birth weight models above, enter Smoking only, Add age and weight, and Full clinical model as the candidate names, place `smoke` in the exposure control, and add the corresponding variables under adjustment. Select `smoke` as the primary exposure to reproduce the estimate-change columns. The generated code records every fitted model and is suitable for direct use in an R script.

Candidate order is deliberate. Put the simplest or primary model first because percentage change is calculated from that model and likelihood-ratio statistics compare sequential pairs. If the fitted samples differ, or a sequential pair is not nested, the rendered table displays only the warning relevant to that comparison. See the gtregression App article for the complete ordinary, Cox, and parametric survival walkthrough.

For Cox and parametric survival models, fit the candidate models with `cox_reg()` or `surv_reg()` first. `compare_models()` then keeps survival-specific columns such as events and Cox concordance.

    cox_m0 <- cox_reg(
      data = lung_data,
      time = time,
      event = status,
      exposures = trt
    )

    cox_m1 <- cox_reg(
      data = lung_data,
      time = time,
      event = status,
      exposures = trt,
      adjust_for = c(age, karno)
    )

    cox_m2 <- cox_reg(
      data = lung_data,
      time = time,
      event = status,
      exposures = c(trt, age, karno, celltype, prior),
      multivariable = TRUE
    )

    cox_model_comparison <- compare_models(
      list(
        "Treatment only" = cox_m0,
        "Add age and performance" = cox_m1,
        "Full clinical model" = cox_m2
      ),
      primary_exposure = trt,
      format = gt
    )

    cox_model_comparison$table

Model comparison
Model | Variables | N | Events | Parameters | AIC | BIC | Best AIC | Best BIC | Log-likelihood | LR chi-square | df | p-value | Concordance | Primary estimate | Change from first
--- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---
Treatment only | trt | 137 | 128 | 1 | 1012.89 | 1015.74 | No | No | -505.44 |  |  |  | 0.525 | 1.02 | 0.00%
Add age and performance | trt + age + karno | 137 | 128 | 3 | 973.76 | 982.31 | No | Yes | -483.88 | 43.13 | 2 | <0.001 | 0.712 | 1.21 | 968.31%
Full clinical model | trt + age + karno + celltype + prior | 137 | 128 | 7 | 962.79 | 982.76 | Yes | No | -474.40 | 18.96 | 4 | <0.001 | 0.736 | 1.34 | 1561.45%
Comparison status: Same analysis sample. Same analysis sample; assessed using retained model row identifiers. Nested-model status: sequential models are nested.
Compare prespecified candidate models; lower AIC or BIC indicates better relative fit among the compared models. Models were fitted to the same analysis sample. AIC, BIC, log-likelihood and likelihood-ratio tests may be interpreted as formal model-comparison statistics when the models are nested as required. Primary estimate change is calculated on the coefficient/log-effect scale before exponentiation and can help assess robustness across candidate models.

`select_models()` compares candidate models step by step. It is useful for exploration, teaching, and sensitivity checks. It should not replace a planned model based on study design or a causal framework.

    selected <- select_models(
      data = birthwt_data,
      outcome = low,
      exposures = exposures,
      approach = logit,
      direction = forward
    )

    selected

    ## $results_table
    ## # A tibble: 6 × 9
    ##   model_id formula          model_terms n_predictors   AIC   BIC logLik deviance
    ##      <int> <chr>            <chr>              <int> <dbl> <dbl>  <dbl>    <dbl>
    ## 1        1 low ~ 1          Intercept …            0  237.  240.  -117.     235.
    ## 2        2 low ~ ptl_cat    ptl_cat                1  226.  232.  -111.     222.
    ## 3        3 low ~ ptl_cat +… ptl_cat + …            2  223.  233.  -109.     217.
    ## 4        4 low ~ ptl_cat +… ptl_cat + …            3  221.  234.  -107.     213.
    ## 5        5 low ~ ptl_cat +… ptl_cat + …            4  217.  234.  -104.     207.
    ## 6        6 low ~ ptl_cat +… ptl_cat + …            5  217.  237.  -103.     205.
    ## # ℹ 1 more variable: selected_vars <chr>
    ##
    ## $best_model
    ##
    ## Call:  stats::glm(formula = fmla, family = family, data = model_data)
    ##
    ## Coefficients:
    ## (Intercept)   ptl_catYes          age        htYes          lwt        uiYes
    ##     1.74560      1.42123     -0.05331      1.90580     -0.01437      0.69193
    ##
    ## Degrees of Freedom: 188 Total (i.e. Null);  183 Residual
    ## Null Deviance:       234.7
    ## Residual Deviance: 205.2     AIC: 217.2
    ##
    ## $all_models
    ## $all_models$model_1
    ##
    ## Call:  stats::glm(formula = fmla, family = family, data = model_data)
    ##
    ## Coefficients:
    ## (Intercept)
    ##       -0.79
    ##
    ## Degrees of Freedom: 188 Total (i.e. Null);  188 Residual
    ## Null Deviance:       234.7
    ## Residual Deviance: 234.7     AIC: 236.7
    ##
    ## $all_models$model_2
    ##
    ## Call:  stats::glm(formula = fmla, family = family, data = model_data)
    ##
    ## Coefficients:
    ## (Intercept)   ptl_catYes
    ##      -1.057        1.463
    ##
    ## Degrees of Freedom: 188 Total (i.e. Null);  187 Residual
    ## Null Deviance:       234.7
    ## Residual Deviance: 221.9     AIC: 225.9
    ##
    ## $all_models$model_3
    ##
    ## Call:  stats::glm(formula = fmla, family = family, data = model_data)
    ##
    ## Coefficients:
    ## (Intercept)   ptl_catYes          age
    ##     0.53225      1.60766     -0.07054
    ##
    ## Degrees of Freedom: 188 Total (i.e. Null);  186 Residual
    ## Null Deviance:       234.7
    ## Residual Deviance: 217.3     AIC: 223.3
    ##
    ## $all_models$model_4
    ##
    ## Call:  stats::glm(formula = fmla, family = family, data = model_data)
    ##
    ## Coefficients:
    ## (Intercept)   ptl_catYes          age        htYes
    ##     0.44082      1.63465     -0.07091      1.28704
    ##
    ## Degrees of Freedom: 188 Total (i.e. Null);  185 Residual
    ## Null Deviance:       234.7
    ## Residual Deviance: 213.1     AIC: 221.1
    ##
    ## $all_models$model_5
    ##
    ## Call:  stats::glm(formula = fmla, family = family, data = model_data)
    ##
    ## Coefficients:
    ## (Intercept)   ptl_catYes          age        htYes          lwt
    ##     2.04619      1.52899     -0.05648      1.82544     -0.01536
    ##
    ## Degrees of Freedom: 188 Total (i.e. Null);  184 Residual
    ## Null Deviance:       234.7
    ## Residual Deviance: 207.4     AIC: 217.4
    ##
    ## $all_models$model_6
    ##
    ## Call:  stats::glm(formula = fmla, family = family, data = model_data)
    ##
    ## Coefficients:
    ## (Intercept)   ptl_catYes          age        htYes          lwt        uiYes
    ##     1.74560      1.42123     -0.05331      1.90580     -0.01437      0.69193
    ##
    ## Degrees of Freedom: 188 Total (i.e. Null);  183 Residual
    ## Null Deviance:       234.7
    ## Residual Deviance: 205.2     AIC: 217.2
    ##
    ##
    ## $direction
    ## [1] "forward"
    ##
    ## $table
    ## a flextable object.
    ## col_keys: `model_id`, `model_terms`, `n_predictors`, `AIC`, `BIC`, `logLik`, `deviance`, `best`
    ## header has 1 row(s)
    ## body has 6 row(s)
    ## original dataset sample:
    ## 'data.frame':    6 obs. of  8 variables:
    ##  $ model_id    : int  1 2 3 4 5 6
    ##  $ model_terms : chr  "Intercept only" "ptl_cat" "ptl_cat + age" "ptl_cat + age + ht" ...
    ##  $ n_predictors: int  0 1 2 3 4 5
    ##  $ AIC         : num  237 226 223 221 217 ...
    ##  $ BIC         : num  240 232 233 234 234 ...
    ##  $ logLik      : num  -117 -111 -109 -107 -104 ...
    ##  $ deviance    : num  235 222 217 213 207 ...
    ##  $ best        : chr  "No" "No" "No" "No" ...

The selected direction is recorded in the formatted table footer. Backward and both-direction searches are available using the same interface.

    select_models(
      data = birthwt_data,
      outcome = low,
      exposures = exposures,
      approach = logit,
      direction = backward,
      format = tibble
    )$results_table

    ## # A tibble: 2 × 9
    ##   model_id formula          model_terms n_predictors   AIC   BIC logLik deviance
    ##      <int> <chr>            <chr>              <int> <dbl> <dbl>  <dbl>    <dbl>
    ## 1        1 low ~ age + lwt… age + lwt …            7  215.  244.  -98.4     197.
    ## 2        2 low ~ lwt + rac… lwt + race…            6  214.  240.  -98.9     198.
    ## # ℹ 1 more variable: selected_vars <chr>

    select_models(
      data = birthwt_data,
      outcome = low,
      exposures = exposures,
      approach = logit,
      direction = both,
      format = tibble
    )$results_table

    ## # A tibble: 6 × 9
    ##   model_id formula          model_terms n_predictors   AIC   BIC logLik deviance
    ##      <int> <chr>            <chr>              <int> <dbl> <dbl>  <dbl>    <dbl>
    ## 1        1 low ~ 1          Intercept …            0  237.  240.  -117.     235.
    ## 2        2 low ~ ptl_cat    ptl_cat                1  226.  232.  -111.     222.
    ## 3        3 low ~ ptl_cat +… ptl_cat + …            2  223.  233.  -109.     217.
    ## 4        4 low ~ ptl_cat +… ptl_cat + …            3  221.  234.  -107.     213.
    ## 5        5 low ~ ptl_cat +… ptl_cat + …            4  217.  234.  -104.     207.
    ## 6        6 low ~ ptl_cat +… ptl_cat + …            5  217.  237.  -103.     205.
    ## # ℹ 1 more variable: selected_vars <chr>

## What To Inspect

- `check_convergence()`: convergence status and maximum fitted probabilities. Use `format = gt` or `format = flextable` for viewing tables.
- `check_collinearity()`: VIF and interpretation. Nested model outputs keep their list structure when formatted.
- `plot_model_fit()`: residual, calibration, observed-versus-predicted, and influence plots for `lm`/`glm` models and stored `uni_reg()` / `multi_reg()` fitted models.
- `check_ph()`: Schoenfeld residual proportional hazards tests for Cox models, including term-level and global tests.
- `compare_models()`: AIC, BIC, log-likelihood, likelihood-ratio tests, sample size, events for survival models, and optional primary-exposure tracking for gtregression candidate models.
- `select_models()`: `$results_table`, `$best_model`, `$all_models`, and `$direction`; `$table` is added when `format = gt` or `format = flextable`.