Author: Thierry Warin

# Getting started with statcanR

#### Thierry Warin

## Overview

`statcanR` connects R to Statistics Canada’s Web Data Service (WDS). This vignette walks through a complete workflow: describing the data you need, choosing a table, understanding its identifier, downloading it, inspecting the result, and optionally saving a CSV copy.

The package supports four common tasks:

1. Find likely tables from an ordinary-language description.
2. Search for exact keywords in the official table catalogue.
3. Download a complete table in English or French into R.
4. Save the downloaded table as a CSV file.

The four public functions have distinct purposes:

Function | Use it when… | Result
--- | --- | ---
`statcan_find()` | You can describe the subject, geography, or period you need | A ranked data frame of likely tables and reasons for each match
`statcan_search()` | You know words that occur in the official table title | An interactive table of exact keyword matches
`statcan_data()` | You want the complete table in R | A data frame
`statcan_download_data()` | You want the data frame and a CSV copy | A data frame with the saved file path attached

The download functions retrieve a complete table, not a filtered selection of observations. A Statistics Canada table can be large. It is therefore useful to identify the right table before starting the download.

An optional fifth function, `statcan_chat()`, wraps `statcan_find()` with a user-configured language model to explain candidates and ask clarifying questions; see Optional: ask a language model for help.

## Two concepts to know first

### Table number and Product ID

Statistics Canada displays table numbers such as `10-10-0001-01`. The first eight digits form the WDS Product ID (PID), `10100001`; the final `01` identifies the displayed view. `statcanR` accepts either the displayed table number or the eight-digit PID:

    table_data <- statcan_data("10-10-0001-01", "eng")
    table_data <- statcan_data("10100001", "eng")

These two calls request the same table.

### Language

Use `lang = "eng"` for an English table and `lang = "fra"` for a French table. The language controls the table contents and labels returned by Statistics Canada; it is not a translation performed by `statcanR`.

## Install or upgrade

The command used for a first installation also upgrades an older CRAN installation:

    install.packages("statcanR")

Then load the package:

    library(statcanR)

If the package was loaded while you upgraded it, restart the R session before calling `library(statcanR)` again. Check which version R will use with:

    packageVersion("statcanR")

Version 0.3.0 preserves the familiar calls from earlier releases. In particular, code that supplies a table number and a language to `statcan_data()` or `statcan_download_data()` continues to work.

## Step 1: describe the table you need

Use `statcan_find()` when you know what data you want but do not know the official title or table number. Write a short request containing as much of the subject, Canadian geography, and period as you know:

    matches <- statcan_find(
      "R&D expenditures in Quebec since 2020",
      lang = "eng",
      n = 5
    )

    matches[, c("title", "id", "score", "match_reason")]

This request is interpreted as:

- a subject: research and development expenditures;
- a geography: Quebec; and
- a date requirement: the table must include 2020.

The result is an ordinary data frame. `rank` orders the candidates, `score` summarizes the strength of each match, and `match_reason` states which clues were confirmed. The score helps with discovery; it is not a measure of data quality. Read the candidate titles because two tables can represent different valid interpretations of the same short request.

When a province or territory appears in the query, `statcan_find()` checks WDS metadata to confirm that the table includes it. `start_date` and `end_date` describe the coverage of the whole table. These checks help choose a table; they do not select rows. The download functions still retrieve the complete table, after which you filter its geography and reference-period columns.

French requests and French catalogue titles are supported too:

    matches_fr <- statcan_find(
      "Dépenses de R-D au Québec depuis 2020",
      lang = "fra"
    )

The parser recognizes common wording and abbreviations, but it is deliberately simple and predictable. A concise request generally works better than a long question. If the results are too broad, add a more specific subject term. If there are no results, remove a detail or try an official keyword.

### Search exact title keywords

Use `statcan_search()` when you know words that occur in the official title. It searches titles without regard to letter case:

    statcan_search(
      c("federal", "expenditures", "objectives"),
      lang = "eng"
    )

The result is an interactive table displayed in the RStudio Viewer or a browser. The most important columns are:

- `title`: the official table title;
- `id`: the table number to pass to a download function;
- `release_date`: the catalogue release date; and
- `lang`: the language searched.

When several keywords are supplied, all of them must occur in the title. This makes searches precise, but it can also produce no matches. If that happens, remove one keyword or use a broader term:

    statcan_search("expenditures", lang = "eng")

Search French titles by using `lang = "fra"`:

    statcan_search(c("dépenses", "fédérales"), lang = "fra")

The catalogue is cached for 24 hours in R’s platform-appropriate user cache directory. `statcan_find()` caches the table metadata used for geography checks for seven days. These caches make repeated searches faster and avoid unnecessary requests to Statistics Canada. Set `refresh = TRUE` only when you specifically need fresh information:

    statcan_find(
      "population in Alberta since 2021",
      lang = "eng",
      refresh = TRUE
    )

If WDS is temporarily unavailable, catalogue searches use the most recent valid cache and issue a warning. If geography metadata cannot be refreshed, `statcan_find()` uses valid cached metadata where possible. Candidates whose geography could not be checked have `geography_match = NA`; the match explanation makes that uncertainty explicit.

### Optional: ask a language model for help

`statcan_chat()` is an entirely optional layer on top of `statcan_find()`. It sends your query and `statcan_find()`’s ranked candidates to a language model you configure, which explains which candidate best matches and asks a clarifying question when the query is ambiguous. The language model never sees or chooses a table number of its own; it only explains the candidates `statcan_find()` already returned.

This feature adds no new package dependencies, and it never makes a network request unless you call `statcan_chat()` yourself. Two providers ship built in, selected with the `provider` argument: `"openai"` (the default) and `"anthropic"` (Claude). The endpoint defaults to the chosen provider and must use `https://` so the key is never sent in cleartext (plain `http://` is accepted only for a loopback host such as `http://localhost`, for a local model). Supply the API key through the provider’s environment variable (`OPENAI_API_KEY` or `ANTHROPIC_API_KEY`, or the generic `STATCANR_LLM_API_KEY`, or the `api_key` argument), since a secret should not be kept in `options()`:

    Sys.setenv(OPENAI_API_KEY = "sk-...")        # OpenAI
    Sys.setenv(ANTHROPIC_API_KEY = "sk-ant-...") # Anthropic (Claude)

Loading the package with `library(statcanR)` never requests these settings, and no key is needed for `statcan_find()` or `statcan_data()`; the key is read only when you call `statcan_chat()`. To set it once and reuse it across sessions without retyping it, add it to your `~/.Renviron` file (open it with `usethis::edit_r_environ()`, then restart R) instead of calling `Sys.setenv()` each time:

    OPENAI_API_KEY=sk-...
    ANTHROPIC_API_KEY=sk-ant-...

R reads `.Renviron` automatically at startup, so the key stays out of your scripts and `.Rhistory`.

    # OpenAI (the default provider)
    statcan_chat("R&D expenditures in Quebec since 2020", model = "gpt-4o-mini")

    # Anthropic (Claude)
    statcan_chat(
      "R&D expenditures in Quebec since 2020",
      provider = "anthropic", model = "claude-opus-4-8"
    )

The `"openai"` provider also covers any OpenAI-compatible server — Groq, Together, OpenRouter, Mistral, vLLM, or a local open-source model served by Ollama or LM Studio — by pointing `endpoint` at it.

A single `statcan_chat()` call is one turn. To answer its `clarifying_question` and keep the discussion going, use `statcan_chat_continue()`. Each follow-up stays scoped to the same candidate tables the first call found (it never re-runs `statcan_find()`, and the model still never invents a table number), and results are chainable. Reassign the same variable each turn — there is no need for `r1`/`r2`/`r3`-style names:

    chat <- statcan_chat("R&D spending in Quebec", provider = "anthropic", model = "claude-opus-4-8")
    chat$clarifying_question
    chat <- statcan_chat_continue(chat, "annual data, since 2015")
    chat <- statcan_chat_continue(chat, "just the total, not by industry")

To search the catalogue again, start a fresh `statcan_chat()` call rather than continuing.

## Step 2: download a complete table

After choosing an identifier, pass it and the desired language to `statcan_data()`. This example uses a relatively small table that is convenient for learning:

    table_data <- statcan_data("10-10-0001-01", lang = "eng")

The function downloads and unpacks the current full-table CSV archive, then returns a data frame. Start by examining its dimensions, names, and first observations:

    dim(table_data)
    names(table_data)
    head(table_data)

The exact columns depend on the table selected. `statcanR` applies a few consistent rules:

- the first column is named `REF_DATE`;
- annual, monthly, daily, and fiscal reference periods are converted to `Date` values when they can be interpreted safely;
- coordinate columns remain character values so leading zeros and compound coordinates are not lost; and
- `INDICATOR` contains the official table title read from its metadata.

For example, you can select observations from 2020 onward with ordinary R subsetting:

    recent_data <- table_data[
      !is.na(table_data$REF_DATE) &
        table_data$REF_DATE >= as.Date("2020-01-01"),
    ]

To download the French version of the table, change the language:

    table_fr <- statcan_data("10-10-0001-01", lang = "fra")

Most source column names remain in the selected language, so do not assume that every English column name has an identical French equivalent.

## Step 3: save a CSV copy when needed

If you only need to analyse the data in the current R session, `statcan_data()` is sufficient. Use `statcan_download_data()` when you also need a CSV file.

Existing two-argument calls save the file in the current working directory:

    table_data <- statcan_download_data("10-10-0001-01", "eng")
    getwd()

This creates `statcan_10100001_eng.csv`. To keep project files organized, create a dedicated directory and pass it through `path`:

    output_dir <- file.path(tempdir(), "statcanR-data")
    dir.create(output_dir, recursive = TRUE, showWarnings = FALSE)

    table_data <- statcan_download_data(
      "10-10-0001-01",
      "eng",
      path = output_dir
    )

    attr(table_data, "statcan_file")

The output directory must already exist. The function returns the same data frame as `statcan_data()` and stores the exact CSV path in its `statcan_file` attribute. The CSV uses UTF-8 encoding, excludes R row names, and writes missing values as empty fields.

## Compatibility with earlier statcanR scripts

The update does not require you to rewrite established calls:

    # This familiar two-argument form remains valid.
    table_data <- statcan_data("10-10-0001-01", "eng")

    # This also remains valid and saves into the working directory.
    table_data <- statcan_download_data("10-10-0001-01", "eng")

The optional `path` argument extends `statcan_download_data()` without changing the meaning of the original arguments. Both hyphenated table numbers and eight-digit PIDs are accepted.

## Troubleshooting

The package validates inputs before downloading and reports network or service problems explicitly. Common issues include:

Message or symptom | What to check
--- | ---
No natural-language results | Keep a clear subject, but remove a geography or date constraint; then inspect broader candidates
No exact keyword results | Try fewer keywords, check the selected language, or use a broader official term
Invalid `tableNumber` | Use a displayed number such as `10-10-0001-01` or an eight-digit PID such as `10100001`
Invalid `lang` | Use exactly `"eng"` or `"fra"`
Output directory does not exist | Create the directory before supplying it through `path`
WDS is unavailable | Check the internet connection and try again later; catalogue search may use a valid cache
Download takes a long time | The function retrieves the complete table, which may be large

Network failures, invalid tables, unexpected API responses, and malformed archives stop with messages that identify the affected Product ID. Temporary files created by a call are removed when it finishes; other files in the R session’s temporary directory are left untouched.

## Reproducible use

WDS provides the current version of a Statistics Canada table, and published observations may be revised. For work that must be reproduced later:

1. record the table identifier, language, package version, and retrieval date;
2. save a local CSV copy of the data used in the analysis; and
3. cite the table and Statistics Canada according to the applicable data licence.

## Data licence and citation

Review the Statistics Canada Open Licence before redistributing downloaded data. To obtain the package’s current citation, run:

    citation("statcanR")