# Connect to a Databricks Workspace

## Defining Credentials

The `{brickster}` package connects to a Databricks workspace in three ways:

1. OAuth user-to-machine (U2M) authentication, either directly or through credentials managed by the Databricks CLI
2. OAuth machine-to-machine (M2M) authentication
3. Personal Access Tokens (PAT)

It’s recommended to use option (1) when using `{brickster}` interactively. If you need to run code via an automated process, use option (2) or (3).

`{brickster}` will automatically detect when a session has Posit Workbench managed Databricks OAuth credentials enabled. For more information about this authentication flow see the section Posit Workbench Managed Databricks OAuth Credentials.

Personal Access Tokens can be generated in a few steps, for a step-by-step breakdown refer to the documentation.

Once you have a token you’ll be able to store it alongside the workspace URL in an `.Renviron` file. The `.Renviron` is used for storing the variables, such as those which may be sensitive (e.g. credentials) and de-couple them from the code additional reading.

To get started add the following to your `.Renviron`:

- `DATABRICKS_HOST`: The workspace URL

- `DATABRICKS_TOKEN`: Personal access token (not required if using OAuth U2M or M2M)

- `DATABRICKS_CLIENT_ID`: OAuth M2M client id (only required for OAuth M2M)

- `DATABRICKS_CLIENT_SECRET`: OAuth M2M client secret (only required for OAuth M2M)

- `ARM_CLIENT_ID`: Azure AD service principal application id (only required for Azure service principal OAuth M2M)

- `ARM_CLIENT_SECRET`: Azure AD service principal client secret (only required for Azure service principal OAuth M2M)

- `ARM_TENANT_ID`: Azure AD tenant id (only required for Azure service principal OAuth M2M)

- `DATABRICKS_WSID`: The workspace ID (docs)

`DATABRICKS_WSID` is only required for the RStudio IDE integration with the connection pane.

Example of entries in `.Renviron`:

    DATABRICKS_HOST=xxxxxxx.cloud.databricks.com
    DATABRICKS_TOKEN=dapi123456789012345678a9bc01234defg5
    DATABRICKS_WSID=123123123123123

For OAuth M2M:

    DATABRICKS_HOST=xxxxxxx.cloud.databricks.com
    DATABRICKS_CLIENT_ID=11111111-2222-3333-4444-555555555555
    DATABRICKS_CLIENT_SECRET=abcdefg1234567890
    DATABRICKS_WSID=123123123123123

For Azure service principal OAuth M2M:

    DATABRICKS_HOST=adb-xxxxxx.xx.azuredatabricks.net
    ARM_CLIENT_ID=11111111-2222-3333-4444-555555555555
    ARM_CLIENT_SECRET=abcdefg1234567890
    ARM_TENANT_ID=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee

With no `auth_type` in a selected `.databrickscfg` profile, `{brickster}` attempts Databricks OAuth M2M (`DATABRICKS_CLIENT_*`), then Azure service principal OAuth M2M (`ARM_*`), then OAuth U2M. When using `.databrickscfg`, set `auth_type = azure-client-secret` in the profile to force Azure service principal authentication.

Note: Recommend creating an `.Renviron` for each project. You can create `.Renviron` within your user home directory if required.

Restarting your R session will allow those variable to be picked up via the `{brickster}` package.

## Using Credentials with `{brickster}`

Authentication should now be possible without specifying the credentials in your R code. You can load `{brickster}` and list the clusters within the workspace using `db_cluster_list()`, to access the host/token use `db_host()`/`db_token()` respectively.

    library(brickster)

    # using db_host() and db_token() to get credentials
    clusters <- db_cluster_list(host = db_host(), token = db_token())

All `{brickster}` functions have their host/token parameters default to calling `db_host()`/`db_token()` therefore we can omit explicit calls to the functions.

    # all host/token parameters default to db_host()/db_token()
    clusters <- db_cluster_list()

When using OAuth U2M or OAuth M2M authentication you don’t define a token in `.Renviron` and therefore `db_token()` will return `NULL`.

## Managing Multiple Credentials

There are two methods that `{brickster}` supports to simplify switching of credentials within an R project/session:

1. Adding multiple credentials to `.Renviron`, each additional set of credentials is differentiated via a suffix (e.g. `DATABRICKS_TOKEN_DEV`)
2. Using a `.databrickscfg` file (primary method in Databricks CLI)

The `use_databrickscfg` option selects which configuration source `{brickster}` reads. It does not sign in to the Databricks CLI or select a named profile. The following example switches the session from `.Renviron` to `.databrickscfg`:

    # uses the CLI-selected default profile, or `DEFAULT` when none is selected
    options(use_databrickscfg = TRUE)

    # values are read from `.databrickscfg`
    db_host()
    db_token()

With its default value of `FALSE`, `{brickster}` does not discover profiles created by `databricks auth login` and does not enable the `databricks-cli` authentication provider. Set the option in `.Rprofile` when `.databrickscfg` should be the normal configuration source. Posit Workbench managed OAuth sessions enable `.databrickscfg` automatically.

### Switching Between Credentials

When `.databrickscfg` is enabled, the selected profile controls the host, credentials, authentication type, and the profile passed to `databricks auth token`. `{brickster}` resolves the profile in this order:

1. `DATABRICKS_CONFIG_PROFILE`
2. The `db_profile` option
3. The CLI default selected by `databricks auth switch`
4. The `DEFAULT` profile

`DATABRICKS_CONFIG_PROFILE` is the standard Databricks unified authentication setting. `db_profile` is the `{brickster}` session-level equivalent.

Profiles enable you to switch contexts between:

- Different workspaces (e.g. development or production)

- Different permissions (e.g. admin or restricted user)

This behaviour works when using credentials specified in either `.Renviron` or `.databrickscfg`:

    # using .Renviron
    db_host() # returns `DATABRICKS_HOST` (.Renviron)

    # switch profile to 'prod'
    options(db_profile = "prod")
    db_host() # returns `DATABRICKS_HOST_PROD` (.Renviron)

    # clear the session-specific profile selection
    options(db_profile = NULL)
    # use .databrickscfg
    options(use_databrickscfg = TRUE)
    db_host() # returns the CLI-selected default, otherwise `DEFAULT` (.databrickscfg)

    options(db_profile = "prod")
    db_host() # returns host from `prod` profile (.databrickscfg)

It is expected that profiles in `.Renviron` will adhere to the same naming convention as default but add an additional suffix.

Here is an example of an `.Renviron` file that has three profiles (default, dev, prod):

    # default
    DATABRICKS_HOST=xxxxxxx.cloud.databricks.com
    DATABRICKS_TOKEN=dapixxxxxxxxxxxxxxxxxxxxxxxxx
    DATABRICKS_WSID=123123123123123
    # dev
    DATABRICKS_HOST_DEV=xxxxxxx-dev.cloud.databricks.com
    DATABRICKS_TOKEN_DEV=dapixxxxxxxxxxxxxxxxxxxxxxxxx
    DATABRICKS_WSID_DEV=123123123123124
    # prod
    DATABRICKS_HOST_PROD=xxxxxxx-prod.cloud.databricks.com
    DATABRICKS_TOKEN_PROD=dapixxxxxxxxxxxxxxxxxxxxxxxxx
    DATABRICKS_WSID_PROD=123123123123125

### Configuring `.databrickscfg`

For details on configuring please refer to documentation from Databricks CLI.

There is only one `{brickster}` specific feature and it is the inclusion of `wsid` alongside `host`/`token`.

Supported OAuth fields in a `.databrickscfg` profile include:

- Databricks service principal fields: `client_id`, `client_secret`
- Azure service principal fields: `azure_client_id`, `azure_client_secret`, `azure_tenant_id`
- Optional auth mode override: `auth_type` (`oauth-m2m`, `azure-client-secret`, `oauth-u2m`, `databricks-cli`)

#### Databricks CLI OAuth profiles

Profiles created by `databricks auth login` use `auth_type = databricks-cli`. The login can be performed from any terminal; the credentials are not scoped to that terminal session. `{brickster}` must run as the same operating-system user and use the same `.databrickscfg` and CLI credential storage.

The `databricks-cli` provider is intentionally available only when the selected `.databrickscfg` profile contains `auth_type = databricks-cli`. Environment configuration cannot activate this provider independently.

Enable `.databrickscfg` and select the profile before making requests:

    options(
      use_databrickscfg = TRUE,
      db_profile = "e2-demo"
    )

    # The host and CLI token are resolved from the same profile.
    db_sql_warehouse_list()

Instead of `db_profile`, set `DATABRICKS_CONFIG_PROFILE`, or select the CLI default outside R:

    databricks auth switch --profile e2-demo

When a profile is selected, `{brickster}` calls `databricks auth token --profile <profile>`. Without a selected profile, `{brickster}` follows the Databricks SDK behavior and asks the CLI for the resolved host. If multiple profiles contain the same host, the CLI refuses to guess; select a profile rather than signing in again. Avoid supplying a different `host` argument while a profile is selected, because the request host and CLI credential would no longer describe the same configuration.

Token acquisition is deferred until an API request is performed. `{brickster}` keeps only the short-lived access token in memory until it expires, then calls the CLI again. The Databricks CLI remains responsible for durable credential storage and refresh. The CLI must be available on `PATH`, or its executable can be provided with `DATABRICKS_CLI_PATH`. `DATABRICKS_CONFIG_FILE` is inherited by the CLI when using a non-default config location.

The `oauth-u2m` mode remains `{brickster}`’s direct browser-based OAuth flow; it does not reuse credentials created by `databricks auth login`.

`wsid` is used by the connections pane integration in RStudio as the underlying API’s require it.

### Posit Workbench Managed Databricks OAuth Credentials

Posit Workbench has a managed Databricks OAuth credentials feature, which allows users to sign into a Databricks workspace from the home page of Workbench when launching a session and then access Databricks resources as their own identity. When in an RStudio Pro session running on Posit Workbench with managed Databricks OAuth credentials selected, `{brickster}` functions using `db_host()`/`db_token()` respectively should just work without needing to specify any credentials in your R code. See the code below as an example.

    library(brickster)
    db_cluster_list()

`{brickster}` will automatically detect when a session has Workbench managed OAuth credentials and then use the `workbench` profile defined in a `.databrickscfg` file at the `DATABRICKS_CONFIG_FILE` specified location. Workbench generates this `.databrickscfg` file in a temporary directory and should not be modified directly.

To use an alternative `.databrickscfg` file, a different `profile`, an alternative env variable `DATABRICKS_HOST` or set an env variable `DATABRICKS_TOKEN`, launch an RStudio Pro session without the Databricks managed credentials box selected.