Sync users and groups from your identity provider

Preview

This feature is in Public Preview.

This article describes how to configure your identity provider (IdP) and Databricks to provision users and groups to Databricks using SCIM, or System for Cross-domain Identity Management, an open standard that allows you to automate user provisioning.

About SCIM provisioning in Databricks

SCIM lets you use an identity provider (IdP) to create users in Databricks, give them the proper level of access, and remove access (deprovision them) when they leave your organization or no longer need access to Databricks.

You can use a SCIM provisioning connector in your IdP or invoke the SCIM APIs to manage provisioning. You can also use these APIs to manage identities in Databricks directly, without an IdP.

Account-level and workspace-level SCIM provisioning

You can configure a SCIM provisioning connector from your identity provider to your Databricks account using account-level SCIM provisioning, or configure SCIM provisioning connectors to each workspace using workspace-level SCIM provisioning.

  • Account-level SCIM provisioning: You can use account-level SCIM provisioning to create, update, and delete users from the account.

  • Workspace-level SCIM provisioning: You can use workspace-level SCIM provisioning to create, update, and delete users from individual workspaces.

Requirements

To provision users and groups to Databricks using SCIM:

  • Your Databricks account must have the Databricks Premium Plan.

  • To provision users to your Databricks account using SCIM (including the SCIM REST APIs), you must be a Databricks account admin.

  • To provision users to a Databricks workspace using SCIM (including the SCIM REST APIs), you must be a Databricks workspace admin.

For more information about admin privileges, see Manage users, service principals, and groups.

You can have a maximum of 10,000 combined users and service principals and 5000 groups in an account. Each workspace can have a maximum of 10,000 combined users and service principals and 5000 groups.

Note

When you use SCIM provisioning, user and group attributes stored in your identity provider can override changes you make using the Databricks admin console, account console, or SCIM (Groups) API.

For example, if a user is assigned the Allow Cluster Creation entitlement in your identity provider and you remove that entitlement using the Databricks Admin Console, the user will be re-granted that entitlement the next time the IdP syncs with Databricks, if the IdP is configured to provision that entitlement. The same behavior applies to groups.

Provision identities to your Databricks account

You can use SCIM to provision users and groups from your identity provider to your Databricks account using a SCIM provisioning connector or directly using the SCIM APIs.

Add users and groups to your Databricks account using an IdP provisioning connector

You can sync users and groups from your IdP to your Databricks account using a SCIM provisioning connector.

If you configure Google Cloud Identity to federate with an external IdP, that IdP may have built-in SCIM integrations. Note that if you use Google Cloud Identity as your only IdP (you do not configure it to federate with an external IdP), there is no built-in SCIM integration.

To configure a SCIM connector to provision users and groups to your account:

  1. As an account admin, log in to the Databricks account console.

  2. Click User Settings Icon Settings.

  3. Click User Provisioning.

  4. Click Enable user provisioning.

    Copy the SCIM token and the Account SCIM URL. You will use these to configure your IdP.

  5. Log in to your IdP as a user who can configure a SCIM connector to provision users.

  6. Enter the following values in your IdP’s SCIM connector:

    • For the SAML provisioning URL, enter the SCIM URL you copied from Databricks.

    • For the provisioning API token, enter the SCIM token you copied from Databricks.

You can also follow these IdP-specific instructions for your IdP:

Add users, service principals, and groups to your account using the SCIM API

Account admins can add users, service principals, and groups to the Databricks account using the SCIM API for Accounts. Account admins call the API on accounts.gcp.cloud.databricks.com ({account_domain}/api/2.0/accounts/{account_id}/scim/v2/) and use a SCIM token.

To get the SCIM token, do the following:

  1. As an account admin, log in to the Databricks account console.

  2. Click User Settings Icon Settings.

  3. Click User Provisioning.

    If provisioning isn’t enabled, click Enable user provisioning and copy the token.

    If provisioning is already enabled, click Regenerate token and copy the token.

See SCIM API 2.0 (Accounts).

Rotate the account-level SCIM token

If the account-level SCIM token is compromised or if you have business requirements to rotate authentication tokens periodically, you can rotate the SCIM token.

  1. As a Databricks account admin, log in to the Databricks account console.

  2. Click User Settings Icon Settings.

  3. Click User Provisioning.

  4. Click Regenerate token. Make a note of the new token. The previous token will continue to work for 24 hours.

  5. Within 24 hours, update your SCIM application to use the new SCIM token.

Provision identities to a Databricks workspace

You can use SCIM to provision users and groups from your identity provider to your Databricks workspace using a SCIM provisioning connector or directly using the SCIM APIs.

Add users and groups to your workspace using an IdP provisioning connector

If you configure Google Cloud Identity to federate with an external IdP, that IdP may have built-in SCIM integrations. Note that if you use Google Cloud Identity as your only IdP (you do not configure it to federate with an external IdP), there is no built-in SCIM integration.

Follow the instructions in the appropriate IdP-specific article:

Add users, groups, and service principals to your workspace using the SCIM API

Workspace admins can add users, groups, and service principals to the Databricks account using the SCIM APIs for workspaces. See SCIM API 2.0.