Configure customer-managed keys for encryption

Preview

This feature is in Public Preview.

Account admins can use the Databricks account console to configure customer-managed keys for encryption. You can also configure customer-managed keys using the Account Key Configurations API.

There are two Databricks use cases for adding a customer-managed key:

  • Workspace storage (the two workspace storage buckets and the GCE Persistent Disk volumes of compute resources).

To compare the customer-managed key use cases, see Compare customer-managed keys use cases.

This feature requires the Premium plan.

What is an encryption keys configuration?

Customer-managed keys are managed with encryption keys configurations. Encryption keys configurations are account-level objects that reference your cloud’s key.

Account admins create encryption keys configurations in the account console and an encryption keys configuration can be attached to one or more workspaces.

You can share a Databricks key configuration object between the two different encryption use cases (managed services and workspace storage).

You can only add an encryption keys configuration to your Databricks workspace during workspace creation.

Step 1: Create or select a key in Cloud KMS

You can use the same Cloud KMS key between the workspace storage and managed services use cases.

  1. Create or select a symmetric key in Cloud KMS following the instructions in Create a symmetric encryption key. The Cloud KMS key must be in the same region as your workspace.

  2. Copy the Resource name for the KMS key.

Step 2: Create a new key configuration

Create a Databricks encryption key configuration object using the Databricks account console:

  1. As an account admin, log in to the account console.

  2. In the sidebar, click Cloud resources.

  3. Click the Encryption keys configuration tab.

  4. Click Add encryption key.

  5. Select the use cases for this encryption key:

    • Both managed services and workspace storage

    • Managed services

    • Workspace storage

  6. In the KMS key ID field, enter the resource name that you copied above.

  7. Click Add.

Step 3: Create a new workspace

Create a workspace using the encryption key configuration that you created. You must have the Google permissions cloudkms.cryptoKeys.getIamPolicy and cloudkms.cryptoKeys.setIamPolicy on the Cloud KMS key to create a workspace with the encryption key configuration.

Note

This section doesn’t show all options for creating a workspace. For more information on other advanced fields, such as those for customer-managed VPCs or custom CIDR values for networks, see Create a workspace using the account console

  1. Go to the account console.

  2. In the sidebar, click Workspaces.

  3. Click Create workspace.

  4. Set the following workspace fields:

    • Workspace name, enter a name for the workspace.

    • Region, select the same region as the Cloud KMS key.

    • Google cloud project ID, enter the project for the workspace’s compute resources, which might be different than the project ID for your Cloud KMS key.

  5. Set customer-managed key specific fields:

    1. Click Advanced configurations.

    2. Under Customer Managed Key, choose the encryption key configuration that you created in the previous steps for Encryption key configuration for managed services or Encryption key configuration for workspace storage or for both.

  6. Click Save.