Manage service principals

This article explains how to create and manage service principals for your Databricks account and workspaces.

For an overview of the Databricks identity model, see Databricks identities.

What is a service principal?

A service principal is an identity that you create in Databricks for use with automated tools, jobs, and applications. Service principals give automated tools and scripts API-only access to Databricks resources, providing greater security than using users or groups.

You can grant and restrict a service principal’s access to resources in the same way as you can a Databricks user. For example, you can do the following:

Give a service principal account admin and workspace admin roles.
Give a service principal access to data, either at the account level using Unity Catalog, or at the workspace level.
Add a service principal to a group at both the account and workspace level, including the workspace admins group.

You can also grant Databricks users, service principals, and groups permissions to use a service principal. This allows users to run jobs as the service principal, instead of as their identity. This prevents jobs from failing if a user leaves your organization or a group is modified.

Unlike a Databricks user, a service principal is an API-only identity; it cannot be used to access the Databricks UI.

Note

Service Principals are managed within Databricks. They do not integrate with Google Cloud Service Accounts.

Databricks recommends that you enable your workspaces for identity federation so that you can manage your service principals in the account. If your workspace isn’t enabled for identity federation, you can create and manage service principals in your workspace.

Who can manage and use service principals?

To manage service principals in Databricks, you must have one of the following: the account admin role, the workspace admin role, or the manager or user role on a service principal.

Account admins can add service principals to the account and assign them admin roles. They can also assign service principals to workspaces, as long as those workspaces use identity federation.
Workspace admins can add service principals to a Databricks workspace, assign them the workspace admin role, and manage access to objects and functionality in the workspace, such as the ability to create clusters or access specified persona-based environments.
Service Principal Managers can manage roles on a service principal. The creator of a service principal becomes the service principal manager. Account admins are service principal managers on all service principals in an account.

Note

If a service principal was created before June 13, 2023, then the creator of the service principal does not have the service principal manager role by default. Ask an account admin to grant you the service principal manager role.

Service Principal Users can run jobs as the service principal. The job runs using the identity of the service principal, instead of the identity of the job’s owner. For more information, see Run a job as a service principal.

For information on how to grant the service principal manager and user roles, see Roles for managing service principals.

Manage service principals in your account

Account admins can add service principals to your Databricks account using the account console.

Add service principals to your account using the account console

As an account admin, log in to the account console.
In the sidebar, click User management.
On the Service principals tab, click Add service principal.
Enter a name for the service principal.
Click Add.

Assign account admin roles to a service principal

As an account admin, log in to the account console.
1. In the sidebar, click User management.
2. On the Service principals tab, find and click the username.
3. On the Roles tab, turn on Account admin or Marketplace admin.

To use service principals as account admins, you must generate Google ID tokens for them. See Authentication with Google ID tokens.

Assign a service principal to a workspace using the account console

To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page. For details, see Add a service principal to a workspace using the workspace admin settings.

As an account admin, log in to the account console.
In the sidebar, click Workspaces.
Click your workspace name.
On the Permissions tab, click Add permissions.
Search for and select the service principal, assign the permission level (workspace User or Admin), and click Save.

Remove a service principal from a workspace using the account console

To remove service principals from a workspace using the account console, the workspace must be enabled for identity federation.

As an account admin, log in to the account console
In the sidebar, click Workspaces.
Click your workspace name.
On the Permissions tab, find the service principal.
Click the kebab menu at the far right of the service principal row and select Remove.
On the confirmation dialog, click Remove.

Deactivate a service principal in your Databricks account

Account admins can deactivate service principals across a Databricks account. A deactivated service principal cannot authenticate to the Databricks account or workspaces. However, all of the service principal’s permissions and workspace objects remain unchanged. When a service principal is deactivated the following is true:

The service principal cannot authenticate to the account or any of their workspaces from any method.
Applications or scripts that use the tokens generated by the service principal are no longer able to access the Databricks API. The tokens remain but cannot be used to authenticate while a service principal is deactivated.
Clusters owned by the service principal remain running.
Scheduled jobs created by the service principal fail unless they are assigned to a new owner.

When a service principal is reactivated, they can login to Databricks with the same permissions. Databricks recommends deactivating service principals from the account instead of removing them because removing a service principal is a destructive action. See Remove service principals from your Databricks account. When you deactivate a service principal from the account, that service principal is also deactivated from their identity federated workspaces.

You cannot deactivate a user using the account console. Instead, use the Account Service Principals API. See Deactivate a service principal using the API.

Remove service principals from your Databricks account

Account admins can delete service principals from a Databricks account. Workspace admins cannot. When you delete a service principal from the account, that principal is also removed from their workspaces.

Important

When you remove a service principal from the account, that service principal is also removed from their workspaces, regardless of whether or not identity federation has been enabled. We recommend that you refrain from deleting account-level service principals unless you want them to lose access to all workspaces in the account. Be aware of the following consequences of deleting service principals:

Applications or scripts that use the tokens generated by the service principal can no longer access Databricks APIs
Jobs owned by the service principal fail
Clusters owned by the service principal stop
Queries or dashboards created by the service principal and shared using the Run as Owner credential have to be assigned to a new owner to prevent sharing from failing

To remove a service principal using the account console, do the following:

As an account admin, log in to the account console.
In the sidebar, click User management.
On the Service principals tab, find and click the username.
On the Principal Information tab, click the kebab menu in the upper-right corner and select Delete.
In the confirmation dialog box, click Confirm delete.

Manage service principals in your workspace

Workspace admins can manage service principals in their workspaces using the workspace admin settings page.

Add a service principal to a workspace using the workspace admin settings

As a workspace admin, log in to the Databricks workspace.
Click your username in the top bar of the Databricks workspace and select Admin Settings.
On the Service principals tab, click Add service principal.
Select an existing service principal to assign to the workspace or create a new one.

To create a new service principal, click the drop-down arrow in the search box and then click + Add new service principal.

Note

If your workspace is not enabled for identity federation, you cannot assign existing account service principals to your workspace.

Assign the workspace admin role to a service principal using the workspace admin settings page

As a workspace admin, log in to the Databricks workspace.
Click your username in the top bar of the Databricks workspace and select Admin Settings.
On the Groups tab, select the Admins group.
Click Add users or service principals.
Select the service principal and click Confirm.

To remove the workspace admin role from a service principal, remove the service principal from the admin group.

Manage workspace entitlements for a service principal

An entitlement is a property that allows a user, service principal, or group to interact with Databricks in a specified way. Entitlements are assigned to users at the workspace level. The following table lists entitlements and the workspace UI and API property name that you use to manage each one. You can use the workspace admin settings page and workspace-level SCIM REST APIs to manage entitlements.

Entitlement name (UI)	Entitlement name (API)	Default	Description
Workspace access	`workspace-access`	Granted by default.	When granted to a user or service principal, they can access the Data Science & Engineering and Databricks Machine Learning persona-based environments. Can’t be removed from workspace admins.
Databricks SQL access	`databricks-sql-access`	Granted by default.	When granted to a user or service principal, they can access Databricks SQL.
Allow unrestricted cluster creation	`allow-cluster-create`	Not granted to users or service principals by default.	When granted to a user or service principal, they can create clusters. You can restrict access to existing clusters using cluster-level permissions. Can’t be removed from workspace admins.
Allow pool creation (not available via UI)	`allow-instance-pool-create`	Can’t be granted to individual users or service principals.	When granted to a group, its members can create instance pools. Can’t be removed from workspace admins.

The users group is granted the Workspace access and Databricks SQL access entitlements by default. All workspace users and service principals are members of the users group. To assign these entitlements on a user-by-user basis, a workspace admin must remove the entitlement from the users group and assign it individually to users, service principals, and groups.

Important

To log in and access Databricks, a user must have the Databricks SQL access or Workspace access entitlement.

You cannot grant the allow-instance-pool-create entitlement using the admin settings page. Instead, use the Workspace Users, Service Principals, or Groups API.

Add or remove an entitlement for a user using the workspace admin settings page

As a workspace admin, log in to the Databricks workspace.
Click your username in the top bar of the Databricks workspace and select Admin Settings.
On the Service principals tab, select the service principal you want to update.
To add an entitlement, select the corresponding checkbox.
To remove an entitlement, deselect the corresponding checkbox.

Deactivate a service principal in your Databricks workspace

Workspace admins can deactivate service principals in a Databricks workspace. A deactivated service principal cannot access the workspace from Databricks APIs, however all of the service principal’s permissions and workspace objects remain unchanged. When a service principal is deactivated:

The service principal cannot authenticate to the workspaces from any method.
Applications or scripts that use the tokens generated by the service principal can no longer access the Databricks API. The tokens remain but cannot be used to authenticate while a service principal is deactivated.
Clusters owned by the service principal remain running.
Scheduled jobs created by the service principal have to be assigned to a new owner to prevent them from failing.

When a service principal is reactivated, they can authenticate to the workspace with the same permissions. Databricks recommends deactivating service principal instead of removing them because removing a service principal is a destructive action.

A deactivated service principal’s status is labeled Inactive in the workspace admin settings page.

You cannot deactivate a service principal using the workspace admin settings page. Instead, use the Workspace Service Principals API. See Deactivate a service principal in your Databricks workspace using the API.

Remove a service principal from a workspace using the workspace admin settings page

Removing a service principal from a workspace does not remove the service principal from the account. To remove a service principal from your account, see Remove service principals from your Databricks account.

As a workspace admin, log in to the Databricks workspace.
Click your username in the top bar of the Databricks workspace and select Admin Settings.
On the Service principals tab, find and click the name.
In the upper-right corner, click Delete.
Click Delete to confirm.

Manage service principals using the API

Account admins and workspace admins can manage service principals in the Databricks account and workspaces using Databricks APIs. To manage roles on a service principal using the API, see Manage service principal roles using the API.

Manage service principals in the account using the API

Admins can add and manage service principals in the Databricks account using the Account Service Principals API. Account admins and workspace admins invoke the API using a different endpoint URL:

Account admins use {account-domain}/api/2.0/accounts/{account_id}/scim/v2/.
Workspace admins use {workspace-domain}/api/2.0/account/scim/v2/.

For details, see the Account Service Principals API.

Deactivate a service principal using the API

Account admins can change a service principal’s status to false to deactivate the service principal using the Account Service Principals API.

For example:

curl --netrc -X PATCH \
https://<account-domain>/api/2.0/scim/v2/ServicePrincipals/<id> \
--header 'Content-type: application/scim+json' \
--data @update-sp.json \
| jq .

update-sp.json:

{
  "schemas": [ "urn:ietf:params:scim:api:messages:2.0:PatchOp" ],
  "Operations": [
    {
      "op": "replace",
      "path": "active",
      "value": [
        {
          "value": "false"
        }
      ]
    }
  ]
}

A deactivated service principal’s status is labeled Inactive in the account console.

Manage service principals in the workspace using the API

Account and workspace admins can use the Workspace Assignment API to assign service principals to workspaces enabled for identity federation. The Workspace Assignment API is supported through the Databricks account and workspaces.

Account admins use {account-domain}/api/2.0/accounts/{account_id}/workspaces/{workspace_id}/permissionassignments.
Workspace admins use {workspace-domain}/api/2.0/permissionassignments.

See Workspace Assignment API.

If your workspace is not enabled for identity federation, a workspace admin can use the workspace-level APIs to assign service principals to their workspaces. See Workspace Service Principals API.

Deactivate a service principal in your Databricks workspace using the API

Workspace admins can change a service principal’s status to false to deactivate the service principal using Workspace Service Principals API. For example:

curl --netrc -X PATCH \
https://<databricks-instance>/api/2.0/preview/scim/v2/ServicePrincipals/<id> \
--header 'Content-type: application/scim+json' \
--data @update-sp.json \
| jq .

update-sp.json:

{
  "schemas": [ "urn:ietf:params:scim:api:messages:2.0:PatchOp" ],
  "Operations": [
    {
      "op": "replace",
      "path": "active",
      "value": [
        {
          "value": "false"
        }
      ]
    }
  ]
}

A deactivated service principal’s status is labeled Inactive in the workspace admin settings page.

Manage tokens for a service principal

To authenticate a service principal to workspace-level APIs on Databricks, a workspace admin can create a Databricks personal access tokens on behalf of the service principal.

Grant the Can Use token permission to the service principal.
Create a Databricks personal access token on behalf of the service principal using the POST /token-management/on-behalf-of/tokens operation in the Token management API. An administrator can also list personal access tokens and delete them using the same API.

Note

It’s not possible to create, list, or manage a token for a service principal from within the Databricks UI.