Manage identities, permissions, and privileges for Delta Live Tables pipelines

This article provides an overview of identities, permissions, and privileges for Delta Live Tables pipelines.

Databricks recommends using Unity Catalog for all new Delta Live Tables pipelines. By default, materialized views and streaming tables created by pipelines configured with Unity Catalog can only be queried by the pipeline owner. See Use Unity Catalog with your Delta Live Tables pipelines.

If your pipelines publish datasets to legacy Hive metastore, see Use Delta Live Tables pipelines with legacy Hive metastore.

For general best practices on identity configurations, see Identity best practices.

What identity is used for pipeline updates?

Delta Live Tables pipelines process updates using the identity of the pipeline owner. Assign a new pipeline owner to change the identity used to run the pipeline.

Databricks recommends setting a service principal as the pipeline owner. See Manage service principals.

Who can run a pipeline update?

Pipeline updates can be run by any user or service principal with CAN RUN, CAN MANAGE, or IS OWNER permissions.

Configure pipeline permissions

You must have the CAN MANAGE or IS OWNER permission on the pipeline to manage permissions. Pipelines use access control lists (ACLs) to control permissions. For a complete list of permissions and their abilities, see Delta Live Tables pipeline ACLs.

  1. In the sidebar, click Delta Live Tables.

  2. Select the name of a pipeline.

  3. Click the Kebab menu, and select Permissions.

  4. In Permissions Settings, select the Select User, Group or Service Principal… drop-down menu and then select a user, group, or service principal.

  5. Select a permission from the permission drop-down menu.

  6. Click Add.

  7. Click Save.

Allow non-admin users to view the driver logs from a Unity Catalog-enabled pipeline

By default, only the pipeline owner and workspace admins can view the driver logs from the cluster that runs a Unity Catalog-enabled pipeline. You can enable access to the driver logs for any user with CAN MANAGE, CAN VIEW, or CAN RUN permissions by adding the following Spark configuration parameter to the configuration object in the pipeline settings:

{
  "configuration": {
    "spark.databricks.acl.needAdminPermissionToViewLogs": "false"
  }
}