Configure access to Azure storage with an Azure Active Directory service principal

Registering an application with Azure Active Directory (Azure AD) creates a service principal you can use to provide access to Azure storage accounts. You can then configure access to these service principals using credentials stored with secrets.

Databricks recommends using Azure Active Directory service principals scoped to clusters or Databricks SQL endpoints to configure data access. See Accessing Azure Data Lake Storage Gen2 and Blob Storage with Databricks and Configure access to cloud storage.

Register an Azure Active Directory application

Registering an Azure AD application and assigning appropriate permissions will create a service principal that can access Azure Data Lake Storage Gen2 or Blob Storage resources.

  1. In the Azure portal, go to the Azure Active Directory service.

  2. Under Manage, click App Registrations.

  3. Click + New registration. Enter a name for the application and click Register.

  4. Click Certificates & Secrets.

  5. Click + New client secret.

  6. Add a description for the secret and click Add.

  7. Copy and save the value for the new secret.

  8. In the application registration overview, copy and save the Application (client) ID and Directory (tenant) ID.

Databricks recommends storing these credentials using secrets.

Assign roles

You control access to storage resources by assigning roles to an Azure AD application registration associated with the storage account. This example assigns the Storage Blob Data Contributor to an Azure storage account. You may need to assign other roles depending on specific requirements.

  1. In the Azure portal, go to the Storage accounts service.

  2. Select an Azure storage account to use with this application registration.

  3. Click Access Control (IAM).

  4. Click + Add and select Add role assignment from the dropdown menu.

  5. Set the Select field to the Azure AD application name and set Role to Storage Blob Data Contributor.

  6. Click Save.