Authentication setup for the Databricks extension for Visual Studio Code

This article describes how to set up authentication between the Databricks extension for Visual Studio Code and your Databricks workspace if you haven’t already configured the extension through project setup. See What is the Databricks extension for Visual Studio Code?.

The Databricks extension for Visual Studio Code implements portions of the Databricks client unified authentication standard, which enables you to configure Databricks authentication once and then use that configuration across multiple Databricks tools and SDKs without further authentication configuration changes.

Databricks recommends OAuth U2M authentication, which is easy to configure using the Databricks extension for Visual Studio Code. See Set up OAuth U2M authentication.

Otherwise if you are using Databricks personal access token authentication, Authenticate access to Databricks with a service principal using OAuth (OAuth M2M), Google Cloud credentials authentication, or Google Cloud ID authentication, see Set up authentication with a configuration profile.

Set up OAuth U2M authentication

Databricks supports OAuth user to machine (U2M) authentication. This enables you to generate short-lived (one hour) OAuth access tokens, which eliminates the risk of accidentally exposing longer-lived tokens such as Databricks personal access tokens through version control checkins or other means. This also enables better server-side session invalidation and scoping.

To complete the setup for OAuth U2M authentication, with your project and the extension opened:

  1. In the Configuration view, click Auth Type, and then click the gear (Sign in to Databricks workspace) icon.

    Sign in to Databricks workspace

  2. If you already have an authentication configuration profile in this list that has the Authenticate using OAuth (User to Machine) label and that you know corresponds to the target Databricks host, select it from the list, and then do the following:

    1. If prompted, complete any on-screen instructions in your web browser to finish authenticating with Databricks.

    2. If also prompted, allow all-apis access.

    3. Skip ahead to Select a cluster for running code and jobs.

  3. In the Command Palette, for Select authentication method, click OAuth (user to machine).

  4. For Select authentication method, select OAuth (user to machine).

  5. Enter some name for the associated Databricks authentication profile.

  6. In the Configuration view, click Login to Databricks.

    Login to Databricks

  7. In the Command Palette, for Select authentication method, select the name of the authentication configuration profile that you just created.

  8. If prompted, complete any on-screen instructions in your web browser to finish authenticating with Databricks. If also prompted, allow all-apis access.

  9. After you have successfully logged in, return to Visual Studio Code.

Set up authentication with a configuration profile

The following instructions assume that you have already set up a Databricks configuration profile with the required fields for your Databricks authentication type. To create a profile, see Databricks configuration profiles. To configure settings for the profile, see the Profile tab or section for your authentication type in Authenticate access to Databricks resources.

To select the configuration profile that you want the extension to use, with your project and the extension open, do the following:

  1. In the Configuration view, click Auth Type, and then click the gear (Sign in to Databricks workspace) icon.

    Sign in to Databricks workspace

  2. In the Command Palette, select your target Databricks configuration profile in the list.

    Tip

    The target workspace host can be changed later by modifying the target workspace setting in the databricks.yml configuration file associated with the project. See Databricks Asset Bundle configuration.

If you do not have a configuration profile, you can create one that authenticates by using a Databricks personal access token for example, as follows:

  1. With your project and the extension open, in the Configuration view, click Auth Type or Login to Databricks, and then click the gear (Sign in to Databricks workspace) icon.

    Sign in to Databricks workspace

  2. In the Command Palette, select Personal Access Token.

  3. Complete the on-screen prompts to do the following:

    1. Enter some name for the associated Databricks authentication profile.

    2. Generate the personal access token in your Databricks workspace and then copy the token’s value.

    3. Return to the extension and enter the copied token’s value.

The extension creates a folder in your project named .databricks with a file named databricks.env, if it does not already exist. This file contains the workspace host that you entered, along with other Databricks configuration details used by the Databricks extension for Visual Studio Code.

The extension also adds a hidden .gitignore file to the project if the file does not exist or if an existing .gitignore cannot be found in any parent folders. If a new .gitignore file is created, the extension adds a .databricks/ entry to this new file. If the extension finds an existing .gitignore file, it adds a .databricks/ entry to the existing file.