Get identifiers for workspace objects

This article explains how to get workspace, cluster, directory, model, notebook, and job identifiers and URLs in Databricks.

Workspace instance names, URLs, and IDs

An instance name is assigned to each Databricks deployment. To segregate the workload and grant access to relevant users only, usually Databricks customers create separate instances for development, staging, and production. The instance name is the first part of the URL when you log into your Databricks deployment:

Workspace

In this case, the instance name is 8757561887652360.0.gcp.databricks.com.

A Databricks workspace is where the Databricks platform runs and where you can create Spark clusters and schedule workloads. Some types of workspaces have a unique workspace ID. If there is o= in the deployment URL, for example, https://<databricks-instance>/?o=6280049833385130, the random number after o= is the Databricks workspace ID. Here the workspace ID is 6280049833385130. If there is no o= in the deployment URL, the workspace ID is 0.

Cluster URL and ID

A Databricks cluster provides a unified platform for various use cases such as running production ETL pipelines, streaming analytics, ad-hoc analytics, and machine learning. Each cluster has a unique ID called the cluster ID. This applies to both all-purpose and job clusters. To get the details of a cluster using the REST API, the cluster ID is essential.

To get the cluster ID, click the Clusters tab in sidebar and then select a cluster name. The cluster ID is the number after the /clusters/ component in the URL of this page

https://<databricks-instance>/#/setting/clusters/<cluster-id>

In the following screenshot, the cluster ID is 0206-231114-irony170.

Cluster URL

Notebook URL and ID

A notebook is a web-based interface to a document that contains runnable code, visualizations, and narrative text. Notebooks are one interface for interacting with Databricks. Each notebook has a unique ID. The notebook URL has the notebook ID, hence the notebook URL is unique to a notebook. It can be shared with anyone on Databricks platform with permission to view and edit the notebook. In addition, each notebook command (cell) has a different URL.

To find a notebook URL or ID, open a notebook. To find a cell URL, click the contents of the command.

  • Example notebook URL:

    https://8757561887652360.0.gcp.databricks.com/?o=8757561887652360#notebook/1451020452210597`

  • Example notebook ID: 1451020452210597.

  • Example command (cell) URL:

    https://8757561887652360.0.gcp.databricks.com/?o=8757561887652360#notebook/1451020452210597/command/1451020452210598

Folder ID

A folder is a directory used to store files that can used in the Databricks workspace. These files can be notebooks, libraries or subfolders. There is a specific id associated with each folder and each individual sub-folder. The Permissions API refers to this id as a directory_id and is used in setting and updating permissions for a folder.

To retrieve the directory_id , use the Workspace API:

curl -n -X GET -H 'Content-Type: application/json' -d '{"path": "/Users/me@example.com/MyFolder"}' \
https://<databricks-instance>/api/2.0/workspace/get-status

This is an example of the API call response:

{
  "object_type": "DIRECTORY",
  "path": "/Users/me@example.com/MyFolder",
  "object_id": 123456789012345
}

Model ID

A model refers to an MLflow registered model, which lets you manage MLflow Models in production through stage transitions and versioning. The registered model ID is required for changing the permissions on the model programmatically through the Permissions API.

To get the ID of a registered model, you can use the Workspace API endpoint mlflow/databricks/registered-models/get. For example, the following code returns the registered model object with its properties, including its ID:

curl -n -X GET -H 'Content-Type: application/json' -d '{"name": "model_name"}' \
https://<databricks-instance>/api/2.0/mlflow/databricks/registered-models/get

The returned value has the format:

{
  "registered_model_databricks": {
    "name":"model_name",
    "id":"ceb0477eba94418e973f170e626f4471"
  }
}

Job URL and ID

A job is a way of running a notebook or JAR either immediately or on a scheduled basis.

To get a job URL, click Jobs Icon Workflows in the sidebar and click a job name. The job ID is after the text #job/ in the URL. The job URL is required to troubleshoot the root cause of failed job runs.

In the following screenshot, the job URL is

https://8757561887652360.0.gcp.databricks.com/?o=8757561887652360#job/2

In this example, the job ID is 2.

Job URL