Monitor usage using cluster and pool tags
To monitor cost and accurately attribute Databricks usage to your organization’s business units and teams (for chargebacks, for example), you can tag clusters and pools.
Note
Cluster tags and pool tags are available, both as defaults and for configuration, in this release of Databricks on Google Cloud, but because billable usage reports are not available for download or log delivery, you cannot yet view usage by tags.
Tagged objects and resources
You can add custom tags for the following objects managed by Databricks:
| Object | Tagging interface (UI) | Tagging interface (API) |
|---|---|---|
| Pool | Pool UI in the Databricks workspace | Instance Pool API |
| Cluster | Cluster UI in the Databricks workspace | Clusters API |
Databricks adds the following default tags to all pools and clusters:
| Pool tag key name | Value |
|---|---|
Vendor |
Constant “Databricks” |
DatabricksInstancePoolCreatorId |
Databricks internal identifier of the user who created the pool |
DatabricksInstancePoolId |
Databricks internal identifier of the pool |
| Cluster tag key name | Value |
|---|---|
Vendor |
Constant “Databricks” |
ClusterId |
Databricks internal identifier of the cluster |
ClusterName |
Name of the cluster |
Creator |
Username (email address) of the user who created the cluster |
On job clusters, Databricks also applies the following default tags:
| Cluster tag key name | Value |
|---|---|
RunName |
Job name |
JobId |
Job ID |
Tag propagation
Tags are propagated to node instances differently depending on whether or not a cluster was created from a pool.
If a cluster is created from a pool, its instances inherit only the custom and default pool tags, not the cluster tags. Therefore if you want to create clusters from a pool, make sure to assign all of the custom cluster tags you need to the pool.
If a cluster is not created from a pool, its tags propagate as expected to node instances.
Cluster and pool tags both propagate to DBU usage reports, whether or not the cluster was created from a pool.
If there is a tag name conflict, Databricks default tags take precedence over custom tags and pool tags take precedence over cluster tags.