September 2022

These features and Databricks platform improvements were released in September 2022.

Note

Releases are staged. Your Databricks account might not be updated until a week or more after the initial release date.

New reference solution for natural language processing

September 29, 2022

The documentation now includes a new reference solution for natural language processing. See Natural language processing.

Audit logs now include events for web terminal

September 22, 2022

Audit logs now include events related to web terminal. There are log entries for the start of a session and the close of a session. See Audit log reference.

Audit logs now include events for managing credentials for Git repos

September 22, 2022

Audit logs now include events related to management of credentials for Git repos. See Audit log reference.

Audit logs are GA

September 22, 2022

Databricks provides access to audit logs of activities performed by Databricks users. Audit logs are now generally available.

Cluster and pool tags now propagate to GCE labels

September 16, 2022

Cluster and pool tags now propagate to Google Compute Engine (GCE) labels, which is an improved way to attribute costs for compute resources to cost centers or projects. Previously, you could use Big Query to attribute costs by using custom and default tags with Google GKE usage metering, but tags did not propagate to the Google Cloud billing console.

You can continue to use GKE usage metering using these tags, but you now have the option of attributing costs using GCE labels, which are a more accurate aggregation of Google Cloud costs for all Databricks compute resources.

See Monitor usage using tags.

Select cluster policies directly in the Delta Live Tables UI

September 12-19, 2022

You can now select a cluster policy in the Delta Live Tables UI when you create or edit a pipeline. Previously, setting the cluster policy for a pipeline required editing the pipeline’s JSON settings.

New data trasformation card on workspace landing pages

September 8, 2022

The new Transform data card on the Data Science & Engineering and SQL landing pages displays data transformation options such as Delta Live Tables and dbt Core.

Delta cache is now disk cache

September 8, 2022

Databricks caching functionality previously referred to as the “Delta cache” has been renamed “disk caching” with no changes to existing behavior. See Optimize performance with caching on Databricks.

View and organize assets in the workspace browser across personas

September 7-21, 2022

You can now view and organize workspace assets such as notebooks, libraries, experiments, queries, and dashboards in the workspace browser across the Data Science & Engineering, Databricks Mosaic AI, and Databricks SQL personas.

New Databricks SQL queries, dashboards, and alerts are visible in the workspace browser. To view and organize existing queries, dashboards, and articles in the workspace browser, users (or admins) must migrate them into the workspace browser.

Databricks Runtime 11.2 and 11.2 ML are GA

September 7, 2022

Databricks Runtime 11.2 and 11.2 ML are now generally available.

See Databricks Runtime 11.2 (EoS) and Databricks Runtime 11.2 for Machine Learning (EoS).

Enhanced Spark egress network policy

September 6, 2022

When you use Databricks with GCP workloads, you can now expect enhanced network isolation with the addition of an enhanced egress policy for the GKE API server and its associated endpoints. The updated policy prevents egress traffic from Databricks Runtime containers to:

  • Databricks Runtime containers in another namespace

    • Databricks controllers

    • GKE API server endpoints