April 2024

These features and Databricks platform improvements were released in April 2024.

Note

Releases are staged. Your Databricks account might not be updated until a week or more after the initial release date.

Bind storage credentials and external locations to specific workspaces (Public Preview)

April 29, 2024

You can now bind storage credentials and external locations to specific workspaces, preventing access to those objects from other workspaces. This feature is especially helpful if you use workspaces to isolate user data access, for example if you have separate production and development workspaces or a dedicated workspace for handling sensitive data.

For more information, see (Optional) Assign an external location to specific workspaces and (Optional) Assign a storage credential to specific workspaces.

Data lineage now captures reads on tables with column masks and row-level security

April 25, 2024

Data lineage now captures reads on tables that include column masks and row-level security. See Filter sensitive table data using row filters and column masks and Capture and view data lineage using Unity Catalog.

New columns added to the billable usage system table (Public Preview)

April 24, 2024

The billable usage system table (system.billing.usage) now includes new columns that help you identify the specific product and features associated with the usage. See Billable usage system table reference.

Delta Sharing supports tables that use column mapping (Public Preview)

April 23, 2024

Delta Sharing now supports the sharing of tables that use column mapping. Recipients can read tables that use column mapping using a SQL warehouse, a cluster running Databricks Runtime 14.1 or above, or compute that is running open source delta-sharing-spark 3.1 or above.

See Add tables with deletion vectors or column mapping to a share, Read tables with deletion vectors or column mapping enabled, and Read tables with deletion vectors or column mapping enabled.

Jobs created through the UI are now queued by default

April 18, 2024

Queueing of job runs is now automatically enabled when a job is created in the Databricks Jobs UI. When queueing is enabled, and a concurrency limit is reached, job runs are placed in a queue until capacity is available. See What if my job cannot run because of concurrency limits?.

Lakehouse Federation supports foreign tables with case-sensitive identifiers

April 15, 2024

Lakehouse Federation is now able to federate foreign tables with case-sensitive identifiers for MySQL, SQL Server, BigQuery, Snowflake, and Postgres connections. See What is Lakehouse Federation.

Serverless SQL warehouses (Public Preview)

April 12, 2024

Serverless SQL warehouses are now in Public Preview. Using the Databricks serverless architecture, a serverless SQL warehouse supports all of the performance features of Databricks SQL. With a serverless SQL warehouse and its performance features, you get:

  • Rapid startup time (typically between 2 and 6 seconds).

  • Rapid upscaling to acquire more compute when needed for maintaining low latency.

  • Query admittance closer to the hardware’s limitation rather than the virtual machine.

  • Quick downscaling to minimize costs when demand is low, providing consistent performance with optimized costs and resources.

See Enable serverless SQL warehouses.

Delta Live Tables notebook developer experience improvements (Public Preview)

April 3, 2024

Databricks has released a set of Public Preview features in Databricks notebooks that assist in the development and debugging of Delta Live Tables code. See Notebook experience for Delta Live Tables development.