November 2024
These features and Databricks platform improvements were released in November 2024.
Note
Releases are staged. Your Databricks account might not be updated until a week or more after the initial release date.
Expanded support for Python debugger in notebooks
November 18, 2024
The built-in Python debugger in Databricks notebooks is now supported on serverless compute and compute configured with shared access mode in Databricks Runtime 14.3 LTS or above. To learn more about the debugger, see Debug notebooks.
Improved web terminal experience
November 18, 2024
Databricks made the following improvements to the web terminal experience:
Support for configuration files: You can now set persistent configurations for your Databricks web terminal with .bashrc configuration files. See Configure your web terminal.
Command cycle history: You can now cycle through commands executed in previous sessions with the “Up” and “Down” arrow keys. This makes it easy to recall and reuse previous commands without retyping them.
Shared Cluster Support: Our improved web terminal is available on shared clusters in Databricks Runtime 15.1 and above.
To learn more about using the web terminal, see Run shell commands in Databricks web terminal.
Query history and query profile support for Delta Live Tables
November 15, 2024
Databricks now offers improved tools for monitoring query performance in Delta Live Tables pipelines. See Access query history for Delta Live Tables pipelines.
Reuse SQL outputs as _sqldf
in subsequent SQL cells
November 15, 2024
In Databricks Runtime 14.3 and above, running a SQL query in a Databricks notebook generates an implicit DataFrame (_sqldf
) that you can use in downstream SQL cells. This adds to the support for using for using the _sqldf
variable in Python cells introduced in Databricks Runtime 13.3. See Explore SQL cell results.
Warehouses system table is now available (Public Preview)
November 15, 2024
The system.compute.warehouses
table records SQL warehouse configurations. Each row is a snapshot of a SQL warehouse’s properties at a specific point in time. A new snapshot is created when the properties change. See Warehouses system table reference.
Cross-platform view sharing (Public Preview)
November 14, 2024
Databricks recipients can now query shared views using any Databricks compute resource. Previously, if a recipient’s Databricks account was different from the provider’s account, recipients could only query a shared view using a serverless SQL warehouse. See Read shared views.
View sharing also now extends to open sharing connectors. See Read data shared using Delta Sharing open sharing (for recipients).
Databricks Runtime 16.0 is GA
November 11, 2024
Databricks Runtime 16.0 and Databricks Runtime 16.0 ML are now generally available.
See Databricks Runtime 16.0 and Databricks Runtime 16.0 for Machine Learning.
Breaking change: Hosted RStudio is end-of-life
November 11, 2024
In Databricks Runtime 16.0 and above, Databricks-hosted RStudio Server is end-of-life and unavailable on any Databricks workspace. To learn more and see a list of alternatives to RStudio, see Hosted RStudio Server deprecation.
Updated and unified read-only code blocks
November 8, 2024
A new read-only code block is now available across the Catalog Explorer, alerts page, and Genie.
Increased Quick Fix recommendations
November 8, 2024
Databricks Assistant Quick Fix now identifies and suggests corrections for a broader range of common SQL and Python errors. With a 50% increase in error detection, Quick Fix can help you debug your code faster. See Quick Fix.
Databricks Runtime 14.2 series support ends
November 5, 2024
Support for Databricks Runtime 14.2 and Databricks Runtime 14.2 for Machine Learning ended on November 5. See Databricks support lifecycles.
Serverless compute for workflows (Public Preview)
November 4, 2024
Serverless compute for workflows is now available in Public Preview. Serverless compute for workflows allows you to run your Databricks job without configuring and deploying infrastructure. With serverless compute for workflows, Databricks efficiently manages the compute resources that run your job, including optimizing and scaling compute for your workloads. See Run your Databricks job with serverless compute for workflows.
Serverless compute for notebooks (Public Preview)
November 4, 2024
Serverless compute for notebooks is now available in Public Preview. Serverless compute for notebooks gives you on-demand access to scalable compute in notebooks, letting you immediately write and run your Python or SQL code. See Serverless compute for notebooks.