August 2023
These features and Databricks platform improvements were released in August 2023.
Note
Releases are staged. Your Databricks workspace might not be updated until a week or more after the initial release date.
Databricks CLI updated to version 0.203.3 (Public Preview)
August 30, 2023
The Databricks command-line interface (Databricks CLI) has been updated to version 0.203.3. For details, see the changelog for version 0.203.3.
Databricks JDBC driver 2.6.34
August 30, 2023
We have released version 2.6.34 of the Databricks JDBC driver (download). This release addresses a number of vulnerability issues by upgrading third-party libraries. In addition, this release fixes the following issues:
Thread leakage when a query fails to connect to the server.
Missing comments in the
REMARKS
column of the table metadata.When a resultset closure operation returns an error, the connector does not clean up the operation handle entries from the heartbeat thread.
The release also improves error message handling by honoring SQLState
and error codes from the server, and checks the server protocol version correctly.
New Google Cloud region: northamerica-northeast1 (Montréal, Canada)
August 31, 2023
Databricks on Google Cloud is now available in the northamerica-northeast1
(Montréal, Canada) region.
Databricks SDK for Go updated to version 0.18.0 (Beta)
August 29, 2023
Databricks SDK for Go version 0.18.0 introduces 17 breaking API changes, adds 30 APIs, and adds about a dozen fields. For details, see the changelog for version 0.18.0.
Databricks SDK for Python updated to version 0.7.0 (Beta)
August 29, 2023
Databricks SDK for Python version 0.7.0 introduces a breaking change for the files API as well as 17 additional breaking changes. This release also adds 14 fields and adds 27 dataclasses. For details, see the changelog for version 0.7.0.
Databricks Terraform provider updated to version 1.24.1
August 29, 2023
Databricks Terraform provider version 1.24.1 includes a fix for verification of workspace reachability. For details, see the changelog for version 1.24.1.
Init scripts on DBFS end of life extended to Dec 1, 2023
August 29, 2023
Databricks previously announced that init scripts on DBFS would reach end of life on Sept 1, 2023. The end of life date has been extended to Dec 1, 2023.
Databricks Runtime 14.0 (Beta)
August 29, 2023
Databricks Runtime 14.0 and Databricks Runtime 14.0 ML are now available as Beta releases.
See Databricks Runtime 14.0 (EoS) and Databricks Runtime 14.0 for Machine Learning (EoS).
Databricks Terraform provider updated to version 1.24.0
August 25, 2023
Databricks Terraform provider version 1.24.0 adds account-level API support for Unity Catalog objects, adds acl_principal_id
to several data sources, and more. For details, see the changelog for version 1.24.0.
Databricks CLI updated to version 0.203.2 (Public Preview)
August 24, 2023
The Databricks command-line interface (Databricks CLI) has been updated to version 0.203.2. For details, see the changelog for version 0.203.2.
Go to definition for functions and variables in Python notebooks
August 24, 2023
In Databricks Runtime 12.2 LTS and above, when a Python notebook is attached to a cluster, you can automatically navigate to the definition of a variable or function that is defined in that notebook. See Go to definition for details.
Databricks Runtime 13.3 LTS is GA
August 22, 2023
Databricks Runtime 13.3 LTS and Databricks Runtime 13.3 LTS ML are now generally available.
See Databricks Runtime 13.3 LTS and Databricks Runtime 13.3 LTS for Machine Learning.
Introducing tags with Unity Catalog
August 22, 2023
You can now apply tags to your securable objects in Unity Catalog. Tags help you organize securable objects and simplify search and discovery of your data assets. See Apply tags to Unity Catalog securable objects.
Volumes support for JARs is in Public Preview
August 23, 2023
You can now use Unity Catalog volumes to store JARs on compute with assigned or shared access modes running Databricks Runtime 13.3 and above. See Install libraries from a volume.
Easier Databricks Repos .ipynb file output commits
August 21, 2023
Databricks Repos has added features to make .ipynb file output commits easier to control and see:
Repos can create an commit_output config file for you.
Notifications in the UI let you know the status of output commits.
From the File menu, you can control whether noteook outputs are committed.
IPYNB notebook support in Databricks Repos is GA
August 21, 2023
IPYNB notebook file support in Databricks Repos is GA. For more information about Repos, see:
Databricks SDK for Go updated to version 0.17.0 (Beta)
August 18, 2023
Databricks SDK for Go version 0.17.0 adds over 30 APIs and renames about 10 APIs. For details, see the changelog for version 0.17.0.
Databricks SDK for Python updated to version 0.6.0 (Beta)
August 18, 2023
Databricks SDK for Python version 0.6.0 adds about 20 dataclasses, adds about 10 APIs, and includes about 20 renames. For details, see the changelog for version 0.6.0.
Databricks CLI updated to version 0.203.1 (Public Preview)
August 16, 2023
The Databricks command-line interface (Databricks CLI) has been updated to version 0.203.1. For details, see the changelog for version 0.203.1.
Unified schema browser is now GA
August 16, 2023
A unified way to navigate your data is now available for notebooks, SQL Editor, and Catalog Explorer, including the following new capabilities:
Condensed version of the tree.
Display of Unity Catalog object count at the schema level when filtering.
Right-click menu to view actions.
Databricks SDK for Go updated to version 0.16.0 (Beta)
August 14, 2023
Databricks SDK for Go version 0.16.0 adds about a dozen fields, removes about a half-dozen fields, and changes the signatures of about a half-dozen APIs. For details, see the changelog for version 0.16.0.
Partner Connect supports Census
August 14, 2023
You can now use Partner Connect to connect your Databricks workspace to Census. For more information, see Connect to Census.
Databricks SDK for Python updated to version 0.5.0 (Beta)
August 11, 2023
Databricks SDK for Python version 0.5.0 changes the signatures of a half-dozen APIs, adds a dozen fields, removes a half-dozen fields, and adds a few dataclasses. For details, see the changelog for version 0.5.0.
Programmatic write support for workspace files
August 11, 2023
Clusters using Databricks Runtime 11.2 and above support writing to workspace files. You can programmatically create, update, and delete files and directories within your workspace.
Databricks CLI updated to version 0.203.0 (Public Preview)
August 10, 2023
The Databricks command-line interface (Databricks CLI) version 0.203.0 improves the auth login
command, and more. For details, see the changelog for version 0.203.0.
Databricks Terraform provider updated to version 1.23.0
August 9, 2023
Databricks Terraform provider version 1.23.0 extends RunJobTask
with additional supported fields in databricks_job
, adds exporter support for databricks_model_serving
, and more. For details, see the changelog for version 1.23.0.
Groups can now be renamed
August 8, 2023
You can now rename existing account groups and workspace-local groups. See Change the name of a group.
Databricks SDK for Go updated to version 0.15.0 (Beta)
August 7, 2023
Databricks SDK for Go version 0.15.0 refactors permissions-related APIs to make them more understandable and easier to use. For details, see the changelog for version 0.15.0.
Databricks SDK for Python updated to version 0.4.0 (Beta)
August 7, 2023
Databricks SDK for Python version 0.4.0 refactors permissions-related APIs to make them more understandable and easier to use. For details, see the changelog for version 0.4.0.
Third-party iFraming prevention configuration setting was removed
August 7, 2023
Databricks reduces the risk of third-party domains from iFraming Databricks by adding appropriate HTTP headers. Third-party iFraming prevention is still enabled by default but Databricks removed the workspace admin setting to disable it. If you previously disabled third-party iFraming for a workspace, the feature continues to be disabled for that workspace. To re-enable third-party iFraming, contact your Databricks account team.
Databricks extension for Visual Studio Code updated to version 1.1.1
August 4, 2023
The Databricks extension for Visual Studio Code version 1.1.1 provides several minor fixes, for instance correctly handling the HOME
environment variable on Windows, and more. For details, see the changelog for version 1.1.1.
LangChain available in 13.1 and above
August 4, 2023
LangChain is available as an experimental MLflow flavor which allows LangChain customers to leverage the robust tools and experiment tracking capabilities of MLflow directly from the Databricks environment.
LangChain is a software framework designed to help create applications that utilize large language models (LLMs) and combine them with external data to bring more training context for LLMs.
Databricks Runtime for Machine Learning includes langchain
in Databricks Runtime 13.1 ML and above.
Feature Engineering in Unity Catalog is Public Preview
August 3, 2023
With Feature Engineering in Unity Catalog, Unity Catalog becomes your feature store. You can use any Delta table with a primary key as a feature table for model training or inference. Unity Catalog provides feature discovery and governance.
Improved error handling for repeated continuous job failures
August 3, 2023
Continuous jobs exceeding a threshold for consecutive failures now use an exponential backoff scheme to efficiently retry job runs and allow jobs to recover from transient issues. Previously, continuous jobs exceeding the failure threshold were paused and not retried.
Run tasks conditionally in your Databricks jobs
August 2, 2023
You can use the new Run if
condition to run a job task based on the status of the task’s dependencies, providing more flexibility when you implement a workflow with a Databricks job. For example, you can use the Run if
condition to run a task when some or all of the task’s dependencies have failed. See Configure task dependencies.
Delta Live Tables is now generally available
August 2, 2023
Delta Live Tables is now generally available on Google Cloud. See What is Delta Live Tables?.
Databricks Terraform provider updated to version 1.22.0
August 1, 2023
Databricks Terraform provider version 1.22.0 adds a databricks_access_control_rule_set
resource, adds databricks_metastore
and databricks_metastore
data sources, and more. For details, see the changelog for version 1.22.0.
Databricks Runtime 13.3 LTS (Beta)
August 1, 2023
Databricks Runtime 13.3 LTS and Databricks Runtime 13.3 LTS ML are now available as Beta releases.
See Databricks Runtime 13.3 LTS and Databricks Runtime 13.3 LTS for Machine Learning.
New Git operations are generally available: Merge branches, rebase and pull with conflict resolution
August 1, 2023
The following Git operations are now generally available in Databricks Repos:
Merge and rebase: You can now Git merge or rebase two branches directly in Databricks. This simplifies the process of combining branches and provides the ability to resolve conflicts that arise during the operation.
Pull with conflict resolution: You can now use Databricks Repos to manage and resolve conflicts that happen when you pull from a remote Git repo using Databricks.
With these powerful new conflict resolution tools, collaborating on code using Databricks Repos becomes even more streamlined and efficient.