October 2022

These features and Databricks platform improvements were released in October 2022.

Note

Releases are staged. Your Databricks account might not be updated until a week or more after the initial release date.

GA: Repos support for non-notebook files

October 31, 2022

Databricks Repos support for non-notebook files is GA. See What are workspace files?.

Deploy models for streaming inference with Delta Live Tables notebooks

October 27, 2022

You can now automatically generate a notebook to use the model for streaming inference via Delta Live Tables. For details, see Configure streaming inference using Delta Live Tables.

Connect to Fivetran from the add data UI

October 27 - November 3, 2022

You can now connect to Fivetran from the add data UI by following the same steps you follow to connect to Fivetran from the Partner Connect portal.

Databricks SQL Driver for Node.js is Generally Available

October 27, 2022

Databricks now provides full support for the Databricks SQL Driver for Node.js. The new version number is 1.0.0.

Changes include:

  • DBSQLClient.openSession now uses the latest protocol version by default.

  • The direct results feature is now available for all IOperation methods that support it. It is on by default.

  • Direct results are now enabled by default. If maxRows is omitted, it defaults to 100,000. To disable direct results, set maxRows to null.

  • The runAsync option is now available for all operations that support it.

  • Logging functionality is added for client-side logging, and a new optional logger param is added to the DBSQLClient constructor.

See the full changelog.

Partner Connect supports connecting to erwin Data Modeler by Quest

October 25-28, 2022

You can now easily create a connection between erwin Data Modeler by Quest and your Databricks workspace using Partner Connect. For more information, see Connect to erwin Data Modeler using Partner Connect.

Optimize performance and control costs by using different pools for the driver node and worker nodes

October 21, 2022

You can now optimize your cluster by attaching it to a different pool for the driver node and worker nodes. For example, you can use a compute-optimized pool for the driver node and a storage-optimized pool for worker nodes. This also helps you control costs by assigning different tags to each instance pool. If you choose, you can use the same pool for drivers and worker nodes.

For more information, see Pool configuration reference.

Databricks Runtime 11.3 LTS and 11.3 LTS ML are GA

October 19, 2022

Databricks Runtime 11.3 LTS and 11.3 LTS ML are now generally available.

Note

LTS means this version is under long-term support. See Databricks Runtime LTS version lifecycle.

See Databricks Runtime 11.3 LTS and Databricks Runtime 11.3 LTS for Machine Learning.

Format Python code in notebooks (Public Preview)

Oct 17-21, 2022

You can now format Python cells in a notebook, making it easier to read and maintain code and enforce coding standards across notebooks. For details, see Format code cells.

DBFS Rest API is available for all customers

October 13, 2022

You can now interact with DBFS using the DBFS Rest API. See DBFS API.

Add data UI provides a central UI for loading data to Databricks

October 11, 2022

The add data UI provides access to common data sources configurations and file upload UIs for all Databricks personas. See Upload files to Databricks.

Create or modify table from file upload page unifies experience for small file upload to Delta Lake

October 11, 2022

Databricks now supports uploading small files to Delta Lake using a UI from all personas. See Create or modify a table using file upload.

Partner Connect Public Preview

October 11, 2022

Partner Connect is now in Public Preview. For more information, see What is Databricks Partner Connect?.

Cluster UI preview and access mode replaces security mode

October 11, 2022

The new Create Cluster UI is in Preview. See Clusters UI changes and cluster access modes.

Partner Connect supports connecting to Hevo Data

October 11, 2022

You can now easily create a connection between Hevo Data and your Databricks workspace using Partner Connect. For more information, see Connect to Hevo Data using Partner Connect.

Account users can access the account console

October 11-31, 2022

Account users can access the Databricks account console to view a list of their workspaces. Account users can only view workspaces that they have been granted access to. See Manage account console settings.

Manage account

Users from workspaces are synced automatically to your account as account users. See How do admins assign users to the account?.

Enable admin protection for Standard clusters (or High Concurrency clusters without table ACLs)

October 7, 2022

Account admins can prevent internal credentials from being automatically generated for Databricks workspace admins on Standard clusters (or High Concurrency clusters without table ACLs).

SQL persona integrated with new search experience

October 6, 2022

Global search is now available in the SQL persona. You can now search for queries, dashboards, and alerts with global search in any persona by name using the new search experience. You can also search for content within a query and see a preview of matching content.

Service principals provide API-only access to Databricks resources (Public Preview)

October 3-6, 2022

Databricks administrators can use the Service Principals API to add API-only service principals to Databricks and grant them access to resources, just like normal users. Administrators can create personal access tokens on behalf of service principals. For details, see Manage service principals.

Top navigation bar in the UI

October 3-6, 2022

The following items have moved from the side bar to the top bar in the UI:

  • User Settings

  • Admin Console

  • Manage Account

  • Help

  • Workspace switcher

Databricks Runtime 11.3 (Beta)

October 3, 2022

Databricks Runtime 11.3 is now available as a Beta release.

See Databricks Runtime 11.3 LTS and Databricks Runtime 11.3 LTS for Machine Learning.

The account console is available in multiple languages

October 1, 2022

The account console is now available in the following languages:

  • English

  • Deutsch

  • Español

  • Français

  • Italiano

  • 日本語

  • 한국어

  • Português (Brasil)

  • Português (Portugal)

To change the account console language, click your username in the console’s top navigation bar, select Settings, and go to the Language settings tab.