March 2024

These features and Databricks platform improvements were released in March 2024.

Note

Releases are staged. Your Databricks account might not be updated until a week or more after the initial release date.

MLflow now enforces quota limits for experiments and runs

March 27, 2024

MLflow now imposes a quota limit on the number of total parameters, tags, and metric steps for all existing and new runs, and the number of total runs for all existing and new experiments. See Resource limits.

The Jobs UI is updated to better manage jobs deployed by Databricks Asset Bundles

March 26, 2024

Because modifications to Databricks jobs deployed by Databricks Asset Bundles should be applied only by updating the bundle configuration, these jobs are, by default, read-only when viewed in the Jobs UI. Previously, by default, these jobs could be modified in the UI and cause unintentional gaps between the configuration in the UI and the bundle configuration. However, an option is provided for cases where you must make emergency changes to a job. See View and run a job created with a Databricks Asset Bundle.

Interactive notebook debugging

March 22, 2024

Databricks now supports interactive Python debugging directly in the notebook for clusters in Single user or No isolation shared access mode. With interactive debugging, you can step through code line by line and view variable values to discover and fix errors in code. For more information, see Debug notebooks.

Self-service sign-up for private exchange providers in Marketplace

March 22, 2024

If you want to publish only private exchange listings in Databricks Marketplace, you can now sign up using a self-service workflow. To publish public listings, you still need to apply through the Databricks partner portal. See Sign up to be a Databricks Marketplace provider.

Databricks Runtime 15.0 is GA

March 22, 2024

Databricks Runtime 15.0 and Databricks Runtime 15.0 ML are now generally available.

See Databricks Runtime 15.0 (EoS) and Databricks Runtime 15.0 for Machine Learning (EoS).

Databricks Repos changed to Git folders

March 21, 2024

The former Databricks Repos feature is now called “Git folders”. If you have existing Repos, they are preserved under the same file system paths. See What happened to Databricks Repos?.

Databricks Runtime 14.1 and 14.2 series support extended

March 20, 2024

Support for Databricks Runtime 14.1 and Databricks Runtime 14.1 for Machine Learning has been extended from April 11, 2024 to October 1, 2024.

Support for Databricks Runtime 14.2 and Databricks Runtime 14.2 for Machine Learning has been extended from May 22, 2024 to October 1, 2024.

See All supported Databricks Runtime releases.

Databricks ODBC driver 2.8.0

March 19, 2024

We have released version 2.8.0 of the Databricks ODBC driver (download). This release adds the following new features and enhancements:

  • Support for JWT assertion as the client credentials for OAuth.

  • Token renew support. For token passthrough authentication, you can now renew your token.

  • Update third party libraries: Arrow 15.0.0 (previously 9.0.0)(on Windows), libcURL 8.6.0 (previously 8.4.0), Zlib 1.3.1 (previously 1.2.13).

  • Accept Undetermined Revocation support.

This release also resolves the following issue in 2.8.0:

  • When UseNativeQuery is set to 1 for a cluster later than DBR 11, the connector returns an incorrect column number after SQLPrepare.

For more information, see the release notes or the Installation and Configuration Guide in the installation package.

Delegate the ability to view an object’s metadata in Unity Catalog (Public Preview)

March 15, 2024

You can now grant users, service principals, and account groups permission to view a Unity Catalog object’s metadata using the new BROWSE privilege. This enables users to discover data without having read access to the data. A user can view an object’s metadata using Catalog Explorer, the schema browser, search results, the lineage graph, information_schema, and the REST API.

The BROWSE privilege can be granted on a catalog or on an external location. Granting BROWSE on a catalog automatically grants BROWSE to all current and future objects within the catalog. A user with the BROWSE privilege does not require USE CATALOG on the parent catalog or USE SCHEMA on the parent schema to view an object’s metadata.

See BROWSE.

System tables now include Marketplace tables (Public Preview)

March 12, 2024

System tables now includes a marketplace schema that stores Marketplace provider data. The schema currently contains the following tables:

  • listing_access_events: Records information about consumers who take action on your listings.

  • listing_funnel_events: Records information about impressions and actions taken on your listings.

For more information these tables, see Marketplace system tables reference. For instructions on enabling the schema, see Enable system table schemas.

System tables now available (Public Preview)

March 11, 2024

You can now access system tables in your Databricks account. System tables are a Databricks-hosted analytical store of an account’s operational data. System tables provide you with easily accessed account-wide observability data.

Admins need to enable each system schema for the tables to be accessible, see Enable system table schemas.

The currently available system tables are:

Databricks Runtime 15.0 (Beta)

March 11, 2024

Databricks Runtime 15.0 and Databricks Runtime 15.0 ML are now available as Beta releases.

Databricks Runtime 14.0 series support ends

March 11, 2024

Support for Databricks Runtime 14.0 and Databricks Runtime 14.0 for Machine Learning ended on March 11. See Databricks support lifecycles.

New computation for sys.path and CWD in Repos

March 8, 2024

We’ve updated how sys.path and the current working directory (CWD) are computed for Python notebooks and files in Repos. There are no functional changes. For sys.path information see the sys.path spec.

Automatic enablement of Unity Catalog for new workspaces

March 6, 2024

Databricks has begun to enable Unity Catalog automatically for new workspaces. This removes the need for account admins to configure Unity Catalog after a workspace is created. Rollout will proceed gradually across accounts. See Automatic enablement of Unity Catalog.