January 2025

These features and Databricks platform improvements were released in January 2025.

Note

Releases are staged. Your Databricks account might not be updated until a week or more after the initial release date.

State store reader is now GA

January 31, 2025

Support for querying Structured Streaming state data and metadata is now GA in Databricks Runtime 14.3 LTS and above. See Read Structured Streaming state information.

Filtering full datasets for large tables is now supported

January 30, 2025

When filtering truncated data in a large table (output larger than 2MB or containing more than 10,000 rows), you can now choose to apply the filter to the entire dataset. See Filter results.

AI Functions are now Public Preview

January 29, 2025

Databricks AI Functions are built-in SQL functions that allow you to apply AI on your data directly from SQL. See AI Functions on Databricks for availability and applicable scenarios.

AI Guardrails is Public Preview

January 29, 2025

Mosaic AI Gateway now supports AI Guardrails to configure and enforce data compliance at the model serving endpoint level.

AI Playground is Public Preview

January 29, 2025

AI Playground is a chat-like environment where you can test, prompt, and compare LLMs. AI Playground is available in Public Preview in us-east1 and us-central1.

Mosaic AI Vector Search is Public Preview

January 29, 2025

Mosaic AI Vector Search is a serverless similarity search engine that allows you to store a vector representation of your data, including metadata, in a vector database. As part of this functionality, you can create auto-updating vector search indexes from data in Unity Catalog and query them with a simple API to return the most similar vectors. See Mosaic AI Vector Search.

Meta Llama 3.3 70B Instruct support for pay-per-token endpoints (Public Preview)

January 29, 2025

Mosaic AI Model Serving now supports Meta Llama 3.3 70B Instruct for Foundation Model APIs pay-per-token endpoints.

Meta Llama model families support for provisioned throughput workloads (Public Preview)

January 29, 2025

Mosaic AI Model Serving now supports the following Meta Llama model families for Foundation Model APIs provisioned throughput workloads.

  • Meta Llama 3.3

  • Meta Llama 3.2 3B

  • Meta Llama 3.2 1B

  • Meta Llama 3.1

Delta Live Tables now supports publishing to tables in multiple schemas and catalogs

January 27 - February 5, 2025

By default, new pipelines created in Delta Live Tables now support creating and updating materialized views and streaming tables in multiple catalogs and schemas.

The new default behavior for pipeline configuration requires that users specify a target schema that becomes the default schema for the pipeline. The LIVE virtual schema and associated syntax is no longer required. For more details, see the following:

Databricks Runtime 16.2 (Beta)

January 27, 2025

Databricks Runtime 16.2 and Databricks Runtime 16.2 ML are now available as Beta releases.

See Databricks Runtime 16.2 (Beta) and Databricks Runtime 16.2 for Machine Learning (Beta).

Comments now support email notifications and @ mentions

January 25, 2025

You can now mention users directly in comments by typing “@” followed by their username. Users will be notified of relevant comment activity through email. See Code comments.

Shortcut to adjust font size

January 25, 2025

You can now use a shortcut to quickly adjust the font size in the notebook, file, and SQL editors. Use Alt + and Alt - for Windows/Linux, or Opt + and Opt - for macOS.

There’s also a developer setting to control the editor font size. Navigate to Settings > Developer > Editor font size and select a font size.

Import workspace files with drag-and-drop

January 24, 2025

You can now drag and drop files and folders to import them into your workspace. Drag-and-drop works on the primary file browser page and the workspace file browser side panel, which is available in the notebook, query, and file editors. See Import a file.

Notebook output improvements

January 23, 2025

The following improvements have been made to the notebook output experience:

  • Is one of filtering: In the results table, you can now filter a column using Is one of and choose the values you want to filter for. To do this, click the menu next to a column and click Filter. A filter modal will open for you to add the conditions you want to filter against. To learn more about filtering results, see Filter results.

  • Result table copy as: You can now copy a result table as CSV, TSV, or Markdown. Select the data you want to copy, then right-click, select Copy as, and choose the format you’d like. Results are copied to your clipboard. See Copy data to clipboard.

  • Download naming: When you download the results of a cell, the download name now corresponds to the notebook name. See Download results.

Faster notebook load times

January 23, 2025

When you first open a notebook, initial load times are now up to 26% faster for a 99-cell notebook and 6% faster for a 10-cell notebook.

Notebooks are now supported as workspace files

January 23, 2025

Notebooks are now supported as workspace files on Databricks Runtime 16.2 and above, and serverless environment 2 and above. You can now programmatically write, read, and delete notebooks just as you would any other file. This allows for programmatic interaction with notebooks from anywhere the workspace filesystem is available. For more information, see Notebooks as workspace files.

Serverless compute for notebooks is GA

January 23, 2025

Serverless compute for notebooks is now generally available. Serverless compute for notebooks gives you on-demand access to scalable compute in notebooks, letting you immediately write and run your Python or SQL code. See Serverless compute for notebooks.

Serverless DLT pipelines is GA

January 23, 2025

Serverless DLT pipelines is now generally available. With serverless DLT pipelines, you can run your Delta Live Tables pipelines without configuring and deploying infrastructure. Databricks efficiently manages the compute resources that run your pipeline, including optimizing and scaling compute for your data processing workloads. See Configure a serverless Delta Live Tables pipeline.

Serverless compute for workflows is GA

January 23, 2025

Serverless compute for workflows is now generally available. Serverless compute for workflows allows you to run your Databricks job without configuring and deploying infrastructure. With serverless compute for workflows, Databricks efficiently manages the compute resources that run your job, including optimizing and scaling compute for your workloads. See Run your Databricks job with serverless compute for workflows.

Failed tasks in continuous jobs are now automatically retried

January 22, 2025

This release includes an update to Databricks Jobs that improves failure handling for continuous jobs. With this change, task runs in a continuous job automatically retry when a run fails. The task runs are retried with an exponentially increasing delay until the maximum number of allowed retries is reached. See How are failures handled for continuous jobs?.

Notebooks: Databricks Assistant chat history available only to user who initiates it

January 22, 2025

In a notebook, the Databricks Assistant chat history is available only to the user who initiates the chat. For more information about privacy and security for Assistant, see Privacy and security.

Update to Databricks Marketplace and Partner Connect UI

January 21, 2025

We have simplified the sidebar by combining Partner Connect and Marketplace into a single Marketplace link. The new Marketplace link is positioned higher on the sidebar for easier access.

Marketplace and Partner Connect.

Databricks JDBC driver 2.7.1

January 16, 2025

The Databricks JDBC Driver version 2.7.1 is now available for download from the JDBC driver download page.

This release includes the following enhancements and new features:

  • Added a new OAuthEnabledIPAddressRanges property that allows clients to override the default OAuth callback port(s), facilitating OAuth token acquisition in environments with network port restrictions.

  • Refresh token support is now available. This enables the driver to automatically refresh authentication tokens using the Auth_RefreshToken property.

  • Added support to use the system’s trusted store with a new UseSystemTrustStore property. When enabled (UseSystemTrustStore=1), the driver verifies connections using certificates from the system’s trusted store.

  • Added UseServerSSLConfigsForOAuthEndPoint property that when it is enabled, it allows clients to share the driver’s SSL configuration for the OAuth endpoint.

  • BASIC authentication is now disabled by default. To re-enable it, set the allowBasicAuthentication property to 1.

This release resolves the following issues:

  • Unicode characters when using IBM JRE with the Arrow result set serialization feature are now properly handled.

  • Complete error messages and causes for error code 401 are now returned.

  • Cloud fetch download handlers are now released when they are finished.

  • Heartbeat threads no longer leak when connections are created using the DataSource class.

  • A potential OAuth2Secret leak in the driver log has been resolved.

  • Query IDs in the driver log are no longer missing.

  • Using OAuth token cache no longer hits tag mismatch bug.

This release includes upgrades to several third-party libraries to address vulnerabilities:

  • arrow-memory-core 17.0.0 (previously 14.0.2)

  • arrow-vector 17.0.0 (previously 14.0.2)

  • arrow-format 17.0.0 (previously 14.0.2)

  • arrow-memory-netty 17.0.0 (previously 14.0.2)

  • arrow-memory-unsafe 17.0.0 (previously 14.0.2)

  • commons-codec 1.17.0 (previously 1.15)

  • flatbuffers-java 24.3.25 (previously 23.5.26)

  • jackson-annotations-2.17.1 (previously 2.16.0)

  • jackson-core-2.17.1 (previously 2.16.0)

  • jackson-databind-2.17.1 (previously 2.16.0)

  • jackson-datatype-jsr310-2.17.1 (previously 2.16.0)

  • netty-buffer 4.1.115 (previously 4.1.100)

  • netty-common 4.1.115 (previously 4.1.100)

For complete configuration information, see the Databricks JDBC Driver Guide installed with the driver download package.

Power Databricks Assistant for Google Cloud Platform with Databricks-hosted models

January 15, 2025

Databricks on Google Cloud Platform now supports Databricks-hosted models for Asssistant. See Use a Databricks-hosted model for Databricks Assistant.

Lakehouse Federation supports Teradata (Public Preview)

January 15, 2025

You can now run federated queries on data managed by Teradata. See Run federated queries on Teradata.

AI Gateway now supports provisioned throughput (Public Preview)

January 10, 2025

Mosaic AI Gateway now supports Foundation Model APIs provisioned throughput workloads on model serving endpoints.

You can now enable the following governance and monitoring features on your model serving endpoints that use provisioned throughput:

  • Permission and rate limiting to control who has access and how much access.

  • Payload logging to monitor and audit data being sent to model APIs using inference tables.

  • Usage tracking to monitor operational usage on endpoints and associated costs using system tables.

  • Traffic routing to minimize production outages during and after deployment.

Databricks Runtime 15.2 series support ends

January 7, 2025

Support for Databricks Runtime 15.2 and Databricks Runtime 15.2 for Machine Learning ended on January 7. See Databricks support lifecycles.

Databricks Runtime 15.3 series support ends

January 7, 2025

Support for Databricks Runtime 15.3 and Databricks Runtime 15.3 for Machine Learning ended on January 7. See Databricks support lifecycles.