Databricks Connect release notes
This page lists releases and maintenance updates issued for Databricks Connect.
The Databricks Runtime version of the cluster must be greater than or equal to the Databricks Connect version. Databricks recommends using the latest version to receive any bug fixes and security updates. For system environment information for Databricks Runtime releases, see Databricks Runtime release notes versions and compatibility.
Databricks Connect for Databricks Runtime 15.4 LTS
Databricks Connect 15.4.1 (Python)
September 2, 2024
Features and improvements:
validateSession()
and thedatabricks-connect test
command now fail when the Databricks Connect version is incompatible with the Databricks serverless compute version. For information about Databricks Connect serverless support, see Install Databricks Connect for Python.
Databricks Connect 15.4.0 (Python)
August 27, 2024
This release of Databricks Connect for Python supports Databricks Runtime 15.4 LTS.
Databricks Connect for Databricks Runtime 15.3
Databricks Connect 15.3.0 (Scala)
July 22, 2024
Features and improvements:
Supported
Dataset.observe
for getting stats of Columns of a DataFrame. See SPARK-47545.Added a new function
timestamp_add
for adding the specified number of units to the given timestamp. See SPARK-48369.Added a new function
timestamp_diff
for getting the difference between the timestamps in the specified units by truncating the fraction. See SPARK-48272.Added a new function
try_mod
for computing the remainder of the division of two numbers, which results in null on division by zero. See SPARK-48954.Added a new function
try_parse_json
, which attempts to parse a JSON string and construct aVariant
value. If the attempt fails, the result isnull
. See SPARK-47922.Changed the default value for
spark.sql.legacy.timeParserPolicy
fromEXCEPTION
toCORRECTED
. Instead of raising anINCONSISTENT_BEHAVIOR_CROSS_VERSION
error,CANNOT_PARSE_TIMESTAMP
will be raised if ANSI mode is enabled.NULL
will be returned if ANSI mode is disabled. See SPARK-47719.
Databricks Connect 15.3.3 (Python)
September 2, 2024
Features and improvements:
validateSession()
and thedatabricks-connect test
command now fail when the Databricks Connect version is incompatible with the Databricks serverless compute version. For information about Databricks Connect serverless support, see Install Databricks Connect for Python.
Databricks Connect 15.3.2 (Python)
August 27, 2024
Features and improvements:
A warning now occurs when the Databricks Connect version is incompatible with the serverless version.
Improved error message when the local Python version does not match the serverless version.
Databricks Connect 15.3.1 (Python)
July 18, 2024
Bug fixes:
Serverless Spark Connect sessions expire after 10 minutes of inactivity. Databricks Connect now automatically closes expired sessions on the client side. Subsequent calls to
DatabricksSession.builder.getOrCreate()
will return a new active connect session.
Databricks Connect 15.3.0 (Python)
July 15, 2024
Features and improvements:
Added a new function
timestamp_add
for adding the specified number of units to the given timestamp. See SPARK-48369.Added a new function
timestamp_diff
for getting the difference between the timestamps in the specified units by truncating the fraction. See SPARK-48272.Added a new function
try_parse_json
, which attempts to parse a JSON string and construct aVariant
value. If the attempt fails, the result isnull
. See SPARK-47922.Changed the default value for
spark.sql.legacy.timeParserPolicy
fromEXCEPTION
toCORRECTED
. Instead of raising anINCONSISTENT_BEHAVIOR_CROSS_VERSION
error,CANNOT_PARSE_TIMESTAMP
will be raised if ANSI mode is enabled.NULL
will be returned if ANSI mode is disabled. See SPARK-47719.
Databricks Connect for Databricks Runtime 15.2
Databricks Connect 15.2.0 (Scala)
July 22, 2024
Features and improvements:
The
split
function now supports column type. See SPARK-47845.Added new functions
is_variant_null
,variant_get
,try_variant_get
,schema_of_variant
, andschema_of_variant_agg
to improve the usability of theVariant
type. See SPARK-47890.Added new functions
collate
andcollation
to access collation capabilities. See SPARK-47395.Extensions are now allowed to log extended information in the explain plan. See SPARK-47289.
Databricks Connect 15.2.3 (Python)
September 2, 2024
Features and improvements:
validateSession()
and thedatabricks-connect test
command now fail when the Databricks Connect version is incompatible with the Databricks serverless compute version. For information about Databricks Connect serverless support, see Install Databricks Connect for Python.
Databricks Connect 15.2.2 (Python)
August 27, 2024
Features and improvements:
A warning now occurs when the Databricks Connect version is incompatible with the serverless version.
Improved error message when the local Python version does not match the serverless version.
Databricks Connect 15.2.1 (Python)
July 18, 2024
Bug fixes:
Serverless spark connect sessions expire after 10 minutes of inactivity. Databricks Connect now automatically closes expired sessions on the client side. Subsequent calls to
DatabricksSession.builder.getOrCreate()
will return a new active connect session.
Databricks Connect 15.2.0 (Python)
July 15, 2024
Features and improvements:
The
split
function now supports column type. See SPARK-47845.Added new functions
is_variant_null
,variant_get
,try_variant_get
,schema_of_variant
, andschema_of_variant_agg
to improve the usability of theVariant
type. See SPARK-47890.Added new functions
collate
andcollation
to access collation capabilities. See SPARK-47395.Extensions are now allowed to log extended information in the explain plan. See SPARK-47289.
Databricks Connect for Databricks Runtime 15.1
Databricks Connect 15.1.0 (Scala)
July 22, 2024
Features and improvements:
Added a new function
try_cast
, which attempts to cast to a different data type. If the attempt fails, the result isnull
. See SPARK-47669.Added a new function
monthname
that returns the three-letter abbreviated month name from a given date or timestamp. See SPARK-46515.
Bug fixes:
Session
stop()
API is idempotent now and will not fail if the session is already closed by the server. See SPARK-48810.The
bloomFilter
function now throws anAnalysisException
for invalid input. See SPARK-46787.
Databricks Connect 15.1.2 (Python)
September 2, 2024
Features and improvements:
Improved error message when the local Python version does not match the Python version of the serverless release.
Databricks Connect 15.1.1 (Python)
July 18, 2024
Bug fixes:
Serverless spark connect sessions expire after 10 minutes of inactivity. Databricks Connect now automatically closes expired sessions on the client side. Subsequent calls to
DatabricksSession.builder.getOrCreate()
will return a new active connect session.Supported Python versions are now limited to 3.10 and 3.11, because the
distutils
dependency has been removed in Python 3.12.
Databricks Connect 15.1.0 (Python)
July 15, 2024
Features and improvements:
Databricks Connect for Python now supports connecting to serverless compute. This feature is available in Public Preview. See Configure a connection to serverless compute.
Added a new function
try_cast
, which attempts to cast to a different data type. If the attempt fails, the result isnull
. See SPARK-47669.Added a new function
monthname
that returns the three-letter abbreviated month name from a given date or timestamp. See SPARK-46515.
Bug fixes:
Session
stop()
API is idempotent now and will not fail if the session is already closed by the server. See SPARK-48810.Supported numpy versions are now limited to 1.15<=numpy<2.0 because PySpark 3.5.0 is incompatible with numpy 2.0.
Databricks Connect for Databricks Runtime 14.3 LTS
Databricks Connect 14.3.4 (Scala)
June 24, 2024
Features and improvements:
A number of internal dependencies that were vendored as part of the Databricks Connect Maven package were not shaded. In this release, all internal dependencies that are not part of the Databricks Connect public API surface are shaded.
Bug fixes:
Fixed the issue where a serial version UID mismatch error occurred when executing queries with UDFs.
json4s
andlog4j
have been declared as dependencies instead of being vendored. This was required as part of the fix for serial version UID.
Databricks Connect 14.3.2 (Python)
May 8, 2024
Bug fixes:
Automatically re-try plan execution in some cases of
SESSION_NOT_FOUND
error from the service.Subsequent calls to
SparkSession.builder.getOrCreate()
afterDatabricksSession.builder.getOrCreate()
will return the active default connect session.
Databricks Connect 14.3.0 (Python)
February 1, 2024
Features and improvements:
Databricks Connect now ships with the
databricks-connect
binary. Users can now validate their Python environment and the connection to the Databricks cluster by running:databricks-connect test
.
Bug fixes:
Limit the number of retries when Spark Connect requests fails to the service.
spark.table()
now accepts Delta table time travel syntax.Aggregate functions run on top of
df.melt()
now work. Seehttps://issues.apache.org/jira/browse/SPARK-46484.DataFrame joins using queries that contain common table expressions (CTE) now work. Seehttps://issues.apache.org/jira/browse/SPARK-46337.
df.withColumnsRenamed()
now retains original dictionary ordering. Seehttps://issues.apache.org/jira/browse/SPARK-46261.df.withColumn()
now accepts an argument ofColumn
type. Seehttps://issues.apache.org/jira/browse/SPARK-46028.The behavior of
df.column()
in Spark Connect is now more aligned with the behavior in classic Apache Spark. Seehttps://issues.apache.org/jira/browse/SPARK-45509.Referencing the join column in
df.drop()
resolves correctly, instead of generating an “ambiguous column” error code. Seehttps://issues.apache.org/jira/browse/SPARK-45770.An object of
Struct
type can now be converted to XML using theto_xml()
method. Seehttps://issues.apache.org/jira/browse/SPARK-44790.Improve the usability of exceptions raised by Spark Connect (for example,
SparkConnectGrpcException
). Seehttps://issues.apache.org/jira/browse/SPARK-45808.
Databricks Connect 14.3.0 (Scala GA)
February 1, 2024
Beginning with Databricks Runtime 14.3 LTS, Databricks Connect is now also generally available for Scala.
Features and improvements:
The dependencies on Apache Spark packages has been removed and replaced by bundling these packages and their transitive dependencies into the Databricks Connect JAR, improving the alignment of Databricks Connect releases with the releases of the Databricks Runtime.
Databricks Connect for Databricks Runtime 14.2
Databricks Connect 14.2.1 (Python)
January 12, 2024
Bug fixes:
Previously missed pyspark module (
pyspark.sql.connect.protobuf
) is now included. Theto_protobuf()
method is now available.
Databricks Connect 14.2.0 (Python)
December 7, 2023
Features and improvements:
The
SparkSession
stop()
method now closes the session on the server, interrupting release running operations and releasing resources.Improved error handling for
addArtifact
API and for cluster start up.
Databricks Connect 14.2.0 (Scala)
December 6, 2023
Features and improvements:
The
SparkSession
stop()
method now closes the session on the server, interrupting release running operations and releasing resources.Improved error handling: Previous generic GRPC and Spark Connect server exceptions now come with more user-friendly messages.
Improved retry logic when waiting for cluster start up.
Databricks Connect for Databricks Runtime 14.1
Databricks Connect 14.1.1 (Python)
January 12, 2024
Bug fixes:
Previously missed pyspark module (
pyspark.sql.connect.protobuf
) is now included. Theto_protobuf()
method is now available.
Databricks Connect 14.1.0 (Scala)
November 7, 2023
Features and improvements:
User-defined functions (UDFs) can now be called by using the new
addCompiledArtifacts()
API. See User-defined functions in Databricks Connect for Scala.Logging improvements.
Databricks Connect for Databricks Runtime 14.0
Databricks Connect 14.0.2 (Python)
January 12, 2024
Bug fixes:
Previously missed pyspark module (
pyspark.sql.connect.protobuf
) is now included. Theto_protobuf()
method is now available.
Databricks Connect 14.0.1 (Python)
September 29, 2023
Bug fixes:
Skip starting PyTorch distribution server.
Improvements:
The AddArtifact API upload speed has been improved.
Databricks Connect 14.0.1 (Scala)
September 22, 2023
Improvements:
The Databricks license file is now included in the JAR.
Databricks Connect 14.0.0
September 20, 2023
Features and improvements:
Queries can now run for longer than 1 hour.
Databricks Connect can now reconnect and resume from network interruptions while waiting for query results.
Ability to interrupt running queries.
Support for all Structured Streaming APIs, including
foreachBatch
.
Databricks Connect for Databricks Runtime 13.3 LTS
Databricks Connect 13.3.2
September 21, 2023
Bug fixes:
We fixed a bug where
createDataFrame()
failed when a large dataset was used.
Databricks Connect 13.3.1
September 18, 2023
Features and improvements:
This release improves support for
mypy
type hints.mypy
provides optional static typing improving the IDE experience and error checking when working on Python code.
Databricks Connect 13.3.0
August 28, 2023
Features and improvements:
The Spark Connect Service can now be disabled by setting the Spark conf on the cluster
spark.databricks.service.server.enabled false
.The
DatabricksSession
class is now available and usable in a Databricks notebook. CallingDatabricksSession.builder.getOrCreate()
is equivalent to using the defaultspark
object.It is now possible to upload local data larger than 128 MB via
createDataFrame()
.
Bug fixes:
The
DatabricksSession.getOrCreate()
API did not work correctly when configuring multiple sessions.
Databricks Connect for Databricks Runtime 13.2
Databricks Connect 13.2.1
August 22, 2023
Bug fixes:
Fix Databricks Connect client not waiting for clusters to start up.
Databricks Connect 13.2.0
July 6, 2023
Features and improvements:
You can now pass in custom headers as part of the Spark Connect request.
A Databricks SDK configuration profile can now be directly specified in
DatabricksSession
by usingDatabricksSession.profile()
.The built-in PySpark REPL now recognizes connection parameters (such as the host and cluster ID) from environment variables and the
.databrickscfg
file. The behavior is similar to the default behaviorwhen a session is created fromDatabricksSession
.
Bug fixes:
DatabricksSession
was able to keep only one connection at any time. Now you can useDatabricksSession
to set up multiple connections to different workspaces or the same workspace but with different credentials.A bug that was affecting auto-completion in Visual Studio Code has been fixed.
Databricks Connect for Databricks Runtime 13.1
Databricks Connect for Databricks Runtime 13.1.0 Python GA
June 2, 2023
Databricks Connect “v2” is now generally available for Python.
Databricks SDK integration which enables authentication with OAuth and Azure Active Director (in Public Preview in the Python SDK).
Support for all pandas API on Spark.
Streaming support except for a few items in structured streaming, such as foreachBatch and streaming event listener.
Full Delta support.
Databricks Connect for Databricks Runtime 13.0
Databricks Connect for Databricks Runtime 13.0.0 Public Preview
Beginning with Databricks Runtime 13.0, Databricks Connect builds upon Spark Connect, enabling developers to use the power of Databricks and their IDE of choice to interactively develop and debug their code in any Databricks workspace.
From this version, Databricks Connect can connect to Unity Catalog’s single-user and shared cluster access modes.
Databricks Connect for Databricks Runtime 12.2 LTS
Databricks Connect 12.2.33
August 20, 2024
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.30
May 30, 2024
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.27
April 25, 2024
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.26
April 8, 2024
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.24
March 14, 2024
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.23
February 27, 2024
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.22
February 14, 2024
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.21
February 2, 2024
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.20
January 24, 2024
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.19
December 13, 2023
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.18
December 4, 2023
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.17
November 7, 2023
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.16
October 30, 2023
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.15
October 16, 2023
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.14
September 28, 2023
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.13
September 11, 2023
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.12
August 28, 2023
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.11
August 18, 2023
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.10
August 7, 2023
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.9
July 19, 2023
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.8
July 5, 2023
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect 12.2.7
June 15, 2023
Databricks Connect client update to support Databricks Runtime 12.2 LTS maintenance release.
Databricks Connect for Databricks Runtime 11.3 LTS
Databricks Connect 11.3.40
August 20, 2024
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.37
May 30, 2024
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.34
April 25, 2024
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.33
April 8, 2024
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.31
March 14, 2024
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.30
February 27, 2024
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.29
February 14, 2024
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.28
February 2, 2024
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.27
January 24, 2024
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.26
December 13, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.25
December 4, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.24
November 7, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.23
October 30, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.22
October 16, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.21
September 28, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.20
September 11, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.19
August 28, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.18
August 18, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.17
August 7, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.16
July 19, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.15
July 5, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.14
June 15, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.13
June 12, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.12
May 16, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.11
May 1, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.10
April 11, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release.
Databricks Connect 11.3.7
March 8, 2023
Databricks Connect client update to support Databricks Runtime 11.3 LTS maintenance release of March 8, 2023.
[ES-589567] Fixes java.lang.NoSuchMethodError error when enabling spark.databricks.io.cache.enabled
Databricks Connect for Databricks Runtime 10.4 LTS
Databricks Connect 10.4.55
August 20, 2024
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.52
May 30, 2024
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.49
April 25, 2024
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.48
April 8, 2024
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.46
March 14, 2024
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.45
February 27, 2024
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.44
February 14, 2024
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.43
February 2, 2024
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.42
January 24, 2024
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.41
December 13, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.40
December 4, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.39
November 7, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.38
October 30, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.37
October 16, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.36
September 28, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.35
September 11, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.34
August 28, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.33
August 18, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.32
August 7, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.31
July 19, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.30
July 5, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.29
June 15, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.28
June 12, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.27
May 16, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
[ES-573304] Fix issues relating to resolving tables from file paths (i.e.
parquet
.”/some/path”)
Databricks Connect 10.4.26
May 1, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.25
April 11, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release.
Databricks Connect 10.4.22
March 8, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release of March 8, 2023.
Databricks Connect 10.4.21
February 20, 2023
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release of February 20, 2023.
[ES-573062] Fixes java.lang.ClassCastException when assigning fields to org.apache.spark.sql.catalyst.json.JSONOptions
Databricks Connect 10.4.18
December 1, 2022
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release of December 1, 2022.
Databricks Connect 10.4.12
September 12, 2022
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release of September 6, 2022.
Databricks Connect 10.4.11
August 25, 2022
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release of August 25, 2022.
Databricks Connect 10.4.10
August 12, 2022
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release of August 12, 2022.
Databricks Connect 10.4.9
July 29, 2022
Databricks Connect client update to support Databricks Runtime 10.4 LTS maintenance release of July 29, 2022.
Databricks Connect for Databricks Runtime 9.1 LTS
Databricks Connect 9.1.67
August 20, 2024
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.64
May 30, 2024
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.61
April 25, 2024
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.60
April 8, 2024
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.58
March 14, 2024
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.57
February 27, 2024
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.56
February 14, 2024
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.55
February 2, 2024
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.54
January 24, 2024
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.53
December 13, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.52
December 4, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.51
November 7, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.50
October 30, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.49
October 16, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.48
September 28, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.47
September 11, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.46
August 28, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.45
August 18, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.44
August 7, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.43
July 19, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.42
July 5, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.41
June 15, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.40
June 12, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.38
May 16, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
May 1, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.37
April 11, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release.
Databricks Connect 9.1.34
March 8, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of March 8, 2023.
Databricks Connect 9.1.33
February 20, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of February 20, 2023.
[ES-573062] Fixes java.lang.ClassCastException when assigning fields to org.apache.spark.sql.catalyst.json.JSONOptions
Databricks Connect 9.1.30
December 1, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of December 1, 2022.
Databricks Connect 9.1.24
September 12, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of September 6, 2022.
Databricks Connect 9.1.23
August 25, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of August 25, 2022.
Databricks Connect 9.1.22
August 12, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of August 12, 2022.
Databricks Connect 9.1.21
July 29, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of July 29, 2022.
Databricks Connect 9.1.20
July 15, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of July 14, 2022.
Databricks Connect 9.1.18
June 16, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of June 12, 2022.
Databricks Connect 9.1.17
June 7, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of June 2, 2022.
Databricks Connect 9.1.16
May 20, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of May 20, 2022.
Databricks Connect 9.1.15
May 5, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of May 5, 2022.
Databricks Connect 9.1.14
April 21, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of April 21, 2022.
Databricks Connect 9.1.13
April 7, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of April 7, 2022.
Databricks Connect 9.1.12
March 29, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of March 29, 2022.
Databricks Connect 9.1.11
March 10, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of March 10, 2022.
Fix
ClassNotFoundException
error message
Databricks Connect 9.1.9
February 8, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of February 8, 2022.
Databricks Connect 9.1.8
January 26, 2022
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of January 26, 2022.
Databricks Connect 9.1.7
January 19, 2022
[ES-216554] Fixed an issue where cluster-attached libraries were not loaded correctly server-side in some cases
Databricks Connect 9.1.5
December 7, 2021
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of December 7, 2021.
Databricks Connect 9.1.4
November 16, 2021
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of November 16, 2021.
Databricks Connect 9.1.3
November 4, 2021
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of November 4, 2021.
Databricks Connect 9.1.2
October 20, 2021
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of October 20, 2021.
Databricks Connect for Databricks Runtime 8.1
Databricks Connect 8.1.14
September 23, 2021
Databricks Connect client update to support Databricks Runtime 8.1 maintenance release of September 23, 2021.
Databricks Connect 8.1.13
September 15, 2021
Databricks Connect client update to support Databricks Runtime 8.1 maintenance release of September 15, 2021.
Databricks Connect 8.1.12
September 8, 2021
Databricks Connect client update to support Databricks Runtime 8.1 maintenance release of September 8, 2021.
Databricks Connect 8.1.11
August 25, 2021
Databricks Connect client update to support Databricks Runtime 8.1 maintenance release of August 25, 2021.
Databricks Connect 8.1.10
August 11, 2021
Databricks Connect client update to support Databricks Runtime 8.1 maintenance release of August 11, 2021.
Databricks Connect 8.1.9
July 30, 2021
Databricks Connect client update to support Databricks Runtime 8.1 maintenance release of July 30, 2021.
Databricks Connect 8.1.8
July 14, 2021
Databricks Connect client update to support Databricks Runtime 8.1 maintenance release of July 14, 2021.
Databricks Connect 8.1.7
June 29, 2021
Databricks Connect client update to support Databricks Runtime 8.1 maintenance release of June 29, 2021.
Databricks Connect 8.1.6
June 15, 2021
Databricks Connect client update to support Databricks Runtime 8.1 maintenance release of June 15, 2021.
Databricks Connect 8.1.5
June 7, 2021
[ES-102916] Support
no_proxy
environment variable in DBConnnect client. Set it to a comma-separated list of fully qualified domain names for which the proxy set inhttps_proxy
orhttp_proxy
environment variable should not be used.
Databricks Connect 8.1.4
May 26, 2021
Databricks Connect client update to support Databricks Runtime 8.1 maintenance release of May 26, 2021.
Databricks Connect 8.1.3
April 30, 2021
Databricks Connect client update to support Databricks Runtime 8.1 maintenance release of April 30, 2021.
Databricks Connect for Databricks Runtime 7.3 LTS
Databricks Connect 7.3.72
September 11, 2023
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release.
Databricks Connect 7.3.71
August 28, 2023
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release.
Databricks Connect 7.3.70
August 18, 2023
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release.
Databricks Connect 7.3.69
August 7, 2023
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release.
Databricks Connect 7.3.68
July 19, 2023
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release.
Databricks Connect 7.3.67
July 5, 2023
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release.
Databricks Connect 7.3.66
June 15, 2023
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release.
Databricks Connect 7.3.65
June 12, 2023
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release.
Databricks Connect 7.3.63
May 16, 2023
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release.
May 1, 2023
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release.
Databricks Connect 7.3.62
April 11, 2023
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release.
Databricks Connect 7.3.59
March 8, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of March 8, 2023.
Databricks Connect 7.3.58
February 20, 2023
Databricks Connect client update to support Databricks Runtime 9.1 LTS maintenance release of February 20, 2023.
[ES-573062] Fixes java.lang.ClassCastException when assigning fields to org.apache.spark.sql.catalyst.json.JSONOptions
Databricks Connect 7.3.55
December 1, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of December 1, 2022.
Databricks Connect 7.3.49
September 12, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of September 6, 2022.
Databricks Connect 7.3.48
August 25, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of August 25, 2022.
Databricks Connect 7.3.47
August 12, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of August 12, 2022.
Databricks Connect 7.3.46
July 29, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of July 29, 2022.
Databricks Connect 7.3.45
July 15, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of July 14, 2022.
Databricks Connect 7.3.43
June 16, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of June 12, 2022.
Databricks Connect 7.3.42
June 7, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of June 2, 2022.
Databricks Connect 7.3.41
May 20, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of May 20, 2022.
Databricks Connect 7.3.40
May 5, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of May 5, 2022.
Databricks Connect 7.3.39
April 21, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of April 21, 2022.
Databricks Connect 7.3.38
April 7, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of April 7, 2022.
Databricks Connect 7.3.37
March 29, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of March 29, 2022.
Databricks Connect 7.3.36
March 10, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of March 10, 2022.
Databricks Connect 7.3.34
February 8, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of February 8, 2022.
Databricks Connect 7.3.33
January 26, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of January 26, 2022.
Databricks Connect 7.3.32
January 19, 2022
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of January 19, 2022.
Databricks Connect 7.3.30
December 7, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of December 7, 2021.
Databricks Connect 7.3.29
November 16, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of November 16, 2021.
Databricks Connect 7.3.28
November 4, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of November 4, 2021.
Databricks Connect 7.3.27
October 20, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of October 20, 2021.
Databricks Connect 7.3.26
October 5, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of October 5, 2021.
Databricks Connect 7.3.25
September 23, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of September 23, 2021.
Databricks Connect 7.3.24
September 15, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of September 15, 2021.
Databricks Connect 7.3.23
September 8, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of September 8, 2021.
Databricks Connect 7.3.22
August 25, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of August 25, 2021.
Databricks Connect 7.3.21
August 11, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of August 11, 2021.
Databricks Connect 7.3.20
July 30, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of July 30, 2021.
Databricks Connect 7.3.19
July 14, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of July 14, 2021.
Databricks Connect 7.3.18
June 29, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of June 29, 2021.
Databricks Connect 7.3.17
June 15, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of June 15, 2021.
Databricks Connect 7.3.16
June 7, 2021
[ES-102916] Support
no_proxy
environment variable in DBConnnect client. Set it to a comma-separated list of fully qualified domain names for which the proxy set inhttps_proxy
orhttp_proxy
environment variable should not be used.
Databricks Connect 7.3.15
May 26, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of May 26, 2021.
Databricks Connect 7.3.14
April 30, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of April 30, 2021.
Databricks Connect 7.3.13
April 27, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of April 27, 2021.
Databricks Connect 7.3.11
March 24, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of March 24, 2021
Databricks Connect 7.3.10
March 9, 2021
[ES-69946] Fix path separator on Windows for
databricks-connect get-jar-dir
[ES-68776] Update Azure Blob File System client
Add support for FlatMapCoGroupsInPandas
Make initial connection process during cluster startup more robust
Note
The Databricks Runtime cluster must be running the latest maintenance release, and the Databricks Connect client must be updated to version 7.3.10 to continue to use the Azure Blob File System and
mapPartitions
andmapElements
RDD operations with Databricks Connect. A restart of the Databricks Runtime cluster is required to update it to the latest maintenance release.
Databricks Connect 7.3.9
February 23, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of February 23, 2021
Databricks Connect 7.3.8
February 4, 2021
[ES-58153] Fix Analysis Exception when creating tables or views based on a query with aggregations.
Databricks Connect 7.3.7
January 20, 2021
Databricks Connect client update to support Databricks Runtime 7.3 LTS maintenance release of January 20, 2021
Databricks Connect 7.3.6
January 12, 2021
[ES-59054] Fix Delta Optimize Table command through Databricks Connect
[ES-53497] Fix Databricks Connect on Databricks Container Services (DCS)
Databricks Connect 7.3.5
December 9, 2020
Databricks Connect 7.3 is now GA.
[ES-49865] Prevent 500 Internal Server Errors due to automatic synchronization of dependent libraries when using Databricks Connect with SBT.
Databricks Connect 7.3.4 Beta
November 24, 2020
[ES-44382] Set
spark.databricks.service.allowTerminatedClusterStart=false
to prevent Databricks Connect from starting a terminated Databricks Runtime cluster.Fix an error where client could not synchronize state with server correctly if
INFO
logging level was disabled.