Databricks Utilities with Databricks Connect for Scala

Note

This article covers Databricks Connect for Databricks Runtime 13.3 LTS and above.

This article describes how to use Databricks Utilities with Databricks Connect for Scala. Databricks Connect enables you to connect popular IDEs, notebook servers, and custom applications to Databricks clusters. See What is Databricks Connect?. For the Python version of this article, see Databricks Utilities with Databricks Connect for Python.

Note

Before you begin to use Databricks Connect, you must set up the Databricks Connect client.

You use Databricks Connect to access Databricks Utilities as follows:

• Use DBUtils.getDBUtils to access the Databricks File System (DBFS) and secrets through Databricks Utilities. DBUtils.getDBUtils belongs to the Databricks Utilities for Scala library.

• No Databricks Utilities functionality other than the preceding utilities are available for Scala projects.

• Databricks Connect for Scala already declares a dependency on the Databricks Utilities for Scala library, so you do not need to explicitly declare this dependency in your Scala project’s build file such as build.sbt for sbt, pom.xml for Maven, or build.gradle for Gradle.

• Authentication for the Databricks Utilities for Scala library is determined through initializing the DatabricksSession class in your Databricks Connect project for Scala.

Tip

You can also use the Databricks SDK for Java from Scala to access any available Databricks REST API, not just the preceding Databricks Utilities APIs. See the databricks/databricks-sdk-java repository in GitHub and also Use Scala with the Databricks SDK for Java.

The following example shows how to use the Databricks Utilities for Scala library to automate a Unity Catalog volume. This example creates a file named zzz_hello.txt in the volume’s path within the workspace, reads the data from the file, and then deletes the file.

import com.databricks.sdk.scala.dbutils.DBUtils

object Main {
  def main(args: Array[String]): Unit = {

    val filePath = "/Volumes/main/default/my-volume/zzz_hello.txt"
    val fileData = "Hello, Databricks!"
    val dbutils = DBUtils.getDBUtils()

    dbutils.fs.put(
      file = filePath,
      contents = fileData,
      overwrite = true
    )

    println(dbutils.fs.head(filePath))

    dbutils.fs.rm(filePath)
  }
}