Developer tools and guidance

Learn about tools and guidance you can use to work with Databricks assets and data and to develop Databricks applications.

Develop code in an IDE

You can connect many popular third-party IDEs to a Databricks cluster. This allows you to write code on your local development machine by using the Spark APIs and then run that code as jobs remotely on a Databricks cluster.

These third-party IDEs include:

To connect other IDEs through JDBC or ODBC, see the JDBC and ODBC configuration guidance.

Work with a Databricks cluster from a notebook

To run Python, R, or Scala code in a notebook to work with file systems, libraries, and secrets from a Databricks cluster, see Databricks Utilities.

Call the Databricks REST APIs

You can use popular third-party utilities such as curl and tools such as Postman to work with Data Science & Engineering workspaces directly through the Databricks REST APIs.

Provision Databricks infrastructure and assets

You can use an infrastructure-as-code (IaC) approach to programmatically provision Databricks infrastructure and assets such as workspaces, clusters, jobs, groups, users, and more. For details, see Databricks Terraform provider.

Follow Databricks development lifecycle patterns and best practices

To manage the lifecycle of Databricks assets and data, you can use continuous integration and delivery (CI/CD), data pipeline, and data engineering tools.

Area Use these patterns and best practices when you want to…
Continuous integration and delivery on Databricks using Jenkins Develop a CI/CD pipeline for Databricks that uses Jenkins.
Managing dependencies in data pipelines Manage and schedule a data pipeline that uses Apache Airflow.
dbt integration with Databricks Transform data in Databricks by simply writing select statements. dbt turns these select statements into tables and views.
DataGrip integration with Databricks Use this integrated development environment (IDE) for database developers that provides a query console, schema navigation, smart code completion, and other features.
DBeaver integration with Databricks Run SQL commands and browse database objects in Databricks by using this client software application and database administration tool.

Use business intelligence tools with data in Databricks

You can connect many popular business intelligence (BI) tools to clusters to access data in Databricks. These tools include:

To connect other BI tools through JDBC or ODBC, see the JDBC and ODBC configuration instructions.