Use IDEs with Databricks

You can connect many popular third-party integrated development environments (IDEs) to a Databricks cluster. This allows you to write code on your local development machine by using the Spark APIs and then run that code as jobs remotely on a Databricks cluster.

Name	Use this when you want to…
dbx by Databricks Labs	Use an open source tool which is designed to provide functionality for rapid development lifecycle and continuous integration and continuous delivery/deployment (CI/CD) on the Databricks platform. dbx simplifies jobs launch and deployment processes across multiple environments. It also helps to package your project and deliver it to your Databricks environment in a versioned fashion. Designed in a CLI-first manner, it is built to be actively used both inside CI/CD pipelines and as a part of local tooling (such as local IDEs).
IDE usage with dbx	Use third-party integrated development environments (IDEs), such as Visual Studio Code, PyCharm, IntelliJ IDEA, and Eclipse, along with dbx by Databricks Labs, to automate Databricks.
dbx sync	Perform real-time synchronization of changes to files on your local development machine with their corresponding files in your Databricks workspaces.

Note

Databricks also supports a tool named Databricks Connect. However, Databricks plans no new feature development for Databricks Connect at this time. Also, Databricks Connect has several limitations.