Use CI/CD

To manage the lifecycle of Databricks assets and data, you can use continuous integration and continuous delivery (CI/CD) and data pipeline tools.

Area

Use these tools when you want to…

CI/CD workflows with Git and Databricks Repos

Use GitHub and Databricks Repos for source control and CI/CD workflows.

Continuous integration and delivery using GitHub Actions

Build a CI/CD workflow on GitHub that uses GitHub Actions developed for Databricks.

CI/CD with Jenkins on Databricks

Develop a CI/CD pipeline for Databricks that uses Jenkins.

Orchestrate Databricks jobs with Apache Airflow

Manage and schedule a data pipeline that uses Apache Airflow.

Service principals for CI/CD

Use service principals, instead of users, with CI/CD systems.