The managed MLflow integration with Databricks on Google Cloud requires Databricks Runtime for Machine Learning 9.1 LTS or above.
MLflow is an open source platform for managing the end-to-end machine learning lifecycle. It has three primary components: Tracking, Models, and Projects. The MLflow Tracking component lets you log and query machine model training sessions (runs) using Java, Python, R, and REST APIs. An MLflow run is a collection of parameters, metrics, tags, and artifacts associated with a machine learning model training process.
Experiments are the primary unit of organization in MLflow; all MLflow runs belong to an experiment. Each experiment lets you visualize, search, and compare runs, as well as download run artifacts or metadata for analysis in other tools. Experiments are maintained in a Databricks hosted MLflow tracking server.
Experiments are located in the Navigate the workspace file tree. You manage experiments using the same tools you use to manage other workspace objects such as folders, notebooks, and libraries.
The following quickstart notebooks demonstrate how to create and log to an MLflow run using the MLflow tracking APIs, as well how to use the experiment UI to view the run. These notebooks are available in Python, Scala, and R.
The Python and R notebooks use a notebook experiment. The Scala notebook creates an experiment in the
With Databricks Runtime 10.3 ML and above, Databricks Autologging is enabled by default for Python notebooks.