Model inference


The managed MLflow integration with Databricks on Google Cloud requires Databricks Runtime for Machine Learning 8.1 or above.

For models registered in Model Registry, you can automatically generate a notebook for batch inference or configure the model for online serving.

For scalable model inference with MLLib and XGBoost4J models, use the native transform methods to perform inference directly on Spark DataFrames. See MLlib example notebooks for several example notebooks that include inference steps.

For other libraries and model types, create a Spark UDF to scale out inference on large datasets. For smaller datasets, use the native model inference routines provided by the library.

If the model is registered in MLfLow, you can create a Spark UDF. Otherwise, use pandas Iterator UDFs to wrap machine learning models.

For streaming applications, use the Apache Spark Structured Streaming API. See the Apache Spark MLlib pipelines and Structured Streaming example.

The following articles provide an introduction to deep learning model inference on Databricks.