This feature is in Public Preview.
In Databricks Runtime 14.1 and above, you can use Structured Streaming to perform streaming reads from views registered with Unity Catalog. Databricks only supports streaming reads from views defined against Delta tables.
To read a view with Structured Streaming, provide the identifier for the view to the
.table() method, as in the following example:
df = (spark.readStream
Users must have
SELECT privileges on the target view.
The following options are supported when configuring streaming reads against views:
The streaming reader applies these options to the files and metadata defining the underlying Delta tables.
Reads against views defined with
UNION ALL do not support the options
Not all views support streaming reads. Unsupported operations in source views include aggregations and sorting.
The following list provides descriptions and example view definitions for supported operations:
Description: Controls column-level permissions
CREATE VIEW project_view AS SELECT id, value FROM source_table
Description: Controls row-level permissions
CREATE VIEW filter_view AS SELECT * FROM source_table WHERE value > 100
Description: Results from multiple tables
CREATE VIEW union_view AS SELECT id, value FROM source_table1 UNION ALL SELECT * FROM source_table2
You cannot modify the view definition to add or change the tables referenced in the view and use the same streaming checkpoint.
The following limitations apply:
You can only stream from views backed by Delta tables. Views defined against other data sources are not supported.
You must register views with Unity Catalog.
The following exception displays if you stream from a view with an unsupported operator:
UnsupportedOperationException: [UNEXPECTED_OPERATOR_IN_STREAMING_VIEW] Unexpected operator <operator> in the CREATE VIEW statement as a streaming source. A streaming view query must consist only of SELECT, WHERE, and UNION ALL operations.
The following exception displays if you provide unsupported options: ``` AnalysisException: [UNSUPPORTED_STREAMING_OPTIONS_FOR_VIEW.UNSUPPORTED_OPTION] Unsupported for streaming a view. Reason: option <option> is not supported. ```