Delta Live Tables introduction
Preview
This feature is in Public Preview.
Delta Live Tables is a framework for building reliable, maintainable, and testable data processing pipelines. You define the transformations to perform on your data, and Delta Live Tables manages task orchestration, cluster management, monitoring, data quality, and error handling.
Instead of defining your data pipelines using a series of separate Apache Spark tasks, Delta Live Tables manages how your data is transformed based on a target schema you define for each processing step. You can also enforce data quality with Delta Live Tables expectations. Expectations allow you to define expected data quality and specify how to handle records that fail those expectations.
To get started with Delta Live Tables:
Develop your first Delta Live Tables pipeline with the quickstart.
Learn about fundamental Delta Live Tables concepts.
Learn how to create, run, and manage pipelines with the Delta Live Tables user interface.
Learn how to develop Delta Live Tables pipelines with Python or SQL.
Learn how to manage data quality in your Delta Live Tables pipelines with expectations.
Learn more about Delta Live Tables:
Use external data sources in your Delta Live Tables pipelines: Data sources
Use the data produced by your Delta Live Tables pipelines: Publish data
Efficiently process continually arriving data in your Delta Live Tables pipelines: Streaming data processing
Use change data capture (CDC) processing in your Delta Live Tables pipelines: Change data capture with Delta Live Tables
Use the Delta Live Tables API: API guide
Use the Delta Live Tables command line interface: CLI
Configure your Delta Live Tables pipelines: Pipeline settings
Analyze and report on your Delta Live Tables pipelines: Querying the event log
Run your Delta Live Tables pipelines with popular workflow orchestration tools: Workflow tool integration
Learn how to use access control lists (ACLs) to configure permissions on your Delta Live Tables pipelines: Access control
Find answers and solutions for Delta Live Tables:
Implement common tasks in your Delta Live Tables pipelines: Cookbook
Review frequently asked questions and issues: FAQ
Learn best practices to develop, manage, and run your Delta Live Tables pipelines: Best practices
Learn how the Delta Live Tables upgrade process works and how to test your pipelines with the next system version: Upgrades