Table protocol versioning

The transaction log for a Delta table contains protocol versioning information that supports Delta Lake evolution. Delta Lake tracks minimum reader and writer versions separately.

Delta Lake guarantees backward compatibility. A higher protocol version of Delta Lake reader is always able to read data that was written by a lower protocol version.

Delta Lake will occasionally break forward compatibility. Lower protocol versions of Delta Lake may not be able to read and write data that was written by a higher protocol version of Delta Lake. If you try to read and write to a table with a protocol version of Delta Lake that is too low, you’ll get an error telling you that you need to upgrade.

When creating a table, Delta Lake chooses the minimum required protocol version based on table characteristics such as the schema or table properties. You can also set the default protocol versions by setting the SQL configurations:

  • spark.databricks.delta.properties.defaults.minWriterVersion = 2 (default)

  • spark.databricks.delta.properties.defaults.minReaderVersion = 1 (default)

To upgrade a table to a newer protocol version, use the DeltaTable.upgradeTableProtocol method:

Warning

Protocol version upgrades are irreversible, and upgrading the protocol version may break the existing Delta Lake table readers, writers, or both. Therefore, we recommend you upgrade specific tables only when needed, such as to opt-in to new features in Delta Lake. You should also check to make sure that all of your current and future production tools support Delta Lake tables with the new protocol version.

-- Upgrades the reader protocol version to 1 and the writer protocol version to 3.
ALTER TABLE <table_identifier> SET TBLPROPERTIES('delta.minReaderVersion' = '1', 'delta.minWriterVersion' = '3')
from delta.tables import DeltaTable
delta = DeltaTable.forPath(spark, "path_to_table") # or DeltaTable.forName
delta.upgradeTableProtocol(1, 3) # upgrades to readerVersion=1, writerVersion=3
import io.delta.tables.DeltaTable
val delta = DeltaTable.forPath(spark, "path_to_table") // or DeltaTable.forName
delta.upgradeTableProtocol(1, 3) // Upgrades to readerVersion=1, writerVersion=3.

Features by protocol version

Feature

minWriterVersion

minReaderVersion

Introduced in

Documentation

Basic functionality

2

1

What is Delta Lake?

CHECK constraints

3

1

Databricks Runtime 7.4 (Unsupported)

Set a CHECK constraint in Databricks

Change data feed

4

1

Databricks Runtime 8.4 (Unsupported)

Use Delta Lake change data feed on Databricks

Generated columns

4

1

Databricks Runtime 8.3 (Unsupported)

Use generated columns

Column mapping

5

2

Databricks Runtime 10.2 (Unsupported)

Column mapping on Databricks