MAX_FILE_PARTITION_BYTES

Applies to: check marked yes Databricks SQL

The MAX_FILE_PARTITION_BYTES configuration parameter controls the maximum size of partitions when reading from a file data source. This affects the degree of parallelism for processing of the data source.

Settings

The setting can be any positive integral number and optionally include a measure such as b (bytes), k or kb (1024 bytes).

You can set this parameter at the session level using the SET statement.

System default

The system default is 128m (128 mebibytes).

Examples

> SET max_file_partition_bytes = 64mb;