MAX_FILE_PARTITION_BYTES
Applies to: Databricks SQL
The MAX_FILE_PARTITION_BYTES
configuration parameter controls the maximum size of partitions when reading from a file data source.
This affects the degree of parallelism for processing of the data source.
Settings
The setting can be any positive integral number and optionally include a measure such as b
(bytes), k
or kb
(1024 bytes).
You can set this parameter at the session level using the SET statement.