Calculate subnet sizes for a new workspace

When you create a new workspace using the account console, Databricks creates a Google Cloud GKE cluster. By default, Databricks decides the IP ranges for the subnets. You can optionally use advanced configurations to set subnet sizes explicitly.

It is important to configure the GKE subnets used by your Databricks workspace accurately because you cannot change them after your workspace is deployed. If the address ranges for your Databricks subnets are too small, then the workspace exhausts its IP space, which in turn causes your Databricks jobs to fail.

To determine the address range sizes that you need, Databricks provides a calculator in the form of a Microsoft Excel spreadsheet (.xlsx format). The spreadsheet returns the results in CIDR format.

Use the Databricks on Google Cloud workspace network sizing spreadsheet

  1. Download the sizing spreadsheet.

  2. Open the sizing spreadsheet in Microsoft Excel.

    GCP network sizing spreadsheet

  3. Enter your data in the two yellow cells:

    • In cell B2, enter the maximum number of nodes that you expect to run in your workspace. Remember to consider both your job clusters and all-purpose clusters. For a comparison of these two cluster types, see Databricks concepts.

    • In cell B6, enter the maximum number of clusters in your workspace. In the worst case where all clusters are Single Node clusters, this would be equal to the number of nodes in your workspace. To size this correctly, first estimate the average cluster size. Next, divide the maximum number of nodes per workspace by that average cluster size.

    Do not modify any other cells in the spreadsheet.

  4. Review the calculated values in the rows below the Output label to size your network:

    • GKE Master subnet size (only used for private GKE clusters): This is always /28 because Google Cloud requires this size.

    • Node subnet size: This is a value in the range /9 through /29, depending on the maximum number of nodes in a workspace (cell B2).

    • Pods subnet size: This is a value in the range /9 through /21, depending on the maximum number of nodes in a workspace (cell B2).

    • Service subnet size: This is a value in the range /16 through /27, depending on the maximum number of clusters in a workspace (cell B6).

    Use these values when you create a new workspace.