Looker

This article describes how to use Looker with a Databricks cluster.

Step 1: Get Databricks connection information

  1. Get a personal access token.
  2. Get the Server Hostname, Port, and HTTP Path field values from the JDBC/ODBC tab for your cluster.

Step 2: Configure Databricks connection in Looker

  1. In Looker, click Admin > Connections > Add Connection.

    Cluster connection parameters
  2. Enter a unique Name for the connection.

    Tip

    Connection names should contain only lowercase letters, numbers, and underscores. Other characters might be accepted but could cause unexpected results later.

  3. For Dialect, select Databricks.

  4. For Remote Host, enter the Server Hostname from Step 1.

  5. For Port, enter the Port from Step 1.

  6. For Database, enter the name of the database in the workspace that you want to access through the connection (for example, default).

  7. For Username, enter the word token.

  8. For Password, enter your personal access token from Step 1.

  9. For Additional Params, enter transportMode=http;ssl=1;httpPath=<http-path>, replacing <http-path> with the HTTP Path value from Step 1.

  10. If you want to translate queries into other time zones, adjust Query Time Zone.

  11. For the remaining fields, keep the defaults, in particular:

    • Keep the Max Connections and Connection Pool Timeout defaults.
    • Leave Database Time Zone blank (assuming that you are storing everything in UTC).
  12. Click Test These Settings.

  13. If the test succeeds, click Add Connection.

Step 3: Begin modeling your database in Looker by creating a project and running the generator

This step assumes that there are permanent tables stored in the database for your connection.

  1. On the Develop menu, turn on Development Mode.

  2. Click Develop > Manage LookML Projects.

  3. Click New LookML Project.

  4. Enter a unique Project Name.

    Tip

    Project names should contain only lowercase letters, numbers, and underscores. Other characters might be accepted but could produce unexpected results later.

  5. For Connection, select the name of the connection from Step 2.

  6. For Schemas, enter default, unless you have other databases to model through the connection.

  7. For the remaining fields, keep the defaults, in particular:

    • Leave Starting Point set to Generate Model from Database Schema.
    • Leave Build Views From set to All Tables.
  8. Click Create Project.

After you create the project and the generator runs, Looker displays a user interface with one .model file and multiple .view files. The .model file shows the tables in the schema and any discovered join relations between them, and the .view files list each dimension (column) available for each table in the schema.

Next steps

To begin working with your project, see the following resources on the Looker website:

Enable and manage persistent derived tables (PDTs)

Looker can reduce query times and database loads by creating persistent derived tables (PDTs). A PDT is a derived table that Looker writes into a scratch schema in your database. Looker then regenerates the PDT on the schedule that you specify. For more information, see Persistent derived tables (PDTs) in the Looker documentation.

To enable PDTs for a database connection, select Persistent Derived Tables for that connection and complete the on-screen instructions. For more information, see Persistent Derived Tables and Configuring Separate Login Credentials for PDT Processes in the Looker documentation.

When PDTs are enabled, by default Looker regenerates PDTs every 5 minutes by connecting to the associated database. Looker restarts the associated Databricks resource if it is stopped. To change this default frequency, set the PDT And Datagroup Maintenance Schedule field for your database connection to a valid cron expression. For more information, see PDT and Datagroup Maintenance Schedule in the Looker documentation.

To enable PDTs or to change the PDT regeneration frequency for an existing database connection, click Admin > Database Connections, click Edit next to your database connection, and follow the preceding instructions.