Get coding help from Databricks Assistant

This article describes how you can use Databricks Assistant to help you code and debug your notebooks, and provides tips on how to get the most out of the Assistant.

What can Databricks Assistant help with?

Databricks Assistant is a context-aware AI assistant that you can interact with using a conversational interface, making you more productive inside Databricks. You can describe your task in English and let the assistant generate Python code or SQL queries, explain complex code, and automatically fix errors. The assistant uses Unity Catalog metadata to understand your tables, columns, descriptions, and popular data assets across your company to provide personalized responses.

Databricks Assistant can help you with the following tasks:

  • Generate code.

  • Debug code, including identifying and suggesting fixes for errors.

  • Transform and optimize code.

  • Explain code.

  • Help you find relevant information in the Databricks documentation.

For information about enabling and using Databricks Assistant, see Use Databricks Assistant. For general information about Databricks Assistant, see DatabricksIQ-powered features.

Assistant command shortcuts for notebooks

In a notebook, Databricks Assistant is available in the Assistant pane or inline in a code cell.

To use Databricks Assistant directly in a code cell, press Cmd+I on MacOS or Ctrl+I on Windows. A text box appears in the cell. You can type a question or comment in English and then press Enter (not Shift+Enter, like you would to run a cell) to have Assistant generate a response.

Inline assistant helps you locate and add enrichment data.

Use slash commands for prompts

Slash commands are a shortcut for creating common prompts.

Prompt text

What Assistant does

/

Displays common commands

/doc

Comments the code in a diff view

/explain

Provides an explanation of the code in a cell

/fix

Proposes a fix to any code errors in a diff view

/findTables

Searches for relevant tables based on Unity Catalog metadata. To find features or feature tables, mention “features” or “feature tables” in the query. See Find features and feature tables using Databricks Assistant.

/findQueries

Searches for relevant queries based on Unity Catalog metadata.

/optimize

Improves inefficient SQL queries.

/prettify

Formats code for readability.

/rename

Suggests updated names to notebook cells and other elements, depending on the context.

/settings

Adjusts your notebook settings directly from Assistant.

When you use /fix or /doc, in the diff window select Accept to accept the proposed changes or Reject to keep the original code. If you accept the proposed code, the code does not automatically run. You can review the code before running it. If the generated code is not what you wanted, try again by adding more details or information to your comment. See Tips for using Databricks Assistant.

Get detailed explanations of code snippets. Use the /explain prompt and include terms like “be concise” or “explain code line-by-line” to request the level of detail that you want. You can also ask Databricks Assistant to add comments to the code.

For code autocomplete, performance may be better using the Assistant pane than in a notebook cell.

Assistant closes automatically if you Accept or Reject the code it generated.

Reference tables in prompts using @

To quickly reference tables in Assistant prompts, use the @ (at) symbol.

Table reference in an Assistant prompt using the @ symbol

Get help with code

Databricks Assistant helps with code, both SQL and Python:

  • AI-based autocomplete in Databricks notebooks, the SQL editor, and the file editor.

  • Data filtering with natural-language prompts.

  • Code debugging with Diagnose Error (Public Preview).

Get inline code suggestions: Python and SQL examples

As you type, suggestions automatically appear. Press Tab to accept a suggestion. To manually trigger a suggestion, press Option+Shift+Space (on macOS) or Control+Shift+Space (on Windows).

Animated GIF of code completion for SQL.
Animated GIF of code completion for Python.

AI-based autocomplete can also generate code from comments:

Animated GIF of code completion from a comment.

Filter data with natural language prompts

Use Databricks Assistant to filter data outputs with natural language prompts.

To filter outputs with natural language, click the Filter icon in the output table and enter a prompt. For example, as in the animated example following, you can prompt “Show me only males over 70.”

Animated GIF shows Databricks Assistant filtering a table of Titanic survivors for males over 70 years old.

Debug code: Python and SQL examples

Preview

This feature is in Public Preview.

To use Databricks Assistant to fix code, do any of the following:

  • Ask a question in the Assistant pane.

  • Click the Diagnose Error button that appears in the cell results when an error occurs.

  • Click Debug to interactively step through the code line-by-line, set breakpoints, inspect variables, and analyze a program’s execution.

The tabs below show examples in Python and SQL code:

Assistant debugging example in Python.
Assistant debugging example in SQL.

When you click Diagnose error, Assistant automatically runs /fix.

Quick Fix

When code returns errors, Quick Fix automatically recommends fixes for basic errors that can be fixed in a single line change.

Click Accept and run to make the recommended fix and continue running your code:

Screenshot of Databricks Assistant inline bug fix UI.

Tips for using Databricks Assistant

This section includes some general tips and best practices when using Databricks Assistant.

Databricks Assistant uses context to provide better answers

Databricks Assistant has access to table and column schemas and metadata. This allows you to use natural language and generate more accurate queries. For example, if a table has a column called State, you can ask Databricks Assistant to generate a list of users who live in Michigan.

Databricks Assistant uses the following context:

  • Code or queries in the current notebook cell or Databricks SQL editor tab.

  • Table and Column names and descriptions.

  • Previous prompt questions.

  • Favorite and active tables.

  • For the diagnose error feature, the stack trace from the error output.

When selecting columns from a DataFrame, you can get more accurate results by providing a starting query. For example, provide a statement like SELECT * FROM <table_name>. This allows Databricks Assistant to get the column names and not have to guess.

Because Databricks Assistant uses your conversation history to provide better and more accurate answers, you can ask Databricks Assistant to alter the output of a previous response without having to rewrite the entire prompt. Use Assistant’s chat history to iteratively clean, explore, filter, and slice DataFrames in the Assistant pane.

Be specific

The structure and detail that Databricks Assistant provides varies from time to time, even for the same prompt. Try to provide Assistant as much guidance as you can to help it return the information you want in the desired format, level of detail, and so on. For example:

  • “Explain this code in a couple sentences” or “Explain this code line-by-line”.

  • “Create a visualization using MatPlotLib” or “Create a visualization using Seaborn”.

Give examples of row-level data values

Because Databricks Assistant does not use row-level data, you might need to provide more detail to prompts to get the most accurate answer. Use table or column comments in Catalog Explorer to add a line of sample data. For example, suppose your height column is in the format feet-inches. To help Assistant interpret the data, add a comment such as “The height column is in string format and is separated by a hyphen. Example: ‘6-2’.” For information about table and column comments, see Add comments to data and AI assets.

If you need to use column data type conversions to run an operation, you might need to provide details. For example: “convert this code from pandas to PySpark, including the code needed to convert the pandas DataFrame to a PySpark DataFrame and changing the data type of column churn from boolean to integer”.

Use Shift+Enter to add a new line in the chat text box

Use Shift+Enter to add a new line in the Assistant chat text box. This makes it easy to format and organize your messages to Databricks Assistant.

Edit and run code in Databricks Assistant chat pane

Run code in the Assistant pane to validate it or use it as a scratchpad. To run code, click run code icon in the upper-left corner of the code box in the Assistant pane.

The tabs below show examples for Python and SQL code:

Run Python code in assistant pane.
Run SQL code in assistant pane.

When you run code in the Assistant pane, output is displayed and the variables become usable in the notebook.

You can also edit the code that Databricks Assistant generates directly in the Assistant chat box before moving the code to the notebook.

Additional information

The following articles contain additional information about using Databricks Assistant: