Use notebooks

A notebook is a collection of runnable cells (commands). When you use a notebook, you are primarily developing and running cells.

All notebook tasks are supported by UI actions, but you can also perform many tasks using keyboard shortcuts. Toggle the shortcut display by clicking the Keyboard Icon icon.

Develop notebooks

This section describes how to develop notebook cells and navigate around a notebook.

About notebooks

A notebook has a toolbar that lets you manage the notebook and perform actions within the notebook:

Notebook toolbar

and one or more cells (or commands) that you can run:

Notebook cells

At the far right of a cell, the cell actions Cell actions, contains three menus: Run, Dashboard, and Edit:

Run IconDashboardEdit

and two actions: Hide Cell Minimize and Delete Delete Icon.

Add a cell

To add a cell, mouse over a cell at the top or bottom and click the Add Cell icon, or access the notebook cell menu at the far right, click Down Caret, and select Add Cell Above or Add Cell Below.

Delete a cell

Go to the cell actions menu Cell actions at the far right and click Delete Icon (Delete).

When you delete a cell, by default a delete confirmation dialog appears. To disable future confirmation dialogs, select the Do not show this again checkbox and click Confirm. You can also toggle the confirmation dialog setting with the Turn on command delete confirmation option in User Settings Icon > User Settings > Notebook Settings.

To restore deleted cells, either select Edit > Undo Delete Cells or use the (Z) keyboard shortcut.

Cut a cell

Go to the cell actions menu Cell actions at the far right, click Down Caret, and select Cut Cell.

You can also use the (X) keyboard shortcut.

To restore deleted cells, either select Edit > Undo Cut Cells or use the (Z) keyboard shortcut.

Select multiple cells or all cells

You can select adjacent notebook cells using Shift + Up or Down for the previous and next cell respectively. Multi-selected cells can be copied, cut, deleted, and pasted.

To select all cells, select Edit > Select All Cells or use the command mode shortcut Cmd+A.

Default language

The notebook’s default language is indicated by a button next to the notebook name. In the following notebook, the default language is SQL.

Notebook default language

To change the default language:

  1. Click the language button. The Change Default Language dialog appears.

    Change default language
  2. Select the new language from the Default Language drop-down.

  3. Click Change.

  4. To ensure that existing commands continue to work, commands of the previous default language are automatically prefixed with a language magic command.

Mix languages

By default, cells use the default language of the notebook. You can override the default language in a cell by clicking the language button and selecting a language from the drop down.

Cell language drop down

Alternately, you can use the language magic command %<language> at the beginning of a cell. The supported magic commands are: %python, %r, %scala, and %sql.


When you invoke a language magic command, the command is dispatched to the REPL in the execution context for the notebook. Variables defined in one language (and hence in the REPL for that language) are not available in the REPL of another language. REPLs can share state only through external resources such as files in DBFS or objects in object storage.

Notebooks also support a few auxiliary magic commands:

  • %sh: Allows you to run shell code in your notebook. To fail the cell if the shell command has a non-zero exit status, add the -e option. This command runs only on the Apache Spark driver, and not the workers. To run a shell command on all nodes, use an init script.

  • %fs: Allows you to use dbutils filesystem commands. For example, to run the command to list files, you can specify %fs ls instead. For more information, see Use %fs magic commands.

  • %md: Allows you to include various types of documentation, including text, images, and mathematical formulas and equations. See the next section.

Explore SQL cell results in Python notebooks natively using Python

You might want to load data using SQL and explore it using Python. In a Databricks Python notebook, table results from a SQL language cell are automatically made available as a Python DataFrame. The name of the Python DataFrame is _sqldf.


  • In Python notebooks, the DataFrame _sqldf is not saved automatically and is replaced with the results of the most recent SQL cell run. To save the DataFrame, run this code in a Python cell:

    new_dataframe_name = _sqldf
  • If the query uses a widget for parameterization, the results are not available as a Python DataFrame.

  • If the query uses the keywords CACHE TABLE or UNCACHE TABLE, the results are not available as a Python DataFrame.

The screenshot shows an example:

sql results dataframe

SQL syntax highlighting and autocomplete in Python commands

Syntax highlighting and SQL autocomplete are available when you use SQL inside a Python command, such as in a spark.sql command.

Include documentation

To include documentation in a notebook you can create a markdown cell, either by selecting Markdown from the cell’s language button or by using the %md magic command. The contents of the cell are rendered into HTML. For example, this snippet contains markup for a level-one heading:

%md # Hello This is a Title

It is rendered as a HTML title:

Notebook HTML title

Collapsible headings

Cells that appear after cells containing Markdown headings can be collapsed into the heading cell. The following image shows a level-one heading called Heading 1 with the following two cells collapsed into it.

Collapsed cells

To expand and collapse headings, click the + and -.

Also see Hide and show cell content.

To expand or collapse cells after cells containing Markdown headings throughout the notebook, select Expland all headings or Collapse all headings from the View menu.

expand-collapse all in the view menu

Display images

To display images stored in the FileStore, use the syntax:


For example, suppose you have the Databricks logo image file in FileStore:

dbfs ls dbfs:/FileStore/

When you include the following code in a Markdown cell:

Image in Markdown cell

the image is rendered in the cell:

Rendered image

Display mathematical equations

Notebooks support KaTeX for displaying mathematical formulas and equations. For example,

\\(c = \\pm\\sqrt{a^2 + b^2} \\)


$$c = \\pm\\sqrt{a^2 + b^2}$$


renders as:

Rendered equation 1


\\( f(\beta)= -Y_t^T X_t \beta + \sum log( 1+{e}^{X_t\bullet\beta}) + \frac{1}{2}\delta^t S_t^{-1}\delta\\)

where \\(\delta=(\beta - \mu_{t-1})\\)

renders as:

Rendered equation 2

Include HTML

You can include HTML in a notebook by using the function displayHTML. See HTML, D3, and SVG in notebooks for an example of how to do this.


The displayHTML iframe is served from the domain and the iframe sandbox includes the allow-same-origin attribute. must be accessible from your browser. If it is currently blocked by your corporate network, it must added to an allow list.

Command comments

You can have discussions with collaborators using command comments.

To toggle the Comments sidebar, click the Comments button at the top right of a notebook.

Toggle notebook comments

To add a comment to a command:

  1. Highlight the command text and click the comment bubble:

    Open comments
  2. Add your comment and click Comment.

    Add commenty

To edit, delete, or reply to a comment, click the comment and choose an action.

Edit comment

Change cell display

There are three display options for notebooks:

  • Standard view: results are displayed immediately after code cells

  • Results only: only results are displayed

  • Side-by-side: code and results cells are displayed side by side, with results to the right

Go to the View menu View Menu to select your display option.

side-by-side view

Show line and command numbers

To show line numbers or command numbers, go to the View menu View Menu and select Show line numbers or Show command numbers. Once they’re displayed, you can hide them again from the same menu. You can also enable line numbers with the keyboard shortcut Control+L.

Show line or command numbers via the view menu
Line and command numbers enabled in notebook

If you enable line or command numbers, Databricks saves your preference and shows them in all of your other notebooks for that browser.

Command numbers above cells link to that specific command. If you click the command number for a cell, it updates your URL to be anchored to that command. If you want to link to a specific command in your notebook, right-click the command number and choose copy link address.

Find and replace text

To find and replace text within a notebook, select Edit > Find and Replace. The current match is highlighted in orange and all other matches are highlighted in yellow.

Matching text

To replace the current match, click Replace. To replace all matches in the notebook, click Replace All.

To move between matches, click the Prev and Next buttons. You can also press shift+enter and enter to go to the previous and next matches, respectively.

To close the find and replace tool, click Delete Icon or press esc.


You can use Databricks autocomplete to automatically complete code segments as you type them. Databricks supports two types of autocomplete: local and server.

Local autocomplete completes words that are defined in the notebook. Server autocomplete accesses the cluster for defined types, classes, and objects, as well as SQL database and table names. To activate server autocomplete, attach your notebook to a cluster and run all cells that define completable objects.


Server autocomplete in R notebooks is blocked during command execution.

To trigger autocomplete, press Tab after entering a completable object. For example, after you define and run the cells containing the definitions of MyClass and instance, the methods of instance are completable, and a list of valid completions displays when you press Tab.

Trigger autocomplete

Type completion, as well as SQL database and table name completion, work in SQL cells and in SQL embedded in Python.

Type Completion — — SQL Completion

In Databricks Runtime 7.4 and above, you can display Python docstring hints by pressing Shift+Tab after entering a completable Python object. The docstrings contain the same information as the help() function for an object.

Python docstring

Format SQL

Databricks provides tools that allow you to format SQL code in notebook cells quickly and easily. These tools reduce the effort to keep your code formatted and help to enforce the same coding standards across your notebooks.

You can trigger the formatter in the following ways:

  • Single cells

    • Keyboard shortcut: Press Cmd+Shift+F.

    • Command context menu: Select Format SQL in the command context drop-down menu of a SQL cell. This item is visible only in SQL notebook cells and those with a %sql language magic.

      Formatting SQL From Command Context
  • Multiple cells

    Select multiple SQL cells and then select Edit > Format SQL Cells. If you select cells of more than one language, only SQL cells are formatted. This includes those that use %sql.

    Formatting SQL From Edit Menu

Here’s the first cell in the preceding example after formatting:

After Formatting SQL

View table of contents

To display an automatically generated table of contents, click the arrow at the upper left of the notebook (between the sidebar and the topmost cell). The table of contents is generated from the Markdown headings used in the notebook.

Open TOC

To close the table of contents, click the left-facing arrow.

Close TOC

View notebooks in dark mode

You can choose to display notebooks in dark mode. To turn dark mode on or off, select View > Notebook Theme and select Light Theme or Dark Theme.

Notebook light or dark mode

Run notebooks

This section describes how to run one or more notebook cells.


The notebook must be attached to a cluster. If the cluster is not running, the cluster is started when you run one or more cells.

Run a cell

In the cell actions menu Cell actions at the far right, click Run Icon and select Run Cell, or press shift+enter.


The maximum size for a notebook cell, both contents and output, is 16MB.

For example, try running this Python code snippet that references the predefined spark variable.


and then, run some real code:

1+1 # => 2


Notebooks have a number of default settings:

  • When you run a cell, the notebook automatically attaches to a running cluster without prompting.

  • When you press shift+enter, the notebook auto-scrolls to the next cell if the cell is not visible.

To change these settings, select User Settings Icon > User Settings > Notebook Settings and configure the respective checkboxes.

Run all above or below

To run all cells before or after a cell, go to the cell actions menu Cell actions at the far right, click Run Menu, and select Run All Above or Run All Below.

Run All Below includes the cell you are in. Run All Above does not.

Run all cells

To run all the cells in a notebook, select Run All in the notebook toolbar.


Do not do a Run All if steps for mount and unmount are in the same notebook. It could lead to a race condition and possibly corrupt the mount points.

View multiple outputs per cell

Python notebooks and %python cells in non-Python notebooks support multiple outputs per cell.

Multiple outputs in one cell

This feature requires Databricks Runtime 7.1 or above and can be enabled in Databricks Runtime 7.1 - Databricks Runtime 7.3 by setting spark.databricks.workspace.multipleResults.enabled true. It is enabled by default in Databricks Runtime 7.4 and above.

Python and Scala error highlighting

Python and Scala notebooks support error highlighting. That is, the line of code that is throwing the error will be highlighted in the cell. Additionally, if the error output is a stacktrace, the cell in which the error is thrown is displayed in the stacktrace as a link to the cell. You can click this link to jump to the offending code.

Python error highlighting
Scala error highlighting


Notifications alert you to certain events, such as which command is currently running during Run all cells and which commands are in error state. When your notebook is showing multiple error notifications, the first one will have a link that allows you to clear all notifications.

Notebook notifications

Notebook notifications are enabled by default. You can disable them under User Settings Icon > User Settings > Notebook Settings.

Databricks Advisor

Databricks Advisor automatically analyzes commands every time they are run and displays appropriate advice in the notebooks. The advice notices provide information that can assist you in improving the performance of workloads, reducing costs, and avoiding common mistakes.

View advice

A blue box with a lightbulb icon signals that advice is available for a command. The box displays the number of distinct pieces of advice.

Databricks advice

Click the lightbulb to expand the box and view the advice. One or more pieces of advice will become visible.

View advice

Click the Learn more link to view documentation providing more information related to the advice.

Click the Don’t show me this again link to hide the piece of advice. The advice of this type will no longer be displayed. This action can be reversed in Notebook Settings.

Click the lightbulb again to collapse the advice box.

Advice settings

Access the Notebook Settings page by selecting User Settings Icon > User Settings > Notebook Settings or by clicking the gear icon in the expanded advice box.

Notebook settings

Toggle the Turn on Databricks Advisor option to enable or disable advice.

The Reset hidden advice link is displayed if one or more types of advice is currently hidden. Click the link to make that advice type visible again.

Share code in notebooks

Databricks supports several methods for sharing code among notebooks. Each of these permits you to modularize and share code in a notebook, just as you would with a library.

For more complex interactions between notebooks, see Notebook workflows.

Use %run to import a notebook

The %run magic executes all of the commands from another notebook. A typical use is to define helper functions in one notebook that are used by other notebooks.

In the example below, the first notebook defines a helper function, reverse, which is available in the second notebook after you use the %run magic to execute shared-code-notebook.

Shared code notebook
Notebook import example

Because both of these notebooks are in the same directory in the workspace, use the prefix ./ in ./shared-code-notebook to indicate that the path should be resolved relative to the currently running notebook. You can organize notebooks into directories, such as %run ./dir/notebook, or use an absolute path like %run /Users/


  • %run must be in a cell by itself, because it runs the entire notebook inline.

  • You cannot use %run to run a Python file and import the entities defined in that file into a notebook. To import from a Python file, see Reference source code files using git. Or, package the file into a Python library, create a Databricks library from that Python library, and install the library into the cluster you use to run your notebook.

  • When you use %run to run a notebook that contains widgets, by default the specified notebook runs with the widget’s default values. You can also pass in values to widgets; see Use widgets with %run.

Reference source code files using git


This feature is in Public Preview.

For notebooks stored in a Databricks Repo, you can reference source code files in the repository. The following example uses a Python file rather than a notebook.

Create a new example repo to show the file layout:

New repo
Example repo

To configure an existing Git repository, see Clone a remote Git repository.

Create two files in the repo:

  1. A Python file with the shared code.

  2. A notebook that uses the shared Python code.

Shared code repo files

The Python file contains the helper.

Shared code Python file

Now, when you open the notebook, you can reference source code files in the repository using common commands like import.

Shared code git file

For more information on working with files in Git repositories, see Work with non-notebook files in a Databricks repo.

Manage notebook state and results

After you attach a notebook to a cluster and run one or more cells, your notebook has state and displays results. This section describes how to manage notebook state and results.

Clear notebooks state and results

To clear the notebook state and results, click Clear in the notebook toolbar and select the action:

Clear state and results

Download results

By default downloading results is enabled. To toggle this setting, see Manage the ability to download results from notebooks. If downloading results is disabled, the Download Result button is not visible.

Download a cell result

You can download a cell result that contains tabular output to your local machine. Click the Download Result button at the bottom of a cell.

Download cell results

A CSV file named export.csv is downloaded to your default download directory.

Download full results

By default Databricks returns 1000 rows of a DataFrame. When there are more than 1000 rows, an option appears to re-run the query and display up to 10,000 rows.

Option to re-run

When a query returns more than 1000 rows, a down arrow Button Down is added to the Download Result button. To download all the results of a query:

  1. Click the down arrow next to Download Result and select Download full results.

    Download full results
  2. Select Re-execute and download.

    Re-run and download results

    After you download full results, a CSV file named export.csv is downloaded to your local machine and the /databricks-results folder has a generated folder containing full the query results.

    Downloaded results

Hide and show cell content

Cell content consists of cell code and the result of running the cell. You can hide and show the cell code and result using the cell actions menu Cell actions at the top right of the cell.

To hide cell code:

  • Click Down Caret and select Hide Code

To hide and show the cell result, do any of the following:

  • Click Down Caret and select Hide Result

  • Select Cell Minimize

  • Type Esc > Shift + o

To show hidden cell code or results, click the Show links:

Show hidden code and results

See also Collapsible headings.

Notebook isolation

Notebook isolation refers to the visibility of variables and classes between notebooks. Databricks supports two types of isolation:

  • Variable and class isolation

  • Spark session isolation


Since all notebooks attached to the same cluster execute on the same cluster VMs, even with Spark session isolation enabled there is no guaranteed user isolation within a cluster.

Variable and class isolation

Variables and classes are available only in the current notebook. For example, two notebooks attached to the same cluster can define variables and classes with the same name, but these objects are distinct.

To define a class that is visible to all notebooks attached to the same cluster, define the class in a package cell. Then you can access the class by using its fully qualified name, which is the same as accessing a class in an attached Scala or Java library.

Spark session isolation

Every notebook attached to a cluster running Apache Spark 2.0.0 and above has a pre-defined variable called spark that represents a SparkSession. SparkSession is the entry point for using Spark APIs as well as setting runtime configurations.

Spark session isolation is enabled by default. You can also use global temporary views to share temporary views across notebooks. See _ or CREATE VIEW. To disable Spark session isolation, set spark.databricks.session.share to true in the Spark configuration.


Setting spark.databricks.session.share true breaks the monitoring used by both streaming notebook cells and streaming jobs. Specifically:

  • The graphs in streaming cells are not displayed.

  • Jobs do not block as long as a stream is running (they just finish “successfully”, stopping the stream).

  • Streams in jobs are not monitored for termination. Instead you must manually call awaitTermination().

  • Calling the display function on streaming DataFrames doesn’t work.

Cells that trigger commands in other languages (that is, cells using %scala, %python, %r, and %sql) and cells that include other notebooks (that is, cells using %run) are part of the current notebook. Thus, these cells are in the same session as other notebook cells. By contrast, a notebook workflow runs a notebook with an isolated SparkSession, which means temporary views defined in such a notebook are not visible in other notebooks.

Revision history

Databricks notebooks maintain a history of revisions, allowing you to view and restore previous snapshots of the notebook. You can perform the following actions on revisions: add comments, restore and delete revisions, and clear revision history.

To access notebook revisions, click Revision History at the top right of the notebook toolbar.

Revision history

Add a comment

To add a comment to the latest revision:

  1. Click the revision.

  2. Click the Save now link.

    Save comment
  3. In the Save Notebook Revision dialog, enter a comment.

  4. Click Save. The notebook revision is saved with the entered comment.

Restore a revision

To restore a revision:

  1. Click the revision.

  2. Click Restore this revision.

    Restore revision
  3. Click Confirm. The selected revision becomes the latest revision of the notebook.

Delete a revision

To delete a notebook’s revision entry:

  1. Click the revision.

  2. Click the trash icon Trash.

    Delete revision
  3. Click Yes, erase. The selected revision is deleted from the notebook’s revision history.

Clear a revision history

To clear a notebook’s revision history:

  1. Select File > Clear Revision History.

  2. Click Yes, clear. The notebook revision history is cleared.


    Once cleared, the revision history is not recoverable.

Version control with Git


To sync your work in Databricks with a remote Git repository, Databricks recommends using Repos for Git integration.

To link a single notebook to Git, Databricks also supports these Git-based version control tools:

GitHub version control

Bitbucket Cloud version control

Test notebooks

This section covers several ways to test code in Databricks notebooks. You can use these methods separately or together.

Many unit testing libraries work directly within the notebook. For example, you can use the built-in Python `unittest` package to test notebook code.

def reverse(s):
    return s[::-1]

import unittest

class TestHelpers(unittest.TestCase):
    def test_reverse(self):
        self.assertEqual(reverse('abc'), 'cba')

r = unittest.main(argv=[''], verbosity=2, exit=False)
assert r.result.wasSuccessful(), 'Test failed; see logs above'

Test failures appear in the output area of the cell.

Unit test failure

You can use widgets to distinguish test invocations from normal invocations in a single notebook.

Widget customize execution

To hide test code and results, select the associated menu items from the cell dropdown. Any errors that occur appear even when results are hidden.

Hiding test code and results

To run tests periodically and automatically, you can use scheduled notebooks. You can configure the job to send notification emails to an address you specify.

Scheduled notebook test

Separate test code from the notebook

To separate your test code from the code being tested, see Share code in notebooks.

An example using %run:

Separating test code

For code stored in a Databricks Repo, you can use the web terminal to run tests in source code files just as you would on your local machine.

Repos testing invocation

You can also run this test from a notebook.

Notebook testing invocation

For notebooks in a Databricks Repo, you can set up a CI/CD-style workflow by configuring notebook tests to run for each commit. See Databricks GitHub Actions.