Delta Sharing

Applies to: check marked yes Databricks SQL check marked yes Databricks Runtime 10.4 LTS and above check marked yes Unity Catalog only

Delta Sharing is an open protocol for secure data sharing with other organizations regardless of which computing platforms they use. It can share collections of tables in a Unity Catalog metastore in real time without copying them, so that data recipients can immediately begin working with the latest version of the shared data.

There are three components to Delta Sharing:

  • Providers

    A provider is an entity which has made data available for sharing.

  • Shares

    A share defines a logical grouping for the tables you intend to share.

  • Recipients

    A recipient identifies an organization with which you want to share any number of shares.

For a detailed guide on how to use Delta Sharing see What is Delta Sharing?.

Providers

Applies to: check marked yes Databricks SQL check marked yes Databricks Runtime 11.3 LTS and above

A data provider is an object representing the organization in the real world who shares the data. A provider contains shares which further contain the shared data. Once a provider has added you as a Recipients you can

Examples 

-- Change the data provider name locally.
> ALTER PROVIDER `Center for Disease Control` RENAME TO cdc;

-- List the shares the provider has granted you access too.
> SHOW SHARES IN PROVIDER cdc;
 vaccinedata

-- Make the share accessible locally as a catalog.
> CREATE CATALOG cdcdata USING SHARE cdc.vaccinedata;

-- Use the data.
> USE CATALOG cdcdata;
> SELECT COUNT(*) FROM information_schema.tables;
  10

Shares

A share is a container instantiated with the CREATE SHARE command. Once created you can iteratively register a collection of existing tables defined within the metastore using the ALTER SHARE command. You can register tables under their original name, qualified by their original schema, or provide alternate exposed names.

You must be a metastore admin or account admin to create, alter, and drop shares.

Examples 

-- Create share `customer_share` only if share with same name doesn't exist, with a comment.
> CREATE SHARE IF NOT EXISTS customer_share COMMENT 'This is customer share';

-- Add 2 tables to the share.
-- Expose my_schema.tab1 a different name.
-- Expose only two partitions of other_schema.tab2
> ALTER SHARE customer_share ADD TABLE my_schema.tab1 AS their_schema.tab1;
> ALTER SHARE customer_share ADD TABLE other_schema.tab2 PARTITION (c1 = 5), (c1 = 7);

-- List the content of the share
> SHOW ALL IN SHARE customer_share;
  name              type  shared_object           added_at                     added_by                   comment partitions
  ----------------- ----  ---------------------- ---------------------------- -------------------------- ------- -----------------
  other_schema.tab2 TABLE main.other_schema.tab2 2022-01-01T00:00:01.000+0000 alwaysworks@databricks.com NULL
  their_schema.tab1 TABLE main.myschema.tab2     2022-01-01T00:00:00.000+0000 alwaysworks@databricks.com NULL   (c1 = 5), (c1 = 7)

Recipients

A recipient is an object you create using CREATE RECIPIENT to represent an organization which you want to allow access shares. When you create a recipient Databricks SQL generates an activation link you can send to the organization. To retrieve the activation link after creation you use DESCRIBE RECIPIENT.

Once a recipient has been created you can give it SELECT privileges on shares of your choice using GRANT ON SHARE.

You must be a metastore administrator to create recipients, drop recipients, and grant access to shares.

Examples 

-- Create a recipient.
> CREATE RECIPIENT IF NOT EXISTS other_org COMMENT 'other.org';

-- Retrieve the activation link to send to other.org
> DESCRIBE RECIPIENT other_org;
  name      created_at                   created_by                 comment   activation_link active_token_id                      active_token_expiration_time rotated_token_id rotated_token_expiration_time
  --------- ---------------------------- -------------------------- --------- --------------- ------------------------------------ ---------------------------- ---------------- -----------------------------
  other_org 2022-01-01T00:00:00.000+0000 alwaysworks@databricks.com other.org https://....    0160c81f-5262-40bb-9b03-3ee12e6d98d7 9999-12-31T23:59:59.999+0000 NULL              NULL

-- Choose shares that other.org has access to
> GRANT SELECT ON SHARE customer_share TO RECIPIENT other_org;