Audit and monitor data sharing using Delta Sharing (for providers)

This article describes how data providers can use Databricks audit logs to monitor Delta Sharing events, including:

• When someone creates, modifies, updates, or deletes a share or a recipient

• When a recipient accesses an activation link and downloads the credential

• When a recipient accesses shares.

• When a recipient accesses data in shared tables.

• When a recipient’s credential is rotated or expires.

For information about how data recipients can audit Delta Sharing events to understand who is accessing which data, see Audit and monitor data access using Delta Sharing (for recipients).

Requirements

To access audit logs, you need to enable audit log delivery. See Configure audit log delivery.

View Delta Sharing events in the audit log

You need to know the bucket and path where your account’s audit logs are delivered.

Delta Sharing events have unityCatalog as their serviceName. Actions that are performed by recipients start with the deltaSharing prefix.

Logged event examples

The following audit event logs an update to the recipient token lifetime. In this example, redacted values are replaced with <redacted>.

{
    "version":"2.0",
    "auditLevel":"ACCOUNT_LEVEL",
    "timestamp":1629775584891,
    "orgId":"3049059095686970",
    "shardName":"example-workspace",
    "accountId":"<redacted>",
    "sourceIPAddress":"<redacted>",
    "userAgent":"curl/7.64.1",
    "sessionId":"<redacted>",
    "userIdentity":{
        "email":"<redacted>",
        "subjectName":null
    },
    "serviceName":"unityCatalog",
    "actionName":"updateMetastore",
    "requestId":"<redacted>",
    "requestParams":{
        "metastore_id":"<redacted>",
        "delta_sharing_scope":"INTERNAL_AND_EXTERNAL"
        "delta_sharing_recipient_token_lifetime_in_seconds": 31536000
    },
    "response":{
        "statusCode":200,
        "errorMessage":null,
        "result":null
    },
    "MAX_LOG_MESSAGE_LENGTH":16384
}

The following audit event logs a recipient’s action to query a table that you, as the provider, have shared with them.

{
        "Version": "2.0",
        "auditLevel": "ACCOUNT_LEVEL",
        "Timestamp": 1635235341950,
        "orgId": "0",
        "shardName": "example-workspace",
        "accountId": "<redacted>",
        "sourceIPAddress": "<redacted>",
        "userAgent": null,
        "sessionId": null,
        "userIdentity": null,
        "serviceName": "unityCatalog",
        "actionName": "deltaSharingQueriedTable",
        "requestId": "<redacted>",
        "requestParams": {
                "Metastore_id": "<redacted>",
                "Options": "{}",
                "Recipient_name": "ip_recipient"
        },
        "Response": {
                "statusCode": 200,
                "errorMessage": null,
                "Result": {
                                "checkpointBytes": "0",
                                "maxRemoveFiles": "0",
                                "path": "file: example/s3/path/golden/snapshot-data0/_delta_log",
                                "scannedAddFileSize": "1300",
                                "jsonLogFileNum": "1",
                                "scannedJsonLogActionNum": "5",
                                "jsonLogFileBytes": "914",
                                "checkpointFileNum": "0",
                                "metastoreId": "<redacted>",
                                "limitHint": "Some(1)",
                                "tableName": "tableName",
                                "tableId": "1",
                                "activeAddFiles": "2",
                                "scannedCheckpointActionNum": "0",
                                "deltaSharingRecipientIdHash": "recipient_hash_id",
                                "tableVersion": "0"
                }
        },
        "MAX_LOG_MESSAGE_LENGTH": 16384
}

Logged event details

The following table lists audited Delta Sharing actions that are delivered to data provider logs. Recipient-initiated events are marked “Recipient action.” Provider-initiated events are marked “Provider action.”

actionName	requestParams
deltaSharingListShares Recpient action	options: The pagination options provided with this request. recipient_name: Indicates the recipient executing the action. is_ip_access_denied: None if there is no IP access list configured. Otherwise, true if the request was denied and false if the request was not denied. sourceIPaddress is the recipient IP address.
deltaSharingGetShare Recpient action	share: The name of the share. recipient_name: Indicates the recipient executing the action. is_ip_access_denied: None if there is no IP access list configured. Otherwise, true if the request was denied and false if the request was not denied. sourceIPaddress is the recipient IP address.
deltaSharingListSchemas Recpient action	share: The name of the share. recipient_name: Indicates the recipient executing the action. options: The pagination options provided with this request. is_ip_access_denied: None if there is no IP access list configured. Otherwise, true if the request was denied and false if the request was not denied. sourceIPaddress is the recipient IP address.
deltaSharingListAllTables Recpient action	share: The name of the share. recipient_name: Indicates the recipient executing the action. is_ip_access_denied: None if there is no IP access list configured. Otherwise, true if the request was denied and false if the request was not denied. sourceIPaddress is the recipient IP address.
deltaSharingListTables Recpient action	share: The name of the share. recipient_name: Indicates the recipient executing the action. options: The pagination options provided with this request. is_ip_access_denied: None if there is no IP access list configured. Otherwise, true if the request was denied and false if the request was not denied. sourceIPaddress is the recipient IP address.
deltaSharingGetTableMetadata Recpient action	share: The name of the share. recipient_name: Indicates the recipient executing the action. schema: The name of the schema. name: The name of the table. predicateHints: The predicates included in the query. limitHints: The maximum number of rows to return. is_ip_access_denied: None if there is no IP access list configured. Otherwise, true if the request was denied and false if the request was not denied. sourceIPaddress is the recipient IP address.
deltaSharingGetTableVersion Recpient action	share: The name of the share. recipient_name: Indicates the recipient executing the action. schema: The name of the schema. name: The name of the table. is_ip_access_denied: None if there is no IP access list configured. Otherwise, true if the request was denied and false if the request was not denied. sourceIPaddress is the recipient IP address.
deltaSharingQueryTable Recpient action	share: The name of the share. recipient_name: Indicates the recipient executing the action. schema: The name of the schema. name: The name of the table. predicateHints: The predicates included in the query. limitHint: The maximum number of rows to return. is_ip_access_denied: None if there is no IP access list configured. Otherwise, true if the request was denied and false if the request was not denied. sourceIPaddress is the recipient IP address.
deltaSharingQueryTableChanges Recpient action	share: The name of the share. recipient_name: Indicates the recipient executing the action. schema: The name of the schema. name: The name of the table. cdf_options: Change data feed options. is_ip_access_denied: None if there is no IP access list configured. Otherwise, true if the request was denied and false if the request was not denied. sourceIPaddress is the recipient IP address.
deltaSharingQueriedTable Recpient action	recipient_name: Indicates the recipient executing the action. is_ip_access_denied: None if there is no IP access list configured. Otherwise, true if the request was denied and false if the request was not denied. sourceIPaddress is the recipient IP address.
deltaSharingQueriedTableChanges Recpient action	recipient_name: Indicates the recipient executing the action. is_ip_access_denied: None if there is no IP access list configured. Otherwise, true if the request was denied and false if the request was not denied. sourceIPaddress is the recipient IP address.
updateMetastore Provider action	delta_sharing_scope: values can be “INTERNAL” or “INTERNAL_AND_EXTERNAL” delta_sharing_recipient_token_lifetime_in_seconds: If present, indicates that the recipient token lifetime was updated.
createRecipient Provider action	name: The name of the recipient. comment: The comment for the recipient. ip_access_list.allowed_ip_addresses: Recipient IP address allowlist.
deleteRecipient Provider action	name: The name of the recipient.
getRecipient Provider action	name: The name of the recipient.
listRecipients Provider action	none
rotateRecipientToken Provider action	name: The name of the recipient. comment: The comment given in the rotation command.
updateRecipient Provider action	name: The name of the recipient. updates: A JSON representation of recipient attributes that were added or removed from the share. Each item includes action (add or remove) and can include name (the new recipient name), owner (new owner), comment, and ip_access_list.allowed_ip_addresses (recipient IP address allowlist).
createShare Provider action	name: The name of the share. comment: The comment for the share.
deleteShare Provider action	name: The name of the share.
getShare Provider action	name: The name of the share. include_shared_objects: Whether the share’s table names were included in the request.
updateShare Provider action	name: The name of the share. updates: A JSON representation of tables that were added or removed from the share. Each item includes action (add or remove), name (the actual name of the table), shared_as (the name the schema and table were shared as, if different from name), and partition_specification (if a partition specification was provided).
listShares Provider action	none
getSharePermissions Provider action	name: The name of the share.
updateSharePermissions Provider action	name: The name of the share. changes: A JSON representation of the updated permissions. Each change includes principal (the user or group to whom permission is granted or revoked), add (the list of permissions that were granted), remove (the list of permissions that were revoked).
getRecipientSharePermissions Provider action	name: The name of the share.
getActivationUrlInfo Provider action	recipient_name: The name of the recipient who opened the activation URL. is_ip_access_denied: None if there is no IP access list configured. Otherwise, true if the request was denied and false if the request was not denied. sourceIPaddress is the recipient IP address.
retrieveRecipientToken Provider action	recipient_name: The name of the recipient who downloaded the token. is_ip_access_denied: None if there is no IP access list configured. Otherwise, true if the request was denied and false if the request was not denied. sourceIPaddress is the recipient IP address.

In addition to the actions and parameters included in the table, the following are also logged:

• Catalog, schema, and table access events. See Unity Catalog events.

• Request parameters that are always present in the audit log for each action:

  • userIdentity.email: The ID of the user who initiated the activity.

  • requestParams.metastore_id: the Unity Catalog metastore that manages the shared data.

Response fields for recipient-initiated events

Responses can include the following key-value pairs under Result:. Values here are examples only. This list is not exhaustive.

"checkpointBytes": "0",
"earlyTermination": "false",
"maxRemoveFiles": "0",
"path": "file: example/s3/path/golden/snapshot-data0/_delta_log",
"deltaSharingPartitionFilteringAccessed": "false",
"deltaSharingRecipientId": "<redacted>",
"deltaSharingRecipientIdHash": "<recipient-hash-id>",
"jsonLogFileNum": "1",
"scannedJsonLogActionNum": "5",
"numRecords": "3",
"deltaSharingRecipientMetastoreId": "<redacted>",
"userAgent": "Delta-Sharing-Unity-Catalog-Databricks-Auth/1.0 Linux/4.15.0-2068-azure-fips OpenJDK_64-Bit_Server_VM/11.0.7+10-jvmci-20.1-b02 java/11.0.7 scala/2.12.15 java_vendor/GraalVM_Community",
"jsonLogFileBytes": "2846",
"checkpointFileNum": "0",
"metastoreId": "<redacted>",
"limitHint": "Some(1)",
"tableName": "cookie_ingredients",
"tableId": "1234567c-6d8b-45fd-9565-32e9fc23f8f3",
"activeAddFiles": "2", // number of AddFiles returned in the query
"numAddFiles": "2", // number of AddFiles returned in the query
"numAddCDCFiles": "2", // number of AddFiles returned in the CDF query
"numRemoveFiles": "2", // number of RemoveFiles returned in the query
"numSeenAddFiles": "3",
"scannedAddFileSize": "1300", // file size in bytes for the AddFile returned in the query
"scannedAddCDCFileSize": "1300", // file size in bytes for the AddCDCFile returned in the CDF query
"scannedRemoveFileSize": "1300", // file size in bytes for the RemoveFile returned in the query
"scannedCheckpointActionNum": "0",
"tableVersion": "0"

Logged errors

Delta Sharing logs the following errors for data providers. Items between < and > characters represent placeholder text.

• Delta Sharing is not enabled on the selected metastore.

  DatabricksServiceException: FEATURE_DISABLED:
  Delta Sharing is not enabled
  

• An operation was attempted on a catalog that does not exist.

  DatabricksServiceException: CATALOG_DOES_NOT_EXIST:
  Catalog ‘<catalog>’ does not exist.
  

• A user who is not an account admin or metastore admin attempted to perform a privileged operation.

  DatabricksServiceException: PERMISSION_DENIED:
  Only administrators can <operation-name> <operation-target>
  

• An operation was attempted on a metastore from a workspace to which the metastore is not assigned.

  DatabricksServiceException: INVALID_STATE:
  Workspace <workspace-name> is no longer assigned to this metastore
  

• A request was missing the recipient name or share name.

  DatabricksServiceException: INVALID_PARAMETER_VALUE: CreateRecipient/CreateShare Missing required field: <recipient-name>/<share-name>
  

• A request included an invalid recipient name or share name.

  DatabricksServiceException: INVALID_PARAMETER_VALUE: CreateRecipient/CreateShare <recipient-name>/<share-name> is not a valid name
  

• A user attempted to share a table that is not in a Unity Catalog metastore.

  DatabricksServiceException: INVALID_PARAMETER_VALUE: Only managed or external table on Unity Catalog can be added to a share
  

• A user attempted to rotate a recipient that was already in a rotated state and whose previous token had not yet expired.

  DatabricksServiceException: INVALID_PARAMETER_VALUE: There are already two active tokens for recipient <recipient-name>
  

• A user attempted to create a new recipient or share with the same name as an existing one.

  DatabricksServiceException: RECIPIENT_ALREADY_EXISTS/SHARE_ALREADY_EXISTS: Recipient/Share <name> already exists`
  

• A user attempted to perform an operation on a recipient or share that does not exist.

  DatabricksServiceException: RECIPIENT_DOES_NOT_EXIST/SHARE_DOES_NOT_EXIST: Recipient/Share '<name>' does not exist
  

• A user attempted to add a table to a share, but the table had already been added.

  DatabricksServiceException: RESOURCE_ALREADY_EXISTS: Shared Table '<name>' already exists
  

• A user attempted to perform an operation that referenced a table that does not exist.

  DatabricksServiceException: TABLE_DOES_NOT_EXIST: Table '<name>' does not exist
  

• A user attempted to perform an operation that referenced a schema that did not exist.

  DatabricksServiceException: SCHEMA_DOES_NOT_EXIST: Schema '<name>' does not exist
  

For a list of auditable events and errors logged for data recipients, see Audit and monitor data access using Delta Sharing (for recipients).