Init script logging

Init script start and finish events are captured in cluster event logs. Details are captured in cluster logs. Global init script create, edit, and delete events are also captured in account-level audit logs.

Init script events

Cluster event logs capture two init script events: INIT_SCRIPTS_STARTED and INIT_SCRIPTS_FINISHED, indicating which scripts are scheduled for execution and which have completed successfully. INIT_SCRIPTS_FINISHED also captures execution duration.

Cluster-scoped init scripts are indicated by the key "cluster".

Note

Cluster event logs do not log init script events for each cluster node; only one node is selected to represent them all.

Where are init script logs written?

If cluster log delivery is configured for a cluster, the init script logs are written to /<cluster-log-path>/<cluster-id>/init_scripts.

Logs for each container in the cluster are written to a subdirectory called init_scripts/<cluster-id>_<container-ip>.

For example, if cluster-log-path is set to cluster-logs, the path to the logs for a specific container would be: dbfs:/cluster-logs/<cluster-id>/init_scripts/<cluster-id>_<container-ip>.

If the cluster is configured to write logs to DBFS, you can view the logs using the File system utility (dbutils.fs).

Every time a cluster launches, it writes a log to the init script log folder.

Important

Any user who creates a cluster and enables cluster log delivery can view the stderr and stdout output from global init scripts. You should ensure that your global init scripts do not output any sensitive information.

Init script events in audit logs

Databricks audit logs capture global init script create, edit, and delete events under the event type globalInitScripts. See Global init scripts events.