Init script logging
Init script start and finish events are captured in cluster event logs. Details are captured in cluster logs. Global init script create, edit, and delete events are also captured in account-level audit logs.
Init script events
Cluster event logs capture two init script events: INIT_SCRIPTS_STARTED
and INIT_SCRIPTS_FINISHED
, indicating which scripts are scheduled for execution and which have completed successfully. INIT_SCRIPTS_FINISHED
also captures execution duration.
Cluster-scoped init scripts are indicated by the key "cluster"
.
Note
Cluster event logs do not log init script events for each cluster node; only one node is selected to represent them all.
Where are init script logs written?
If cluster log delivery is configured for a cluster, the init script logs are written to /<cluster-log-path>/<cluster-id>/init_scripts
.
Logs for each container in the cluster are written to a subdirectory called init_scripts/<cluster-id>_<container-ip>
.
For example, if cluster-log-path
is set to cluster-logs
, the path to the logs for a specific container would be: dbfs:/cluster-logs/<cluster-id>/init_scripts/<cluster-id>_<container-ip>
.
If the cluster is configured to write logs to DBFS, you can view the logs using the File system utility (dbutils.fs).
Every time a cluster launches, it writes a log to the init script log folder.
Important
Any user who creates a cluster and enables cluster log delivery can view the stderr
and stdout
output from global init scripts. You should ensure that your global init scripts do not output any sensitive information.
Init script events in audit logs
Databricks audit logs capture global init script create, edit, and delete events under the event type globalInitScripts
. See Global init scripts events.