Data governance for the data lakehouse

The architectural principles of the data governance pillar cover how to centrally manage data and access to it.

Data governance lakehouse architecture diagram for Databricks.

Principles of data governance

  1. Unify data management

    Data management is the foundation for executing the data governance strategy. It involves the collection, integration, organization, and persistence of trusted data assets to help organizations maximize their value. A unified catalog centrally and consistently stores all your data and analytical artifacts, as well as the metadata associated with each data object. It enables end users to discover the data sets available to them and provides provenance visibility by tracking the lineage of all data assets.

  2. Unify data security

    There are two tenets of effective data security governance: understanding who has access to what data, and who has recently accessed what data. This information is critical for almost all compliance requirements for regulated industries and is fundamental to any security governance program. With a unified data security system, the permissions model can be centrally and consistently managed across all data assets. Data access is centrally audited with alerting and monitoring capabilities to promote accountability.

  3. Manage data quality

    Data quality is fundamental to deriving accurate and meaningful insights from data. Data quality has many dimensions, including completeness, accuracy, validity, and consistency. It must be actively managed to improve the quality of the final data sets so that the data serves as reliable and trustworthy information for business users.

  4. Share data securely and in real-time

    Data sharing plays a key role in business processes across the enterprise, from product development and internal operations to customer experience and regulatory compliance. Increasingly, organizations need to share data sets, large and small, with their business units, customers, suppliers, and partners. Security is critical, as is efficiency and instant access to the latest data. Using an open and secure exchange technology helps to maximize the pool of potential exchange partners by removing the barriers of vendor technology lock-in.

Next: Best practices for data governance

See Best practices for data governance.