Microsoft Purview Data Governance

From a marketing and GTM standpoint, Microsoft Purview has mainly helped us understand where our data actually comes from and whether we can trust it. When you're working with reports that combine retail performance, campaign metrics, and commercial data, having visibility on data lineage and ownership makes a big difference.

++ Positive: - The application is quick and easy to set-up - "Managed" infrastructure types such as managed integration runtimes and managed private endpoints are available, making management of infrastructure components a breeze - It is a very cost efficient data cataloging tool, with almost no upfront costs and no extra costs for additional users. - Purview integrates really well with the Azure landscape - Purview works great if the current design suits your governance structure. Friction starts if your existing Governance does not align with the current line of thinking of Purview and if some customizability is required. -- Improvement points: - In general, Purview feels like a promising application in development that currently has many small shortcomings that should be addressed. # UI - The UI is not very intuitive. The landing page is a 'getting started' page which is not useful for general users and also not useful for admins after initial set-up. It takes multiple clicks for a user to start a real search for data (in any other catalog this is zero). The UI does not allow for any customization, so it is not possible to link to any company specific resources on the homepage. - Purview has two search bars, one persistent search bar in the lint that only results in searching for Microsoft documentation, and a search bar hidden behind a 'discovery' tab that actually results in searching for data products. Users use the persistent search bar in the lint often, resulting in them getting lost without actually finding data. - The UI is not context aware. Normal users can see menu's to which they have no access, leading to confusing. Also, data stewards sometimes end-up in the wrong menu where they cannot edit data products. # Data Products - Once a data product is created in a Governance Domain, it is not possible to move the data product to another governance domain. - Currently, the 'contacts' field under a data product is tightly coupled with the 'data product owner' field, allowing only a one-to-one relationship. This design limits flexibility and does not reflect common governance models where a data product has one owner but multiple relevant contacts (e.g. data stewards, subject matter experts). Allowing separate management of contacts would improve usability and better support real-world organizational structures. Moreover, customers need to be able to use this expert field to route tasks in access request workflows in the future. - In Purview, a data product consists of one or more "data assets" (e.g. containers, schema's, reports). These assets are presented in a flat list in the data product. For large data products containing many assets, this is not so useful. The ability to show hierarchy and/or relationships between assets would be helpful. - It is not possible for users to rate data products or to provide feedback on data products. This feature is possible for data assets though. - While data products have to be "unpublished" in order to be edited, after which they have to be "published" again, it is not possible to see an audit trail in the UI, or to roll backt to older versions. # Search Behavior - The current default search behavior does not support partial term matching for data products, resulting in zero results even when a relevant item exists. This limits discoverability and undermines the core purpose of a data catalog: making data accessible and searchable. While a natural language search option exists, it is hidden behind a toggle which is not intuitive for new users. Making natural language search a default behavior or improving the visibility and usability of natural language search would significantly enhance user experience and adoption. - Data asset search contains many legacy components. Searching for a container name can result in thousands of matches, as any path of any file in a container contains the name of the container. No prioritization in search results seem to happen. And why are ADLS2 containers called File Systems in Purview? # Custom metadata - While it is possible to create custom metadata attributes, the functionality is weak. No people-select type of field exists and it is not possible to create dynamic lists. Furthermore, filtering on these custom attributes is not available everywhere and otherwise hidden, further limiting it's usability. An example is that a "date" field exists, while the filter only allows the user to select items "last 24 hours" until "1 year ago", while the filter should logically be a datepicker. # Scanning - Scanning Fabric and Databricks is only cumulative: items that are deleted in the source, are not deleted in Purview. Furthermore, deleted items are not tagged in any way. - Automated data classification is a great functionality, but not available for all source types, such as Fabric - Scoped scans are not available for all source types. # Glossary - An old and a new business glossary exist with different functionality. The old glossary could be linked to columns of assets, the new glossary still can't. One coherent solution is required. # Access requests - The request access form and workflow are currently not customizable. This feature is on the roadmap though. - Real integration with access management systems, or actual access provisioning, is currently not possible. # Infrastructure - Purview only allows for one Purview resource per tenant. Thus, you are always working in production. # Adoption reporting - Adoption reporting is focused on the "data asset" experience, but not on the new "data products". It is not possible to see what users are searching for (and not finding) on data product level.