Emily Washington, Senior Vice President of Product Management, Infogix, writes about the importance of data governance
The most significant obstacle preventing organizations from realizing the full potential of their data assets today is the widespread data disorder. Companies have quickly accrued massive amounts of data, and adopted big data environments to store it. And while insights might be buried within all that raw data; if no one knows where it came from, how to find it, what it means or if they can trust it, it will remain untapped and untouched.
To prevent data assets from becoming data liabilities, organizations are increasingly recognizing the need to implement a data governance framework to establish a baseline of data understanding and set data quality benchmarks to ensure the integrity, usability, and value of their data.
Making Data Governance Work for You Data governance is the formal orchestration of people, processes, and technology that enables an organization to leverage data as an enterprise asset. Raw data is largely without value, but it can become an organization’s most important asset when it is refined and understood. It can then be used to generate critical insights resulting in improved business decisions across an enterprise to increase revenue, reduce risk, and drive competitive advantage.
Data governance is the mechanism for enabling this transformation, regardless of the data environment. However, big data environments, such as data lakes, are particularly susceptible to systemic issues around data quality, data lineage, and appropriate usage and meaning, given the predominance of unstructured and semi-structured data. Data governance, in a nutshell, provides business users with the data literacy and structure they need to turn raw data into real intelligence.
The Integral Role of Data Governance in Big Data Environments Data governance is a multi-faceted concept, but it provides the tools and processes to foster data understanding throughout an enterprise. It is a comprehensive program, not a project, and should include a core set of solutions to provide a proper governance foundation.
These solutions include a business glossary, data dictionaries, and data lineage to define data, terms, and business attributes, as well as data sources, usage, relationships, and interdependencies. Data governance should also clearly assign accountability and ownership among data stakeholders, stewards, and owners, as well as a mechanism for managing inquiries and resolving issues.
Historically, data governance has been closely associated with ensuring regulatory compliance, and while that is true, the role of data governance is far broader in the age of big data. For instance, metadata management is a crucial part of governance, and metadata plays an important role for organizations to discover analytic insights. Data governance also plays a critical part in data quality efforts, as organizations continue to struggle with how to assess, improve, and report the quality of their data.
The challenges that today’s broad-based data governance helps organizations overcome—those of data accessibility, usability, meaning, and quality—all increase exponentially in the world of big data. Big data environments are potential treasure troves for insights, but without proper governance, accountability, and organizational collaboration and support, they can be black holes of unused data.
The key to governing these environments is to manage and define that data throughout the entire data supply chain—an effort that begins as data is ingested into organizations and enters any internal environment, whether a data warehouse or a data lake, and continues throughout the data lifecycle. Throughout the data supply chain, key questions need to be addressed. These include:
- Transparency and traceability are key elements that can be tracked through metadata and data lineage—where did the data come from, through what processes and systems has it moved within the organization, and how may it have been transformed?
- Data quality is an ongoing concern as data is considered for analysis. With potential transformations, what do we know about this data and can it be trusted? Is it accurate, consistent, and reliable? Can business users depend on it to generate accurate analysis and insights?
- Accessibility and understanding are fundamental for business users. Without them, it’s like having a warehouse full of equipment and materials, but no key to the door and no instructions for how to use what’s inside—nothing will get built, or what does get built will likely fall apart. What data is available? How is it used internally? How is it defined, and what are its related business terms? Do these definitions differ among lines of business or departments? Data should be clearly categorized, organized, and available to users, and well-defined so the right data is selected for the right task.
- Ownership and collaboration are critical. It isn’t enough to know where the data came from and what it is; there needs to be ongoing responsibility and accountability for those assets. Who owns the data? Data owners and stewards must be clearly defined so that the business users have resources to turn to, for questions regarding use and applicability.
A comprehensive data governance program will answer all of these questions, and provide a solid framework so that the extracted organizational data is reliable, understandable, and usable. Failure to do so can result in business decisions based on bad or incomplete data, which can be costly in both, revenue and loss of reputation, productivity, and missed opportunities.
Data Governance Isn’t Just About Data Management, it’s About a MindsetData governance is gaining importance not just due to the increasing volume and velocity of data and emergence of big data environments, but also because of growing regulatory complexities and the unrelenting challenge of securing data quality to produce quality outcomes. And while governance has historically been relegated to IT and compliance, it is today’s business leaders who need to leverage data and analytics to gain competitive advantage and improve the bottom line.
We’re experiencing a democratization of data in the sense that data is no longer just a concern of IT, as resource constraints and business demands drive a growing need for business user empowerment through self-service capabilities. Business needs to have data right at their fingertips to solve business problems, so they may quickly turn that data into actionable insights.
This is why data governance is so critical in the age of big data. The need and use of data is increasingly in the hands of business users, which means that there needs to be a solid framework that defines all aspects of data and its utilization.
The Right Approach to Data Governance Successful governance requires a business-oriented, centralized data governance model that focuses on an organization-wide understanding of data assets across the entire enterprise. When combined with proper tools, organizations can facilitate a broad and comprehensive understanding of their data, enabling data owners, data stewards and data consumers to manage and apply data to extract maximum business value effectively.
As data is exchanged and used across applications, integrations, and interfaces; it is imperative to guide its journey with effective data stewardship and data governance across all relationships, details, and dependencies. Using a community approach brings together two of your most critical business assets: people and data.
By bridging the business and technical divide, partnerships can be built among data producers, enablers, and consumers by clearly outlining everyone’s roles and responsibilities and establishing full transparency into all aspects of your organization’s data assets.