Importance of Data Governance

Data governance is the process of managing the availability, usability, integrity, and security of data in the organization's systems, based on internal data standards and policies that also control data usage.

Effective data governance ensures that data is consistent, reliable, and not misused. It is increasingly critical as organizations face new data privacy regulations and increasingly rely on data analytics to help streamline operations and drive business decision-making.

A well-organized data governance model includes a governance team, a steering committee that acts as the governing body, and a group of data stewards. They all work together to create the standards and policies to govern the data, as well as the implementation and compliance procedures that are carried out primarily by data stewards. Executives and other representatives of an organization's business operations, as well as IT and data management teams, also participate.

While data governance is a core component of an overall data management strategy, organizations should focus on the desired business outcomes derived from a governance program rather than the data itself.

Why is data governance important?

Without effective data governance, data inconsistencies across different systems in an organization may not be resolved.

For example, customer names may appear differently in sales, logistics, and customer service systems. That could complicate data integration efforts and create data integrity issues that affect the accuracy of business intelligence (BI), business reporting, and analytics applications.

Additionally, data errors may not be identified and corrected, further affecting the accuracy of BI and analytics. There are several reasons why organizations should have data governance:

• Avoid inconsistent silos in different departments or units.
• Agree on data definitions that can be shared and understood across the organization.
• Improve data quality through common efforts to identify and correct errors.
• Increase the credibility of the analyzes and offer decision makers valuable information
• Implement and enforce policies that help prevent errors and misuse of information
• Ensure alignment with privacy policies, data protection and other regulations

Poor data governance will also hamper compliance initiatives, which could cause problems for organizations that need to comply with new data protection and privacy laws, such as the European Union's GDPR and the European Union's Consumer Privacy Act. California (CCPA).

A data governance program typically results in the development of common data definitions and standard data formats that apply across business systems, increasing data consistency for business and compliance uses.

Benefits and objectives of data governance

A key goal of data governance is to break down data silos in an organization. These silos commonly build up when individual business units implement separate transaction processing systems without centralized coordination or a common organizational data architecture.

data governance has as purpose harmonize the data in those systems through a collaborative process, with stakeholders from the different business units involved.

Another government objective is to ensure that data is used correctly, both to prevent the introduction of data errors into systems and to block potential misuse of personal data about customers and other confidential information.

This can be achieved by creating uniform data usage policies, along with procedures to monitor usage and enforce policies on an ongoing basis. Additionally, data governance can help strike a balance between data collection practices and privacy regulations.

In addition to more accurate analytics and stronger regulatory compliance, the benefits of data governance include better data quality; lower data management costs; and increased access to data needed by data scientists, other analysts, and business users.

Ultimately, data governance can help improve business decision-making by providing better information. This leads to competitive advantages, higher revenues and profits.

Data governance roles

In most organizations, multiple people are involved in the data governance process. That includes business executives, data management professionals, and IT staff, as well as end users who are familiar with the relevant data domains in an organization's systems. These are the key players and their main governance responsibilities.

Chief Data Officer

The chief data officer (CDO), if there is one, is often the senior executive who oversees a data governance program and has high-level responsibility for its success or fracaso. The role of the CDO includes securing approval, funding, and staffing for the program, playing a leadership role in its establishment, monitoring its progress, and acting as an advocate for it internally. If an organization does not have a CDO, another C-suite executive will typically serve as executive sponsor and perform the same duties.

Data Governance Manager and Team

In some cases, the CDO or an equivalent executive (a director of enterprise data management, for example) may also be the director of the practical data governance program. In others, organizations appoint a data governance administrator or specifically direct them to run the program. Either way, the program manager typically leads a data governance team that works on the program full-time. Sometimes more formally known as the data governance office, it coordinates the process, leads meetings and training sessions, tracks metrics, manages internal communications, and performs other management tasks.

Data Governance Committee

However, the government team does not generally make decisions about policies or standards. That is the responsibility of the data governance committee or council, which is made up primarily of business executives and other data owners. The committee approves the fundamental data governance policy and the associated policies and rules on aspects such as data access and use, as well as the procedures to implement them. It also resolves disputes, such as disagreements between different business units over data definitions and formats.

data administrators

Data steward responsibilities include monitoring data sets to keep them in order. They are also in charge of ensuring that the policies and rules approved by the data governance committee are implemented and that end users comply with them. Workers with knowledge of particular data domains and assets are typically appointed to handle the data management function. That's a full-time job at some companies and a part-time position at others; There may also be a mix of IT and business data managers.

Data architects, data modelers, and analysts and engineers

Data architects, data modelers, and data quality analysts and engineers are also part of the governance process. Additionally, business users and analytics teams need to be trained on data governance policies and data standards so that they can prevent misuse or misuse of data. You can learn more about data governance roles and responsibilities and how to structure a governance program in a related article.

Components of the Governance Framework

A data governance framework consists of policies, rules, processes, organizational structures, and technologies that are implemented as part of the governance program.

It also details items such as the program's mission statement, its goals, and how its success will be measured, as well as the decision-making responsibilities and return for the various roles that will be part of the program.

Therefore, given so much information, it must be documented and shared internally to show how it works, so that it is clear to all those involved from the beginning.

On the technology side, data governance software can be used to automate aspects of program management. While governance tools are not a required component of the framework, they aid in its development by supporting schedule and workflow management, collaboration, governance policy development, process documentation, creation of data catalogs and other functions. They can also be used in conjunction with data quality, metadata management, and master data management (MDM) tools.

Implementation of data governance

The initial step in implementing a data governance framework involves identifying the owners of the various data assets in an enterprise and having them or designated individuals on their teams participate in the governance program. The CDO (Chief Data Officer), executive sponsor or data governance manager takes the lead in creating the structure of the program, working to staff the data governance team, identify data stewards and formalize the committee of government.

Once the structure is finished, the real work begins. Governance policies and data standards should be developed, along with rules that define how data can be used by staff with access.

• Data mapping and classification. Mapping data across systems helps document data assets and how data flows through the organization. Different data sets can be classified based on factors such as whether they contain personal information or other sensitive data. Classifications influence how government policies are applied.
• Business Glossary. A business glossary contains definitions of business terms and concepts used in an organization, for example, what constitutes a active client. By helping to establish a common vocabulary for business data, business glossaries contribute to governance efforts.
• Data Dictionary. Data dictionaries or catalogs collect metadata from systems and use it to create an indexed inventory of available data assets that includes data source information, search capabilities, and collaboration tools. Information about data governance policies and automated mechanisms to enforce them can also be incorporated into catalogs. (watch Keys to a data-driven organization)

Better Practices

As restrictions can be placed on how data is handled and used, it can become a challenge in organizations. A common concern among technology and data management teams is that business users may be perceived as “data police”.

To promote user buy-in and avoid resistance to governance policies, experienced managers recommend that programs be business-driven, with data owners involved and the governance committee making decisions about frameworks. of standards, policies and rules.

Data governance training and education is a necessary component of initiatives, particularly to familiarize business users and data analysts with data usage rules, privacy mandates, and their responsibility to help maintain consistency. of the data sets.

Ongoing communication with corporate executives, business managers, and end users about the progress of a data governance program is also a must, through a combination of reports, email newsletters, workshops, and other outreach methods.

There are a number of factors that help the success of a governance framework:

• A focus on business value and organizational results.
• Internal agreement on data responsibility and decision rights.
• A model based on trust in the origin and preservation of data.
• Transparent decision making that conforms to a set of ethical principles.
• Risk management and security included as core components of governance.
• Permanent education and training, with mechanisms to measure its effectiveness.
• A collaborative culture and governance process that encourages broad participation.

Professional associations that promote best practices in data governance processes include DAMA International and the Organization of Data Governance Professionals. The Data Governance Institute, an organization founded in 2004 by then-consultant Gwen Thomas, published a data governance framework template and a variety of guides on best governance practices.

Data governance challenges

The former are often the most difficult because different parts of an organization typically have divergent views of key business data entities, such as customers or products.

These differences need to be resolved as part of the governance process, for example by agreeing on common definitions and data formats. It is often a tense and contentious effort, so the data governance committee needs a clear dispute resolution procedure.

Other common challenges organizations face include the following items shown below.

show your commercial value. This challenge starts from the beginning. On an ongoing basis, demonstrating business value requires the development of measurable metrics, particularly improvements in data quality. That could include the number of data errors resolved on a quarterly basis and the revenue gains or cost savings that result from them. Other common data quality metrics measure accuracy and error rates on data sets and related attributes such as data integrity and consistency.

Support for self-service analytics. The self-service BI and analytics movement has created new governance challenges by putting data in the hands of more users in organizations. It is necessary to ensure that data is accurate and accessible to users, while ensuring that users (business analysts, executives, citizen data scientists, and others) do not misuse or conflict with data. data security and privacy restrictions. Streaming data used for real-time analysis further complicates this challenge.

Governing big data. The implementation of big data systems also adds new governance needs and challenges. Data governance programs traditionally focused on structured data stored in relational databases, but now must deal with the mix of structured, unstructured, and semi-structured data that big data environments often contain, as well as a variety of data platforms. data, including Hadoop and Spark. systems, NoSQL databases and cloud object stores. Additionally, big data sets are often stored in raw form in data lakes and then filtered as needed for analytical uses.

Key pillars of data governance

Data governance programs build on additional aspects of the overall data management process, including the following.

• Data Management. An essential responsibility of the data steward is to be responsible for a part of an organization's data, in areas such as data quality, security and use. Data stewardship teams are typically formed to help guide and execute the implementation of data governance policies. They are typically made up of data-savvy business users who are subject matter experts in their domains, although the data steward may also hold a technology role. Data administrators collaborate with data quality analysts, database administrators, and other data management professionals, while working with business units to identify data requirements and issues.

• Data quality. Improving data quality is one of the biggest driving forces behind governance activities. Accuracy, integrity, and consistency of data across systems are critical characteristics of successful initiatives. Data cleansing, or data cleansing, is a common element of data quality. It corrects data errors and inconsistencies, and also correlates and removes duplicate instances of the same data elements, thus harmonizing the various ways in which the same customer or product may appear in systems. Data quality tools provide these capabilities through data profiling, analysis, and comparison capabilities, among other features.

• Master data management. MDM (Marter Data Management) is another management discipline that is closely associated with governance processes. MDM initiatives establish a master set of data about customers, products, and other business entities to help ensure data is consistent across different systems in an organization. MDM is a natural fit with government. However, like government programs, MDM efforts can create controversy in organizations due to differences between departments and business units on how to format master data. Additionally, the complexity of MDM has limited its adoption compared to data governance. But the combination of the two has led to a shift towards smaller-scale MDM projects driven by data governance goals.

• Use cases. Effective governance is at the heart of managing the data used in operating systems and BI and analytics applications. It is also an important component of digital transformation initiatives and can help with other corporate processes, such as risk management, business process management, and mergers and acquisitions. Compliance with GDPR and CCPA privacy directives is another new use case for data governance. Disseminating these success stories internally helps establish a common culture in organizations.

Data Governance providers and tools

Data governance tools are already available from various providers.

That includes major technology providers like IBM, Informatica, Information Builders, Oracle, SAP, and the SAS Institute, as well as data management specialists like Adaptive, ASG Technologies, Ataccama, Collibra, Erwin, Infogix, and Talend.

In most cases, governance tools are offered as part of larger suites that also incorporate metadata management features and data source functionality.

Data catalog software is also included in many of the data governance and metadata management platforms.

As in the previous case, it is available from specialized providers with an independent product, for example Alation, Alteryx, Boomi, Cambridge Semantics and Data.world.