Poor Data Governance: Why Customer Data Integration Projects Fail Part III
Recently, many a large enterprise has embarked on a Customer Data Integration initiative to gain unified views of its customers and their relationships across products, locations and business lines. With the demand for data integration hubs increasing in the past few years, several CDI vendors now vie for a leadership position in the emerging market. As these vendors offer an array of packaged solutions with rich features and services, it is easy to lose sight of the most fundamental requirement of a CDI hub -- sound customer data management.
Creating unified customer views across conflicting, disparate data sources is the raison d' être for all CDI implementations; therefore, ensuring that the data in the hub is reliably consolidated, its exceptions properly stewarded and data policies properly implemented -- in short, the data is well governed -- has to be one of the most critical requirements of the solution. Yet, companies often evaluate these data governance capabilities inadequately in their vendor selection process only to regret their decision later in the implementation phase.
Therefore, it is critical for enterprises to review the architecture of a CDI solution closely to determine whether it meets all their data governance needs in the long run. So, as you embark on a solution, consider these three factors that underlie data governance capabilities: data reliability, data stewardship and governance regimes.
Make sure these are all part of a flexible architecture platform.
Data Reliability: Built-in or Bolted-on?
Most application-centric CDI solutions that focus primarily on the operational use of data often underestimate the harder challenges of building a reliable high-volume customer hub in the first place or of managing different data governance regimes across multiple lines of business. While external tools can be loosely integrated to such solutions to cleanse and match data, the more intractable problem is that of merging matched records to create the "best of breed" master record for each customer. Essentially, a CDI solution that promises a scalable, operational hub but has bolted-on data quality tools is going to incur ever increasing costs for data management over time.
In order to deliver a "golden" or master record for each customer and its various affiliations, a system must dynamically assess reliability across all data sources -- based on user-defined parameters and intrinsic data properties -- and ensure that only the most reliable content survives at the cell-level of the master record.
For instance, if the call center begins collecting e-mail addresses when confirming orders, this data attribute may be more reliable than the e-mail addresses submitted by customers at the Web site. The ability to rapidly adjust the system to survive the call center e-mail address over the Web site e-mail address is a critical architectural component of any CDI system. Moreover, such cell-level survivorship for data reliability must be built into the core product architecture and should not be sacrificed as the customer hub scales to millions of customers. Ultimately, how well the end-users accept a customer data hub depends on sustaining a high level of data reliability, even as the hub grows in volume or as new data sources are added.
Data Stewardship: Out of the Box or Custom?
A business cannot implement an operational customer hub in the absence of data stewardship -- as soon as data begins to flow through the supported business processes, exceptions and errors begin to flow as well. Therefore, any customer hub acting as a data integration platform must offer business capabilities to monitor and handle such exceptions either by business analysts and end-users or by anyone designated as a data steward.
Today, many CDI solutions overlook the real-time configurable rules needed to trigger alerts about the exceptions that are created during data flow. Often managing such exceptions requires full user interfaces for complex data stewardship tasks such as investigating the history and lineage of certain master records in question.
This may be the only way to ensure that the user acceptance and data hub reliability remains high. In other circumstances, an enterprise may choose to build a specific user interface against the programming interfaces of the master hub in order to suit its needs.
In either case, an adaptive solution must deliver rules-based configurability with best-in-class data stewardship consoles as well as programming interfaces to handle all data exception and reliability needs.
Data Governance Regime: Central, Distributed or Both?
Most CDI solutions are designed to create a single silo, an operational hub to serve a handful of applications, as opposed to one that can scale to the needs of the enterprise. While a focused deployment may be the desired option for central data governance, it usually does not address all the business needs for governance and compliance across an enterprise. Often, certain data attributes (such as privacy preferences) need central control and exception handling whereas other attributes are best left under local management.
In addition, security and access to data attributes in the hub will vary by individual roles within each organization and by organization at large. In fact, to support the broad range of business requirements across business lines, there may be multiple data governance regimes required for different data attributes, all within a single enterprise.
An adaptive approach must be based on a distributed architecture whereby multiple hubs can be deployed to integrate different data sources and support different processes, yet be able to share data across one another based on any number of hub-and-spoke or peer-to-peer governance regimes.
This offers a line of business, yet another dimension of flexibility to share some but not all data -- each based on its own data reliability and governance policies. With full rules-based configurability and data stewardship interfaces, a broad range of data governance regimes can be supported.
Data Governance Requirements: Coded or Configurable?
As the pace of change in business continues to increase, so does the complexity of maintaining data quality and reliability across the enterprise. In master data hubs that are custom-built or based on fixed models, it is very difficult to implement changes in either business logic or process because the rules reflecting the business conditions and requirements are usually custom-coded or tightly bound to the underlying fixed data model. To automate the consolidation of customer data in an intelligent and ongoing manner and maintain the highest level of data quality and reliability in the face of business changes, it is critical for an enterprise to be able to manage the repeatable business rules for its data cleanse, match and merge processes effectively.
Consequently, an adaptive master data solution must capture and manage these changing business conditions without the IT team having to custom code these rules each time change is desired. In addition, all the business rules for data consolidation, change detection and write backs must be easily configured and rapidly modified, without additional programming, to reflect specific business conditions and the associated data needs. This results in a solution that can be governed as business changes occur during the normal course of operation, without compromising the data integrity across multiple source systems.
In summary, data must be the central focus of any CDI project, and data reliability, stewardship and governance regime must never be an after-thought of a comprehensive CDI solution. Enterprises must closely review the details of a vendor's offering in this area to reduce their risk of project failure and the total cost of CDI implementation.