Data Governance – a key enabler of cloud strategy and adoption

As the volumes of data created and used by businesses continues to grow, cloud-based infrastructure appears to be the logical solution to increasing demands for processing power and storage.
Cloud platforms offer scalability, flexibility and promise to open up new levels of insight and analysis to organisations seeking a competitive edge or point of difference. To realise these benefits, however, requires careful planning and implementation: traditional data governance frameworks must be extended and enhanced to ensure that the new complexities of the cloud environment are taken into account while regulatory and compliance requirements are also met.

Some of the key challenges include:

1. Fragmentation of architecture

Implementing cloud-based solutions almost always requires a degree of integration with existing systems. Often organisations will find themselves with a mix of cloud and on-premises architecture, which introduces extra complexity as data must be able to move between these systems. As well as the challenges of interoperability, the needs and practices of different business units can introduce further complexity. For instance, without a centralised approach to data governance, organisations may find themselves trying to manage conflicting or redundant data flows across multiple platforms with specific ingestion and integration requirements servicing specific areas of the business.

2. Data integration

Cloud-based infrastructure, when coupled with the right processing tools, offers a tantalising vision of structured, unstructured and semi-structured data all ingested and analysed with minimal development or configuration efforts. In reality, hybrid architecture and multiple technologies demand a rigorous approach to data quality and integration. Without a consistent and managed approach to master data, data quality, and metadata, organisations can easily find themselves with siloed and incompatible data. While the hope is that access to multiple data sources via a single platform will allow for improved analysis, the reality is that combining disparate and inconsistent data will yield inaccurate results, leading to frustration and disappointment when the promised insights fail to emerge.

3. Data Duplication

The capacity of cloud to store and transfer large volumes of data in multiple formats can lead to the creation of multiple bespoke copies of data for different business uses. While enabling projects to customise their data has benefits, organisations must be prepared to track and monitor these copies to ensure that they remain compliant with data quality and master data requirements if they are to be useable for business reporting. A fragmented approach to data governance allows overlapping but inconsistent data copies to be created. In turn, the metadata captured and maintained about each copy is specific to its project, making it a very complex undertaking to attempt to determine the relative reliability of each copy when choosing a source for business reporting activities.

4. Security and privacy

The obligation to maintain the security and privacy of data remains whe norganisations implement cloud solutions. Security breaches and unauthorised access of data can lead to significant reputational harm and fines from regulators. When choosing vendors and moving to cloud-based services, organisations must be confident that they will be able to apply appropriate levels of protection for all their data assets. This includes ensuring that sensitive data that is not permitted to be stored or processed offshore remains locally held and that permissions are enforced. Additionally, organisations need to consider the degree to which they are able to make sensitive data discoverable via indexing or available as part of self-service BI and reporting.

Case study: Enterprise Data Warehousing

A large organisation with a complex data environment implemented an Enterprise Data warehouse but did not invest sufficiently in governance at the time that the project commenced. No centralised controls over the data model or business facets were put in place, which allowed overlapping entities with inconsistent copies of data to be created. The inconsistent approaches to data quality maintenance and metadata collection led to problems in monitoring and reporting on data quality across the platform. Core data sets were not managed as master data, compromising the ability of the organisation to create a single view of customers and core services. Additionally, multiple and overlapping data flows were created to service particular projects. Over time, it became complex and costly to make changes on the platform. Build and maintenance costs also increased due to the fragmented nature of the architecture.

Figure 1: A sample data governance framework

Data governance offered ways to resolve a number of these challenges. The first step was to implement a centralised approach to the hybrid data architecture. A single governance model also allowed for consistency to be introduced when managing copies of data between the data lake and access layers. In turn, this improved reliability which as an outcome enabled improved analytics and business reporting. Finally, a data governance group was introduced to support data ownership throughout the organisation. This centralised governance enabled the development of authoritative master data, which supported projects such as customer journey modelling.  

Holistic data governance

To effectively manage the complexities of working in a cloud-based or hybrid environment, data governance must be embedded into any cloud adoption strategy. To be effective, however, governance must be an enterprise-wide undertaking in order to avoid the problems of fragmentation and conflicting standards highlighted above. Many organisations will benefit from undertaking an audit of current data governance practices and cataloguing current data holdings in order to gauge the existing levels of maturity and identify any centres of excellence whose practices can be leveraged.

Figure 2: The role of data governance

A comprehensive data governance strategy will address issues such as data architecture, master data, data quality, metadata, and the use of appropriate standards to support the organisation to ensure that data is fit-for-purpose, discoverable and useable to the extent allowed by privacy and security controls. At the same time, data governance responsibilities, performed by such roles types including data custodians or champions and data owners, can augment strategic efforts by providing ownership and accountability as policies are developed and implemented.

Cloud-based solutions have great potential for organisations with growing needs in storage space and processing power. However, it is naïve to imagine that migrating “to the cloud” solves the familiar problems of managing data effectively. Rather, cloud implementations offer an opportunity for organisations to review their data governance practices, build policies and processes that will set a solid foundation and enable cloud-based solutions to leverage their full potential.

Want to stay up-to-date?

Stay on trend and in the know when you sign up for our latest content