{"id":40439,"date":"2020-10-15T08:02:29","date_gmt":"2020-10-15T07:02:29","guid":{"rendered":""},"modified":"2025-05-14T13:05:30","modified_gmt":"2025-05-14T12:05:30","slug":"how-to-build-and-deliver-an-effective-data-strategy-part-3","status":"publish","type":"post","link":"https:\/\/newed.any0.dpdns.org\/en-gb\/industry\/blog\/cross-industry\/2020\/10\/15\/how-to-build-and-deliver-an-effective-data-strategy-part-3\/","title":{"rendered":"How to build and deliver an effective data strategy: part 3"},"content":{"rendered":"

In part one<\/a> of this three part blog series, we took a detailed look at the importance of building a data strategy. We started laying out the steps to build a successful strategy that builds a data-driven culture and aligns with business outcomes. In part two<\/a>\u00a0we covered capability and maturity models. In this final blog we\u2019re focussing on how you can execute a strategy, by taking a principled approach for the end-to-end data lifecycle and aligning technology choices accordingly. We\u2019ll also look at the importance of ethics and responsibility in data and analytics.<\/p>\n

Take a principled approach to your data strategy<\/h2>\n

With\u00a0<\/span>data governance<\/span>\u00a0in mind<\/span>,\u00a0<\/span>the following diagram shows the essential stages for data lifecycle,\u00a0<\/span>please consult the attached document\u00a0<\/span>on\u00a0<\/span>\u201c<\/span>A Guide to Data Governance<\/span>\u201d<\/span>.<\/span>\u00a0An effective data strategy must have provisions for data governance, but they\u00a0<\/span>are\u00a0<\/span>mut<\/span>ually inclusive, but not the same thing.<\/span>\u00a0<\/span><\/p>\n

For the purpose of this blog, l<\/span>et\u2019s take a closer look at the considerations to take when building design principles for the four layers<\/span>\u00a0that are key in your<\/span>\u00a0data strategy, focused towards delivering business outcome and value from data.<\/span>\u00a0<\/span><\/p>\n

1. Data ingestion<\/h3>\n

A key consideration for data ingestion is the ability to build a data pipeline extremely fast, from requirements to production, in a secure and compliant manner. Elements such as metadata driven, self-service, low-code technologies to hydrating your data lake are key. When building a pipeline, consider design, the ability to do data wrangling, scale compute and also data distribution capabilities. In addition, having the right DevOps support for the continuous integration and delivery<\/a> of your pipeline is critical. Tools such as Azure Data Factory<\/a> support a plethora of on-premises, Software as a Service and data sources from other public clouds. Having such agility from the get-go is always helpful.<\/p>\n

2. Storage<\/h3>\n

Data needs to be tagged and organised in physical and logical layers. Data lakes are key in all modern data analytics architectures. The Hitchhikers guide to the data lake<\/a> is an excellent resource to understand different considerations that companies need to make. Organisations need to apply appropriate data privacy, security and compliance requirements based on data classification and the industry compliance requirements they operate in. The other key considerations are cataloguing and self-service to aid organisation level data democratisation to fuel innovation, guided by appreciate access control.<\/p>\n

Choosing the right storage based on the workload is key, but even if you don\u2019t get it right the first time, the cloud provides you options to failover quickly and provide options to restart the journey reasonably easily. You can choose the right database<\/a> based on your application requirements. When choosing an analytics platform<\/a>, make sure you consider the ability to process batch and streaming data.<\/p>\n

3. Data processing<\/h3>\n

Your data processing needs will vary according to workload, for example most big data processing<\/a> has elements of both real-time and batch processing. Most enterprises also have elements of time series processing requirements and the need to process free-form text for enterprise search capabilities.
\nThe most popular organisational processing requirements come from Online Transaction Processing (OLTP). Certain workloads need specialised processing such as High Performance Computing (HPC), also called \u2018Big Compute\u2019. These use many CPU or GPU-based computers to solve complex mathematical tasks.<\/p>\n

For certain specialised workloads, customers can secure execution environments such as Azure confidential computing<\/a> which helps users secure data while it\u2019s \u2018in use\u2019 on public cloud platforms (a state required for efficient processing). The data is protected inside a Trusted Execution Environment (TEE), also known as an enclave. This protects the code and data against viewing and modification from outside of the TEE.\u00a0 This can provide the ability to train AI models using data sources from different organisations without sacrificing data confidentiality.<\/p>\n

4. Analytics<\/h3>\n

Extract, transform, load construct or otherwise called ETL (or ELT depending on where the transform happens) relates to online analytical processing (OLAP), and data warehousing needs. One of the useful capabilities here is the ability to automatically detect schema drift. Consider end-to-end architectures like automated enterprise BI with Azure Synapse Analytics and Azure Data Factory. To support advanced analytics including Machine Learning and AI capabilities, it is key to consider the reusability of platform technologies already in use for other processing requirements of similar nature. Here is a quick start guide<\/a> with working example for end-to-end processing.<\/p>\n

Batch processing on Databricks, R, Python or for deep learning models are common examples. Compute, storage, networking, orchestration and DevOps\/MLOps are key considerations here. For super large models, look at distributed training of deep learning models on Azure<\/a> or the Turing Project.<\/a> You also need to consider the ability to deal with data and model drift<\/a> too.<\/p>\n

Azure Enterprise Ready Analytics Architecture<\/a> helps collate all the four layers together with people, process, security, and compliance. We also suggest using the recommended architectures from Azure Landing Zones<\/a> to get started. It uses the Microsoft Cloud Adoption Framework and culminates our experience working through thousands of large scale enterprise deployments.<\/p>\n

Now that we have covered the four stages,\u00a0<\/span>the following\u00a0<\/span>representation shows the key capabilities needed on top of your data platform to provide end to end data governance capability.\u00a0<\/span>\u00a0<\/span><\/p>\n

Here are some resources to help you build a data strategy that accounts for governance and also allows for seamless <\/span>innovation.<\/span>\u00a0<\/span><\/p>\n