How Data Quality Is Powering M&S Food's Transformation

Callum Staff, Head of Data Science and Analytics, Marks & Spencer

Callum Staff, Head of Data Science and Analytics, Marks & Spencer

Data quality and governance is often viewed as bureaucratic, nice-to-haves in a project or organisation. However, if well implemented, properly owned by business teams, and integrated into existing processes, it drives significant, and crucially measurable, value placing data at the heart of decision making.

M&S is a 139-year-old retailer, and to make sure we are around for another 139 years we need to adapt and transform how we do things while staying true to what makes M&S special. In short that means protecting the magic of M&S while modernising the rest. Protecting the magic means holding true to the things our customers love about M&S: our leading product innovation, our quality and exceptional sourcing standards, and our longstanding reputation for doing the right thing by our people and the planet. Modernising the rest means reshaping our business and removing the impediments to growth, with our focus in the M&S Food data team on ensuring our teams are properly equipped to make the informed decisions, which drive our transformation.

However, up until recently we have not been able to depend on the quality or accessibility of data to achieve this. So in March 2021 we initiated a two-and-a-half-year data quality change programme to improve our data governance and quality. The programme is far from over, but we are already seeing the benefits. For example, we now have reliable and easily accessible product attribute data enabling us to automate a dashboard our nutritionists use. This now takes seconds instead of weeks, meaning our nutritionists now have the latest data at their fingertips helping them make informed decisions on our health strategy – a key pillar in our food transformation. This is one example of a number of areas where the programme is overhauling our data through its three key workstreams – the 3 Ps:

• People – setting teams up in the right way and equipping them with the right skills to effectively tackle data quality and governance problems

• Processes – obtaining technology and designing frameworks, before embedding both within teams, to allow the correct management of data as a business asset

• Programmes – doing the work! Supporting key programmes of work as part of M&S’ wider transformation but also delivering one off projects to move critical datasets to an acceptable baseline quality The key purpose of the programme and the 3 Ps – outlined in more detail - is for us to manage our data with the same respect as the physical products in our stores, where our customers expect exceptional quality and sourcing standards.

People

Increasing data quality and governance capability starts with developing teams’ skills. This has started with our Food Data Quality team – moving their skillset from manual data entry and correction to skills akin to Commercial Analysts or BI Developers. The first step was freeing up the team’s time to conduct more value-add data quality analysis by better integrating systems or using Robotic Process Automation, changing system architecture to cut out processes entirely, or passing the responsibility of data entry or data incident management to the business owners of the process who have the full context. We want to give teams autonomy to be creative and focus on problem solving, not data entry.

"Protecting the magic means holding true to the things our customers love about M&S: our leading product innovation, our quality and exceptional sourcing standards, and our longstanding reputation for doing the right thing by our people and the planet"

This has freed up time to focus on more valuable analysis. To support this we are investing in development programmes to develop existing team members’ analytical skills, chiefly technologies such as SQL, PowerBI, and the data quality tool Ataccama, and analytical curiosity - developing, exploring, and communicating the results of hypotheses. In terms of collaboration, the team works within a wider data team, consisting of data science and reporting functions, allowing us to leverage a range of skills to tackle data quality problems.

More widely, we’re clarifying accountability of data within business teams – identifying data owners who are commercial responsible for ensuring quality data. This is not about adding lines to a job description but demonstrating to people the commercial impact of managing their data well. As the other side of the bargain, we are providing them with the processes, tools, and people support to manage that data effectively – as demonstrated in the sections below.

Processes

To maximise productivity from empowered individuals and teams with the right skills, the right tools need to be available. Across all teams associated with data management and across all stages of the data lifecycle, we are looking to embed data governance as efficiently as possible.

A key role of the programme governance has been to prioritise what to tackle first by identifying a range of ‘data domains’. These align to business functions, such as marketing, supply chain, or nutrition, as opposed to aligning to software and systems. Saying ‘we are going to tackle supply chain data’ is far more tangible to business colleagues than saying ‘we are going to tackle data in this software’. We have visualised this for colleagues with a tree-like diagram, split first into reference and operational data, then high level domains like the areas mentioned above, and then sub-domains within those.

After identifying our domains our first initiative has been to unify where all over our data sits for analytical purposes. We are in the process of moving from multiple, disparate on-premise data stores to one cloud solution, known as BEAM. This is already yielding far more effective and efficiency data governance and analysis, and after identifying data domains, is an absolute cornerstone to our programme.

Accessibility to data governance processes for all colleagues is an important aim of the programme, and a day-to-day example of this is the backlog where we record data quality issues. Supported by our reporting team, the data quality team have built an application allowing users to log data quality issues as they discover them to assist us with triaging and prioritisation. By crowdsourcing data issues we can be more responsive to business needs in our support and gain a far truer picture of the state of data in M&S Food.

The initiatives above have been foundational in better managing our data, but it is two pieces of technology that have improved our data quality analysis capabilities. Firstly, we have implemented a tool called Ataccama, which allows assessment of large datasets against business-defined rules, scheduling of these tests at specific frequencies, and workflow management to notify data owners of resulting issues. This automation of data quality assessments has supplemented our typical data analysis suite such as SQL, Python, and PowerBI. Secondly, we have a tool called Alation, allowing us to create a Wikipedia of data. This has made understanding our data more accessible to colleagues through data definitions, lineage, calculations, and limitations. The development of resources like this are important in facilitating usage and so are part of our ‘definition of done’ for datasets. In both instances other tools are available, but it is the capability development that is the important point here.

Pharma companies want to see that Operational Excellence and Continuous Improvement organizations and plans are in place and efficient

The final element of the process section is about measuring the value delivered from the scheme. We are developing weekly reporting for business colleagues on the state of data quality in each data domain and using data science methods to measure the commercial impact of improving our data quality. These capabilities allow us to track overall data quality improvements, hopefully demonstrating long term sustainable changes, but also allow us to respond when domains require additional support.

Programmes

The people and process sections are about empowering our colleagues and making sure we have the right systems in place to access the data they need for well support decision making – and ensuring we act always with our customers’ front of mind. The programmes section is about putting this into practice. As part of M&S’ wider transformation journey, there are several transformation workstreams within the Food business where data quality improvements are a central element so that we can ultimately better serve our customers. The best example of where data quality improvements have underpinned value generation is the transformation of our Food supply chain, both the overall operating model and the systems that support it – across the supply chain good quality data is vital for human and automated-decision making.

As well as project delivery, there is a wealth of data used to support daily operations which also requires management, such as ensuring there is alignment in store opening hours across different systems – our website, supply chain, and labour planning for example. Based on business priorities, we are conducting audits of critical datasets – assessing the state of data, delivering short-term remedial work on data where possible, and if required detailing longer term structural changes to ensure sustainable data quality (e.g. building data validation into a data entry tool), measuring the operational and commercial impact of data quality changes, and then embedding them with business teams. This final stage is the crucial one in achieving sustainable data management – ensuring teams are knowledgeable, empowered, and equipped enough to lead in managing their data, with support from specialist teams such as the Food Data Quality team.

Our programme is far from finished, but we can already start to see business benefits, and are aiming by September 2023 to be in a place where data is governed as part of business processes, not outside of them. This is not a capital delivery project of a piece of technology, nor a people programme to change organisational structures, but ultimately a change in culture. We are aiming to change attitudes towards data – if you own a sellable product or business process, you own the data associated with it, and we will support you in leveraging value from it.