Introduction

We’ve led data and software teams at Google, GWI, Pleo, and Monzo and spoken with thousands of data teams over the past years. Over again we see the same struggles from data teams – they lack a framework to support the growing expectations of use cases from data in their companies.

More is expected from data teams #

Data is used for business-critical use cases #

Data has evolved from being a nice-to-have to becoming a critical component of core business processes. Data used for decision-making is important and if data is incorrect it may lead to wrong decisions and over time a loss of trust in data. But data-forward businesses have data that is truly business-critical. If this data is wrong or stale you have a hair-on-fire moment and there is an immediate business impact: Tens of thousands of customers may get the wrong email as the reverse ETL tool is reading from a stale data model. You’re reporting incorrect data to regulators and your C-suite can be held personally liable. Or your forecasting model is not running and hundreds of employees in customer support can’t get their next shift schedules before the holidays.

Data stacks are getting more complex #

15% of dbt projects have more than 1,000 models, and 5% have more than 5,000 models. These aren’t just large numbers—they represent growing data ecosystems connected to hundreds of upstream data sources and feeding hundreds of downstream destinations. Data Mesh has gained traction as a solution for scaling data work but teams struggle translating the principles into actions.

This scale comes with a cost. Speed and agility take a hit, leading to frustration both within the data team and among stakeholders. Collaboration becomes increasingly challenging as no one has full visibility into the entire codebase. Meetings start consuming more time than actual data work. Ensuring quality becomes more difficult across an expanding surface area, resulting in a rise in user-reported errors. SLA performance also declines as job failures become more frequent, with retrospectives offering little to reverse the trend.

Lack of testing framework #

The root cause of the problem is that data teams are not well-equipped to take a strategic approach to building reliable data. This leads to tests being placed sporadically, with little consideration for the use case of the data, and unclear ownership, creating ‘broken windows’—tests that remain failing for too long.

As a result, data teams become overwhelmed by alerts, stakeholders are often the first to uncover issues, and confidence in the value of testing begins to erode.

The Data Product Reliability Workflow #

We recommend you think systemically about the reliability of your data stack through a 5-step framework – from defining data use cases as products, setting ownership & severity, deploying strategic tests & monitors, and establishing quality metrics. We call it the Data Product Reliability Workflow.

In this guide, we’ll work you through actionable steps you can take to adopt the Data Reliability Workflow at your company, including specific examples, tips & tricks, and pitfalls.

• Setting expectations: A framework to define data products to manage tiers of data assets and SLAs for maintenance.
• Proactive testing & monitoring: We build a testing and monitoring framework that maximizes errors caught and minimizes alerts.
• Ownership with rapid response: Developing scalable ownership and efficient incident management processes to quickly resolve issues.
• Continuous improvement: Establishing feedback loops and learning processes to continuously enhance data reliability practices.

Reliable data isn’t just an operational requirement, it’s a competitive advantage. By prioritizing trust and quality, your team will be trusted to own new critical processes and move faster.

Happy reading!

Petr & Mikkel