Building critical data products? Sign up for our upcoming guide

Logo

Building Reliable Data Products

A practical guide for building analytical data products that power business-critical automation or customer-facing applications.

Foreword

When analytics datasets power operations or customer facing systems there is no stakeholder to flag the errors. When data goes wrong, the business goes wrong. It puts a new demands on data teams to ensure data products work 24/7.

Neither anomaly monitoring, data contracts, data diffing or data testing is a silver bullet to achieve data reliability. Instead, we need a strategy that considers how these techniques fit together.

— Petr Janda, CEO & founder of SYNQ

Table of Contents

Chapter 1

The Reliability Challenge

Outlining the clear relationship between data reliability and business impact.

Chapter 2

Setting expectations

A framework to define data products to manage tiers of data assets and SLAs for maintenance.

Chapter 3

Building the Foundation

An approach to robust platform architecture that creates the right interfaces.

Chapter 4

Proactive Testing & Monitoring

We build a testing and monitoring framework that that maximises errors caught and minimises alerts.

Chapter 5

Ownership with rapid response

Developing scalable ownership and efficient incident management processes to quickly resolve issues.

Chapter 6

Continuous Improvement

Establishing feedback loops and learning processes to continuously enhance data reliability practices.

About the Authors

After leading data and software teams at Google, GWI, Mathem, Pleo, Monzo or Epidemic Sound we have seen what it takes to deliver data products that have profound impact on the business and can't fail. We've accumulated countless learnings and best practices that we're transforming into a framework to build more reliable data products.

Petr Janda

Petr Janda

CEO, co-founder

Mikkel Dengsoe

Mikkel Dengsoe

Chief Evangelist, co-founder

Robert Sahlin

Robert Sahlin

Product Lead