Ever since reading Toby’s announcement on The future of DataOps, we were excited. The SQLMesh approach to data transformation resonated strongly with us.

SQLMesh detaches models from physical tables through a novel concept called virtual data environments, which opens opportunities for new efficiencies and workflows. Similarly at SYNQ, we’ve been building observability from day 1 with data models as first-class citizens, assuming that tables will freely change in the background, an approach different from the other platforms on the market—this was a great fit.

SYNQ now integrates with SQLMesh, becoming the first data observability platform that supports their data transformations with modern data observability. We’re committed to continue supporting analytics engineers across platforms and look forward to expanding this new integration together with Tobiko and our shared customers.

What is SQLMesh #

SQLMesh is a next-generation data transformation and modeling framework. It aims to be easy to use, correct, and efficient and is maintained by the Tobiko Data team. It helps you scalably, reliably, and safely modify your data pipelines because it understands SQL and can make intelligent updates instead of stringing scripts together. SQLMesh boasts several future-proof features, such as automatic data contracts, virtual data environments and snapshots, extensive change summaries (before updates are applied!), and column-level lineage out of the box.

What is SYNQ #

SYNQ is a ‘Data Product Observability Platform’ built for teams that own business-critical data. By embedding the concept of Data Products into observability, SYNQ helps teams manage data quality in the context of business-critical use cases. With models and metrics as core constructs, SYNQ enables observability workflows that go beyond traditional table-centric approaches, supported by features like anomaly monitoring, data domains, ownership, incident management, and data quality reporting.

Why SQLMesh matters #

We’ve seen teams adopting dbt to transform how they work with data. SQLMesh takes many of the same concepts but also brings new ideas to the table such as:

• 💡Transpile SQL (a.k.a. SQL translation) to run transformations across different engines.
• 💡Understand code impacts on tables before running transformations, saving time, and cost.
• 💡Trace how columns in one table contribute to others for detailed upstream and downstream impacts.
• 💡Execute only impacted transformations instead of re-running entire DAGs.
• 💡Enable Python-based workflows for tasks like ML and geocoding.

We’re excited about those too, as they tackle some of the pain points we hear from our customers!

SQLMesh & SYNQ #

Integrating both platforms lets you model your data following DevOps best practices and combine it with an observability layer to ensure that your data is reliable. SYNQ automatically ingests all SQLMesh model properties and metadata so you can manage data products and ownership definitions seamlessly, whilst deploying monitors through code.

Combining both tools helps you model your data following DevOps best practices while ensuring you deliver reliable data products to the business.

SQLMesh + SYNQ – build and model reliable data with a unified workflow

The following workflows demonstrate why SQLMesh and SYNQ play well together:

• SQLMesh models as Data Products–SYNQ automatically ingests data assets from SQLMesh, including SQL models, snapshots, and seeds, and lets you build data products that give you visibility of the health across your data stack.
• Enhance tests and audits with monitors–use SQLMesh model tags to define SYNQ monitor deployments and combine tests and audits with SYNQ’s advanced anomaly detection to catch both ‘known unknowns’ and ‘unknown unknowns’.
• Activate ownership–use SQLMesh model properties such as owner or tags to bring in ownership to SYNQ. Route test or audit issues alongside other data issues and avoid dispersed alerting workflows.
• Manage data incidents–treat test or audit failures as incidents, track downstream impacts, and manage communication from detection to resolution.

– SQLMesh — learn more about SQLMesh here.
– SYNQ — learn more about SYNQ’s SQLMesh integration.

To learn more, join us for our fireside chat on January 23rd where Petr Janda (Founder and CEO of SYNQ) and Tobias (Toby) Mao (Co-Founder and CTO of Tobiko) will discuss our partnership in more detail.