Auditing analytics without manual test plans: four approaches

You can stop maintaining manual test plans, but you can't stop specifying what "correct" means. Every audit tool moves that work somewhere: scripted journeys, a schema in code, a baseline learned from live traffic, or warehouse tests. The right choice depends on which of those you can afford to maintain, and on where your breaks actually happen.

Disclosure: I am the Chief Revenue Officer at Trackingplan, one of the tools in this category. I evaluate it with the same criteria as the others, and I say where it is not the best fit.

How I compared them. Eight tools, the same eight criteria: validation approach, what you have to configure, when it detects a break, root-cause help, CI/CD integration, coverage, pricing model, and best fit. Everything comes from each vendor's public website and documentation, read on 26 October 2026, with links below. I have not run a side-by-side test, so the weaknesses are structural, not benchmark results.

The four approaches solve different problems

The usual "best tag auditing tools" list mixes products that don't do the same job. They differ on two things: where they look and who defines what is correct.

1. Scheduled synthetic testing (ObservePoint, DataTrue). A robot browser crawls pages or replays scripted journeys and checks what tags fire. You define correct with rules and journeys. It looks at the browser, but only on the paths you scripted.
2. Schema-driven governance (Avo, Amplitude Data, the product that came from Amplitude's acquisition of Iteratively in May 2021). The tracking plan becomes code: developers call generated, typed functions, and CI fails if an event is missing. You define correct in the plan, before anything ships.
3. Live-traffic observability (Trackingplan). A tag or SDK listens to the requests your site, apps and servers already send to analytics and ad vendors. The tool infers a baseline from that traffic and alerts on deviations. It looks at production, at everything real users trigger.
4. Warehouse data quality (Monte Carlo, Great Expectations, dbt tests). Checks run on tables after data lands in BigQuery or Snowflake. They see what was modeled, never what the browser sent.

None of them replaces the others completely. A crawler cannot see a payment flow nobody scripted. A schema cannot see a GTM tag a marketer added on Friday. Live monitoring cannot see a broken dbt model. A warehouse test cannot see a pixel that fired before consent.

Amplitude's own guide to data validation techniques, first published on the Iteratively blog in 2020, makes the case for validation at the client, the pipeline and the warehouse. That layered view still holds. What has changed since is how much of the client layer you can cover without writing the tests yourself.

The comparison matrix

The first table covers how each tool works. The second covers fit and cost.

Two status changes matter if you read older comparisons. GX Cloud is no longer sold: Great Expectations announced in May 2026 that FICO is acquiring it, that it stopped being publicly available on 1 June, and that Fivetran is now the steward of GX Core. And Fivetran and dbt Labs completed their merger on 1 June 2026. dbt Core and dbt Cloud are now one product, called simply dbt.

Scheduled synthetic testing: ObservePoint and DataTrue

Both tools send a real browser to your site and check what fires. Neither watches what real users do. That is a strength before launch and a gap afterwards.

ObservePoint

ObservePoint runs audits, which crawl pages, and journeys, which follow scripted paths through forms and funnels. You attach tag and variable rules that say what should fire and with which values.

• Strengths: it can test GTM changes before publishing by swapping a development container into the page. Its consent testing checks whether your CMP actually blocks trackers before consent or after a refusal, simulating EU and US visitors. For a privacy or legal team, scheduled, documented scans are evidence.
• Weaknesses: it only checks the pages and journeys you configured. Audits and journeys can be scheduled from every few minutes to quarterly, but only check what you configured. Journeys need upkeep, and they auto-pause after repeated errors when the site changes. App coverage goes through a device proxy, not real users. CI/CD runs through the API, not a native plugin.
• Not for: teams that want to know whether real users' events broke today.

DataTrue

DataTrue runs coverage scans and step-by-step journeys in real browsers, and it is partly a managed service: it builds the tests from a solution design you approve. Its data layer validation checks that the object exists, has the right structure and holds the right values at every step.

• Strengths: strong data layer checks, including pushing edge-case values to see how tags react. It tests before publishing against an unpublished GTM or Adobe Tags container, and from CI through its API. It covers native apps with SDKs and server-side GTM output. Pricing is public.
• Weaknesses: "continuous monitoring" here means scheduled re-tests, about hourly at best, not real-traffic monitoring. Test visits reach your vendors and can inflate analytics unless you filter them. Bot-protected sites need to allowlist its IPs.
• Not for: small budgets. The entry plan is $16,000 a year.

If you need the manual end of this spectrum too, extensions and one-off audits, the GA4 audit and error-detection tools guide covers them. OptimizeSmart's tag auditing tools roundup makes a useful point about this group: a crawler that only reads HTML can't confirm what actually fires, which is why real-browser tools exist.

Schema-driven governance: Avo and Amplitude Data

These tools stop bad events at the source. Developers don't write track("Purchase", {...}) by hand. They call a generated, typed function, and the build fails if the event is missing or wrong.

Avo

Avo generates type-safe tracking wrappers with compile-time and runtime validation. avo status fails in CI when a planned event has no call in the code. Its Inspector watches production event schemas and flags missing properties or mixed types.

• Strengths: errors are caught before release, by the developer who made them. It works across many analytics destinations, not one vendor. The free plan is enough to try it properly.
• Weaknesses: it only works if developers adopt the generated functions and someone keeps the plan current in Avo. I found nothing showing it covers pixels or tags added in GTM, outside the code.
• Not for: marketing-led implementations that live mostly in a tag manager.

Amplitude Data

Amplitude Data governs the taxonomy of an Amplitude project. The Ampli wrapper enforces event names and property values, ampli status runs in CI, and schema settings can flag or reject unplanned events at ingestion.

• Strengths: validation at three points (code, CI, ingestion) with no extra vendor if you already pay for Amplitude.
• Weaknesses: it only covers data Amplitude ingests. GA4, ad pixels and consent are out of scope. Amplitude's own docs warn that an empty error list doesn't mean every violation is fixed.
• Not for: anyone whose main analytics or ad stack is not Amplitude.

Live-traffic observability: Trackingplan

Trackingplan listens to the requests your code already makes to analytics and ad vendors. It discovers events, properties and destinations from real traffic, so there is no tracking plan to write before it starts. It installs with a tag, an SDK or a server-side hook, and covers web, iOS, Android and server-side GTM.

• Strengths:
  • Zero configuration to start. The baseline is inferred, and the inferred plan doubles as living documentation.
  • Upstream of the reports. It sees raw requests before they reach GA4, ad platforms or the warehouse, including consent signals and pixels added outside the code.
  • An agent on top. Trackingplan says its agent answers questions about the implementation in plain language and runs root-cause analysis and explains the fix.
• Weaknesses:
  • It only sees traffic that happens. A rare flow nobody visits stays unchecked until someone does.
  • An inferred baseline learns from what the site already sends. An event that has always been wrong looks normal until you define the expected value.
  • It needs a learning period, two days according to its pricing page.
  • There is no native CI step today. The Regression Testing API is deprecated. The documented path is to point your existing automated tests at a staging environment.
  • There is no free plan, only a trial, and the agent consumes paid credits on top of the session-based fee.
• Not for: low-traffic sites with few releases, or teams that need to prove compliance on flows real users rarely reach.

A published case. Trackingplan's customer story with CSA, an agency in Havas Media Network, was published in May 2026 and shows this approach from the agency side. CSA had planned its own tool since around 2018 to check Google Analytics data against its tracking specification, and never got it built. With live-traffic monitoring, it extended checks beyond Google Analytics to Floodlight tags, Meta and TikTok pixels and server-side integrations without writing tests. The team says it now detects and fixes incidents in under 24 hours. It also uses the daily record of the data layer to show whether a break came from a client's release or another vendor. It is the vendor's own case study, with the customer's quotes; I have not verified its figures independently.

Warehouse data quality: Monte Carlo, Great Expectations and dbt

These tools check what landed in the warehouse. They are the right layer when the risk is a broken pipeline or model. They are the wrong one for a tag that never fired, which leaves no row, or a pixel that ignored consent.

Monte Carlo

Monte Carlo learns normal freshness, volume and schema patterns from warehouse metadata, with no configuration for table monitors. It traces field lineage by parsing SQL. Its troubleshooting agent follows that lineage and checks recent job failures and pull requests. Monitors can be deployed as code from CI.

• Weaknesses: a missing event shows up, at best, as a volume anomaly with no browser-side cause. Pricing is not public.

Great Expectations (GX Core)

GX Core is an open-source Python library: you write expectations, and a checkpoint runs them when your orchestrator calls it.

• Weaknesses: everything is written by hand, and it detects nothing between pipeline runs.

dbt tests

dbt data tests are SQL assertions on models and sources: unique, not null, accepted values, relationships. On the dbt platform, CI builds and tests only the models a pull request changes.

• Weaknesses: as with GX, every test is a rule someone wrote and maintains, which is the test plan again, in YAML.

All three are the right answer when your marketing reporting runs on a modeled warehouse. They are a complement to a collection-layer tool, not a substitute. The marketing data observability guide explains that split and how to route the alerts.

How to choose: start from where your breaks happen

Look at your last five tracking incidents before you look at any vendor. Where did each break start, and who found it? The answer points to the approach.

• Pre-launch audits and compliance evidence → ObservePoint or DataTrue. You need repeatable, documented runs on defined paths: a consent audit for legal, a release gate before a redesign.
• Developer-enforced schema governance → Avo, or Amplitude Data if Amplitude is your analytics. Your events live in product code, developers will adopt generated functions, and you want breaks stopped before merge.
• Real-time production monitoring with automated diagnosis → Trackingplan. You ship often, many teams touch tags, and you can't write a test for every change across web, apps, GTM and server-side.
• Warehouse-level data quality → Monte Carlo, Great Expectations or dbt tests. Your reporting depends on models, and the incidents are late tables and broken joins, not missing tags.

Three trade-offs decide between them:

• Maintenance moves, it doesn't vanish. Journeys break when the site changes. Schemas need someone to own the plan. Inferred baselines need someone to say which deviations matter. Pick the maintenance your team will actually do.
• Coverage versus certainty. Synthetic tests are certain about few paths. Live monitoring covers every path users take, with no certainty about paths they don't.
• Who owns it. Scheduled testing suits an analytics or QA team, schemas need engineering, and warehouse tools belong to data engineering. A tool nobody owns produces alerts nobody reads.

Most mature teams combine two: one tool before release and one in production, or one on collection and one on the warehouse. If you don't have a written specification yet, start there. A solution design document makes every approach above more useful.

Questions to ask in a trial

Ask every vendor the same questions, and ask for the answer on your site:

• What do I have to write or maintain when we add an event, a page or a destination?
• How long between a break in production and the alert, and what sets that delay?
• Which of our platforms are covered: web, iOS, Android, server-side GTM, ad pixels?
• How do you test consent: granted, denied and no choice, by region?
• What runs in CI today, natively, and what needs your API?
• What does the alert say about the cause: which tag, which release, which parameter?
• What drives the bill: pages scanned, sessions, events, seats or monitors?

A vendor that can only answer in a demo account is answering a different question.

Three scenarios

These are illustrative situations, not client cases.

An e-commerce team launching a new checkout before Black Friday. The risk is concentrated: one flow, one deadline, and revenue tracking that has to be right on day one. Scripted journeys are the best fit. DataTrue or ObservePoint can test the checkout against the staging container before publishing, check the data layer at each step and re-run after every fix. After launch, the question changes to whether real users' purchases keep arriving complete, which is a monitoring question.

A SaaS analytics team shipping 15 releases a month. Nobody can update a test plan that fast, and the spreadsheet tracking plan is already out of date. Trackingplan fits when the team wants discovery from real traffic and a plan that documents itself. If engineering is willing to adopt typed functions, Avo is the alternative that stops errors before merge rather than after release. The two can work together: schema in code, monitoring for what the code doesn't own.

An agency managing 30+ clients with inconsistent setups. There are two valid models. ObservePoint gives a standardized audit process and reports a client can read, at the cost of building journeys per client. Trackingplan scales with no per-client configuration, at the cost of only seeing what each client's traffic exercises. The choice depends on what the agency sells: audits as a deliverable, or ongoing monitoring as a retainer.

Questions buyers ask

Can live monitoring replace pre-release QA completely? Not on its own. Monitoring detects a break after it reaches the environment it watches. Running it on staging with your existing automated tests narrows the gap, but a launch-critical flow still deserves a scripted test.

Do I still need a tracking plan if a tool infers one? You need a definition of correct for the events that drive money decisions. An inferred plan is a good starting draft and good documentation. It can't tell you that a value has always been wrong.

Where does Google Tag Manager's own preview mode fit? It checks one session, yours, on the pages you open. It is a development tool, not an audit. See the GTM audit essentials for what to review in the container itself.

What to check this week

• List your last five tracking incidents, and for each one where it started and who found it first.
• For your three most important conversion events, write down what "correct" means: required parameters, types and destinations.
• Ask each shortlisted vendor to show one of those incidents being detected in a trial, on your site, not in a demo account.