Diagnose an activation or checkout drop
A metric fell yesterday. The job is to find where it fell, for whom, and what to check next — not to announce a cause before anyone has checked it.
Who does it
A product or growth lead notices the drop. An analyst or engineer confirms it, and the owner of the affected flow decides what happens next.
What data it needs
- Signup, funnel-step and order events in a supported store, with platform and app version on each event.
- A defined metric — for example, activation = signup → first order within 24 hours.
- Release dates, ideally in the catalog's business context, so the timing can be compared.
Step by step
| Step | Status |
|---|---|
| Check that the data is fresh and that no event changed name or shape | Manual today |
| Compare the metric with a comparable window (same weekday, 7-day baseline) | Live trends and dashboards |
| Split by platform, version and channel to isolate the cohort | Live breakdowns and cohorts |
| Find the funnel step where users stopped | Live funnels |
| Line the window up against releases and operational changes | Manual today |
| Ask "why was activation down?" in plain language and get the steps above done for you | In build Ask |
| Get flagged automatically when the drop starts | Next Watch |
| Rank hypotheses and test them | Later Dig |
What the answer includes
Both the absolute and the relative change. 31.4% → 24.5% is a drop of 6.9 percentage points, or 22% relative, and the two are easy to confuse. The answer also shows the definition and its version, the period and comparison window, the affected population and its size, data freshness, and the chart behind each claim.
The release timing is a hypothesis. The answer names it alongside the alternatives still open — an OS update, an SMS-provider outage, a tracking change — and what would separate them.
The decision it supports
Who investigates, what they check first, and whether to roll back, hold, or run a controlled test. It does not support declaring a cause on timing alone.
Questions this page doesn't answer are the ones we want to hear. Ask on a call, or write to contact@depsmith.com.
Request a demo