Launch: The next generation of Cancel Flow A/B Testing
With Cancel Flow A/B Testing 2.0, you get deep statistical rigor, documented decisions, and extensive LTV tracking.
Cancel Flow A/B testing has been one of our most-used advanced features since its launch two years ago. Teams across SaaS, media, and e-commerce run experiments every month to optimize their retention strategy.
But as our customers' experimentation programs matured, so did their questions. They wanted deeper statistical rigor. They wanted to track what happens after a customer accepts an offer. They wanted confidence intervals, winner declarations, and revenue impact—all in one place.
Welcome to Cancel Flows A/B Testing 2.0.
What's new
Real-time statistical confidence. Every test now calculates and displays statistical significance as data rolls in. Clear badges tell you exactly where you stand: "Not Significant," "Approaching Significance," or "Statistically Significant at 99%." You'll know when your results are ready for action.
Choose your primary success metric. Different experiments optimize for different goals. Now you select what matters most to your test:
- Revenue per Exposure
- Save Rate
- Pause Acceptance Rate
- Discount Acceptance Rate
- Reactivation Rate
- LTV Extension
Your selected metric drives the confidence calculations and determines winner declarations.
Two-phase testing for the full picture. A 7-day enrollment period assigns customers to variants. Then a 30-day tracking window captures what happens next: reactivation rates, lifetime value, and total revenue impact. You get the complete story, not just the first chapter.
Automatic winner declarations. When your test reaches statistical significance, you'll see it immediately. The system identifies the leading variant, shows you why, and lets you make the final call—then push live with one click.
The Exposure Stream. A real-time view of every customer session: which variant they saw, what offer they received, how they responded, and the revenue impact. Exportable and filterable.
Smarter notifications. Awareness for when your tests go live, when winners emerge, and when tracking windows complete.
How it works
- Create your experiment. Name it, document your hypothesis, select your primary metric, and choose which Cancel Flow to test.
- Review the timeline. See exactly when you'll reach 95% confidence based on your flow's traffic volume. Know when enrollment ends and when tracking completes.
- Monitor progress. Watch metrics evolve in real-time. See confidence rise, compare variants side-by-side, explore the retention timeline.
- Make your decision. When tracking completes, review final results across all metrics. Select your winner, document your rationale, and push live.
The winning variant becomes your active Cancel Flow. The decision rationale becomes part of your team's institutional knowledge.
Built for teams who take experimentation seriously
This release is for customers who've pushed us to go deeper—who run multiple tests per quarter, who want to know not just if something worked but why and by how much.
With A/B Testing 2.0, you get deep statistical rigor and clear outcomes weeks after your test concludes.
More to come
This release establishes the foundation for a new generation of experimentation tools at Churnkey. We're bringing this same level of depth across everything—and we'll have more to share soon.
Want to see it in action? Book a walkthrough, check out the docs, or start a free trial.
