A/B testing with statistical discipline
An experimentation programme with statistical discipline. Hypotheses, MDE, stopping rules and honest reporting even when a test hasn't won.
The problem we solve.
Badly run A/B testing doesn't just fail to help: it does damage. Making decisions based on "winners" without statistical significance, without a calculated sample size or without measuring the effect on secondary metrics is worse than not testing at all.
At Hi Labs we treat every test as a scientific experiment: a clear hypothesis, a defined MDE, agreed stopping rules and honest reporting. When a test is inconclusive, we say so. When it is conclusive, we stand by it.
What is included
- Hypothesis design and prioritisation (ICE / PIE)
- Implementation with VWO, AB Tasty, Optimizely or GrowthBook
- Sample size and duration calculation per test
- Cross-browser, cross-device and cross-funnel QA
- Reporting with confidence intervals, not misleading percentages
- Post-rollout impact tracking
Our process for A/B Testing.
Hypothesis
Every test starts from a clear hypothesis supported by research or prior data.
Design
We calculate the sample, define the primary metric and stopping rules before launching.
Launch
Thorough QA, gradual rollout and real-time incident monitoring.
Decision
Rigorous statistical analysis, a clear recommendation and a roll-out plan.
What sets a sustained CRO programme apart
It isn't the tool. It's the discipline: one hypothesis per test, one primary metric, documentation of every experiment (including the ones that failed) and a repository of learnings that grows over time.
For mid-traffic clients we recommend a cadence of 2-4 tests per month. For high-traffic ecommerce, 8-12. The key is never to test more than you can analyse with rigour.
Other services within Growth & Optimisation
CRO Agency
End-to-end conversion rate optimisation (CRO): audit, experimentation and analytics across your digital product.
View service →SEO & AI Search / GEO
Ranking on Google and visibility in AI search (ChatGPT, Perplexity, AI Overviews) through technical SEO, content and GEO.
View service →CRO Audit
Quantitative and qualitative diagnosis of the funnel to detect the friction that costs the most to fix and the one that pays back the most.
View service →Headless Ecommerce
Migrations to Shopify Hydrogen, Commercetools or Magento headless with top-tier Core Web Vitals and real, measured conversion.
View service →Web analytics
GA4, server-side tagging, Looker Studio dashboards and attribution models that give you real visibility over the business.
View service →