Evidence library · Real experiments
A/B test examples: guess the winner
24 published A/B tests and UX experiments, from checkout and pricing to AI assistants. Each one shows both versions. Make your call before you open the result: that's how the judgment builds.
By Sarit Elisha, founder of Irreplaceable · Updated October 10, 2026 · Every result links to its source
Friction
- Large e-commerce retailer · c. 2009Forced registration or guest checkout: which sold more?Register to buy vs Continue as guest
- Etsy · 2012Pages or infinite scroll? Etsy's search results experimentPaginated results vs Infinite scroll
- EA · Maxis · SimCity · 2013Should SimCity keep the $20-off banner on its pre-order page?With promo banner vs No promo
- StubHub · Blake, Moshary, Sweeney & Tadelis · 2021Show ticket fees upfront or at checkout? StubHub's testFees included upfront vs Fees added at checkout
Persuasion
- Obama 2008 campaign · 2007Obama 2008 sign-up page: video or photo, Sign Up or Learn More?Video + “Sign Up” vs Family photo + “Learn More”
- GOV.UK × NHS × Behavioural Insights Team · 2013Which organ donor prompt registered more donors?Social proof + photo vs Reciprocity question
- HMRC × Behavioural Insights Team · 2011Which tax reminder letter got more people to pay?Standard letter vs Local social norm
- Amazon · c. 1998Recommendations in the shopping cart: help or distraction?Clean cart vs Cart + recommendations
- 37signals · Highrise · 2010Long feature page or big customer photo? Highrise's signup testLong-form page vs Person page
- Hotel field experiment · Goldstein, Cialdini & Griskevicius · 2008Which hotel card got more guests to reuse their towels?Help the environment vs Join your fellow guests
- Petrified Forest National Park · Cialdini et al. · 2006Which sign stopped visitors taking petrified wood?Don't remove the wood vs Many visitors have taken wood
- Opower home energy reports · Allcott · 2011Energy bills: does comparing you with your neighbors change usage?Your usage only vs Compared with neighbors
- UK Courts Service × Behavioural Insights Team · 2012Which text reminder got more people to pay court fines?Standard reminder vs Reminder with their name
Craft & speed
- Microsoft Bing · 2012Short or long ad titles: Bing's search ads experimentShort ad title vs Longer ad title
- Microsoft Bing · c. 2013Can a slightly darker blue change revenue? Bing's color testCurrent colors vs Darker blue, lighter caption
- Microsoft Bing · 2012Does 100 milliseconds of page speed change revenue?No, 0.1s is unnoticeable vs Yes, measurably
Humans + AI
- HBS × BCG · 758 consultants · 2023Consultants with GPT-4 or without: who did better work?Without AI vs With GPT-4
- HBS × BCG · same study · 2023When AI's answer sounds right but isn't: who got it right?Without AI vs With GPT-4
- NBER · 5,179 support agents · 2023AI assistants for support agents: who gained the most?The most experienced agents vs Novice, less skilled agents
- Univ. of Washington × Microsoft Research · CHI 2021Do AI explanations help people decide better?People caught more AI mistakes vs People accepted the AI more, right or wrong
- Harvard SEAS · CSCW 2021Decide first, then see the AI: does it reduce blind trust?Less over-reliance, and people liked it vs Less over-reliance, but people liked it least
Choice design
- Columbia × Fordham · Science 2003Opt-in or opt-out: which framing produced more organ donors?Opt-in vs Opt-out
- US employer 401(k) plan · Madrian & Shea · 2001Opt in or enrolled by default: what happened to retirement savings?Opt in vs Enrolled, can opt out
- Grocery store tasting booth · Iyengar & Lepper · 200024 flavors or 6: which jam display sold more?24 flavors vs 6 flavors
Practice the call
Call experiments like these every day.
Irreplaceable gives you a few real experiments a day to call before you see the result, and tracks how your judgment improves. Start with a free baseline: 12 decisions, about 12 minutes.
Take the free baseline No card needed.