UX experiments / Craft & speed
Craft & speed · A/B test · replicated on 32M users
Can a slightly darker blue change revenue? Bing's color test
Ad titles in the current blue, or a slightly darker blue with slightly lighter captions?
Make your call. Which version won, and why? Decide before you open the result. Guessing first is what trains your judgment.
Made your call? See the result
More than $10M a year in revenue.
What happened
Everyone was skeptical, so Bing replicated the test on 32 million users. The results held.
Why it worked
Contrast and hierarchy change what people read and click, even when nobody can say why.
The takeaway
Visual craft is measurable. Treat color and contrast as product decisions.
Source
- Kohavi & Thomke · HBR 2017, summary (maaw.info)
More experiments to call
Practice the call
Call experiments like these every day.
Irreplaceable gives you a few real experiments a day to call before you see the result, and tracks how your judgment improves. Start with a free baseline: 12 decisions, about 12 minutes.