UX experiments / Humans + AI
Humans + AI · Staggered rollout study
AI assistants for support agents: who gained the most?
Customer support agents got an AI assistant. Who gained the most?
Make your call. Which version won, and why? Decide before you open the result. Guessing first is what trains your judgment.
Made your call? See the result
Average +14% issues resolved per hour; +34% for novices; little effect on the most experienced.
What happened
The assistant spread the practices of top agents to newer ones. Customer sentiment and retention also improved.
Why it worked
AI packages what the best people do and hands it to everyone.
The takeaway
If your skill can be learned from your own output, AI will spread it. Build the part that can't.
Source
More experiments to call
Practice the call
Call experiments like these every day.
Irreplaceable gives you a few real experiments a day to call before you see the result, and tracks how your judgment improves. Start with a free baseline: 12 decisions, about 12 minutes.