Humans + AI · Experiment · 199 participants

Decide first, then see the AI: does it reduce blind trust?

Harvard SEAS · CSCW 2021

To reduce blind trust in AI, make people decide first, then show the AI's answer. What happened?

Version A: Less over-reliance, and people liked it — ALess over-reliance, and people liked it

Version B: Less over-reliance, but people liked it least — BLess over-reliance, but people liked it least

Make your call. Which version won, and why? Decide before you open the result. Guessing first is what trains your judgment.

Made your call? See the result

The designs that reduced over-reliance the most got the least favorable ratings.

What happened

People preferred and trusted the systems that felt less demanding, even when they performed worse with them.

Why it worked

Thinking is effort, and people rate effort as worse UX.

The takeaway

The design people prefer isn't always the one that protects them. Make the call, and be able to defend it.

Source

Buçinca, Malaya & Gajos · To Trust or to Think (CSCW 2021) (arxiv.org)

More experiments to call

Practice the call

Call experiments like these every day.

Irreplaceable gives you a few real experiments a day to call before you see the result, and tracks how your judgment improves. Start with a free baseline: 12 decisions, about 12 minutes.

Take the free baseline No card needed.