Does feedback help people make better choices?
It is tempting to think that helping people during a choice task must improve it. If we tell respondents something about their earlier answers, or nudge them toward being consistent, surely the data get better. In recent work I looked at this more carefully, and the answer is not that simple.
Feedback can help. It can focus attention, remind people what they said, and reduce careless mistakes. But feedback can also steer. If we tell people they were inconsistent and invite them to fix it, we may be pushing them toward a kind of consistency that suits our model rather than their real preferences. In that case the data look cleaner while telling us less about what people value.
The distinction I find useful is between helping someone answer the question we asked and quietly changing the question. Feedback that reduces confusion, or that clarifies what a task involves, tends to help. Feedback that signals a right way to answer tends to guide people toward it, and we should be honest that we have then measured something partly of our own making.
None of this is an argument against supporting respondents. It is an argument for being deliberate. If we add feedback, we should be clear about what it is meant to do, test whether it changes the results, and report that it was used. Cleaner data are not the goal. Data that reflect real preferences are the goal, and sometimes those are a little messier.
Back to top