Attention, attribute ordering, and what people actually read

Bringing eye-tracking evidence into preference measurement: what moves when a choice task is reordered, and what that means for the estimates.

Choice modelling assumes the respondent read the alternatives. Every coefficient in a discrete choice experiment is an answer to the question how much does this attribute matter, and that answer only means what we say it means if the person looked at the attribute.

They frequently did not. Eye tracking shows attention across a choice task is uneven and patterned: the top of the table is read, the bottom is sampled, and some attributes are skipped almost entirely by some respondents and treated as decisive by others. The pattern is stable enough to model and consequential enough to change estimates.

Order is not neutral

The clearest demonstration is reordering. Take the same attributes, the same levels, the same design, and move one attribute from the second row to the seventh. The estimated coefficient changes. Nothing about the underlying preference has changed, so either the elicitation or the interpretation is wrong.

Schematic of fixation patterns under two attribute orders Two identical choice tables with the attribute rows in different orders. In each, circles mark fixations and their size marks dwell time. Attention concentrates in the upper rows in both cases, so the attribute placed near the top receives more attention than the same attribute placed near the bottom. Order 1 Option AOption B Out-of-pocket cost Waiting time Continuity of care Consultation length Travel required Order 2 Option AOption B Travel required Consultation length Continuity of care Waiting time Out-of-pocket cost
Schematic, not data. Circles are fixations and their size is dwell time. The attention pattern is held constant across the two panels and only the row order changes, which is the point: cost is read closely on the left and barely reached on the right, though the respondent's preference for cost has not changed.

What the process evidence buys

Read more

A check on the design. If a whole attribute goes unattended by most respondents, its coefficient is not measuring a preference. Knowing that before publication is better than after.

A model of how, not just what. Attention data supports models where the decision rule itself varies: some people compare alternatives attribute by attribute, some evaluate each alternative whole, and some eliminate on one attribute and never look further. These produce different choices from identical preferences.

A reason to standardise reporting. If order effects are real, then attribute order belongs in the methods section alongside the design, and reordering across respondents belongs in the design itself.

Software

The analysis pipeline for the attribute-ordering work, including the sequence analysis of fixation data, is on GitHub.

Back to top