Eye tracking companies, including GazeHawk, frequently use heatmaps to present the results of a study to customers. The reason for this is simple: heatmaps instantly communicate where study participants looked and for how long. They tell you the hot spots of the site and the places that might be getting overlooked.

That said, heat maps have some major drawbacks. In particular, heatmaps usually:

  • Eliminate the element of time from eye tracking. Heatmaps do not communicate when a user looked at something, only that he or she did so at some point. Moreover, heatmaps invite the reader to forget that this is a problem, since there is often no indication that this axis is getting left out.
  • Do not distinguish between a single person looking at a spot for a long time and a group of people looking at a spot for a moment. This ambiguity can lead to all sorts of problems interpreting the heatmap. We find that unless warned about this problem, readers tend to conflate the results of individual participants with that of the group as a whole and assume that most people’s individual heatmap looks pretty much like the aggregate heatmap. This is usually not true.

When you combine these two problems, you can get large differences between what a study’s “first-glance” aggregate heatmap suggests and what the data actually support. Here’s an example: a few months back, we conducted a study tracking the eye movement of people on NBC.com. The heatmap below shows the combined views of all participants in the study:

Suppose that we’re interested in making sure users understand the layout of the site and what content is available to them. At first glance, everything appears to be working as intended. The screen-spanning Miss USA images up top get a lot of attention, as does the “Spotlight on NBC” spot and top-right ad. Users followed the center column of large pictures down, occasionally flicking to the right or left columns as particular bits of content caught their eye.

Almost everything on the page was viewed at some point, and we can see a few clear but unsurprising trends in the distribution of views, such as a preference for large images and human faces. Mission accomplished?

Not at all. Check out these heatmaps from a few individual participants in the study:

While all participants looked down the green central column of the site, there doesn’t appear to be any pattern besides that (this is true of all participants, not just these four). Some looked to the left, some looked to the right – and others didn’t look anywhere at all, quite unlike what the combined heatmap suggests. If we only looked at the combined heatmap, we might think that the site is fine as-is. After reviewing the individual heatmaps, however, we get the impression that the site might have a bit too much going on; after the middle column, people did not know where to look.

While we’re aware of their flaws, at GazeHawk we think that heatmaps are not beyond redemption. Keeping their weaknesses in mind when using them can prevent misinterpretation and ensure that your conclusions are meaningful. And, if all else fails, you can always review individual participant data and watch the eye tracking as a video.

Next week we’ll look at ways to cluster eye tracking data so that the heatmaps don’t have these problems. Until then, don’t get burned.

GazeHawk provides eye tracking services using ordinary webcams. They are the industry leader in economical eye tracking, with the ability to run higher quality studies at lower costs. Learn more about their products or request additional information.
This entry was posted in Uncategorized and tagged , , . Bookmark the permalink.

6 Responses to Don’t get burned by heatmaps

  1. Tyr Grisfal says:

    Why anyone would not run simple clustering on the per-user data is beyond me. Yes, it’s easier to just integrate everything – but you lose so much useful information.

    • Lionel Barrow says:

      That’s more or less what we’re doing next week. The catch is that there are many different ways to cluster these data. Do cluster based on where the users looked alone? Do you take time into account? We’ll be examining at least two ways of clustering, possibly three.

  2. Joakim says:

    What about adding a time slider to the heat map visualization to enable sliding in x second aggregations. Plus maybe normalizing the times for the users. That should give a better grip on the site usage by time, still using the heat map.

    • Brian Krausz says:

      We do actually offer a slider in our reporting system. The problem is that interactive visualizations are harder to show to other. You can export the individual subset heatmaps, but if there were a way to contain more temporal data in a single image it would be very powerful.

  3. Pingback: Why look at 30 heatmaps when 3 will do | GazeHawk Blog

  4. Pingback: Beyond heatmaps: visualizing eye tracking data | GazeHawk Blog

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>