Eye-Tracking Zoomable Maps

This post was joint authored by Paolo Battino and Rich Treves.

One of the evaluation techniques we said we would employ in our Google Research Project (links to search query) is eye-tracking.  Eye-tracking software is usually designed to record the position of your gaze on the screen, assuming the content of the screen only changes in a predictable manner. This means that current eye-tracking software is good for understanding actions on a web page e.g. did the user spend more time looking at side menu, header or main content?  This is because the screen is divided into static areas and the time spent looking at each area can be easily calculated.

Problem with Eye-Tracking: Unfortunately, this does not work with a map (or a virtual globe) and you want to keep track of the geographic location observed by the user.  In this situation, XY coordinates on screen recorded by the eye-tracker do not directly map onto Lat Long coordinates because the user can zoom, pan and tilt the map ‘camera’. 

Solution:  We have developed a solution entirely based on software developed for this project.  See example below:  

    

Heat maps showing density of eye fixations on a Google Earth map.  

Reading down, the screen shots represent zooming in.  

Red = High density, Blue = Low

Subjects were tested in a mock up of an educational situation.  They were shown (in a Google Earth tour) how to identify a special type of valley and then asked to find one in a given area.  The heat-maps show where on the surface of Google Earth the user was looking at during the experiment independent of zoom level/tilt/pan position. 

Heat Map Script: The heat-map script, developed by Patrick Wied, is particularly efficient in showing “the big picture” (top) but also shows dynamic rendering when the user zooms in.  The screen shots themselves are from a Google Map mashup with all the usual zoom and pan controls.

HowTo:  The solution we describe here only works with Google Earth as it requires the Google Earth API.  

Summary of the Process:

1)   During the experiment, the eye-tracker records each fixation in terms of X,Y tuple together with a very accurate timestamp (this is important).

2)   During the experiment, a custom script records the position of the Google Earth ‘camera’ which is producing the view  on screen.  It polls the Google Earth API every 200 milliseconds or so and every entry is timestamped.

3)   After the experiment, on a webpage using the Google Earth API we reproduce exactly the same view displayed during the exercise by feeding Google Earth the logs from  [2].

4)   Using the timestamp of each log entry, we look up the eye-tracking logs to find out if there was a fixation recorded at exactly that time.

5)   We then use the X,Y screen coordinates to poll Google Earth and transform those coordinates into latlongs. In effect we ‘cast’ a ray from a specific location on screen onto the virtual globe.

6)   Using the API we record the lat long from the end of the cast ray and put it into a database (see diagram below)

7)   This data is processed to render the heat-map.

There are obvious far more technical details that this but for the moment we thought we'd just get the idea out.

Problems with Eye-Tracking Maps:  There are a couple of inherent issues to do with eye-tracking virtual globe maps that have already occurred to us:

• High Altitude Zooms:  At both high and low altitude the fixation is captured as a point, at a high zoom the user may be looking at a larger feature.  A circle polygon would better represent the fixation at altitude.
• Tilt inaccuracy: In a situation where the user is highly tilted, the inherent inaccuracies of the eye-tracking kit get amplified - a small change in eye position can represent a large variation in distance on the ground. 

In the particular case study we've discussed today we don't think either of these are a particular issue but they need to borne in mind in other situations.