Eye tracking

Eye tracking is the process of measuring either the point of gaze ("where we are looking") or the motion of an eye relative to the head. An eye tracker is a device for measuring eye positions and eye movements. Eye trackers are used in research on the visual system, in psychology, in cognitive linguistics and in product design. There are a number of methods for measuring eye movements. The most popular variant uses video images from which the eye position is extracted. Other methods use search coils or are based on the electrooculogram.

History

In the 1800s, studies of eye movements were made using direct observations.

In 1879 in Paris, Louis Émile Javal observed that reading does not involve a smooth sweeping of the eyes along the text, as previously assumed, but a series of short stops (called fixations) and quick saccades. This observation raised important questions about reading, which were explored during the 1900s: On which words do the eyes stop? For how long? When does it regress back to already seen words?

File:ReadingFixationsSaccades.jpg

An example of fixations and saccades over text. This is the typical pattern of eye movements during reading. The eyes never move smoothly over still text.

Huey built what might be the first eye tracker, using a sort of contact lens with a hole for the pupil. The lens was connected to an aluminum pointer that moved in response to the movements of the eye. Huey studied and quantified regressions (only a small proportion of saccades are regressions), and show that only a portion of the words in a sentence were actually fixated.

The first non-intrusive eye trackers were built by George Buswell in Chicago, using beams of light that were reflected on the eye and then recording them on film. Buswell made systematic studies into reading and picture viewing.

In the 1950s, Alfred L. Yarbus did important eye tracking research and his 1967 book is one of the most quoted eye tracking publications ever. For example he showed the task given to a subject has a very large influence on the subject's eye movements. He also wrote about the relation between fixations and interest:

"All the records (…) show conclusively that the character of the eye movements is either completely independent of or only very slightly dependent on the material of the picture and how it was made, provided that it is flat or nearly flat." The cyclical pattern in the examination of pictures "is dependent not only on what is shown on the picture, but also on the problem facing the observer and the information that he hopes to gain from the picture."

This study by Yarbus (1967) is often referred to as evidence on how the task given to a person influences his or her eye movements.

"Records of eye movements show that the observer's attention is usually held only by certain elements of the picture. (…) Eye movements reflect the human thought processes; so the observer's thought may be followed to some extent from records of eye movements (the thought accompanying the examination of the particular object). It is easy to determine from these records which elements attract the observer's eye (and, consequently, his thought), in what order, and how often."

"The observer's attention is frequently drawn to elements which do not give important information but which, in his opinion, may do so. Often an observer will focus his attention on elements that are unusual in the particular circumstances, unfamiliar, incomprehensible, and so on."

"(…) when changing its points of fixation, the observer's eye repeatedly returns to the same elements of the picture. Additional time spent on perception is not used to examine the secondary elements, but to reexamine the most important elements."

In the 1970s, eye tracking research expanded rapidly, particularly reading research. A good overview of the research in this period is given by Rayner..

In 1980, Just and Carpenter formulated the influential Strong eye-mind Hypothesis, the hypothesis that "there is no appreciable lag between what is fixated and what is processed". If this hypothesis is correct, then when a subject looks at a word or object, he or she also thinks about (process cognitively), and for exactly as long as the recorded fixation. The hypothesis is too often today taken for granted by beginning eye tracker researchers.

During the 1980s, the eye-mind hypothesis was often questioned in light of covert attention, the attention to something that one is not looking at, which people often do. If covert attention is common during eye tracking recordings, the resulting scan path and fixation patterns would often show not where our attention has been, but only where the eye has been looking, and so eye tracking would not indicate cognitive processing.

According to Hoffman, current consensus is that visual attention is always slightly (100 to 250 ms) ahead of the eye. But as soon as attention moves to a new position, the eyes will want to follow.

We still cannot infer specific cognitive processes directly from a fixation on a particular object in a scene. For instance, a fixation on a face in a picture may indicate recognition, liking, dislike, puzzlement etc. Therefore eye tracking is often coupled with other methodologies, such as introspective verbal protocols.

Tracker types

Eye trackers measure rotations of the eye in one of several ways, but principally they fall into three categories.

One type uses an attachment to the eye, such as a special contact lens with an embedded mirror or magnetic field sensor, and the movement of the attachment is measured with the assumption that it does not slip significantly as the eye rotates. Measurements with tight fitting contact lenses have provided extremely sensitive recordings of eye movement, and magnetic search coils are the method of choice for researchers studying the dynamics and underlying physiology of eye movements.

The second broad category uses some non-contact, optical method for measuring eye motion. Light, typically infrared, is reflected from the eye and sensed by a video camera or some other specially designed optical sensor. The information is then analyzed to extract eye rotation from changes in reflections. Video based eye trackers typically use the corneal reflection (the first Purkinje image) and the center of the pupil as features to track over time. A more sensitive type of eye tracker, the dual-Purkinje eye tracker, uses reflections from the front of the cornea (first Purkinje image) and the back of the lens (fourth Purkinje image) as features to track. A still more sensitive method of tracking is to image features from inside the eye, such as the retinal blood vessels, and follow these features as the eye rotates. Optical methods, particularly those based on video recording, are widely used for gaze tracking and are favored for being non-invasive and inexpensive.

The third category uses electrical potentials measured with contact electrodes placed near the eyes. The most common variant of this is the electro-oculogram (EOG) and is based on the fact that the eye has a standing electrical potential, with the cornea being positive relative to the retina. This potential is not constant, however, and its variation causes the EOG to be somewhat unreliable for measuring slow eye movements and fixed gaze positions. The EOG is most useful for measuring the rapid, saccadic eye movements associated with gaze shifts and is the method of choice for measuring REM during sleep.

Technologies and techniques

The most widely used current designs are video-based eye trackers. A camera focuses on one or both eyes and records their movement as the viewer looks at some kind of stimulus. Most modern eye-trackers use contrast to locate the center of the pupil and use infrared and near-infrared non-collimated light to create a corneal reflection (CR). The vector between these two features can be used to compute gaze intersection with a surface after a simple calibration for an individual.

Two general types of eye tracking techniques are used: Bright Pupil and Dark Pupil. Their difference is based on the location of the illumination source with respect to the optics. If the illumination is coaxial with the optical path, then the eye acts as a retroreflector as the light reflects off the retina creating a bright pupil effect similar to red eye. If the illumination source is offset from the optical path, then the pupil appears dark.

Bright Pupil tracking creates greater iris/pupil contrast allowing for more robust eye tracking with all iris pigmentation and greatly reduces interference caused by eyelashes and other obscuring features. It also allows for tracking in lighting conditions ranging from total darkness to very bright. But bright pupil techniques are not effective for tracking outdoors as extraneous IR sources interfere with monitoring.

Eye tracking setups vary greatly; some are head-mounted, some require the head to be stable (for example, with a chin rest), and some function remotely and automatically track the head during motion. Most use a sampling rate of at least 30 Hz. Although 50/60 Hz is most common, today many video-based eye trackers run at 240, 350 or even 1000/1250 Hz, which is needed in order to capture the detail of the very rapid eye movements during reading, or during studies of neurology.

Eye movements are typically divided into fixations and saccades, when the eye gaze pauses in a certain position, and when it moves to another position, respectively. The resulting series of fixations and saccades is called a scanpath. Most information from the eye is made available during a fixation, but not during a saccade. The central one or two degrees of the visual angle (the fovea) provide the bulk of visual information; the input from larger eccentricities (the periphery) is less informative. Hence, the locations of fixations along a scanpath show what information loci on the stimulus were processed during an eye tracking session. On average, fixations last for around 200 ms during the reading of linguistic text, and 350 ms during the viewing of a scene. Preparing a saccade towards a new goal takes around 200 milliseconds.

Scanpaths are useful for analyzing cognitive intent, interest, and salience. Other biological factors (some as simple as gender) may affect the scanpath as well. Eye tracking in HCI typically investigates the scanpath for usability purposes, or as a method of input in gaze-contingent displays, also known as gaze-based interfaces.

Eye tracking vs. gaze tracking

Eye trackers necessarily measure the rotation of the eye with respect to the measuring system. If the measuring system is head mounted, as with EOG, then eye-in-head angles are measured. If the measuring system is table mounted, as with scleral search coils or table mounted camera (“remote”) systems, then gaze angles are measured.

In many applications, the head position is fixed using a bite bar, a forehead support or something similar, so that eye position and gaze are the same. In other cases, the head is free to move, and head movements are measured with systems such as magnetic or video based head trackers.

For head-mounted trackers, head position and direction are added to eye-in-head direction to determine gaze direction. For table-mounted systems, such as search coils, head direction is subtracted from gaze direction to determine eye-in-head position.

Eye tracking in practice

A great deal of research has gone into studies of the mechanisms and dynamics of eye rotation, but the goal of eye tracking is most often to estimate gaze direction. Users may be interested in what features of an image draw the eye, for example. It is important to realize that the eye tracker does not provide absolute gaze direction, but rather can only measure changes in gaze direction. In order to know precisely what a subject is looking at, some calibration procedure is required in which the subject looks at a point or series of points, while the eye tracker records the value that corresponds to each gaze position. (Even those techniques that track features of the retina cannot provide exact gaze direction because there is no specific anatomical feature that marks the exact point where the visual axis meets the retina, if indeed there is such a single, stable point.) An accurate and reliable calibration is essential for obtaining valid and repeatable eye movement data, and this can be a significant challenge for non-verbal subjects or those who have unstable gaze.

Each method of eye tracking has advantages and disadvantages, and the choice of an eye tracking system depends on considerations of cost and application. There is a trade-off between cost and sensitivity, with the most sensitive systems costing many tens of thousands of dollars and requiring considerable expertise to operate properly. Advances in computer and video technology have led to the development of relatively low cost systems that are useful for many applications and fairly easy to use. Interpretation of the results still requires some level of expertise, however, because a misaligned or poorly calibrated system can produce wildly erroneous data.

Choosing an eye tracker

One difficulty in evaluating an eye tracking system is that the eye is never still, and it can be difficult to distinguish the tiny, but rapid and somewhat chaotic movements associated with fixation from noise sources in the eye tracking mechanism itself. One useful evaluation technique is to record from the two eyes simultaneously and compare the vertical rotation records. The two eyes of a normal subject are very tightly coordinated and vertical gaze directions typically agree to within +/- 2 minutes of arc (RMS of vertical position difference) during steady fixation. A properly functioning and sensitive eye tracking system will show this level of agreement between the two eyes, and any differences much larger than this can usually be attributed to measurement error.

Applications

A wide variety of disciplines use eye tracking techniques, including cognitive science, psychology (notably psycholinguistics, the visual world paradigm), human-computer interaction (HCI), marketing research and medical research (neurological diagnosis). Specific applications include the tracking eye movement in language reading, music reading, the perception of advertising, and the playing of sport. Uses include:

• Cognitive Studies
• Medical Research
• Human Factors
• Computer Usability
• Translation Process Research
• Vehicle Simulators
• In-vehicle Research
• Training Simulators
• Virtual Reality
• Adult Research
• Infant Research
• Adolescent Research
• Geriatric Research
• Primate Research
• Sports Training
• fMRI / MEG / EEG
• Commercial eye tracking (web usability, advertising, marketing, automotive, etc)
• finding good clues
• Communication systems for disabled
• Improved image and video communications

Commercial applications

In recent years, the increased sophistication and accessibility of eye tracking technologies have generated a great deal of interest in the commercial sector. Applications include web usability, advertising, sponsorship, package design and automotive engineering. In general, commercial eye tracking studies function by presenting a target stimulus to a sample of consumers while an eye tracker is used to record the activity of the eye. Examples of target stimuli may include websites, television programs, sporting events, films, commercials, magazines, newspapers, packages, shelf Displays, consumer systems (ATMs, checkout systems, kiosks), and software. The resulting data can be statistically analyzed and graphically rendered to provide evidence of specific visual patterns. By examining fixations, saccades, pupil dilation, blinks and a variety of other behaviors researchers can determine a great deal about the effectiveness of a given medium or product. While some companies complete this type of research internally, there are many private companies that offer eye tracking services and analysis.

The most prominent field of commercial eye tracking research is web usability. While traditional usability techniques are often quite powerful in providing information on clicking and scrolling patterns, eye tracking offers the ability to analyze user interaction between the clicks. This provides valuable insight into which features are the most eye-catching, which features cause confusion and which ones are ignored altogether. Specifically, eye tracking can be used to assess search efficiency, branding, online advertisements, navigation usability, overall design and many other site components. Analyses may target a prototype or competitor site in addition to the main client site.

Eye tracking is commonly used in a variety of different advertising media. Commercials, print ads, online ads and sponsored programs are all conducive to analysis with current eye tracking technology. Analyses focus on visibility of a target product or logo in the context of a magazine, newspaper, website, or televised event. This allows researchers to assess in great detail how often a sample of consumers fixates on the target logo, product or ad. In this way, an advertiser can quantify the success of a given campaign in terms of actual visual attention.

Eye tracking provides package designers with the opportunity to examine the visual behavior of a consumer while interacting with a target package. This may be used to analyze distinctiveness, attractiveness and the tendency of the package to be chosen for purchase. Eye tracking is often utilized while the target product is in the prototype stage. Prototypes are tested against each other and competitors to examine which specific elements are associated with high visibility and appeal.

One of the most promising applications of eye tracking research is in the field of automotive design. Research is currently underway to integrate eye tracking cameras into automobiles. The goal of this endeavor is to provide the vehicle with the capacity to assess in real-time the visual behavior of the driver. The National Highway Traffic Safety Administration (NHTSA) estimates that drowsiness is the primary causal factor in 100,000 police-reported accidents per year. Another NHTSA study suggests that 80% of collisions occur within three seconds of a distraction. By equipping automobiles with the ability to monitor drowsiness, inattention, and cognitive engagement driving safety could be dramatically enhanced. Lexus claims to have equipped its LS 460 with the first driver monitor system in 2006, providing a warning if the driver takes his or her eye off the road.

Since 2005 Eye tracking is used in Communication systems for disabled allowing the user to speak, mail, surf the web and so with only the eyes as tool. Eye control works even when the user has involuntary movement as a result of CP or other disability, those who wear glasses or many other characteristics that limit the effectiveness of older eye control systems.

Research: Journals, conferences, publications

Because of the wide variety of application areas, there are few common research journals or conferences for eye-tracking research. Results from research on eye movements often end up in very different channels. There are a number of recurring research conferences, however.

ECEM - the European Conference on Eye Movements, biannual

• ECEM 2007, ECEM 2005, ECEM 2003, ECEM 2001

SWAET - the Scandinavian Workshop on Applied Eye-tracking, annual

• SWAET 2007, SWAET 2006

Vision in Vehicles, biannual

• Vision in vehicles home page

ETRA - Eyetracking Research and Applications, biannual

• ETRA 2006, ETRA 2004 ETRA 2002 ETRA 2000

COGAIN - Communication by Gaze Interaction, annual

• COGAIN

References

• Adler FH & Fliegelman (1934). Influence of fixation on the visual acuity. Arch. Ophthalmology 12, 475.
• Buswell, G.T. (1922). Fundamental reading habits: A study of their development. Chicago, IL: University of Chicago Press.
• Buswell G.T. (1935). How People Look at Pictures. Chicago: Univ. Chicago Press 137–55. Hillsdale, NJ: Erlbaum
• Buswell, G.T. (1937). How adults read. Chicago, IL: University of Chicago Press.
• Carpenter, Roger H.S.; Movements of the Eyes (2nd ed.). Pion Ltd, London, 1988. ISBN 0-85086-109-8.
• Cornsweet TN, Crane HD. (1973) Accurate two-dimensional eye tracker using first and fourth Purkinje images. J Opt Soc Am. 63, 921-8.
• Cornsweet TN. (1958). New technique for the measurement of small eye movements. JOSA 48, 808-811.
• Deubel, H. & Schneider, W.X. (1996) Saccade target selection and object recognition: Evidence for a common attentional mechanism. Vision Research, 36, 1827-1837.
• Duchowski, A. T., "A Breadth-First Survey of Eye Tracking Applications", Behavior Research Methods, Instruments, & Computers (BRMIC), 34(4), November 2002, pp.455-470.
• Eizenman M, Hallett PE, Frecker RC. (1985). Power spectra for ocular drift and tremor. Vision Res. 25, 1635-40
• Ferguson RD (1998). Servo tracking system utilizing phase-sensitive detection of reflectance variations. US Patent # 5,767,941
• Hammer DX, Ferguson RD, Magill JC, White MA, Elsner AE, Webb RH. (2003) Compact scanning laser ophthalmoscope with high-speed retinal tracker. Appl Opt. 42, 4621-32.
• Hoffman, J. E. (1998). Visual attention and eye movements. In H. Pashler (ed.), Attention (pp. 119-154). Hove, UK: Psychology Press.
• Holsanova, J. (forthcoming) Picture viewing and picture descriptions, Benjamins.
• Huey, E.B. (1968). The psychology and pedagogy of reading. Cambridge, MA: MIT Press. (Originally published 1908)
• Jacob, R. J. K. & Karn, K. S. (2003). Eye Tracking in Human-Computer Interaction and Usability Research: Ready to Deliver the Promises. In R. Radach, J. Hyona, & H. Deubel (eds.), The mind's eye: cognitive and applied aspects of eye movement research (pp.573-605). Boston: North-Holland/Elsevier.
• Just MA, Carpenter PA (1980) A theory of reading: from eye fixation to comprehension. Psychol Rev 87:329–354
• Liechty,J, Pieters, R, & Wedel, M. (2003). The Representation of Local and Global Exploration Modes in Eye Movements through Bayesian Hidden Markov Models. Psychometrika, 68 (4), 519-542.
• Mulligan, JB, (1997). Recovery of Motion Parameters from Distortions in Scanned Images. Proceedings of the NASA Image Registration Workshop (IRW97), NASA Goddard Space Flight Center, MD
• Ott D & Daunicht WJ (1992). Eye movement measurement with the scanning laser ophthalmoscope. Clin. Vision Sci. 7, 551-556.
• Posner, M. I. (1980) Orienting of attention. Quarterly Journal of Experimental Psychology 32: 3-25.
• Rayner, K. (1978). Eye movements in reading and information processing. Psychological Bulletin, 85, 618-660
• Rayner, K. (1998) Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124, 372-422.
• Riggs LA, Armington JC & Ratliff F. (1954) Motions of the retinal image during fixation. JOSA 44, 315-321.
• Riggs, L. A. & Niehl, E. W. (1960). Eye movements recorded during convergence and divergence. J Opt Soc Am 50:913-920.
• Robinson, D. A. A method of measuring eye movement using a scleral search coil in a magnetic field. IEEE Trans. Biomed. Eng., vol. BME-l0, pp. 137-145, 1963
• Wright, R.D., & Ward, L.M. (2008). Orienting of Attention. New York. Oxford University Press.
• Yarbus, A. L. Eye Movements and Vision. Plenum. New York. 1967 (Originally published in Russian 1962)

Commercial eye tracking

• Chandon, Pierre, J. Wesley Hutchinson, and Scott H. Young (2001), Measuring Value of Point-of-Purchase Marketing with Commercial Eye-Tracking Data. [2]
• Duchowski, A. T., (2002) A Breadth-First Survey of Eye Tracking Applications, 'Behavior Research Methods, Instruments, & Computers (BRMIC),' 34(4), November 2002, pp.455-470.
• National Highway Traffic Safety Administration. (n.d.) Retrieved July 9th, 2006, from [3]
• Pieters, R., Wedel, M. & Zhang, J. (2007). Optimal Feature Advertising Under Competitive Clutter, Management Science, 2007, 51 (11) 1815-1828.
• Pieters, R., & Wedel, M. (2007). Goal Control of Visual Attention to Advertising: The Yarbus Implication, Journal of Consumer Research, 2007, 34 (August), 224-233.
• Pieters, R. & Wedel, M. (2004). Attention Capture and Transfer by elements of Advertisements. Journal of Marketing, 68 (2), 2004, 36-50.
• Weatherhead, James. (2005) Eye on the Future, 'British Computer Society, ITNOW Future of Computing,' 47 (6), pp. 32-33 [4]
• Wedel, M. & Pieters, R. (2000). Eye fixations on advertisements and memory for brands: a model and findings. Marketing Science, 19 (4), 2000, 297-312.
• Wittenstein, Jerran. (2006). EyeTracking sees gold in its technology. [Electronic Version]. San Diego Source, The Daily Transcript, April, 3rd, 2006. [5]

See also

• Eye movement
• Eye movement in language reading
• Eye movement in music reading
• Fovea
• Gaze-contingency paradigm
• Peripheral vision
• OpenInterface Platform, OpenInterface Multimodal Interaction Designer Framework