A new TAU–Ichilov study shows that tracking eye movements can assess memory more accurately than verbal reports, with potential use for infants, Alzheimer’s patients, and brain injury victims.

Researchers from Tel Aviv University and Tel Aviv Sourasky Medical Center (“Ichilov”) have measured subjects’ memory without asking whether they remembered something or not – -simply by tracking their eye movements as they watched animation videos. The study demonstrated that people actually remember more than they report. Moreover, this method can be used to measure memory in subjects who cannot speak— including infants, patients with brain injuries, and even animals.

The groundbreaking study was led by Dr. Flavio Jean Schmidig, Daniel Yamin, Dr. Omer Sharon, and Prof. Yuval Nir from the Sagol School of Neuroscience, the Gray Faculty of Medical and Health Sciences, and the Fleischman Faculty of Engineering at Tel Aviv University, as well as the Sagol Brain Institute at the Tel Aviv Sourasky Medical Center (“Ichilov”). The paper was published in Communications Psychology.

Beyond Traditional Memory Tests

“Memory is usually tested through direct questioning, with subjects verbally reporting whether they remember a certain event,” explains Dr. Flavio Schmidig, currently completing his postdoctoral research in Prof. Yuval Nir’s lab at TAU. “For example, a subject might be shown a picture and asked if they remember having seen it before. However, this type of testing cannot be performed on animals, infants, patients with advanced Alzheimer’s, or people with head injuries who cannot speak. In this study we wanted to test memory in a more natural way, without asking people to remember.”

“Gaze Memory” Illustration by Ana Yael

Inside the Experiment

In the study, 145 healthy subjects watched specially created animation videos that included a surprising event –  for example, a mouse suddenly jumping out of the corner of the frame. Tracking the subjects’ eye movements  across two separate viewings of the same films, the researchers found that during the second viewing, subjects shifted their gaze toward the area where the surprising event was about to occur. A comparison of eye movement data with verbal memory reports indicated that gaze direction was in fact a more accurate measure.  In some cases, subjects said they did not remember the mouse, yet their gaze indicated that they did.

“The study proves that tracking eye movements can be an excellent alternative to verbal questions such as ‘Do you remember this?’,” says Daniel Yamin. “In a series of experiments, we demonstrated that gaze direction is a very sensitive gage of memory. Even when subjects said they didn’t remember, their gaze direction showed they did. This means that sometimes people remember, but can’t say that they remember. By using AI machine learning techniques, it is possible to infer automatically, from just a few seconds of eye tracking, whether someone has seen a video before and formed a memory of it.”

“When I ask you if you remember,” adds Dr. Sharon, “you might give any of several answers: yes, no, not sure, etc. But when you look to the left of the frame due to a vague memory that something is about to happen there, finer nuances can be discerned. Now we have a tool for testing to what extent memory is present. Our new method is also more natural than traditional memory tests.”

Looking Ahead

“The results of this study are especially relevant when verbal reports on memory cannot be obtained,” adds Prof. Yuval Nir, the study’s supervisor. “We believe that in the future this new method may be used for measuring memory functions in infants, Alzheimer’s patients, and people with brain injury whose speech ability has been impaired. Gaze direction can be simply detected by the camera of a laptop or smartphone as the subject views a video – with no need for large, sophisticated equipment. The method has the potential for identifying memories even in situations that have so far been out of reach for us as scientists and clinicians.”