Eye movements alone are enough to identify a reader’s interaction with text, according to new research from Haifa’s Technion-Israel Institute of Technology Faculty of Data and Decision Sciences.
Reading is an all-pervasive, practiced skill that is indispensable for successful involvement in modern society. When reading, our eyes move over the text in a saccadic fashion, where there are periods of time in which the gaze is stable at a specific location, called fixations, and rapid transitions between fixations called saccades.
This sequence of fixations and saccades is generally hypothesized to contain rich information about how readers interact with text. Automatic decoding of such information is currently a growing area of research.
People read texts with different goals. Whether it’s a newspaper or magazine article, the Internet, a novel, a cooking recipe, or a scientific paper, each type of text can be approached with various intentions. Two common reading goals are general comprehension (regular reading) and the search for information. The researchers developed computational models that combine eye-tracking with text processing.
In daily life, a reader may have one or several goals that they pursue with respect to the text. For example, they may read the text closely or skim it to obtain the gist of the text’s content, they may proofread it, or they may be seeking specific information of interest. Each goal can have a profound impact on online linguistic processing and on the corresponding eye movement behavior while reading.
These models can accurately detect a reader’s purpose with about 90% accuracy and nearly 80% accuracy within just two seconds from the moment the reading started, the team maintained.
The team’s findings were presented at the 63rd Association for Computational Linguistics conference, one of the world’s most prestigious gatherings in the field of natural language processing, which was recently held in Vienna.
The research was led by doctoral student Omer Shubi, together with master’s student Cfir Hadar, under the supervision of Assistant Prof. Yevgeni Berzak, who heads the Technion’s Language, Computation, and Cognition Lab and joined the university in 2021 after completing his doctorate and postdoctoral research at the Massachusetts Institute of Technology. They also published their findings in the journal ACL Anthology under the title “Decoding reading goals from eye movements.”
Berzak, who was born in Moscow and was brought to Israel as a child with his younger brother in 1990 by their parents – his father an aviation engineer and his mother a social worker – joined the faculty in 2021.
He earned a bachelor’s degree at the Hebrew University of Jerusalem in cognitive science, a master’s degree in computational linguistics at the University of Saarland in Germany and the University of Nancy in France, followed by a doctorate in computer science from the prestigious Massachusetts Institute of Technology in 2018 and a postdoctoral position at MIT’s Department of Brain and Cognitive Sciences.
He speaks five languages – Hebrew, Russian, English, German, and French – and said he doesn’t confuse them when he speaks. “I was always interested in languages as a youth. Coming to Israel, I had to learn Hebrew, so I was even more interested,” he recalled.
“I FOCUS in my interdisciplinary research on how humans acquire and process language by combining linguistic and cognitive theory, computational modeling, and behavioral and neuroimaging studies. I also examine how natural language processing in machines can be brought closer to human linguistic abilities by utilizing insights and data from human language processing.
“This study is part of a broader research program in which we are developing AI models that infer, in real time and from eye movements alone, key aspects of the reader’s linguistic knowledge, their interaction with the text, the difference between a first and a second reading, the readability of a given text, and even the specific information the reader is seeking,” Berzak explained in an interview with The Jerusalem Post.
“These studies pave the way for new methods in assessing linguistic knowledge, personalizing texts according to the reader’s linguistic and reading proficiency, improving accessibility to textual information for various populations, and more.”
Popularity of eye-tracking technologies
Eye-tracking technologies are becoming increasingly widespread, affordable, and accurate, with some now available on common devices like tablets and smartphones. The researchers hope these developments will speed up the adoption of their models, benefiting both users and content providers in fields such as education, government, and media.
“We did the study solely online, with people reading from a computer screen. Everything is digital today. Keep your eyes on the text. Everything is digital. There are characteristics that are universal to many languages,” he said.
For example, when one reads English, the eyes focus on less text on the left but three times more text on the right. When reading Hebrew, it does the opposite. “We didn’t test Chinese or Japanese that go down and then up, or Russian or Amharic. But we are working on eye movements of people who read English as a second language and on predicting a person’s mother tongue,” the linguistic expert noted.
“We asked participants if they were looking for data or not, what kind of information, and how much they understand. I don’t know of any possible connection between [the] speed of reading and intelligence. It depends on all kinds of variables. Speedreading works differently; it takes a lot of practice, and there are several techniques for doing it,” Berzak added.
“We haven’t yet worked with children, but we hope to do so soon, and in the future, [we hope to work] with the deaf and sign language, the elderly, people with cognitive problems, and even those with autism who have learning disabilities.”
The research could lead to websites like those of municipalities, for example, quickly recognizing with their eye movements what information they are seeking and tailoring the text to specific users.
“I am less interested in companies determining what products people are interested in. Although everyone with a smartphone can be followed, privacy is important to us,” he concluded.