Firms are increasingly tracking users’ eye movements, to the discomfort of ethicists

When Facebook reinvented itself into Meta in October 2021, it was widely reported that Meta would be focusing on virtual reality or VR by being at the forefront of the metaverse.

But Meta has not given up on the world of bricks and mortar yet, as reflected by the company’s massive investment in augmented reality or AR glasses.

My research considers smart real estate and human-computer interactions in smart environments.

Meta is only one among many companies betting that the future of physical space will involve merging with digital space, resulting in an augmentation of our reality. Apple, Google, Snap, Microsoft and a string of other tech companies are working on AR wearables: AR glasses, smart contact lenses and AR headsets.

My research considers smart real estate and human-computer interactions in smart environments.

Meta is only one among many companies betting that the future of physical space will involve merging with digital space, resulting in an augmentation of our reality. Apple, Google, Snap, Microsoft and a string of other tech companies are working on AR wearables: AR glasses, smart contact lenses and AR headsets.

Insight into the subconcious

As part of its Reality Labs, Meta spearheads Project Aria, which drives the pilot development of AR glasses under the umbrella of a research experiment undertaken with academic partners. The company promises that users will be able to use AR glasses to switch on a lamp by simply staring at it and being able to find their keys quickly.

However, there is one dimension of AR wearables that developers of such devices tend to downplay or ignore altogether: it is eye tracking and what information related to the way we interact with the world through our gazes and eye movements are captured and analyzed.

Psychologists have long identified that eye movements are unfiltered signals, giving insight into humans’ subconscious cognition.

Understanding attention

Eye tracking in the context of AR devices has received much interest from Big Tech. Eye tracking was originally designed as a methodology to help researchers understand and record visual attention in a research lab environment as far back as the 19th century.

It has been customarily applied to cognitive psychology, marketing research and, more recently, human-computer interactions where it can facilitate the life of patients with disabilities.

Modern eye trackers generally use a method known as corneal reflection, where a near-infrared light is used to illuminate the eyes, causing a reflection that is detected by a high resolution camera. Advanced image programming then identifies the point of gaze and the stimuli, making it possible to draw a heat map of where a person was looking in a given environment. In addition, data captured include pupil position, blinking patterns and eye movements.

In recent years, the range of eye tracking applications has considerably broadened, from driver monitoring systems, attention management in education, health care for the elderly, e-commerce website design and even video games as a tool to build “emotional journeys” for players.

However, these applications are usually carried out as part of product development or research projects, not as inbuilt features in devices aimed at the consumer market.

Privacy is not enough

Indeed, embedding eye trackers in consumer-driven AR devices is taking what was originally a scientific methodology into the real world. Developing AR wearables with eye tracking possibilities for the mass market epitomises the unrelenting appropriation of humans’ most intimate living spaces by technology.

It is easy to brush aside the issue by claiming that eye tracking is necessary for users to get the full benefit of AR. For example, Project Aria’s developers explain that for AR glasses to work, “they need to have a good sense of where you are, what you’re looking at, and what action you might want to take”.

To preempt users’ concerns, Meta’s ethicists insist on privacy. However, focusing on data transparency and advanced research to anonymise eye tracking data will not prevent eye trackers from monitoring users’ interactions with the outside world at levels of consciousness we are not even aware of.

Augmented reality is big business with an unparalleled ability to monetize our very being in the built environment. Meta has reportedly invested billions of dollars in what it calls the “holy grail” of fully fledged AR glasses for all.

Undoubtedly, with AR wearables, what drives its implementation is the potential for monetization through targeted advertising.

Evading responsibility

Companies developing eye-tracking products tend to dodge responsibility by asking for self-regulation of the nascent AR industry.

My research on the implementation of pervasive technologies in the built environment shows that in the context of utilitarian trade-offs imposed by embedded technologies on users in smart environments, self-regulation does not work.

Whether users prefer to get satisfaction from AR at the expense of their freedom, or to be free at the expense of their satisfaction, is the key question.

The use of eye-tracking technology should be strictly controlled by external regulators. Users should always have the legally defined right and ability to make informed choices about opting into eye-tracking whenever they use wearables in both augmented and virtual realities.

That is absolutely crucial to make sure that immersive technology does not lead to a most dystopian future.

This article first appeared on The Conversation.