Car Makers Open Pandora’s Box With Facial Recognition

Facial recognition technology has become increasingly common in the world of consumer electronics, with Apple having introduced its Face ID system for unlocking the iPhone all the way back in 2017. The idea of porting this type of biometric information for use in the automotive world, however, is much more recent.

Genesis, Korea’s nascent luxury brand and a driving force in the EV world, first brought facial recognition to market with its GV60 electric SUV, allowing drivers to unlock the vehicle with a smile. Sony and Honda have announced plans to follow suit, and Ford has filed a patent application for “enhanced biometric authorization.” (In a statement, Ford acknowledged the patent, but said patent applications “aren’t necessarily an indication of new business or product plans.”) In another implementation, many automakers already make use of attention-tracking technology that constantly scans the driver to determine their eye and face position and ensure that they are focused on the road ahead. This seemingly harmless safety feature gives them a foothold into further exploration of how biometric information could be harvested and used by the vehicle.

At the root of each of these applications are two core assumptions. The first is that drivers are comfortable with having their biometric data scanned and stored by the company that builds their car (and then, potentially, shared with any of the dozens of partners that automakers monetize that data with). The second, and seemingly more basic assertion, is that this technology actually works as designed.

The reality is that neither of those pillars of facial recognition hold up to any real scrutiny — and it’s precisely the obscurity of the details surrounding the implementation of this technology that has allowed it to skate, relatively unexamined, into showrooms across the country.

Recognizing Facial Failure

Why would a car company implement something like facial recognition in its vehicles if it doesn’t actually work as advertised? The trouble is tied to the fact that while leading facial recognition systems are capable of 99% accuracy, that statistic only applies to identifying white male faces. In fact, if you happen to be a woman or a person of color — or both — the viability of facial recognition technology can drop drastically, sometimes to as low as 65%. That’s roughly one in three attempts to unlock your door with your face ending with you having to dig into your pocket for the key fob.

The key to facial recognition’s failure is a function of the data sets it is used to train on, with leading systems having drawn from primarily Caucasian photographs. This constraint, which is certainly not advertised by tech companies seeking to make inroads with their latest products, has resulted in consistent misidentification by law enforcement agencies using facial recognition software, with multiple false arrests of innocent Black citizens making headlines over the past several years. The problems are so pervasive that major cities across the country are banning the use of facial recognition by government agencies.

If police and municipal services don’t have access to functional facial recognition, then what are the odds a car company can rise above its serious failure rate in a commercial application — particularly one where owners could be left unsafe, locked out of their cars, in terrible weather far from home?

It’s the “legitimate” use of biometric data that presents the more pervasive, and much less visible threat.

Speaking of weather, how do hats, sunglasses, scarves and masks factor into a vehicle’s facial recognition algorithms? In winter climates, it’s rare for individuals to leave the house without at least some type of head or facial covering, and summer sun dictates the use of eye protection and hats across most of the country. With existing driver attention monitors often blocked by something as simple as steering wheel position, it’s hard to imagine a world where flawless facial recognition is able to deal with these common accessories.

Finally, there’s the problem of aging, something none of us can stop, and a major issue for facial recognition software. It can take as little as five years before your faceprint “wears out” and you lose access to whatever’s sitting behind that biometric wall.

The Mass Delusion of Driving

Behind the wheel, we’re overconfident, inattentive and over-reliant on tech. According to those who study driver behavior, instead of solving these problems, we’re making them worse.

Dude, Where’s My Data?

Even in a world with flawless facial recognition, the question of what, exactly, happens to a customer’s biometric data, and the full scope of the dossier this type of surveillance system might be gathering on a driver, is just as thorny.

To its credit, Genesis claims that no biometric data ever leaves the confines of its vehicles, as it exclusively uses local storage of the faceprint. What happens if the vehicle is stolen, however? How safe is that data from potential thieves who can sell off a valuable part of your identity alongside rims, stereos and engines as they pick apart your ride?

As alarming as having the GV60’s facial database in the hands of criminals might seem to potential customers, it’s the “legitimate” use of biometric data by partner companies that presents the more pervasive, and much less visible threat. People’s eyes generally glaze over just before they click “accept” on any terms of service associated with the software powering a new mobile device or app, but we enter into tacit agreements with automakers about how our information is being used simply by sliding behind the wheel.

The vast majority of brands are repackaging information about how you use and what you do in your car, how it interacts with your phone, where (and how often) you drive, and then selling it to third parties. Almost all of this occurs under the radar, with no details provided about what digital protections are afforded owners, how secure the data is or what systems are in place to prevent it from being shared with law enforcement without a warrant.

Adding biometric data to this mix essentially casts your faceprint into the sphere of digital commerce with almost zero control over how it is used. Then there’s the fact that facial recognition is the thin end of a surveillance wedge that unlocks even deeper excavation of your private information. A vehicle doesn’t have to record audio of your in-car conversations if it can simply read your lips, for example, skirting the law at every turn while mining your life for all it is worth.

The communal character of automobiles only accentuates how problematic this is. Even if owners were made aware of, or required to agree to terms of service about how their vehicle harvests the details of their personal lives, it’s unlikely that every passenger, or even every family member, who borrows the keys will have done the same.

You Are the Product

Perhaps the most important thing to remember in this entire discussion about the merits and drawbacks of facial recognition, and the gathering of biometric information by automakers, is that all of this is occurring inside of a product that you are paying for. You are actually handing over your hard-earned dollars for the privilege of having your personal life stripped of any and all valuable data points. The profits generated by that invasion of privacy in no way benefit you other than through distracting marketing gambits that reference “convenience” as though it somehow balances the scale.

As drivers increasingly become profit centers to be exploited by car companies diving head first into the same kind of business model that powers online giants like Google and Amazon, it’s imperative to ask the question: what’s in it for you? Is it really necessary for us to subject ourselves to constant surveillance behind the wheel so that a multi-billion dollar company might pad its bottom line, all while we expose ourselves to the inequities and insecurities associated with these systems?

Perhaps if one was forced to “click to accept” terms of service that flashed that very question across the dashboard before each and every drive, automakers might be held more accountable for disguising invasive technologies as must-have features.

Update – February 9, 2024: Ford’s plans for facial recognition technology were clarified: the company is in the patent process, but hasn’t announced any business or product plans.