Apple is facing our masked future. This week, the company began testing new software for the iPhone that will allow device owners to unlock the device using a face shield. There is a catch, however, that aligns with Apple’s strategy of pinning people to different Apple products and highlights how challenging it can be to develop accurate facial recognition technology: the new facial unlock feature requires an Apple Watch.
The first beta developer of iOS 14.5 includes updates for application tracking controls and Siri along with the face mask function. Application manufacturers typically gain early access to the latest version of iOS to launch or refit their applications well before the formal release of the software. (Brave souls who don’t care about the risk of potentially blocking their iPhones can also sign up for public beta releases.) The full version of the software is expected to be made available to the general public this spring.
This means that when most people install the latest version of iOS on their iPhones, we will be wearing masks for a year or more to help prevent the spread of the coronavirus. Compared to all the other ways the pandemic has changed our lives, having to use a method other than Apple’s facial ID to unlock the iPhone is not a major inconvenience. Still, it is frustrating to hold the phone close to your face and remember that the facial ID will not work because of the mask. The promise of facial recognition technology – which coexists with very valid concerns about its misuse and error rates for people with darker skin – is that it should get smarter and better over time.
With the next software update, Apple is more or less shifting the authentication burden to the Apple Watch. If you’re using a newer iPhone model (one with a facial ID) and you’ve installed iOS 14.5 beta software, and If you’re using an Apple Watch with watchOS 7.4, raising the locked phone to your face will trigger some communication between the phone and the watch. The phone will be unlocked. The watch will also display a notification that the phone has been unlocked. An iOS developer described for WIRED as an experience similar to unlocking a Mac with an Apple Watch.
Like 9to5 Mac notes, this is the second change that Apple has made to its face ID authentication system to accommodate masks. Last spring, the company released software that made it easier to avoid using Face ID while wearing a mask, showing the iPhone password screen after the first Face ID failure. However, these updates have their limitations. The new Face ID feature with face mask will only work with phone unlocking. Therefore, if you use Face ID for Apple Pay transactions or to sign in to third-party applications, you will still need to authenticate in some other way.
But the big question is why Apple is relying on the Apple Watch to unlock iPhones with masks, instead of releasing software that simply recognizes the bare part of a person’s face. At the time of publication, Apple had not yet answered WIRED’s questions about this. Experts say there are many ethical and technical considerations when implementing any facial recognition technology, but when it comes to performing facial recognition on partially covered faces, it is particularly challenging.
Anil K. Jain, who researches computer vision, machine learning and biometric recognition at Michigan State University, says that despite the progress of facial recognition in the past five to 10 years, he is still “susceptible to occlusion, that is, which part the face is not visible. In most cases, technology assumes that the person will remove their glasses and facial covers, the lighting would be homogeneous and the expression would be neutral, like a passport photo ”.