Facebook Can Now Recognise You Without Even Seeing Your Face

You know how you do that thing where you cover your face in pictures so even if your friends put them up on Facebook (without your permission, obviously) no one will be able to tell who you are? Turns out Facebook knows exactly who you are even without seeing your face.

New Scientist reports that Facebook’s got a new algorithm that allows recognition without needing to resort to anything as basic as facial features. Instead, the experimental (meaning it’s coming soon, don’t worry) algorithm looks at other recognizable characteristics to pick out who’s who in all those party pics that were shot in low light at bad angles.

From New Scientist:

“There are a lot of cues we use. People have characteristic aspects, even if you look at them from the back,” [Yann] LeCun says [head of Artificial Intelligence]. “For example, you can recognise Mark Zuckerberg very easily, because he always wears a gray T-shirt.”

The research team pulled almost 40,000 public photos from Flickr - some of people with their full face clearly visible, and others where they were turned away - and ran them through a sophisticated neural network.

The final algorithm was able to recognise individual people’s identities with 83 per cent accuracy. It was presented earlier this month at the Computer Vision and Pattern Recognition conference in Boston, Massachusetts.

While this could be a great feature for Facebook’s business (“moments” may be using this technology in the near future), it also raises some concerns about privacy. For instance, why does Facebook need to know who’s in every photo or what you’re doing? What if it starts tagging you without permission? Yes, it’s great that Facebook can alert you to photos of you being posted, but as New Scientist points out, the flipside of that coin may be a little bit more sinister.

“If, even when you hide your face, you can be successfully linked to your identify, that will certainly concern people,” says Ralph Gross at Carnegie Mellon University in Pittsburgh, Pennsylvania, who says the algorithm is impressive. “Now is a time when it’s  important to discuss these questions.”

The head of Facebook’s artificial-intelligence research lab says the software can identify users 83% of the time even if their face isn’t visible

We are just beginning to come to grips with the idea that computers and algorithms can recognize our faces, and the implications that has for privacy. Now the head of Facebook’s artificial-intelligence research lab says that an experimental algorithm he helped develop for the giant social network can recognize you with a high degree of accuracy even if your face is hidden from the camera.

Yann LeCun, an expert in computer vision and pattern recognition who was hired by Facebook  FB -2.47%  in 2013, presented his research at a recent conference in Boston. He told New Scientist magazine that he wanted to see whether the same kinds of algorithms used for facial recognition could be tweaked to recognize people from other physical characteristics—their body type, the way they stand, etc.

“There are a lot of cues we use. People have characteristic aspects, even if you look at them from the back,” said LeCun, a former BellLabs researcher who helped develop the algorithm used by many U.S. banks to verify handwriting on checks. “For example, you can recognise Mark Zuckerberg very easily, because he always wears a gray T-shirt.”The research took 40,000 public photos from the social network, some of which showed people with their faces fully visible to the camera and others with their faces partially or fully hidden. After running them through the recognition filter, LeCun said the system could determine a user’s identity with 83% accuracy. Using its existing algorithms, Facebook has said that it can recognize you with 98% accuracy—in fact, its software can identify you in one picture out of 800 million in less than 5 seconds.

Companies like Facebook are interested in facial recognition so that they can help users organize their photos, the way the social network wants to do with its recently launched Moments feature—which automatically sorts your pictures into different categories, and can detect when you and your friends upload photos of the same event. The description of the new service says:

“Moments groups the photos on your phone based on when they were taken and, using facial recognition technology, which friends are in them. You can then privately sync those photos quickly and easily with specific friends, and they can choose to sync their photos with you as well.”

Google is also focusing on similar features, which can recognize people, places and things in pictures and automatically organize or tag them for search. LeCun said that his recognition algorithm could be useful in helping people find out when someone else uploads a photo of them to Facebook, even if their face is not visible, which would help the privacy-conscious keep track of where their pictures are being published.