Facebook's new tools to help the blind navigate social media

Naquela Wright's life took an unexpected turn when she lost her eyesight as a teenager, but even when her world became immersed in darkness, the New Jersey resident didn't want to quit social media.

"I'm not even going to lie. I was 18 years old and I was like a Facebook junkie," said Wright, who is now 23. "Facebook is a good outlet to say how you feel and what you think."

Navigating Facebook was a challenge at first. Diagnosed in 2010 with pseudotumor cerebri, a rare health condition in which pressure increases around the brain and can result in the loss of vision, Wright learned how to use a screen reader to peruse the site through the tap of keystrokes and sound of a robotic voice. Still, when a friend tags her in a photo, Wright often has no clue what the image shows.

Now Facebook is trying to solve this problem by harnessing the power of artificial intelligence to create new tools that not only describe items in a photo but allows users to ask what's in an image.

Social media is not only becoming more popular but visual, filled with a stream of photos, GIFs and videos, which sometimes leave the blind and the visually impaired out of what their friends and family are posting. Nearly 2 billion photos are shared daily across Facebook, Instagram, Messenger and WhatsApp, and the world's most popular social network has more than 8 billion video views daily.

"I can have a basic picture in my mind of what's going on in the picture and now I can comment on my own," said Wright, who got to try out the new tools that are still being tested. "Of course, it's different, but it's something more than I had."

The online social networking service is testing these features in 32 languages, and if all goes well, hopes to publicly release the new tools early next year.

"Social media is becoming more and more important whether it be for advertising, networking or keeping in touch with family and friends. The same is true for people who are blind and visually impaired," said Lee Huffman, editor-in-chief of a technology magazine run by the American Foundation for the Blind.

An estimated 285 million people are visually impaired globally, according to the World Health Organization, and research conducted by Facebook showed that blind users have trouble figuring out what's in a photo because the description isn't clear or doesn't exist.

At Facebook's Building 20 in Menlo Park, a series of gadgets including a braille display sit in an "Empathy Lab," allowing the company's employees to walk in the shoes of people who face barriers when attempting to access the site.

Facebook engineer Matt King, who is part of the company's accessibility team, is blind so he knows firsthand what's it's like to use social media without sight. Browsing through Facebook more than five years ago was so frustrating for the engineer that he gave his wife the password to his account and she would start "liking" and commenting on posts using his name.

Since then, Facebook has made it easier to skim through the content on its website with a screen reader by improving HTML headings, adding alternative text for images, launching keyboard shortcuts, and more. Using artificial intelligence to describe photos, King said, is only a part of these ongoing efforts.

Demonstrating the new photo features, King scrolls through his News Feed, which is filled with images of kids, profile pictures of his biking buddies and a couple who just got engaged.

"This image may contain two people, smiling outdoors. Link. Image," a female voice named Vicki said.

In another demonstration, a baby is smiling in a bathroom. "What is the baby doing?" a male voice heard in a video asks. "Brushing teeth," a robotic female voice responds.

"I wouldn't want all my friends to describe their photos for me," King said. "It's adding something to their life that makes it harder for them to post photos. It adds friction and I don't want to add friction."

With 1.5 billion users, Facebook isn't the only social media company that wants to improve its website for the visually impaired. Along with Facebook and other major tech firms, Twitter and LinkedIn have their own accessibility teams and belong to an initiative called "Teaching Accessibility."

Jeff Wieland, Facebook's head of accessibility engineering, said the group wants to educate more engineers, especially early on in college, about designing products that are compatible with the disabled and others.

"We really don't want accessibility to be the luxury of a handful of companies," Wieland said. "We want everything around the world to be built with accessibility in mind."

Blind runner Simon Wheatcroft joined Facebook during its early years as a way to keep in touch with his friends. He then signed up for Twitter and in 2011 used the microblogging site to find runners willing to help him train for a 100-mile ultramarathon.

Nowadays, the 33-year-old who lives in Doncaster, England, notices that his social media feeds are flooded with more photos.

"The majority of the time, I'll just skip it and not get involved in the conversation," said Wheatcroft. "But if it's something I really want to know then I'll ask a sighted person what's in the photo."

There are mobile apps such as TapTapSee that can identify objects in photos, helping the blind "see" what's in front of them with the help of technology, and also can help them take pictures. But Wheatcroft chooses to snap photos the old fashioned way: touching an object and then moving backward with his phone. He also recently joined Facebook-owned Instagram and although it's difficult to consume photos, Wheatcroft still enjoys creating images.

"We all capture our lives through social media with photos at a point in time and my children might be interested in seeing this stream of consciousness in the future," he said.

As for the new features that Facebook is testing, Wheatcroft thinks it's about time.

©2015 San Jose Mercury News (San Jose, Calif.)
Distributed by Tribune Content Agency, LLC.