Akiko Ishii was referred to a job training program where she hoped to learn how to use a computer and work in an office. Instead, she was told she should probably study to be a massage therapist, that it would be a good profession for someone like her – someone who was blind. Ishii was having none of that.
Newly blind following a surgery at age 30, Ishii was determined to achieve her goals. That determination paid off with a job in the human relations department of an IT company in Tokyo. Now, she and her husband have a company that gives classes ranging from yoga to technology know-how for people who are blind or have low vision.
Ishii manages a very full life, and in recent years has been able to do it more easily with her “assistant,” as she calls Seeing AI, the free Microsoft app for iOS for those who are blind or who have low vision. The camera app uses “channels,” or modes, to read printed text, currency, and describe physical objects, product labels and colors, among other things. Now, coinciding with the United Nations International Day of Persons with Disabilities, Microsoft is expanding Seeing AI support from English to another five languages: Dutch, French, German, Japanese and Spanish.
Now I will be able to read in Japanese. And I can also read my mail and other communication and correspondence on paper, so that will be super exciting.
The additional language support will make Seeing AI, which uses artificial intelligence (AI), more accessible to the millions of people around the world who are blind, and the growing number of people with low vision.
“Now I will be able to read in Japanese. And I can also read my mail and other communication and correspondence on paper, so that will be super exciting,” says Ishii, who is eager to use Seeing AI’s Short Text channel, which speaks text as soon as it appears in front of the camera, and the Documents channel, which reads larger articles of printed text.
Previously, “If any communication was on paper, I would have to have somebody read it to me,” she says. “But now I’ll be able to read it myself.”
That’s exactly the kind of comment that means so much to Saqib Shaikh, Microsoft project lead and co-founder of Seeing AI, who is also blind.
“More than the technology itself, the thing that has really touched me is the way that people have taken the features of Seeing AI and incorporated it into their personal lives,” Shaikh says. “I get the most joy hearing from customers about how they’re using Seeing AI.”
That includes an email from a woman who used the app to read handwritten notes that her mother, now passed away, had written to her years ago. Or a father who used the Short Text feature at a zoo to read information on placards and signs to his young son.
“The father was feeling a bit down before because he couldn’t really interact at the zoo, tell his son what was around him,” Shaikh says. “But with Seeing AI, he was able to read the signs. And suddenly now, he could have that father-son moment of teaching his son where the monkeys live, and what they eat and all that kind of thing.”
The thing that has really touched me is the way that people have taken the features of Seeing AI and incorporated it into their personal lives.
Seeing AI launched in English in 2017 and is available in 70 countries. The app leverages AI technology, and so far, has helped people with more than 20 million tasks. Developed with inclusive design in mind, the app is tested by and with the disability community via nonprofit organizations around the world.
Among those testers is Manuel Pereira of France, who was born blind. He is the head of the digital accessibility department of the AVH association, which supports people who are blind or who have low vision with access to employment and other services. While France requires accessibility in many physical areas, the reality is very different when it comes to mobile and web apps, he says, among the reasons he is happy about a French version of Seeing AI.
He thinks of Seeing AI as his personal “toolkit.” “There are a lot of useful applications on it – the ability to read short text, scan documents, detect light, and more. What’s impressive in this application is that you have a lot of tools you can use in your daily life to give you more autonomy,” he says.
“When I receive a letter, for example, I can use this application very easily to know who sent me this letter very quickly – and I can choose to read all the letter immediately or to read it later, if it’s not very important.”
Having Seeing AI in his native language will also be a huge help to culinary aficionado Cristian Sainz of Spain.
Sainz, 50, lost his vision after a car accident when he was 18. Today, he is head of digital workplace for National Organization of the Spanish Blind (ONCE), a foundation that helps people who are blind or with other visual impairments.
A cook and former restaurant owner, Sainz says one of the features he likes most about Seeing AI is that he can use it to learn the exact measurements he needs when he is cooking. He has a scale that speaks the weights of food, but it only speaks measurements by degrees of 5 grams.
“Sometimes with a recipe, I need 1 gram, not 5, and I can get that information in seconds by pointing my phone with Seeing AI’s Short Text channel to the scale’s screen and Seeing AI then reads it out loud. And I’ve been able to start using recipes that I couldn’t before.”
A Seeing AI feature that was developed in response to user requests – the ability to explore photos by touch and hear descriptions of images from family pictures to those on social media – was added earlier this year. With the feature, “It’s very easy to know the information that is sent in photos,” Sainz says. “I swipe my finger on the screen, and it describes where the objects are in a scene.”
Florian Beijers in the Netherlands agrees. Until now, photos had little meaning for the 27-year-old, who was born blind.
“Now I can sort of ‘see’ where something is in the picture, what else is in there, how they’re related to each other,” he says. “The geek in me appreciates having that information.”
The geek in him led Beijers to become a web developer, something, he says, often surprises prospective employers when they meet him. While most understand “blind people aren’t sitting behind the windows doing nothing anymore,” Beijers says, “I still encounter companies that are still in the, ‘Hey, do blind people use computers?’ stage.”
Having Seeing AI in Dutch will be a “game changer” for making daily tasks easier in a country where accessibility is not as good as it should be, he says.
In restaurants, he says, menus often aren’t accessible. “There’s not always a web-based version of it you can look up on your phone.”
Beijers says for him, Seeing AI’s Document channel, which provides audio guidance to capture a printed page and recognizes the text, along with its original formatting, “is very useful, especially for a menu.”
Domingos de Oliveira, a digital accessibility expert in Germany, says there’s a phrase for Seeing AI in his language. “We call universal tools like Seeing AI ‘Schweizer Taschenmesser,’” or a Swiss Army knife, he says.
De Oliveira, who was born blind, says he “didn’t really have any other choice but to intensively engage with technology” when he was younger. “I couldn’t write homework by hand or write letters, so I was interested in technology from the very beginning, just to overcome the barriers that come with blindness.”
He works for the organization Aktion Mensch, which raises money and supports projects for children and adults with disabilities.
The German version of Seeing AI will increase the use of the app dramatically.
“Many Germans did not use Seeing AI because their English is not good enough to understand the English voice” that has been part of the app, he says. “The German version of Seeing AI will increase the use of the app dramatically.”
Akiko Ishii thinks it will be the same in Japan. In Tokyo, accessibility awareness is high, she says, but “as you go outside of Tokyo, then it gets very difficult.” There is little technology being utilized in rural areas, she says, and with Seeing AI being free to use, it will be a big help.
Ishii regularly uses the app’s Scene and Person channels to recognize and describe the people around her, including her 3-year-old daughter, Ami. “I use that feature to get the phone to tell me what she’s doing,” Ishii says.
Or, if Ami asks Mom for chocolate, Ishii can go to the kitchen and use the app to “read the bar code of the chocolate, so I will know exactly which chocolate she wants.” It is the kind of independence that is priceless, she says.
“One of the things I love about Seeing AI is that once you have this one application, with tons of features, then you can use it for anything,” Ishii says. “Seeing AI becomes my assistant and provides me with support that I need so that I can focus on taking care of my daughter.”