'Be My Eyes' Phone App Testing ChatGPT To Help Blind Customers

Exploring the unknown can be overwhelming if you are blind. But for Jesper Holten, a new powerful tool is giving him greater freedom.

Danish phone app Be My Eyes is testing using ChatGPT, an artificial intelligence chatbot, to analyze images and explain them to customers.

"It tends to be in daily situations like when I'm cooking food and want to have a look at tins," says Holten.

The app sends the photos to ChatGPT’s image-to-text generator. Holten demonstrated how it works by taking a picture of a can of sardines. Then the app analyzed it and a voice from his phone described the image, saying, “Virtual volunteer: This is a picture of a can of John West sardines. The label is green and yellow and there is a wrinkle on the top of the can."

The app currently connects the blind with one of millions of sighted volunteers to help understand the picture and can translate around 180 languages. Be My Eyes hopes to roll out the new ChatGPT feature soon.

"I see a potential for people to have more independence. For some people, it can even be difficult to ask other people for help," says Holten.

The founder of Be My Eyes, Hans Wilber, believes AI’s potential is endless, especially when the new feature finally rolls out. "I have really had a hard time sleeping since this because I think there is so many possibilities where this could go,” says Wilber.

And those possibilities excite Holten. "If the AI technology can help me gaining or regaining that level of confidence, that would really be something," he says.