Smartphone cameras may help visually impaired users

Smartphone cameras nowadays are fully-featured and increasingly efficient. Frontal and wide-angle lens, vanshing cameras, ever higher resolutions and “panoramic” modes – the possibilities are endless. We are used to think of smart cams as devices deigned to produce entertainment in the form of picture taking; yet these devices may play a pivotal role and help visually impared people with many of their everyday hassles.

Visual handicaps and smartphone cameras are not the worlds apart that one may assume. Smart cams developed for visually impared users are old news; one of the first examples was the OrCam MyEye – a wearable camera that sticks to glasses for an enhanced visual experience. This device premiered in 2017, but accessible technology has come a long way since then.

Unsurprisingly it’s Mountain View that takes the challenge up yet another notch. Google has aimed always to shape the role of advanced technology into the near future, and accessibility is but one of the many cards up its sleeve.

LookOut, Google’s new app for smartphone cameras

We’ve told you that this new app allows visually impaired users to interact with their smartphone cameras in new and unexplored ways. But how does LookOut work? Let’ find out!

LookOut turns smartphone cameras into an actual additional eye for the users with visual handicaps, allowing them to be fully functional and overcome everyday obstacles. It can be a formidable help with urban mobility, grocery shopping and the basic housekeeping chores, which are often challenging and frustrating for the visually impaired.

Smartphone cameras, on their half, have all-powerful definitions that often exceed the perception of the human eye. However, the images alone may not be enough to overcome the impasse of poor eyesight or visual handicaps.

Some phone apps may turn the camera into an actual magnifying glass, but if the user is severely impaired, even this trick might not be enough.

Hence Google’s new LookOut app, with its synch of advanced technologies thriving to improve the lifetyle of users with disabilities.

Smartphone cameras are the “eye”….

As we’ve seen, Google’s LookOut app has brought smartphone cameras to yet new heights. As a tool to aid the visually impaired, these devices are invaluable. The ever increasing definition often results in highly realistic and richly detailed pictures.

Modern day smartphone cameras reproduce flawless images in just about any light conditions, both indoors or in dim light..

Furthermore, many smartphone cameras can create high definition panoramic shots from single pictures. As for all the technical finetunings, nowadays it is often left to the auto-focus and auto-mode of the individual smartphone.

The astonishing visual input notwhistanding, LookOut does something else – analyzing the visual elements and drawing a textual description of each photo. For example, the app can estimate the distance between two objects, spell out the indications on a label, or recognize the color of traffic lights.

As the first real app that relies not just on OCR, but actually “reads” photos, LookOut is going to play a pivotal role in making life more accesible and reliable for users with visual handicaps.

…and the Artificial Intelligence is “the mind”

Una app che legge le foto, quindi. Ma come è possibile estrapolare dati da un insieme di pixel?

You may ask, how can LookOut translate pictures into words? The answer lies in Google’s Artificial Intelligence Project. The Mountain View Labs are hip-deep at work on enhancing their AI. Back into 2018, smartphone cameras could already synch with the Artificial Intelligence through Google’s Clips. Technology has come a long way since then, as has the interaction between smartphone cameras and Google’s AI. If the smart cams are LookOut‘s eyes, the Artificial Intelligence may very well be its mind, processing the visual input just as a human brain would. From the camera roll, the AI can recognize shapes and colors, characteristics and even coordinates, assisting the visually impaired in their everyday life.

LookOut then translates the AI’s output into words through the phone’s own voice synthesizer, which enables the app to interact with the user, guiding him – or her – through every hardship and hassle.

LookOut can help visually impaired users in the kitchen, read out printed texts like medical receipt or leaflets, and even tell apart coins and banknotes whilst shopping. It’s not the first app of its kind – but by far the most complete and reliable tool for assisting users with visual handicaps.

This post is also available in: Italiano