Microsoft and the University of Amsterdam in the Netherlands have designed a talking-camera app that describes the world for the visually impaired.
The app, called Seeing AI, uses a smartphone's camera and computer vision to recognise people, objects and scenes in the environment. It then narrates what it "sees" to the user. For example, if pointed at a person, it can identify who they are, their emotional state and age, and how far away they are standing.
It can also recognise household products, scan documents and identify different denominations of dollar bills – despite them all being the same size and colour.
"Beyond performing the essential task of optical character recognition, the app is also able to guide the user to move the camera left or right to get the target in the shot," tech lead Saqib Shaikh told The Verge at a Microsoft event in London.
The prototype of Seeing AI was first presented last year in March, but Microsoft has now released the free app to all iOS users in the US.
Designing with empathy is crucial to supporting marginalised consumers and breaking free from social stigma. As highlighted in our Diversity Outlook 2017/18 Innovation Platform, the market for products targeting overlooked minority groups is growing, as design in urban environments becomes more inclusive.
For more on emerging products and services that are sensitive to the needs of those with physical disabilities, see Diversity Outlook Scenario.