Huawei has released StorySign, an Android app that can add Auslan, Australia’s version of signing, to selected kids story books. It leverages the power of Huawei’s AI technology and cloud, HiAi.

Yellow Wiggle Emma Watkins has joined as an ambassador for the StorySign app. As a passionate advocate for sign language, Emma is committed to supporting young children of the deaf community.

StorySign (free Android app) requires the purchase of the featured book. At launch it has Eric Hill’s bedtime story ‘Where’s Spot’. Other books will follow. The app focuses on deaf and hearing-impaired children from, 3-6 years old.

Storysign

Kyle Miers, CEO of Deaf Australia, said,

“Deaf children don’t learn to read in the same way as hearing children. Many struggle to learn how to read because they can’t match words with sounds. The deaf community needs accessible content to address children’s literacy development needs. StorySign helps to address this. Using AI and innovative technology could enrich the lives of Australian deaf children and their families in a meaningful way.”

To develop StorySign Huawei collaborates with:

  • Deaf Australia
  • European Union of the Deaf
  • British Deaf Association (UK)
  • Irish Deaf Society (Ireland)
  • Federation of Parents of Deaf Children (NL)
  • National Confederation of Deaf People (Spain)
  • National Federation of France for the Deaf(France)
  • Ente Nazionale Sordi (Italy)
  • German Association of the Deaf (Germany)
  • Portuguese Federation of Association of the Deaf(Portugal)
  • Federation of Flemish Deaf Organisations(Belgium)
  • Swiss Federation of the Deaf (Switzerland

Lisa Connors, Huawei Australia Corporate Responsibility Manager, said,

“Huawei believes in the power of AI and how that technology can make a positive difference in the world. We created StorySign to help make it possible for families with deaf children to enjoy the truly magical moments of story time.”

How does StorySign work?

Huawei in conjunction with animation specialists Aardman – the award-winning creators of Wallace & Gromit – and in partnership with local charities and publishing partner Penguin collaborate on the project.

Storysign

The child or parent opens the app and clicks on the selected book from the StorySign library. Then they hold their phone up to the words on the page and our friendly avatar – Star– signs the story as the printed words. This helps parents and children to learn to read together and at their own pace.

What else do you need?

It works best on the Huawei Mate 20 Pro, but Huawei states that it should work with any late-model Android handset that has the power and camera quality to support the app.

Storysign

Huawei says using the Mate 20 Pro adds

  • NATURAL READING – HiAI’s Image Recognition allows children to hold the phone up to a 45-degree angle to the book and the app will still recognise words perfectly. 
  • HIGH ACCURACY – HiAI’s OCR (optical character recognition) allows the app to function with a greater degree of accuracy. 
  • SPEED – The AI performance will also power the speed at which pages from the book can load in the app, allowing for improved user experience.

HiAI is Huawei’s a mobile terminal–oriented (smartphone) artificial intelligence (AI) computing platform.

Its first implementation was on the Huawei Mate 10 and P20 (both use the Kirin 970 Soc).

It exists as a replacement to Google Assistant, Google Lens or other voice assistants like Amazon Alexa, Apple Siri or Samsung Bixby in countries where these services and not always available.

HiAI is specifically for China, but it will likely come to the rest of the world. Although Huawei seems very content to use Google Services where they are available.

It has

  • Computer Vision (CV) Engine: CV is the capabilities by which computers simulate the human visual system to sense the ambient environment, and determine, recognise,and understand the composition of space. The capabilities include image super-resolution, facial recognition, and object recognition. Also called HiVision.
  • Automatic Speech Recognition (ASR) Engine: ASR converts human voice into text, to facilitate further parsing and understanding by computers. The capabilities include speech recognition, speech conversion,and text-to-speech (TTS).
  • Natural Language Understanding (NLU) Engine: NLU combines with the ASR engine, to enable Apps to understand human voice or text,performing communication or natural actions. The capabilities include word segmentation, text entity recognition, emotive tendency analysis, and machine translation. Also called HiVoice.

GadgetGuy’s take: StorySign

Great use of HiAI to help deaf or hearing-impaired children learn to sign.