The App that will interpret baby language

Saturday June 25 2016

It takes only 15 seconds for the app to “listen” to a crying

It takes only 15 seconds for the app to “listen” to a crying baby and to make an analysis. photo by Enru Lin 

By Enru Lin

What’s baby saying? A team in Taiwan made an app to tell parents if their baby is hungry, tired, in pain or needs a fresh diaper.

Over three years, researchers collected about 300,000 sounds from 100 newborn babies at the National Taiwan University Hospital’s branch in Yunlin County, Taiwan.

They made a smartphone app called Infant Crying Translator, a cloud-based program that can decode the crying of babies from the moment they are born until they are six months old.

It takes only 15 seconds. Tap “Record” in the app and a clip of the infant’s cry is uploaded to a cloud database. The file is quickly compared to an audio library and a verdict pops up onscreen.

After winning an innovation award from Taiwan’s government in 2014, Infant Crying Translator for Android and iPhone went on sale in 2015 and now has some 10,000 users worldwide.

Better accuracy
The app isn’t perfect, but results are getting better all the time.

According to user feedback, accuracy reaches 92 per cent for babies under two weeks old. For babies under two months, the accuracy of the app was once low but now can be as high as 85 per cent. For a four-month-old baby, accuracy is 77 per cent and rising.

“When we first launched, the app wasn’t particularly accurate for babies older than two weeks. Back then, the database only had audio of very young babies that we gathered from a hospital,” said Chuan-Yu Chang, a head researcher.

“Now the library has many, many more sound files uploaded by users, and it can make increasingly better judgments for a wider range of ages.” In addition, a machine-learning algorithm lets parents personalise the app for their baby.

“Based on the actual situation for each baby, the cloud database will make unique data models to improve the recognition rate,” he said.

High tech, low birthrate
Chang is from Taiwan, a high-tech island responsible for assembling some of the world’s top information and communication technology products.

It’s also a place where the birthrate has slipped to one of the lowest in the world, as a high cost of living prompts more Taiwanese to put babies on hold in favour of careers.

The Population Reference Bureau reported that the 2015 fertility rate – the average number of babies that women have during their childbearing years – was 1.2 in Taiwan, behind the world average of 2.5 and the Asia average of 2.2.

The Taiwanese government has enacted measures like childcare subsidies and free daycare, to only a modest effect.

Chang’s team at Yunlin University of Science & Technology hopes that baby translation can do its part to reverse the decline, which Taiwan’s president has declared a matter of national security.

“There are many ways to apply the programme so that it is even more helpful for parents,” Chang said.

Developers are working with the government to incorporate the baby translator in a mobile suite for expecting mothers, which is aimed at making parenthood less intimidating.
They are also working with a leading Taiwanese original equipment manufacturer (OEM) to create a bedside appliance that lets parents monitor their baby from afar.

The device, slated for release this summer, has a built-in microphone that can detect crying and then automatically switch on the translation program. The result is beamed to the parents, who can come running with exactly what the child needs. “Even if parents are busy and cannot be right beside the baby, they can know how their child is doing,” Chang said.

Babies without borders

Surprising finding. Chang said that while developing the app, a surprising finding has been that babies born in different countries sound mostly the same, at least in the earliest stage of life. “Newborn babies of different nationalities cry in mostly the same ways,” he said, adding that the same goes for male and female newborns.

Global market. Chang’s team is convinced that a reliable baby reader has an important place beyond Taiwan and in the global market.

Clarity and relief. Everywhere in the world there are experienced parents, who even on a bad day could outperform the app. But for first-time parents of any nationality, the app could offer some clarity and relief when faced with an inconsolable newborn.
“From my own experience as a father, I know that sometimes when the baby cries, the parents feel a bit like crying, too,” Chang (below) said. “Humans have emotions and they make mistakes. The app doesn’t get flustered. It simply reads the data.”

The China Post, Taiwan