Scientists create apps for special needs persons

Nanyang Technological University, Singapore (NTU Singapore), global video game developer YOOZOO Games and Singapore's Association for Persons with Special Needs (APSN) have jointly developed three mobile applications to help persons with special needs learn about social and emotional intelligence.

Following a nine-week pilot led by NTU Singapore involving 140 special needs students from APSN, the mobile apps were developed further in collaboration with YOOZOO Games.

Professor Ong Yew Soon, Director of Data Science & Artificial Intelligence Research Centre (DSAIR) at NTU Singapore, who led the project, said that the team's main goal was to use artificial intelligence (AI) for societal good.

"The benefits of these apps to special needs students are substantial," said Prof Ong. "Prior to this, learning about emotional intelligence was more of a pen-and-paper exercise. Now, they can enjoy a personalized learning experience, as well as leverage the ability of AI as an assistive tool to recognize and respond to feelings while picking up social and emotional skills in a fun and engaging manner."

"The project showcased a three-way collaboration between NTU, our industry partner YOOZOO, and APSN to use AI for the betterment of society. YOOZOO helped us to scale up the project with APSN by enhancing technology readiness and translation of research with target user experience and scalability to market in mind," he said.

Mr Stanley Tang, Acting CEO of APSN said: "APSN is glad to be part of this pilot program with NTU and YOOZOO Games, with the adoption of the AI-powered digital apps in three of our APSN schools: APSN Chaoyang School, APSN Katong School and APSN Tanglin School.

"These apps have enhanced the learning experience of our students with special needs, having personalisation and gamification injected into the lessons. We hope that such a collaboration will inspire more to step forward and join us in building and advocating an inclusive society."

Dr. Goh Chi Keong, Director of YOOZOO AI Innovation, said: "We are excited to partner with institutes like NTU to foster AI applications and education. The coming together of APSN's domain understanding, NTU's software IntelliK, the students and faculty from NTU, as well as YOOZOO's translational AI expertise, have really paved the way for positive societal impact, particularly during COVID-19.

"This is an extremely meaningful and exciting endeavor, and we will continue to contribute where it matters to society," Dr. Goh added.

Developed for mobile phones and tablet on both Android and iOS, the three applications are:

• EmojiCapcha—A child-friendly quiz app that rewards users for making facial expressions that match the emojis displayed on screen. It lets users learn different expressions and recognize emotions in a fun and casual way.
• Happy Bird—A fresh take on the all-time classic Flappy Bird, Happy Bird Lost Journeys is about an astro-bird who got a little lost in many different worlds. Avoid obstacles as you fly an astro-bird with your smile.
• Betterfly—The app allows users to create their own butterflies that come to life when they smile. This interactive app allows children to express and understand their emotions better through different visual effects.

Each of the apps developed on the made-in-NTU IntelliK AI Emotion Sensing Platform, can detect facial expressions using the mobile device's front-facing camera and then match them to the emotions they portray.

IntelliK is a creation tool which enables users to make AI powered digital apps and 'gamified' learning or training content without requiring any coding skills. It was created and developed by DSAIR and undergraduates from NTU's School of Computer Science and Engineering.

Thumbs up from APSN students for fun learning from home during COVID-19

Following the pilot, the apps have been released for use by students in APSN, who have given positive feedback on their enjoyment and educational factor.

Miss Nurrifahirah Binte Ahmad Sulimi, a Secondary 3 student from APSN Tanglin School said: "Through the apps, I learned more about different emotions and most importantly, how to better express myself."

Master Josh Koh, a Primary 4 student from APSN Katong School said: "I like Emoji Capcha. It is easy to use and I can see my face on the screen. It shows different emotions like happy, sad, angry and surprise."

Miss Chua Jie Yi, a Primary 6 student, from APSN Chaoyang School, added: "The games are so fun! I just want to keep on playing and learning."

Prof Ong also highlighted the positive impact of the learning apps during the ongoing COVID-19 pandemic.

"In a COVID-19 situation where students and teachers may be physically distanced, virtual learning and training can continue to take place since any personalized content customized by teachers can be accessed by individual students through their mobile phones or preferred devices that are conducive for learning," he said.

Custom-made experiences for different purposes and users

The IntelliK apps developed for special needs persons at APSN allow their teachers to personalize and customize them for contextual and progressive learning.

Educators can also adjust the apps' difficulty level, choose from several social scenarios, and customize the types of images used in the activities. For example, EmojiCapcha's Familiar Faces function allows for the emojis to be replaced with photos of family and friends.

Educators can also collect data such as scores and response time for users to display emotions, as measures of personal improvement.

Prof Ong said that although the apps are currently available to students at APSN, there are plans to work with other organizations to release them to a larger audience, including adults and the elderly.

Continuing with the theme of collaboration, NTU and YOOZOO are also announcing the new Data Science and AI (DSAI) Gold Medal, which is sponsored by YOOZOO to recognize the academic excellence of the top student from Bachelor of Science in DSAI. The award will be effective from 2021.

Elaborating on the award, Dr. Goh added, "We believe it is important to have top talent joining the industry to drive us forward. With the joint effort of universities and the business sector, we look forward to empowering better lives with AI."