Ahead of International Autism Day on April 4, Cox wanted to empower those on the autism spectrum to develop more meaningful connections online. So we created a tool that enabled them to have better, deeper conversations with their loved ones.
Project Convey is a video chat prototype designed to help people on the autism spectrum better identify non-verbal social cues. It recognizes an emotion on someone’s face and translates it into an emoji, allowing someone with autism to better understand the emotions and nuances of the conversation.
Our team used AI cloud services like IBM Watson to analyze both facial and verbal expressions in real-time. Using pitch analysis and speech-to-text allowed us to interpret someone’s verbal expression sentiment and the emotion behind their words. And interpreting facial expressions, particularly facial micro-expressions, helped capture the sentiment behind what they were saying.
With the release of the prototype coinciding with International Autism Day, the response has been overwhelmingly positive. It’s amazing to be able to use technology to make real connections and bring people closer together, and we loved seeing the joy it brought to those with autism of all ages.