A team of researchers in the U.S. has developed an artificial intelligence system that can detect the tone of conversations and analyze speech. This portable application could be useful for people with anxiety disorders or Asperger's.
A single conversation can be interpreted in a variety of ways. A new AI system emulates the human ability to identify emotions and the tone of speech to establish whether conversations are sad, happy or neutral. This brings scientists closer to developing a potential solution.
The AI system was developed by a team of researchers at the Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Institute of Medical Engineering and Sciences (IMES) at Massachusetts Institute of Technology in Boston.
In a news release, published Feb.1, the researchers explain the main functions of the system, which takes the form of an application that can be loaded into a smartwatch or wearable device.
The scientists tested the system on participants wearing a Samsung Simband wearable. This device is capable of capturing high-resolution psychological waveforms to measure physical parameters such as changes in body temperature, blood pressure and heart rate, as well as movements of the arms and legs. The system also captured audio data and text transcriptions to analyze the speaker's tone, pitch, energy and vocabulary.
The team then captured 31 conversations, each a few minutes long, and tested two algorithms. The first established the overall nature of the conversation (happy or sad), while the second classified each five-second snippet of every conversation as positive, negative or neutral. For example, the algorithm associated long silences and monotone tones with sadder themes, while more energetic and varied speech patterns were classified as happier conversations.
The system analyzed audio signals, physiological data and text transcriptions to determine the general tone of speech or conversation with 83 per cent accuracy.
"As far as we know, this is the first experiment that collects both physical data and speech data in a passive but robust way, even while subjects are having natural, unstructured interactions. Our results show that it's possible to classify the emotional tone of conversations in real-time," explains Mohammad Ghassemi, co-author of the study.
This new technology could serve as a "social coach" for people with anxiety or Asperger syndrome, a milder Autism Spectrum Disorder characterized by difficulties in communication and social interaction.
The researchers' findings will be presented at the Association for the Advancement of Artificial Intelligence (AAAI) conference in San Francisco next week.
