Real-Time Social Feedback using Non-Verbal Audio Visual Cues

Technology Overview

When human beings communicate, their non-verbal behavioral cues contain a lot of information about the social state they are in. This system can extract these non-verbal cues and infer human behavior while the conversation is going on. The objective is to provide useful feedback that can help people rectify any anomaly in their communication. Different platforms like smartphone, Skype and smart glasses will be used to provide feedback. The system can be integrated with a humanoid robot to make sure that the robot is aware of the social state of the people involved in communication. Using this scheme, human robot interaction can be enhanced.

Technology Features & Specifications

Our system uses lapel microphones for audio data acquisition and Microsoft Kinect sensor to acquire video and depth data. Acquired data is processed to extract non verbal audio visual cues such as speaking percentage, interruptions, pitch, posture and hand gesture usage etc. The social state is inferred using machine learning classifiers, which are trained in a supervised manner using carefully annotated data sets. To provide feedback we use different platforms such as smartphone, Skype and smart glasses. A humanoid robot is also used for experiments aimed at enhancing human robot interaction using this system.

Potential Applications

There are two main applications for this system:- Communication training Call center training Sales person training Interview training Personalized feedback for self-improvement Social robotics To enhance human robot communication Enabling robots to mediate meetings/conversations

Market Trends and Opportunities

Companies are always on the lookout for systems that can help improve their employees’ communication skills. Systems like ours can help companies train their employees to be better communicators. The avenue of robotics in service industry has recently opened up; system like this can help enhance human robot interaction. 

Customer Benefits

Our system can help companies improve the communication skills of their employees by training them on certain communication aspects that they require to perform better. It can also help save costs if companies use an automated system for employee training. If robots become part of our daily lives, human behavior awareness will enable the robots to behave more naturally.

Make an Enquiry