Technology
Facial Expression Recognition
In human–robot interaction automatic detection of facial expressions can make the interaction between humans and robots more natural and user-friendly. Our ultimate goal is to enable robots to automatically recognize action units which describe muscle-based facial actions and emotions like happiness, sadness, anger in-the-wild where people are allowed to move freely around the robot.
End-to-end Social Reinforcement Learning
Deep reinforcement learning have made robots to learn how to perform tasks from raw sensory input possible by learning representation autonomously. Currently, we are developing deep interactive reinforcement learning method to facilitate Haru to learn state representation and how to behave at the same time from evaluative feedback provided by a human user. The resulting learning algorithm can be applied to to learn Haru’s empathic response from human multimodal data during interaction.
Performance Prediction and Profiling
A robot with the capacity to predict and profile the task performance of a human is useful in a large number of industrial and domestic applications. The relevant implemented toolbox is termed P3M, which stands for Performance Prediction and Profiling Methodology. Haru, when equipped with P3M, is inherently able to: (i) track the performance and behavioural skills of a human, by observation, while performing a task, and (ii) use non-verbal, gestural and visual cues to indicate how one’s performance can be improved.
Generative Time Models
The ability to predict the duration of an activity provides valuable information for Human-Robot Interaction. GTMs (Generative Time Models) consist computational models that accomplish the above. GTM, can predict the duration and other temporal properties of an activity by observation and enables the robot to inform its collaborators on how to perform a task more efficiently and to better respond during turn-taking in a collaborative task settings.
Bio-Inspired Robotics for Enhanced Social Interaction
Design and development of biologically-inspired applications via soft robotics technologies for reproduction of life-like motion performances. Ultimate goal is the future use of innovative structure and actuation equipment for improving the expressivity and motion performance of socially interactive robotic platforms.
Automatic Prose Generation with Language Models
In order for social robots to successfully communicate with humans, they need to generate appropriate responses. This requires understanding the context of a conversation and incorporating it into prose generation. We are working to construct a language model that generates content-aware prose by predicting what words comes next given the previously generated words as input by using deep neural networks to learn from billions of words of Web text. The resulting language model will be applied in a variety of Haru’s functions, including small talk, interactive storytelling, and expressive humor delivery.
A Cognitive Architecture for Social Interaction
Most, if not all, human cognitive mechanisms play a role in the social interaction with other agents. By building a cognitive architecture that is fundamentally social, we also build on the knowledge and expertise in cognitive science: How do humans perceive social agents? What factors facilitate interaction? Not only does this lead to a robot that it is fundamentally social in its behaviors, it might also help us learn a little more about ourselves.
