Dr Zou Di Received a Silver Medal in the International Innovation and Invention Competition (IIIC) Taiwan 2020

Dr Zou Di from the Department of English Language Education received a silver medal in the International Innovation and Invention Competition (IIIC) Taiwan 2020. IIIC is a renowned event of innovations in the Greater China Region, attracting over 400 innovations from all over the world including Asia, the United States and European countries, every year.

Dr Zou’s awarded project is about an emotion recognition system based on the deep neural network. It can recognise students’ emotions during their learning by adopting a novel multi-branch deep convolutional neural network that integrates multi-modal data sources. Specifically, the multi-modal data sources include electroencephalogram, face, and eye data collected by a physiological data collector, an eye-tracker, and a motion sensor, respectively. Furthermore, the proposed neural network model not only enables automatic and effective feature extraction but also provides flexibility to be extended to various emotions. From the pedagogical perspective, the identified emotions can be explored for evaluating mental status, measuring learning engagement, and providing personalised scaffolding.

Its features include:

• High accuracy of emotion recognition achieved by a novel multi-branch deep convolutional neural network;
• Reliable emotion recognition via the integration of multi-modal data sources, including EEG, face, and eye data;
• Automatic and effective feature extraction from multi-modal data sources facilitated by the end-to-end structure; and
• Flexibility for identifying various emotions by extending the circumplex model of emotions.