About Us

Objectives

The Multimodal Analysis Lab research team consists of social scientists and computer scientists working together to develop prototype software for modelling, analyzing, storing and retrieving meaning from images, video texts and interactive digital sites constructed through the use of multiple semiotic resources (e.g. language, visual imagery, gesture, movement, music, sound, three-dimensional objects and so forth). 

The specific objectives of the Multimodal Analysis Lab research programme are: 

• To develop interactive digital media (IDM) technology for multimodal analysis of images, video texts and interactive digital sites.

• To develop IDM technology for the analysis of print and digital learning materials and classroom activities

• To investigate the integrative dynamics of complex meaning-making practices. 

• To examine the relative affordances and simultaneous constraints of different forms of technology. 

The research will advance humanities research through the development of specially designed tools and digital platforms to dynamically model and analyse meaning in multimodal discourse.  In addition, the research will advance existing computer science research in multimedia analysis which tends to depend on low-level feature information within specific domains of activity.

The interdisciplinary research programme in the Multimodal Analysis Lab has downstream R&D applications for industry, business and government because searching, managing and retrieving information from images, video texts and interactive digital sites is a significant problem with the rapid advance of digital technology. In addition, the research programme will advance understanding of the relations between technology and patterns of thinking. 

For further information, please download Multimodal Analysis Lab.