NUS Ascent Feb 2008

Multimodal Analysis Lab, Interactive & Digital Media Institute

(Download original
ascent article here


The proliferation of visual images and video texts taking place
through the rapid advance of digital technology is a major problem
for government, industry and business. Efficient practices for managing
data in text-based documents have been developed. However, visual
images and video texts remain elusive with regards to management,
coding and searching for information. The problem intensifies, as
videosharing websites become the standard forum for exchanging information.
Dynamic digital texts clearly require theoretical concepts and analytical
approaches which extend beyond those developed for text-based data.
Therefore, social scientists and computer scientists are working
together in the newly established Multimodal Analysis Lab in the
Interactive & Digital Media Institute (IDMI) to develop new approaches
to multimodal and multimedia data analysis, storage and retrieval.
IDMI is a university-level research institute for the development
of cutting-edge digital media technologies.


Social scientists and computer scientists do not usually work together
to solve problems. However, research collaboration between the two
groups in the Multimodal Analysis Lab, each sharing and expanding
their own knowledge base and expertise, will advance existing approaches
to multimodal and multimedia analysis. The specific objectives of
the research programme in the lab are:

  1. to develop prototype interactive digital media
  2. technology for multimodal analysis of
    1. images, video texts and interactive digital sites, and
    2. print and digital learning materials and classroom activities;
  3. to investigate the integrative dynamics of complex meaning-making
    practices; and
  4. to examine the relative affordances and constraints of different
    forms of technology for knowledge creation and the enactment of
    social relations.

The research programme will advance arts and social science research
through the development of specially designed tools and digital platforms
to dynamically model and analyze multimodal discourse and events.
In addition, the research will advance existing computer science
approaches to multimedia analysis which tend to depend on low-level
feature information within specific domains of activity. Finally,
research in the Multimodal Analysis Lab will advance our understanding
of the relations between technology and ways of thinking and being.
The Multimodal Analysis Lab is currently housed at 9 Prince George’s
Park, National University of Singapore. The official launch of the
lab will take place in January 2008.


For further information, please contact Multimodal Analysis Lab
Director, Associate Professor Kay O’Halloran (email:; and visit
the Multimodal Analysis Lab website: