An ESRC National Centre for Research Methods Node:

Multimodal Methodologies for Researching Digital Data and Environments† (£1.34 Million)

Directed by Professor Carey Jewitt and Dr Jeff Bezemer

with Dr Sara Price, Professor Gunther Kress, Dr Andrew Burn, Professor Roger Kneebone, Dr Diane Mavers, Dr Neil Selwyn, Dr Niall Winters, Dr Caroline Pelletier

Digital technologies are profoundly changing society. These changes create significant challenges and opportunities for social scientists. An education researcher, for instance, now asks: How can I investigate the interactive whiteboards and visualizers used in schools to facilitate learning? A sociologist may ask: How do digital technologies change the way people organise their work? They may both ask: How can we make use of the digital artefacts that the people we study produce such as websites, games, mobile devices, virtual environments, 'touch based' technologies, and videos, as well as the digital data automatically produced by technologies, such as CCTV video recordings? Multimodal methodologies offer ways to address these urgent and timely questions by attending to what people say, write, draw, design, and look at, how they lay out and navigate through rooms, websites and other spaces, how they use their hands and other parts of their bodies to interact with computers, devices and other people in face-to-face encounters. In short: multimodal methodologies allow social scientists to study how people behave and interact in contemporary digital environments.

The Multimodal Methodologies Node will further develop multimodal methodologies for social scientists, providing systematic ways to investigate all modes of communication used in digital environments, whether they are sites of learning, work, or 'social' sites (e.g. Facebook). The work of the Node is focused on five themes:

- Capturing and analyzing digital data: How to gather materials such as video-recordings and logs of people's online presence and how to systematically analyze these materials using multimodal methodologies?

- Dissemination: How to present materials such as video and analysis of the video to social scientists when writing up research findings in a journal article?

- Mixed methods: How to reconcile different perspectives and different sources of evidence on materials (for instance a video recording of someone browsing the internet and a log capturing mouse movements)?

- Time and space: How do you make sense of the activities that people engage in if they take place in an unfixed time and space frame? (For instance, responses to a video posted on YouTube)

- Body: How do we investigate people's physical or virtual co-presence in digital envirionments? For instance, how dowe make sense of the use of 'avatars' in virtual worlds which extend the sense of a person's body.†

The objectives of the Node are to:

- establish a strategic focal point for the development, delivery and dissemination of† multimodal methodologies, training and capacity building;

- set up and carry out research in digital environments to try out and develop new multimodal methodologies

- provide a coherent program of training and capacity building activities in multimodal methodologies for social science researchers;†

- build a social science research community that enhances the UK's profile and leading position in multimodal methodologies and digital technologies.

The Node will deliver a coherent program of training and capacity building activities and research projects.

These include seminars, lectures, introductory courses and summer schools, and online discussion and support for early and mid career social scientists from a range of academic disciplines, government, private sector organisations and the public sector. The two research projects provide a testing ground for new multimodal methodologies and focus on how people use new technologies in different digital environments. One research project is located in an operating theatre, where screen technologies are used to look inside people's body cavities, and another one in a classroom, where Geographic Information System technologies and touch screen interactive tables are used to facilitate learning.