Multimodality in Social Interaction

We investigate how people create meaning and understanding through social interaction. We focus on the multimodal and multisensory resources used in interaction as situated and sociomaterial practice.

This program has a broad interest in all kinds of situations where social interaction occurs - from private life to institutional contexts. We are interested in all kinds of social phenomena such as how identity is produced, how understanding is achieved, how knowledge is displayed, how perception is accomplished, etc. We investigate phenomena through in-depth ethnographic methods, and by looking at how the phenomena gets accomplished in and through interaction.

We provide basic scientific knowledge about how human interaction and sociality in natural situations arise and develop sequentially, where not only language but also bodies, things, space and the material surroundings are central for accomplishing activities.

We are grounded in interactionist theories that goes back to Garfinkel (1967) and Goffman (1983) and which further developed in ethnomethodology and conversation analysis (Sacks et al., 1974). We are particularly interested in how interaction is bodily (Streeck 2009), multimodal (Goodwin 2017), multisensorial (Mondada


Representation of data in publications

The main method is video ethnography where we record situations, process data in collections and transcribe not only speech but also bodily actions. We analyze transcribed examples in publications, from which we develop the scientific findings. In connection with ethnographic work, we also use observations and interviews that are typically included as background understanding.


We examine how not just a single form of communication such as linguistic structure is used to create meaning and understanding, but how it always takes place in a context of other modalities such as intonation and the body's many forms of communicative actions, which together create wholes of meaning (gestalts).

Senses and sensory loss

We examine how the senses are given meaning in social interaction. We look at how vision, hearing, touch, taste, smell and the proprioceptive (balance) sense are made relevant in social interaction. We also investigate how people with sensory loss such as blindness interact in social contexts and typically also how technology is included.


We are interested in developing a new understanding of how perception is not only understandable as a mental or internal bodily phenomenon, but instead can be seen as linked to practical circumstances and be a socially observable phenomenon.

Nonhuman actors

We investigate not only how people create sociality together, but also how sociality is created with other nonhuman actors. We have particularly focused on sociality between humans and dogs, but other animals such as cats and horses can also be examined in this research program.

Sociomateriality and new technology

We investigate how materiality in the form of the built environment, office landscapes, furnishings in houses and in urban environments, objects and technologies always to some extent form part of and influence social interaction. All sorts of media, screens, technologies are examined as forming part of assemblages.

Distributed agency

We also examine how agency is distributed between people and sociomateriality. We focus on the production of assemblages that, as units consisting of a large number of elements, enable the accomplishment of actions and activities.


There is always identity at stake when people meet, and we investigate how identity is created on a micro level through multimodal actions. We do not assume that any specific identity is at play (e.g. being a leader), but investigate which actions and orientations from others co-construct identities.


We examine how trust between people is created through social actions. We focus on how trust as an interactional phenomenon is achieved, created, maintained or lost in conversations between employees and residents of (re)socializing institutions, which indicators of trust or distrust can be identified and converted into educational tools for practitioners.






Garfinkel, H. (1967). Studies in Ethnomethodology. Prentice Hall.

Goffman, E. (1983). The Interaction Order: American Sociological Association, 1982 Presidential Address. American Sociological Review, 48(1), 1–17.

Goodwin, C. (2017). Co-Operative Action. Cambridge University Press.

Laurier, E. (2013). Before, in and after: Cars making their way through roundabouts. In Interaction and Mobility Language and the Body in Motion. De Gruyter. 

Mondada, L. (2021). Sensing in Social Interaction: The Taste for Cheese in Gourmet Shops. Cambridge University Press. 

Sacks, H. L., Schegloff, E. A., & Jefferson, G. (1974). A Simplest Systematics for the Organization of Turn-Taking for Conversation. Language, 50(4), 696–735.

Streeck, J. (2009). Gesturecraft: The manu-facture of meaning. John Benjamins Pub.




Name Title Phone E-mail
Due, Brian Lystgaard Associate Professor - Promotion Programme +4535335929 E-mail
Nielsen, Ann Merrit Rikke Assistant Professor - Tenure Track +4535337181 E-mail

Affiliated researchers 

  • Morris, David 
  • Nielsen, Mie Femø 
  • Pedersen, Bolette Sandford
  • Stæhr, Andreas Candefors 
  • Toft, Thomas L.w.

External researchers

  • Munk, Timme B. 

Head of research group