ACDH-CH Lecture 7.3

John Bateman
Universität Bremen

The relevance of multimodality theory for the future of the digital humanities

The Digital Humanities (DH) are increasingly engaging with artefacts and performances that go well beyond much of the field's origin in written texts. Questions concerning how to deal with this extension of interest are legion. However, many of the theories and practices common in the field are still based on models developed for text in a rather traditional sense. This raises crucial issues with respect to materiality, to dynamic media, to combinations of media and, for all of these, to the combined workings of theory, models, scale and visualisation. In this talk I set out the answers to these questions that have been developed over the past 10 years in the until now quite separate area of multimodality theory, particularly the applied semiotic account of multimodality introduced in Bateman/Wildfeuer/Hiippala (2017). In this framework, we rework many fundamental semiotic principles without the assumption that our primary object of concern is language, rebuilding from the ground up a general treatment of how materials (including 'digital' materials) can be made to carry meanings that signify for their communities of users.

A particular concern of this work has been achieving an anchoring in empirical studies: this involves establishing a systematic relationship between theory and descriptions at scale. Particular ways of organising collections of rich multisemiotic artefacts and performances are made to follow directly, and in step with, theory.

The talk will draw on examples from several media to show how this can then guide the practice of everyday studies of complex multimodal meaning-making.

The lecture will take place as part of the University of Vienna lecture series Introduction to Digital Humanities.