Digital Storytelling and Multimodal Scholarship

When scholarship moves to the screen, it can combine text, image, sound, data, and interactivity. Multimodal scholarship and digital storytelling explore how arguments and narratives can be made in these mixed media, and how academic publishing must change to accommodate them.

Definition

The creation and study of scholarly and narrative works that combine multiple communicative modes in digital media, together with the changes such forms demand of academic publishing and evaluation.

Scope

Covers scholarly and narrative forms that combine multiple modes — text, image, audio, video, interactivity: multimodal scholarship, digital storytelling, and the transformation of academic publishing and peer review for networked, multimedia work. Includes the theory of multimodality and debates over how new forms are evaluated and credited.

Core questions

• How can argument and narrative be made across text, image, sound, and interaction?
• What does multimodality theory tell us about meaning-making?
• How must publishing and peer review change for multimodal work?
• How should non-textual scholarship be evaluated and credited?

Key concepts

• Multimodality
• Digital storytelling
• Scholarly communication
• Open peer review
• Mode and affordance

Key theories

Multimodality

Kress argued that meaning is made through many modes beyond language — image, layout, sound — each with its own affordances, a foundation for analyzing multimodal scholarship.

Generative, multimodal digital humanities

Burdick and colleagues described the digital humanities as producing new, design-rich, multimodal forms of scholarly knowledge beyond the printed monograph.

Reforming scholarly publishing

Fitzpatrick argued that academic publishing and peer review must adapt to networked, multimodal scholarship and more open, iterative models of review.

History

Multimodality theory, developed by Kress and others in social semiotics, gave tools for analyzing mixed-media meaning. In the digital humanities, Fitzpatrick's Planned Obsolescence (2011) and Burdick et al.'s Digital_Humanities (2012) argued for new multimodal scholarly forms and for reforming the publishing and review systems around them.

Debates

Recognition of multimodal scholarship

Whether and how multimodal and interactive scholarly works can be peer-reviewed, preserved, and credited within systems built for printed text.

Key figures

• Gunther Kress
• Kathleen Fitzpatrick
• Anne Burdick
• Johanna Drucker

Seminal works

• kress2010
• burdick2012
• fitzpatrick2011

Frequently asked questions

Why not just publish digital scholarship as a normal article or book?

Some scholarship depends on interactivity, media, or data that print cannot hold. Multimodal forms preserve those qualities, but they strain conventional publishing, peer review, and preservation, which is why reform of scholarly communication is part of this topic.