A Multimodal Scrapbook About Multimodal Literacy

Author: Various

Citations in the document

Use these multimodal resources to inspire comments and replies about your own multimodal learning experiences and how you might design multimodal learning and composing for your students.

2

Paragraph 2 changes

With all due respect the the resources we have collected here, a careful reading of each of them is not our intention. Instead, we want you to look quickly on the left side of NowComment’s 2-panel pane set-up and find a few things that resonate with your own learning and your plans for students. Once identified, go to the right side and, in both comments and replies, describe how something you created in a recent workshop or seminar is an example of multimodal literacy and learning. Then begin to think with your peers about how this might be applied in your classrooms this year.

With all due respect the the resources we have collected here, a careful reading of each of them is not our intention. Instead, we want you to look quickly on the left side of NowComment’s 2-pane set-up and find a few things that resonate with your own learning and your plans for students. Once identified, go to the right side and, in both comments and replies, describe how something you created in a recent workshop or seminar is an example of multimodal literacy and learning. Then begin to think with your peers about how this might be applied in your classrooms this year.

1. Teachernerdz. “Angela Stockman about Multimodality in Writing.” Twitter, Twitter, 17 July 2021, https://twitter.com/teachernerdz/status/1416364810325962752?s=20&t=DGpklSugmbdQpAvp4xV85Q.

2. Coppola, Shawna. “Writing, Redefined in a Nutshell, a Comic Strip by Shawna Coppola.” Stenhouse Blog, 26 Aug. 2019, https://blog.stenhouse.com/writing-redefined-in-a-nutshell.

3. Babak, Larissa. “A Pedagogy of Multiliteracies: Designing Social Futures.” YouTube, 6 Apr. 2015, https://youtu.be/lMOMf1uNdYk.

4. Ryan, Cathy. “Retelling Stories by Creating a Multimodal Text: Cummins Area School.” YouTube, Teaching and Learning in South Australia, 16 Sept. 2013, https://youtu.be/3yMT3pCEQIs.

5. Serafini, Frank. “Multimodal Literacy.” Faculti – Multimodal Literacy, 2 Dec. 2020, https://faculti.net/multimodal-literacy/.

6. The Why, What, and How of Multimodal Literacies
Dr. Frank Serafini, Professor – Arizona State University
University of Wyoming Literacy Conference 2018

www.frankserafini.com

Foundational Questions

Why Should We Teach Multimodal Literacies?
What do We Teach?
How do We Teach It?

Multimodal(ity): A Brief Definition

• as an adjective to describe a particular type of text
• as a theoretical perspective
• as a framework for analyzing texts, events, & social practices

Multimodal Ensembles

• Texts that operate across multiple modes: (image, language, music, animation, etc.)
• Mode = a socially-shaped, material means for making meaning
• Some modes are temporal (language, music) and some are spatial (image, sculpture) and some are both (picturebooks)

Why Should We Teach Multimodal Literacies?

• The world told has become the world shown – Gunther Kress
• A Shift from Page To Screen
• From Monomodal to Multimodal Texts
• From Static to Interactive Texts

Assertion #1

The texts children interact with today have become more visual, digital, and multimodal.

Analogue-Based Multimodal Texts
Digitally-Based Multimodal Texts
Transmedial Texts

Assertion #2

Because of these shifts, we have to redefine what we mean by “text” & what counts as reading.

Redefining Texts
Text as Written Language
Text as Visual Object
Text as Multimodal Ensemble
Text as Cultural Artifact

The Case for Multimodal Literacy

• Since the texts readers read have become radically different, the strategies readers will need are radically different.
• We need to make a shift from supporting “readers” to supporting “reader-viewers”
• Understanding how texts work is as important as deciding what texts mean.

What Do We Teach?: A Multimodal Curriculum
Re-Envisioning Reader Competencies
Re-Thinking Analytical Perspectives

Assertion #3

We have to help students develop new competencies for designing, producing, navigating, and
interpreting multimodal texts.

Developing Multimodal Competencies

• Call attention to various aspects of multimodal texts that often go unnoticed.
• Teach reading-viewing strategies in the context of making sense of the texts students encounter each day in and out of school.
• Help students understand how different modes work individually and collectively across multimodal texts.

Learning Processes (van Leeuwen )

• Understanding Texts
• Engaging Personally with Texts
• Connecting to Texts
• Engaging Critically with Texts
• Experimenting with Texts
• Reflecting / Reconsidering

Assertion #4

Students need a variety of analytical perspectives from which to consider and interpret
multimodal texts.

Analytical Perspectives

• Help readers approach visual and multimodal texts in different ways.
• Serve as interpretive frameworks to help readers understand what to attend to.
• Provide a vocabulary or metalanguage for discussing various elements and features ofvisual and multimodal texts.

3 Analytical Perspectives

• Perceptual – Noticing, Navigating, Naming Elements of Multimodal Texts
• Structural – Grammar and Conventions of Visual Images & Multimodal Texts
• Ideological – The Socio-Cultural Contexts of Visual Images and Multimodal Texts

Additional Theoretical Perspectives

• Art History and Criticism
• Media and Cultural Studies
• Systemic Functional Linguistics
• Visual Grammar
• Picturebook Theories
• Visual Discourse Analysis
• Multimodal Analysis
• Multimodal Interaction Analysis
• Semiotics and Social Semiotics

Assertion #5

Multimodal pedagogy should be an optimal blend of explicit instruction and student
exploration and experimentation.

Pedagogical “Points of Entry”

3 Points of Entry

• Art Movements: Various techniques and styles from the history of art criticism
• Visual Grammar: A grammar for images based on systemic functional linguistics
• Design Elements: Elements used as spatial and linguistic means for organizing texts

Art Movements

• Realism
• Folk Art
• Modern Art
• Surrealism

Questions to Ask

• How does the artistic style add to the theme, mood, or setting of the story?
• What aspects of the artistic movement can be used to better understand the narrative?
• How would your interpretations of the book change if the art style changed?

Visual Grammar

• Developed by Kress and van Leeuwen (1996)
• Based on Halliday’s (1977) framework of systemic functional linguistics
• Describes grammars and conventions of visual images based on the framework of grammars and conventions of language

Point of View (Positioning)

How viewers of an image are positioned.
Has socially determined meaning potentials.
Provides viewer with their “window on the world.”
May suggest certain power relationships.

Questions to Ask

• Where are the characters located in the image?
• Are objects or people above, below or straight on?
• Where are you positioned as the viewer?
• How does your position as viewer affect your interpretations?

Viewer Interactions

• How is the viewer being asked to interact with the characters in the image?
• Demand – requires active response to characters in an image.
• Offer – placed in voyeuristic position, viewers look at what characters are looking at.

Questions to Ask

• Are the characters looking at you or away from you?
• How does this affect your relationship with the characters?
• What are you being asked to consider?
• How would your interpretation of the image change if the characters looked at you ornot?

Visual Design Features

• Speech / Thought Bubbles
• Upfixes
• Impact Stars
• Motion Lines
• Reduplication

Questions to Ask

• What visual design elements did you notice?
• How did these elements help you make sense of the picturebook or other text?
• What functions do these elements serve? What do they do for the text and for us asreaders?

Endpapers: Different Roles of Endpapers

• Connections to Story
• Thematic
• Part of the Narrative

Questions to Ask

• Are the endpapers the same in front and back?
• If they are the same or single colored pages, is there a thematic connection to thenarrative?
• If they are different, how do the endpapers add to the narrative?

Some Final Thoughts

• As texts continue to evolve into digital and multimodal ensembles, the strategies readers will need to be successful will have to evolve as well.
• Teachers need to expand their own interpretive competencies if they expect to support their students interpretive competencies.
• Texts are always encountered within particular social practices and contexts, and these practices and contexts influence how we understand them.

Professional Resources
Reading the Visual – Serafini
Reading Contemporary Picture Books – Lewis
Looking at Pictures in Picture Books – Doonan
Postmodern Picturebooks – Sipe and Pantaleo
How Picturebooks Work – Nikolajeva and Scott
Reading Images – Kress & van Leeuwen
Picture This – Bang
Literacy in the New Media Age – Kress
Practices of Looking – Sturken and Cartwright
Handbook of Visual Analysis – van Leeuwen & Jewitt
Visual Methodologies – Rose

Thank You!
www.frankserafini.com

7. O’Brien, Annemaree. “Creating Multimodal Texts.” Creating Multimodal Texts, https://creatingmultimodaltexts.com/.

creating multimodal texts

resources for literacy teachers

Introduction and how to use this resource

Storytelling in any format is about making meaning. While the essence of the stories we tell may remain the same, the ways in which we can now share these stories have changed dramatically with the development of digital communication technologies. Access to simple, easy to use media production tools and resources in conjunction with the potential for immediate and universal online publication has significant implications for literacy thinking and practice.

This website is an evolving compilation of resources designed to support the development of student multimodal authoring at all year levels. It features examples of different types of student multimodal compositions to demonstrate the rich variety of choices available, along with practical resources to support text production.

What does creating multimodal texts mean?

Creating is defined in the Australian Curriculum as ‘the development and/or production of spoken, written or multimodal texts in print or digital forms’ and is an embedded literacy expectation across all disciplines.

Multimodal is defined in the Australian Curriculum as the strategic use of ‘two or more communication modes‘ to make meaning, for example, image, gesture, music, spoken language, and written language.

What is a multimodal text?

While the development of multimodal literacy is strongly associated with the growth of digital communication technologies, multimodal is not synonymous with digital. The choice of media for multimodal text creation is therefore always an important consideration.

A multimodal text can be paper – such as books, comics, posters.

A multimodal text can be digital – from slide presentations, e-books, blogs, e-posters, web pages, and social media, through to animation, film and video games.

A multimodal text can be live – a performance or an event.

And, a multimodal text can be transmedia– where the story is told using ‘multiple delivery channels’ through a combination of media platforms, for example, book, comic, magazine, film, web series, and video game mediums all working as part of the same story. Transmedia is a contested term and Henry Jenkins is worth reading for more background. Jenkins argues that transmedia is more than just multiple media platforms, it is about the logical relations between these media extensions which seek to add something to the story as it moves from one medium to another, not just adaptation or retelling. Transmedia enables the further development of the story world through each new medium; for example offering a back story, a prequel, additional ‘episodes’, or further insight into characters and plot elements. (Jenkins, 2011). It also can require a more complex production process.

Development of multimodal literacy knowledge and skills

To enable our students to effectively design and communicate meaning through such rich and potentially complex texts, we need to extend their (and along the way, our own) multimodal literacy knowledge and skills. Skilled multimodal composition requires new literacy design skills and knowledge to enable students to make informed choices within and across the available communication modes to effectively construct meaning.

REPORT THIS AD

Creating a multimodal text, a digital animation, for example, is a complex meaning design process requiring the strategic orchestration of a combination of modes such as image, movement, sound, spatial design, gesture, and language. The process of constructing such texts is also truly a cross-disciplinary literacy process, drawing on digital information technologies and The Arts (media, music, drama, visual arts, design) to bring meaning to life.

About this website

The multimodal text examples here describe different media possibilities – both digital and on paper and provide links to examples of student work and production guides.

Print-based multimodal texts include comics, picture storybooks, graphic novels; and posters, newspapers and brochures.

Digital multimodal texts include slide presentations, animation, book trailers, digital storytelling, live-action filmmaking, music videos, ‘born digital’ storytelling, and various web texts and social media. The level of digital technology requirements range from very simple options such as slide presentations through to complex, sophisticated forms requiring a higher level of technical and digital media skills. The choice is yours depending on your skill and experience, level of confidence, and the resources and tools available to you.

These examples of different types of student multimodal composition are provided as ideas and starting points, and may also provide models for introducing new forms of ‘writing’ to your students.

The technical construction of digital multimodal texts is always a significant consideration for teachers. Practical information about communication technology resources and digital media tools is provided where possible to support the successful implementation of multimodal authoring in the literacy classroom.

Please explore these resources as a starting point to develop ideas to suit your own situation.

How this website works

Practical

Production processes is an overview of the three production stages in creating a multimodal text.

Media resources provides links to a wealth of digital audio and image resources which can be used under creative commons licensing.

Copyright and Attribution provides information about how to ethically source and use digital materials responsibly.

Pedagogy

Modes and meaning systems explores the key meaning-making systems we can use to create meaning

Pedagogy provides a brief guide to teaching creating multimodal texts.

Visual literacy provides a closer look at one of the key meaning-making modes, with the aim to develop a shared metalanguage for talking about how visual meaning is constructed.

Reading multimodal texts provides resources for deconstructing and analysing how different modal systems work to create meaning in a text.

DMU Timestamp: July 18, 2022 19:48