An Interconnected Framework for Assessment of Digital Multimodal Composition (English Education, July 2021)

Author: Ewa McGrail, Kristen Hawley Turner, Amy Piotrowski, Kathryn Caprino, Lauren Zucker, and Mary Ellen Greenwood

McGrail, Ewa, et al. “National Council of Teachers of English.” NCTE, English Education, July 2021, https://library.ncte.org/journals/ee/issues/v53-4/31483.

Drawing from the Beliefs for Integrating Technology into the English Language Arts Classroom, as well as prior scholarship on digitally mediated communication, rhetorical studies and composition, assessment, and digital literacies, this theoretical article presents a framework for creating and assessing digital multimodal compositions. The Interconnected Framework for Assessment of Digital Multimodal Composition conceptualizes digital multimodal composing through three inter-connected and layered domains: audience, mode and meaning, and originality. Though the three domains are defined individually, they are inextricably linked within the recursive processes and products of digital multimodal composing to contribute to intended meaning. The authors describe and justify the domains, present assessment considerations, and conclude with implications for practice and suggestions for designing assessments relevant to context and task.

Over the last few decades, the field of English teacher education has embraced the idea that literacy involves the social practices and abilities that enable readers and writers to understand multiple ways of representing, communicating, and interpreting texts and ideas from different contexts and in different modes (Swenson et al., 2006). As stated in the Beliefs for Integrating Technology into the English Language Arts Classroom (Bartels et al., 2019), “Literacy means literacies” (p. 307) in today’s world. As such, English teachers are charged to “model classroom use for engaging and critiquing texts” that make use of “digital and networked technologies” and to “design assignments, activities and assessments” (Bartels et al., 2019, p. 308). Yet, both English teachers and the educators who instruct them struggle with how to assess digital multimodal creations. While there is much research on multimodal practices in teacher education and language arts classrooms (Jocius, 2020; Lammers & Van Alstyne, 2019; Smith, 2017), as well as some guidance on assessment—in, for example, the Multimodal Assessment Project

(MAP) framework from the National Writing Project (NWP) (Eidman-Aadahl et al., 2013)—critiquing and assessing multimodal creations has received little attention from English education and literacy researchers. Furthermore, digital multimodal composing is not the same as print-based composing; there is a difference between a digital video essay and a traditional written or typed essay, so traditional assessment frameworks may not be helpful (Burke & Hammett, 2009; McGrail & Behizadeh, 2017).

As members of the ELATE Commission on Digital Literacy and Teacher Education who have researched multimodal and digital writing for close to two decades, and based on our own experiences with students who regularly produce multimodal compositions, we are interested in how teachers can move from print-based assessment frameworks to those that support writers in digital spaces. This support begins with understanding how to critique, assess, and provide feedback to digital writers, yet there is limited research in this area.

To address this gap, we propose an interconnected framework comprised of three domains. This framework leads to assessment considerations for each domain of multimodal composition. It provides an approach for English teachers and teacher educators, as well as their students, for critiquing and assessing multimodal creations in their own contexts. In exploring the framework in our own teaching as English teacher educators and English teachers, we use authentic student examples (with all students’ names used with permission) to articulate definitions of each domain and specific questions that might be asked by both writers and assessors of multimodal texts.

Dilemmas in Examining and Assessing Digital Multimodal Composing

Today’s technologies enable and encourage digital multimodal creation that includes images, audio, video, graphics, and alphabetic text to convey messages and to encourage interaction and collaboration with a communicative purpose and real audience (Anderson & Jiang, 2018; Hicks, 2013; Ito et al., 2009; Lammers & Van Alstyne, 2019; Weiser, Fehler, & González, 2009). Through the lenses of New Literacies and multimodality, we recognize that digital composing and its resulting creations share the following characteristics:

>They are multimodal (New London Group, 2000), blending and inhabiting different modalities (e.g., visual, oral, textual) and genres (e.g., poetry mashup, digital story), and they are disseminated in a variety of media (e.g., audio recording, social media post).

> They are outcomes of an authentic writing task with a communicative purpose (Duke et al., 2006; Lindblom, 2015; Wiggins, 2009).

> They incorporate some degree of interaction and/or collaboration with a real audience utilizing various means of communication (McGrail & Behizadeh, 2017).

> They invite diverse forms of creativity and challenge traditional notions of originality of multimodal texts (Law, 2020; McGrail & McGrail, 2010).

Examples of this kind of composition come from Lammers and Van Alstyne’s study (2019) of high school students learning to identify and write for their audiences through participation “as readers and writers on a variety of popular websites, such as Fanfiction.net, Wattpad, and Tumblr” (p. 653). Because much of the authentic composition was done anonymously or using screen names unknown to their teachers, the students’ compositions were not assessed. Though students earned credit for participating, the authors stated, “We did not evaluate their creative writing” (p. 656), which Silseth and Gilje (2019) would describe as a “decoupling of production practices and assessment practices” (p. 28). This example identifies the need for educators to consider both how to honor the norms and practices of composing in digital spaces and how to assess multimodal compositions.

Examples also come from the Community Inquiry and Mobile Learning (CIML) project (Rish et al., 2018), where adolescents participating in a community service program digitally mapped assets of interest to them (e.g., recreational activities) throughout their community. The creative process included data collection, mappings software, digital photos, walking tours, and conversations with researchers to represent the findings spatially, geographically, and multimodally. The authors did not address assessment, and it is not clear how teachers would evaluate such Story Maps with adolescents’ learning processes taking place along many “trajectories and physical, virtual, and educational mobilities that connected places” (Rish et al., 2018, p. 138).

These examples demonstrate the difficulty of assessing digital multi-modal compositions and the need for a framework to do so, since assessment allows for feedback that helps writers to grow. Although the word assessment is used three times in the Beliefs statement (Bartels et al., 2019), little guidance or import is placed on this aspect of integrating technology into the English language arts and literacy classroom. How, then, can teachers of multimodal writing begin to tackle the assessment issue?

Research on Teachers Negotiating Assessment of Multimodal Composition

Tan et al.’s (2020) literature review showed that teachers in higher education and K–12 contexts do not have sufficient knowledge in assessing multimodal composing, focusing instead on language or text even when they assess multimodal compositions. In an earlier study, Wierszewski (2013) observed that teachers’ evaluative comments in assessing multimodal creations overlapped with the comments typically made in traditional print essays. However, the researcher identified six criteria needed to assess student multimodal compositions: “creativity, grammar, idea development, movement, multimodality, and technical execution [italics in original]” (Discussion, para. 6). Wierszewski (2013) noted that, even when student multimodal compositions included at least two modalities, teachers were less attentive to this multimodality. Similarly, Tan et al. (2020) found that little attention had been given to “intermodal complementarity (how modes operate together to create meanings) (Painter & Martin, 2011) in the teachers’ assessment practices and frameworks [italics in original]” (p. 104).

In part, it seems that tackling the issue of assessment requires developing knowledge of how writers craft digital compositions. Hicks, Turner, and Stratton (2013) and Turner and Hicks (2017) articulated the importance of knowledge of technology and how it intersects with procedural and declarative knowledge of form and substance, yet the model presented does not address assessment criteria. In his 2015 book, Hicks provided an assessment protocol that prompts conversation about digital writing focused on the following questions: “What do you see/notice?; What is working in this piece/composition?; What does it make you wonder/what questions does it raise?” (p. 16). These questions may turn teachers’ eyes from more traditional assessment criteria to consider the unique features of digital compositions; however, a more specific framework for discussing digital multimodal texts may better support teachers in offering specific and effective feedback. Leaders from the National Writing Project (Eidman-Aadahl et al., 2013) outlined domains for multimodal writing assessment that included the “language of evaluation, the language of instruction” (para. 1): artifact, context, substance, process management and technique, and habits of mind. As with Hicks (2015), this work highlighted the importance of process in digital creation, as well as the need for conversation about that process in assessment. What is less clear, however, are the criteria for what makes a piece of digital writing effective.

What domains might support conversations that critique the artifact itself while still acknowledging the importance of process? How might domains engage multimodal writers and assessors in examining the relationships within and across domains to create meanings? Turning back to the Beliefs statement provides a guide to answering these questions. For example, the concepts of audience, multimodality, and repurposing are mentioned multiple times, suggesting their import for consideration in teaching and assessing digital multimodal compositions. Taken in conjunction with the literature, we can define the domains that support the development of specific criteria for assessment of digital multimodal writing: (1) audience, (2) mode and meaning, and (3) originality. We explore each domain in turn after first sharing how they work in an interconnected, and often messy, way.

Interconnected Framework

Audience, mode and meaning, and originality represent three domains of multimodal composition. While it is possible to define each and to articulate assessment considerations within each domain, these three domains work together in creating a holistic composition and meaning. The domains are both layered and interconnected rather than easily separated as discrete components. The framework presented here builds on earlier work (e.g., (Eidman-Aadahl et al., 2013; Wahleithner, 2014) to show that digital com-posing is often a messy act of communication where these three domains come together holistically.

Composing is recursive (Flower & Hayes, 1981) and multimodal composing is even more so (Fulwiler & Middleton, 2012; Jocius, 2020) as it “allows for sequential multimodal representation of thoughts and ideas” (Bruce, 2009, p. 443) to be continuously adjusted “to ensure that modes are working together” (Jocius, 2020, p. 152). This means that writers may not consider each domain equally or in linear and fixed form, choosing to privilege one domain depending on the context of the writing task. For example, on social media, audiences can be interactive, a key feature of the context that may make the domain of audience more prominent in the composition process. Yet, writers must consider mode and meaning, as well as originality, as they compose since all three domains contribute to the whole composition.

Figure 1 is a visual representation of the proposed framework. The overlapping blocks communicate the interconnectedness of the three domains while showing that digital multimodal composing is not linear. The incomplete circle indicates that there is movement in the act of digital multimodal composing, allowing for the flow of ideas within and across domains while bringing unity and wholeness to a composition.

Audience

Because of social networking and digitally mediated communication, the concept of audience has expanded to include multiple and diverse audiences (boyd, 2014). Fanfiction writers, remix creators, and users of social media platforms (e.g., Facebook) engage in conversation and collaboration online where meaning, interpretation, and ideas are “augmented, altered, and amended” (Hoffman, 2015, p. 34) in ways that help both writers and audience. Within such contexts, writers’ awareness of the audience is situated in a complex and multidirectional relationship with the audience. For example, the actual audience is not always clear on social network sites, as the audience writers imagine might not read or follow their posts (Lunsford & Ede, 2009). However, there might be “invisible audiences” (boyd, 2014, p. 31) and “unwanted audiences” (Lunsford & Ede, 2009, p. 55), that is, audiences who are not intended or recognizable, since diverse audiences with different interests and expectations inhabit the same social network environments (Weiser et al., 2009). This complex environment can lead to what Marwick and boyd (2011) referred to as the “context collapse” or the “flattening of [multiple] audience[s] . . . into single contexts” (p. 114).

In discussing the writer-reader relationship shift in academic writing, Long (1980) argued that writers create the audience they wish to have by asking themselves questions that aid their process of “inventing” the audience: “Who do I want my audience to be?” (p. 225) and “What attitudes, ideas, actions are to be encouraged?” rather than beginning with the traditional question of “Who is my audience?” (p. 226). While today’s writers ask similar questions to create the audiences they desire for their digital multimodal compositions, they use a wider range of writing tools and modalities (e.g., images, audio, graphics) than the writers of the past. They also attract larger, more diverse, and more fluid audiences for their writing (see boyd, 2014; Marwick & boyd, 2011). The biggest difference, however, is that the role of the audience has expanded and that audience can and often does interact with the writers. Thus, writers must be able to discern who they are interacting with and how they wish to possibly manage and negotiate their interaction with the audience.

Assessment Implications for the Proposed Framework

Given the changing dynamics of the audience described above, assessment considerations of audience awareness in multimodal composing should include the ability to articulate the imagined versus actual audience (Lunsford & Ede, 2009), as well as who is actually reading and responding to writers’ digital multimodal creations (boyd, 2014). Writers should be able to discuss how they negotiate the power dynamics and the potential conflicts in these imagined and actual audiences, in addition to the goals they have for a given rhetorical situation (Charlton, 2014). Writers should also be able to reflect on “the nature of engagement and collaboration with the audience . . . [and] . . . the point in the composition process (e.g., creation and distribution)” (McGrail & Behizadeh, 2017, p. 35). Finally, writers should clarify if they did or did not use the audience’s feedback in efforts to improve their compositions. The following questions can guide writers and assessors in these considerations:

> What conceptions of audience(s) do writers have?

> How do writers utilize audience interactivity to achieve their writing goals?

> How do writers negotiate power dynamics and potential conflicts in imagined and actual audiences?

Domain in Practice: Audience

To explore the domain of audience in practice, we turn to TreasureofRead-ing(Liu & Dou, 2015), a website codesigned by Zhongjian and Meredith, two Master of Education students at a private institution in the northeast. For this assignment, students were instructed to design a teaching tool using technology to assist English Language Learners

in their acquisition of content-area vocabulary. Zhongjian and Meredith chose to design a digital textbook that featured two original stories written for elementary students, “A Love Story” and “A Tale of a Pirate.” (Scan the QR code in Figure 2 with a smartphone to be directed to the Treasure of Reading website [Liu & Dou, 2015].) Zhongjian and Meredith created digital picture books for each story, using the website Storybird, and audio-recorded a read-aloud file to accompany each book (see Figure 3). They also created digital flashcards with vocabulary words from each story using Quizlet and embedded the flashcards on their website (see Figure 4). The website, created with Google Sites, included two options for navigation: an upper-level menu bar and three large icons on the homepage (see Figure 5).

In examining the product itself through the lens of the audience domain, we first explore the question, “What conceptions of audience(s) do the writers have?” The language on the website clarifies that the writers crafted this text with multiple audiences in mind: elementary students, teachers, and their professor. For example, on a subpage titled “About this Website Textbook,” the authors made an extended argument for why teachers should adopt their texts to use with ESL students. Here, they clarified the intended audience for the texts themselves (second through fourth graders) and shared suggestions for classroom implementation.

They also described intentional design choices made for their audience, for example, using simple, colorful icons and menus to make the website accessible and inviting for children. Though the navigation and website headings were not entirely intuitive, the authors explained that they considered various scenarios when designing the site. They organized the content so that a teacher could present it to their class on a single screen, or alternatively, so that students could move through the website independently. One could continue to investigate the writers’ conceptions of their audience(s) by reviewing individual components of the text, including the stories themselves and their corresponding materials.

Next, we consider the question, “How do writers utilize audience interactivity to achieve their writing goals?” The authors of the website shared their goal of helping ESL students’ literacy development by utilizing three strategies: “integrat[ing] reading, writing, listening, and speaking with each other,” “contextualizing vocabulary,” and “provid[ing] interest-ing instructional materials” (Liu & Dou, 2015, About this Website Textbook section, para. 1). With this goal of engaging readers in multiple ways, the writers incorporated a variety of interactive elements within their website. The pairing of a picture book and accompanying audio-recording invited students to practice reading and listening. The clickable stories required students to hover over each page with their cursor to “flip” the pages of the book and, similarly, “flip” the digital flashcards with a click to reveal the vocabulary word or its accompanying definition. Students could click basic controls (e.g., play/pause, volume) to listen to the embedded audio files. Though a comments box appeared at the bottom of some of the subpages, the comments feature was disabled.

Lastly, we turn to the question, “How do writers negotiate potential conflicts in imagined and actual audiences?” Though the website was designed with multiple audiences in mind, the navigation was not intuitive for either group. For example, the “Surprise” heading in the menu bar leads users to an “About this Website Textbook” page written for an audience of teachers. It is possible, though, that student readers might click on the “Surprise” menu and find their way to a page not designed for them to utilize. Similarly, the writers used the word “Lessons” as a menu heading, which could suggest “lesson plans” to teacher-readers; however, the “Les-sons” page was geared toward students, housing reading material for each story, referred to as Lesson 1 and Lesson 2.

Considering this example of multimodal writing through the lens of our framework, we see some missed opportunities. For example, we wonder about the authors’ decision to disable the comment feature and what that might reveal about their perceptions of audience. In this case, the instructor did not prompt the creators to explain this rhetorical choice, so our analysis of the text itself does not reveal such information. The last question within the audience domain, with its distinction between imagined and actual audiences, necessitates that the creators of a text consider their varied audiences and share those reflections explicitly with their instructor. Gaps such as these point to the framework’s ability to raise questions about instructional possibilities that could enhance both the creators’ and readers’ experiences.

Mode and Meaning

Composing any piece necessitates considering the mode best suited to the piece’s intended audience. Kress (2005) defined mode as “the culturally and socially produced resources for representation” (p. 6), including “artifacts, experiences, [and] technological tools” (Anderson & Kachorsky, 2019, p. 24). In other words, mode refers to the means and tools of composing—and these tools exist in social and cultural contexts. Kress (2005) distinguished mode from media by defining mode as the means by which compositions are shared with an audience and messages are disseminated to readers, viewers, and listeners. As Kress pointed out, modes have affordances and limitations that impact the meaning of a composition since the mode serves as the form of the composition.

So, what does it mean to combine modes to communicate a message? How can writers strategically use the affordances and limitations of different modes to create meaning? Mills et al. (2020) demonstrated that students can use multimodal strategies to communicate attitudinal meanings through systems of appraisal, as designed by Martin and White (2005), which create positive and negative feelings and attitudes. In studying culturally and linguistically diverse twelfth graders completing multimodal assignments, Smith (2019) pointed out that disparities in technical skill can affect how students are able to create meaning and that collaboration can be a way to develop these skills. In digital multimodal writing, writers draw on knowledge of form, substance, and technologies to craft compositions that produce a desired meaning and effect on an audience (Hicks et al., 2013). The connection between mode and meaning, then, is an important aspect of multimodal composition.

Through technical skill, writers can realize the affordances (and limitations) of various modes, making discrete choices to create meaning. The more writers know about how tools can be used, the more writers can make strategic decisions about which tools to use and how to use them (Hicks et al., 2013). Therefore, in crafting multimodal compositions, writers must draw on knowledge of different modes and their affordances, knowledge of particular genres and the conventions of those genres, and knowledge of technologies that allow the multimodal composition to be created and distributed. This knowledge intersects with the content to be shared. Meaning evolves from the attention to this intersection, so choices of mode facilitate meaning for the audience.

Assessment Implications for the Proposed Framework

In assessing multimodal compositions, teachers can consider how elements are combined in a situation to communicate a message to the audience. Whithaus (2005) pointed out that

the ability to see a relation between textual and visual elements are some of the skills that will develop as students become more proficient with multimodal literacies. The trick will be to design assessment systems that are flexible enough to adapt as these forms of writing emerge. (p. 16)

Among his suggested assessment criteria, Neal (2011) included relationships among modes, media, and texts. Modes working together is key to the persuasiveness of a multimodal composition.

To assess multimodal compositions, teachers must understand the how and why of students’ decisions in addition to looking at the composition itself. The following questions will help writers and assessors with considerations of mode and meaning:

> How do writers make choices about the modes they compose in and the media they use to disseminate their multimodal compositions?

> How do writers combine modes in ways that communicate their meaning to an audience? Are the combinations of modes persuasive in communicating the intended meaning?

> How do writers choose the media to get their message to their audience?

Domain in Practice: Mode and Meaning

In response to a task that challenged teacher candidates to examine how identity influenced their learning, Kieran, a Master of Arts in Teaching candidate at a private university in the northeast, created an Instagram profile titled “and.and.and.and.and.and.and” where she experimented with multimodal representation. Each post included an image of an impactful text and her written reflective experience associated with that text. In all, a series of 15 posts traced her identity development and the “dichotomies [that] inevitably led to the hierarchization, and sometimes exclusion, of practices and concepts” (see Figure 6).

Looking at Kieran’s piece, we first ask, “How do writers combine modes in ways that communicate their meaning to an audience?” and “Are the combination of modes persuasive in communicating the intended meaning?” Kieran provided a series of short posts analyzing works of literature and philosophy. While a reader might expect an Instagram page to rely on images or video, each post on this page relied on text. The reader is led to navigate through the series of posts as a gallery of analysis. Each post is viewed by clicking on a colored tile with the title of that analytical post. Though the format suggests that the reader move through the posts sequentially, it is possible to click on any particular post in a nonlinear order.

By presenting a series of analytical posts rather than a long essay, Kieran has made a choice in mode that leads to and underscores her meaning: her thoughts on identity and the emotional nature of academic work.

Kieran calls this Instagram account “a rhizomatic visual representation of self.” For her, the study of literature is about emotion, and by viewing the posts one by one, the reader explores her gallery of literature as an exploration of her identity.

To examine the question, “How do writers make choices about the modes they compose in and the media they use to disseminate their com-positions,” it is useful to hear from Kieran herself. In correspondence with her professor, she stated,

My representation of voice in this project also relates to my use of the rhizome as a means to structure my identity and project. No one part is separate from or more valuable than the whole. The last post was meant to be open-ended, because I do not see this inquiry as resolved.

Her choice in mode—an Instagram profile—allowed her to show movement over time, the duality of the rhizome that sat at the heart of her project, and an unresolved ending. The pairing of image and text further supported the dichotomies she felt in her identity exploration.

The final question of “How do writers choose the media to get their message to their audience?” is more difficult to explore in this project. The Instagram profile that Kieran created has only two followers, one of them her professor. The profile is open, yet she did not share it broadly enough to recruit an audience.

As assessors, we might surmise that she did not want to share these deeply reflective moments beyond the course. This assumption, however, lies in our review of her product, not in Kieran’s reflection on her process.

As is true with most writing, in simply looking at the product, the thinking behind the decisions made during composition is unclear to an assessor. If the reader makes assumptions, those assumptions may be incorrect. As Neal (2011) suggested, it may be useful for instructors to include a written reflection, in which the writer reflects on their writing process and decision-making, to be turned in with digital multimodal writing assignments to more fully assess elements of the domain.

Originality

Since Aristotle’s time, philosophers have contemplated the idea that new ideas are simply “old ideas revived” (Else, 1957, p. 424). The long history of seeing possibilities in creating new from old suggests that originality in multimodal compositions can be defined, in part, through the concept of remixing. Originality in multimodal composition does not require a reliance on producing an absolutely original product but rather creating a product that is informed by that which already exists but is transformed in a new, fresh way. Traces of what has already been produced can be seen within creative multimodal compositions, and the act of remixing helps multimodal writers to create meaning (Jocius, 2020; Smith, 2017). As such, multimodal compositions require reflection on preconceived ideas about creativity since it is in the creation of new meaning that they can be assessed for originality. In her seminal book on copyright and fair use in education, Hobbs (2010) explained that transformativeness is important in “repurposing copy-righted materials as part of the creative process” (p. 8). Without transforming the meaning and use of existing media, writers may be guilty, at minimum, of ethical violations and potentially of plagiarism and copyright infringement. Assessing originality in a remixed composition, then, requires evaluation of transformativeness. Hobbs (2019) indicated that current educational practice does not necessarily support student learning of the ethical imperative of highlighting authorship in copying and remixing material under fair use doctrine. Part of the assessment of originality in multimodal compositions, then, is to trace influences (or actual media) that have been remixed into the new, original composition. Writers should be able to articulate how their remixed piece transforms the original sources.

Assessment Implications for the Proposed Framework

Assessing originality within multimodal text compositions requires assessors and composers to move beyond traditional conceptions of creativity and originality, given that the multimodal text may or may not have components never before presented in a text. When considering derivative elements within the multimodal text, assessors should consider the text’s transformativeness, that is, the ways in which the composer repurposed an original text in a unique, creative manner. Writers and assessors might consider the following questions:

> In what ways does the composition embody originality as it relates to multimodal composition?

> From what derivative source or sources did the writer gain inspiration? What derivative source or sources did the writer use in the multimodal text?

> How did the writer alter any texts used in a transformative way?

Discussions between assessors and composers should facilitate opportunities to think critically about choices made and better understand what it means to be creative and original in multimodal texts.

Domain in Practice: Originality

To consider the domain of originality in practice, we turn to Jenna’s (H., 2019a) project titled “The Museum of ED372’s Trip Abroad,” a virtual multimodal museum created as part of a teacher education course at a small private college in the mid-Atlantic. (Scan the code in Figure 7 with a smartphone to be directed to Jenna H.’s virtual museum.) The museum was created as part of a unit in which candidates produced a multimodal text their students could use as a mentor text for a project of their own. The candidates then created a teacher assessment guide (H., 2019b) to help fellow educators think about how to assess their students’ multimodal compositions. After studying abroad the summer prior, Jenna decided to create a virtual museum to teach about the places she visited. She created a main lobby (see Figure 8) from which visitors could move to other rooms via hyperlinks: Room of Strained Peace, Education Room, The London Room, or The Curator’s Office.

Once visitors made their decision, they were brought to a room. The London Room (see Figure 9), for example, included a gallery of 10 images. If visitors clicked on the photograph of Jenna in a telephone booth, they were taken to another slide (see Figure 10), which provided both alphabetic text and the same image of Jenna in the telephone booth.

First, we ask, “In what ways does the composition embody originality as it relates to multimodal composition?” Jenna’s multimodal composition lends itself to analysis through the lens of originality in several ways. Jenna used original photographs that either she or others took. In addition, she composed hyperlinked paths and wrote descriptions of the artifacts selected to highlight how she wanted her audience to move through the museum.

Second, we ask, “From what derivative source or sources did the writer gain inspiration or what derivative source or sources did the writer use in the multimodal text?” In Jenna’s assessment guide, she had to indicate her mentor texts, that is, the works from which she took inspiration as she composed her virtual museum. She cited two mentor texts: the Metropolitan Museum of Art’s online gallery and a YouTube video titled “Virtual Museums with Google Slides” (Lee, 2016).

As required for the assignment, Jenna acknowledged that she gained inspiration in myriad ways from multiple sources. One can see that the side-by-side format from the Met’s website layout mirrors the layout Jenna used in her museum. In her assessment guide, Jenna suggested that her students could use language similar to that on the Met’s website when composing their virtual museums. When one views the mentor YouTube video, one can see that the lobby design looks quite similar to Jenna’s design. In her assessment guide, Jenna discussed how students could learn about using links, setting up an exhibit, and having a curator’s office.

Finally, we ask, “How did the writer alter any texts used in a transformative way?” Jenna’s decision to place her title “The Museum of ED 352’s Trip Abroad” (see Figure 11) on top of an original image exhibits transformation. It is important to note that, in post-assignment conversations, Jenna suggested this cover photo and others were found on Google. Whereas her text does include some original photos (e.g., those taken by her and put into frames found online via Google slides), other photos (e.g., the cover slide) came from a Google search.

Had the idea of transforming texts been more of a purpose of the assignment or been made more explicit during the composition process, Jenna may have altered texts in more transformative ways. This assignment did not privilege originality in its inception or process, however, so it is not surprising that Jenna’s multimodal text does not reflect such transformation. In fact, her previous instructor, Kathryn, contacted Jenna during the writing of this article to ask questions about originality because they were not made clear within the final product or assessment guide. When particular multimodal texts lend themselves to assessing originality, teachers should engage students in conversations and reflective work on the transformative use and citation of source material.

Implications for Practice from the Interconnected Framework

The proposed interconnected framework seeks to put the Beliefs for Integrating Technology into the English Language Arts Classroom (Bartels et al., 2019) into action by providing English teacher educators, English teachers, and composition and literacy instructors a way to think about digital multimodal composition and assessment. The framework draws from prior research in areas such as multiliteracies (New London Group, 2000), modality (Kress, 2005), audience (boyd, 2014; Marwick & boyd, 2011), and remixing (Jocius, 2020; New London Group, 2000; Smith, 2017). Whereas earlier work on developing a framework for multimodal assessment focuses on process and writer’s dispositions (Wahleithner, 2014), this proposed framework lays out domains with questions for both writers and readers/assessors to consider and seeks to be flexible for use with a variety of texts and for a variety of pedagogical purposes, demonstrating the contextual nature of multimodal composing and evaluation. In particular, English teacher educators might work with the framework to develop English teachers who see themselves as multimodal composers and who feel competent assessing their students’ multimodal compositions.

The questions associated with each domain suggest that assessment must occur throughout the process of composition, including the writer’s reflection on that process, and not simply focus on the created product (see Table 1). Even before the composing process begins, students can use the framework to analyze mentor texts, learning about the ways in which domains function in a particular multimodal composition so that they may apply these understandings in their work. Some domains and their correlating questions are more fitting for particular assignments, which makes the proposed framework adaptable to particular rhetorical situations, assignment purposes, student and teacher decisions, and the audiences for which the compositions are created.

Analyzing the given multimodal composition examples through this framework communicates clearly that writers need to be involved in the assessment process. As such, the framework gives power and voice to student composers by inviting them into conversation about what counts as effective multimodal compositions and how to evaluate such texts. By permitting assessment practices to be co-constructed and co-negotiated by teachers and students, the framework’s questions provide the structure for and invite multidirectional dialogue concerning the domains of audience, mode and meaning, and originality and of their interconnectedness at different points. With multiple touchpoints for instructors to reach out for the writer’s perspective of the piece, and possibly the writer’s interpretations of peers’ and authentic audience’s feedback, the framework allows teachers to respond, give feedback, and support writers as they make decisions about their multimodal compositions. For instance, when we look at the example of the virtual museum, we realize that the composer did not necessarily consider originality. Had the student been prompted more explicitly to do so, however, the composer’s process and completed product may have reflected attention to that specific domain. Similarly, in the example of the website, we acknowledge author choices about audiences were not part of the assessment. Had the choices authors made regarding the audience been explicitly built into the assignment and assessment, the website’s comment feature may have been enabled. Neither of these changes would automatically make better multimodal compositions; the point is simply that drawing attention to a particular domain may alter compositional choices and, therefore, the composition.

Whereas including students in assessment can add complexity to the assessment process, it also creates space for renegotiating the power dynamics of assessment in the classroom. This requires a shift in the traditional power dynamics (Silseth & Gilje, 2019) and roles and relationships related to assessment (Anderson & Kachorsky, 2019), allowing power to flow not only from teacher to student but also from student to teacher, and perhaps even from the audience (McGrail & Behizadeh, 2017). The latter approach would require “providing specific directions for external reviewers and student writers regarding assessment criteria expectations, as well as description of student writer and reviewer roles during this process” (McGrail & Be-hizadeh, 2017, p. 35).

Ultimately, the interconnected framework for assessment of multi-modal compositions affords composers and educators myriad opportunities, calling for a negotiated approach to assessment between teachers and students. It permits different entry points for instruction, composing, and assessing that can reflect the space in which the composer is creating a multimodal composition. It also supports the composer’s reflection on using the domain concepts and questions as resources when developing and evaluating their digital multimodal compositions. As multimodal composition and composing practices continue to evolve and adapt to new technologies and new contexts, and because communities will continue to evolve and adopt norms that will become the practices of future writers, the framework will no doubt be re-visited and re-envisioned over time.

Note

The authors would like to acknowledge Ivy Yeh Cox for her assistance in finalizing the graphic design of the framework. They also would like to thank the students who generously shared their work samples.

References

Anderson, K. T., & Kachorsky, D. (2019). Assessing students’ multimodal compositions: An analysis of the literature. English Teaching: Practice and Critique, 18(3), 312–334.

Anderson, M., & Jiang, J. (2018). Teens, social media & technology 2018. Pew Research Center. https://www.pewinternet.org/2018/05/31/teens-social-media-technology-2018/

Bartels, J., Beach, R., Connors, S., Damico, N., Doerr-Stevens, C., Hicks, T., Labonte, K., Loomis, S., Lynch. T. L., McGrail, E., Moran, C., Pasternak, D., Piotrowski, A., Rice, M., Rish, R., Rodesiler, L., Rybakova, K., Sullivan, S., Sulzer, M., & Zucker, L. (2018). Beliefs for integrating technology into the English language arts classroom. National Council of Teachers of English Position Statement. http:// www2.ncte.org/statement/beliefs-technology-preparation-english-teachers/

boyd, d. (2014). It’s complicated: The social lives of networked teens. Yale University Press.

Bruce, D. L. (2009). Writing with visual images: Examining the video composition processes of high school students. Research in the Teaching of English, 43(4), 426–450.

Burke, A., & Hammett, R. F. (2009). Assessing new literacies: Perspectives from the classroom. Peter Lang.

Charlton, C. (2014). The weight of curious space: Rhetorical events, hackerspace, and emergent multimodal assessment. Computers and Composition, 31, 29–42.

Duke, N. K., Purcell-Gates, V., Hall, L. A., & Tower, C. (2006). Authentic literacy activities for developing comprehension and writing. TheReading Teacher, 60(4), 344–355.

Eidman-Aadahl, E., Blair, K., DeVoss, D. N., Hochman, W., Jimerson, L., Jurich, C., Murphy, S., Rupert, B., Whithaus, C., & Wood, J. (2013). Developing domains for multimodal writing assessment: The language of evaluation, the language of instruction. In H. A. McKee & D. N. DeVoss (Eds.), Digital writing assessment and evaluation(Ch. 7). Computers and Composition Digital Press/Utah State University Press.

Else, G. F. (1957). Aristotle’s poetics: The argument. Oxford Press.

Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College Composition and Communication, 32(4), 365–387.

Fulwiler, M., & Middleton, K. (2012). After digital storytelling: Video composing in the new media age. Computers and Composition, 29(1), 39–50.

H., J. (2019a). The Museum of ED 352’s Trip Abroad. https://docs.google.com/ presentation/d/1BF0tEYKz85k7LFVh3OLvucas7ZAUis7F8pW75min3c/ edit#slide=id.p

H., J. (2019b). Assessing writers with virtual museums. https://docs.google.com/ document/d/10KocnIeb-cbO9Syb7dwUgWwGvEi9NBeeI_MD8AQ73QA/ edit?usp=sharing

Hicks, T. (2013). Crafting digital writing: Composing texts across media and genres. Heinemann.

Hicks, T. (2015). Assessing students’ digital writing: Protocols for looking closely. Teachers College Press.

Hicks, T., Turner, K. H., & Stratton, J. (2013). Reimagining a writer’s process through digital storytelling. Learning Landscapes, 6(2), 167–184.

Hobbs, R. (2010). Copyright clarity: How fair use supports digital learning. Corwin.

Hobbs, R. (2019). Ethical practices: Meditation. In K. H. Turner (Ed.), The ethics of digital literacy: Developing knowledge and skills across grade levels (pp. 155–158). Rowman and Littlefield.

Hoffman, M. (2015). Peer response, remixed: Authentic peer response through audio technology. English Journal, 104(4), 32–36.

Ito, M., Baumer, S., Bittanti, M., Boyd, D., Cody, R., Herr-Stephenson, B., Horst, H. A., Lange, P. G., Mahendran, D., Martinez, K. Z., Pascoe, C. J., Perkel, D., Robin-son, L., Sims, C., & Tripp, L. (2009). Hanging out, messing around, and geeking out: Kids living and learning with new media. MIT Press.

Jocius, R. (2020). The CLICK model: Scaffolding multimodal composing for academic purposes. Language Arts, 97(3), 146–158.

Kress, G. (2005). Gains and losses: New forms of texts, knowledge, and learning. Computers and Composition, 22(1), 5–22.

Lammers, J. C., & Van Alstyne, J. H. (2019). Building bridges from classrooms to networked publics: Helping students write for the audience they want. Journal of Adolescent&AdultLiteracy, 62(6), 653–662.

Law, L. (2020). Creativity and multimodality: Analytical framework for creativity in multimodal texts (AFCMT). Linguistics and Human Sciences, 14(1), 60–93.

Lee, D. (2016). Virtual museums with Google slides. YouTube. https://www.you tube.com/watch?v=5XNevQHfvjc

Lindblom, K. (2015). School writing vs. authentic writing. Writers who care. https://writerswhocare.wordpress.com/2015/07/27/school-writing-vs-authen tic-writing/

Liu, Z., & Dou, K. (2015). Home.Treasure of Reading. https://sites.google.com/site/ treasureofreading/home

Long, R. C. (1980). Writer-audience relationships: Analysis or invention? College Composition and Communication, 31(2), 221–226.

Lunsford, A., & Ede, L. (2009). Among the audience: On audience in an age of new literacies. In M. E. Weisler, B. Fehler, & A. M. González (Eds.), Engaging audience: Writing in an age of new literacies(pp. 42–72). NCTE.

Martin, J., & White, P. (2005). The language of evaluation: Appraisal in English. Macmillan.

Marwick, A. E., & boyd, d. (2011). I tweet honestly, I tweet passionately: Twitter users, context collapse, and the imagined audience. New Media and Society, 13, 114–133.

McGrail, E., & Behizadeh, N. (2017). K–12 multimodal assessment and interactive audiences: An exploratory analysis of existing frameworks. AssessingWriting,31, 24–38.

McGrail, E., & McGrail, J. P. (2010). Copying right and copying wrong with Web2.0 tools in the classroom. Contemporary Issues in Technology &Teacher Education,10(3), 257–274. http://www.citejournal.org/vol10/iss3/languagearts/article1. cfm

Mills, K., Stone, B., Unsworth, L., & Friend, L. (2020). Multimodal language of attitude in digital composition. Written Communication, 37(2), 135–166.

Neal, M. (2011). Writingassessment and the revolution in digitaltexts and technolo–gies. Teachers College Press.

New London Group. (2000). A pedagogy of multiliteracies: Designing social futures. In B. Cope & M. Kalantzis (Eds.), Multiliteracies: Literacy learning and the design of social futures(pp. 9–37). Routledge.

Painter, C., & Martin, J. R. (2011). Intermodal complementarity: Modelling affordances across image and verbiage in children’s picture books. In H. G. Wen (Ed.), Studies in functional linguistics and discourse (pp. 132–158). Higher Education Press.

Rish, R., Cun, A., Gloss, A., & Pamuk, M. (2018). Community inquiry with mobile asset mapping. In D. Herro, S. Arafeh, R. Ling, & C. Holden (Eds.), Mobile learning: Perspectives on practice and policy (pp. 119–142). Information Age Publishing.

Silseth, K., & Gilje, O. (2019). Multimodal composition and assessment: A sociocul-tural perspective. Assessment in Education: Principles, Policy & Practice, 26(1), 26–42.

Smith, B.E. (2017). Composing across modes: A comparative analysis of adolescents’ multimodal composing processes. Learning, Media and Technology, 42(3),

259–278.

Smith, B.E. (2019). Collaborative multimodal composing: Tracing the unique partnership of three pairs of adolescents composing across three digital projects. Literacy, 53(1), 14–21.

Swenson, J., Young, C. A., McGrail, E., Rozema, R., & Whitin, P. (2006). Extending the conversation: New technologies, new literacies, and English education. English Education, 38(4), 349–367.

Tan, L., Zammit, K., D’warte, J., & Gearside, A. (2020). Assessing multimodal literacies in practice: A critical review of its implementations in educational settings. Language and Education, 34(2), 97–114.

Turner, K. H., & Hicks, T. (2017). Argument in the real world. Heinemann.

Wahleithner, J. M. (2014). The National Writing Project’s Multimodal Assessment Project: Development of a framework for thinking about multimodal composing. Computers and Composition, 31, 79–86.

Weiser, M. E., Fehler, B. M., & González, A. M. (Eds.) . (2009). Engaging audience: Writing in an age of new literacies[Preface]. NCTE.

Whithaus, C. (2005). Teaching and evaluatingwriting in the age of computers and high-stakes testing. Lawrence Erlbaum Associates.

Wierszewski, E. (2013). Something old, something new: Evaluative criteria in teacher responses to student multimodal texts. In H. A. McKee & D. N. DeVoss (Eds.), Digital writing assessment and evaluation(Chapter 5). Computers and Composition Digital Press/Utah State University Press.

Wiggins, G. (2009). Real-world writing: Making purpose and audience matter. English Journal, 98(5), 29–37.

Ewab McGrail, PhD, is an Associate Professor of Language and Literacy at Georgia State University. She coauthored Student Blogs: How Online Writing Can Transform Your Classroom, and she examines digital writing and multi-modal assessment; copyright and media literacy; technol-ogyandthedigitaltextinteachingandlearning;aswellas the representation of underprivileged and marginalized groups in popular culture and mass media. She has been a member of NCTE since 2004.

Dr. Kristen Hawley Turner (@teachkht) is Professor and Director of Teacher Education at Drew University in New Jersey. She is the co-author of Connected Reading: Teaching Adolescent Readers in a Digital World and Argument in the Real World: Teaching Students to Read and Write Digital Texts and editor of Ethics of Digital Literacy: Developing Knowledge and Skills across Grade Levels. A member of NCTE for two decades, she is also the founder and director of the Drew Writing Project and Digital Literacies Collaborative.

Amy Piotrowski, PhD, is an assistant professor of English education at Utah State University. Before going into teacher education, she taught middle school English language arts, high school English, and college composition. Her scholarly interests focus on digital literacies, young adult literature, and teacher education. Amy has been a member of NCTE since 2001.

Kathryn Caprino (@KCapLiteracy), PhD, is a former middle school and high school English teacher and current assistant professor of PK–12 New Literacies at Elizabethtown College. A trained reading specialist, she teaches courses in literacy methods and assessment. Her current scholarship focuses on the teaching of writing; children’s, middle grades, and young adult literature; and technology integration. Kathryn has been a member of NCTE since 2004.

Lauren Zucker, PhD, teaches high school English in New Jersey and education courses at Drew University and Fordham University. Her recent work explores the differences in adolescents’ attitudes and approaches to print and digital reading tasks. She is the coeditor of New Jersey English Journal and has been a member of NCTE since 2008.

Mary Ellen Greenwood is a lecturer of English at Utah State University, where she is also pursuing a PhD in Curriculum and Instruction. Her scholarly interests focus on New Literacies, critical pedagogy, popular culture, and learning outcomes for college composition. She has been a member of NCTE.

DMU Timestamp: April 07, 2022 14:13