Multimodal pedagogy

Chart using the five modes of communication to explain multimodality.

Multimodal pedagogy is an approach to the teaching of writing that implements different modes of communication. Multimodality refers to the use of visual, aural, linguistic, spatial, and gestural modes in differing pieces of media, each necessary to properly convey the information it presents.

The visual mode conveys meaning via images and the visible elements of a text such as typography and color. The aural mode refers to sound in the form of music, sound effects, silence, etc. The linguistic mode includes written and spoken language. The spatial mode focuses on the physical arrangement of elements in a text. The gestural mode refers to physical movements such facial expressions and how these are interpreted. A multimodal text is characterized by the combination of any two or more modes to express meaning.

Multimodality as a term was coined in the late 20th century, but its use predates its naming, with it being used as early as Egyptian hieroglyphs and classical rhetoric. Compositionists and writing theorists have been exploring how the five modes of communication interact with each other and how multimodality can be used in the teaching of writing since the 20th century.

Multimodal pedagogy encourages the use of these modes as teaching tools in the classroom to facilitate learning. Although lack of experience with new technologies and limited access to resources can make multimodal instruction difficult for teachers, it is important for students to learn to interpret and create meaning across multiple modes of communication in order to navigate a multimodal world.

Background

Some compositionists, such as Jason Palmeri, have suggested that writing has always been multimodal, with writing always containing different modes, such as visual, aural, spatial, and gestural. Multimodal pedagogy in that sense has existed since before multimodality was properly named in the mid-1990s.

The School of Athens by Raphael portrays Ancient Greek rhetoricians Aristotle and Plato.

Rhetoric began as the art of oral persuasion, and classical rhetoric was always meant to be delivered via voice. In the 1960s, auditory art as it related to writing pedagogy began to be studied by compositionists. In his article “What Do We Mean When We Talk about Voice in Texts?” Peter Elbow introduced his audible voice theory. Elbow's theory posits that words are multimodal and that readers experience sound in their heads even as they read silently. Audible voice theory argues that reading out loud lets writers understand how voice plays a part in writing and how text sounds to others. This understanding of speech can then improve understanding of communication and writing.

In the 1970s, the Process Theory of Composition focused on writing as a process. Linda Flower and John Hayes studied problem finding and solving, and argued this was a creative cognitive activity that writing and art had in common. Flower and Hayes also argue that writing is multimodal thinking, because writers don't think in just words. Writing includes the forming of ideas, creation of organization and rhetorical goals, and experiencing the world through our senses.

Kathleen Blake Yancey contends that literacy “is in the midst of tectonic change” and that technology has resulted in an increase in multimodal genres in writing. In one of the National Council of Teachers of English's position statements, they state that all texts are multimodal, and that composing as a whole has changed as a result of technology and its advances. The NCTE asserts that multimodality should be part of definitions for composition and that excluding multimodality is outdated.

Currently, print based literacy has undergone a transformation into hyphenated, plural or multiple literacies, acknowledging the diversity of media and information sources. Both technology and literacy are not mutually exclusive, now existing as a merged vocabulary used in current educational debates. Instructional practices, reading, learning, writing and literacy practices as a whole are being transformed, causing concern amongst teachers and educational educators, to be mediated by microcomputers, the Internet and educational software. The process of reading and writing has shifted, even though the core principles have not, going from print text into multimodal text-image information.

Teachers understand that multimodality composition assignments provide students with valuable opportunities to build their rhetoric skills. An example is an assignment that challenges students to "translate" a text based ethnography to a photo essay, which can cause a change to research ethics. In various scenarios are students able to build the following skills: to properly consider the forms of communication required in every given circumstance, to select resources and materials that will aid in the creation of an effective text, must consider the audience and meet the objectives in correspondence to them, have the finished text be and avenue for improved communication, and to have the text influence positive action.

Goals

Multimodal pedagogy aims to help students express themselves more accurately within their work. This approach allows students to engage deeply with their learning process, possibly increasing their investment in their work by identifying the modes that best suit their subject or personal preferences. With the different kinds of multimodal texts, it allows students to look at other forms of media with the thought of multimodality. Students' individual processing of texts shows different ways of understanding and using multimodality in learning.

Students have four primary learning styles: visual, auditory, reading/writing, and kinesthetic, each with specific sources of learning. Visual learners are those that get their learning from anything that stimulates their eyes. They primarily use infographics, videos, and illustrations as their source of learning. Aural learners like to use anything they can simply listen to in order to take in information. Their sources of learning mainly come from podcasts, an audiobook, and group discussions. Reading and writing is the most traditional form of multimodal learning. These learners use documents, books, and PDF's as their primary sources. Lastly, kinesthetic learning is one that gets its learners active. It commonly uses multiple learning types together at once. The main ways of learning are through demonstrations and multimedia presentations.

Multimodal pedagogy aids in enhancing students' comprehension of topics and issues by allowing them to explore information from various perspectives through different modes.

Application to the college writing classroom

Multimodal pedagogy can be applied in many different areas within the college writing classroom. Assignments that incorporate the multimodal modes of communication (visual, spatial, gestural, audio, and linguistic) encourage students to think about which method communicates ideas and information in the most efficient way. Technological advances have facilitated access to and creation of new learning materials for students, and multimodal pedagogy makes use of these innovations.

Students who have a harder time engaging with traditional teaching methods may engage better with multimodal materials. Assignments that use multiple modes of communication increase learning and comprehension skills. In addition, multimodal pedagogy allows for the development of multiliteracy skills and modal adaptability using a creative approach.

Multimodal pedagogy can be implemented into required and supplementary learning materials in the form of podcasts, video essays, infographics, or graphic novels (to name a few).

Podcasts help the students learn the importance is lingual communication, which incorporates word choice, tone of delivery, and organization of phrases and ideas. It also teaches the importance of audio — music, sound effects, ambient noise, silence, and volume — when it comes to conveying information.

Zines are one type of multimodal projects created by students.

Graphic novels exemplify the use of the visual communication mode, utilizing color, layout, style, size, and perspective to convey a story or message. The use of visual and graphic novels in the college writing classroom can increase engagement in the material whilst promoting Visual Literacy. Illustrations strengthen information by depicting its content or supplementing it, as seen in The Fragile Framework, an academic comic book published by The International Weekly Journal of Science 'NATURE'.

Zines allow students to engage in multimodal text creation in way that is accessible and inexpensive. Students are able to cut and paste images and text into pamphlet pages requiring that they make choices regarding the visual, linguistic, and spatial aspects of the text and examine these modes in relation to another.

Educational institutions have implemented multimodal pedagogy directly through curriculum design. Universities such as Iowa State and Georgia Tech use a WOVE curriculum in their composition courses. The acronym stands for written, oral, visual, and electronic communication. This curriculum recognizes the interconnectedness of these communicative modes and is designed to prepare students with a range of literacy skills that extends across multiple modes and media.

Challenges

Inexperience with new media poses a challenge for teachers in multimodal classrooms. Some teachers, such as Bremen Vance of Iowa State University, express anxiety and discomfort in incorporating new modes and technologies as they lack the skills to confidently engage with these, let alone guide students in their engagement. The rapid development of new media available to teachers further complicates the task of staying up to date and equipped with the skills to utilize these. While the multimodal pedagogical approach has expanded what qualifies as writing and how teachers can go about its instruction, thorough knowledge and planning is required in order to be effectively implemented. Writing instructors need to be able to contextualize their use of multimodal texts and lead students to rhetorically analyze how these create meaning.

Lack of resources has limited the ability of public school teachers to integrate new technologies that facilitate multimodal learning in their classrooms. Often times, they are forced to resort to older technologies and poorly functioning equipment such as overhead projectors and art supplies.