Multimodal texts are communications that use more than one semiotic system, or mode, to convey a message. These modes can include written language, visual imagery, audio, spatial arrangements, and gestures. The integration of these different modes allows for a more complex and enriched communication experience. Multimodal texts can be found across various platforms, such as books, advertisements, websites, films, and video games. They are particularly common in digital environments, where technology enables the easy combination of various media forms. By engaging multiple senses, multimodal texts can enhance comprehension, engagement, and memory. Let's explore the definition, examples, and analysis of multimodal texts.
Multimodal text definition
A multimodal text is a text that creates meaning by combining two or more modes of communication, such as print, the spoken word, sound, and images. Multimodal texts can be digital (websites, blogs) or print-based (graphic novels, newspapers, magazines). Multimodal texts can even be live performances.
Importance of multimodal texts
Multimodal texts are important because, these days, communication is shaped by a range of different, and overlapping, technologies. It is essential that we are able to understand multimodal texts so that we can take advantage of this.
Furthermore, combining different modes of communication to create meaning is an essential skill. Understanding them helps us develop a critical perspective of the media we consume.
Modes of multimodal text
There are five modes that can be combined to create multimodal texts:
- Linguistic
- Visual
- Gestural
- Spatial
- Aural
Linguistic
The linguistic mode refers to any sort of written or spoken text.
The linguistic mode is the most common mode because it can be used in a variety of ways. Think of all the places where you find written or spoken texts!
Visual
This refers to images and characters, such as marks, signs, and symbols. The visual mode involves the use of color, layout, style, size, and perspective.
Think of road signs, for example. They seem purely visual because they don't use any language or sound. Signs rely on the visual mode to get their message across. However, they also use the spatial mode. That's why we can say that signs are multimodal texts.
Gestural
The gestural mode is about movement and how we interpret it. Examples of gestural modes include facial expressions, hand gestures, body language, and interactions between people.
The gestural mode is important in face-to-face conversations, in the theater, concerts, and other live performances, as well as in YouTube videos and other platforms. This mode is usually combined with the other modes.
Spatial
The spatial mode refers to the organisation of elements in a space. It also deals with the physical closeness between people and objects.
Here are two examples of the spatial mode:
Fig 1. - Theatre stages are examples of a spatial mode
The stage is on a platform and the audience sits in chairs below, facing it. Here, the power is in the hands of the performers. This spatial order allows the audience to forget about their surroundings and to focus on what is happening on stage.
The audience sits in a circle that surrounds the stage. The performers and the audience are on the same level. This spatial order allows each audience member to get a unique view of what is happening on stage. Additionally, sitting in-the-round makes the viewers aware of each other, and enables their participation.
Both of these uses of the spatial mode create different experiences for the audience.
Aural
The aural or audio mode focuses on sound. This includes music, sound effects, ambient noises, silence, the tone of a person's voice, the volume of a sound, emphasis, and accent.
Some examples of the aural mode are radio programs, podcasts, and audio plays.
Examples of multimodal text
Multimodal texts are compositions that incorporate multiple modes of communication, including written text, visual images, audio, and sometimes even interactive elements. Here are a few examples:
Graphic Novels: These blend written text with visual images to tell a story. For example, Maus (1991) by Art Spiegelman.
Websites: Most websites are multimodal, combining text, images, video, and interactive elements.
Film: Films use moving images, spoken words, sound effects, and music to create a multimodal experience.
Video Games: Video games combine visuals, sound, text, and interactive gameplay.
Infographics: These visual representations use images and text to explain complex information quickly and clearly.
Advertisements: Most advertisements, whether print or digital, combine text, images, and sometimes sound to sell a product or service.
Let's look at the different types of multimodal text along with these examples.
Types of multimodal text
There are four types of multimodal text:
| Types of multimodal texts |
|---|
| Types of multimodal texts | Multimodal text examples | Mode combination |
| Print-based | Comic books, graphic novels, posters, newspapers | Linguistic and visual |
| Digital | Internet-based: Social media content, blogs, online gamesOffline: films, animation, video games | Internet-based: usually linguistic and visualOffline: linguistic, visual, gestural, and aural |
| Live | Theatre, concerts, poetry readings | Linguistic, gestural, spatial, and aural (and sometimes visual) |
| Transmedia | Board game on an app, or an app that requires you to interact with the real world | All five modes, or variances of a combination |
Print-based multimodal text
This is a printed text that combines different modes.
Comic books, graphic novels, picture storybooks, posters, leaflets, magazines, and newspapers are all examples of print-based multimodal text. They combine the linguistic mode and the visual mode.
Digital multimodal text
This is a virtual text that combines different modes.
Digital texts can be internet-based, such as social media content, blog posts, vlogs, and online games. They can also be found offline. Some examples are films, animations, e-books, video games, and presentation slides.
Digital multimodal texts can combine different modes. For instance, films and video games normally combine the linguistic, visual, gestural, and aural modes, whilst blog posts and e-books usually only use linguistic and visual modes.
Live multimodal text
This is a text that is performed live.
Live multimodal texts include theatre, opera performances, and concerts. Book premieres, poetry readings, and lectures are also examples of live multimodal texts. These usually combine four of the five modes: linguistic, gestural, spatial, and aural. Sometimes they can combine all five modes.
Transmedia multimodal text
This is a text that tells a story or sends a message through a combination of media platforms (each of which uses various modes.) A transmedia text is a multimodal text within a multimodal text.
Each added platform in a transmedia multimodal text further develops the story or idea and enhances the user experience.
Some games can be effective transmedia multimodal texts:
Have you ever played a board game that required the use of an app? The physical game and the app are two different platforms. Additionally, the board game itself combines different modes: linguistic, visual, gestural, and spatial. Perhaps the app uses the remaining aural mode, whilst also incorporating other modes, such as the visual and the linguistic.
Maybe you have played a mobile game that required you to engage in physical activity outside of the digital world. Think of Pokémon GO, for instance.
Both of these are examples of transmedia multimodal text.
Transmedia texts are also known as transmedia storytelling. American researcher Henry Jenkins was the first to define this kind of text back in 2003.
Multimodal text analysis
To analyse a multimodal text, you should first recognise its different modes and be aware of the context they appear in. Then, you can proceed by pointing out those modes and using them as evidence to make an argument. You should also describe the modes in detail.
Multimodal text analysis example
Here is an example analysis of a multimodal text. In this case a street sign:
Fig 2. - A stop sign is an example of a multimodal text
The use of the linguistic mode –the written word 'stop'– reveals the message of the street sign. The drivers who see the sign need to stop driving. The visual mode supports the linguistic mode: the layout of the sign is white letters on a red background which is framed in white. This simple but effective layout draws attention and the use of red enhances the importance of what is written (that it is important to stop).
The spatial mode positions the sign on a road in what seems to be the countryside, where it is visible and not obstructed by other signs or buildings. This tells us that the drivers can immediately notice the sign and do as it says: to stop. This is important to do in the space where the sign is situated and will prevent accidents.
Multimodal Text - Key Takeaways
- A multimodal text is a text that creates meaning by combining two or more modes of communication, such as print, spoken word, audio, or image.
- Understanding multimodal texts and the ability to create multimodal texts are important skills.
- There are five modes used in multimodal texts: linguistic, visual, gestural, spatial, and aural.
- There are four types of multimodal text: print-based, digital, live, and transmedia.
- The analysis of a multimodal text includes recognising its different modes and using those modes as evidence to make an argument.
Explanations 
Exams 
Magazine 
