StudySmarter: Study help & AI tools
4.5 • +22k Ratings
More than 22 Million Downloads
Free
Multimodal texts are communications that use more than one semiotic system, or mode, to convey a message. These modes can include written language, visual imagery, audio, spatial arrangements, and gestures. The integration of these different modes allows for a more complex and enriched communication experience. Multimodal texts can be found across various platforms, such as books, advertisements, websites, films, and video games. They are particularly common in digital environments, where technology enables the easy combination of various media forms. By engaging multiple senses, multimodal texts can enhance comprehension, engagement, and memory. Let's explore the definition, examples, and analysis of multimodal texts.
Explore our app and discover over 50 million learning materials for free.
Lerne mit deinen Freunden und bleibe auf dem richtigen Kurs mit deinen persönlichen Lernstatistiken
Jetzt kostenlos anmeldenNie wieder prokastinieren mit unseren Lernerinnerungen.
Jetzt kostenlos anmeldenMultimodal texts are communications that use more than one semiotic system, or mode, to convey a message. These modes can include written language, visual imagery, audio, spatial arrangements, and gestures. The integration of these different modes allows for a more complex and enriched communication experience. Multimodal texts can be found across various platforms, such as books, advertisements, websites, films, and video games. They are particularly common in digital environments, where technology enables the easy combination of various media forms. By engaging multiple senses, multimodal texts can enhance comprehension, engagement, and memory. Let's explore the definition, examples, and analysis of multimodal texts.
A multimodal text is a text that creates meaning by combining two or more modes of communication, such as print, the spoken word, sound, and images. Multimodal texts can be digital (websites, blogs) or print-based (graphic novels, newspapers, magazines). Multimodal texts can even be live performances.
Multimodal texts are important because, these days, communication is shaped by a range of different, and overlapping, technologies. It is essential that we are able to understand multimodal texts so that we can take advantage of this.
Furthermore, combining different modes of communication to create meaning is an essential skill. Understanding them helps us develop a critical perspective of the media we consume.
There are five modes that can be combined to create multimodal texts:
The linguistic mode refers to any sort of written or spoken text.
The linguistic mode is the most common mode because it can be used in a variety of ways. Think of all the places where you find written or spoken texts!
This refers to images and characters, such as marks, signs, and symbols. The visual mode involves the use of color, layout, style, size, and perspective.
Think of road signs, for example. They seem purely visual because they don't use any language or sound. Signs rely on the visual mode to get their message across. However, they also use the spatial mode. That's why we can say that signs are multimodal texts.
The gestural mode is about movement and how we interpret it. Examples of gestural modes include facial expressions, hand gestures, body language, and interactions between people.
The gestural mode is important in face-to-face conversations, in the theater, concerts, and other live performances, as well as in YouTube videos and other platforms. This mode is usually combined with the other modes.
The spatial mode refers to the organisation of elements in a space. It also deals with the physical closeness between people and objects.
Here are two examples of the spatial mode:
A traditional theatre.
The stage is on a platform and the audience sits in chairs below, facing it. Here, the power is in the hands of the performers. This spatial order allows the audience to forget about their surroundings and to focus on what is happening on stage.
A theatre-in-the-round.
The audience sits in a circle that surrounds the stage. The performers and the audience are on the same level. This spatial order allows each audience member to get a unique view of what is happening on stage. Additionally, sitting in-the-round makes the viewers aware of each other, and enables their participation.
Both of these uses of the spatial mode create different experiences for the audience.
The aural or audio mode focuses on sound. This includes music, sound effects, ambient noises, silence, the tone of a person's voice, the volume of a sound, emphasis, and accent.
Some examples of the aural mode are radio programs, podcasts, and audio plays.
Multimodal texts are compositions that incorporate multiple modes of communication, including written text, visual images, audio, and sometimes even interactive elements. Here are a few examples:
Graphic Novels: These blend written text with visual images to tell a story. For example, Maus (1991) by Art Spiegelman.
Websites: Most websites are multimodal, combining text, images, video, and interactive elements.
Film: Films use moving images, spoken words, sound effects, and music to create a multimodal experience.
Video Games: Video games combine visuals, sound, text, and interactive gameplay.
Infographics: These visual representations use images and text to explain complex information quickly and clearly.
Advertisements: Most advertisements, whether print or digital, combine text, images, and sometimes sound to sell a product or service.
Let's look at the different types of multimodal text along with these examples.
There are four types of multimodal text:
Types of multimodal texts | ||
---|---|---|
Types of multimodal texts | Multimodal text examples | Mode combination |
Print-based | Comic books, graphic novels, posters, newspapers | Linguistic and visual |
Digital | Internet-based: Social media content, blogs, online gamesOffline: films, animation, video games | Internet-based: usually linguistic and visualOffline: linguistic, visual, gestural, and aural |
Live | Theatre, concerts, poetry readings | Linguistic, gestural, spatial, and aural (and sometimes visual) |
Transmedia | Board game on an app, or an app that requires you to interact with the real world | All five modes, or variances of a combination |
This is a printed text that combines different modes.
Comic books, graphic novels, picture storybooks, posters, leaflets, magazines, and newspapers are all examples of print-based multimodal text. They combine the linguistic mode and the visual mode.
This is a virtual text that combines different modes.
Digital texts can be internet-based, such as social media content, blog posts, vlogs, and online games. They can also be found offline. Some examples are films, animations, e-books, video games, and presentation slides.
Digital multimodal texts can combine different modes. For instance, films and video games normally combine the linguistic, visual, gestural, and aural modes, whilst blog posts and e-books usually only use linguistic and visual modes.
This is a text that is performed live.
Live multimodal texts include theatre, opera performances, and concerts. Book premieres, poetry readings, and lectures are also examples of live multimodal texts. These usually combine four of the five modes: linguistic, gestural, spatial, and aural. Sometimes they can combine all five modes.
This is a text that tells a story or sends a message through a combination of media platforms (each of which uses various modes.) A transmedia text is a multimodal text within a multimodal text.
Each added platform in a transmedia multimodal text further develops the story or idea and enhances the user experience.
Some games can be effective transmedia multimodal texts:
Have you ever played a board game that required the use of an app? The physical game and the app are two different platforms. Additionally, the board game itself combines different modes: linguistic, visual, gestural, and spatial. Perhaps the app uses the remaining aural mode, whilst also incorporating other modes, such as the visual and the linguistic.
Maybe you have played a mobile game that required you to engage in physical activity outside of the digital world. Think of Pokémon GO, for instance.
Both of these are examples of transmedia multimodal text.
Transmedia texts are also known as transmedia storytelling. American researcher Henry Jenkins was the first to define this kind of text back in 2003.
To analyse a multimodal text, you should first recognise its different modes and be aware of the context they appear in. Then, you can proceed by pointing out those modes and using them as evidence to make an argument. You should also describe the modes in detail.
Here is an example analysis of a multimodal text. In this case a street sign:
The use of the linguistic mode –the written word 'stop'– reveals the message of the street sign. The drivers who see the sign need to stop driving. The visual mode supports the linguistic mode: the layout of the sign is white letters on a red background which is framed in white. This simple but effective layout draws attention and the use of red enhances the importance of what is written (that it is important to stop).
The spatial mode positions the sign on a road in what seems to be the countryside, where it is visible and not obstructed by other signs or buildings. This tells us that the drivers can immediately notice the sign and do as it says: to stop. This is important to do in the space where the sign is situated and will prevent accidents.
A multimodal text is a text that creates meaning by combining two or more modes of communication, such as print, spoken word, audio, and images.
Multimodal text examples include newspapers, websites, and magazines.
The four types of multimodal text are print-based multimodal text, digital multimodal text, live multimodal text, and transmedia multimodal text.
There are also five modes that are used in multimodal texts. These are the linguistic mode, the visual mode, the gestural mode, the spatial mode, and the aural mode.
To analyse multimodal texts, you should recognise the different modes that are used in it while being aware of the context they appear in. You can make an argument by pointing out the modes and using them as evidence. You should also describe the modes in specific detail.
Multimodal texts are important because, by combining different modes of communication that complement each other, they create more meanings.
The ability to read and create multimodal texts is an important skill that can help us find new ways of expression and gain cross-disciplinary literacy.
Which of the five modes of multimodal text appear in a picture storybook?
A. Linguistic
What type of multimodal text is the game Pokémon GO?
Transmedia
Which of the five modes of multimodal text appear in a film?
A. Visual
What type of multimodal text is a magazine article?
Print-based
Which of these is a part of the aural mode?
Accent
Which of these does NOT refer to the linguistic mode?
Layout
Already have an account? Log in
Open in AppThe first learning app that truly has everything you need to ace your exams in one place
Sign up to highlight and take notes. It’s 100% free.
Save explanations to your personalised space and access them anytime, anywhere!
Sign up with Email Sign up with AppleBy signing up, you agree to the Terms and Conditions and the Privacy Policy of StudySmarter.
Already have an account? Log in