Multimodal Texts

Multimodal texts are communications that use more than one semiotic system, or mode, to convey a message. These modes can include written language, visual imagery, audio, spatial arrangements, and gestures. The integration of these different modes allows for a more complex and enriched communication experience. Multimodal texts can be found across various platforms, such as books, advertisements, websites, films, and video games. They are particularly common in digital environments, where technology enables the easy combination of various media forms. By engaging multiple senses, multimodal texts can enhance comprehension, engagement, and memory. Let's explore the definition, examples, and analysis of multimodal texts.

Multimodal Texts Multimodal Texts

Create learning materials about Multimodal Texts with our free learning app!

  • Instand access to millions of learning materials
  • Flashcards, notes, mock-exams and more
  • Everything you need to ace your exams
Create a free account
Table of contents

    Multimodal text definition

    A multimodal text is a text that creates meaning by combining two or more modes of communication, such as print, the spoken word, sound, and images. Multimodal texts can be digital (websites, blogs) or print-based (graphic novels, newspapers, magazines). Multimodal texts can even be live performances.

    Importance of multimodal texts

    Multimodal texts are important because, these days, communication is shaped by a range of different, and overlapping, technologies. It is essential that we are able to understand multimodal texts so that we can take advantage of this.

    Furthermore, combining different modes of communication to create meaning is an essential skill. Understanding them helps us develop a critical perspective of the media we consume.

    Modes of multimodal text

    There are five modes that can be combined to create multimodal texts:

    • Linguistic
    • Visual
    • Gestural
    • Spatial
    • Aural

    Linguistic

    The linguistic mode refers to any sort of written or spoken text.

    The linguistic mode is the most common mode because it can be used in a variety of ways. Think of all the places where you find written or spoken texts!

    Visual

    This refers to images and characters, such as marks, signs, and symbols. The visual mode involves the use of color, layout, style, size, and perspective.

    Think of road signs, for example. They seem purely visual because they don't use any language or sound. Signs rely on the visual mode to get their message across. However, they also use the spatial mode. That's why we can say that signs are multimodal texts.

    Gestural

    The gestural mode is about movement and how we interpret it. Examples of gestural modes include facial expressions, hand gestures, body language, and interactions between people.

    The gestural mode is important in face-to-face conversations, in the theater, concerts, and other live performances, as well as in YouTube videos and other platforms. This mode is usually combined with the other modes.

    Spatial

    The spatial mode refers to the organisation of elements in a space. It also deals with the physical closeness between people and objects.

    Here are two examples of the spatial mode:

    A traditional theatre.

    Multimodal Text Traditional Modes Theatre Stage StudySmarterFig 1. - Theatre stages are examples of a spatial mode

    The stage is on a platform and the audience sits in chairs below, facing it. Here, the power is in the hands of the performers. This spatial order allows the audience to forget about their surroundings and to focus on what is happening on stage.

    A theatre-in-the-round.

    The audience sits in a circle that surrounds the stage. The performers and the audience are on the same level. This spatial order allows each audience member to get a unique view of what is happening on stage. Additionally, sitting in-the-round makes the viewers aware of each other, and enables their participation.

    Both of these uses of the spatial mode create different experiences for the audience.

    Aural

    The aural or audio mode focuses on sound. This includes music, sound effects, ambient noises, silence, the tone of a person's voice, the volume of a sound, emphasis, and accent.

    Some examples of the aural mode are radio programs, podcasts, and audio plays.

    Examples of multimodal text

    Multimodal texts are compositions that incorporate multiple modes of communication, including written text, visual images, audio, and sometimes even interactive elements. Here are a few examples:

    1. Graphic Novels: These blend written text with visual images to tell a story. For example, Maus (1991) by Art Spiegelman.

    2. Websites: Most websites are multimodal, combining text, images, video, and interactive elements.

    3. Film: Films use moving images, spoken words, sound effects, and music to create a multimodal experience.

    4. Video Games: Video games combine visuals, sound, text, and interactive gameplay.

    5. Infographics: These visual representations use images and text to explain complex information quickly and clearly.

    6. Advertisements: Most advertisements, whether print or digital, combine text, images, and sometimes sound to sell a product or service.

    Let's look at the different types of multimodal text along with these examples.

    Types of multimodal text

    There are four types of multimodal text:

    Types of multimodal texts
    Types of multimodal textsMultimodal text examplesMode combination
    Print-basedComic books, graphic novels, posters, newspapersLinguistic and visual
    DigitalInternet-based: Social media content, blogs, online gamesOffline: films, animation, video gamesInternet-based: usually linguistic and visualOffline: linguistic, visual, gestural, and aural
    LiveTheatre, concerts, poetry readingsLinguistic, gestural, spatial, and aural (and sometimes visual)
    TransmediaBoard game on an app, or an app that requires you to interact with the real worldAll five modes, or variances of a combination

    Print-based multimodal text

    This is a printed text that combines different modes.

    Comic books, graphic novels, picture storybooks, posters, leaflets, magazines, and newspapers are all examples of print-based multimodal text. They combine the linguistic mode and the visual mode.

    Digital multimodal text

    This is a virtual text that combines different modes.

    Digital texts can be internet-based, such as social media content, blog posts, vlogs, and online games. They can also be found offline. Some examples are films, animations, e-books, video games, and presentation slides.

    Digital multimodal texts can combine different modes. For instance, films and video games normally combine the linguistic, visual, gestural, and aural modes, whilst blog posts and e-books usually only use linguistic and visual modes.

    Live multimodal text

    This is a text that is performed live.

    Live multimodal texts include theatre, opera performances, and concerts. Book premieres, poetry readings, and lectures are also examples of live multimodal texts. These usually combine four of the five modes: linguistic, gestural, spatial, and aural. Sometimes they can combine all five modes.

    Transmedia multimodal text

    This is a text that tells a story or sends a message through a combination of media platforms (each of which uses various modes.) A transmedia text is a multimodal text within a multimodal text.

    Each added platform in a transmedia multimodal text further develops the story or idea and enhances the user experience.

    Some games can be effective transmedia multimodal texts:

    Have you ever played a board game that required the use of an app? The physical game and the app are two different platforms. Additionally, the board game itself combines different modes: linguistic, visual, gestural, and spatial. Perhaps the app uses the remaining aural mode, whilst also incorporating other modes, such as the visual and the linguistic.

    Maybe you have played a mobile game that required you to engage in physical activity outside of the digital world. Think of Pokémon GO, for instance.

    Both of these are examples of transmedia multimodal text.

    Transmedia texts are also known as transmedia storytelling. American researcher Henry Jenkins was the first to define this kind of text back in 2003.

    Multimodal text analysis

    To analyse a multimodal text, you should first recognise its different modes and be aware of the context they appear in. Then, you can proceed by pointing out those modes and using them as evidence to make an argument. You should also describe the modes in detail.

    Multimodal text analysis example

    Here is an example analysis of a multimodal text. In this case a street sign:

    Multimodal Text A stop sign with a blurred green background StudySmarterFig 2. - A stop sign is an example of a multimodal text

    The use of the linguistic mode –the written word 'stop'– reveals the message of the street sign. The drivers who see the sign need to stop driving. The visual mode supports the linguistic mode: the layout of the sign is white letters on a red background which is framed in white. This simple but effective layout draws attention and the use of red enhances the importance of what is written (that it is important to stop).

    The spatial mode positions the sign on a road in what seems to be the countryside, where it is visible and not obstructed by other signs or buildings. This tells us that the drivers can immediately notice the sign and do as it says: to stop. This is important to do in the space where the sign is situated and will prevent accidents.

    Multimodal Text - Key Takeaways

    • A multimodal text is a text that creates meaning by combining two or more modes of communication, such as print, spoken word, audio, or image.
    • Understanding multimodal texts and the ability to create multimodal texts are important skills.
    • There are five modes used in multimodal texts: linguistic, visual, gestural, spatial, and aural.
    • There are four types of multimodal text: print-based, digital, live, and transmedia.
    • The analysis of a multimodal text includes recognising its different modes and using those modes as evidence to make an argument.
    Frequently Asked Questions about Multimodal Texts

    What is a multimodal text?

    A multimodal text is a text that creates meaning by combining two or more modes of communication, such as print, spoken word, audio, and images.

    What is an example of multimodal text?

    Multimodal text examples include newspapers, websites, and magazines.

    What are the types of multimodal text?

    The four types of multimodal text are print-based multimodal text, digital multimodal text, live multimodal text, and transmedia multimodal text.


    There are also five modes that are used in multimodal texts. These are the linguistic mode, the visual mode, the gestural mode, the spatial mode, and the aural mode.

    How to analyse multimodal texts?

    To analyse multimodal texts, you should recognise the different modes that are used in it while being aware of the context they appear in. You can make an argument by pointing out the modes and using them as evidence. You should also describe the modes in specific detail.

    How effective is it to use multimodal text?

    Multimodal texts are important because, by combining different modes of communication that complement each other, they create more meanings. 


    The ability to read and create multimodal texts is an important skill that can help us find new ways of expression and gain cross-disciplinary literacy.

    Test your knowledge with multiple choice flashcards

    Which of the five modes of multimodal text appear in a picture storybook?

    What type of multimodal text is the game Pokémon GO?

    Which of the five modes of multimodal text appear in a film?

    Next
    1
    About StudySmarter

    StudySmarter is a globally recognized educational technology company, offering a holistic learning platform designed for students of all ages and educational levels. Our platform provides learning support for a wide range of subjects, including STEM, Social Sciences, and Languages and also helps students to successfully master various tests and exams worldwide, such as GCSE, A Level, SAT, ACT, Abitur, and more. We offer an extensive library of learning materials, including interactive flashcards, comprehensive textbook solutions, and detailed explanations. The cutting-edge technology and tools we provide help students create their own learning materials. StudySmarter’s content is not only expert-verified but also regularly updated to ensure accuracy and relevance.

    Learn more
    StudySmarter Editorial Team

    Team Multimodal Texts Teachers

    • 9 minutes reading time
    • Checked by StudySmarter Editorial Team
    Save Explanation

    Study anywhere. Anytime.Across all devices.

    Sign-up for free

    Sign up to highlight and take notes. It’s 100% free.

    Join over 22 million students in learning with our StudySmarter App

    The first learning app that truly has everything you need to ace your exams in one place

    • Flashcards & Quizzes
    • AI Study Assistant
    • Study Planner
    • Mock-Exams
    • Smart Note-Taking
    Join over 22 million students in learning with our StudySmarter App