2.1.1 Multimodal PPT
First of all, we will explain the definition of modality. Mills (2006:25) hold thatmodality refers to the interaction style between human senses(vision, hearing, tactilesensation) and external surroundings(people, animals, plants and machines). Modality isthe carrier for the resources or discourse analysis. Modality refers to the interaction tool orchannel such as the olfactory modality, gustatory modality, visual modality, auditorymodality and tactile modality from the perspective of sense classification (Gu Yueguo,2007). When we talk about the modality, we need to know about another two terms:medium and mode. The medium is a kind of technological tool which is used to transmitinformation. Mode refers to the communication ways such as the electronic mode, writtenmode, verbal mode and so on (Du Yinwei, 2019). The interaction in a single sense is thesingle mode. The interaction among more than two senses is multimodality.
Zhu Yongshen (2007) states that modality refers to the communication channels andmedia, which includes language, and other nonverbal symbol systems such as image, color,audio and video. Gu Yueguo(2007) proposes that modality refers to an interaction stylebetween people and environment through senses. The interaction of a single sense can becalled as monomodality(single mode). The interaction of multiple senses is called asmultimodality(multiple modes).
......................
2.2 Theoretical Foundation
2.2.1 Multimodal Discourse Analysis Theory
Multimodal Discourse Analysis theory was firstly proposed by Harris (1952:56-59).We can call it as Systemic-Functional Semiotics (Zhu Yongsheng,2007).Harris(1952:56-59) not only analyzed the internal regulation of discourse analysis but alsoexplored the relation between discourse analysis, ideology and cognitive style. Thefundamental point for MDA theory is social semiotics which is based on Halliday's (1985)Systemic Functional Grammar Theory. Halliday (1985) indicated that there are 3meta-functions of language which include conceptual function, interpersonal function anddiscourse function. From the aspect of MDA theory, verbal and nonverbal language havethe above three functions. MDA theory extends its focus from text or words to picture,graph, color, font, audio and other symbolic resources. Some researchers delve into theMDA theory and provide experiences for the application of MDA theory in teaching.
Stock (2004) proposes a framework of modes and sub-modes in printed media. Thisframework includes sensory channels, core mode, media variants, modes, sub-modes andfeatures. Sensory channels include visual channel, audio channel and so on. Core mode islanguage. The main media variant is writing. Modes include non-verbal resources, meansand typography resources. Sub-modes include color, font, type, gesture, distance, topic,speech acts and so on. Features include saturation, purity, modulation, differentiation,directness and so on. Zhang (2009) puts forward a framework of multimodal discoursesubsystems. This framework is proposed from five levels: culture level, context level,meaning level, form level and media level. In the culture level, culture context includesideology and genre. In the context level, context includes the tenor of discourse, the field of discourse and the mode of discourse. In the meaning level, discourse meaning levelincludes textual textual and interpersonal meaning. In the form level, form includeslanguage, vision, hearing and sense. Relation is composed of complementary relation andnon-complementary relation. In the media level, language includes para-language andverbal language. Nonverbal language includes body language and non-body language.Furthermore, Zhang (2009) further explores the media level. Media system is composed oflanguage and non-language. Language includes para-language and verbal language.Non-language includes body language and non-body language. Body language includesface, action, metaphor and limb. Non-bod
