In Halliday’s work, the term ‘meaning potential’ is just about synonymous with ‘language’. Learning a language is ‘building up a meaning potential’ (1978: 30) and what is built up is a system of choices that constitutes the ‘reality’ of a culture (ibid, 123 – the inverted commas are Halliday’s) and circumscribes what we can mean.
In multimodality, another related term has come into use, ‘affordance’. It is taken from the ecological perception theory of J.J. Gibson (1979). According to Gibson, affordances are the potential uses of a given object, which stem directly from their material, observable properties. However, different observers will pick up different affordances, depending on their needs and interests. Although the affordances are objectively there, perception is selective and subjective. It should be added that Gibson primarily referred to the behaviour of animals in their environment, and that he regarded the environment as not meaningfully structured: “the environment does not communicate with the observer who inhabits it…it is entirely up to us to perceive it” (ibid: 127).
The concept of ‘affordance’ was then taken up in the field of design, especially in the design of human-computer interfaces, which sought to create ‘intuitive’ interfaces where “the user knows what to do just by looking…no picture, label or instruction is required” (Norman, 1999: 9). Interfaces thus came to be imagined as if they were the natural environment Gibson had in mind, there to be used according to our needs and interests, and not themselves motivated by any form of intentional design.
The use of the term ‘affordance’ in multimodality is near synonymous with ‘meaning potential’ in Halliday’s sense. It refers to the potentialities and constraints of semiotic modes, to ‘what they can express’. The difference with Halliday is a new emphasis on the materiality of the signifier, a differentiation between the “affordances of the material stuff” and “the work done in social life with that material over very long periods” (Kress, 2010: 80). This allows Kress to explore how the same materialities may be used differently in different kinds of ‘social work’, how sound, for instance, is used differently in speech and music, and how different materialities, even though ultimately expressing the same cultural ‘reality’, do so differently on the basis of their modal affordances.
Van Leeuwen uses the term affordance somewhat differently. He uses ‘meaning potential’ to refer to uses of material semiotic resources that have been socially ratified, whether by design or by dint of custom and practice, resources that have congealed into modes, and he uses ‘affordance’ to refer to semiosis, to new uses of semiotic resources. In his view, semiosis is based on our physical experience of the material properties of signifiers in parametric systems, for instance the properties of voice quality such as pitch level and range, loudness, roughness, breathiness, tension and vibrato, all of which afford meaning on the basis of our bodily experience, which tells us, for instance, that tension may derive from excitement, anxiety, exertion, etc., and allows us to select from this complex of experiences what is relevant at a given moment in a given context (Van Leeuwen, 2009)
The extension of the term affordance has been criticized (eg Oliver, 2005), and perhaps it is true that social semiotics tends to walk a tightrope between the individualized perception-action nexus that is so characteristic of contemporary technology and that is now increasingly used to model human behaviour, and the Hallidayan emphasis on language as the realization of cultural context and as fundamentally social; and between a very localized view of communication in which sign-makers, on the basis of ‘interest’, select features from the resources they have at hand to represent what, to them, “is criterial, at that moment, and in that context” (Kress and Van Leeuwen, 2006: 11), and, on the other hand, the social construction of ‘what we can mean’ and what we may mean, living as we are in “social structures that are inevitably marked by power differences” (ibid: 11).
Citing this entry
van Leeuwen, Theo. 2015. “Multimodal meaning potential.” In Key Terms in Multimodality: Definitions, Issues, Discussions, edited by Nina Nørgaard. www.sdu.dk/multimodalkeyterms. Retrieved dd.mm.yyyy.
References
Gibson, J.J. (1979) The Ecological Approach to Visual Perception. Boston: Houghton Mifflin
Halliday, M.A.K. (1978) Language as social semiotic. London: Arnold
Kress, G. (2010) Multimodality – A social semiotic approach to contemporary communication. London: Routledge
Kress, G. and Van Leeuwen, T. (2006) Reading Images – The Grammar of Visual Design. London: Routledge
Norman, D.(1999) Affordances, Conventions and Design. Interactions, May: 38-43
Oliver, M. (2005) The Problem with Affordance, E-Learning and Digital Media 2(4): 402-413
Van Leeuwen, T. (2005) Introducing Social Semiotics. London: Routledge
Van Leeuwen,T. (2009) ‘Parametric systems: The case of voice quality’, in C. Jewitt ed. The Routledge Handbook of Multimodal Analysis. London: Routledge, pp. 68-78.