Jewitt, Carey, et al. “Exploring Learning through Visual, Actional and Linguistic Communication: The Multimodal Environment of a Science Classroom.” Educational Review 53 1 (2001): 5-18.

From the abstract:

It suggests that learning is realised through the interaction between visual, actional and linguistic communication (i.e. learning is multimodal) and involves the transformation of information across different communicative systems (‘modes’), e.g. from speech to image. It demonstrates that learning is a process of selection, adaptation and transformation motivated by the interests of pupils and the context of learning” (5).

This finding is quite important to my research, since it suggest that it is this multimodality of seeing, doing, and discussing that is most effective for learning. Furthermore, students–when motivated by their own interest in the subject matter–adapt to certain modes, in part by selecting and transforming information from one mode to another.

Jewitt, et al. note that whereas the dominant view of learning generally sees it as a primarily linguistic accomplishment, they took a more multimodal approach, applying the term “learning” to all methods of making meaning, that is how both instructors and students convey meaning in the classroom. The authors define “mode” as “organized, regular, socially specific means of representation…” (5).

Coming form a clearly social semiotic perspective, the authors see students as very involved in the learning process in that they reshape the information conveyed by the classroom instructor to create their own meaning.

The field of social semiotics is primarily concerned with communication as an inherently social phenomenon (Hodge & Kress 1988). The conceptual framework of social semiotics accounts for how modes of communication (speech, writing, image, gesture, three-dimensional models, etc.) are used.

As I’ve noted in earlier discussions (posts) on social semiotics, it has much to do with choice in that meaning is created as a consequence of choice: the communicator selects a certain method of communication and a certain message, both of which have an affect on the message that the student (or other conversant) receives and interprets. In this way, learning can be greatly affected by the selected mode of delivery.

The authors state that communication is naturally multimodal; for example, we use facial expression and gesture when speaking. However, there are so many different methods of communication, that many of those modes are often removed; for example, the telephone removes visual. Most textual communication removes visual and aural modes of communication, as well as the temporal or immediate nature of it. However, there are variations of this as well, such as texting or IM chat, which offer that immediacy in communication. The online video conversation (OVC) offers audio, visual, social distance (simulated to some extent), but does not offer the immediacy.

Continuing with the idea of social semiotics and sign-making, the process of learning can be seen as occurring in the way that students make meaning (or signs) through their interpretation of the message received from the instructor. What the students focus on is somewhat affected by choice in that their it is selective, focusing on the points in which they find the most interest. As Jewitt et al. state, “[W]e treat teacher and pupils on par, both producing new signs in response to the signs of the other, reflecting their different interests” (7).

The article makes a number of other points that are relevant to my study, such as the suggestion that using multimodality in the classroom highlights the benefits of thinking about what mode may be ‘best suited’ to a task and the different cognitive and representational demands different modes place on pupils. (17). However, the authors’ study largely focuses on how multimodality affects learning, which is beyond the scope of my study.