A Study on Verbal and Image Relations in Multimodal Texts From the Perspective of Intersemiotic Complementarity

Yong HU, Qing QIU


Multimodal texts play a key role in the context of new media with the acceleration of modern technology of communication. Instead of merely keeping an eye on the verbal semiotic, recent developments in multimodal studies place a particular focus on the multi-semiotic texts involving some other non-verbal semiotics. The primary focus of this paper is placed on the textual meaning which is realised in both visual and verbal semiotics within children’s picture books. In this paper, the textual features of visual components are first identified throughout the database, and it then moves to the analysis of compositional intersemiotic complementarity between verbal and visual semiotic systems to create messages in multimodal texts.


Multimodal texts; Intersemiotic complementarity; Verbal and image relations

Full Text:



Barthes, R. (1977). Rhetoric of the image. In R. Barthes (Ed.), Image–music–text. London: Fontana.

Chang, C. G. (2004). English idioms and interpersonal meanings. Guangzhou: Sun Yat-sen University Press.

Halliday. M. A. K. (1985). An introduction to functional grammar. London: Arnold.

Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. London: Longman.

Halliday, M. A. K., & Matthiessen, C. M. I. M. (2004). An introduction to functional grammar (3rd ed.). London: Arnold.

Kress, G., & van Leeuwen, T. (1996). Reading images: The grammar of visual design. London: Routledge.

Martinec, R., & Salway, A. (2005). A system for image–text relations in new (and old) media. Visual Communication, 4(3).

Painter, C., & Martin, J. R. (2011). Intermodal complementarity: modelling affordances across image and verbiage in children’s picture books. In G. W. Huang (Ed.), Research in functional linguistics and discourse analysis (Vol.3). Beijing: Higher Education Press.

Painter, C., Martin, J. R., & Unsworth, L. (2013). Reading visual narratives: image analysis of children’s picture books. London: Equinox.

Royce, T. (2007). Intersemiotic Complementarity: A framework for multimodal discourse analysis. In T. Royce and W. Bowcher (Eds.), New directions in the analysis of multimodal discourse. Mahwah, New Jersey: Lawrence Erlbaum Associates.

DOI: http://dx.doi.org/10.3968/11938


  • There are currently no refbacks.

Copyright (c) 2020 Canadian Social Science

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Share us to:   


  • How to do online submission to another Journal?
  • If you have already registered in Journal A, then how can you submit another article to Journal B? It takes two steps to make it happen:

1. Register yourself in Journal B as an Author

  • Find the journal you want to submit to in CATEGORIES, click on “VIEW JOURNAL”, “Online Submissions”, “GO TO LOGIN” and “Edit My Profile”. Check “Author” on the “Edit Profile” page, then “Save”.

2. Submission

  • Go to “User Home”, and click on “Author” under the name of Journal B. You may start a New Submission by clicking on “CLICK HERE”.
  • We only use three mailboxes as follows to deal with issues about paper acceptance, payment and submission of electronic versions of our journals to databases: caooc@hotmail.com; css@cscanada.net; css@cscanada.org

 Articles published in Canadian Social Science are licensed under Creative Commons Attribution 4.0 (CC-BY).


Address1020 Bouvier Street, Suite 400, Quebec City, Quebec, G2K 0K9, Canada.

Website: Http://www.cscanada.net Http://www.cscanada.org 
E-mailcss@cscanada.net, css@cscanada.org

Copyright © Canadian Academy of Oriental and Occidental Culture