Shaun Bangay: Virtual Reality and Computer Graphics

Conversion of Natural Language Fiction to Graphical Scenes

Abstract

Virtual environments have proven to be increasingly popular in recent times, specifically with regard to entertainment. However, it has been noted that the process in constructing such environments remains largely manual, requiring the tedious construction of such environments using manual mouse-based systems. One suggested technique for automating the construction of virtual environments is through the use of existing descriptions found in fictional literature. In addition to providing scene descriptions, other aspects could be extracted from literature such as the verbal discourse between characters, or script. Also character attributes, as well as movements, may be extracted from such sources.

This project proposes to cover the following topics:

Script generation: construct a script based on direct speech in the literature, attempting to resolve all pronoun references such as "he" and "she" so that a coherent, fully labelled verbal discourse can be produced
Information extraction: construct and test models of discourse in order to extract scene information, such as the objects present in a scene
Movement: analysis of discourse to attempt to build a timeline of actions to be performed in the scene

Participants

Kevin Glass
Shaun Bangay
Bruce Alcock

Journal Articles

[1] Kevin Glass and Shaun Bangay. A method for automatically creating 3D animated scenes from annotated fiction text. IADIS International Journal on Computer Science and Information Systems, 4(2):103-119, Spring 2009. [PDF] [BibTeX]

[2] Kevin Glass and Shaun Bangay. Evaluating and improving morpho-syntactic classification over multiple corpora using pre-trained, "off-the-shelf", parts-of-speech tagging tools. South African Computer Journal, 40:4-10, June 2008. Also published in PRASA 2007: Proceedings of the 18th Annual Symposium of the Pattern Recognition Association of South Africa. [PDF] [BibTeX]

Conference Papers (peer reviewed)

[1] Kevin Glass and Shaun Bangay. Automating the creation of 3D animation from annotated fiction text. In Proceedings of the IADIS International Conference on Computer Graphics and Visualization 2008, pages 3-10, July 2008. Best paper award. [PDF] [BibTeX]

[2] Kevin Glass, Shaun Bangay, and Bruce Alcock. Mechanisms for multimodality - taking fiction to another dimension. In Afrigraph '07: Proceedings of the 5th international conference on Computer graphics, virtual reality, visualisation and interaction in Africa, pages 135-144, Grahamstown, South Africa, October 2007. ACM Press, New York, NY, USA. [PDF] [BibTeX]

[3] Kevin Glass and Shaun Bangay. A naive salience-based method for speaker identification in fiction books. In PRASA 2007: Proceedings of the 18th Annual Symposium of the Pattern Recognition Association of South Africa, pages 1-6, November 2007. [PDF] [BibTeX]

[4] Kevin Glass and Shaun Bangay. Evaluating and improving morpho-syntactic classification over multiple corpora using pre-trained, "off-the-shelf", parts-of-speech tagging tools. In PRASA 2007: Proceedings of the 18th Annual Symposium of the Pattern Recognition Association of South Africa, pages 19-24, November 2007. Also published in South African Computer Journal, 40:4-10, June 2008. [PDF] [BibTeX]

[5] Kevin Glass and Shaun Bangay. Constraint-based conversion of fiction text to a time-based graphical representation. In SAICSIT '07: Proceedings of the 2007 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countries, pages 19-28, Fish River Sun, South Africa, October 2007. ACM Press, New York, NY, USA. Best paper award. [PDF] [BibTeX]

[6] Kevin Glass and Shaun Bangay. Hierarchical rule induction for speaker identification in fiction books. In SAICSIT '06: Proceedings of the 2006 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countries, pages 31-40, Somerset West, South Africa, October 2006. South African Institute for Computer Scientists and Information Technologists, South Africa. [PDF] [BibTeX]

[7] Kevin Glass and Shaun Bangay. Evaluating parts-of-speech taggers for use in a text-to-scene conversion system. In SAICSIT '05: Proceedings of the 2005 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countries, pages 20-28, White River, South Africa, September 2005. South African Institute for Computer Scientists and Information Technologists, South Africa. [PDF] [BibTeX]

[8] Kevin Glass and Shaun Bangay. Building 3d virtual environments from natural language descriptions. In Poster presented at SAICSIT '05: Proceedings of the 2005 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countries, pages 20-28, White River, South Africa, September 2005. South African Institute for Computer Scientists and Information Technologists, South Africa. Best Poster Presentation. [PDF] [PNG] [BibTeX]

PhD Theses

[1] Kevin Glass. Automating the Conversion of Natural Language Fiction to Multi-Modal 3D Animated Virtual Environments. PhD thesis, Department of Computer Science, Rhodes University, Grahamstown, South Africa, August 2008. [PDF] [BibTeX]

External links

Automating the Conversion of Natural Language Fiction to Multi-Modal 3D Animated Virtual Environments