Play and Performance Interfaces for Culture and Games

Focusing on the relationship between performance and content in digital media, we will integrate the analysis, evaluation and design of cultural and entertainment applications with the development of interfaces for play, performance, and storytelling. Research/creation occurs along three axes: spatial play, gestural play and vocal/audio play.

1) Spatial Play considers the body’s movement in physical space in relation to digital media. What technological innovation will be needed to design true "full-body" games? How do the physical, material, and information spaces of engagement in museums and other public sites affect participants' experiences and understandings?

2) Gestural Play considers discrete movements of bodies in relation to screen-based media and interactive installations, raising questions of how the body and its gestural motion can affect the player’s relationship to digital representations. We explore how gestural interfaces may be used to generate meaning and support expression, embodied learning, and engaged participation.

3) Vocal/Audio Play considers voice as an important input channel in interfaces. We explore vocality as a corporeal practice that enhances engagement and interactivity. How can vocal interfaces control or augment experiences like gameplay? How can vocal play create new categories of expressive speech that influence a range of cultural applications?

PlayPr encompasses several projects under the rubrik of spatial, gestural and vocal play. Most milestones will refer to multiple projects.

• The Gestural Games Group (G3) focuses on the creation and study of embodied gestural games, a platform for evaluating novel controllers, and a platform for creating sensor-based responsive spatial environments.
• Ludic Voice (LuV) explores vocal play through two game prototypes and a live performance platform, focusing on voice-as-controller and on expressive qualities of vocal recitation.
• A Microethnographic effort explores gameplay at home, as well as providing resources for evaluating or socially intervening with game prototypes.
• A Playful Museum Guide project embeds play and technology appropriation in a particular sort of cultural space.
• The TUNE project explores the use of tangible objects embedded in physical space for user interpretation of the sort of open-ended narratives typical of games and play experiences.
• StoryTrek, meanwhile, embeds narrative in space, to be performed by mobile actor/readers.
• Ongoing work on Augmented Reality Games (ARGs) creates hybrid spaces that tell stories about historical places through gameplay.
• Cutting Edges allows end-users to reinterpret and re-edit short films through active bodily engagement.
• The Ozone Media Choreography System™ can create responsive environments using dynamic topological models. It contributes an infrastructure for rapid prototyping of live movement scenarios in dance and architectural experiments, using theatrical as well as phenomenological strategies.
• Our work explores five interlinked themes: Embodiment, Play, Performance, Social Production, and Narrative.

Embodiment

Within HCI and Ubicomp there is now a well-established interest in the body and physical space [3] as well as in tangible, ambient, and haptic interactions with computation [5]. We are increasingly concerned with the ways that subjects are politicized and socialized in their embodied relationships with and through technology. A focus on embodied interaction does not necessarily imply the design of interfaces for unencumbered physical movement (such as the Microsoft Kinect), but rather an understanding of bodies, and their social embeddedness, as loci of action, knowledge and collaboration.

Expressiveness

While digital systems are typically thought of as tools for symbolic representation (i.e. text, code, icons), we are interested in how such systems may support expressiveness that is not necessarily representational. Here we focus on non-symbolic expression that occurs in the space for interpretation around code and symbol systems – the contextually meaningful that cannot be said in so many words. In space, this may manifest as the production-in- action of the space [2], as opposed to the static, designed configuration of it. In embodied interaction this can manifest as an “excess” of gesture beyond that which controls the interface.

Play

HCI and Ubicomp have seen an increasing emphasis on play as a core design concern and an important mode of interaction through and with technology [4]. Play here can be thought of in the usual sense of enjoyable recreational activity. But we would argue that “play” in the mechanical sense, meaning looseness, or wiggle room, is also a useful concept [1]; indeed that “wiggle room” is what allows for expressiveness and play-in-the-usual-sense. We investigate how play can be supported (or not) by material, interactional and symbolic properties of designed systems. While play is obviously a core concern of game systems, we contend that playfulness also has a role in the design, use, and appropriation of all technical systems.

Performance

Collaborative systems depend on users’ performance of collectively visible, accountable and interpretable actions beyond that needed as computer input, as resources for others as well as ourselves [7]. We posit performance as enabled by and contributing to collective play and expressiveness, where the acting out of physical and social embodiment fosters the mutual understanding needed to perform comprehensibly. Coupling and interaction enable performance through a digital system.

Social Production

Social production is typically concerned with the ways in which collective human interaction gives meaning to technologies in their social contexts. We agree with the importance of such processes, but we also take “production” more literally. There is an increased interest in low cost, hackable and repurposable technologies [6] and design for appropriation by users or non-specialists. We both use such technologies in our prototyping process, and wish to support our users’ collective repurposings.

Narrative

All interactions with technology involve a narrative dimension even if it is simply a matter of representation (the interpretation of signs) and justification (the reasons for action). Narrative design or narrative in the context of design treats narrativity as an implicit aspect of digital interaction. How do we makes sense of what narrative is in digital experiences?  What are models and methods of narrative design?

REFERENCES

1.Brewer, J., Williams, A., and Dourish, P. (2007) A handle on what's going on: combining tangible interfaces and ambient displays for collaborative groups. Tangible and Embedded Interaction 2007. February 2007. Baton Rouge, Louisiana, USA.

2.de Certeau, M. (2002). The Practice of Everyday Life: University of California Press.

3.Dourish, P. (2001) Where the Action Is: The Foundations of Embodied Interaction. MIT Press.

4.Gaver, W. (2002) Designing for Homo Ludens. 13 Magazine No. 12. June, 2002.

5.Ishii H. & Ullmer, B. (1997). Tangible Bits: Towards seamless interfaces between people, bits and atoms. Proceedings of the CHI '97 Conference on Human Factors in Computing Systems, 234-241. New York, NY: ACM Press.

6.Mellis, D. A., M. Banzi, D. Cuartielles und T. Igoe: Arduino: An Open Electronics Prototyping Platform. In: CHI 2007, San Jose, USA, Apr. 2007. ACM Press.

7.Nadeau, B., and Williams, A. Tactful Interaction: Exploring Interactive Social Touch Through a Collaborative Tangible Installation. Tangible and Embedded Interaction 2009. February 2009. Cambridge, UK.